Written by Nejde on 04.11.2023 at 22:26

Written by Ball Fondlers on 04.11.2023 at 19:28

The point that the existence and high visibility of a top 20 on the site leading to rating abuse is 100% correct. I haven't seen a really good counter argument to this

It's not 100% correct because rating abuse is not only related to the top 20 lists. There are users who boost all bands they like with 10s no matter how obscure they are and tank bands they dislike with 1s.

This does not contradict the hypothesis that the existence of the Top 20 lists induces rating abuse. This hypothesis does not rule out rating abuse of a cause other than this existence, it merely claims that the one leads to the other.

Written by Ball Fondlers on 04.11.2023 at 19:28

I like this mde guy, I hope he sticks around.

The point that the existence and high visibility of a top 20 on the site leading to rating abuse is 100% correct. I haven't seen a really good counter argument to this

Thank you for your kind words, and, even more importantly, for eloquently reiterating the point I was trying to make. You have expressed the wish that I stick around, and you've certainly helped the cause immensely. However, as they say, be careful what you wish for.

Written by Nejde on 04.11.2023 at 22:26

Written by Ball Fondlers on 04.11.2023 at 19:28

The point that the existence and high visibility of a top 20 on the site leading to rating abuse is 100% correct. I haven't seen a really good counter argument to this

It's not 100% correct because rating abuse is not only related to the top 20 lists. There are users who boost all bands they like with 10s no matter how obscure they are and tank bands they dislike with 1s.

I didn't say it's the only reason for rating abuse

Written by Roman Doez on 03.11.2023 at 11:40

Slight nitpick right there, the average ratings on RYM are very different than on MS.

Yeah, you're totally right. I shouldn't have compared the ratings, RYM is a different kettle of fish. But this is not because RYM's average ratings are different, but rather because of the exact reason why they are different.
What I simply didn't know (I don't use RYM) is that they've been using a weighting algorithm for quite some time now. So what the site shows as “Average” is not the arithmetic mean of all ratings submitted, but rather the result of a pretty non-transparent weighting process.

What I've learned today is that the weighting factor assigned to each user can be 0 (new users and anyone with a suspicious rating distribution), 0.25-0.50 (users who have not logged in for a while, people who only visit the site occasionally and users with less than 50 album ratings), 1.0 (active users with balanced ratings who have been there for a long time) or 1.25 (active users who write a review every now and then). An interesting detail is that you can't see your own weighting factor; you have to ask a moderator to find out.
But that's not all, as the weighting factor is just one part of the site's anti-abuse measures. There's also a popularity weighting and other stuff running in the background. Apart from a few insiders, no one knows exactly what the entire algorithm does.

RYM's statement on this matter:

Quote:
RYM assigns a score to each album based on many factors, including weighted ratings.
[...] The actual scores as well as the chart algorithm is not public (to prevent abuse of the system).

Yet, I think this system is legitimate because RYM is financed through advertising, donations, member fees and commissions. Only the current number of active users is relevant, because former users do not pay premium member fees, they do not click on annoying pop-up ads and they do not purchase music for which RYM receives a commission. From an economic perspective, it makes total sense to suppress the impact of ancient ratings in order to make the charts appear more attractive to active users. RYM's weighting system is effective, unbiased and it operates fully automatic. There's nothing wrong with that.


In any case, it was wrong to directly compare the ratings of MS and RYM, and I would like to apologize for that. I just wasn't aware that the RYM algorithm existed and that it had such a big impact.
(which is, by the way, undisputed - anyone who takes a look at old RYM ratings using Wayback Machine can easily check it out)
I'm also sorry for this far too long and rambling post, but I didn't want to leave your legitimate objection unanswered.

^Isn't that kind of weighting sort of what mde is talking about...?

Written by Starvynth on 07.11.2023 at 22:13

RYM is a different kettle of fish.

I guess I now know why I'd rather opt for a transparent, non-complex, equally-weighted-for-all-members, ad-free, free-of-charge metal storm rating algorithm instead.

Written by Karlabos on 07.11.2023 at 22:21

^Isn't that kind of weighting sort of what mde is talking about...?

I don't think so. Didn't he say the exact opposite?*
He believes that it's primarily the "posers" who try to manipulate the charts. But the truth is that most average ratings have dropped significantly since the weighting system was introduced (just compare Opeth in 2006 with the current situation).
What emerges from this observation is that RYM's algorithm primarily eliminates high user ratings. So it's not about the laymen, but the fanboys who are desperately trying to push their favorite bands. This applies to RYM and Metal Storm alike, and it has always been the main aspect of rating abuse.

If you can't believe this: the proportion of MS users with an average rating of 9.0 or higher is 43%, while only 0.7% of all MS users have an average of less than 5.0/10.

*edit: mde017 is of course absolutely right that the direct impact of a 1/10 rating on the Top 20 is much stronger than the effect of a 10, because only two 1s are needed to balance out seven 10s. That's a serious issue we need to tackle.

Written by Starvynth on 07.11.2023 at 23:15

the direct impact of a 1/10 rating on the Top 20 is much stronger than the effect of a 10, because only two 1s are needed to balance out seven 10s. That's a serious issue we need to tackle.

But this issue is not inherent in our rating algorithm, it rather results from the distribution of ratings across all members and an overall average rating of around 8. It would not be an issue if the overall average was at 6 or better 5.5. But how do you want to tackle that? You could put a weight at ratings, as a function of the distance from the average rating; a greater distance gets a lower weight. But I am not sure I would favor such complexity in our rating algorithm.

Written by Starvynth on 07.11.2023 at 23:15


He believes that it's primarily the "posers" who try to manipulate the charts.
What emerges from this observation is that RYM's algorithm primarily eliminates high user ratings. So it's not about the laymen, but the fanboys who are desperately trying to push their favorite bands. This applies to RYM and Metal Storm alike, and it has always been the main aspect of rating abuse.

"I'm receiving some feedback again regarding my use of the term 'poser.' Just to clarify, I intentionally used this derogatory term to highlight a specific behavior rather than targeting individuals. My intention was to draw attention to an issue that I feel strongly about.

For example, I may not personally listen to Slipknot, but I wouldn't consider giving them a low rating just to boost my favorite band's position. That would be a loser move. Doing so would make me feel like I'm being insincere and, in a way, posing as asomeone who actually devoted time listening to them. ( On top of it I do not care about ranking - music is not a sport - rating is only useful if promotes good music)
On the other hand I find it hard to comprehend why some 'fanboys' would give a perfect 10 rating to their favorite bands is being described a problem in first place. As you pointed out, the impact of a 10 rating is much smaller than that of a 1 or even a 5. If they genuinely love a band, giving it a 10 seems reasonable. The question is, what should they rate to beloved band - an 8? Well I am officially announcing I am going to give to whole discography of Death 10s and I will feel zero guilt about it. You cannot go wrong with any of their albums.

In regards to the algorithm on RYM, I have mixed feelings. I understand why they use a weighting algorithm to reward users who rate extensively and diversely. From a commercial standpoint, it might be a smarter approach compared to what we do here. I acknowledge that our approach is more transparent but might deter some users. On the other hand, RYM, even after giving abusers a weight of 0, still allows them to participate on the site and 'vote' in some way."

Written by Redel on 07.11.2023 at 22:22

I guess I now know why I'd rather opt for a transparent, non-complex, equally-weighted-for-all-members, ad-free, free-of-charge metal storm rating algorithm instead.

Same here. I'd rather have real people keep an eye on rating abuse than trust an anonymous and fully automated algorithm. On the other hand, I believe that the sheer volume of user votes cannot be consistently controlled by one or two people alone. It's a bottomless pit.

Written by Redel on 08.11.2023 at 20:12
But this issue is not inherent in our rating algorithm, it rather results from the distribution of ratings across all members and an overall average rating of around 8. It would not be an issue if the overall average was at 6 or better 5.5.

I see where you're coming from, but I was talking about the Top 20 because it originally sparked this discussion. And there's something that you may not be aware of: it takes 20 votes, a visible band profile and an average rating of at least 8.0 to be in the Top 20. This has nothing to do with the general distribution of ratings, it's part of the site's programming.

Alkaloid's Numen is a very good and current example, as the album just made it into the Top 20.

10 - 8 votes
9 - 24 votes
8 - 31 votes
7 - 18 votes
6 - 5 votes
5 - 2 votes
4 - 0 votes
3 - 1 vote
2 - 0 votes
1 - 0 votes

Total: 89 votes
Average: 8.011

In this particular case, a single 5/10 would be enough to knock the album out of the Top 20 again, because the new average would then be 7.977. A single 1/10 would reduce the average to 7.933, which would require 3 additional 10s to get back to a straight 8.0.
This example shows how fragile the Top 20 reacts to low ratings, as everything below 6/10 cannot be compensated by 9s and 10s in the same number.

Written by Redel on 08.11.2023 at 20:12
But how do you want to tackle that? You could put a weight at ratings, as a function of the distance from the average rating; a greater distance gets a lower weight. But I am not sure I would favor such complexity in our rating algorithm.

I believe there's only one option: weighted ratings based on the deviation of each user's personal average from a reasonable mean value (which has yet to be determined) in combination with the measures we already have in place. Without moderators who examine the extreme examples of rating abuse and without a community that reports these cases, it simply doesn't work.
I don't think this kind of weighting would be particularly complex, it would rather be a straightforward, fair and transparent way to minimize the negative impact of rating abuse.

^ that would just be an weighted average, though. You are simply using a non-linear scale for averaging, making it less likely for every album to attain too bad or too good ratings. But since this applied to every album, it wouldn't really change anything when it comes to overvoting or undervoting, it would just hide it a little.

Ultimately, while I agree there are some issues with the voting system, I'd say the best wold be leaving as it is and deal with the obvious vote abuses one by one, just like the staff has been doing. In my view mde is trying to make it so that the votes accuratly represent the musical value of the album, but that's not what a voting system should do. The voting system should depict the popular view of the album. The actual quality can only be judged by the listener himself.

Written by Starvynth on 08.11.2023 at 23:31

Alkaloid's Numen is a very good and current example, as the album just made it into the Top 20.

10 - 8 votes
9 - 24 votes
8 - 31 votes
7 - 18 votes
6 - 5 votes
5 - 2 votes
4 - 0 votes
3 - 1 vote
2 - 0 votes
1 - 0 votes

Total: 89 votes
Average: 8.011

Seeing this made me curious to see who rated this album a 3. The user was first warned for rating abuse by Redel and later by me with a 60 day ban and deleted ratings for trying to manipulate the Top 20. Now I permanently revoked the user's permission to rate because he was still tanking and boosting albums in the Top 20 or albums being close to be in it. This is a prime example of the exact same thing that is being discussed here now. I haven't deleted the ratings (yet) so if anyone is interested to see what it looks like, be my guest.

And dealing with rating abuse by myself, considering the size of our database and all albums featured, is a Herculean task and it's impossible for me to catch everyone abusing our rating system. But I'm doing what I can even if it's just a drop in the ocean.

Written by mde017 on 08.11.2023 at 20:14
I'm receiving some feedback again regarding my use of the term 'poser.' Just to clarify, I intentionally used this derogatory term to highlight a specific behavior rather than targeting individuals. My intention was to draw attention to an issue that I feel strongly about.

Yeah, and I feel strongly about a forum that can do without insults that may be taken personally. You mentioned specific albums and everyone could see which individual users you were directly or indirectly targetting, because our album ratings are not anonymous.
But let's leave it at that. I don't think you have any bad intentions.
Peace!

Written by mde017 on 08.11.2023 at 20:14
For example, I may not personally listen to Slipknot, but I wouldn't consider giving them a low rating just to boost my favorite band's position. That would be a loser move. Doing so would make me feel like I'm being insincere and, in a way, posing as asomeone who actually devoted time listening to them. ( On top of it I do not care about ranking - music is not a sport - rating is only useful if promotes good music)

Agreed. That's exactly how I see it too.

Written by mde017 on 08.11.2023 at 20:14
On the other hand I find it hard to comprehend why some 'fanboys' would give a perfect 10 rating to their favorite bands is being described a problem in first place. As you pointed out, the impact of a 10 rating is much smaller than that of a 1 or even a 5. If they genuinely love a band, giving it a 10 seems reasonable. The question is, what should they rate to beloved band - an 8? Well I am officially announcing I am going to give to whole discography of Death 10s and I will feel zero guilt about it. You cannot go wrong with any of their albums.

There's a very simple answer: if everyone only rated their favorite bands' discographies and always gave a perfect score, all ratings would be absolutely worthless. The truly perfect, revolutionary, extraordinary albums would be lost in an endless sea of mediocrity and a 10/10 would no longer have any meaning.
Seriously, people who can't tell the difference between a 7 and a 10 shouldn't even start diluting album ratings with their votes.
(Please note that I'm not talking about your intentions regarding Death's discography. Feel free to do whatever you want.)

Written by mde017 on 08.11.2023 at 20:14
I acknowledge that our approach is more transparent but might deter some users. On the other hand, RYM, even after giving abusers a weight of 0, still allows them to participate on the site and 'vote' in some way."

Thousands of RYM users have no idea that their ratings have no impact at all. Also, RYM uses an algorithm whose exact functionality is unknown to the public, but which even takes the general level of popularity of artists into account when calculating average ratings and charts. If that's not a reason to deter users from participating, then I don't know what is.

Written by Starvynth on 08.11.2023 at 23:31


And there's something that you may not be aware of: it takes 20 votes, a visible band profile and an average rating of at least 8.0 to be in the Top 20.

Yes, I was aware of this rule, but I claim it is irrelevant for my point. Imagine there was no such rule. How would the Top 20 lists look like? There would be more albums with an even higher rating, making the lists even more affected by single, very low individual ratings. The Top 20 will by nature always display the top end of the rating distribution.
This is all just a matter of the high level of the overall average rating. Our barrier for entry into the Top 20 lists just mitigates this effect but it is not the cause for its existence.

Edit: Just saw, I misread your post. Forget what I just wrote. It is probably nonsense.
I thought I read you were talking about a minimum number of votes for the Top 20 and my response would have applied to that. But with a minimum average score things are obviously different, and you are right. Sorry for the confusion.

Written by Starvynth on 09.11.2023 at 01:03

Yeah, and I feel strongly about a forum that can do without insults that may be taken personally. You mentioned specific albums and everyone could see which individual users you were directly or indirectly targetting, because our album ratings are not anonymous.
But let's leave it at that. I don't think you have any bad intentions. Peace!

I don't think you have any bad intentions either so peace
However, employing derogatory language can be a useful tool, particularly when describing a behavior rather than targeting an individual. I believe it's important to call out such behavior if one aims to bring about change. Additionally, both Nejde and yourself did not hesitate to confront me, even if it meant that I might take your criticisms personally, when my behavior didn't align with your preferences. This appears to be a double standard.

You've previously mentioned that if I use derogatory language, I should be prepared to accept the consequences. Shouldn't the same apply to someone engaging in rating abuse? If they're participating in such actions, is it not fair for them to face harsh words in return?

One mistake I can admit to was referencing specific albums and bands (if you check my favorite bands, you'll notice they aren't even included). I thought providing examples would clarify my point, but it seems to have opened a can of worms. Personally, I don't care whether the latest Dying Fetus album or any other is in the top 20 or not, but that specific mention led the discussion in an unproductive direction

Written by Starvynth on 09.11.2023 at 01:03

There's a very simple answer: if everyone only rated their favorite bands' discographies and always gave a perfect score, all ratings would be absolutely worthless. The truly perfect, revolutionary, extraordinary albums would be lost in an endless sea of mediocrity and a 10/10 would no longer have any meaning.
Seriously, people who can't tell the difference between a 7 and a 10 shouldn't even start diluting album ratings with their votes.

Well, that argument doesn't hold up. It assumes that the existence of a group automatically leads to a dominating impact on rankings. People vote differently; some only vote for their favorite bands' discographies and consistently give perfect scores. Then there's another group, like myself, who only vote for music they genuinely listen to frequently, resulting in a higher distribution, albeit generally with high scores (after all, not many masochists listen to music they don't enjoy).

You'll also find critic types—individuals who consume a vast amount of music and feel compelled to explore numerous albums. Their distribution will be high, and they'll likely rate more albums than group two. Finally, there's group four: abusers. Their distribution is high, but for all the wrong reasons, as they mischievously assign low scores. My main concern lies with this last group, as mentioned in our previous discussion. My take on group one, or 'fanboys,' is that they may elevate the overall score average but won't significantly impact the differences between rated albums (although it might lean towards favoring more well-known bands)

Written by Starvynth on 09.11.2023 at 01:03

Thousands of RYM users have no idea that their ratings have no impact at all. Also, RYM uses an algorithm whose exact functionality is unknown to the public, but which even takes the general level of popularity of artists into account when calculating average ratings and charts. If that's not a reason to deter users from participating, then I don't know what is.

The trick is that they don't publish user weights, leading people to assume their rankings carry significant impact. Judging by the number of votes on RYM per album, it seems that not many users are detracted. Additionally, popularity is just one of the factors influencing rankings. You can easily examine the Esoteric ranking, which I personally enjoy doing—it's a great way to discover new music. Actually if you think about it makes perfect sense as this approach safeguards smaller bands from being unfairly downranked by abusers fixated on rankings. Their maligned behavior has a lesser effect on bigger bands due to the sheer number of votes they receive. Anyway lets leave this at that. We will not impact RYM algorithm here and as mentioned I have mixed feelings about it so I don't intend to raise RYM flag.

Written by Karlabos on 08.11.2023 at 23:43

^ that would just be an weighted average, though. You are simply using a non-linear scale for averaging, making it less likely for every album to attain too bad or too good ratings. But since this applied to every album, it wouldn't really change anything when it comes to overvoting or undervoting, it would just hide it a little.

Sorry if I didn't make myself clearer, but that's not what I meant, and that's not what a smart weighting algorithm should do. It makes no sense to balance album scores by weighting each and every user rating. It's also not about automatically limiting the impact of every single extreme rating - it's about reducing the impact of users who hand out 1s and 10s like candy, but nothing else in between.

weighting factor = (1+(64/(user average*user average)-(1/8*user average)))*ABS(ROUNDUP(1(ROUNDDOWN(user average;0))-1;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/2;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/3;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/4;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/5;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/6;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/7;0))+((user average*user average)-user average)/30*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/6;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/7;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/8;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/9;0))+ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/2;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/3;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/4;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/5;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/8;0))*ABS(ROUNDUP(1/(ROUNDDOWN(user average;0))-1/9;0))

This is just one of many examples of a possible algorithm – a simple quadratic equation that weights a 10/10 (but it could be any other rating, the result would not differ) by a user average of 1.0 to 10. Yeah, it looks pretty wild and random and it's way too long, but that's because I opted for an algorithm that uses very simple and universal mathematical operations, but no IFs, no WHENs and no lookups, so it's easily convertible.




As you can see, it does exactly what it's supposed to do: it protects the votes of users with unsuspicious average ratings, but weights everything else. This algorithm would basically take users with an average below 3 out of the game - their votes would either have no impact at all or only a very negligible influence.
The core of the algorithm is a predefined base value, in this particular case I chose 7. In a specified range around that value - in this case from 6 to 8 - the weighting is 100%. That means, users with an average rating within this range will not be affected by the algorithm at all, regardless of whether the individual user rating at hand is a 1 or a 10.
I've done offline testing with quite a few algorithms and combinations thereof, and some, like the above example, work wonders for albums that suffer from rating abuse on one or both ends of the spectrum.

Anyway, this is just me playing around with a spreadsheet application and 2 Million user ratings.
As long as there's no consensus – and that's the hard part – we'll have to deal with rating abuse the good old-fashioned way, just like we always have. But my point is that it is possible to implement a weighting system that is effective, fair and transparent, but doesn't disregard thousands of user votes just because people haven't logged in for a while.

Written by Karlabos on 08.11.2023 at 23:43
Ultimately, while I agree there are some issues with the voting system, I'd say the best wold be leaving as it is and deal with the obvious vote abuses one by one, just like the staff has been doing. In my view mde is trying to make it so that the votes accuratly represent the musical value of the album, but that's not what a voting system should do. The voting system should depict the popular view of the album. The actual quality can only be judged by the listener himself.

After spending way too many hours with dry statistics, dull figures and boring algorithms, I tend to agree with all of this. Nevertheless, I believe that as long as a 10/10 (or 1/10) from a user who awards this very special rating only once or twice a year has just as much impact as a 10 (1) from a random multi account with a 10.0 (1.0) average, we cannot really speak of fair album ratings.

Written by Starvynth on 18.11.2023 at 13:03


Sorry if I didn't make myself clearer, but that's not what I meant, and that's not what a smart weighting algorithm should do. It makes no sense to balance album scores by weighting each and every user rating. It's also not about automatically limiting the impact of every single extreme rating - it's about reducing the impact of users who hand out 1s and 10s like candy, but nothing else in between.

Explanation

Oh, I see. So basically you are grading each user's voting system and making it count more or less depending on whether it's a fair voting system or not. In this case you chose an arbitrary threshold of 6-8 but you could also have done it other way.

I guess it makes sense. Of course, it's still exploitable, but most of the time the new accounts that are created for vote boosting wouldn't bother reading the rules in order to go through the trouble of exploiting it, so I think ultimately this is something that could work...?

But yeah, one could argue the voting system grading system is arbitrary and then there would be a quest for finding the perfect way to grade a user voting system... Which would boil down to the same impossible problem of finding a perfect voting system.

Written by Starvynth on 18.11.2023 at 13:03

...example for a weighting algorithm on the users' album ratings...

I like the idea of such a weighting scheme on the users' ratings and I like its outcome. I would also like to see it implemented on the site I guess, but here I see a problem, which you might already have come across too, as you mention the critical absense of a consensus.
The tricky part I would say is the fact that this algorithm -- and basically any other weighting algorithm I can think of now -- implies a normative decision as to what is an acceptable range into which a user's average rating has to fall and outside of which it is unacceptable in the sense that you will be punished if you fall in there. You have in your example parameterized this range to be 6.0 to 8.0, and I am pretty sure you would say that these values are basically arbitrary. But we do need to have a consensus on the exact parameters of this range. Why did you not choose 5.0 to 7.0? Or put differently, why should we punish users with average ratings of 5.9 but not users with average ratings of 7.9? As of now our rules of voting dont say that your average ratings have to fall in any range at all. Implementing such an algorithm would mean that we have to add the acceptable range to the rules accordingly. And I see a lot of potential for controversy there.

Written by Redel on 18.11.2023 at 16:07
I like the idea of such a weighting scheme on the users' ratings and I like its outcome. I would also like to see it implemented on the site I guess, but here I see a problem, which you might already have come across too, as you mention the critical absense of a consensus.
The tricky part I would say is the fact that this algorithm -- and basically any other weighting algorithm I can think of now -- implies a normative decision as to what is an acceptable range into which a user's average rating has to fall and outside of which it is unacceptable in the sense that you will be punished if you fall in there. You have in your example parameterized this range to be 6.0 to 8.0, and I am pretty sure you would say that these values are basically arbitrary. But we do need to have a consensus on the exact parameters of this range. Why did you not choose 5.0 to 7.0? Or put differently, why should we punish users with average ratings of 5.9 but not users with average ratings of 7.9? As of now our rules of voting dont say that your average ratings have to fall in any range at all. Implementing such an algorithm would mean that we have to add the acceptable range to the rules accordingly. And I see a lot of potential for controversy there.

Yes, that's exactly what I meant with "that's the hard part".

For me personally, it's very obvious that any average that falls below our definition of average (6.0) should be weighted in some way, but one could as well argue that the arithmetic mean of our scale is 5.5 and therefore any weighting of a 5.8 cannot be justified.
Another example... I have a very strong opinion that people with an average of over 8.5 don't really understand the rating system, but some users only rate albums from their own collection and therefore automatically have a relatively high average. I think the decision to only rate your own collection is absolutely legitimate - that's exactly how I do it. Why should this behavior be punished?
In addition, the current average of all user averages is 8.7. Is this very high value part of the problem or should we use it as a benchmark for determining which average ratings are acceptable and which are not?

But the best example of all is your personal distribution of ratings, because it's the prime example of a perfectly balanced distribution with an average of 6.0. This can't be a coincidence, and I suspect that you are taking the definition of average very literally, and that you're trying to adapt your album ratings to our system as best as possible. It should be clear to everyone that this truly enviable ambition should not be punished, but purely mathematically, you are only 0.19 points away from JimmyMagnetar, who has been reported as an "annoying rating abuser" more often than any other current MS member.

The whole thing is also a psychological problem. I mean, the purpose of sites like Metal Storm is to share and promote good music. Nobody is here to spend more of their valuable time with mediocre music and bad albums. A 1/10, but also a 4/10 or sometimes even a 6/10, can therefore easily be recognized as rating abuse, while we generally associate a 9 or a 10 with something utterly positive. Conveying that not every 10 has a positive impact on the fairness of the rating system is therefore much more difficult than making it clear to people that not every 6 is a personal and malicious attack on someone's favorite band or genre.

Written by Starvynth on 18.11.2023 at 18:52

For me personally, it's very obvious that any average that falls below our definition of average (6.0) should be weighted in some way, but one could as well argue that the arithmetic mean of our scale is 5.5 and therefore any weighting of a 5.8 cannot be justified.

For me it is not very obvious that ratings of users averaging below 6.0 (or 5.5) should be weighted downwards. What if someone really likes only a handful of albums very much (9-10) and at the same time rates hundreds of albums "average" or "not good". Would that not be a legitmate rating behavior? Well, anyhow, one can construct an infinite number of cases that are debatable, there will never be a solution for all possible cases of rating behavior, I guess.

Written by Starvynth on 18.11.2023 at 18:52

some users only rate albums from their own collection and therefore automatically have a relatively high average. I think the decision to only rate your own collection is absolutely legitimate - that's exactly how I do it. Why should this behavior be punished?
In addition, the current average of all user averages is 8.7. Is this very high value part of the problem or should we use it as a benchmark for determining which average ratings are acceptable and which are not?

Yes, exactly, very true. I have been wondering about these questions as well. And I havent come to an ultimate answer yet.

Written by Starvynth on 18.11.2023 at 18:52

But the best example of all is your personal distribution of ratings

Well, my own rating distribution is definitely not an entirely random outcome. Indeed I am constantly keeping an eye on it when rating albums, seeing to that it doesnt get out of balance. I am doing that because I interpret the scale in relative terms (I like albums more or less than others), I dont do an evaluation in absolute terms. And by this balance I intend to constrain myself to not put an excessive amount of either, above or below average ratings.
But that is not more than my own choice. I am aware that almost nobody else is going down this path -- and that is absolutely fine.
And, yes, the day will come when someone will report me for rating abuse, definitely, because of my 5 or 6 rating on some of their all-time favourite albums.

Written by Starvynth on 18.11.2023 at 18:52

The whole thing is also a psychological problem

Absolutely -- what can ultimately be considered wrong with calling a product of art, done by human beings, a "masterpiece" and expressing your unconditional love for it? Criticism on the other hand conveys a negative tone, and most people dont want to hear that when talking about art. Plus, effects of inflation of your own evaluation when you just tend to praise everything and are unable to say what you dislike... who cares!? Nobody is ever going to ask you which bands you dislike, you will just be asked about your favourite ones.

Written by Starvynth on 18.11.2023 at 18:52

JimmyMagnetar, who has been reported as an "annoying rating abuser" more often than any other current MS member.

I am one of those who reported him more than once before becoming staff. Since his registration 3,5 years ago he hasn't posted a single comment in the forums but his has rated 5241 albums.
Of those ratings only 29 are 10s, 173 are 9s and 520 are 8s which is 722 votes or 13% of his total ratings. But then on the lower scale 75 are 1s, 154 are 2s, 225 are 3s and 655 are 4s which in turn is 1109 votes or 21% of his total ratings. This just seems like he hates music in general. My problem is that he always has rated albums on release day and not just one or two but more often than not over half of the albums released that day and the ratings are given during the night or early in the morning. This means that he with all certainty just skims through all the albums released and rates them after just listening to a couple of minutes at most and if he doesn't like it, hands out ratings between 1-4. This makes his votes void of any value.

So I still stand by my opinion that this IS rating abuse because when the rest of the users drop in to check new releases they just see the score and most likely don't take notice of the little grey text under it that shows the number of votes. This makes a lot of good albums being disregarded as 'bad' or worse and might not be checked out by other users because of the low rating. So in the end we have ONE user who has a great impact on what other users will check out or not, at least when it comes to albums from lesser known bands. Because if I personally look at his 1-4 ratings, I haven't even heard of most of the bands. With bigger bands and big releases that get many votes is the only time his low ratings do any good because they counter all the 10s given. But if I keep looking at his ratings he doesn't really give low ratings to big or well known bands unless their albums actually are deemed bad by the community so he doesn't really help the average ratings there either.

Ultimately we have a user spamming ratings, undermining our rating system and not contributing to the site or the community in any other way. Another example of this is NICK VAN ORTON. He had almost 10000 ratings of which over half were 9s and 10s. I finally took away his permission to rate and deleted his ratings. I would honestly like to do the same with JimmyMagnetar, but because of all the low ratings he so frivolously hands out.

Written by Nejde on 19.11.2023 at 13:05

So I still stand by my opinion that this IS rating abuse because when the rest of the users drop in to check new releases they just see the score and most likely don't take notice of the little grey text under it that shows the number of votes. This makes a lot of good albums being disregarded as 'bad' or worse and might not be checked out by other users because of the low rating. So in the end we have ONE user who has a great impact on what other users will check out or not, at least when it comes to albums from lesser known bands.

But this is a concern of a more general scope than the rating behavior of a single user. It is the power of individual ratings in case the overall number of ratings is small. There are other mechanisms available to deal with this problem. All votes on new albums could e.g. be hidden for some time after release, or even the possibility for all users to vote on new releases could be turned off during this time, or average ratings -- plus the single ratings -- on all albums could only be displayed if they reach a threshold of a minimum number of votes.
Implementing such a mechanism would appear more fair to me than making single users responsible for its occurence, even if they contribute to its effect in a relevant manner (by rating very early after release and often very low).

Written by Nejde on 19.11.2023 at 13:05

Written by Starvynth on 18.11.2023 at 18:52

JimmyMagnetar, who has been reported as an "annoying rating abuser" more often than any other current MS member.

My problem is that he always has rated albums on release day and not just one or two but more often than not over half of the albums released that day and the ratings are given during the night or early in the morning. This means that he with all certainty just skims through all the albums released and rates them after just listening to a couple of minutes at most and if he doesn't like it, hands out ratings between 1-4. This makes his votes void of any value.

Death Metallus is another such user. Hell, he even voted for the new Farsoth album before it's even been released.

Written by F3ynman2000 on 19.11.2023 at 19:08

Written by Nejde on 19.11.2023 at 13:05

Written by Starvynth on 18.11.2023 at 18:52

JimmyMagnetar, who has been reported as an "annoying rating abuser" more often than any other current MS member.

My problem is that he always has rated albums on release day and not just one or two but more often than not over half of the albums released that day and the ratings are given during the night or early in the morning. This means that he with all certainty just skims through all the albums released and rates them after just listening to a couple of minutes at most and if he doesn't like it, hands out ratings between 1-4. This makes his votes void of any value.

Death Metallus is another such user. Hell, he even voted for the new Farsoth album before it's even been released.

Maybe he got a promo?

A rather obvious case. 10s for the discography of one band. 10s and 9s for all top 20 albums of 2023 except Fires In The Distance, which gets a 1.
https://metalstorm.net/users/album_votes.php?user_id=323292

I feel like I've reported this guy before. Reserves his 1s for three albums, which happen to be in the top 20 of 2023
https://metalstorm.net/users/album_votes.php?user_id=335712

Written by F3ynman2000 on 25.11.2023 at 16:38


I feel like I've reported this guy before. Reserves his 1s for three albums, which happen to be in the top 20 of 2023
https://metalstorm.net/users/album_votes.php?user_id=335712

A great example supporting my perspective. Clearly a fan of melodic black and death metal, the rating is reasonable if sticking to preferred genres. However, it seems they can't help himself not to try to influence the top 20 rankings by ratin down bands clearly doesnt listen to.

Written by mde017 on 26.11.2023 at 00:30

A great example supporting my perspective. Clearly a fan of melodic black and death metal, the rating is reasonable if sticking to preferred genres. However, it seems they can't help himself not to try to influence the top 20 rankings by ratin down bands clearly doesnt listen to.

Bro will never give up on trying to make his point

Written by mde017 on 26.11.2023 at 00:30

However, it seems they can't help himself not to try to influence the top 20 rankings by rating down bands clearly doesn't listen to.

Exactly an example of abuse.

Written by F3ynman2000 on 25.11.2023 at 16:38

A rather obvious case. 10s for the discography of one band. 10s and 9s for all top 20 albums of 2023 except Fires In The Distance, which gets a 1.
https://metalstorm.net/users/album_votes.php?user_id=323292

This user got warned and had all his Top 20 and :wumpscut: ratings deleted.

Written by F3ynman2000 on 25.11.2023 at 16:38

I feel like I've reported this guy before. Reserves his 1s for three albums, which happen to be in the top 20 of 2023
https://metalstorm.net/users/album_votes.php?user_id=335712

And this one also got warned. The mentioned 1s and the only 10 he had given this year, which was an album ranked lower in the Top 20 than the other three, were all deleted, four in total.

I will also mention that I keep track of all users reported here that needs some kind of action from me and they will be handled accordingly if the same rating behaviour continues.

Thank you for your service.

The voting from these two users seems very suspicious to me:
https://metalstorm.net/users/BasilioSatan/profile
and
https://metalstorm.net/users/steliosd/profile