How did you derive X?

Written by Nimlot on 13.09.2015 at 14:13

How did you derive X?

I did all the calcs in Excel.
There was a master 'X' cell which was linked into the formula of every album rating calculation. I just toggled this value until the top album (which turned out to be Agalloch - The Mantle) was at a rating of exactly 10.

It doesn't have a big influence on the list, as it is the same value for every album. If I change X, the order would remain the same, but the rating of everything shifts. The top album has a weighted rating of 8.4 if I change to X=8.00 for example.

I like your list. Great effort! However, I would have kept the multiple albums. The way you made it, its not a best album list but more a best artists list. Nevertheless, very interesting. Hm, another interesting step might be to correct for the vote abuses by not counting votes that are less than x (with x=4?).

Thanks, I see what you mean now. It sure is interesting to see a list like this.

Written by Nihil Aeternum on 13.09.2015 at 14:52

I like your list. Great effort! However, I would have kept the multiple albums. The way you made it, its not a best album list but more a best artists list. Nevertheless, very interesting. Hm, another interesting step might be to correct for the vote abuses by not counting votes that are less than x (with x=4?).

Thanks.
Yeah I was torn between keeping them or not. You're right, I guess it's technically not a best album list now. I still have the original list on a separate tab. Exclusions, and their original ranking, include the following (ie. The album at #100 in the list above is actually #130):

3) Agalloch - Ashes Against The Grain
13) Saturnus - Martyre
16) Opeth - Ghost Reveries
19) Be'lakor - Of Breath And Bone
20) Pain Of Salvation - The Perfect Element, Part I
24) Porcupine Tree - Fear Of A Blank Planet
25) Nevermore - This Godless Endeavor
44) Septicflesh - The Great Mass
46) Windir - Likferd
50) Leprous - Tall Poppy Syndrome
59) Saturnus - Saturn In Ascension
67) Opeth - Deliverance
72) Insomnium - Above The Weeping World
73) Kamelot - Epica
75) Porcupine Tree - Deadwing
79) Agalloch - Marrow Of The Spirit
80) Omnium Gatherum - Beyond
81) Ne Obliviscaris - Citadel
85) Mors Principium Est - The Unborn
86) Enslaved - RIITIIR
92) Seventh Wonder - The Great Escape
95) Riverside - Second Life Syndrome
101) Ayreon - The Theory Of Everything
107) Opeth - Damnation
109) Ayreon - 01011001
111) Haggard - Awaking The Centuries
117) Ghost Brigade - Isolation Songs
121) Insomnium - One For Sorrow
124) Opeth - Watershed
128) Moonsorrow - Voimasta Ja Kunniasta

Written by Nihil Aeternum on 13.09.2015 at 14:52

Hm, another interesting step might be to correct for the vote abuses by not counting votes that are less than x (with x=4?).

Which doesn't take into account the vote abusers that rate stuff with 10.

How much time you spend on it? while you did I was shagging your GF

Written by Moose on 13.09.2015 at 15:10

3) Agalloch - Ashes Against The Grain
...
128) Moonsorrow - Voimasta Ja Kunniasta

Wow, cool, thanks a lot! Your list also shows that more recent albums are completely under-represented in the "official" Top 200.

Written by Marcel Hubregtse on 13.09.2015 at 15:36

Written by Nihil Aeternum on 13.09.2015 at 14:52

Hm, another interesting step might be to correct for the vote abuses by not counting votes that are less than x (with x=4?).

Which doesn't take into account the vote abusers that rate stuff with 10.

Yes, that's right. A better way would be to delete the accounts that are just generated for vote abuse, at least the obvious ones that have only 10 and 1 ratings. Another solution against vote abuse would be even better and probably also simple to implement for MS: just do not count votes from accounts that exceed a given standard deviation. No account must be deleted, very (and I say very) strange voting behaviors just don't affect the rating of an album.

Written by Nihil Aeternum on 13.09.2015 at 17:21
just do not count votes from accounts that exceed a given standard deviation. No account must be deleted, very (and I say very) strange voting behaviors just don't affect the rating of an album.

That could work, but some vote abusers might stick to low or high rating for their duplicate accounts. So the standard deviation wouldn't be as high as you'd expect in all cases. Interesting and semi-effective idea tho!

Really interesting list, great job mate I'll be using this to see if I've missed anything

^ Or the voting system could be programmed so as to multiply every grade by a constant factor just like is done in this list? That way there wouldn't be that linear decline of the highest score...
Of course, nuking abusers would still be needed, since counting their votes plus scaling the grades would result into unfair scores for the top album candidates.

Both your reason for creating and rating system are top notch! Phenomenal post!

Written by LeKiwi on 13.09.2015 at 17:26

Written by Nihil Aeternum on 13.09.2015 at 17:21
just do not count votes from accounts that exceed a given standard deviation. No account must be deleted, very (and I say very) strange voting behaviors just don't affect the rating of an album.

That could work, but some vote abusers might stick to low or high rating for their duplicate accounts. So the standard deviation wouldn't be as high as you'd expect in all cases. Interesting and semi-effective idea tho!

Yes, I haven't thought about that. So it would be better to count the 10 votes and the 1 votes and divide their sum by the total number of votes. If the result is larger than about 0.75 it is clearly vote abuse (however voting with 9 and 2 would be a "workaround" and "new accounts" with only a few votes are more affected). So one would probably need a more complicated system with more "expert knowledge". However, as long as you do not delete accounts but just do not consider their votes, the worst case is not that bad and fair-minded users will not be disgusted.

Would be interesting to extend your analysis back to the early days of metal and make a de-biased Top 200. But you have probably better ways to spend your free time ;-) Looking at your list, I may have missed some great releases. Your list is already a great service for everybody in the community who is looking for the best albums. Much appreciated.

Great idea, but I can definitely see this is a hard concept to perfect, given the number of variables which could be considered. However as someone who gets rather infuriated by this growing mentality of short-sighted vote abuse, I absolutely love how you've decided to go about executing this.

Written by deadone on 15.09.2015 at 04:18

The reasons for this intrigue me immensely.

In the last days, I have made myself some thoughts about the decrease of the mean ratings during the last years, also considering your interesting attempts for explanation. I don't know whether my ideas are right or wrong, but maybe they are useful in this discussion. First, I don't think that the releases became worse in the last years, not at all. But ratings do not reflect the quality of a release but its acceptance in the whole(!) MS community. And this is where I think that the main problem lies. It's the diversification of styles, tastes, and metal sub-communities. When I remember back almost 30 years ago, almost every metal fan found Master of Puppets to be great. And even today's power metal fans say it is. At the same time, almost every metal fan found Helloween's Keeper II album to be great. And even today's thrash metal fans say it is. Look at thrash and power metal now. If a new great power metal album comes out (let's assume that this could be the case), power metal fans rate it 9 or 10. Thrash metal fans (and fans from other subgenres) rate it 8 or 7 or even lower. So, overall, it gets a quite low rating. But its the same visa versa. A great thrash album also gets a quite low rating because of low-voting power metal fans (and from the fans of other subgenres). However, it is fair for me to give such an album a low rating if I don't like it. Why should I rate an album that I do not like 9 or 10 just because it is one of the best of a subgenre I don't like so much? It's my personal and biased rating. But, this way, only albums that break the subgenre barrier, like Axioma Ethica Odini, manage to get high ratings nowadays. So, in short, the lower ratings are the result of a seemingly increasing fragmentation and incompatibility in substyles and subtastes. In my arguments above, you can replace thrash and power with any other substyle you like (or don't like). Overall, I don't think that this must necessarily be a problem (as vote abusers are), but it shows clearly that a correction / de-biasing of the charts might be useful, as Moose did it here. (But how could this become official MS policy?)

By the way, for the de-biasing I would use the top 25(?) albums average of each year with more than 100(?) votes and then find a function that describes the trend, to correct it afterwards. This way it becomes independent of singular great albums in singular years. Maybe I will give it a try during my next holidays. (If somebody gives me those yearly averages, however, I can find out the (current) de-biasing function quite rapidly. Extracting the numbers from the website is the hard work.)

This list is a nice idea, although as others have said it's probably not the most effective in dealing with vote abusers. Truth be said people have been complaining about vote abusers on this site for AGES. Maybe something should finally be done.

Guys check out RYM's way of doing it, simple as all hell. All Ivan would need to do is write a few lines of code (lol but I'm no programmer so I'm probably wrong).. But it looks simple enough
http://rateyourmusic.com/list/sharifi/weighting_faq/

Basically its an invisible weighting system new users have shit all weighting compared to a regular/older member/reviewer for example a new user's vote of 10 will be 10% as strong as an old regular's vote of 10. Also any users who have mainly 1's and 10's in their vote history will have shit all weighting.

So we could go all statistical with standard deviations etc. BUT what is even easier is a hidden weighting system, hardly any maths involved then - yet equally if not more so effective.

Although I do like that standard deviation idea.

Nitpick: Persefone have two albums on the list.

Otherwise, extremely good list. Essentially works out as a "21st century metal albums you must hear before you die" list that's unbiased and accurate. As much as album ratings are and should be about personal preference, and it's OK to not like whatever music you don't like (I tried Lateralus and didn't get it at all), I do believe that some albums are empirically very good, and I'm seeing a lot of those on here.

Suspended Animation by John Petrucci (2005) should be added. It has a rating of 8.96 from 80 votes but is not in the charts because the "band profile" is invisible. Right, who is John Petrucci?

Written by [user id=142921] on 22.09.2015 at 15:28


Sounds like a founder of a pasta brand.

I never liked that brand. Preferred John Frusciante pasta, personally.

Lot of great stuff on this list.

Written by [user id=4365] on 22.09.2015 at 15:37

I never liked that brand. Preferred John Frusciante pasta, personally.

Without the sauce on top, Petrucci pasta tastes quite different. And since it has 9 stars, it should be in the Guide Michelin ;-)

Written by Nihil Aeternum on 24.09.2015 at 10:57

Without the sauce on top, Petrucci pasta tastes quite different. And since it has 9 stars, it should be in the Guide Michelin ;-)

9 stars?! Does that mean it's 3 times tastier than food served at The Fat Duck, Noma or elBulli? I need to get me some of that.

Your "calculation" is absolutely absurb and not logical to me.
The average rating dropped to 8.26 is normal and just, it said that the quality of metal music has been dropping for the 10 years in between, there is nothing to do with "abusing votes". Please bear in mind that many albums are underrated in music industry due to the poor marketing plan and many are overrated because of manipulation caused by stupid commercial magazines. Members give higher ratings to some albums that you don't like would be considered abusing their votes ? Very wrong.

It's not about members not liking albums, but members giving a 1 to an album just to get a 'competing' album higher. For example, if you look at the votes on 'rust in peace' you'll find many of the people voting a 1 are also voting 'blackwater park' a 1 and both metallica albums a 10. It's quite unlikely that they think MoP and RtL are amazing albums warranting a 10 while another trash metal album of similar style is a 1 and just so happens to be slightly above their 2 favorites in the top 200.

I think the ultimate rule of thumb is just to take any rating, including your own, with a grain of salt. Don't obsessed over any such list too much and remember music is not a sport or academic test; ratings and lists are ultimately futile and pointless in "measuring quality". Use them to introduce yourself to some music that is generally considered "good" - that's pretty much the only reason I look at the top albums lists.

As for the general trend of rating decline, I don't at all agree that it truly reflects a decay in the quality of new releases each year. For example, I think many would agree with me that 2015 easily kicks 2014's ass musically, but the ratings of top albums continue to decline; personally, my top favourite album of 2014 wouldn't even make it to my top 10 for this year, and there are still 3 months of highly anticipated releases to go!

Written by Dr. Strawberry on 24.09.2015 at 21:26

Your "calculation" is absolutely absurb and not logical to me.
The average rating dropped to 8.26 is normal and just, it said that the quality of metal music has been dropping for the 10 years in between, there is nothing to do with "abusing votes". Please bear in mind that many albums are underrated in music industry due to the poor marketing plan and many are overrated because of manipulation caused by stupid commercial magazines.

Are you saying that you think the rating decline is simply due to worsening quality of music?

The fundamental assumption of this list is that musical quality remains relatively consistent over time. So if you believe this to be untrue, then I suppose the list/calculation would seem illogical.

I think there is enough evidence to suggest something else is going on. Metalstorm was only launched in 2000, so I find it difficult to believe that the albums released in say 2000-2005 were subject to the same scrutiny and voting conditions as those of recent years. The calculation I've used is an admittedly 'rough' method of accounting for this.

I was perhaps hasty in labeling it as 'voting abuse', as Metalstorm seem to do a decent job of removing wrongdoers.

Written by Dr. Strawberry on 24.09.2015 at 21:26

Members give higher ratings to some albums that you don't like would be considered abusing their votes ? Very wrong.

I haven't singled out individual albums. I have factored the albums based on the average of the year they were released.
I'm not even a fan of Agalloch, but they are number 1 on the list. My musical tastes didn't have any influence.

Written by Demise on 24.09.2015 at 22:18

It's not about members not liking albums, but members giving a 1 to an album just to get a 'competing' album higher. For example, if you look at the votes on 'rust in peace' you'll find many of the people voting a 1 are also voting 'blackwater park' a 1 and both metallica albums a 10. It's quite unlikely that they think MoP and RtL are amazing albums warranting a 10 while another trash metal album of similar style is a 1 and just so happens to be slightly above their 2 favorites in the top 200.

Probably you're too obsessed with rating, there is no competition at all... , unless Metallica members came in here trying to sabotage Megadeth
Generally listeners/ metal fans gave some album a 1-4 was probably due to their personal taste/ preference (they really didn't like those music composition, even felt annoying...), nothing should be "rectified by third-party".
Making a list and call it A "Controversial" Top 100 better as there are many so called classic albums came with extreme ratings.