20 Years of Music Reviews. What Has Changed?
By: Lionel Emilion
Tags: Music, Data Science, Data Visualization, Music Review, Data

Some of the questions we are hoping to answer:

How nostalgic are we really when it comes to music?

Do users and critics usually agree in their assessments?

What does the future of reviews look like?

Is the music industry gender biased?

In this part we will tackle the first of these and to start of, let’s ask a simple question:

What was the best year for music?

Let’s see what Pitchfork has to say about this. We plot the average album score on Pitchfork of the past six decades:

only years with more than 5 releases were considered

the top 10 years measured by average score

There it is: 1986!

Who knew the answer could be this simple…

It should be no surprise to most people that the highest average score is from a year before Pitchfork started operating online. Of course nobody in the Pitchfork team will spend hours listening to Eddie Murphy’s 1986 “Party All The Time” just to write an extensive review about how uninspiring the instrumentals were. The disc has been forgotten and should stay forgotten. (Actually…scrap that. Treat yourself. You deserve this)

On the other hand it is by no means a surprise, that at some point in their 20 year history, someone at Pitchfork has decided to give us their take on The Beatles’ white album and it also shouldn’t come as a surprise either, that it will score very highly on the list (a 10).

However unsurprising this result is, let’s visualize it:

plotting the nostalgia factor

There is a very clear trend to be seen here:

The longer the time between an album release and its review, the better the score is on average. Just like we suspected. If an album covered on pitchfork is over 30 years old it has an average score of about 8.5, whereas a new album’s average score is closer to 7.0. This trend is also clearly visible for albums released after 1999, when Pitchfork was already writing online reviews.

Can we thus conclude, that reviewers are biased towards older albums? Well… not quite (yet). The point we made earlier about reviewers being more likely to pick out a (subjectively) good old album to feature in their review as opposed to a bad one (often dubbed as “survival bias”) remains valid. But we now have strong evidence suggesting that this theory is indeed supported by the data we are given. This also explains, why many people tend to say that the best years in music lie behind us. Our judgement is subconsciously skewed by the disproportionate amount of positive reviews we see of past albums, as well as by our own selective memory of only “the greats”. These at least are two factors explaining the phenomenon.

To answer the question whether or not reviewers are really influenced by their own nostalgia, let’s have a look at one more graph. We want to find out how many perfect scores (10) have been given out to albums every release year. That way the results are not taking into consideration the proportion of bad albums and are not biased in that regard.

A few stats:

Out of the ten years with the most amount of “perfect albums”, seven years are from before 1999, while only three years are from after 1999, when Pitchfork started reviewing online.

On average a year from before 1999 has 2.44 perfect albums whereas a year from after 1999 only has 1.94

According to this measure 1977 was the best year for music

However, upon closer inspection we can see that many of the albums are actually compilations or re-releases of much older projects. When we sort by the original release date, the graph is even more skewed!

A few updated stats: