World Happiness Reports make this Data Scientist Unhappy
By: Jamie Ferreira
Tags: United Nations, Happiness, Data Science, Tableau, Statistics

Every year the United Nations release the World Happiness Report leading readers to believe that they have successfully ranked all countries by happiness. These reports always capture the attention of the media and is most certainly captivating, but the reality is that these ‘rankings’ are built on statistical misuse.

The happiness index for each country is calculated by asking citizens:

‘Please imagine a ladder with steps numbered from 0 at the bottom to 10 at the top. The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you. Which step of the ladder would you say you personally feel you stand at this time’

And then averaging the responses to get a score between 0 and10.

This approach has already drawn some concerns from a philosophical standpoint, with some arguing that this line of questioning leads responders to overvalue income and material possessions rather than happiness. I’ll leave that line of thinking to my friends in the social sciences and for the purpose of this article I will only go over the statistical shortcomings:

Shortcoming Number 1 — The Rankings are too precise

It’s just not feasible to survey every person in each country, so the next best thing is to survey a random subset of people and use those results to represent the entire population of the country. Although the average happiness of the random sample would closely represent the average happiness of the entire population (assuming the sample size is large enough) it will not be an exact representation. Statisticians get around this by reporting a 95% percent confidence interval (CI) which means that there is a 95% chance the true mean (in this case happiness) lies in that range. For many countries, their respective sample happiness scores are very close and there is a significant overlap in the 95% confidence intervals. Consider the following example:

Spain and Saudi Arabia 2020 Happiness Score

We can see that the happiness score are very similar for both Spain and Saudi Arabia and it would be very irresponsible to conclude that Spain is happier than Saudi Arabia or vice versa.

However, in order to achieve a ranking system, this inaccurate conclusion is repeated across many countries.

In this case Saudi Arabia is ranked above Spain despite there being no data supporting this conclusion.

This is an example of over precision, meaning that the level of accuracy associated with random polling is not enough to justify a ranking system.

A more appropriate use of the data would be to categorise each country as either thriving, struggling or suffering. This is what the Gallup (original organisation that ran the poll) concluded with the data.

To illustrate the effect over precision can have on the happiness rankings, I ran 1000 simulations where each country was randomly allocated a new happiness score (and consequently rank) based on the 95% confidence interval reported. I have linked below a dashboard I have created, where for each country you can view the lowest and highest possible rank of each countries based on these simulations, as well as the span between the lowest and highest rank.

Countries with a difference span less than 5

In the above visualisation, only countries with a difference between the highest and lowest rank in the simulations being less than 5 are coloured. We can see that this only covers a handful of countries which highlights how volatile and inconsistent these rankings are!

Shortcoming Number 2 — The happiness score is biased

Since the measure of happiness is based on quantifying qualitative data rather than pure quantitative measures, it opens the door for many biases that have the potential to derail any analysis. There are many underlying factors that influence someone’s response when you ask them to rate their life out of 10. For example, many Asian cultures have a tradition of humility whereas in Scandinavian cultures people tend to be more inclined to tell others how happy they are. This could have contributed to the high rankings of Nordic countries whilst Asians countries such as Japan (62nd) and South Korea (61st) are lower than expected.

These biases unfortunately are part of any analysis, especially those involving qualitative responses, however if they are properly understood and minimised than reliable conclusions can still be drawn. For example, these biases may limit the ability to compare happiness between countries but from the data we can still conclude that developing countries are generally less happy then developed countries.

These statistical shortcomings are enough to make any data scientist cringe. What would have been more appropriate with the data at hand would have been the identification of broad categories like what Gallup produced. The world happiness report is an example of how not fully understanding the limitations of statistical processes can result in misleading and unreliable analysis.

For my simulation code please see this GitHub Link: https://github.com/jama001/World-Happiness-Rankings

My name is Jamie Ferreira and I am an aspiring data scientist majoring in Data Science and Business Analytics at the University of Sydney. Feel free to leave any comments below and connect with me on LinkedIn: https://www.linkedin.com/in/jamie-ferreira-942374186/