Differences in variance. Data people love averages. They love…
By: Thomas Richardson
Tags: Variance, Data Visualization, Data Science

Differences in variance

An unappreciated source of insight in our data

Data people love averages. They love means, they love medians. Which countries are the happiest on average? What is the average life expectancy of someone with a rare disease? What is the median income in a country? What is the average price of an engagement ring?

Means and medians are great because they tell us what is generally going on with our data. If your data is normally distributed (and often if it’s not) then an average will tell you most of what you need to know. As a result, most of our statistical tests, like t-test, ANOVA etc are built to look at averages like the mean, and how they might differ between groups. But there’s another variable, staring us in the face, that doesn’t get nearly as much love: the variance. And it has much to teach us!

2 ways that groups can differ

Here are 2 groups of people, given some fictitious test. I’ve plotted their distributions:

I just generated some random, fake data in R and plotted it using the ggplot package (Image by author)

We see that the groups differ in their mean. This kind of difference would be clearly picked up by a t-test or regression. But there’s another way the groups could differ:

Image by author

Here, group 1 has a wider distribution than group 2. It has greater variance. A t-test or regression would not capture this; they’d only tell you the groups don’t differ on average. But as you can see, that doesn’t mean the groups aren’t different! I’ll show you how you can test for this below.

You might think you haven’t encountered this concept before. But you’ve probably heard of the GINI coefficient, right? It measures wealth inequality in a society. The concept is the roughly same: some societies might have the same average wealth, but in one society that wealth may be distributed unevenly. A society where everyone earned the exact same wage would have a variance of 0 after all!

Implications in real data: the case of sex and IQ

One of the few areas where variance differences have been investigated is in the area of IQ. Studies have generally found that while IQ differences between men and women are either small or non-existent, men show consistently higher variance in IQ compared to women. I illustrate this below with data I simulated using the real variances of a study of over 80,000 Scottish teens (what a dataset!). I also give the R code so you can do it yourself:

set.seed(140293) male = rnorm(5000,100,14.9)

female = rnorm(5000,100,14.1)

We set a specific seed so that you get the same random numbers as me, and as a result the same randomly generated data. I then create 2 sets of IQ data, 5000 males and 5000 females, both normally distributed with a mean of 100. The standard deviation of the male data is 14.9 and the female is 14.1. Remember that the standard deviation is the square root of the variance.

Next, we pull the data together into a tibble and reformat it so it’s easy to work with, which is easy using the tidyverse R packages:

install.packages('tidyverse')

library(tidyverse) IQ_data = tibble(male, female) %>%

pivot_longer(cols = c(male, female),

names_to = 'sex',

values_to = 'IQ') %>%

mutate(sex = factor(sex))

Then we make a frequency polygon to compare the distributions for males and females:

IQ_data %>% ggplot( aes(x = IQ, colour = sex) )+

geom_freqpoly(size = 1)

Image by author

The peak of the female distribution is higher than the males, indicating that there are more females scoring close to 100. The male distribution is ever so slightly higher at the tails, indicting that there are more males scoring extremely high and extremely low on IQ. But the differences seem small.

t.test(IQ ~ sex, IQ_data)

Running a t-test, there are no sex differences in mean IQ (p = 0.38). However, the groups might differ at the tails. Let’s investigate.

Tail ratios

We can calculate a tail ratio by taking the ratio of males to females above or below a certain threshold. We take all individuals with an IQ of 130 or higher (generally considered to be very smart) and count the number of males and females. We do the same for those with an IQ of 70 or lower (generally considered unintelligent).

IQ_data %>% filter(IQ >= 130) %>% group_by(sex) %>% count IQ_data %>% filter(IQ <= 70) %>% group_by(sex) %>% count

Running the code you’ll see there are 95 extremely smart males in our dataset and 66 extreme smart females (59% male). This gives a male: female ratio of 1.44 very smart males for every very smart female.

Likewise, there are 104 very unintelligent males in our data and 84 females (55% male), or a male to female ratio of 1.24. So even though males and females don’t differ on average, there are more males among the smartest, and least smart groups.

Going further: Levene’s test

Ratios are all well and good, but we might want to know if the difference is statistically significant. There are a few tests for this, but the most common is a Levene’s test for equality of variances, which I get from the car R package:

install.packages('car')

library(car)

leveneTest( IQ ~ sex, IQ_data)

We get an F value of 5.54 and a significant p value of 0.02, confirming our result that males have higher variance in IQ than women.

Implications

What other questions could we answer by comparing variance?

If a drug increases the average health of a patient, but also increases the variance of health among those that take it, that may signal the drug is helping some but not others.

Perhaps you have 2 products sold in a number of stores. The products may sell the same number of units per store on average, but one product might have higher variance in units sold, indicating that some stores sell many whereas others sell few. This could have implications for understanding your customers and where to sell your product.

Policies that do not affect the average happiness of people in a country may still be good for society if they reduce the variance of happiness, which we might call “happiness inequality”.

Often a particularly low variance can be a sign of a good product. Rory Sutherland likes to argue that the best sat-nav is one that minimises variance in journey times, not necessarily the one that can find the shortest average journey times. When going to the airport to catch a flight, you don’t want a route that could take between 30 minutes to 2 hours, even if it is the quickest route on average.

When investigating representation of women and minorities. E.g. the mean % of women in top managerial roles might be 50% across companies, which some would interpret as achieved gender equality. However, if there is a very large variance in this percentage between companies, it may indicate that many companies have far more than 50% female managers and other companies could have far fewer.

Conclusion

We’ve seen here that groups can differ in their variance as well as means, and that this can lead to new insights in our data. In my experience this really is a neglected area, so cool results might be relatively easy to find. It also doesn’t require any more data than you already have, which is nice. If you find any interesting variance differences, be sure to tag me on Twitter, so I can see them. I’m excited to see what you find!