ANOVA Explained by Example. Manually Calculating an ANOVA Table…
By: Eric Onofrey
Tags: Wine, Statistics, Python, Visualization, Coding

ANOVA Explained by Example

Background

Image by Author

Analysis of variance, typically referred to as ANOVA, is something often glossed over in introductory statistics classes. With today’s technology, it’s easy to let the computer do all of the calculations and just look at the p-value at the end of a busy-looking table like the one above. In this article, I’ll go through what these numbers are and what ANOVA is actually doing.

The main idea behind ANOVA is to test if there is a statistically significant difference in a the measured means between subgroups of a sample. The concept dates back to around 1920 with a British statistician named Ronald Fisher, who was analyzing massive amounts of data on crop yields. Fisher used ANOVA to prove there was a significant difference in the average weight of potatoes when different fertilizers were used.

To demonstrate ANOVA in the present day, I’ll be looking at the prices of wine rather than the weights of potatoes. The data for this example is from Kaggle’s Wine Review dataset, which contains thousands of reviews on bottles of wine from WineEnthusiast. I’ve filtered it down to a subset of 3,000 reviews for wines from the United States, Italy, and France (1,000 per country) and will use ANOVA to test if there is a statistically significant difference in the average price between countries of origin.

A quick plot of the price distribution by country shows three pretty similar distributions, with the majority of prices around $25 a bottle.

Image by Author

Taking the mean of each country, we see that France has the most expensive wine on average, followed by Italy and then finally the US. But the question is, is this a genuine difference in price between countries or could this just be due to sample variation? This is where ANOVA comes into play.

wine_df.groupby('country').mean()['price'] country

France 37.242

Italy 35.286

US 33.776

Name: price, dtype: float64

ANOVA

The four steps to ANOVA are:

1. Formulate a hypothesis

2. Set a significance level

3. Compute an F-Statistic

4. Use the F-Statistic to derive a p-value

5. Compare the p-value and significance level to decide whether or not to reject the null hypothesis

1. Formulate a Hypotheses

As with nearly all statistical significance tests, ANOVA starts with formulating a null and alternative hypothesis. For this example, the hypotheses are as follows:

Null Hypothesis (H0): There is no difference in the average price of wine between the three countries; they are all the same.

Alternative Hypothesis (H1): The average price of wine is not the same between the three countries.

Note, this is an omnibus test, meaning if we are able to reject the null hypothesis it will tell us that a statistically significant difference exists somewhere between these countries, but it won’t tell us where it is.

2. Set a Significance Level

The significance level, or alpha, is the probability of rejecting our null hypothesis when it actually holds true. In other terms, it’s the probability of making a Type I error.

Typically, one should weigh the costs of making a Type I vs. a Type II error to determine the best alpha for an experiment, but for this toy example I’m just going to use the standard .05 for our α value.

3. Compute an F-Statistic

The F-statistic is simply a ratio of the variance between samples means to the variance within sample means. For this ANOVA test, we’ll be looking at how far each country’s average wine price is from the overall average price, and dividing that by how much variation in price there is within each country’s sample distribution. The F-statistic formula is below, which may look complicated until we break it down.

Image by Author

SSB = Sum of squares between groups. This is the summation of the squared difference between each group’s mean and the overall mean times the number of elements per group. For this example, we take the mean of each country’s wine price, subtract it from the overall mean, square the difference and multiply by 1,000 (the number of data points per country).

Image by Author

SSW = Sum of squares within groups. This is the summation of the squared difference between the group-mean and each value in the group. For France, we would take the mean price of French wine, then subtract and square the difference for each bottle of French wine of the thousand data points in that group.

Image by Author

DoFB = Degrees of freedom between groups, simply the number of groups minus 1. We have three different countries we are comparing, so the degrees of freedom here is 2.

DoFW = Degrees of Freedom Within Groups, simply the number of data points minus the number of groups. We have 3,000 data points and three different countries, so this is 2,997 for this example.

Dividing the sum of squares for a group by its degrees of freedom yields the mean squares for that group, and the F statistic is just a ratio of the mean squares between over the mean squares within.

Below I manually calculate these values in Python, and end up with an F-Statistic of ~4.07.

Sum of Squares Between: 6039.73

Sum of Squares Within: 2226089.55

Degrees of Freedom for SSB: 2

Degrees of Freedom for SSW: 2997

F-Statistic: 4.06567

4. Using the F-Statistic, compute a p-value

Once we have our F-statistic, we plug it into an F-distribution to get a p-value. You can find a table for these values in the back of any statistics book or there are far-easier online calculators that will do this for you. With our specific degrees of freedom, an F-Statistic of 4.07 yields a p-value .0172.

Image by Author

5. Compare the p-value and significance level to decide whether or not to reject the null hypothesis

Our p-value signifies that assuming the null hypothesis (all countries have the same mean price of wine) is true, there is roughly a 1.7% chance of seeing the data we have by sheer sampling chance. By setting our significance level, or alpha, at 5% before all of this, we said that we would be willing to accept a 5% chance of rejecting the null when it is true. Since our p-value is below our pre-determined significance level, we can reject the null hypothesis and say that there is a statistically significant difference in the mean price of wine between countries.

Remember that ANOVA is an omnibus test, meaning because we are able to reject the null we know that a difference exists among the average wine prices between countries, but not exactly where. For finding where the difference lies, we would then conduct hypothesis tests between two countries at a time.

Automatic ANOVA

Like with most things in life, Python has an intuitive solution for conducting ANOVA with the statsmodel library. The below code does all the calculations in this article and outputs a summary table complete with an F-statistic and p-value.

Image by Author

Thanks for reading, full code and data can be found on my GitHub page.