Why ANOVA is Essential to Data Science with a Practical Example | by Betty LD | Towards Data Science

What you will learn from this article:
Why you should care about fundamental statistics.
Why ANOVA and not another method.
How to easily run ANOVA on your data with python (repo here).
Have you already had an uncomfortable feeling about the conclusion you reach from the data you collect? I did.
Even after cleaning, processing, comparing the data, I could hear this devilish voice in me: “Is your conclusion correct?”, “Is your data really representative or is it just lucky samples?”.
This recurrent “Is this data enough to draw a conclusion?” was constantly floating in my head.
I knew that my data created ideas but no truth. They were only opinions, nothing reliable enough to convince people — even not myself.
One day, I looked for help to get rid of this uneasy feeling, and run into this short video which changed my perspective:
Then I realized one thing, “you don’t always need more data, maybe you just need statistics”.
With statistics, you can control the uncertainty hidden behind your data. With statistics, you can know what the margin of error of your guess on the population made from the sample data is.
In this article, I won’t detail the following fundamentals:
probability distribution,
sampling distribution,
large and small sample estimation,
hypothesis testing.
I will dive directly into ANOVA: let’s start by why you should care about it.
You are a manager. Sales went great and your monthly budget increased. You want to improve the working conditions of the sales team and got an idea (Eurêka!): how about offering breakfast every Monday?
Wait a minute, there is a very important meeting every Monday morning, and you need everybody to be aware. Since you hesitate about the food quantity you should provide, you decide to run an experiment and find which breakfast fits the best by yourself.
One group of 15 employees was randomly assigned to each of the three breakfast types:
No breakfast (5 people)
Light breakfast (5 people)
Full breakfast (5 people)
After eating, their attention span in minutes is recorded during the morning meeting.
In academic statistics, we call this experiment a “single factor experiment”. The factor of interest is the meal type which has three levels (no, light, full). Since it is a single factor experiment, each level can be called a treatment because they represent the three populations we investigate.
Collect the results (15 measurements in total) in a pandas DataFrame (full notebook here)
import pandas as pdresp_var = 'attention_span'treatments = "meal_quantity"experiment = {'No breakfast' : [8,7,9,13,10],             'Light breakfast' : [14,16,12,17,11],              'Full breakfast' : [10,12,16,15,12]}exp_df = pd.DataFrame(experiment)exp_df.index.name = "observation"exp_df
Compute the mean, standard error, and the confidence interval for each treatment with numpy.mean, scipy.stats.sem, scipy.stats.t.interval or use directly researchpy library:
import researchpy as rprp.summary_cont(exp_df)
Reminder: the Standard Error (SE) is the standard deviation (SD) of the sample mean (often called std). It measures how far the sample mean of the data is likely to be from the population mean. It measures how much discrepancy there is likely to be from your sample compared to the population mean [Investopedia]. The Central Limit Theorem approximates it to the ratio of standard deviation divided by the square root of the sample size.
Then, visualize your data to get insights:
1. A plain histogram of the data
exp_df.plot.hist()
2. A kernel density estimation
Smoothen histogram with a mix of Gaussian distributions to better visualize the data shape:
import seaborn as snsstack_df = exp_df.stack().reset_index()stack_df = stack_df.rename(columns={'level_0': 'id',                                    'level_1': 'treatment',                                    0:'span'})graph = sns.displot(stack_df, x='span', hue='treatment', kind="kde", fill=True)
3. Boxplots:
One of the most reliable ways to compare the data at a glance:
ax = sns.boxplot(y=stack_df["span"], x=stack_df["treatment"])ax.set_title('Boxplot')
Ok. From these samples made of the attention span of 5 people for each breakfast type, you can draw a rough conclusion about all your employees: they are more attentive when they eat breakfast rather than when they don’t. You can clearly see on the boxplot that the average attention span is obviously higher for “light” and “full breakfast” than for “no breakfast” in blue.
Now, you should ask yourself:
Is this sample enough to draw any conclusion about all the employees?
What is my level of confidence in that conclusion?
Reformulated in statistical terms:
Are the responses collected from these 15 experimental units giving me some meaningful insights or confirming an initial hypothesis?
How these three different levels of factors of interest affect the average response?
ANOVA answers partially all these questions since the “ANalysis Of VAriance” judges the effect of one factor (meal quantity) on the experimental response (attention span).
I don’t want to use ANOVA, can’t I just compare the means?
I don’t want to use ANOVA, can’t I just compare the means?
The mean is the most honest (but naive estimation) you can make about the population you study from the data you collected. The pitfall is illustrated below, where the distribution of Figure 1 and Figure 2 have the same means (0, 9, 16) but their variance differs (small variance for Figure 1 and large variance for Figure 2).
In Figure 1, you can say that the population means of the three categories are different with pretty high confidence, but that is not the case for Figure 2 where the data is more spread out, even if the means in both figures are exactly the same. In the case of Figure 2, each population means could be anywhere inside its distribution and since the distributions overlap each other, we cannot know which mean is higher.
2. Can’t we just use Z or t-test hypothesis testing?
No, because of three reasons:
- If you just compare two means at a time, you waste the information from the other means in the study.
- When you compare two means at a time and make all pairwise hypothesis testing, the probability of rejecting the null hypothesis when it is true increases. This is because the more t-tests are conducted, the greater is the likelihood of getting a false significance [2].
- If you need a test for each pair, for 5 means, you would need 10 tests.
Conclusion: if you have plenty of time and good luck on your side, you can do hypothesis testing. Otherwise, run an ANOVA.
I just spoke about Z and t-test statistics, so I will insert a quick reminder about about them.
The test statistic is a tool used to make a conclusion about a population estimator (mean, variance, etc) from a sample and estimate the error coming along with this estimator (because it is very rare to be able to collect all the population data).
To do that, you postulate an initial guess (alternative hypothesis Ha) which goes against the general opinion (null hypothesis Ho) and check if:
- The data confirms your hypothesis.
- The data rejects your hypothesis.
The hypothesis testing tells you if there is enough evidence in your data to make a conclusion (alternative hypothesis) over the current opinion. More particularly it indicates how small or large the difference should be between the current opinion estimator (Ho) and the estimator from your data to reject the current opinion (Ha).
In the analysis of variance, we use the F-distribution which is the distribution used for comparing two population variances, and the hypothesis testing focuses on the equality of the means.
In this article, we explore one way (one factor) ANOVA.
But if you think that one way is too boring, you can switch to a two way ANOVA by increasing the number of factors of interest and study each of them separately and their interaction.
For example:
Factor 1: the meal quantity (three levels: no, light, full).
Factor 2: the employee genders (two levels: women or men).
We have 2 factors so it becomes a two-way ANOVA (3x2). But we won’t cover it in this article.
Samples are randomly and independently selected from their respective normal population.The populations are distributed with equal variance.
Samples are randomly and independently selected from their respective normal population.
The populations are distributed with equal variance.
1. Check if your data is mount shaped with the display techniques shown above, and if the mount shape is not obvious, ensure at least that the samples’ sizes is equal [1, p.427]. Or run normality of variances tests like:
Q-Q plot, to show the distribution of the data against the expected normal distribution. If the distribution is normal, dots should follow the line.
Shapiro test tests the following null hypothesis: the data is normally distributed. If p-value is less than 0.05, you can reject the hypothesis and the assumption of the normality of your data.
2. Run variance homogeneity tests:
Levene test for the equality of variance. It tests the null hypothesis that the variances of the treatment samples are equal.
Now we know: why we do ANOVA, and ensured that we can do ANOVA, let’s do it!
At this point, you understand that ANOVA is a tool to compare the means of k different treatments, answering the following questions:
Are the k means all the same? Or at least one is different?
If one is different, with how much confidence can I assert it?
In our example, we have three levels for breakfast so k=3, and the population means are mu_i for i=1,2,3. It gives the hypothesis:
The question:
Does the meal quantity affect the average attention span of my employees?
becomes:
Do my sample data show that at least one type of breakfast gives a better average attention span?
To answer these questions, we will compare the mean (mu_i) of the attention span for each breakfast, taking into account their variability (sigma_i).
To know the reliability of our conclusion, we need to understand the amount of variability in the data. In the analysis of variance, you divide the total variation in the response measurement (the attention span) into portions that may be attributed to various factors of interest (the meal quantity). This total variation is called Total Sum of Square TSS:
with
xij the jth measurement (j is between 0=1 and 5) in the ith sample (we have 3 samples),
x\ the mean of the sample i.
TSS can be decomposed into:
Between-group variance represents the variance of the means.
Within-group variance represents all the data variance respectively to its sample mean.
With
i the index of the sample (from 1 to k=3) for (no, light, full) respectively,
ni the number of observation in the i sample,
xi\ the mean of the sample i,
x\ the mean of all observation,
si the standard deviation of the sample i.
Then you convert the Sum of Squares (SS) into an estimate of the variation called the Mean Square (MS), by dividing it by its degree of freedom.
The degree of freedom is defined by the number of observations minus one. Therefore, the degree of freedom are respectively:
n-1 for TSS (total), because the total sum involves n observations.
k-1 for SST (treatment), because the sum square for treatments involves k squared observations of sample means.
n-k for SSE (error), because the sum square for error has (n_1–1) + (n_2 -1) + ... + (n_k-1) observations.
In our example, n-1 is 14 degrees of freedom, k-1 is 2 degrees of freedom and n-k is15–3=12 degrees of freedom.
Finally, we use the ratio of MST/MSE as the F-test statistic.
1.Chose a Level Of Confidence, LOC=1-alpha% (often alpha=0.05 and LOC=95%).
alpha = 0.05dof_mse = n-kdof_mst = k-1F_alpha = scipy.stats.f.ppf(q=1-alpha, dfn=dof_sst, dfd=dof_sse)
Here: F_0.05= 3.885.
2.Test statistic: compute F.
F = 4.932584...
3.Rejection region: reject Ho if F > F_alpha.
print('We reject Ho!') if F_alpha<F else print('We fail to reject the null hypothesis')
In practice, run directly:
We obtain the ANOVA table:
From the output ANOVA table, you can read the conclusion: our test statistic F is about 4.93 and F_0.05 is about 3.89, therefore F>F_0.05 and we can reject the null hypothesis with a level of confidence of 1–0.05= 95%. It means that the three average attention spans are different from at least one of the others.
You can also use the p-value and reject the null hypothesis if p < 0.05. Here, p=0.027 < 0.05 therefore we reject the null hypothesis and reach the same conclusion.
The conclusion from ANOVA: breakfast type matters on the employee attention span since the average attention span differs for at least one of the meal.
Because thanks to ANOVA, we know that the average attention span can be ranked, the next step is to know if we can say that a light breakfast is better than a full breakfast with 95% confidence using Scheffe or Tukey test.
The article is dense, but running ANOVA is pretty fast-forward. I built a library to run it from any dataframe. Few lines of code will do all the operations (and more) shown in this article:
Check the repo here, clone it, install it in a virtual environment, and run as many ANOVA you want!
“If you want to find the most sold item among these products, you should take into account the variability when comparing the mean.”
“ANOVA tells you if there is a difference, but not where the difference lies.”
“You better check if your sample data is at least shaped mounted and run a few tests of variance equality.”
“I am feeling anxious now. I run an ANOVA but I am afraid to look at the results... Imagine if I cannot draw any conclusion from the data I just collected?”
[1] Mendenhall, W., Beaver, R. J., & Beaver, B. M. (2006). Introduction to probability and statistics. Belmont, CA: Thomson/Brooks/Cole.
[2] Triola, Mario F, and Mario F. Triola. Elementary Statistics with Multimedia Study Guide. Boston: Pearson Addison Wesley, 2008. Print.
[3] Investopedia, Standard Error of the Mean vs. Standard Deviation: The Difference.