Demystifying Confidence Interval and Margin of Error
By: Ratul Ghosh
Tags: Interview, Data Science, Towards Data Science, Mathematics, Statistics

I recently came across this survey by the Gallup International Association Corona Tracker

We all love a conspiracy theory! 36% globally blame foreign power for COVID-19. The margin of error for the survey is 3 at 95 per cent confidence level. More than 20,000 interviews were conducted globally for this survey.

I tried to find more about the Confidence Interval and Margin of Error but they seem to be quite hard to understand for someone without in-depth knowledge of statistics. The challenge is most of the explanations rely on a lot of jargon which a layman might not know about.

In this article, I would try to explain the concepts margin of error and confidence intervals without too much mathematics and fancy statistics terms.

Photo by Element5 Digital on Unsplash

Let say a news agency is trying to figure out what percentage of the population would vote for a political party A in the upcoming election. As it is almost impossible to interview the entire population, they randomly select a sample of say 100 people. Out of those 100 people interviewed, 43 are willing to vote for A.

Let’s convert the problem in mathematical terms. We are going to say, sampling someone who is going to vote for A is equivalent to sampling 1, and sampling someone who is not going to vote for A is equivalent to sampling 0. It means that out of those 100 samples 43 are 1 and the remaining 57 are 0.

Now let’s define some mathematical symbols:

Here you might wonder why the denominator for calculating the standard deviation is n-1 instead of n. This is done using Bessel Correction to remove the bias in the estimation of the population variance. Refer here for more information.

For a population with mean μ_p(not known)and standard deviation σ_p(not known), the Central limit theorem tells us the following:

If we take a sufficiently large number of random samples from the population with replacement, then the distribution of the sample means will be approximately normal.

The mean of the sampling distribution can be used to estimate the true population mean, μ_p

The sample standard deviation of the sampling distribution(σ_x) is given by σ_p/√n( i.e., the standard deviation of the population divided by the number of observations in the sample).

Important side note: The above estimate would only hold if our samples are random i.e. they represent the whole population.

To calculate σ_x, we need the standard deviation of the population(σ_p) which is unknown. Instead, we estimate σ_p using the standard deviation of the sample(σ_s) after removing the bias using Bessel correction.

Calculation of Confidence Interval(CI) and Margin of Error:

From the above distribution, let’s find an interval i.e. a range of values that we can be 95% certain contains the true population mean(μ_p, whose value is the same as the mean of the sampling distribution(μ_s)). It is not quite correct to say that there is a 95% chance that the population mean(μ_p) lies within the interval. The population mean(μ_p) has a fixed value which we don’t know. If we repeatedly perform the same experiment, it wouldn’t change the value of the population mean(μ_p). Therefore it isn’t correct to ask about the probability that the population mean(μ_p) lies within a certain range. In contrast, the confidence interval we compute depends on the sample collected. If we repeat the experiment, the confidence interval would almost certainly be different. So we should ask about the probability that the interval contains the population mean.

A 95% confidence interval simply means, out of repeated random samples, there is a 95% chance that the true population mean(μ_p)lies within the interval.

Confidence interval is the most widely used method of interval estimation in frequentist statistics and is often confused with credible interval, an analogous concept in Bayesian statistics. Refer here for a more technical explanation of the differences: What’s the difference between a confidence interval and a credible interval?

Let’s now find the value of z such that the area of the shaded region is 0.95. z is the number of standard deviations away from the sample mean, also known as the critical value.

Formulas for calculating CIs take the general form:

CI = Sample mean ± z value × Standard error of the mean (SEM) CI = Sample mean ± z value × (Standard deviation/√n)

If we are calculating the 95% CI of the mean, the z value to be used would be 1.96.

In our dummy example, the Margin of error is 0.098, or approximately 10%(1.96*σ_x ).

The above statement can be interpreted as “We’re 95% confident that the interval between 0.332(0.43–0.098) and 0.528(0.43+0.098), capture the true mean proportion of the population that would vote for A.” This implies that were the sampling process to be repeated over and over with random samples from the same population, then 95% of the calculated intervals would be expected to contain the true value.

Misconception: