What is Central Limit Theorem
By: Chetan Ambi
Tags: Machine Learning, Statistics, Central Limit Theorem, Artificial Intelligence, Data Science

The Central Limit Theorem is one of the important theories in Statistics and Probability. In this article, you will understand what is Central Limit Theorem, why it is important, what it tells us, and how it used, etc.

Central Limit Theorem (CLT)

The Central Limit Theorem states that “given a dataset with unknown distribution, the sample means will approximate the normal distribution”.

What does this mean?

Let’s consider a population of size m. You don’t know the distribution of the population. It can be normal, binomial, exponential, uniform, or some random distribution. But if we were to draw n samples with sample size, say 50, the distribution of means of these samples (or simply, sampling distribution) will approximate a normal distribution.

Let’s look at an example. In the below example, population follows a uniform distribution . When you plot the sample means, the central limit theorem kick-in and follows the normal distribution.

Assumptions of Central Limit Theorem

Random Sampling: Samples must be chosen randomly.

Samples must be chosen randomly. Independence Assumption: Samples should be independent of each other. If the samples are chosen randomly we can assume that the independence assumption is met.

Samples should be independent of each other. If the samples are chosen randomly we can assume that the independence assumption is met. 10% Condition: The sample size should be no more than 10% of the population when sampling is done without replacement.

The sample size should be no more than 10% of the population when sampling is done without replacement. Sample Size Assumption: The sample size should be sufficiently large enough. If the population data is skewed, then the sample size should be large. If the symmetric then the sample size can be small.

Why Central Limit Theorem is important?

Why should you study Central Limit Theorem? How it will be useful to you. Let’s try to understand this with the help of these two important properties of CLT.

Normality Assumption: This is the very definition of CLT that ‘given a dataset with unknown distribution, the sample means will approximate the normal distribution’. As the sample size increases, the sampling distribution converges more and more closely to a normal distribution. This assumption holds good even when the unknown data is not normally distributed. As we already discussed, the normal distribution has very interesting properties that help you solve many statistical hypotheses.

Precision of Estimates: If you were to draw sampling distribution with varying the sample sizes you will notice that, as sample size increases, the sample mean will come closer to the real population mean. In other words, as the sample size increases the estimate* of the population mean will be more precise. This also means that the smaller the sample size, the less accurate or precise your estimation of the population mean.

*Note: We almost always don’t know about the population parameters. So, we always estimate the population parameters i.e. sample mean, sample standard deviation.

Conclusion

Hope you have understood what is Central Limit Theorem and why it is important.