Demystifying statistics for Data Scientists — Central Limit Theorem
By: Sandeep Mellacheruvu
Tags: Statistics, Machine Learning, Analytics, Data Science

The Central Limit Theorem is perhaps the most important theorem in the development of modern statistics. It enables us to estimate the population characteristics using a sample dataset.

In this article, we will explore the workings of the theorem. We will also go through some python code to verify the central limit theorem using some visualizations.

Before deep diving into the theorem, let's understand some of the basic terminologies of statistics used in the theorem.

Note: You can find the code samples for this article at https://github.com/sandeepvja/statistics-repo

Basic Terminology

Population

Population represents all possible data items in a particular data set. In general, we don't have access to the full population.

Since we don’t have the population dataset, we can't accurately calculate the population dataset metrics like mean, variance etc. But using the Central Limit Theorem, we can estimate them (as closely as possible).

Example: The population data for Student Dataset represents all the students’ information in the world.

2. Sample

A sample is a subset of the entire population. We generally have access only to the sample.

Example: A student dataset of the University of Maryland is a sample from the whole population of student dataset.

3. Probability Distribution / Distribution

A probability distribution is simply a mathematical formula through which we can describe the entire dataset.

4. Gaussian Distribution

Gaussian Distribution (Normal Distribution) is a probability distribution which follows a bell shape similar to the image shown below. The distribution is perfectly symmetric around its mean.

Normal Distribution

Now let's try to understand the Central Limit Theorem,

Central Limit Theorem

Central Limit Theorem states that given a sufficiently large sample size from a population with a finite level of variance, the means of all samples from the same population will follow a Gaussian Distribution.

In short, the theorem states that from a Population dataset, if we take a lot of random samples of sufficiently large size, and take the mean of each of these samples, all those means will follow a Gaussian distribution.

This theorem is central to many of the statistical techniques that we will see in the later articles. But first, let's try to verify the Central Limit Theorem using python.

First, let's assume that we know the population from which we have to gather the samples from. For simplicity, let's take our population as numbers ranging from 1 to 6.

Now, let's create a python function for generating the samples and calculate their means, which takes the following arguments:

The size of the population The size of each sample to take from the population (sampling with replacement). Please take a look at this article if you want to understand sampling with replacement and without replacement The number of such samples to take

import pandas as pd def getSampleMeans(population_size, sample_size, num_samples):



population_range = range(1, population_size + 1)



population_data = pd.Series(population_range) sample_means = [population_data.sample(sample_size, replace=True).mean() for i in range(num_samples)]



return sample_means

Let's also create another function which creates a histogram from the sample means generated by the above function using plotly

def plotHistogram(sample_means):

# Import plotly

import plotly

import plotly.plotly as py

import plotly.graph_objs as go

data = [go.Histogram(x=sample_means)]

plotly.offline.init_notebook_mode()

plotly.offline.iplot(data, filename='jupyter/Normal')

The last two lines of the code snippet above are needed for the jupyter notebook to show the plotly visualization.

Now, according to the Central Limit Theorem, the sample means from the same population will follow a Normal Distribution.

Let's assume a population of numbers ranging from 1 to 6. And take 1 sample each time and take a mean of those samples and plot the histogram.

sample_means = getSampleMeans(population_size = 6, sample_size = 1, num_samples = 10000)

plotHistogram(sample_means)

The above code will plot a histogram similar to below plot

Histogram with sample size 1 for a population of 6

But wait !! This doesn’t look like a normal distribution !!

Why doesn't Central Limit Theorem work?

Notice that in the theorem definition above, the statement clearly specifies, sufficiently large sample size from the population.

We took only one sample from the population, and since it is insufficient, it is not working.

Now lets try to increase, the sample size to 2 and the code below,

sample_means = getSampleMeans(population_size = 6, sample_size = 2, num_samples = 10000)

plotHistogram(sample_means)

Then we will see a histogram similar to below.

Histogram with sample size 2for a population of 6

Now, this looks slightly better and closer to the Normal Distribution curve, but we can do much better with a higher sample size. Since we are taking sampling with replacement, we can take a sample size larger than the population size as well.

sample_means = getSampleMeans(population_size = 6, sample_size = 100, num_samples = 10000)

plotHistogram(sample_means)

Histogram with sample size 100 for a population of 6

You can now see that the above sample means approximately converged with the normal distribution.

You can further vary the number of samples, size of the population or the number of rep’s and see how the sample means start converging into a Gaussian Distribution.

We will further explore the applications of Central Limit Theorem in the upcoming articles.