Sampling Distributions with Python | by Luís Roque | Medium | Towards Data Science

In a series of weekly articles, I will be covering some important topics of statistics with a twist.
The goal is to use Python to help us get intuition on complex concepts, empirically test theoretical proofs, or build algorithms from scratch. In this series, you will find articles covering topics such as random variables, sampling distributions, confidence intervals, significance tests, and more.
At the end of each article, you can find exercises to test your knowledge. The solutions will be shared in the article of the following week.
Articles published so far:
Bernoulli and Binomial Random Variables with Python
From Binomial to Geometric and Poisson Random Variables with Python
Sampling Distribution of a Sample Proportion with Python
Confidence Intervals with Python
Significance Tests with Python
Two-sample Inference for the Difference Between Groups with Python
Inference for Categorical Data
Advanced Regression
Analysis of Variance — ANOVA
As usual, the code is available on my GitHub.
We often find ourselves wanting to estimate a parameter for a population, for instance, its mean or standard deviation. Usually, we cannot collect data from the overall population. In this case, the only way for us to estimate the parameters of our population is by random sampling from it. We define the size of the sample n and calculate a statistic for each sample. This statistic is what we use to calculate the parameter of the overall population. Notice that the statistic that we calculate for each random sample could be far from the population value because it is a random sample. That is why we perform this process a series of times, which we can call trials.
Let’s define a straightforward example. We have a population of 100 athletes with numbered shirts with distinct values from 0 to 99. First, we calculate the parameters for our population, μ, and σ.
import numpy as npimport matplotlib.pyplot as pltfrom scipy.stats import bernoulli, norm, poissonfrom scipy import statsfrom scipy.stats import kurtosis, skewimport seaborn as snsX = np.random.choice(np.arange(0, 100), 100, replace=False)print(f'μ={X.mean()}')print(f'σ={X.std()}')μ=49.5σ=28.86607004772212
We take 100 random samples of size 5.
rs = []for i in range(100):    rs.append(np.random.choice(X, 5))rs = np.array(rs)# 5 examples of our random samplesrs[:5]array([[87, 73, 61, 91, 99],       [10,  5, 46, 72, 92],       [ 1,  3,  7, 59, 88],       [ 1, 42, 87, 21, 42],       [65,  9, 54, 88, 67]])
For each one, we calculate some statistic; in this case, the sample mean x̄. Thus, x̄ s an array of 100 values (the mean value of each sample). Let’s print the first 5 values and then plot a histogram to understand the sampling distribution's shape better. In fact, this is the sampling distribution of the sample mean for a sample size equal to 5.
x_bar = rs.mean(axis=1)print(x_bar[:5])plt.hist(x_bar, bins=100);[82.2 45.  31.6 38.6 56.6]
We will start this section by creating two Random Variables (RV), a Bernoulli RV and a Binomial RV (if you are unfamiliar with the details, please see my previous articles from this series).
The problem at hand is the following, there are 20,000 distinct universes where Rick and Morty live and we want to calculate the proportion of Mortys that we can find if we visit one random universe. Let X be the following Bernoulli RV:
We already saw in a past article that
and
Now, let’s define a new RV, which is equal to the sum of 10 independent trials of X. This is a Binomial RV with
Now it is time to visit some universes and look for Morty there randomly. Imagine that we visit 10 universes each time, which are indeed samples that we are taking from the overall population of universes. Notice that we can consider these samples independent even if we choose distinct universes for the sample of 10. This is due to a very small number of samples compared to the population. There is, in fact, a 10% rule to assume independence in a random sampling without replacement from a population of a certain size.
Time to look for Morty.
p = 0.6n = 10X = bernoulli(p)Y = [X.rvs(n) for i in range(10000)]plt.hist(np.sum(Y, axis=1));
And there you go, this is the sampling distribution of the sampling proportion. How can we calculate its mean and standard deviation? The expected value for your sample proportion is the proportion of Mortys that we find in our visits, showing an unbiased estimation for the population parameter. In the same way, the standard deviation of the sample proportion is the standard deviation of our binomial random variable Y divided by n.
print('Empirically calculated expected value: {}'.format(np.mean(np.mean(Y, axis=1))))print('Theoretical expected value: {}'.format(p))Empirically calculated expected value: 0.5977399999999999Theoretical expected value: 0.6print('Empirically calculated standard deviation: {}'.format(np.std(np.mean(Y, axis=1))))print('Theoretical standard deviation: {}'.format(np.sqrt(p*(1-p)/n)))Empirically calculated standard deviation: 0.15472844728749785Theoretical standard deviation: 0.15491933384829668
There you go; we arrive at very similar values. First, by performing a large number of trials of 10 visit tours to random universes. Secondly, by computing the theoretical values for this sampling distribution. In this article, we will understand exactly why these two approaches yield similar results.
There are two conditions to consider a sampling distribution approximately normal in its shape.
Let’s try an example. You ran a music store and received 100 new CDs every week (consider these to be a random sample). Your supplier states that they are delivering approximately 10% of Rock CDs. You calculate the weekly proportion of Rock CDs in the weekly sample.
n = 100p = 0.1print(n*p)print(n*(1-p))10.090.0
The conditions are met. Let’s plot it and look at the resulting distribution.
X = bernoulli(p)Y = [X.rvs(100) for i in range(10000)]normal = np.random.normal(p*n, np.sqrt(n*p*(1-p)), (1000, ))density = stats.gaussian_kde(normal)n_, x, _ = plt.hist(normal, bins=np.linspace(0, 20, 50),                    histtype=u'step', density=True) plt.close()plt.hist(np.sum(Y, axis=1), density=True)plt.plot(x, density(x));
Looks good and approximately normal. What if instead of a 10% probability of receiving Rock CDs, we were informed that it changed to 3%?
n = 100p_ = 0.03print(n*p_)print(n*(1-p_))3.097.0
The first condition is not met.
X = bernoulli(p_)Y_ = [X.rvs(100) for i in range(10000)]normal = np.random.normal(p_*n, np.sqrt(n*p_*(1-p_)), (1000, ))density = stats.gaussian_kde(normal)n_, x, _ = plt.hist(normal, bins=np.linspace(0, 20, 50),                    histtype=u'step', density=True) plt.close()plt.hist(np.sum(Y_, axis=1), density=True)plt.plot(x, density(x));
In fact, our distribution is skewed to the right.
Why are these conditions important? These are relevant when we need to answer questions about the probability of sampling proportions. For example, returning to our example and considering the initial information of 10% probability of receiving Rock CDs. Consider that you find on your first trial that 12% of the CDs were Rock CDs. Assuming that the true proportion is informed by your supplier, what would be the probability that more than 12% of the sample you searched consists of Rock CDs?
We already know that the sampling distribution is approximately normal. So we need to build our normal distribution and compute P(p>0.12).
print('Empirically calculated expected value: {}'.format(np.mean(np.mean(Y, axis=1))))print('Theoretical expected value: {}'.format(p))Empirically calculated expected value: 0.09973800000000001Theoretical expected value: 0.1print('Empirically calculated standard deviation: {}'.format(np.std(np.mean(Y, axis=1))))print('Theoretical standard deviation: {}'.format(np.sqrt(p*(1-p)/n)))Empirically calculated standard deviation: 0.029996189024607777Theoretical standard deviation: 0.030000000000000002n = 100p = 0.10p_ = 0.12print(f'P(p>0.12)={1-norm.cdf(p_, p, np.sqrt(p*(1-p)/n))}')P(p>0.12)=0.252492537546923
In the first section, we have calculated the number o Mortys that we found in samples of our population of universes using two different approaches. The first by deriving the theoretical properties of a sampling distribution of sampling proportions. The second one by estimating the parameters using the statistics of the sampling distribution. In this section, we will understand better the second approach.
Let’s start by defining the population mean μ:
where N represents the population size or the 20,000 universes. The sample mean ȳ can be defined as,
where n represents the sample size or one of the simple random samples of size 10 that we have drawn from the overall population.
p = 0.6n = 10X = bernoulli(p)Y = [X.rvs(n) for i in range(10000)]# One random sampley_bar = np.mean(Y[0])y_bar0.9
Notice that ȳ is a statistic that we use to infer the population parameter. Nevertheless, these could be different as we are randomly taking samples from the population.
We want to get the intuition on the Central Limit Theorem (CLT), and then we will get to its interest and applications. But, first, let’s define a clearly non-normal distribution.
elements = np.arange(6)probabilities = [0.3, 0., 0.05, 0.05, 0.2, 0.4]X = np.random.choice(elements, 100, p=probabilities)sns.histplot(X, stat='probability');
Now, we can draw samples from it. We are going to draw samples of size 4 and calculate its mean.
s_1 = np.random.choice(elements, 4, p=probabilities)print(s_1)x_bar_1 = np.mean(s_1)print(x_bar_1)[0 5 4 0]2.25
Let’s do the same procedure 10,000 times.
s = []n=4for i in range(10000):    s.append(np.random.choice(elements, n, p=probabilities))s = np.mean(np.asarray(s), axis=1)print('Kurtosis: ' + str(np.round(kurtosis(s),2)))print('Skew: ' + str(np.round(skew(s),2)))print('---')print('μ=' + str(np.round(np.mean(s), 2)))print('σ=' + str(np.round(np.std(s), 2)))sns.histplot(s, stat='probability');Kurtosis: -0.36Skew: -0.27---μ=3.06σ=1.06
The distribution above does not look like our original distribution anymore. The CTL tells us that as you take more samples and calculate their mean, the resulting distribution will approximate a normal distribution. We can measure it by the value of the skewness and kurtosis, which for a normal distribution should be zero. A positively skewed distribution has a tail to the right, while a negative one has a tail to the left. If the distribution has positive kurtosis, it has fatter tails than the normal distribution; conversely, the tails would be thinner in a negative scenario.
What the CLT also tells us is that the approximation becomes better as the sample size increases. So let’s test it by increasing the sample size from 4 to 20 and then to 100.
s = []n=20for i in range(10000):    s.append(np.random.choice(elements, n, p=probabilities))s = np.mean(np.asarray(s), axis=1)print('Kurtosis: ' + str(np.round(kurtosis(s),2)))print('Skew: ' + str(np.round(skew(s),2)))print('---')print('μ=' + str(np.round(np.mean(s), 2)))print('σ=' + str(np.round(np.std(s), 2)))sns.histplot(s, stat='probability');Kurtosis: -0.1Skew: -0.09---μ=3.06σ=0.47
s = []n=100for i in range(10000):    s.append(np.random.choice(elements, n, p=probabilities))s = np.mean(np.asarray(s), axis=1)print('Kurtosis: ' + str(np.round(kurtosis(s),2)))print('Skew: ' + str(np.round(skew(s),2)))print('---')print('μ=' + str(np.round(np.mean(s), 2)))print('σ=' + str(np.round(np.std(s), 2)))sns.histplot(s, stat='probability');Kurtosis: -0.08Skew: -0.03---μ=3.05σ=0.22
Notice how the last plot resembles a normal distribution. There is a convention that a sample size bigger than 30 is enough to approximate a normal distribution. Also, notice that the value of the mean and standard deviation of the sampling distribution that we have been plotting for the different sample sizes. Not surprisingly, the mean is the same as the original distribution. But the value for the standard deviation is indeed strange, as it has been shrinking. In fact, this is an important property to take note of: the standard deviation of our sampling distribution of sample means is the square root of the standard deviation of the original population divided by the sample size. Thus, we can write it as:
print("Theoretical value: " + str(np.sqrt(X.var()/n)))print("Empirically computed: " + str(s.std()))Theoretical value: 0.20789179878003847Empirically computed: 0.2151977077573086
This process is not specific to the sample mean; we could be, for instance, calculate the sample sum.
s = []for i in range(10000):    s.append(np.random.choice(elements, 100, p=probabilities))s = np.sum(np.asarray(s), axis=1)print('Kurtosis: ' + str(np.round(kurtosis(s),2)))print('Skew: ' + str(np.round(skew(s),2)))sns.histplot(s, stat='probability');Kurtosis: -0.04Skew: -0.02
I hope that now you see why the normal distribution is so often used to model different processes. Even if you do not know the distribution of a process that you want to describe statistically, if you add or take the mean of your measurements (assuming that they all have the same distribution), suddenly you get a normal distribution.
The problem that we faced in this article is the estimation of parameters of a population. Very often we can not collect enough data to represent the entire population and so, we need to find another way to estimate its parameters. The method that we explored draws random samples from the population. From the random samples, we calculate statistics which we afterward use to infer the parameters of the population. We looked at sampling distributions of sampling proportions and sampling distributions of sample means.
We also explored the CLT. It tells us that independently of the shape of the original distribution of a process that we want to describe statistically, the mean or sum of samples taken from this original distribution will approximate a normal distribution for a sample size larger than 30.
You will get the solutions in next week’s article.
Rick takes an SRS of 75 citizens on a distant planet to see what proportion of citizens sampled are satisfied with their standard of living. Suppose that 60% of the 1,000,000,000 citizens who live on the planet are satisfied with their living standards. What are the mean and standard deviation of the sampling distribution of the proportion of citizens who are satisfied with their standard of living?A certain planet with over 1,000,000 households has a mean household income of $1,000,000 with a standard deviation of $150,000. Rick plans to take random samples of 700 households and calculate the sample mean income. Then, calculate the mean and standard deviation of the sampling distribution of the sample mean.Rick is performing quality control tests on different portal guns since there is some variability in the manufacturing process. For example, a certain gun has a target thickness of 5mm. The distribution of thicknesses is skewed to the right with a mean of 5mm and a standard deviation of 1mm. A quality control check on this part involves taking a random sample of 35 points and calculating the mean thickness of those points. What is the shape of the sampling distribution of the sample mean thickness? What is the probability that the mean thickness in the sample is within 0.2mm of the target value?
Rick takes an SRS of 75 citizens on a distant planet to see what proportion of citizens sampled are satisfied with their standard of living. Suppose that 60% of the 1,000,000,000 citizens who live on the planet are satisfied with their living standards. What are the mean and standard deviation of the sampling distribution of the proportion of citizens who are satisfied with their standard of living?
A certain planet with over 1,000,000 households has a mean household income of $1,000,000 with a standard deviation of $150,000. Rick plans to take random samples of 700 households and calculate the sample mean income. Then, calculate the mean and standard deviation of the sampling distribution of the sample mean.
Rick is performing quality control tests on different portal guns since there is some variability in the manufacturing process. For example, a certain gun has a target thickness of 5mm. The distribution of thicknesses is skewed to the right with a mean of 5mm and a standard deviation of 1mm. A quality control check on this part involves taking a random sample of 35 points and calculating the mean thickness of those points. What is the shape of the sampling distribution of the sample mean thickness? What is the probability that the mean thickness in the sample is within 0.2mm of the target value?
def geomcdf_1(p, x):    # implementing first approach    prob = 0    for i in range(x-1):        prob+=p*(1-p)**i    return probdef geomcdf_2(p, x):    # implementing second approach    prob = 1-(1-p)**(x-1)    return prob
You have a standard deck of cards, and you are picking cards until you get a Queen (you replace the cards if they are not Queens). What is the probability that you need to pick 5 cards? And less than 10? And more than 12?
You have a standard deck of cards, and you are picking cards until you get a Queen (you replace the cards if they are not Queens). What is the probability that you need to pick 5 cards? And less than 10? And more than 12?
# Exactly 5p = 4/52p**1*(1-p)**40.055848076855744666# Less than 10geomcdf_2(p, 10)0.5134348005963145# More than 121 - geomcdf_2(p, 13)0.3826967066770909
2. Jorge conducts inspections on freezers. He finds that 94% of the freezers successfully pass the inspection. Let C be the number of freezers Jorge inspects until a freezer fails an inspection. Assume that the results of each inspection are independent.
# Our probability of success is actually the probability of failing the inspectionp=1-0.94p**1*(1-p)**40.04684493760000003
3. Pedro makes 25% of the free kicks shots he attempts. For a warm-up, Pedro likes to shoot free kick shots until he makes one. Let M be the number of shots it takes Pedro to make his first free-kick. Assume that the results of each shot are independent. Find the probability that it takes Pedro fewer than 4 attempts to make his first shot.
p=0.25# We can use our two functionsgeomcdf_2(p, 4)0.578125geomcdf_1(p, 4)0.578125# which are computing1-0.75**30.578125# and0.25+0.25*0.75+0.25*0.75**20.578125
4. Build a function that computes the Poisson PMF without using any functions from external packages besides np.exp from numpy. Choose some parameters and compare your result with the pmf function from scipy.
def fact(k):    k_ = 1    for i in range(1, k+1):        k_ *= i    return k_def poisson_pmf(k, λ):    return np.exp(-λ)*λ**k/fact(k)poisson_pmf(1, 2)0.2706705664732254poisson.pmf(1, 2)0.2706705664732254
5. Build a function that computes the Poisson CDF without using any external package. Choose some parameters and compare your result with the cdf function from scipy.
def poisson_cdf(x, λ):    p = []    for i in range(x+1):        p.append(poisson_pmf(i, λ))    return sum(p)poisson_cdf(2, 2)0.6766764161830635poisson.cdf(2, 2)0.6766764161830634