Bootstrapping for Inferential Statistics | by Samarth Agrawal | Towards Data Science

Bootstrap is a powerful, computer-based method for statistical inference without relying on too many assumption. It’s just magical to form a sampling distribution just from only one sample data. No formula needed for my statistical inference. Not only that, in fact, it is widely applied in other statistical inference such as confidence interval, regression model, even the field of machine learning.
In this article we will primarily talk about two things
Building Confidence IntervalsHypothesis Testing
Building Confidence Intervals
Hypothesis Testing
Link to the github for the code and dataset.
In real world — we don’t really know about our true population. For that it could be the entire population of planet or past, present and future transactions of a company. We just don’t know the real value of parameter. So we rely on sampling distributions to infer something about the parameter for these large populations.
There are a lot of different names for hypothesis test and the way we build confidence intervals:
T-Test
Two sample t-test
Z-test
chi-squared Test
Bootstrapping approach can be used in place of any of these. We will walk through an example to see how will use a sampling distribution to build a confidence interval for our parameter of interest.
Objective : Let’s say we want to get an idea on what’s the mean height of the people who drinks coffee
We have a dataset of all the coffee drinkers. `coffee_full.csv`
coffee_full = pd.read_csv(‘coffee_dataset.csv’)
From that i will create a sample dataset of 200 datapoints. lets call it coffee_red. This because in reality we never get full population
coffee_red = coffee_full.sample(n=200)
Proportion of coffee drinkers in our dataset
[coffee_red[‘drinks_coffee’]==True].mean()---Output---0.595
Average height of coffee drinkers in our dataset
coffee_red[coffee_red[‘drinks_coffee’]==True]['height'].mean()---Output---68.119 inches
Let’s bootstrap from sample to create a confidence interval
# Let's first create an empty list for storing means of bootstrapped samplesboot_means = []# Let's write a loop for creating 1000 bootstrapped samplesfor i in range(1000):    bootsample = coffee_red.sample(200, replace=True)    bootsample_mean = bootsample.[bootsample[‘drinks_coffee’]==True]['height'].mean()    boot_means.append(bootsample_mean)boot_means = np.array(boot_means)
For a 95% confidence interval — we cut-off 2.5% from each side and then these values from the sampling distribution that will give us the range where we believe the parameter would be “with 95% confidence”
# we build 95% in the middle portionnp.percentile(boot_means, 2.5), np.percentile(boot_means, 97.5)---Output---(66.00, 67.59)
We can interpret these values as the bounds where we believe the mean height of all coffee drinkers in the population to be with 95% confidence
To visualize this, let’s look at the following graph
plt.hist(boot_means, alpha=0.7);plt.axvline(np.percentile(boot_means, 2.5), color='red', linewidth=2) ;plt.axvline(np.percentile(boot_means, 97.5), color='red', linewidth=2) ;
We are 95% confident that the mean height of all the coffee drinkers is between 66.0 and 67.59 inches
Let’s go back and see what the population mean actually was
coffee_full[coffee_full[‘drinks_coffee’]==True][‘height’].mean()---Output---66.44 inch
We can see that our original mean height is within our 95% confidence interval
There are many traditional approaches for building confidence intervals which are already built in python. Here’s a good stackoverflow link
import statsmodels.stats.api as smX1 = coffee_red[coffee_red['drinks_coffee']==True]['height'] X2 = coffee_red[coffee_red['drinks_coffee']==False]['height']cm = sm.CompareMeans(sm.DescrStatsW(X1), sm.DescrStatsW(X2))print (cm.tconfint_diff(usevar='unequal'))
Notice that the intervals for bootstrapping method and the built in using the traditional method are nearly identical
As a data scientist you have to first translate the questions into what it’s called hypothesis. Then you need to use the data to justify which hypothesis is likely to be true.
Let’s say we wanted to ask a questions whether the average height for all the coffee drinkers is greater than 70 inches.
Here we will have to define what are called as Null hypothesis and Alternate hypothesis. Null hypothesis is something that is assumed to be true by default even before we collect data. Alternate hypothesis is something that we are trying to prove using data
We can bootstrap a sample set of data and compute the sample mean again and again, build a sampling distribution and confidence interval to determine what are the reasonable values for the population mean with some level of confidence
means = []for i in range(10000):   bootsample = sample_df.sample(n=150, replace = True)   bootsample_mean = bootsample[bootsample['drinks_coffee']==True]   means.append(bootsample_mean['height'].mean())means = np.array(means)np.percentile(means, 2.5), np.percentile(means, 97.5)---Output---(66.00, 67.59)
We can see that our 95% confidence interval is (66.0 , 67.59). Value of 70 is beyond 95% confidence interval so it is safe to say that alternate hypothesis is not true. We fail to reject our null hypothesis
We start by assuming that null hypothesis is true. If we were to simulate the values closest to the null hypothesis we would know what the sampling distribution will look like
We are going to simulate from the normal distribution in this case
null_vals = np.random.normal(loc = 70, scale = np.std(means), size =10000)
70 is our hypothesized mean
Each of the simulated draws here represent a possible mean from the null hypothesis. We can now ask the question where the sample mean falls in this distribution
plt.hist(null_vals, alpha=0.7);plt.axvline(sample_mean, color = 'red', linewidth=2);
We can see that it falls far below this distribution from the norm and we don’t think it probably came from this null hypothesized value. Hence we can say that our null hypothesis is true
p_value = (null_vals > sample_mean).mean()p_value--Output--1.0
Large p-value suggest that we can not reject our null hypothesis. which means that our population mean is indeed less than or equal to 70
We calculate p value as the proportion of the simulated draws that are larger than our sample mean
Graphically p-value here indicates area under the curve for all the values greater than the highlighted red line. which is basically all the data
Please feel free to write your thoughts / suggestions / feedback
Quite an informative article on the foundations of Bootstrapping- Introduction to bootstrap method
Stackoverflow link for T-test-Confidence interval for t-test