Hypothesis Testing with Numpy
By: Mathanraj Sharma
Tags: Hypothesis Testing, Numpy, Python, Data Science, Hypothesis

Howdy fellow Data Science enthusiasts, we all do hypothesis testing to infer the results from a sample data from a larger popular. The test helps us to find whether our Null Hypothesis is True or false.

First, let us understand when we do a hypothesis test. We do it when we can draw a conclusion neither from Graphical EDA nor by comparing summary statistics. In such situations, we simulate a hypothesis from the dataset we have. One way to do this is to permutate the dataset we have. Before moving further let us see how to do permuting

def permutation_sample(data1, data2):

#Generate a permutation sample from two data sets.

# Concatenate the data sets: data

data = np.concatenate((data1, data2)) # Permute the concatenated array: permuted_data

permuted_data = np.random.permutation(data) # Split the permuted array into two: perm_sample_1, perm_sample_2

perm_sample_1 = permuted_data[:len(data1)]

perm_sample_2 = permuted_data[len(data1):] return perm_sample_1, perm_sample_2

You can use this function in a “for loop” to produce a number of sample sets you need. Note, here I have written this function, inference the samples from two datasets. Tweak the function as you need.

Now since we know how to simulate the hypothesis using permutations lets test it. Remember testing a Hypothesis is an assessment of how reasonable the observed data are, assuming the Hypothesis is True. It is important to understand what about the data we are assessing and how we quantify it. The answer to this is Test statistics.

“ Test statistic is a single number that can be computed from the observed data and from the data we have simulated from the null hypothesis”. It is going to help us to compare what the hypothesis predicts and what we actually observed from the data. We can choose the test statistic which can help us to answer the question which we are trying to solve using hypothesis. If the hypothesis is true the samples’ test statistics and original data’s test statistic will be identical or the difference between them will be too small. Now let us see how we can use python to compute this test statistic replicates.

def draw_perm_reps(data_1, data_2, func, size=1):

"""Generate multiple permutation replicates.""" # Initialize array of replicates: perm_replicates

perm_replicates = np.empty(size) for i in range(size):

# Generate permutation sample

perm_sample_1, perm_sample_2 = permutation_sample(data_1, data_2) # Compute the test statistic

perm_replicates[i] = func(perm_sample_1, perm_sample_2) return perm_replicates

In the above function, you can tweak the number of data sets you are going to pass. In the for loop, for each iteration, it will compute permutated samples using the previous function we wrote and then computes the test statistic. You can pass any function (like np.mean(), np.var(), or your own function) which will compute your test statistic.

Then we plot those values in a histogram. Then we mark the test statistic value from original data on the histogram. If we tally up the area right to the original data’s test statistic, it will give us the p-value. It can be done python using,

P-Value, Red line represents the original data’s test statistics.

# Compute p-value: p

p = np.sum(perm_replicates >= test_statistic_of_original_data)/ len(perm_replicates)

For example, if we get a p-value 0.05, we can say that we are 95% confident that our hypothesis is true. i.e (100 -(0.05 X 100)).

P-Value is the probability of obtaining the value of your test statistic that is at least extreme as what was observed, under the assumption of the null hypothesis. Note, that p-value is not the probability of Hypothesis being True.

In summary, let us write down the Pipeline of Hypothesis testing.

Clearly, state the null hypothesis Define your test statistic Generate many sets of simulated data assuming the null hypothesis is true Compute the test statistics for each simulated data The p-value is the fraction of your sets for which the test statistic is at least as extreme as for the real data

Here I have shared a few explanations and python snippets on how to do hypothesis testing. Hope you find it helpful. Let me know your feedback.