Package: bootstrap-CIHT
bootstrap-CIHTA Python module to calculate confidence intervals and perform hypothesis tests on the mean or proportion of one or two groups using bootstrap sampling.Selected (hopefully useful) visualizations of the data and null distribution are provided viamatplotlib.IndexDescriptionParametersAttributesMethodsUsageDependenciesInstallationExampleAdditional TestsHistoryLicenseDescriptionclassbootstrap_CIHT.Bootstrap_CIHT(data, data_col, num_vars=1, null_mean=0.0, group_col=None, group1=None, group2=None, samples=10000, alpha=0.05, h_sides=2, h1_dir=None)Parametersdata (Pandas dataframe):Unprocessed user data. Should contain at least a column with the data to analyze. For
proportions, data should be coded as 0 or 1. If a two-group comparison, should also have
a column containing group membership labels.data_col (string):Name of column containing the data to be analyzednum_vars (int):Number of groups (1 or 2); default 1null_mean(float):For one-group test, comparison mean or proportion; default 0.0Should be left at 0.0 for two-group comparisons.group_col (string):For two-group comparison, name of column with group labelsgroup1 (int, bool, string):For two-group comparison, label for first (control) groupgroup2 (int, bool, string):For two-group comparison, label for second (experiment) groupsamples (int):Number of bootstrap samples; default 10000alpha (float):Sets size of confidence interval (100*(1-alpha)); default 0.05h_sides (int):One- or two-sided confidence interval & hypothesis test; default 2h1_dir (string):Inequality direction for 1-sided alternative hypothesis:'greater' -> experiment parameter > control parameter'less' -> experiment parameter < control parameterNone (default) -> 2-sided comparisonAttributes:df (Pandas dataframe):Dataframe derived from user-supplieddatafor use in analysisMethods:fill_data()Extract data for analysis from user-supplied dataframe. Performed automatically on instantiation.Args:NoneReturns:df (Pandas dataframe):Dataframe containing data to be analyzed ("data" column) and group membership ("group" column) for two-group comparisonget_bootstrap_sample()Create sampling distributions using bootstrapping.Args:NoneReturns:For one-group analysis:experiment_mean (numpy array):Bootstrapped means of the group of interestFor two-group comparison:experiment_mean (numpy array):Bootstrapped means for experiment groupcontrol_mean (numpy array):Bootstrapped means for control groupdiffs (numpy array):Differences in means of bootstrapped samplescalculate_CI(means)Compute and print 100*(1-alpha) confidence intervals for the relevant sampling distribution. For a one-group case, this is for the mean of the data. For a two-group case, it is for the difference in means.Args:means (numpy array):Contains either the bootstrapped mean (for a single group) or difference in means (for two groups)Returns:CI (tuple):Upper and lower confidence limitscalculate_hypothesis_test(means)Calculate p-value for a 1- or 2-side hypothesis test comparing the sample mean to the null mean. For two-group comparisons, difference in means is compared to a null mean of zero.Plots the observed statistic on a histogram of the null distribution.Args:means (numpy array):Contains either the bootstrapped mean (for a single group) or difference in means (for two groups)Returns:p_value (float):Probability of the observed statistic given the null hypothesisplot_hist_CI(means, CI, bins=10)Plot the sampling distribution with confidence intervals as vertical red lines. For the single-group case, also plots the null mean as a vertical green line.Args:means (numpy array):Contains either the bootstrapped mean (for a single group) or difference in means (for two groups)CI (tuple):Confidence limits.bins (int):Number of bins in the histogram; default 10Returns:Noneplot_distribs(experiment_mean, control_mean, bins=10)For a two-group comparison, plot histograms of the control and experimental sampling distributions on a single graph.Args:experiment_mean (np array):Bootstrapped samples of mean for experiment groupcontrol_mean (np array):Bootstrapped samples of mean for control groupbins (int):Number of bins in the histogram; default 10Returns:NoneUsageDependenciesnumpy : 1.19.1pandas : 1.1.3matplotlib : 3.3.1additionally for unit tests:scipy 1.5.2Developed & tested inPython 3.8.5InstallationAssuming workingnumpy,pandas, andmatplotlib:> pip install bootstrap_CIHTExampleThis uses a 'toy' dataset (available in thetestsfolder) with two groups: "success" and "failure". The success group is 1000 samples from a normal distribution with mean 0.0 and standard deviation 0.5. The failure groups is 1000 samples from a normal distribution with a mean of 1.0 and standard deviation of 0.5.Note that each pop-up plot window will need to be closed for the script to continue.>>>importpandasaspd>>>importnumpyasnp>>>frombootstrap_CIHTimportBootstrap_CIHTasBS>>>np.random.seed(42)#for reproducibility>>>df=pd.read_csv('nml2.csv')>>>bs1=BS(df,'numbers',num_vars=2,group_col='outcome',group1='failure',group2='success')>>>bs1.df.head()#the processed datadatagroup00.248357success1-0.069132success20.323844success30.761515success4-0.117077success>>>bs1.df.tail()datagroup19951.535075failure19960.986739failure19970.559063failure19980.918467failure19990.627549failure>>>exp_mean,ctrl_mean,diffs=bs1.get_bootstrap_sample()>>>CIs=bs1.calculate_CI(diffs)2-sidedCIlimits:2.597.595%2-sidedCIfordifferencesinmeansofnumbers:(-1.0697323775784746,-0.9825440488702097)>>>p_val=bs1.calculate_hypothesis_test(diffs)experimentalgroup:success;mean:0.0097controlgroup:failure;mean:1.0354differenceofmeans,samplingdistributionSD:-1.0258,0.0223H0:Differenceinmeanssuccessvsfailure=0H1:Differenceinmeanssuccessvsfailure<>0p-value:0.000000>>>bs1.plot_hist_CI(diffs,CIs,bins=20)>>>bs1.plot_distribs(exp_mean,ctrl_mean,bins=20)The plot produced bycalculate_hypothesis_testis:The plot fromplot_hist_CIis:Finally,plot_distribsproduces:Additional TestsThetestsfolder contains unit tests intest.py. These use the same toy data as above and are run via:>python -m unittest test.pyThere are also three other files (test0.py,test1.py,test2.py,test3.py) that reproduce exercises from a Udacity Introduction to Statistics course I previously took. The source.csvfiles are included, as are the Jupyter notebooks with the original exercises and results.Note that the exercise intest0.pyappears to produce results that vary a fair amount with each re-sampling, so you may get inconsistent CIs and p-values.HistoryCreated April 12, 2021LicenseLicensedunder theMIT License.