Package: cached-contingency
CachedContingencyPython 3.9+ classes to compute and cacheFisher's exact testandBoschloo's testmore efficiently.InstallationThis package requires at leastPython 3.9.pipinstallcached_contingencyIdeaI have to compute lots of these tests and want to accelerate the process. There are two optimizations that came to my mind:My contingency tables often have identical column sums, so many tests can be recycledSome contingency tables are equivalent and only have to be computed onceFisher's test:abcd,acbd,badc,bdac,cadb,cdab,dbcaanddcbaare equivalent (pvalue,notodds ratio)Boschloo's test:abcd,badc,cdabanddcbaare equivalentFurthermore, sometimes, one has to re-run tools. In these cases, all previously computed results can be recycled.As cache, an SQLite database is used.ExecutionReplace equivalent contingency tables with the same contingency tableFind all tests that are not cached yetCalculate them in parallel, using all CPU coresAdd them to the cacheReturn resultsUsageSet the location of the cache database:exportKEY_VALUE_STORE_DB=/custom/path.db# default: ~/.cache/keyvaluestore.dbCalculate single tests:fromcached_contingencyimportCachedFisher,CachedBoschloo,odds_ratiofromscipy.statsimportfisher_exact,boschloo_exactfromnumpyimportisclose# Create class (automatically creates database if none exists yet)cf=CachedFisher()# Calculate Fisher's testpval_cache=cf.get_or_create(74,31,43,32)odds_ratio_cache=odds_ratio(74,31,43,32)# This is equivalent to:odds_ratio_calc,pval_calc=fisher_exact([[74,31],[43,32]])assertisclose(pval_cache,pval_calc)assertisclose(odds_ratio_cache,odds_ratio_calc)# Create class (automatically creates database if none exists yet)cb=CachedBoschloo()# Calculate Fisher's testpval_cache=cb.get_or_create(74,31,43,32)# This is almost* equivalent to:pval_calc=boschloo_exact([[74,31],[43,32]]).pvalueassertisclose(pval_cache,pval_calc)*: Not exactly equivalent: My function never returns pvalues greater than 1 and never returns nan as pvalues.
(Seescipy issue.)Calculate multiple tests:fromcached_contingencyimportCachedFisher,CachedBoschlooimportpandasaspdimportnumpyasnp# Create class (automatically creates database if none exists yet)cb=CachedBoschloo()# Create test DataFrame, column names are important!np.random.seed(42)test_df=pd.DataFrame([(np.random.randint(200)for_inrange(4))for_inrange(5)],columns=['c1r1','c2r1','c1r2','c2r2'])print(test_df)#    c1r1  c2r1  c1r2  c2r2# 0   102   179    92    14# 1   106    71   188    20# 2   102   121    74    87# 3   116    99   103   151# 4   130   149    52     1# Calculate multiple Boschloo's testsresult_df=cb.get_or_create_many(test_df)print(result_df)#    c1r1  c2r1  c1r2  c2r2          pval# 0   102   179    92    14  3.442564e-20# 1   106    71   188    20  1.144156e-12# 2   102   121    74    87  9.692791e-01# 3   116    99   103   151  3.821222e-03# 4   130   149    52     1  1.831830e-14# If you run this again, the results will be loaded from cache:result_df=cb.get_or_create_many(test_df)print('Like a flash!')Advanced usage:Alternative way to specify the path to the database via Python and change number of CPUs:fromcached_contingencyimportCachedFishercf=CachedFisher(db_path='/custom/path.db',n_cpus=1)