Package: babbab
babbabThe two purposes ofbabbabare:To be thesimplesttool for Data Analysts/Statisticians toanalyzeA/B tests.To return thesimplestresults for Stakeholders/Non-Statisticians tounderstand.babbaban acronym ofBAyesianBeta-BinomialA/Btesting (BaBBAB), but it's spelled in lowercase (babbab) because it doesn't like shouting.InstallThis should work in vanilla Python +3.8.pipinstallbabbabA quick exampleLets assume we are testing changing the background color of our app from grey to green. Lets say we sell subscriptions to a paper magazine. We want to know if changing the background color will increase sales. To do so, we assign 50% of our users to the new app design with a green background (The Variant Group), while other 50% stay in the old grey design (the Control group). We managed to pull these 4 numbers out our tracking into Python:control_sold_subscriptions=200control_users=40316variant_sold_subscriptions=250variant_users=40567Becausebabbabis awesome you can just run:importbabbabasbabplot,statement,trace=bab.quick_analysis(control_sold_subscriptions,control_users,variant_sold_subscriptions,variant_users)And geteverything you need.Inplotyou will find a matplotlib figure. You can change the title and labels in thequick_analysisfunction.Instatement, you will get a string that is intended to be interpreted verbatim by Non-Statisticians.Intrace, you will get anarviz InferenceDataobject, in case you want to explore the run further.In the signature ofquick_analysisyou can configure the statistics and the aesthetics of most of this.MotivationA/B tests (or controlled experiments) are an increasingly popular way of incrementally improving websites, desktop, and mobile apps. AtMultilayerwe have analyzed probably hundreds, with a miriad of different tools and statistical methodologies.In our experience, when companies A/B tests, the biggest problems they encounter are around interpreting the results and acting appropiately on them. There are plenty of statistical libraries out there that do A/B testing right (babbab actually usesPyMCin the background). However, sharing statistics (like p-values) with non-statisticians can lead to confusion andmisuseof results.Whatbabbabtries to cover is the "last mile" of the A/B test analysis: Interpreting and communicating the results for them to be actionable.In other words - Whybabbabis awesomeGet 4 numbers in, get a statistically valid statement that you can repeat to your manager verbatim, and a plot you can understand.Get 4 numbers in + some labels, and you will get the aboveanda plot you can shareanda statement you can C&P in the company chat.Add a bit more work, and you have your own custom built AB testing dashboard/tool.Stop worrying about your peers and yourself misinterpreting stats.TODO list - Soft LaunchThis project is currently soft launched, which means that while you should feel free to use it and share it with friends, please let us know you are using it and please don't discuss it in social media as it's not ready and it's kind of a shy piece of software.Tasks to get out of this phase:Add example results (plot, statement) to the READMEAdd example with labels to READMEAdd docstringsMaybe?Sphinx or RTD Documentation