Package: py-ab-testing
AB TestingTermsPackage API ReferencesFor React:@appannie/react-ab-testingInstallationUsageFor Vanilla Javascript:@appannie/ab-testingInstallationUsageJavascript Hashing Utils@appannie/ab-testing-hash-objectInstallationUsageFor Python:py-ab-testingInstallationUsageConfigration HashingPrepare config file BEFORE make it publicIn runtimeConfiguration File ReferenceConfiguration File Hashing and Protecting private informationExample configuration and common use caseExcluding a user from an experimentInternal whitelistCreditsAB TestingThe AB Testing library implements multi-variance testing with a deterministic algorithm. It doesn't require any complex backend or database. Supporting both Javascript and Python.The AB Testing library segmentating users under different "cohorts", so you could use anif-else-like statement to implement different features base on the segmentation.There're 2 ways of segmentation:Percentage based segmentation: Randomly allocate certain percent of user to a "cohort". The allocation algorithm guarantees to produce stable results within a singleexperiment, and produce a different set of users for anotherexperimentin a random way. You can also further filter the user set using the keyallocation_criteria. Take a look at this example:first 25% users, it will always result into the same group of users inexperimentA, but results in another completely different group of users forexperimentB.User profile based segmentation: Manually assign a specific group of users to a cohort. This can be used to achieve features like "expose a new feature to the company emplyees for internal testing". Note the profile based segmentation works independently from the percentage based method. They doesn't interfere each other.The segmentation decisions are made base on a configuration file. The file should bejsonformated and hosted centralized place. More details about the configuration file below.The config file can be optionalyhashedso the content is not reversable. In this case you can host the file publicly, for example, on you CDN server, or in a publich S3 bucket.TermsExperiment: A subject that you want to do ab-testing with. You can have multiple experiments on going at the same time, the library can handle them independently.Cohort: Inside an experiment, users are classified into groups. The groups are called Cohorts.control: The name of the default cohort. Which is, the cohort returned when the user doesn't matche any rule on any cohorts, or the experiment just doesn't exists.Package API ReferencesFor React:@appannie/react-ab-testingReact binding for the vanilla Javascript package, featuring similar functionality.Installationnpminstall@appannie/react-ab-testing# oryarnadd@appannie/react-ab-testingUsageWrap your app with theABTestingControllerimport{ABTestingController}from'@appannie/react-ab-testing';constMyApp=({user})=>{constprofile={persona:user.persona,employee:user.isEmployee,};return(<ABTestingControllerconfig={testConfig}userId={user.id}userProfile={profile}><App/></ABTestingController>);};The required props are:config: the configuration object.userId: a unique identifier for your current user. This ID should be the same across visits to make sure your user always end up in the same cohorts. It can astringor anumber.userProfile: a key/value map used to force include a user in given cohorts.Then within your app, check the cohort a user is assigned to using theuseCohortOfhook.import{useCohortOf}from'@appannie/react-ab-testing';constComponent=()=>{constcohort=useCohortOf('experiment-name');switch(cohort){case'blue':return<BlueButton/>;case'red':return<RedButton/>;// 'control' is the default cohort. All experiments have a control cohort.case'control':default:return<Default/>;}};For Vanilla Javascript:@appannie/ab-testingCore Javascript package that implementing ab-testing SDK APIs.Installationnpminstall@appannie/ab-testing# oryarnadd@appannie/ab-testingUsageimport{Experiments}from'@appannie/ab-testing';constprofile={persona:user.persona,employee:user.isEmployee,};constexperiments=newExperiments(config,user.id,profile);constcohort=experiments.getCohort('experiment-name');switch(cohort){case'blue':console.log('user in the blue cohort');break;case'red':console.log('user in the red cohort');break;// 'control' is the default cohort. All experiments have a control cohort.case'control':default:console.log('user in the control (default) cohort');break;}Javascript Hashing Utils@appannie/ab-testing-hash-objectAn helper library that implements hashing for the configurations (user profile and the configurationforce_includesection) in case you want to host the config file publicly but still keep the content private, or just don't want to leave PIIs been transmitted in plain text.The hashing algorithm used issha256, and we're using a universal implementation (create-hash) that'll work both in Node and the browser.Installationnpminstall@appannie/ab-testing-hash-object# oryarnadd@appannie/ab-testing-hash-objectUsageimport{hashObject}from'@appannie/ab-testing-hash-object';constprofile=hashObject({persona:user.persona,employee:user.isEmployee,},salt);constforceIncludeCondition=hashObject({persona:['data analyst'],employee:['yes'],},salt);You'll want to make sure the salt is the same on your server and your client for the encoding result to be the same.For Python:py-ab-testingPython implementation for the same APIs in@appannie/ab-testingand@appannie/ab-testing-hash-objectInstallationpipinstallpy-ab-testing# orpipenvinstallpy-ab-testingUsageNote: Theconfigvariable holds andictwithconfiguration file format that documented here.fromABTestingimportABTestingControlleruser_profile={'persona':user.persona,'employee':user.isEmployee,}controller=ABTestingController(config,user.id,user_profile)cohort=controller.get_cohort('experiment-name')ifcohort=='blue':do_something()elifcohort=='red':do_something_else()else:do_default_behavior()Configration HashingSimilar to the Javascript SDKs, the package comes with an optional util for hashing private information withsha256.Prepare config file BEFORE make it publicfromABTesting.utilsimporthash_dictconfig['salt']=saltforexperimentinconfig['experiments']:forcohortinexperiment['cohorts']:cohort['force_include']=hash_dict(cohort['force_include'],salt)In runtimefromABTesting.utilsimporthash_dicthashed_user_profile=hash_dict({'persona':user.persona,'employee':user.isEmployee,},salt)# Make sure config is hashed with the same saltcontroller=ABTestingController(config,user.id,hashed_user_profile)Configuration File Reference{// The version isn't used right now, it is used to allow potentially breaking configuration// change in the future. Current version is "1.0"version:"1.0",experiments:[{// Unique name used to refer to an experiment; used within `useCohortOf`name:'experiment-name',cohorts:[{// Name of a cohort. These are the values returned by `useCohortOf`name:'blue',// The force_include section is used to force users into given cohorts if their// userProfile key. In many keys are used, *any* match will force include the user in// the cohort.// The force_include rules are checked in the order of the cohorts in the array and// the first match wins.// This section is optional.force_include:{persona:['data analyst']},// The users are allocated to values in a range of 0 to 100. The allocation config// represents the slice of users allocated to a cohort. You can have multiple// allocation ranges.// Every range needs to be unique and not overlap other ranges in any other cohort.allocation:[[0,25]],// The allocation_criteria key allows us to further filter the set of users by enforcing// a criteria that must be valid before the cohort is approved. In this case,// the user must have an email domain of data.ai or appannie.com.allocation_criteria:{email_domain:['appannie.com','data.ai']}},{// "control" is the default cohort. All experiments always have a control cohort.// All users not allocated to other cohorts will be assigned to "control" by default.name:'control'},]}]}At Data AI, we're maintaining this configuration in yaml format to reduce the syntax noise. We have an automated CI task to encode/encrypt and push the final configuration to a public S3 bucket from where our SDK retrieve the configuration.Configuration File Hashing and Protecting private informationYou shouldn't list userprivate information (PII)within the configuration file as these are most likely public.If you need to segment users using emails or information you wish/need to keep private, we suggest you to securely encrypt them.We're providing an extra module@appannie/ab-testing-hash-objectaiming to simplify this flow. This module will encode the keys and the values of your user profile. You then need to call it with the samesaltwhile preprocessing your configuration and once when defining the user profile:import{hashObject}from'@appannie/ab-testing-hash-object';constprofile=hashObject({persona:user.persona,employee:user.isEmployee,},config.salt);Example configuration and common use caseExcluding a user from an experimentYou can do this by force including a user or a group of users in thecontrolcohort.experiments:-name:my_experimentcohorts:-name:controlforce_include:# Here we'd prevent any google.com users from seeing the experimentemail_domain:-google.com-name:cohort_Aallocation:-[0,10]Internal whitelistForce include users withemail_domainequal toappannie.comunder the desired cohort.experiments:-name:my_experimentcohorts:-name:control-name:cohort_Aforce_include:# Force all Data AI user under the given cohortemail_domain:-appannie.comCreditsMade with ❤️ byZhang TaoandSimon Boudriasfrom the Data AI Beijing office.Available for public use under the MIT license.