Patent ID: 11887149
Assignee: KLAVIYO INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 1:
2. A method for selecting a winner of an A/B test, comprising:
obtaining historical data related to the A/B test;
creating a histogram based on the historical data, the histogram including bins and weights;
creating a Dirichlet distribution by computing concentration parameters of the Dirichlet distribution from the weights of the histogram;
executing the A/B test with an associated metric of the A/B test with a user profile;
receiving new data collected based on the execution of the A/B test and the associated metric;
allocating the new data into same bins as the bins of the histogram of the historical data yielding a new data bin count for each bin of the allocated new data;
computing a posterior Dirichlet distribution comprising updating the Dirichlet distribution using the same bins and the new data bin counts of each bin and the concentration parameters of the Dirichlet distribution in closed form vie conjugate prior formulae;
inferring corresponding central tendencies of samplings of a metric distribution, wherein each central tendency of the corresponding central tendencies is determined by sampling the posterior Dirichlet distribution;
constructing an overall utility distribution for each arm of the A/B test by combining the central tendency of each sampling of the metric distribution with a corresponding sampling of a conversion probability distribution; and
determining a winning arm of the A/B testing by comparing the overall utility distribution of each arm with each other arm of the A/B test.