AB tests with Symfony 5 made easy
Tags: php, symfony, abtest
Published: 2020-10-09 17:16:15

The company I work for is a data-driven company. It is necessary for us to make the best decisions and aim for our core value: being customer-centric.

Naturally, when we iterate on a feature, we study the data to define how to improve it. And because we can’t improve what we don’t measure, we measure the results of our new version with the previous version.

To do this, we almost systematically perform AB tests that we couple with our events database.

> AB tests are not for experimenting things, but to measure them.

And since it's [HacktoberFest](https://hacktoberfest.digitalocean.com/), we published one of our tools in open source here: [Travaux.com VariantRetriever](https://github.com/travaux-com/VariantRetriever/)

Next part of this article is to start your first AB test, like changing the subject of an email when a new user registers.
The experiment we are doing will be called `welcome-email-experiment` and it will have a `control` version (the actual version) and a `participant` (or variant) version (the new version we want to try).

When we will send this email, we’re going to require the VariantRetriever and ask it: for this experiment and this user identifier, tell me which variant it should have.

## Lets run a new AB test with Symfony 5
First step is of course to install the AB test package in your project:
```composer require travaux-com/variantretriever```

Because it’s not a Symfony bundle, you don’t need to declare it in your AppKernel file, but you still need (recommanded in fact) to declare it as a service.

We’re going to declare it as a service with the Symfony container factory system to have our “variant retriever” with all information about our running AB tests. (`welcome-email-experiment` and `display-cta`).

```yaml
services:
    App\FeatureFlag\VariantRetriever:
        factory:   ['@Travaux\VariantRetriever\Factory\VariantRetrieverFactory', 'createVariantRetriever']
        public: true
        arguments:
            - welcome-email-experiment:
                - control-email: 50
                - participant-email: 50
            - display-cta:
                - control-cta: 75
                - participant-cta: 25
```
The service name is actually `App\FeatureFlag\VariantRetriever` but I strongly encourage you to customize it.

Now you’re able to use the [Symfony autowiring](https://symfony.com/doc/current/service_container/autowiring.html) to retrieve the VariantRetrieverInterface and use it in your command handler.
And so on, how to use it ?

```php
$affectedVariant = $variantRetriever
           ->getVariantForExperiment(
                  new Experiment('my-ab-test'), 
                  (string) $user->getId()
           );
```

`VariantRetrieverInterface` actually came with the getVariantForExperiment method where you will have to define the experiment you want to test (by just filling the experiment name) and the “user identifier” who just need to be a string, so it can be a UUID v4 or an auto-incremented integer.

It will return you a `Variant` value object that contains the name of the affected variant.

Now you can imagine select the right translation key to use depending on the affected variant for your email:

```php
$experimentTranslationsKeys = [
       'control' => 'email.welcome.subject_control',
       'variant' => 'email.welcome.subject_variant',
];
$userVariant = $variantRetriever
           ->getVariantForExperiment(
                  new Experiment('welcome-email-experiment'), 
                  (string) $user->getId()
           );
$emailSubject = $experimentTranslationsKeys[(string) $userVariant];
```
This package will ensure you to always retrieve the same variant for your user, without any database or cache.

You’re now able to run your first AB in your Symfony project 🎉

## Go beyond this example
As you can see, we’ve use the [Symfony container factory](https://symfony.com/doc/current/service_container/factories.html) system to instantiate our VariantRetriever, but it can also be declared with [Symfony method call](https://symfony.com/doc/current/service_container/calls.html) like this:

```yaml
   Travaux\VariantRetriever\Retriever\VariantRetrieverInterface:
        class: Travaux\VariantRetriever\Retriever\VariantRetriever
        calls:
            - addExperiment:
                - !service
                    class: Travaux\VariantRetriever\ValueObject\Experiment
                    arguments:
                        - 'welcome-email-experiment'
                        - !service
                            class: Travaux\VariantRetriever\ValueObject\Variant
                            arguments: ['control', 100]
                        - !service
                            class: Travaux\VariantRetriever\ValueObject\Variant
                            arguments: ['variant', 0]
```

# How does it work internally ?
Thanks to [php crc32](https://www.php.net/manual/en/function.crc32.php) function, we’re able to process a string that return everytime the same integer value. So on, even if it’s a big number, we’re able to reduce it under a range of 0 to 99 that match the rollout percentage given by one of our variant.

Also, the `Travaux\VariantRetriever\Retriever\VariantRetriever` class is not final to allow you to extend it. Like injecting a PsrLogger and/or the event dispatcher to keep a trace of which experiment have been affected to which user. You can also add you own method to retrieve a variant based on your User entity object like this:

```php
public function getUserVariantForExperiment(
    Experiment $experiment,
    User $user
);
```
Feel Free to contribute to this project or give feedback.