A/B testing a Vue app with AlephBet
Tags: abtesting, vue, node, sqlite
Published: 2022-04-06 12:45:53

In this article, we will set up A/B testing for a simple Vue.js app and process the experiments on a Node.js server, saving them in a SQLite database. 

Before we get started, if you need a refresher on A/B testing or want to learn about how A/B testing is used by Netflix to drive user experience, I can recommend this [series on the Netflix tech blog](https://netflixtechblog.com/decision-making-at-netflix-33065fa06481).

All code of this post is on [Github](https://github.com/leo-pfeiffer/vue-ab-testing).

## Goal
The idea is to build a simple counter app with a button that increments a counter with every click. The A/B test could (for example) test if users prefer a blue button or a green button.

When the user clicks the button, the event is posted to the node server, which will store the results in a database for later analysis.

While the set up is simple, it should demonstrate the principles involved relatively clearly.

## Technologies
- Vue 2 app on which we want to perform A/B testing
- Node/Express server to process experiment results
- SQLite database to store experiment results
- [AlephBet](https://github.com/Alephbet/alephbet) framework for A/B testing

## Setting up the Vue project
First, set up a basic Vue project. We will use Vue 2 here.

```
vue create client
```

Next, we will make some changes to the `HelloWorld` component, which will be the subject of our tests.  The Vue component will have a single button and counter. The `.is-blue` and `.is-green` CSS classes will be used later for the A/B test.

```vue
// client/src/components/HelloWorld.vue

<template>
  <div class="hello">
    <p id="counter"> {{ counter }}</p>
    <button id="increment-btn" @click="increment">Increment</button>
  </div>
</template>

<script>
export default {
  name: 'HelloWorld',
  data() {
    return {
      counter: 0,
    }
  },
  methods: {
    increment: function() {
      this.counter++;
    }
  }
}
</script>

<style scoped>
#counter {
  font-size: xxx-large;
}

#increment-btn {
  border: none;
  padding: 15px 32px;
  font-size: x-large;
  margin: 4px 2px;
  cursor: pointer;
}

.is-blue {
  background-color: #34495e;
  color: white;
}

.is-green {
  background-color: #41b883;
  color: white;
}

</style>
```

## Setting up AlephBet

As mentioned before, we will use [AlephBet](https://github.com/Alephbet/alephbet) for managing the A/B tests.

### Configuration

First, install the dependency:

```bash
npm install alephbet
```

Next, we can configure our tests. Set up a new file in the Vue project `src/analytics/ab-testing.js`.

After importing AlephBet, we will set up a wrapper method `makeExperiment` to create new experiments.

```javascript
// client/src/analytics/ab-testing.js

const AlephBet = require("alephbet");

/**
 * Set up a new A/B testing experiment with AlephBet
 * @param name Name of the experiment
 * @param variants Object of the experiment variants
 * @param adapter Adapter of the experiment
 * */
const makeExperiment = (name, variants, adapter) => {
    return new AlephBet.Experiment({
        name: name,
        variants: variants,
        tracking_adapter: adapter,
    });
};
```

AlephBets uses Google Analytics by default as the experiment adapter. Since we want to set up our own backend, we need a custom adapter. For now, we will simply log all events to the console.

```javascript
// client/src/analytics/ab-testing.js

/**
 * Wrapper for an A/B testing adapter for AlephBet experiments.
 * */
const makeAdapter = () => {
    return {
        experiment_start: async function (experiment, variant) {
            console.log(experiment, variant, 'participate')
        },
        goal_complete: async function (experiment, variant, event_name) {
            console.log(experiment.name, variant, event_name)
        },
    };
};
```

### Defining the variants

Next, we can define our experiment variants. We will store these in a JS object `experimentVariants` with the key being the name of the experiment.

```javascript
// client/src/analytics/ab-testing.js

// Experiment variant presets
const experimentVariants = {
    "button color": {
        green: {
            activate: function () {
                document.getElementById("increment-btn").className = "is-green";
            },
            weight: 50,
        },
        blue: {
            activate: function () {
                document.getElementById("increment-btn").className = "is-blue";
            },
            weight: 50,
        },
    },
};
```

In the inner object, we define two variants `blue` and `green`. Each variant has an `activate` function that is called by AlephBet when the variant is activated for a user. In our case, the `activate` function adds the `.is-green` or `.is-blue` CSS class, respectively, to the increment button.

The `weight` specifies the likelihood that a user is allocated the variant.

Lastly, export the two wrapper methods and the object with the variants.

```javascript
module.exports = {
    makeExperiment: makeExperiment,
    makeAdapter: makeAdapter,
    experimentVariants: experimentVariants,
}
```

### Configuring the Vue component

The actual experiment is configured in the HelloWorld component, specifically in the `mounted` method of the component. 

Start by importing the functions we just created as well as AlephBet. We also need to define the variable for the goal in the most outer scope of the component.

```javascript
// client/src/components/HelloWorld.vue

import {
  experimentVariants,
  makeAdapter,
  makeExperiment,
} from "@/analytics/ab-testing";
import AlephBet from "alephbet";

let goal;
...
```

The `goal` variable captures the completion of the goal of the experiment - in this case when the user clicks the button.

In the `mounted` method, set up the goal with the experiment. We set `unique: false` since we want every click to be registered and not merely the first click.

```javascript
// client/src/components/HelloWorld.vue

...
  mounted() {
    const name = "button color";
    const variants = experimentVariants[name];
    const adapter = makeAdapter();
    const experiment = makeExperiment(name, variants, adapter);
    goal = new AlephBet.Goal("button clicked", {unique: false});
    experiment.add_goal(goal);
  },
...
```

Finally, we need to actually register the goal completion when the button is clicked. As the `increment` function is called on click, we can simply add one line to that method.

```javascript
// client/src/components/HelloWorld.vue

...
increment: function() {
      goal.complete()
      this.counter++;
    }
...
```

That is the basic set up of the client complete. Fire up your application and head over to `localhost:8080`. You should now either see a blue or a green increment button. AlephBet actually stores the variant in localStorage so that one user is always shown the same variant. Hence, if you'd like to see the other variant, delete the `alephbet` entry from localStorage and refresh the page (you might have to do this a few times until you randomly get allocated to the other group).


![Screenshot of the BLUE variant](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ylxsfmp7f7n3kxue727m.png)

![Screenshot of the GREEN variant](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9wx31q4zf4i1u83lcmj7.png)

If you open the console, you will also notice that the adapter logs the start of the experiment as well as every goal completion, that is every button click.

Next on the list is the set up of our experiment tracking server.

## Node.js server setup

Start by setting up a second directory `server` on the same level as the `client` Vue app, then setup the npm project and install the dependencies.

```
mkdir server
cd server
npm init
npm install express cors sqlite3
```

### index.js

Next, create a file in `server` called `index.js` and add the following content:

```javascript
// server/index.js

const express = require("express")
const app = express()
const cors = require('cors')

app.use(cors())

// Server port
const PORT = 5555;

// Start server
app.listen(PORT, () => {
    console.log(`Server running on port ${PORT}`)
});

// Root endpoint
app.get("/", (req, res, next) => {
    res.json({"message":"Ok"})
});

// A/B testing endpoint
app.post("/track_experiment", (req, res) => {
    const experiment = req.query.experiment;
    const variant = req.query.variant;
    const event = req.query.event;

    if (experiment === null || variant === null || event === null) {
        res.status(400);
        return;
    }

    console.log(experiment, variant, event);
    res.json({"message":"Ok"})
})

// 404 not found for other requests
app.use(function(req, res){
    res.status(404);
});
```

We won't go into detail here, but essentially we're setting up a simple server running on PORT 5555 with the `/track_experiment` endpoint, to which we can send our experiment events from the counter app.

### database.js

To persistently store the experiment results, we use a simple SQLite database. The setup here is very basic and could (should!) be improved but it is sufficient for this proof of concept.

In the `server` directory create the following file `database.js`:

```javascript
// server/database.js

const sqlite3 = require('sqlite3').verbose()

const DB_FILE = "db.sqlite"

let db = new sqlite3.Database(DB_FILE, (error) => {
    if (error) {
        // Error opening db
        console.error(error.message)
        throw error
    }
    else{
        console.log('Connected to the SQLite database.')

        const sql = `
        CREATE TABLE experiment (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            name text, 
            variant text, 
            event text
            );
        `
        db.run(sql, (err) => {/* table already exists */});
    }
});

module.exports = db
```

This sets up a single table `experiment` with the columns `id`, `name`, `variant`, and `event`.

Now that we've set up the database, we can fill out the rest of the API endpoint.

### Back to index.js

First, import the `db` object at the start of the file.

```javascript
// server/index.js
...
const db = require("./database.js")
...
```

We can now update the `/track_experiment` to insert the incoming experiment data into the database. The final endpoint should look like this.

```javascript
// server/index.js

...

// A/B testing endpoint
app.post("/track_experiment", (req, res) => {
    const experiment = req.query.experiment;
    const variant = req.query.variant;
    const event = req.query.event;

    if (experiment === null || variant === null || event === null) {
        res.status(400);
        return;
    }

    // Insert into database
    const sql = 'INSERT INTO experiment (name, variant, event) VALUES (?, ?, ?)'
    const params = [experiment, variant, event];

    db.run(sql, params, function (error, result) {
        if (error){
            res.status(400).json({"error": error.message})
            return;
        }
        res.json({
            "message": "success",
            "data": params,
            "id" : this.lastID
        })
    })
})

...
```

We can try this out by starting the server `node server/index.js` and sending a test request to the endpoint with curl.

```
curl --request POST "http://localhost:5555/track_experiment?experiment=myname&variant=myvariant&event=myevent"

> {"message":"success","data":["myname","myvariant","myevent"],"id":1}%  
```

Success!

## Integrating the Vue app with the server

Server and DB are running, thus we can now connect the client to the server.

Head back to the `client` directory. We will edit the `ab-testing.js` file.

We first need to add a method to post the event to the tracking server.

```javascript
// client/src/analytics/ab-testing.js

/**
 * Post an experiment result to the tracking server.
 * */
const postResult = (experiment, variant, event) => {
    let URL = "http://localhost:5555/track_experiment"
    URL += `?experiment=${experiment}&variant=${variant}&event=${event}`
    fetch(URL, {
        method: 'POST'
    }).catch(console.error)
}
```


Almost done. Now, in the `makeAdapter` wrapper function we want to use this new method. Update the code as follows:

```javascript
// client/src/analytics/ab-testing.js

...

const makeAdapter = () => {
    return {
        experiment_start: async function (experiment, variant) {
            postResult(experiment.name, variant, 'participate')
        },
        goal_complete: async function (experiment, variant, event_name) {
            postResult(experiment.name, variant, event_name)
        },
    };
};

...
``` 

That's it. Any experiment results will now be posted to the server and saved in the database:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w05ov09lf5h39nb4q7yv.png)
 

## Final thoughts

Setting up basic A/B tests with AlephBet isn't rocket science. While this demo project is fairly simple, I hope it serves as a decent introduction.

Feel free to reach out for feedback!

Happy A/B testing 🎉