How to conduct an A/B test experiment in Go using feature flags
Tags: featuremanagement, featureflags, go, configcat
Published: 2023-03-24 22:44:44

When it comes to releasing new features, it is often difficult to anticipate how users will react and interact with them. This is where A/B testing comes in useful. It provides a mechanism to test and evaluate two variations of an app to determine which is better by releasing them to an isolated user segment before a full deployment. This level of flexibility allows developers to quickly experiment with new features without affecting the production environment.

## What is an A/B Test

An A/B test is a type of testing in which two variations of an app are released for testing to an isolated user segment before being deployed to the entire user base to collect metrics and feedback. One variation is designated **variation A**, and the other is designated **variation B**. Variation A usually represents the current version of the app and is considered a control or benchmark variation, while variation B represents the version of the app with the new feature to be tested.

## Conducting a demo A/B test

Here are two variations of a [demo website](https://github.com/configcat-labs/ab-testing-in-go-sample) built with Golang. I aim to determine if changing the design of the home page will influence a higher number of visits to the **Store page**.

![A/B Versions](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gcw8rs764foit23z9c2c.png)

Before we dive deeper, let's take a moment to discuss a few additional technologies that will help streamline the A/B testing process.

## The role feature flags play

Feature flags are virtual switches that you can integrate into your app to control what features users can and cannot see. An added benefit of using feature flags is that you can easily switch from one variation to the other with ease and target specific user segments when releasing new features. They let you launch new features and change your software configuration without (re)deploying code.

Let's see how we can set this up using [ConfigCat's feature flag service](https://configcat.com). For context, ConfigCat supports simple feature toggles, user segmentation, and A/B testing and has a generous free tier for low-volume use cases or those just starting out.

## Creating feature flags for the test

Let's create two feature flags. One will be responsible for showing the old home page (Variation A) and the other will be responsible for showing the new home page (Variation B).

To create the feature flags, you'll need to sign up for a [free ConfigCat account](https://app.configcat.com/signup).

**1.** In the dashboard create two feature flags as shown below:

![Adding feature flags](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ktmbxcrcoljkxrnasbm3.png)

**2.** I'll turn off the **newHomePage** feature flag, for now, to focus on collecting metrics from the old home page first. More on this in a later step.

## Creating user segments

**1.** I'll create two user segments within my ConfigCat dashboard. Each segment will consist of 50 users represented by their email addresses. One segment will see variation A and the other will see variation B.

![Adding user segments](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/phepdasfdwt724jc1ec9.png)

**2.** I'll link the feature flags to the segments. In this way, when a flag is switched on, the variation it controls will only be shown to users in the segment it is linked to.

![Targeting user segments](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h82y9s28s8et143b64a5.png)

## Integrating the feature flags into Go

Let's look at how we can integrate the feature flags into the Golang app.

**1.** Install the ConfigCat's Go SDK, then add it to the import statement in `main.go`:

```sh
go get github.com/configcat/go-sdk/v7
```

```go
import (
  "github.com/configcat/go-sdk/v7"
)
```

**2.** Create the client using your ConfigCat's SDK Key. You should keep this key private and outside of your code base.

```go
client := configcat.NewClient("YOUR_CONFIGCAT_SDK_KEY")
```

**3.** Add the user object. Here, I am using the user's email address. This is required to help ConfigCat to decide whether the user belongs to a particular segment.

```go
var user = &configcat.UserData{Email: "email1@example.com"}
```

**4.** Create variables to store the value of each feature flag and create if-statements to render the variation of the home page it is responsible for:

```go
func homePageHandler(w http.ResponseWriter, r *http.Request) {
	isNewHomePageEnabled := client.GetBoolValue("newhomepage", false, user)
	isOldHomePageEnabled := client.GetBoolValue("oldhomepage", false, user)

	if isNewHomePageEnabled { newHomePageTemplate.Execute(w, nil)} 
	if isOldHomePageEnabled { oldHomePageTemplate.Execute(w, nil)} 

}
```

With that set, let's look at how we can collect metrics for the test.

## Collecting metrics

I plan to collect metrics for **variation A** over a period of three weeks and then do the same for **variation B**. To do this, I will use a product analytics tool called [Amplitude](https://amplitude.com/).

**1.** [Sign up](https://analytics.amplitude.com/signup) for a free Amplitude account.

**2.** Switch to the **Data** section by clicking the dropdown at the top left.

**3.** Click the **Sources** link under **Connections** in the left sidebar, then click the **+ Add Source** button on the top right.

**4.** Select the **GO SDK** from the SDK sources list and enter the following details to create a source:

![Creating a source](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wmlcdrui2im336rzh4yp.png)

**5.** You should be automatically redirected to the **implementation page**. We'll revisit the instructions on this page soon but first, let's add an event.

## Adding an event

To keep track of the number of visits to the store page, I'll create an event. In a later step, I'll trigger and log this event to Amplitude when the store page is rendered.

**1.** Click the **Events** link in the left sidebar under **Tracking Plan** to access the events page.

**2.** Click the **+ Add Event** button at the top right to create an event and fill in the following details:

![Adding an event](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/amgrai3196em152iy6cm.png)

**3.** Click the **Save changes** button in the left sidebar.

## Integrating Amplitude into Go

Click the Implementation link in the left sidebar to see the integration instructions page.

**1.** Install the Amplitude CLI with the following command:

``sh
npm install -g @amplitude/ampli
``

**2.** Install the Amplitude Go SDK:

```sh
go get github.com/amplitude/analytics-go
```

**3.** Run the following command to pull the necessary configurations we added in Amplitude's dashboard into the Golang app.

```sh
ampli pull
```

## Sending the page visit event to Amplitude

**1.** Import Amplitude in `main.go`

```go

import (
	"github.com/configcat-labs/ab-testing-in-go-sample/ampli"
)
```

**2.** Call the **PageView** event method on the **ampli** instance to send an event when the store page is visited:

```go
func storePageHandler(w http.ResponseWriter, r *http.Request) {
	ampli.Instance.PageView("user@example.com") // New*
	storePageTemplate.Execute(w, nil)
}
```

Let's check the logged requests to make sure the Go app is connected to Amplitude.

## Checking for logged requests

**1.** Under **Connections** in the left sidebar, click on **Sources**.

**2.** Clicking on the **Visit the store** button on the home page will redirect the user to the store page. When the store page is rendered the event will be logged to Amplitude as shown in the **Successful Requests** graph:

![Successful requests chart](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l9pujqb1ywbn0ba83orp.png)

## Setting up an analysis chart

To effectively analyze the incoming events from the app, I'll set up an Analysis chart. This will give me visual insights into how each variation is performing in real-time.

**1.** Switch to the **Analytics** dashboard, by clicking the dropdown arrow on the top left next to **Data**.

**2.** In the analytics dashboard, click the **New** button in the left sidebar.

**3.** Select **Analysis**, then select **Segmentation**.

**4.** Select the event as shown below:

![Selecting an event for the analysis chart](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mgizyr5xkqcjvoscfh9u.png)

**5.** Click **Save** on the top right to save the chart.

I'll leave this to run for three weeks to see how **Variation A** performs. When the three-week period is up, I'll switch off the **oldHomePage** feature flag and turn on the **newHomePage** feature flag which will release **variation B** to users in **User segment B**.

![Switching off the old home page feature flag](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cacvg1l5tiqm8btf6k79.png)

## Analyze the test results

Depending on how long you decide to run your tests, at the end of the time period you can use the **Compare to past** dropdown option at the top right of the analysis chart to compare the two variations. For example, if the total number of visits to the store page was 40 for the past three weeks for variation A, you can compare that to the three-week period where you released variation B to determine which is performing better.

![Snapshot of compare to past dropdown options](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kh0te7gnj4vi95ocwz63.png)

To keep track of your feature flag changes you can [add it to your charts on Amplitude](https://configcat.com/docs/integrations/amplitude/#:~:text=Installation%E2%80%8B&text=Open%20the%20integrations%20tab%20on,check%20your%20charts%20in%20Amplitude.).

## Conclusion

By test-driving new features through an A/B test, we can see what works and what doesn't. This method allows you to test and refine features without affecting the entire user base. Feature flags play an important role in this process by allowing us to switch between variations and set up user segmentation. ConfigCat's dashboard is simple to use allowing you and your teammates to get up and running with feature flags quickly. You can learn more about the various languages and frameworks that ConfigCat supports [here](https://configcat.com/docs/sdk-reference/overview/).

Stay on top of the latest posts and announcements from ConfigCat on [Twitter](https://twitter.com/configcat), [Facebook](https://www.facebook.com/configcat), [LinkedIn](https://www.linkedin.com/company/configcat/), and [GitHub](https://github.com/configcat).