Dynamically routing user requests to multiple CloudFront origins
Tags: aws, serverless, s3, contentdelivery
Published: 2024-02-29 07:35:11

We can use Lambda@Edge functions to dynamically change CloudFront origins based on request parameters and perform production testing with a small subset of users.

## 1. The scenario

Say we have a new website and want to test it with real users in a production environment. We allow a subset of users access to the new page while the majority should still go to the existing site. This testing type is known as **A/B testing** or split testing.

In this scenario, we host the website on <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html" class="link" target="_blank" rel="noopener noreferrer">S3</a>. We have two buckets, one for the existing site and another for the new one, which we want to test. We have also set up a <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-overview.html" class="link" target="_blank" rel="noopener noreferrer">CloudFront distribution</a> in front of the buckets for optimization and security reasons. The buckets are **private**, and only the distribution can access them through an <a href="https://dev.to/aws-builders/securing-s3-uploads-and-downloads-with-origin-access-control-3fl0" class="link" target="_blank" rel="noopener noreferrer">origin access control</a>. The website files are in the **root** of the buckets, so no `v2` folder exists for the new variant. Both the existing and new sites should be available under the same URL.

It's a requirement that 10% of the users should view the new site via the **same** custom domain name as the existing page. We need to perform the user selection and redirection inside the AWS infrastructure.

## 2. Solution overview

The CloudFront distribution has **two origins**, one for the existing bucket and another for the new one. It also has the **default behavior** with the `*` path pattern and the **existing bucket** defined as the origin.

CloudFront can deploy <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-at-the-edge.html" class="link" target="_blank" rel="noopener noreferrer">Lambda functions</a> to the points of presence around the world and invoke them in various situations. We can use these functions to modify the requests and responses originated from both the users and the origin.

We can create two Lambda functions.

First, we decide if the user is lucky to be in the 10% who has the privilege to access the new page. We want to remember these users, so we add a **cookie** to their requests. We invoke this function on the **viewer request**.

The second function assigned to the **origin request** will check if the cookie is present. If so, the function will change the default origin (pointing to the existing website bucket) to the new one.

## 3. Settings and code

Let's see how we can set up this solution.

### 3.1. CloudFront settings

We need a **custom origin request policy** that forwards our cookie called `X-New-Page-Test` (the name can be different) to the origin. We want the second (origin request) Lambda@Edge function to decide if it needs to redirect the request to the new origin.

### 3.2. Viewer request Lambda

We associate the first Lambda function to the **viewer request** (configurable on the `Behavior`).

An example code can look like this:

```ts
import { CloudFrontRequestEvent, CloudFrontRequest, CloudFrontResponse }
from 'aws-lambda';

const NEW_PAGE_COOKIE = 'X-New-Page-Test';

export const handler = async (event: CloudFrontRequestEvent):
  Promise<CloudFrontRequest | CloudFrontResponse> => {
  const request = event.Records[0].cf.request;
  const cookies = request.headers.cookie;

  // 1. Check if the cookie is present
  const hasNewPageCookie = cookies?.find((cookie) =>
    cookie.value.includes(NEW_PAGE_COOKIE));
  if (hasNewPageCookie) {
    // The cookie is present, and the function has nothing to do here.
    // The origin request function will handle it.
    return request;
  }

  // 2. If the cookie is not present, we decide if the user should access
  // the new page
  const shouldGoToNewPage = Math.random() >= 0.9;
  if (shouldGoToNewPage) {
    const response = {
      status: '302',
      statusDescription: 'Found',
      headers: {
        'cache-control': [
          {
            key: 'Cache-Control',
            value: 'no-store',
          },
        ],
        'set-cookie': [
          {
            key: 'Set-Cookie',
            value: `${NEW_PAGE_COOKIE}=true`,
          },
        ],
        location: [
          {
            key: 'Location',
            value: request.uri,
          },
        ],
      },
    };
    // 3. We send the request back to the browser with the cookie
    return response;
  }

  // 4. The user doesn't belong to the tester group, so no cookie is set.
  // The request proceeds as intended.
  return request;
};
```

The function does a couple of things.

It checks if the `X-New-Page-Test` cookie is in the request (1). If so, it means that we already set it up earlier. The request can proceed with the cookie, and the origin request Lambda@Edge function will re-route it to the new bucket.

If the cookie is not in the request, the function decides if it should add to it (2). The `Math.random()` function returns a random number between 0 and 1. If the number is greater than or equal to 0.9, the user will belong to the 10% who get access to the new version. In this case, we add the `X-New-Page-Test` cookie and respond with a `302` status code. It will indicate to the browser that it should redirect the request to the URL specified in `Location`. But the value of the `Location` header is the same as before, so the browser will call the same URL again. This time, the cookie will be part of the request (3).

If the user turns out to belong to the 90% based on the `Math.random()` function's outcome, we don't set the custom cookie. Everything will go as usual this time, and we return the original `request` object (4).

### 3.3. The origin request Lambda

The second Lambda@Edge function will redirect the user to the new site if the `X-New-Page-Test` cookie is in the request.

Its code can look like this:

```ts
import { CloudFrontRequestEvent, CloudFrontRequest } from 'aws-lambda';

const NEW_PAGE_COOKIE = 'X-New-Page-Test';

export const handler = async (event: CloudFrontRequestEvent):
  Promise<CloudFrontRequest> => {
  const request = event.Records[0].cf.request;
  const cookies = request.headers.cookie;

  // 1. Check if our cookie is in the request
  const hasNewPageCookie = cookies?.find((cookie) =>
    cookie.value.includes(NEW_PAGE_COOKIE));
  if (hasNewPageCookie) {
    // 2. Change the relevant request fields to the new value
    const newPageDomainName = 'NEW_BUCKET_NAME.s3.eu-central-1.amazonaws.com';
    request.headers.host = [{ key: 'Host', value: newPageDomainName }];
    request.origin = {
      s3: {
        domainName: newPageDomainName,
        authMethod: 'none',
        customHeaders: {},
        path: '',
        region: 'eu-central-1',
      },
    };

    return request;
  }

  // 3. The cookie is not in the request, so go to the original page
  return request;
};
```

It starts similarly to the first function. We check if the `X-New-Page-Test` cookie is part of the request (1).

If so, we change the `host` header and `domainName` field in the `origin` object to the domain name of the second bucket that contains the new version (2). We then return the modified `request` object. CloudFront will then know that the request should go to the second origin.

If the cookie is not in the request, we return the same `request` object we received from CloudFront. In this case, the request will proceed to the existing bucket as usual.

## 4. Considerations

I want to highlight some points regarding the solution.

### 4.1. Lambda permissions

CloudFront can only invoke the functions if we add the `edgelambda.amazonaws.com` service to the `Principal` element. So the **trust policy** of both Lambda functions' **execution role** should look like this:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "edgelambda.amazonaws.com",
          "lambda.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
```

The **permission policy** is a different discussion. In our case, the `AWSLambdaBasicExecutionRole` AWS-managed policy is sufficient as the functions don't call any other AWS service.

### 4.2. Other Lambda requirements

For a Lambda function to become **Lambda@Edge**, we should be aware of some caveats:

<ul>
  <li>The functions must be deployed to <strong>us-east-1</strong> regardless of the buckets' regions. Because CloudFront is a global service, it <q>lives</q> in Northern Virginia.</li>
  <li>The maximum allowed function <strong>timeout</strong> is 5 seconds.</li>
  <li>Lambda@Edge only supports the <strong>x86_64</strong> architecture. We can't use Graviton-based processors at the moment for Lambda@Edge.</li>
  <li>We must specify a <strong>qualified function ARN</strong> when we configure Lambda@Edge in the CloudFront behavior. It means that we must create a numbered version of the function and can't use <code class="inline-code">$LATEST</code>.</li>
  <li>We can't use <strong>environment variables</strong> in the functions.</li>
</ul>

### 4.3. Logs in the closest region

Lambda@Edge functions send their logs to a log group in the region **closest** to the user. It's `eu-central-1` for me, even if I have deployed the functions to `us-east-1`.

So, if your first thought is that CloudFront doesn't trigger the function, check CloudWatch Logs in the region closest to you before starting debugging.

We can always aggregate the logs into a single log group in a dedicated region, but it's beyond the scope of this article.

### 4.4. Not the only way to solve the problem

As always, the solution described above is not the only way to do A/B testing with CloudFront. Requirements are always different, and there might be a solution that suits a specific use case better.

That said, you might find an easier way to perform a similar logic. A different solution may be a better fit for your scenario, which is it's OK. We can solve one problem in multiple ways. I added some links to the end that provide some alternative solutions.

## 5. Summary

We can use Lambda@Edge functions to perform live A/B testing with CloudFront. We want to send a group of users to the new version of our website by adding a custom cookie to the request. A Lambda function can inspect the request, and if the cookie is present, it can dynamically point the request to the new origin.

## 6. Further reading

<a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-examples.html#lambda-examples-content-based-routing-examples" class="link" target="_blank" rel="noopener noreferrer">Content-based dynamic origin selection - examples</a> - Dynamically selecting origin for different scenarios

<a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-examples.html#lambda-examples-a-b-testing" class="link" target="_blank" rel="noopener noreferrer">Example: A/B testing</a> - Path-based A/B testing in CloudFront

<a href="https://grrr.tech/posts/2021/cloudfront-a-b-tests/" class="link" target="_blank" rel="noopener noreferrer">Performing A/B tests on static websites using Cloudfront and Lambda@Edge</a> - Great article on the same problem with a slightly different solution

<a href="https://docs.aws.amazon.com/lambda/latest/dg/getting-started.html" class="link" target="_blank" rel="noopener noreferrer">Getting started with Lambda</a> - How to create a Lambda function

<a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html" class="link" target="_blank" rel="noopener noreferrer">Creating a distribution</a> - How to create a CloudFront distribution