How I Setup CI/CD Pipeline For Cloudflare Worker
Tags: devops, programming, serverless, tooling
Published: 2022-06-01 00:09:26

Today, most of my projects on GitHub are kept up to date with [Renovate](https://github.com/renovatebot/renovate). With auto-merging enabled, I want to have enough confidence that the automated dependencies update would not cause any regressions.

Testing Cloudflare Worker is a bit out of whack. Don’t get me wrong, I love Cloudflare Worker. However, the [existing solution](https://blog.cloudflare.com/unit-testing-workers-in-cloudflare-workers/) out there feels a little bit unintuitive. On top of that, [dollarshaveclub/cloudworker](https://github.com/dollarshaveclub/cloudworker) is no longer actively maintained, which is a bummer.

_TL;DR: This post talks about how I set up a CI/CD pipeline for a Cloudflare Worker project using GitHub Action and [Grafana k6](https://k6.io/)._

---

## Overview

Previously, [I built a URL shortener clone](https://jerrynsh.com/i-built-my-own-tiny-url/) with Cloudflare Worker. Using this existing project ([GitHub](https://github.com/ngshiheng/atomic-url) link), we shall look into setting up a CI/CD pipeline for it along with simple integration tests.

Our GitHub Action CI/CD pipeline is rather straightforward. The stages (jobs) are as follows:

![Example CI workflow on GitHub](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tlx6zq6yn5d4m4s8w80i.png)

Example CI workflow on GitHub

1. Lint check or unit testing
2. Deploy to the Staging environment
3. Run the integration tests on our Staging environment
4. Run [semantic release](https://github.com/semantic-release/semantic-release) and deploy to the Production environment

## Wrangler Config

To start, we’ll have to modify our [existing `wrangler.toml` file](https://github.com/ngshiheng/atomic-url/blob/v1.1.8/wrangler.toml). Remember, we need a Staging environment to deploy to run our integration test against it:

```toml
[env.staging]
name = "atomic-url-staging"
workers_dev = true
kv_namespaces = [
  { binding = "URL_DB", id = "ca7936b380a840908c035a88d1e76584" },
]
```

Here are the things to take note of:

-   `name` — Make sure that the `name` has to be unique and alphanumerical (`-` are allowed) for each environment. Let’s name our Staging environment `atomic-url-staging`.
-   `worker_dev` — Our integration tests run on `<NAME>.<SUBDOMAIN>.workers.dev` endpoint. Hence, we need to deploy our Cloudflare Worker by setting `workers_dev = true` ([reference](https://developers.cloudflare.com/workers/platform/environments/#publishing-to-workersdev)).
-   `kv_namespaces` — Atomic URL uses KV as its database to store shortened URLs. Here, I chose to use a preview namespace KV as the test database. Why? Simply because it is the same development KV namespace that I use during local development (when running `wrangler dev`). Of course, you could just use a regular KV namespace. Just make sure that you’re not using Production KV `id`. [Read how to create a KV namespace](https://developers.cloudflare.com/workers/wrangler/cli-wrangler/commands/#kv_namespaces).

Next, we’ll also need to create a Production environment to deploy to with our production KV namespace:

```toml
[env.production]
name = "atomic-url"
route = "s.jerrynsh.com/*"
workers_dev = false
kv_namespaces = [
  { binding = "URL_DB", id = "7da8f192d2c1443a8b2ca76b22a8069f" },
]
```

The Production environment section would be similar to before except — we’ll be setting `worker_dev = false` and `route` for production.

### Deployment

To deploy manually from your local machine to their respective environment, run:

-   `wrangler publish -e staging`
-   `wrangler publish -e production`

Though, we’ll look into how to do this automatically via our CI/CD pipeline using GitHub Actions.

Before we move on, you may find the complete `wrangler.toml` configuration [here](https://github.com/ngshiheng/atomic-url/blob/f1ff25a00c1430314b45dde855497b58831b8e3a/wrangler.toml).

Oh, here’s a [cheat sheet](https://developers.cloudflare.com/workers/wrangler/configuration/#configure-wranglertoml) to configure `wrangler.toml`. I highly recommend you make use of this!

## Grafana k6

For integration testing, we’ll be using a tool known as k6. Generally, k6 is used as a tool for performance and load testing. Now, bear with me, we’re not going to integrate any load tests into our CI/CD pipeline; not today.

Here, we’ll be running [smoke tests](https://k6.io/docs/test-types/smoke-testing/) for this project whenever new commits are pushed to our `main` branch. A smoke test is essentially a type of integration test that performs a sanity check on a system.

In this case, running a smoke test is sufficient enough for me to determine that our system is deployed without any regression and can run on minimal load.

### What to test

Basically, here are a couple of things that we want to check as part of our smoke test for our URL shortener app in [groups](https://k6.io/docs/using-k6/tags-and-groups/#groups):

-   The main page should load as expected with a 200 response status

```js
group("visit main page", function () {
    const res = http.get(BASE_URL);

    check(res, {
        "is status 200": (r) => r.status === 200,
        "verify homepage text": (r) =>
            r.body.includes(
                "A URL shortener POC built using Cloudflare Worker",
            ),
    });
});
```

-   Our main POST API endpoint `/api/url` should create a short URL with the original URL

```js
group("visit rest endpoint", function () {
    const res = http.post(
        `${BASE_URL}/api/url`,
        JSON.stringify({ originalUrl: DUMMY_ORIGINAL_URL }),
        { headers: { "Content-Type": "application/json" } },
    );

    check(res, {
        "is status 200": (r) => r.status === 200,
        "verify createShortUrl": (r) => {
            const { urlKey, shortUrl, originalUrl } = JSON.parse(r.body);
            shortenLink = shortUrl;
            return urlKey.length === 8 && originalUrl === DUMMY_ORIGINAL_URL;
        },
    });
});
```

-   Make sure that when we visit the generated short URL, it redirects us to the original URL

```js
group("visit shortUrl", function () {
    const res = http.get(shortenLink);

    check(res, {
        "is status 200": (r) => r.status === 200,
        "verify original url": (r) => r.url === DUMMY_ORIGINAL_URL,
    });
});
```

Finally, we configure out `BASE_URL` to [point to our newly created Staging environment](https://github.com/ngshiheng/atomic-url/blob/f1ff25a00c1430314b45dde855497b58831b8e3a/scripts/test.js#L15) in the previous section.

To test locally, simply run `k6 path/to/test.js`. That’s all! You may find the [full test script here](https://github.com/ngshiheng/atomic-url/blob/f1ff25a00c1430314b45dde855497b58831b8e3a/scripts/test.js).

In case you are thinking about running load tests, read [how to determine concurrent users](https://k6.io/blog/monthly-visits-concurrent-users/) in your load test.

## GitHub Actions

I’ll be glossing over this section as it is pretty straightforward. You may refer to the final [GitHub Actions workflow file here](https://github.com/ngshiheng/atomic-url/blob/f1ff25a00c1430314b45dde855497b58831b8e3a/.github/workflows/ci.yml).

Let’s piece together everything we have. Below are the GitHub Actions that we’ll need to use:

-   [wrangler-action](https://github.com/cloudflare/wrangler-action/tree/1.3.0) — For deploying Cloudflare Worker using Wrangler CLI
-   [k6-action](https://github.com/marketplace/actions/k6-load-test?version=v0.2.0) — For running k6

One thing to note about the workflow file — to make a job depend (need) on another job, we’ll make use of the [`need syntax`](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idneeds).

### Actions Secrets

This project requires 2 Actions secrets:

1. `CF_API_TOKEN` — To be used by Wrangler GitHub Action to automatically publish our Cloudflare Worker to its respective environment. You can [create your API token](https://dash.cloudflare.com/profile/api-tokens) using the `Edit Cloudflare Workers` template.
2. `NPM_TOKEN` — This project also uses [semantic-release](https://github.com/semantic-release/npm) to automatically publish to [NPM](https://www.npmjs.com/). To enable this, you will need to create an `NPM_TOKEN` via [npm create token](https://docs.npmjs.com/getting-started/working_with_tokens#how-to-create-new-tokens).

To add it to your GitHub repository secrets, check out [this guide](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository).

If you had taken a look at the final workflow file, you may have noticed the syntax `${{ secrets.GITHUB_TOKEN }}` and wondered why I didn’t mention anything about adding `GITHUB_TOKEN` to our project Actions secrets. Turns out, it is [automatically created and added to all of your workflows](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret).

---

## Closing Remark

Understandably, serverless platforms are generally known to be hard to test and debug. However, that doesn’t mean that we should ignore it.

So, what’s next? Right on top of my head, we could do better. Here are a couple of improvements that we can make:

1. Add a job/stage that automatically rolls back and reverts commit when the smoke tests fail
2. Create an individual testing environment upon PR creation so that we can run smoke tests on them
3. Probably overkill for this project — implementing [canary deployment](https://octopus.com/docs/deployments/patterns/canary-deployments) sounds like a good challenge

## References

If you’re looking into unit testing Cloudflare Workers, here are my recommendations:

-   [https://findwork.dev/blog/testing-cloudflare-workers/](https://findwork.dev/blog/testing-cloudflare-workers/)
-   [https://blog.cloudflare.com/unit-testing-workers-in-cloudflare-workers/](https://blog.cloudflare.com/unit-testing-workers-in-cloudflare-workers/)
-   [https://blog.cloudflare.com/unit-testing-worker-functions/](https://blog.cloudflare.com/unit-testing-worker-functions/)

Here’s a decent video about setting up an ideal yet practical CI/CD pipeline:

-   [https://www.youtube.com/watch?v=OPwU3UWCxhw](https://www.youtube.com/watch?v=OPwU3UWCxhw)

---

_This article was originally published on [jerrynsh.com](https://jerrynsh.com/how-i-setup-ci-cd-pipeline-for-cloudflare-worker/)_