Operation Pact or: How I Learned to Stop Worrying and Love Contract Testing
Tags: testing, devops, pactflow
Published: 2022-09-20 08:44:37

I heard about Contract Tests in detail at the Ministry of Testing Meetup. Abhi Nandan made an excellent introduction talk [Contract testing in Polyglot Microservices](https://youtu.be/PWM8iLqpVdA?t=129).

I liked the idea and decided to try this approach on my current project. I must say that the devil is in the details. In theory, everything is simple until you try the integration in practice on an existing project.
It is the same with algorithms: when you are watching how the developer on YouTube solves the task, everything seems straightforward. You immediately hit a snag while sitting down to solve the problem alone.

Today I want to focus on practical examples, as much as possible close to real-world conditions. I encourage you to try it on your own to gain experience and see if these approaches are worth applying to your projects. We will use the [realword](https://github.com/gothinkster/realworld) project on GitHub, an Exemplary Medium.com clone powered by different frameworks and tools.

I like using this project for the demo because it's much more complex than `HelloWorld`, where everything works most of the time. True-to-life problems give your extensive experience.

## What is so unique in contract tests?

> [Contract tests](https://docs.pact.io/) assert that inter-application messages conform to a shared understanding documented in a contract. Without contract testing, the only way to ensure that applications will work correctly together is by using expensive and brittle integration tests.

You can hear opinions: Contract tests often make sense when you have a microservice architecture with 10-50 services.
I'm afraid I have to disagree with that. If you have at least one Provider (back-end with API) and Consumer (front-end), it already makes sense to try this testing approach.

I suggest starting exploration with **bi-directional contract tests**.

> [Bi-Directional Contract Testing](https://docs.pactflow.io/docs/bi-directional-contract-testing/) is a type of **static contract testing**.
> Teams generate a consumer contract from a mocking tool (such as Pact or Wiremock) and API providers verify a provider contract (such as an OAS) using a functional API testing tool (such as Postman). Pactflow then statically compares the contracts down to the field level to ensure they remain compatible.

Why is it better to start with BDCT? Bi-Directional Contract Testing (BDCT) allows you to use existing Provider Open API. If your current project has a back-end service with Open API specification, you are lucky to leverage it as a Provider contract.

## 1. Provider Contract

Let's configure publishing of **[OAS spec](https://github.com/gothinkster/realworld/blob/main/api/openapi.yml)** from `realworld` app to pactflow.

Requirements:
- Create a [free account](https://pactflow.io/try-for-free/) on Pactflow.io;
- Install [Docker](https://www.docker.com/products/docker-desktop/) on your machine.

We want to be able to publish Open API spec to Pactflow locally and from CI/CD. To do that we will need the following: API key from Pactflow, `docker` and `make` commands installed.

Consider the [Makefile](https://github.com/drakulavich/realworld/blob/main/Makefile):
```makefile
# Makefile
PACTICIPANT ?= "realworld-openapi-spec"

## ====================
## Pactflow Provider Publishing
## ====================
PACT_CLI="docker run --rm -v ${PWD}:/app -w "/app" -e PACT_BROKER_BASE_URL -e PACT_BROKER_TOKEN pactfoundation/pact-cli"
OAS_FILE_PATH?=api/openapi.yml
OAS_FILE_CONTENT_TYPE?=application/yaml
REPORT_FILE_PATH?=api/README.md
REPORT_FILE_CONTENT_TYPE?=text/markdown
VERIFIER_TOOL?=newman

# Export all variable to sub-make if .env exists
ifneq (,$(wildcard ./.env))
    include .env
    export
endif

default:
	cat ./Makefile

ci: ci-test publish_pacts can_i_deploy

# Run the ci target from a developer machine with the environment variables
# set as if it was on CI.
# Use this for quick feedback when playing around with your workflows.
fake_ci:
	@CI=true \
	GIT_COMMIT=`git rev-parse --short HEAD` \
	GIT_BRANCH=`git rev-parse --abbrev-ref HEAD` \
	make ci

## =====================
## Build/test tasks
## =====================

ci-test:
	@echo "\n========== STAGE: CI Tests ==========\n"

## =====================
## Pact tasks
## =====================

publish_pacts:
	@echo "\n========== STAGE: publish provider contract (spec + results) - success ==========\n"
	PACTICIPANT=${PACTICIPANT} \
	"${PACT_CLI}" pactflow publish-provider-contract \
	/app/${OAS_FILE_PATH} \
	--provider ${PACTICIPANT} \
	--provider-app-version ${GIT_COMMIT} \
	--branch ${GIT_BRANCH} \
	--content-type ${OAS_FILE_CONTENT_TYPE} \
	--verification-exit-code=0 \
	--verification-results /app/${REPORT_FILE_PATH} \
	--verification-results-content-type ${REPORT_FILE_CONTENT_TYPE} \
	--verifier ${VERIFIER_TOOL}

deploy: deploy_app record_deployment

can_i_deploy:
	@echo "\n========== STAGE: can-i-deploy? 🌉 ==========\n"
	"${PACT_CLI}" broker can-i-deploy --pacticipant ${PACTICIPANT} --version ${GIT_COMMIT} --to-environment test

deploy_app:
	@echo "\n========== STAGE: deploy ==========\n"
	@echo "Deploying to test"

record_deployment:
	"${PACT_CLI}" broker record-deployment --pacticipant ${PACTICIPANT} --version ${GIT_COMMIT} --environment test

## =====================
## Misc
## =====================

.env:
	cp -n .env.example .env || true
```

Run `make .env` to create `.env` file. You will need to update two variables:
- `PACT_BROKER_BASE_URL`
- `PACT_BROKER_TOKEN`

_[Optional]_ We may try to publish the contract from the local machine using `make fake-ci`. This target under the hood executes the following stages:
1. `ci-test` runs functional tests (skipped in our case)
2. `publish_pacts` uses pactflow docker image to publish current OAS to Pactflow broker. `GIT_COMMIT` and `GIT_BRANCH` environment variables are used to set the version tag and the branch on the broker side.
3. `can_i_deploy` verifies that the published contract is compatible with dependent consumers. Hence we are safe to deploy.

And the last piece. Let's create a [GitHub workflow](https://github.com/drakulavich/realworld/blob/main/.github/workflows/pact.yml) to publish OAS on each push to the repository:
```yaml
# .github/workflows/pact.yml

name: Pact CI

on:
  pull_request:
  push:
    branches:
      - main
    paths-ignore:
      - '*.md'

concurrency:
  # For pull requests, cancel all currently-running jobs for this workflow
  # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

env:
  PACT_BROKER_BASE_URL: https://drakulavich.pactflow.io
  PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  GIT_COMMIT: ${{ github.sha }}

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Inject slug/short variables
      uses: rlespinasse/github-slug-action@v4

    - name: Tests
      run: GIT_BRANCH=${GITHUB_REF_NAME_SLUG} make ci

  deploy:
    runs-on: ubuntu-latest
    needs:
      - verify
    if: ${{ github.ref == 'refs/heads/main' }}
    steps:
      - uses: actions/checkout@v3

      - name: Inject slug/short variables
        uses: rlespinasse/github-slug-action@v4

      - name: 🚀 Record deployment on Pactflow
        run: GIT_BRANCH=${GITHUB_REF_NAME_SLUG} make deploy
```

GitHub will use the same `Makefile`. Please, have a look at the Deploy job from the workflow above. We are executing `make deploy` to explicitly mark the deployed version for the test environment on the Pactflow broker.

We will provide the environment variable to access Pactflow differently. Don't forget to add the secret `PACT_BROKER_TOKEN` in Settings — Secrets — New repository secret.

![PACT_BROKER_TOKEN repository secret](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o2iwzk5ffnouwmmtluy9.png)

Ok. We are ready to commit the files! If everything is fine, the pipeline should be green.

![Pipeline to publish OAS to Pactflow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tqh83q2tgkfahe63aaw0.png)

Open your Pactflow workspace and check out the Provider Contract. You should be able to open Open API Swagger spec.

![Open API Swagger on Pactflow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h4bgx8kest8tqk1bq9w1.png)

So, we finished with the first part. We have CI/CD for Provider Contract. Let's move on to the second part.

## 2. Consumer contract

We will use [ts-redux-react-realworld-example-app](https://github.com/angelguzmaning/ts-redux-react-realworld-example-app) project which implements the front-end in TS for realword app using React/Redux.

If you want to follow the same approach, I suggest you to check [**pact-workshop-js**](https://github.com/pact-foundation/pact-workshop-js) for better understanding.

1) We'll start by installing `@pact-foundation/pact` dependency:
```sh
npm install --save-dev @pact-foundation/pact
```
2) Create directory `src/services/consumer/`
3) Create `src/services/consumer/apiPactProvider.ts`
Here we'll describe the pact specification format and name of our **pact**icipants (participants).

```ts
// src/services/consumer/apiPactProvider.ts

import path from 'path';
import { PactV3, MatchersV3, SpecificationVersion } from '@pact-foundation/pact';

export const { eachLike, like } = MatchersV3;

export const provider = new PactV3({
  consumer: 'ts-redux-react-realworld-example-app',
  provider: 'realworld-openapi-spec',
  logLevel: 'warn',
  dir: path.resolve(process.cwd(), 'pacts'),
  spec: SpecificationVersion.SPECIFICATION_VERSION_V2,
});
```
4) Create `src/services/consumer/api.pact.spec.ts`

Let's start with something tiny and achievable. We would like to check `/tags` endpoint.

```ts
// src/services/consumer/api.pact.spec.ts

describe('API Pact tests', () => {
  describe('getting all tags', () => {
    test('tags exist', async () => {
      // set up Pact interactions

      // Execute provider interaction
    });
  });
});
```

From [**pact-workshop-js**](https://github.com/pact-foundation/pact-workshop-js) you will learn that each Pact test contains two parts:
1. Setup Pact interactions.
2. Execute the interaction using the API client.

The first part might look like:
```ts
// src/services/consumer/api.pact.spec.ts
// ...
      const tagsResponse = {
        tags: ['reactjs', 'angularjs'],
      };
      
      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'tags exist' }],
        uponReceiving: 'get all tags',
        withRequest: {
          method: 'GET',
          path: '/tags',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(tagsResponse),
        },
      });
// ...
```

We created a stub for the response `tagsResponse`. Then we added interaction to the provider object from `src/services/consumer/apiPactProvider.ts`. The essential parts relate to the expected request and how the provider will reply.

In the second part we will add the test execution:
```ts
// src/services/consumer/api.pact.spec.ts
// ...

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const tags = await getTags();

        expect(tags).toStrictEqual(tagsResponse);
      });
// ...
```

Ultimately we need to change the URL for the API client and execute the request after it. In our case we have to alter axios `baseURL`, because all API interactions explicitly use axios. For example, `getTags()` implementation:
```ts
export async function getTags(): Promise<{ tags: string[] }> {
  return guard(object({ tags: array(string) }))((await axios.get('tags')).data);
}
```

We want to verify that response is the same as a prepared stub. After collecting all parts inside the test file we will get the following:
```ts
// src/services/consumer/api.pact.spec.ts

import axios from 'axios';
import { provider, like } from './apiPactProvider';
import { getTags } from '../conduit';

describe('API Pact tests', () => {
  describe('getting all tags', () => {
    test('tags exist', async () => {
      const tagsResponse = {
        tags: ['reactjs', 'angularjs'],
      };

      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'tags exist' }],
        uponReceiving: 'get all tags',
        withRequest: {
          method: 'GET',
          path: '/tags',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(tagsResponse),
        },
      });

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const tags = await getTags();

        expect(tags).toStrictEqual(tagsResponse);
      });
    });
  });
});
```

Let's run `make fake-ci` to check how is going:
```sh
Computer says yes \o/ 

CONSUMER                             | C.VERSION          | PROVIDER               | P.VERSION  | SUCCESS? | RESULT#
-------------------------------------|--------------------|------------------------|------------|----------|--------
ts-redux-react-realworld-example-app | d981433+1663657371 | realworld-openapi-spec | 95dbd23... | true     | 1      
```

Hooray! It's a huge step towards contract testing.
Now let's try to test another endpoint `GET /user`. We will start with the same ideas: create a stub, prepare interactions and execute API client call:

```ts
// src/services/consumer/api.pact.spec.ts
// ...

  describe('getting current user', () => {
    test('user exists', async () => {
      const tUser: User = {
        email: 'jake@jake.jake',
        token: 'jwt.token.here',
        username: 'jake',
        bio: 'I work at statefarm',
        image: 'https://i.stack.imgur.com/xHWG8.jpg',
      };
      
      const userResponse = {
        user: tUser,
      };
      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'user has logged in' }],
        uponReceiving: 'get user',
        withRequest: {
          method: 'GET',
          path: '/user',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(userResponse),
        },
      });

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const user = await getUser();

        expect(user).toStrictEqual(tUser);
      });
    });
  });

// ...
```

If you run `make fake-ci`, you will get an error:
```sh
Computer says no ¯_(ツ)_/¯
```

Follow the generated report link and you will see the details:
```
Request Authorization header is missing but is required by the spec file
```
![Error on Pactflow side](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/193au4k1xeebmij4cr58.png)

We missed Authorization header that is required to check logged in user. Let's add the stub (1), headers to the expected request (2) and axios defaults (3):
```ts
// src/services/consumer/api.pact.spec.ts

      // (1)
      const authToken = {
        Authorization: 'Token xxxxxx.yyyyyyy.zzzzzz',
      };

// ...

        uponReceiving: 'get user',
        withRequest: {
          method: 'GET',
          path: '/user',
          headers: authToken, // (2)
        },
// ...
      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;
        axios.defaults.headers.Authorization = authToken.Authorization; // (3)
```

After next `make fake-ci` attempt you will get the new error:
```sh
FAIL src/services/consumer/api.pact.spec.ts
  ● Console

    console.error
      Error: Cross origin http://localhost forbidden
```

I found the fix for it [on StackOverflow](https://stackoverflow.com/a/67880430/4894316). We need to add the following line on the top of the test file:
```
axios.defaults.adapter = require('axios/lib/adapters/http');
```

Next `make fake-ci` should be successful.

![Pactflow broker UI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6uehg4ngsar8b3gjqx1u.png)

## Summary
We have learned how to configure Provider and Consumer for bi-directional contract testing. We used RealWorld project to practice integration with Pactflow broker. And finally, we wrote a couple of consumer tests for ts-redux-react-realworld-example-app. We also learned how to debug the tests using `make fake-ci` command. All of the examples are available on GitHub. Feel free to try and post your questions if you face the issues.

## Materials
- [RealWorld repo](https://github.com/drakulavich/realworld) with Open API spec (provider contract)
- [ts-redux-react-realworld-example-app](https://github.com/drakulavich/ts-redux-react-realworld-example-app) with pact tests (consumer)
- [pact js workshop](https://github.com/pact-foundation/pact-workshop-js)