Contract-based testing 3: Accept, or get out
By: Sander Van Beek
Tags: Testing, Paradigm, Contract Testing, Provider Driven, Provider

In part 1, I’ve talked about what contract-based testing is. In part 2, I continued with why we should prefer contract-based testing over end-to-end testing.

In this part, I want to start with how contract-based testing works. While doing so, I’ll also go into the details of the provider-driven approach. Sadly, the consumer-driven approach will have to wait until part 4.

Image by Vilas Pultoo

If you remember from part 2: When contract-based testing, we couple each application with a single file: A contract. The contract is the only coupling between applications. We will then use the contract when testing our application in isolation.

Central contract repository

Contract-based testing only works if both the provider-side and consumer-side use the same contract. This means that sharing contracts has to be done right.

To effectively share contracts, we use a central contract repository. This is a place where everyone can discover, publish, and download contracts. The central repository is essential in making sure that everyone is always using the latest contracts. To this end, you must never share contracts via mail, chat, or any other medium that is not the central repository.

The central contract repository is the only infrastructure an organization needs to test contract-based, regardless of scale.

Where to start

Image by Vilas Pultoo

Contract-based testing calls for two things to be made for every integration; the integration and its contract. When we start with writing the contract, we creatively call it contract-first. If we start with building the integration instead, it’s called implementation-first.

When testing contract-based it’s always better to work contract-first. A contract is a way to encode specifications. Starting with the contract allows us to use the contract as the specification document during the development cycle. While the interface is being built, we can use the computer-readable nature of contracts to create the tests for our interface. Making the contract a fundamental part of our tests helps us to deliver quality fast.

For existing interfaces, you are working implementation-first by definition. Working this way negates a bunch of the process advantages you get from working contract-first, but you can still test contract-based. How we handle implementation-first depends on your approach to the contract.

Approaches

In part 1, I mentioned that there are two ways to approach the contract; provider-driven and consumer-driven. It’s finally time to take a closer look at these approaches to contract-based testing.

Provider-driven contract-based testing

Image by Vilas Pultoo

In the provider-driven approach, the provider writes the contract. The provider makes sure that the contract matches the implementation. When they’re sure about the contract, they’ll share the contract with all consumers. The consumers use the contract as a part of their tests.

Image by Vilas Pultoo

When there are multiple consumers they will all use the same provider contract. This makes the approach great for interfaces with multiple consumers.

The contract

A provider-driven contract contains the definition and documentation of the interface. Only the provider can change it. It describes all the technical details like endpoints, HTTP status codes, response body structure, etc.

Technical details are great for computers, but they won’t be the only ones using the contract. To make sure those pesky humans can also use it effectively, the contract contains examples and descriptions. This makes writing a provider contract feel like writing the documentation of the interface, just in a standardized format.

A high-quality provider-driven contract …

Covers the entire interface. It omits nothing.

Is as specific as possible so it leaves no room for assumptions. Note that is this goes for both humans and computers. Humans, in particular, are notorious for making wrong assumptions. But computers are also known to do this through defaults.

Adopts all standards and guidelines. Think of the contract standard itself, additional company standards, and general best practices.

Where to start

Working contract-first allows us to use the contract as a fundamental part of our interfaces tests from the get-go. This allows the provider to catch any misalignments between the interface and the contract throughout the interfaces development cycle. Any misalignment will cause tests to fail on the provider-side.

For existing interfaces, you would generally be best off by generating the contract from the implementation code. This allows the consumer to reap most benefits of the contract, while the provider barely has to change a thing. However, generating a contract comes at the cost of a lower quality contract, especially for human readers.

Both approaches described here ensure that the implementation and the contract are aligned at all times. No matter where you start, you must always make sure that the interface implementation and the contract are aligned with each other. This is essential as it allows consumers to safely assume that the contract describes the truth.

Provider responsibilities

Image by Vilas Pultoo

In the provider-driven approach, the provider has full control over all aspects of the interface at every stage. “With great power comes great responsibility”. The provider’s responsibilities include:

Writing the contract

Part of this is ensuring that the contract is of high quality. This should not be glossed over as a high-quality contract directly leads to high-quality contract tests for both the provider and the consumers. Building the interface

This makes you a provider in the first place. Aligning the contract with the interface

Alignment between the interface implementation and the contract that describes it is essential. It allows consumers to safely assume that the contract describes the reality of the implementation. Omitting alignment will cause the consumer to test with bad stubs, which will lead to production issues. Publishing the contract

The final responsibility of the provider is to publish the aligned, high-quality contract to the central contract repository.

Consumer responsibilities

Image by Vilas Pultoo

In the provider-driven approach, the consumer starts with an assumption: The contract describes the interface correctly. The consumer is responsible for testing and implementing the integration on their end based on the contract. Their responsibilities include:

Downloading the latest contract

Every test-run the consumer downloads the latest contract from the central repository. This makes sure that the consumer is never testing with an outdated contract. Consuming as described in the contract

When consuming the consumer must never deviate from the contract. Anything that is not described in the contract is subject to change at any given time without notice upfront. When encountering undocumented behaviour, it’s fair to ask the provider to include it in their contract. An incomplete contract is a quality issue for which the provider is responsible. Creating all stubs

The consumer writes tests to make sure their application works as expected. These tests include stubs for the interface. This allows the tests to be executed in an isolated environment. The consumer bases these stubs on the contract. Note that the provider is not responsible for providing any stubs. Validating stubs against the contract

Validation ensures a certain quality level in the stubs and will prevent the stubs from drifting out of sync with production. After the stubs are validated, we can use them in the consumer’s tests like normal stubs are. Doing this will catch most consumer-side issues before they make it to production.

Summing up