Contract Testing with Pact.io
By: Domenico Angilletta
Tags: Testing, API, Quality Assurance, Microservices, Software Development

How to deal with testing external services. With a Ruby on Rails example.

Imagine you work on a complex product that requires multiple smaller applications to work with each other, for example an eCommerce platform made of a service for user profiles, inventory, payments, recommendations, etc… and each of those services is developed and maintained by a different team in your organization. In an environment like this, how can you make sure that a change on one application does not break another application that relies on it?

In this article I am going to to show you how you can use consumer-driven contract tests using pact.io to solve this problem, allowing you to test each service in isolation, thus in an efficient way, by stubbing external requests and at the same time ensuring that those “stubs” are an accurate representation of the external service.

Photo by Chris Liverani on Unsplash

Contract testing is a methodology for ensuring that two separate systems (such as two microservices) are compatible with one other. It captures the interactions that are exchanged between each service, storing them in a contract, which can then be used to verify that both parties adhere to it. (pactflow.io)

Before going into contract testing and pact.io, let’s first see other approaches and why they might be less efficient or secure than the contract testing approach.

End-to-end Testing

One way of making sure that a change does not break any other application is to test the full platform functionality end-to-end by doing the real calls between the services.

The problem with this approach is that it is slow, expensive and potentially frustrating.

In order to detect a failure on the consumer side when the provider changes, you would need to run all tests that possibly involve calling the part of application you are changing. It can be very tricky to determine subsets of functionalities that can be affected, thus you would end up running the full test suite for the whole platform. Those tests would have to perform real calls between services, create database entries, move data around, and for this reason they are very slow.

On top of that, they tend to be unstable, since they rely on networking and probably also on data being available on the different services. Whenever you have a failing test you have to figure out if it is a random failure due to some networking or service unavailability, if any data changed, or if it is really caused by the changes you want to release.

This can block or at least delay releases. One of the biggest advantages of microservices is that it allows a big organization to create small sized teams that work independently from each other on different services, so that they can implement new features and deploy them as soon as they are ready, without having to coordinate with any other team. If you can’t release your change because you get blocked by flaky tests or slow end-to-end runs, you may slow down the general development speed, and lower the developer happiness.

The main reason for microservices is speeding up development

— Adrian Cockcroft, former cloud architect at Netflix, in Scalable Microservices with Kubernetes

Stub external requests and test each service in isolation

Since end-to-end tests are slow and not always reliable, normally the interaction with an external service is faked inside the test suite of each application. Based on the technology you are using there are different ways to do this. In Ruby on Rails for example you could use a gem like VCR, to record the real response to an external service the first time you run the test, and afterwards use the stored response (cassette) instead of calling the external service again. Or you could use Webmock to manually define the response that you want to get when you make a specific call.

Mock external service with Webmock

This makes the test suite faster, more reliable and isolated from external factors. But that’s also the problem. The test suite would not fail if the external service suddenly changes the API. In the example above, if the user profile service removes the email field from the response, the test would still pass, but because of this change our application would start failing when interacting with the remote service for real.

So the question is: how can you make sure that those mocks actually represent the current version of the external service? Let’s see how pact.io and consumer-driven contract testing solve the problem.

Consumer-driven Contract Testing

The basic idea is this: whenever you need an interaction between a consumer and a provider, create a contract for it, which specifies that for a given request, the provider has to respond in a certain way.

Example of contract using Pact

The contract that defines the interaction is used by the consumer to mock the requests to the external service, and by the provider to ensure that whatever change it makes, it still respects the contract. With this in place, removing the email field from the response would make the provider be not compliant with the contract anymore, and should prevent the release of the change.

Example Application

On Pact.io you can find libraries for many different languages, which will help you to generate the contract, use it to mock the response for the consumer tests and verify changes on the provider side.

I am going to use Ruby on Rails with the pact-ruby gem and, like in my previous article, it will be about a blog application that interacts with an external service, in this case a user profile service that provides user specific data.

Consumer: Blog

Let’s start with the consumer application.

The Remote::User class defines a find class method, that calls the remote UserProfile service to fetch user data. It then uses name and email contained in the response to build a new User instance, after transforming their values to lower case. Notice that the base URL of the external service is defined as an environment variable, which will come in handy when mocking the service using pact .

The webmock_example_spec.rb we saw earlier shows how you would test this code using the Webmock gem.

Next let’s see how to we can test this code by still stubbing the remote service, but at the same time generating a contract JSON that will be used by the provider to ensure the API it implements matches the mock behavior. First of all we need to add pact to our Gemfile and bundle install . See the readme for more details.

Now we can create a mock service on localhost:1234 , which will respond to our application’s queries over HTTP as if it was the real User Profile Service app. It also creates a mock provider object which we will use to set up our expectations. The method name to access the mock service provider will be whatever name we give as the service argument — in this case user_profile . The value we give to service_consumer and has_pact_with will set the Consumer and Provider values in the generated contract pact.json file.

Ok, we have a mock server running on localhost:1234 , which we can access using the object user_profile in order to add expectations, basically saying how the mock server should reply given certain conditions and requests.

The blog application sends a request to the User Profile service at the URL specified in the environment variable USER_PROFILE_BASE_URL . We want to stub this request in our tests with the mock server we defined before. We can do that by using a test environment variables configuration, that sets USER_PROFILE_BASE_URL=lcalhost:1234 . I use dot-env to define different configuration files for each of the environments, but you can also set it in your shell.

The way of specifying the expectations looks very similar to the Given-When-Then style used for example by Cucumber. First you define a precondition given('...') or ProviderState how it is called in pact, then the event upon_receiving('...').with(method: get, path: '...') , and finally the expected result will_respond_with(...).

After running bundle exec rspec you should see your specs pass, meaning that it correctly used the mocked service, and a new file should appear in spec/pacts which specifies the contract.

It is very easy to see how the definition of the mock-service in user_profile_provider_helper.rb and the endpoint mock in user_spec.rb map to this contract. At this point you might want to publish the contract to a pact-broker so that the provider service can verify that his API complies to the contract. You could run your own broker server or you could use pactflow.io which allows you to setup one in the cloud. To get more details about how to publish the contract to a pact-broker checkout the Pact Broker Client gem. For the scope of this article, we will just access the contract file locally, without using a broker.

Provider: UserProfile

Now let’s see on the provider application, how to verify that the API is compliant with the contract.

The implementation of the API endpoint is trivial. It just looks up the user by id, and returns it as JSON.

In this case pact is not used to mock any external service. Instead we need to define and link all contracts the API should comply to, so that we can run a verification task that will ensure that the consumer expectations are met.

Before that, we need to do some setup tasks: first add pact to your gems, then add require pact/tasks to your Rakefile, which will allow you to run rake pact:verify . See pact-ruby ’s readme for more details.

To specify the contract to which the provider has to comply we add a new file spec/support/service_consumers/pact_helper.rb

We indicate that the User Profile application honors a pact with the Blog application, and that the contract can be reached at a specific pact_uri . Since the contract we defined earlier contains a provider state, a user with id 100 exists , we need to tell the provider application how to translate this into code: create a new user record with id 100 before running the test, and delete it after the test is over. And that’s it. Let’s run the verify task.

The task confirmed that the API honors the pact with the Blog application. Last step, let’s remove the email field from the response to be sure that the verifier actually is able to detect the change.

Let’s run the verifier task again.

As expected the verify task returns a failure and more specifically it tells us that the “email” key was not found.

Conclusion

In this article we saw how contract testing can allow you to test your application in isolation by stubbing an external service call, and at the same time translate the stubbed call into a pact contract JSON, which in turn can be used by the external service, the provider, to verify that its API meets the consumer expectations.

You can checkout the full code used in this example here.

References