Contract testing at a glance

Testing microservices

Testing is a key ingredient in a successful CI/CD pipeline, especially in the microservices world. We test our classes in unit tests, test business processes in integration tests, even test our UI with visual tests. Our service is tested fully end to end — from frontend JavaScript code to our backend code and database.

However, services aren’t usually a single player in our deployment. Services are often interdependent, either directly by performing http requests to another service or indirectly by sending a message via a message bus.

How do I know that my newly deployed service won’t cause failures in services that depend on it, because I changed my API signature? How do I know that another service I depend on won’t cause my service to fail because now it’s sending a different message?

The simple solution is some form of end-to-end test. Simply deploy your service and its dependencies to a testing environment and run tests to verify that things are still working as expected across the environment.

This is fine when you have a handful of microservices. What happens when you have dozens? Hundreds? Thousands?

This is fine

The dependency chain between services can grow and become an expensive, time consuming and laborious undertaking — I just want to test my service before deployment.

Enter Contract Testing

Contract testing is a testing paradigm that aims to solve this issue. Instead of running expensive end-to-end tests, we define a contract between 2 entities: the consumer and the provider.

A contract describes the interaction between both sides, as a set of requirements that the consumer has from the provider. It’s analogous to interfaces in OOP — The interface is the contract between the calling code (consumer) and the implementing class (provider).

For example, the consumer may declare that when it performs a POST request to a certain endpoint, and that it will receive a response with an ‘OK’ HTTP status and a JSON body:

"consumer": {
    "name": "my-consumer"
  },
  "provider": {
    "name": "my-provider"
  },
  "interactions": [
    {
      "_id": "fd0c3e907b1e128d241810303938b04884b3f242",
      "description": "Get data from provider",
      "request": {
        "method": "POST",
        "path": "/some/path"
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": true
      }
    }]

The contract can also define that this JSON will contain the fields we require in the correct format and type.

Once we have a contract, both consumer and provider can verify themselves:

1. The consumer will receive a mocked response based on the contract.
2. The provider will verify that it can generate the expected response according to a given request.

The important concept to take away from this is that both of these tests occur independently of each other. We test the consumer and provider separately, whenever we need and want, without the need to deploy both of them to a test environment.

This is the power of contract testing!

So how do we actually do it? For that there’s Pact.

What is PACT ?

Baby don’t hurt me, no more

Pact is a contract testing tool; it defines a format for describing a contract (a JSON file) and an API.

On the consumer side, the Pact API will generate a contract based on the consumer requirements from a specific provider and set up a mock http server that returns a result accordingly.

The consumer can then call this HTTP endpoint and verify that it can actually handle the response correctly.

For the provider side, we load the contract file and let Pact perform the HTTP call to the provider service. Pact then verifies that the provider actually returned the correct response based on the contract.

How does the consumer and the provider share the contract between them?
These are 2 different services that can even reside on different code repos.

For that Pact has another tool in its belt, which is called the Pact Broker. Essentially, this is a repository of contracts; when a consumer generates a contract it publishes it to the broker. A provider then can ask the broker for relevant contracts, using a rest API.

Once a contract succeeded or failed verification, we update it in the pact broker.

The broker serves a very important task — it’s the leading authority on whether or not we can actually deploy a service to an environment. Only the broker knows which versions of services work well with each other.

Pact at Tipalti

So how are we using Pact at Tipalti? How does Pact affect our CI/CD pipeline?

First, there are some things to consider:

1. Versioning — each consumer / provider in a contract verification is versioned by the git commit id of the relevant code.
2. Tags — each consumer / provider in a contract verification is tagged with the relevant branch name and any environment it was deployed to (QA, sandbox, production).

This information both allows us to understand the participants of the contract verification and allows us to query the Pact broker.

Let’s take a look at the flow:

Contract tests

The contract tests differ between consumers and providers:

Consumers will verify that they can handle the provider’s response according to that contract.

Providers will query the Pact broker for the relevant consumers and check that the provider responds correctly to each one of their contracts; we search for contracts that are tagged as “prod”, since we want to be sure that we won’t break our production environment with our changes.

These are implemented as unit tests, based on a set of test suites we prepared as part of our testing framework.

Local development / Pull request

Contract tests will run as part of local development and PR tests, the same as any other tests. The results of the contract verification is not published to the Pact broker since the code is not yet part of one of our main development branches.

Post PR

After a PR is completed, we trigger another run of the contract tests on the target branch.
This time the contract test results are published to the Pact broker and tagged accordingly.

Consumers also trigger running contract tests for all relevant providers that are tagged with “prod”. This verifies the contract on both ends (consumer and provider).

Since these builds run in the background, after the consumer PR, we notify the build results via Slack to the relevant team.

Release pipeline

The first step of the release pipeline is to check if we can deploy the current commit. This is done via the aptly named Pact broker “can-i-deploy” command line.

We run this tool to verify that the commit id (which is also the version of the consumer / provider) can be deployed to the “prod” environment:

pact-broker can-i-deploy --pacticipant "MyServiceName" --version 23jsa45bg --to "prod" additional parameters omitted

If this step succeeds, we can continue in our release pipeline and deploy our service.
On each deployment to an environment we also tag the contract with the name of the environment.

If this step fails, this means that our service has an invalid contract and we can’t continue in our release pipeline — thus achieving our goal of protecting our production environment from failing due to our service.

TL;DR

Contract testing is an important tool in our testing arsenal that allows us to easily verify the interactions between our microservices. We use Pact and Pact broker as our contract testing tools to generate, verify and publish our contract tests.