Welcome to a new post series! Here, I'm going to investigate building a CD pipeline for a system using consumer-driven contract testing, with Pact (and Pact Broker).

This first post introduces the theory of contract testing, then proceeds to set up a Java Spring Boot API provider app, a React consumer app, generates a consumer-driven contract and then performs provider verification.

📖 Click here for a fully-worked example of the code in this blog post

Why Do Consumer-Driven Contract Testing?

A single component rarely does it all itself - normally relying on other components for service functionality and data. Traditionally, testing this will involve deploying all the components and performing "end-to-end" verification of user workflows, checking that the system works in its integration. As appealing as this sounds, it's difficult to get right. End-to-end tests have a habit of being brittle, difficult to maintain and time-consuming, due to the complexities of setting up and then observing behaviour in unpredictable environments.

Larger systems will have components developed and maintained by many different teams, adding to the challenge of getting everything into working order. Without supporting processes to give teams the confidence they can deploy their piece of the whole without breaking anything, practices develop which make the overall system more rigid: heavy focus on end-to-end testing, co-ordination of deployments, big-bang releases. Bottlenecks, bureaucracy and delays to feedback.

Even with highly automated environment provisioning and observability tooling, teams require faster feedback than the above situation can provide. To shift testing "left" is to query whether the assurances sought from a particular stage of testing can be done earlier in the development process. Do those end-to-end tests need to be done as end-to-end tests? Or could the same failure signal be accurately derived in a simpler environment, that provides faster feedback and reduces the need for rework?

Enter Pact: a programming language-agnostic, consumer-driven contract testing tool. Its purpose is to answer the question "can I safely deploy my service"? It can answer this through contracts - a relationship between a service provider and a consumer, from the consumer's perspective. The consumer expresses its expectations: for a given requet, what's form should a response take, what constraints must be upheld, and what's used?

These contracts are shared with the provider. They shift testing left by:

  • Giving a provider a means to test whether it still fulfills all its contractual obligations, at build-time, in isolation from other services
  • Supplying deploy-time assurances that a new component can be safely released.

Rather than replacing the entire web of deployments and performing end-to-end tests, it's possible to switch out individual components independently and safely. If the contracts are satisfied, the components are compatible.

Pact doesn't remove the need for all end-to-end testing. But combined with a good set of acceptance tests that cover the functional behaviour of a component in isolation, it gets you most of the way there. Of course, there are still operational concerns that Pact doesn't concern itself with. For example, blue/green deployments or staggered rollouts. It can certainly provide the initial compatibility sanity check prior to a deployment. You may still wish to run some test transactions through the integrated system and observe the trace from source to sink(s). The important thing is that end-to-end tests are used where they're most appropriate!

A Worked Example

Time to get started! Firstly, I will set up a Spring Boot application to act as a provider for an API. This API will be consumed by a React application. Then, I'll generate the contract on the consumer side as part of an automated test run, supply it to the provider, and perform provider verification testing of that contract.

Provider-Side API

I used the Spring Boot Initializer to create a Spring Boot app. Adding the Spring Web dependency pulls in spring-boot-starter-web alongside spring-boot-starter-test, which gives me enough to get started with a simple REST API.

When developing API services I normally add a simple 'ping' API call, that in the early days can serve as a proto-healthcheck endpoint. Call /ping, get "pong" back.

The Spring Boot starter kit unzips to contain a single class file, which kickstarts the application. For simplicity's sake, I annotated this with @RestController so I can define the endpoint as part of this class:

@SpringBootApplication
@RestController
@CrossOrigin
public class DemoApplication {

	@GetMapping("/ping")
	String home() {
		return "pong";
	}

	public static void main(String[] args) {
		SpringApplication.run(DemoApplication.class, args);
	}
}

Now, this application will build: mvn clean install. run: java -jar target/provider-0.0.1-SNAPSHOT.jar and serve on http://localhost:8081/ping. Fantastic!

Consumer-Side Setup

I generated a React-based web application using Create React App. I used the Typescript template for additional developer-time type information:

npx create-react-app pact-consumer --template typescript

I defined a class that calls the /ping endpoint on the provider, using the XHR library of the day: npm install --save axios @types/axios:

import axios, { AxiosResponse } from "axios";

class PingService {
    constructor(readonly baseUrl: string) { }

    getPing(): Promise<AxiosResponse> {
        return axios.get(`${this.baseUrl}/ping`, {
            responseType: 'text'
        });
    }
}
export default PingService;

And I then hooked this into the main React component, so that "pong" is displayed on screen following a load:

import React, { useState, useEffect } from 'react';
import logo from './logo.svg';
import './App.css';
import PingService from './pingService';

function App() {
  const [message, setMessage] = useState("");

  useEffect(() => {
    new PingService('http://localhost:8081')
      .getPing()
      .then(r => {
        setMessage(r.data);
      });
  }, []);

  return (
    <div className="App">
      <header className="App-header">
        <img src={logo} className="App-logo" alt="logo" />
        <p>{message}</p>
      </header>
    </div>
  );
}

export default App;
Create React App, displaying text from API response

Defining a Consumer-Driven Pact

As discussed, it's the responsibility of the consumer to define its interactions with and expectations of a service provider.

With Pact, an API request and response expectation is encoded as a contract, known as a Pactfile. The process for generating these is simple, and should run as part of the consumer's automated test suite.

Consumer makes API requests to mock provider

A Pact test sets up a mock server, then specifies that for a given request, a particular response should be sent back. The test will then execute consumer code to make this API call and handle the mocked response. The overall interaction is written to a Pactfile, which is then committed to source control and shared with the provider. The test may then perform additional assertions, such as checking the consumer code handles the request correctly.

To set up a Pact mock provider service, I pulled in the following dependencies:

npm install --save @pact-foundation/pact @pact-foundation/pact-node jest-pact

Let's look at what this interaction looks like, as captured in a Jest test:

import { AxiosResponse } from "axios";
import { pactWith } from 'jest-pact';
import PingService from './pingService';

pactWith({ consumer: 'pact-consumer', provider: 'pact-provider', cors: true }, provider => {
  let pingService : PingService;

  beforeEach(() => {
    pingService = new PingService(provider.mockService.baseUrl)
  });

  describe('ping API', () => {
    beforeEach(() =>
      provider.addInteraction({
        state: "ping",
        uponReceiving: 'a request',      
        withRequest: {
          method: 'GET',
          path: '/ping',
        },
        willRespondWith: {
          status: 200,
          body: "pong",
        },
      })
    );

    it("sends a request", async () => {
      const response: Response = await pingService.getPing();
      await expect(response.status).toBe(200);
    });
  });
});

Note that I had to add the cors: true option to get Axios to play nicely with access to the Pact mock provider server, running on a random localhost port.

Using a neat mini-DSL expression, an interaction is defined. The request (verb and path) alongside the response (status and body text) are specified. Notice there's also a state field - this will be useful in future when we touch on provider states.

On running the tests with npm test, the contract is outputted to a pact directory within the consumer project.

{
  "consumer": {
    "name": "pact-consumer"
  },
  "provider": {
    "name": "pact-provider"
  },
  "interactions": [
    {
      "description": "a request",
      "providerState": "ping",
      "request": {
        "method": "GET",
        "path": "/ping"
      },
      "response": {
        "status": 200,
        "headers": {
        },
        "body": "pong"
      }
    }
  ],
  "metadata": {
    "pactSpecification": {
      "version": "2.0.0"
    }
  }
}

Changes to a Pactfile then form part of code review on the consumer side to ensure that interactions have been correctly captured. I'll cover in a future post how to make the most of a contract test - in general, aim to express only what's needed and what's used on the consumer side. Don't fall into the trap of testing business rules in the contract.

Provider Verification

Finally, over to the provider side. In an ideal world, I'd want a mechanism that facilitates the sharing of Pactfiles between consumer and provider automatically. This is crucial if there are separate teams involved in the consumer/provider divide, but also reduces operational overhead. I shall address this properly later in the series. For now, I'm simply going to copy/paste the Pactfile over into the Provider's source code. As it's a testing-time resource, Java convention dictates that src/test/resources is an appropriate location for this file to live.

Pact provider verification sets up a mock consumer to run against the provider. This is essentially the inverse of the consumer-side testing above. Consumer-side, a mock provider records interactions against it and outputs a Pactfile. On the provider-side, a mock consumer takes a Pactfile as an input and uses this to make API calls to the Provider, ensuring those responses conform to the contract.

If, for example, the above service replied with "ping" instead of "pong" - this would violate the contract!

Mock consumer uses contracts to make API calls to provider

There are two ways of performing provider verification. Both are at automated testing time, but the difference is in how they are triggered. For demonstration purposes, I'm going to show both. They are:

  1. Use the pact-cli to set up a mock consumer
  2. (If supported by your programming language) Incorporate a library to set up a mock consumer as part of the test suite.

Firstly, to keep everything ticking over, I need a small change to cope with provider states. As this is a simple API service with a hard-coded response, I don't yet need to think provider states other than to define a no-op POST endpoint. To the singular DemoApplication class, I've added:

@PostMapping("/provider-states")
public void providerStates(@RequestBody Map body) {
  System.out.println(body.get("state"));
}

As more complexity is added to the application, it will become necessary to setup the provider to behave differently based on incoming pacts. Provider State verification is Pact's mechanism for controlling this whilst maintaining test isolation. I'll touch on this more in a later article.

Now, let's cover those two options.

Dockerised, with Pact-CLI

The generic way to run Pact provider verification is using the CLI tool. I opted to Dockerize the Spring Boot application, so that it can be packaged and run alongside the pact-cli tool in a docker-compose setup.

version: "3"
services:
  app:
    image: pact-provider:latest
    ports:
      - "8081:8081"
  pact:
    image: pactfoundation/pact-cli:latest
    depends_on:
      - app
    links:
      - app
    volumes:
      - ./src/test/resources/pacts:/tmp/pacts
    command: >
      verify
      --provider_base_url http://app:8081
      --provider-states-setup-url http://app:8081/provider-states
      --provider pact-provider
      --wait 15
      /tmp/pacts/pact-consumer-pact-provider.json

The tests can then be ran with docker-compose up --abort-on-container-exit --exit-code-from pact. The additional argument ensures that after test completion, everything is torn down, and the exit code from the pact-cli image is used as the overall exit code. Without this argument, the Spring Boot service will remain up and accessible on localhost:8001.

In the docker-compose file, note that I'm mounting the /src/test/pacts volume into /tmp/pacts of the pact-cli image and then instructing pact-cli, through its verify command, which Pactfile to verify, and against which service. As docker-compose uses DNS to references running containers, the app is accessible on http://app:8081, as that's the name I gave that container within the compose file.

I personally prefer this solution as it's neat, generic, and ties closely with how it will run in CI (spoilers!!).

Pact Provider Verification as a Spring/JUnit Test

In JVM-land, it is alternatively possible to incorproate Pact provider verification as part of a JUnit suite.

I'll demonstrate what is needed for a bare-bones provider verification test - but to repeat for clarity - you only need one or the other!

First, I declared the test-scoped dependency for the junit5spring Pact provider library:

<dependency>
  <groupId>au.com.dius.pact.provider</groupId>
  <artifactId>junit5spring</artifactId>
  <version>4.1.17</version>
  <scope>test</scope>
</dependency>

Next, I set up a new ProviderVerificationTests class. This is configured to run as a Spring Boot test on a random port, so a @BeforeEach block is necessary to instruct the Pact provider context what port to look for on localhost.

With JUnit5, it's possible to autogenerate tests from a @TestTemplate, which means that given the contents of a @PactFolder, individual tests can be auto-generated for each interaction in the Pactfiles.

Finally, provider state set up is handled . As this isn't being used yet in my Pactfiles, I can go ahead and create an empty method.

The resulting test class is:

package co.uk.samhogy.pact.provider;

import au.com.dius.pact.provider.junit5.HttpTestTarget;
import au.com.dius.pact.provider.junit5.PactVerificationContext;
import au.com.dius.pact.provider.junit5.PactVerificationInvocationContextProvider;
import au.com.dius.pact.provider.junitsupport.Provider;
import au.com.dius.pact.provider.junitsupport.State;
import au.com.dius.pact.provider.junitsupport.loader.PactFolder;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.TestTemplate;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.boot.web.server.LocalServerPort;

@Provider("pact-provider")
@PactFolder("pacts")
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class ProviderVerificationTests {
  @LocalServerPort
  private int port;

  @BeforeEach
  void before(PactVerificationContext context) {
      context.setTarget(new HttpTestTarget("localhost", port));
  }

  @TestTemplate
  @ExtendWith(PactVerificationInvocationContextProvider.class)
  void pactVerificationTestTemplate(PactVerificationContext context) {
    context.verifyInteraction();
  }

  @State("ping")
  public void setupPingState() {

  }
}

If you've complex Spring set up then this option may be more suitable than using pact-cli directly.

Next Steps

I now have barebones front- and back-end applications with a contract for a simple API call, with provider verification of that contract. For demonstration purposes I covered two ways of performing provider verification in Java - one in docker-compose with pact-cli, another directly in JUnit.

The next stages are to:

  • Get this working on Gitlab CI
  • Create a more complex interaction and utilise provider states
  • Share Pactfiles between the consumer and provider in a better way than copy/paste
  • Demonstrate how contracts can be used to perform deploy-time safety checks

In the next post, I'll move onto Gitlab!