All the hard work in setting up consumer and provider apps with a fully-integrated CI/CD workflow is now done. Hooray! Now though, it’s time to turn our eyes towards good practice when it comes to writing contract tests for realsies.
The contract that I generated for the provider’s
/ping endpoint - that it returns a hard-coded piece of text - is a bit lacking to put it politely. For demonstration purposes of setting up a workflow, it’s sufficient. As a reminder, this is what it looks like:
Pretend you didn’t see that, as we discuss how to make the most from contract testing.
What Should Contracts Test?
Contract testing and functional testing are distinct activities. Be careful about mixing them. Whenever a consumer writes a contract, it places constraints upon the provider’s API. This isn’t itself a bad thing - the purpose of a contract test is to encode the shared understanding about usage, so that an automated signal exists for breaking changes. The constraint is “don’t break my usage”. But this constraint, enforced too particularly, limits the ability for the API to evolve.
Contract testing veers into functional testing when the contracts end up expressing the business logic of a provider. Instead, contracts should be written from the perspective of the consumer: how is the API used? What must be constrained to prevent a breakage in the consumer application?
In the example above, the consumer app is calling the simple
/ping endpoint and displaying the response text. Does it matter if that changes from ‘pong’ to something elese? As the consumer app currently uses it - not at all! The constraint can be loosened into a type - the contract expresses the response must be a string.
There will be times when an exact value will be necessary to assert, I’m sure of it. There may be times where a certain format is expected, such as a URL containing
https:// (so from a consumer’s perspective, their usage in an
<a> tag works correctly). There will also be times where a particular structure to the response object needs to be specified in the contract.
Pact contract tests let contracts be expressed through:
- Exact values
- Types (string, boolean, integer, uuid, and some built-in date-time matching!)
- Object/array ‘shapes’
The exercise of consumer-driven contract tests is not for a consumer to dictate how an API behaves. It’s to have a conversation and form the shared understanding. Constraints placed in a contract should be expressed as loosely as possible, whilst still giving convidence that the consumer use case is satisfied. Keep this principle in mind while developing contract tests to ensure the contracts don’t become proxy-functional tests.
Taking a step back, it’s common to see a
Given / When / Then pattern form in testing. Also expressed as
Arrange / Act / Assert, the idea is that sometimes there’s some pre-test setup that’s needed to establish the context of the item under test. Contract tests are no different - think of applications that communicate with a database, or have user-driven workflows. When contract testing a particular API, it’s useful to be able to request that the provider performs some setup code first.
This is where provider states are useful. The simple
/ping example from earlier in the series did specify a provider state - but it was being ignored. So let’s do something more interesting!
I’m creating a new API so that for a given user, it returns some favourite quotations from TV shows. The rules of the contract are that:
- When there’s no user, return 404
- When there’s a specified user, return a list of quotations
- Quotations need to have the quote text, an author and a source in the form of a URL.
Why is there no need to contract test the empty list case, you may ask? Well, this is where it’s worth recalling the line between contract and functional testing. For me, that leans closer to the functional testing side of things, because the contract around the ‘shape’ of the array satisfies the contract. So, I opted to omit this case.
On the consumer side, I can express this as a contract test:
Here, I’m using a mix of Pact features. The source URL is specified using a (rather naive) regex - as it’s important to the consumer app this link is fully-resolvable. Using
eachLike is a handy convenience method for specifying an array of JSON objects, based on a sample object. This sample object uses strings for the
author, but unlike the earlier contract for
/ping, these aren’t absolute values. Rather, they’re matching the type of string, but using the sample values as an example.
Recall I’m using the
jest-pact library, which is a wrapper on
pact-js. This means after each test I don’t need to call
verify myself. But, I do need to
await the call to
provider.addInteraction. As there’s multiple of these now, I opted to move individual interactions into each test, rather than in the catch-all
Note that each test has a different
state value. These are provider states which must be implemented on the provider side, during provider verification.
There’s now a Plain Ol’ Java Object to represent a quote:
Alongside a service class which has a pre-canned Quote response. As a
@Component, it’s injectable in the Spring Boot context and effectively made a singleton. This means that I can define a public method to modify the pre-canned response - which is what I’ll be doing later during provider verification to satisfy the provider states.
Finally, the new
GET endpoint for quotes. This uses Spring Boot dependency injection to
@Autowire the singleton instance of
QuoteService. Upon receiving a call to get quotes, it gets the list from the service. If it’s empty a 404 error is returned. Otherwise, the list is returned as a JSON array. Sure, this isn’t production-quality either, but it demonstrates the contract well enough.
Now, onto provider verification. Note that I’d previously mentioned the provider application performs two variants of provider verification: one using
pact-cli (where the app is treat as a black box), another using
junit5-spring (which uses JUnit and can interact with the Spring setup at test-time). This is complete overkill - choose one of them only. I only did this for completeness in an earlier post. But, I’ll continue, and implement provider states in both ways.
Provider Verification via pact-cli
To the DemoApplication class, I added a POST endpoint to handle provider states. Provider verification, when performed using the pact-cli tool, posts a JSON object for each provider state it encounters:
For convenience sake, I mapped a POJO object for this. For the provider states known about, the QuoteService’s quote list is modified:
pact-cli verify, there’s an option to pass in
Provider Verification via junit5-spring
Curiously, I couldn’t find a way to get the
junit5-spring library to work with a provider state url, so I ended up duplicating the provider state setup in the tests. As this is a Spring Boot test, the test class itself can utilise the DI context, so it’s possible to inject the same instance of
QuoteService that the application receives, and configure it appropriately. Each provider state is given its own method to perform its setup, and annotated with
@State("<the state name>") so that it is called at the appropriate time.
When running Provider verification, the output looks quite like the
Given / When / Then format. This shows that it’s useful to think of the provider state in that way. Don’t overspecify the provider state - exactly how the provider wants to implement that is their business - but it forms part of the conversation and shared understanding between the consumer and provider. My provider state represents a use-case: “a comedy-loving user”, rather than an overspecified “a user who has a Frank Reynolds quotation saved”.
Verifying a pact between pact-consumer and pact-provider Given no user a request for quotes with GET /quotes returns a response which has status code 404 Given a comedy-loving user a request for quotes with GET /quotes returns a response which has status code 200 has a matching body
Well, this has been fun! Over four posts, I’ve built a full contract testing workflow using Pact Broker and Gitlab. It’s been fun! I’ve learnt a lot. I hope you have too. Please get in touch (details in footer) if you’d like to discuss more in private, or have any suggestions for where to go next.
In my opinion, contract testing is an essential step in creating, maintaining and operating a distributed service-oriented architecture. Done well, it avoids the need for “big bang” deployments and costly phases of manual end-to-end testing. With integrations such as Pact Broker, it can also act as a vehicle for Continuous Delivery by letting individual services be deployed to an environment with surety they’re compatible with the existing setup in that environment. Obviously, other contract testing techniques than Pact exist, but what I like especially about the Pactflow is how it’s specifically designed to be an enabler of good practices.