Building an API consumer-first with Pact
At the start of April I decided to finally take the leap and try and build my own app, based on my personal project of automating my CV with FRESH and HackMyResume.
I’m currently going through the idea validation stage of the app but as the app will ultimately need a backend I decided to invest some time building a basic API with Create, Read, Update & Delete (CRUD) functionality for the CV data.
This CRUD API was the perfect excuse for me to try out Pact, a tool I’d used previously for testing my API consumer code in another project but never for it’s purpose as a consumer-driven contract testing solution.
I really enjoyed using Pact and the paradigm shift it offered so I thought I’d share my experience using it and I’ve created a demo project you can download to try it out yourself.
What is consumer-driven contract testing?
Contract testing is the act of validating a contract, in the case of an API this would be the contract of what the API endpoint expects to receive and what it returns based on the input.
It’s common practice when building an API to produce some form of document, either in the form of documentation or API definition using a tool like Swagger.
While Swagger is a great tool (and it compliments Pact well) it has some flaws:
- The Swagger document (even those generated as part of the code base) can become incorrect if the implementation is updated but the documentation is not
- The Swagger document is usually used one-way with the API consumer, using it to test the contract and the API provider never verifying the contract is maintained
- As an API provider it’s hard to visualise which services are consuming the API and which properties they are actually using as there’s no means of the consumer telling Swagger this
Consumer-driven contract testing flips this top-down, one-way communication flow with the API consumer defining how they are using the API and the API provider using the definition to test that it honours the ‘pact’.
This is a powerful idea as it means that the API provider is free to update any endpoints or properties that are not used by any of the API’s consumers, allowing the API to evolve while maintaining quality.
Pact’s Broker service also provides a great visualisation of the different consumers for an API provider which helps in scaled micro-service based deployments where an API provider could have numerous consumers all requiring different properties.
Designing the API
As mentioned earlier, Swagger is a great tool and it’s still by far the best tool I’ve ever used for jumping in to sketch out an API, so I started by creating the following Swagger doc for the API.
Usually the Swagger document would be used to create a mock server for a consumer to use, to test its API integration code, but instead I used Pact.
Building the API consumer
It can feel a bit odd to build the API consumer before there’s even an API to consume but when working in a distributed team I’ve done this a number of times with both front-end and back-end teams building against the same API definition, integrating later which has always caused issues (something Pact solves).
I built a new module in my front-end repository to integrate with the to-be API and created a test suite for that module in Jest, using Pact to define the request to be sent to the API provider and the expected response.
Pact then uses these definitions (called a pact) to create a mock server that the front-end API requests will be sent to and will return the expected response, allowing the functions to be integration tested without the API provider needing to be accessible.
One gotcha to look out for is when you have tests that match against the same response status, say a test for a 401 returned because the user doesn’t send an authentication header and a 401 because the authentication token is invalid, in this instance Pact will only add the first interaction for the response code to the final pact.
I then published the pact to the Pact Broker, which is a server that manages the pacts for other services to validate against, test results and relationships between the API consumers and providers.
Setting up the Pact Broker
Pact offer a Docker image for the Pact Broker which is perfect for local testing as it can be spun up quickly but they also provide a free tier on their paid for service Pactflow that allows for 5 pacts to be held.
I’ve created a Docker Compose file you can use to set up the Pact Broker which will make the service available at http://localhost:9292 (use
test for both the username and password to log in).
Once the pact publish job is configured to upload the pact to the Pact Broker and the pact is published, the consumer’s pact will be available in the Pact Broker’s UI.
The next step is to use the Pact Broker to allow the API provider to download the API consumer’s pact, verifying whether the API provider honours that pact and uploading the results to the Pact Broker.
Building the API provider using Pact for TDD
The ability to download the API consumer’s pact and verify conformance shows the power of consumer-driven contracts as now we’re able to use the already verified contract in order to practise Test Driven Development (TDD).
The test suite for an API provider is less verbose compared to the one built for the API consumer. This is because the concerns of these test suites are verifying the pact is honoured instead of defining it.
Once the test suite is run the test suite results will be uploaded to the Pact Broker and results will be visible for everyone to see. This means that should the tests fail then it’s easy to see which consumers are impacted by changes.
Red, Green, Refactor
In order to get my initial test suite passing for the API I relied heavily on stubbed values but of course the final app would use a proper authentication service to do this, which can be hard to manage when using Pact.
Pact’s JSON structure is static, so authentication tokens and other potentially dynamic data needs to be available to both the consumer and provider at time of the tests running.
Pact suggest a few ways to mitigate the issue such as using long lived authentication tokens or to have authentication as a separate layer, allowing Pact to run against the unauthenticated endpoints.
When refactoring my API to use firebase, I opted to keep the stubbed values but use Jest’s excellent mocking features to build mocks for the
firebase-admin modules, returning the stubs within the mocks.
This allowed me to move all my test logic out of the controllers and into those underlying functions and is much better than my previous approach of using a strategy pattern to return stubs based on the
test (Jest sets this at runtime).
As the API grows in functionality and number of consuming services it becomes important to manage test data and ensure a consistent set of values being used for behaviour flags for stubbed data.
I managed this by creating another test data module that stored all the request and response payloads as well as values for authentication tokens and IDs, allowing me to change any test values without breaking the tests due to data mismatches.
In my demo project I use this technique also, with the test data being found in the
By using consumer-driven contract testing I was able to verify my front-end would work with my API before I had even built the API itself and then used the pact from the consumer to build the API using TDD.
As someone who has been on multiple teams where we’ve either been given a Swagger document that turned out to be out of date or had to spend time reworking API integrations due to the back-end developers not interpreting the Swagger document the same way the front-end developers did Pact is an amazing solution.
In my app development I’ve used Pact’s Pactflow service in my Continuous Integration (CI) in order to verify the pact on every push, using the app’s version straight from the
package.json to ensure versioning is reported properly.
As my app grows in size I look forward to seeing how Pact scales as more consumers get built to handle more focused jobs using the data from the CRUD API.
Check it out yourself
I’ve created a demo project that contains a API consumer, API provider and a Docker Compose file to run the Pact Broker that can be used to play with Pact and see how consumer-driven contracts testing can bring a higher level of quality to API powered applications.