Building an API consumer-first with Pact

02/05/2020 — Testing, Automation, JavaScript, Software Development — 6 min read

child interacting with a robot — Photo by Andy Kelly on Unsplash

At the start of April I decided to finally take the leap and try and build my own app, based on my personal project of automating my CV with FRESH and HackMyResume.

I’m currently going through the idea validation stage of the app but as the app will ultimately need a backend I decided to invest some time building a basic API with Create, Read, Update & Delete (CRUD) functionality for the CV data.

This CRUD API was the perfect excuse for me to try out Pact, a tool I’d used previously for testing my API consumer code in another project but never for it’s purpose as a consumer-driven contract testing solution.

I really enjoyed using Pact and the paradigm shift it offered so I thought I’d share my experience using it and I’ve created a demo project you can download to try it out yourself.

What is consumer-driven contract testing?

Contract testing is the act of validating a contract, in the case of an API this would be the contract of what the API endpoint expects to receive and what it returns based on the input.

It’s common practice when building an API to produce some form of document, either in the form of documentation or API definition using a tool like Swagger.

While Swagger is a great tool (and it compliments Pact well) it has some flaws:

The Swagger document (even those generated as part of the code base) can become incorrect if the implementation is updated but the documentation is not
The Swagger document is usually used one-way with the API consumer, using it to test the contract and the API provider never verifying the contract is maintained
As an API provider it’s hard to visualise which services are consuming the API and which properties they are actually using as there’s no means of the consumer telling Swagger this

Consumer-driven contract testing flips this top-down, one-way communication flow with the API consumer defining how they are using the API and the API provider using the definition to test that it honours the ‘pact’.

This is a powerful idea as it means that the API provider is free to update any endpoints or properties that are not used by any of the API’s consumers, allowing the API to evolve while maintaining quality.

Pact’s Broker service also provides a great visualisation of the different consumers for an API provider which helps in scaled micro-service based deployments where an API provider could have numerous consumers all requiring different properties.

Designing the API

As mentioned earlier, Swagger is a great tool and it’s still by far the best tool I’ve ever used for jumping in to sketch out an API, so I started by creating the following Swagger doc for the API.

1openapi: 3.0.0
2# Added by API Auto Mocking Plugin
3servers:
4  - description: SwaggerHub API Auto Mocking
5    url: https://virtserver.swaggerhub.com/colinfwren/pactDemo/1.0.0
6info:
7  description: API for demoing Pact
8  version: "1.0.0"
9  title: Pact Demo API
10  contact:
11    email: lolpointerexception@gmail.com
12  license:
13    name: Apache 2.0
14    url: 'http://www.apache.org/licenses/LICENSE-2.0.html'
15tags:
16  - name: public
17    description: Available to everyone without logging in
18  - name: private
19    description: Available only to the logged in user
20paths:
21  /accounts/:
22    post:
23      tags:
24        - public
25      summary: Create an account for the user
26      operationId: signupUser
27      description: |
28        Using the provided email and password, create the user account
29      requestBody:
30        description: User's email and password
31        required: true
32        content:
33          application/json:
34            schema:
35              $ref: '#/components/schemas/LoginDetails'
36      responses:
37        '201':
38          description: User account successfully created
39          content:
40            application/json:
41              schema:
42                type: object
43        '400':
44          description: Account already exists for that user
45          content:
46            application/json:
47              schema:
48                type: object
49                properties:
50                  message:
51                    type: string
52                    description: error message
53                    example: Account already exists
54        '406':
55          description: User didn't send application/json content-type header
56  /sessions/:
57    post:
58      tags:
59        - public
60      summary: Log the user in with the credentials provided
61      operationId: loginUser
62      description: |
63        Using the provided email and password, logs the user in to the system
64      requestBody:
65        description: User's email and password
66        required: true
67        content:
68          application/json:
69            schema:
70              $ref: '#/components/schemas/LoginDetails'
71      responses:
72        '201':
73          description: User successfully logged in, returns Token for authentication
74          content:
75            application/json:
76              schema:
77                type: object
78                properties:
79                  token:
80                    type: string
81                    description: Auth token for user to use with subsequent calls to server
82                    example: dead-b33f-1337-beef
83        '400':
84          description: User provided incorrect details
85          content:
86            application/json:
87              schema:
88                type: object
89                properties:
90                  message:
91                    type: string
92                    description: error message
93                    example: Account doesn't exist
94        '406':
95          description: User didn't send application/json content-type header
96    delete:
97      tags:
98        - private
99      security:
100        - BearerAuth: []
101      summary: Log the user out
102      operationId: logoutUser
103      description: Log the user out of their current session
104      responses:
105        '204':
106          description: User successfully logged out
107        '401':
108          description: User is unable to be logged out, as they aren't logged in
109  /dogs/:
110    get:
111      tags:
112        - private
113      summary: Lists user's dogs
114      operationId: getDogs
115      security:
116        - BearerAuth: []
117      description: |
118        Gets the dogs for the logged in user
119      responses:
120        '200':
121          description: User fetched all their dogs successfully
122          content:
123            application/json:
124              schema:
125                $ref: '#/components/schemas/DogList'
126        '401':
127          description: User is unable to fetch their dogs, as they aren't logged in
128    post:
129      tags:
130        - private
131      summary: Creates a new dog
132      operationId: createDog
133      security:
134        - BearerAuth: []
135      description: |
136        Creates a new dog for the logged in user
137      requestBody:
138        description: Doggo information
139        required: true
140        content:
141          application/json:
142            schema:
143              $ref: '#/components/schemas/Dog'
144      responses:
145        '201':
146          description: User successfully created dog, returns ID of the new dog
147          content:
148            application/json:
149              schema:
150                $ref: '#/components/schemas/DogList'
151        '400':
152          description: User sent bad data to create dog with, returns a list of paths to the incorrect data
153          content:
154            application/json:
155              schema:
156                $ref: '#/components/schemas/DogUpdateErrors'
157        '401':
158          description: User is unable to create dog, most likely due to not being logged in
159        '406':
160          description: User didn't send application/json content-type header
161  /dogs/${dogId}:
162    get:
163      tags:
164        - public
165      summary: Returns dog
166      operationId: getDog
167      description: |
168        Returns the dog with the provided ID if in system
169      parameters:
170        - in: path
171          name: dogId
172          description: ID (uuid) of the dog to return
173          required: true
174          example: dead-beef-1337-beef
175          schema:
176            type: string
177      responses:
178        '200':
179          description: Dog with the provided ID
180          content:
181            application/json:
182              schema:
183                  $ref: '#/components/schemas/Dog'
184        '404':
185          description: No dog in the system with provided ID
186    delete:
187      tags:
188        - private
189      summary: Deletes a dog
190      operationId: deleteDog
191      security:
192        - BearerAuth: []
193      description: |
194        Deletes the dog with the provided ID if in system and belongs to the user
195      parameters:
196        - in: path
197          name: dogId
198          description: ID (uuid) of the dog to return
199          required: true
200          example: dead-beef-1337-beef
201          schema:
202            type: string
203      responses:
204        '204':
205          description: User successfully deleted dog
206        '401':
207          description: User isn't logged in
208        '403':
209          description: User is trying to delete a dog that doesn't belong to them
210        '404':
211          description: No dog in the system with provided ID
212    patch:
213      tags:
214        - private
215      summary: Updates dog info
216      operationId: updateDog
217      security:
218        - BearerAuth: []
219      description: |
220        Updates the dog with the provided ID if in the system and belongs to the user
221      parameters:
222        - in: path
223          name: dogId
224          description: ID (uuid) of the dog to update info of
225          required: true
226          example: dead-beef-1337-beef
227          schema:
228            type: string
229      requestBody:
230        description: Sets of changes to make to the dog's info
231        required: true
232        content:
233          application/json:
234            schema:
235              $ref: '#/components/schemas/DogInfoUpdates'
236            example:
237              - compatibility:
238                  cats: Hates the things
239      responses:
240        '200':
241          description: User successfully updated dog's info
242          content:
243            application/json:
244              schema:
245                $ref: '#/components/schemas/Dog'
246        '401':
247          description: User isn't logged in
248        '400':
249          description: User sent bad data to perform update with, returns list of paths to incorrect objects
250          content:
251            application/json:
252              schema:
253                $ref: '#/components/schemas/DogUpdateErrors'
254        '403':
255          description: User is trying to update the info of a dog that doesn't belong to them
256        '404':
257          description: No dog in the system with provided ID
258        '406':
259          description: User didn't send application/json content-type header
260components:
261  securitySchemes:
262    BearerAuth:
263      type: http
264      scheme: bearer
265  schemas:
266    LoginDetails:
267      type: object
268      properties:
269        email:
270          type: string
271          description: User's email to log in with
272          example: test@test.com
273        password:
274          type: string
275          description: User's password to log in with
276          example: correctHorseBatteryStaple
277    DogList:
278      type: array
279      items:
280        type: object
281        properties:
282          id:
283            type: string
284            description: ID of the dog in the system
285            example: dead-b33f-1337-beef
286    DogUpdateErrors:
287      type: array
288      items:
289        type: object
290        properties:
291          path:
292            type: string
293            description: JSONPath to incorrect data
294            example:  0.age
295          error:
296            type: string
297            description: Reason why data is incorrect
298            example: "Age is expected to be a number"
299    DogInfoUpdates:
300      type: array
301      items:
302        type: object
303        properties:
304          name:
305            type: string
306            description: The candidate's name as it should appear on the dog
307            example: Jane Q. Programmer
308          compatibility:
309            $ref: '#/components/schemas/DogCompatibility'
310          age:
311            type: number
312            description: How old the dog is
313            example: 2
314    Dog:
315      type: array
316      required:
317        - name
318      items:
319        type: object
320        properties:
321          name:
322            type: string
323            description: The candidate's name as it should appear on the dog
324            example: Jane Q. Programmer
325          compatibility:
326            $ref: '#/components/schemas/DogCompatibility'
327          age:
328            type: number
329            description: How old the dog is
330            example: 2
331    DogCompatibility:
332      type: object
333      properties:
334        cats:
335          type: string
336          description: How well does the dog get on with cats
337          example: Not very well
338        kids:
339          type: string
340          description: How well does the dog get on with kids
341          example: Really well
342        mailmen:
343          type: string
344          description: How well does the dog get on with the mailman
345          example: Best buds

Swagger definition for a Doggo API, part of the demo project

Usually the Swagger document would be used to create a mock server for a consumer to use, to test its API integration code, but instead I used Pact.

Building the API consumer

It can feel a bit odd to build the API consumer before there’s even an API to consume but when working in a distributed team I’ve done this a number of times with both front-end and back-end teams building against the same API definition, integrating later which has always caused issues (something Pact solves).

I built a new module in my front-end repository to integrate with the to-be API and created a test suite for that module in Jest, using Pact to define the request to be sent to the API provider and the expected response.

getDogs.spec.js

1describe('getDogs', () => {
2  describe('Get Dog list for authenticated user', () => {
3    beforeEach(async () => {
4      updateAuthenticationToken(authenticationToken);
5      const interaction = {
6        state: 'The user has authenticated',
7        uponReceiving: 'a request for all Dogs the user has created',
8        withRequest: {
9          path: collectionEndpoint,
10          method: 'GET',
11          headers: {
12            authorization: `Bearer ${authenticationToken}`,
13          },
14        },
15        willRespondWith: {
16          body: dogList,
17          status: 200,
18          headers: {
19            'Content-Type': 'application/json; charset=UTF-8',
20          },
21        },
22      };
23      await provider.addInteraction(interaction);
24    });
25    it('Returns the list of Dogs for the authenticated user', async () => {
26      const response = await getDogs();
27      expect(response).toEqual(dogList);
28      await provider.verify();
29    });
30  });
31});

Example Pact test, the interaction is described and then validated against a mock server

Pact then uses these definitions (called a pact) to create a mock server that the front-end API requests will be sent to and will return the expected response, allowing the functions to be integration tested without the API provider needing to be accessible.

One gotcha to look out for is when you have tests that match against the same response status, say a test for a 401 returned because the user doesn’t send an authentication header and a 401 because the authentication token is invalid, in this instance Pact will only add the first interaction for the response code to the final pact.

I then published the pact to the Pact Broker, which is a server that manages the pacts for other services to validate against, test results and relationships between the API consumers and providers.

1const publisher = require('@pact-foundation/pact-node');
2const path = require('path');
3
4const opts = {
5  providerBaseUrl: 'http://localhost:8080',
6  pactFilesOrDirs: [path.resolve(process.cwd(), 'pacts')],
7  pactBroker: 'http://localhost:9292',
8  pactBrokerUsername: 'test',
9  pactBrokerPassword: 'test',
10  consumerVersion: '1.0.0',
11};
12
13publisher.publishPacts(opts);

The JS implementation of Pact has a simple way of publishing pacts, other implementations may differ

Setting up the Pact Broker

Pact offer a Docker image for the Pact Broker which is perfect for local testing as it can be spun up quickly but they also provide a free tier on their paid for service Pactflow that allows for 5 pacts to be held.

I’ve created a Docker Compose file you can use to set up the Pact Broker which will make the service available at http://localhost:9292 (use test for both the username and password to log in).

1version: "3.3"
2services:
3  broker:
4    image: pactfoundation/pact-broker:2.52.2-1
5    environment:
6      PACT_BROKER_DATABASE_ADAPTER: sqlite
7      PACT_BROKER_DATABASE_NAME: pact_broker.sqlite
8      PACT_BROKER_BASIC_AUTH_USERNAME: test
9      PACT_BROKER_BASIC_AUTH_PASSWORD: test
10      PACT_BROKER_BASIC_AUTH_READ_ONLY_USERNAME: test
11      PACT_BROKER_BASIC_AUTH_READ_ONLY_PASSWORD: test
12      PACT_BROKER_PORT: "9292"
13    ports:
14      - 9292:9292

Spinning up the Pact Broker in Docker is really easy

Once the pact publish job is configured to upload the pact to the Pact Broker and the pact is published, the consumer’s pact will be available in the Pact Broker’s UI.

Pact report — Once the consumer’s pact is published to the Pact Broker it’s available to be verified by the provider

The next step is to use the Pact Broker to allow the API provider to download the API consumer’s pact, verifying whether the API provider honours that pact and uploading the results to the Pact Broker.

Building the API provider using Pact for TDD

The ability to download the API consumer’s pact and verify conformance shows the power of consumer-driven contracts as now we’re able to use the already verified contract in order to practise Test Driven Development (TDD).

The test suite for an API provider is less verbose compared to the one built for the API consumer. This is because the concerns of these test suites are verifying the pact is honoured instead of defining it.

provider.spec.js

1describe('Pact verification', () => {
2  const port = 8080;
3  const opts = {
4    provider: 'doggo-api-provider',
5    providerBaseUrl: `http://localhost:${port}`,
6    pactBrokerUrl: 'http://localhost:9292',
7    pactBrokerUsername: 'test',
8    pactBrokerPassword: 'test',
9    publishVerificationResult: true,
10    providerVersion,
11  };
12
13  beforeAll(async () => {
14    apiServer.listen(port, '0.0.0.0', () => {
15      console.log(`${apiServer.name} listening at ${apiServer.url}`);
16    });
17  });
18
19  afterAll(async () => {
20    apiServer.close();
21  });
22
23  it('Should validate the expectations of consumers', () => {
24    return new Verifier().verifyProvider(opts);
25  });
26});

The provider test downloads the pacts it needs to verify, runs the server, tests them and publishes the results

Once the test suite is run the test suite results will be uploaded to the Pact Broker and results will be visible for everyone to see. This means that should the tests fail then it’s easy to see which consumers are impacted by changes.

passing pact report — Once the provider runs it’s test the results of the pact are published to the Pact Broker

Red, Green, Refactor

In order to get my initial test suite passing for the API I relied heavily on stubbed values but of course the final app would use a proper authentication service to do this, which can be hard to manage when using Pact.

Pact’s JSON structure is static, so authentication tokens and other potentially dynamic data needs to be available to both the consumer and provider at time of the tests running.

Pact suggest a few ways to mitigate the issue such as using long lived authentication tokens or to have authentication as a separate layer, allowing Pact to run against the unauthenticated endpoints.

When refactoring my API to use firebase, I opted to keep the stubbed values but use Jest’s excellent mocking features to build mocks for the firebase/app , firebase/auth and firebase-admin modules, returning the stubs within the mocks.

This allowed me to move all my test logic out of the controllers and into those underlying functions and is much better than my previous approach of using a strategy pattern to return stubs based on the NODE_ENV being test (Jest sets this at runtime).

Test data

As the API grows in functionality and number of consuming services it becomes important to manage test data and ensure a consistent set of values being used for behaviour flags for stubbed data.

I managed this by creating another test data module that stored all the request and response payloads as well as values for authentication tokens and IDs, allowing me to change any test values without breaking the tests due to data mismatches.

In my demo project I use this technique also, with the test data being found in the common folder.

Summary

By using consumer-driven contract testing I was able to verify my front-end would work with my API before I had even built the API itself and then used the pact from the consumer to build the API using TDD.

As someone who has been on multiple teams where we’ve either been given a Swagger document that turned out to be out of date or had to spend time reworking API integrations due to the back-end developers not interpreting the Swagger document the same way the front-end developers did Pact is an amazing solution.

In my app development I’ve used Pact’s Pactflow service in my Continuous Integration (CI) in order to verify the pact on every push, using the app’s version straight from the package.json to ensure versioning is reported properly.

As my app grows in size I look forward to seeing how Pact scales as more consumers get built to handle more focused jobs using the data from the CRUD API.

Check it out yourself

I’ve created a demo project that contains a API consumer, API provider and a Docker Compose file to run the Pact Broker that can be used to play with Pact and see how consumer-driven contracts testing can bring a higher level of quality to API powered applications.

Demo project to show off Pact