A tale of two teams and how they helped shaped the value testing provided

Now I’ve changed client at work I feel like I can address something that I’ve seen happen twice at my last engagement and a fair amount in my career.

Every team I’ve been in has had a disproportionate number of developers to testers and has expected the testers to conduct manual testing and build an automated test suite.

When I was a developer I was always willing to help the testers with the automation side of things as I had an interest in it but also as I saw it as a means of helping the team adopt a more BDD approach.

The reason I wanted to free them up was because every team I’ve been on has had a period where there’s a lengthy test backlog due to the developers hitting a decent velocity and being encouraged by project management to focus on their cycle time and not the team’s.

Here’s how this way of thinking has impacted two teams I’ve been in.

Testing in a team that doesn’t trust test

The team consisted of 6 developers (including me at the start) and 3 testers and we were working on a launching the client’s web offering in a new territory.

We all started on the project at the same time, but the testers struggled to get the UI test framework up and running due to a lack of the testers’ knowledge of the stack and due to needing to hit delivery deadlines, we were told to time-box our assistance of the testers.

It wasn’t until about two months after I moved from development to test within the team that the test framework and a subset of the tests were in a stable enough state to be used for evaluating the product’s adherence to a set of rules we had to meet to be certified in the territory we were entering.

I suggested in a retrospective if we could add the test suite as a quality gate to ensure there was no regression against these rules that we had covered but I was met with a harsh criticism of the test team’s work.

“I’m not having your tests prevent my code from being released to live. I’m not going to be punished because a test fails due to you not implementing it properly” — One of the developers on my team showing that ‘one team’ spirit

This response really pissed me off.

It wasn’t because I had some deep pride in the code I’d written and I felt this was a personal attack but because the team’s management seemed more interested in the development team shipping features than actually ensuring those features met the strict governance guidelines we were meant to work in.

As time went by things were made even worse within the team as the developers started to change existing functionality, which had been automated against and would change the attributes we had been using, making the tests brittle but not directly seeing the consequence of their changes due to no CI running the tests on each push.

I expressed my disappointment of the dismissal of test to the team lead who explained to me why there was a lack of trust within the team.

They mentioned that because the team hadn’t had any visibility of test runs for nine months they just didn’t believe the test team were doing anything useful and during that time the developers had really hit a good velocity which would be ruined by having to suddenly solve teething issues around introducing tests.

One ray of hope did come out of that conversation however.

One of the issues the team was facing was that they suddenly found themselves deploying to four environments instead of the one the developers were used to and they weren’t able to evaluate the environment’s viability quick enough — this sounded like a prime candidate to win back some trust in testing!

I then built a quick API testing suite that covered some of the functionality of the regression suite using Postman and ran this daily to give the team the ability to see how stable the environment was and as if by magic the team became more receptive to the test team’s work.

Unfortunately however by the end of my time on that team we still hadn’t managed to get the regression tests run as part of the development process.

I get by with a little help from my (developer) friends

The team consisted of 3 developers and one tester (me) and we were working on building a new service to bridge two systems.

Similar to the other team we all started at the same time but unlike the other project none of the team could access the upstream or downstream systems for a month (and even then they were unstable for a few more months).

Having seen how poorly test was implemented into the development lifecycle in the other team I proposed we work in a way that the definition of done is only met when there’s an automated end-to-end test for new functionality.

This was relatively well received within the team with only one developer being dismissive of the approach but it became hard to deliver in the intended manner due to the lack of system access.

This eventually led to a development pattern where the developers would work on their work items then pick up a new one leaving me to attempt to get it working against the limited access we had which of course led to a massive test backlog.

One of the developers also started to get sloppy with their work.

Instead of taking the time to run their code in the rudimentary environment I had been able to pull together (and made available to the team) they had programmed, unit and integration tested against an interface that wasn’t the correct one so when I finally got around to testing the functionality it didn’t work and they got upset as they’d built a lot more functionality on top of this incorrect interface.

Eventually the upstream and downstream systems became stable and accessible and the end-to-end test suite was able to actually work end-to-end but then I faced a new issue.

Because of the backlog of 44 work items sitting in the test column and the developers velocity I was unable to realistically clear it and catch up with developers.

I raised this issue at a retrospective and to my surprise the developers agreed to give me 20% of their time to help shift the backlog and to look at hiring an additional tester to help in the long term.

With the developers on board I faced a new issue of refactoring the code base to make their life easier but this was relatively easy and they were happy writing the test cases after this was done.

This help and the additional tester in the team allowed me to not only get the tests implemented but also to make the test results visible in JIRA (using Zephyr) so that as the developers worked on a work item the associated test case would hold the status of the most recent test run.

I was also freed up to build reporting in confluence using the Zephyr widgets to report the system’s compliance with the feature set the stakeholders wanted and when we scaled up from one environment to six it became really trivial to report on these too.

The turning point

In the first team the turning point came when I was able to show that testing wasn’t just about checking a developers work after it was built and instead could be used to help discover the viability of different environments.

Of course it was too late in the project and I never got to show them that the the automated regression suite would also help them in a similar matter as they built the main product but that turning point really helped prove test to the team.

However this proof didn’t come soon enough and the team didn’t fully adopt the test team’s work into the main development cycle.

In the second team I was fortunate in the regard that the developers were willing to help me clear the backlog and as they did they saw how the end-to-end tests were going to provide value to the team.

Because of their help, the end-to-end test suite went from just a set of tests run against the system to the primary means of CI tests, checking deployments, checking new environments and showing the team’s work met all the stakeholder’s needs.

The take away

Hopefully this tale of two teams has helped to illustrate how a development team’s approach to the test team being behind the developers can help shape the amount of value testing gives to the team.

A more important idea I’d like to drive home though is that by ensuring the development team’s definition of done includes testing activities (and this isn’t abandoned when there’s issues getting test automation set up) that the value of test can be delivered and proved in the first sprint and that this stops the rift happening in the first place.