In early 2019 one of the Engineering Managers in my organization encouraged me to attend the Rapid Software Testing Applied online course. Initially, I was sceptical about how beneficial a remote course on testing methodology might be for me. I had several years of experience working in testing roles and I was quite comfortable with the way we were approaching testing in my current product team. I wasn’t sure that there was much more for me to learn or things that would change my opinions on testing and the roles of testers. I was quite wrong in these assumptions.

By the end of 2019 I and several colleagues were enthusiastically presenting a business case to management to advocate for further RST training within our organisation. In this article, I’ll highlight some of my experiences of the applications of the RST methodology, the challenges it’s helped to overcome and how I’ve seen it benefit even very experienced testers.

Improving Strategies

“Your strategy contains the reasoning behind the testing that you do.” (Kaner, Bach and Pettichord, 2002, p. 236)

There is a well-established expectation that testers should be experts in the creation of test strategies for the products they work on. It’s still listed as an (if not the) key responsibility for the majority of testing related job specifications. However, it can be challenging for testers to define what the process of creating a strategy should actually look like in a practical sense. Is it just defining tests in advance of performing them? Is it deciding what tools and automation frameworks should be used?

 

One of the immediate benefits RST brought to our organisation was a new perspective on testing strategies. The Heuristic Test Strategy Model encouraged us to review and challenge our process of creating strategies. Prior to being introduced to this model most of our testers were already familiar with many of the common test techniques it mentions. They could easily articulate what these techniques were and how they would use them to test a product. However, what they were going to focus their testing on and the reasons why they would do this was often less clear.

The Product Elements and Quality Criteria introduced in the HTSM became a powerful tool for framing our thinking around testing and helping us clearly define our reasoning. Specific tools such as Product Coverage Outlines and Risk Lists allowed us to capture these thought processes. Our strategies were no longer just conceptual and general, they were lightweight documents specific to the context of our products.

Risky Business

As testers, we frequently talk about our work in the context of risks. The terms ‘risk-based testing’ or ‘risk-focused testing’ have become commonplace in the testing community. Whilst not being unique to RST, a core aspect of the methodology is risk analysis.

We found that the Heuristic RiskBased Testing approach could be applied to our existing software products with very little overhead. Prior knowledge of statistical or quantitative analysis methods to identify and communicate risks weren’t required.

Through experimenting with HRBT, we realised the importance of the social process involved in risk analysis. Discussing risks as a cohort of testers was exciting and a great way to share knowledge. However, at times it could become an echo chamber of testers opinions and ideas.

“Risk communication and risk management efforts are destined to fail unless they are structured as a two-way process. Each side, expert and public, has something valid to contribute.” (Slovic, 1987, p.285)

In order to make our risk analysis and risk based testing really effective we needed to involve more diverse perspectives outside of testing roles. Collaborative exercises such as the RST inspired Risk Storming have helped to enable this. It’s now common in our organisation for testers to host risk analysis sessions for their product teams and stakeholders. Our discussions around product risks now involve more people who can actually help identify, rationalise and mitigate those risks. An additional benefit we’ve noticed is that these discussions provide opportunities for testers to subtly introduce or reiterate concepts such as Quality Criteria and Heuristics to our teams.

Jumping into the Unknown

Our organisation went through a rapid phase of growth during 2019. We hired lots of new engineers, rapidly formed new teams and developed new ways of working. As a result our testers occasionally needed to support multiple product teams, each with their own unique context and level of testing maturity.

The challenges of this situation prompted us to experiment with the Test Jumper role described by James Bach. This role proposes ways in which testers can add lasting value to teams, without needing to be embedded long term for the full lifecycle of a project or product. When combined with newly acquired knowledge of RST our testers were able to succeed in these high-pressure temporary roles, despite working in the context of unfamiliar products, domains and technologies.

Continuous Learning and Community

In the two years since we first began to adopt the RST methodology, the context of our organisation has continued to change. As a result, we’ve found that we’ve frequently had to revisit the RST course materials, either to introduce them to new people, to revise our understanding or to improve how we are using them.

At the same time, we’ve noticed more voices in the global community advocating for context-driven and heuristic-based approaches to testing. Conor Fitzgerald’s talk at Test Bash Brighton 2019 highlights this and also hints at a growing interest amongst testers in the wider topics of cognitive science and psychology.

As the trends for adopting microservices architectures and DevOps methodologies continue, the expectations of testing roles are changing. There is pressure on testers to provide value whilst not impacting the release cadence of product teams. Clokie describes the wider consequences this can have.

“The difficulty for testers is identifying what their role is, when testing is expected to be a part of everything.” (Clokie, 2017, p.8)

The techniques offered by RST alone are not a direct solution to this challenge. However, the critical mindset it encourages and the shared language it provides can help testers to make lasting improvements in an organisation.

References

Andrew January

Andrew January is a Senior Test Engineer at Simply Business. He began his career in software testing in 2010. Since then he has worked at several London tech companies, coaching agile teams in the ways of testing and leading testing communities of practice. Outside of testing he has an interest in building Python applications for hobby projects.