Challenging areas in Agile testing maturity

First posted on the Polteq website.

A couple of years ago, I had to assess the testing maturity of a company that was practicing Agile/scrum. The maturity model that I needed to use was TPI Next. At the end of the assessment the model showed a lot of areas for improvement. The improvements that would result in a more mature organization according to the model, would in my opinion not benefit the organization. So why did my opinion and the model differ so much? The reason was the Agile/scrum context. The model steers an organization to more structure, where Agile/scrum asks for flexibility. The context of this company required a different view on what needed to improve. That’s when Polteq decided to put effort in creating a test improvement model for this specific context: TI4Agile .

Since the introduction of TI4Agile a couple of years ago, it has been successfully applied many times in different contexts. I have analyzed the results of these assessments and found common challenges with Agile testing maturity that are interesting to share. It appears that many organizations have a low maturity in the following areas, which are a subset of the twelve key areas of TI4Agile:

• Test management
• Test process
• Test automation
• Interaction

Test management and test process share some problems in the transition to Agile development. Both areas were introduced into the traditional testing world to provide more structure to testing. The flexibility and adaptability that is needed in the current Agile context, requires large changes in how to approach test management and test processes.

Test automation has gained a more prominent role in the Agile context, to facilitate iterative and incremental development with fast feedback loops. The increase in importance is recognized by organizations, but often lacks professionalism. Therefore this area lacks maturity.

And last but not least, self-organizing teams require more and better interaction between the team members. Scrum facilitates the interaction in the different meetings that form the basis of the development process. To gain most value out of these meetings, the purpose of these meetings must be clear to all participants.  The team members must be able to switch between very technical topics and complex business situations.

In future posts I will dive deeper into each of the areas to provide more insight in the specific challenges of Agile testing maturity.

Advertisement

Noordertest 2012

Last week the Noordertest conference took place in Groningen. With three groups of five parallel presentations and one keynote, each of the 160 attendees could find something interesting.

Polteq provided two presentations for the conference. I presented about Test Improvement for Agile together with Edze Knol and Ruud Teunissen presented about how to properly do test automation. Edze and I were in the first group of presentations, so after a general introduction to the conference we had to kick-off.

Test Improvement for Agile proved to be a hit, since we had more people than we had seats 🙂 We started with a short introduction to Agile and SCRUM, followed by a short introduction on test improvement. After the introduction we provided some more depth information on three of our key areas:

• Teamwork
• Test management
• Defect management

In teamwork we seek for collaboration, trust and the willingness to work outside your comfort zone. About test management you can read more in my previous post and defect management should be to support the team in stead of the business.

After our presentation I attended “The fragility of agility” by Lloyd Roden. It was nice to see that he pointed out the same groups of improvable items that we dealt with in our presentation.

Parallel was the presentation about proper test automation by Ruud Teunissen. He told the audience that we need to make sure that test automation bridges the gap between testware and the system under test. Make decisions on what you want to achieve and not on e.g. the tools that are already present in an organization. Remember that test automation is a form of development and should be treated as such.

Next was a workshop on how to ask questions. Main points were to make sure that you ask the question you really want to ask and then listen to the answer. Don’t add your own information while asking questions, so you will get the real answer in stead of what you want to hear.

Finally the keynote – also by Lloyd Roden – about challenges in software testing. Here Lloyd presented eight challenges in the software testing world. Learning, skills and communication where of course part of the challenges.

I really enjoyed the conference!

Test conferences

The number of events that deal with software testing seem to increase. This globalisation is great news for the testing community! More and more opportunities to share knowledge and learn from others. Last year my presentation on how to test mobile apps got to several stages and this year the release of the book Cloutest provides some opportunities for presentation slots. A presentation on Cloutest has been accepted for EuroStar and for ChinaTest. Maybe some more events will follow.

Next to Cloutest, my stage appearances will focus around Agile. China is starting to go Agile and ChinaTest has accepted two Agile proposals next to my Cloutest presentation. These will deal on how to work as a tester in Agile teams and how to improve testing in Agile settings.  The Testnet fall event also focuses on Agile, the proposal that got accepted for Testnet deals with Agile and outsourcing.

It’s great to have a lot of proposals accepted, now to prepare each presentation! For the Agile part, there are some basics that need to be dealt with in all the presentations, so let’s think about reusability 🙂 Working at Polteq provides a great setting to discuss about several testing topics. It’s great to get a lot of support and help. For Agile we recently started a discussion group, helping to see common problems and different solutions in Agile settings. Every meeting is an eye opener for me and I hope we will have a lot of informative meetings to come.

Test improvement in Agile/Scrum*

This is the translation of my article for Polteq in TestnetNieuws 2012-1.

Testnet exists fifteen years, in this time testing and the context of testing has changed a lot. This should have influenced test improvement models, but are the current test improvement models fit for Agile/Scrum? Most of the models are based on the tradition development and test methodologies  (waterfall) . The key areas that get addressed by these models depend largely on the phases in these development models. One change we see within Agile/Scrum is the phases that are used… Above all, we see that not only the phases have changed, but the role of testing too. A test manager needs a new point of view on Agile projects and for the testers in Scrum teams we see new demands. Testing has become a role in stead of a function! To assess the maturity of testing, we need to change the models and incorporate Agile/Scrum into the models.

Maturity

To be able to improve, we need to asses the current maturity of Agile/Scrum and of testing. Testing is not “just a phase” but part of the whole. We are thinking about the following levels of maturity:

• No Agile/Scrum (it’s called Agile, but it is waterfall);
• Agile/Scrum process in place (all useful meetings and artifacts are introduced, we see little waterfalls with more and more Agile elements)
• Agile/Scrum is understood and applied (good incremental/iterative development incorporating the Agile principles)

To achieve optimal improvement, not only testing has to be addressed. However in this article the focus will be on some important testing aspects.

Test knowledge

The first prerequisite is a vast knowledge of testing. The evolution of testing is depicted in the following figure. At first testing was at the pioneering phase; everything had to be invented. After that the structured approaches started to emerge. The logical next step was to let go of the structure and use the gained (structured) knowledge in an Agile way. When starting to work Agile before learning the structure, the testing basics will miss and we keep reinventing “test wheels”.

The tester

Agile/Scrum has different demands for testers than waterfall projects. The test needs to do more tasks, which previously were dealt at higher levels in the organization. For instance planning and estimation are now dealt with at team level. We need more proactive testers. Since test basis usually does not exists when a sprint starts, the tester needs to actively help to get it. Don’t wait behind your desk until the specifications are ready! Help the team by creating clear Definitions of Done per backlog item.

Test management

The test manager will have a facilitating role in Agile/Scrum. The role might be a little less test manager and somewhat more team manager. So supply the teams with people that have enough test expertise! However there will be enough work in the area of testing for the test manager. When the product backlog gets assembled, the test manager should think and communicate about risks. This will help to find the dependencies between backlog items as early as possible and will help to decide what to do in the regression tests and/or end-to-end tests. The test manager has the overall view on risks! When multiple teams exist, this is a very important aspect. Another tasks for the test manager is creating generic test approach. For Agile/Scrum this can consist of how to deal with critical issues, demands for test automation and minimal mandatory reporting. Make sure to know the context, it can be organisation wide or only for the Agile projects. Creating a generic approach will give the business more insight and the business still feels like there is some control.

Planning

Sad enough, testing is often only partly involved. Planning sessions take place without involving testers or with not enough respect for the testers. Resulting in incorrect estimations and too little thought about the risks. Testers need to be involved in all phases within Agile/Scrum. Preparations like assembling and prioritizing the product backlog also needs input from testing.

Test automation

An important element of testing within Agile/Scrum is test automation. This is necessary to keep up in the incremental process, since the regression test will become bigger and bigger. Within test automation there are a lot of improvement possibilities. Good automation isn’t as easy as it seems. The foundation of good automation is a sound architecture (test framework). Make sure that the test automation is kept up-to-date.

Business commitment

The organization as a whole needs to support the new way of working. Controlling based on reports is no longer possible, so the business needs to actively communicate with the teams. This will lead to guiding the teams in the right direction while they still have enough freedom to be Agile. Teams can still do some reporting to help involve the business. Most tools that teams use to keep track of progress and managing defects have reporting facilities. Make sure that the reports deliver what the business needs to know!

Conclusion

It is important to realize that the focus and approach of testing within Agile/Scrum is different. The team is responsible for quality, so testing has become a team responsibility. All people in the team will be testing, but the testers still have the testing expertise. So especially the testers can help to improve the testing within Agile/Scrum.

*) Agile/Scrum is Agile with Scrum