Challenging areas in Agile testing maturity

First posted on the Polteq website.

A couple of years ago, I had to assess the testing maturity of a company that was practicing Agile/scrum. The maturity model that I needed to use was TPI Next. At the end of the assessment the model showed a lot of areas for improvement. The improvements that would result in a more mature organization according to the model, would in my opinion not benefit the organization. So why did my opinion and the model differ so much? The reason was the Agile/scrum context. The model steers an organization to more structure, where Agile/scrum asks for flexibility. The context of this company required a different view on what needed to improve. That’s when Polteq decided to put effort in creating a test improvement model for this specific context: TI4Agile .

Since the introduction of TI4Agile a couple of years ago, it has been successfully applied many times in different contexts. I have analyzed the results of these assessments and found common challenges with Agile testing maturity that are interesting to share. It appears that many organizations have a low maturity in the following areas, which are a subset of the twelve key areas of TI4Agile:

• Test management
• Test process
• Test automation
• Interaction

Test management and test process share some problems in the transition to Agile development. Both areas were introduced into the traditional testing world to provide more structure to testing. The flexibility and adaptability that is needed in the current Agile context, requires large changes in how to approach test management and test processes.

Test automation has gained a more prominent role in the Agile context, to facilitate iterative and incremental development with fast feedback loops. The increase in importance is recognized by organizations, but often lacks professionalism. Therefore this area lacks maturity.

And last but not least, self-organizing teams require more and better interaction between the team members. Scrum facilitates the interaction in the different meetings that form the basis of the development process. To gain most value out of these meetings, the purpose of these meetings must be clear to all participants.  The team members must be able to switch between very technical topics and complex business situations.

In future posts I will dive deeper into each of the areas to provide more insight in the specific challenges of Agile testing maturity.

Advertisement

Testing needs to change

Abraham Lincoln once said:

“The dogmas of the quiet past are inadequate to the stormy present. The occasion is piled high with difficulty, and we must rise with the occasion. As our case is new, so we must think anew and act anew.”

To tackle the changes in the development processes, testing must grow along with these changes. The shift in most companies is from waterfall (or traditional) development to Agile development. To enable this growth, testing should go from an industrial, manufacturing model towards an agricultural model. You cannot predict the outcome of testing, you can just – like a farmer – create the conditions under which testing begins to flourish. Keep in mind that you can predict the outcome of a single test, but not of the complete testing process. To make testing flourish, you need to adapt testing to the context. In some settings it seems enough to reform your testing, but in most cases a transformation is needed.

I advise starting with a clean slate (transforming) over changing your current process (reforming). Aligning with the new context is easier when you are not bound by decisions made in the old context. Start off with thinking about what you need and not what you have! So what do you need? Your context can have a lot of (conflicting) demands, such as:

• Short time to market
• High quality
• Low costs
• High auditability

From these demands, look at efficient and effective ways to arrange your testing process along with the development process. You must not only find out when to test what, but also how this testing should take place. Do you still need the process driven testing, with testing techniques to get you test cases that can check if the product meets its requirements? Or do you need exploratory testing to validate that you built the right product? Or do you need both, or other approaches?

In my opinion you need to look at your product from multiple angles. So in your process, you should find a combination of approaches that best fits your needs. None of the approaches is completely wrong or right, they are just different. So why not try to take the best of all worlds?

A good assessment is about improving

Improvements require an assessment, either formal or informal.  A good assessment tells you what you are doing right and which areas are open for improvement. If you do not plan to act on the improvement areas, then why do you do an assessment? So the target for an assessment is twofold. First you need to know where you are, next you need to know where you want to go. The road from where you are to where you want to be can only be defined by improvement actions.

So how do you do a good assessment of any situation? You need to ask questions and more importantly, you need to listen to the answers and make sure you understand them. Every situation you want to assess, has its own context. So get to know the situation and the context. The clearer the view you achieve on a situation in its context, the better your assessment. When you define  improvement actions, you need to determine where you want to go. This target situation, needs as much context and information as possible. The best way from A to B can only be defined if we have a common understanding of A and B.

Beware when you define improvement actions! Not every action is suitable for every situation. In other words: each problem has multiple solutions. The solution must solve the problem and it must fit in the context. Only then it will prove to be valuable. The solutions that work in every context are usually abstract. These will not actually help you to achieve your goals, but are a good starting point. When making these solutions more concrete, you need to change them to fit the current context. Sometimes when you try to do this, the solution will not fit your context. So is this the right way to go? Yes, since knowing what not to do is also valuable.

At Polteq we always use assessments in support of improvements. The available improvement models contain important areas for testing. The models also provide a lot of questions to assess the areas. But we do not stop there, we keep asking questions outside the model! The areas with their questions provide nice guidelines, but a good assessment depends on the assessors. The assessors must take the context into account and therefore questions in the model can become either more or less important. While assessing, we try to capture the situation in its context. This allows us to write specific improvement actions which are fit for your situation.

How do you find the right improvement actions?

Noordertest 2012

Last week the Noordertest conference took place in Groningen. With three groups of five parallel presentations and one keynote, each of the 160 attendees could find something interesting.

Polteq provided two presentations for the conference. I presented about Test Improvement for Agile together with Edze Knol and Ruud Teunissen presented about how to properly do test automation. Edze and I were in the first group of presentations, so after a general introduction to the conference we had to kick-off.

Test Improvement for Agile proved to be a hit, since we had more people than we had seats 🙂 We started with a short introduction to Agile and SCRUM, followed by a short introduction on test improvement. After the introduction we provided some more depth information on three of our key areas:

• Teamwork
• Test management
• Defect management

In teamwork we seek for collaboration, trust and the willingness to work outside your comfort zone. About test management you can read more in my previous post and defect management should be to support the team in stead of the business.

After our presentation I attended “The fragility of agility” by Lloyd Roden. It was nice to see that he pointed out the same groups of improvable items that we dealt with in our presentation.

Parallel was the presentation about proper test automation by Ruud Teunissen. He told the audience that we need to make sure that test automation bridges the gap between testware and the system under test. Make decisions on what you want to achieve and not on e.g. the tools that are already present in an organization. Remember that test automation is a form of development and should be treated as such.

Next was a workshop on how to ask questions. Main points were to make sure that you ask the question you really want to ask and then listen to the answer. Don’t add your own information while asking questions, so you will get the real answer in stead of what you want to hear.

Finally the keynote – also by Lloyd Roden – about challenges in software testing. Here Lloyd presented eight challenges in the software testing world. Learning, skills and communication where of course part of the challenges.

I really enjoyed the conference!

Test conferences

The number of events that deal with software testing seem to increase. This globalisation is great news for the testing community! More and more opportunities to share knowledge and learn from others. Last year my presentation on how to test mobile apps got to several stages and this year the release of the book Cloutest provides some opportunities for presentation slots. A presentation on Cloutest has been accepted for EuroStar and for ChinaTest. Maybe some more events will follow.

Next to Cloutest, my stage appearances will focus around Agile. China is starting to go Agile and ChinaTest has accepted two Agile proposals next to my Cloutest presentation. These will deal on how to work as a tester in Agile teams and how to improve testing in Agile settings.  The Testnet fall event also focuses on Agile, the proposal that got accepted for Testnet deals with Agile and outsourcing.

It’s great to have a lot of proposals accepted, now to prepare each presentation! For the Agile part, there are some basics that need to be dealt with in all the presentations, so let’s think about reusability 🙂 Working at Polteq provides a great setting to discuss about several testing topics. It’s great to get a lot of support and help. For Agile we recently started a discussion group, helping to see common problems and different solutions in Agile settings. Every meeting is an eye opener for me and I hope we will have a lot of informative meetings to come.