Is a test manager still needed in Agile development?

This is the translation of my article for Polteq in TestnetNieuws 2012-2.

Within Agile the responsibilities shift more to the executing teams. For instance the planning and estimation of the work that used to be done by management, now lies completely within the teams. So is there enough work left to justify the function test manager? To be able to judge this we need to do a couple of things. First determine what a test manager did and which parts of his work will stay within Agile. Next is to determine if new responsibilities and tasks arise fir the test managers.

Who does what?

The first tasks that come in mind when I think of the traditional test manager are:

• Create and update test plans;
• Set the test policy;
• Initiate and execute product risk analysis;
• Estimate the work;
• Create planning;
• Report on risks, progress and quality;
• Manage the test team;
• Manage the test project.

Mostly organizing tasks dealing with testing. When we start working Agile, many of these task will be picked up by the Agile teams. These teams will have the responsibility to estimate and plan the work that is provided to them. A clear indication the the work of the test manager will be less… Whenever the Agile teams execute their planning sessions they need to think about risks, because these risks will influence the amount of work. Therefore the product risk analysis will be split up into smaller parts and will be executed every iteration by the teams.

Next the test plans, test policy and reporting. One of the Agile principles states: “Working software over comprehensive documentation”. So we need to find out how much of the plans and reports is still valid in Agile and how we can minimize this. Test plans cannot become comprehensive documentation! These plans usually leave little room for flexibility and flexibility is just what the Agile teams need. They need to be able to adjust their plans every iteration to fit their needs. The policy will be dealt with later on in this article.

Reporting is also placed in the Agile teams. The teams report on their status every day at the daily stand-up; everyone talks about what they will do and which impediments they have. The burn-down chart – a graph to show how many work is still left in the current sprint – should also be part of the stand-up. It helps to track the progress and well functioning teams will adjust their plannings according to the information of the burn-down chart.

The Agile test manager

The Agile test manager will focus more on people management and therefore has the responsibility to pick the right people for the different teams. The test manager needs to make sure that they hire people fit for Agile, testers have the opportunity to attend essential courses and that sufficient growth possibilities are present within the organization. Beware that not every good (or even excellent) tester is fit for Agile!

Managing the test team has become more complex. At first we had a test team that worked together, but the testers are now divided over different teams. In stead of keeping one plate in the air, a multitude of plates should be kept in the air. It is reasonable to assume that this aspect of test management takes more time than before.

Another important aspect is facilitating knowledge sharing. Of course the knowledge should be shared within the Agile team to enable good and efficient work of the team. But when there are multiple teams, these teams can also learn a lot from each other. The test manager should have a test meeting once every couple of weeks to let the testers of the different teams learn from each other. In this way the teams can benefit from problems and best practices from other teams.

Thinking about risks within the team is very important, but this should also be done at higher levels within the organization. During creation of the product backlog and the release planning (cross-iteration activities), the test manager should make sure that a risk analysis is performed. The impact of different backlog items on each other should become clear. Some items are prerequisite for others and should be planned like that. In projects with multiple teams it often occurs that multiple teams effect the same parts of the system, this risk needs to be addressed during release planning and the sprint planning sessions. In short, we need a helicopter view on the project with a focus on quality and risks.

The business needs a view on product quality. With Agile testing we do not need to reinvent the wheel, a structured testing approach still offers insight into quality. The Agile test manager needs to create a generic test approach that deals with several subjects. Think of how to register defects, dealing with non-functionals and how to do end-to-end testing. The amount of freedom that the teams get to fill in how they deal with these topics depends on a number of factors like risks, the maturity of testing, the maturity of the team, etc. The approach should offer space for the teams to make their own decisions.

Conclusion

Small project with only one or two teams don’t need a full-time test management job, one of the more experienced testers in the teams will be able to fill the role of test manager. Organizations with large and/or a lot of projects have enough room for a full Agile test manager function. So the role of a test manager will not disappear!

The focus of test management is more on people management. Next to that on facilitating and controlling the quality of testing. We need generic guidelines to manage the testing within the organization and the test manager needs to supply these. These guidelines should not limit the freedom of the Agile teams too much.

Thinking about risks still is important. This should not only be done within the Agile teams, but also cross-iteration and even cross-project. The test manager can offer the helicopter view and look at risks on several levels. It’s very important to realize the risks with multiple parallel Agile teams.

So: Yes, in the Agile world we need test management!

Testnet summer school

Last wednesday was the Testnet summer school. A nice day of workshops about several testing topics. There were three workshops on the Certified Agile Tester training, one of them was mine. Since I like to improve my teaching, I decided to follow the other two workshops too.

The first workshop by Bart Bouwers dealt with user stories. The important thing to take away from this workshop was the lifecycle of a user story. From the initial setup as a high level requirement to the go-live moment. The second workshop was my own workshop for Polteq on Agile test management. I shared with the attendees that the role of the test manager probably won’t disappear, but it will change to more people management and more facilitating. The last workshop I attended was by Cecile Davis. We were with a very small group, but had a lot of fun discussing about several Agile topics that came out of the group.

I really enjoyed myself at this great event and hope that Testnet will continue to organize the summer school.

Test conferences

The number of events that deal with software testing seem to increase. This globalisation is great news for the testing community! More and more opportunities to share knowledge and learn from others. Last year my presentation on how to test mobile apps got to several stages and this year the release of the book Cloutest provides some opportunities for presentation slots. A presentation on Cloutest has been accepted for EuroStar and for ChinaTest. Maybe some more events will follow.

Next to Cloutest, my stage appearances will focus around Agile. China is starting to go Agile and ChinaTest has accepted two Agile proposals next to my Cloutest presentation. These will deal on how to work as a tester in Agile teams and how to improve testing in Agile settings.  The Testnet fall event also focuses on Agile, the proposal that got accepted for Testnet deals with Agile and outsourcing.

It’s great to have a lot of proposals accepted, now to prepare each presentation! For the Agile part, there are some basics that need to be dealt with in all the presentations, so let’s think about reusability 🙂 Working at Polteq provides a great setting to discuss about several testing topics. It’s great to get a lot of support and help. For Agile we recently started a discussion group, helping to see common problems and different solutions in Agile settings. Every meeting is an eye opener for me and I hope we will have a lot of informative meetings to come.

Test improvement in Agile/Scrum*

This is the translation of my article for Polteq in TestnetNieuws 2012-1.

Testnet exists fifteen years, in this time testing and the context of testing has changed a lot. This should have influenced test improvement models, but are the current test improvement models fit for Agile/Scrum? Most of the models are based on the tradition development and test methodologies  (waterfall) . The key areas that get addressed by these models depend largely on the phases in these development models. One change we see within Agile/Scrum is the phases that are used… Above all, we see that not only the phases have changed, but the role of testing too. A test manager needs a new point of view on Agile projects and for the testers in Scrum teams we see new demands. Testing has become a role in stead of a function! To assess the maturity of testing, we need to change the models and incorporate Agile/Scrum into the models.

Maturity

To be able to improve, we need to asses the current maturity of Agile/Scrum and of testing. Testing is not “just a phase” but part of the whole. We are thinking about the following levels of maturity:

• No Agile/Scrum (it’s called Agile, but it is waterfall);
• Agile/Scrum process in place (all useful meetings and artifacts are introduced, we see little waterfalls with more and more Agile elements)
• Agile/Scrum is understood and applied (good incremental/iterative development incorporating the Agile principles)

To achieve optimal improvement, not only testing has to be addressed. However in this article the focus will be on some important testing aspects.

Test knowledge

The first prerequisite is a vast knowledge of testing. The evolution of testing is depicted in the following figure. At first testing was at the pioneering phase; everything had to be invented. After that the structured approaches started to emerge. The logical next step was to let go of the structure and use the gained (structured) knowledge in an Agile way. When starting to work Agile before learning the structure, the testing basics will miss and we keep reinventing “test wheels”.

The tester

Agile/Scrum has different demands for testers than waterfall projects. The test needs to do more tasks, which previously were dealt at higher levels in the organization. For instance planning and estimation are now dealt with at team level. We need more proactive testers. Since test basis usually does not exists when a sprint starts, the tester needs to actively help to get it. Don’t wait behind your desk until the specifications are ready! Help the team by creating clear Definitions of Done per backlog item.

Test management

The test manager will have a facilitating role in Agile/Scrum. The role might be a little less test manager and somewhat more team manager. So supply the teams with people that have enough test expertise! However there will be enough work in the area of testing for the test manager. When the product backlog gets assembled, the test manager should think and communicate about risks. This will help to find the dependencies between backlog items as early as possible and will help to decide what to do in the regression tests and/or end-to-end tests. The test manager has the overall view on risks! When multiple teams exist, this is a very important aspect. Another tasks for the test manager is creating generic test approach. For Agile/Scrum this can consist of how to deal with critical issues, demands for test automation and minimal mandatory reporting. Make sure to know the context, it can be organisation wide or only for the Agile projects. Creating a generic approach will give the business more insight and the business still feels like there is some control.

Planning

Sad enough, testing is often only partly involved. Planning sessions take place without involving testers or with not enough respect for the testers. Resulting in incorrect estimations and too little thought about the risks. Testers need to be involved in all phases within Agile/Scrum. Preparations like assembling and prioritizing the product backlog also needs input from testing.

Test automation

An important element of testing within Agile/Scrum is test automation. This is necessary to keep up in the incremental process, since the regression test will become bigger and bigger. Within test automation there are a lot of improvement possibilities. Good automation isn’t as easy as it seems. The foundation of good automation is a sound architecture (test framework). Make sure that the test automation is kept up-to-date.

Business commitment

The organization as a whole needs to support the new way of working. Controlling based on reports is no longer possible, so the business needs to actively communicate with the teams. This will lead to guiding the teams in the right direction while they still have enough freedom to be Agile. Teams can still do some reporting to help involve the business. Most tools that teams use to keep track of progress and managing defects have reporting facilities. Make sure that the reports deliver what the business needs to know!

Conclusion

It is important to realize that the focus and approach of testing within Agile/Scrum is different. The team is responsible for quality, so testing has become a team responsibility. All people in the team will be testing, but the testers still have the testing expertise. So especially the testers can help to improve the testing within Agile/Scrum.

*) Agile/Scrum is Agile with Scrum

Attended the Testnet testing dojo

So, the 4th of April I attended my first testing dojo. Fun to see that most people are up for their first time. A large group of people showed up, but we had a relatively small room to work in.

After a very short introduction of Huib Schoots and Peter van Tulder we started testing in groups. The test object was a (not too structured) website. Testing without any specification was harder than I expected. Where to start? What to test? What was the intention by releasing this website? Without asking any of these questions upfront, we started… We had some structure on how to go through the website, but it was a really messy site. So after half an hour of testing all teams came together for a debriefing. People found bugs, but were they bugs?

Before testing another website, we got some leads on how to get focus on what to do. One suggestion was to use SFDEPOT to address different aspects of the software. Luckily the second test object had a clearer scope 🙂 We only got 15 minutes to test this one, so we focussed on a part of the functionality. Though we had more structure this time, I still had trouble in creating structure in what to do. Luckily the partnering provided guidance and in the end I felt we did the right things.

This dojo has opened my eyes on how much I rely on scripted testing and I will need to practice more on my exploratory skills. I had a great evening and hope Testnet will do this more often (hopefully in smaller groups)!