Experiences at EuroSTAR 2012 (part 2)

In the previous post I described my experiences at the Tuesday of EuroSTAR 2012. In this post I will continue my EuroSTAR experience with the Wednesday.

Changing Management Thinking – John Seddon

A nice quote from this talk about changing management thinking: “The primary cause of failing is management”. Managers tend to make decisions that are not in the benefit of projects. For example when you want to decrease the costs, managers start managing on costs… This actually increases the costs in most cases. However, when you manage on value, this will more often decrease the costs. A very useful story that John told us, was about chicken wings and spare ribs. Management of a large chain of restaurants decided to replace the spare ribs as a starter with chicken wings, since the chicken wing had a larger margin. Customers were disappointed and asked the waiters if they could get a small portion of ribs (still available as a main) as a starter. The waiters want to please their customers, so they say that it is possible. Then the fun part will start… The waiter needs to put the starter in the cash register and since there is no starter of spare ribs listed, they choose to file it under chicken wings. Since management reads the registers and sees that chicken wings are sold very often, they order more chicken wings for their restaurants. A fine example of failing management.

Adventures in Test Automation – Breaking the Boundaries of Regression Testing –

 John Fodeh

John provided information on automated monkey testing. The presentation was supported by using some scenes from the IT Crowd to inform us on automation.  Automated monkey testing p roved to be an easy to understand concept: by randomizing each step, you are simulating monkey testing. The problem of course is that it is easy to miss out on obvious defects, it does not effectively emulate real scenarios and debugging lng test runs can be quite a pain. They felt the need to create more intelligent monkeys by creating somewhat more expectable behavior via the use of state tables with probabilities per action.

Evolving Agile Testing – Fran O’Hara

After a short introduction on Agile and SCRUM, Fran started off on requirements. When we start to talk about user stories, we should try to find out about acceptance criteria for the story. This serves several goals:

• Define the boundaries for a user story/feature
• Help the product owner to find out what it is that delivers value
• Help the team gain understanding of the story
• Help developers and testers to derive tests
• Help developers when to stop adding functionality to a story

Fran reminds us to keep these acceptance criteria at a relatively high level, so do not lose yourself in too much details. Detailing will be done in e.g wireframes, mockups or validation rules. Another place where we find detailing is in the automated acceptance tests. Try to find examples that support your acceptance criteria.

Next Fran stresses the fact that we still need test strategy in Agile. We need to think about the minimal tests in the sprints (automated unit, automated acceptance, manual exploratory) a

nd sometimes need to do some additional testing e.g for non-functionals, feature integration or business processes. The testers themselves need to have broad knowledge (more than just testing) and deep knowledge in testing. This requires a ‘technical awareness’.

Testing of Cloud Services; The Approach: From Risks to Test Measures – Kees Blokland & Jeroen Mengerink

Kees and myself presented Cloutest® our approach to testing cloud services. We started off with an introduction to cloud computing to set the context. To properly introduce the concept, we decided to use the definition provided by NIST. After this into our approach. We identified 143 risks that arise when using cloud computing and grouped these risks into categories:

• Performance
• Security
• Availability & Continuity
• Functionality
• Manageability
• Legislation & Regulations
• Suppliers & Outsourcing

Since 143 risks is quite a lot, we decided to give a limited set of examples of risks and detail these. For instance there occurs a performance risk, since a cloud service usually has several customers. So it’s not only you as a customer that is putting load on the service, but also other users of other customers. This will influence the performance of the service. Imagine your webshop hosted at the same hosting provider that hosts WikiLeaks… The huge amounts of traffic that a new publication on WikiLeaks will generate, might result in your webshop not being available due to performance problems of the service.

With testing we provide methods to mitigate risks, so that is what we did too. The good news is that we can still use a lot of what we have learned over the years. Some techniques need to be tweaked to fit in the cloud context, but they are very useful. Next to the tweaked measures, we also describe some new measures that we have used at our clients. We grouped the measures too:

• Selection
• Performance
• Security
• Manageability
• Availability & Continuity
• Functional
• Migration
• Legislation & Regulations
• Production

How to test the scalability of a cloud service??? Providers promise scalable services and customers pay per use, so if you need more, you will get more. With traditional load testing, we can gradually increase the load and see how the system responds. This can be applied to the service too, but it will scale. You will see the point where the scaling starts in your response times, they will drop when more performance is added at the service. Check around the boundary of the scaling point to see if the billing is also scaled.

We see that test starts earlier, the scope is wider and testing will not stop in production.

Inspirational Talk: Sky is not the limit: Copenhagen Suborbitals – Peter Madsen

The inspirational talk was very nice, however not very test related. It showed us that with the right vision and perseverance you can reach goals that seem to be unreachable. Peter showed us how he built a homemade submarine and a homemade rocket.

Advertisement

Experiences at EuroSTAR 2012 (part 1)

Since Kees Blokland and I got selected for EuroSTAR 2012 with our presentation on Cloutest by Polteq, this was my second EuroSTAR and it was a nice experience. Next to the presentations I met a lot of friends and other interesting people. Since I seem to be unable to read all my notes, not all presentations I visited will have a substantive report.

Test innovation for Everyone – Alan Page

The first keynote! Since innovation starts with generating ideas and combining these ideas to make something new, testers are really fit for this job. So why do we start to innovate… mostly to solve a problem.  You usually start to think about how to solve the problem, but are we solving the right problem? Make sure to try a lot, but keep realizing if you are doing the right thing.

Testing Spreadsheets – Felienne Hermans

Since I know Felienne, but hadn’t seen her in years I really enjoyed seeing her as a speaker at the conference. She started of by giving us some facts on spreadsheets:

• 95% of all U.S. firms use spreadsheets for financial reporting
• 90% of all analysts in industry perform calculations in spreadsheets
• 50% of the spreadsheets is used to make important decisions

After knowing these facts, start realizing that spreadsheets are almost never tested… Looking at some spreadsheets, you can notice that testing spreadsheets can prove to be quite difficult. Lots of the formulas in the sheets are too complex to evaluate, some contain fixed number that can mean anything and of course a lot of assumptions exist in the formulas. So when we test spreadsheets we should focus on these risks! Testing the functions can visualized by using guards and/or conditional formatting. When doing this, make sure you know what you are doing and that you use the right visualization.

Unconventional influences – Alan Richardson

My notes on this presentation seem to be a bit blurry, but that might be my conventional way to capture the unconventional influences. Two statements in my notes are ghosthunting and e-prime. Okay, that unconventional indeed… The influence of ghosthunting (as I recall it) was the use of several independent observers. These observers need to match their observations with each other and in this process they can find either explanations for their observations or anomalies. E-prime is helpful when we need to report on our observations.  It makes the observations less biased, by not using the verb are. The statement might become harder to read, but at least we will have to think a lot about what we really mean to say. This encourages not to use definitions, but always provide the context. Make it simple and explain what you mean.

The Certified Agile Tester exam

I received some questions about the Certified Agile Tester exam and decided to answer them in a blog post, so that more people could benefit from this. The course will prepare you for all the parts of the exam, but if you will pass will depend on a lot of factors. At the moment of writing, the exam is available in English and in German. You must be able to answer the questions in these languages too. For the normal courses it is no problem to have some spelling or grammar errors as long as the correctors can understand what you mean, but the demands are higher for the train-the-trainer courses.

Theory

i have some questions about the nature of the examination for CAT as the description is pretty vague . What does an open questions exam mean exactly? Can you detail it a bit ? Like how many questions in the 3h, what s the general scope of the questions ? How your knowledge of written English language affects your mark(and your ability to express yourself in English for non native English speakers) ? Maybe an example of similar question to one at the exam?

The theoretical part of the exam consists of 10 open questions, directly related to the contents of the course and 3 scenarios that also have a couple of questions (this number varies). During the course you will get example questions that can be discussed with the class. Giving an example of a scenario is quite hard, but the open questions could look like:

“Name 1 statement from the agile manifesto and describe how it affects the testers in an agile team”

You will then be awarded four points for an answer containing one of the statements of the manifesto and at least three explanations of the effect on testers.

Practice

Second part of the examination aka a practical section where the examinee’s testing skills are put to the test, how exactly is that done ? There are 100 ways to test practical knowledge in my mind , how exactly it is done here ? Again maybe you can give a theoretical example in the style of the exam.

The practical part consists of testing a website which is described by several user stories. You act as if you are the tester in a team and start working on planning your time, deciding what and how to test followed by actually testing the website. The increments to the website can be deployed, so you will see some bug fixes and some new bugs throughout the exam. Note that you have to write down a lot, to give the corrector a good impression on what you planned to do, what you actually did and why you did or did not do certain things. During the course we will work on a website too, this will prepare you for the exam.

Focus

The certification is focus on the Scrum “flavor” of agile development or it s general including Extreme Programming ,Crystal etc? I m asking mostly because from what i ve read already some of the “flavors” (in this case Extreme Programming) diminish the role of the professional tester to the point that it s just a hat the developers wear at some points in a iteration.

The main focus of the course is on the Scrum flavor. We do talk a bit about some other flavors and some of the questions in the theoretical part can deal with another flavor than Scrum. By the way, I do not agree that with Extreme Programming the professional testing is just a hat the developers wear. Never completely rely on the books you read, in practice you will see more use for testers!

Testing in China

Last 9 days I’ve been in Shanghai together with Martin Pol for Polteq. Luckily the typhoon didn’t reach the part of Shanghai we were in, we had some wind and rain, but nothing to extreme. We’ve presented some tutorials and tracks at ChinaTest 2012, but also needed to visit a couple of companies to talk about testing. So there was a lot to do in very little time. At one company you speak with a selected group of a couple of people, where at other companies you have a room filled with about a hundred people and a video connection to other offices where also a hundred people are attending our presentation on testing. No matter what the setting was, we would get a lot of questions, since everyone wanted to learn as much as they could from our visit. This was also the case for the conference.

Together we presented a full day tutorial on structured testing. It was great to notice how people at first were a bit hesitant to ask question, but during the day dared more and more. Some concepts that require little explanation in Europe, required more in China and the other way around too. The next day of the conference Martin had one half day tutorial on compact test process improvement and I had two half day tutorials. Both of my tutorials dealt with Agile testing. The first one was on how to work as a tester in scrum teams. At some parts of the tutorial I had groups of people discuss together on some topic and then present their findings to the whole group. Usually I can follow what people talk about and try to guide them a bit in the direction I want them to think, but my Chinese is not good enough to understand the discussions 😉 So I just had to trust that they were talking about the topics I wanted them to. When we started discussing about the topics in English it became clear that they did. The other tutorial was on how to improve testing in Agile (mostly scrum) settings. What struck me was that most questions dealt with automation and tools and not the actual job of testing.

The third day of the conference started with key notes from Martin Pol and Richard Bender. After these keynotes we had to do some more company visits. The last day of the conference we both needed to do some presentations. I presented on how to test mobile apps and on the Polteq approach for testing cloud computing: Cloutest. Both of these topics proved to be very popular! Martin presented on outsourcing for a crowd that seemed to grow during the presentation. The conference ended with a set of lighting talks, where Martin told that we need to save the testing skills and I provided some fast insight into Agile.

Usually I try to provide some information on the different presentations I attended at a conference, but since these were all in Chinese there is not much that I can say about them.

I’d like to thank the organizing committee for a great conference and hope to be able to come back for the 2013 edition!

Certified Agile Tester

Last December I became a trainer for Certified Agile Tester (CAT). Following the course has been very intensive, but really great! Four full days of training, with a couple of hours every night to keep up with the provided material, followed by an exam. The effort paid off and I passed the exam, so since then Polteq can offer the training!

Giving this training feels great. Seeing how your students evolve in using Agile and getting more comfortable every day is very satisfying. This course actually has all the elements that I mentioned in my post on good courses. Every day starts with explaining the theory (and exam material) to the students alternated with discussions. All the students actively participate in the discussions, allowing them to learn from each other’s experience. Another part of the day is a practical exercise, where the students work together in teams to experience what working in Agile teams is like.

Not only the students are learning from this course. Every discussion we have, provides me with more examples that I will be able to use the next time. Tomorrow will be the last day of exam preparation and I want to wish my students all the luck with their exam on Tuesday.