Agile Testing Days 2013

In average it was very good conference. Several speeches had low quality bud these were exceptions and such ones are on every conference.

Agile Testing Days 2013 has a word Testing in the title but it was not only about testing.

My original plan how to write this blog post was to do it as summary keynote by keynote, speech by speech. Then I went trough my notes trying to put them into mind map where each branch was one session. After a while I recognized that I was not able to capture relationship between them – there were just too many connections. Hmm, ok, what about to switch it? I created main branches from topics and it works. Here it is.

I have to admit that I was struggling with the order. Planning, prioritization, etc? Yes. But who are the stakeholders? I am one of them, my team(s) are next and then everybody in our testing (agile) community who is interested to read it. Perfect, we have the order.

Team, Trust, Fun

The most important idea (concept) to take from these three days is about people and teams. We need to trust each other having fun daily. Sounds quite naturally and easy, right? Focus! This is the hardest aspect of every success.

Here are three quotations, which I value the most:

  •  “Humans do not want to disappoint others. So we prefer to deliver under stress and in hurry before not delivering at all.”
  • “Live is too short to do something we do not like.”
  • “Emotions are not accurate!”
  • “Stop it!”

 Great software is build by great people. Great people trust each other but this trust needs to be built.

Foundation of trust

  1. Trust yourself
  2. Trust others
  • Character
    • Clear expectations
    • Meet commitments
    • Show results
  • Disclosure
    • Open and constructive feedback
    • Can we tell the truth?
  • Capability
    • Feasibility of expectations

People are different but behaviors of each individual are not changing too often and it is even possible to measure them.

Important is that there are two types of thinking

  • Fast – we do it kind of automatically and we do not even think about it.
  • Slow – it is the one when we need to analyze, think about, etc.

We use both of them in different situation and important is to be able to know which one brings bigger value in concrete situation.

Team culture needs to be protected and evolved.

  • Focus on the goal not on the role
  • Rely on each other to meet commitments
  • Sometimes: Develop team sub-culture which is protected by somebody (usually by scrum master)

Team members can be effective only if they can fully focus on the goal. Context switching is the highest risk. Investigate what are the most frequented causes why somebody needs to switch context too often and fix it. Also try to reduce the noise to protect basics.

There is always a need for management. But it should not be management by orders. That is a different topic but at least I would like to state following ideas.

  • When was the last time your manager inspired you?
  • Manager of issues -> Manager of inspiration.
  • (As manager) never ever underestimate your power.
  • Change needs to have a reason.

People in IT has very often problem with communication so it is important to know how to talk and how to ask questions. Most important rule is to do not assume answer when we are asking. It is natural we have usually our own opinion but this opinion is not necessary share by others. Keep your mind open and listen people carefully. Then ask yourself if it is answer you are looking for.

Your questions should have some logical sequence

  1. Opening question
  2. Keep talking
  3. Suggestions
  4. Activations

You should always identify all following aspects of answer.


Learning by doing

“Good choices come from experience, experience comes from bad choices.”

In general we need to do things to learn something. Theory is not enough, vision is not enough, or planning is not enough.

Do it now and expect mistakes. There are more likely opportunities.

There are only several best practices and even these do not necessarily fit everyone. Make your own Good practices and evolve them.

When you know that something is good, do it and protect with any costs. When you know that something is wrong, stop it.

“You need to do something (long) enough to be able to say enough of that – no more.”

 One joke: What do have Pinky and Brain similar with software development?

  • Their project is always limited to one episode.
  • They always fail.
  • They always survive.
  • They never give up.



I will again start with several quotations:

  • “Your job as a tester is not to verify software. It is to verify the world is actually changing fast enough.”
  • “When I played with, oh sorry, when I tested the system.”
  • “World is not black and white. There are plenty of other colors and everything is about context.”
  • “Show me the money.”

Business value in testing

Why is testing squeezed on waterfall projects? Probably because it does not bring any value. Or to be precise, it does not bring any business value. Functional testing itself is only measure.

Is it different on Agile projects? I hope it is. But we still have one common problem. Most of the stakeholders do not see the business value in testing. And it is our fail mostly.

“For whom are we testing?”

Think about each of your stakeholder and find out their expectations of testing. Clear, short and descriptive Test Strategy will help them understand how your team would like to achieve required quality. Reporting is also very important - you need to deliver them similar information but on different level.

Responsibilities regarding quality need to be clearly negotiated in the team and everybody must commit to his part. And we are back talking about trust.

“Testers do not take any advantage from developer`s unit tests. We just do not trust them and so we test everything ourselves.”

Sounds familiar? Stop it!

Testing approaches

There are various testing approaches or types (I heard that more than 100). Of course we do not use them all in daily bases but they can be divided according this chart.

Functional testing

Agile does not mean we ignore traditional approaches like functional testing. Of course there are still very important. But we also focus on benefits from others (mostly talking about exploratory testing) and we have a time for them thanks to automation of deterministic ones. Next chapter will be whole about exploratory testing but I would like to spend a minute on functional before.

Test cases are kind of connected with manual functional testing and I consider them to be very important topic. How abstract they should be? Traditionally they have name, pre-conditions, steps and expected results. This is a lot of work so we need to always think about theirs business value. I can see a benefit of them if we are talking about some very complex systems with complicated flows, etc. But even there we should derive them from requirements to minimalize our work.

My definition of test case is “artifact, which adds the required minimum up to requirements to be able to test them in efficient way and provide readable results.” In other words – if your requirements are not good, test cases will not improve them.

So back to traditional test case model and try to remove expected results. We can easily do it because result is to fulfill requirement and whatever kind of requirement should have acceptance criteria. Next step is to remove test steps. You should be able to test something if you understand requirements and your business domain. Preconditions? Some times needed.

In my case test cases are in form of Mind Map, structured requirements with notes regarding testing. Some times I need also few end-to-end scenarios.

Always have in my mind: Everything needs to bring some value. This is easily applicable on test cases too.

Exploratory testing

“Exploratory testing is similar to exploring a town. It is boring and without risks when you are in the center. You probably know it very well there and nothing can surprise you. Sometimes you would like to go to neighborhoods and play there, even to do something craze. Like knocking on each door and test when somebody shoots you.”

Exploratory testing is a powerful weapon we have in our toolbox. But it is very often misunderstand as something like “free for all”. No rules, no plan, no vision… Stop it!

Exploratory testing is activity where we apply and use all our skills. We always have a vision and plan and there are based on our experience from all other techniques we use, other projects and systems we worked on, etc. Moreover we are able to deliver readable and clear results of exploratory testing using different approaches like sessions, runs, etc.


“Aim for the position when automation is an option. Than it is optional.”

 “The goal is not automation. It is confidence.”

 Agile without automation is very hard and probably not effective. But automation is not dogma. Your testing approach needs to be good before you will adopt automation because automated chaos is only faster chaos.

Aiming for coverage can be also very tricky. We need to consider many things regarding what tests should be automated and when. Sometimes it is not even possible (technical limitations), sometimes it is not economic (changing requirements, EMTE).

But automation definitely gives you confidence and time to do something else, like exploratory and usability testing.

Measures and reports

“Every measure can be fooled. Every report can be misleading.”

“The value of information we provide equals the value of decision or change it made later.”

“We have two identical cars here. First one was tested so you can be pretty sure that all critical things are working. Second was not tested. Which one will you choose? And by the way first one is more expensive.”

There is one concrete measure regarding testing which is really ridiculous. It is coverage. I am not saying we do not need metrics based on coverage. But these cannot be used in general view. Example: test coverage should be 80%. Does it mean in average or for every single feature? It is always about context and scale.


Everything needs to be measured carefully to be able to provide reasonable reports. Different stakeholders require different approach. Goal is always the same – provide information that is easy to understand and to activate required action.

Steering group or similar level of management is usually not interested in details. High-level results per big areas are probably enough. Make sure you highlight important problems or risks if there are any.

We are used to use green and red color in reports very often. Red is ok and it indicates big problems. But what is the green for? Passed? But does it mean it is really working or rather that there were no observations during our testing? I leave it open…

Visualization is the key to success. Probably because sight is the most powerful perception.

We measure a lot of things during development life cycle but it usually ends with UAT. But green UAT is not enough. We should also measure delivered value after go-live.

And never ever compare over more teams.

Bug hunt

Do not focus on bugs but look for the causes. More issues have very often the same cause. There is no value to report several of them but we should rather find the common reason.


Risks are everywhere and risk mitigation is kind of never ending story. Risks are also in software development.

The first thing is to find them, but than we need to categorize them because it is simply impossible to deal with all of them. There are two main attributes of each risk – impact and likelihood. The easiest way how to visualize likelihood is in the percentage scale from 0 to 100. Risks with likelihood zero or one hundred are a bit strange because there are more likely facts. You should deal with and mitigate risks that have high probability and big impact. On the other hand these with low probability and low impact are not important. What about rest of them? Be aware of them and regularly review them.

One of the big risks in software development is scope. In agile we know that very often is hard to deliver all requirements so we prioritize project backlogs and negotiate this with all stakeholders. But usually we speak about whole requirements so we limit possible delivered business values.

“Not only which story, but also how much of each.”

And remember, “No” is very often the best answer.



Good requirements are key to successful project. There are plenty of ways how to capture them when most frequented are user stories, scenarios, examples, use cases, etc.

Probably everybody at least heard about user stories. It is short sentence in specific pattern: “"As a <role>, I want <goal/desire> so that <benefit>".

It is very important to keep in mind that such short sentence is only placeholder for communication. It is not a complete requirement. It is impossible to develop according to it, nor to test using it.

So what needs to be there to have testable user story?

First to consider is the “why” part of user story. It means to understand what is the business value to be delivered and how it will be used. Than you need to understand the role by the mean of persona. Finally is the goal of the use story = “what” part.

This is still not enough. It is nearly impossible to capture all required details in such short sentence so you will need some more space for it and probably also some scenarios and examples.

And finally user story is not complete without acceptance criteria. These are also very important for testing as minimum to be tested.

Next aspect of each user story is size. Delivery of big user stories often fails. On other hand small user stories miss context. Good practice is to measure failures by size and find out the best size for your team.

Another very good practice is to avoid unnecessary variations of one requirement. It adds complexity and in the most of the cases one variations is enough.


Agile in general

There were several topics regarding Agile in general (sometimes related to testing, sometimes not).

  • “It is a desperate shame how often and how much is agile manifesto misunderstand or even abused”
  • “Companies move to agile from wrong reasons.”
  • “Hang the rules!”

In general I have a feeling that word Agile is kind of in trend now even if a lot of people does not understand it.


Agile manifesto

This is the most important document only the one which really matters.

Everybody knows these four magic sentences with the word “over” in the middle. But it does not mean we do the left side and right side is banned. It means there is a value on both sides but we value left one more.

Each of these sentences has own challenges.

  • Individuals and interactions over processes and tools
    • What about distributed teams?
  • Working software over comprehensive documentation
    • Find a balance (documentation is needed)
    • Stable vs. no-stable teams
  • Customer collaboration over contract negotiation
    • Contract is never good, but there is always some contract
    • There is no trust if there is something like "but contract said"
  • Responding to change over following a plan
    • Change customers and mature them
    • Some customers have no flexibility to adopt Agile.
    • Sometimes they just cannot

There are no rules in Agile. There are guidelines, recommendations, case studies, but no rules! Following something blindly means only adding more work. Any change requires reason.

“Vision is not enough!!! Here are the first steps…”



Retrospective is one of the most important activities on Agile projects. If you do not evaluate what are you doing you have no change to improve anything.

Retrospective is time when team shares their feelings and ideas and plan actions. But do not wait till retrospective if something is not working. Tell it just in time when it is happening.

It is very important to keep your attitude and do not change it each retrospective. Actions need to followed and measured. If there is no action after retrospective it has no sense.

Good idea is to have more sections than only good and to improve, something like breaks and accelerators.


Out of topic

There was quite a lot of topics or activities I really like.

Lean coffee

It is a discussion over coffee when all participants suggest their topics. Several of them are chosen by voting and discuss in limited time. After for example 4 minutes is voting if a group wants to continue with current topic or if they would like to switch to the next one.

This is a nice way how to discuss things in informal way and I really liked these discussions. I saw that we are dealing with very similar problems and we shared our ideas.


Agile games

There are plenty games we can use during Agile development or to demonstrate Agile principles.

We all know for example planning poker but there are more of them.

1) Order the backlog as different personas

You have always the same backlog but you should order it by business value each time as different persona. It demonstrate that business priorities of different people can be really different and there are only several when they all match. These are where you should start.

2) Coin games – estimations

You have coins of different size and several tools for water drops. Goal is to estimate how many drops fits on the one coin. After while you will discover there are quite a lot of variables.

3) Paper planes

In a team you need to build planes from paper to achieve different goals (like distance, rotation, time in the air…). You have always only one try per iteration and than you have a retrospective and planning. This is kind of communication training.



Mind Maps

I use mind maps for very long time and it is nice to see I am not alone. There are the way I am thinking and how I can capture it because they have structure.

Slides are different approach, which is sometimes also needed – but there are very sequential so use them when this is advantage.



Videos are very good way how to demonstrate some idea. There was a good activity when we collect all videos we are using with description what each should demonstrate.

And here is my favorite one. It is kind of summary of my feelings and I will apply this as much and as often as possible.


  • Gert-Jan Bartelds 11.11.2013 15:18


    you report the following quote, which I also heard:
    “Your job as a tester is not to verify software. It is to verify the world is actually changing fast enough.”

    Do you recall the context and the presenter who said it? I do not, sadly…



Přidat komentář

* Nezapomeňte na povinné pole.