A short recap
In my recent post about proactive testing I promised to share my method for creating lean test cases. So as I like to keep my promises here it is; I hope you’ll find it useful.
I’ll start out with how this all came about, feel free to skip to the “lean in all aspects” section if you don’t like listening to how people overcome their problems.
Some background
I used to dislike writing test cases a lot! I found it painstaking how much work was involved just to document what was in my head. I used to think this could be time better spent doing other things because I did waste a lot of time writing these, too much time, time that could have been spent testing!
So like everything I thought could be done better I started trying to think of idea’s on how this could be improved. Obviously it starts out with a question “What is the actual problem here?”. The problem being my dislike for writing test cases & my desire to test instead! So looking back now it was obvious that I felt burdened by writing test cases on functionality which I already had available to test. I probably felt that for every test case I’d written I could probably have found some defects on that testable functionality.
Dilemma solved!
At the start of each new release I’d always have a bit of spare time which I could use to get other projects done. This time could have been spent writing my test cases right? If I had some requirements I could start writing down test conditions. So that was it, I’d solved my dilemma of having to write test cases while all I wanted to do was test, I’d just write them before the functionality was written.
Requirements review
So the release went out & the requirements for the next release came in. This was my chance to see if this made me feel any better. So out came the requirements & on went the thinking cap. I began jotting down my test ideas & looking at what types of testing would be required for this project. I also began noticing shortcomings in the requirements & things I knew just wouldn’t work. I also began to generate idea’s! Why do we do this? Or would it not be more user friendly to do it this way? On and on it went, I was having loads of fun.
So I looked over what I’d noted down after reviewing these requirements & noticed that half of it mapped out as an overview of how I’d test this functionality & the other half was good feedback for the BA’s & managers to consider. I’d found that on one hand I could now easily write up my test plan’s & work out how much time I needed to write up test cases. On the other I’d came away with a lot more useful feedback than I’d ever had before, some which might just make this project that little bit more successful if all went to plan. I was very pleased & most of all I was having great fun.
Introducing test cases early
Next stop was the test cases I’d hated writing before, this time around though it proved to be very enjoyable. A few days later I’d written up all my test cases & while I knew they’d probably have to change a bit I was quite happy in that not only was it more enjoyable, as they were being written I’d been spotting lots more gaps & potential issues with the requirements. The BA’s really appreciated the feedback & I was happy that I was more involved with them to.
So the release came and went, I’d begun my first steps into proactive testing without realising that I was being proactive. I’d noticed a big downturn in defects but I’d also spent a lot of time chasing developers to run these test cases I’d written1. This seemed like a better way of working, making good use of a quite period & contributing heavily to the team.
Lean in all aspects
So your probably wondering when the lean part comes in right? We’ll begin touching on it now I promise
But how many test cases?
Previously our entire team appeared to have their own ways of creating test cases with no set standard. Some people opted for a test case for every one to three test conditions, resulting in hundreds of test cases. Others would bunch more conditions into test cases but for each step in the process that changed they’d always have to write a new test case. I didn’t think this was very efficient, or fair at the same time for those people who’d have to re-run them if they ever became part of the regression test suite. I remember once two testers responsible for an important new feature got asked to write some test cases for it & came back with over eight hundred! Crazy & looking into these each had one or two conditions at most for each test case. How many test cases you’ve written means nothing, James Christie wrote an excellent article on this “But How Many Test Cases”. Could you ever imagine asking someone to run those eight hundred test cases with a straight face? Crazy talk!
So that was the first challenge getting the rest of the team on board with how I’d thought these should be written. I’d arranged a meeting & asked that everyone submit how they thought a test case should look. We discussed the pro’s & con’s of each template & grouped our favourite couple, then discussed these some more. Thankfully they all seemed bought into my suggested format.
Mind mapping
So what was the format? We’ll it’s evolved slightly since then so we’ll just discuss what it looks like at present. Firstly I think it’s important to discuss how they are designed.
So like I’d said before I generally I try to take a proactive approach by generating these up front. If you’ve paid any attention to requirements at the start of a project you’ll know they can vary greatly from someone’s vision to very detailed documentation. So you’ll know the extent of your test cases will be limited by the available requirements, something which we’ve been lucky with at my company allowing us to provide very good test cases up front for developers to run prior to committing code.
Once I’ve gave feedback on the requirements & they’d become more stable I’d crack open my favourite mind map tool, in this case Xmind. Mind maps are excellent, previously I’d been writing all my test cases in our test management tool & not seeing as much interlink between the different area’s of the feature as my thoughts were constrained to my current test case. I’d also have to jump in and out of test cases whenever I’d thought of new conditions for a test case I wasn’t currently creating or editing. With a mind map as you build it up you just see all the links in the feature so you’re fully aware of its integration points & influences.
With Xmind you can add extensive notes to each node on the mind map, this is what I used to write up all my test conditions for that part of the feature, to later by dumped into the test management tool. If I think of a new test condition for some other part of the feature I simply select and edit the notes of that node on the mind map. What took a little time before now takes seconds. Likewise as everything is visibly in front of me my mind begins to naturally consider more aspects of the feature in turn generating better test conditions, that I might previously not have thought of when using a test management tool to write these.
Monkey Madness!
For me my map shows my path through the application/feature, or steps as you’d call them in a test case. When it comes a time to put my conditions into a test management tool the map becomes the test case folder hierarchy, making it easier to navigate test cases when running them in the future. If my tool needed me to add in steps for some reason (mine doesn’t & I don’t) I’d simply copy the folder structure onto my steps in a brief format e.g. Make Order > Add Billing Details > Confirm. Let’s keep the fluff to a minimum that’s why I myself don’t use steps, people are intelligent & in my app can see that the folder structure mimics the steps they would have to take.
As you’ll imagine with the nodes being paths or areas through the application or feature we can cover a broader feature area than before. We’re not training monkeys by telling them “for this condition you’ll need to select X, then after a second Y will appear” but to do that you need another step oh crap lets write another test case for this! No we don’t need to hold people hand’s, we can let people think for themselves. As long as the test conditions make sense people will understand that they might need to deviate from the beaten path to test something.
Adding in types of testing
For the conditions I generate these on nodes on my mind map. I split them up into the types of testing I would do. When I say a testing type it would be something like Usability, Extensibility, Security and so on. I generally dump a generic template of all testing types into each node at the start as a note so I can at least attempt to consider if any conditions need to exist for that testing type. From splitting these test conditions into different types of testing, you’ll find that you think more, as such generating much better test cases for that functional area, since your paying much more attention to what you’ll need to test. I also add in a rules section for everything that the requirements expect, I guess you could call these our acceptance conditions.
Increased requirements review
My final section section at the end of my notes is for questions I might have about the requirements. The good thing about Xmind here is that if I know something on my mind map has some unanswered questions noted, I can highlight it in a different colour such as red. Then I can easily see that I need to resolve a few questions for that area at some point. You’ll find when writing test cases up front even if you’ve done a review & feedback of the requirements you’ll still find lots of other things you hadn’t considered that will need some discussion with a stakeholder at some point. These could be gaps, risks or suggested improvements; you’ll see it’s very handy to be able to collate everything you’ve done in one place with a mind map.
Generating the test cases
So you might be thinking these are all very conditional based, your correct! That’s all I want. I’ll take these once I’m happy with them from the mind map & put them directly into my test management tool as-is. Like I said my steps will be my folder hierarchy or dependant on your tool a very brief copy of the mind maps path to the node to replicate the steps the user would make. If I need some setup tasks done before running these test cases I’ll put a link to them at the start of my conditions.
Requirements change they always do & often are not strictly followed, that’s why I tend to leave all my test conditions in my mind map right up until the last minute. It’s easier to change & add conditions in the map, it’s less easier to do so once they’ve reached your test management tool. With the addition of a better testing mindset using the mind map you’ll be more aware of the impact of a requirements change, as such you’ll be able to write better conditions around this.
I’d considered doing a demonstration of all this using a simple application but I think I’ve covered almost everything & hopefully made sense. If anyone would like me to do a demonstration from requirements, to maps, through to the end test cases I’d be happy to, just leave a comment or drop me an email & I’ll do this as a later follow up post.
Summary
So I hope from the description & the screenshots I’ve provided you’ll be able to see the benefit of keeping your test cases as lean as possible. For a quick recap here’s the why:
- Mind mapping
- Increases creativity
- Reduces test case creation time
- Increases visibility of the bigger picture
- Very flexible to changing requirements
- Can highlight areas of concern (or be marked for a follow up to any questions).
- Grouping conditions into types of testing
- Generate much better test conditions
- Provides more coverage
- Using templates of testing types makes you at least consider that type of testing, when writing conditions.
- When re-run these often result in new conditions being added & defects found due to the increased awareness
- Lean test cases
- Easy to dump from the map into a test management tool
- If available the folder hierarchy can become your steps
- Blend in easily with exploratory testing. Prevents a script monkey mentality.
- Much lower cost to generate and maintain, whilst yielding better results.
That’s it for this post, I hope you enjoyed it as much as I did writing it, thanks for reading.
- I discussed previously how I’d solved this in the article “Tales from the trenches: Proactive Testing” [↩]
Related posts:
Interesting approach thanks for sharing that.
You might also drop the test cases completely and go into exploratory testing once you’ve got the initial mindmap.
Since it’s all mapped out quite nicely you can then assign different threads to different testers, either before or after adding the test types. I’m sure some testers will come up with more test types as they go exploring which would feed back into the original map.
Nice stuff, I’ll share it with my team.
Thanks Thomas,
For myself I use them to give to developers as checks prior to committing code. So for myself I’d probably not feed back any extra test into these once the code had been complete. We have a 2nd model of lean test cases we are working on just now for our regression test’s. I might talk about that when see the results of using it.
Let me know how you team gets on if they decide to try this out, thanks again for your comment
Darren.
Good stuff Darren! I’ll need to try X-mind – it looks like a really good user interface too. I’m going to give that a try as I think it could fit in really well against the way our backlog is documented.
Stephen
Hi Stephen,
Be sure to let me know how you get on, I’m keen to learn if anyone finds any other tricks to make this better. Thanks for your comment
Cheers,
Darren.
As I twitted, that was truly a world-class example of test case mindmapping. Thanks for sharing it, Darren!
And XMind is a great tool, for many reasons: http://www.xmind.net/share/agareev/7-1-reasons-why-i-use-xmind/
Albert, I’m truly humbled for you to say such a thing. Also honoured that you’d think that coming from someone who does such an excellent job teaching his experiences to others.
Thanks, and thanks for taking the time out to read & comment on this.
Excellent experience report.
One piece of homework before the Rapid Testing class:
I’ll take these once I’m happy with them from the mind map & put them directly into my test management tool as-is.
a) Practice saying “I’ll take these once I’m happy with them from the mind map & put them directly into another test management tool as-is.”
b) Practice asking, “Hey, wait a second… why would I bother doing that?”
After that, we’ll see if we can give you the final vaccination against testcasitis.
—Michael B.
Hi Michael,
Due to the nature of this late reply, I’ve been there and done your course already as you know
I don’t think I’ve been vaccinated against testcasitis, nor do I think I ever will be. The process followed in doing this approach for me reaps so many rewards it would be crazy for me to give it up
From a pro-active view point I think they do an excellent job for our team currently.
Your spot on for the test management tool aspect & believe you me I’d give that up in a minute if it didn’t provide such good historical information, stats & collaboration for execution in release cycles.
Perhaps Xmind will read this comment & see a sales opportunity & unique selling point
Thanks Michael for taking the time out to read this and comment.
Darren.
Hi, Darren…
It was great to meet you; thanks for your participation in the course. Sorry if I said “Eh?” frequently, but with those dulcet Glaswegian tones, I bet you’ve heard “Eh?” before.
Note that I didn’t say “test cases”, but “testcasitis”. In doctor-talk, “-itis” refers to “undesirable swelling of”. Keep ‘em lean and compact such that they’re the right fit for you and your organization and you’ll do fine. But do keep up with the questioning loop, “Can I do this just as (or more) effectively AND cheaper and/or faster?”
—Michael B.
Interesting, but I am still preferring to write a “daily report” stuff to show my work. Not a t the end of the day but during while I test.
To organize the ideas a simple document works better.
Anyway maybe could be used in different way though also for people that like ET.
Sebi
Hi Eusebiu,
Thanks for reading this, like you say not everyone likes using scripted tests. I’d hope you’d see the benefits thought from a proactive testing viewpoint.
Cheers,
Darren.
Thanks Darren for sharing this!
Great report!
I am with Michael here regarding the test cases, I thought it was brilliant until the part where you put everything in the test case management tool. I was wondering why? You have it all in place already!
This might sound harsh, but if the test case management tool is needed, this looks a bit like Lean Waterfall (or Lean V-model) to me. I.e. it is like a Lean way of getting Big Design Up Front (BDUF). Or am I misinterpreting?
Hi Henrik,
Like I said to Michael, give me a mind map that can do the things our team enjoys from the test case management tool & I’ll jump right in & never use a TCMT again
I’d say for myself your probably correct, in that I do that BDUF. However it really depends on the person doing this & how they choose to approach it. For example if you developing against work packages, you could consider writing these just before the work package begins. Then the requirements would be more set in stone & you could still be proactive in giving these up front to the developers to run before committing code without the BDUF. The only downside I could see with that is proactive requirements feedback, you’d still find the problems with the requirements it would just be a little more cost to fix them, again that depends on how your team works, for us it would be considered un-costed work at that stage.
Thanks for taking the time out to share your views & thanks for reading this
Darren.
Hi Darren,
Thanks for your reply; and I also read the reply you gave to Bolton.
It makes sense to me, though I am not sure I would have done it exactly in your way. I need to think more about this!
Cheers,
Henrik
excellent write up… well explained. … I want to try this tool… will definitely give it a try.
Shilpa, thanks for your comment, please do let me know how you get on
Darren,
I’m intrigued about “…give me a mind map that can do the things our team enjoys from the test case management tool…”
What does the team enjoy from your test case management tool? How does it help them?
It’s a genuine question, I’d like to see if we miss a trick here or if it’s just works in your context.
Thanks,
Thomas
Hi Thomas,
For us the test case management tool allows us a historical report on testing we executed. It’s very helpful, because whenever we’re asked at a later date if any testing has been done in an area or scenario, we can easily see this.
As our company supports many version streams the tool we use also allows us to see this historical report on these different streams.
Another useful thing is the collaboration it provides, we can quickly generate a module of charters here and give them to a group of testers to execute in an auto generated test plan by the tool. You see we don’t have to be restricted by a tool by assuming we can only use it in a traditional scripted sense, like most do, we can make the tool work around our own approach.
We can also quickly generate statistics from the tool, and while the numbers it produces don’t essentially mean anything they can at least highlight if there is a potential problem in an area, worth further investigation. If we think of SBTM this could lead to new session coming in and lower priority ones dropping out of scope for our given time constraints.
Very worthwhile question thanks for asking.
Cheers,
Darren.
Hi Darren,
Can you explain a bit more about your statistics which, on the one hand, you have said don’t essentially mean anything yet you go on to say they highlight potential problems.
Are these problems to do with test coverage – and they are being highlighted by the fact that there is a low number of test cases in those areas? If so, how do you communicate the significance of the coverage issues? For instance you could have an area where very few tests have been written because the risk of failure is perceived to be low or the business impact is low.
Basically, how do you get at the story behind the figures to highlight the issues? This is something which I am genuinely very interested in because I have not found a really satisfactory way of presenting such information to my stakeholders without using prose.
Thanks,
Stephen
Hi Stephen,
The most useful statistic for us is defects. Our test management tool stores our tests in a folder hierarchy from which we usually lay our tests out in the same manner that a user would use to navigate our product. So when it comes to executing tests we can see if an area is showing a high density of defects & then consider what action we take. For example do we continue testing? Or let the main issues be fixed first? Or are the defects so unrelated that it warrants further investigation?
There are many factors to consider, one of the easy one’s being the maturity of that feature. If it was mature and suddenly had a cluster of defects raised in it’s area it might highlight other problems, such as the amount of testing being done when changes are made to mature code, who made the changes? why? and so on..
The main thing is spotting trends, e.g. defect clusters, over runs for test case execution, reduced ROI. It doesn’t essentially mean anything, but prods you to at least ask “Is there a problem here?” When I say ask, I don’t direct the asking of the question from anyone in particular, many will assume this is a management, lead or senior figures role, I think that in a good test team everyone should be doing this.
As for test coverage, this is a problem area in our team which we are still trying to tackle. I’ve a couple of idea’s around it, one in particular I’m very keen to try out, another lean method which I think would work well for us. I’ll be sure to blog about it should it be a success or not, with the later to show to others the why & how at least.
Reporting likewise is something we’re still working on improving, we don’t have a perfect solution as yet, it’s still in the trial error process.
Cheers,
Darren.
Hi Darren,
Thanks for the explanation. It sounds as if you are working in a team for whom this approach to statistics is likely to work because you obviously are delving into the figures to answer “is there a problem here?”.
I am suspicious of figures being bandied about without an explanation or understanding of the story behind the figures. Identifying that there is a problem in a particular area – and being able to communicate what that problem is – rather than just giving a Defect Density Percentage is the key here.
I’ll let you know if I ever get a breakthrough on communicating problem information in writing without a fair bit of prose.
Regards,
Stephen
Hi Stephen,
I’m with you on that, caring about defect density as a metric is a sure way to lose track of what’s important. Using it for awareness though can prove to be a useful tool in your toolkit.
We should arrange a conference call about the reporting issue, it’s something I’d like to perfect, if such a thing as perfect reporting is even possible. If you want to progress it we can see who’s interested in participating, lets make good use of our community
Cheers,
Darren.
What a brilliant post Darren ! It din’t allow me to stop until I finished reading it completely. Comments and Replies have added more value to it. Thanks a bunch for sharing it.
Couple of days before I was asked by my Manager to suggest her some ways where we can improve the Test Design Productivity that actually means the Number of Test Cases one Tester can write in a day.. or say average time for a tester to write one test case. I was thinking of coming up with something like Mindmaps but I was clueless on convincing her its benefits. Your blog has made it just clear and easy for me .
While reading your post the only thing which was running in my mind was to find out.. “how would mindmap make it easy to write scripted test case or say make it less time consuming” and then I figured it out from what I interpreted from your blog.
Even though “Scripted Test Cases” are part of our Deliverables ( at my work place) I personally feel that spending at least half of the day on Mind Map before actually writing down test cases would make it very easy for testers to write their scripts. Since , once you have a clear picture ready.. writing it down remains just a clerical work.
This also made me to recall my school days when we used to get questions on describing some cycles or procedures etc. The best way I had chosen to answer such questions was drawing the figures first (accurate figures I mean) and then just describing those figures in words. As looking at the figure and explaining it in words used to make it very easy for me.
Some folks used to make poor attempts of recalling what they had mugged up .. and that was funny part .
Thanks again for this brilliant post.
Hi Lalitkumar,
Thanks for the comment and I’m glad you found it useful
Visuals can play a big role in test design, and for the most part they work pretty well. This example deals with test case design. Using mind maps for pure test idea generation is also worthwhile, and these of course can later be broken down into test cases. Most importantly I think, is that if you’re going to the effort of designing test cases, then you should be doing it in a proactive manner, otherwise, I feel it’s wasteful, and therefore test ideas with exploratory investigation around those areas, would be a much better fit.
Thanks,
Darren.
Hi Darren McMillan,
I dont have the test management tool, i just use excel sheets for testing (manual)
How can i use mindmaps with respect to testing now ?
Hi Srinivas,
It should be no different, in fact perhaps your test cases in your excel document are simply replaced by the mind map.
You can use coverage notations in mind mapping tools to mark out how much testing you have completed on the map. I have been meaning to blog about this for a long time now, but sadly have not been able to find the time to do so.
I’ll drop you an email when I do get round to blogging about it.
Thanks,
Darren.
Hi Darren McMillan,
Interesting topic on Mind Maps..
I have tried To use XMinds..
I dont have the test management tool, i just use excel sheets for testing (manual)
How can i use mindmaps with respect to testing now ?
Hi Darren,
I think replacing formal test cases with mind maps is a great idea. How would mind maps fit when developing automation for the regression suite?
Hi Vincent,
If you lay your checks out as a mind map (which is slightly different from the style of this post), you can use markers in your mind mapping tool, to mark which checks (test conditions) are suitable for automation. From that you can then change the marker, when the check has been automated.
I had actually planned and half written a post around this, which has been in draft for over a year now, if I ever find the time to complete it, I’ll drop you an email.
Thanks,
Darren.
Hi,
This is swapna from Hyderabad.
This is a very good article. I’ll need to try X-mind. But in my company, We manage testcases by the test management tool called HP Quality Center ( QC ). Usually everyone prefer to write testcases in an Excelsheet and export to QC. So when I had tried to Export from X-Mind, I was stuck with few queries as below:
1. Is exporting from X-Mind to excelsheet possible? or is exporting directly from X-Mind to QC possible? If possible through any mindmapping technique, please suggest
2. To Export, I was asked to purchase X-Mind. Is it not an open source? If there is any best mindmapping tool which is open source for all, please suggest.
I was given a challenge of Implementing this mindmapping technique using QC overall the team by my manager. Kindly suggest