Background
Ajay Balamurugadas, Anand Ramdeo, Anna Baik, Eusebiu Blindu, Markus Gärtner, Mike Scott & Zeger Van Hese all joined me in what I’d say was the most fun & inspirational testing session I’ve ever participated in! European Weeknight testing session number two, had me on a testing high for at least a few days afterwards.
The mission was simple: Test the site Text2MindMap, making notes of your session in the form of a mind map & recording any bugs you’d found. Simple stuff really & being focused on mind mapping I was pretty excited by it. I’d also had some recent exposure to mind mapping in a testing session such as this when I’d paired with Albert Gareev that previous weekend during a Weekend Testing Americas session. Albert wrote a good write-up about that on his blog, so I’ll not go into that, but that was also very enjoyable & it gave me the chance to work with someone I think very highly of.
Pairing wish list
In fact Albert ticks off at least one name I’d had on my wish list to pair with at some point. There are many more on that wish list, I’ll not mention them here though. For me, the trend in my wish list is not people who talk good about testing, it’s those that demonstrate it via good ideas that they’ve posted about or excellent experience reports; people who essentially teach others. These people really excite me & sadly there are not enough like them out there teaching others through their experience, instead of just talking the testing talk.
Under Pressure
Anyway back on topic, I’d joined the session thirty minutes late & after a quick review of what had been discussed already & the mission I quickly set about testing the application & recording my findings using my favourite mind map tool Xmind.
For myself having arrived late I felt under severe pressure. I’d given myself half the allocated testing time & having signed up to take the initial Miagi-Do testing challenge with Mr Gärtner I was keen to impress
Testing Models
So what do you do in such a situation? We’ll I’d never really appreciated the concept of testing models fully until that day. I did like the concept & Michael Bolton made them sound very attractive & essential tools for any tester when I’d attended his Rapid Software Testing course. However for me I’d just assumed everyone had concepts of types of testing to try out in their heads & had never really related it to a model. It is though, and even if I didn’t have a written model of my own to follow at that point in time I did have my mental testing model which was invaluable in guiding my testing approach.
For me my model boiled down to the following: Claims, Documentation, Usability, Accessibility, Functional, Security, Performance, Limitations, Code. A simple quick model which I could exercise to an extent within the thirty minutes I had.
My plan was simple; exercise each aspect of my model as quickly as possible, not dwelling on anything for too long. Of course such testing would not catch everything (no testing does); it would however provide high coverage & feedback rapidly for the time allocated.
Times up
The thirty minutes passed quickly & before I knew it, it was time for the debrief. Everyone posted their maps & we discussed aspects of each, interesting defects found, test approaches & such. You can see from the mind map below how I got on
I find it’s always good when testing a UI to firstly work out all area’s of inputs. You’ll want to focus your attentions on these first. Then take note of all outputs & possible inputs which results in an output. From these you can quickly apply elements of your testing model, providing a quick initial sanity of the functionality.
Inspectional Testing
Markus got the chance to demonstrate his Inspectional Testing idea which I’d thought looked very interesting & forced me to go read his blog post’s about it, which I’d had on my to-do list having only skimmed them previously. It’s a good idea & as I’ve commented on his blog, could also be used in a proactive sense.
It really highlighted for me that idea’s shown in execution are more likely to get buy in from others than those ideas that we simple discuss & assume will be adopted. This brings me back to some thoughts I’ve been having around the initial proactive testing post I put up a few months back. We’ve nearly completed a full release cycle using these proactive techniques combined with its awareness (not control) metric, so expect a follow up post from me on how they’ve been perceived & how useful they’ve proven to be.
You can see below from Markus’s mind map how he applied Inspectional Testing to drive out his initial test charters.
Reporting with mind maps
I liked how Markus reported what passed & didn’t pass, I could quickly see from this what area’s he’d covered or not. It got me thinking about SBTM using mind maps as a form of reporting for test sessions. You can easily see what Markus covered & if we decided it required another session of testing then as a tester I could easily come in & decide which areas needed more attention or not initially.
Dangers of adding to a working process
As a tester though I can quickly see the downfall in reporting what I’ve done as I’m doing it. My mind when testing is like a volcano ready to explode, I have so many ideas going on it’s often difficult to keep track. I find that having a mind map, or a notepad handy to take notes of these idea’s or different threads that I’d like to investigate works well, as if I don’t give them attention immediately I’m in danger of losing them.
So that is obviously the glaring danger of adding in another aspect into your working model, by this ongoing reporting you run a large risk of losing test ideas. Obviously for some reporting what they’re doing results in new test ideas. I guess it’s down to the individual & highlights why a testing approach or method of reporting should never be forced upon anyone.
If you’re really keen on the reporting aspect of what was tested but don’t want your testers to lose valuable test ideas when testing, then using a tool such as Test Explorer would be useful. Having used this myself I can honestly say it’s an excellent aid when doing session based testing. It allows you to focus solely on testing while leaving the reporting aspect up to it. It’s not free though, that’s the only downside.
Using others models
Jumping back into testing models slightly Anna Baik explored the application using a Heuristic Test Strategy Model, in this case (although I’m not sure) I think it was The Test Eye’s excellent Software Quality Characteristics 1.0. You can see from her mind map below how this proved useful in allowing her to test aspects she might not have considered without it.
Template mind map models
During the session I’d suggested that a really clever tester would have a template mind map pre-prepared which would provide them with a base list of aspects to be aware of when testing. Anna said she’d considered this also & had thought about putting the contents of a HTSM into a map to use. I’m certainly going to document my testing model in the form of a mind map after the holidays; I’ll publish it here once I do so.
What things stood out?
So what cool stuff did people find? We’ll an excellent & very common issue was found by Anand Ramdeo. He simply disabled JavaScript on his browser & low & behold the application stopped working. I’d noticed when I’d had a quick glance at the source code some JavaScript dependencies, however I failed to follow up on that with the test that Anand done
Error handling
So what’s the big deal then if we obviously need JavaScript enabled to run this application? Well it’s simple really; the application doesn’t provide any feedback to say what the problem is. It’s good practice when coding to account for possible errors, in this case a simple message to alert the user they needed to enable JavaScript on their browser to use the application would have sufficed. In fact it’s often a good question to ask a programmer “What error handling code do you have?” Seriously it’ll at the minimum get them to consider possible errors that might occur.
My assessment
Myself I found the application to be ok for basic usage. It did as it claimed, coverts text to mind maps & in that respect it did a good job at the minimal levels of usage. I wouldn’t recommend this to anyone though for serious usage. Why? This is why:
- Under volume it simply renders unreadable mind maps.
- Under depth certain nodes will disappear.
- It has no undo which for me is basic usability & with the text area being its main area of input could result in users losing data.
- On page hyperlinks don’t have a target of new, resulting in you losing all unsaved data
For me those are all basic functionality essentials. The data integrity aspect is a major image concern for all companies & in this respect it fails on this twice. Sure it’s meant to be a bit of fun, however if the creator wanted to seriously market this then they’d have invest a lot more time on it.
Eusebiu Blindu killed the application stone dead with his million character test Markus Gärtner also did some locale testing using accented characters which was cool.
Taking it local
For me though this was really an exciting, thought provoking & inspiring session. So much so that the next day I setup two laptops in a room & invited my team mates to try it out in two groups of two
The mission was the same as mine & of the two groups only one managed to get the application up and running before their laptop crashed ten minutes later. The other pair couldn’t get a new version of flash installed to run the application. Failure? Not quite! Both groups paired up & produced a textual report which I’ve converted into a mind map below.
I talked to them all afterwards & their perception was that it was a failure as they couldn’t test the application. To me it was a great success! They didn’t give up; they were resourceful in pairing up & using the knowledge and skills of both groups together. In fact the four of them managed to find issues with an application they couldn’t test. Not only that they planned in possible tests for a future test session, which could uncover further defects. I think they did excellent & was chuffed they didn’t just give up when the technical problems occurred.
Wrap up
So to wrap this up what did I learn from this session & what can others learn?
- Developing your own testing model is essential
- Using other available models can be helpful
- Creating a template testing model in the form of a mind map could prove to be really useful
- Inspectional Testing is a good idea & should be explored further.
- Any new input into a working process can have serious negative side effects
- Mind maps could be useful for reporting/debrief/follow-up sessions with SBTM
- Good error handling is important, let’s not forget about it.
Most importantly, there is a lot to be learnt from pairing with others. Even reflecting upon your own approach can provide ideas & insights into what you could improve upon. For myself I’ve came away from this looking to produce a base template testing model mind map to aid my future testing in testing sessions like this & also for my day to day job. Something which I think will prove invaluable for me in the future, so I’m very happy.
I hope to see some of my readers at the next week night testing session after the New Year. Drop Mike Scott a message on twitter for more details if you’re interested, or just keep an eye on his tweets for an announcement on this or any future sessions.
On a parting note, Ajay Balamurugadas is a freak of nature! Seriously I’ve been reading all the reports from Weekend Testing sessions & he appears to be attending every one Ajay, you’re only human, the sleep will have to catch up with you sooner or later
Thanks for reading.
Related posts:
Hi Darren,
Excellent post; it sounds as though you got a lot out of the session!
Your observation about the value of testing models is an excellent one and I think it is worth repeating that a testing model does not have to be written down – it can be your own mental model that you run through in your mind.
I’m interested, though, to note that your team members thought their mission was a failure. Was this because they could not get the application to work? To me, not being able to get the application to work at all is very valuable feedback because it tells us something about the way the application has been built and the fact that dependencies have not been carried through into the final package. Should those dependencies exist? Has there been a mistake in the packaging process?
It would be great to come onto one of the weeknight sessions but unfortunately most week night evenings I have things on. I try and do the Saturday ones though – European and American where possible. They are all great sessions and, as you have observed, the value of pairing up with other craftsmen and women cannot be over-estimated.
Regards,
Stephen
Hi Stephen,
Thanks for your comment
I think it’s worthwhile saying again that we don’t need to write our models down My problem previously the concept of stating I was using a model was that I’d always considered these testing thoughts I had as common sense really; after all everyone thinks about all these aspects when testing right? That’s what I realised though that not everyone does & having a model stated or learning from other peoples models can be very valuable.
More so I think there is great value to be learned from others by showing how they exercise their models in practise. I hope to do a few post in the future around this from my own experiences; I’d love to see others doing so also.
For my team mates even after I’d had a discussion with them about their results individually, from some I could still sense they felt it was still a failure. The fact is, in all testing lessons not everyone can see clearly the benefits & lessons to be learnt from what they have done. The dice game James Bach & Michael Bolton do so often on their course is a key example of this, only some will see the real benefits from it. Others will just laugh it off as a bit of fun, or worse still come away feeling their time has been wasted.
As for the dependency on Flash 9, I’m not sure why that is there at all. I noted it down as a limitation & had I been evaluation this product in a professional sense it would certainly have been something I would have discussed further with the stakeholders. Perhaps there is a reason for it, or perhaps it is simply an unneeded limitation.
Hopefully we’ll get the chance to pair up at a session at some point
Thanks Again.
Darren.
Great write-up of what sounds like a fascinating session. I keep wanting to try weekend testing or weeknight testing and the timing has never worked out but maybe I can join the next one. I’m rather intimidated by it, my real value add is in domain knowledge and working with customers and programmers, not so much in any brilliant spur of the moment testing technique. So I probably won’t be good at this, but I’m sure I’d learn something.
We use mind mapping here to prepare for testing our more complex stories and projects. I use mind mapping for lots of things. So it’s great to see more examples of using mind mapping for testing prep as well as reporting test results.
Hi Lisa,
Thanks for taking the time out to read it; I’m glad you’ve been enticed a little more
I honestly couldn’t recommend these sessions enough. Myself, I’ve only recently began attending them, previously with only weekend session being available I was never able to attend any as my weekends are time for me and my family, however Weekend Testing Americas start quite late on in UK time so I’ve managed to attend one of those so far. Week Night Testing was a great idea & is something which is much easier for me to attend.
If you’re intimidated a little by it I’m sure the community would absolutely love the chance to learn from you as a session facilitator. I’m sure you could come up with something really worthwhile which we could all learn from. I know I’d love the chance to gain some of your knowledge for sure!
Either way it’s very worthwhile & if you get the chance to attend any jump at it
As for your mind mapping for complex stories I’d love to chat about that off line I’ll drop you a message on twitter about it.
Cheers,
Darren.
Sad to read that your colleagues thought it had been a failure. To cite from Quality Software Management Vol. 2:
Your colleagues could have taken the information and turned it into a meaningful test report by stating the particularities of their laptops, and why the application didn’t work under these special circumstances. Maybe it shouldn’t have to work at times, but I’m sure there is one or another occasion where it should have worked and didn’t.
Replying to Lisa, on attending Week[end|night] testing sessions, you should try it out. It’s less intimidating than a Testing Dojo where all your thoughts are exposed once in front of the keys. In Weekend Testing you are all alone in the first hour, and can still learn a lot how others tackled the testing challenge. And I am sure that Lisa would provide meaningful meaningful insights in this part also for other attendees.
Hi Markus,
The two laptops I provided to allow them to test the application were problematic, one crashed, the other just couldn’t install any software at all & was very unresponsive. In that respect for both it was the fault of the hardware and not the application under test.
You’re correct though, it is information & they did from that go on to highlight the limitation of Flash 9 being a problem. The aspect of the application really requiring this version of flash of not could be debated but like you say it’s information worth highlighting.
Thanks for reading & taking the time out to comment, hopefully I’ll also see you at the next session after the new year
Cheers,
Darren.
Hi, Darren!
Thanks for the great experience reports we all can learn from.
I see your concerns regarding “Dangers of adding to a working process”, and, indeed, mindmapping activity will be distracting itself… But! If we do paired testing, we get all the advantages without losing testing insights, if testers really collaborate instead of going on their own. “Parallel processing” instead of “multi-tasking”.
For a duet of testers, one, let’s call “tester-primo”, is the one who is actively engaged with an app, but “tester-secundo” is in position that allows seeing forest through the trees. Therefore tester-secundo can do all the mapping based on colleague’s notes, and own observations, and test ideas. And then the roles can be switched for a second session, so secundo becomes primo, while they keep adding to the same map.
Let’s try “testing duet” approach next time?
Cheers,
Albert
Hi Albert,
Thanks I’m glad you enjoyed it I’m up for another pairing session I’ll drop you an email or chat with you on Skype next time I know I’ll be able to attend a Weekend Testing Americas session & hopefully you’ll be available then as well
Your duo idea sounds like it could work; often just from reviewing someone else’s work you can feedback so much into it. Sounds good, I’m looking forward to it!
Cheers,
Darren.