Ah! A Friday Challenge!
The 19th of November marked the day of the first Friday challenge in our office. The idea was that we’d have one each week & these would be fun, short, simple & provide people with the chance to learn & discuss what they’d tried to solve. I must say when I heard about the idea of doing one each week I thought it might be a little bit too much! None the less one sprung to mind that I thought I’d give a try.
So with fun in mind, this challenge was all about those nasty little things that we call assumptions, which at times can be useful but more often than not dangerous & wasteful.
The challenge ran in parallel with our R&D team & also on twitter with the testing community. I’ll start of with our team before taking it global.
The challenge
So the challenge was very simple I’ll recap it below:
“The world could end in five minutes! You’re currently in the office using a program that a wannabe programmer “Tiffany Swift” has developed for your team. It requires you to be present to press a key every 3 minutes. Your boss is expecting the results from this program when it completes! He’s stressed it’s very important.
What do you do?”
I already knew what I was looking for obviously. Each of these would be found simply from people questioning any assumption they’d made. In fact, between doing this with our R&D team & on Twitter each aspect of this challenge was cracked! I was impressed with that, because there is a lot that we can derive from this in the form of assumptions.
What the locals did!
So I’ll start with a summary of the responses from our R&D team. Be warned some of them are a bit silly; it appears to be mandatory to crack at least one joke with every email they’d sent 
Recruitment
- Our product manager suggested using “name bias” as a recruitment policy. Apparently he wouldn’t hire someone with the name Tiffany Swift *tut tut*
Automation
- Three people suggested using automation to prevent you from having to press the key manually. I’ve included these below.
- Does the requirement to constantly press a button match the initial requirements and could this not be carried out via automation?
- Do what Homer did in the Simpson’s – get one of those little nodding bird toys to press the key for you!
- I’d spend the 2 minutes, 59.9 seconds between key presses training a monkey to do the job.
Compliance
- Two people suggested just continuing to use the program until it produced your results.
Documentation
- Someone wrote some test cases & highlighted what to do if the world ended or not.
Selfishness
- Someone suggesting nuking the site from orbit! I’m calling them selfish as I’d assume they’d not be on earth when they did that. Either that or they’re just stupid
- Someone also suggested building a time machine. Now, if you’ve ever watched Star Trek you’ll know this is a really bad idea!
Requirements
- Someone questioned the purpose of the program.
- The same person also questioned who it was intended for.
Approach
- Someone suggested sitting Tiffany down and using the program with her, taking notes & questioning aspects of the program while she uses it.
Assumptions
- Most people assumed the program was related to the world ending
- Most people assumed the world would end.
- Most people assumed pressing a key on a program every three minutes was acceptable
Risk
- Two people questioned the probability of the world ending in five minutes
- One person questioned the relationship between the world ending and the actual program
- Someone wanted to know how the program would make the world end.
- Someone stated that the world could end at any time!
Design
- Two people questioned whether the time you pressed the key at mattered
- Someone asked if you could just hold down the key
- Two people wanted to know how long the program took to complete
- Three people wanted to know what results it produced
- Someone questioned why it produces these results
Other
- Someone highlighted that if Tiffany was hot, they’d work harder for her!
So some pretty good stuff came from our R&D team, for me they had cracked all but one aspect of the challenge. However a technique was suggested which would have probably cracked that last aspect as well.
Thoughts from afar
I’ll move on then to the testing community who had also taken me up on my challenge on twitter.
Thank you to: Michel Kraaij, Peter Haworth-Langford, Simon Morley, Stefan Kläner, Stephen Hill, Steven Hedley& Thomas Ponnet who all joined in & despite my best attempts to put them off they still kept coming back with more questions like all good testers do.
I kept a feed of everyone’s responses to the challenge here. There were some very interesting questions & funny remarks in this; I’d recommend giving it a read if you have time.
So here’s a breakdown of the responses from the testing community that differed from our R&D team:
Requirements
- Someone asked what type of testing the program was doing
- Someone asked for the non functional requirements of the program
- Someone asked for more information & got it!
- Someone asked if they were using or testing the program
Stakeholders
- Two people asked who it was important to
- Someone asked the boss to provide quality related questions
- Someone asked why the boss cared about the results
Compatibility
- Someone wondered if the modifier keys on Windows / Apple would be any different.
Program
- Someone asked what would happen if we didn’t press a key
- Someone asked if you could press the same key more than once
- Someone quizzed why we had to press the key every three minutes
- Someone asked what benefit the program provides
- Someone thought outside the box & asked if pressing his car key would work
Usability
- Someone suggested raising a usability issue against the program!
Automation
- Two people expressed that it should be automated, one declaring “no testing occurs here”
Selfishness
- Someone decided you should tell your boss to do it, and then do a runner!
Doomsday
- After what seemed like forever someone finally questioned how the world would end
- Someone questioned how the program was related to the world ending
When to say no
- Someone related the test with a monkey! Then stated they could waste their whole weekend pressing this key
- Someone expressed that they would be reluctant to accept the application if they had to press it every three minutes
Time frame
- Someone questioned the time frame of five minutes
Approach
- Someone stated I’m testing without a mission, as after quizzing me they’d realised I’d no idea what the program did!
- Someone suggested using Michael Bolton’s “I feel stuck” heuristic
Techniques
- Lots of people expressed boundary tests
So between the two groups of participants there were a lot of responses, with some pretty good questions asked. A lot of people made assumptions and never bothered to question them, leading them astray! Remember assumptions can be dangerous!
Breaking it down
So let’s quickly look at how you could have solved the challenge.
To solve the first part we could have looked at the challenge as a whole, and broke it into sections.
1. The world could end in five minutes!
2. You’re currently in the office using a program that a wannabe programmer “Tiffany Swift” has developed for your team. It requires you to be present to press a key every 3 minutes. Your boss is expecting the results from this program when it completes! He’s stressed it’s very important.
We can see there are two main things; the fact that the world could end & a program. From breaking this up we can begin to raise questions on this & some people did do this. A few people directly questioned the relationship between the fact that the world could end & the program. That’s one of the things I was looking for; they did in fact have no relationship.
We could further break this down now that we know the relationship is void.
1. You’re currently in the office using a program that a wannabe programmer “Tiffany Swift” has developed for your team.
2. It requires you to be present to press a key every 3 minutes.
3. Your boss is expecting the results from this program when it completes! He’s stressed it’s very important.
But why press the button?
Some people considered the technical ability of Tiffany which was good. Others focused on the fact that you had to press this key every three minutes. A bunch of people suggested using automation to solve this, which was also very good! Although not quite what I was looking for. Michael Johnston suggested observing Tiffany using the application and taking notes & questioning her about the program, which in itself could be looked on as a type of usability evaluation. This was really good, by taking this approach we could have found one of the other aspects I was looking for. Stephen Hill said he would just raise a usability issue against the program! You’re correct, why do we have to press this stupid button in the first place anyway? Sure, automation would provide a workaround to the problem, but the issue here is having such a bad usability flaw in the first place. That was Tiffany’s flaw, she wasn’t a programmer & while she had created a useful program she never could figure out how to stop it from forcing you to press a key to continue every three minutes.
Reminds me of a time…
We don’t even have to raise a bug always; we may find that just by communicating our problem with others, someone with the skills to help may be able to fix it easily. Or better still, they might provide a more effective solution or alternative.
I remember once my product manager at that time had asked me to write up a script to extract some stats from our session logs. The aim of this script was to find the amount of database calls being made at any given time when using our product. It was fun, I enjoyed writing up this little script to do this, and the next day I sent out an email to the rest of our R&D team showing them this script that I’d made, hoping it would be useful for them also. I got a reply pretty quickly from a developer “Oh that’s a lot like an awk script I wrote a year or so back!”, he then went on to share his script with the rest of the team as well. His script was excellent; he’d spent a lot more time on it than me & had made it do lots of cool stuff. To me that had highlighted the power of communication. Just from discussing what we do, have done, or are doing, we most often come away with more useful knowledge, which could have been lost had we not communicated it in the first place. Now this developer on the other hand, could have shared his useful script a long time ago, which he probably viewed as a personal tool to aid his job. That script is now heavily used by the rest of the team
Times running out!
Anyway back to the challenge & sorry for jumping off topic slightly. There was only one other assumption that I was hoping someone would question & someone did eventually. One of our developers Stuart Chalmers pointed out that the world could end at any time & this had made no sense to him in the context of this challenge. Ah! Most people assumed that the fact the world could end was fact! Only one person had thought to question that assumption. Could, would, might, possible, won’t? In hindsight keyword based testing would probably have been useful here.
So that’s all I was looking for. Someone to determine the program and the fact that the world could end had no relationship. Likewise the fact that the world could end didn’t really matter, it could end this second for all we know Finally having to press a button on a program every three minutes is a waste of people’s time! Let’s fix that so we don’t have to press the button
Lessons learnt
Are we done yet? Well you could be if you like; I mean the challenge is solved now after all. However you might like to read on a bit more to see what we can learn from this, after all that’s the best part of any challenge
The challenge highlighted to me that people skip usability quite easily if they hadn’t considered it as an aspect in their model. Some people provided workarounds to the usability issue, but no one had a solution. My company is excellent at considering usability at the requirements stage, they also consider it at design & after this some people will highlight a few things here and there. So for me I think my company would probably benefit by inputting usability feedback at the later stages. One of the things I’d been looking at doing myself was usability evaluations for our new features with end users. This would help for sure; we could also look at reviewing standards such as Elmer and possibly training up some people on community recognised courses. Even though we are good at considering usability as a business, I think you can & always should improve where possible. To me these seem like cheap suggestions to try, so hopefully we will.
Questions, questions, questions!
Twitter’s testing community or at least those that joined in on the challenge, wow! What can I say? You don’t give up do you? At times it must have felt like you were asking questions to a brick wall! Such were my responses. Probably the most useful question anyone could ask came from Michel Kraaij who had simply asked if there was any more information. He got his extra information & asked for more, from which I told him I didn’t have anymore but dropped him a subtle hint! He missed the hint which was easy to miss anyway, but as a tester that’s an excellent question to ask! Just even asking people questions in the first place is a powerful technique in itself. Have you any more information? What testing would you recommend here? What tests have you done? What does the customer think? You could go on forever! I think though some people have a fear of looking silly. Remember, you’ll only look silly the first time you ask something. Even then it might be just yourself who thinks they look silly.
Michael Bolton told us a story at his Rapid Software Testing course & the details that I can recall are sketchy, but from what I remember of the story he was a product manager who asked questions all the time. I mean he’d spend his day asking a whole pile of questions to different people so he could gather up the information he needed to do his job. After some time, people began asking him the questions. He became the go to guy! Why? Well people thought he knew most things, even though he was the one doing all the asking! Funny thing that, but hey, in the end through communication from asking all those questions he probably did know most things, as by asking those questions he was learning.
What problem are we solving here?
So what other cool questions got asked? I quite enjoyed Stefan Kläner’s observation that I was testing without a mission, as after all I was just pressing a button waiting for the program to finish. Was I testing though? Was I even checking? I don’t think I was doing either, in fact I think my valuable time was being wasted by this programs flaw. A few people asked who the program was important to & who actually cared about its results. Both of these are excellent questions, both could help you gather more information on your users & stakeholders.
Another good one from Stefan Kläner was asking about what benefit the program would provided? After all we build programs to solve people’s problems right? If there isn’t a problem to solve, then why are we wasting valuable time & money on it? That’s a question I use often “What problem is this trying to solve?”
Help!
A few people questioned the timeframe which was good; it’s always good to know what our influences on time are e.g. budget, resource, key events and so on. Michel Kraaij also suggested he might use the “I Feel Stuck!” (Michael Bolton) or “Mission Rejected” (Cem Kaner) heuristic. I liked his response a lot because often testers don’t know when to raise the white flag & admit defeat. Often just a break or some help is all that’s needed. Sometimes even, it might just be impossible to continue. To me asking for help is probably the most important one here, I’d consider that a key skill in itself, knowing your limits and being able to ask for assistance. Its problem solving after all, just asking someone for help, you’ll solve it more efficiently. Always asking though, that’s another thing! Don’t do that
So that’s it, I’m sure I could pick out many other things to discuss from this, but I think I’ve went on long enough. Plus my fingers are getting sore now from pressing all these buttons!
Thank you to everyone who joined in it was fun & thanks to all those who took the time to read this
Related posts:
Good write-up!
As I remember, my thinking on the challenge:
My first assumption was that “the world could end” and the program were not related – it was not explicitly stated that they were connected. I didn’t feel this was the first assumption I needed to look at, so I didn’t ask about it (but left it in the backpocket.)
I didn’t automatically assume the button was a direct input to the program (it didn’t state it was) – maybe somebody observed “a button push” and entered something into the program. I wanted to test this link (I was also thinking of the non-compliance to the timeframe input – as I’d assumed there was no connection to the world ending). Hence, my first question was related to pressing a totally unrelated button – actually a serious testing question, to confirm/frame the target. Answering and following-up that question would have given me a lot to go on…
Hi Simon,
Thanks for joining in! Also thanks for taking the timeout to comment
I felt bad at the time when all these questions from different people were being posted to me on Twitter. I had a hard time trying to respond to everyone & you’re correct your question was serious & excellent! I remember doing a similar thing with Michael Bolton when I attended RST last month; during the dice game I took away the dice & asked him to tell me a number for the table itself. Serious question! I also wanted to see if the dice at that point mattered in his pattern.
I’m hoping someone will do another challenge on Twitter that I can join in on this time. Its good fun being on the other end, however I love participating in them more
Cheers,
Darren
Good write-up Darren. I must admit I immediately discounted the ‘world is going to end’ thing as a bit of artistic licence and that it had nothing to do with the matter in hand. Essentially I treated it as ‘noise’ akin to amateur dramatics
!
I think we can waste a lot of time testing things that should not even be present in the application. Sometimes they are there because of bad design, others because of poor implementation practice. I learned the lesson of concentrating early on user experience (UX) a few years ago when I discovered that nothing was actually done with a person’s initials after they were entered into a piece of software. There was no point, therefore, in asking for the user to enter his/her initials!
Great challenge – I really enjoyed it!
Stephen
Hi Stephen,
Amateur dramatics
The bold text & the impact of the sentence were indeed there to lead people astray 
The problem you describe is a common one in a lot of applications, specifically ones with poor spec up front & many developers working around the code area in a iteration or over a prolonged time. Likewise products evolve and wasteful code is sometimes not removed leading to the same problem you’d experienced. It’s a serious issue & something which is hard to keep present when testing in the back of your mind. Metamorphic testing helps a lot here I find, even adding it as a technique to include in your testing model, will make sure you consider it more often. Time is not infinite sadly, I often find myself dropping many things out of my model, this one though I always keep, it’s very productive & by it’s nature doesn’t require to be run often.
Thanks for participating, reading & commenting
It was fun to see everyone’s responses over twitter on this.
Cheers,
Darren.
First of all: thanks for the great idea of doing a “interactive” twitter-challenge.
As I read your first tweets about the challenge my first thought was, that the “world could end”-part is out of scope. (As your co-worker stated: we cannot know it.) As a result I ignored it completely. But you are right, if we take sth. for granted or sth. seems to be obvious, we should nevertheless speak about it, as it might not be obvious to someone else. Lesson learned!
As simple as the challenge may sound, there is a lot to learn from it. I guess a lot of “testers” would have performed a similar task, when their boss would ask them to, but this challenge is a perfect example, that saying “no” is sometimes necessary.
And when a tester executes a program without a mission or not being able to interpret results, he surely provides data and no information to his boss.
I somehow fear, that there are actually “QA teams” who perform “tests” similar like this one. But on the other hand, it is great to see how many ideas you collected from the community.
Really enjoyed participating!
Hi Stefan,
Thanks for your kind words & thank you for joining in
I agree, sometimes it’s good to say no. I think though what is better, is debating your objections in the first place & providing alternatives or solutions. If we can’t think of any ourselves, we can make use of our team & arrange a meeting or consult others in the field, be they near or afar.
In this scenario the tester knew nothing of the program, what it did, why & who for. Like you said he had no mission, to find yourself in a position such as this (I’m sure some people do) I think personally says a lot about that person. I’d ask why am I in this position? Or why is my team or even department in this position? Why am I not valued?
I’ve worked with indirectly a team in such a position before. Largely though I’d say they did nothing to justify removing themselves from that position. If you promote and offer such low level services & then provide a lower level service itself you’re onto a downward spiral.
I’ve also worked again indirectly with a talented team who had no trust placed upon them, not by us but by their direct employers. Changes that some people there had tried to influence on their processes & strategies fell on deaf ears.
To me it’s very sad when talent is neglected. It’s even sadder when the talentless lower the bar in our profession & continue to gain employment.
Thanks for the comment Stefan, its thought provoking to say the least.
Cheers,
Darren.
Hi Darren,
Well late for a comment I know…eek…but thanks for the challenge! I was actually on holiday when I checked twitterverse and followed the tweets for the challenge. As I followed I did notice that nobody was mentioning the world ending, so I kinda mentioned it ‘tongue in cheek’…although there were a couple of things that crossed my mind…The application could have been a nuclear power station or something which could be dangerous…World ending? Maybe for some people. I didn’t know that when I asked the question. I am now wondering if humour can be a way of extracting information in a non-formal way (although you have to be careful) Simon Morley has already talked about humour and test ideas (I think!) but what about humour and the team…important?. Also I think I asked if ‘Tiffany Swift’ drank coffee, I thought we could have a quick chat with a coffee, and some of my questions may get answered instantly. Proactive testing? maybe…
Thanks again for the Challenge!
Peter
Hi Peter,
Thanks for the comment. I’ll often comment on post’s weeks or months after they’ve been published. It’s hard to keep up to date with my Google Reader at times
My boss also mentioned sitting with Tiffany to have a discussion. Direct communication with developers is an excellent way of gathering information & finding gaps, issues & generating new test ideas. Had you asked me to sit down & have a chat with her over a cup of Coffee you could have been on to a real winner
I agree with the humour aspect; we can often be far too serious about things. Sometimes adding a little humour will drop peoples defences & allow us to forge strong bonds, with the end result being people that will want to actively help you. Of course there is humour & taking it too far and being the office clown; we don’t want to be that
Thanks again & hopefully you can link me to Simon’s post I’d be interested in reading it.
Cheers,
Darren.