I don’t usually post this sort of thing, but today I found a blocking bug that required some technically understanding of why it was occurring to overcome. Thankfully this wasn’t found on a client site, so I can post it here.
Please view the video then come back and read the rest. Don’t worry, I’ll wait..
So obviously ScotRail have put some thought into the key tasks a user will do when visiting their website. Having a journey planner directly on the homepage is a huge thumbs up for user experience and usability. Sadly the positioning of their error message “Please enter both origin and destination stations” actually blocked my progression. I was stuck!
I had two options:
- Disable JavaScript and have to make sure I guess the format of the station names correctly since I wouldn’t have autocomplete.
- Crack open Firebug and remove the HTML holding the error message!
I opted for number two.
It gets better though, it turns out this issue only affects users who use the keyboard to key down the stations auto-complete list. I only realised this when I switched the debugger on to step through what it was doing. It makes you wonder though, since website builds are relatively cheap due to the huge amount of companies competing against each other, how many of these companies actually employ professional testers to test their sites. I’m not saying every company who hired a tester would have caught this, but it’s something that has always been on my testing check-lists since way back when I first started out in this craft. After all keyboard navigation plays an important role to not only power users, but also to assistive technologies such as screen readers, so you would hope most would also consider keyboard navigation in their testing check-list.
In the end this blocker for me probably only affected a minority of users, since most people who use keyboard navigation for components for which this is commonly available are either:
a) Using a screen reader and will therefore be unaffected (unless partially sighted using it to aid navigation, though even then they’ll likely quickly overcome the hurdle).
b) Experienced in the affordance of common components (power users), which will also be a small minority.
Still, a blocker though, unless you can work out how to overcome it! I guess that’s the important part of this, when you prevent a user from achieving their aim, their already limited emotional involvement with your site diminishes. Some might venture further, in search of a solution – some though will simply give up and leave. Depending on the context and earning potentials of your site, simple little things like this could be the difference between success or failure. An article that sums this up brilliantly and one I reference a lot when discussing usability is the three hundred million dollar button, by Jared Spool.
It also highlights why it pays as a tester, to investigate the why and the how behind the bugs you are going to raise. You’ll not only produce more helpful bug reports which in turn make those who read them happier, but you’ll also gain a lot more credibility within your team.
Thanks for reading.
No related posts.
Darren,
Great post. I love it. Specifically, I love that:
- The bug is a combinatorial bug (my favorite kind) / (e.g., in order to trigger it, you needed “Select from Dropdown menu” = Yes “Navigate using” = Keyboard)*,
- You explained it so well and included a clear video of it as well, and
- You clearly described how Firebug helped isolate the issue.
Keep up the good work.
* Given that you only needed these 2 test inputs to appear together in the same test case, that makes it a “pair-wise” defect instead of a higher strength combinatorial bug that could only be triggered by, say, 4 specific test inputs all being tested in the same test case in the case of a 4-way bug.
Hi Justin,
Thanks for taking the time to comment.
I agree and certainly a tool like Hexawise would expose this immediately. It would still need the person inputting the variables to remember to account for keyboard navigation though. So even with the most usable, useful tool there is still the ability for the user to make mistakes themselves by not being able to identify hidden variables.
Thumbs up though, your tool is one of the few aimed at testers I’d recommend.
Thanks,
Darren.
Nice find and write up
I think my Snack Test would have caught this
I had to Google that one Phil, but I get it after reading
Forced keyboard navigation from having a snack in the one hand, which also provides useful snacks for the billions of bacteria occupying the average keyboard. Hopefully this will not be to your downfall 
Ah, ScotRail the company who clearly doesn’t care about your journey, electronic or actual. Oddly enough I noticed the following additional anomalies when I visited the site:
- Clicking ‘Buy Tickets’ (entering nothing) gave a two line error – enter origin & destination, and return date and time, therefore obscuring even more of the page
- The hour for the return journey was blank (not terribly contentious, but why blank the hour field and not the minute? )
- There was a red box around the ‘return journey minutes’ despite having a valid pre-populated value in there. If hour was the offending field then why not just highlight the hour field as that’s what needs addressed.
- There was a red box around the ‘return journey date’ Why? Can’t I return today? The same date was in the outbound date box and didn’t cause a problem.
Nice Del
I didn’t poke about, but it sounds like from your short time poking and prodding that perhaps they didn’t hire any testers after all.