Overview

Tonight I had the opportunity to attend my 2nd Scottish Usability Professionals’ Association event in Edinburgh.  It was a good night which saw two speakers from the University of Abertay delivering two very different talks.

I am not going to talk about both.  Instead I would like to share my notes and thoughts on Dr Jackie Archibald’s talk titled: Emotional Avatars – What and Why, which I found extremely interesting and thought provoking.

Emotions

The talk saw Jackie begin by explaining the role emotion plays in modern day human computer interactions.  She discussed aspects such as perception, cognition, creativity and our ability to cope as reactions triggered by our emotions.  She also discussed how our current emotions can affect our memory, ability to plan, or complete tasks, and our ability to make decisions.  She went on to explain how displaying your emotions can often be seen as a negative thing.  People may questions your ability to work logical, or even think rationally.  However, she explained that emotion is important, and can aid interaction in an intelligent way.

Avatars

With the reasoning out of the way, next came the interesting part – emotional avatars.  She explained that an emotional avatar, was simply a virtual character who could display, and / or recognise emotions.  The use for them she suggested could be in education, training and as help systems.  She provided a example of a virtual avatar which was used in New Zealand to help teach kids working from home, which could help guide them through lessons.  This was called Easy with Eve, and supposedly could detect facial patterns in students and react to them.

She provided a few more examples, of avatars currently in use:

• Asking Anna, the IKEA bot
• Ask Rachel, Tesco
• Virtual agent Sasha, Kaspersky

Jackie, showing Virtual agent Sasha, the Kaspersky bot.

All very interesting and sharing one common trait – they are all provide support desk type feedback.  Perhaps this is how future FAQs (frequently asked questions) will turn out?  None from what I could gather had any concept of emotions, but simply how to display them.

Now the talk became more interesting as Jackie went on to explain the research three of her students had done around this area.

Dealing with frustration

The first student, John Hart had completed a research project titled “Dealing with frustration”.   He began by surveying opinions on virtual agents such as the Asking Anna IKEA bot.  The general feedback not surprisingly, was that people were not overly impressed by them.

For the next step of his project, John developed a travel website which intentionally frustrated users with introduced delays and broken links.  As people used the site it was noted their frustration levels began to rise.

His final step was to develop an emotional avatar, which would see if it could assist in reducing user frustrations.  He would then be able to introduce an agent to help assist the user, using affective language via a text based conversation panel.

What he found was:

• Users were both frustrated with the avatar and the website.
• Avatar frustrations seemed to stem from its limited knowledge base, meaning it struggled to answer certain questions asked of it during the studies.
  • The avatar provided no help to overly frustrated users.
  • Moderately frustrated users which it successfully responded to, noted reduced frustration levels to some extent.

Interesting, but without seeing the research myself it is hard to make much of it.  The approach appears open to a lot of bias and assumptions.  The fact that surveys were employed as a research tool, doesn’t hold much weight really in my opinion.  How, or when he decided to inject the avatar as well, I’m unsure of.  Still, it is very interesting and opens up ideas for how these could be used as a means of online engagement with users based upon some sort of emotional rules engine.

Improving self service systems via emotional virtual agents

Next up was PHD student Chris Martin, who was doing a research project titled: Towards improvement of self service systems via emotional virtual agents.  This project looked into how the user experience of a self service system such as a supermarket self service checkout.  The idea being that an empathetic system could be developed, which could understand, reason and react to peoples emotions.

The first stage of his project involved filming (with permission) people using supermarket self service checkouts, and noting their facial reactions while using them.  These would be logged as facial action units and when reviewed patterns could be identified.  It was found that the two most common action units related to anger.

For his next stage, knowing that anger was the most common response to these systems, he wanted to determine the most appropriate response an agent should provide to the user.  To do this he created a system which would mimic a self service checkout, but intentionally block the users from completing their task.  At this point, an affective agents (emotional avatar) would be introduced, and their response to the avatar would be recorded.  Seven different types of avatar were used to determine which was most suitable.  Each of these had a different facial expression:

• Anger
• Disgust
• Fear
• Neutral
• Happy
• Sadness
• Surprise

Of these, the one found to be the most inappropriate to use was disgust.  The one which received the most positive reaction was the neutral avatar.  I sought clarification after the talk if text was displayed to the user to match these differing emotions, but it appears that only an avatar image was presented to the user.  I assume some processes were still available to the user at this point to progress or cancel the checkout process though.

Usable security via affective feedback

The final research project titled: Usable security via affective feedback, was being done by a student named Lynsay Shepherd.  This one though was only just starting, so there wasn’t much to cover research wise, as none had really been done yet.

This project would look at how systems could improve risk awareness in end users using affective feedback.  Users would then become more aware of the risks their behaviour could introduce.

Thoughts

All in all, it was a really interesting talk and for that reason alone I’m glad I took the time out to attend.  I struggle though to see how avatars using current technology can react to end users.  I did ask the question and Jackie provided a decent enough response, although perhaps not quite what I was looking for:

• Biometrics could be used to determine user emotions.
• Mouse devices with hand grip detection, to determine frustration.
• Cameras which monitor facial reactions.
• Language used during text interaction with the avatar.

The last point certainly seems valid.  What would this be useful for though?  FAQ type systems?  Could you possibly drive site navigation via a keyword driven avatar, returning differing results depending on gauged emotions?

My thoughts with current technology are that outwith user inputs such as direct textual conversation, it is hard to gauge emotion.  I do think though that some form of rules engine could be developed to determine emotions and react to the user in different ways.  For example if the user exhibits some form of navigation pattern while using a website, which indicates that they might be lost, perhaps an avatar would help negate negative emotions, and if it successfully helps them find where they need to go, perhaps their mood may then even be positive.  Or, what if the user is found to be on a checkout process for some time, perhaps an avatar could be injected onto the page to provide assistance.

Regardless, I think relying upon hardware for now, is a non starter.  Perhaps as devices become more camera ready ( mobile is certainly miles ahead of desktops currently), then detecting facial emotions might not be so far away.  There is a big assumption of course that people will want their devices monitoring them, which I think is very unlikely to be a given, for many obvious reasons.

There is serious potential though for emotional avatars.  I’m still unsure where though.  The world certainly could do without another Microsoft Clippy that’s for sure!  With some thought though and making use of behavioural interaction modules that new content management systems are now beginning to make available, I think there could be room for these emotional avatars to build empathic relationships with users.

Thanks for reading.

Gradually engaging your audience

Ever since I first heard Luke Wroblewski talking about the concept of Gradual Engagement, I had an immediate “Oh, why didn’t I think of that?” moment.

To put it simply, Gradual Engagement is the process of allowing a user to explore what you have to offer, before deciding if they like it enough to actually sign up for it.

Gradual Engagement doesn’t force a user to register first.

Software Test Pro, forced engagement. When attempting to access a podcast, I’m forced to engage into an unwanted membership, be that free or not.

If it does, you don’t even realize it has occurred!

geni.com’s old design used to make the family tree creation process double as a transparent sign up process.

Why should you be forced into an unwanted engagement, when all you want to do is get a feeling for something?

A glimmer of hope?

Crazy huh?  I’m sure CEO’s all around the world are scratching their heads thinking, surely if they register first, I’ll have more chance of keeping them?  No!  Half of the battle is getting people to register in the first place.  The sight of a big ugly old registration form is enough to make a lot of people turn around and run for the hills.  Furthermore, so many forms capture needless information just to fulfill some pointless metric, or open up avenues for the company to spam you.  Yes, users feel harassed at being asked for more information than they should do!  Why should they trust you?

Forcing users to jump over fences to get information that they want is not the way forward!

Most importantly, Gradual Engagement isn’t new!  It’s been around for a decades.  Remember shareware, freemium business models and the more recent spin liteware (no exact wiki reference sadly)?  These are all similar concepts.  Get people interested first before making them part with their hard earned cash.  The phrase Gradual Engagement is simply far more self-explanatory, and in turn makes itself more useable as a medium for making others see sense.

Why should you care?

So what on earth has all of this got to do with Software Testing?  Well if you think testing is merely about checking if software works or not, then I’ve probably lost you somewhat.  Don’t worry though; testing is much, much more!  Testing is among many things, caring about the success or failure of your product.  Part of the success may be investigating how people will interact with your software, and trying to proactively avoid them wanting to throw in the towel at the first hurdle.  By removing the hurdle in the first place, you give your product a fighting chance of actually being adopted by someone and not simply thrown away.  Good testers care about these things.  They think holistically, not just about what’s in front of them today or tomorrow.

A new playfield for the losers and winners

So moving on, we are starting to see consoles of all things getting their own apps.  Fantastic!  I can fast forward 20 years into the future and see myself never having to leave my couch!  The wonderful thing about consoles though, is that it’s a new paradigm to play with.  It is all very new so we can expect many great successes and many great failures.  For Gradual Engagement there is one excellent example of both already.

NetFlix provide a video application for the Xbox, which allows you to watch media content.  After downloading and launching the app you’ll be asked if you are a member or not.  Fair enough, but if you aren’t a member your next screen, instead of being a navigation paradigm to explore and preview their vast array of content (which they so want to show off), instead you are presented with a sign up for a free trial option.  So not only do you have to fill in a form of some kind, but you’ll also have to engage in a trial which you may, or may not remember to cancel should you no longer wish to use it.  For all you know, you might not even like it once you’ve signed up for it, so why the hell would you want to go through the hassle in the first place?  The damndest thing though, is that unlike web paradigms which have lots of interactive elements, the Xbox app is a linear process, so I can only ever go forward (sign up for trial), or back (exit app).  Pretty stupid, if you ask me!

ILoveFilm on the other hand, also have an Xbox application, but wait!  Listen carefully and keep this between ourselves for now!  ILoveFilm let you preview their content, they let you watch movie trailers, they even let you search for films you might want to watch and better still, you haven’t had to give them any information yet!  It is your choice if you want to proceed further and engage with them in the form of a mutually agreed contract.  You haven’t been forced into something that you might regret later!

Spread the word!

So go forth and tell the tale of the benefits of Gradual Engagement to anyone who’ll listen.  Make your company some more money and don’t forget to tell them who told them about it when you ask for that bumper pay rise!

Thanks for reading.

I don’t usually post this sort of thing, but today I found a blocking bug that required some technically understanding of why it was occurring to overcome.  Thankfully this wasn’t found on a client site, so I can post it here.

Please view the video then come back and read the rest.  Don’t worry, I’ll wait..

So obviously ScotRail have put some thought into the key tasks a user will do when visiting their website.  Having a journey planner directly on the homepage is a huge thumbs up for user experience and usability.  Sadly the positioning of their error message “Please enter both origin and destination stations” actually blocked my progression.  I was stuck!

I had two options:

1. Disable JavaScript and have to make sure I guess the format of the station names correctly since I wouldn’t have autocomplete.
2. Crack open Firebug and remove the HTML holding the error message!

I opted for number two.

It gets better though, it turns out this issue only affects users who use the keyboard to key down the stations auto-complete list.  I only realised this when I switched the debugger on to step through what it was doing.   It makes you wonder though, since website builds are relatively cheap due to the huge amount of companies competing against each other, how many of these companies actually employ professional testers to test their sites.   I’m not saying every company who hired a tester would have caught this, but it’s something that has always been on my testing check-lists since way back when I first started out in this craft.  After all keyboard navigation plays an important role to not only power users, but also to assistive technologies such as screen readers, so you would hope most would also consider keyboard navigation in their testing check-list.

In the end this blocker for me probably only affected a minority of users, since most people who use keyboard navigation for components for which this is commonly available are either:

a) Using a screen reader and will therefore be unaffected (unless partially sighted using it to aid navigation, though even then they’ll likely quickly overcome the hurdle).

b) Experienced in the affordance of common components (power users), which will also be a small minority.

Still, a blocker though, unless you can work out how to overcome it!  I guess that’s the important part of this, when you prevent a user from achieving their aim, their already limited emotional involvement with your site diminishes.  Some might venture further, in search of a solution – some though will simply give up and leave.  Depending on the context and earning potentials of your site, simple little things like this could be the difference between success or failure.  An article that sums this up brilliantly and one I reference a lot when discussing usability is the three hundred million dollar button, by Jared Spool.

It also highlights why it pays as a tester, to investigate the why and the how behind the bugs you are going to raise.  You’ll not only produce more helpful bug reports which in turn make those who read them happier, but you’ll also gain a lot more credibility within your team.

Thanks for reading.