Prototyping: Investigating a testers role.

Summary

In the following post we’ll look at attempting to identify the role a tester plays in a prototyping team.  We’ll discuss feedback provided by the testing community and devise an initial approach that a tester could take.

Background

My workplace has decided to put a big effort on prototyping for the next couple of months.  It’s something new for us as a company as we’ve always just went away and built fully functioning software that’s been asked for.  We’ve never had to prototype anything before.

So certainly it’s a step out of the comfort zone for everyone involved.  One which will require a lot of thought and collaboration from all parties involved.

For myself as a tester it makes me question where I belong in this team, since a traditional functional testing effort will have less importance here; something which would have took up a large amount of my time previously.  I can certainly think of places where I can provide value; however there is still a lot of uncertainty in my mind as to the role we play.  It’s not a traditional test effort for sure, and requires a new approach.

So what role does a test play in a team developing prototypes?

It’s something both myself and my colleague Stuart MacDonald wanted to find out.  So we sat down and had a brief fifteen minute discussion on what roles we could play to provide value to the team.  Jotting down a mind map of our ideas on some paper, we quickly identified some crucial areas we could provide assistance in.

Insights from the community

Now I’m sure other people out there will have a lot more experience than us when it comes to the role a tester plays in a prototype team.  As such I turned to twitter for assistance asking “How does a tester fit within a prototype team?”  A bunch of people responded, and the insights they provided were very helpful for me, helping me clarify thoughts in my head and introduce new ones.

Proactive Testing

Albert Gareev highlighted the need for prospective testing skills.  Prospective being something I refer to as proactive testing.  Certainly, proactive/prospective testing is essential here!  But to what extent?

Feeding back on requirements

As testers a key role we play is gathering information, assessing it and providing feedback on it.  For a prototype team we will have some form of requirement, or even a vision if you like?  So thinking proactively around the testing we could do for a requirement, we could review whatever information we currently have and gather any addition information that we need, review this and provide feedback on it.  Or for a vision we could do exactly the same but obviously put more emphasis on information gathering and collaboration (questioning stakeholders) in order to provide our feedback.

So now as a result the feedback, or feedback cycles we have provided will have helped stabilise scope, reduce risk, and most fundamentally assisted in determining exactly what the end goal of the prototype will be.

Collaboration and determining business expectations

Is that as far as we can go proactively?  Of course not!  Prototyping will evolve over time.  A good approach to prototyping will involve lots of interaction with it’s stakeholders to make sure essentially that the prototype meets their expectations, or at least indicates early when expectations are not achievable.

An approach we can take as testers here is to begin thinking of the difficult questions a stakeholder might ask of a prototype, and asking them early!  From doing so we reduce the risk of developers being embarrassed and the stakeholder being upset.

Our teams are being asked to produce something demonstrable, which can be picked up and demoed to customers by our demo team.  As such we already have an expected outcome placed upon us from the start.  So we could begin asking difficult questions around the business expectations of the prototypes such as:

  • “Are you happy to invest X amount of time in throwaway but demonstrable code?”
  • Do you expect anything to be reusable from this prototype?
  • Are you happy to invest knowing that what is being asked might not be achievable?
  • If you do expect some re-use what are the non functional requirements of this prototype.
  • Are you willing to fully commit to making these a success via frequent collaboration?

Feeding back on visuals

Rob Lambert and Peter Haworth-Langford stated that testers could play key roles in providing input to designs, process flows and UI.

I agree, and although some might question if a testers role is to input on design decisions at all, I think it can be essential at times.  We might make suggestions for improvement here around usability issues we’ve identified, or realise that a piece of design or in fact the overall design itself doesn’t actually provide any business benefit.

Process flows can be mapped out early from requirements if we have wire frames.  If we don’t we can communicate with the stakeholders to determine the process flow of the proposed prototype.  From assessing these flows we can identify usability concerns early, for example could the flow to point X be quicker if we combined points C and D?

UI, of course there is a lot we could feed back on early here.  We could even check if our competition is doing something similar?  What makes their paradigms special?  How do we make ours better?

Preventing rejected systems

The following two I liked a lot:

  • Anders Dinsen stated “The tester understands the objectives, but questions the means to get there because he sees the problems.”
  • Jamie Cuthbertson stated that a tester’s role should be to challenge assumptions, viewing things from a different perspective and making suggestions for improvement to the product.

Anders highlights the need for a tester to think of the bigger picture.  It’s a difficult job that in itself, and for a prototype we’d need to fully understand what the business expectations of the prototype are first before we can begin to see if we are on the correct path.

Jamie correctly highlights the need to challenge assumptions.  I’d like to add to this “assumptions that matter”.  Certainly for a prototype we will be looking at things from a different perspective.  I’m still unsure myself what perspective we’ll take, but for now I see myself and my colleague Stuart being focused on making sure that the end delivery satisfies business needs as opposed to individuals visions, or worse still conflicting visions.  After all it’s easy for a prototype to free-fall into something which sounds and looks pretty but delivers no business value as it doesn’t actually solve a problem, or one that’s worth solving.  Peter Haworth-Langford highlighted on this when he said that we should be making sure this is actually something someone wants.

Peter Haworth-Langford also added that a tester’s role might be to help a customer or stakeholder formulate their concept.  I think that’s a very good point.  We could collaborate with them to generate scenarios, persona’s and solid use cases around their proposals.  This in turn would help identify if their was a solid business case for the prototype after all.

Functional test involvement?

Basim Baassiri highlighted that testers could provide feedback by doing exploratory tests.  Stephen Hill said something similar when he stated that they could look for areas where the prototype might break.  I agree to an extent here, and if the prototype is being demonstrated as its being developed we could provide assistance here to avoid embarrassment during the demonstrations.  Of course exploratory testing doesn’t just stop at functional testing, from providing feedback to the requirements we will be exploratory testing as well.

Alternative prototyping

Prototype ipad made of foam from 2001

Markus Gärtner asked a series of questions to help him understand the definition of prototyping in my company.  In one question I responded that traditional functional testing would be of less importance here, to which he asked “So, how would you test a teleporter like the one from NCC-1701 D?”  To which I replied that it depends on what this prototype teleporter was required to do.  As a prototype it might not even need to teleport at all.

A famous modern example of prototyping without actually having a working model comes from none other than Apple, who had created several initial prototypes of a proposed iPod design out of foam.  Drawing the controls on the front they used these foam models as if they were actual real working mp3 players for some time.  Walking around with them and pretending to interact with the controls.  This allowed them to understand if the actual look and feel could be appealing, prior to investing in a working model.

So what is the role of a tester in a prototyping team?

To get more of an understanding of the role I’ll be playing I’ve generated a mind map below.

Mind map showing the role of a tester in a prototyping team. Click to view

This image has been cropped, please select it to see it in an expanded size.

Suggested approach

So the Twitter testing community provided me with a lot of help in identifying what a testers role should be in a prototyping team.  As the above is just a collection of my thoughts on peoples responses I’d like to see if I can model an approach of some kind, and hopefully you can help me some more by providing feedback on that.

Step 1:  Clarify business expectations

We need to quickly identify what the expectations on a prototype are from the business.  Hopefully this will have been done already, but if not we’ll have to input these into the chain of command.  Some example questions here might be:

  • What deliverables are you expecting during or at the end of the prototyping phase?
  • Are you expecting re-useable code?
  • If something is not realistically achievable in the given constrains how will you react?

Step 2:  Feedback on requirements or gather an understanding of visions.

Obviously as information gatherers we will identify and feedback on potential issues, gaps and risks with the information we have.  Most importantly we’ll be in constant communication with the key parties involved in this prototype, keeping an eye out for conflicting information.

Step 3:  Validate designs

Often a prototype can have little or no requirements.  We can assist both the developers and the stakeholders here by helping (in collaboration with the stakeholder) produce real life scenarios of how a customer would use the proposed prototype.  We might produce persona’s as well, to highlight the different kinds of user of a system, and in doing so validate our paradigms against these, and uncover potential usability issues.

Step 4:  Determine functional test involvement if any

We might be able to assist with functional testing of a prototype prior to stakeholder demonstrations.  There is a high possibility that our input here might not be required at all.

Step 5:  Ask the difficult questions early

This can be two fold.  We can assist developers by putting ourselves into a stakeholders / business mindset and thinking of possible questions they might ask of this prototype.  If the prototype has frequent progress demonstrations (and it should have) to stakeholders, we can ask the difficult questions of them also.  For example the prototype might be at risk because a clear decision has not been on a paradigm to use, we can ask why that decision has not been made and when it will be made highlighting that the project is at risk by the decision delays, and clarifying that their expectations will not be met if this progresses.

Step 6:  Track expectations

As a prototype might have one or more stakeholders, combined with other people wanting to input their ideas, we’ll need to do our best to keep track of the initial expectations, and make sure if these change that everyone is aware of it and in agreement.

Step 7: Be aware that business expectations may change

Even the best laid out plans for a prototype may become obsolete overnight if the expectations of the business change.  They might like the concept so much that they decide it should be part of the product, and ask that you begin building it in production standard code, which comes with a large test effort in itself.  Likewise the concept might be sold to a customer and require all current prototyping efforts to be dropped to work on developing this sold concept for the customer.

Thanks for reading.

Related posts:

  1. Dice game: Key life skills for testers