The power of Claims Testing

By
Darren McMillan
– December 8, 2010

Some background

Two weeks back Zeger Van Hese refreshed something in my memory which I’d almost forgotten about.  A simple little testing technique, which in itself can be very powerful & often essential in my honest opinion.

So what is it?  Well it’s very simple really, people make claims about software all the time; our job as testers is to confirm, dispute or reject those claims.  You’ve probably done this many a time and not even realised it, I know I have.  Up until now I’ve never noted it down as a technique to add to my testing model.

Now I’ve heard about claims testing three times now in the past three months.  Once in James Whittaker’s book on Exploratory Testing, another time during the Rapid Software Testing training course I’d attended by Michael Bolton when he’d described the FDSFSCURA heuristic mnemonic and finally when I’d participated in the first ever European Midweek Testing session & Zeger Van Hese showed how he’d used it in his testing approach.  Now although I’d valued it as a technique, it wasn’t until I saw Zeger show how powerful it could be in his approach that I thought, yeah I really should start using this all the time.

Identifying claims

So how do you find these claims?

Documentation

Well an excellent place to start is your products documentation.  Really just pick it up and start identifying what claims it makes.  These could be as simple as a compatibility list.   Does it really work in that database claimed that no one has ever tested?  Oh that browser we claim to support hasn’t been tested either, we still use the previous version.  If you’re clever you’ll skip the reading part altogether and sit down and talk with someone on your documentation team who’ll probably be able to point you in the right direction reading wise.

Demo

Most companies have a demo team; go talk to them & see what claims they have or will be making about your new features.  The team I’m currently working in is developing a new key product feature, which hopefully will make our company lots of money.  Our demo team have been sitting anxiously waiting to get their hands on this to start showcasing it in their demos.  They’ll have some claims we’ll need to be weary of, something which I’ve scheduled in to be looked at.

Sales

Likewise your sales team much like your demo team will be making all sorts of claims.  It’s probably best to even just ask how they plan to sell this to customers, from that itself you’ll gain many valuable insights.  If you think of the context of a claim it might be ok for your sales team to claim your feature can do something that it currently can’t.  Why?  Well the feature may be extended for that customer by another team.  If you’d missed that valuable insight your sales team just gave you then let me rephrase.  A sales team’s claims can point to key extensibility scenarios that your feature should be capable of handling.

Proactive claims testing

A common problem in lots of products features is that they’re developed to fulfil a requirement.  Now requirements change over time & that’s when the extensibility of a feature can become a problem.  Sure anything can be extended with some work.  Should we have to invest ridiculous amounts of time to do that though?  If it’s developed in such a way that extensibility is fairly easy, or at least not made difficult!  Then we’ve saved our self’s a lot of time & money from some thoughtful consideration up front.

If we as testers can communicate prior to development of a feature with our sales team, we can find at least some of those extensibility points for free, from a quick chat.  Even better still we can influence design by simply asking if those points will be extendible?  I find myself that even just asking the people working on the feature how they plan to make it extensible helps iron out these problems early at a much lower cost.

You can look upon your sales teams claims as a unique opportunity to do some cheap proactive testing.1

I’ve talked more about this than I’d intended to, so I’ll pass in another place to look for product claims before wrapping this post up :-)

Development

Development claims.  Those do happen right?  Sure they do!  Once I remember my feature team lead at the time being cornered for a statement from our CEO on how much faster a new developer tooling feature would make developing web pages for our deployment teams.  He eventually gave in & said a figure of around 40%.   Yes, a very bold claim!  One that we later had to prove & luckily enough his magic number, which I’m sure he’d debate, did turn out to be around 40% faster :-)  Obviously there was no trickery involved here & this was of course a very valid experiment.

It’s easy to forget the impact a claim from the development team to others, be that your CEO, customer, management and so on, can have.  It can be a struggle to keep track of them, and really you just have to try your best; read all your emails, talk to people on your team, communicate with your customers & stakeholders regularly.

So without going into this too much I think you can see how powerful a technique Claims Testing can be.  Nothing new, however hopefully I’ve provided you with a few insights on how to make good use of it & I hope you’ll provide me with a few as well.  Thanks for reading :-)

  1. A technique to help prevent issues from entering code in the first place..  Proactive meaning you provide methods to test prior to any code being committed into your products code base, as opposed to reactive.  [↩]

Related posts:

  1. Proactive testing: Tips from Michael Bolton
  2. Tales from the trenches: Proactive testing
  3. Tales from the trenches: Lean test case design
  4. Tales from the trenches: Keyword based testing
Tags: , ,