Monday, May 5, 2014

Not everything that counts can be counted, and not everything that can be counted counts. (Albert Einstein)

I once put this quote on a report I put together for a client who was really into counting.  It may not have been the best idea, but it did get their attention.  And several years later, it looks like they may actually have found a place at their table for qualitative research.

I use a couple of arguments when I have to stand up for the legitimacy of qualitative research with more numeric types (those who are really into surveys or analytics, for example). First, I simply point out that we’re both simply creating some sort of feedback loop. The one real advantage of usability testing is that we can get that feedback before something launches.

At this point, I often have to make a distinction between usability testing and QA. What I typically stress here is that usability testing can happen at any point in the cycle (from pieces of paper to production systems) and that it has a much broader focus (not just, “is it broken?”).

Next, I usually point to qualitative work that they may be familiar with. If they’re marketing types, that usually means focus groups. (Though usability engineers may have trouble with these, marketeers typically do appreciate this method.)

I then make the point that there is often a real trade-off between numbers and richness of data. Methods that emphasize the former (web analytics, say) tend to give you a real good feel for what’s happening, but often don’t tell you why it’s happening.

Finally, I like to point to the famous graph that Jakob Nielsen (with help from Bob Virzi) came up with:


I also typically mention – in an offhand way – that this is a perfect example of an asymptotic curve (and ask them where they think the asymptote would be). It’s usually at this point, when they’re so bowled over with my brilliance, that I can get them to agree to anything. (That’s a joke, by the way.)


Einstein also said:


No comments:

Post a Comment