It’s a lot like putting on a play. And, like a play, when the people start showing up (observers and users, in our case), there’s absolutely no backing out.
(Even when I’ve actually had my project team pull out [because of major switches in strategy, prototypes that just won’t work], I’ve still been able to get something out of it. Typically, I’ll turn it into an interview, or maybe a test of what some competitors offer, or maybe a test of something totally different. But with all the users recruited and paid for, you really need to do something.)
So, with all this complexity, it’s inevitable that something will go wrong. I don’t think I’ve ever had a prototype that hasn’t had some glitch in it. Heck, I’ve even had production systems go down on me. As for users, there will always be no-shows, or poor recruits, or late arrivals, or the ones who just won’t talk. On the technology side, cameras sometimes don’t work, feeds go black, and recording software crashes. And all that’s not even taking into consideration user error – i.e., the poor facilitator who’s trying to do a million things at once.
The important thing to realize, though, is that every test is going to have some issue. At the same time, however, you will still get tons of great data. Now, some of that data might have to be thrown out, or some of it might have to be taken with a grain of salt, but it is truly amazing how much even the most imperfect of tests will give you.
The real challenge often is getting your observers to understand all this. And, sometimes, that starts right off the bat. One thing that I like to tell them is that “I’ll test whatever you can get me” and that “I can guarantee we’ll get something out of it.” Overall, though, my goal is to get them to relax, let the test happen, and concentrate on the results.
Laura is the president of the wonderfully named Users Know, as well as the author of UX for Lean Startups and Build Better Products
No comments:
Post a Comment