Monday, August 25, 2014

It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them. (Steve Jobs)

Great point about the focus groups, Steve.  I was wondering, though, how you were able to uncover exactly what people wanted.

Focus groups, used properly, are actually one way to do that.  You might, for example, get people to talk about their experience with some task.  That’ll get at the existing systems, whether they’re working or not.  But it’ll also likely get at things that are outside the current systems.  And addressing all that can help you come up with some really innovative ideas.  In other words, ask them what their problems are (whether that’s with the systems they use currently or has absolutely nothing to do with those).  Don’t ask them for solutions.  Coming up with those is your job.

Just to make this a little concrete, I worked on a project once where my team was tasked with completely redesigning the mortgage application process.  As part of that effort, we put together some focus groups with people who had just been through that process – good, bad, indifferent, with our company, with other companies, mostly online, mostly offline …  The pain points came up quickly, easily, and in great volume.  Further, many of these were not being addressed by anyone else out there.  They typically had to do with things like communications, and paperwork, and motivation – and not necessarily with the systems these people worked with.  The solutions we came up with were rather proprietary, so I really shouldn’t share them here.  Suffice it to say, though, they were pretty darn good.  And I’m not sure we would ever have come up with them on our own.

Now, an even better way to generate ideas is through ethnography.  That’s just a fancy way to say “field study,” and all that involves is watching people complete their own tasks in their own environment.  In the session itself, you should concentrate on observing, simply capturing all the reality that happens right in front of your eyes (there’ll be plenty).  Later, you can analyze all that data ‘til the cows come home.  Once again, though, what you’ll be looking for primarily are the pain points, the gaps – along with creative ways to address them.

Does Apple do something like that?  I do know that they love their genius designer paradigm.  So, maybe they have some genius way to get at those user pain points as well.  I also know that they like to design for themselves.  And seeing that iPhones and iPads and so on are meant for pretty much everyone, they can probably get away with that.  You and I though – with our specialized audiences who we may or may not have anything in common with – may not be so lucky.  Finally, I do know that Apple is extremely secretive – heck, even paranoid.  So, we may never actually know.  Anyway, unless you’re one of those Apple geniuses yourself, I would definitely recommend some of that upfront research – whether a focus group, field study, or what have you.

Friday, August 8, 2014

A successful test is a test that uncovers many problems. (Rolf Molich)

I often get asked in the elevator or the break room or the wash room how “that test is going.” Even after all the years, I’m afraid I have to admit that I never know quite what to say.

If it’s someone on the project team, they basically want to know that everything’s hunky-dory and that I have essentially uncovered no problems [yeah, right]. If it’s another usability engineer, I might delightedly share that the system’s a piece of crap, or that my 1:00 appeared to have had a few drinks with lunch, or that that method they suggested is working well. If it’s a developer, they may simply want to know that the prototype worked.

For me, though, what I’m really looking for is consistency. What I really like is when the first three or four or five users all have trouble with widget X, or label Y, or screen Z. Even though we are dealing with pretty small numbers, that consistency makes me feel much more confident.

(Now, there are never any shortage of problems that only one or two users might uncover. What I do in that situation is take a closer look at these issues and see if the user has uncovered something valuable or if they are simply an outlier. That’s always a judgment call, though, unfortunately.)

My ability to tie a problem quickly, directly, and clearly to a specific field or interaction or missing piece of information also makes me feel good inside. Some of the hardest reports to write are those where there was obviously an issue, but what it was and what was causing it weren’t so clear. Of course, it’s always a great feeling when I’ve solved these knotty problems, but there can definitely be a fair amount of anxiety leading up to that moment.

One final thing … Yes, most of my tests do find plenty of problems.  Plenty, though, also uncover many things that work. And some even uncover plenty of good things and very few problems. And I’ve always believed that’s just as important to present as when there’s nothing but issues.

Unlike a lot of usability engineers, I do like to emphasize the positive. I just find my report recipients are a lot more open to what I have to say if it’s not all negative. So, maybe Rolf’s quote should read, “A successful test is a test that uncovers many things,” or “A successful test is a test that uncovers a lot of stuff,” or … Ah, forget it, it’s still a pretty darn good quote just as is.