Show, don't tell (Percy Lubbock)

I’m intimately familiar with this one primarily because my wife is a novelist. It’s one of the most basic ideas in fiction. A good example might be revealing a character’s true nature by an act or a quote of theirs. Which certainly beats simply saying the character is mean, or brave, or witty, or whatever.

Now, how could that possibly apply to user research? Isn’t a good usability report all about "just the facts"? And though I do work a lot with personas, aren’t they something very different from characters in novels?

Maybe a better way to say this might be, “Do the math.” It’s usually not enough to simply say that screen X didn’t work or task A failed. If you do nothing else, you have to at least offer some detail. There’s a reason something failed or didn’t work, right? Now, what was that exactly?

If your client can get their head around those reasons, that might be all that’s required. So, that button was hidden, that path was pretty convoluted, that copy was really hard to read. I get it.

It helps too, though, if you have some evidence. First, you can just throw some numbers in there. Even though you’re probably talking about the small numbers that come with qualitative studies, it is helpful to know that everybody had that problem, or most did, or over half …

It’s even better, though, when you can share quotes and clips. Those can really bring home the idea that these are real users having real problems. It’s not just my take on it. I’m not just editorializing here. Quotes and clips really do speak for themselves.

So, you’re probably now asking yourself who the heck was Percy Lubbock. Well, the interwebs are telling me that he was, in fact, a British literary critic (and a CBE to boot!). His The Craft of Fiction popularized this adage way back in the 1920’s.

It’s an idea, though, that’s been expressed by a number of different writers:

• Mark Swan (American playwright and scriptwriter): "Show–not tell”
• Anton Chekhov: “Don't tell me the moon is shining; show me the glint of light on broken glass.”
• Ernest Hemingway:  “Show the readers everything, tell them nothing.”

My favorite writer after Shakespeare 
(and my wife, of course)

If you want information, feign ignorance. (unknown)

Ever researched something that you don’t yourself understand? No matter how hard you try?

Though most of my work has been with straight-up, average consumers, I do get the occasional something way out there. I also know that there are researchers who deal with this every day (poor things). 

Since researchers tend to come from all over the place, this could mean dealing with medical devices if your degree is in psychology, accounting if your background is comp sci, nuclear engineering if you have a BA in English …

For me, ex tech writer that I am, it’s been a challenge on almost everything outside of that nice, cozy (mostly financial) consumer space. Personally, I’ve worked on stuff that might normally require an MBA, or 5 years in the insurance business, or a degree in statistics …

How to deal with it? I think the first thing is to admit to yourself that you won’t understand it all. I know that can be tempting, but I think it’s better to concentrate instead on your knowledge of research. Let that be your SME superpower. 

Second, don’t try to cover up your ignorance. That can backfire, resulting in misunderstandings, embarrassment, and bad research results. 

Ironically, it actually takes a fair amount of self-confidence to admit you don’t know anything. (In general, humility is another wonderful superpower to have.)

Finally, don’t beat around the bush. Just come right out and fess up. It’s really no biggie. In fact, it really helps in making it clear where everyone stands, getting you the info you ultimately need, and making the SME feel like a big-shot. 

But, if you’re still worried about how you’ll come across, here’s another anonymous quote I found on the Interwebs: “Ignorance is lack of knowledge, not intelligence.” Indeed, you have to be pretty darn smart to be able to talk with the experts effectively. Just make sure you’re taking the right tack right from the get-go.

Where that last metaphor comes from (and, no, I don’t understand it at all)

Method shouldn’t dictate results. (me)

I looked all around for a better quote than this, perhaps from someone a little higher up in the UX firmament than yours truly. Unfortunately, though, you’re stuck with me.

It is one of my all-time favorite topics however. I find it applies to a lot of newer researchers especially, but to more seasoned ones as well, especially if they are short on time or just forget.

What I am talking about here? Well, I guess it’s just a plea for a little more high-level thinking. Let me give you an example…

Say you’re writing up a usability test you just ran. Now, a traditional approach would be just to report things task by task. You know, task 1 had an 87% completion rate, as well as these particular issues. Same deal with task 2, task 3 & all on down the line.

But what about those issues that came up on all – or at least on multiple – tasks? Just mentioning them for each task doesn’t really give the reader an idea of how pervasive – and important – they might be.

Another approach – especially if you have, say, a single task with multiple steps or pages – is to go screen by screen. Screen x had these issues, screen y these, screen z none, yada yada yada. Unfortunately, you’re going to have the same problems with that approach as well. What issues spanned multiple screens? What are some of the larger problems here? 

I guess a lot depends on how you think about your research. If it’s just like a punch list for a new house, say, go for that bottom-up approach. If, however, you’d like to engage with your audience at a higher level – maybe even learn ‘em a few things – you might want to sum things up a little more.

If, for example, terminology is an issue across tasks or screens, simply put in a slide solely on that topic. Yes, you can mention where that appears, but I think it’s better that the team knows this is a larger issue. Heck, they might even want to do a review of all the screens or functions to make sure it’s taken care of everywhere. After that, they may even go so far as to apply that learning to their next project, and the one after that, and the one after that…

I do find that there are topics that seem to go across multiple projects & multiple teams – nav, graphics, accessibility, mental models, TMI, affordances … In fact, you can even start to track some of these and even address them upfront, separate from any particular project. In other words, help your team address root causes, not every little nit.

Same thing applies to surveys, focus groups, and other research methods as well. As an example, I just read a report that separated out quantitative survey results into each individual question, then tackled verbatims separately. How nice it would have been if the researcher had combined those two, identifying some larger themes along the way. That seems to be something that most researchers do naturally on interviews and ethnographies. I’m not sure why that wouldn’t apply all around.

So, in sum, don’t leave your results half-baked. Go that extra step and give them something that’s easy and pleasant to digest. And, whatever you do, don’t just give ‘em a recipe! Go ahead and plop that amazing dish right on their plates in front of them.

Tell me, why does it seem that all my favorite metaphors involve food?

Just do it. I don’t want an experience. (user)

I just had a user offer up this quote on a DMV website. She had just run across a message saying, “Just a moment while your experience is loading.” She was trying to pay a bill, and the site was being a little slow, hence the interesting variation on the standard “loading” message.

The same user was also on Vanguard, trying to make sense of some message that asked her to “migrate off the legacy platform.” “What’re they talking about?” she asked me. “How are you understanding it?” would be my typical response in a situation like this.

Jargon used to be the sole province of techies or industry insiders. You know, the error message that sounded like a leftover from the space program – “Invalid response: abort or retry?” Or perhaps the casual use of terms like “widget,” “RSS,” “cache,” “patch,” “API,” “firewall” or “SSL.” I think we’re all familiar with this sort of thing sneaking into the UI.

Now, ever heard of the terms “deposit account,” “external transfer” “HELOC” or “ACH”? Those are all nuggets I’ve run across in my career in banking. I’m sure each industry has a list of something similar.

Well, UX must be mature, because I’m starting to see jargon from there now as well. “Experience,” “interaction,” “journey,” “hamburger menu,” “onboarding,” “breadcrumbs,” “responsive” … 

Yup, not only do users have to contend with the techies and the industry insiders. They now have to contend with us. The irony is palpable.

Now, you might be wondering what I’m doing testing DMV and Vanguard. Actually, I’m not.

That particular user? To paraphrase an old vaudeville joke, that was no user – that was my wife. Sounds like I’ve got her trained pretty well now to recognize poor design.

And here I always thought that quote came from Groucho

It’s kind of a pain in the button. (anonymous user)

Actually, the user didn’t really say that. What they really said was that something was a “pain in the butt.”

My transcription software, though, rendered it as “pain in the button.” Which, in its own way, is absolutely perfect.

To be honest, I don’t think I could live without transcription. Sure, I could simply watch the tapes. That, though is an extremely linear experience.

Honestly, there’s going to be so much filler – my prep, polite chitchat, reading scenarios, the user’s making their way through the easy part of the task, irrelevant diversions …

But with a transcript, if I’ve got a note about something interesting the user said about a certain topic, I can just search on that topic and go right there. Alternatively, if I’ve captured part of a quote, I can just search on any distinguishing word in that quote.

Now, if the test was unmoderated, I can also scroll through the transcript faster than I can go through tape. And that’s even if I play the tape at 2X (which, honestly, can be a little hard to interpret). Once again, I can just ignore all the filler, and go straight to where there’s something with a little more meat on it.

Now, no transcription software is perfect. So, I do get my fair share of stuff like this, usually with non-native English speakers (poor things):

• “I'm a little bit confusing.”
• “Sorry, I have a little bit, uh, destruction.”
• “So I'm confident that I have the DF finger, right?”
• “So again, uh, notifications for competent for Diva.”
• “APIs are, are variable submitted to change and accurate as okay as female reduced enemy, okay? A bonus plus a great threat.”

That said, I do love my transcripts. 

Oh, and by the way … Transcript or not, I’ve had users comment on pain in butts, asses, keisters, tuchuses & dupas. And for those of you who don’t speak Polish, well, that’s exactly what that last one was. So, don’t say I never taught yez nuthin, ‘kay?

Just think of user researchers as big pillows on the hard, small bicycle seat of terrible UI

Being surprised is a strategy. (Dani Niro)

In fact, it’s the whole point behind user testing.

For some reason, though, most design teams go into testing thinking they know exactly what will happen. And what that typically is is that there will be no issues at all and that the users will love everything unreservedly.

I, on the other hand, know there will always be something no one accounted for. And that’s because I’ve been in the business for 35 years and have interacted with over 4,000 users. It’s also something I actually always look forward to. I figure, if nothing else, I did manage to learn something new that day.

Why the different perspective? Well, one simply has to do with our job roles. For me, my job is to find issues, so they can be fixed up before release. If there aren’t any, great! Realistically speaking, though, I’m not sure that that’s ever happened in those 35 years. I figure even Michelangelo’s David has got a little crack in it somewhere.

For the design team, they need to build things to spec and have various groups – management, legal, compliance, risk – sign off on it. They typically go into testing thinking that they’ve done a great job, both from their own perspective and from the perspective of all those different approvers.

In other words, I look for "bad stuff." They try to avoid "bad stuff" at all costs.

Depending on how truly user-centered a company is, the feedback from actual users that comes out of a usability test can come as a nasty – or a pleasant – surprise. Nasty because the team was going into it with the totally wrong attitude. Pleasant because they went into testing seeing it as an opportunity to improve their product, as well as their own skills … but also to still feel some validation when things do test well.

BTW, this strategy need not be limited to user testing. Any interaction with real users – focus groups, ethnography, card sorts, surveys – can provide a different perspective, put a check on errant assumptions, and introduce a little humility into the process.

Dani is a VP & head of Consumer Lending Delivery at Citizens Bank

The user experience is not siloed. (Anonymous)

 Over my 35-year career, I’ve worked at several small companies. There, I’ve typically worn many hats – tech writer, instructional designer, usability specialist … And that’s for pretty much anything that came through the door – banking, health, transportation, sports, food …

I’ve also worked at a larger company where I wore the same hats, but for only one of their many products. Finally, I’ve also worked at a large company where I was a user researcher on a single product.

That last one is the worst. For one, I love variety. Now, it turns out I did do a lotta pinch-hitting, so it wasn’t really all that bad.

What I did not like, though, was that the user experience itself was so siloed. For example, the storefront page was its own little group, as was applying for an account, and then servicing the account.

Needless to say, users simply didn’t think that way. For example, take when I tested those online account applications. I really couldn’t just throw the user into that flow.

First, they really had to have some idea of what they were applying for. (Believe me, I tried it. All I got, though, was just endless questions about that product.)

So, I typically started them on the page that describes that particular product. Even then, interestingly, I might have users who wanted to know a little bit about the company, so I might typically just let them at that.

Finally, after the account application was done, we typically threw them into our authenticated space, so they could check out what they just signed up for. Now, there might not be any more there than the funds they just transferred in, but the users sure appreciated being able to play around and check things out. In other words, it was a complete, wholistic, realistic experience for them.

Now, if I wrote my reports or shared my results according to the existing internal silos, I’d have something for:

• Storefront
• Corporate 
• Account application
• Authenticated

Often, unfortunately, I’d only have time to write up the thing the team was interested in (in this case, the application process). There was so much more to share though.

Sometimes, I would have some time to write up some of that other stuff. I couldn’t, however, always get the team to receive it with open arms. I guess it just didn’t fit on a roadmap, count as a strategic initiative or … maybe it simply just wasn’t someone’s yob. Sigh ...