Being surprised is a strategy. (Dani Niro)

In fact, it’s the whole point behind user testing.

For some reason, though, most design teams go into testing thinking they know exactly what will happen. And what that typically is is that there will be no issues at all and that the users will love everything unreservedly.

I, on the other hand, know there will always be something no one accounted for. And that’s because I’ve been in the business for 35 years and have interacted with over 4,000 users. It’s also something I actually always look forward to. I figure, if nothing else, I did manage to learn something new that day.

Why the different perspective? Well, one simply has to do with our job roles. For me, my job is to find issues, so they can be fixed up before release. If there aren’t any, great! Realistically speaking, though, I’m not sure that that’s ever happened in those 35 years. I figure even Michelangelo’s David has got a little crack in it somewhere.

For the design team, they need to build things to spec and have various groups – management, legal, compliance, risk – sign off on it. They typically go into testing thinking that they’ve done a great job, both from their own perspective and from the perspective of all those different approvers.

In other words, I look for "bad stuff." They try to avoid "bad stuff" at all costs.

Depending on how truly user-centered a company is, the feedback from actual users that comes out of a usability test can come as a nasty – or a pleasant – surprise. Nasty because the team was going into it with the totally wrong attitude. Pleasant because they went into testing seeing it as an opportunity to improve their product, as well as their own skills … but also to still feel some validation when things do test well.

BTW, this strategy need not be limited to user testing. Any interaction with real users – focus groups, ethnography, card sorts, surveys – can provide a different perspective, put a check on errant assumptions, and introduce a little humility into the process.

Dani is a VP & head of Consumer Lending Delivery at Citizens Bank

The user experience is not siloed. (Anonymous)

 Over my 35-year career, I’ve worked at several small companies. There, I’ve typically worn many hats – tech writer, instructional designer, usability specialist … And that’s for pretty much anything that came through the door – banking, health, transportation, sports, food …

I’ve also worked at a larger company where I wore the same hats, but for only one of their many products. Finally, I’ve also worked at a large company where I was a user researcher on a single product.

That last one is the worst. For one, I love variety. Now, it turns out I did do a lotta pinch-hitting, so it wasn’t really all that bad.

What I did not like, though, was that the user experience itself was so siloed. For example, the storefront page was its own little group, as was applying for an account, and then servicing the account.

Needless to say, users simply didn’t think that way. For example, take when I tested those online account applications. I really couldn’t just throw the user into that flow.

First, they really had to have some idea of what they were applying for. (Believe me, I tried it. All I got, though, was just endless questions about that product.)

So, I typically started them on the page that describes that particular product. Even then, interestingly, I might have users who wanted to know a little bit about the company, so I might typically just let them at that.

Finally, after the account application was done, we typically threw them into our authenticated space, so they could check out what they just signed up for. Now, there might not be any more there than the funds they just transferred in, but the users sure appreciated being able to play around and check things out. In other words, it was a complete, wholistic, realistic experience for them.

Now, if I wrote my reports or shared my results according to the existing internal silos, I’d have something for:

• Storefront
• Corporate 
• Account application
• Authenticated

Often, unfortunately, I’d only have time to write up the thing the team was interested in (in this case, the application process). There was so much more to share though.

Sometimes, I would have some time to write up some of that other stuff. I couldn’t, however, always get the team to receive it with open arms. I guess it just didn’t fit on a roadmap, count as a strategic initiative or … maybe it simply just wasn’t someone’s yob. Sigh ...

Anything seems possible if you don't know what you're talking about. (Anonymous)

“So, we need to slice and dice those users by age, household income, geography, gender, eye color, and Zodiac sign. And can we get 50 of each? And they’ll need to do all 63 tasks in that email I sent you 10 minutes ago. Oh, and can we get a full report by tomorrow?”

You know, sometimes it’s best if the user researcher leads the research effort. I don’t know. Just an idea. Might work.

I must admit, this only seems to come from folks who are brand new to usability. You know, the people who think usability testing is QA, or a focus group, or a marketing survey, or a combination of all three, or whatever.

So, education is definitely key in these situations. How to go about that though?

I’m always tempted to just say I’ve been doing this for 35 years, stopped counting at 4,000 users, and actually know what I’m doing. So, just shut up and let me do my job!

Now, that would be fun. There are, though, a couple of things that are a little more realistic that I’ve found particularly useful over the years. 

For one, if there are some unfamiliar names and faces in the meeting, is simply to ask if everyone’s familiar with usability research. If not, I can head a lot of this sort of thing off at the pass by giving a nice, simple, accurate definition. 

Another is to make sure I’m not alone. In other words, ensure that there are some people at the meeting who do know what a usability test is. Encourage them to pipe in. If they’re higher up the food chain than you, have them preface the meeting with a few words. In other words, feel free to gang up on your troublemakers. There is strength in numbers. 

And if it really comes down to it, my trump card is to ask, in the politest way possible of course, how they would feel if I were to create their wire frames for them, or write their content deck, or make their GANTT chart. We’re all professionals here, right? 

Don’t fall in love with your work – fall in love with your users. (Me)

At least I think that that quote’s mine. A quick Google search did leave me with something similar from Dana Chisnell: “Want your users to fall in love with your designs? Fall in love with your users.” Great minds think alike, I guess.

Now, my question is, Why does this even need to be pointed out? The whole idea of being user-centric just seems so very basic.

As an official UX old fart, maybe I can give a little insight into this. Interestingly, it actually wasn’t always that way.

One change I’ve seen over the years is the rise of digital creatives. Back in the day, there simply weren’t that many of them around. And those that were there were often stuck doing things like writing manuals or designing the packaging that your fancy new software came in.

One thing I’ve noticed about today’s digital creatives is that they don’t always have their beginnings in UX. Now, that’s nothing new in this field. I myself have a background in English. Other researchers I know come from physics, education, philosophy, linguistics, journalism, even theatre …

A very common background for digital creatives these days seems to be from working at ad agencies. In that particular context, users are definitely part of the equation. Often, though, that focus is a little limited. Will someone open up this email? Will they pay attention to that commercial? Will anybody bother to pick up and read this brochure?

More important, though, is the client. As long as they’re happy, you’re golden … and pretty much ready to move on to your next client.  

Now, in house, marketing is definitely tracked. That’s always, though, been kind of a once-it’s-out-there kind of thing. It’s not really something that’s done beforehand, which is what usability testing is all about.

Up to that point, though, decisions seem to be very intuitive. Yes, what you’ve come up with will get batted around in a crit or some other meeting. It’s all still, though, very much at a gut level. 

Another issue is that agency experience focuses very much on selling. Now, there’s definitely a need for that on every website. There are, though, much more practical things to worry about – can I pay my bill, order tickets, make an appointment, find some particular piece of information …

And, here, the emphasis is less on how clever or appealing your design or words might be, but more on whether it works,  whether the user can get their task done. And a good way to do that is to focus less on yourself and your work and more on who whatever you’re designing is meant for. 

Happy Valentine’s Day!

Every time I hear an executive demand that someone "produce the next iPod," I want to respond, "Can you make yourself the next Steve Jobs?" (Carl Turner)

To be honest, all I really want the people at the “top of the house” (I hate the business jargon, but you know what I mean) to just leave me alone.   

And, over the years, I’ve been pretty lucky. There was only one time where I really had to deal with meddlesome execs.  Boy, though, was it a doozy.

What I was dealing with was a C-suite exec who wanted to incorporate some social media trope on a financial services site. I won’t tell you what that was, as it might give the game away. So, let’s just call them “awards.”

I think the idea was to gamify the boring task of handling one’s money. Now, there’s nothing wrong with gamification. It just needs to be done in the right context. Social media is a great example of that context, as are things like self-improvement, learning, and so on. Finances, however, can be a little dicey. 

Looking at an old report, I see that my users agreed with me: “You know, like I don't really expect to have to a earn an award. You know, this isn't a game – this is my bank account.”

They also pointed out how task-oriented they typically are on a finance site: “I’m here for one real purpose.  This is just extra noise that I don’t need when I’m really just trying to do some online banking.”

They also mentioned that they wanted to get something out of it:

• “It doesn’t look like I gain anything from them, which is another reason why they’re not super interesting to me.  If it was something like, once you’ve been a member for two years, you’ll get a higher interest rate, then I might be interested in the badges.”
• “So, unless it's gonna be pertinent to things that are happening to me that are affecting my bank account, I probably wouldn't wanna see them over there.”

Finally, users felt a little talked down to: “It’s like they're trying to gamify saving, which I feel like is kind of silly, and like they're treating me kind of like a child.”

Did they pay attention to my results? Of course not. It launched with few if any changes.

What’s interesting is that, once it was launched, the feedback was about the same. I lost track of what happened from there, I’m afraid.

Here’s hoping, though, that the exec got some understanding that that wasn’t such a brilliant idea after all, and that he should leave that sort of stuff to the professionals who he’s hired and pays good money to. Heck, I could have let him know both of those things right off the bat, and saved the company a lot of time and money.

Carl’s LinkedIn page lists him as a “chaos wrangler,” though I know him better as a usability engineer

Research that isn’t shared is research that hasn’t been done. (Lindsey Redinger)

It’s so tempting to just throw a research report over the transom. I’ve usually got another project in the works (if I’m not already juggling it), and it’s so much easier to not deal with politics and difficult personalities.  I just want to say, “Here’s the results,” and move on.  Prep, facilitation, and analysis are a lot more fun, and much more basic to what I do.

I figure the least I can do, though, is hold a meeting to go over the results.  I figure that it’s at least important to field questions, to make sure everyone understands what you’re trying to get across, to make sure there are no outstanding issues.

But, you know, for that to go over well, it really does help to get the team involved before that point.  And that means observers, and debriefs, and topline reports, and sharing tapes, and updates in team meetings.  Heck, it all really starts with good intake and planning meetings.

But it’s not all about the stuff that happens up to the report either. What happens after is just as important.  I’ve found that if you really want to have an impact, and actually have someone address the issues that came up in your research, there is no shortage of follow up you have to do.  And that means design meetings, and prioritization meetings, and technical meetings, and emails, and Slack messages, and hallway discussions.  

Writing a good report is kind of like getting a fish on the line.  That’s nice, but you’ve also got to hook him, land him, filet him, and cook him.  Now, I’m not sure how a fishing metaphor got in here but, heck, I’ll take it.

Maybe a better way to look at it is through writing fiction (something my wife does, and where that expression "throwing it over the transom" comes from).  It’s not enough just to send it to a publisher.  You may have to snag an agent first.  And to snag an agent, you’re going to have to do some schmoozing.  These days, you may even have to print, sell, and market it yourself.  I’m not sure an Emily Dickinson would have much luck in 2022.

To get back to UX research though …  One thing I’ve found over the years is that research ≠ report.  Honestly, sometimes it seems that once you’ve put your final touches on that great report of yours, your job has only really just begun.

Lindsey is head of research at Etsy

Love means never having to say you’re sorry. (Love Story)

Whuh? Huh? What the heck does that have to do with usability?

Oh, also, it’s a terrible quote. My wife and I have been married for almost  30 years. We say “sorry” quite a bit (and “thank you” as well).

Now, if this means you’re automatically forgiven, I guess that’s okay. They could have been a little more explicit though. Can you tell I'm an engineer?

I digress … How exactly does this apply to user research? For me, it kind of reminds me of my users. I’m sure we’ve all experienced – and probably been frustrated by – test participants who say things like, “I’m not good at computers” or “I shoulda got that” or “Oh, that’s my fault.”

Heck, the type even made it onto my typology of users. They’re called the “Charlie Brown” type. Note to millennials … Charlie Brown was the main character in the Peanuts cartoon. He was a sad sack character famous for his bad luck, morose disposition & for blaming himself when things go wrong.

I approach this user in several, graduated ways. If it’s just once or twice, I just let it go. If it comes up again, I usually give them a little encouragement – you know, “You’re doing just fine,” “This is good feedback,” “This is a test of the system, not of you” … 

If it’s still persistent (and these users can be persistent) and really starting to get in the way, I usually go into full pep talk mode. And that’s something along the lines of, “You can do no wrong here today. If there’s an issue, it’s an issue with the [site / app / software]. And I want to know about it.” I might also mention that they are the perfect user for this system, and if they can’t use it, other people won’t be able to either.

When it comes down to it, I really do love my users. All I want to do is make them feel comfortable sharing their thoughts – and never having to say they’re sorry for anything.