Friday, February 16, 2018

However beautiful the strategy, you should occasionally look at the results. (Winston Churchill)

And isn’t that what usability testing is all about? 

Of course, there are other ways to get that information as well, especially these days. For one thing, you can simply put it out there and monitor the heck out of it. 

A kind of old-fashioned way to do that is Voice of the Customer – popping up surveys, including “give feedback” links, and so on. These days, you can even monitor social media. It’s all good stuff, but there’s also no shortage of possible drawbacks. I already wrote a post going over those in great detail, so I won’t repeat all that here.

You could also take a look at web analytics. I wrote about that, to some degree, in another post. The main thing with analytics, though, is that, though you might have tons of data showing exactly where users went and what they did as they actually used your system, you totally lack why they did so. And that can be pretty darn important.

It’s the same problem with A/B testing. Once again, you get reams of real data, all tied directly to the bottom line, and in this case, totally actionable. You are, however, relying totally on correlation – which, in my opinion, means relying totally on conjecture. We know that A performed better, but why? How can we tell with just the numbers? What were users thinking when they went with A?

And then there’s usability testing. It’s the prime way to get feedback before release, but it also provides super-rich data that can be used to completely understand your users, identify and understand their issues, and then come up with legitimate solutions to address those issues. 

That said, if you are getting some kind of feedback, you are way ahead in the game. What really worries me are those companies out there who basically don’t do any of this.

By the way, when Churchill was talking about strategy and results he was actually not talking about usability testing (I know, hard to believe). I would imagine he was talking about something like war plans, or economic strategy, or political campaigns. Unfortunately, we may never know. There is actually no evidence that Churchill ever said that (and I’m not sure who did). But it certainly does sound like something he would have said.


Bet you had no idea that the Daleks were a secret weapon during WWII
(or that Churchill looked anything like that)

Thursday, February 15, 2018

Mobile is a magnifying glass for your usability problems. (Josh Brewer)

There are designers and developers and UX researchers out there whose whole world seems to be totally encompassed within the mobile space. Now, that is definitely a fact of life these days. At the same time, though, that’s also a tad unfortunate as well.

For some of these folks, mobile is a brave new world, where they are intrepid explorers creating their own rules. Now, mobile definitely is terra nova. It is not, however, terra incognita.

For someone who’s been doing this as long as I have, mobile is really just the latest whatever. All the things that we learned about green screens, CLIs, GUIs, the Internet … they pretty much all still apply. Yes, there are new things like gestures, but affordances, mental models, proximity, hierarchy, information density, legibility … all that great stuff has not somehow magically disappeared overnight.

Reinventing the wheel is not the point with mobile. In fact, designers need to think, not about being innovative so much, but more about handling the old principles within the additional constraints that mobile involves. 

And what are those? Well, the main one by far is simply the huge difference in screen size. And what does that mean relative to UX? More than anything, I think it takes the old KISS formula and makes it absolutely paramount. Complexity that you might have been able to get away with on a nice big screen is simply going to blow up in your face on mobile.

Less central – but certainly not unimportant – issues include awkward input, new environmental contexts, and several others. There’s also one thing I’ve noticed which probably doesn’t need to be an issue at all. For some reason, some designers took the new context of mobile and decided to just ditch the idea of affordances altogether. 

What do I mean by that? I just seem to see missing pieces on every mobile test I run – or as simply a user myself. In fact, just this morning, I was testing out a prototype for a test I’m running next week on an iPhone …

First, though, I had to clean the screen up a bit, and get rid of some old icons on the home screen. Now, how to do that? Why press and hold those icons, of course (I somehow remembered that from the last time I had to test an iPhone). That puts you into delete mode, where the icons wiggle and have litte x’s in their top-left corners. Okay, this I can figure out – an x is a pretty darn clear affordance. But what do I do to return to normal mode? I would imagine I would press a wiggly icon again. Nope, you press the Home button (had to really think about that one).

Now I can type in the URL of my new prototype. I do that, bring it up, then wonder how I can put an icon for it on my home screen. Oh, it’s the little box icon with an arrow in it (had to ask an iPhone user for that). What do I click now (interestingly, the iPhone user couldn’t help me there). Oh, that little bar of icons on the bottom – looks like I can slide it over (had to ask an iPhone expert for that one). Now, why didn’t they signal that somehow? You know, like with a >, or a little bit of the next icon, or practically anything? 

And all this is before I even get to start in on that prototype. And after that, I then get to try it all over again on an Android, which I least am familiar with, but which also has its own way of doing things, and its own set of affordances to ignore as well. Ugh, it’s gonna be a long day …


Josh was once the principle designer at Twitter

Tuesday, January 9, 2018

Our inventions are wont to be pretty toys, which distract our attention from serious things. (Henry David Thoreau)

Over the past few years, I’ve noticed something rather disturbing.  I noticed it with my 20-something sons, my wife, and even at times myself. 

People seem to be addicted to their phones and computers. I don’t mean this in a metaphorical way, but very literally.

I actually followed up these observations with my own, very informal field research. Out to get a bite to eat for lunch on the streets of my mid-size metropolis, I counted the number of single pedestrians (people tend not to do this in crowds) who were busy staring at their phones. I did this several times, and don’t recall ever getting a number under 50%. And this is people walking, on crowded city streets – and generally fairly oblivious to other walkers, cars, signals, or anything other than their phones. 

With the exception of a few (mostly older) friends, however, I’ve mostly kept these thoughts to myself . I mean, I’m in high tech. I work for an online-only bank. I do usability tests all the time on mobile phones. The people I work with are all just a little older than my kids (and are really into their mobile devices). Ergo, the last thing I’d like to be known as is a Luddite.

Recently, though, I see the tide turning, just a little. This morning, I read an article about two investors threatening to sue Apple because they made “addictive devices.”. And, just a few weeks before, former Apple exec Tony Faddell got in the news bemoaning his role in making what is basically a tech version of casinos or cigarettes or opioids.

There’s also no shortage of books out there on the subject as well. You remember those things, don’t you? It used to be that these were few and far between. Lately, though, it seems like that’s all I’ve been reading. Just looking at my reading list, I’ve got Think Before You Like, The Know-It-Alls, The Hacking of the American Mind, iDisorder, iGen, Technically Wrong … So, at least I feel like I’m not alone anymore. 

But if you think about it, this is probably a necessary, and overdue, correction. There’s really nothing surprising here. Basically, there is no shortage of examples of society rushing to adopt technical wonders – especially those that might be highly financially remunerative – focusing solely on the positives, and without really thinking about any other possible implications. 

So, what I’m saying is not that smartphones are bad necessarily, but that we really do need to realize that they might come with some costs, and that we need to be super thoughtful about how we use them.


Wednesday, December 13, 2017

If you have no critics, you’ll likely have no success. (Malcolm X)

I never thought I’d be quoting Malcolm X in a blog about usability. In fact, I originally had no idea I was quoting him. I had found this quote in a fortune cookie, of all places. It was only when I Googled the quote that I found that it was from the father of Black radicalism.

It is, however, a quote that can be applied to many different fields. Within usability, I see it applying in a number of situations.

Most obviously, this sounds like something I’d share with my project teams. I’ve found, over the years, that good designers can’t wait to get real user feedback. They tend to have thicker skins, and can roll with the punches. And I like to point out, and congratulate them, on their ability to do so. 

Needless to say, we can also turn the tables on ourselves. Once again, I’ve found that the better usability engineers are the ones who are always learning things and looking for a better way to do their jobs. They tend to practice what they preach, and let humility be their guiding principle. But this probably just comes from being a social scientist. I’ve found that, in every test I run, I learn something new – whether that’s about computers, people, or myself.

Finally, I think this maxim applies to acceptance of usability in general. Having been in this profession for 30 years allows me to take the long, historical view. In particular, I remember way back when the resistance came from the techies (a time which Alan Cooper so brilliantly captured in the title of his book The Inmates Are Running the Asylum). 

Next, it seemed to come from graphic designers. Though they did us a service by adding more pleasure and delight to the user experience, they tended to divorce those considerations from more practical considerations like usability and company profit. Jakob Nielsen, for some reason, seemed to be the brunt of a lot of their disapproval.

Lately, it seems resistance is coming from the business side. In particular, I worry that all a site is these days is a sales or purchase funnel, where users are hustled along without any time to explore or ask questions – to me, at least, the online equivalent of a used car lot. Business does have the numbers on their side – with analytics, big data, and A/B testing – but do I worry that they sometimes may be missing the forest for the trees.

And then there's Agile ... Not only does this seem like the return of the techies, but it seems like, this time, they've teamed up with the business side against us.

Ah well, it’s always something, or somebody, isn't it? I actually think the dialectical nature of all this is good for usability. It shows that we can adapt, incorporate other viewpoints, and even act as a mediator sometimes.

 



And you thought I was making that up

Thursday, November 9, 2017

Web users ultimately want to get at data quickly and easily. They don’t care as much about attractive sites and pretty design. (Tim Berners-Lee)

Wow! I never thought I’d be disagreeing with Sir Tim Berners-Lee. But there you go …

Now, there was a time when I would have heartily agreed with him. And my guess is that this quote is probably from a long time ago as well.

Yup, it’s hard to believe, but there was a time when information was paramount. The Internet was simply a place where you went for data, plain and simple. I think the idea was that having all the information in the world at your fingertips was enough. Anything that could possibly get in the way – and that included design and aesthetics – did indeed get in the way, and really shouldn’t be there.

(You can still see that approach in sites like Craig’s List, the Drudge Report, and even Jakob Nielsen’s Alertbox. In fact, Jakob just put out an article on what he calls “brutalist” web design – borrowing the term from architecture – discussing this very style.)

Fortunately or unfortunately, something called eCommerce then occurred. Yup, companies started using the Internet to sell things. (It wasn’t just random people and organizations dumping data there anymore.) Further, those companies wanted to use all the methods at their disposal to convince you to buy their stuff. 

At the same time, web coders started introducing all sort of techniques that could make websites less just pure HTML and more along the lines of print or TV or whatever the client desired. Finally, there were also studies done that showed that users preferred more aesthetically pleasing designs. In fact, comparative testing of the same site in a “brutalist” style and a more aesthetically pleasing one actually made users think that the more aesthetically pleasing ones were more usable as well.

The age of the graphic designer was at hand (and, honestly, we’ve never really looked back). Unfortunately, some of those graphic designers got a little carried away. In fact, given free rein, these folks went a little over board, and started making sites that were, not only attractive, but “different,” “fresh,” and even “challenging” as well. In other words, the design had ceased to help the user and business achieve their goals, but interfered with those, becoming something of an end in itself.

Now, here’s the thing …Usability and aesthetics need not be in conflict. In fact, a really great design will have them working hand in hand, seamlessly, to help the user and the business (if not necessarily the graphic designer) meet their respective goals as efficiently and effectively as possible.


Hard to believe, but I think that screen was Photoshopped!

Monday, October 9, 2017

The most valuable question could be the one that’s not asked. (Anonymous)

And that’s why I’m not so crazy about surveys.  Sure, you can include an Other field and, yes, you can test your survey out beforehand.  Heck, you can even do some qualitative research (e.g., a focus group) upfront to help guide the creation of your survey.  But there will still be plenty of things you miss, par example:
  • Respondents may not be able to express their issues in words.  Believe me, “This sucks!” and “I hate this!”  are not going to provide you with a lot of actionable data.
  • They may also have gotten so used to work-arounds that they might not think to ever bring up the issue the work-around is for.  Along those lines, they may have also no idea that something could even be better.  
  • More importantly, they may simply fall prey to the limits of human memory, not remembering, for example, that terrible issue that came up last month and that they’ve successfully managed to totally repress since then.  
  • Finally, you simply cannot assume – no matter how diligent you’ve been – that you have encapsulated their whole universe.  Unless you are a user yourself – and your team, further, represents every persona for that user out there – there will still be plenty of things you just can’t possibly foresee.

How to get around this problem?  Well, why not go straight to the users themselves?  In particular, why not let the user tell you or, even better, demonstrate how they feel, think, and behave.  That’s where in-depth interviews, usability tests, and ethnography come in.

Yes, I do typically have a list of things I want to cover when I do these kinds of studies.  What I’ve found, though, is that, if it matters to the user, they’ll be certain to let you know.  And this will come, not from your asking about it, but from them volunteering – either free-form or prompted by some specific task they’re doing.

Now, if something does not come up – and my team really wants some feedback on it – I will probe.  I always tell my team, though, that this is really an instance of a bird in the hand being worth two in the bush.  In other words, if the user brings up or uncovers something on their own, that’s going to be a lot more valuable than if I force them to come up with a response.  

In fact, I usually tell my clients that there is a definite hierarchy of feedback, based on how the feedback came about:
  1. Independently, through a task
  2. Independently, through interviewing
  3. Through probing, but only at the highest level (e.g., “How did that go?”)
  4. Through probing a little deeper (“Any thoughts on page X?”)
  5. Through direct questioning (“What did you think of field Y?”)

Note that I would never go any further than that last instance.  In other words, “Did you understand what that was for?” “Is that link prominent enough?” and “Is that the best place for that button?” are simply leading the witness, and I won’t stand for them.

Now, do surveys have a purpose?  My, of course, they do.  If it’s numbers you’re after, surveys are up there with web analytics and A/B testing.  Note, though, that quantitative methods may all lack two very important things:
  • The ability to follow up
  • The ability to get at the real reason why

And that’s why I usually recommend triangulation – quantitative and qualitative, surveys and tests, analytics and interviews …  And, believe me, that should about cover it all.




Thursday, September 7, 2017

A wealth of information creates a poverty of attention (Herb Simon)

Information density is one of my favorite issues. I’m pretty sure it crops up on every test I run.

And the particular problem I run into is typically the one that Simon points out here. If I had a dollar for every time I’ve written that n users couldn’t find x, and that we should “consider simplifying the UI,” I’d be a (very slightly) richer man. 

It’s also pretty obvious where it comes from too. Marketeers and execs are famous for wanting to provide as many features as possible, but to also make sure that everything is just one click away. Faced with this kind of “logic”, I try to point out the total fallacy of the old one-click rule and the primacy of information scent, but it can sometimes be a struggle.

I also like to point out that simpler interfaces are just easier to use. I think, these days at least, most people do get that. I only need to point to Google, or apps, or Nest, or even Tindr to make my point. 

I also, though, like to get them to think about the user’s experience as a whole. And that means not just whether users see each marketer or exec’s particular pet feature. 

I try to get across that a little here, a little there amounts to something I like to call “additive design,” a guaranteed way to get a system or site or app to sink under its own weight. I try to get them to consider that, as Shlomo Benartzi has written, “every new feature comes with a cost.”

Finally, I also like to tie in what I call the simple, basic “yuck factor.” My users are great at providing me with ammunition for that. If I had a dollar for every quote where a user basically says “TMI!” …  Even better, I’ve found that these quotes are typically some of the most pithy ones I get.

What I really try to get across to my clients in this instance, though, is that this visceral reaction on the user’s part has some very real consequences. In addition to making things harder to find, it oftentimes make the user not even want to try, and just give up. And that - and just clutter in general - can have a major negative impact on brand perception. And we all know how worked up marketeers and execs can get about that.


Herb Simon was quite the guy. A Nobel Prize winner, he also won the Turing Award, and made major contributions to psychology and sociology as well. He’s also the father of the “think-aloud protocol,” the basis of pretty much every usability test that's run. I was lucky enough to have met him in grad school.