Thursday, March 29, 2018

There are three kinds of lies: lies, damned lies, and statistics. (Benjamin Disraeli)

I have a mixed relationship with statistics. 

On the one hand, I know their an important part of our history. I also know there some of us out there who still do a bang-up job with them (Jeff Sauro and Jim Lewis come immediately to mind), Finally, I also realize that stats definitely have their use.

I myself, however, am much more qualitative. I generally like to leave stats to things where they are a much better fit – surveys, web analytics, A/B testing, big data ...  I’m more than happy with my rich, why-based qualitative data.

That said, I do use stats.  Because of the qualitative nature of my data and the much smaller number of participants I typically have, though, my stats tend to be much more on the lighter side.  For example, if I’ve got my typical 10 users, I think it’s good for my audience to know if it was 2 users who had a certain problem versus 9 of them.  

I also typically eliminate any 1-offs. I find they typically draw too much attention to themselves by their mere inclusion.  That also has the added benefit of making my report easier to write, and easier to read as well. 

I also might do error rates.  Once again, though, I try to keep it simple.  If the user aces a particular task, that’s a 100.  If they veer off somewhere but eventually get back on track, or if they complete everything but miss 1 small part, that’s a 50.  If they give up and look to me (and even then complete it with some help from me), that’s a 0.  I add ‘em all up, and then generally use a simple academic grade scale – 90 and above is an A, 80 to 90 a B, and so on.  I just want to give my audience a rough idea of how the system performed.  (BTW, time on task – because of think aloud – means absolutely nothing to me.)

Finally, I also like to do a SUS. Note, though, that scores for individual tests mean almost nothing on their own.  I do keep a running total of all tests we run however. And that let’s me compare 1 particular score to all the rest, to an average for the type of system or user group, or for previous tests of the same thing. Very helpful.

So, yes, I do use stats.  But only in the simplest, most direct ways that pretty much anyone – business types, developers, writers, graphic designers … could understand.  


I have no idea why he’s on a phone case, but there you have it

Tuesday, March 20, 2018

Do the right thing. (Digital Equipment Corporation)

My first job out of grad school was at Digital Equipment Corp., or DEC. For those of you who haven’t been around as long as I have, I like to tell people that it was the number two computer manufacturer in the world at that time, and was something of the Google of its day.

Google, in fact, has a similar motto, “Don’t be evil.” Well, DEC’s is a little more positive, of course. But it's still the same general idea.

Now, we might contrast those two that with the motto of Facebook. Their motto was originally “Move fast and break things” (they’ve since come up with some bland official pablum). 

Well, I guess they've managed to do both, haven't they? They moved very fast, and they seem to have broken many things – the ability to go more than a few minutes without looking at your phone, any sense of personal privacy on the Internet, journalistic objectivity, the democratic process, civil discourse …

I think this is particularly ironic as Facebook is a social media company – a company who’s purported purpose is to bring people together. Of course, there is also their business model. And that revolves around addiction cum targeted marketing – basically hooking people then delivering them to large corporations so they can extract your hard-earned dollars (and yen, and euros, and yuan) from you in the most efficient manner possible. You know, maybe their motto ought to be, “Profits before people.”

So, what does all this mean for the average usability engineer out there? Well, there was a time when our main job was to make things easier to use, for users. In essence, we made the world a friendlier, nicer place. These days, though, it seems mostly to be to sell, sell, sell. That moral high ground we used to occupy is steadily eroding. 

So, what can we do about it? Well, when your company, or project team, or colleagues, appear to be blindly headed down that path of moving fast and breaking things, perhaps you can be the still, small voice of conscience. In other words, don’t be afraid to raise your hand. Ask them about the implications of what they’re proposing, what effect something might have on your brand, how that UI change will fit into the customer journey. Remind them, too, that there is more to the world than profits, that a handsome paycheck does not absolve you of all moral obligations, that there is still a possibility for UX to make the world a better place.


The only wrong thing former DEC CEO Ken Olson did was to bet against the PC  :^(

Wednesday, March 7, 2018

Agile is twenty cement mixers pouring feverishly while somebody looks in the Yellow Pages under "architect." (Roger Belveal)

I have incredibly mixed feelings about Agile

On the one hand, I know people are making it work with usability. I have even done something like that myself.

On the other hand, though, I know it was developed without usability in mind. Now, you can make it work with usability, but you really do have to make a separate, concerted effort.  It is not baked in.

Out-of-the-box Agile does not guarantee usability. This worries me, as I am often struck by the zealotry of a lot of Agile advocates.  In fact, there seems to be a certain fundamentalism within Agile that implies that you have to do Agile only in its purest form.

So, what does Agile have instead? Well, there are the user stories. And they are a great way to turn requirements into something that ties in real users trying to do real tasks with real goals in mind. So, that does loop the user in when it comes to creation and design at least.

But how about the evaluation part? Yes, Agile does a great job making a tweak here, a tweak there, then rushing it out to market to see if it “moves the needle” with actual users. A/B testing is an even more sophisticated version of that.

Note, though, that this is totally correlational. You know that something worked better, but you have no idea why. Design, then, becomes a simple shot in the dark. Throw it on the wall and see if it sticks. Run it up the flagpole and see who salutes. I was hoping we had come a little further than that in all these years. Sigh …

Now, there’s plenty out there about how to make usability work within Agile, and it does make a lot of sense (and, like I said before, I’ve done it myself). I do really worry, though, about the people who aren’t super-sophisticated about UX who hear about Agile, can’t wait to jump on the bandwagon, and want to “do it right.” Unfortunately, that last bit often means not even thinking about something that took years to finally become adopted in IT and which has proven its value many times over. For me, it almost feels like starting from scratch in a way. Maybe I’m naïve, but I honestly thought I’d never have to make those arguments again.


In addition to being a kick-ass UX professional,
Roger Belveal is also a pretty talented artist
(these are his iPhones made out of concrete and steel)