I’ve been doing this for 30 years now. I’ve got over 3,000 users under my belt. I also just so happen to have been in the same industry for 20-some years. And I’m still surprised by at least something on every test I run.
Now, maybe that’s because I’m particularly dense, and haven’t been able to learn much over all those years and users. (And it is true that there are plenty of things that I am not surprised by – things that I see over and over.)
But if you think about it, there are a ton of reasons why even Jakob Nielsen would share my same sense of surprise. First, every test is different. Even if you’re doing a second round on the same stuff, you’ll at least find something new on those changes you made from the first round, right?
Second, technology is constantly changing. I actually started out testing mainframe green screens. These days, I’m testing voice-activated applications on smart phones. Who woulda thunk it?
Third, people change as well. Though I am a firm believer in another of Nielsen’s quotes, the one that states that “technology changes; people don’t,” I still have seen many changes over the years in users as well. In fact, I think it would be pretty darn interesting to take those users I tested on mainframes and plop them down in front of an iPhone and see what happens. Yes, the basic human hardware of brains and fingers and eyes has not – and will not – change, but users’ experiences and cultural contexts certainly have.
Most importantly, though, usability testing takes what may be the two most complicated things around – humans and computers – and throws them together. With that formula, there’s absolutely no telling what’s going to come up.
But that’s what makes it so fun. Honestly, the fact that tests uncover surprising results is why I’m still around. If I wasn’t getting surprised and learning something new on each test I run, I probably would have quit a long time ago.
“If you don’t get any unexpected findings from usability testing, you didn’t run the study right” is another of Nielsen’s great quotes