One swallow does not a summer make. (Aesop)

This one is actually posted in the observation room of my usability lab. It’s a subtle, tasteful reminder that you might want to consider coming to more than just that one usability test, Mr. or Ms. Team Member. 

Yes, I know you’re busy. Yes, I realize that usability tests are a little bit like baseball games – lots of boredom punctuated by rare, brief flurries of excitement that are easy to miss.

You do realize, though, that that one user you saw may not be totally representative of all 10 I will be bringing in this week, right? In other words, if you happen to be at the very first session, there’s really no need to completely redesign the system (or get all defensive or jump off the roof) before the 10:30.

And when it’s time for the report out, do please be a little circumspect when you’re tempted to talk for 10 minutes about the one user who had that one problem that – hate to break this to you – nobody else actually had. The rest of us did see that person, saw a whole bunch of other people as well, and determined that that original user might just be an outlier.

Yup, that’s right. The rest of the team actually attended most of the tests. And, as a matter of fact, I personally happened to attend all of them. In addition, I was paying attention the whole time as well. Finally, I spent probably twice as much time reviewing my notes, looking at the tapes, trying to figure out what it all meant, and putting it all in a form that you could easily digest.

Now, don’t get me wrong. I’m really happy you could make that particular session. And, no, I am not taking attendance. It’s just that the more users you see, the more you’ll understand and the more refined your subsequent judgments will be. No, you don’t have to attend every darn one. Heck, 3 or 4 might be enough to give you a good idea whether what you’re seeing is representative or not. 

(By the way, I actually have not found this to be a real problem for the actual members of the project team. Interaction designers, information architects, writers and even graphic designers are usually there for the duration. It’s often the managerial or business types who are guilty here. And that’s okay. In general, if these types take my report seriously and act on the findings, it’s not totally essential that they see it with their own eyes.)

And, no, Aesop was not blind. I understand 

the sculptor just had trouble “doing eyes.”