Heck, even though I call myself a “usability engineer,” I know what I do is honestly pretty iffy engineering. And I should know – I put up with real engineering school for two years before calling it quits.
What usability does, however, have in common with “real” engineering is a focus on practical solutions and on real data. Now, there was a time when that data was pretty darn hard even for usability – basically, error rates and time on task. Usability engineers found, though, that that hard data was lacking an important component. That data really didn’t tell anyone why users failed, why things took so long, what project teams could do to make things better. Softer, more qualitative data, however, did.
So, you may run across clients who still insist on that hard data, especially if they have a quant background (for example, from business or marketing). In that case, you have to sell the richness of qualitative over the certainty of quantitative. And for some clients, you will definitely have to overcome the idea that qualitative data is less pure, less perfect. In those situations, I like to emphasize what we do with the data – in particular, that soft data can be a lot more actionable than hard. (It also typically eliminates the conjecture that actually comes when teams move on from gathering their hard data and then try to interpret what it means and how to come up with solutions.)
A similar issue usability engineers have to deal with has a lot to do with the numbers thing. I cover that in “Not everything that counts can be counted, and not everything that can be counted counts” (which is a quote from Albert Einstein).
Finally, there is the issue of the perfect test. And I’ve talked about that before in, “The test must go on” (I’ve got Laura Klein down for that one).
Ultimately, the final decision can come down to running an imperfect test or never getting around to running that perfect one. And we all know that there's nothing perfect about the latter.
Usability is really the art of the possible. We do what we can. Like I tell my clients, give me what you’ve got, and I’ll be sure to get you something of value in return.
But then again, there’s this!