Usability's benefits, after all, have proved difficult to quantify so far. Manning says usability testing ultimately won't gain much ground. Surprising as it may be, he says, quality hasn't ranked as the primary priority in software purchases. We buy buggy software all the time. Moreover, usability testing can run into difficulty because it's often conducted with a small sampling of testers.
Users vary widely in their skills and intelligence.
One particularly savvy or dim user can skew the results abnormally Laskowski says the common reporting format would at least elicit enough detail from vendors so that customers would know how much experience a vendor's testers have. The customers would also see which features were tested, so they could determine whether the vendor tested the features they want. Even users directly involved in the project concede that it may not achieve the ideal goal of making usability data as visible in software purchasing as nutrition data is in food purchasing. But if the effort ultimately results only in heightened awareness about the importance of usability testing, he says, that will be a major benefit in its own right. It's a worthy effort, Manning says.