We are often asked which method is best for my needs, User Testing or Expert Usability Reviews? The simple answer is, it depends on your objectives and available budget.
If we were thinking of the two side by side, we could expect to get some good results from both. If we were to liken the two to a premier league football team, the expert usability review would be a mid-table workmanlike team, good at what they do, but not title or trophy contenders. Whereas, the User (Usability) Testing study would be a Chelsea, or perhaps Manchester City, a team at the top of their game.
The main differences between the two are:
User Testing (or Usability Testing) is primary research that is based on one-to-one interviews with users that match your user persona profiles. They work through a set of scenarios to test specific user journeys and the process is filmed. This is then analysed to identify potential blockers in a website, mobile app or prototype. You’re able to identify nuances and detailed, specific information throughout the test.
Expert Usability Reviews are undertaken by a usability specialist who inspects a website or digital property to identify potential usability issues at a high level. They follow a list of best practice recommendations (perform an heuristic evaluation) to identify design breaches and give you a check-list of actionable insight pointers to make your website easier to use.
Usability Testing is generally more in-depth and will identify some genuine issues that may not have been apparent from an initial Expert Usability Review. Identifying these problems could have a real financial impact on a revenue generating website or mobile application, that could be easily rectified.
Completing an Expert Usability Review and comparing it to User Testing often identifies that the usability expert may not have identified all of the problems or that elements that you thought were issues were not in fact difficult for users to tackle. No matter how good the expert is in evaluating a website, they are making assumptions based on their knowledge, but not necessarily on the habits of your target user groups and potentially serious issues could be overlooked. They could also lack domain expertise that real users would identify.
There is some overlap with Expert Usability Reviews and User testing and research by Fu, Salvendy and Turley in 2002 estimated this to be around 41%.
An Expert Review will generally identify more issues and they are often more commonly performed as they are cheaper and quicker to complete. It may only take a few days to inspect a site, analyze the information write this into a report. Because of this, they’re often preferred to usability testing, especially when time and budget constraints are paramount.
Usability testing requires more time to organize and plan and will be more expensive as you will need to recruit and incentivise users from a specific target audience and this can be costly. It is without doubt worth the effort though, as the results will undoubtedly provide you with a truer picture of problems that your target customers will encounter.
It is often inconceivable to understand how a modern website can go live without having checked that is genuinely usable. We would liken this to buying a house without looking inside or to continue the football analogy, buying an expensive striker without knowing his form or track record. This is crazy.
A further advantage of usability testing is that conjecture is reduced as you receive powerful and direct user feedback. The video user testing sessions can provide a valuable persuasion tool for reluctant stakeholders that are difficult to dispute.
An Expert Usability Review can be a good start on the road to improved User Experience Design, but key elements such as the Information Architecture and overall website and navigation should be tested early with real users and refined throughout the build process. There can be no substitute for this face-to-face validation process or the tangible benefits that it accomplishes.
