Usability experts have an annoying shared trait: they tend to answer a lot of questions by saying it depends. Each website’s unique content, audience, goals, and style ensure that there are no universal solutions to even the most basic questions. This means that there is only one way to know if your site really works for users: you have to test it. Fortunately, if conducted right, these tests shouldn’t require a lot of money or time.
When professionals begin designing a website, they typically follow a series of guidelines or design patterns that help ensure the result is easy to use. In the case of simple interactions, such as a log-in form, this method almost always results in a usable design.
However, when the interface becomes complicated or highly specific to an individual site, it’s unlikely that any existing guidelines or patterns will provide enough direction. At this point, the designer cannot—or, at least, should not—be sure that the interface will work: what they have produced is simply an educated guess at a usable design. Even worse, many inexperienced (and some experienced) designers will base their work on assumptions or personal preferences—either their own or their clients’. Now you can drop the educated and just call the design a guess.
Most major websites rely on user testing to determine if their features work for visitors. Although the specifics vary with each implementation, user-testing basics are generally the same. A representative sample of subjects are exposed to a website (or a portion of a website), data is collected based on the subjects’ interactions, and the results are analyzed to measure effectiveness.
Big sites like Amazon and Netflix collect data from huge numbers of users in a constant stream of specialized tests. If Amazon wants to test a feature with 5,000 new users, they only need to turn the feature on for 45 seconds and 5,000 new users will see it. (Being an 800-pound gorilla has its benefits.) Of course, big websites tend to have big budgets, and this type of ongoing testing isn’t cheap.
But while these tests generally produce a lot of great data, more data doesn’t always mean better results. Just recently Google tested its new social media tool Buzz with 20,000 people. Unfortunately, these 20,000 people were all Google employees—who have a specific set of shared experiences and web-usage patterns, to put it mildly—and this testing therefore failed to turn up a major usability flaw: Buzz automatically connects you to people you have emailed. The resulting—and immediate—storm of complaints had Google publicly apologizing and rushing to make changes.
By any measure, Google’s user testing for Buzz was a massive failure. Think you could do better with five subjects and a couple days? You should.
The idea that user testing can be done on the cheap isn’t new. Jakob Nielsen’s 1989 paper on discount user testing is often seen as the start of the low-cost usability movement, which also gets called guerilla usability.
The basic premise is simple: although larger studies are useful, discount testing produces valuable results and a far greater return on investment. (Predictably, Nielsen goes further and suggests that discount testing often produces better results than bigger studies because it encourages an iterative design process.) In fact, one of Nielsen’s more interesting findings was that while better studies produce better results, even poorly executed quick tests provide significant value.
At Orbit, we’ve distilled a variety of research and approaches into three main principles for guerilla usability.
If conducted intelligently, discount usability can provide crucial data to improve your site’s design and greatly improve its effectiveness. Paper prototyping can produce results in a few hours of work, and tests on a live site can be completed and analyzed within a couple days.
These quick guerilla techniques won’t guarantee that your site is perfect, but they will ensure you catch the biggest problems. And that’s perhaps the most important principle of all: doing something is much better than doing nothing. Every user test you perform—however modest—takes you further beyond the guess and closer to the knowledge that your site works.