Automated Usability Testing

At our most recent South Florida User Experience Meet-up we discussed automated usability testing. I wanted to post the slides here for those who missed them and provide a brief description of the methodology for those who are unfamiliar.

There are many types of usability tests, moderated live tests, wizard of oz test, remote moderated test, and automated un-moderated test for example. So what makes automated testing unique? It is one of the few types of usability test that doesn’t require a moderator.  Instead, the test is created and then sent out through various means to multiple people. Each participant takes the same test, much like they would an online survey. Then once the testing period has ended the facilitator closes the study and reviews the results.

This is a great method for answering specific questions and measuring task success without the need for the extensive scheduling and recruiting that goes with live moderated test sessions. In addition, many of the application available will do most of the metrics and results analysis on your behalf.

However, you can’t just jump in and ask questions and expect good results. As with any test, careful preparation is required. Tasks must be vetted, recruiting methods determined, and question makeup analyzed in order to remove bias, ensure actionable results, and organizes a representative audience.

These slides briefly discuss some of these points. They were accompanied with a live demo when they were presented, but for those interested I recommend checking out many of the sites that offer free demo versions like Loop11 to get more acquainted.