IA Summit 11: DIY Mobile Usability Testing

The last session I attended on Day 2 was a fantastic session by Belén Barros Pena and Bernard Tyers on “Do it Yourself Mobile Usability Testing“. They walked through the positives, and negatives of mobile usability testing, and then listed out all potential constraints. They then demonstrated a solution – a usable, cheap, and useful way of preforming mobile usability testing on apps using a self constructed holder, and dual camera capture. 

My Notes

• Three things need to be taken into consideration on any mobile testing
  • Phone
  • Context
  • Connection
• Phone
  • Phones vs. ability to complete a web task
    • Feature Phone – 38%
    • Smart Phone – 55%
    • Touch Phone – 75%
  • Handset usability will affect test results, not necessarily because the software is bad, but because the hardware is difficult
  • Test with the participants phones whenever possible.
    • If not possible include training and warm up tests
• Context
  • Field versus the lab which is better? Experts Disagree
    • However, all agree that the field is more expensive, difficult and time consuming
    • Based on field constraints, lab testing is typically considered okay.
  • If doing testing in the field
    • Run pilot tests
    • Test the software as close to a completed stage as possible
• Connection
  • Do not test over wifi – it’s unrealistic
  • Compensate participants for data costs
• Recording Options
  • Wearable equipment like microphones and cameras on hats
    • Allows in field testing but difficult and expensive
  • Screen capture applications
    • Apps that can share the screen with a computer then record on the computer
    • Not one app currently supports all platforms
    • Expensive
    • Participants may not want installed
  • Document Cameras
    • Most popular, and well documented for mobile testing
    • Not cheap
    • Participants must keep phone in camera range and flat
  • Mounted device with camera
    • Natural interaction with phone
    • Not cheap and can be hard to build
    • Bulky & heavy making single hand use difficult
• The above analysis results in the following needs
  • Easy, cheap, portable, repeatable, using participant phones, supporting multi and single hand use, supporting phone rotation, screen capture, and facial capture, video quality
• Solution
  • Two small webcams, Mecanno builder parts, a usb cord, windows recording software, and putty
  • Amcap – software that allows multiple camera feeds into one computer simultaneously
  • Camstudio – screen recording
  • Screen flow
  • Total cost: 196.40!
  • By creating a holder around the phone with the Mecanno parts, a small camera can be attached that sits above the phones pointing down at the screen. This allows a view of the phone and also the participant’s fingers, and is still easy to hold, use and rotate the phone. Then another camera is placed on a desk or flat surface recording the participant face.
  • Both recordings are streamed into one screen that records the test.

Slides:

Video of them constructing the phone holder: