Usability Testing for Mobile Devices

It’s easy to do usability testing on desktop machines. Laptops with built-in microphones and webcams partnered with screen-recording software make it simple to capture the experience of using a website — from the obvious stuff, like the movements, scrolling, and clicks of a volunteer’s mouse, to the subtler stuff of facial expressions and voices. But what about testing user experience on a mobile device? This is a challenge that has resulted in some interesting innovations.

Why Mobile Usability Testing is Different

There are a couple of reasons why the “standard” approach we take to usability testing — which is really optimized for desktop use cases, anyway — won’t work very well on mobile devices:

1. There are no good screen-capture software options for mobile devices (yet). It may be that a mobile version of something like Camtasia is in the works, but if so, it’s going to have to overcome some challenges. For instance, an app running on an iPhone that captures the screen would have to be able to run in the background, so that if the user begins running a different app, it will still capture what is happening on the screen. The ways that mobile operating systems handle “multitasking” might make this a bit less straightforward than running multiple applications simultaneously on a desktop machine. But like I said, I am not aware of any screen sharing or capturing software for mobile devices at this time (would welcome correction).
2. But even if there was, screen-sharing is not enough for mobile device usability testing. On a desktop, we use the webcam to capture the face of our volunteers, the microphone to capture the sound — our instructions and questions and our volunteer’s feedback — and the screen-capture software to record what the volunteer is seeing and the movements of their cursor across the screen. All in all, that’s a pretty comprehensive approach. If the same information could be captured on a mobile device, we’d run into problems pretty quickly. For example, if the camera on the front side of a smartphone could be engaged along with screen-capturing software of some kind, it would not pick up very helpful information. It would actually capture a strange angle — looking up, diagonally, at the users face, which will either be foreshortened enough to not capture facial expressions with any value, or offscreen entirely depending upon the position of the user and how far away from them they hold the phone. But as they use it, their own hands will mostly fill the frame and make much of that footage useless. We need a solution that can capture what is happening on the screen as well as what is happening above and near the screen, specifically with the user’s hands. A camera looking out from the device is not set up to do this well.

So, what can we do?

DIY Solutions for Mobile Usability Testing

A few months ago, I was looking for some information on this while helping someone do some research on mobile usability testing for libraries. We discovered that people had been building mini rigs using all kinds of materials to position an external camera over the mobile device that would capture the screen and the user’s hands without compromising their ability to use the device in a natural way. If you think about it, all kinds of setups could be constructed, but it’s tricky to make one that still allows the user to pick up the device and hold it in their hands as they would their own — instead of being required to sit and operate a device that is being held for them in some way, for example.

Here is the first one I found:

Image credit: Harry Brignull

This rig was created by Harry Brignull (He calls it a “sled.”) for £5 with the help of some local plastic merchants.

Image credit: Nick Bowmast

This one (above) is a slight improvement over Harry’s first version, created by his friend Nick. Pretty slick, right? I wanted to build something similar but didn’t have the time to track down the people who could help me bend some acrylic in the same way. But, I at least had an idea of the shape I needed to emulate.

So here is my “budget” version. The camera is a very cheap one that had an adjustable arm that screws into a base. I removed that and found a strip of thin steel at my local hardware store that had threads that matched the camera’s screw. I then mounted some velcro on the bottom half of the steel strip and attached the alternate side of velcro to a cheap case. All that together made this functional, but less attractive, rig.

I’d still like to try and make something fancier at some point, though 😉

Of course, I could also just purchase something like this:

Image credit: Mr. Tappy

This is a product called Mr. Tappy. Pretty impressive, right? Here’s a video of it in action:

In my research, I also found another, albeit rougher, approach. The MailChimp team blogged about using the webcams already on their laptops and iPads to record mobile device use sessions. Basically, they’re turning them backward and “giving them a hug.” See here for details on that.

Would love to hear about any other solutions you’ve come up with!