UX Brighton - Mobile usability testing
Notes from two talks at UX Brighton by <a href=\"http://www.twitter.com/rajarjan\">Raj Arjan</a> from <a href=\"http://hcid.soi.city.ac.uk/cityinteractionlab/\">City interaction lab</a> at the City University London and <a href=\"http://www.twitter.com/silverfoxyboy\">Walt Buchan</a>, director of user research at <a href=\"http://www.cxpartners.co.uk/\">CX Partners</a>. The event was kindly hosted by <a href=\"https://twitter.com/FabricaGallery\">Fabrica Gallery</a> which worked really well.
Both talks were packed full of kit that you may want to buy to help you test a mobile experience. Here's a list of the kit and a summary of the main points of both of the talks. I've intermingled some of my thoughts and past experiences of testing.
- Testing in the wild is hard you could follow someone around all day waiting for them to need to use the app or site. But that's going to be expensive and unproductive.
- You could go and watch people using a mobile device in their own home, which is probably going to be quite successful and give you more insight than sticking them in a lab. If you are trying to test a second screen experience this would be particularly helpful.
- Home baked solutions are good. You want them to be as flexible as possible. What you really want is people using their own phone if possible, it's going to remove all sorts of familiarity barriers.
- You may have to accept that the video that you get won't be as perfect as it would be in a normal lab setup. You may not be able to get a picture of their face as well as the screen, you're testing. You'll need to decide which is more important. Maybe start with a shot of their face, so you can remember who they are when you are reviewing the video.
- Why are you filming the test? Is it to show people what happened? Or so you can remember what happened yourself? If it's the former, the video that you take is important, It's there for you're client. Therefore your probably going to want to put more effort in to getting the set up right. Having said that, the last time I conducted some research and presented the video's back to senior stakeholders. I got a real feeling that they were not really that interested, even though I was only showing them small important parts of the sessions. They trusted me to do the right thing for them and just wanted the headlines so that they could make decisions based on that information. Where this video came in useful is sharing it around the team, so that they can reflect on the things people did.
- What you are really trying to achieve is, a situation whereby the test subjects are unaware of the camera in the room. You want something that doesn't get in the way of the person. For that reason any sled based approach can be difficult. That doesn't mean you should go all covert and start hiding stuff in the walls MI5 style.
- Raj had some nice solutions for PC and Mac setups. Both solutions give you a good cheep way to get started.
- Elmo visualiser - circa £300 Place the phone below the camera and pipe the video input in to your computer then record the video with a screen capture app. I've used a similar much less lo-fi approach to this with two iPhones in the past, for testing a prototype. All you need is a jumbo tube of smarties, or something like that and two iPhones. Place one iPhone on the table, with the top butted up against the tube of smarties. Then place the other phone on the top of the tube of smarties with the camera pointing at the other phone. Test and adjust and necessary, so you get the other phone in focus. You can then record what you or the participant is doing and show it to other people on the project. The downside to this is that the video is upside down so you will need to rotate it somehow, I just held the phone the other way up with rotation lock turned on. Coincidentally, if you have a lot of remote meetings. This looks like something that could solve some remote collaboration issues.
- Muvi mini camcorder £35. Get the test subject to wear the camera around their neck. Your probably not going to want to rely on this as the only video for the test as the quality will probably not be great for £35. This also looks like a great little tool for "guerrilla" research.
- Mr Tappy A sled that you can mount the phone on and then point a camera at the screen. The problem with anything like this - as I said above - is it gets in the way of the natural use of the phone.
- MOD1000 $410 (with international shipping) Another sled.
- Machine cameras Tiny cameras that give you a very clear picture from a distance, you can pan and zoom from your Mac. I'm not sure how necessary the pan and zoom will be. In my experience things happen in a split second in a test and unless you are very quick once you have zoomed in the moment will have passed. Walt chucked this one in saying he was aspiring to it, so there is not much more info than that. You will need to get the right lens as well.
- Display recorder An app that is only available on a jail broken iPhone. The problem with a phone that has been jail broken is they tend to crash more often, not ideal for a test situation.
- Reflection app An app which will broadcast the screen of the iPhone so you can record it directly from your phone and would make display recorder obsolete.
- Mirror Op The same as reflector but for Android.
- UX Recorder Coming soon to iPhone captures the screen and touch gestures. It also uses the front facing camera to capture the users face. If you're just designing an iPhone app this is perfect and looks like it has the ability to render all other iPhone test solutions redundant.
Mobile user research is a strange beast. You're probably going to have to accept that you're not going to be able to be that scientific with a test of this nature. You may have to accept that you can't adequately get a picture of the users face. I think it's important to remember that testing of this kind is part of the design process. Design is about making mistakes and tests like this are just a good way of finding those mistakes.
Comment on Twitter