Usability testing is arguably one of the most useful tools a UX designer can use to gather real insights, as the Nielsen Normal group put it, “the only way to get UX design right is to test it.”
This article will dig deeper into one of our favourite types of testing, remote usability testing.
“If you want a great site, you’ve got to test”
Don’t Make Me Think — Steve Krug
Remote testing is a hot topic at the moment, as people learn how efficient, easy and powerful it can be. When we first started the studio, we had the preconception that it wasn’t as insightful as in-person lab-based testing, however, we’ve found quite the opposite is true, and it is now one of our favourite tools in our armoury.
The main drawback between remote and in-person we’ve found is how heavily reliant it is on technology. For example, you might find participants haven’t installed software properly, there can be a lag in actions and worst of all poor internet connection. Although things are constantly improving and getting better, we’ve already seen a massive shift and development this year alone. The main advantage of remote testing, and a big one at that, for both the facilitator and participator is convenience. But there are also several others:
The major pros to remote testing:
- Ability to facilitate the test from almost anywhere, and at any time
- It’s a lower commitment for the participant
- Scheduling tends to take less time, both for the participant and the facilitator
- No travel time
- Reach a wider demographic, especially those in different countries
- Can feel more casual and less like an interview (the choice to wear pyjamas helps)
- Usually costs less
- Higher participation rate (less likely of a no-show)
Another interesting point, in which Nate Bolt and Tony Tulathimutte describe in the book Remote Research. Is the fact that when doing in-person testing, the user most of the time is in an unnatural state. Whereas with remote testing you can be testing a product in the real context of where a participant could be using it. “In-person research will introduce an element of the unnatural. After all, you’re either asking someone to leave their typical context (home, work, the train) and to recreate their needs in an unfamiliar space”.
Moderated vs Unmoderated
There are two main types of remote testing, moderated and unmoderated. In a nutshell, moderated is when you interact directly with the participant, whilst unmoderated is when you have no direct contact during the test and leave them to complete the test without your support, through a service such as UserFeel, Userlytics or UserTesting.
“Remote-moderated testing can yield a lot of valuable information from users.”
Lean Product Playbook — Dan Olsen
We tend to favour moderated as the data is much richer, you’re able to gather in-depth qualitative feedback, you’re able to speak to the user directly, build a relationship and really dig deep into their thought process. Through this, you’re picking up nuanced behaviours, tone of voice, and task and time context. You also have the ability to probe at new subjects that might spring up, which makes the research more flexible and dynamic, and enables you to explore behaviours that might have been unforeseen during planning. Lastly, there’s also more upfront work with unmoderated due to having to prepare instructions for them to follow, and they really must be crystal clear as you won’t be there to guide them if any issues arise.
“The quality is not going to be the same as doing a moderated test, but I’ve been favorably impressed” — Rocket Surgery Made Easy — Steve Krug
Unmoderated is a great way of gaining quick-and-dirty results, and you can of course use both methods. There are some advantages over moderated. You can typically get results quicker, there’s less time having to recruit and schedule (can usually be done on the platform), you’re able to test multiple users at the same time as you’re not restricted to your availability, and lastly there’s little risk of verbally influencing the participant.
UXmatters have a great article on the topic of unmoderated testing.
Process
The process is essentially the same as in-person testing, we dive a little deeper in a previous article. But the main difference that you do want to focus on, as I’ve already alluded to, is to make sure all the tech you are using works, and that everything is planned to a tee. Running a couple of pilot tests before the real thing will make sure everything syncs well.
“Almost everything about remote testing is the same as testing in person: you choose what to test, write scenarios, follow the script, ask them to think aloud, probe and so on”.
Rocket Surgery Made Easy — Steve Krug
Similarly to in-person testing, you’re not limited to what you can test, paper prototyping can be completed successfully by emailing a printout and going through it together. Testing live apps or live demos can also be done, but we always find clickable prototypes work the best. Live testing can be scary — think of Tesla’s bulletproof Cybertruck unveiling! We tend to either use Figma, InVision or Marvel for the clickable prototype, it’s quick and easy to do, and most importantly the participants don’t need to download any software, it’s literally just accessed through any browser. Another plus with Marvel specifically, is that they have a new awesome user testing tool that can be plugged into the prototype, to further geek out on data and get some really cool insights. Lookback.io is another great alternative.
After knowing what you’re testing and how, picking a good video conferencing tool is imperative, not only for seeing the participant but also to record and monitor (using screen share) their actions with the prototype or product. Google Meet/Hangouts, Zoom, GoToMeeting or Skype are all good options. Whereby, a new kid on the block, is also a good one, as there’s no need to create an account or install any software.
A great source for remote user testing tools:
Lastly, you want a way to take notes, it’s advisable to have someone else do this, as it keeps your role solely on being a kickass facilitator. Writing notes manually is great, but also any of the popular note-taking apps will do. A Google Doc or Sheets can be useful if collaborating with a team, but if you hate spreadsheets (as I do) using a digital whiteboard such as Mural or Miro is an excellent collaborative and fun solution.
If you prefer Miro here’s an alternative, kindly shared by AJ&Smart.
Conclusion
To conclude it’s clear to see how powerful and useful remote testing can be, for the main reason of pure convenience it will help adoption and make it easier to facilitate, as well as potentially gathering more relevant users due to them being a lower commitment. The quality of the insights has been proven not to be hindered or affected by not having face-to-face time, if it makes it easier for the facilitator but also the participant then it’s clearly a no-brainer. It’s never been easier, so get remote testing now!
To read a more thorough post on Usability Testing as a whole, check out this article.
“You should always try to involve users in some way, whatever the constraints of your budget or deadlines”.
Smashing UX Design – Jesmond Allen and James Chudley.
Books mentioned
As always we would love to hear your thoughts, do let us know in the comments below if you have any questions or want to know more!
Would you like to work with us? We are a friendly bunch, come and say hello 👋
Much ❤️ 🧡 💛 💚
