Usability Testing is Essential to UX Design!
Usability testing is one of the best things people can do to improve web sites (or almost anything they’re creating that people have to interact with).
Rocket Surgery Made Easy — Steve Krug
Usability testing, and testing in general, is often overlooked and a lot of the time it gets pushed back until the very last moment of a design, or even the build itself. I’ve been a culprit of this myself, the easy thing to do is to skip testing altogether, and just hope that it will all work out. I did shy away from it as I initially felt like it wasn’t the best use of time, especially as the pure process of creativity was what I loved to do. I did feel it was an inefficient use of time, and I felt it was only really needed at the end of the process. Why would people want to see my napkin sketches? Plus it can be nerve-racking, having your baby that you’ve been working so hard on, to be scrutinised by strangers. The thought of having to potentially redo your hard work can influence this bad judgement, and push you into a vicious cycle of never wanting to test, ever. Another blocker for not wanting to test is the excuse of a lack of resources and time, however, it really can be done on a shoestring budget, all that’s really needed is a computer, and of course a good internet connection .
“You should always try to involve users in some way, whatever the constraints of your budget or deadlines”.
Smashing UX Design — Jesmond Allen and James Chudley.
Why
I’ve really realised the importance of testing, and how time invested early on can actually save you time, not waste time. The key is to test early and often, but instead of long dragged out days, they’re quick and efficient sessions. As Steve Krug highlights in Don’t Make Me Think “testing one user early in the project is better than testing 50 near the end”. This is especially important before getting into development, as a design change can take minutes or hours whilst a development change could take days or weeks. Testing and constant iteration is the key to creating the best and most polished design possible, taking constant learning’s from customers will undeniably create a world-beater. Steve Krug compares this to comedians, and how the only way they can truly improve on their craft is by practising in front of live audiences regularly, they then do one major show at the end of the project where everything is fine-tuned, and ready to be videoed and recorded for the masses.
Now there’s no need to test hundreds of people, that can be left for quantitive tests such as surveys. In reality as Jakob Nielsen highlights, you only really need to test with a maximum of 5 people at a time. That seems to be the magic number, as you’ll usually begin to see repetition and the same insights or frustrations emerge. Jake Knack is his book Sprint encountered the same theory, “by the time we observe the fifth customer, we’re just confirming patterns that showed up in the first four interviews”. You also don’t want to just test 5 users once, and never test again, you should test on every major stage of the process. Or even test elements in the design, such as a new novel experience or something that you believe might cause friction. Testing can really begin before even any design has started. You can test the idea, even competitors sites to see what they do right. Also testing rough sketches and even napkin scribbles can be useful, sometimes it can be more beneficial as people are more likely to give honest feedback as they can see clearly it’s a work in progress. They also won’t feel bad about criticising the design, nor distracted by minute details, and can focus purely on the experience. But to reap all the benefits of testing it really is all about testing, early and often. By having this strategy in place it will help the adoption rate of testing and make it feel habitual, you’ll realise how little time it can take, especially after becoming more experienced in the process.
“The fact is, it just works…if you sit somebody — almost anybody — down and have them try to use what you’re building, they’ll inevitably encounter some of the problems that most people are going to encounter”
Rocket Surgery Made Easy — Steve Krug
The word testing can sometimes be a scary thought, but in reality, it’s really just about talking and hearing peoples thoughts. I heavily believe in the importance of gaining peoples opinions and feedback, understanding not only, the user experience of the product but also more importantly if it’s something that people want (market fit). This shouldn’t make you anxious, even if the idea isn’t world-changing, by talking to people you’ll find ways in which it could potentially fit a need, there’s always new ideas and it’s only by talking to others that will increase the chances of discovering the eureka moments. Educating people in testing and feedback is one of the number one things we always look to do at Fello, especially startups at an early stage.
Jake Knack in his book Sprint emphasises the importance of testing and feedback when he quotes the great story of Airbnb. That when they first started off “despite their efforts, the new service wasn’t catching on. They had a few customers and a little revenue, but they weren’t growing, and — bringing in just $200 a week…they tracked down a handful of their customers. Then they interviewed them…when they returned to the office, Joe and his co-founders spent a week fixing the most glaring problems, and then released a new version to their customers. Revenue doubled to $400 a week…so they did another round of interviews and another round of improvements. Revenue doubled again to $800, then $1,600, then $3,200 a week. That growth didn’t stop.”
The importance of testing can’t be overstated, you can be the best designer in the world but until you get the design in front of real people, especially your user demographic you will never really know how well it works. Most people working on the design and product, are heavily invested in it, they see it on a daily basis so it’s hard to have an unbiased viewpoint. To get the best out of the design, you have to test, if you don’t then you’re really just making assumptions.
“One person told me that after his team saw the recording, they made one change to their site which they later calculated had resulted in $100,000 in savings”.
Don’t Make Me Think — Steve Krug
How
In a usability test there are usually two major aspects you want to be testing:
- Do people get it — is it obvious what the website or app’s purpose is, do they get what it’s about.
- Key tasks — testing key sections or interactions in the app, especially important ones that will be used regularly, such as the home page. You simply ask the user to do something, and you watch how they complete the task.
Typically there are four stages — planning, doing, analysing and reporting. Planning is an integral part but also quite simple, you should make sure you know if the test will be done in-person or remote, what tech you’ll be using to conduct the test, what recording device and note taking you’ll be doing. Once all the logistics are in place, the two main things you’ll need to focus on is having a list of tasks so that you’re prepared and know exactly what the objectives are. And of course, participants that fit your target demographic. However try not to be overly broad on your selection, as long as the people are along the right lines of your user base. Although not a must, when choosing users it is always beneficial to screen them beforehand, this can be done through email, call or even sending a survey to understand more about them. Gathering their full name, contact point (email), timezone, location and their experience with the product or similar products is handy to know. You can then already start to build a rapport with the participant, by giving an introduction to what you are trying to achieve, explaining to them how the test will work, how long it will take, and so on.
“Testing with one user is 100% better than testing with none”
Rocket Surgery Made Easy — Steve Krug
There’s no one-size-fits-all answer for finding users, you can essentially get them anywhere and even be pretty creative in doing so. From using Slack channels, social media groups, your network or even going directly to the source. For example, if the product is aimed at senior’s, then you could go to senior centres, libraries and churches. However when choosing users try and refrain from picking people that know too much about what you’re testing, as you won’t be able to get a fresh pair of eyes and real solid feedback. One thing you will need to do to get users to participate is to offer an incentive, usually, this is monetary such as a voucher, but it can also be product specific such as discounts and offers. It’s important to keep the invitation light, simple and casual, not discussing the product beforehand can be useful (dependent on what you’re testing) to keep biases to a minimum, especially as capturing first impressions can be handy. Before the user test, make sure to send out an email with all the important information in writing, such as the date and time, location and if they need to prepare anything.
Once you have your users and the list of tasks you’re pretty much ready to go. Conducting a test run with a colleague or friend beforehand is definitely advisable, to double-check all the tech works, and your tasks are easy to follow and not over-bloated. Having a prewritten script at hand is recommended, of course, you can deviate from it and it doesn’t have to be word for word, but having one will make sure you don’t forget anything important. Having this script is also useful to avoid any bias, as well as making sure you keep a straight linear journey without jumping around between tasks. Lastly, you really don’t want your test any longer than 60 minutes, but 45 minutes seems to be the optimum, and the script will aid in keeping to that timeframe. A good reference of a script can be found here.
“45 to 60 minutes is often the best range…it gives you enough time to build a rapport and cover a wide range of questions without fatiguing your participant”
A Project Guide to UX Design — Russ Unger and Carolyn Chandler
Now probably one of the most important aspects of the user test itself is making the participant feel at ease, you really want them to feel as comfortable as possible. To do this build rapport by having a chat, briefly learn about their interests, you can then begin to start talking about subjects that are more relevant to the product you’re testing. If for example, the product is within fitness, you can ask if they keep active, do they attend a gym and what fitness products do they use. Lastly, remind them that it’s not an interview or even a test on them, they can do no wrong, the test is actually on the product and actually finding flaws is in a way a positive. Possibly obvious, but another way of making the user feel at ease is sitting beside them, sitting opposite will make it feel more like an interview. This of course wouldn’t be applicable if conducting the test remotely, which can feel inherently less formal, people are usually more relaxed and comfortable — having the choice to be in Pyjamas probably helps.
Recording the test is always super useful, for reference or if you ever need to double-check your notes, but don’t forget to get the users consent first. If you’re able to get someone else to write notes for you, then that would be ideal, as it means you are completely dialled into the test. Just a tip when note-taking, handwriting can sometimes be better as clicking away at a computer’s keyboard can sometimes be distracting.
During the test it’s beneficial to get the user to talk out loud while going through the tasks, this will help with the recording, but more importantly, understand their thought process. When giving the user a task to do try and keep as neutral as possible, refrain from influencing them in any way. Use tactics such as only using open-ended questions, avoiding multiple-choice or binary questions, avoiding leading questions, and try and not express any opinions, keeping a poker face is usually a good tactic. Also during the usability test avoid asking too many questions or probing as this will interrupt the flow, try and stick to your tasks and script and only probe when it’s something critical.
Once the test is complete, you’ll want to wrap up with a quick debrief and capture any last thoughts, impressions or questions, lastly show your gratitude for their time and effort, and remind them they did perfectly and that their insights will be very useful (even if it wasn’t). After this you’ll want to analyse the feedback, doing this as soon as you can will make sure all the points are fresh in your mind.
The key to analysing, is not to take every suggestion and comment as gospel, but instead listen, take notes and then decide on popular comments that have been said by multiple people, and obviously act on any alarming UX flaws or frustrations. Quoting Jack Knack, a good rule of thumb is “when two or three people out of five have the same strong reaction — positive or negative — you should pay attention”. It’s also always good practice to always take user feedback with a grain of salt, as customers don’t always know exactly what they want, they might suggest things they like over what they need. The challenge sometimes can be deciphering this to really pinpoint what they’re trying to achieve and what an impact it has on the experience.
“If I asked customers what they wanted, they would have asked for a faster horse.”
— Henry Ford
An interactive and fun way is to gather all the feedback into sticky notes, you can then efficiently categorise and prioritise these in order. Steve Krug in the book Don’t Make Me Think outlines the importance of grabbing “the low-hanging fruit”, essentially looking for things in each round that are cheap wins that are easy and quick to implement, or those that require a little effort but are super visible. He also explains that “it’s important to keep in mind that even a minor change can have a major impact. Sometimes the real challenge isn’t fixing the problems you find — it’s fixing them without breaking the parts that are already working”. He also emphasises (as already mentioned) the importance of fixing the big problems that emerge, ones that become obvious after a few testers have noticed them, he quite humorously calls these the “head slappers”.
Your last step is creating a report of the results, this can be as simple as a list of clear, concise and actionable bullet points. Or it can be more robust, thorough and visual, where you could have annotated screenshots of the product, alongside user quotes and even pictures of their faces. Having the issues outlined, with a description on each and the severity on the UX can be incredibly useful.
To quickly summarise the steps of usability testing:
- Have a list of tasks or objectives to test. Writing a script can be useful
- Plan logistics, where and when. And also the tech you’ll be using, such as recording
- Recruit participants, and offer an incentive
- Day of test: Friendly chat, get consent over recording and then start, at the end don’t forget to thank them, and get any final thoughts
- Analyse insights ASAP, choose quick wins or major flaws first
- Report your findings
- Plan for your next test!
And that’s a wrap 🌯
In conclusion, it’s obvious to see how integral testing is, without it you’re really not doing your design justice. It might look great, but when it goes out into the real world there’s much less of a chance of it being a success. And the great thing is that you don’t need to hire a professional to do it, all you need is to be patient, empathetic and a good listener. As Steve Krug says in Rocket Surgery Made Easy, usability testing will get you “insights that enable you to improve what you’re building”. Spending time in testing is an investment which you’ll be able to reap the rewards into the future. What’s better, having your design aimlessly sitting in Dribbble or sit in the App Store with millions of downloads? Having an understanding of research and testing can really elevate your game, and take you to the next level. Be the best UX designer you can and test!
Books mentioned
As always we would love to hear your thoughts, do let us know in the comments below if you have any questions or want to know more!
Would you like to work with us? We are a friendly bunch, come and say hello 👋
Much ❤️ 🧡 💛 💚
