Avoiding Bias in User Research
A large portion of a UX/Product Designer’s job is to make user-oriented design decisions. UXers do this by understanding the users by looking into user data and insights, and taking the user’s perspectives. Piece of cake. Except, not really.
Making an objective, user-oriented design decision based on well-researched truth is one of the most crucial skills any UXer should have. But it often does not receive the attention it deserves as many aspiring UXers tend to overly focus on the step-by-step methods of UX design and research.
Throughout my career, I have noticed many designers (including myself) fall into cognitive biases, often because they are unaware of what they may look like or how to avoid it. In this post, I would like to share what I have learned about the common cognitive biases designers may often face and how I prevented myself from being affected by them.
Confirmation bias
Confirmation Bias is the most common form of bias UX designers and researchers face during assessing user research or user test results. A person under confirmation bias tends to favor information that confirms or proves one’s previously existing beliefs or biases over those that contradict them.
For example, a designer may think users reading expanded multiple long articles is a better UX for an in-app experience for reading news articles. The designer will easily agree with user insights that favor the full view of articles as they align with the designer’s initial thoughts. However, the same designer will feel hard to agree with opposing insights that favor a less-loaded view of the articles. In disagreement with the user interaction of clicking/tapping the View More button, the designer may try to actively disprove the opinions that do not align with their belief.
Confirmation biases are preventable by avoiding leading questions and preparing open-ended questions that allow researchers to explore different contexts and diverse insights.
False consensus bias
Closely related to confirmation bias, the false consensus is a form of bias designers face when researchers think their subjective opinion represents that of the majority. Some might start to get some idea on this, and yes, it is the case where that one client says: “I am the perfect user, and this user research is wrong.”
Suppose a designer is a very skilled dog walker and designing for a dog walking service application. Being an experienced dog walker, the designer may hold a subjective opinion that it is not necessary for the walkers to know the size of the dogs. However, in multiple interviews with potential users, many pointed out that they need the dog’s size information. However, the designer may still think the users will not find the dog’s size information necessary and the opposing insights are wrong. As shown, the designer with false consensus bias will think one’s subjective opinion represents that of most users, while unjustly downsizing the opposing opinions.
Detecting and preventing false consensus bias can be done by communicating with users with different experiences and viewpoints from the designer’s social circle. By listening to diverse source information, being more aware of other perspectives and making objective decisions is possible.
Primary bias and Recency effect
Primary and Recency bias often happens while evaluating multiple objects or events when one’s mind gives more emphasis on first (Primary), or last (Recency) subjects.
Primary Bias often occurs during user study sessions that require designers to observe the users. At the beginning of a research session, the designer may find a trait or an impression from the first research participant. Due to this first impression, the designer may receive a false impression that all participants shared the specific trait. Under the effect of Primary Bias, one’s first experience followed by strong cognitive stimuli (first impression) often overwrites the other experiences impacting how one remembers the entire event.
Similar to Primary Bias, the Recency effect is also often found in user study settings. Those who are affected by recency bias tend to put a stronger emphasis on a more recent event compared to those that happened before. In Recency bias, one may get the impression that the insight from the latest user research would feel more important than the insights previously acquired.
Designers can avoid Primary Bias and Recency effects in user research settings by documenting the research process by making recordings and keeping detailed notes on each session with users during the process. Through real-time documentation during the research, one can create references to go back and assess anytime. This process lowers the chances of memories being overwritten and decreases the number of biased decisions.
Signing off
Staying vigilant towards our own psychology is not the easiest part of a designer’s job because biases are often unnoticeable to oneself. But being aware of how one’s mind works and how our own mind can affect the design decisions can be our first step toward understanding ourselves better as designers to make objective design decisions that more accurately capture user insights.
