Bias in User Research: how to avoid that?
There is a huge problem in the results of almost any research—it is not the correspondence between the true information (what, in fact, what the user thinks) and what they say during the research process.
Because of this, situations very often arise when research becomes not an accelerator of the product development cycle, but, on the contrary, a time bomb that will explode when the product / feature goes into production.
Researchers are often biased data catalysts, making mistakes that force users to say not what they think, but what they want to hear from them.
To better understand what I am talking about, I will cite the 3 most common mistakes that are made in the process of conducting research:
Confirmation bias
This is the most common and dangerous mistake. It arises when we approach research with already biased inputs.
A simple example:
— «Why are you doing research, what is your goal?»
— «We want to prove that {hypothesis № 1} is in demand and users cannot live without it».
We already say in the hypothesis itself that «users cannot live without it», which means that the answer is already in the hypothesis itself, and, most likely, we will pay attention to the answers of users who will confirm its confirmation, ignoring those who will express the opposite point of view.
The sunk cost trap
This is a very common situation that can be observed not only in research, but also in life. How many times have you been unable to quit reading a boring book, because «yes, there are only 250 pages left, I’ve already read more than half, it’s a pity to quit …».
The fact is that it is very difficult for people to give up something when some resources have already been invested in it. It’s the same in product development. The product has already analyzed the market, competitors, put forward hypotheses; the designers have already drawn the layout in Figma; it remains only to «CONFIRM» this hypothesis by research. And I assure you, every interview will be perceived as biased, the purpose of which is just a meaningless and formal procedure.
Non-representative sample
This problem is related to the non-representativeness of the sample with which the study is conducted. The situation when we take respondents from one segment, and ignore the rest, drawing conclusions based on data consisting of 1 segment.
Each segment (if you have segmented your audience in detail) distinguishes from each other not only in terms of social media, but also in terms of the ultimate purpose of using your product, behavior within the product, and other important factors. It is wrong and biased to draw conclusions based on only one segment.
How to avoid bias problems?
- Be open-minded. It is necessary to take a sober approach to the research process. This means that you need to treat any information that users say in the same way. Without accents and focus on certain things. This will help make the final research findings «cleaner» and more objective.
- Think critically. It should always be borne in mind that each person has their own opinion. As a researcher, you don’t need to voice your opinion; you need to listen to what users are saying and understand the reasons why the user thinks this way and not otherwise.
- Don’t build expectations. Every person has a mental model in their head. And it is important, when the research process is going on, not to try to impose your mental model on the user. This is a complete failure and one hundred percent biased data that cannot be used to confirm or disprove product hypotheses.
- Choose words. The most important thing in research is the thoughts of the users. And it is important that all the questions and the course of the research do not turn into a «search for the right answers». There are no correct answers in research. There is data to work with and not distort it.
Conclusions
An unbiased approach to research and honest, «clean» user feedback is the best any researcher can strive for in order to bring maximum value to their company and users.
