Tackling The Inclusivity and Algorithmic Bias Paradox In Design and Tech
Discussion with Martha Cotton, Managing Director of Fjord
Every year a design consultancy (owned by Accenture Interactive), Fjord, creates a Trends Report, similar to John Maeda’s Design in Tech Reports. Reports like this give us the sense of what’s relevant today and what’s going to lead in future markets.
This year’s theme for Fjord is Value and Relevance. Based in their report, “Digital is now so widely adopted that its novelty has worn off. We are at an innovation plateau — a flat point in the S-curve before new products and services become mainstream. Some will be powered by artificial intelligence, such as hyperpersonal living services; others will be driven by a shift to the circular economy and new cultural norms around data, identity and well-being”.
The topic of discussion is the “Inclusivity Paradox”. From my point of view, this is being more intentional with how we decide something has relevance or not in our lives by tackling the root issue of “inclusive” products. Fjord has noticed that “people expect organizations to see and engage with them as individuals.” The paradox behind being more “inclusive” and “personal” is that organizations inadvertently exclude others and when trying to speak out about this, organizations risk making users feel even more excluded by saying something that lets them down even more. This can prevent organizations from being more open to creating solutions for people who really need them because they feel comfortable designing for the “majority”.
I had the opportunity to sit down and chat with Martha Cotton, a group design director at Fjord/Accenture, who is responsible for developing design research practices in North America. I asked her questions about inclusive products and existing practices of where it is now and solutions they have discovered which can lead the direction to tackling the issue of designing with bias and excluding users.
The Interview
For the new designers out there, can you tell me about Fjord and what they do?
I am the lead for design research at Fjord, working with 1000 designers all over the world and being well equipped to do research work. This year I was also on the team who worked on the trends report. I did qualitative research and was the main author the Trends Inclusion report.
In the trends report, you say that there is a risk of being inclusive as we inadvertently exclude others. How have you seen this happen?
The reason for this paradox is because we increasingly have tools and AI. These tools allow us and organizations to be more inclusive, but by being more inclusive, the danger is excluding others. Couple that with “industries inability to want to see people as individuals and people’s demand to be seen as individuals”
One example is the huge gap in the fashion industry to answer the calls for muslim people. There is a reason there is little to no fashion for muslims. The fashion industry here views muslim women as one homogenous group. This is a paradoxical thing that’s happening in other industries. You see in it clients; they think about millennials and generation z, and expect an entire generational cohort to want the same things. It is how a lot of organizations think as they think through products and services to bring to market.
A major call to action is that we need to meet people where they are as quest to be individuals. AI can help do us better, but before that happens, we need to think of people beyond their demographics.
What kind of methods do you have in mind for designers and organizations to design ethically?
One of things we do a lot at Fjord is when we design for large groups of people, we think about the mindset they have during a specific experience. In financial services, we developed something called “money mindsets”, in which people, regardless of income, demographic, age, have a distinct mindset to saving and approaching money. Right now, Banks see different mindsets as “doing it wrong” if users are using their product in a way that one might except, like using products that specifically catered towards certain mindset (i.e people who have jobs).
Designing products for one major group of users leaves other people out. What about depressed people, people are out of their jobs that are using financial products? We need to identify those mindsets to better serve these people. They matter.
You mention mindful design as a “fast rising” initiative. How can existing methods tackle the inclusivity paradox?
I love this idea of “mindful design” and it comes from the design goals trend. The simple idea is that people are overwhelmed with physical and digital clutters. Organizations need to do less with more. This is a simple statement to make, but it’s hard to do and a hard design challenge. If you recognize these people, add less in their lives but still make yourself relevant, how do you do that? First, it is important to not shout and overwhelm the user with a plethora of features that would be helpful in theory but is not a healthy way to engage with an organization and consumer. I might tune it out, and start to get annoyed.
Yesterday, I was on an airline and in the hour before I caught my flight, I got pinged and notified from the airline, the app and travel agent. It was continual, and though this doesn’t happen all the time, it was still annoying to be bombarded with notifications. Overall, this doesn’t set yourself for emotional engagement. Let’s do less of that and focus on curating the experience.
Sometimes getting more out of the digital experience is doing less. Silence of gold. The emotional response people have around the world with digital clutter has started a trend that people are thinking about mental health of people who are digitally engaged (example is iPhone screen time statistics, have it on your phone weekly). Governments are exploring cost of social media on young people and in additional to the consumer, the feeling of being overwhelmed and other mental health concerns have been becoming more and more relevant.
One other fascinating point is that we used to talk about the digital divide. This was a socioeconomic way of seeing how people had digital tools and people who don’t. Now, it’s people who have access to digital tools and people who have the luxury to not access them and design that into their own lives.
As a result of finding peace in potentially toxic behaviors when consuming technology, people have been opting out of digital subscriptions or trying out new products. I-phone’s Screen Time to monitor and limit phone use as well as ones for Instagram and Youtube have been popular for the use of decreasing the time one spends on technology.
There is a cool product called IRL glasses. They look like sunglasses but you can’t look any screen when you put them on. These literally force us to not use a digital interface. Interesting product responses as a response to digital products ideally catered towards us.
Have experienced the inclusivity issues within your team and the company?
One thing I was researching with implications in healthcare. AI is going to be amazing in accurately diagnosing illness in patients. One of the things that has happened is diagnosing melanoma. They have access to huge date sets that allow doctors to diagnose melanoma, but data set is based on their people’s skin. This is great, but not inclusive because there is no data on the people who don’t have fair skin. This is good but not inclusive at all. In the long run, it is dangerous for people of color because they won’t get the same treatment as the people with fair skin.
When chatting about the melanoma dilemma and biases in the space medical space, examples relating to tech where companies have excluded other groups (algorithmic bias) is Google Search where back in 2009, where searching up certain terms yielded racist results, Snapchat’s racist filters, Airbnb causing gentrification while catering towards vacationers and people with lots of money, and Facebook’s Year of Review in showing pictures of loss and disaster where its intention was to showcase only the “good” moments.
What do you think gets in the way of designing more inclusive products, especially for underrepresented groups and minorities?
I think there is usual constraint of cost of locations. Lots of times, organizations start to spin if this has to work for everybody. It is really hard to have one product that works for one group no matter what. The investment that has to go into to inclusive design would be prohibited. Not because they don’t have great intent. It’s expensive. Lot of upfront work that needs to go into inclusive design, going out into the world and understanding people in context. For example, around money mindsets, we found groups of people with different experience with money from different socioeconomic situations and understand people’s experience with money. This was definitely worth the effort because we gained a more robust understanding of money.
If you are able to meet groups of people, there are economic and altruistic benefits.
To pitch this idea to stakeholders, maybe invest time into how important understanding people is and turning that into opportunity for our clients, so you can argue you can make that investment. A method I have found useful is getting to know how executives or stakeholders have achieved business goals in the past and if it’s based on what I am trying to pitch (i.e. user research), I can point out and ask in greater detail such as, “I see that you achieved x by going to users workplaces. What did that entail as I am proposing for us to something similar in order to help us achieve our business objective.”
Any tips on having a more inclusive mindset when gathering research and designing products for the design community?
When designing or building something, ask yourself: What assumptions I am making about these people and am I confident that this is right? This is so important but not instinctual for people to do.
Check your assumptions because that is essentially what organizations do. They assume they know what people want and move forward with that. They move with assumptions based on their own world view. Sometimes that works but to design inclusively, that won’t happen.
Go into the world. If in retail, put yourself in the experience and watch people shop. Ask yourself what do you know and go to the place where the people you are serving and engage with will interact with your product. If you are a hospital administrator, go sit in the patient room. This sounds so simple but I don’t think it happens very often. That is the first thing to do.
Ending Notes
Fjord alludes to artificial intelligence helping overcome this paradox of inclusivity, and though most people believe that is the problem to most problems, embedding inclusivity always starts from human input. If we are inherently bias in the choices we make within our products, the output will be inherently bias regardless of how robust the technology is. Until then, when we can effectively use artificial intelligence to solve all of our problems, we need to check our assumptions and for organizations, they have to evolve their approach that goes beyond the data, and not basing product decisions on simply increasing engagement and revenue.
Here are the solutions that Fjord suggests to do or what we may seeing more this year. I enjoy the fact that these points are rather applicable and don’t rely solely on using trending methods as a copout for actually solving the problem now.
1. Marry quant and qual data
When designing services, carefully mix human insights with data to breathe more color into facts that are often black and white. Learn the differences between qualitative insights and quantitative statistics at scale and how each can make the other more powerful.
2. Focus on mindsets over segmentation
Move away from traditional marketing approaches that treat people as a homogenized group according to their demographics. Instead, focus on the mindsets that group people together based on their motivations, attitudes and behaviors.
3. Become a Living Business
Living Services are personalized services that adapt to user needs in real-time context. To be able to deliver them, you need to rewire your business by putting humans at the center and strive for ultimate customer relevance — you need to become a Living Business.
Check out my Skillshare Course on UX Research and learn something new!
To help you get started on owning your design career, here are some amazing tools from Rookieup, a site I used to get mentorship from senior designers.
