9 Strategies to Be a More Effective Researcher that I Learned from Jury Duty

Not too long ago I had the opportunity to sit on a Jury in the NY Federal court. In addition to witnessing democracy at work, what I observed from watching the attorneys gave me great insight into my own career as a UX researcher. They put on a show on how to interview people to uncover fact, and how to help their stakeholders (judge and jury) understand the content well enough to make their own conclusions, based on those facts. I immediately saw the parallels to techniques I use for interviewing participants and to help my stakeholders see data clearly. I have since used this observation to hone my skills and increase the impact that my research has on my stakeholders’ decisions.

So, what did I learn about courtroom interviewing that can be applied as a lesson for moderating a research interview?

Lesson 1: It’s your witness

The first thing I observed was the obvious difference of approach in questioning with your own witness (Direct Examination) vs your opponent’s witness (Cross Examination). In Direct, the attorney seems to allow the witness to speak as much as possible, with an open-ended but guided interview approach; the goal seems to be to uncover a clear story in an unobtrusive and natural manner, free from bias. In contrast, Cross Examination is much more leading; the goal seems to be to have the witness contradict themselves, or even to say something you want them to but they do not.

Direct examination questions are short, open ended and designed to allow the witness to tell what they know in their own words. The lawyer was careful not to lead the witness, not to name any names, or ask about any events until they were brought up by the witness. In fact, there are court rules requiring this question approach, put in place to ensure the witness can tell their story. Hearing these details directly from the witness is much more powerful, and left no doubt in the judge or jury’s mind whether the witness was speaking truth.

In Practice

A UX Researcher should always approach an interview the same way an attorney approaches a Direct Examination. We are advocates for the user; assuming you have recruited correctly, it is your client on the stand. Our interviews should be open ended enough to allow the user to tell their own story, in their own words. Hearing them volunteer that ‘this feature would be used all the time’ is so much more powerful than hearing them say ‘yes’ after you ask the same question directly.

In contrast, the aim in Cross Examination is to corner the witness into saying what you want to make your point. The attorney will often do most of the talking, and will leave the interviewee with only the option to say ‘yes’ or ‘no’, so it’s the attorney’s version that is told to the jury rather than the witness’s. The attorney may challenge the witness’s memory, instinct, and position, possibly confusing the witness and making it look like they don’t know. It might not be getting them to tell a non-truth, but to think about what they believe in another way, trying to make them look unsure and untrustworthy. These are not things a UX Researcher should do in an interview. Get out of the way and let the witness tell all.

Lesson 2: Warm up the witness

With victims on the stand, the prosecuting attorney always began by asking basic questions to make the witness comfortable. The prosecutor asked every victim about their job, family, where they are from, and how they got to NY. These witnesses were nervous. A crime had been committed against them, and this was their chance to hold the culprit accountable. The interview began with softballs, to get them over the shock of where they were, and into a place where answering the tough questions would be easier and less emotional. This may have bored my fellow jurors to sleep, but it made for a better interview, with more natural, and complete answers.

In Practice

Start with easy questions. Whether they have come to your lab or office, or you are visiting theirs, having you ask all these questions can be strange and cause anxiety. Let them get comfortable and they will open up for a much better interview. Even if it seems boring to you, it’s important to them.

Lesson 3: Make sure everyone is fully engaged

I wasn’t kidding when I said some jurors may have been bored to sleep. At one point during the trial, the defense attorney shot up, frantically asked to approach the bench, and began whispering to the judge. We were sent to the jury room so they could discuss. When we came back, we were given a lecture by the judge about how we needed to pay attention to all testimony if we are to make a fair judgement. Apparently, a juror had dozed off and the defense had motioned for a mistrial because one of the 12 of us might have missed a few minutes. There is a very strict rule that all evidence must be heard and considered by the jury. And that is fair, this is people’s lives we are talking about here! He didn’t get the mistrial, but we all stayed on our toes after that.

In Practice

You should request, and encourage, stakeholders to observe your research so they can fully understand your findings. Asking every stakeholder to watch every minute of research might be going overboard, but I do find it’s better to have them observe more than less. I have a rule: if you have time to observe one session then you have time to observe two. Of course it’s better if they observe them all. I try to avoid situations where a stakeholder observed the anomaly, made a bad conclusion, and therefore can’t believe the generalized results. The best defense against these types of mistaken conclusions is to make sure they attend at least a couple of sessions and have seen as much ‘evidence’ as possible.

Lesson 4: Follow the plan, naturally

Each attorney systematically asked questions and followed a script. Before long, I could start predicting the questions the attorney would ask next, and how they would ask them. I was surprised to see that, like research interviews, a certain kind of witness got a certain line of questioning, and it could get repetitive. (They don’t bore you with that stuff on Law and Order!)

The line of questioning was usually in the same order, but not always. Sometimes the witness would go off topic, or get ahead of themselves. Our lawyers never asked a question out of the blue or changed the subject to get to their questions. It was a procedure cloaked in a conversation, and they kept the conversation flow natural. I could tell when they were starting to steer the conversation toward a certain topic. They would ask a few set-up questions to make sure that the key question felt natural. The attorneys made sure to circle back and cover the topic because they knew that if the jury had only heard about “topic X” from a couple of witnesses, the jury would not be sure to have a complete view of that topic. We might wonder if witness number five would have said differently, breeding doubt. Any small doubt can sink the prosecutor’s case.

In Practice

In a customer interview, you too should have a plan — a guide, with a clear list of things to cover. You might not get to them in exact order, but you better be sure to get to all of them in the time you have. Otherwise, you won’t have the data you need at the end of the interview. While doubt is more fatal to a prosecutor’s goal, the replacement of doubt with certainty is a powerful tool for getting stakeholders to act on your research.

Lesson 5: Ask key questions in a deliberate, standard way

This ensures that the jury identifies the key questions, and reduces the variability in responses which might require further analysis and interpretation. Lawyers knew this would help the jury to easily compare answers across witnesses, making it easy even for novices to identify patterns and agreement across individual accounts. Their goal was to make sure all jurors were on the same page.

In Practice

Like jurors, stakeholders observing research are not as familiar with the line of questioning and might not be as good at identifying patterns in conversations. Make it obvious what key topic is being addressed and help your stakeholders follow along. Prep your observers by providing an interview guide, or at least an agenda, so they know what to expect. When you think observers might be struggling to keep up, pause and confirm with the participant ‘now we are talking about X’ or use key words in your conversation with the participant as much as a signal to stakeholders as to keep the interview moving along.

Lesson 6: Be thorough, leaving no room for doubt

Several times, after a simple question, I thought to myself “the witness just said that in a different way.” Then I realized, the attorney was just being thorough, and safe. Most likely, they knew exactly what the witness was going to say, but they had to be sure that the jury knew too. I may have been thinking it was a dumb question, but who knows what the other jurors thought. Not to mention, I can’t tell you if there were times I did not pick up on this. In either case, it would have served the attorney well to ask again to make sure I heard the answer he needed me to hear.

In Practice

Get clarity even when you yourself may not need it — for the benefit of others and to erase all doubt. These attorneys took nothing for granted, and neither should a researcher. Asking again might seem unnatural, but it leaves no room for error. Ask the same question in a different way to make sure you fully understand what they mean. Or, one of my favorite techniques, just repeat what they said with inflection at the end (to signal it is a question rather than a statement) and listen to them explain further.

Lesson 7: Focus on the facts

The always exciting ’objection’ revealed rules and norms of prosecutorial behavior that I believe should be rules and norms of interviewing customers. They focus on only allowing the witness to tell you what they know, not what they think. These norms are in place because the jury are not expert in interpreting interviews and do not know when fact ends and opinion begins.

Here are some objections I heard, and the instructions given to the jury when it was sustained:

• Hearsay — Pay no attention to what the witness believes other people think, you can’t trust that they read the minds of another. This is not fact.
• Foundation — Pay no attention to the witness’ account of what others did, other than things the witness saw with their own eyes. This is not something the witness can reliably tell you, unless they were there.
• Speculation — Pay no attention to what the witness thinks may or may not have happened. This is to be decided by the jury, after all witnesses are called.
• Cumulative — Pay no attention to what the witness has put together for the jury. The jury needs to see the facts and put them together for themselves. This one needs explanation, so here is an example: a participant might say “every day I need to do X when I get to work in the morning” (fine, this is a fact) and “when I get to work, I boot up my computer” (fine, this is a fact) and “so X needs to be on the boot-up screen” (objection, this is cumulative because they put those two facts together to form an opinion— let your stakeholders form their own opinion, based on facts alone).
• Non-responsive — The witness is answering a different question than the attorney asked, which may confuse the jury. This often led to a restating of the question, and the judge instructing the witness to answer directly.

In Practice

Abide by the norms and rules of courtroom objections. There is nothing worse than coming into the back room, engaging stakeholders and wondering how they got THAT out of the interview. Keep the common courtroom objections in mind as you form your questions to avoid this unfortunate situation. Think of your stakeholders as the jury. Choose your questions carefully to reduce the likelihood that answers will fall into one of the above categories.

When a response requires an objection, be sure to shine a light on the reason with your follow-up questions. This will help your stakeholders discern fact from opinion. A simple “So do you know s/he did that, or are you speculating?” can help.

Lesson 8: Wait until all the evidence is heard before drawing conclusions

At the end of each day, the judge sent us home with these instructions: “do not come to any conclusions regarding the case and do not discuss the case.” We were not allowed to even think about, let alone, discuss which way were leaning until all witnesses were called to the stand.

In Practice

It is dangerous to make conclusions after each participant in UX research. Wait until you have interviewed all your participants, and then come up with conclusions (insights) that are not attributable to a single interview, but to several. Make sure your stakeholders are doing the same. Once an impression is made, it will take twice as much data to unmake it! Avoid the pitfalls of confirmation bias by delaying having anything to confirm for as long as possible.

Lesson 9: Clearly present the full body of evidence to make your final case

The Prosecuting attorney used a PowerPoint presentation to support his closing argument, much like many of us do at the end of a research project! He recognized that we needed to be led to the right conclusion. He reminded us of the charges and the witnesses; stated that it was clear the accused was guilty of each; he methodically took us through each charge, one by one, presenting quotes and exhibits as evidence for guilt on each; and ended with a summary. Each part played its role in helping the jury see the case his way.

In Practice

Researchers might not have an opinion going into the research (in fact it is usually better if they don’t). But, they certainly have one coming out. An important part of our work is to help our stakeholders see the evidence clearly, and make appropriate conclusions. To help this, make sure to tell a good story in your final presentation and follow the example of the attorney, who essentially presented the methods, executive summary, detailed results, and conclusions. Each element plays its part in leading your audience to the right conclusions: methods show you talked to the right people; the summary sets them up to absorb the key details; detailed results such as quotes and descriptions help support your conclusions; and conclusions clearly lay out what they should take away. An incomplete final presentation might be a hindrance to stakeholders absorbing the evidence and making the right conclusions.

But, not everything was directly transferable

I found many parallels between interviewing in the courtroom and interviewing your target user in UX research. There was much to learn and reflect on during my time as a juror. However, not everything translates exactly. One key difference to bear in mind is that attorneys do a lot of prep work and pre-interview their witnesses. As researchers, we have not, so we have no idea where the interview might go. Sometimes, we need to be more flexible and allow the interviewee to guide what we focus on. Sometimes, what we get in the unexpected diversion is extremely valuable.

Another key difference is that, as UX researchers, we are working for pure justice; we have no pre-defined bias about the “verdict”, as long as it accurately reflects what we found in our research. Attorneys have a client, and a clear end-goal for the trial; an attorney will do what they can, within the law, to win their case. This is why you hear ‘objection’ so often in the courtroom.

In the end, UX Researchers are as much judge and jury as they are attorney, and therefore must approach their witness in a more neutral way. We must also check ourselves and avoid any objectionable interviewing techniques, because there is no one else to do that for us. The burden to find the facts and to do so in a fair way are both on the UX Researcher, as opposed to the checks and balances of attorney and judge.

Nonetheless, I believe that the strategies outlined above can help any UX Researcher be more effective and impactful. As a researcher, we can all learn from legal professionals and improve our ability to interview people to uncover fact and help stakeholders understand the content well enough to make the right conclusions, based on those facts.