What to keep in mind when creating a research discussion guide
By
Doaa Badran
March 21, 2023
Today we thought we’d share some super helpful insights that we’ve recently learned from our very own Senior Researcher here at Digital of Things, Doaa Badran.
We sat down with Doaa to get the lowdown on what she thinks is the key to writing research questions using a discussion guide. This is basically a set of pre-planned questions and topics that guide an interview and help elicit the responses you need.
Here are the key highlights when you’re building a high-quality discussion guide that provides actionable qualitative data for gauging a user’s pain points, needs, and desires.
Start with deep-diving into the client’s brief objectives, who they’re targeting and why.
Begin by asking what the client’s research objective is in order to decide on a suitable approach and methodology (user testing, user interviews, or a combination of both). Sometimes clients already have a strong hypothesis on paper, which helps us understand what to focus on in the discussion.
The first step is to identify the methodology used, which will help us shape the questions. For instance, questions for group sessions would be somewhat different from user interviews where you already have the undivided attention of one participant in order to delve deep.
The second step is to know your target audience to help you choose the correct wordings, probes, prompts, and relevant examples.
Doaa is a firm believer in grouping the questions of your discussion guide in a logical order – this is to avoid overlapping certain topics that might make the next question redundant. Structuring your questions also helps avoid biased responses from your users. For example, if you asked users about their perceptions of a specific product and then asked them about other competitors to this product, you run the risk of their answer being influenced by what they think you want to hear based on the product you’ve just shown them.
Using the funnel approach, Doaa recommends starting with very broad questions and ending with specific questions. Begin with basic inquiries into their general perceptions (ie. the first thing that comes to mind, likes/dislikes) and then start focussing on general behaviour towards the product/service, followed by the main topical questions about the product or platform at hand. That way the conversation feels easy and naturally flows. rather than being thrown off by odd questions and sequencing.
The key is to make the user feel comfortable enough to open up and give authentic and reliable answers. For example, a client may want to test an app prototype and gain feedback on customer perceptions. Once you know the objective of the study, you can then start to get a broader idea of the information areas you need to investigate through specific and precise questioning. For this example, we first want to start broad (ie. “What are the first things that come to mind when we think of this app?” or “What can be seen as positive or negative?”), then ask specifically about the stimulus being tested.
That being said, a moderator should always remember that a discussion guide’s flow and order can change based on the user’s answers and unexpected insights.
A moderator must also be flexible and accept that an answer may come before asking the question in the designed flow. Based on the broad research objective, it is up to the moderator to decide if altering the flow or adding questions will change or bias the answers.
A moderator is responsible for wording their questions mindfully in order to avoid swaying opinions and leading the question in the direction you want it to go. If you’re asking for an opinion on your new product’s prototype, wording the question as “Did you like the prototype?” might influence them to provide positive feedback. Instead, try to keep the question generic, neutral, and non-leading, ie. “What do you think of the prototype?”. Then follow up by asking about what they liked or disliked about the prototype.
It’s also important to write probing and prompting questions rather than presumptuous ones, for example, “Can you tell me a bit more about how that makes you feel?” or “Is there a reason you did this?” rather than “Did you like this product because of XYZ?”’.
We can only summarise or conclude when wrapping up the conversation between sections. This should be done after telling users “I am just going to wrap up what you said, please correct me if I missed or did not understand correctly”. This usually gives users a chance to add more points as you present them with everything they mentioned.
She explains that the brain is split into two: one that requires a lot of thought and mental processing and the other side of the brain that triggers spontaneous immediate reactions (in other words, subconscious answers you’ve probably not ever thought of before). As a moderator and user, the side you manage to capture depends on the types of questions you ask.
To get the user to stop, pause, and ponder on something for a while before replying, make sure you ask questions that trigger conscious thought rather than spontaneity, like: “Tell me why you connected these together?” or “What made you choose this over that?”
Whether you’re conducting a one-to-one interview or a group session like a focus group, it’s important to be an efficient moderator in the conversation. Although it’s easy to automatically analyse a user’s answer or jump to conclusion in the middle of the discussion, remember to slow down, take your time, and remind yourself you’re the moderator and the conversation facilitator. You can put your data-led researcher mind back on once the session is over.
Other challenges may be specific to group session discussions due to the nature of them being a deep conversations with multiple subjects. Some users tend to dominate the conversation while others fade into the background and agree with whatever is being said by the louder and more active members of the group. To eradicate this problem, a moderator needs to take control of the group and give everyone a chance to speak up without allowing the session to be dominated by a few individuals.
Some projective techniques and games like grouping exercises and psycho drawing can help harmonise the group as they encourage them to work together or individually ensuring everyone gets heard.
When curating a discussion guide, it’s vital you take the time to come up with the right kind of questions and think about the order and flow of the guide. It is vital to do dry runs as they usually make you realise issues with questioning, including unclear wordings, repetitive questions, missing follow-ups or probes, and most importantly whether the questions fit the specified duration of the session.
