One of the best ways to learn bias-free interviewing techniques is to look at example conversations and notice the mistakes made in each. As you read through these examples, note the mistake the interviewer is making and the type of bias it introduces. You will also be shown alternative approaches that will better serve you.
6 mistakes and how to avoid them
Mistake 1: talking about your idea, not about their life/behavior
One of the most common mistakes is to talk about how great your idea is and how successful it is going to be. Imagine starting an interview with a question like this:
If you ask a question like this, you make it difficult for the interviewee to disagree. Now that they know you are spending your life working on this idea, they have two choices:
Respond with something like “I think you are wasting your life.”
Be polite and go along with everything you say (whether they actually agree or not).
The solution here is to not talk about your idea at all. Rather, ask them about their lives. Ask them if they have a certain problem, how they currently solve it and what they did last time it happened. Such a conversation might look like this:
There is one huge benefit to asking the customer to talk about the past and what they did last time something of interest happened: you almost always get reliable information (the truth!). If you can start a conversation like this without indicating that you have a huge vested interest in them responding a particular way, then the responses carry far less risk of being biased.
Mistake 2: pitching
Your main goal when conducting customer interviews is to get actionable insight. You want to know which problems are really worth solving, which events trigger the customer to take action, and what they currently do to solve the problem.
If you start selling your particular idea or solution too soon, then you won't gather enough information about the problem. Pitching your solution so that the interviewee likes it is not the goal. You are there to listen and learn, not to convince the interviewee. You are conducting interviews to see how someone else perceives the world, not to change their perception.
Warning signs that you are pitching:
You are interrupting the interviewee. In case it is not obvious, you should never, ever do this. Let them talk.
The interviewee doesn't "get it."
They pay you (and not your product) compliments. Interviewees often pay a compliment as a way of stopping you from talking / pitching and to prompt an end to the conversation.
In the example above, you also see how to fix this. The interviewer self-corrects and changes the nature of the conversation from 'pitching' to asking about the past.
Mistake 3: not digging
Look at the interview below. The mistake here is that the interviewer is asking a set list of questions (possibly interviewing from a script). This is a missed opportunity because the interviewee is indicating that they really care about a certain problem (or job-to-be-done).
The obvious solution is to dig deeper with appropriate questions at the moments where the interviewee signals that a problem is particularly painful.
When the interviewee says, "We use reports for many other things also," an appropriate question would be "What else do you use your current reports for?"
When the interviewee says "This is the worst part of my day," you absolutely have to know more about that. Why is it the worst part of their day? In what way exactly is it painful? How much time is wasted? Which part of the process is broken?
One technique that can help is the five whys. This is where you ask the question "Why?" five times to get to an extra depth into your understanding of the problem. A five whys might look like this:
Our release was two days late!
Why was it two days late? Because one big feature took longer than expected to finish.
Why did that feature take longer? Because we had problems testing it.
Why did we have problems testing it? Because we struggle to test features with external dependencies.
Why are features with external dependencies a struggle? Because our test environment isn't set up right.
Why isn't our environment set up correctly? We have never prioritized it as important work!
I sometimes feel like asking the question "why" five times in a row can be too much. It doesn't feel natural and it doesn't feel like a conversation. Interviews should feel like conversations and conversations are supposed to flow. If you have ever witnessed a child asking their parents "why" about absolutely everything, you'll know that constant questioning can lead to fatigue!
Therefore, I think it is possible to get to five levels of "depth" or "digging," but you don't always have to use the word "why." You can use "what," "when," 'how," "where," or any combination. I refer to this as the "five somethings" approach.
Mistake 4: taking words literally
Look at the interview below and consider how much weight you can give to the interviewee's responses.
You have to be wary of generic responses like "we always" or "we never." They are not sources of information you can trust. If you get a generic response, ask about the last time something happened.
Future and conditional answers are even more dangerous. If an interviewee says something like "I would definitely buy this," you might be inclined to think this is the best interview ever. However, any answer that uses a future or conditional tense cannot be trusted.
When customers talk about the future, they talk about their aspirations for the future. This is distinct from their current reality. For example, the idea of a home gym is something that I think sounds really cool and I would love to have one. The reality is that if you asked me to pay for that product today, I would say "no."
Mistake 5: not deciding
One mistake you can make is to expect the interviewee to tell you if your idea is good or bad. This is not the interviewee's decision to make.
Rather, it is the responsibility of the interviewer to dig into the customer's current and past situation (like a private detective) and then decide whether to rely on their input.
The interviewer must uncover the customer's motivations, processes, anxieties, purchase history, budget, and likes/dislikes. By understanding this context, the interviewer can then decide if this is a problem worth solving and what the customer's job-to-be-done is. The interviewer can also decide if the interviewee is a potential future customer.
Look at the two interviews below. They show two customers responding to the same question. The first response is short and pays a compliment (a known warning sign!). There is no usable information, and you need to dig into the past to understand what they lose by not solving the problem. The second response provides some amazing information. They have five full-time people working on the problem (i.e., they already spend money on solving it), and they still lose business. Therefore, if you had an effective solution, they would likely be delighted.
In these two contexts, the first customer does not have any problem worth solving (although you can dig deeper to see for more details), and the second customer certainly has a major problem worth solving. The point is that the interviewer must decide. You can't take customer responses literally; you must decide based on the information revealed.
Mistake 6: non-customers
The best way to illustrate the mistake of interviewing non-customers is to look at the interview snippet below:
The first answer tells us that the interviewee is not a customer for the problem/solution/product you are investigating. Therefore, the second answer provides information from someone who has said that they would never buy your product.
When someone indicates they are just not a customer, it's okay to end the interview at that point. No further answers can give you any value. You need to interview the next person, not force this existing person to make up answers to questions they don't care about!
Let's recap!
There are several elements to achieving real insight from customer interviews. You have to:
Identify a problem that you believe is worth solving.
Find people who have that problem.
Interview them using methods that eliminate bias.
Use their input to decide if the problem is worth solving or not.
In this lesson, you have learned what you need to do step 3 and 4 correctly. You have learned to recognize and correct the common type of interviewer mistakes that introduce bias and how to correct those. You have also learned how to recognize and disregard certain types of interviewee responses such as conditional and future feedback.
In the next chapter, we will look at signals that prove an interviewee really does have a problem that they want to solve as a high priority.
