Seeking Clarity in Questions and Answers

2 links

You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.

Seeking Clarity in Questions and Answers

I’m not interviewing for the right answer to the questions I ask. Instead, I want to see how the candidate thinks on their feet and whether they can engage in collaborative problem solving with me. So I always frame interview questions as if we were solving a real-life problem, even if the rules are a little far-fetched.Eric Ries, author of The Lean Startup*

The best interview questions are short and clear. Short and clear questions ensure candidates answer the question you meant, not just those they’ve previously thought about and are primed to respond to.

Good interviewers use a balance of open and closed questions. Open (or “open-ended”) questions, such as “How would you approach that challenge?”, allow the candidate to determine the direction of the discussion and show how they handle ambiguity. Open questions do not have a yes, no, or otherwise simple answer. Closed questions have specific answers, such as “How many months did you spend on this project?” These questions are good for gathering basic facts or seeking clarification.

Interviewers may need to probe into ambiguous or obfuscated answers, which are often signs that the candidate either doesn’t know the answer or is trying to hide something (perhaps avoiding an admission of fault, in a behavioral interview). Ambiguity requires specific or direct follow-up questions.

Sometimes a candidate may say something that just doesn’t make sense given the context shared in the interview. For example, they may describe previous work experience in a way that sounds problematic to the interviewer, but the candidate doesn’t acknowledge the problem. This sometimes happens because the candidate may not recognize that they’ve left out key context or details. This situation requires further questions, even leading ones, to clarify whether the candidate behaved correctly but just failed to communicate clearly during the interview. Any time a possible exculpatory explanation exists, a good interviewer will pursue it to avoid later questions about whether their assessment was correct.

Eric Ries stresses, “No matter what question you’re asking, make sure it has sufficient depth that you can ask a lot of follow-ups, but that it has a first iteration that’s very simple.” This gives you the opportunity to probe further into what the candidate may or may not know. Ries distinguishes between degrees of “not-knowingness” that give you additional signal:

  • The candidate doesn’t know, but can figure it out. Perhaps it’s been some time and they can’t recall all the details, but they have a strong intuition that leads them in the right direction.

  • They don’t know, but can deduce it given the key principles. If you fill the candidate in on the basic rules, can they reason from there? Would that change the way they approach the problem at hand?

  • They don’t understand the question. Ries notes that “Most questions require a surprising amount of context to answer. It doesn’t do you any good to beat someone up by forcing them through terrain that’s too far afield from their actual area of expertise.” If it is their area of expertise, you know you’ve hit a point to consider in your evaluation. If it’s not, it’s best to move on—it’s not fair or efficient to ask them to keep struggling through at that point.

Hinting and Helping Candidates Shine

A key part of extracting signal is making sure the candidate is able to put their best foot forward. An interviewer’s job entails setting up the candidate to shine—but it’s up to the candidate to actually shine. While the interviewer/interviewee relationship has an inherent adversarial component, if it’s clear that the interviewer is trying to set the candidate up for success, it can de-stress the process. Strategies might include outlining the flow of the interview before it begins, making it clear what the interviewer is looking for, and clarifying how the candidate can best provide answers. Giving appropriate guidance and hints can help get the candidate unstuck.

Much like the process of scaling the question, good hints are metered out carefully, starting with more general hints and gradually getting more specific. An interviewer who solves the problem for the candidate definitely doesn’t intend to hire them, so hints that answer for the candidate aren’t appropriate unless it’s clear they are not a fit. It’s also important to time hints properly—not too early, before the candidate has time to think; and not too late, when the interview runs overtime. This timing will vary by question (some questions take longer to answer than others), but if a candidate is clearly stuck for more than a few minutes, it’s usually time to get them unstuck. Also note that letting a candidate flounder for too long restricts what information you can gather as an interviewer. A candidate stuck for 45 minutes on a single aspect of a question might have performed brilliantly on the rest of the question, but for you to discover that, you’d have to give an appropriate hint so that they can move along.

Hints can take multiple forms:

You’re reading a preview of an online book. Buy it now for lifetime access to expert knowledge, including future updates.
If you found this post worthwhile, please share!