Conducting Interviews

You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.

Why Interviews?

The purpose of the interview process is twofold:

1. Companies assessing candidates. Once candidates have passed through the top of the funnel, companies need to gauge candidate-company fit based on deeper skills assessment; suitability for the role, including values alignment; and the candidate’s interest in the role, team, and company. In some cases, a candidate will not be screened at the top of the funnel; this occurs most frequently with senior candidates and referrals, whom the company will get to know first through more casual interviews.

2. Candidates assessing companies. Interviews typically provide the best opportunity for candidates to evaluate whether the company is a good fit for them. Interviews offer the candidate’s first opportunity to interact directly with people on the team and to meet a larger part of the company or team as they progress further through the pipeline. At Google, internal research showed that interactions with interviewers are the top-mentioned factor in candidate feedback—more important than recruiter interactions, company benefits, or even type of work.

The interview process for technical roles typically involves some kind of technical skills assessment, including writing code and answering coding questions, as well as answering non-coding technical questions and nontechnical behavioral and situational questions. Much of the software industry emphasizes technical interviews—the portion of the interview process that covers practical technical skills—as the main mechanism for candidate assessment.

A technical interview is a portion of the hiring process for technical roles that assesses technical skills and thinking through coding questions and technical non-coding questions. The technical interview may take any number of forms, including technical phone screens, onsite interviews, hands-on coding interviews, take-home assignments, and prior work assessments.

​confusion​Some people use the term technical interview to refer to the entire interview process for technical roles, but more precise usage leads to greater clarity, especially in complex interview processes.

In the past, interviews for technical candidates rarely focused on nontechnical skills like collaboration, communication, and empathy; but today more companies embrace the importance of these skills, for all roles. Academic and industry research shows that diverse teams with people who value communication, equal participation, and good EQ outperform teams with only highly skilled or high-intelligence members.** Amanda Sopkin’s post, “Are Technical Interviews a good measure of software engineering ability?” describes the path that led the industry to focus on narrow skills assessment and the ways in which that is changing for the better. And an excerpt from Laszlo Bock’s Work Rules! explains the thinking (and data) behind Google’s eventual embrace of structured behavioral and situational interviews as a means to get the full picture of a candidate’s ability.

​caution​There’s plenty of research showing that interviews are poor predictors of on-the-job performance.*** Every interview process will filter candidates according to some criteria; a poorly designed interview process will favor candidates who are simply good at interviewing in a particular setting that does not assess fit for the role. Interviews that are too specific—covering one skill set, being prescriptive about languages, focusing on brain teasers, or not asking any behavioral questions—also fail to gather accurate or complete signal. In contrast, a well-designed and well-run process mirrors the actual work the person would do at the company, focuses on how they go about solving problems, and includes behavioral assessments that test for things like communication ability.

Interviews are noisy, and biases are often present and unaccounted for. No process can eliminate bias and noise completely, but adherence to principles and a commitment to structured interviewing will help reduce bias and get clearer signal out of interviews, while creating a better candidate experience.

​important​There is no magic formula when it comes to interviewing for technical roles. It helps to let interview and assessment formats flow from the kind of signal a company needs to gather for the role; knowing what you’re listening for can turn down the noise. The best formats for technical interviews—and the balance between technical and nontechnical interviews—also depend on candidate sourcing, what part of the funnel a candidate enters into, whether they’re hiring for a senior role, the company’s resources, values, and philosophies, and the availability of a given candidate. These differ from company to company, role to role, and, in some cases, candidate to candidate, and so the technical interview loop that works well for one company in one situation might not work at all for another.

On the whole, a well-designed and well-run interview process—consistent with our hiring principles—will:

• Be predictive. Accurately assess the candidate’s fitness for performance in a specific role.

• Be fair. Minimize noise and bias while not being easily faked, gamed, or manipulated.

• Be respectful. Make good use of the time invested by the candidate and by everyone at the company who participates.

• Be positive. Leave the candidate with a favorable experience regardless of whether they receive or accept an offer.

Interviews may not be a perfect tool, but with proper design and execution, the right mix of questions can fulfill most of these core principles.

The Interview Loop

The interview loop is the series of conversations and tests a company designs to assess a candidate’s fitness for a role. Interview loops are most effective when they are standardized—to the extent possible, the process puts every candidate for a role through the same loop. This practice helps to calibrate expectations of candidates, interviewers, recruiters, and hiring managers.*

At the outset of an interview process, the company may change some specifics of the interview loop after the first couple of candidates to account for unforeseen gaps or to adjust expectations based on how the first round of candidates perform. When hiring for roles that no one at the company has held before—for example, hiring the company’s first QA engineer—the loop may need multiple rounds of adjustments.

A company’s style and priorities inform the structure of interviews. To balance candidate experience with the signal you need to gather, traditional interview loops likely include one or two phone screens and an onsite interview. Effective interview processes increase time and effort demands on the candidate as they progress rather than being front-loaded with “hoop-jumping” exercises, and they also avoid any surprises for the candidate, which the hiring manager or recruiter can facilitate by telling the candidate what to expect from the process.

A sample interview loop might include:

1. Recruiter screening call or meeting. The recruiter screens for general role fit, and fact-finds to prepare the rest of the loop. (This could instead be done by a hiring manager, especially if the company doesn’t have recruiters.)

2. Technical phone screen. A phone conversation tests for core skills, typically basic programming or technical concepts. (Some companies may conduct two phone screens to gather more signal, or so a second interviewer can offer perspective.)

3. Take-home evaluation. This is a variant of the basic skills screen that may take the form of an online challenge or simpler take-home test. (Many companies skip this.)

4. Onsite interviews. These typically extend half or most of a day and include three to six interviews in several formats, covering:

   a. in-person coding questions

   b. non-coding technical questions

   c. behavioral questions

   d. wrap-up conversation with the hiring manager that includes questions, concerns, or loose ends, and sets expectations on next steps

   e. some kind of social event, like lunch with the team.

5. Interviewer feedback. Each interviewer offers written feedback on the candidate, and/or discussion among the interviewing panel.

6. Post-interview follow-ups. Calls, meetings, and possibly second onsite visits allow the hiring team to assess anything not yet covered or to gather more signal on something interviewers disagree about.

7. Reference checks. The hiring manager or interviewers call past employers and colleagues to verify aspects of the candidate’s experience.

8. Decision and negotiation. The company gives either a rejection or an offer. An offer leads to negotiation and acceptance or rejection by the candidate.

​important​Note that this is one of many possible paths. In particular, outbound candidates or senior candidates may necessitate a more selling-focused, organic process that begins with conversations rather than skills assessments and moves more quickly to onsite interviews.

Figure: Example Interview Loop With Targets

Source: Kevin Morrill

A company may offer trial employment, in which the candidate joins the company for a short probationary period while the employer decides whether to offer them a full-time position. During the trial employment period, the candidate may occupy an internship or contractor position, and both the candidate and the company have the opportunity to get a more complete feel for whether there’s a fit. However, many candidates cannot accommodate trial employment requests, especially if they have existing jobs or other commitments.*

Preparing Candidates

An interview is most effective when each party approaches it with a mutually agreed purpose: to get to know each other and to assess fit. Candidates need to know how long the interview will be, what kinds of questions to expect, and if possible, some insightful detail about the people they are going to talk to.

This typically includes sharing the general structure of the process. As the interviewer, you may let the candidate know the types of questions to expect (specific coding knowledge? or probing previous work?). Offering tips for how your company likes to get answers and a bit about what you’re looking for will help them rise, not sink. This kind of transparency leads to a better overall candidate experience and can reduce stumbles due to anxiety, which can lead to false negatives.

Candidates benefit from preparation at multiple stages: before the process begins (you can tell them what to expect by email or phone); at the start of each change in interview format; and at the end, to explain what’s next. Preparing candidates to do well also entails checking in along the way to confirm that they understand what to expect, asking if they have any questions, and offering encouragement.

At the outset of any of these conversations, it helps to break the ice with some small talk and compliment the candidate on specific work they’ve done that’s gotten them this far in the process. A nervous candidate who feels unwelcome or out of place won’t give an accurate signal of their abilities and capabilities.