Selecting Interview Formats

6 minutes, 1 link
From

editione1.0.8

Updated August 24, 2022

You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.

Technical interviews can be conducted in person or remotely, with one interviewer or a small group, synchronously or asynchronously, on a whiteboard, a laptop, or as a take-home test. Which technical interview formats will be most effective—and how those interviews should be conducted—depends on what signals the company finds most useful to gather. Each format collects different information, and each has pros and cons, pitfalls, and associated time and effort investment from both sides. When choosing a format, the company should consider whether it can be scaled and still conducted in a way that is fair to and useful for the candidates.

Deciding on formats can be a balancing act between time savings for the company and interviewers, quality of the signals that are truly predictive, and the need for a positive candidate experience.

Many processes will involve a combination of several formats for technical interviews:

  • Phone screens or video call screens.

  • Take-home tasks or assignment.

  • One or more onsite visits with in-person meetings, one-on-one interviews, pair or panel interviews, or working sessions. Onsites can be technical or nontechnical, or both.

  • Fully remote meetings.

Generally, formats move from screening (which help disqualify poor fits) to assessments (to qualify and measure candidate-company fit). Assessments include the technical or nontechnical. As a candidate advances through the interview process, the interview can ask more theoretical questions. Along the way, companies sell to candidates—but efforts to sell usually increase as a candidate moves through the pipeline, all the way to the onsite. Effort on both sides increases the further a candidate gets. But be wary of adding too much to the interview loop in the hopes of further defining the signal. There is a point of diminishing returns, and strong candidates are unlikely to stay through a never-ending process. If you aren’t getting the signal you need, it’s time to look carefully at each format you’ve chosen and figure out what’s not working.

Table: Onsite and Remote Formats

SynchronousAsynchronous
OnsiteOnsite conversation, Whiteboarding, Onsite codingOnsite task or project
RemotePhone screens, Remote conversation, Remote codingTake-home coding project, Portfolio or prior work sample

Table: Types of Interviews by Format

FormatEffort and signalStrengthsWeaknesses
Qualification call (general)LowOrients candidate; Sells them on processMay lose good candidates who need more selling or personal touch up front
Phone screen (technical)LowTime-efficient on both sides for disqualifying a candidate, especially if interviewer is technical and trainedFalse negatives high if interviewer is not technical (HR or recruiter); May lose good candidates who need more selling or personal touch up front; Difficult to do well, i.e. minimize both false positives and false negatives
Remote conversationMediumLogistically easier; Less costly; Ideal for remote workersLess warm and personal for candidate; Less signal for company on personality and team chemistry
Onsite conversationMediumLargest volume of signal; Good for assembling signal from many interviewers at once; If done well, effective and time-efficient on both sidesNeed a properly trained interviewing team and processes; Logistical costs
Onsite WhiteboardMediumGood for architectural questions; Can work for coding as well without requiring language-specific setupWhiteboards are not representative of actual coding Whiteboard coding is controversial and disliked by many candidates
Remote coding (Coderpad style)MediumMuch more representative of real programming than whiteboard coding Better signals on knowledge of a specific programming languageFewer human cues make it more impersonal and stressful; Coding in a new environment can be awkward and stressful for some
Onsite coding or pair programmingMediumSimilar to Coderpad style, but can be more friendly or personalRequires skill and preparation from the interviewers for consistency
Onsite task or projectHighGiving the candidate a little more time and a project can be highly representative of real workHarder to structure and do well; Can be confusing or stressful for candidates
Take-home coding projectHighMuch more representative of real programming than whiteboard coding; Optionally can pay candidate for their timeDemands a lot of time from candidate; Hard to scale and avoid cheating Biased against those without flexibility to do take-homes
Portfolio or prior work sampleMedium to highCan be an excellent signal if it exists; Open source work can also show popularity and utility of the workMost candidates do not have this

Figure: Interview Format Usage

Based on data from their experience with over 900 engineers, TripleByte found that interviewing practices and formats vary quite a bit across companies.

Source: TripleByte

controversyThe “best” interview format for coding questions is a notoriously controversial topic. While many companies rely primarily on onsite, face-to-face coding interviews, a significant fraction of engineers consider “whiteboard coding” to be intimidating, stressful, and not predictive of job performance. On the other hand, alternatives like take-home tasks have drawbacks as well.

So how do you decide which style to use? Given the trade-offs, the answer may differ by company, by role, and sometimes even by candidate. For example, an internship-style arrangement might work for an entry-level candidate without a lot of work experience, while a laptop-based interview may help a more experienced engineer showcase their skills. Some companies combine methods (for instance, having a whiteboard interview as well as a laptop-based one). You might also offer candidates a choice of which assessment method they prefer, though that significantly increases the company’s work to develop, maintain, and evaluate the different assessment methods in a fair way.

importantWhen choosing or evaluating formats, consider how each format relates to general hiring and interviewing principles.

According to Scott Woody, Dropbox used to do a lot of take-homes, because they found they got some of the strongest signal possible on a candidate’s ability to solve problems in unique ways. But they had to start moving away from them because they couldn’t contain the cheating.* If we apply this back to the principles, we see how it was easy (if disappointing) to make the decision to switch formats—it’s not fair because some people are cheating, and it’s not effective because you can’t assess people’s work properly or comparatively. The company had gone about their commitment to the format in the right way: they built up a robust process around take-homes, wrote and used detailed rubrics, conducted blind assessments to minimize bias, and despite all of this, encountered an unavoidable pitfall. They wisely pivoted.

In reality, no technique is perfect, and the bulk of ineffectiveness in the industry results from poor adherence to principles in both design and execution. A well-designed and properly conducted whiteboard interview is more effective than a sloppy laptop interview or a take-home exercise that frustrates a candidate. Select whatever format will cover the skills that correspond to the requirements of the job. This will protect efficiency, effectiveness, and the candidate experience. When in doubt, you may wish to revisit both the hiring principles (of candidate experience, accurate assessment, fairness, and efficiency) as well as the interviewing principles (of minimizing bias, being predictive of on-the-job work, avoiding questions that can be gamed, and being mindful of both your and your candidates’ time).

Technical Phone Screens

A technical phone screen is an early conversation between a candidate and a current employee of a hiring company with technical ability. Most commonly, technical phone screens are conducted by engineers at the hiring company, but they may also be conducted by others, such as recruiters with deep technical knowledge. The content of a technical phone screen can range from a simple conversation to live coding.

confusion Note that for some candidates—particularly high-value candidates, referrals, and outbound candidates—interviewers may begin with a more general first conversation to get to know the candidate and gauge their interest. Depending on their experience, these candidates may skip technical phone screens altogether.

importantIn general, technical phone screens are better for disqualifying obviously poor candidates than for qualifying them. Secondarily, they can help identify potential superstars whom interviewers will want to focus additional energy on courting.

You’re reading a preview of an online book. Buy it now for lifetime access to expert knowledge, including future updates.
If you found this post worthwhile, please share!