editione1.0.8
Updated August 24, 2022You’re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.
The people included in an interview loop convey a great deal about your company to candidates. These people must successfully interview candidates while also positively representing the company. The choice of interview panel members has an outsize effect on the candidate’s ultimate assessment of a company and thus plays a significant role in their final decision should you extend an offer.
Your entire team should conduct interviews. Everybody. If you don’t want some people to interview, ask yourself why. If you’re worried about how they’re representing the company, there’s a bigger issue at hand.Marco Rogers, veteran engineering manager*
It’s beneficial to have everyone involved in some way in the recruiting process. Exposure to the recruiting process provides an opportunity to familiarize the people who work at your company with the expectations you have for employees, because the interview process filters for people who would do well at your company. That said, successful interviews require that interviewer roles be determined carefully based on the skills of individual interviewers. Some people excel at evaluating a candidate’s ability to demonstrate a specific skill set. Other roles are purely recruitment—social events like lunch with candidates or getting on the phone to answer their questions. Not everyone has the skills or qualifications to do both.
story “The interview process is an opportunity for everyone to better understand the company’s values and how it evaluates, and it helps you wrestle with questions like, ‘How do I level up and evaluate my own performance?’ The interview process should map to your performance review process. What the performance review grades you on, the interview process should evaluate on. These should be aligned.” —Scott Woody, former Director of Engineering, Dropbox
dangerNote that forcing people to interview candidates without getting their buy-in on the importance of interviewing can be a bad experience for everyone involved. Interviewers will feel like their time is being wasted, and candidates will be exposed to people who don’t want to talk to them. This thread on Blind has several comments about how employees who have been forced to conduct interviews against their wishes create a bad experience for candidates: “interviewing candidates became an unpleasant burden no one really enjoys, but [everyone has] to do anyway to report in the next performance cycle.” Likewise, letting truly low-quality candidates progress into your funnel wastes interviewers’ time and causes them to lose buy-in. Ideally, interviewers will view the process as a high-impact activity and as a privilege they have to earn and maintain.
Also note that while, ideally, everyone should be involved in the interview process to some degree, not everyone is suited to conduct interviews. If an engineer has a poor bedside manner that could sour the candidate’s opinion of the company, that person should not be interviewing. Such people often self-identify as being poor interviewers or make it known that they dislike interviews strongly. Keeping these people as interviewers can be extremely costly—they might take their frustration out on the candidate. Alternately, some might not be aware that interviewing isn’t their strong suit. In these cases, their interview conclusions tend to be either very positive or very negative but offer low signal about why. Internal recruiters often get to know interviewers’ styles, so if you’re assembling a panel, it can be helpful to talk to them about who might need to be added or removed.
importantIf someone is not happy at the company or appears to be bitter about being asked to interview, they shouldn’t be interviewing. Anyone who has given their notice to leave the company should never interview candidates.
story “There are certainly people who shouldn’t be interviewing, and it’s people who are not as self-reflective as they should be—who are not aware of their biases or are unwilling to engage in a conversation about bias. Just being aware goes a long way to reducing their effect. There are so many people who say, ‘I don’t believe in biases.’ That person should not be charged with evaluating talent, assessing skills, or judging others in any way.” —Benjamin Reitzammer, freelance CTO
Good interviewers are always communicating their excitement and passion for the company—they are excited about the company and have their own compelling reasons to work there. They likely have a captivating story about why they work there and what will excite a candidate about both the company and the role itself (if the role is on the same team as the interviewer). These personal stories help humanize the company.
dangerIt’s important not to over-sell your company, however; gross misrepresentations about the day-to-day reality of the company will cause pain and trouble for everyone after a candidate joins and learns the interviewer misled them.
caution Ensuring that your interviewers represent the diversity of your company creates a better interview process. There is a balance here. Your intention might be to ensure that you gather a diverse set of opinions on every candidate, but it’s critical to avoid overloading underrepresented folks by asking them to take on a disproportionate number of interviews (and also thus taking time away from their other day-to-day responsibilities). Creating an inaccurate representation of your company’s diversity does no good; misleading candidates can result in low retention rates, not to mention resentment. See Diversity and Inclusion in Tech for more.
It’s probably not reasonable to have senior engineers involved in every level of the hiring process, even if they could be calibrated to interview any level of candidate—senior time is worth a lot. Instead, hiring managers may wish to consider how to deploy levels of seniority and determine the best use of senior engineers’ time in any given situation.
Having had the comparative experiences of working with hundreds of colleagues and numerous hiring decisions, senior engineers are equipped to better measure and evaluate certain signals like trustworthiness and long-term potential or talent. Senior engineers may join later in the hiring process to conduct behavioral interviews or more complex technical non-coding interviews to assess architectural questions, or to dive deep into why a candidate made certain technical decisions in the past. Engineer seniority at the company is particularly important because it lends the process the perspective of deeper understanding of company values.
On the other hand, many technical skills are easier to assess with less experience and only basic training, so it’s acceptable for junior engineers to conduct earlier screens. That said, technical assessments can go wrong: junior engineers may not have sufficient experience to lend awareness of where they can make high-confidence assessments, where they may have inadvertent bias, or where they may be prone to proceed with overgeneralizations. The best interviewers are able to put their egos aside and ask questions of the candidate—but not all junior engineers are willing to appear as if there is something they don’t understand.
controversy Should junior candidates ever interview senior candidates? Asking a junior engineer to evaluate a senior engineer’s merit can be fraught. Even if the interviewer has been properly trained, they might misunderstand the abilities necessary to do a job they’re unfamiliar with, especially if they are separated from the candidate by fifteen years of experience. Having a junior engineer involved may also hurt both the candidate experience and the candidate’s perception of the company—for example, the more senior person might encounter ill-formed questions from a more junior interviewer and make a reverse judgment, like “this person doesn’t value me.”
cautionA common pitfall for junior interviewers is to grade candidates’ answers based on how they themselves would solve the problem; they may be very literal in interpreting right and wrong responses based on a rubric. Senior and out-of-the-box-thinking candidates often do go “off book” on interview questions, and this can signal serious creativity. However, interviewers should be able to distinguish brilliant out-of-box thinking from unwillingness to do necessary laborious or technical work. This isn’t always easy and usually requires follow-up questions. Experienced interviewers tend to find these interviews less stressful to conduct than more junior people do, and they are more likely to be able to dig deep enough to get the right signal.
Creating established rubrics for each interview format and making interviewers aware of potential biases will help protect against these pitfalls.
Better candidate experiences result from having some interviewers be individuals the candidate would work with directly if hired. This can give the candidate a sense of the team environment and also what their day-to-day will be like. Interviewers who know that they might work with a given candidate tend to be more invested and engaged.
Some companies compose the interview panel entirely of the team the candidate would be working with. Others include a mix of those who will work closely with the new employee and trusted individuals from other areas of the company. Having more people from the team increases buy-in and incentive alignment—the team is part of the process and will have to work with the candidate. The team will best know what the candidate needs to be capable of to succeed. On the other hand, this can lead to situations where separate teams within a company develop different hiring philosophies. It can also make it harder to schedule candidates because it reduces the pool of available interviewers.
The balance between local empowerment of individual teams and global company standards will vary from company to company based on size and culture.
importantIf you are hiring for a senior role with direct reports, it is wise to include one or more of the people who will work for the candidate.
In every interview I’ve ever had with another company, I’ve met my potential boss and several peers. But rarely have I met anyone who would be working for me. Google turns this approach upside down. You’ll probably meet your prospective manager (where possible—for some large job groups like ‘software engineer’ or ‘account strategist’ there is no single hiring manager) and a peer, but more important is meeting one or two of the people who will work for you. In a way, their assessments are more important than anyone else’s—after all, they’re going to have to live with you. This sends a strong signal to candidates about Google being nonhierarchical, and it also helps prevent cronyism, where managers hire their old buddies for their new teams. We find that the best candidates leave subordinates feeling inspired or excited to learn from them.Laszlo Bock, Senior VP of People Operations, Google*
story “If being able to guide or mentor others is something you expect of senior engineers, having more junior people interview them can be a good way to figure out how well they can explain things and how they treat people who are junior to them, which can be useful signal.” —Ryn Daniels, Senior Software Engineer, HashiCorp
If you decide that it’s important to the team or company that everyone be involved in the interview process, keep in mind that everyone needs to be trained. Interviewer training and calibration are necessary for juniors and seniors on a per-role basis.
story “It’s ridiculous to expect engineers to be competent interviewers with little to no training. To the extent you can be good at interviewing, it comes from repetition, learning the failure modes and bolstering against those. Experience just makes you much better at navigating the myriad situations. If this is your first remote interview, you’re not going to be able to tell if it’s bad because of the candidate or because it’s remote. Training often doesn’t exist, or it’s done really poorly. If you can’t invest in training engineers to interview, you can’t expect to get any real signal on candidates.” —Scott Woody, former Director of Engineering, Dropbox
Effective interviewing is a learned skill. It requires a mix of technical knowledge, emotional intelligence, and thinking on your feet, while being fair and rational. None of this comes easily, and doing it all at once is really difficult. Having experienced employees who have done interviews elsewhere go through your organization’s interviewer training will help ensure alignment.
Calibration is the process of developing the ability to accurately assess whether a candidate will succeed in a role. A calibrated interviewer will not only be able to assess a candidate’s interview performance but draw an informed conclusion about whether the candidate should be hired for a given role.