The Trouble with Resumes

5 minutes, 7 links

Youโ€™re reading an excerpt of The Holloway Guide to Technical Recruiting and Hiring, a book by Osman (Ozzie) Osman and over 45 other contributors. It is the most authoritative resource on growing software engineering teams effectively, written by and for hiring managers, recruiters, interviewers, and candidates. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, over 800 links and references, commentary and future updates, and a high-quality PDF download.

The Trouble with Resumes

While hiring teams often view resumes as the primary means of making top-of-funnel decisions, filtering candidates by their resumes is noisy, biased, and inefficient. Although little academic research exists on the predictive use of resumes in software engineering hiring specifically, research has been done on the function of resumes across all industries. An 85-year meta-analysis found that resumes (along with other traditional assessment methods like interviews) are very poor predictors of subsequent job performance.

Engineering managers tend to harbor the suspicion that engineers are better at reading technical resumes than recruiters: โ€œWeโ€™re domain experts! We donโ€™t need to rely on proxies and can actually tell what the candidate can do!โ€ But even the most experienced, qualified engineers arenโ€™t always on the same page when it comes to what makes a good technical hire. In fact, the more experience you gain, the higher the risk that you may be stuck in your own way of seeing things. This makes the static data of a resume a particularly poor alignment tool.

Interviewing.io recently conducted a study* in which researchers removed all personally identifying information (name, contact information, dates, et cetera) from a set of resumes, and showed them to hundreds of recruiters and engineers. For each resume, researchers asked participating recruiters and engineers just one question: โ€œWould you interview this candidate?โ€

On average, participants correctly guessed which candidates were strong* 53% of the time, and there was no statistically significant difference between engineers and recruiters. Moreover, and even more importantly, peopleโ€™s errors werenโ€™t consistent. In other words, everyone disagreed about what a good candidate looked like in the first place.*

Figure: Distribution of Resume Scores

โ€‹Distribution of resume scores in interviewing.io studyโ€‹

Source: interviewing.io

Academic and industry research has consistently proven that bias is a common problem in resume filtering as well. In a 2003 study on resumes by the National Bureau of Economic Research, experimenters submitted identical resumes to a series of help-wanted ads in the Boston and Chicago areas where the only difference between resumes was candidate name; some names were thought of as traditionally white and others as traditionally Black. Resumes belonging to candidates with โ€œwhite-soundingโ€ names received 50% more callbacks. These results were repeated in a 2016 study at Harvard Business School, targeting 16 metropolitan areas and including traditionally Asian names. Another study showed bias along class and gender lines in law firms.

โ€‹candidateโ€‹ As a candidate, you will increase your chances of being judged by your strongest attributes by doing everything in your power not to be filtered by your resume alone. If you have a resume with no name-brand schools or companies, the reality is that youโ€™ll face stiff odds at companies with many applicants who do. And even if you have a degree from an elite university and employment with a top company, youโ€™ll still have more luck advancing through the hiring process at any company if you can get into the funnel later, via a warm introduction, referral, or any other connection you can find. A few things to consider when building a resume:

  • Attention to detail. Avoid typos, poor organization, or other easy-to-fix formatting problems. These can mean automatic rejection. Many people looking at a resume think, โ€œIf you canโ€™t write one page without four typos, how are you going to do other work where attention to detail matters?โ€

  • Provide context. When describing work, include context on your role (and how it fit into the team) and the business impact. had so reviewers know why it was important and how you performed. An ineffective, mechanistic description would be: โ€œWorked on Java backend of the AcmeTronix 7000.โ€ Something like this is far better: โ€œWas main developer of backend of Acmeโ€™s e-commerce platform used for $1M/month in transactions. Led team of four.โ€

  • Look forward. Highlight what you want your job to be, by writing your resume to show your aspirations, not just what you factually have done in the past.

Online Challenges

Online challenges (or coding challenges) are tests that a company sends to prospective candidates in order to screen them for specific criteria early in the hiring process, usually before phone screens or more in-depth interviews. Online challenges typically involve a small set of predetermined programming problems. Either the company or a third-party company will have created them, and the hiring team will expect candidates to complete them in a specified amount of time and without any assistance.

When candidate volume is high, screening mechanisms like online challenges can make sense; they are easier for companies to scale and assess than other types of evaluations, and they are less biased than traditional screening mechanisms like resumes. Hiring teams use online challenges to identify unqualified leads.

Companies can learn a lot about candidates by giving them a homework assignment in which they have to write code to solve a problem. These technical screens are almost always scored automaticallyโ€”once the candidate writes and submits code, the assessment system runs it against a bunch of tests to verify that the code outputs are correct and that the code runs efficiently.

Youโ€™re reading a preview of an online book. Buy it now for lifetime access to expert knowledge, including future updates.
If you found this post worthwhile, please share!