Software development interviews don’t have to suck
If there was a yearly contest to proclaim the industry with the most abusive interview process, software development would win every year. Interviewers ask candidates to jump through hoops to prove their worthiness. I have been in interviews with me on one side of the table, 12 engineers on the other side taking turn asking coding questions. In a different occasion, a screener asked me to code on a voice-only phone call; that was followed by 3 rounds of 6 hours interview loops. Such drills are ridiculous, abusive, and prove weakness and insecurity in the hiring process. A strong hiring manager needs to have the skills to decide much faster if a candidate is a good fit. There is no reason to torture job applicants, other than being insecure and unable to make a decision.
As a hiring manager, I have decided to give job applicants the same respect I give to my team. I am determined to set the stage for potential employees well before the employment relationship starts. The interview must be effective, but it also has to be a good experience for the candidates. It must leave applicants with a great impression of the company and its people. I strive for people to know, from the beginning of the interview process, that it will be interesting and fun, feel good during its whole duration, and leave feeling like they learned something new, regardless if they get an offer or not.
Over the years I have refined my interview process and had great success with the following six step formula:
Step 1. An on-site one-hour screening session with the hiring manager.
I wrote extensively about this important topic in The art of Screening Candidates.
Step 2. If the candidate succeeds with the screening, I give him or her a programming challenge to do at home.
This coding challenge is perhaps the most important part of the process. It gives candidates a chance to do their best work without feeling pressured or intimidated. The no time limit and no supervision is a form of managed trust and sets the relationship tone from the beginning. The hiring manager sends the message that the company trusts people to do their best work without cheating, and in return, it asks for the time commitment to do the challenge.
I give the challenge as a one-page document containing the following elements:
- A Clear description of the problem to solve, its requirements and expected result. The challenge needs to be simple enough to be doable in 2 to 4 hours, but complex enough to need the coding skills needed for the job. It is beneficial to give some freedom of interpretation to see if the candidate chooses to make assumptions, ask questions or apply good judgment. Make the problem original, with no solutions readily available on the internet.
- A statement that the candidate can use a language of choice (give a list of preferred languages to set the stage) and no hard time limits apply. However, remind them that the first qualified candidate to finish the interview process will be hired. That puts some pressure without creating an arbitrary deadline. It also reinforces the need for good judgment and self-discipline for a candidate to be successful.
- A statement that any code or algorithm taken from the internet must be disclosed. Include the rule that the candidate should not use any library or copied & pasted code that solves the core logic of the exercise. This is mostly a time saver for the candidate since the problem you give is unique (see element #1).
- The expectation that the code needs to be easy to build and execute on a specific type of system (i.e., *nix). The candidate must include all the build scripts. You will be building, running and testing the code, and you must set the expectation that the code needs to be usable.
- Expectations that a readme file must be included with notes, instructions on how to build and run the code, technologies used and a list of assumptions made. You will be reviewing the quality of the readme file as part of the quality evaluation of the solution.
- Hiring manager contact information and encouragement to ask questions. This part has two goals. The first one is to test if the candidate is willing to ask questions when something is unclear. Asking questions prevents assumptions and is a critical skill for an any good software engineer. The second goal is to establish an open communication channel between the hiring manager and the potential new employee. That communication channel must be open from the beginning to set the stage for future potential interactions.
- Clear instructions on how to submit the solution and a confirmation protocol. I use a file sharing solution, but choose something that works for you. Make sure to ask the candidates to send you an email when they submitted their solution; also, let them know that you’ll acknowledge when you received it. This confirmation protocol is to avoid someone to submit a zipped file that never makes it to the destination, and for the two of you to wait for each other indefinitely (I learned this the hard way.)
Make sure to personally do the challenge as you expect candidates to do for you. When you are satisfied with it, test it with the members of the team that will review the solutions. Refine it until you’re happy. Candidates will find issues with the challenge itself, and you must make revisions to incorporate solutions. I’ve released many revisions of the challenge I use, and I fully expect a new revision every few candidates.
You might think that there is too much trust given to applicants you don’t know yet, and they could cheat. In practice, that is not a problem. The trust given is well-managed. If a candidate asks friends to help, it becomes painfully obvious during step 5. Other than being waste of time, it does not substantially increase the risks of hiring the wrong person.
If the candidate searches on Google for answers, they’ll have little luck finding anything particularly useful. As mentioned, the programming challenge needs to be crafted to be unique, with specific corner cases. Using answers from similar problems found on the internet is useless. It results in obvious misfit solutions that do not resolve the specific problem given. During the code review process, it becomes obvious if somebody copied and pasted code from the internet without disclosing it. You gave trust and expect integrity. Lack of integrity is not acceptable and is an automatic disqualification from moving forward.
When I hand the challenge to the candidates, I explain in details the steps of the interview process. The goal is to be absolutely transparent and eliminate any unnecessary stress due to uncertainty. Keep in mind that you are setting the stage of a relationship with a potential future team member. You want that relationship to be as clear as possible from the first day.
Step 3. The candidate submits the code when he or she is ready.
When you receive the solution, you must act immediately with a basic validation. Any time wasted is disrespectful, and might cost you the candidate. Unzip the file and follow the instructions in the readme to run the code. If you have any problems let the candidate know the nature of the problem and ask him or her to fix it. Once you have something you can work with, send the package to your team for a complete review. Let the candidate know immediately when you accept the submission and give him or her a timeline for communication of next steps. Do not let them wait without knowing what to expect. Transparency and respect are fundamental during all stage of the process.
Many years ago I interviewed with Google, and they kept me in the dark for prolonged periods of time. I had no idea what was going on, and eventually, I told them “no thanks” even before hearing from them how I did. Regardless how awesome you think your company is, it’s not any better than the way you treat people, including job candidates.
Step 4. The review Team runs, tests and code-reviews the solution.
For this step, I have developed a simple automated test that executes a set of cases and prints a report. Having the report before the team starts the review process, helps tremendously to look for issues. Usually, I run the tests as part of the initial verification; then, I submit the results to the review team along with the solution package as submitted by the candidate. I found that a well-crafted set of tests makes it almost unnecessary for the review team to run the submission, saving them a bunch of time.
Step 5. If the solution is good, invite the candidate for a 3 hours technical interview with 3 groups of two engineers each.
I ask the interview Team to start the interview from the programming challenge and go from there, mostly focusing on the challenge. Asking questions about the solution and explore improvements is a great way to break the ice. Focusing on the challenge ensures that candidate and interviewer stand on a common ground of understanding. This method is designed to avoid making the interview process into a lottery, where you might or might not be lucky to get the right CS question. Too often interviews test interviewing skills, and not the skills required for the job. The goal of an interview is not to find something the candidate does not remember from his or her college days. The goal is to decide if the candidate can be successful as a developer in your organization.
I always end an interview loop with a 30 minutes one-on-one between hiring manager and applicant. That is the time for the candidate to ask any last-minute questions, give feedback about the interview process, and to let them know what to expect next.
In total, this process takes the candidate 4 to 5 hours on-site, and 2 to 4 hours at home. It costs the company 1 to 10 man-hours, for the full process to unfold. The average is 1 to 3 man-hours, as most candidates do not pass the first screening or the programming challenge.
Step 6. Debrief the candidate with the interview team, striving to give the candidate an answer within 24 hours.
As soon as possible after the interview loop (same day is best), have a 30 minutes meeting with the interview team to discuss. Everyone should talk about their experience with the candidate, and cast a “hire” or “no hire” vote. The final decision is in the hands of the hiring manager. When there is uncertainty, err on the side of no-hire, as hiring the wrong person is expensive. If someone in the interview team is strongly against hiring the candidate, and the hiring manager chooses to ignore that feedback, the interview team should be reassessed. I see no reason to have somebody in the interview team whose feedback is unimportant enough to be ignored. It is rude for the interviewer, and you’d be better off making the team smaller and the interview loop shorter.
Interviewing is a difficult process. There is no reason to make it abusive, frightening or rude. Software engineers should be suspicious of companies that lack respect for job candidates. That lack of respect often translates into a lack of respect for the employees, which is unacceptable.
For hiring managers, strive for absolute transparency, open communication and think of it as a way to create fans of the company, not enemies.