The Fallacy of Extreme Tech Interviews
Life of a software engineer.
It must have been more than 15 years ago. I was working as a developer for a company in Seattle. I spent my days writing code, designing technical solutions, leading projects, and interviewing job applicants for engineering positions. I loved my job. The interviewing part was a nice distraction from hours of uninterrupted programming sessions. It was more enjoyable than the average meeting, more social than coding, and I got to meet lots of interesting and like-minded people.
On a gloomy evening of February, my calendar informed me that I had one more thing to do before going home. I had almost forgotten. I had to interview someone for a Senior Software Engineer position. His name was Bob.
Bob had applied a couple of weeks earlier and was scheduled to be interviewed by six engineers for one hour each. As one of them, I had the shared responsibility of deciding if he was the right person for the job. I was also one of his roadblocks to a potentially prosperous career in my company.
I entered the small conference room reserved for the interview. Bob was sitting, tapping his fingers on the desk. He got up with an awkward jolt and shook my hand attempting to smile. His hand was sweaty, and he could not look at me in the eyes for more than two seconds. I looked at him for a moment, and said, “It’s nice to meet you. Can I get you a glass of water?”
“Yes, maybe, if it’s not an issue,” he mumbled.
His résumé was stellar, and he had a long list of accomplishments. He had been in the software industry for a decade and worked in a few companies I was familiar with. Everything about Bob screamed software engineering success. However, something was off. In the interview room, he reminded me of a child in the presence of a frightening stranger. Or a prisoner walking toward the execution room.
I was the last person in the interview loop. Before my time-slot, he already had to resolve many programming problems. I had no idea what the other interviewers asked. From the scribbles on the whiteboard, it looked like someone made him write C code to reverse a linked list. That is a very standard interview question. So standard that made me wonder why we were still asking it. I also wondered how often interviewers asked him the same question in the past. One? Ten? I mentally counted how often I had to write that code on the job. Zero.
Looking at the whiteboard, I could also recognize a drawing of a room, switches, and light bulbs. Somebody must have asked the infamous light bulb puzzle; it goes something like this:
“You are in a chamber where you see one closed-door and three light switches. The door opens to a separate room where there are three incandescent light bulbs. With the door closed the light bulbs are not visible from the room you are standing in. The door forms a perfect seal with no gaps.
You know that each switch activates a different light bulb. You don’t know which switch goes to which bulb. Your goal is to figure it out.
When you start, you know that all the switches are off. You can do whatever you want with them, but you can only open the door one time. Once you open the door, you cannot touch the switches anymore. You need to deduce which switch controls each bulb. How do you do that?”
What just happened here.
Bob looked tired. I sat down and made small talk to release some of the tension. I asked, “How is your day so far?” He looked at me, shrugged and said, “I am not sure. It was a long day. I think I screwed up one of the coding questions. I am not very good at coding on a whiteboard, so this is not easy for me. Also, it took me a while to solve the light bulb puzzle, but I think I got it right. I am not sure. I guess I’ll find out soon.” I was shocked. Nobody told him if his answer was correct.
That’s when it dawned on me. Was I interviewing Bob to know if he could do the job we needed him to do? Or was I interviewing him to find out if he is good at writing code on a whiteboard and resolving logic puzzles? Are the two things the same? What if they aren’t? The interviewer who gave him the bulb problem neglected to tell him if the solution was correct. Why? What if the usual tech interview process is good only at testing interview skills? Maybe the industry got it all wrong. What if we convinced ourselves of something that is not reality? How often don’t people get the job because they are not good at interviews? How much talent gets wasted this way?
That evening I didn’t ask Bob my usual coding question. I decided that there was no point and spent my hour with him talking about past projects and his passion for technology. I also brainstormed a real problem with him; a problem I was trying to resolve before the interview. We made some progress together. He left relieved that I didn’t get him to code on the whiteboard. He was intrigued by the discussion and surprised that the last hour wasn’t pure torture.
When the interview team met to discuss, a few people were not thrilled with his performance at the whiteboard. They said he didn’t seem confident and stated that they didn’t like how he named variables. They also said that he couldn’t write clearly, so it was hard to read his code. “Since when are we interviewing people for their handwriting skills?” I asked.
Mark, the interviewer who asked the bulb question, said Bob was nervous and struggled, but resolved the puzzle. I asked him why Bob didn’t seem to know that. Mark thought about it a moment and said that there was no reason to tell him. It would not have helped to assess the candidate, so he forgot to mention it. “So, does it mean that it doesn’t matter how the candidate feel? When did we get so uncaring?” I thought.
Two things happened that day. The first one was that it was decided Mark should not interview people. He didn’t like it anyway, and he wasn’t very good at it. Respect for a candidate is paramount, and anything less is unacceptable. The second was that I made a strong case for Bob, along with another interviewer, and in the end, he got the job. Bob ended up being with the company for several years and was a stellar developer who did great things.
The abnormal normal.
The company I worked for at the time was far from being an extreme case. It was average in the industry, if not above average. It allowed engineers to interview people in whatever way they wanted. Leadership gave no direction or training.
Interviewers feel safe and in control. They often act like they need to show their superiority, stake out the territory, and prove who’s in control. As a result, they treat candidates with hostility. It reminds me of a fraternity hazing, a third-degree examination, or an “ordeal by trick-question.” Applicants are questioned until they show weakness.
Things must change.
First, tech interviews must identify if an applicant has the technical skills necessary to do the job, and the soft skills to be in the organization. Second, tech interviews should be viewed as a test for both the interviewer and the interviewees. Third, tech interviews should display the character of the company, and give the candidate an idea if they want to be part of the team. Finally, tech interviews should not be tests of endurance. They should be a professional exploration of skills, knowledge, and character.
Interviewing candidates is a science and an art. As an interviewer, you must test their abilities. You also want them to enjoy the experience, and you want them to get to know your team and like your people.
If an interviewee has a rough experience, bad things happen. Even if you offer them a job, they will refuse it. If you don’t give them a job, you’ll find your company bad-mouthed on social media.
But, if a candidate has a good experience, good things happen. If they don’t get the job, they’ll talk to their colleagues and friends about your company. As a result, hard-to-reach people will start applying. If you do present them with a job offer, they will start with gusto, excited to join an excellent team.
It was my experience with interviews like Bob’s that convinced me. Companies must create interview processes both productive and enjoyable. Years later, as a manager, I invested a lot of time refining an interview process that was fun and fruitful. It took many trials and errors; it is in constant evolution, but the process I use now gave me excellent results. I shared it in a post titled “6-Step process to hire happy software developers and create fans” (AKA “Software development interviews don’t have to suck.”)
A solution for the light bulb puzzle.
If you are wondering how to resolve the light bulb problem, you should think about it. It is a fun little puzzle. Here is a possible solution. Do you see other ones?
- Label the switches with S1, S2, and S3.
- Turn S1 and S2 on. Don’t touch S3.
- Wait 10 minutes.
- Turn S2 off.
- Open the door.
- The light bulb connected to S1 is the one that is lit.
- The one connected to S2 is off and hot when touched.
- The bulb connected to S3 is off and cold.
If you enjoy this kind of logic puzzle, or if you are interested in more tech interview history and stories, you might want to consider reading “How Would You Move Mount Fuji?: Microsoft’s Cult of the Puzzle – How the World’s Smartest Companies Select the Most Creative Thinkers.“
As for all my industry stories, I changed names, time frames, and any information that could identify real people, businesses involved, and intellectual property. Additionally, I never write stories about my current employer or anything that is recent or potentially damaging. I make absolutely no exceptions to this rule. Plese, refer to my disclaimer page for more info.