Interviews are a very vulnerable display of ourselves and I am extremely humbled by the fact that some 400 odd engineers sat with me to interview, and taught me how to do it right.
I've previously put together a team of brilliant engineers from the likes of Google, Facebook & Amazon for a seed funded startup as the product manager and set up an entirely new tech team from scratch for FusionCharts as a product consultant. Now I'm doing it at scale, as the co-founder at Adaface where we automate first round technical screening with our chatbot, Ada. Since it has been a core part of my job for several years now, I've been working on designing an interview process that I'd want to go through myself, one that helps us find the engineers best suited for the role, and most importantly an interview where candidates leave the building with a smile. This post is a summary of what I've learnt so far, and I'll lay out actionable insights that hiring managers can incorporate into their hiring process.
I'll divide my learnings into 3 sections:
- Tech interview
- Making a decision
- Interview Etiquette
1. Ask questions that helps you score candidates granularly
So how do we decide what questions to ask? Don't ask trick questions just for the sake of a filter. Candidates get pissed. And t doesn't give you any meaningful signal about the ability of the candidate. No one's happy.
Just because there's a question that only 5% of engineers can solve, doesn't mean that those 5% are the top 5% most suited for the role. Also, this way of measuring developer skills' has an inherent bias against more experienced developers.
- Don't let each interviewer decide what questions to ask candidates randomly.
- Have questions for all levels of scoring 1, 2, 3, 4 (increasing levels of difficulty/ years of expertise)
- Each question needs to be solved by all interviewers before we start interviewing candidates. And then allow candidates a generous allowance of 3x the time to solve that question.
- The idea behind the asking particular questions in an interview is to understand the candidate's ability in a given skill. Which inherently means that we can't ask everyone all questions/ the same questions- that would be waste of time. If a candidate couldn't solve 2 questions of medium difficulty, there's no point discussing a difficult question just for the sake of it. Asking an easier question will give you useful data about where the candidate stands on that skill.
2. Have a consistent scoring mechanism
After a bit of iteration, I chanced upon what I like to call assume-the-candidate-is-exceptional way to score. Here's how it works:
- When you present a question, assume that the candidate will solve it in the best possible way and give them a 5
- Throughout the course of the discussion, add negative points for things that matter and the candidate doesn't do well on.
- For e.g. for a role which requires a good grip on algorithms, -1.5 for being able to arrive only at an unoptimised answer on attempt #2, -1.5 if we had to give the candidate a very revealing hint to move forward, -2 if they give up too soon, -0.5 if they miss edge cases and so on.
- Add the numbers at the end of the discussion for that question, and pick the next question accordingly.
3. Candidates are their best judges of their own skills
Most candidates know how good they are with a particular skill, but they usually self-report incorrectly for 2 reasons: bias and misaligned scales.
There's an easy way to fix them both:
Bias → ask them to prove it with data.
Misaligned scales → tell them about your scale and ask them to grade to your scale. Most engineers are smart to recalibrate if you explain your scale.
I typically explain my skill expertise scale to candidates:
- You know the syntax and keywords enough to google, use Stack Overflow and get the job done
- You have experience handling an end to end project using the skill. Can be verified by going through their code, screenshots, understanding scale/ impact/ challenges of the project.
- You know internals of the skill, design patterns and currently perform code reviews for other engineers in your company for that skill. Can be verified by going through their code reviews (if possible) and asking what design patterns they look for.
- Zen mode: You are writing libraries and rolling out new versions of this skill.
And I ask candidates how they would grade themselves in this scale for all the skills they know. Almost all of the candidates score themselves within an error range of -0.5 of what I end up scoring them at. Use this to your advantage to know what skills to focus more on and what to spend less time on.
4. Ask at least one question where they struggle.
This might sound mean to some extent and maybe it is. But if you ask a candidate 10 questions and they solve all 10 easily, they will perceive it to be an easy interview and will associate that with how challenging the work might be. That might lead to them not being very excited about the role. The harder it is to get somewhere, the more we appreciate it. Also, it gives you a lot of meaningful data on how they react in a difficult/ stressful situation.
Obviously that does not mean that we have to ask them hard questions and make them feel like they lost. The intention is quite the opposite. The candidate and the interviewer are evaluating each other. Let's push each other to have atleast one challenge that's new so that the discussion has taught us something new, irrespective of the result.
So I end up asking atleast one question (I have a handy list of some questions that are really good/ tough) to good candidates and leave them with that.
Making a decision
1. Create a scorecard as you go and check for bias later
Don't write the scorecards post interview. You will forget a lot of what happened during the interview, the scoring won't be as granular. At the same time, make sure you check for bias at the end. Did you score someone higher just because they are from a tier 1 college? Did you score them higher or lower because they are of opposite/ same gender? Or because they were very similar to you?
2. Force yourself to say yes or no by choosing better scales.
We don't like to take ownership of things we didn't signup for. It is hard to take a stance of rejecting or hiring someone and a lot of the times we end up saying 'maybe', so that someone else needs to make that decision. Force yourself to not do that. One way to do that is choosing a good scale. If you choose a scale of 1-5, most of the interviewers choose 2 or 3 or 2.5 or 2.75 (anything that makes them comfortable to be in the zone of 'I don't know let someone else make the decision').
Here's the scale I think interviewers should use for the final yes/ no decision: 1 - 4
- If they're hired, I quit
- Don't hire them
- Hire them
- Hire them, else I quit
The pros of this scale are that there are no maybe's and there are polarizing yes's or no's. When every interviewer follows this scale, it makes easy to discuss a candidate in the context of hiring them.
3. Hire. Don't get too caught up in finding the absolute best. Be okay with making calculated "maybe" hires.
There are hiring managers who never compromise on "bar"/ "quality" of candidates. But at the end of they day we need to execute. We need to move closer towards the company's mission. So how do we decide whether to hire or not? Build a decision framework that gives results (actual hires) and course correct as you go. To this end, I believe a humane version of hire fast, fire fast strategy is better than not hiring anyone.
If we always hire everyone with a rigid framework, we'll end up with a similar thinking, non-diverse and eventually boring group. Sometimes, go with the gut and make few 'maybe' or even 'no' hires if the project needs that but be very mindful of how many such hires you're making. Who will their manager be? What are their responsibilities? Setup 1-1s to check how they're performing. Set up processes/ training to help them turn their negatives around or double down on their positives.
Here's a hiring strategy that has worked quite well for me in the past:
- If any interviewer said 4, even if someone else said 2, hire.
- If no one said 4 and atleast one interviewer said 1, don't hire.
- If atleast 2 interviewers said 3, hire.
4. Set processes to revisit your hiring framework
We need to have set meetings on the calendar to visit our hiring framework and its results. Analyze if our assumptions have been correct. Are the "perfect" hires performing as expected? What happened to the "maybe" hires- how are they performing?
5. Get your team to sit with the candidate to see if there's a mutual cultural fit.
Your friends probably know if your current partner is good for you (or not) and might actually see your breakup way before it actually happens. It's the same with a team and a new hire. Your team might give you good data on what they think about the hire culturally. It also gives the candidate a chance to experience the company's culture before making a final decision.
1. Create a short document on the pre-requisite skills and how you evaluate them. Share it with the candidates beforehand.
What's a pre-requisite or a must have skill? If a candidate doesn't have this skill, we are not going to hire them. Ideally, we won't even interview them if we can help it.
This is a bit of work. But do it. Ideally, you need to have this locked down with all interviewers. Do not let each interviewer decide for themselves what's important for the role.
Share it in the interview invitation email that you send to candidates. Once I started doing this, the conversation with candidates was much more effective. Remember that this might be constantly changing for the same role, so talent acquisition teams and hiring managers need to keep iterating this.
For example: hiring first iOS engineer? Probably want them to must have knowledge of Swift. Second iOS engineer? Maybe Swift isn't a must-have this time? They can learn on the job.
Tricky spot: Hiring for a role with no must-have skills like full-stack engineer. In cases like these dig deeper to what kind of work they should have previously done. How many projects/ delivered-impact/ time did the person work on any particular skill. If the person kept switching between languages every quarter without any high-impact projects, they might be a beginner in everything.
Once we've narrowed down on the must have skills, we also need to define the expertise range. Should they be experts in the skill, and be able to deploy new libraries by themselves? Should they already be doing code reviews in that skill? Should they have experience with one production application in that? What is the bare minimum we need?
2. Break the ice. You first.
It takes time for candidates to settle in. So we shouldn't expect them to start with talking about themselves. Have a pitch about yourselves, the company and the role handy (write it down). Preferably the short, medium and long versions. Start with a short introduction of yourself and set the stage for them.
3. ASK them for their communication style and adapt to it
This is probably one of the most important things in an interview, and what most of us get wrong. Especially because every how-to-smash-a-tech-interview guide out there tells candidates that they need to think out loud. But not everyone works that way. Some interviewers expect candidates to walk them through their thought process as they solve a question, and that's detrimental for the category of engineers who don't work that way.
I understand why that is, and it makes the interviewer's job easy if the candidate talks out loud about what they are thinking, it's much easier to see how they are thinking about optimizing step by step and score accordingly. But, we need to learn how to work in a way that's most conducive for the candidate to perform at their best.
I believe the best way is to give candidates all options upfront. "Here's a few ways we can do this, what would you prefer?" These usually ease up immediately.
4. Interview should end with leaving the candidate excited and inspired.
Ideally everyone, more so if the candidate could be a great fit. Talk about what impact YOU created in the company so far and what your next projects are. Nothing beats honesty and personal experience. Tell them why are you excited to be in the company and what inspires you. Prove with data. If you are not excited/ inspired everyday about your job, you probably shouldn't interview. That is one reason, I would never ask someone who has spent only 3-4 months so far in the company to interview.
5. Please be kind.
I'll end with this, if you do nothing else right, please make sure you're not insulting a candidate irrespective of circumstances. An investor might have pulled out at the last moment, you might be going through a breakup or whatever else, nothing is a good enough justification to make a candidate feel bad in an interview setting.
The part of your life where you're interviewing for roles can be extremely stressful. And a bad interview experience can kill someone's confidence, you never know what someone is already going through. Irrespective of how qualified (or not) the candidate is for the role, please make sure each candidate is leaving the interview with a smile. That's the least we can do for someone who took so much time out just to interview with us.
There are enough companies in the world that have ridiculous interview processes that cause undue stress to candidates. Please be an example of how it can be better.