78% of experienced programmers can't code this problem simpler than FizzBuzz
April 20, 2021
Anyone remotely involved in recruiting has their own horror story about candidates with perfect resumes being invited to interviews, only for the hiring manager to figure out that they can't code. The Spray and Pray application process deployed by candidates leads to each corporate job opening attracting 250 resumes on average. This forces companies to use an automated screening process to avoid spending countless hours interviewing candidates.
A FizzBuzz style screening is critical to prevent hiring managers from wasting their time interviewing candidates who can't code.
Traditional assessment tools use coding puzzles and niche algorithm questions for the screening, which creates a lot of angst among programmers about having to go through irrelevant screening assessments.
While it's great if someone's good at puzzles, this is not a strong indicator of how good of an engineer someone is/ how good they're going to be in the role (for most roles). Also, this way of evaluating developer skills has an inherent bias against more experienced developers, and developers from non-CS backgrounds.
The main reason we started Adaface is that traditional tech assessment platforms are not a fair way for companies to evaluate engineers. We started Adaface to help companies find great engineers by assessing on-the-job skills required for a role.
We deploy subject matter experts to go over a customer's job description to understand the ideal candidate persona and select the most relevant questions from our database to be included in the assessment.
Over the last few years, several enterprises globally have switched to Adaface, and we've had a chance to take a closer look at whether the approach works. We intend to share our learnings with the community, to help us uncover insights better and fast-track our progress towards the goal of building candidate friendly assessments.
One of the experiments we had at Adaface was to test if it makes sense to include easier or more difficult coding questions in a coding test (in a 45 min assessment, you can include only 1 question so doing a mix of both is not an option and we're not keen on 120 min assessments).
One of our favorite questions to include in coding tests is a very straight-forward question which requires the candidate to use just an array and dictionary/ map. When our team created this question, it was categorized under "Easy" (as the question is attempted by candidates and more data points come in, the difficulty level is updated automatically). Now that this question has been attempted by a significant number of candidates, we can share some results. Here goes:
Since this question is currently being used by some of our clients, we can't reveal the exact question. However, here's what one would need to do to solve the question:
This should give you a clear picture of what the question would be like, and the difficulty level of the question.
These candidates are not truck drivers or plumbers without any programming experience who applied for the role without reading the job description. These are pre-screened engineers who have 2+ years of experience, and who have applied for roles at companies like Morgan Stanley, Amazon, PayPal, the Singapore government, and the likes.
You're right, it doesn't.
The idea behind the coding test is to just filter out candidates who are definitely unqualified, which this question does a great job at. Ideally you want to assess their ability to write clean, maintainable code with a simple design, work collaboratively with the team to arrive at solutions to difficult problems etc. But that's not the coding test's job.
Before setting up a 2 hour interview with a candidate you want to know if the candidate can write code. That is the job of a coding test.
Again, you're right.
A take home assignment can do a much better job of capturing what the role will be like, so the evaluation will be better. However, most companies that use take home assignments report a completion rate of 30% (since they're too long).
If you're seeing better completion rates, you should probably stick to using a take home assignment. For the others- candidates are busy, and if you're adamant about using long assignments as the first step of your hiring process you might lose out on good candidates.
An easy coding question is a great way to filter out unqualified candidates who can't code. Nothing difficult- just a simple coding exercise to go through the motions of writing code. That way you can make sure the coding test is short (~45 to 60 mins) and doesn't drive away your candidates.
Looking to screen candidates for your tech roles? Check out Adaface.
Deepti is a co-founder at Adaface. Her online persona is extroverted, but in real life she is terribly introverted and you can startle her just by calling out her name.