At a dinner I attended this past week, the discussion turned to a CEO’s frustrations regarding a particular hire. He said something to this effect:
I’m trying to hire a VP of People and find that the interview process is a poor way to evaluate candidates for this role. Everything seems so subjective. How can we add rigor and objectivity to the hiring process for such a senior person, in a role as broad as human resources?
Hard to measure
There are few objective measures of performance, in general. Certain roles are more evidence-based than others. If you’re a programmer, your Github profile says a lot about your competency, but interviewers can get even more granular with techniques like pair programming. To take it a step further, I used to work with a CEO who would invite candidates to bring a piece of their own code into the company and pair on that with an employee. The idea behind this technique was to remove all elements of a “simulation” in the interview. The candidate was doing real work on his side project, but doing so alongside his potential co-workers.
Sales can also be fairly objective. What was your quota, and did you hit it? Why or why not? How did you rank against other sales people in your org at the same period in time? I’ve found that high performers in sales tend to continue to perform well, and that the opposite is also true.
Categories where evidence-based interviewing is harder include human resources, marketing (brand, not performance), business operations, and finance. Even if you have OKRs in a field like HR — say, employee retention — how do you know if employees stayed because the HR boss implemented some awesome new culture initiative, or because they were waiting for their stock to vest? The outcome is the same, but the root cause analysis is nearly impossible to conduct. Frustratingly, the more senior the hire, the larger the disconnect between performance and its objective measurement.
Case in point
Another CEO at our dinner, a former BCG consultant, suggested the case method as an effective a job simulation. The goal, he argued, was to get as close to an authentic interaction with the candidate as possible, and he found the case method to be quite effective.
For those of you unfamiliar with the case method of interviewing, here is Wikipedia’s description (emphasis mine):
A case interview is a job interview in which the applicant is given a question, situation, problem or challenge and asked to resolve the situation. The case is often a business situation or a business case that the interviewer has worked on in real life.
After the applicant is given information about the case, the applicant is expected to ask the interviewer logical questions that will help the applicant understand the situation, probe deeper into relevant areas, gather pertinent information and arrive at a solution or recommendation for the question or situation at hand.
Case interviews are mostly used in hiring for management consulting and investment banking jobs. Consulting firms use case interviews to evaluate candidate’s analytical ability and problem-solving skills; they are not looking for a “correct” answer but for an understanding of how the applicant thinks and how the applicant approaches problems.
Questions are generally ambiguous and require interviewees to ask questions or make assumptions to make a reasonable, supported argument to their solutions. Candidates are expected to demonstrate reasoning rather than to produce the exact answer.
Over several decades, the world’s top banks and consulting firms have converged on this method as the best way to evaluate talent with little prior signal, since most new hires are recent graduates. Its advantages are clear:
- Cases are inspired by actual business scenarios and are therefore as close as it gets to a simulation of real life. (I’m excluding the “How many ping pong balls fit in a Boeing 747?”-style question. I personally find those annoying and unproductive.)
- Cases get the candidate talking about a topic for which she cannot prepare. You get a lot more truth from people when they’re thinking on their feet, and the interaction is less artificial.
- Because there is no “correct” answer, the case method leaves open room for creativity and non-traditional thinking. The best candidates really shine in this light.
- The ambiguity of the case method highlights those candidates who exhibit clarity of thought and problem solving skills, which are essential for any job, but especially for management.
I can imagine some disadvantages too. Case interviews are notoriously hard to conduct. Without adequate training, most employees are likely to do it improperly, yielding a bad experience for the candidate and little usable information for the interviewer. In addition, some senior hires may balk at the idea of role playing — feeling like the entire idea is beneath their pay grade. You have to decide if that’s the type of person you want to hire.
Is the case method a good fit for evaluating talent at a startup? For junior hires in hard-to-measure roles, I’d say it’s worth trying. For senior hires, I’m not necessarily convinced, but a few CEOs seem to be honestly considering it.
If you’re currently using the case method in any capacity at your company, I’d love to hear from you and will update this post with anecdotes.
Authors
