In a recent article published in The New York Times, Jason Dana, a professor at the Yale School of Management, argued that job interviews are “utterly useless” in identifying the best candidates for a job.
He had students interview other students and then predict their grade point averages for the following semester. The prediction was to be based on the interview, the student’s course schedule and his or her past GPA (they explained to interviewers that the past GPA was historically the best predictor of future grades at their school). In addition to predicting the GPA of the interviewee, the subjects also predicted the performance of a student they did not meet, based only on that student’s course schedule and past GPA. In the end, the GPA predictions were significantly more accurate for the students they did not meet. The interviews had actually been counterproductive.
While it is provocative to suggest that law firms should do away with interviews, that isn’t the correct conclusion: Research is clear that poorly planned, unstructured interviews are poor predictors of on-the-job success. It is proven that objective candidate evaluation methods work much better than the typical interview to establish competence.
While efforts have been made to train lawyers to interview and create a process that yields better candidates, the process has remained relatively unstructured for most firms. Unstructured interviews are just that — there are free-flowing questions with no consistency between different candidates and no rating scale. While firms might favor these interviews because they are easier (due to the fact that interviewers can “wing it”) and they allow interviewers to “get to know” candidates better, these same features make this type of interview subjective, less accurate, open to legal challenges and less effective for analyzing skills and competencies.
In a structured interview, all candidates are asked the same questions in the same order, all candidates are evaluated using a common rating scale and all interviewers agree beforehand on acceptable answers. Structured interviews vastly improve the odds that candidates have equal opportunities to provide information and are assessed in a consistent manner.
Research also shows that law firms that hired recent law school grads with high scores on their structured interview were 11% more likely to stay with the hiring firm for more than four years. When it came to lateral hires, that difference was 14%. Of course, all of this sounds great in theory, but how do you actually create an effective structured interview process?
1. Determine core competencies to be assessed during the interview.
Firms first need to understand what they are looking for in candidates and develop a list of characteristics or capabilities that differentiate them strategically from others — these are your firm’s core competencies. Successful performers will exhibit behaviors in line with those core competencies. You can identify these competencies by reviewing your mission statement, looking at attorney reviews and thinking through what your business does. Some sample core competencies include innovation, quality, efficiency, client service and teamwork. Start with what your firm does and what capabilities clients’ value.
2. Choose the interview format and develop questions.
Once you’ve determined your core competencies, the next step is to develop questions and an interview process that will determine whether the candidate’s attributes align with your core competencies. Behavioral-based interviewing is the most effective way to do this. Behavioral interviews use past employment-related situations to predict future performance. So, instead of asking candidates how they would behave, ask candidates how they did behave. In this case, interviewers look for information on how a candidate handled a situation, instead of learning what a candidate might do in the future. With this in mind, develop behavioral-based questions to determine whether a candidate possesses the competencies you’re looking for.
It is very easy to fall back on the tried and true questions that are typically asked, like “Tell me about yourself,” “What are your strengths?” and “Where do you see yourself in five years?” These questions, unfortunately, are largely ineffective. Again, the ideal questions are designed to elicit whether a candidate exhibits your firms’ core competencies.
3. Develop rating scales to evaluate candidates.
The evaluation and feedback step in the process is where the “structure” part generally falls apart. It doesn’t help to develop core competencies and corresponding behavior-based interview questions if you haven’t developed a structure for how feedback is delivered after the interview. Creating a defined rating scale and objective evaluation process is integral to an effective structured interview; not having one allows too much room for bias and judgment.
While upfront work is required to develop effective rating scales and agreed-upon good and bad responses to interview questions, these additional steps are essential to sustaining an effective interview process. When developing a rating scale, law firms should identify positive and negative traits that demonstrate whether a candidate is proficient in a core competency, then come up with example behaviors that do or don’t exemplify the competency and, finally, evaluate each candidate based on that rating scale. This will create a standard evaluation process across all interviewers.
4. Create a method to solicit feedback — a candidate evaluation.
Once you have created your scale, then develop a candidate evaluation form that allows interviewers to give their feedback. The interviewer should immediately rate each candidate after their interview based on the rating scale you constructed and then make a recommendation as to whether they approve an offer. This can be as simple or detailed as fits the culture of your firm, but the important thing is that interviewers are evaluating candidates on core competencies and in accordance with what you have predetermined are acceptable answers.
5. Create an interviewer’s guide and train interviewers.
Interviewing isn’t an easy process, and interviewers need guidance for how to do it correctly. You will need to train your interviewers on your new interview process. It’s best to create a detailed interviewer’s guide so that attorneys have a roadmap to use during the process. The interviewer’s guide should provide general instructions about the interview process, a summary of common rating biases and rating mistakes to avoid, and general tips for good interviewing. The guide should also provide information specific to the particular interview, including definitions and proficiency levels of each competency being assessed, interview questions to ask and a sample of the rating scale. Training will standardize the process and make sure everyone is following the new protocol you’ve established.
Identifying qualified candidates that align with your firm’s core competencies is essential to the growth and success of your business. While the competitive candidate market can make anyone eager to relax their interview process in favor of easing the experience for candidates, firms should think about the long-term benefits of choosing the right attorneys. Structured interviews will ultimately provide a consistent and value-aligned rating system that best positions firms for hiring success.
