After the completion of our inaugural OCI season, we are excited to share that 5 pilot law firms, including Cooley, Katten, Shearman & Sterling, Holland & Hart, and Proskauer, launched their own custom and distinct filter of Thine’s assessment in connection with OCI, and over 2,500 students from 70+ law schools completed it.
We recently hosted an interactive Q+A webinar with some of our inaugural clients to speak about their experience deploying the assessment. Key topics included how they built internal consensus and overcame objections, when they administered the assessment, student reaction, and ways in which they used the information.
From that conversation, five key takeaways emerged.
Involve senior stakeholders early to overcome potential objections, expedite adoption, and increase buy-in around use of results
Introducing new technology into law firms can be challenging. For Thine’s pilot group of law firms, getting senior stakeholders involved in the process early was key. Allowing them an opportunity to provide their opinions in early conversations with Thine helped inform the build out of the firm’s custom assessment filter. The enthusiasm of these early adopters proved valuable in getting the buy-in of other members of the firm.
Before introducing the idea of an assessment to a larger audience, it was important to educate the key stakeholders on the science behind the assessment and get resolution around internal objections, deployment plans, and candidate messaging to avoid getting derailed. Common objections included concerns that law students would all answer similarly or that students would be able to guess a “correct” answer. Each firm designed their own unique filter, which resulted in a wide distribution of results. Seeing the variety of assessment responses allayed concerns about students’ ability to game the system.
Another concern was that the assessment could introduce bias into the recruitment process. However, Thine’s assessment has been rigorously tested to ensure that it has no statistically significant adverse impact on legally protected demographic groups. The use of big data validation, empirical testing, and EEOC compliance helped recruiting committees see that the assessment would actually remove bias rather than introduce it.
Creating the firm’s assessment filter was straightforward and allowed for diverse perspectives to be reflected
Firms used profile creation as a key way to gain internal buy-in and increase the likelihood that recruiting teams embraced the results.
Thine’s process allowed lawyers from different practice areas and experience levels, as well as business service professionals (recruiting, professional development, DE&I) to work together to determine the personality traits and competencies most important at their firm. Professional development teams provided valuable insight around the prevalence of certain traits and competencies in existing lawyers at a given firm.
The process was streamlined and took most firms between 3 and 4 hours to complete. Firms found that there was a lot of internal consensus around the personality traits. Firms emphasized the importance of choosing traits that are relevant but that are also differentiators. As expected, each firm’s filter was distinct.
Firms paid special attention to trends that were either universally low or high for applicants and will adjust their filters accordingly when preparing for next year’s recruitment season.
Deploying the assessment earlier in the process increases potential uses for the results
Firms deployed the assessment at different times during the OCI cycle. Most pilot firms asked students to complete the assessment before their callback interviews. One firm deployed the assessment before OCI interviews and for all write-in candidates, which allowed the firm to interact with students at more law schools than they historically would have. Going forward, most of the pilot firms expect to deploy the assessment earlier in the process to provide greater flexibility in the number of students included and how they use the results.
Most students completed the assessment without needing a reminder. As a result, it did not cause delays in the recruiting schedule.
Students were willing to take assessments and excited to see an additional data point that provides a more holistic picture
Assessments are relatively new for students going through OCI, so there was an initial curiosity about what they measured and how the information would be used. As more firms rolled them out, students became more comfortable taking them. Students communicated an appreciation that some firms were open to changing up the OCI process, and looking beyond grades and law school ranking.
“[Students] mostly commented on their appreciation that the firm is endeavoring to be more equitable in its evaluation of candidates.”
- Pilot Participant
Firms also thought that introducing students to competencies early on would create a comfort level that will be helpful throughout their careers. The report that students received around their top strengths and developmental areas allowed them to learn from the assessment and have more meaningful discussions with potential employers around the different ways they might align.
The assessment helped firms reach students and schools that they hadn’t in the past
Firms who deployed the assessment early in their process were able to cast a wider net and see a larger and more diverse applicant pool. In this way, the assessment was used as an additional data point for students and wasn’t used to exclude candidates from consideration. For candidates that might have historically been discounted because of grades or law school ranking, the assessment provided firms with another basis for determining whether to extend an offer or not.
“What I love about this tool is it's just, again, that additional data point that you might need to get somebody over whatever might be on a transcript or something else that is holding a hiring team back.”
- Pilot Participant
About Thine’s Assessment
Thine's personality assessment measures a candidate’s traits and competencies. When a candidate completes the assessment, those results are filtered through a firm’s unique filter, which has been customized based on their hiring preferences. Law firms receive a report for each candidate highlighting how closely aligned they are with a firm’s preferred filter.