When programs of study look to implement CASPer® as an assessment tool, they’re often doing so because they’ve seen what happens when students are not screened for skills and qualities like communication, professionalism and empathy.
More often than not, these programs seek out CASPer® because they’re looking for ways to eliminate the problematic outcomes they’ve experienced as a result of gaps in their admissions processes.
We often discuss the advantages of assessing non-academic competencies. Today, we’re discussing the consequences of not doing so. Read on to learn about the repercussions experienced by programs that don’t screen for people skills, and how tools like CASPer® can help address them.
Consequence: Problematic Students
This is the repercussion we hear about most: admitting applicants who turn out to be ill-suited and problematic.
Picking the best students is a difficult task for admissions offices and despite best intentions and efforts, it’s typical that one or two students get admitted who, perhaps, shouldn’t have been.
Sometimes it’s the fault of weak spots in the screening process. In other cases, it’s applicants who know how to “game the system”; they shine on paper and can survive a half-day of interviews without major red flags.
It may not seem like one or two problematic students in a class of 125 really makes a difference, but the difference can be huge. These are students who show up late to class, who don’t work well with their peers, and who lack professionalism. Academic affairs departments spend countless hours trying to deal with these students, which is a drain on program resources. For these ill-suited students, underperformance takes a toll on them too, emotionally, mentally, and even financially. They end up dropping out or dragging their feet through the program.
During the admissions process, grade point average (GPA) and standardized tests aren’t often accurate indicators of best fit. These problematic students are often amongst the brightest. Additional screening for non-academic competencies is required so that admissions teams can more accurately identify well-suited students earlier in the process.
Consequence: Lack of Confidence in the Screening Process
Popular modes of screening for non-academic abilities include personal statements and reference letters — tools that, as we have explored, lack robust evidence proving their effectiveness. In fact, literature has shown these methods to be unreliable, inconsistent and poor indicators of non-academic abilities. They may be helpful for learning about an applicant’s past, but they can’t be relied upon as accurate indicators of people skills.
Despite the lack of supporting evidence, both personal statements and reference letters are still widely used, perhaps because of an admissions department’s fear of change, belief in tradition or because up until about three years ago, there weren’t many accessible alternative screening tools available.
That’s not the case anymore. Admissions officers around the world are increasingly educating themselves on the psychometrics required to build tools that assess qualities like communication, teamwork, collaboration, ethics and empathy. Terms such as ‘reliability’ and ‘predictive validity’ are becoming commonplace in the world of admissions. Because of these changes, admissions teams who still choose to use tools like personal statements and reference letters may find themselves and their processes open to scrutiny and a lack of confidence.
Consequence: Lack of Diversity
There has long been an understanding that GPA and tests like the MCAT disadvantage individuals that are underrepresented in medicine (UIMs), but the extent was never known. New York Medical College conducted a study reviewing the impacts of GPA, MCAT and SJTs on the diversity of its applicant pool and, in 2016, introduced CASPer® to its admissions process.
NYMC received approximately 13,000 medical school applications (which represented about 20% of all medical school applicants in the U.S.). They looked at the number of UIM applicants included in that total, and compared it with the number of UIM applicants received by the school in 2015, before CASPer® was implemented.
From the results, Dr. Juster, associate dean of admissions at NYMC, observed the following:
- Amongst various income groups, CASPer® scores were equivalent;
- CASPer® scores for UIM applicants were lower, but the difference was much less pronounced than the differences observed in GPA and with MCAT;
- Applicants with paid employment before the age of 18 (considered an element of disadvantaged status) had higher CASPer® scores than those who did not have paid employment before the age of 18; and,
- Women scored higher on CASPer® than men, which is opposite to the gender differences observed with MCAT.
In summary: compared to GPA and MCAT, an SJT like CASPer® was found to be more effective in promoting diversity.
The repercussions for not screening applicants based on people skills is ever growing. Now that alternative screening methods such as situational judgement tests are emerging, we are seeing these problematic outcomes being addressed. This helps admissions officers, applicants and the professions for which these applicants will one day be a part of.
Published: March 7, 2018
By: Diana Ibranovic, M.A.
Marketing Manager at Altus Assessments