The process of admitting students to high-stakes professional programs is no easy feat. A dedicated team of school faculty and staff is often required to manage and coordinate the entire process. These teams sift through and review hundreds — sometimes thousands — of applications, all from qualified candidates. It’s a long and intensive job.
Dr. Michael Roscoe knows this well. As founding director, chair, and associate professor of Physician Assistant (PA) Studies at the University of Evansville in Evansville, Indiana, he oversees many facets of the program, including admissions.
For Roscoe, creating the program was an opportunity to apply his 15+ years in academics and 7.5 years of previous director experience. That included using the Multiple Mini-Interview (MMI) for assessments, which he had used for professionalism screening at his previous institution.
“The MMI we use [at Evansville] has six 10-minute non-clinical scenarios and one 20-minute group interview scenario, which has been very successful,” Roscoe says. “But the problem doesn’t lie there. It lies in the way we decide what applicants get the opportunity to interview.”
Grade point average (GPA) and graduate record examinations (GRE) are used to determine whether a candidate should proceed to the interview stage. Non-cognitive tools, such as personal essays and reference letters, are also often used to provide a more in-depth look at the applicant, but Roscoe hasn’t found either tool to be useful. When he arrived at Evansville, he worked with the faculty to reconsider the use of these tools. In order to spearhead the change to the admissions process, he had to prove to the faculty the non-cognitive tools did not work.
Roscoe had his faculty grade the personal essays of a pool of real applicants, to help determine which of those applicants would be selected for an MMI. Based on a combination of GPA, GRE, and the essays, his team chose 78 applicants to take the MMI, only to find that about 20 of those candidates performed extremely poorly. Despite being very smart, these candidates were unable to demonstrate the skills Roscoe and his faculty were probing for in the MMI stations.
How did these 20 or so applicants make it to the MMI stage? Because of their essays. The essays were used to make decisions about candidates, yet they did not predict any value in MMI performance.
To further prove the ineffectiveness of the essays, Roscoe took it upon himself to conduct a small case study to prove this point further. He went back and brought in two applicants that met all the academic thresholds but didn’t rank high enough to make it to the interview stage initially. He reviewed their applications again, taking a look at portions of the application that weren’t weighed, such as communication throughout the process and extracurriculars. These individuals stood out to him and he had a feeling given the opportunity to take the MMI, they would do great. Turns out, he was right.
This was the “ah-ha!” moment that helped Roscoe convince his faculty to use a different non-cognitive evaluation before the interview, one that was reliable and would provide some sort of correlation to future performance.
Roscoe’s goal was to ensure that this evaluation was meaningful so that the limited interview opportunities weren’t given to candidates who didn’t have the non-cognitive performance necessary. He had previously researched non-cognitive screenings and heard about the CASPer® test through the Physician Assistant Education Association (PAEA) and the National Commission on Certification of Physician Assistants (NCCPA).
Roscoe found that CASPer® offered the defensibility he was looking for and brought it to his faculty for consideration. After a presentation with the CASPer® team, Roscoe and the faculty were convinced. CASPer® will be implemented in the Evansville PA program starting April 2018.
Published: January 4, 2018
By: Diana Ibranovic, M.A.
Marketing Manager at Altus Assessments