How One of America’s Top Pediatric Hospitals Is Changing Its Graduate Medical Education Admissions for the Better

With a medical staff of over 1,000 specialists, Phoenix Children’s Hospital in Arizona is one of the largest pediatric healthcare centers in the U.S. To grow its services and expand its coverage, the hospital actively looks for standout promising physicians for its graduate medical education program.

The process of finding and admitting the next cohort of world-class physicians isn’t easy. Sometimes, it’s even flawed. The problem with many admissions processes is that they are “very old school in how they do things,” says Dr. Brig Willis, the chief medical education officer and the designated institutional official at Phoenix Children’s Hospital.

In the hospital’s graduate medical education programs, the admissions team looks at academic metrics that include information provided by medical schools, a dean’s letter and evaluative documents, usually in the form of letters of recommendation. These metrics pose a few issues.

One of the issues pertains to dean’s letters, which Dr. Willis says can be “controversial entities” due to the use of a code-word system. He explains that while these letters are “supposed to be summative evaluations of [applicants’] medical school careers outside of grades,” many schools will use a keyword near the end of the letter as a way to slyly indicate what academic quartile students fall within. Stating it plainly isn’t allowed. And because some schools refuse to use the code-word system at all, there’s a lack of consistency that makes the comparison of letters difficult.

Then there are traditional candidate interviews, which often don’t reveal accurate information about the quality of a candidate and end up costing both applicants and hospitals thousands of dollars.

Another challenge that the hospital’s graduate education admissions process has faced is within the application process for its fellowships, which utilizes a letter of recommendation instead of a dean’s letter. That means the admissions decision is largely based on grades, the United States Medical Licensing Examination (USMLE) scores and the content of the recommendation, which amounts to an insignificant amount of hard data to review.

Together, these metrics fail to capture the full picture of not only a candidate’s intelligence but also their professionalism and ethical competency, which are key qualities of good physicians. It’s these inefficiencies and challenges that inspired Dr. Willis to take a look at a more reliable admissions tool —  the Situational Judgement Test (SJT).

Dr. Willis had begun developing his own SJT before he began working in graduate medical education. Then he came across CASPer and found it was a “more mature version” of what he was trying to build. “The concept is something I’ve been an advocate of for years,” says Dr. Willis. “We need to get at those qualities that actually matter most in selecting whether someone’s going to be a good physician, which are empathy, ethical decision-making, professionalism, and [etcetera].”

Thanks to Dr. Willis’ advocacy, CASPer has been introduced to Phoenix Children’s Hospital’s admissions process, enabling the admissions team to make more informed, holistic decisions about applicants. Of particular interest to Dr. Willis is CASPer’s ability to help enhance the diversity of admitted students, without directly screening for race, gender and socio-economic background; by reducing the reliance on cognitive metrics which are known to disadvantage applicants from certain backgrounds. Enhancing diversity is one of the hospital’s “big pushes” for its graduate education programs.

The admissions process is crucial to improving the quality of healthcare systems. CASPer aims to support that process by making it more equitable, accurate and effective. “If you can have a pool of [students] who are more mature, more professional, more ethical, the entire quality of your program goes up,” says Dr. Willis. “It’s going to improve patient satisfaction, it’s going to improve teamwork in the hospitals, patient outcomes … it’s a big deal.”