Two doctors were found guilty of torturing patients at an Ontario maximum-security mental health facility, which housed an estimated 1,240 patients over a 17-year period. The techniques used included solitary confinement, the administration of hallucinogens and delirium-inducing drugs, and brainwashing methods developed by the CIA.
That same month, a former nurse was sentenced to eight concurrent life terms for using insulin to injure and kill nursing home residents in southwestern Ontario between 2007 and 2014. Her actions ultimately resulted in eight deaths.
Cases like these are, fortunately, quite rare, but they do highlight how essential it is for our medical system to ensure that healthcare workers are adhering to the highest ethical principles while also providing compassionate and empathic care. Research has shown that for physicians the majority of disciplinary actions by medical boards are attributed to issues of professionalism.
Doctors who are empathic and warm are perceived to be more competent; from a patient’s perspective, competency and empathy go hand-in-hand. In addition to a correct diagnosis, a proper treatment plan and medical knowledge, patients also value whether doctors listen and care, have patience, take them seriously, and treat them with respect and dignity.
Healthcare professionals are required to pass quality checks to ensure that they have both the knowledge and the personal qualities needed to do their jobs well. The biggest hurdle they must clear happens at the very beginning of their careers — being selected by a medical program.
Medical programs are the first major filter to shape the kinds of people that end up working in the healthcare system. It makes sense, then, that the medical education system should reflect the needs and the wants of the people who use the system, by selecting the smartest and the most compassionate students and then training them to become the kinds of doctors and nurses who treat patients with kindness and respect.
Focusing efforts on improving these professionalism traits in-program show mixed results. Undergraduate medical programs strive to foster the interpersonal skills and empathy of their students, but evidence suggests their efforts aren’t entirely successful. Research shows that student empathy actually declines during the third year of medical school, when the curriculum shifts to patient-care activities — a time when empathy is the most essential.
So why the decline in empathy? It’s been attributed to a number of reasons: an intimidating educational environment, negative educational experiences, sleep deprivation, experiences of bullying in medical school and the promotion of programs to focus excessively on science and inadequately on the art of patient care.
These concerns about how our medical education system is structured are legitimate, but there might be fewer concerns if medical programs were selecting students who already are highly compassionate and empathic. But are they?
Medical programs typically implement a two-step screening process: a pre-screen to select which applicants get invited to interview, and a second screen to select which students receive an offer of acceptance after the interviews are completed. It is not feasible to interview every applicant as medical programs receive far too many applications, so the pre-screening stage is crucial in reducing the applicants to a manageable amount for interview. Interviews are costly for both applicants and administrators, so programs want to make sure that they are inviting their best candidates to take up each valuable spot. The average acceptance rate into medical school in the United States is less than 6%, with a bulk of applicants failing to pass the pre-screen stage, so this is the first major filter in shaping our healthcare professionals.
The goal of the prescreen is to ensure that candidates are well-rounded and possess both a strong foundation of medical knowledge and the necessary personal and professional competencies in becoming an effective physician. Most medical schools pre-screen applicants using scores from their MCAT and undergraduate GPA as indicators of their scientific knowledge and cognitive aptitude, and research shows that they do a great job in predicting the academic performance of medical students.
The assessment of personal and professional competencies have been more of a challenge, as many of the tools that are targeted to evaluate these traits have not been very successful. Tools like reference letters and personal statements have been found to be unreliable and provide little utility in predicting any sort of important outcomes in medical school. Many qualified students don’t even get a chance to showcase their personal qualities as they get filtered out by the excessively high academic requirements — with some schools having an average acceptance GPA of over 3.9!
We need to find another, better way to assess prospective students. The good news is that medical programs and their governing bodies have realized this and have invested more resources into finding better methods for assessment of the “non-cognitive” (also known as “non-academic” or “personal/professional”) characteristics of their applicants. A number of tools have been developed to do just that, with varying degrees of success:
The Personal Qualities Assessment (PQA) was developed by a team of researchers in Australia to measure both the academic abilities and personal qualities of applicants. The test consists of three sections: a mental agility test, a moral orientation test and the NACE scale, which assess levels of narcissism, aloofness, (self-)confidence and empathy. Although the scale demonstrates good psychometric properties, the test has limited ability to predict professionalism outcome measures.
The MOR (Hebrew acronym for “selection for medicine”) is an innovative assessment centre developed in Israel to assess candidates’ personal and interpersonal attributes. The test is quite extensive and is comprised of three tools: a role-playing challenge where candidates go through a group of behavioral stations that simulate a challenging encounter with patients, a series of essay-writing tasks that involve real-life ethical dilemmas and a biographical questionnaire aimed at assessing a candidate’s past experiences and emotional awareness.
MOR has shown some promising results: it was found to predict students’ first-year GPAs in medical school, which is comprised of courses that mostly focus on factual knowledge but also teach some interpersonal skills. But implementing MOR is a challenge due to its cost: the test is very expensive to administer, requiring many test officers, numerous evaluators and a central location to house all the applicants. There is also a heavier burden on students since they are required to travel to the assessment centres.
The UKCAT text-based Situational Judgement Test (UKCAT-SJT) is a situational judgement test (SJT) designed in the UK to test applicants’ personal characteristics, such as perspective-taking, resiliency and team involvement. The test is comprised of 68 multiple-choice questions where students are presented with a series of hypothetical scenarios. In one format, they’re asked to rate the appropriateness of a series of responses, and in another format, they’re asked to rate whether a particular concern is something important to take into account when navigating the situation.
A number of studies have demonstrated good psychometric properties of the UKCAT-SJT, and it was found to predict supervisor ratings of performance in tutorials amongst medical students. At the moment, the UKCAT-SJT is combined with the UK Clinical Aptitude Test (UKCAT), which is only administered in UK test centres. This makes it difficult for rural applicants to access the test, and for medical programs outside of test centre areas to adopt the test.
CASPer® is another SJT, designed at McMaster University in Canada, to evaluate the personal and professional competencies of medical school applicants. CASPer® is currently being utilized by a number of medical programs in the United States, Canada and Australia, with 100,000 applicants having taken the test by the end of this application cycle – representing about 50% of all medical school applicants in the United States. There are 12 sections on the test: eight hypothetical scenarios presented in video format, three probing questions that students answer in an open-ended format, and four behavioural description questions where students must describe a specific situation from their history and answer a few follow-up questions about that situation.
Studies have demonstrated good psychometric properties of CASPer®, and it’s been shown to predict performance on national licensure examinations of personal and professional characteristics three to six years after admission to medical school. It’s even been proven to predict applicant rankings for surgery residency. CASPer® is designed to allow students to take the test in the comfort of their own homes, minimizing the costs for both medical institutions and applicants and widen access for ethnicity and economically diverse applicants.
Medical programs seem to be doing an excellent job of selecting the smartest applicants, but not the most compassionate. With the spreading adoption of tools like UKCAT-SJT, MOR, and CASPer®, medical programs seem to be heading in the right direction, but they are still part of the minority and not the norm. The right tools can help these institutions pick the most well-rounded candidates, the ones who go on to serve patients with empathy and compassion at hospitals and clinics. While most of us would prefer to be treated by someone like Dr. Oz, with the current state of the medical admissions process, we’re more likely to be treated by someone like Dr. House.
Published: March 22, 2018
By: Christopher Zou, Ph.D.
Education Researcher at Altus Assessments