Looking Forward, Glancing Back: Our Journey Through 2018 and into 2019
Researching, educating, and developing. We spent 2018 focusing on reflection and growth, and the hard work paid off. We improved our process, helped our partner programs successfully integrate CASPer into their admissions process, and gave future students a platform to demonstrate their personal and professional skills.
Building on our base of strong partner program engagement and data analysis, CASPer, and our team at Altus Assessments, succeeded beyond our expectations in 2018. We provided testing for over 80,000 applicants, from 162 countries, helping a diverse group of applicants from around the world show that they are more than just book smart.
All of this growth is thanks to our dedicated team, and their leadership in supporting our partner programs, educating students and institutions, and researching what’s next. However, like any good researcher, we need to look at the data from the past before moving forward. So, come with us for our Altus Assessments’ 2018 year-in-review.
Great students are more than just book smart. Our assessment tool helps institutions assess their applicants’ people skills, but we also want to make sure those applicants feel confident in the process. Institutions are able to evaluate, and applicants are given a data driven platform to demonstrate their full skill set.
With 169 test slots, and a record single test sitting of 3877 applicants this year, we have a lot of data to work with. In 2018, we are very proud to say that applicants generally had a positive perception of CASPer before taking the test. However, those perceptions became even more positive after completing the test. That’s a milestone very few assessment tools can achieve.
Students found CASPer to be fairer, more enjoyable, less stressful, and more effective after completing the test. This shows that giving students the ability to demonstrate all of their skills is not only beneficial for the institution, but for the applicant as well.
We redesigned the TakeCASPer.com website to be much more user-friendly, help applicants find answers to questions about the CASPer test, and more importantly, how to prepare for it. We received positive feedback on the change and will continue to integrate these user experience learnings into other areas of the CASPer platform.
Understanding that applicants had a positive experience was just the first step in our goal to provide a great assessment tool. We also looked to improve our methods to make sure the test was easy for applicants to access and navigate and provided reliable, data-driven results institutions trust.
Situational judgement tests (SJTs) are one of the most commonly used assessments for employee selection, but we have proven they also provide meaningful information about candidates to academic programs and their possibility of future success. Our 2017-2018 admissions cycle gave us the data to show that our process is a reliable, feasible, and acceptable way to assess non-cognitive skills for healthcare and teachers’ education admissions as well as residency selection.
Using multiple methods to ensure assessment reliability, such as proctoring and webcam monitored testing sessions, the internal consistency of CASPer scores was exceptionally high. We smoothed out our online platform, with 92% of applicants reporting no technical issues during testing. And for those who did report technical issues, we were pleased to find the issues were attributed to their devices.
Most importantly was the value the students found in taking the CASPer assessment. Students were satisfied with the process overall, and valued the ability to demonstrate their personal characteristics for admissions.
As much as Altus Assessments believes that applicants need a platform to demonstrate their professionalism, personality, empathy, and other people skills, we believe that the tools to assess these skills need to be data driven. So, we are constantly researching and developing to make sure the data gives institutions consistent and reliable information on applicants.
Our 2018 run shows that from one testing occasion to the next, our scores remain reliable and reproducible. This is thanks to our amazing team of raters, made up of professionals, faculty, and individuals from special interest groups, who work with us to improve the rating process, timeline, and our focus on maintaining high interrater reliability.
CASPer raters go through a rigorous onboarding and training process to ensure the highest quality ratings that are on time and accurate. We worked diligently with our raters and it led to amazing results. Our goal for 2018 was to provide test result turn-around in three weeks, and we met this goal 100% of the time. As we continue to expand our team of raters, we hope to improve the efficiency of this process even more.
While our raters are able to produce reliable results on time, it’s thanks to our implementation of the Global-Z scores systems that increased our partners’ ability to use these results. By moving away from raw scores, and focusing on providing CASPer Z-scores, we were able to improve test score reliability and simplify the logistics process for our partner institutions.
Altus Assessments has always focused on educating institutions on the importance of people skills. 2018 was an incredible year in regards to this ongoing goal. From workshops about the importance of personal competencies for Engineering, to presentations on how situational judgement testing promotes diversity in Nursing, we have been busy delivering educational content that demonstrates the value of personal skills, and educates institutions on how to assess these skills.
Through these seminars, workshops, presentations, and reports, we have been able to demonstrate the high cost of medical residency selection, and how to decrease that cost and improve the process. We’ve shown our partners how open-ended questions on situational judgement tests improve the selection process by demonstrating the importance the “why” of the answer has instead of the “what”. And, most importantly, we’ve been able to demonstrate that the keys to applicant success are found in their professional and personal skills as much as in their academic results.
Altus Assessments started in medical school admissions, using CASPer as a reliable tool to identify applicants with the right personal qualities for medical training and to later become effective physicians. In 2017, we were happy to be expanding our skills and system into the realm of education where we are able to help institutions select candidates who will become effective teachers.
2017 saw Victoria State in Australia adopt the CASPer testing system and have continued its use through 2018. They saw the importance of selecting applicants who bring value to their institutions, and to the professional sphere of education in Australia. Like medicine, where 95% of patient complaints are related to professionalism and not knowledge in the field, problems with teachers are rarely related to their teaching ability. CASPer can now help institutions identify candidates who will be amazing educators academically and professionally.
Partner Program Success
As mentioned above with our use of Z-scores and improving our turn-around period for results, we spent 2018 working with our partners to improve their experience with CASPer and show them how to use our data-driven results to increase the quality of their applicant pool.
We developed webinars coaching pre-health advisors on how to prepare applicants for CASPer. We also delivered value by working with our partner programs on research initiatives. This provided them the opportunity to share their research findings through reports to their committees, in journal articles and at education conferences around the globe.
Speaking of research initiatives, our very notable milestones are anchored around the 24 accepted submissions to education conferences we completed in 2018. These submissions included presentations, workshops, symposiums and posters, all focusing on different areas of the admissions process and how it can be streamlined and improved. This success has provided us with the opportunity to facilitate several discussions in the admissions space contributing to, and often leading, the overall conversation happening amongst the research community.
Looking to 2019
As great as 2018 was, we never stop looking for ways to make our process more effective for applicants and institutions alike. 2019 will see developments in our Altus Assessments website, TakeCASPer account platform for applicants, applicant support and The Admissions Summit.
We are redesigning the Altus Assessments website to provide useful resources that programs visiting our website can use when assessing their own admissions processes. The website will include research updates as well as taking a closer look at the Altus Assessments team. On top of this website overhaul, we are improving the user flow for applicants signing up to take the CASPer test on the account platform and we will be working to update our support resources so that applicants are equipped with the information they need even faster! This will help address questions and concerns applicants may have from registration to results distribution.
The Admissions Summit will be our way of giving back to the admissions community. This two-day conference, focused on navigating healthcare admissions challenges together, is being held in Toronto on June 10th and 11th, 2019. We hope to see many admissions professionals gather to network, learn, and share their experiences.
The milestones and growth that we were able to achieve in 2018 are a direct result of the amazing talent of our Altus Assessments team. Our researchers prepared presentations and results to explore and aide the field of admissions. We worked diligently with partner institutions to improve their process and our own. We came together to improve the functionality of CASPer on every level. And, most importantly, we had an amazing time doing it!
At Altus Assessments, we continue to surprise ourselves with how quickly we are expanding and evolving. It’s thanks to the amazing people who built our foundation, and the top-talent we brought on as we grew that made this an effective, driven, and enjoyable place to work in 2018.