By: Claire Touchie (@Drctouchie)
Conflict of Interest: Dr. Touchie was the Chief Medical Education Officer for the Medical Council of Canada, the organization that develops and administers the licensing examinations in Canada.
In North America and in other jurisdictions, there are standardized high-stakes point-in time assessments as pre-requisites for licensure and certification. These are usually in place to assure the public that a minimum standard of knowledge plus or minus skills have been attained. Think US medical licensing examinations (USMLE) or the Medical Council of Canada qualifying examinations or the certification exams of the Royal College of Physicians and Surgeons of Canada. Licensing authorities use the information generated from these examinations in order to make licensing decisions.
A recurring question comes up: Are these examinations still necessary in the era of CBME? Fair question. Schools and residency programs are busy implementing CBME with curricula designed around desired outcomes and a program of assessment guiding the learners to reach these goals. Students and residents are presumably being observed more frequently, provided with feedback to continuously improve and there should be some sort of final attestation by their schools/programs that they have met the said outcomes required to move on.
So why continue with these high-stakes point-in-time exams? Arguments for such examinations include that these exams set expectations for learners and provides standardized information that schools/programs may not provide licensing authorities at the individual level (e.g., that they have failed or are struggling in certain areas). There are studies demonstrating that weak or failing candidates on these exams also have a higher likelihood of practice issues such as complaints and mis-prescribing than those who succeed or perform better1. Arguments against standardized exams are that they are expensive and that the vast majority of ‘home-grown’ candidates pass these exams (reference). Depending on the examination blueprint and design, they may actually take learners away from what programs consider important to learn.2
Aren’t local program of assessment sufficient? Programmatic assessment is described as an arrangement of assessment methods planned to optimize its fitness for purpose3. Assessments must be fit for purpose and medical schools, for example, do not necessarily have the same intended purpose as licensure. In fact, when you have reviewed one medical school in North America, you have reviewed one medical school. Although there are stringent accreditation standards, each school serves a unique community, faculty and learners. Assessments vary, standards for these assessments vary, university policies on assessment types and grading systems vary. So how can the public be assured that a student from school/program A has reached a level similar to school/program B? In addition, at least in Canada, not all programs have adopted CBME with rigorous programs of assessment yet.
So has the time come to discount high-stakes point-in-time examinations over programs of assessments? The answer should not be ‘either/or’ but should consider both. In fact, high-stakes point-in-time examinations should be part of a robust program of assessment. In keeping with the spirit of programs of assessments and assessment for learning, it is time that high-stakes examinations provide meaningful feedback to learners and to programs to encourage further development of identified weaknesses and to help programs help learners. Not just for those who have clearly not met the standard but to all in the name of continuous improvement. Medical schools, residency programs, certification and licensing examination organizations must work together to align the various assessment strategies such that they make sense in the learning continuum and provide meaningful feedback to foster ongoing learning.
So, no, I don’t think that time’s up for high-stakes point-in-time examination. But I do think that it is time to rethink them so that they complement other types of CBME assessment approaches for the benefit of learners, programs and ultimately the patients we look after.
About the author: Claire Touchie, MD, MHPE, FRCPC is a Professor of Medicine at the University of Ottawa, Ottawa, Ontario, Canada
References:
1. De Champlain A, Ashworth N, Kain N, Qin S, Wiebe D, Tian F. Does pass/fail on medical licensing exams predict future physician performance in practice? A longitudinal cohort study of Alberta physicians. J Med Reg 2020;106:17-26
2. Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student perspectives on the “Step 1 Climate” in preclinical medical education. Acad Med 2019;94:302-3041
3. Van der Vleuten CPM, Schuwirth LWT. Assessment of professional competence: From methods to programmes. Med Educ 2005;39:309-317
