This week the Key Literature in Medical Education post examines a controversial topic (myth?) regarding the clinical performance of international medical graduates when compared to domestically-trained physicians. Read the abstract below for more details.
To hear the actual podcast, subscribe on iTunes here OR stream the podcast at our new webpage here.
As always, we’re interested in your comments on the show + recommendations for future articles.
Anywhere, anytime, KeyLIME.
– Jonathan (@sherbino)
KeyLIME Session 69 – Article under review:
View/download the abstract here.
Norcini J, Boulet JR, Opalek A, Dauphinee WD. The Relationship Between Licensing Examination Performance and the Outcomes of Care by International Medical School Graduates. Academic Medicine 2014, 89 (8): 1157-62
Reviewer
Jason Frank (@drjfrank)
Background
IMGs (International Medical Graduates) make up about 25% of the workforce in both the US & Canada. Some authors have identified concerns that IMGs may not perform as well as domestic graduates, so some jurisdictions have required IMGs to write multiple national examinations to assess their competence. In the US, IMGs are asked to complete several assessments, including the USMLE Step 2 Clinical Knowledge (S2CK) exam. What is not known is if this assessment correlates with IMG performance in practice?
Purpose
These #meded gurus (Norcini, Boulet, & Dauphinee are assessment giants with an interest in assessment, especially of IMGs) set out to determine if IMGs USMLE S2CK scores had any relationship with patient outcomes.
Type of paper
Research: validation study; retrospective observational study
Key Points on the Methods
The authors correlated the S2CK scores of 2535 IMG practicing physicians in Pennsylvania, USA with the in-hospital mortality of their 60,958 admissions for ACS or CHF between 2003-2009. The authors say they chose these two cardiac conditions because they feel they are common, are treated often by IMGs, and have important mortality rates. The data was from the Pennsylvania Health Care Cost Containment Council, which excluded Veterans, psychiatric, and nursing facilities. Patients with high acuity on a statistical index were excluded, as were those with missing data, etc (~15%). The authors took pains to control for patient characteristics, facility, case volume, and time from graduation. The 173 facilities had 4 to 18,262 hospitalizations. The physicians came from 48(!) different practice specialties. Multivariate analysis and some other fancy stats were used to model the impact of the IMGs on case mortality.
Key Outcomes
In this study, after some statistical adjustments for case and physician characteristics, each point on the S2CK was correlated with a 0.1-0.4% decrease in mortality. Each standard deviation change in score (20 points) was associated with 4% change in mortality risk.
Once again, these authors found that better outcomes correlated with practice volumes (each additional hospitalization treated lowered patient mortality 0.1%!). Lower mortality was also associated with MD board certification.
Key Conclusions
The authors conclude that this is yet another study that shows the predictive validity of well-designed high-stakes national exams.
Spare Keys – other take home points for clinician educators
This is another powerful study that belongs in every clinician-educator’s “meded outcomes” folder.
1. Competence is as competence does. Once again, we see the predictable association between practice volume and patient outcomes.
2. This paper foreshadows the 21st century era of meded literature, in which we will be asked to show the connections between features of our system and patient outcomes.
3. It is also another example of high-quality outcomes literature, along with works by David Asch and Robin Tamblyn. These are landmark papers showing that physicians in the North American environment can often be stratified based on their characteristics correlating with hard patient outcomes (eg mortality risk of 4% with each SD).
Access KeyLIME podcast archives here
