This commentary outlines the 6 factors that endanger the replicability of medical education research: Linda calls it a must read for all #meded researchers!
Click here to listen to her discuss it with her co-hosts.
———————————————————————–
Reference
Hope, Dewar, Hay. Is there a Replication Crisis in Medical Education Research? Acad Med. 2021 Mar 16: Published Ahead of Print.
Reviewer
Linda Snell (@LindaSMedEd)
Background
In fields ranging from cancer therapy to psychology only 15-35% of sometimes ‘landmark studies’ could be replicated.
Medical education research is important: it contributes to policy and influences practitioner behavior. As in other fields, if findings cannot be replicated it suggest the original research was not credible, and may lead to unhelpful or even harmful changes to medical education.
But what if you were told that up to half the results of studies in med ed could not be reproduced? The implications are scary – we teach, students learn, they apply it in practice, and there are patient outcomes.
Purpose
The authors “consider a major threat to the integrity of research in many fields, explain and provide examples of the risk it poses to medical education, and offer practical advice, again with examples, on what medical educators and clinicians can do about it.”
Key Points on the Methods
None, this is a commentary.
Key Outcomes
6 factors that endanger the replicability of medical education research:
- small sample sizes, not represented, underpowered
- small effect sizes, statistical vs education significance
- exploratory designs, ‘data mining’
- flexibility in design choices, analysis strategy, and outcome measures, possibly chosen post-hoc
- conflicts of interest, lack of funding
- very active fields with many competing research teams, with pressure to publish early.
Therefore it is plausible that MER is experiencing a replication crisis.
Key Conclusions
The authors conclude… research opportunities to study interventions involving the development of AE. Gaps in the development and validation of measurement tools.
Spare Keys – other take home points for clinician educators
All researchers in med ed should read this!
Access KeyLIME podcast archives here
The views and opinions expressed in this post and podcast episode are those of the host(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page