During this week’s KeyLIME podcast session, Jason Frank grapples with the question: “What is the most effective way to prepare senior medical students for the transition to residency?” As of late, preparatory courses, especially so-called Surgical Boot Camps, have become a trend. This cohort study attempts to fill some of the gaps in our understanding of the effects of such courses on learning and performance in residency.
Check out the podcast here (or on iTunes!)
————————————————————————–
KeyLIME Session 156:
Listen to the podcast
Read the episode abstract here.
Wunder JA, Brandt CP, Lipman JM. A surgical residency 
preparatory course for senior medical students leads to earlier independence in ACGME competencies. Am J Surg. 2017 Nov 8. [Epub ahead of print]
Reviewer: Jason Frank (@Drjfrank)
Background
When do the most junior trainees become competent for specific abilities? My Chief resident’s answer years ago was “…when the senior residents go home…” What is the most effective way to prepare senior medical students for the transition to residency? For this, there are many answers and efforts and failures at medical schools worldwide. Recently, a trend has been the growth of special preparatory courses dedicated to those going into a certain type of residency, especially so-called Surgical Boot Camps. Previous literature has shown that these improve trainee confidence and scores on knowledge tests, but the effects rapidly fade (Antonoff, Acad Med 2012). What is not known, is whether these intensive investments in preparation for junior residency training provide a kick-start to learning in PGME, or lead to earlier independence.
Purpose
This study attempted to fill some of the gaps in our understanding of the effects of such courses on learning and performance in residency. Wunder et al, from Case Western Reserve University in Cleveland, USA set out to determine whether “participation in a surgical residency preparatory course…leads to earlier unsupervised…performance than those who have not participated in such a course.”
Type of Paper
Research: Observational; Cohort Study
Key Points on Methods
Kudos to the authors for taking advantage of a natural difference among surgical residents affiliated with their Department of surgery, in terms of observing those who had taken the local surgical boot camp or not. Their protocol involved an open invitation to any surgically-interested local senior med student to sign up for a 1-month elective offered annually. 9 enjoyed the intensive curriculum, then agreed to participate in the study (“Group A”). These were compared to two controls. “Group B” were local students who did not sign up for the prep course but matched to surgery, and “Group C” were those who matched to surgery from outside the local region and did not do a boot camp.
All participants were surveyed every 2 weeks for 1 year asking to self-report when they first performed a defined list of procedures and management competencies a) with supervision, and b) unsupervised. The list was from the ACGME Intern Competencies list and included such items as running an ACLS code and performing dialysis access. An item was “achieved” when the trainee reported doing it once unsupervised. Time to independent performance and number of achievements were the primary outcomes.
Note: As clever as the authors were, there are a number of whopping threats to validity here, including:
1. No randomization (keener learners will sign up for boot camps and be more keen about learning in residency too!)
2. Time on task leads to greater learning in ANY human endeavour
3. Results are all self-report, with huge social-desirability bias at every step of the way.
4. No clear criteria for achievement of competence, just independence. Permission to perform independently may have more to do with staffing after hours than ability.
5. Rotation schedule and other opportunities for learning could account for huge differences in achievements.
6. Despite the frequent surveys, there is still huge risk of recall bias.
7. Finally, some of the priority items (not made by the authors) make me wonder if they are really fair for interns (dialysis catheters, really?).
Key Outcomes
The study involved very small numbers. Participation rates for the groups were 9 (100%), 9 (69%), & 12 (24%).
The authors report two major outcomes.
- First, the average number of competencies achieved by the end of PGY1 was ~12 for every group (12.7 vs 12.4 vs. 11.7); a wash.
- Secondly, the average time to first performance of unsupervised items was much less in Group A: 43.6 days earlier vs Group B, 49 days earlier than Group C. Ranges were huge, from 6.1-112 days and from 11.5-165 days, respectively.
About half of the data was “lost”: many surveys went incomplete.
It is also notable that certain items often not achieved in PGY1 from the list: ACLS code, compartment syndrome, central venous catheters, dialysis access, art lines, chest tubes, paracentesis, & endotracheal intubation.
Key Conclusions
The authors conclude that an intensive 4-week surgical preparatory course can lead to faster achievement of unsupervised performance of key competencies.
Spare Keys – other take home points for Clinician Educators
1. The authors get high marks for taking advantage of a natural cohort effect. We should all be looking for these opportunities.
2. This study, and many other observational ones like it, are threatened by small-n, non-randomized effects that limit the validity of the results.
3. Teaching works. Time on task works.
4. This boot camp phenomenon informed our own College’s creation of a mandatory Transition to Discipline period for the start of all residency training.
5. This study inspires some connections to mastery learning approaches, in which intense investments in trainee development lead to improved performance in the clinical setting.
6. Learners tend to achieve similar outcomes over time.
7. Boot camps don’t have to be so focused on procedures / basic technical skills.
Access KeyLIME podcast archives here
Check us out on iTunes