#KeyLIMEpodcast 133: The Observer Effect in Meded – The Learners’ View

LaDonna K, Hatala R, Lingard L, Voyer S, Watling C. Staging a performance: Learners’ perceptions about direct observation during residency. Medical Education. 2017. May;51(5):498-510

Reviewer: Jason R. Frank (@drjfrank)

Background

This elegant paper explores the lived experience of physicians who have had direct observation while performing clinical tasks.

Direct observation (DO) is currently discussed with renewed emotions in meded discussions: ennui for apprenticeship days long past, enthusiasm for the promise of a return to DO under CBME, anxiety that such observation is impossible in 21^st century healthcare.

Few would challenge the concept that DO has fantastic potential to provide real-time, authentic data for teachers to assess learner progress, provide really useful feedback for improvement, and document abilities. However, has DO really lived up to its promise when applied to clinical residency education? This paper goes there.

Purpose

The authors set out to…explore residents’ experiences of being observed during their training, including their perceptions of the impact of DO on their performance, behavior & learning.

Type of Paper

Research: Qualitative study

Key Points on Methods

This paper is a wonderful example of the application of constructivist grounded theory to a meded phenomenon. The authors recruited trainees from two Canadian medical schools to participate in structured interviews. A constant comparative method was used to analyze transcribed, deidentified data iteratively. The trainees from the second school were interviewed to triangulate initial findings. (The paper is silent on how the second group were recruited). This continued to sufficiency. The description is exemplary, but longish.

Key Outcomes

This paper is rich in detail, but the essential themes identified by trainees included:

1. DO was uncomfortable, and often created emotional discomfort in trainees (“made me sweat”)
2. DO often involved unclear expectations, leading to further discomfort (lack of clarity, unpredictability)
3. Trainees felt greater discomfort when observations where infrequent, and there was little relationship with the observer
4. Trainees felt obligated to alter their behavior, from patient-centred to task-centred in order to maximize the assessment score (inauthentic)
5. Trainees felt they were more likely to fumble a task while being observed (self-conscious)
6. Trainees felt a loss of rapport with patients while being observed (disrupt the dynamic)
7. Trainees felt that DO was always related to assessment, and rarely had a focus on learning
8. Feedback / observations had less credibility if there was less rapport with the supervisor, the observation was inauthentic, or the comments were perceived as an attack on their self-concept as a professional.
9. On the positive side, learners did identify the potential benefits of a trusted observer building confidence, identifying blind spots, or energizing learning “with adrenaline”.

At the same time, trainees offered distinct conceptualizations of “supervision” (active, hands on, safety net, task-focused, efficiency-focused) vs “observation” (hands off, coaching).

The authors offer an extensive discussion, including waxing into the literature of self-concept / professional identity formation as it relates to the “observer effect”.

Key Conclusions

The authors conclude that we have much work to do on Direct Observation. To maximize the effectiveness, credibility, and value of DO, we need to:

1. ensure a safe learning environment, with a “feedback alliance” between the observer and learner, that generates trust;
2. clearly articulate expectations of the DO;
3. how the DO relates to assessment.

I would add a 4^th one:

4. Make DO frequent, low-stakes, to make the DO “normal”, relaxed, and an authentic part of everyday work.

Spare Keys – other take home points for clinician educator

1. This is a great example of a grounded theory paper
2. This is the first of an expected avalanche of papers on Direct Observation in meded (I hope)
3. This is a key paper for your CBME library

Shout out

Thanks to the journal Medical Education for this pre-publication manuscript, which we enjoy selecting from those they offer to us.

Shout-out to Lara Varpio PhD, for her letters clarifying some of the concepts of qualitative research that we have covered in past episodes. We welcome the learning too!

Access KeyLIME podcast archives here

Check us out on iTunes!