“Comparison is the thief of joy.” – Theodore Roosevelt
In my April post, I shared some thoughts about the way we often provide feedback to our residents. To summarize, I pondered our use of normative-referenced scales (i.e. assessments that position an individual compared to a cohort), such as ITERs for work-based assessment. These scales are often not scored as they are intended (grade inflation) and are often not consistently used among faculty (hawks and doves).
I made the suggestion that we consider doing away with numbers in these work-based assessments, and instead place more focus ensuring robust formative feedback with the narrative comments provided to learners. I closed by saying that I would follow-up with my perceptions about some resident perspectives on this, and why I think they may also favour this approach.
I believe that part of the challenge when we use these types of assessments is that if a resident is told they are a “3” (on a 5-point scale) or “meeting expectations”, that is likely at odds with their world view. Many residents likely equate a “3” or “meeting expectations” with a “C” grade in undergrad, or in other words “mediocrity” – and no one got into medical school with C grades. Hence it might be hard to conceptualize oneself as a “C” resident. Yet this equivalent to “meets expectations” is supposed to be the most frequently assigned score for residents on our ITERs, at least at my institution (as faculty we recognize this tension and hence, on our part, why there is often grade inflation).
Does this really matter though? Perhaps. The issue may not only be that residents overly focus on their score or position on a scale, but if a score is at odds with their world view, it may affect how they process any narrative constructive feedback provided at the same time. Feedback recipients’ perceptions are important to consider as they are the lens through which a learner will hear feedback,1-3 and may alter what they do with that feedback.3,4 Therefore those who are receiving “3”s (or lower), and potentially most in need of feedback, may be less likely to use it if the associated grade is incongruent with how they perceive themselves. Enhancing formative feedback and removing scores from most assessments may help with this possibility.
Consideration of this is important in light of the shift towards competency-based medical education, as the provision of frequent, timely feedback to residents will be the cornerstone of this movement. From the Royal College’s website,5 Competency by Design (CBD – Canada’s version of competency-based medical education) will afford learners the following:
“For residents, CBD will lead to more frequent formative assessment and feedback from supervisors by expert faculty, and allow trainees to advance more rapidly to additional skills and knowledge modules as they achieve the outlined competencies. It will also provide the opportunity for residents to identify learning needs and course correct if required.” 1
Given the importance of formative feedback in CBD, it will be essential to be thoughtful about how feedback is framed and presented to residents for it to be maximally effective in the clinical setting. To be truly formative, structured feedback needs to occur continuously during rotations (rather than only at the end) and offer specific, action-oriented guidance for residents about what they are doing well, and why, and what they can improve on, and how they can achieve this. Not only is there a need for faculty development in this area, which is already ongoing, but there needs to be resident development too; such that receiving frequent constructive feedback about performance is both expected and optimally utilized. Pass / fail (or similar) summative judgements can be made based on compilations of these interactions. Numbers attached to each individual assessment do not seem necessary to achieve this goal.
References:
- Sargeant J, Mann K, van der Vleuten C, Metsemakers J. ‘Directed’ self-assessment: practice and feedback within a social context. J Contin Educ Health Prof. 2008 Winter; 28:47–54.
- Archer JC. State of the science in health professional education: Effective feedback. Med Educ. 2010 Jan; 44(1):101-8.
- Eva KW, Armson H, Holmboe E, Lockyer J, Loney E, Mann K, Sargeant J. Factors influencing responsiveness to feedback: On the interplay between fear, confidence, and reasoning processes. Adv in Health Sci Educ Theory Pract. 2012 Mar; 17(1):15-26.
- Sargeant J, Eva KW, Armson H, Chesluk B, Dornan T, Holmboe E, Lockyer JM, Loney E, Mann KV, van der Vleuten CPM. Features of assessment learners use to make informed self-assessments of clinical performance. Med Educ. 2011 Jun; 45(6):636-47.
- Competence by Design (CBD): Moving towards competency-based medical education. The Royal College of Physicians & Surgeons of Canada. Available on-line at: http://www.royalcollege.ca/portal/page/portal/rc/resources/cbme . Accessed August 9, 2015.
