By: Olle ten Cate (@olletencate)
One of our PhD students at UMC Utrecht, Wilma Kleijer, is interested in the considerations of nurse-preceptors when they decide to entrust students with care tasks. In an interview study she found that preceptors acknowledge that they weigh in more than \’objective\’ information, i.e. scorable data on assessment forms, when determining a student\’s readiness to work by themselves (paper under review). They often express that as a \’gut feeling\’ about the student. Such judgment reminds of rapid diagnostic impression formation, such as what happens in emergency rooms or on ICU wards about patients. Sibbald and colleagues found that emergency clinicians can distinguish the sick from the not-sick by visual appearance quickly, and pretty reliably, and they interpret this as System-1 processing.1 This is an important skill in emergency setting, but not easily explained and taught and it may take experience with many instances of practice to acquire it.
A farmer\’s tacit knowledge
Geert Mak, a popular Dutch journalist and author of historical non-fiction in Europe, reported how villages and their inhabitants\’ skills disappeared in 20th century western nations. He explained how in the past \”..a youthful farmer not only had to learn how to milk smoothly, skillfully repair his own tools and appliances, and perform thousands of other skills, he had to learn how to trust his own intuition, by \’seeing\’ that a cow was sick, \’feeling\’ that the weather had turned, \’knowing\’ when the harvest had to be reaped from the land.\”2 Why they would see, feel or know this was difficult to convey to inexperienced others, let alone non-farmers.
Gut and intuition
Trusting your intuition means using gut feelings, also in evaluations of trainees and deciding to trust them with clinical tasks. Experienced farmers, diagnosticians and educators have in common that we call them \’experts\’ and we regularly rely on their judgment. Expert judgment cannot always be explained3 and relying on experts requires trust. If we call their judgment \’subjective\’, there is always a risk of opposition. Subjectivity has the connotation of not being valid, not defensible in case of formal disputes, and to be disregarded at all cost. However, subjectivity is increasingly acknowledged as important, maybe even inherent to any expert judgment. The German psychologist Gigerenzer claims that decisions based on intuition and gut feelings can even outperform rational decisions in effectiveness.5
The bad connotation of the word \’subjective\’ in assessment arises when personal prejudice, based on irrelevant information, weighs into judgment and leads to unfair decisions about learners. Measured across groups of assessed learners this might be called, with a statistical term, construct-irrelevant variance. Indeed, even with an individual learner, what you want to avoid is weighing in subjective feelings that are not relevant for the construct of interest (i.e. \’competence for a task\’ or \’readiness to perform without supervision\’). But discarding all subjectivity in judgment would be throwing away the baby with the bathwater. Even fairness in decision making may benefit from subjectivity, as that adds to what can be \’objectively measured\’ and expressed in numbers.6
Disentangling components of subjectivity
So the trick will be to disentangle bias from legitimate subjectivity. That, of course, will not be easy. Faculty training and reflection on personal judgment may help, focused on learner qualities that should count in entrustment.7 Next, important decisions should not be based on individual observations, but on a collection of data from different sources. Clinical Competency Committees, or Trained Entrustment Groups are charged with making decisions on the data available. \’Objective\’ portfolio data are not always comprehensive8,9 and committee members may supplement these with expert opinions. When different observers pool their impressions to arrive, after discussion, at collective, intersubjective judgment, chances are that the ‘bias’ component of subjectivity will decrease and the ‘legitimate’ component remains.
———————————————————————————-
About the author: Olle ten Cate PhD is a senior scientist at the Center for Research and Development of Education Universiteit Utrecht, the Netherlands.
References
1. Sibbald M, Sherbino J, Preyra I, Coffin-Simpson T, Norman G, Monteiro S. Eyeballing: the use of visual appearance to diagnose ‘sick.’ Med Educ. 2017;51(11):1138-1145. doi:10.1111/medu.13396
2. Mak G. Hoe God Verdween Uit Jorwerd (Translated in 2000 as: Jorwerd: The Death of the Village in Late Twentieth-Century Europe). Amsterdam: Altas; 1996.
3. van Enk A, ten Cate O. “Languaging” tacit judgment in formal postgraduate assessment: the documentation of ad hoc and summative entrustment decisions. Perspect Med Educ. 2020;9(6):373-378. doi:10.1007/s40037-020-00616-x
4. Ten Cate O, Regehr G. The Power of Subjectivity in the Assessment of Medical Trainees. Acad Med. 2019;94(3):333-337.
5. Gigerenzer G. Gut Feelings. The Intelligence of the Unconscious. London, UK: Penguin Books; 2007.
6. Valentine N, Durning SJ, Shanahan EM, van der Vleuten C, Schuwirth L. The pursuit of fairness in assessment: Looking beyond the objective. Med Teach. 2022;(EarlyOnline). doi:10.1080/0142159X.2022.2031943
7. ten Cate O, Chen HC. The ingredients of a rich entrustment decision. Med Teach. 2020;42(12):1413-1420. doi:10.1080/0142159X.2020.1817348
8. Touchie C, Kinnear B, Schumacher D, et al. On the validity of summative entrustment decisions. Med Teach. 2021;43(7):780-787. doi:10.1080/0142159X.2021.1925642
9. Brown DR, Moeller JJ, Grbic D, et al. Entrustment Decision Making in the Core Entrustable Professional Activities: Results of a Multi-Institutional Study. Acad Med. 2022;97(4):536-543. doi:10.1097/acm.0000000000004242
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our \’About\’ page
