#KeyLIMEPodcast 227: The Hitchhiker’s Guide to Clinical Performance Indicators

Reference

Smirnova A et al. Defining and Adopting Clinical Performance Measures in Graduate Medical Education: Where Are We Now and Where Are We Going? Academic Medicine. 2019 May;94(5):671-677

Reviewer

Jason R. Frank (@drjfrank)

Background

Regular listeners of KeyLIME will know that #meded (and health professions training more generally) is a field and enterprise widely criticized for unacceptable variation in its product, namely practicing clinicians. Our leaders remind us that this is an age of accountability, and our profession is asked to innovate to improve competence, care, safety, and the systems that produce them.

Enter data. Lots of data. If you have an electronic health record or a digital portfolio or even a great big box of paper with stuff about your trainees, you have data. The tricky part is what to do with it? How can we convert the data all around us into meaningful intelligence that can inform teaching, learning, assessment, curriculum, systems design, accreditation, and more? Enter smart people who think about Clinical Performance Indicators (CPIs).

Purpose

This paper, Smirnova et al, is a team of heavy hitting #meded authors. (In particular, I encourage you to watch the clever work of Dan J Schumacher from Cincinnati.)

The group set out to describe how CPIs, as measures of the activities, behaviours, and abilities of clinicians, can inform medical education and health care quality improvement.

Key Points on the Methods

This is essentially a narrative review by a learned group. They chose to focus on the implications for residency education, but they suggest their points apply to any phase of a clinician’s career, and any workplace-based learning setting. The paper is essentially framed as a charter and state of the art overview of CPIs for #meded.

Key Outcomes

Why do we need CPIs? The authors make the case that CPIs are needed to:
• Align #meded with clinical care quality;
• Justify societal investment in #meded via quality markers;
• Connect education to clinical care process indicators as well as patient outcomes;
• Provide feedback loops to teaching, learning, and curriculum;
• Assess learner performance, progression of competence;
• Help evaluate learning environments and accredit programs;
• Attempt to alleviate unacceptable variations in care between training settings;
• Promote patient safety; and,
• Inform #meded research, program evaluation, and CQI.

Levels of CPIs can include :
1. Individual clinician / learner
2. Teams
3. Settings
4. Programs
5. Institutions
6. Systems
7. Specialties
8. Nations

For each of these levels, CPIs can inform education via:
• feedback loops,
• guides for further learning and reflection,
• personal insights,
• tracking progress
• aggregate clinician performance over time to guide assessment, evaluation, curriculum renewal,
• clinical processes, and patient outcomes,
• characteristics of learning environments

Challenges with CPIs for #meded include:
• need for collaborations across multiple healthcare stakeholders, including educators, patients, learners, health services researchers, economists, digital systems managers, and patient safety/QI leaders;
• need for taxonomies for a lingua franca among training communities;
• need to define CPIs that are clinically relevant, educationally sensitive, and tailored to the learner’s stage of development;
• need to overcome measurement issues, including capturing meaningful data from existing digital systems (e.g. EMRs and registries);
• need to address analysis issues, especially attribution (the effect of a given individual’s behaviours in care) and contribution (the ways a given individual’s abilities can impact), as well as aggregation and nesting;
• need to take into account complexity issues in health care systems, and the interdependence of individual health professionals’ input into care, as well as the contextual influence;
• protecting privacy of people’s data; and,
• linking data sets in useful ways.

Key Conclusions

The authors conclude that measures of trainee performance are essential ingredients to enhancing competence and ensure safe and effective care. They argue the way forward includes:

1. Developing a common taxonomy of CPIs and relevant patient outcomes by specialty;
2. Defining uniform measures for comparative research;
3. Linking #meded and CPI measures to provide new insights into training systems; and,
4. Developing iterative loops to improve measures, definitions, and measurement systems

Spare Keys – other take home points for clinician educators

1. We do live in an era of accountability, metrics, and digital databases, CPIs are coming, so #meded leaders need to plan for it.
2. CPI papers are also coming to the #meded lit, a torrent is on the way. I highly recommend the work of the authors.
3. What would happen if we correlate entrustment measures to CPIs? Stay tuned.

Access KeyLIME podcast archives here