#KeyLIMEpodcast 121: ACGME Accreditation Reveals Key Themes for #MedEd

Wagner R, Koh NJ, Patow C, Newton R, Casey BR, Weiss KB, on behalf of the CLER Program. Detailed Findings from the CLER National Report of Findings 2016. Journal of Graduate Medical Education. 2016 May;8(2 Suppl 1):35-54.

Reviewer: Jason Frank (@drjfrank)

Background

Accreditation, the enterprise and process to judge and enhance the quality of an educational program via comparison to third-party standards, is undergoing a quiet revolution. Accreditation is evolving away from an overwhelming emphasis on process measures to also focus on other important aspects of health professions training, such as program outcomes and learning environment. However, accreditation reform suffers from a few challenges. Two I will mention here: 1) a lack of bold innovation, and 2) few published papers to build on. Enter the ACGME’s CLER (Clinical Learning Environment Review) initiative.

Purpose

The authors of this paper describe the ACGME’s CLER initiative and the first national report on the patterns found in 6 areas in US residency clinical learning environments, namely:

1. Patient safety
2. Health care quality
3. Care transitions (aka handovers)
4. Supervision
5. Fatigue management & duty hours
6. Professionalism

Type of Paper

Program Evaluation

Key Points on Methods

The ACGME (Accrediting Council for Graduate Medical Education) is the accrediting body for residency education (GME aka PGME) in the US. CLER emerged as part of an evolving package of reforms intended to enhance American residency training programs and shift the accreditation emphasis to outcomes (and away from process measures).

The ACGME moved to mandatory reporting of resident progress on competency-based milestones a few years ago, and at the same time decreased the number of on-site surveys of programs. They also added a new time of survey of institutions focused on features of the clinical learning environment, and CLER was born.

This report was generated from the aggregate findings of reviews of 297 meded institutions overseeing 8,878 residency programs (3 to 148 per site) between 2012 and 2015. This covered 111,482 trainees (range 8 to 2,216; median 241).

Survey teams used an accreditation technique sometimes called a tracer, in which surveyors interview groups and then go on walking rounds to seek validity evidence for patterns that were suggested. Multiple lines of evidence are combined to provide a greater picture. CLER teams interacted with a wide variety of officials and professionals, from 1000 executives, 8755 residents, 7730 faculty, 5599 program directors, as well as nurses, pharmacists, social workers, etc. Data collection was from discussions, surveys, interviews, and anonymous audience response systems. Quantitative scores were compared for groups using simple stats.

Key Outcomes

Looking at the patterns from these 297 institutions, here are some highlights the authors found:

1. Patient Safety: large variations in all aspects of patient safety; 96.8% of residents reported have some patient safety education; there usually was a method for reporting incidents; 95.5% of trainees reported a safe environment to report; but few <20% ever did themselves;
2. Quality: about 3 quarters of interviewed populations reported any knowledge of QI priorities; few trainees were familiar with even basic QI terminology (e.g. PDSA); about 3 quarters of trainees said they did a QI project; only about half of participants reported knowing about the priorities to improve health care disparities;
3. Handovers: 82% of trainees reported that handovers were a priority area for improvement; ~84% reported using some kind of standardized process for inpatients, and 90% for end of shifts;
4. Supervision: more than 90% of trainees reported feeling confident in their scope of activity without direct supervision; 47% of PDs reported managing issues related to supervision and patient safety;
5. Fatigue & Duty Hours: 95.5% of trainees reported receiving education on fatigue management (mainly on the first week of orientation), where only 67% of faculty reported the same; 8% of PDs discussed patient safety incidents related to fatigue; faculty expressed concern about a “shiftwork mentality” after duty hour reforms;
6. Professionalism: 66.4% of executives reported incidents relating to professionalism; 92.8% or residents reported some education related to professionalism; 16% of residents felt they had been asked to compromise their integrity for an authority.

These findings reflect a program evaluation methodology, and there are a number of threats to validity.

Key Conclusions

The authors conclude that the CLER visits have provided rich data on 6 important aspects of the learning environment in the US that can be used by system and institutional leaders and others to act.

Spare Keys – other take home points for clinician educator

1. This paper is in a rare category of meded paper: data from an accreditation study. We need more to inform accreditation practices, and measures of educational outcomes.

2.We also need data on clinical learning environments. There is a lot rich material here to inspire future interventions by clinician educators.

3. As we’ve said before, JGME is a great new meded journal. Check it out.

Shout out

Kudos to Tom Nasca and his team at ACGME for committing to innovations and sharing results with the community.

Access KeyLIME podcast archives here

Check us out on iTunes!