#KeyLIMEPodcast 264: Within a WiSCoR of Coaching

Reference

Vande Walle et. al., Development and Assessment of the Wisconsin Surgical Coaching Rubric. JAMA Surg. 2020 Apr 22

Reviewer

Jason R. Frank (@drjfrank)

Background

In his classic and hugely popular 2011 essay in the New Yorker “Personal Best”, surgeon-author-influencer Atul Gawande asks why doctors don’t have formal coaches. Comparing our pursuit of world-class expertise to other human endeavours—athletes, singers, etc—he sets out to find his own, pioneering surgical coach to take his game to a new level. It is a must read.

Have you had an amazing coach? In what part of your life? Was it a coach to enhance your medical expertise? Was your coach “good”? Effective? How would you know? It is nearly a decade since Gawande recruited his coach, so have we progressed in our #meded coaching enterprise?

Purpose

Enter Dr. Vande Walle & co from the University of Wisconsin Department of Surgery…They designed a tool to measure the performance of a surgical coach, the Wisconsin Surgical Coaching Rubric (WiSCoR). In this study, they set out to “evaluate the validity” of the WiSCoR to assess peer surgical coaching.

Key Points on the Methods

The authors defined peer surgical coaching as “an approach to continuous professional development that uses adult learning theory to support a surgeon’s individual performance improvement…” via pairing surgeons with a trained surgeon coach. The process involved collaborative analysis of practice, and constructive feedback for enhanced performance. Core competencies were identified through the development of the coaching programs. The authors shared that they used Cook & Messick as their validity framework to evaluate the novel assessment tool called WiSCoR. [Reminder: “Validity arguments” are made across multiple domains of evidence: Content, Response Process, Internal Structure (reliability), Associations with other variables, and Consequences.)

The authors assessed the performance of surgical coaches in 2 statewide surgical coaching programs (Wisconsin Surgical Coaching Program & Michigan Bariatric Surgery Collaborative Coaching Program) between Nov 2014-Feb 2018, as rated on the WiSCoR. There were 8 (35%) coaches in the Wisconsin and 15 (65%) in the Michigan programs, respectively. The programs coached 38 surgeons during the study period.

Sessions were video recorded and transcribed so that multiple raters could score the coaching performances.

Key Outcomes

Using Messick and Cook, they described how they developed the “content evidence” of the instrument: literature review, field observations, and interviews of coaches in various fields. These were then reviewed by surgical experts and refined. The subsequent 4 domains of the WiSCoR include:

1. Shared responsibility & equal exchange
2. Uses guided self-reflection
3. Provides constructive feedback
4. Guides goal setting & action-planning.

Each domain was rated 1-5, with 5 the level expected of a professional coach.

Validity of ratings was assured via rater training, scoring discussions, and scoring calibrations.

Mean score using the instrument was 3.23 (range 1-5). 282 WiSCoR scores demonstrated high interrater reliability (Gwet weighted agreement coefficient) and modest correlation with coachee ratings (r=0.22) over 106 sessions.

The authors found no other measure to correlate with except coachee ratings.

Consequential evidence consisted of providing feedback to the coaches via sharing the scores.

Key Conclusions

The authors conclude…The WiSCoR is a reliable and valid measure of surgical coaching.

Access KeyLIME podcast archives here

The views and opinions expressed in this post and podcast episode are those of the host(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page