For the fourth year, we are collaborating with the ALiEM Faculty Incubator Program to serialize another volume of Educational Theory made Practical. The Faculty Incubator program a year-long professional development program for educators, which enrolls members into a small, 30-person, mentored digital community of practice (you can learn more here); and, as part of the program, teams of 2-3 participants author a primer on a key education theory, practically linking the abstract to practical scenarios.  

They have published their first and second e-book compendium of this blog series and you can find the Volume 3 posts here (the e-book is in progress!) As with the previous iterations, final versions of each primer will be complied into a free eBook to be shared with the health professions education community. 

Your Mission if you Choose to Accept it:

The ALiEM Faculty Incubator Program would like to invite you to peer review each post. Using your comments, they will refine each primer. No suggestion is too big or small – they want to know what was missed or misrepresented. Whether you notice a spelling or grammatical mistake, or want to suggest a preferred case scenario that better demonstrates the theory, they welcome all feedback! (Note: The blog posts themselves will remain unchanged.)

This is the fifth post of Volume 4!  You can find the previous posts here: Cognitive Load Theory; Epstein’s Mindful Practitioner; Joplin’s Five-stage Model of Experiential Learning and Maslow’s Hierarchy of Needs.

Miller’s Pyramid

Authors: Eric Blazar (@eblazar); Vimal Krishnan (@drvimalkrishnan);
Shivani Mody (@shivanimody)

Editor: Daniel Robinson (@simdoc)

Main Authors or Originators: George Miller (1990)

Part 1: The Hook

It’s busy monday afternoon and the number of patients in the waiting room is only increasing.  The attending is paired with a PGY 2 EM resident.  Both the attending and resident are completing charts when the triage nurse hastily interrupts stating, “this very nice woman in bed 6 appears to be having a stroke and she’s inside the tPA window.” On impulse, the attending and resident both stand and calmly walk over to evaluate the patient. Before them is a mid-50s appearing, well kept woman sitting calmly in the stretcher. The resident starts taking the history as the attending observes from the doorway. The patient states that she is experiencing difficulty speaking and left arm weakness, that started while working. Her symptoms began 2 hours prior to arrival. The resident takes a lengthy and thorough history and performs a physical examination. During the history, the attending notices the patient is having difficulty with word finding, is unable to resist gravity in the left upper extremity and has slight weakness in the left lower extremity. The resident turns to the attending and states that the patient needs an emergent CT scan and places the order in the computer. The attending reminds the resident that a set of vital signs and a point-of-care (POC) glucose are necessary prior to CT scan. Aside from noted hypertension, the patient’s vitals and glucose are appropriate and the patient is taken to the CT scanner. While the patient is in the CT scanner, the attending physician discusses the case with the resident. The attending posits the role of thrombolytics and the resident states that if the CT scan is negative for an acute bleed, thrombolytics may be indicated. The patient returns from the CT scanner and radiologist calls shortly after stating the CT is unremarkable for acute pathology. The attending asks the resident to question the patient about any contraindications she has to thrombolytics. The resident states he is unsure of the contraindications. The attending also asks if blood pressure control is necessary as the blood pressure is 260/130.  The resident states that he is unsure of the blood pressure goals and also states he is not sure how to best treat the blood pressure in this patient, if necessary. The attending starts the patient on a labetalol drip for blood pressure control. 40 minutes have passed since the patient’s arrival and it has been determined by the treatment team that thrombolysis is indicated. After a long discussion with the patient and her family, the decision is made to administer thrombolysis. Now 2 hours and 50 minutes following onset of symptoms, the resident sits down to order the medication and states to the attending that he is not sure what the dosing is or how thrombolytics is administered. The attending places the order. Following administration, the patient is admitted to the critical care/neuro intensive care setting at the hospital. While both the resident and the attending are charting, the attending physician asks the resident to reflect on the case. The resident tells her, “I’ve really only ever taken care of one or two acute stroke patients and today was my first time administering thrombolytics. I feel like I know a lot about stroke but I’m just not comfortable managing it in the ED.”

Part 2: The Meat

Overview

Multiple studies in medical education have demonstrated that performance on examinations does not reflect proficiency in patient care. That is, strong written testing performance demonstrating a strong grasp of medical science knowledge does not necessarily translate to excellent patient care.³ Miller’s Pyramid provides a framework for assessing clinical competence in medical education beyond test-taking or memorization. The pyramid, sometimes called Miller’s Triangle, can assist clinical teachers, as well, in matching learning outcomes with expectations of learner performance at specific training levels. The framework, shown in Figure 1, starts with the wide base of “knows” representing knowledge. The pyramid transitions up to “knows how” or competence. Next, “shows how”  is demonstration of performance. Finally the ultimate point of the pyramid is “does” or action. Miller’s pyramid demonstrates the importance of skills beyond knowledge in medicine emphasizing the delivery of care and skills necessary for excellence in medicine.¹  The pyramid focuses on the importance of acquiring knowledge to perform a task in practice.⁴ It also presents a standardized step-wise approach to clinical pedagogy and trainee assessment.¹

Background

Created in 1990 by George Miller¹, Miller’s pyramid (Figure 1) was initially created and utilized as a framework to clinically assess trainees at varying levels.¹ Starting with “knows”, the base of the pyramid is based on knowledge and Miller opines is the easiest to assess. Simply, standard medical knowledge is tested through board examinations and medical school testing. These tests are considered incomplete assessment tools in Miller’s view as he states in his original opinion piece that medicine requires more than knowledge of science to master excellence in medical care. In essence, Miller emphasizes the art of medicine.  Miller views knowledge as the introductory and basis for clinical competence.  As a learner, ”knows”/knowledge in tube thoracostomy, for example, would be the ability to state indications and contraindications for placement or demonstrate understanding of the pathophysiology of congestive heart failure (CHF).²

The second stage of the pyramid is “knows how”. This is the assessment of the ability to analyze and interpret the data obtained or as Miller classifies, competence.¹ Competence can be assessed by testing on simulated patients or performing procedures in simulation. The US Medical Licensing Examination (USMLE) clinical skills (CS) assesses this second pyramid level of competence. The learner in this phase is able to describe the steps in placing a chest tube or discuss the steps in managing CHF.²

The third stage is “shows how”. Miller describes this as assessing the performance of the learner in clinical situations. Performance assessment is the learner’s ability to apply knowledge and to analyze data when faced with a patient situation.¹ Rather than working through the physical examination on a standardized patient as in “knows how”, standardized patients can be used to ascertain not only data gathering but formulation and processing outside the clinical setting. Other attempts at assessment of clinical performance are Objective Structured Clinical Examination (OSCE) and Mini-Clinical Evaluation Exercise for trainees (Mini-CEX).² Assessment on the hospital wards by clinical teaching faculty fall under the performance assessment. However, Miller cautions against this “shows how” type of assessment as he states there is declining direct observation of the learner and most of the learner skills are interpreted during rounding or discussion of disease etiologies.¹  In the 1960/70s, studies found bedside teaching and direct observation of the learner occurred in about 75% of patients.  In contrast, Gonzalo et al in 2009 found a sharp decline in bedside direct observation teaching, occurring only in about 25% of precepted patients.¹¹ Stickrath et al found in their study, “Teams infrequently … taught physical examination skills (14.6%), evidence-based medicine topics (7.2%), or learner-identified topics (3.2%). Many commonly performed activities occurred infrequently at the bedside.”¹¹ With this move away from direct involvement and observation by the educator, Miller opines that assessment of the learner in their ability to “shows how” is mere inference.

The apex of Miller’s pyramid is “does”. “Does” represents action and assessment of the physician’s ability to function independently in the clinical setting.¹  Miller discusses that this is the most difficult to measure. Current attempts to measure action in the clinical setting include patient satisfaction surveys, demonstration of post procedure care and other aspects including functional status, cost effectiveness and intermediate outcomes.^2,7  

Miller’s pyramid presents a graded method of assessment for learners. In 1990, he was suggesting a paradigm shift hoping to refocus medical education and stress the importance of assessment outside of just knowledge. In today’s medical education ecosystem, Miller’s pyramid aids in understanding the question we are testing and insures the assessment is valid.

Modern takes or advances

Since 1990, Miller’s idea has been adapted to continue to fit the changes in the field. As Miller hoped from his original opinion piece, medicine has changed paradigms with a marked greater focus on teaching the art as well as the science of medicine. As the learner familiarizes themselves with concepts and skills, there is understanding that comes before having strict knowledge of the subject. Two underpinned awareness levels, “heard of” and “knows about” have been added to Miller’s pyramid below “know”. (Figure 2) For instance, knowing of a chest tube but not knowing the indications for placement is a precursor knowledge set.⁶

Cruess et al added a fifth level above “does”. “Is” should reflect not only clinical competence but the presence of a professional identity.⁸ Medical education evolved to consider the “attitude” part along with knowledge and skills, which is reflected in Miller’s prism (Figure 3). This adds a third dimension of professional authenticity onto the original Miller’s Pyramid. Professional authenticity focuses on “attitudes”, “skills” and “knowledge” on a spectrum of novice to expert. Thus, “does” with expert level attitude, skills and knowledge is the apex of Miller’s prism which was remodeled in 2009, nineteen years later.⁶

Rethans et al. recognized that both internal and external factors influence performance and wanted to further focus on “performance-based assessment measuring what doctors do in actual professional practice.”⁹ Thus, the Cambridge model (Figure 4) was created as a modern adaptation of Miller’s pyramid.⁹  Beyond the apex of the pyramid, “does” or competence, both system and individual related influences will affect physician performance.  System influences consist of any driving factor affecting or being dictated by the community. These influences consist of guidelines, government programs, time available to spend with patients and even patient expectations. All these factors influence a physician’s daily performance. Test utilization and adherence to guideline metrics are attempts to measure daily performance. Individual influences, such as personal and mental health and occupational relationships, will also influence the physician’s overall performance.^8,3 Thus, Rethans states that one’s individual competence plus individual and system based external forces come together to dictate a physician’s overall performance.

Other examples of where this theory might apply in both the classroom & clinical setting

Miller’s pyramid can be used as the transition from classroom to clinical setting. As constant learners of new knowledge, physicians are always maturing through the pyramid from knows to does. While on a grander scale, practicing medicine is a competency that has been divided into specialities, Miller’s pyramid can also be used on a much smaller level. Starting large in the US, medical students select and match into residency programs and then test in many various ways and gain board certification for a specialty or multiple specialty practices. Individual practice and board certification are progressions up the pyramid as one becomes able to clinically demonstrate mastery of a medical specialty. However, on a smaller scale the pyramid becomes applicable for new knowledge physicians gain at conferences or through other continuing education to update their skills. For instance, if an older attending uneducated in ultrasonography that already performs at a “does” level in Emergency Medicine attends a conference or workshop to develop ultrasound competency, that provider will progress up the assessment levels of the pyramid through testing, then observing and then on shift QI as they improve and increase their ultrasound skills. Thus, assessment must be separated into graded measurable goals for various skills and knowledge bases as skill mastery progresses for providers.

Annotated Bibliography of Key Papers

Miller G. The assessment of clinical skills/competence/performance. Academic Medicine. 1990;65(9):S63-7.¹

Miller’s original manuscript where he presents his pyramid for clinical assessment. (Knows, knows how, shows how, does). Miller suggests that testing is antiquated and not getting to the center of necessary assessment. He suggests that while testing knowledge is important, an advanced learner must know how and then show and finally do. He suggests different manners for assessment at each level of the pyramid.

Rethans J, Norcini J, Baron-Maldonado M et al. The relationship between competence and performance: implications for assessing practice performance. Med Educ. 2002;36(10):901-909.⁹

Rethans et al further expanded on Miller’s pyramid of assessment and further theorized that competence is not solely influenced by performance. Rethans demonstrates how there are both systemic and individual external factors that play significant roles in influencing perceived and observed competence. Systemic influences include any determinant from the governing body or hospital of employment as well as government influences on health care. Individual influences, such as provider wellness and health, mood, patient mood and wellness, among many others also equally play a role in performance and thus competence. Rethans decided that an inverted pyramid would best show that performance is based on competence as well as systemic and individual influences equally.

Limitations

While Miller’s pyramid has become widely utilized as the basis for benchmarking training assessment and performance, as with any education theory, Miller’s pyramid does have some limitations. First, Miller himself notes the difficulty in assessing “does”. To assess physicians in their work environment while they are actively treating patients is challenging, expensive and time consuming. The underlying test/observation would never be the same for two different providers as no two patients, patient encounters, or care environments are the same. Thus, comparison becomes quite difficult when evaluating proficiency at the top of the pyramid.^3,7 As case mix and number of cases vary, complexity and attribution also contribute to the complexity of measuring. Physicians are not the sole dictators of patient outcomes and thus only monitoring outcomes is inherently flawed.^3,7  However, at this time in health care, patient outcomes seems to be how physician proficiency and assessment is monitored.³

The pyramid is based on the assumption that assessment in actual practice environments are a better reflection of routine performance as opposed to preset/ artificial testing conditions.⁷ While this framework is widely accepted, Miller even notes in his original manuscript that he is basing his pyramid on conjecture, as most frameworks are. He assumes that observation in reality is a more effective truth than simulated or testing environments.

Miller’s pyramid also is limited in that it assumes that competence predicts performance.  Demonstrating competence in the “does” apex of the pyramid is assumed to predict future good performance. Often times, there are other influences on one’s performance. A physician or learner’s performance can be greatly dictated by availability, time, energy level, mood, the environment and the patient.⁴

Part 3: The Denouement

Let’s revisit the opening case. We had a PGY 2 well respected resident near mismanage an acute stroke patient. The attending noted he demonstrated fairly significant knowledge gaps in the management of acute thrombotic stroke. The attending physician provided feedback to the resident the following day in an email stating that he took much too extensive of a history and delayed the patient’s CT scan. He also forgot to obtain vital signs and a POC glucose. He was unable to manage the patient’s blood pressure and was unsure of many of the contraindications to thrombolysis administration. The attending provided this feedback in a constructive and non-threatening manner and provided resources from EM text books as well as a visual youtube lecture and a podcast to augment the resident’s learning. The attending was focusing on moving the learner from “knows” to “knows how”.

One week later during EM didactics the resident approaches the attending stating the resources were excellent and he feels much more confident in his acute stroke knowledge.  The attending offers a simulated case as practice to the resident. Later that day in the sim lab, the residents goes through 2 cases of acute thrombotic stroke and displays his comfort with his new knowledge. The attending is very pleased with the resident’s improvement and the learner has moved to “shows how.”  During the debriefing from the simulation, the attending talks about the limitations of simulation as well as the need for continued assessment of the resident when managing acute CVA patients in the ED. They develop a way for the resident to ask the attendings for attention and feedback when managing other acute stroke patients on shift in the ED.

Don’t miss the sixth post in the series, coming out Tuesday, March 10, 2020!

PLEASE ADD YOUR PEER REVIEW IN THE COMMENTS SECTION BELOW

References

1.Miller G. The assessment of clinical skills/competence/performance. Academic Medicine. 1990;65(9):S63-7. doi:10.1097/00001888-199009000-00045

2. Ben.edu. http://www.ben.edu/college-of-education-and-health-services/nutrition/upload/ assessment_and_feedback_for_residents_and_students-1.pdf. Published 2019. Accessed May 1, 2019.

3. Ramani S, Leinster S. AMEE Guide no. 34: teaching in the clinical environment. Med Teach. 2008;30(4):347-364. doi:10.1080/01421590802061613

4. Carr S. Assessing clinical competency in medical senior house officers: how and why should we do it?. Postgrad Med J. 2004;80(940):63-66. doi:10.1136/pmj.2003.011718

5. Pangaro L, ten Cate O. Frameworks for learner assessment in medicine: AMEE Guide No. 78. Med Teach. 2013;35(6):e1197-e1210. doi:10.3109/0142159x.2013.788789

6. Cheek B. The Miller pyramid and prism. Gp-training.net. http://www.gp-training.net/training/educational_theory/adult_learning/miller.htm. Published 2016. Accessed June 23, 2019.

7. Norcini J. ABC of learning and teaching in medicine: Work based assessment. BMJ. 2003;326(7392):753-755. doi:10.1136/bmj.326.7392.753

8. Cruess R, Cruess S, Steinert Y. Amending Miller’s Pyramid to Include Professional Identity Formation. Academic Medicine. 2016;91(2):180-185. doi:10.1097/acm.0000000000000913

9. Rethans J, Norcini J, Baron-Maldonado M et al. The relationship between competence and performance: implications for assessing practice performance. Med Educ. 2002;36(10):901-909. doi:10.1046/j.1365-2923.2002.01316.x

10. Gonzalo JD, Masters PA, Simons RJ, Chuang CH.Attending Rounds and bedside case presentations: medical student and medicine resident experiences and attitudes. Teach Learn Med. 2009;21(2):105–110. doi:10.18043/ncm.76.3.174

11. Stickrath C, Noble M, Prochazka A, et al. Attending Rounds in the Current Era: What Is and Is Not Happening. JAMA Intern Med. 2013;173(12):1084–1089. doi:10.1001/jamainternmed.2013.6041

The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page