Blogging about Blogs: Curating High-Quality Blogs and Podcasts for Graduate Medical Education – The ALiEM AIR Series

Hi everyone, this is Teresa Chan reporting for the ICE blog. In the lead up to medical education’s first ever social media consensus conference, ICE has been connecting with various online education groups to find out what they’ve been up to.

Today, ICE talks to the Academic Life in Emergency Medicine (ALiEM) blog. With > 50k visits per month, ALiEM is a prominent education organization. While I am an associate editor at ALiEM, I still think what they’ve done is pretty cool, especially in light of this Annals of Emergency Medicine article, which highlights the need for integration of social media into residency curriculae. [1]

Until next time. – Teresa (@TChanMD)

By Andy Grock,  Nikita Joshi (@njoshi8), Eric Morley (@emericmorley) and Michelle Lin (@M_Lin)

 In 2008, the Council of Residency Directors (CORD) in Emergency Medicine and the Accreditation Council for Graduate Medical Education (ACGME) endorsed a change in the education hours required for Emergency Medicine (EM) residency training programs. This new recommendation (PDF) allows residencies the option to decrease their conference time from five to four hours per week, in exchange for one hour of asynchronous learning outside the classroom. This one-hour is termed Individualized Interactive Instruction (III).

For asynchronous educational material to be eligible for III credit, the ACGME states that all four of the following criteria should be fulfilled:

1. The program director must monitor resident participation.
2. There must be an evaluation component.
3. There must be faculty oversight.
4. The activity must be monitored for effectiveness.

Anecdotally,residency programs have struggled to determine what digital and print-based resources might be eligible for III learning. Major obstacles include identifying high-quality, educational content pertinent to resident education, providing ongoing faculty oversight, and monitoring resident participation.

Because Academic Life in Emergency Medicine (ALiEM) is the most-read educational blog by U.S. EM residents [2], we sought to address these issues on a national level. What we came up with was the Approved Instructional Resources (AIR) series.

We decided to filter the often-overwhelming amount of online educational content currently available, rather than create new content. To approach this in a more rigorous process, we first created a 9-person editorial board with graduate medical education leaders from across the country to help peer-review and identify quality content. We follow the monthly systems-based curriculum already established and in use by CORD.

Selection of AIR Module Content

To create the list of highlighted blogs/podcasts for the module, we followed these operational steps:

1. We first compiled a list of relevant blog posts and podcasts published within the past 12 months. These posts and podcasts were curated based on the Social Media Index (SM-i), which was previously created by the ALiEM team to help identify the most popular open-access blogs and podcasts in EM and Critical Care. Content in these resources, presumably are already vetted by the “crowd” as high quality and valuable based on crowdsourcing principles.
2. This initial list of blogs/podcasts are submitted to the entire AIR editorial board to grade using our AIR Grading Instrument (see below).
3. The best blogs/podcasts are selected, based a clear cut-point in the scores.
4. A multiple choice question is created for each selected blog/podcast using Google Forms.
5. An ALiEM blog post announces our selection, along with the embedded quiz which automatically collects user data.

AIR Grading Instrument

We created a scoring tool consisting of five categories, using a seven-point scale with behavioral anchors. The five categories were:

1. Best Evidence in Emergency Medicine (BEEM) Score [3]: Assuming that the results of this article are valid, how much does this article impact EM clinical practice? Although this rating system was validated for journal publications, the rating scale seemed to be relevant for blog/podcasts as well.
2. Content accuracy: Do you have any concerns about the accuracy of the data presented or conclusions of this article?
3. Educational utility: Are there useful educational pearls in this article for residents?
4. Evidence based medicine (EBM): Does this article reflect EBM and thus lack bias?
5. Appropriate references: Are the authors and literature clearly cited?

Returning back to the III criteria discussion, we thus feel that the AIR Series meets these criteria.

1. The program director must monitor resident participation. We give read-access to program directors or their designee to the Google Docs that records resident completion of the module.
2. There must be an evaluation component. We have a post-module knowledge-based quiz.
3. There must be faculty oversight. Not only are the AIR Editorial Board screening the content for accuracy and resident value, but we are also replying to resident questions in timely fashion (typically within 24-48 hours) on the blog. Automatic email-notifications through the blog comments section makes this possible.
4. The activity must be monitored for effectiveness. We have an internal quality improvement process as well as a post-module survey to incorporate learner feedback.

To date, the AIR Series is the first national effort in EM to provide a free, rigorously-designed resource to curate high-quality blogs and podcasts specifically for graduate medical education. By publishing modules on a monthly basis, we hope to minimize the siloed, redundant efforts spent on the local levels to address the III issue.

——————-

References

1. Scott, K. R., Hsu, C. H., Johnson, N. J., Mamtani, M., Conlon, L. W., & DeRoos, F. J. (2014). Integration of Social Media in Emergency Medicine Residency Curriculum. Annals of emergency medicine. In press. PMID: 24957931
2. Mallin M, Schlein S, Doctor S, Stroud S, Dawson M, Fix M. A survey of the
   current utilization of asynchronous education among emergency medicine residents in the United States. Acad Med. 2014 Apr;89(4):598-601. PMID: 24556776
3. Carpenter CR, Sarli CC, Fowler SA, Kulasegaram K, Vallera T, Lapaine P, Schalet G, Worster A. Best Evidence in Emergency Medicine (BEEM) rater scores correlate with publications’ future citations. Acad Emerg Med. 2013 Oct;20(10):1004-12. PMID: 24127703