In this episode, Jon takes a break from the KeyLIME formula and goes back to the archives. While the gang normally choses papers hot off the press, they chose this literature from 2008 that focuses on Internet-based instruction – certainly a timely topic as many struggle to transition their curricula on-line. Learn more about the study here.
————————————————————————–
KeyLIME Session 262
Reference
Cook et. al., Internet-Based Learning in the Health Professions A Meta-analysis JAMA. 2008 Sep 10;300(10):1181-96
Reviewer
Jon Sherbino (@sherbino)
Background
Ok. Let’s start with some obvious statements. Times are strange. Innovation is required. And everything education is going on-line.
So, is everything better with tech? (Yep, a leading question, but I have the mic here.). What is special about an on-line platform? What elements of effective learning are inhibited or enhanced with an on-line forum compared to an in-person forum?
In this episode we break slightly from the KeyLIME formula and go back to the archives. Normally, we choose papers hot-off-the-press. But to help KeyLIMErs wrestling with COVID and the transitioning of curricula on-line, we’re turning to literature from 2008. (Cue the irony about the future of learning by looking at literature from the past.)
Purpose
“To summarize the effect of Internet-based instruction for health professions learners compared with no intervention and with non-Internet interventions.”
Key Points on the Methods
In brief, the methods are exemplary, adhering to the PRISMA guidelines.
Key points:
- Studies in any language from 1990 (WWW came on-line in 1991) to 2008 were included.
- Studies must report an evaluation using Kirkpatrick’s hierarchy of outcomes and have a comparator (including NO INTERVENTION).
- Single group pretest-postest
- 2 group randomized and nonrandomized
- Parallel group
- Cross over designs
- Or “tack-on” studies
- 9 databases searched (all of our favourites)
- Studies selected in a tiered fashion, independently by 2 raters (ICC 0.71). Final agreement was by consensus.
- A standard data extraction form used, where judgement was required, data was independently abstracted (ICC 0.77).
Key Outcomes
From an initial search of 2045 citations (+ 148 via hand search), 201 articles were selected.
Heregeneity was high (12 >0.79); thus, a random effects model was used for the meta-analysis.
Study quality (scored using nodified Newcastle-Ottawa scale for cohort studies) was poor to moderate.
Content included:
- diagnostic (e.g. ECG, PFT) and therapeutic (e.g. medications) content ,
- basic sciences (e.g. histology, anatomy),
- ethics,
- evidence-based medicine, conduct of research and biostatistics,
- communication skills, and
- systems-based practice.
Health professionals included:
- medical students,
- medical residents,
- physicians in practice,
- nursing students,
- nurses in practice,
- dental students,
- dentists in practice,
- pharmacy students, and
- pharmacists in practice
Internet interventions (mainly) included:
- self-study modules
- virtual patients
- on-line discussion with peers or instructors
Education techniques used by internet interventions included:
- high interactivity
- practice exercises
- repetition with longitudinal access
- tutorial and group discussion
Non-internet interventions (mainly) included:
- in-person course
- paper modules
- SPs
For passive control studies, internet based interventions had a large effect.
- 00 (95%[CI], 0.90-1.10; P < .001; n=126 studies) for knowledge outcomes,
- 85 (95% CI, 0.49-1.20; P _<.001; n=16) for skills,
- 82 (95% CI, 0.63-1.02; P <.001; n=32) for learner behaviors and patient effects.
For comparison to in-person control, internet-based interventions demonstrated equivalence.
- 10 (95% CI, −0.12 to 0.32; P=.37; n=43) for satisfaction,
- 12 (95% CI, 0.003 to 0.24; P=.045; n=63) for knowledge,
- 09 (95% CI, −0.26 to 0.44; P=.61; n=12) for skills
- 51 (95% CI, −0.24 to 1.25; P=.18; n=6) for behaviors or patient effects.
No important treatment-subgroup interactions were identified. Notably, internet-based interventions with greater repetition had lower effect sizes than balanced repetition. The authors caution that the multiple comparisons conducted in the subgroups increase the risk for bias and random effect.
Key Conclusions
The authors conclude…
“Although conclusions must be tempered by inconsistency among studies and the possibility of publication bias, the synthesized evidence demonstrates that Internet- based instruction is associated with favorable outcomes across a wide variety of learners, learning contexts, clinical topics, and learning outcomes. Internet-based instruction appears to have a large effect compared with no intervention and appears to have an effectiveness similar to traditional methods.”
Spare Keys – other take home points for clinician educators
As the authors allude to in the limitations section, internet-based learning is merely a platform. It allows scale, asynchronous access, and gives voice to non-traditional teachers, among other things. See, for example, MOOCs, social media and FOAMed. However, the design of internet-based resources is dependent on the structure of best learning practices/theory and must be attentive to context/content. As an analogy, the first person to use a Kindle did not write a Pulitzer-prize winning novel.
Access KeyLIME podcast archives here
The views and opinions expressed in this post and podcast episode are those of the host(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page