Successful scholarship is a team sport, and culture is paramount in determining successful research outcomes. But what about culture? Read on, and check out the podcast here.
————————————————————————–
KeyLIME Session 189:
Listen to the podcast.
Reference:
Rola Ajjawi, Paul ES Crampton & Charlotte E Rees What really matters for successful research environments? A realist synthesis Journal of Medical Education 2018 [ePub ahead of print]
Reviewer: Jonathan Sherbino (@sherbino)
Background
(Editor’s Note: This paper is reviewed as part of an agreement between KeyLIME and Medical Education. The KeyLIME hosts receive a number of accepted articles pre-publication . We choose a single paper and review it. There are no restrictions on the process; we have editorial autonomy. There is no financial relationship.)
While research prizes (e.g. the Karolinska – the most prestigious prize for medical education research) acknowledge an individual scholar (and believe me each of the past winners deserve the acknowledgement!!), the reality is that successful scholarship is a team sport. Important and rigorous scholarship requires collaboration, resources, protected time, internal review, mentorship, leadership, institutional valuation… etc. etc. etc. In other words, culture is paramount in determine successful research outcomes. I guess that seems obvious. But what elements of culture? How do you unpack this abstract construct? Good news… this is the purpose of this article.
The context for this review is the massification of education, increasing competition for funding and a managerial culture in higher education that adopts an actuarial approach (e.g. quantification of impact as determined by superficial measures that lead to determination of future support).
Purpose
“While several narrative and systematic reviews have begun to identify the characteristics of research-favourable environments, these reviews have ignored the contextual complexities and multiplicity of environmental characteristics. Therefore, the current synthesis adopts a realist approach to explore what interventions work for whom and under what circumstances.”
Key Points on Method
This systematic review uses a realist approach as the guiding conceptual framework. (As background, a realist review borrows techniques familiar to a systematic review. However, it diverges from the goal of finding an overarching, unifying, explanation to answer “what works,” “for whom” and “in what context?” A realist review looks for middle-range theories (cf Merton) that have enough abstraction to be distinct from the abundant minor hypotheses that are assumed within a program of research, yet provides enough specificity to be useful and easily tested, when compared to an abstract, unifying theory. A realist synthesis acknowledges that an intervention is influenced by context to trigger an outcome. The relationship between intervention and outcome is not direct. A realist synthesis attempts to describe the context-mechanism-outcome configuration. By nature, realist syntheses are typically iterative. Realist philosophy is situated between positivism and constructivism.)
This study was exemplary in following the RAMESES (Realist and Meta-narrative Evidence Synthesis: Evolving Standards) Guidelines.
- Clarifying scope: Search terms for relevant literature were informed by a UK higher education research excellence framework that articulates key elements. The introduction also includes a number of previous systematic reviews that helped refine the research question.
- Search for evidence: Medline, ProQuest, Scopus, CINAHL and Web of Science were searched from 1992 to 2016 with a comparison across medical education, education and medicine. An initial search revealed 8500+ articles that was reduced to 420 after title and abstract screening. Included articles were reviewed in detail by a single reviewer. Ambiguity was checked by a second reviewer. Ten percent of all articles were reviewed by a second reviewer to ensure reliability of the inclusion process.
- Quality assessment: Manuscripts were screened for relevance (“contributes to theory building and/or testing”), leaving 100 articles. All of these articles were original research and excluded reviews or commentaries. The MERSQI (quantitative framework) and CASP (qualitative framework) were used. 47 articles remained after this step. 5 additional papers were excluded because they lacked a CMOC. 42 papers were included in the review.
- Data extraction: CMOCs were coded using a common code developed by consensus by the authors. The code was iteratively modified as a component could be conceptualized as either a mechanism or outcome depending on the framing of the study.
- Data synthesis: An iterative approach to the determination of patterns and themes as adopted and compared to the initial program theory (based on previous literature reviews) to modify this theory.
Key Outcomes
4 papers were from medical education, 18 from education and 20 from medicine. 26 were qualitative studies, 11 quantitative studies and 5 mixed-methods. The contexts included all inhabited continents except South America.
CMOCs were consistent across disciplines, while an intervention could fire either positive or negative mechanism leading to a positive or negative outcome, all dependent upon context.
The following four interventions could be either a positive or negative mechanism, depending on context:
- A research strategy was positive if it enabled resources and valued research, yet could be negative if excessively emphasizing outputs, rewards or incentives.
- Researchers require internal motivation, protected time and access to capacity building activities to be successful. Knowledge deficits, poor leadership support, and limited networks inhibit an individual researcher.
- Resources (income, infrastructure, facilities) were overwhelmingly identified as positive CMOCs, with their absence impairing research.
- Collaboration was crucial in a trusting, respectful, supportive relationship. Competitive cultures or cultures with poor communication undermine collaboration leading to decreased productivity.
The following three interventions were positive mechanisms:
- Time (e.g. protected time, workload pressures)
- Identity (e.g. professional and social identity of the individual as it relates to their research to ensure motivation and engagement)
- Relationships
No outcomes were identified relating to research impact (e.g. citation indices, number of grants etc.), despite the predominance of this metric of research success.
For the interested reader Figure 3 graphically shows the interplay among all of the CMOCs.
The authors acknowledge a significant limitation: the search strategy was limited to ensure feasibility.
Key Conclusions
The authors conclude…
“Researchers, research leaders and research organisations should prioritise protected time for research, enculturate researcher identities, and develop collaborative relationships to better foster successful research environments. Future research should explore further the interplay between time, identity and relationships.”
Access KeyLIME podcast archives here