In this episode, the co-hosts review a guide that outlines the methods for an economic evaluation as well as a model for analysis of its results. A timely topic in a world where transparency of costs in healthcare is increasingly demanded – listen to the hosts discuss at this link.
————————————————————————–
KeyLIME Session 235
Reference
Maloney et al., AMEE Guide No. 123 – How to read studies of educational costs Medical Teacher. 2019 May;41(5):497-504.
Reviewer
Linda Snell (@LindaSMedEd)
Background
These days we live in a world of healthcare and HPE where resources are limited – choices must be made; there are increasing demands for transparency; there is more scrutiny of costs and efficiency of spending.
The goal is not necessarily low-cost education, it is optimizing education value… that ‘teaching and assessment deliver optimal educational value for a given amount spent’. Costlier may be better if there is a better outcome.
Purpose
‘…to introduce educational decision-makers to the economic concept of cost, and how to read research about educational costs in HPE to inform cost-conscious decision-making.’
Key Points on the Methods
This is a guide – there are no ‘methods’ per se.
The authors review the common study designs used for this research and how they are presented then offer a model for appraising the results of an economic evaluation.
Key Outcomes
There are a lot of helpful definitions:
A partial economic evaluation – what was spent for an activity to occur (educational inputs).
A full-economic evaluation – what was spent, and what was obtained in return (outcomes).
What goes into ‘costs’ – the ‘ingredients’
- Start-up costs – new cost for change
- Ongoing (future) costs
- ‘Sunk costs’ – past expenses
Equipment and materials: Equipment purchase, maintenance, depreciation, lifespan, donations.
Personnel cost: Academic staff; volunteers; administrative
Facility costs: rental, cost, maintenance
Required client inputs: Learner costs (transportation, meals, course registration, books, software, hardware)
Other program inputs: Information technology
Sometimes info not available and costs must be modelled.
Four questions for appraising the results of an economic evaluation (Box 3) – (similar questions to the critical appraisal of a research study)
1. Can I trust the results?
a. Were the costs and outcomes properly measured and valued? (all were considered and properly estimated – commonly overlooked is faculty time; properly quantified – e.g. hours measured with logs)
b. Was appropriate allowance made for uncertainties in the analysis?
2. What are the results telling me?
a. What were the incremental costs and learning outcomes of each strategy? How much benefit or gain can be obtained from what additional cost.
b. How much does allowance for uncertainty change the results?
3. Could these results be transferred to my context?
a. Could I expect similar learning outcomes in my situation? Enough info to see if it can be replicated; similarity of settings?
b. Could I expect similar costs in my situation? Similar contexts?
4. Should I change my practice?
a. Are the educational benefits worth the costs? Make a 2×2 table with cost and benefits
b. Does the change align with my educational context? There are many factors beyond cost and effectiveness, e.g. capacity and motivation to change, strategic goals, current capabilities.
There is an excellent glossary.
Key Conclusions
The authors conclude… ‘Data published from economic evaluations can be a powerful decision-making aid and the number of studies that examine the cost and value of HPE is increasing.
Educational decision-makers create change in teaching and learning practices, and require increasing skill in understanding, appraising, and considering study findings to optimize value.
Spare Keys – other take home points for clinician educators
We all need to think about costs.
Access KeyLIME podcast archives here