Title: Developing Realist Economic Evaluation Methods (REEM) and Guidance to Evaluate the Effectiveness, Costs, and Benefits of Complex Interventions.
Funder: Department of Health and Social Care

Abstract: Background: It is increasingly recognised that the ‘big problems’ in health and social care require well-designed complex solutions and robust evaluation, which itself is often complex. Realist evaluations were designed to take account of that complexity, offering an explanation of ‘what works, for who, in which circumstances and why?’. Concurrently, policymakers and research funders require economic appraisals to accompany evaluations of complex interventions to inform difficult decisions in the context of resource scarcity. However, realist evaluations do not tend to explicitly capture the economic consequences of interventions and economic evaluation methods often ignore context and do not capture variations in resource use or outcomes across groups, or recognise the implications this may have for the relationship between resource use and outcomes This research will develop realist economic evaluation methods (REEM) and guidance to better understand and evaluate the benefits and costs of complex interventions. It will bring together realist and economic evaluation to enable evaluators to establish what works, for who, in which circumstances, why, and with what related resource impacts and opportunity costs. Research Question and Research Plan: This research addressed the following questions in three phases: • Phase 1 (Months 0-12): What are the theoretical, methodological and practical similarities and differences between realist and economic evaluations? This phase will use literature scoping, online discussion boards (including short activities), and facilitated virtual meetings to develop provisional REEM guidance for piloting. • Phase 2 (Months 12-30 months): What lessons can we learn from using REEM in practice to improve it? This phase will pilot REEM and investigate it’s feasibility in research across three evaluations. • Phase 3 (Months 30–36): How can we use empirical and expert knowledge to produce consensus REEM guidance? This phase will use a Consensus Development Conference with Modified Delphi to agree the core elements of REEM and produce finalised guidance. Patient and Public Involvement and Engagement (PPIE): PPIE will be co-led by a Public Involvement & Community Engagement Manager and member of the public. As well as being involved in the research throughout, PPIE members will have the opportunity to engage in a university level module on PPIE in research and will co-produce PPIE summary notes for REEM. Dissemination and anticipated impact: The outputs from this research will be REEM guidance, a checklist and summary notes for use by PPIE members. These outputs will allow REEM to be utilised by academics and scrutinised by research funders. Additional outputs include: peer reviewed academic journal articles, conference presentations, webinars for research funders policy makers and commissioners. Short term impact will be facilitated through: stakeholder involvement from the outset of the research, development of international academic networks, use of established international academic reach. Long term impact will include the use of REEM to: make better-informed commissioning and policy decisions; reduce research waste, and targeting interventions to those that need them.

Lay abstract: Background When people get sick or injured, or need help to live at home or to take part in the community, services like hospitals, health centres and social services are there to help them. Different treatments and services work for different people. Some kinds of research help to work out what works best for different people in different situations. Research can also help to work out which services provide the best value for money - which might be different for different people. The information from research helps politicians and government departments to make good decisions about what services should be funded. It also helps service providers (doctors, nurses, social workers and so on) to work out exactly what services and treatments should be provided for different people. Providing the right services to the right people leads to better outcomes. At the same time, it makes better use of funding for health and social care. Evaluation is one type of research. Until now, two different kinds of evaluation have been used to look at ‘what works for whom’, and what provide the best value for money. Realist evaluation tries to understand what works for which people, in what circumstances, exactly how it works and why it doesn’t work for other people. Economic evaluation tries to understand the costs and the benefits of services, in financial terms. This research project wants to work out how to bring the two types of evaluation together. Combining them will help researchers to work out what works, for whom, in what circumstances, at what cost, and with what benefits. Combined, the methods will be called Realist Economic Evaluation Methods (REEM). The project will also develop advice (called ‘guidance materials’) on how to use the methods. How will we do this? The project will be conducted in three phases. The first phase will review existing evidence about the problems in bringing the two kinds of evaluation together, and what has been learned from previous attempts to do so. The findings will be provided to groups of experts from different countries. Together, they will work out how to bring the two types of evaluation together, what research methods could be used, and how to use them. This stage will provide draft guidance materials. The second phase will trial the combined methods in three small evaluation studies. The aim is to test the methods and refine how to use them in practice. We want to work out their strengths and weaknesses, and the value of using them. Service staff, patients and the public will be involved in each of the studies. The third phase will bring together the lessons from the studies. We will bring the experts together again to reach agreement about what has been learned and what that means for how to use REEM in future. We will revise the draft guidance materials. The guidance will be shared with other researchers and evaluators, the organisations that fund research, and decision-makers in health and social care and in government departments. Why should we do this? We believe that REEM will help provide better information for decision-makers. This is important because health and social services are expensive and there is never enough money to pay for everything. Better informed decisions should result in better services, better outcomes, and better value for money. Five patient and public representatives reviewed and helped to edit this information sheet.

Research activity codes: 8.4 RESEARCH DESIGN AND METHODOLOGIES (HEALTH SERVICES)
Health categories: GENERIC HEALTH RELEVANCE