Title: Evaluation of the drugs Recovery Payment by Results Programme of Piloting
Funder: Department of Health (England)

Abstract: The process evaluation will provide an in-depth understanding of how and whether the PbR model works, and its impact on the wider landscape of provision of related services. We will undertake a multi-faceted process evaluation, providing detailed case-studies of the eight PbR pilots, comparing and contrasting approaches. Phase one will undertake a selective review of the relevant research literature and a mapping exercise using key informant interviews, existing administrative data, local needs assessments and emerging policy and practice documents. Changes resulting from the introduction of PbR will be captured. The initial stage of the research will include a strong formative element: feeding back to key messages on the implementation process, so that early findings can contribute to the development of the pilots. Phase two will include an extensive programme of in-depth interviews with purposively sampled stakeholders and users to gauge their views and experiences of the development and delivery of the PbR pilots. IPCR and RAND have high levels of technical competence in evaluation, qualitative research in relevant fields, and in the analysis of data, including structured, thematic analysis of large qualitative data-sets. As the evaluators for the Peterborough Social Impact Bond, the team has unique expertise in the evaluation of emerging PbR models, has proven skills in drawing out the implications of this research for policy and practice, and in supporting the implementation of evidence-based change. The impact evaluation will assess the extent to which individual client outcomes are improved in PbR pilot areas. To achieve sufficient statistical power, centrally available sources of secondary data will be used to compare client outcomes in pilot areas with those in comparator sites. During the scoping phase the team will consult with stakeholders to confirm access to suitable sources of secondary data. Thereafter, samples will be drawn from treatment data and, using privacy-protecting techniques, will be matched with secondary datasets including Police National Computer and ONS mortality records. Survival /time-to-event analyses will determine whether outcomes in PbR areas are improved and, if so, quantify the observed differences. The team has long-standing experience in this area and includes expertise in addiction epidemiology, in linkage of secondary datasets to assess impact, in developing key treatment indicators and research to develop methods to measure the impact of treatment on mortality and offending. The research will provide an early indication for policymakers of the impact of the PbR pilots and will develop approaches suitable for future national use in surveillance of PbR schemes. The economic component will examine the effects of the scheme on commissioner and provider behaviours. We will identify matched comparator sites, and use a difference-in-differences design to derive estimates of the consequences of introducing PbR. Market analysis will then be undertaken in the pre- and post-intervention periods and in the pilot and comparator sites, using a variety of measures of market concentration (number of providers and the Herfindahl index). The consequences for service users will be measured on three sets of metrics: the incentivised metrics; metrics that are related to the incentivised metrics in the key domains of interim and final treatment outcomes, re/offending, employment, and health; and measures that reflect aspects of social value outside these key domains. The latter two groups will be analysed to see whether providers focus on the targets but miss the point and whether there is diversion of effort away from unincentivised domains and into incentivised domains or whether there are positive spillovers onto these areas. As well as evaluating the average effects of the scheme, we will examine the distributional impact of the scheme across types of service users.


Research activity codes: 8.1 ORGANISATION AND DELIVERY OF SERVICES
Health categories: STROKE;NEUROLOGICAL;RESPIRATORY;METABOLIC AND ENDOCRINE;CARDIOVASCULAR