A multistage mixed-methods evaluation protocol for the national testing response during the COVID-19 pandemic in England
Authors: Naidoo Reshania , Lambert Ben , Voysey Merryn , Shretta Rima , Keene Claire , Wanat Marta , Andersen-Waine Billie , Dahal Prabin , Stepniewska Kasia , Hounsell Rachel , Molyneux Sassy , Rowe Emily , Pinto-Duschinsky Sarah , Yenidogan Gulsen , Fowler Tom , White Lisa J ,   the EY-Oxford Health Analytics Consortium
Published: October 2022

Introduction

In 2020, the UK government established a large-scale testing programme to rapidly identify individuals in England who were infected with SARS-CoV-2 and had COVID-19. This comprised part of the UK government’s COVID-19 response strategy, to protect those at risk of severe COVID-19 disease and death and to reduce the burden on the health system. To assess the success of this approach, the UK Health Security Agency (UKHSA) commissioned an independent evaluation of the activities delivered by the National Health System (NHS) testing programme in England. The primary purpose of this evaluation will be to capture key learnings from the rollout of testing to different target populations via various testing services between October 2020 and March 2022 and to use these insights to formulate recommendations for future pandemic preparedness strategy. In this protocol, we detail the rationale, approach and study design.

Methods and analysis

The proposed study involves a stepwise mixed-methods approach, aligned with established methods for the evaluation of complex interventions in health, to retrospectively assess the combined impact of key asymptomatic and symptomatic testing services nationally. The research team will first develop a Theory of Change, formulated in collaboration with testing service stakeholders, to understand the causal pathways and intended and unintended outcomes of each testing service and explore contextual impacts on each testing service’s intended outcomes. Insights gained will help identify indicators to evaluate how the combined aims of the testing programme were achieved, using a mixed methods approach.

Ethics and dissemination

The study protocol was granted ethics approval by the UKHSA Research Ethics and Governance Group (reference NR0347). All relevant ethics guidelines will be followed throughout. Findings arising from this evaluation will be used to inform lessons learnt and recommendations for UKHSA on appropriate pandemic preparedness testing programme designs; findings will also be disseminated in peer-reviewed journals and at academic conferences. This will be the first evaluation to produce a portfolio of evidence in relation to the testing effectiveness and public health impact of the national testing programme in England, encompassing behavioural, economic, equity and public health impacts. These findings will strengthen the evidence base with regards to the effectiveness of COVID-19 testing and identify which aspects are necessary to prioritise in mitigating future pandemic threats when deploying a complex public health intervention such as testing.

Transparency declaration

The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate and transparent account of the study being reported; no important aspects of the study have been omitted, and any discrepancies from the study as planned have been explained.

Strengths and limitations of this protocol

Strengths of this mixed methods evaluation protocol include the use of theory-based, complex evaluation approaches and an iterative and participatory approach with the stakeholder (UKHSA) to the evaluation process.

Given the scale and complexity of the COVID-19 testing response in England, there is a scarcity of previous relevant research, either in England or appropriate international comparators, warranting the mixed methods evaluation approach we will employ.

To the best of the authors’ knowledge, this is the first national-scale evaluation of the COVID-19 testing programme in England to incorporate the broadest scope of testing services, a programme that formed an integral part of the UK pandemic response strategy. The approach proposed could be applied to the evaluation of pandemic responses in other contexts or to other types of interventions.

Whereas most complex interventions are ideally accompanied by a prospective evaluation design initiated at the time of the intervention or earlier, this study will predominantly comprise a retrospective evaluation and is therefore limited by the quality of existing research and the data available to the research team at the time of conducting the evaluation, within the specified eight-month period allocated by UKHSA. As the UK government is in the process of consolidating data and policy related to the COVID-19 pandemic and subject to an independent inquiry, certain datasets may not be available to the researchers at the time of conducting the evaluation.

The scope of testing services to be evaluated and the selection of methods has been guided by the study sponsor team within UKHSA and must be achievable within the timeframe of the funding allocated to the study (eight months). Therefore, some trade-offs had to be made in terms of selecting research methods that would be feasible within this time constraint. For future evaluations, a mixed methods approach could be complemented by qualitative interviews with members of the public to gauge their experiences of testing and test-related behaviours, as well as an evaluation of other testing services that were out of scope for this research, including in prisons, the private sector and the events testing programme.