Title: Improving the evidence base of healthcare policy interventions through the stepped-wedge cluster randomised trial using mixed methodologies
Funder: Department of Health and Social Care

Abstract: Background: The stepped-wedge-cluster-randomised-trial (SW-CRT) is a novel randomised design in which an evaluation of an intervention is performed in the midst of a routine roll-out. The design presents an unprecedented opportunity to increase the robustness with which healthcare-policy interventions are evaluated and there has been a large growth in the numbers of these trials.However, there is a dearth of methodological-research on the SW-CRT, and the research that does exist has concentrated on sample-size. There is almost no methodological research on the conduct and analysis of the SW-CRT, and this means these studies are at risk of selection biases (due to poor conduct of the trial) and analysis biases (due to poor analysis of the data).Analysis and selection biases are both potential risks in a SW-CRT due to the fact that the study is run over an extended period of time, crucially with more clusters exposed to the intervention during later periods of the study. Analysis biases are a threat to the validity of the estimate of the treatment effect as analysis must adjust for secular trends - to avoid misattributing a secular trend to the treatment effect. Selection biases are a threat to validity because recruitment or identification of participants often has to be made with knowledge of treatment allocation thus risking lack of allocation concealment.Aims: This research will identify how trialists using a SW-CRT can avoid compromising estimates of treatment effects due to selection and analysis biases. Specifically: Objective 1 (WP1): How can design and analysis of a SW-CRT mitigate potential biases due to temporal trends? 1a: Identify how to effectively adjust for underlying temporal trends at the analysis stage. 1b: Identify how to restrict the random allocation so as to effectively balance key covariates.Objective 2 (WP2): How can the conduct of a SW-CRT mitigate potential biases due to selection biases? 2a: Identify whether published SW-CRTS are at risk of selection biases. 2b: Identify the determinants and barriers to selection biases. 2c: Provide a narrative synthesis of the recommendations on when it is appropriate to not obtain participant consent to mitigate selection biases. Objective 3 (WP3): When can the SW-CRT provide a similar level of evidence to that of a parallel cluster-randomised-trial (CRT)? 3a: Identify, using optimal allocation and analysis methods, under what circumstances the estimate of treatment effect from a SW-CRT is as robust as that of a parallel-CRT. 3b: Identify, in real life scenarios, when the estimate of treatment effect from a SW-CRT is as robust as that of a parallel-CRT. Plan of investigation: These separate and bespoke work-packages will use a mixture of mathematical simulation, empirical and narrative reviews, and qualitative research.Extensive simulation studies will explore the degree of robustness of commonly used analysis methods to provide an accurate estimate of the treatment effect and whether other analysis models can add any improvement. Methodological reviews will be conducted to explore actual threats of selection bias; and a qualitative study will explore barriers to overcoming these threats. A narrative review will synthase recommendations on when it is appropriate to take a complete enumeration of the cluster, using waiver of consent, to mitigate selection biases.Findings from these two work-packages will then be triangulated to determine when the level-of-evidence generated from a SW-CRT is as robust as that from a parallel-CRT, under both optimal and real-life scenarios.Benefits to the NHS: This fellowship aims to fill a methodology-gap and will provide the foundation for pragmatic evaluations of NHS healthcare-policy interventions, leading to better decisions of how to run these trials, to ultimately benefit patients.

Lay abstract: Why this research is needed:Researchers constantly look for ways to improve patient's health and quality-of-life. Before new treatments or polices are introduced, they need to be tested. Researchers need to make sure they conduct these testing studies carefully. A particular intervention might be effective and improve the quality-of-life for patients, but if the results of the study have not been measured properly, people who make decisions about care may make incorrect decisions.One particular type of study design is called a stepped-wedge-cluster-randomised-trial (SW-CRT). This is a new study design which is particularly valuable for evaluating changes in the way care is delivered. The new policy is gradually rolled-out so that every group (e.g. hospital) receives the new way of care in the end.More and more studies are now using the SW-CRT. However, more research needs to be done so we can better understand the study and how it can be used in the best ways to get the best results.What this research aims to do: At the end of this research, we will know more about the stepped-wedge-cluster-randomised-trial. We will know: How to design the study so that groups of people or data that are included before the change in policy are very similar to those included after the change in policy.How to measure and look at the results from the study so researchers can be more confident about whether the new policy is working. Making sure we can compare apples with apples In studies where participants are split into groups, it is important that the different groups are as similar as possible, before and after the new policy is rolled-out. The SW-CRT study design seems to be more prone to this not happening and researchers need to know the best ways to ensure this doesn't happen. One option, for example, is to include data from all people in the study, even if they don't know they are included or may not have given consent. Because the people in the study won't be directly getting any new medicines, this might be considered to be a reasonable approach. Making sure we can make the best and most accurate measurements Sometimes it is difficult for researchers to work out whether it is the specific intervention that is working or whether other changes are taking place. In a SW-CRT, researchers collect more data from groups that get the intervention at a later stage in the study (the SW-CRT is gradually rolled-out to all groups). Researchers need to understand how to analyse results of the study so they only measure whether the new policy is effective. They need to avoid measuring other changes that might be taking place at the same time.How we will do the research In the study, we will learn more about SW-CRT study design. We will: Find out whether there is more of a risk that the groups that take part in the study are different;Find out when researchers should ask people if they agree to being part of a study;Learn more about how to analyse results so that only the impact of the policy is analysed. We will use different ways to learn more about the study. This includes:Doing mathematical simulations;Looking at what we already understand about the study;Interviewing people who use the study;Asking patients and the public who might be included in a study. We will come up with guidelines and recommendations on how to use SW-CRT and how to analyse the results. What will the research mean?Doing any study costs money, and if we know more about the study and the best ways to use it, the study will be more effective. Our guidelines and recommendations will make sure that researchers do the SW-CRT in the best way and that they can make the best measurements and analysis. Researchers will be more confident in the results, which will mean that people who make decisions on care will be more confident in using the results.Ultimately, this means that when decisions are made that affect patient care, the options have been tested carefully and measured effectively so we know they work and have the potential to improve the health and quality-of-care for patients.Sharing our findings We will make sure that the results of this research are communicated in accessible ways to different groups: Researchers that use these studies; People that commission studies; Patients and groups that might be involved in a study.

Research activity codes: 8.4 RESEARCH DESIGN AND METHODOLOGIES (HEALTH SERVICES)
Health categories: GENERIC HEALTH RELEVANCE