TITLE: Completeness in reporting of surrogate primary endpoints in Randomised Controlled Trials: A targeted review protocol
AUTHORS: Anthony Muchai Manyara, Sathish Venkatasamy Dhayalan, Philippa Davies, Amber Young, Imad Adamestam, Onyeka Obuaya, Valerie Wells, Christopher J Weir, Jane Blazeby, Rod S Taylor, Oriana Ciani

[DESCRIPTION]
Introduction

Using a surrogate endpoint as a substitute for a patient-relevant final outcome enables randomised controlled trials (RCTs) to be conducted more efficiently. However, there is currently no consensus-driven reporting guideline for RCTs using a surrogate endpoint as a primary outcome. Therefore, we are developing SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) extensions to improve the design and reporting of these trials. As an initial step, a targeted review will identify participants to contribute to a Delphi consensus process and document current reporting of surrogate endpoints in trial protocols and reports.

Methods and analysis

We will search for RCT reports and protocols published from 2017 to mid-2022 in six high impact general medical journals (Annals of Internal Medicine, BMJ, Journal of the American Medical Association, New England Journal of Medicine, Lancet, and PLoS Medicine) and two journals that commonly publish protocols: BMJ Open and Trials. Analysis will be done using frequencies and simple thematic analysis.

Ethics and dissemination

Ethical approval is not required. The review will support the development of SPIRIT and CONSORT extensions for reporting surrogate primary endpoints (surrogate endpoint as the primary outcome) and future evaluation of the impact of the extensions. The findings will be published in an open-access publication.

[GUIDELINES]
Introduction

Well designed, conducted and reported randomised controlled trials (RCTs) provide rigorous evidence for evaluating health interventions. To generate such rigorous evidence that meets scientific, statistical, legal, and ethical considerations, trials have increasingly become resource and time intensive [1]. Consequently, use of surrogate endpoints instead of patient relevant final outcomes can improve trial efficiency, i.e. shorter follow-up, smaller sample size hence lower cost [1]. Furthermore, surrogates that truly inform subsequent patients longer term endpoints are particularly valuable in pragmatic trials where long-term follow up is difficult - e.g., trials of interventions to reduce long term complications of diabetes. Despite this efficiency offered by surrogate endpoints, they are concerns about their limitations. Compared to patient relevant final outcomes, surrogate endpoints in trials have been found to overestimate health benefits [2]. Furthermore, some approvals of interventions based on surrogate endpoints has led to roll out of interventions with no benefit or that were harmful due to the surrogate not being in the causal pathway of the disease or unintended effects of the intervention [3, 4]. Given these limitations, trials that use primary outcomes which are surrogate endpoints should be clear on having used a surrogate, its validity in predicting intervention effect and the uncertainty and risks associated with its use [5]. However, most trials that use surrogate primary endpoints are not transparent in this regard: an analysis of 626 trials published in 2006 and 2007 found that of the 109 using a surrogate primary endpoint, 62 (57%) reported that the primary outcome was a surrogate endpoint and only 38 (35%) discussed the validity of the surrogate [6]. Therefore, we aim to update this review and analyse the current practice in the use and reporting of surrogate endpoints in RCTs. This review is part of our project that aims to develop consensus-driven SPIRIT and CONSORT extensions for reporting surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE – see [7] for full project protocol. The project has four phases: literature reviews; e-Delphi survey; consensus meeting; and knowledge translation. This targeted review is part of the literature reviews phase [8] and will serve two purposes:

To identify researchers that have used surrogate endpoints in RCTs of any design in the last five years, and invite them to participate in an e-Delphi survey
To document current reporting of surrogate endpoints in trial protocols and reports as a baseline for future evaluation of SPIRIT-SURROGATE and CONSORT-SURROGATE

Ethics and dissemination

Ethics approval is not required for this review. Findings of the review will be published in an open access journal.

Amendments

This protocol can be amended in the course of the review. When amended, a date and rationale of amendment will be provided.


Funding

The review is part of the development of SPIRIT and CONSORT extensions funded by the Medical Research Council (grant number MR/V038400/1). Onyeka Obuaya is funded by the Medical Research Council, University of Edinburgh and University of Glasgow, as part of the Precision Medicine Doctoral Training Program (grant number MR/W006804/1).

Acknowledgments

We are indebted to Prof. Amber Young (University of Bristol) for her contribution the to the planning and conduct of the CONSORT/SPIRIT-SURROGATE project and who sadly passed away in September 2022.

Timeline of the review

We anticipate the review will be completed and submitted for publication in 9 months – see Table 1
 
     Activity      Period        Literature search      July 2022        Screening       July-October 2022        Data extraction and synthesis      November-January 2023        Write-up and submission      February-March 2023   

References 
Umscheid, C.A., D.J. Margolis, and C.E. Grossman, Key concepts of clinical trials: a narrative review. Postgraduate medicine, 2011. 123(5): p. 194-204.
Ciani, O., et al., Comparison of treatment effect sizes associated with surrogate and final patient relevant outcomes in randomised controlled trials: meta-epidemiological study. BMJ : British Medical Journal, 2013. 346: p. f457.
Grimes, D.A. and K.F. Schulz, Surrogate end points in clinical research: hazardous to your health. Obstet Gynecol, 2005. 105(5 Pt 1): p. 1114-8.
Thomas R. Fleming and David L. DeMets, Surrogate End Points in Clinical Trials: Are We Being Misled? Annals of Internal Medicine, 1996. 125(7): p. 605-613.
Ciani, O., A. Manyara, and R.S. Taylor, Need for better reporting of trials with surrogate endpoints: SPIRIT|CONSORT-SURROGATE extensions. Journal of Epidemiology and Community Health, 2022. 76(9): p. 769-770.
la Cour, J.L., J. Brok, and P.C. Gøtzsche, Inconsistent reporting of surrogate outcomes in randomised clinical trials: cohort study. BMJ, 2010. 341: p. c3653.
Manyara, A.M., et al., Protocol for the development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE. BMJ Open, 2022. 12(10): p. e064304.
Manyara, A.M., et al., Scoping and targeted reviews to support development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: protocol. BMJ Open, 2022. 12(10): p. e062798.
Covidence, Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia.
Page, M.J., et al., The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 2021. 372: p. n71.


Appendix 1: Checklists

Checklist for trial protocols
 
     Item      Checklist      Completeness check – Adequately,   Inadequately or not reported        1      SPIRIT 1 [M]: State that   primary outcome(s) is considered a surrogate endpoint               2      SPIRIT 2 [M]: State the   participant/patient relevant final outcome(s) that the surrogate endpoint is   substituting and predicting for                3      SPIRIT 3 [M]: State the   practical reason(s) for using a surrogate endpoint as a primary outcome                4      SPIRIT 4 [M]: Justification   for selected surrogate: Evidence of validation                5      SPIRIT 5 [M]: Justification   for selected surrogate: Evidence of being specific to setting used e.g.,   intervention, disease, population                6      SPIRIT 6 [M]: Clarify if the   sample size calculation is explicitly informed by statistical metrics of   surrogate validity (such as the surrogate threshold effect (STE) or its   equivalent)               7      SPIRIT 7 [N]: State if trial   participants will be informed before enrolment that trial was powered to evaluate   an intervention’s effect using a surrogate endpoint (rather than a patient   relevant final outcomes)               8      SPIRIT 8 [M]: Comment on   whether the trial sample size and follow up period is sufficient to   adequately capture potential harms of the intervention being tested                9      SPIRIT 9 [N]: State if there   explicit to plans to extend follow up or conduct subsequent analyses/studies   to verify benefit of current findings on the patient relevant final outcome          
 
Checklist for trial reports
 
     Item      Checklist Items      Completeness check – Adequately,   Inadequately or not reported        1      State that primary outcome(s)   is considered a surrogate endpoint                2      State the participant/patient   relevant final outcome(s) that the surrogate endpoint is substituting and   predicting for                3      State the practical reason(s)   for using a surrogate endpoint as a primary outcome                4      Justification for selected   surrogate: Evidence of validation                5      Justification for selected   surrogate: Evidence of being specific to setting used e.g., intervention,   disease, population                6      Clarify if the sample size   calculation was explicitly informed by statistical metrics of surrogate   validity (such as the surrogate threshold effect (STE) or its equivalent)               7      State if trial participants   were informed before enrolment that trial was powered to evaluate an   intervention’s effect using a surrogate endpoint (rather than a patient   relevant final outcomes)               8      If the primary outcome is a   composite outcome that includes a surrogate endpoint, report the intervention   effect on all components               9      Comment on whether the trial   sample size and follow up period is sufficient to adequately capture   potential harms of the intervention being tested                10      State if there are explicit   to plans to extend follow up or conduct subsequent analyses/studies to verify   benefit of current findings on the patient relevant final outcome               11      Provide an estimate of the   predicted effect based on the observed effect on the surrogate endpoint, and   if not possible then a qualitative assessment                12      Interpretation of findings of   the trial in the context of using a surrogate primary endpoint including its   known validity and the potential benefit-risk ratio of the tested   intervention for participants                13      If surrogate endpoint and   patient relevant final outcome data were collected in the trial, state the   open access arrangements for the data for future secondary research including   the statistical evaluation of the surrogate          
 

[STEPS]
SECTION: Working definition
1. Our working definition of a surrogate endpoint is ‘A biomarker or intermediate outcome used to substitute for a patient or participant relevant final outcome (i.e., severe morbidity; health related quality of life or mortality) and reliably predicts benefit or harm based on epidemiologic; therapeutic; pathophysiologic; or other scientific evidence’. As per this definition, we will consider intermediate outcomes including symptoms and functional outcomes to be surrogate endpoints – see Figure 1 for health outcome framework below.   
 
  
 
SECTION: Search strategy and study selection
2. We will include RCT reports, and protocols of pragmatic and explanatory trials that use a primary outcome which is a surrogate endpoint (including if it’s part of a composite outcome) published in the last five years (2017 to mid-2022) in six high impact general medical journals (Annals of Internal Medicine, BMJ, Journal of the American Medical Association, New England Journal of Medicine, Lancet, and PLoS Medicine) and two journals that commonly publish protocols: BMJ Open and Trials. 
SECTION: Search strategy and study selection
3. Journal names will be searched through PubMed and searches limited to “randomised controlled trial” and publication years (2017 to June 2022). 
SECTION: Search strategy and study selection
4. Identified records will be exported to Covidence [9] where title, abstract and full text screening will be done by two reviewers. 
SECTION: Search strategy and study selection
5. Study selection will be performed independently by two reviewers with a third reviewer providing a decision where there is lack of agreement. 
SECTION: Inclusion criteria
6. Full RCT protocols or reports including those describing follow up of completed trials. 
SECTION: Inclusion criteria
7. Primary outcome is a surrogate endpoint (self-reported or considered so by reviewers) including clinical scales that measure both surrogate endpoints (e.g., symptoms and functional outcomes) and final outcomes (i.e., severe morbidity or death). Where the primary outcome is not explicitly stated, but all the outcomes are considered surrogates, such articles will also be included. Furthermore, trial protocols or reports whose primary outcome is a composite that includes a surrogate endpoint will be included. Finally, records with multiple final outcomes that include a surrogate endpoint will be included.
SECTION: Exclusion criteria
8. We will exclude the following RCT protocols and reports:
SECTION: Sample size and data extraction
9. Based on the total number of included and relevant RCTs (protocols and reports), we will calculate the proportion of these studies that are based on surrogate primary outcome.
SECTION: Sample size and data extraction
10. From the included RCTs, we will select a random sample of 100 protocols and 100 reports that used a surrogate endpoint as a primary outcome for data extraction and synthesis.
SECTION: Sample size and data extraction
11. Data from sample of included trials and protocols will be extracted independently by a single reviewer and checked for accuracy by a second reviewer.
SECTION: Sample size and data extraction
12. We will extract study characteristics such as author, journal, year of publication, RCT hypothesis (superiority/non-inferiority/equivalence), RCT design (parallel, cross over, cluster, and number of arms), study setting (e.g., primary care, community), study country, clinical/research area, type of surrogate (i.e., biomarker or intermediate), intervention type (e.g., drug, screening), comparator (e.g., placebo, usual care), sample size, length of follow-up and funder – see https://osf.io/jr53b.
SECTION: Sample size and data extraction
13. We also extract data on completeness of reporting (see below).
SECTION: Assessment for completeness in reporting
14. We will use a list of reporting items synthesized from our recent scoping review to analyse for completeness in reporting of trials and protocols.
SECTION: Assessment for completeness in reporting
15. Items include statement of using a surrogate endpoint, rationale, and justification for using the selected surrogate, use of surrogate treatment effect or equivalent in sample size calculation, interpretation of findings in the context of limitations of surrogate endpoints among others – see full list of items in Appendix 1.
SECTION: Assessment for completeness in reporting
16. Initially we will do a pilot on the data extraction tool using six random records. Where possible and applicable, text will be extracted from papers to support coding decision.
SECTION: Assessment for completeness in reporting
17. After the pilot, synthesis for completeness will be done for all records by one reviewer and validated by a second reviewer.
SECTION: Data synthesis and presentation
18. Given the nature of this review, synthesis of results will primarily be descriptive. Presentation of results will include PRISMA flow-chart [10] of the whole process, counts and percentages summarising study characteristics and completeness of reporting per item, and if need be, tabulation of text to support assessment of completeness judgements.
SECTION: Exclusion criteria
8.1. with two primary (co-primary) outcomes that include both a surrogate and a final patient-relevant outcome, e.g., progression free survival (surrogate) and survival (final outcome).

SECTION: Exclusion criteria
8.2. with a primary outcome that is a clinical scale that measures both surrogate endpoints (symptoms and functional outcomes) and final outcomes (severe morbidity or death) is dichotomised to compare severe morbidity versus moderate/no morbidity than we will consider it a final outcome and exclude.
SECTION: Exclusion criteria
8.3. that are  feasibility or pilot trials with solely feasibility objectives.
SECTION: Exclusion criteria
8.4. with a primary outcome that is solely safety.
SECTION: Exclusion criteria
8.5. that solely report cost or process evaluation analyses.
SECTION: Exclusion criteria
8.6. reporting a secondary analysis of a trial.
SECTION: Exclusion criteria
8.7. reporting non-health outcomes e.g., intelligence quotient.