# Methods and results used in the development of a consensus-driven extension to the Consolidated Standards of Reporting Trials (CONSORT) statement for trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE)

Mahrukh Imran, Linda Kwakkenbos, Stephen Mccall, Kimberly Mccord, Ole Fröbert, Lars Hemkens, Merrick Zwarenstein, Clare Relton, Danielle Rice, Sinéad Langan, et al.

► To cite this version:

Mahrukh Imran, Linda Kwakkenbos, Stephen Mccall, Kimberly Mccord, Ole Fröbert, et al. Methods and results used in the development of a consensus-driven extension to the Consolidated Standards of Reporting Trials (CONSORT) statement for trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE). BMJ Open, 2021, 11 (4), pp.e049093. 10.1136/bmjopen-2021-049093. hal-04369495

HAL Id: hal-04369495

https://hal.science/hal-04369495v1

Submitted on 6 Feb 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# BMJ Open Methods and results used in the development of a consensus-driven extension to the Consolidated Standards of Reporting Trials (CONSORT) statement for trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE)

Mahrukh Imran,¹ Linda Kwakkenbos,² Stephen J McCall,³,⁴,⁵ Kimberly A McCord,⁶ Ole Fröbert,⁷ Lars G Hemkens,⁶ Merrick Zwarenstein,⁸,⁹ Clare Relton,¹⁰ Danielle B Rice,¹,¹¹ Sinéad M Langan,¹² Eric I Benchimol,¹³,¹⁴,¹⁵ Lehana Thabane,¹⁶ Marion K Campbell,¹⁷ Margaret Sampson,¹⁸ David Erlinge,¹⁹ Helena M Verkooijen,²⁰,²¹ David Moher,²² Isabelle Boutron,²³,²⁴,²⁵ Philippe Ravaud,²³,²⁴,²⁵ Jon Nicholl,²⁶ Rudolf Uher,²⁷ Maureen Sauvé,²⁸,²⁹ John Fletcher,³⁰ David Torgerson,³¹ Chris Gale,³² Edmund Juszczak,³,³³ Brett D Thombs,¹,³⁴

**To cite:** Imran M, Kwakkenbos L, McCall SJ, et al. Methods and results used in the development of a consensus-driven extension to the Consolidated Standards of Reporting Trials (CONSORT) statement for trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE). *BMJ Open* 2021;11:e049093. doi:10.1136/bmjopen-2021-049093

► Prepublication history and supplemental material for this paper is available online. To view these files, please visit the journal online (http://dx.doi.org/10.1136/bmjopen-2021-049093).

Received 18 January 2021
Revised 16 March 2021
Accepted 18 March 2021

<figure><img src="image_2.png" /></figure>

Check for updates

© Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY. Published by BMJ.

For numbered affiliations see end of article.

**Correspondence to**
Dr Brett D Thombs;
brett.thombs@mcgill.ca

## ABSTRACT

**Objectives** Randomised controlled trials conducted using cohorts and routinely collected data, including registries, electronic health records and administrative databases, are increasingly used in healthcare intervention research. A Consolidated Standards of Reporting Trials (CONSORT) statement extension for trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE) has been developed with the goal of improving reporting quality. This article describes the processes and methods used to develop the extension and decisions made to arrive at the final checklist.

**Methods** The development process involved five stages: (1) identification of the need for a reporting guideline and project launch; (2) conduct of a scoping review to identify possible modifications to CONSORT 2010 checklist items and possible new extension items; (3) a three-round modified Delphi study involving key stakeholders to gather feedback on the checklist; (4) a consensus meeting to finalise items to be included in the extension, followed by stakeholder piloting of the checklist; and (5) publication, dissemination and implementation of the final checklist.

**Results** 27 items were initially developed and rated in Delphi round 1, 13 items were rated in round 2 and 11 items were rated in round 3. Response rates for the Delphi study were 92 of 125 (74%) invited participants in round 1, 77 of 92 (84%) round 1 completers in round 2 and 62 of 77 (81%) round 2 completers in round 3. Twenty-seven members of the project team representing a variety of stakeholder groups attended

## Strengths and limitations of this study

► We followed a five-step process to develop Consolidated Standards of Reporting Trials statement extension for trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE), consistent with Enhancing the QUALity and Transparency Of health Research guidance.

► Items were informed by reporting guidelines on similar research designs, a scoping review, a three-round Delphi process and expert members of the guideline development team.

► CONSORT-ROUTINE was reviewed and tested at various stages of the development by project team members and key stakeholders.

► The limited methodological literature on trials conducted using cohorts and routinely collected data was a limitation in developing the extension.

► Similar to other reporting guidelines, CONSORT-ROUTINE will require re-evaluation and revisions over time to ensure that it is kept up to date with evolving methodology and practice of trials using cohorts and routinely collected data.

the in-person consensus meeting. The final checklist includes five new items and eight modified items. The extension Explanation & Elaboration document further clarifies aspects that are important to report.

**Conclusion** Uptake of CONSORT-ROUTINE and accompanying Explanation & Elaboration document will improve conduct
of trials, as well as the transparency and completeness of reporting of trials conducted using cohorts and routinely collected data.

## BACKGROUND

The use of reporting guidelines, including the Consolidated Standards of Reporting Trials (CONSORT) statement, improves the transparency and completeness of reports of results from randomised controlled trials (RCTs).¹⁻⁴ The CONSORT statement helps to facilitate critical appraisal and interpretation of RCTs by providing guidance to authors on a minimal set of items that should be reported for all trials.⁵ The CONSORT 2010 guideline aimed to improve the reporting of two-arm parallel group RCTs. Extensions of the CONSORT statement have been developed to encourage better reporting of other trial designs, including, for instance, multiarm parallel group randomised trials, cluster trials, pilot and feasibility trials and pragmatic trials.⁶⁻⁹

There is a growing interest in RCTs conducted using cohorts or routinely collected data, including registries, electronic health records (EHRs) and administrative databases.¹⁰⁻¹⁴ In a cohort, a group of individuals is gathered for the purpose of conducting research, whereas routinely collected data refer to data initially collected for purposes other than research or without specific a priori research questions developed before collection.¹⁵ ¹⁶ Trials may use a cohort or routinely collected data for: (1) identification of eligible participants, (2) outcome ascertainment and (3) to implement an intervention, or for a combination of these purposes. For example, in registry-based RCTs, a registry could be used to identify eligible participants for a trial, for the collection of participant baseline characteristics and as the source of outcome data; some registries have used interactive technology to actively flag participants for RCT enrolment as patient data are entered into the registry.¹² In some EHR trials, the EHR itself is used to implement an intervention. For example, one RCT tested an intervention to reduce antibiotic prescribing by feeding back personalised antibiotic prescription data to primary care physicians.¹⁷

The use of cohorts and routinely collected data may make RCTs easier and more feasible to perform by reducing cost, time and other resources.¹⁸ ¹⁹ It may also facilitate the conduct of trials that more closely replicate real-world clinical practice. These trial designs, however, are relatively recent innovations, and published RCT reports may not describe important aspects of their methodology in a standardised way. Trials conducted using cohorts and routinely collected data share certain elements with conventional RCTs, but there are also distinctive elements to report that are not covered in the CONSORT 2010 statement. The REporting of studies Conducted using Observational Routinely-collected Data (RECORD) statement

provides guidance on reporting of studies conducted using routinely collected data but does not address RCT-specific methodological and reporting considerations.²⁰ Research conducted using routinely collected data presents unique methodological challenges that are often insufficiently reported, but there is scant guidance on methods and reporting of trials conducted using routinely collected data or cohorts.²¹ ²²

An extension to the CONSORT statement for RCTs conducted using cohorts and routinely collected data was developed using methods recommended for developing reporting guidelines.²³ This article describes, in detail, the consensus-based development process. The main aims of this article are to: (1) describe the methods and processes used in the development of the CONSORT Extension for Trials Conducted Using Cohorts and Routinely Collected Data (CONSORT-ROUTINE)²⁴ and (2) describe decisions made to arrive at the final checklist and the accompanying Explanation & Elaboration statement.

## METHODS

The project was registered with the Enhancing the QUALity and Transparency Of health Research (EQUATOR) network.²⁵ We followed the EQUATOR network's guidelines for recommended methods and processes for developing, disseminating and implementing healthcare reporting guidelines.²³ These methods have been used in the development of other similar EQUATOR guidelines. Figure 1 illustrates the five parts of the development process for this guideline.

<figure><img src="image_3.png" /><figcaption>**Figure 1** Development process of the CONSORT Extension for Trials Conducted Using Cohorts and routinely Collected Data (CONSORT-ROUTINE).</figcaption></figure>
## Project phase 1: project launch, establishment of team and funding

### Need for the guideline and literature review

An initial informal review of reports of published protocols and reports of trials using cohorts and routinely collected data by BDT and LK suggested that there appeared to be deficiencies in reporting of such trials. For instance, many reports did not adequately describe the cohort or database from which trial participants were recruited, processes used to link participants across databases were not always provided and it was sometimes unclear whether trial outcomes were assessed by the triallists or ascertained via existing databases used to conduct the trial. A review of the EQUATOR website and published literature indicated that there was no existing reporting guideline for these types of trials. The RECORD statement addresses reporting issues related to routinely collected data but does not include guidance on reporting of trials. Many trials conducted using routinely collected data are pragmatic or use cluster designs, for instance, but CONSORT extensions for those types of trials do not address issues germane to the use of cohorts or routinely collected data to conduct trials.⁷⁹

### Project launch and identification of CONSORT-ROUTINE project members

Initial discussions on developing a CONSORT extension for RCTs conducted using cohorts occurred in November 2016 at the Trials within Cohorts symposium in London, UK (LK, MZ, CR and BDT).²⁶ Discussions continued virtually and key people involved in cohort-embedded trials or the EQUATOR network were approached during December 2016 (HMV, DM, IB, PR, JN, RU and DT). It was suggested that trials conducted in registries had many characteristics similar to those in cohorts, and there was agreement to include registry-based trials in the extension. People with expertise in registry-based trials were approached in March 2017 (OF, LT, MKC and DE), and an experienced librarian (MSam) and patient representative familiar with trials conducted using cohorts (MSau) were also included in the group at that point.

The project was registered on the EQUATOR website in April 2017. During the preparatory phase, while developing searches and reviewing example publications, we became aware that trials conducted using EHRs and administrative databases also shared similar characteristics with trials in cohorts and registries, and it was decided to expand the scope to trials conducted using cohorts and routinely collected data. In July 2017, triallists, who were leading the development of a reporting guideline for EHRs, joined the project group (EJ and CG). Given the relevance of their previous work and their expertise (LH, SL, DM and EIB), authors who had been involved in the development of the RECORD statement were invited to join the team.²⁰ Several doctoral students also joined

the project team (SJM, KAM and DBR). A steering committee comprising of 10 members with key expertise for consultation was established. A research coordinator (MI) was hired in April 2018 to manage the project, and an experienced journal editor was invited to join (JF). The group communicated regularly throughout the process via a number of virtual meetings, using an online platform to conduct teleconferences, as well as through email discussions.

### Rationale for developing one checklist versus four different checklists for trials conducted using cohorts, registries, EHRs and administrative databases

Team members discussed the advantages and disadvantages of creating individual checklists for each of the four types of data versus a single checklist for all four. It was determined that, although there are some differences in the implementation of trials across the different types of data sources, the methodological principles are similar, and there is substantial overlap in the design, conduct and factors that may influence interpretability. Thus, the steering committee reached consensus to develop a single statement, addressing any differences by including 'if applicable' to items in the checklist that may not apply to all trial designs and to clarify differences in the Explanation & Elaboration publication as deemed necessary.

### Funding

The project team obtained its main source of funding from a grant from the Canadian Institutes of Health Research Institutes (CIHR) to support the development of the guideline (BDT, OF, EJ, LK, CR; Grant #PJT-156172). EJ and CG also obtained funding from the UK National Institute of Health Research Clinical Trials Unit Support Funding - Supporting efficient/innovative delivery of NIHR research. In addition, funding to hold the face-to-face meeting was provided by a Planning and Dissemination Grant from CIHR (BDT and LK; Grant #PCS - 161863) and by contributions from Queen Mary University of London, the University of Sheffield, McGill University and the Lady Davis Institute for Medical Research of the Jewish General Hospital in Montreal, Canada.

A project protocol was developed and published.²²

## Project phase 2: scoping review

A preliminary 'long list' of possible reporting items was formulated by LK and KAM based on review of the CONSORT 2010 statement items, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE)²⁷ and the RECORD statements,²⁰ as well as discussions with steering committee members. The STROBE and RECORD statements were considered the most relevant to this project because of their focus on reporting of observational studies and non-interventional studies using routinely collected data.

A scoping review was conducted to identify: (1) articles on the methodology or reporting of RCTs conducted using cohorts or routinely collected data that could
inform the development of new items or modification of existing CONSORT items; and (2) trial reports to identify aspects of reporting that need improvement and examples of good reporting of potential checklist items that could be used to support CONSORT-ROUTINE.²⁸ We searched for relevant articles on trials conducted using cohorts, registries, EHRs and administrative databases from 2007 to 2018. After screening articles for inclusion and exclusion at the abstract and full-text level, 10 people from the team independently reviewed the included papers and provided suggestions for modifications or additional reporting guideline items until no new ideas emerged (saturation). Suggestions were added in a standardised, shared spreadsheet. At the same time, team members provided examples of good reporting for each proposed item or item modification. Additionally, the review helped us to create a list of authors with experience in these trial designs as potential participants for the Delphi study. Search terms used in the scoping review are shown in online supplemental file 1.

## Project phase 3: Delphi study

The objectives of our Delphi study were: (A) to obtain feedback on the importance of including each candidate item in CONSORT-ROUTINE; (B) to improve the wording of items considered important; and (C) to elicit suggestions for additional items not in the existing list. We aimed to engage key stakeholders across different sectors and backgrounds. There are not fixed guidelines on the sample size of Delphi studies, and the ideal number of participants may depend on the complexity of the topic, the likely heterogeneity of relevant experiences and viewpoints, and resources available to manage the data generated.²⁹⁻³¹ Many studies use small groups of experts (eg, <20), but we believed that a larger group with diverse expertise would best complement the knowledge of the project team. Thus, we sent out an invitation to reporting guideline developers (including those involved in previous CONSORT extensions), funders, journal editors, patient representatives, trial methodologists, epidemiologists, meta-research authors, ethicists, biostatisticians and clinical triallists who were identified by members of the project team. We also encouraged recipients of the invitation to forward the invitation to other potentially interested stakeholders.

The Delphi surveys were built and hosted using an online survey platform in Qualtrics. During registration, we gathered demographic and professional background characteristics of participants, including geographical location, self-identified stakeholder group (eg, clinical trials user, clinical triallist and methodologist), employment sector, years of experience in trials research and research experience in trials conducted using cohorts or routinely collected data.

Registered participants received a link to access each of the three rounds of the Delphi survey. In each round, we asked participants to rate their perceptions about the importance of each suggested reporting item by ranking

<figure><img src="image_4.png" /><figcaption>**Figure 2** Example of a round 1 Delphi survey item as presented in the online survey. CONSORT, Consolidated Standards of Reporting Trials.</figcaption></figure>

items based on how essential they are for reporting on a 1–5 Likert scale (1=not essential; 5=essential). There is not consensus on the ideal number of Likert categories or groupings for decision-making, but it is common to use between 4-point and 7-point scales.³⁰

Responses were categorised as follows:

1–2=low score (item should not be part of CONSORT-ROUTINE checklist).

3=moderate (item should be discussed).

4–5=high score (item should be part of CONSORT-ROUTINE checklist).

Participants also had the option to select ‘Not my expertise’ for items if they believed that they did not have the appropriate level of expertise to rate an item. Figure 2 shows a screenshot of an example proposed modification item from the survey:

Items from the CONSORT 2010 statement for which modifications were initially not proposed were also included in the survey so that participants could provide comments or make recommendations for modifications to these items. For all items (proposed modifications and CONSORT 2010 items), we provided participants with the opportunity to give open-ended feedback, using freetext boxes provided at the bottom of each survey page and at the end of the survey. At the end of the survey, participants were asked to provide any additional items that they believed would be important for reporting in trials conducted using cohorts and routinely collected data but that had not been included in the proposed set of new and modified items.

We launched round 1 of the survey on 4 February 2019 with 2 weeks to provide responses. Round 2 was launched on 4 March 2019, and round 3 was launched on 1 April 2019. After each round, the Qualtrics built-in analysis software was used to generate a distribution of scores and to aggregate group results for each item (mean score, maximum and minimum score, SD, variance
and percentage ratings of 1–5 ranking for items) and summary statistics were circulated among all participants. Individual responses were not fed back. In addition, a bar chart with the ratings and counts for each item was created. Following each round of the survey, the CONSORT-ROUTINE steering committee members reviewed the survey results independently and then met via teleconference to discuss and analyse the results of the survey. During these meetings, decisions were made on how to address comments from participants by modifying, adding or combining items. Notes were also made on comments that reflected a need for explanation in the Explanation & Elaboration companion to the checklist.

We predefined consensus as at least 2/3 of responders rating the importance of an item as ‘high’ or ‘very high’. Items that reached consensus for inclusion were not rated again in the next round. For some items that did not reach consensus, the wording of items was revised based on participants’ suggestions. Items that did not reach consensus were rated again in the next round in their original or revised form. Reports summarising the Delphi results were circulated after each round including summary statistics such as counts, means, SD and variances for the responses on each item. Reminder emails were sent 1 week prior to the deadline and extensions were provided if requested for all three rounds in order to maximise participation.

Since the Delphi Study was advisory, all items were reviewed and vetted again at the in-person consensus meeting, and comments provided by participants of the Delphi Study were taken into consideration while making decisions to include or exclude items.

## Project phase 4: in-person consensus meeting and development of checklist publication

A 2-day in-person consensus meeting was held on 13–14 May 2019 in London, UK. The purpose of the meeting was to discuss the Delphi results, make decisions on items to retain in the final checklist, make any necessary modifications to items and suggest reporting aspects that should be addressed in the Explanation & Elaboration documentation supporting the checklist. The meeting was attended by 26 members of the CONSORT-ROUTINE Group.

We used approaches similar to those used in previous consensus meetings for other guidelines. Participants were provided with the results of the initial long-list generation and the Delphi study in advance of the meeting. At the meeting, steering committee members first presented the background and an update on work done to date, in order to facilitate the discussions. Session chairs then separately presented items from the preliminary checklist, results of the Delphi study and feedback from stakeholders, after which the group discussed in an open forum. Decisions were made on items to be modified or added based on the following criteria: (1) whether they addressed elements unique to trials conducted using cohorts or routinely collected data versus elements applicable to any trial and (2) whether

they reflected information that should be included in a minimum reporting set of items. Notes were taken, and the discussions were audio-recorded to ensure that the content was accurately captured.

Following the consensus meeting, refinement of the content and wording of the items was continued through online group discussions with CONSORT-ROUTINE project team members. The initial version of the checklist was pilot-tested by circulating it among stakeholders in order to assess its usability and to identify any challenges that might arise while applying the checklist. Pilot-testing the checklist also provided insight into issues that should be addressed in detail in the Explanation & Elaboration statement.

## Project phase 5: publication, dissemination and implementation

As with several previous CONSORT extensions, it was decided to publish the reporting checklist with a detailed Explanation & Elaboration statement in the same document.⁶⁻⁹ The Explanation & Elaboration statement is intended to provide an in-depth explanation of the scientific rationale for each recommendation, together with an example of clear reporting for each item.

In addition to publication of the reporting guideline checklist and Explanation & Elaboration material, to attempt to maximise uptake, we will undertake additional dissemination activities, including presentations and workshops at conferences and other venues. We also plan to seek endorsement of the guideline by journal editors. Research has shown that formal endorsement and adoption of the CONSORT statement by journals is associated with improved quality of reporting.² Studies conducted by members of our team have benchmarked pre-extension reporting completeness in trials conducted in cohorts, registries, EHRs, and administrative databases.³²⁻³⁴ There were not enough examples of completed cohort-embedded trials for benchmarking reporting.

The final CONSORT-ROUTINE checklist has been published.²⁴

## Patient and public involvement

One of the members of our CONSORT-ROUTINE team, MSau, is a patient organisation leader. She has been involved in working with researchers to establish a cohort of patients living with the rare disease scleroderma, which supports RCTs of trials of online rehabilitation, self-management and psychological intervention programmes.

## RESULTS

### Stage 2: scoping review and initial long list of potential items

The scoping review sought methods articles and reports of trials conducted using cohorts, registries, EHRs or administrative databases.
Cohorts

The database search identified 1185 publications, of which 1062 were excluded after title and abstract screening and 37 after full-text review. A total of 86 studies were included in the scoping review, including 15 papers on methodological considerations of using cohorts for conducting RCTs. All trials used the cohort for both identification of patients and outcome ascertainment.

Registries

The search identified 234 publications, of which 143 received full-text review. A total of 106 publications were eligible, including 95 trial reports or protocols (both identification of patients and outcome ascertainment (n=27); identification of patients only (n=28); outcome ascertainment only (n=40)) and 11 papers on methodological considerations.

Electronic health records

The search identified 2085 citations, of which 548 studies were reviewed at the full-text level. A total of 289 eligible publications, including 263 trial protocols or reports (both identification of patients and outcome ascertainment (n=169); identification of patients only (n=38); outcome ascertainment only (n=56)) and 26 articles that described methodological considerations.

Administrative databases

The search identified 663 citations, of which 151 full texts were reviewed. There were a total of 117 trial protocols or reports included (both identification of patients and outcome ascertainment (n=57); identification of patients only (n=1); outcome ascertainment only (n=58)) and one paper on methodological considerations.

Delphi study results

Of 125 people invited to take part in the Delphi study, 115 people registered via an online survey, and 92 (74%) provided responses on the items in round 1. Figures 3 and 4 present the types of stakeholder groups that completed round 1 of the Delphi study and the type of trials conducted using cohorts or routinely collected databases with which they had familiarity. Participants belonging to more than one category had the option of checking multiple options in the survey.

Round 1

Of the 92 participants who completed the round 1 survey, 90 provided valid ratings and two provided comments but not ratings. Of the 27 items rated in round 1, 14 reached consensus to be included in discussions at the consensus meeting; the other 13 did not reach consensus and were included in round 2. Based on round 1 feedback, a total of 11 items were modified for review in round 2, including

<figure><img src="image_5.png" /><figcaption>**Figure 3** Professional roles reported by participants who completed round 1 of the CONSORT-ROUTINE Delphi study (%). Participants could report more than one role. CONSORT-ROUTINE, Consolidated Standards of Reporting Trials Extension for Trials Conducted Using Cohorts and Routinely Collected Data.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Figure 4** Participants of round 1 of the CONSORT-ROUTINE Delphi study by type of cohort or routinely collected database with which they had familiarity (%). Participants could report more than one. CONSORT-ROUTINE, Consolidated Standards of Reporting Trials Extension for Trials Conducted Using Cohorts and Routinely Collected Data.</figcaption></figure>

two items that were combined into one. No items were excluded from the checklist.

### Round 2

Of the 92 participants who completed round 1, 77 (84%) completed the round 2 survey. Of the 13 items rated, 2 reached consensus for inclusion in consensus meeting discussions, and 11 did not reach consensus in round 2. Based on round 2 feedback, eight items were modified prior to round 3.

### Round 3

Of the 77 people who completed round 2, 62 (81%) completed round 3. Of the 11 items in round 3, five items reached consensus in round 3. The remaining six items did not reach consensus after the three rounds.

There were several new items suggested via the Delphi process but not added to the potential item list. The main reasons why some items were suggested but not incorporated were:

1. The suggestion was encapsulated in CONSORT 2010 items, was already captured by proposed new or modified items or could be captured by further modifying new or modified items.

2. The suggestion was not specific to trials conducted using cohorts and routinely collected data and, thus, was recommending a change to the CONSORT 2010 checklist, which was not the task of the CONSORT-ROUTINE group.

Summary results of the three rounds can be accessed at: https://osf.io/4zh6f/

### In-person consensus meeting

Table 1 summarises the CONSORT-ROUTINE group's discussions and advisory decisions for each of the items that was discussed during the in-person meeting. If there were differing opinions on the inclusion or exclusion of items and consensus could not be reached, voting was implemented by the session chair, with an 80% threshold for inclusion in the checklist as part of the minimal set

of recommended reporting items. The key recommendations that emerged were as follows:

► Proposed modification to CONSORT 2010 items: it was recommended to retain proposed modifications to seven CONSORT 2010 items. These modifications pertained to differences in mechanisms used to conduct trials using cohorts or routinely collected databases. As in previous CONSORT extensions, some of the recommended changes end with 'if applicable' to show that some information which authors are being asked to report might not be relevant or applicable for their particular RCT or the particular type of data that was used in the RCT.

► Proposed additional items: consensus was reached to include six additional items and to add a new subheading, 'Cohort or routinely collected database', to the checklist.

A recurrent discussion point was the need to minimise adding new items to the abstract unless they are essential due to word limits imposed by journals. A suggestion was made to expand the explanatory text of the Explanation & Elaboration document for nine unchanged CONSORT 2010 items to clarify additional requirements for reporting aspects of the trial without modifying the item: item 1a (identification as a randomised trial in the title), item 4b (settings and location where the data were collected), item 5 (intervention), item 13b (losses and exclusions after randomisation), item 14a (dates of recruitment/follow-up), item 15 (baseline data), item 20 (limitations), item 21 (generalisability) and item 24 (study protocol). For the abstract, there was an agreement to include an additional item to the abstract for naming the cohort or routinely collected database (item 1c). This item was later merged with item 1b from the CONSORT 2010 checklist after discussion with the project team (table 1). Thus, the final extension checklist included eight modified items and five new items.²⁴

### CONSORT-ROUTINE pilot test

The preliminary version of the checklist was pilot-tested by 17 people who had been previously involved in conducting trials using cohorts and routinely collected data. Based on feedback received from the pilot test, there were minor
<table><caption>**Table 1** Consensus meeting discussions and advisory decisions for the checklist items</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td>Title and abstract</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>1a</td><td>1a</td><td>Identification as a randomised trial in the title.</td><td>Identification as a randomised trial in the title, including that it was a trial conducted using a cohort or routinely collected source of data (modified).</td><td>Not reached.</td><td>Discussed the need for a modification to the original item. It was noted that multiple databases or types of databases could be used to conduct a trial and stating all would not be feasible as journals might have title length restrictions.<br/>Decision: do not include the modification and retain the CONSORT 2010 item; expand the E&amp;E text for clarification.</td><td>Identification as a randomised trial in the title.</td></tr><tr><td></td><td>1b</td><td>1b</td><td>Structured summary of trial design, methods, results and conclusions (for specific guidance see CONSORT for abstracts).</td><td>The source(s) of data used to conduct the trial should be specified in the abstract (additional).</td><td>Reached for inclusion.</td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.<br/>Note: additional item 1c was later merged with this item (see further).</td><td>Structured summary of trial design, methods, results and conclusions (for specific guidance, see CONSORT for abstracts). Specify that a cohort or routinely collected data were used to conduct the trial and, if applicable, provide the name of the cohort or routinely collected database(s).</td></tr><tr><td></td><td></td><td></td><td></td><td>If linkage between multiple sources of data was conducted for the study, this should be clearly stated in the abstract (additional).</td><td>Not reached.</td><td>Noted the importance of stating the cohort or routinely collected database(s) used to conduct the trial in the abstract, if not in the title.<br/>Decision: include the suggested new item with revisions.<br/>Note: the item was later merged with item 1b from CONSORT 2010.</td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>The proportion of participants offered and the proportion that accepted the intervention should be reported (for trials conducted using the cohort multiple RCT design) (additional).</td><td>Reached for inclusion.</td><td>Mixed views on the necessity of reporting the suggested new item in the abstract. Agreed that linkage is important to report in the body of the paper but not necessarily the abstract.<br/>Decision: do not include the suggested new item.<br/>Mixed views on the necessity of reporting the suggested new item in the abstract. Agreement that the information is important to report but not essential for the abstract due to word count restrictions. In addition, this applies to one trial design used in cohorts but not all cohort trials and not trials using other types of data. The item was merged with CONSORT 2010 item 13a (14a in the final extension checklist) pertaining to participant flow.<br/>Decision: do not include the suggested new item in the abstract but include in item 14a in the final extension checklist.</td><td></td></tr></tbody></table>

Continued
Continued

<table><caption>Table 1 Continued</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td colspan="8"><strong>Introduction</strong></td></tr><tr><td>Background and objectives</td><td>2a</td><td>2a</td><td>Scientific background and explanation of rationale.</td><td></td><td></td><td>Discussed the importance of reporting the rationale for conducting the trial using a cohort or routinely collected database but decided against modifying original CONSORT 2010 item.<br>Decision: retain the CONSORT 2010 item.</td><td>Scientific background and explanation of rationale.</td></tr><tr><td></td><td>2b</td><td>2b</td><td>Specific objectives or hypotheses</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>Specific objectives or hypotheses.</td></tr><tr><td colspan="8"><strong>Methods</strong></td></tr><tr><td>Trial design</td><td>3a</td><td>3a</td><td>Description of trial design (such as parallel, factorial) including allocation ratio.</td><td>Description of trial design (such as parallel and factorial) including allocation ratio, the source(s) of data used to conduct the trial (such as cohort and registry) and how the data are used within the trial (such as identification of eligible trial participants, trial outcomes) (modified).</td><td>Noted that key elements of the study design and cohort or database(s) used for the trial should be stated early in the methods section, as well as the extent to which the database was used in the trial. Decision: include the modified item.</td><td>Description of trial design (such as parallel and factorial) including allocation ratio, that a cohort or routinely collected database(s) used to conduct the trial (such as electronic health record and registry) and how the data were used within the trial (such as identification of eligible trial participants and trial outcomes).<br>Important changes to methods after trial commencement (such as eligibility criteria), with reasons.</td></tr><tr><td></td><td>3b</td><td>3b</td><td>Important changes to methods after trial commencement (such as eligibility criteria), with reasons.</td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td></td><td></td></tr><tr><td>Cohort or routinely collected database (additional header)</td><td>Routine-1</td><td></td><td></td><td>Description of the source(s) of data used to conduct the trial, including the setting, locations, relevant dates, periods of recruitment, follow-up and data collection (additional).</td><td>Reached for inclusion</td><td>Agreed on the importance of reporting the item.<br>Decision: include the suggested new item.</td><td>Name, if applicable, and description of the cohort or routinely collected database(s) used to conduct the trial, including information on the setting (such as primary care), locations and dates (such as periods of recruitment, follow-up and data collection).</td></tr><tr><td></td><td></td><td></td><td></td><td>Describe indicators of the quality of the source(s) of data used to conduct the trial including what types of quality checks have been performed and the entity responsible for the data (additional).</td><td>Reached for inclusion.</td><td>Mixed views on the necessity of the suggested new item. There were concerns that 'quality' is vague and the term 'accuracy and completeness' may better clarify the intent of the item. It was acknowledged that the accuracy and completeness of the cohort or database is important to report while (1) selecting participants and (2) ascertaining outcomes.<br>Decision: do not include the suggested new item as a standalone item. The item was merged with extension items 5a and 7b (pertaining to participant selection and outcome ascertainment) in the finalised checklist.</td><td></td></tr></tbody></table>
<table><caption>Table 1 Continued</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td></td><td></td><td></td><td></td><td>Describe modifications to the data collected in the source(s) of data used to conduct the trial, such as adding data items, if applicable (additional).</td><td>Reached for inclusion</td><td>Agreed that the suggested item is not necessarily unique to trials conducted using cohorts and routinely collected data. Decision: do not include the suggested new item; expand the E&amp;E text for clarification.</td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>Describe additional sources of data used to conduct the trial, if any (additional).</td><td>Reached for inclusion.</td><td>Mixed views on the necessity of the suggested new item as it is not unique to trials conducted using cohorts and routinely collected data. Decision: do not include the suggested new item; expand the E&amp;E text for clarification.</td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>Give the eligibility criteria, the sources and methods of selection of participants, and methods of follow-up (for trials conducted using cohorts or registries) (additional).</td><td>Reached for inclusion</td><td>Discussed the importance of reporting the eligibility criteria for inclusion in the cohort or routinely collected database(s), but there was concern that elements related to follow-up are not specific to trials conducted using cohorts and routinely collected data. Decision: include the suggested new item with revisions; expand the E&amp;E text for clarification of other aspects.</td><td>Eligibility criteria for participants in the cohort or routinely collected database(s).</td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td>Suggestion to integrate wording from RECORD checklist for clarity. Decision: include the suggested new item adapted from RECORD.</td><td>State whether the study included person-level, institutional-level or other data linkage across two or more databases and, if so, linkage techniques and methods used to evaluate completeness and accuracy of linkage.</td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td>Mixed views on the necessity of the item as some believed that ethics considerations are beyond the scope of CONSORT, and ethics does not appear in CONSORT 2010. The group agreed to include the item as consent is an important issue with unique aspects in these trials, but that this should be presented as part of trial participants section. Decision: include the suggested new item with revisions and move to section 'Trial participants' as item 5c.</td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>Detail any use of record linkage across sources of data, the methods of linkage and methods of quality evaluation, if applicable (additional).</td><td>Reached for inclusion.</td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td><td>Describe if (and how) participants were informed about the potential use of their data in randomised trials (additional).</td><td>Not reached</td><td></td><td></td></tr></tbody></table>
<table><caption>Table 1 Continued</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td>Trial participants (modified header)</td><td>4a</td><td>4a</td><td>Eligibility criteria for participants.</td><td>Eligibility criteria for trial participants (modified).</td><td></td><td>Agreed to merge with suggested new item (see next row).<br>Decision: merge with suggested new item, 'Provide details of how eligible clusters/ participants were identified from the source(s) of data used to conduct the trial'.</td><td>Eligibility criteria for trial participants, including information on how to access the list of codes and algorithms used to identify eligible participants, information on accuracy and completeness of data used to ascertain eligibility and methods used to validate accuracy and completeness (eg, monitoring, adjudication), if applicable.</td></tr><tr><td></td><td>4b</td><td>4b</td><td>Settings and locations where the data were collected.</td><td>Provide details of how eligible clusters/participants were identified from the source(s) of data used to conduct the trial (additional).</td><td>Reached for inclusion.</td><td>Suggested merging with CONSORT 2010 item 4a (5a in final checklist) and address accuracy and completeness of data.<br>Decision: merge the suggested new item with CONSORT 2010 item 4a (5a in the final checklist).</td><td></td></tr><tr><td></td><td></td><td></td><td>Routine-4</td><td>Settings and locations where the trial data were collected (modified).</td><td>Reached for inclusion.</td><td>The word 'trial' was dropped as the header 'Trial participants' clarifies the intent of the item.<br>Decision: retain the CONSORT 2010 item; expand the E&amp;E text for clarification.</td><td>Settings and locations where the data were collected.</td></tr><tr><td></td><td></td><td></td><td></td><td>Details of information provided to participants from the source(s) of data who are selected for recruitment or inclusion in the trial, including any differences in information provided across trial arms (additional).</td><td>Not reached.</td><td>Extended discussions on the importance of the item as it might only be applicable to the controlled multiple RCT design.<br>Agreement to formulate as a general item on consent as item 5c.<br>Decision: do not include the suggested new item. The consent item was simplified and moved to this section. Expand the E&amp;E text for clarification of consent issues.</td><td>Describe whether and how consent was obtained.</td></tr><tr><td>Interventions</td><td>5</td><td>5</td><td>The interventions for each group with sufficient details to allow replication, including how and when they were actually administered.</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item; expand the E&amp;E text for clarification.</td><td>The interventions for each group with sufficient details to allow replication, including how and when they were actually administered.</td></tr><tr><td>Outcomes</td><td>6a</td><td>6a</td><td>Completely defined prespecified primary and secondary outcome measures, including how and when they were assessed.</td><td>Describe how the source(s) of data was used to implement the intervention, if applicable (eg, for trials conducted using electronic health records) (additional).</td><td>Reached for inclusion.</td><td>Debated the necessity of the new item as it is only applicable to trials conducted using electronic health records that may be used as intervention tools.<br>Decision: do not include the suggested new item; expand the E&amp;E text for clarification.</td><td>Completely defined prespecified primary and secondary outcome measures, including how and when they were ascertained and the cohort or routinely collected database(s) used to ascertain each outcome.</td></tr></tbody></table>

Continued
<table><caption>Table 1 Continued</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td rowspan="2">ROUTINE-5</td><td rowspan="2"></td><td rowspan="2"></td><td>Provide source(s) of data for each outcome (additional).</td><td>Reached for inclusion.</td><td>Suggestion to merge with CONSORT 2010 item 6a.<br/>Decision: item merged with CONSORT 2010 item 6a (7a in the final checklist).</td><td rowspan="2">Information on how to access the list of codes and algorithms used to define or derive the outcomes from the cohort or routinely collected database(s) used to conduct the trial, information on accuracy and completeness of outcome variables and methods used to validate accuracy and completeness (eg, monitoring, adjudication), if applicable.</td></tr><tr><td>Provide a list of codes and algorithms used to define (and/ or derive) the outcomes as online supplemental information, including validation, if applicable (additional).</td><td>Not reached.</td><td>Acknowledged the importance of reporting the list of codes and algorithms for ascertaining outcomes along with the accuracy and completeness of data and validation.<br/>Decision: include the suggested new item with revisions.</td></tr><tr><td></td><td></td><td></td><td>Detail any adjudication or external validation of data items from the source(s) of data used to conduct the trial, if applicable (additional).</td><td>Reached for inclusion.</td><td>Acknowledged the importance of reporting the item. There was agreement that validation should be reported while selecting participants and ascertaining outcomes and included as part of items 5a and 7b of extension checklist.<br/>Decision: address elements of proposed item as part of items 5a and 7b in the final checklist.</td><td></td></tr><tr><td>6b</td><td>6b</td><td></td><td>Any changes to trial outcomes after the trial commenced, with reasons.</td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>Any changes to trial outcomes after the trial commenced, with reasons.</td></tr><tr><td rowspan="2">Sample size</td><td>7a</td><td>7a</td><td>How sample size was determined.</td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>How sample size was determined.</td></tr><tr><td>7b</td><td>7b</td><td>When applicable, explanation of any interim analyses and stopping guidelines.</td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>When applicable, explanation of any interim analyses and stopping guidelines.</td></tr><tr><td colspan="8" style="font-weight: bold; text-align: left;">Randomisation</td></tr><tr><td>Sequence generation</td><td>8a</td><td>8a</td><td>Method used to generate the random allocation sequence.</td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item; expand the E&amp;E text for clarification.</td><td>Method used to generate the random allocation sequence.</td></tr><tr><td></td><td>8b</td><td>8b</td><td>Type of randomisation; details of any restriction (such as blocking and block size).</td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>Type of randomisation; details of any restriction (such as blocking and block size).</td></tr><tr><td>Allocation concealment mechanism</td><td>9</td><td>9</td><td>Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned.</td><td>Mechanism used to implement the random allocation sequence, describing any steps taken to conceal the sequence until interventions were assigned,<br/>such as using automated random sequence generation concealed within source(s) of data (modified).</td><td>Discussion to clarify wording of the item.<br/>Decision: include the modified item with revisions.</td><td>Mechanism used to implement the random allocation sequence (such as embedding an automated randomiser within the cohort or routinely collected database(s)), describing any steps taken to conceal the sequence until interventions were assigned.</td></tr></tbody></table>
<table><caption>Table 1 Continued</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td>Implementation</td><td>10</td><td>10</td><td>Who generated the random allocation sequence, who enrolled participants and who assigned participants to interventions.</td><td></td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>Who generated the random allocation sequence, who enrolled participants and who assigned participants to interventions.</td></tr><tr><td>Blinding</td><td>11a</td><td>11a</td><td>If done, who was blinded after assignment to interventions (eg, participants, care providers, those assessing outcomes) and how.</td><td></td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>If done, who was blinded after assignment to interventions (eg, participants, care providers, those assessing outcomes) and how.</td></tr><tr><td>Statistical methods</td><td>12a</td><td>12a</td><td>If relevant, description of the similarity of interventions.</td><td></td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>If relevant, description of the similarity of interventions.</td></tr><tr><td></td><td>12b</td><td>12b</td><td>Statistical methods used to compare groups for primary and secondary outcomes.<br/>Methods for additional analyses, such as subgroup analyses and adjusted analyses.</td><td></td><td></td><td>No suggested modification.<br/>Decision: retain the CONSORT 2010 item.</td><td>Statistical methods used to compare groups for primary and secondary outcomes.<br/>Methods for additional analyses, such as subgroup analyses and adjusted analyses.</td></tr><tr><td colspan="8"><strong>Results</strong></td></tr><tr><td>Participant flow (a diagram is strongly recommended)</td><td>13a</td><td>13a</td><td>For each group, the numbers of participants who were randomly assigned, received intended treatment and were analysed for the primary outcome.</td><td>Describe in detail the numbers of clusters/participants in the source(s) of data used to conduct the trial, number screened for eligibility, randomly assigned, offered and accepted interventions (eg, cohort multiple RCTs), received intended treatment and analysed for the primary outcome (modified).<br/>Describe any linkage of multiple sources of data, including the number of clusters/participants successfully linked (additional).</td><td>Reached for inclusion.</td><td>Suggestion to form a committee to draft example flow diagram and oversee the E&amp;E.<br/>Decision: include the modified item; committee to oversee the E&amp;E development.</td><td>For each group, the number of participants in the cohort or routinely collected database(s) used to conduct the trial and the numbers screened for eligibility, randomly assigned, offered and accepted interventions (eg, cohort multiple RCTs), received intended treatment and analysed for the primary outcome.</td></tr><tr><td></td><td>13b</td><td>13b</td><td>For each group, losses and exclusions after randomisation, together with reasons.</td><td></td><td>Reached for inclusion.</td><td>Debated the necessity of the item as a stand-alone item as linkage was addressed in Item 4c. Suggested to include the number of clusters/participants successfully linked as part of the flow diagram.<br/>Decision: do not include the suggested new item; expand the E&amp;E text for clarification.</td><td>For each group, losses and exclusions after randomisation, together with reasons.</td></tr></tbody></table>

Continued
<table><caption>Table 1 Continued</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td rowspan="2">Recruitment</td><td>14a</td><td>14a</td><td>Dates defining the periods of recruitment and follow-up.</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>Dates defining the periods of recruitment and follow-up.</td></tr><tr><td>14b</td><td>14b</td><td>Why the trial ended or was stopped.</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>Why the trial ended or was stopped.</td></tr><tr><td>Baseline data</td><td>15</td><td>15</td><td>A table showing baseline demographic and clinical characteristics for each group.</td><td>A table showing baseline demographic and clinical characteristics for eligible participants who participated in the trial and those who did not (additional).</td><td>Reached for inclusion.</td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>A table showing baseline demographic and clinical characteristics for each group.</td></tr><tr><td>Numbers analysed</td><td>16</td><td>16</td><td>For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups.</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups.</td></tr><tr><td>Outcomes and estimation</td><td>17a</td><td>17a</td><td>For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% CI).</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% CI).</td></tr><tr><td rowspan="2">Ancillary analyses</td><td>17b</td><td>17b</td><td>For binary outcomes, presentation of both absolute and relative effect sizes is recommended.</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>For binary outcomes, presentation of both absolute and relative effect sizes is recommended.</td></tr><tr><td>18</td><td>18</td><td>Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing prespecified from exploratory.</td><td>If outcomes for eligible patients in the existing source(s) of data who were not included in the trial are known, they should be reported (additional).</td><td>Not reached.</td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing prespecified from exploratory.</td></tr><tr><td>Harms</td><td>19</td><td>19</td><td>All important harms or unintended effects in each group (for specific guidance see CONSORT for harms).</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>All important harms or unintended effects in each group (for specific guidance see CONSORT for harms).</td></tr></tbody></table>

Continued
<table><caption>Table 1 Continued</caption><thead><tr><th>Section/topic</th><th>CONSORT 2010 item</th><th>CONSORT ext. item</th><th>CONSORT 2010 item</th><th>Suggested modified or additional extension items</th><th>Consensus status (Delphi)</th><th>Summary of the discussion, decisions and suggestions made during the CONSORT-ROUTINE in-person consensus meeting</th><th>Final checklist item to be included in CONSORT-ROUTINE</th></tr></thead><tbody><tr><td colspan="8"><strong>Discussion</strong></td></tr><tr><td>Limitations</td><td>20</td><td>20</td><td>Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses.</td><td>Discuss the implications of using data that were not created or collected to answer the specific research question(s) (additional).</td><td>Reached for inclusion.</td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.<br>Discussed that using routinely collected data is not necessarily a limitation, and the content of this item should be addressed in the 'Interpretation' section.<br>Decision: do not include the suggested new item; merge with CONSORT 2010 item 22 (23 in the final checklist); expand the E&amp;E text for clarification in the 'Generalisability' section.</td><td>Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses.</td></tr><tr><td>Generalisability</td><td>21</td><td>21</td><td>Generalisability (external validity and applicability) of the trial findings.</td><td></td><td></td><td>No suggested modification. Agreement to elaborate on the representativeness of the cohort or routinely collected database(s) used for the trial, including issues related to characteristics of eligible cohort or database participants who do not agree to participate in trial.<br>Decision: retain the CONSORT 2010 item; expand the E&amp;E text for clarification.</td><td>Generalisability (external validity and applicability) of the trial findings.</td></tr><tr><td>Interpretation</td><td>22</td><td>22</td><td>Interpretation consistent with results, balancing benefits and harms and considering other relevant evidence.</td><td></td><td></td><td>Item merged with the proposed new item 'Discuss the implications of using data that were not created or collected to answer the specific research question(s)'.<br>Decision: include the modified item.</td><td>Interpretation consistent with results, balancing benefits and harms and considering other relevant evidence, including the implications of using data that were not collected to answer the trial research questions.</td></tr><tr><td colspan="8"><strong>Other information</strong></td></tr><tr><td>Registration</td><td>23</td><td>23</td><td>Registration number and name of trial registry.</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item.</td><td>Registration number and name of trial registry.</td></tr><tr><td>Protocol</td><td>24</td><td>24</td><td>Where the full trial protocol can be accessed, if available.</td><td></td><td></td><td>No suggested modification.<br>Decision: retain the CONSORT 2010 item; expand the E&amp;E text for clarification.</td><td>Where the full trial protocol can be accessed, if available.</td></tr><tr><td>Funding</td><td>25</td><td>25</td><td>Sources of funding and other support (such as supply of drugs), role of funders.</td><td></td><td></td><td>Reached for inclusion.<br>Suggested minor revision to the item.<br>Decision: include the modified item with revision.</td><td>Sources of funding and other support for both the trial and the cohort or routinely collected database(s), role of funders</td></tr></tbody></table>

CONSORT, Consolidated Standards of Reporting Trials; E&amp;E, Explanation &amp; Elaboration; RCT, randomised controlled trial; ROUTINE, Extension for Trials Conducted Using Cohorts and Routinely Collected Data.
modifications made to the wording of two items for clarity (item 1b and item 9) in the final checklist.²⁴

## DISCUSSION

We have developed a consensus-driven extension to the CONSORT 2010 Statement for RCTs conducted using cohorts and routinely collected data.²⁴ CONSORT-ROUTINE contains minimum reporting requirements with appropriate flexibility as described in the Explanation & Elaboration part of our checklist document. This article described how we reached the final checklist and Explanation & Elaboration text and provides information on the decision-making process. We anticipate this paper will help others who may learn from our experiences and may apply this to the development of future guidelines or extensions.

There were several important strengths to our approach. A consensus-driven Delphi methodology, which is recommended when developing healthcare reporting guidelines by the EQUATOR network, was used to develop the extension.²³ We engaged with key stakeholders in trials research and potential end-users of the resultant CONSORT-ROUTINE reporting guideline throughout the development process. The process involved participants from a wide range of scientific disciplines and with diverse experience in conducting trials using different cohorts and routinely collected databases. As with other CONSORT-related guidelines, the inclusion of CONSORT Group members (IB, DM and PR) was intended to ensure consistency in the use of recommended methods in the development, dissemination and implementation of the extension. We recorded high response rates of 74% (92 respondents), 84% (77 respondents) and 81% (62 respondents) in Delphi rounds 1, 2 and 3, respectively. In addition, the number of registered participants and responders is larger than in most Delphi surveys used to develop healthcare reporting guidelines.<fnref n="8" /> ³⁵ ³⁶ Finally, we achieved a high degree of consensus that was consistent across Delphi survey rounds for the majority of the items.

There are also limitations to consider. One is that most participants were academic researchers with primary roles in trials research, and despite our broad engagement efforts, the number of participants from some stakeholder groups was small. One patient was included as a member of the reporting guideline development team, but no patients participated in the Delphi exercise. It is possible that perceptions about the importance of items might have differed across different stakeholder groups that might have favoured the inclusion or exclusion of certain items. Nonetheless, our project group included people from diverse backgrounds with expertise in using different types of data sources, who oversaw the development process to ensure that the checklist was equally applicable to, and representative of, all four types of data sources. A second is that our scoping review was not designed to capture each and every trial conducted using routinely collected data. This was in part because of the lack of accepted specific Medical Subject Headings terms

to identify these studies, or any research using routinely collected data, and the limited number of completed trials and methodological articles on these trial designs. For our purposes, it was not necessary to capture all trials that had been conducted using cohorts or routinely collected data, and we believe that we were able to capture a significant number of important trial reports and methodology papers that served as a basis for the development of our extension. A third is that the CONSORT-ROUTINE group predominantly consisted of members from high-income countries, which might have led to decreased applicability of the checklist for trials conducted in other settings. Finally, as with all reporting guidelines, ours will require re-evaluation and revisions over time to ensure that it is kept up to date with evolving research and knowledge on these trial designs.

## CONCLUSION

CONSORT-ROUTINE has now been developed and can be used to support comprehensive reporting of RCTs conducted using cohorts or routinely collected data. The extension statement contains minimum requirements of reporting that we encourage researchers to report. A baseline assessment of the completeness and reporting of these trial designs is being conducted, and the impact of the extension will be assessed in the coming years. While we anticipate that CONSORT-ROUTINE may need to be updated with the evolution of research methods, we hope the guideline will improve the reporting of RCTs conducted using cohorts and routinely collected data, enhance their interpretability and credibility of their results, improve their reproducibility, indirectly facilitate their robust design and conduct and lead to improved patient care.

## Author affiliations

<footnote n="1">¹Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Québec, Canada</footnote>

<footnote n="2">²Behavioural Science Institute, Clinical Psychology, Radboud University, Nijmegen, Netherlands</footnote>

<footnote n="3">³National Perinatal Epidemiology Unit Clinical Trials Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK</footnote>

<footnote n="4">⁴Center for Research on Population and Health, Faculty of Health Sciences, American University of Beirut, Ras Beirut, Lebanon</footnote>

<footnote n="5">⁵Institute of Applied Health Sciences, School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, Aberdeen, UK</footnote>

<footnote n="6">⁶Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland</footnote>

<footnote n="7">⁷Faculty of Health, Department of Cardiology, Örebro University, Örebro, Sweden</footnote>

<footnote n="8">⁸Department of Family Medicine, Western University, London, Ontario, Canada</footnote>

<footnote n="9">⁹IC/ES Western, London, Ontario, Canada</footnote>

<footnote n="10">¹⁰Centre for Clinical Trials and Methodology, Barts Institute of Population Health Science, Queen Mary University, London, UK</footnote>

<footnote n="11">¹¹Department of Psychology, McGill University, Montreal, Quebec, Canada</footnote>

<footnote n="12">¹²Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK</footnote>

<footnote n="13">¹³Department of Pediatrics and School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada</footnote>

<footnote n="14">¹⁴ICES uOttawa, Ottawa, Ontario, Canada</footnote>

<footnote n="15">¹⁵Children's Hospital of Eastern Ontario Research Institute, Ottawa, Ontario, Canada</footnote>

<footnote n="16">¹⁶Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada</footnote>

<footnote n="17">¹⁷Health Services Research Unit, University of Aberdeen, Aberdeen, UK</footnote>
**Correction notice** This article has been corrected since it was published. Deatils regarding final CONSORT-ROUTINE checklist under section Project phase 5: publication, dissemination and implementation has been corrected.

**Twitter** Lars G Hemkens @LGHemkens, Eric I Benchimol @ericbenchimol, Margaret Sampson @msampso, David Moher @dmoher and Chris Gale @DrCGale

**Contributors** MI, LK, OF, LGH, MZ, CR, SML, DM, MSam, CG, EJ and BDT were involved in initial phases of study conception, design of the search strategy and development of conceptual frameworks. SJM, KAM, DBR, EIB, LT, MKC, DE, HMV, IB, PR, JN, RU, MSau, JF and DT provided regular feedback on each of these steps. MI wrote the first draft with LK and BDT. All authors made provided critical revisions to the development of this manuscript and approved the final version.

**Funding** The development of CONSORT-ROUTINE was supported by the Canadian Institutes of Health Research (CIHR; PJT156172; PCS-161863), and the UK National Institute of Health Research (NIHR) Clinical Trials Unit Support Funding – Supporting efficient/innovative delivery of NIHR research (Principal Investigator (PI): EJ, co-PI: CG). DBR was supported by a Vanier CIHR Graduate Scholarship; SML was supported by a Wellcome Senior Clinical Fellowship in Science (205039/Z/16/Z); EIB was supported by a New Investigator Award from CIHR, the Canadian Association of Gastroenterology and Crohn's and Colitis Canada, and the Career Enhancement Program of the Canadian Child Health Clinician Scientist Program; RU was supported by the Canada Research Chairs Program (Award #231397); CG was supported by the UK Medical Research Council through a Clinician Scientist Fellowship; and BDT was supported by a Tier 1 Canada Research Chair, all outside of the present work.

**Disclaimer** The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

**Competing interests** None declared.

**Patient consent for publication** Not required.

**Provenance and peer review** Not commissioned; externally peer reviewed.

**Data availability statement** All data relevant to the study are included in the article or uploaded as supplemental information.

**Supplemental material** This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

**Open access** This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any

purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

## ORCID iDs

Ole Fröbert http://orcid.org/0000-0002-5846-345X
Danielle B Rice http://orcid.org/0000-0001-5615-7005
Sinéad M Langan http://orcid.org/0000-0002-7022-7441
David Moher http://orcid.org/0000-0003-2434-4206
Chris Gale http://orcid.org/0000-0003-0707-876X
Brett D Thombs http://orcid.org/0000-0002-5644-8432

## REFERENCES

1. Altman DG. Transparent reporting of trials is essential. *Am J Gastroenterol* 2013;108:1231-5.

2. Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. *Cochrane Database Syst Rev* 2012;11:MR000030.

3. Cobo E, Cortés J, Ribera JM, et al. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. *BMJ* 2011;343:d6783.

4. Moher D, Jones A, Lepage L. Consolidated standards for reporting of trials). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. *JAMA* 2001;285:1992-5.

5. Schulz KF, Altman DG, Moher D. Statement: updated guidelines for reporting parallel group randomised trials. *BMJ* 2010;2010:c332.

6. Juszczak E, Altman DG, Hopewell S, et al. Reporting of multi-arm parallel-group randomized trials: extension of the CONSORT 2010 statement. *JAMA* 2019;321:1610-20.

7. Campbell MK, Elbourne DR, Altman DG, et al. CONSORT statement: extension to cluster randomised trials. *BMJ* 2004;328:702-8.

8. Eldridge SM, Chan CL, Campbell MJ, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. *BMJ* 2016;355:i5239.

9. Zwarenstein M, Treweek S, Gagnier JJ, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. *BMJ* 2008;337:a2390.

10. Anderson GL, Burns CJ, Larsen J, et al. Use of administrative data to increase the practicality of clinical trials: insights from the women's health Initiative. *Clin Trials* 2016;13:519-26.

11. James S, Fröbert O, Lagerqvist B. Cardiovascular registries: a novel platform for randomised clinical trials. *Heart* 2012;98:1329-31.

12. James S, Rao SV, Granger CB. Registry-based randomized clinical trials-a new clinical trial paradigm. *Nat Rev Cardiol* 2015;12:312-6.

13. Relton C, Torgerson D, O'Cathain A, et al. Rethinking pragmatic randomised controlled trials: introducing the "cohort multiple randomised controlled trial" design. *BMJ* 2010;340:c1066.

14. van Staa T-P, Dyson L, McCann G, et al. The opportunities and challenges of pragmatic point-of-care randomised trials using routinely collected electronic records: evaluations of two exemplar trials. *Health Technol Assess* 2014;18:1-146.

15. Morrato EH, Elias M, Gericke CA. Using population-based routine data for evidence-based health policy decisions: lessons from three examples of setting and evaluating national health policy in Australia, the UK and the USA. *J Public Health* 2007;29:463-71.

16. Moskowitz A, Chen KP, Cooper AZ, et al. Management of atrial fibrillation with rapid ventricular response in the intensive care unit: a secondary analysis of electronic health record data. *Shock* 2017;48:436-40.

17. Hemkens LG, Saccilotto R, Reyes SL, et al. Personalized prescription feedback to reduce antibiotic overuse in primary care: rationale and design of a nationwide pragmatic randomized trial. *BMC Infect Dis* 2016;16:421.

18. Mc Cord KA, Ewald H, Ladanie A, et al. Current use and costs of electronic health records for clinical trial research: a descriptive study. *CMAJ Open* 2019;7:E23-32.

19. Wachtell K, Lagerqvist B, Olivecrona GK, et al. Novel trial designs: lessons learned from thrombus aspiration during ST-segment elevation myocardial infarction in Scandinavia (TASTE) trial. *Curr Cardiol Rep* 2016;18:11.

20. Benchimol EI, Smeeth L, Guttmann A, et al. The REporting of studies conducted using observational routinely-collected health data (RECORD) statement. *PLoS Med* 2015;12:e1001885.

21. Hemkens LG, Benchimol EI, Langan SM, et al. The reporting of studies using routinely collected health data was often insufficient. *J Clin Epidemiol* 2016;79:104-11.

<footnote n="18">¹⁸Library Services, Children's Hospital of Eastern Ontario, Ontario, Ottawa, Canada</footnote>

<footnote n="19">¹⁹Department of Cardiology, Clinical Sciences, Lund University, Lund, Sweden</footnote>

<footnote n="20">²⁰University Medical Center Utrecht, Utrecht, Netherlands</footnote>

<footnote n="21">²¹University of Utrecht, Utrecht, Netherlands</footnote>

<footnote n="22">²²Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada</footnote>

<footnote n="23">²³INSERM, Paris, France</footnote>

<footnote n="24">²⁴Centre d'Épidémiologie Clinique, Hôpital Hôtel Dieu, Assistance Publique-Hôpitaux de Paris, Paris, France</footnote>

<footnote n="25">²⁵Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité, Paris, France</footnote>

<footnote n="26">²⁶School of Health and Related Research, University of Sheffield, Sheffield, UK</footnote>

<footnote n="27">²⁷Department of Psychiatry, Dalhousie University, Halifax, Nova Scotia, Canada</footnote>

<footnote n="28">²⁸Scleroderma Society of Ontario, Hamilton, Ontario, Canada</footnote>

<footnote n="29">²⁹Scleroderma Canada, Hamilton, Ontario, Canada</footnote>

<footnote n="30">³⁰British Medical Journal, London, UK</footnote>

<footnote n="31">³¹York Trials Unit, Department of Health Sciences, University of York, York, UK</footnote>

<footnote n="32">³²Neonatal Medicine, School of Public Health, Faculty of Medicine, Imperial College London, London, UK</footnote>

<footnote n="33">³³Nottingham Clinical Trials Unit, University of Nottingham, Nottingham, UK</footnote>

<footnote n="34">³⁴Departments of Psychiatry; Epidemiology, Biostatistics and Occupational Health; Medicine; and Educational and Counselling Psychology; and Biomedical Ethics Unit, McGill University, Montreal, Quebec, Canada</footnote>
22 Kwakkenbos L, Juszczak E, Hemkens LG, et al. Protocol for the development of a CONSORT extension for RCTs using cohorts and routinely collected health data. *Res Integr Peer Rev* 2018;3:9.

23 Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. *PLoS Med* 2010;7:e1000217.

24 Kwakkenbos L, Imran M, McCall SJ, et al. Consort extension for the reporting of randomised controlled trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE): checklist with explanation and elaboration. *BMJ* 2021;373:n857.

25 Equator Network. Available: http://www.equator-network.org/library/reporting-guidelines-under-development/ [Accessed 2 Nov 2020].

26 Relton C, Burbach M, Collett C, et al. The ethics of 'Trials within Cohorts' (TwiCs): 2nd international symposium. *Trials* 2017;18:1–10.

27 von Elm E, Altman DG, Egger M, et al. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. *BMJ* 2007;335:806–8.

28 Kwakkenbos L, Imran M, McCord KA, et al. Protocol for a scoping review to support development of a CONSORT extension for randomised controlled trials using cohorts and routinely collected health data. *BMJ Open* 2018;8:e025266.

29 Trevelyan EG, Robinson PN. Delphi methodology in health research: how to do it? *Eur J Integr Med* 2015;7:423–8.

30 Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. *Inf Manage* 2004;42:15–29.

31 Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. *J Adv Nurs* 2000;32:1008–15.

32 McCall SJ, Imran M, Hemkens LG. A review of randomised controlled trials conducted using electronic health records with an assessment of completeness and transparency of reporting: study protocol, 2019. Available: https://osf.io/p6wa4/ [Accessed 2 Nov 2020].

33 Mc Cord KA, Imran M, McCall SJ. Systematic analysis of the reporting transparency and completeness of randomized controlled trials using registries: a protocol, 2019. Available: https://osf.io/x75gf/ [Accessed 2 Nov 2020].

34 Imran M, Mc Cord KA, McCall SJ. Protocol for a completeness and transparency of reporting study of randomised controlled trials conducted using administrative databases, 2019. Available: https://osf.io/dp23x/ [Accessed 2 Nov 2020].

35 Agha RA, Fowler AJ, Rajmohan S, et al. Preferred reporting of case series in surgery; the PROCESS guidelines. *Int J Surg* 2016;36:319–23.

36 Vohra S, Shamseer L, Sampson M, et al. CONSORT extension for reporting N-of-1 trials (CENT) 2015 statement. *BMJ* 2015;350:h1738.