A scoping review identifies multiple comments
suggesting modifications to SPIRIT 2013 and
CONSORT 2010

Camilla H Nejstgaard, Isabelle Boutron, An-Wen Chan, Ryan Chow, Sally Hopewell, Mouayad Masalkhi, David Moher, Kenneth F Schulz, Nathan A Shlobin, Lasse Østengaard, et al.

<figure><img src="image_1.png" /></figure>

To cite this version:

Camilla H Nejstgaard, Isabelle Boutron, An-Wen Chan, Ryan Chow, Sally Hopewell, et al. A scoping review identifies multiple comments suggesting modifications to SPIRIT 2013 and CONSORT 2010. Journal of Clinical Epidemiology, 2023, 155, pp.48-63. 10.1016/j.jclinepi.2023.01.003. hal-04086343

HAL Id: hal-04086343

https://cnam.hal.science/hal-04086343

Submitted on 2 May 2023

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire **HAL**, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ELSEVIER

<figure><img src="image_4.png" /></figure>

Journal of Clinical Epidemiology 155 (2023) 48–63

REVIEW ARTICLE

# A scoping review identifies multiple comments suggesting modifications to SPIRIT 2013 and CONSORT 2010

Camilla H. Nejstgaard<sup>a,b,*</sup>, Isabelle Boutron<sup>c,d</sup>, An-Wen Chan<sup>e</sup>, Ryan Chow<sup>f</sup>, Sally Hopewell<sup>g</sup>, Mouayad Masalkhi<sup>h</sup>, David Moher<sup>i,j</sup>, Kenneth F. Schulz<sup>k</sup>, Nathan A. Shlobin<sup>l</sup>, Lasse Østengaard<sup>a,b,m</sup>, Asbjørn Hróbjartsson<sup>a,b</sup>

Accepted 12 January 2023; Published online 18 January 2023

## Abstract

**Objectives:** To identify, summarize, and analyse comments on the core reporting guidelines for protocols of randomized trials (Standard Protocol Items: Recommendations for Interventional Trials [SPIRIT] 2013) and for completed trials (Consolidated Standards of Reporting Trials [CONSORT] 2010), with special emphasis on suggestions for guideline modifications.

**Methods:** We included documents written in English and published after 2010 that explicitly commented on SPIRIT 2013 or CONSORT 2010. We searched four bibliographic databases (Embase and MEDLINE to June 2022; Web of Science and Google Scholar to April 2022) and other sources (e.g., the EQUATOR Network website, the BMC Blog Network, and the BMJ rapid response section). Two authors independently assessed documents for eligibility and extracted data on basic characteristics and the wording of the main comments. We categorized comments as ‘suggestion for modification to the wording of an existing guideline item,’ ‘suggestion for a new item,’ or ‘reflections on challenges or strengths.’ We provided a summary and examples of the proposed suggestions and categorized comments into those that were directly linked to empirical investigations, were continuations of previous methodological discussions, or reflected new methodological developments.

**Results:** We assessed full text of 2,320 potentially eligible documents and included 93 documents with 114 comments. In total, 37 comments suggested modifications to existing guideline items. The participant flow section of CONSORT 2010 received the most comments (eight comments made different suggestions, e.g., one comment suggested to add numbers on nonrandomized screened participants). There were 46 comments suggesting new items. Multiple suggestions were related to trial interventions (eight comments made different suggestions, e.g., one comment suggested to add content on cointerventions), blinding (six comments suggested to add content on risk

Funding source: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data statement: The dataset used and analysed during this study is available upon request.

Declarations of interest: I.B., A-W. C., S.H., D.M., K.F.S., and A.H. are part of the joint executive group for the planned updates of SPIRIT and CONSORT. D.M. and A.H. are members of the editorial board of the Journal of Clinical Epidemiology.

**Conflict of Interest:** The authors declare no additional conflicts of interest.

**Author Contributions:** The review protocol was developed by C.H.N., I.B., A-W. C., S.H., D.M., K.F.S., L.Ø., and A.H. C.H.N., R.C., M.M., N.A.S., L.Ø., and A.H. assessed documents for inclusion and/or extracted data. C.H.N. performed the data analysis, and all authors participated in data interpretation. C.H.N. and A.H. wrote the draft review and all authors contributed in revising the review.

* Corresponding author. Centre for Evidence-Based Medicine Odense (CEBMO) & Cochrane Denmark, University of Southern Denmark, J.B. Winsløws Vej 9b, 3rd Floor, 5000 Odense, Denmark. Tel.: +45-23-60-44-74.

E-mail address: cahansen@health.sdu.dk (C.H. Nejstgaard).

<footnote><sup>a</sup>Centre for Evidence-Based Medicine Odense (CEBMO) and Cochrane Denmark, Department of Clinical Research, University of Southern Denmark, Denmark</footnote>

<footnote><sup>b</sup>Open Patient Data Explorative Network (OPEN), Odense University Hospital, Denmark</footnote>

<footnote><sup>c</sup>Université Paris Cité, Centre of Research in Epidemiology and Statistics (CRESS), Inserm, France</footnote>

<footnote><sup>d</sup>Cochrane France, France</footnote>

<footnote><sup>e</sup>Department of Medicine, Women's College Research Institute, University of Toronto, Canada</footnote>

<footnote><sup>f</sup>Faculty of Medicine, University of Ottawa, Canada</footnote>

<footnote><sup>g</sup>Oxford Clinical Trials Research Unit/Centre for Statistics in Medicine, University of Oxford, Oxford, UK</footnote>

<footnote><sup>h</sup>School of Medicine, University College Dublin, Ireland</footnote>

<footnote><sup>i</sup>Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Canada</footnote>

<footnote><sup>j</sup>School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Canada</footnote>

<footnote><sup>k</sup>Department of Obstetrics and Gynecology, School of Medicine, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA</footnote>

<footnote><sup>l</sup>Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA</footnote>

<footnote><sup>m</sup>University Library of Southern Denmark, University of Southern Denmark, Odense, Denmark</footnote>
of unblinding), statistical methods (five comments made different suggestions, e.g., one comment suggested to add content on blinding of statisticians), and participant flow (seven comments made different suggestions, e.g., three comments suggested to add content on missing data). Half (53%) of the suggestions were directly linked to empirical investigations. Six (7%) suggestions were continuations of previous methodological discussions and five (6%) suggestions reflected new methodological developments related to conflicts of interest and funding, data sharing, and patient and public involvement.

**Conclusion:** The issues raised provide context to authors, peer reviewers, editors, and readers of trials using SPIRIT 2013 and CONSORT 2010 and inform the planned updates of the core guidelines. © 2023 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

**Keywords:** SPIRIT; CONSORT; Reporting guidelines; Clinical trials; Comment; Reporting

# 1. Introduction

Results from randomized trials have a profound impact on patient care. When appropriately designed, conducted, and reported they provide trustworthy measurements of the effect of healthcare interventions. Reporting the ratio-nale and core methods in a protocol is a key part of con-ducting a randomized trial. A protocol enables appropriate assessment of the trial before it begins and full appraisal of the conduct and results after completion [1]. The importance of protocols is increasingly being recog-nized and some journals, such as BMJ, require authors of completed randomized trial reports to submit their study protocol alongside the report of the completed trial [2]. For completed trials, a similarly important step is the adequate reporting of protocol changes, results, interpreta-tions, and conclusions, typically in a trial report. Adequate reporting of a trial enables appraisal of risk of bias, the appropriate applicability of its results, and inclusion of data in a meta-analysis.

Comprehensive, clear, and transparent reporting is a pre-requisite for trial findings to reliably inform patient care and patient decision-making [3]. Unfortunately, important as-pects are often not reported adequately [4]. Studies have shown that reporting of trial outcomes [5], blinding [6], sam-ple size calculations [7], and allocation concealment [8] is often incomplete in trial protocols and completed reports.

To facilitate full reporting of information in trial proto-cols, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline was developed in 2013 [1]. Similarly, to facilitate full reporting of completed trial reports, the Consolidated Standards of Reporting Trials (CONSORT) was developed in 1996 with subsequent up-dates in 2001 and 2010. The purpose of updating CON-SORT was to prevent misinterpretation of specific items and to take into account new methodological developments [3].

SPIRIT 2013 and CONSORT 2010 are endorsed by many journals, regulators, research funders, editorial groups (e.g., the International Committee of Medical Jour-nal Editors), and patient groups [9,10] and are used routinely by trial authors. Several years have passed since the release of the guidelines, and the context of clinical trial

methodology is continually evolving. Similarly, open sci-ence practices related to clinical trials (e.g., data manage-ment plans) have started to be mandated and/or recommended. Therefore, the executive group for SPIRIT and CONSORT is planning to jointly update both reporting guidelines [11].

Several researchers have commented on the usability of SPIRIT 2013 and CONSORT 2010. For example, Laursen et al. suggested to include recommendations to report run-in periods [12], and duVaure et al. commented on the lack of recommendations to report authors’ financial con-flicts of interest (and not only funding source) [13]. We thought it relevant to provide an overview of such comments.

The objective of this scoping review was to identify, summarize, and analyse comments on SPIRIT 2013 and CONSORT 2010, with special emphasis on suggestions for guideline modifications.

# 2. Methods

## 2.1. Terminology

We use the term ‘documents’ to refer to published texts that include comments on SPIRIT 2013 or CONSORT 2010. This may cover, for example, empirical studies, edi-torials, and blogs.

We use the term ‘comments’ to refer to explicit state-ments on SPIRIT 2013 or CONSORT 2010 expressed in the included documents.

## 2.2. Protocol and registration

This scoping review was based on a predefined protocol (available from OSF [14]). We reported the review in accor-dance with the Preferred Reporting Items for Systematic re-views and Meta-Analyses extension for Scoping Reviews reporting guideline [15].

## 2.3. Eligibility criteria

We included documents written in English and pub-lished after 2010 that explicitly commented on either
## What is new?

### Key findings

* We identified 114 published comments on SPIRIT 2013 and CONSORT 2010. Comments commonly proposed modifications to the wording of the participant flow section of CONSORT 2010 (N = 8) and proposed new content related to the intervention (N = 8), blinding (N = 6), statistical methods (N = 5), and participant flow (N = 7) sections of both SPIRIT 2013 and CONSORT 2010.

* Half (53%) of the suggestions were directly linked to empirical investigations, whereas others were continuations of previous methodological discussions (7%) or reflected new methodological developments (6%).

### What this adds to what is known?

* We used multiple approaches to identify documents. Our review precedes the planned update of the reporting guidelines. Many of the identified comments suggested adding new content to SPIRIT 2013 or CONSORT 2010 and none suggested deleting items. SPIRIT 2013 and CONSORT 2010 aim to address the minimum content relevant for all randomized trial protocols and reports. The forthcoming revision of the guidelines needs to prioritize which new items are the most important and which changes can be implemented without introducing unnecessary complexity.

### What is the implication and what should change now?

* The issues raised may provide context to authors, peer reviewers, editors, and readers of trials using SPIRIT 2013 and CONSORT 2010 and inform the planned updates of the guidelines.

SPIRIT 2013 or CONSORT 2010. To be included, the documents had to mention SPIRIT 2013 or CONSORT 2010 somewhere in the text (i.e., not necessarily in the title or abstract). We only included comments on the most recent version of CONSORT (either stated directly by the document authors or verified by us based on the reference lists). We included opinion pieces (e.g., commentaries, letters, and editorials) and empirical studies and literature reviews. We also included suggestions for modifications and comments sent to members of the steering group of SPIRIT or CONSORT or posted on key websites (e.g., the EQUATOR Network website, https://www.equator-network.org/).

To be included, documents had to suggest modifications to SPIRIT 2013 or CONSORT 2010 or reflect on their challenges or strengths. Documents commenting on other aspects of the reporting guidelines (e.g., explaining the basis of the guidelines) were excluded. Thus, we excluded empirical studies that investigated the use of the guidelines but did not suggest modifications. We excluded documents that had only generic comments (e.g., briefly mentioning that CONSORT 2010 is a resource) or had only comments on implementation or endorsement of SPIRIT 2013 or CONSORT 2010. We excluded ‘peripheral remarks’ on the reporting guidelines (defined as very short remarks, i.e., no more than one sentence). We also excluded documents that had only comments related to SPIRIT or CONSORT extensions unless the suggestions were also relevant for the main guidelines. Finally, we excluded documents with comments suggesting the development of new extensions (e.g., suggestion to develop reporting recommendations for surgical adverse events [16]) and comments addressing application of SPIRIT 2013 or CONSORT 2010 on a specific subgroup of trials (e.g., suggestion to modify CONSORT 2010 to therapeutic medical devices [17]).

## 2.4. Information sources and search for documents

In a previous study investigating comments on the Cochrane tool for assessing risk of bias in randomized trials, it proved challenging to strike an appropriate balance between search sensitivity and specificity [18]. Therefore, we used multiple approaches to identify documents. The search strategy was developed in liaison with a search specialist (L.Ø.).

First, we performed a systematic basic search in two databases: Embase and MEDLINE (from January 2010 to June 2022). We used the search strategies in Appendix 1. We reviewed the search strategy using the PRESS 2015 Evidence-Based Checklist [19] in collaboration with a search specialist.

Second, we performed a focused search among documents citing SPIRIT 2013 or CONSORT 2010. We used Web of Science (from January 2010 to April 2022) to identify studies citing any of the SPIRIT 2013 or CONSORT 2010 statement or explanation publications listed on the SPIRIT or CONSORT websites where a Digital Object Identifier was available. We used the refine search function and the search terms in Appendix 2 to identify potentially eligible documents.

Third, we used Google Scholar (from January 2010 to April 2022) to conduct full text searches. We used standard phrases from the statement and explanation publications of the reporting guidelines and comments identified through the database searches (Appendix 3). For each search, we sorted the search records by relevance and stopped screening when no additional documents had been identified for a substantial amount of the sorted documents (> 50 records).
Fourth, we searched key websites (e.g., the EQUATOR Network website [20]), blogs (e.g., the BMC Blog Network [21]), and responses or comments to the SPIRIT 2013 or CONSORT 2010 statement or elaboration papers (e.g., through the BMJ rapid response section [22]) for additional documents (from January 2010 to April 2022). We searched proceedings from Cochrane Colloquia (covering conferences from 2010 to the most recent in 2020) [23] for conference abstracts with SPIRIT or CONSORT in the title and/or abstract.

Finally, we read reference lists of key publications and inspected personal files of the authors of this review and the executive group for SPIRIT and CONSORT.

## 2.5. *Selection of documents for inclusion*

From the searches in Embase and MEDLINE, duplicates were removed using EndNote and search records were managed using Covidence. One author (primarily C.H.N. or L.Ø.) screened titles and abstracts of all search records for obvious exclusion. One author (C.H.N. or N.A.S.) electronically screened full texts to exclude records with no mentioning of SPIRIT 2013 or CONSORT 2010, with sole mentioning of SPIRIT or CONSORT extensions, or not written in English. Two authors (C.H.N. and either R.C., M.M., or L.Ø.) independently screened remaining full texts of potentially eligible documents.

One author (C.H.N.) performed the additional searches. All eligible documents identified through these searches were verified for inclusion by a second author (R.C. or M.M.).

## 2.6. *Data extraction*

Two authors (C.H.N. and either R.C. or M.M.) independently extracted data and coded comments from each included document. Data were extracted into a pilot tested Excel sheet. We extracted the following basic characteristics of each document: first author, publication year, publication type (e.g., editorial or empirical study), and reporting guideline considered (i.e., SPIRIT 2013, CONSORT 2010, or both). We also extracted the exact wording of the main comments from each included document. One document could contain several comments. We defined separate comments as comments related to separate items or sections of the reporting guidelines.

We categorized comments as ‘suggestion for modification to the wording of an existing guideline item,’ ‘suggestion for a new item,’ or ‘reflections on challenges or strengths.’ We noted the SPIRIT 2013/CONSORT 2010 topic the comment was addressing (e.g., methods or results), and, when relevant, the specific SPIRIT 2013/CONSORT 2010 item number. For comments categorized as ‘suggestion for modification to the wording of an existing guideline item’ and ‘suggestion for a new item’, one author (C.H.N.) phrased a key point and this was verified by two

authors (R.C. and M.M.) independently. Moreover, one author (C.H.N.) categorized the importance of such comments (e.g., suggestion directly linked to an empirical investigation) and this was verified by a second author (A.H.).

## 2.7. *Synthesis of results*

We noted the numerical distribution of included documents and comments within each category and theme. We categorized each comment into SPIRIT 2013/CONSORT 2010 themes to merge similar topics. We qualitatively mapped the themes addressed by the documents. For comments categorized as ‘suggestion for modification to the wording of an existing guideline item’ or ‘suggestion for a new item’, we provided a summary and examples of the proposed suggestions.

We furthermore categorized comments into those that were directly linked to an empirical investigation, were continuations of previous methodological discussions, or reflected new methodological developments.

For comments categorized as ‘reflections on challenges or strengths’, we listed relevant comments in a table and provided a brief overview.

# 3. Results

We electronically or manually assessed full text of 2,320 potentially eligible documents (1,366 identified through MEDLINE and Embase, 954 identified through other sources) and included 93 published documents with 114 comments (Fig. 1 and Appendix 4). The documents were either empirical studies (69%) or opinion pieces (31%). Most of the 93 documents (87%) had comments that were primarily intended for CONSORT 2010 (Table 1). The majority of the 114 comments were related to the methods section (48 comments, 42%) or results section (31 comments, 27%) of the reporting guidelines.

## 3.1. *Suggestions for modifications to the wording of existing guideline items*

In total, 37 comments were suggestions for modifications of existing guideline items. The comments covered all main sections of the reporting guidelines but were mostly related to the methods (14 comments) and results (12 comments) sections. The topic receiving the most comments was the participant flow section of CONSORT 2010 (item 13a and 13b). In eight comments, the authors suggested modifications to the enrolment (e.g., add numbers on nonrandomized screened participants), allocation (e.g., clarify reasons for exclusion following randomisation), follow-up (e.g., add duration of follow-up), analysis (e.g., add number of participants with and without the measured outcome), and design (e.g., remove connecting line between follow-up section and analysis section) parts of the
<figure><img src="image_5.png" /></figure>

*Number of records identified in the databases; †Number of full text records identified through other sources. These searches were performed by one author, and we did not record exclusion reasons. All eligible documents were verified for inclusion by a second author.

<caption>**Fig. 1.** Flow chart of the inclusion of documents.</caption>

CONSORT 2010 flow diagram (Table 2 and Appendix 5). Furthermore, multiple comments were related to the intervention (three comments made different suggestions, e.g., two comments suggested to include details on timing of the treatments), statistical methods (three comments made different suggestions, e.g., one comment suggested to include detailed guidance on the use and reporting of subgroup analyses), the harms results (three comments made different suggestions, e.g., one comment suggested to require all harms to be reported rather than just important harms), and interpretation of findings (four comments made different suggestions, e.g., one comment suggested to include a statement on blinded interpretation) parts of SPIRIT 2013 and CONSORT 2010 (Table 2 and Appendix 5).

## 3.2. Suggestions for new items

In 46 comments, suggestions for new items were made. The comments were mostly related to the methods section (21 comments). The intervention section of SPIRIT 2013 and CONSORT 2010 received the most comments. In eight comments, the authors made different suggestions. For example, comments suggested to add content on quality of the drugs used, intervention implementation strategies, and cointerventions (Table 3 and Appendix 6). Furthermore, multiple comments were related to the blinding (six comments suggested to add content on risk of unblinding), statistical methods (five comments made different suggestions, e.g., one comment suggested to add content on blinding of statisticians), and participant flow (seven comments made
<table><caption>**Table 1.** Characteristics of documents commenting on SPIRIT 2013 or CONSORT 2010</caption><thead><tr><th>Document characteristics</th><th>N (%) of included documents</th></tr></thead><tbody><tr><td>Publication type</td><td></td></tr><tr><td>Empirical study<sup>a</sup></td><td>64 (69)</td></tr><tr><td>Opinion piece<sup>b</sup></td><td>29 (31)</td></tr><tr><td>Document publication year</td><td></td></tr><tr><td>2010–2012</td><td>25 (27)</td></tr><tr><td>2013–2015</td><td>20 (22)</td></tr><tr><td>2016–2018</td><td>19 (20)</td></tr><tr><td>2019–2022</td><td>29 (31)</td></tr><tr><td>Reporting guideline considered</td><td></td></tr><tr><td>SPIRIT 2013</td><td>3 (3)</td></tr><tr><td>CONSORT 2010</td><td>81 (87)</td></tr><tr><td>Both SPIRIT 2013 and CONSORT 2010</td><td>9 (10)</td></tr><tr><td>Number of comments per document<sup>c</sup></td><td></td></tr><tr><td>1</td><td>84 (90)</td></tr><tr><td>2</td><td>4 (4)</td></tr><tr><td>3</td><td>3 (3)</td></tr><tr><td>5</td><td>1 (1)</td></tr><tr><td>8</td><td>1 (1)</td></tr></tbody></table>

<sup>a</sup> Empirical studies include, for example, systematic reviews, experimental studies, and cross-sectional studies.

<sup>b</sup> Opinion pieces include, for example, commentaries and letters.

<sup>c</sup> Number of comments in each published document (one document may contain multiple comments on SPIRIT 2013 or CONSORT 2010).

different suggestions, e.g., three comments suggested to add content on missing data), and parts of SPIRIT 2013 and CONSORT 2010 (Table 3 and Appendix 6).

### 3.3. Importance of the proposed suggestions

Half (53%) of the suggestions for modifications of existing guideline items (37 comments) and new items (46 comments) were directly linked to empirical investigations. In most cases, the authors conducted an empirical study investigating reporting in a sample of randomized trials and proposed changes to SPIRIT 2013 and/or CONSORT 2010 on the basis of their findings. For example, Sweetman et al. investigated reporting in 80 trials and found that protocol violations were often under-reported. Therefore, they suggested to modify existing items on protocol amendments and provide explicit reporting requirements on protocol violations [24]. Moreover, Zhang et al. investigated reporting in 221 trials and found that reporting of subgroup analyses was neither uniform nor complete. Therefore, they suggested to modify existing items on statistical methods and include more detailed guidance on the use and reporting of subgroup analyses [25].

A few of the suggestions for modifications of existing guideline items (37 comments) and new items (46 comments) were continuations of previous methodological

discussions. In six comments (7%), the authors reacted on the removal of risk of unblinding from the CONSORT 2001 version [26]. Finally, five comments (6%) made suggestions that reflected new methodological developments. Two comments suggested to include reporting requirements related to trial authors' financial conflicts of interest and funding amount [13,27], two comments suggested to include reporting requirements of data sharing [28,29], and one comment suggested to include reporting requirements on patient and public involvement [30].

### 3.4. Reflections on challenges and strengths

In total, 28 comments reflected on challenges of SPIRIT 2013 and CONSORT 2010 and most of these were either related to the methods section (12 comments), results section (six comments), or were generic comments not related to any specific section (nine comments). In addition, three comments reflected on strengths of CONSORT 2010 (Table 4).

## 4. Discussion

### 4.1. Summary of main findings

Comments on SPIRIT 2013 and CONSORT 2010 made multiple suggestions for modifying the wording of existing guideline items and adding new items. The comments covered all aspects of the reporting guidelines but were often related to the methods or results section. Several comments proposed modifications to the wording of the participant flow section of CONSORT 2010 and several comments proposed adding content related to the intervention, blinding, statistical methods, and participant flow sections of SPIRIT 2013 and CONSORT 2010. Half of the suggestions were directly linked to empirical investigations and additional few comments were continuations of previous methodological discussions or reflected new methodological developments.

### 4.2. Strengths and weaknesses

We used multiple approaches to identify documents. Our review precedes the planned update of the reporting guidelines. However, it is challenging to search for documents with comments as not all are indexed in standard databases. We may not have identified all eligible documents but it is unlikely that any missing comments would change our main findings or qualitative conclusions.

### 4.3. Other similar studies

Our review complements previous studies that have focused on adherence and endorsement of the reporting guidelines. Empirical studies have reported that adherence to the CONSORT 2010 reporting guideline is suboptimal in, for example, addiction trials [31], COVID-19 trials [32], and in trials on statins or fibrates for diabetic
<table><caption>**Table 2.** Suggestions for modifications to the wording of existing guideline items (37 comments in 29 documents)</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Item number</th><th>Document ID</th><th>Comment primarily intended for</th><th>Key point</th></tr></thead><tbody><tr><td>Title and abstract, comment related to title</td><td>SPIRIT 1 CONSORT 1a</td><td>Nicholls 2022</td><td>CONSORT</td><td>Include a statement on whether the trial report is primary or nonprimary (e.g., a subgroup analysis)</td></tr><tr><td>Other information, comment related to registration</td><td>SPIRIT 2a CONSORT 23</td><td>Reveiz 2010a</td><td>CONSORT</td><td>Include the timing of registration (prospectively or retrospectively registered trial)</td></tr><tr><td>Other information, comment related to funding</td><td>SPIRIT 4 CONSORT 25</td><td>Siddiq 2019</td><td>CONSORT</td><td>Include the funding amount or budget allocation, rewards, and reimbursements</td></tr><tr><td>Introduction, comment related to background and objectives</td><td>SPIRIT 7 CONSORT 2b</td><td>Nicholls 2022</td><td>CONSORT</td><td>Include details on the primary or nonprimary nature of the trial report (i.e., whether results are addressing primary objectives or secondary objectives only)</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comment related to participants or study setting</td><td>SPIRIT 9 CONSORT 4b</td><td>Patterson 2010</td><td>CONSORT</td><td>Include an estimate of the potential study population (as an addition to 'assessed for eligibility') and details on planned and actual recruitment sites and strategies</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comments related to participants or eligibility criteria</td><td>SPIRIT 10 CONSORT 4a</td><td>Cals 2011 Yelland 2019</td><td>CONSORT SPIRIT and CONSORT</td><td>Include details on the similarity between participants in the current trial and participants in previous trials that established efficacy of the control intervention<br>Include information on repeat participation</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comments related to interventions</td><td>SPIRIT 11a CONSORT 5</td><td>Bryant 2014 Cals 2011 Hoffmann 2014</td><td>CONSORT CONSORT CONSORT</td><td>Add details on the interventions, including: level (individual, cluster, or both), timing of treatment, content of intervention materials, administration and delivery (who and where), intervals between delivery of intervention components, any tailoring or standardization of the intervention, rationale for the type of control, and information on whether the control is identical to that in any previous trials that established efficacy</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comment related to outcomes</td><td>SPIRIT 12 CONSORT 6b</td><td>Downey 2016</td><td>CONSORT</td><td>Include guidance on assessing composite outcome measures and the justifiable basis for changing outcome measurements</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comments related to sample size</td><td>SPIRIT 14 CONSORT 7a</td><td>Cals 2011 Zakeri 2018</td><td>CONSORT CONSORT CONSORT</td><td>Include details on whether and how clustering by care providers or centres was addressed<sup>a</sup><br>Include rationale for the hypothesized effect size (ideally based on evidence from prior research)</td></tr><tr><td>Methods: Assignment of interventions, comments related to blinding</td><td>SPIRIT 17a CONSORT 11a</td><td>Blanco 2018 Funada 2022</td><td>CONSORT CONSORT</td><td>Delete the phrase 'if done'<br>Modify blinding items to separate and report each blinding</td></tr><tr><td>Methods: Data collection, management, and analysis, comments related to statistical methods</td><td>SPIRIT 20a CONSORT 12a<br>SPIRIT 20b CONSORT 12b</td><td>Cals 2011 Rivoirard 2016<br>Zhang 2015</td><td>CONSORT CONSORT CONSORT</td><td>Include specification on whether a one-sided or two-sided confidence interval approach was used<br>Include advice on checking if the tests used in the results section were consistent with those described in the methods section<br>Include more detailed guidance on the use and reporting of subgroup analyses</td></tr><tr><td>Ethics and dissemination, comment related to protocol amendments</td><td>SPIRIT 25 CONSORT 3b</td><td>Sweetman 2011</td><td>CONSORT</td><td>Include explicit reporting requirements on protocol violations</td></tr></tbody></table>

(Continued)
<table><caption>**Table 2. Continued**</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Item number</th><th>Document ID</th><th>Comment primarily intended for</th><th>Key point</th></tr></thead><tbody><tr><td>Ethics and dissemination, comment related to protocol</td><td>SPIRIT 31c<br>CONSORT 24</td><td>Reveiz 2010b</td><td>CONSORT</td><td>Include statement that protocols can be submitted in languages other than the language of the journal</td></tr><tr><td rowspan="10">Results, comments related to participant flow</td><td rowspan="10">CONSORT 13a, 13b and flow diagram</td><td>Cals 2011</td><td>CONSORT</td><td rowspan="10">Suggested modifications for existing parts:<br>Enrollment:<br>- Number of participants or units approached to take part in the trial, the number which were eligible, and reasons for nonparticipation<br>- More details on the process of inclusion<br>- Numbers on nonrandomized screened participants<br>- Details on participants in screening, eligibility, approached, and randomized stages<br>Allocation:<br>- Number of care providers or centres performing the intervention in each group<br>- Number of patients treated by each care provider or in each centre<br>- Add reasons for exclusion following randomisation<br>Follow-up:<br>- Time spent on the trial for all participants<br>- Number of participants with unknown primary end point status and/or unknown vital status<br>- Number of participants who discontinue study intervention<br>- Duration of follow-up<br>- More guidance on the difference between loss to follow-up and discontinuation of the intervention<br>Analysis:<br>- Number of participants with and without a measured outcome<br>- Number of participants included or excluded in analysis<br>- Categories to allow distinction between number analysed and number for whom the outcomes were known and imputed<br>Suggested modifications for the design:<br>- Remove connecting line between follow-up section and analysis section</td></tr><tr><td>Campbell 2016</td><td>CONSORT</td></tr><tr><td>Deo 2011</td><td>CONSORT</td></tr><tr><td>Hopewell 2011</td><td>CONSORT</td></tr><tr><td>Kearney 2017</td><td>CONSORT</td></tr><tr><td>Péron 2013</td><td>CONSORT</td></tr><tr><td>Rønsbo 2021</td><td>CONSORT</td></tr><tr><td>Wilson 2018</td><td>CONSORT</td></tr><tr><td>Results, comment related to baseline data</td><td>CONSORT 15</td><td>Cals 2011</td><td>CONSORT</td><td>Include baseline data for each group at individual and cluster levels</td></tr><tr><td rowspan="2">Results, comments related to harms</td><td rowspan="2">CONSORT 19</td><td>Gorrell 2016</td><td>CONSORT</td><td>Avoid the term 'side effect', prefer the term 'adverse event'</td></tr><tr><td>Jull 2020</td><td>CONSORT</td><td>Require all harms to be reported rather than 'important harms'</td></tr><tr><td></td><td></td><td>Ting 2010</td><td>CONSORT</td><td>Place emphasis on patient's own first hand impression of adverse symptoms during trial participation and follow-up (rather than clinician impression)</td></tr></tbody></table>

(Continued)
<table><caption>**Table 2. Continued**</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Item number</th><th>Document ID</th><th>Comment primarily intended for</th><th>Key point</th></tr></thead><tbody><tr><td>Discussion, comment related to limitations</td><td>CONSORT 20</td><td>Cals 2011</td><td>CONSORT</td><td>Include reflections on the choice of comparator, lack of or partial blinding, and unequal expertise of or recruitment by care providers or centres</td></tr><tr><td rowspan="4">Discussion, comments related to interpretation</td><td rowspan="4">CONSORT 22</td><td>Bacchetti 2010</td><td>CONSORT</td><td>Include guidance on how misinterpretation should be avoided (e.g., by clarifying that interpretation should take into account the estimated effects and its confidence interval rather than focusing on the P value)</td></tr><tr><td>Jellison 2019</td><td>CONSORT</td><td>Include language discouraging spin</td></tr><tr><td>Järvinen 2014</td><td>CONSORT</td><td>Include an explicit statement about whether the authors conducted blinded interpretation</td></tr><tr><td>Nicholls 2022</td><td>CONSORT</td><td>Include a statement on the primary or nonprimary nature of the trial report</td></tr></tbody></table>

retinopathy [33]. Moreover, studies have reported that journal endorsement of previous versions of CONSORT and its extensions may improve reporting quality [34,35] and that the use of a CONSORT-based peer review tool (COBPeer) may improve peer reviewers ability to detect inadequate reporting in randomized trials [36].

The methods used in our scoping review are similar to those used in evaluations of other methodological tools. Jørgensen et al. summarized published comments on the first version of the Cochrane tool for assessing risk of bias in randomized clinical trials to provide a basis for a revision of the tool. They identified 68 comments that were categorized as per whether they expressed strengths, challenges, or suggestions to changes of the tool [18].

### 4.4. Mechanisms and perspectives

Many of the 114 comments suggested adding new content to SPIRIT 2013 or CONSORT 2010 and none suggested to delete items. SPIRIT 2013 and CONSORT 2010 aim to address the minimum content relevant for all randomized trial protocols and reports. The forthcoming revision of the guidelines needs to prioritize which new items are the most important and which changes can be implemented without introducing unnecessary complexity.

New methodological developments have emerged or received increasing attention since the development of SPIRIT 2013 and CONSORT 2010. Three themes, trial authors' financial conflicts of interest, data sharing, and patient and public involvement, were highlighted in the included comments. Additional themes such as remote clinical trials [37], patient-centric trials [38], basket trials, umbrella trials, and platform trials [39,40] may also be worth considering, although such issues may be better addressed in extensions than in the main reporting guidelines.

The threshold for publicly commenting on problems with SPIRIT 2013 or CONSORT 2010 may be lower than for commenting on strengths. This may explain why we found a larger number of comments on challenges than comments on strengths, which contrasts the wide use of both reporting guidelines. Authors who comment may also have a particular interest in a specific topic that may not otherwise meet the threshold for inclusion in a checklist of minimum content for all randomized trials.

### 4.5. Implications

SPIRIT 2013 and CONSORT 2010 are among the most well-known reporting guidelines. CONSORT has been listed as a top health research milestone in the 20th century [41]. Both SPIRIT 2013 and CONSORT 2010 are endorsed by hundreds of journals [9,10] and several prominent organizations, such as the International Committee of Medical Journal Editors [42]. We suggest that our findings provide context to users of both guidelines (e.g., trial authors, peer reviewers, editors, and readers of trials).

Furthermore, developing and revising health research reporting guidelines is an extensive process involving a series of steps including, among others, to seek feedback and criticism from various stakeholders [43]. Therefore, we suggest that the issues raised are considered for the planned updates of SPIRIT 2013 and CONSORT 2010.

## 5. Conclusion

We identified 114 comments on SPIRIT 2013 and CONSORT 2010, covering all aspects of the reporting guidelines but often related to the methods or results sections. Modifications were suggested to the participant flow section of

<footnote>a Comment also applicable to SPIRIT 2013 item 20a and CONSORT 2010 item 12a.</footnote>
<table><caption>**Table 3.** Suggestions for new items in SPIRIT 2013 or CONSORT 2010 (46 comments in 42 documents)</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Document ID</th><th>Comment primarily intended for</th><th>Key point</th></tr></thead><tbody><tr><td rowspan="2">Other information, comments related to funding</td><td>Leichsenring 2017</td><td>CONSORT</td><td>Include content on trialists allegiance (e.g., whether the treatment or etiological model was developed or advocated by one of the authors, whether the therapists were trained or supervised by one of the authors, whether the therapists orientation matches with trial conditions, and whether the treatments were structurally comparable in relation to duration or dose)</td></tr><tr><td>duVaura 2014</td><td>CONSORT</td><td>Include content on author's financial conflicts of interest</td></tr><tr><td>Other information, comment related to roles and responsibilities</td><td>Conroy 2015</td><td>CONSORT</td><td>Include content on trial steering committees</td></tr><tr><td>Introduction, comment related to background and objectives</td><td>Gambrill 2011</td><td>CONSORT</td><td>Include content on problem framing (e.g., medicalization of common concerns)</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comment related to participants</td><td>Ntala 2013</td><td>CONSORT</td><td>Include content regarding the methods of recruitment</td></tr><tr><td rowspan="2">Methods: Participants, interventions, and outcomes, comments related to interventions</td><td>Armijo-Olivo 2020</td><td>CONSORT</td><td>Include content on defining compliance with treatment, the methods of tracking this, and the monitoring and reporting of percentage of compliance and handling of partial compliance in the analyses</td></tr><tr><td>Golomb 2010</td><td>CONSORT</td><td>Include content on the test agent (i.e., describe test agent or drug in detail, give full constituents by weight for chemical compounds, describe appearance)<br/>Include content on (placebo) control (i.e., describe (placebo) control treatment in detail, give full constituents by weight for chemical compounds, stipulate appearance and any differences from the test drug, and state what other factors might render the experience of the control distinctive from the test agent)</td></tr><tr><td rowspan="7">Methods: Assignment of interventions (for controlled trials), comments related to blinding</td><td>Johnson 2020</td><td>CONSORT</td><td>Incorporate TIDieR items</td></tr><tr><td>Levack 2020</td><td>CONSORT</td><td>Include content regarding the rationale for why an experimental intervention might be more effective than the comparison intervention</td></tr><tr><td>Newton 2015</td><td>SPIRIT and CONSORT</td><td>Include content on quality of drugs and medical products used in the trial</td></tr><tr><td>Rudd 2020</td><td>SPIRIT and CONSORT</td><td>Include content on intervention implementation strategies</td></tr><tr><td>Shaheed 2021</td><td>CONSORT</td><td>Include content on cointerventions (covering type of intervention and frequency or duration of use)</td></tr><tr><td>Verhagen 2011</td><td>CONSORT</td><td>Include content on measuring and reporting adherence to the interventions</td></tr><tr><td>Bian 2011</td><td>CONSORT</td><td>Include content on procedures intended to prevent, record, and deal with cases of overt unblinding, including success of blinding or risk of unblinding</td></tr><tr><td rowspan="5">Methods: Data collection, management, and analysis, comments related to statistical methods</td><td>Colagiuri 2010</td><td>SPIRIT and CONSORT</td><td></td></tr><tr><td>Colagiuri 2019</td><td>SPIRIT and CONSORT</td><td></td></tr><tr><td>Webster 2021</td><td>CONSORT</td><td></td></tr><tr><td>Armijo-Olivo 2020</td><td>CONSORT</td><td>Include content on reporting type, purpose, conduct, and consistency of results when different analyses (intention to treat, per protocol, as treated) are conducted</td></tr><tr><td>Cro 2020<sup>a</sup></td><td>SPIRIT and CONSORT</td><td>Include content on when statisticians become unblinded to the results or data</td></tr><tr><td>Kahan 2021</td><td>SPIRIT and CONSORT</td><td>Include content on estimands (i.e., what treatment effect is being estimated for the primary outcome)</td></tr><tr><td>Stevely 2015</td><td>CONSORT</td><td>Include content on bias correction for the early withdrawal of treatment</td></tr></tbody></table>

(Continued)
<table><caption>**Table 3. Continued**</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Document ID</th><th>Comment primarily intended for</th><th>Key point</th></tr></thead><tbody><tr><td></td><td>Yelland 2018</td><td>SPIRIT and CONSORT</td><td>Include content on how recruitment, randomization, and treatment errors are handled</td></tr><tr><td>Methods: Monitoring, comment related to data monitoring</td><td>Korn 2011</td><td>CONSORT</td><td>Include content on reporting reasons for not prespecifying formal inefficacy monitoring or reporting reasons for not following prespecified monitoring guidelines</td></tr><tr><td rowspan="2">Ethics and dissemination, comments related to research ethics approval</td><td>Owyang 2021</td><td>CONSORT</td><td>Include content on patient and public involvement</td></tr><tr><td>Strech 2014</td><td>CONSORT</td><td>Include content on ethical assessment</td></tr><tr><td>Ethics and dissemination, comment related to consent or assent</td><td>Pretto-Lazarova 2021</td><td>CONSORT</td><td>Include content on informed consent for special population groups (e.g., children)</td></tr><tr><td rowspan="2">Ethics and dissemination, comments related to dissemination policy</td><td>Barbui 2016</td><td>CONSORT</td><td rowspan="2">Include content on data sharing (covering what to share, when, and how)</td></tr><tr><td>Smith 2015</td><td>CONSORT</td></tr><tr><td rowspan="10">Results, comments related to participant flow</td><td>Armijo-Olivo 2020</td><td>CONSORT</td><td>Include content on attrition (random or nonrandom, timing, and number and reasons for participant dropout)</td></tr><tr><td>Fielding 2016</td><td>CONSORT</td><td rowspan="3">Include content on reporting missing data and retention data, including how missing data are handled (covering justification of the methods used and a missing data sensitivity analysis)</td></tr><tr><td>Hussain 2017</td><td>CONSORT</td></tr><tr><td>Wadood 2019</td><td>CONSORT</td></tr><tr><td>Glasgow 2018</td><td>CONSORT</td><td>Suggestions for new sections in the CONSORT flow diagram:<ul><li>Include level of setting and staff with data on participation and representativeness</li><li>Include level of setting and intervention sustainability after project support ends</li></ul></td></tr><tr><td>Heidari 2012</td><td>CONSORT</td><td>Include content on sex differences</td></tr><tr><td>Kearney 2022</td><td>CONSORT</td><td>Include content on imputed primary outcome data and reasons for imputation</td></tr><tr><td>Laursen 2019</td><td>SPIRIT and CONSORT</td><td>Include content on run-in periods</td></tr><tr><td>Yelland 2018</td><td>SPIRIT and CONSORT</td><td>Include content on number and type of recruitment, randomization, and treatment errors that occurred during the trial</td></tr><tr><td>Results, comment related to baseline data</td><td>Furler 2012</td><td>CONSORT</td><td>Include content on reporting socioeconomic status characteristics of trial participants</td></tr><tr><td>Results, comment related to numbers analysed</td><td>Armijo-Olivo 2020</td><td>CONSORT</td><td>Include content on participant exclusions from the analysis, reasons for exclusion, and imputation methods</td></tr><tr><td>Results, comment related to outcomes and estimation</td><td>Schriger 2012</td><td>CONSORT</td><td>Include content on using distributions to depict each subject's continuous outcome</td></tr><tr><td>Results, comment related to harms</td><td>Konwar 2022</td><td>CONSORT</td><td>Include content on safety statistics</td></tr><tr><td>Discussion, comment related to generalisability</td><td>Buchtele 2022</td><td>CONSORT</td><td>Include content on pragmatism</td></tr><tr><td>Discussion, comment related to interpretation</td><td>Glujovsky 2016</td><td>CONSORT</td><td>Include content on clinical importance</td></tr></tbody></table>

<footnote>ª Key point based on quote from included document combined with personal communication with authors. In addition to the comments included in the table, one comment was not related to reporting of trial protocols or reports as the authors suggested to mention guest or ghost authorship in the CONSORT 2010 statement, for example, by referring to the ICMJE guidelines (Shaw 2010).</footnote>
<table><caption>**Table 4.** Comments on challenges or strengths in SPIRIT 2013 or CONSORT 2010 (31 comments in 28 documents)</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Document ID</th><th>Comment primarily intended for</th><th>Quote</th></tr></thead><tbody><tr><td colspan="4">Comments on challenges</td></tr><tr><td rowspan="3">Methods: Participants, interventions, and outcomes, comments related to interventions</td><td>Baker 2010</td><td>CONSORT</td><td>"It should be noted that the CONSORT criteria (as well developed as they are) cover only a subset of issues related to intervention evaluation"</td></tr><tr><td>Page 2017</td><td>SPIRIT and CONSORT</td><td>"[...] CONSORT lists "interventions" as one of 22 items to address, asking authors to provide "precise details of the intervention intended for each group and how and when they were actually administered". [...] Although SPIRIT also requires an intervention description as one of 33 items, it does require more specific reporting guidelines, adding detail on modifications, adherence, and concomitant interventions to the description. Such general instruction about the reporting of therapeutic interventions often leaves researchers uncertain of exactly how much to report [...]"</td></tr><tr><td>Torgerson 2020</td><td>CONSORT</td><td>"The statement [CONSORT] has a single item related to the trial intervention, but fulfilment of this item does not always signify inclusion of sufficient information for researchers to reproduce the interventions in subsequent studies or for health care providers to apply these interventions in the clinical setting"</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comment related to interventions, comment related to outcomes</td><td>Kyte 2014</td><td>SPIRIT</td><td>"[...] existing PRO [patient-reported outcome] guidance for protocol writers lacks consistency and is difficult to access, whereas PRO-specific protocol items are not fully addressed by the current SPIRIT statement"</td></tr><tr><td rowspan="5">Methods: Assignment of interventions, comments related to blinding</td><td>Cals 2011</td><td>CONSORT</td><td>"The CONSORT group has stopped advocating testing for blinding in the 2010 Statement. This removal is in our view rather unfortunate because it hinders a more thorough assessment of the success of blinding, although we agree with the remark in the 2010 main Statement that it is of limited value when assessed after the primary outcome has been reached"</td></tr><tr><td>Hemilä 2010</td><td>CONSORT</td><td>"There is evident citation bias in the article by Moher et al. [the CONSORT explanation and elaboration paper]. When arguing that the lack of blinding causes bias in controlled trials, they refer to an old study which supports their preconceptions [Karlowski 1975], ignoring the evidence which indicates that the old study was erroneously analyzed. In addition, they ignore an extensive meta-analysis which analyses the effect of blinding on 60 clinical conditions [Hróbjartsson 2010]"</td></tr><tr><td>Hopton 2011</td><td>CONSORT</td><td>"[...] the decision to eliminate the recommendation of CONSORT to report "how the success of blinding was evaluated" is understandable, yet the criticism of the decision as a philosophy of "let us give up because it is difficult to do or to interpret" is equally justified. The new CONSORT recommendations go some way to simplifying what should be included, but leave the assessment of blinding to be done by the readers of reports rather than those writing them"</td></tr><tr><td>Kolahi 2010</td><td>CONSORT</td><td>"[...] we are concerned by the removal in the most recent iteration of CONSORT of the recommendation that authors report measures of the blindness"</td></tr><tr><td>Webster 2021</td><td>CONSORT</td><td>"[...] the change in the CONSORT recommendation from asking researchers to report on success of blinding (if measured) to not asking, seems to have been based on arguments that may deserve revisiting. [...] However, the fact that CONSORT sites an article by Sackett as the reason for removing it, in which he claims that testing the success of blinding is a 'mug's game' could be interpreted as a reason to avoid reporting on the success of blinding"</td></tr></tbody></table>

(Continued)
<table><caption>**Table 4. Continued**</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Document ID</th><th>Comment primarily intended for</th><th>Quote</th></tr></thead><tbody><tr><td rowspan="3">Methods: Data collection, management, and analysis, comments related to statistical methods</td><td>Cashin 2021</td><td>CONSORT</td><td>"There are no specific reporting guidelines for studies that use mediation analysis [...] CONSORT [...] do not cover the additional aspects of study design, analysis, and effects that should be reported in a mediation analysis"</td></tr><tr><td>Kahan 2020</td><td>SPIRIT</td><td>"[...] however, the primary aim of these guidelines [SPIRIT] is to describe what information should be included in the protocol or Statistical Analysis Plan rather than describe exactly how the analysis should be designed. As such, these guidelines do not offer a prescriptive approach for how analysis strategies should be designed to limit p-hacking"</td></tr><tr><td>Palys 2013</td><td>CONSORT</td><td>"The CONSORT description of intent to treat (ITT) also lacks precision, as authors often misrepresent what they did by referring to analyses as ITT even when some randomized patients have been excluded from these analyses [...] Nor does CONSORT call for adequate handling of missing data, so analyses based on the assumption of missing data being completely random, which is virtually unattainable in practice, can still get full credit, even if they do not also use Lachin's conservative worst-rank approach [Lachin, Control Clin Trials 1999] for validity"</td></tr><tr><td rowspan="5">Results, comments related to participant flow</td><td>Armijo-Olivo 2020</td><td>CONSORT</td><td>"Although standard guidelines to report RCTs [randomized controlled trials] exist, such as the CONSORT statement, a lack of clear guidance on how to report and evaluate attrition and compliance related biases still remains"</td></tr><tr><td>Elm 2014</td><td>CONSORT</td><td>"The general implication of this statement [the CONSORT explanation for reporting screening counts] is that by collecting screening data [in the CONSORT flow diagram] it will be possible to demonstrate the generalizability of the findings and lack of selection bias in the study subjects. This premise is, however, a relatively crude assessment because the counts themselves tell us nothing about actual specific clinical characteristics of patients with the disease of interest not included in the study"</td></tr><tr><td>Palys 2013</td><td>CONSORT</td><td>"The Chalmers scale [Chalmers, Control Clin Trials 1981] calls for the log of patients screened for the trial but not randomized. This key element of trial quality, so fundamental to understanding the success of randomization, masking, and allocation concealment (3) is not an element of CONSORT"</td></tr><tr><td>Patterson 2010</td><td>CONSORT</td><td>"In an attempt to quantify 'volunteer' bias, the widely endorsed CONSORT statement [...] encourages use of a flowchart depicting the number assessed for eligibility who declined to participate. [...] Important as this information is, it sheds little light on processes preceding assessment of eligibility, potentially masking 'preselection' bias"</td></tr><tr><td>Boers 2010</td><td>CONSORT</td><td>"In my view, summary tables and graphs are essential in conveying the principal messages of a study, so it is a pity that nothing of these results and considerations have made it into the CONSORT guidelines"</td></tr><tr><td>Results, comment related to outcomes and estimation</td><td>Tfelt-Hansen 2018</td><td>CONSORT</td><td>"[...] some of the recommendations and debated points in the CONSORT statement are irrelevant. For example, withdrawal from an RCT due to AEs [adverse events] is a highly relevant parameter for prophylactic drugs tested for treating migraines, but not for the acute-administration drugs, which are typically administered as a single dose"</td></tr></tbody></table>

(Continued)
<table><caption>**Table 4. Continued**</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Document ID</th><th>Comment primarily intended for</th><th>Quote</th></tr></thead><tbody><tr><td>Ethics and dissemination, comment related to consent or assent</td><td>Kremer 2013</td><td>SPIRIT</td><td>"In your guidance [of SPIRIT] you are presenting an example schedule requiring a written informed consent as the second-earliest action. [...] I have difficulties in understanding why a written consent on participation in a randomized clinical trial should be obtained just during an initial visit. In fact, many patients (and maybe physicians) would refuse such coercion. I think that patients insisting on time for consideration are the reason for many violations in the dates when consent was in fact obtained. Auditors and inspectors will criticize "wrong" dates as violation of the protocol. But why should the protocol always require written consent at that point? So, please explain how to come to a written informed consent at the initial visit without coercion or undue influence?"</td></tr><tr><td rowspan="10">Generic comments on challenges of SPIRIT or CONSORT</td><td>Barnard 2015</td><td>CONSORT</td><td>"Furthermore, the checklist may not be sufficiently explicit. [...] The CONSORT "Explanation and Elaboration" document provides this information [explicit recommendations for reporting the intervention], but authors may be overwhelmed by reading this 28-page document mixing guidance on why each item is important and should be reported, statistics of inadequate reporting, and guidance on how to report it with examples of adequate reporting"</td></tr><tr><td>Barnes 2015</td><td>CONSORT</td><td>"The Explanation and Elaboration documents are meant to help authors understand the checklist. However, these documents are very long (more than 30 pages) and they combine explanations about why the item should be reported, how the item is reported in the literature, what should be reported, and examples of adequate reporting. Consequently, the important information is buried in the manuscript. The use of a template shell with a clear and explicit reminder of what should be reported when authors are writing their manuscript could be useful to increase adherence to the guidelines"</td></tr><tr><td>Blanco 2018</td><td>CONSORT</td><td>"[...] it is possible that authors are not attentive to the requirements of CONSORT or, despite their efforts to be compliant with the requirements, they are struggling to interpret certain items or the level of detail that is required"</td></tr><tr><td>Dijkers 2015</td><td>CONSORT</td><td>"The list of items in CONSORT [...] is insufficient to guide complete reporting, especially for nondrug research"</td></tr><tr><td>Du 2021</td><td>CONSORT</td><td>"It also helps to make the current CONSORT statement more user friendly"</td></tr><tr><td>Ghosn 2019</td><td>CONSORT</td><td>"Updates of the main CONSORT are not planned with a concomitant update of existing extensions. The delay in updating extensions can be long. Consequently, an extension not based on the updated CONSORT checklist can be confusing and difficult to use because of inconsistencies in the numbering, content, and wording of items"</td></tr><tr><td>Jones 2017</td><td>CONSORT</td><td>"The limitations of CONSORT must also be considered: for example, two studies may score similarly in CONSORT even if one shows markedly better outcomes than the other, as CONSORT does not evaluate the magnitude of reported outcomes. Given these inconsistencies, does the CONSORT statement have any clinical relevance?"</td></tr><tr><td>Palys 2013</td><td>CONSORT</td><td>"[...] CONSORT is not sufficient for a valid trial or for good reporting. In fact, when used as a checklist for trial quality, CONSORT is essentially a subset of the more complete Chalmers scale [Chalmers, Control Clin Trials 1981]"</td></tr><tr><td>Rademaker 2020</td><td>CONSORT</td><td>"It is debated that CONSORT does not include all items to properly assess trial quality, such as the lack of a randomization log, the lack of precision in the intention to treat (ITT) description, and the lack of adequate handling of missing data"</td></tr></tbody></table>

(Continued)
<table><caption>**Table 4. Continued**</caption><thead><tr><th>SPIRIT or CONSORT section and topic</th><th>Document ID</th><th>Comment primarily intended for</th><th>Quote</th></tr></thead><tbody><tr><td colspan="4">Comments on strengths</td></tr><tr><td>Title and abstract, comment related to abstract</td><td>Dex 2010</td><td>CONSORT</td><td>"I am particularly gladdened by the addition of item 1B [in CONSORT] requiring a structured summary in line with the CONSORT abstracts recommendation"</td></tr><tr><td>Methods: Participants, interventions, and outcomes, comment related to sample size</td><td>Ruan 2022</td><td>CONSORT</td><td>"Scientific and reasonable estimation of sample size recommended by CONSORT can avoid false negative results owing to minimized sample size, and waste of resources induced by excessive sample size"</td></tr><tr><td>Other information, comment related to registration</td><td>Dex 2010</td><td>CONSORT</td><td>"However, the most important additions to the new 2010 CONSORT statement are the requirement for prospective trial registration and the making available to the editorial panel of the trial protocol. [...] Items 23 and 24 [in CONSORT, related to registration and protocol] can only improve the quality of not only trial conduct but also trial reporting"</td></tr></tbody></table>

CONSORT 2010 and new content was suggested to the sections on intervention, blinding, participant flow, and statistical methods. The issues raised may provide context to authors, peer reviewers, editors, and readers of trials using SPIRIT 2013 and CONSORT 2010 and inform the planned updates of the guidelines.

## Acknowledgments

We thank Benjamin Speich, Chenchen Xu, Ariel Barrientos, and Evan Tang for assisting in assessing search records on a related project aimed at identifying empirical literature on SPIRIT 2013 and CONSORT 2010. A few of the studies identified through this project were assessed for inclusion in this scoping review.

## Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.jclinepi.2023.01.003.

## References

[1] Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gotzsche PC, Krleza-Jeric K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 2013;158:200–7.

[2] Jones G, Abbasi K. Trial protocols at the BMJ. BMJ 2004;329:1360.

[3] Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c332.

[4] Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet 2005;365:1159–62.

[5] Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457–65.

[6] Hrobjartsson A, Pildal J, Chan AW, Haahr MT, Altman DG, Gotzsche PC. Reporting on blinding in trial protocols and

corresponding publications was often inadequate but rarely contradictory. J Clin Epidemiol 2009;62:967–73.

[7] Chan AW, Hrobjartsson A, Jorgensen KJ, Gotzsche PC, Altman DG. Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ 2008;337:a2299.

[8] Pildal J, Chan AW, Hrobjartsson A, Forfang E, Altman DG, Gotzsche PC. Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study. BMJ 2005; 330:1049.

[9] CONSORT Statement. Endorsers - journals and organizations. Available at http://www.consort-statement.org/about-consort/endorsements. Accessed February 7, 2023.

[10] SPIRIT Statement. Endorsement. Available at https://www.spirit-statement.org/about-spirit/spirit-endorsement/. Accessed February 7, 2023.

[11] Hopewell S, Boutron I, Chan AW, Collins GS, de Beyer JA, Hróbjartsson A, et al. An update to SPIRIT and CONSORT reporting guidelines to enhance transparency in randomized trials. Nat Med 2022;28:1740–3.

[12] Laursen DRT, Paludan-Müller AS, Hróbjartsson A. Randomized clinical trials with run-in periods: frequency, characteristics and reporting. Clin Epidemiol 2019;11:169–84.

[13] duVaure CB, Boutron I, Perrodeau E, Ravaud P. Reporting funding source or conflict of interest in abstracts of randomized controlled trials, no evidence of a large impact on general practitioners' confidence in conclusions, a three-arm randomized controlled trial. BMC Med 2014;12:69.

[14] Nejstgaard CH, Boutron I, Chan A-W, Hopewell S, Moher D, Schulz KF, et al. The evolving context of reporting randomised trials. A scoping review of comments on SPIRIT 2013 and CONSORT 2010. Available from OSF. 2022. https://doi.org/10.17605/OSF.IO/YB98V.

[15] Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018;169:467–73.

[16] Meghelli L, Narducci F, Mariette C, Piessen G, Vanseymortier M, Leblanc E, et al. Reporting adverse events in cancer surgery randomized trials: a systematic review of published trials in oesophago-gastric and gynecological cancer patients. Crit Rev Oncol Hematol 2016;104:108–14.

[17] Chen W, Yu J, Zhang L, Su G, Wang W, Kwong J, et al. Quality of reporting in randomized controlled trials of therapeutic cardiovascular medical devices. Surgery 2019;165:965–9.

[18] Jørgensen L, Paludan-Muller AS, Laursen DRT, Savović J, Boutron I, Sterne JAC, et al. Evaluation of the Cochrane tool for assessing risk
of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews. Syst Rev 2016;5:80.

[19] McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol 2016;75:40–6.

[20] Equator Network. Enhancing the QUALity and transparency of health research. Available at https://www.equator-network.org/category/blog/. Accessed February 7, 2023.

[21] BMC. Blog Network. Available at https://blogs.biomedcentral.com/. Accessed February 7, 2023.

[22] BMJ. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials - all rapid responses. Available at https://www.bmj.com/content/340/bmj.c332/rapid-responses. Accessed February 7, 2023.

[23] Community C. Cochrane colloquium abstracts. Available at https://abstracts.cochrane.org/. Accessed February 7, 2023.

[24] Sweetman EA, Doig GS. Failure to report protocol violations in clinical trials: a threat to internal validity? Trials 2011;12:214.

[25] Zhang S, Liang F, Li W, Hu X. Subgroup analyses in reporting of phase III clinical trials in solid tumors. J Clin Oncol 2015;33:1697–702.

[26] Schulz KF, Altman DG, Moher D, Fergusson D. CONSORT 2010 changes and testing blindness in RCTs. Lancet 2010;375:1144–6.

[27] Siddiq H, Pentapati KC, Acharya S. Adherence of randomized controlled trials to consolidated standards of reporting trials 2010 guidelines: a survey of randomized controlled trials published in 2011-2016 in 3 periodontology journals. J Evid Based Dent Pract 2019;19:260–72.

[28] Barbui C. Sharing all types of clinical data and harmonizing journal standards. BMC Med 2016;14:63.

[29] Smith CT, Hopkins C, Sydes MR, Woolfall K, Clarke M, Murray G, et al. How should individual participant data (IPD) from publicly funded clinical trials be shared? BMC Med 2015;13:298.

[30] Owyang D, Bakhsh A, Brewer D, Boughton OR, Cobb JP. Patient and public involvement within orthopaedic research: a systematic review. J Bone Joint Surg Am 2021;103:e51.

[31] Vassar M, Jellison S, Wendelbo H, Wayant C, Gray H, Bibens M. Using the CONSORT statement to evaluate the completeness of reporting of addiction randomised trials: a cross-sectional review. BMJ Open 2019;9:e032024.

[32] Yin Y, Shi F, Zhang Y, Zhang X, Ye J, Zhang J. Evaluation of reporting quality of randomized controlled trials in patients with

COVID-19 using the CONSORT statement. PLoS One 2021;16:e0257093.

[33] Mozetic V, Leonel L, Leite Pacheco R, de Oliveira Cruz Latorraca C, Guimarães T, Logullo P, et al. Reporting quality and adherence of randomized controlled trials about statins and/or fibrates for diabetic retinopathy to the CONSORT checklist. Trials 2019;20:729.

[34] Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 2012;11:MR000030.

[35] Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors' implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ 2012;344:e4178.

[36] Chauvin A, Ravaud P, Moher D, Schriger D, Hopewell S, Shanahan D, et al. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med 2019;17:205.

[37] Alemayehu D, Hemmings R, Natarajan K, Roychoudhury S. Perspectives on virtual (remote) clinical trials as the “new normal” to accelerate drug development. Clin Pharmacol Ther 2022;111:373–81.

[38] Sharma NS. Patient centric approach for clinical trials: current trend and new opportunities. Perspect Clin Res 2015;6:134–8.

[39] Park JJH, Siden E, Zoratti MJ, Dron L, Harari O, Singer J, et al. Systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols. Trials 2019;20:572.

[40] Strzebonska K, Waligora M. Umbrella and basket trials in oncology: ethical challenges. BMC Med Ethics 2019;20:58.

[41] Gabriel SE, Normand SL. Getting the methods right - the foundation of patient-centered outcomes research. N Engl J Med 2012;367:787–90.

[42] International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. Available at https://www.icmje.org/icmje-recommendations.pdf. Accessed February 7, 2023.

[43] Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med 2010;7:e1000217.