Clinical trial NCT03119376: Web-based Tool to Improve the Assessment of Reporting (COBPeer)
Official title: Evaluation of an Online Tool (COBPeer) Dedicated to Junior Peer Reviewers for Assessing the Reporting of Randomized Controlled Trial Reports
Study type: INTERVENTIONAL
Overall status: COMPLETED
Phase: NA
Conditions: Peer Review
Keywords: peer review
online tool
Randomized Controlled Trial
Lead sponsor: Assistance Publique - Hôpitaux de Paris
Sponsor class: OTHER
Collaborators: name: University of Oxford
name: University of Ottawa
name: University of California, Los Angeles
First posted: 2017-04-18
Enrollment: 120
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Allocation: NON_RANDOMIZED
Intervention model: PARALLEL
Primary purpose: OTHER

Brief summary: The peer review process is a cornerstone of biomedical research publication. Despite being essential, the assessment of the completeness of the reporting and the identification of switched outcomes are not appropriately performed. Furthermore, these tasks do not suppose a high level of expertise and could be performed by junior researchers.

To assess the completeness of reporting and identified switched outcome(s), junior peer reviewers could use a simple online tool (COBPeer) based on the CONSORT (Consolidated Standards of Reporting Trials) 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCTs. The tool would feature bullet points eliciting the meaning of each checklist item. The aim of this study that will compare the accuracy of junior peer reviewers using the tool to that of usual peer reviewers when evaluating the completeness of reporting and switched of primary outcome(s) in reports of RCTs

Detailed description: BACKGROUND: Context Inadequate reporting is a frequent cause of waste of research. To overcome this issue, the CONSORT statement, an evidence-based, minimum set of recommendations for reporting RCTs was developed in 1996. These guidelines have since been updated in 2001 and more recently in 2010. Many journals endorse the CONSORT statement. Some journals provide recommendations to authors to follow the CONSORT guidelines and some editors enforce the use of the CONSORT guidelines by requesting authors to submit a checklist in either the submission or acceptance stage. Nevertheless, inadequate reporting remains.

HYPOTHESE: To assess the completeness of reporting, junior peer reviewers could use a simple online tool based on the CONSORT 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCT

OBJECTIVE: 1) Develop an online tool and training module dedicated to junior peer reviewers for a) assessing the completeness of reporting of key items and b) identifying switched primary outcome(s) in reports of RCTs. 2) Compare the performance of junior peer reviewers using this tool with the usual peer-review process in identifying inadequate reporting and switched primary outcome(s) in reports of RCTs.

STUDY DESIGN: randomized ratio 1:1 controlled trial with two parallel arms. Participants will be invited to participate by invitation.

ELIGIBILITY: eligible participants are juniors peer reviewers. Junior peer reviewers are defined as early stage researchers: master students, PhD students, residents involved in clinical research during their study, and clinicians who have never reviewed a manuscript

INTERVENTION: the peer-review tool will be based on 10 items of the CONSORT Statement. This tool reminds the peer reviewer of the CONSORT item which must be reported by explicating it with a series of questions. The training module will explain each item to be evaluated with examples followed by a series of exercises from published randomized trial extracts.

MANUSCRIPT SELECTION: sample of randomized controlled trials published by Annals of Emergency Medicine, British Medical Journal, British Medical Journal Open and BioMed Central series medical.

EVALUATION BY THE JUNIOR PEER REVIEWERS: the junior peer reviewer who has accepted to participate and who has passed the training with success, will have to evaluate 1 article of the selected randomized sample.

EVALUATION OF THE USUAL PEER REVIEWERS: the evaluation by the usual peer-reviewers will be obtained by an analysis of the comments of the peer-reviewers which are available on line or obtained from the publisher.

GOLD STANDARD: peers of researchers who are experts in conducting systematic reviews will evaluate each article independently from other evaluations. Disagreements will be resolved by consensus.

PRIMARY OUTCOME: the mean number of items accurately classified per manuscript

SECONDARY OUTCOMES: the mean number of items accurately classified per manuscript for the 10 CONSORT items; the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).

SAMPLE SIZE: 120 manuscripts and 120 peer reviewers juniors.

Eligibility criteria: Inclusion Criteria:

* master students,
* PhD students,
* residents involved in clinical research during their study,
* clinicians who have never reviewed a manuscript

Exclusion Criteria:

* none

Primary outcomes:
measure: the mean number of items accurately classified per manuscript
timeFrame: through study completion, an average of 6 month
Secondary outcomes:
measure: the mean number of items accurately classified per manuscript for the 10 CONSORT items
timeFrame: through study completion, an average of 6 month
measure: the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).
timeFrame: through study completion, an average of 6 month
Other outcomes:
Locations:
city: Paris
country: France
facility: Hotel Dieu, 1, place du parvis de notre dame
Study officials:
References:
citation: Chauvin A, Ravaud P, Moher D, Schriger D, Hopewell S, Shanahan D, Alam S, Baron G, Regnaux JP, Crequit P, Martinez V, Riveros C, Le Cleach L, Recchioni A, Altman DG, Boutron I. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med. 2019 Nov 19;17(1):205. doi: 10.1186/s12916-019-1436-0.
citation: Chauvin A, Moher D, Altman D, Schriger DL, Alam S, Hopewell S, Shanahan DR, Recchioni A, Ravaud P, Boutron I. A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials. BMJ Open. 2017 Sep 15;7(9):e017462. doi: 10.1136/bmjopen-2017-017462.