Data extraction error in pharmaceutical versus non-pharmaceutical interventions for evidence synthesis: Study protocol for a crossover trial
Authors: Zhu Yi , Ren Pengwei , Doi Suhail A.R. , Furuya-Kanamori Luis , Lin Lifeng , Zhou Xiaoqin , Tao Fangbiao , Xu Chang 
Published: September 2022

Background

Data extraction is the foundation for research synthesis evidence, and it is often time- and labor-consuming, and prone to errors. Whilst data extraction errors frequently occur in the literature, an interesting phenomenon was observed that data extraction error tend to be more common in trials of pharmaceutical interventions compared to non-pharmaceutical ones. This phenomenon has not been verified by high-quality evidence, the elucidation of which would have implications for guidelines, practice, and policy.

Methods and analyses

We propose a crossover, multicenter, investigator-blinded, trial to elucidate the potential variants on the data extraction error rates of meta-analyses with pharmaceutical against non-pharmaceutical interventions. Eligible 90 participants would be 2ndyear or above post-graduate students (e.g., masters, doctoral program). Participants will be randomized to one of the two groups to complete pre-defined data extraction tasks: 1) group A will contain 10 randomized controlled trials (RCTs) of pharmaceutical interventions; 2) group B will contain 10 RCTs of non-pharmaceutical interventions. Participants would then be assigned to the alternative group for another round of data extraction, after a 30 mins washout period. Finally, those participants assigned to A or B group will be further 1:1 randomly matched based on a random-sequenced number, for the double-checking process on the extracted data. The primary outcome will be the data extract error rates of pharmaceutical intervention group and non-pharmaceutical group,beforethe double-checking process, in terms of the cell level, study level, and participant level. The secondary outcome will be the data error error rates of pharmaceutical intervention group and non-pharmaceutical group,afterthe double-checking process, again, in terms of the cell level, study level, and participant level. Generalized linear mixed effects model (based on the above three levels) will be used to estimate the potential differences in the error rates, with a log link function for binomial data. Subgroup analyses will account for the following factors i.e., the experience of individuals on systematic reviews, and the time used for the data extraction.

Ethics and dissemination

This study has been approved by the institutional review board of Anhui Medical University (No.83220405). Findings of the study will be presented at international scientific meetings, and publish in peer-reviewed academic journal.

Trial registration

Chinese Clinical Trial Register Center (Identifier: ChiCTR2200062206).

Strengths and limitations of the study

This will be the first trial to compare data extraction error rates in pharmaceutical intervention and non-pharmaceutical intervention studies for research synthesis.

This will be the third randomized trial on the strategy of data extraction in the world and the first in the Asia-Pacific region.

The use of a crossover design provides a valid way to reduce the potential impact of heterogeneous contexts of the studies and thus is expected to provide robust evidence to support better evidence synthesis practice.

We will restrict the participants to 2ndyear post-graduate students or above to ensure the feasibility of the trial; this restriction will no doubt impact the representativeness of the samples.

A group of useful strategies (eg. use U disk and isolate signal etc.) should be taken to minimize the impact of the possible sharing of the completed extraction table among the participants.