Evaluating The Impact and Reporting of Attrition in Randomized Controlled Trials: A Systematic Review of The Top Medical Journals
Creators: name: Haq, M
name: Grolleau, Francois-Camille
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Medicine and Health Sciences

description: Background: “Intention to treat” (ITT) is a well-defined strategy for the analysis of randomized controlled trials (RCTs) that compares patients in the groups to which they were originally randomly assigned.1 Specifically, ITT corresponds to the inclusion of all patients in the group they were originally randomized to, regardless of whether they satisfied the inclusion criteria, the treatment received, and subsequent withdrawal or deviation from the protocol. In contrast, attrition in the analysis is defined as the exclusion of any participants who were randomly assigned.2 Attrition, therefore, constitutes a specific type of violation of the ITT principle. As formalized in the literature of causal inference,3 attrition can weaken the internal validity and/or the external validity of RCTs conclusions depending on the reason for exclusion of participants from the analysis: If both patients’ characteristics and the randomization group determine exclusion from the analysis, then RCTs’ internal and external validity decrease concomitantly. If only patients’ characteristics—but not the randomization group—determine exclusion from the analysis, then RCTs’ external validity decreases while internal validity withstands. In other words, RCTs’ conclusions hold but for a different population than that originally envisioned. If neither patients’ characteristics nor the randomization group determine exclusion from the analysis (i.e., missingness from the analysis is completely at random), then the estimated treatment effect is more variable as the sample size decreases but internal and external validity are unaffected. In the medical literature, however, the terminology “attrition bias” does not distinguish between these three mechanisms.4 Objectives: The aims of this study are to assess, in 1:1 two parallel arms RCTs published in the last six month in the top four medical journals: 1.The distribution of absolute and relative attrition levels 2.The reporting quality of attrition in RCTs 3. The evidence for attrition bias and its correlation (or the lack of) with attrition levels. METHODS: We will register the protocol OSF(The Open Science Framework) as per guidelines. We will conduct and report this study following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.6 Sources of Information and Search Strategy: We will search MEDLINE from September 1, 2021, to March 1, 2022, to identify all RCTs published in the following four major general journals—The New England Journal of Medicine, The Lancet, JAMA, The British Medical Journal. We will use medical subject headings, in various combinations, supplemented with free text to increase sensitivity and the Cochrane sensitivity and precision-maximizing search strategy for identifying RCTs. Eligibility Criteria: Eligible studies will be the superiority trials that randomized patients 1:1 into two parallel arms and had a statistically significant result on a binary primary outcome. We will exclude cluster randomized trials and trials with time-to-event primary outcomes because 1) the statistical issues for these are different to those in individually randomized and for non-time-to-event outcomes and 2) we want to compare our results to a previous review.4 We will also exclude cross-over, and multi-arm (&gt; 2 arms) trials, pilot and feasibility studies, letters, commentaries, corrections, and papers that were not primary reports of clinical trials. Study Selection: One reviewer (AH) will screen the titles and abstracts and review the full text of all potentially eligible citations using the freely available software Rayyan. Fifteen percent of the studies will then be randomly selected for independent assessment by a second reviewer (FG). Using a pilot-tested Microsoft Excel spreadsheet one reviewer (AH) will extract data from the full texts of eligible studies. Specifically, for each eligible RCT, AH will extract the following data: 1.The total number of patients randomized in each arm. 2.The total number of patients from each arm included in the primary analysis. 3.The number of events and nonevents for each arm 4.In each arm, the number of patients excluded from the primary analysis because of i) withdrawals, ii) dropouts, iii) protocol violations. 5.The authors’ claim for their primary analysis—categorized as i) ITT ii) modified ITT, iii) no claim, iv) other claims. 6.Whether adjustment techniques were used for the primary analysis (e.g., multivariate analysis). 7.Did the trial included only COVID-19 patients 8.Was the trial a vaccine trial? Statistical Analysis: For descriptive statistics, we will report the median and interquartile range or mean and standard deviation of continuous variables as well as the number of occurrences with proportions represented as percentages for categorical variables. We will assess the evidence for attrition bias by testing the null hypothesis of equal attrition levels in the intervention and control arm via the Fisher exact test. We will plot the evidence of attrition bias (i.e., one minus the P-value) against the total proportion of attrition from both arms. We will use Pearson’s correlation coefficient to evaluate the correlation between evidence for attrition bias and the total proportion of attrition from both arms. All analyses will be performed using R or Python. All tests of significance will be two-tailed, and P-value of less than 0.05 will be considered significant. REFERENCES 1. Hollis, S. &amp; Campbell, F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ 319, 670–674 (1999). 2. Nunan, D., Aronson, J. &amp; Bankhead, C. Catalogue of bias: attrition bias. BMJ Evid Based Med 23, 21–22 (2018). 3. Mansournia, M. A., Higgins, J. P. T., Sterne, J. A. C. &amp; Hernán, M. A. Biases in Randomized Trials: A Conversation Between Trialists and Epidemiologists. Epidemiology 28, 54–59 (2017). 4. Hewitt, C. E., Kumaravel, B., Dumville, J. C., Torgerson, D. J., &amp; Trial attrition study group. Assessing the impact of attrition in randomized controlled trials. J Clin Epidemiol 63, 1264–1270 (2010). 5. Dumville, J. C., Torgerson, D. J. &amp; Hewitt, C. E. Reporting attrition in randomised controlled trials. BMJ 332, 969–971 (2006). 6. Liberati, A. et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 339, b2700 (2009).