Lost Evidence From Registered Large Long-Unpublished Randomized Controlled Trials: A Survey

Background: Empirical studies have demonstrated that many randomized controlled trials (RCTs) remain unpublished (13). Besides publication in journals, trial results also may be disseminated by posting on ClinicalTrials.gov, a public, searchable registry. Loss of information is most worrisome when the largest and thus potentially most informative trials both remain unpublished and lack posted results long after completion. Objective: To assess the largest preregistered RCTs that were still both unpublished and lacked reported results at ClinicalTrials.gov more than 46 months after completion. We evaluated whether these trials subsequently were published or posted any results. These studies are an important part of the larger nonpublication problem that affects large and small, preregistered and non-preregistered trials. Methods and Findings: We searched ClinicalTrials.gov for studies with a start date after 1 June 2007 that were completed before 1 June 2012. From these trials, we identified those that were unpublished and did not have results posted on ClinicalTrials.gov as of 22 April 2016. We then selected the 500 largest trials in terms of participant enrollment. We excluded nonrandomized, cluster-randomized, and nonpreregistered trials. For eligible trials, we extracted ClinicalTrials.gov information on the interventions compared, condition or disease, actual sample size enrolled, industry funding, country, and contact person. We also used ClinicalTrials.gov to search for the contact person's e-mail address and recent publications. We examined whether any eligible RCTs were published or posted results at ClinicalTrials.gov between April 2016 and January 2019 (time points chosen for convenience and not for any specific rationale). Whenever possible, we verified nonpublication by e-mail communication with investigators. Two reminders were sent to each nonresponder. Of 146 large RCTs without main publications listed or results posted on ClinicalTrials.gov as of April 2016, 60 had in fact been published by that time (58 were retrieved from PubMed; for the other 2, investigators indicated publications with different National Clinical Trial [NCT] numbers). Two additional trials had duplicate ClinicalTrials.gov records, and 84 were indeed unreported (had neither published nor posted results) as of April 2016. Of these 84, only 14 (17%) were published within 32 months, by January 2019 (Figure). Searches at clinicalstudydatarequest.com yielded no data for the 70 unreported trials. Figure. Flow chart of eligible trials. * For 58 trials, we found published papers; for another 2, investigators responded that they had been published with different National Clinical Trial (NCT) numbers. Only 3 of these studies had results posted at ClinicalTrials.gov. Investigator e-mail addresses were found for 29 of the trials apparently unreported by January 2019. Of the 29 researchers contacted, 15 replied, 2 of whom responded (as mentioned earlier) that their RCT was published in 2011. We had missed these 2 reports because they referred to the ClinicalTrials.gov registered protocol with a different NCT identifier. Results of only 3 of the 70 long-unreported studies were posted on ClinicalTrials.gov as of January 2019. The remaining 67 long-unreported trials had a median enrollment of 765 participants (interquartile range, 618 to 1071 participants) and were unreported for a median of 9 years (interquartile range, 8 to 10 years) after completion. The total number of participants was 87883. Enrollment varied between 511 and 11000 participants. Most trials (58 of 70 [83%]) recruited adults, 11 enrolled a mixed population, and 1 included only children. Fifty-five trials (79%) evaluated pharmacologic interventions. Major disease categories included cardiovascular disease (n= 13), infection (n= 12), psychiatric diseases (n= 9), women's health and obstetrics (n= 6), allergy (n= 5), and neurologic diseases (n= 5). Thirty-nine trials (55%) were conducted in the United States, 4 in China, 3 in France, 3 in the United Kingdom, and 21 elsewhere. With 1 exception, all trials with results published from 2016 to 2019 were completed in 2011 to 2012, not earlier (Table). We also indicatively investigated whether among 10 trials randomly selected from the 70 (on NicVAX [Nabi Biopharmaceuticals], WC3011 vaginal gel, tonabersat, Dimebon [Medivation], eplivanserin, naloxone, PYY3-3, FAB-6011, and A/Anhui/05), any larger trials evaluating the same treatment comparison for the same disease or condition were registered and listed as completed at ClinicalTrials.gov. We found larger studies for only 1 of our randomly selected trials (on eplivanserin for insomnia), which had negative results and led to a licensing application withdrawal. Table. Publication of Trials Unreported Until April 2016 Discussion: Large trials that are unreported for almost 4 or more years after completion are unlikely to be published later or to post results on ClinicalTrials.gov. The loss of evidence from these trials pertained to almost 90000 participants. Efforts are still needed to ensure that all completed large trials are reported, including incentives and rewards for publication and transparency (4) or penalties for nonpublication. The unreported trials we identified typically were the largest to assess the specific treatment comparison for the specific disease or condition. In fields with several available treatments, unreported large trials may represent a substantial share of the evidence. For example, we found 5 long-unreported large trials on antidepressants. The largest meta-analysis on the same question found data on only 32 trials of equal size (5). Our study had limitations. We used only 1 registry. Furthermore, we could not determine the exact proportion of the total evidence represented by these unreported trials. Large unreported trials threaten the credibility of the available, published evidence. One should be cautious in combining published trials and interpreting them when extensive results remain undisclosed.