dataesr/scientific-paragraphs-categorization huggingface dataset:
A further step was achieved in 2007 with the US Food and Drug Administration Amendments Act (FDAAA 801) requiring phase 2-4 trials of FDAapproved drugs, devices, or biologics to post results on the federally funded registry ClinicalTrials. gov within 1 year of completion  [16, 17]. According to the law, "a table of values for each of the primary and secondary outcome measures for each arm of the clinical trial, including the results of scientifically appropriate tests of the statistical significance of such outcome measures" should be posted  [18]. However, it seems that many trials with results posted fail to report a an estimate of treatment effect or p-value. In this study, we aimed to assess how many superiority clinical trials with results posted at ClinicalTrials. gov report statistically significant results and whether the proportion of significant results differed for trials with no treatment effect estimate or p-value posted.
Extraction of data from ClinicalTrials. gov The following characteristics were extracted from the records downloaded from ClinicalTrials. gov: 1. General characteristics: we collected the phase of the trial (phase 3 or 4), type of intervention assessed (e. g. , drug, biological, or device), type of control (i. e. , placebo, no treatment, or active control), sponsor, and collaborators. We considered that there was an industry sponsorship if the sponsor or one of the collaborators was industry. We also extracted sample size and primary completion date (i. e. , date of final collection for the primary outcome). 2. Location: we collected the countries where the trial was conducted and whether the trial was conducted in a single country and the number of centers involved.
Results posted: we collected the date when results were first received and whether the study was likely subject to the FDAAA. This characteristic is based on an algorithm developed by the US National Library of Medicine. Then, for the first primary outcome reported in the Results section, we collected whether a treatment effect estimate and/or p-value was reported and, if yes, we extracted them. Reporting of statistically significant results for the first primary outcome reported For each trial with a treatment effect estimate and/or p-value reported, we evaluated whether results reported were statistically significant or not. To do so, we relied on the p-value reported at ClinicalTrials. gov and considered p-values <0. 05 as statistically significant. Five percent has been the most commonly used threshold for statistical significance in clinical intervention research  [21].
Calculation of p-values for trials with no treatment effect estimate or p-value reported For trials with no treatment effect estimate or p-value reported, we calculated whenever possible the treatment effect estimate and corresponding p-value using the results per arm posted at ClinicalTrials. gov. This calculation was possible only for binary and continuous outcomes when sufficient data were reported at ClinicalTrials. gov. For binary outcomes, we calculated relative risk with 95% CI and the corresponding p-value from the number of events and number of participants analyzed reported per arm. For continuous outcomes, we calculated mean difference with 95% CI and the corresponding p-value from the mean and standard deviation (SD) or standard error (SE), which we transformed to SD as well as the number of participants analyzed reported per arm.
Extraction of results per arm from ClinicalTrials. gov and calculation of treatment effect estimate with 95% CI and the corresponding p-value was systematically performed by two reviewers working independently (JS and CR) using Revman 5. 3 (Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014). Then, all results were compared and any disagreement was resolved by consensus with the help of a senior researcher (AD) if needed. We also considered a p-value <0. 05 as statistically significant. Calculation of treatment effect estimate and corresponding p-value was not possible for censored outcomes (e. g. , progression-free survival), repeated measurements of the outcome over time (e. g. , change in mean bone mineral density from baseline assessed at 12, 24, and 36 months), other situations for which there were several observations or events per patient because these situations required individual patient data, or when there was no event in both groups.
Statistical analysis Categorical variables were described with frequencies (percentages) and quantitative variables with median (Q1-Q3). To assess the statistical significance of the results, we focused on p-values rather than on treatment effect estimates with 95% CI, and graphically represented the density plots of p-values for trials by whether a treatment effect estimate and/or p-value was reported or not. For this, we considered p-values <0. 0001 equal to 0. 0001. We aimed to identify trial characteristics associated with reporting a p-value <0. 05 for the first primary outcome at ClinicalTrials. gov. To do, so, we first compared trial characteristics by whether results were reported to be statistically significant or not using chi-square tests for categorical variables and Wilcoxon tests for continuous variables.
Then, we used a multivariate logistic regression model to identify factors independently associated with the reporting of statistically significant results. All variables that were statistically significant on univariate analyses were entered into multivariate analysis except single-center, which was closely related to singlecountry status. Statistical analysis involved use of SAS 9. 3 (SAS Inst. , Cary, NC, USA). The density plot was created with R 3. 2. 3 (R Foundation for Statistical Computing) using a syntax provided in a previous article  [23].
Identification of completed randomized trials with a superiority hypothesis Figure  1  shows the selection process. From the 192,175 studies registered at ClinicalTrials. gov on 2 June 2015, 17,536 had results posted, with 9640 (55%) likely to be subject to the FDAAA. We identified 2823 completed phase 3 or 4 randomized trials with a superiority hypothesis. Reporting of treatment effect estimate and/or p-value at ClinicalTrials. gov Among the 2823 eligible trials, 1400 (50%) had a treatment effect estimate and/or p-value reported at Clinical-Trials. gov, and 1423 trials (50%) had only results per arm reported. Characteristics of trials by whether a treatment effect estimate and/or p-value was reported or not are reported in Table  1. Briefly, as compared with trials with no treatment effect estimate or p-value reported, those with a treatment effect estimate and/or p-value reported were more likely to be phase 3 trials (68% versus 51%), to have an inactive comparator (59% versus 36%), to have an industry sponsorship (83% versus 74%), to involve several countries (41% versus 21%) and several centers (66% versus 49%), to be subject to the FDAAA (68% versus 59%), and to have a larger median sample size (270 versus 156).
Reporting of statistically significant results for the first primary outcome When a treatment effect estimate and/or p-value was reported (N = 1400 trials), results were reported as Fig. 1  Flow chart of the selection of trials. RCT randomized controlled trial, SD standard deviation statistically significant for 844 trials (60%), with a median p-value of 0. 01 (Q1-Q3: 0. 001-0. 26) (Fig. 2 ). For trials with no treatment effect estimate or p-value reported at ClinicalTrials. gov (N = 1423), we could calculate treatment effect estimate and the corresponding p-value for 929 (65%). From data posted at ClinicalTrials. gov, p-values could not be calculated for 494 trials (35%), mainly because of the reporting of percentages only for binary outcomes (n = 187), no reporting of mean (±SD) for continuous outcomes (n = 95), censored data (n = 61), and repeated measurements over time (n = 52).
Factors associated with reporting statistically significant results for the first primary outcome Table 2 compares trial characteristics by whether statistically significant results were reported or not. Trials with significant results reported were more likely to be phase 3 trials (74% versus 53%, p < 0. 0001), to have an inactive comparator (64% versus 40%, p < 0. 0001), to have an industry sponsorship (totally or partially) (88% versus 74%, p < 0. 0001), to be subject to the FDAAA (70% versus 61%, p < 0. 0001), and to have a larger sample size or to be a multicenter trial involving several countries than those reporting a p-value ≥0. 05 or not reporting treatment effect estimate or p-value. On adjusted multivariate analysis, reporting statistically significant results at ClinicalTrials.
Discussion Nearly half of the studies with results posted at Clini-calTrials. gov are not required to do so as they do not seem to be subject to the FDAAA. Nevertheless, more attention should be paid to having more complete and transparent results posted. Only half of the completed trials with results posted at ClinicalTrials. gov report a treatment effect estimate and/or p-value for the first primary outcome posted, with 60% of these trials reporting significant results. In contrast, the proportion of trials with significant results seemed much lower when no treatment effect estimate or p-value was reported. Factors independently associated with reporting statistically significant results at ClinicalTrials. gov were being a phase 3 trial, using an inactive comparator, having industry sponsorship, and involving multiple countries.
ClinicalTrials. gov is the most widely used clinical trial registry worldwide, with studies registered from 190 countries, and for now it is the only one allowing standardized posting of results  [17, 18]. Clinical. Trials. gov represents a crucial source of information on trial results. Previous studies showed that it allows access to results not yet published and to more complete results than in corresponding published articles  [24] [25] [26]. A recent study has compared results posted at Clinical-Trials. gov to corresponding FDA reports for new drug approval trials and found large concordance between both sources  [27]. In this study, we found that results of many trials are posted although not required, because nearly half of the trials with results posted did not seem to be subject to the FDAAA according to the algorithm developed by the US National Library of Medicine.
e. , for analysis of censored outcomes or repeated measurements over time). Therefore, for about one third of trials, systematic reviewers may be unable to use the results reported at ClinicalTrials. gov when treatment effect estimates or p-values are not reported. Our results also suggest that the proportion of trials with significant results differs between trials reporting or not a treatment effect estimate and/or p-value. Although we could not calculate treatment effect and corresponding p-values for all trials not reporting these, our results suggest a much lower proportion of trials with significant results when no treatment effect estimate or p-value is reported. These results may reflect a selective posting of treatment effect estimate and/or p-value when results are statistically significant.
This lack of transparency may have consequences when interpreting the results posted at ClinicalTrials. gov because physicians and decision-makers may be more likely to rely on trials with p-values already reported. We identified some factors associated with reporting significant results at ClinicalTrials. gov. Reporting of significant results for the first primary outcome posted were more frequent for trials sponsored by industry than academic sources, which is consistent with previous studies finding that industry trials are more likely than public trials to report significant results in published articles  [32, 33]  and at ClinicalTrials. gov [31]. In addition, reporting of significant results was also more common for phase 3 trials and for trials with an inactive control. This finding is not surprising for trials with inactive control treatment because treatment effect estimates are expected to be higher in this situation than when the experimental intervention is compared to an active treatment.
This study has important implications. It highlights the importance of having complete results posted, including the posting of treatment effect estimate and/or p-value, to avoid any misinterpretation about the benefits of interventions. Because of the poor compliance with these requirements, it becomes necessary to improve their implementation. Systematic checking of results posted and automatic mailing to responsible parties may help improve the completeness of results posted. A recent article showed that sending emails to responsible parties of completed trials that do not comply with the FDAAA legal requirement to post results significantly improved the posting rate at 6 months  [34]. In April 2014, the European Union voted to adopt the Clinical Trials Regulation, which requires the registration of all clinical trials conducted in Europe and the posting of trial summary results in the European Clinical trials Database (EudraCT) within 1 year after trial completion  [35, 36].
This is a crucial step toward more transparency and this initiative should take advantage of the body of literature available for ClinicalTrials. gov and compliance with the FDAAA during the implementation process. Our study has some limitations. We only looked at the first primary outcome with results posted, so our study is not representative of all results posted at Clini-calTrials. gov. For trials with no treatment effect estimate or p-value reported, we attempted to calculate the treatment effect and corresponding p-values from data reported per arm, adopting the viewpoint of systematic reviewers. However, we could not do this for about one third of trials with no treatment effect estimate or p-value reported because of insufficient elements reported or because the data were repeated measures or censored, situations for which individual participant data are necessary.
Conclusions Only half of completed trials with results posted at ClinicalTrials. gov had a treatment effect estimate and/ or p-value reported for the first primary outcome, with significant results in 60% of these. The proportion of trials with significant results seemed much lower when no treatment effect estimate or p-value is reported, which may suggest a selective posting of treatment effect estimate and/or p-value when results are statistically significant. More efforts should be expended to improve the transparency in posting results at ClinicalTrials. gov. Data are presented as n (%) unless indicated. FDAAA US Food and Drug Administration Amendments Act, Q1-Q3: Quartile 1-Quartile 3.
Acknowledgements We thank  Elise Diard  ( Centre de Recherche Epidémiologie et Statistique, Inserm U1153, French Cochrane Center ) and  Elodie Perrodeau ,  M. Sc. ( Centre de Recherche Epidémiologie et Statistique, Inserm U1153, French Cochrane Center ) for their help with the figure. ED and EP did not receive any compensation for their role in the study.
0001 Academic only 100 (12) 506 (26) Countries Single country 361 (43) 1267 (64) <0. 0001 Multiple countries 353 (42) 507 (26) Not reported 130 (15) 205 (10) Location At least one site in the USA 493 (59) 1175 (59) 0. 003 No site in the USA 221 (26) 599 (30) Not reported 130 (15) 205 (10) Subject to the FDAAA Yes 588 (70) 1206 (61) <0. 0001 No 201 (24) 732 (37) Not reported 55 (6) 41 (2) Centers Multicenter 581 (69) 1044 (53) <0. 0001 Single center 133 (16) 730 (37) Not reported 130 (15) 205 (10) Sample size Median (Q1-Q3) 305 (139-541) 179 (62-405) <0. 0001 Time to results first received at Median (Q1-Q3) 13. 0 (11. 6-26. 3) 16. 5 (12. 0-30. 9) <0. 0001 ClinicalTrials. gov (month) Data are presented as n (%) unless indicated. FDAAA US Food and Drug Administration Amendments Act, Q1-Q3: Quartile 1-Quartile 3.
Background: Estimates of life expectancy are a key input to cost-effectiveness analysis (CEA) models for cancer treatments. Due to the limited follow-up in Randomized Controlled Trials (RCTs), parametric models are frequently used to extrapolate survival outcomes beyond the RCT period. However, different parametric models that fit the RCT data equally well may generate highly divergent predictions of treatment-related gain in life expectancy. Here, we investigate the use of information external to the RCT data to inform model choice and estimation of life expectancy. Methods: We used Bayesian multi-parameter evidence synthesis to combine the RCT data with external information on general population survival, conditional survival from cancer registry databases, and expert opinion. We illustrate with a 5-year follow-up RCT of cetuximab plus radiotherapy v.
radiotherapy alone for head and neck cancer. Results: Standard survival time distributions were insufficiently flexible to simultaneously fit both the RCT data and external data on general population survival. Using spline models, we were able to estimate a model that was consistent with the trial data and all external data. A model integrating all sources achieved an adequate fit and predicted a 4. 7-month (95% CrL: 0. 4; 9. 1) gain in life expectancy due to cetuximab. Conclusions: Long-term extrapolation using parametric models based on RCT data alone is highly unreliable and these models are unlikely to be consistent with external data. External data can be integrated with RCT data using spline models to enable long-term extrapolation. Conditional survival data could be used for many cancers and general population survival may have a role in other conditions.
3  Parametric distributions differ in their flexibility: the exponential requires a constant hazard, the Weibull has monotonically increasing or decreasing hazards, while the log-normal and log-logistic allow for ''bowl'' or ''hat'' shaped hazards. 1  Sometimes, the proportional hazard assumption is relaxed by fitting an accelerated failure time (AFT) model, or by fitting unrelated models to each arm, also known as Fitted Separately to Each Arm (FSEA). 3 Model selection is not straightforward, and models that fit equally well to the observed RCT data may give very different estimated mean survival gains. 6 This is due to mean survival being very sensitive to the tails of the survival distribution, which are usually not captured within RCT follow-up periods. There is a well-recognized need to improve extrapolation of survival data for use in the context of cost-effectiveness analysis  7  and there have been several attempts to do this using data external to the trial.
We consider two types of external information to inform extrapolation of the control arm: population data on overall survival of an age-and sex-matched cohort, and cancer registry information on conditional survival of a matched cohort of head and neck cancer patients. A third external source is information on the relative treatment effect, derived from an analysis of the literature and treatment mechanisms. We begin with a description of the illustrative dataset and, to motivate the paper, we compare the performance of a series of standard parametric survival models on goodness of fit and predicted gain in life expectancy. We then briefly review what is known about the clinical epidemiology of head and neck cancer and the mechanisms underlying treatment with radiotherapy and cetuximab, and provide some preliminary analyses of data from cancer registries and recent meta-analyses, which justify our use of conditional survival data and the external information on treatment effect.
MOTIVATING EXAMPLE Bonner and others  22  conducted an RCT to compare radiotherapy plus cetuximab v. radiotherapy alone, with overall survival recorded over a 5-year follow-up period. We have used a method  23  that accurately reconstructs the life-table data from the Kaplan-Meier curves published in that paper. These data are used throughout the paper and are shown in the Web Appendix. In the analyses that follow, we consider how to extrapolate both arms in order to inform a potential CEA of cetuximab, aimed at a target population represented by the Bonner trial. The trial was carried out between 1999 and 2002 on treatment-naı ¨ve patients in several countries. The patient sample was 80% male with median age of 57 years, range 34 to 83 years. The most common site of primary tumors was oropharynx (59.
33, 34   ccording to the above registry studies, this excess mortality persists for at least 15 years. To confirm the relevance of these results in the present context, we constructed a cohort of cancer patients matched to the Bonner trial population for For each parameter, or log parameter if the parameter was required by definition to be positive, the prior was assumed to follow a normal distribution with a mean of 0 and a variance of 1000. age, gender, cancer site, and date of diagnosis  35  using registry data from the Surveillance Epidemiology and End Results (SEER) database. 36 ne-year conditional survival in this cohort is shown alongside conditional survival in the trial (Figure  2A ). The results accord closely with previous work reported above, and show the close agreement of trial and SEER data.
Boundary knots were placed at the extreme ends of the data. We also specified one internal knot in the RCT segment of the data, and another internal knot in the period between the end of the trial (5 years) and the end of the external data (40 years). The knots were placed in the mid-point on the log-time scale. External information was added incrementally: data on general population and the SEER database for the control arm were first included separately, then together, and finally information was also added on the hazard ratio. External information was introduced by specifying the relationship between the parameters estimated by the external information, and the parameters of the survival distributions, whether these were parametric or spline functions. This was implemented by writing the likelihoods for the external data in terms of the parameters of the (extrapolated) survival model that also gives the likelihood for the trial data.
Estimation Estimation was carried out by Bayesian Markov Chain Monte Carlo (MCMC) simulation using WinBUGS  42  and WBDev. 43 Observed and fitted survival, conditional survival, and relative hazard ratios were visually compared. D and DIC statistics were computed within the MCMC simulation and transformed using a constant to the scale of the parametric models to allow comparisons. 10 These statistics were recorded, separately for the RCT data and each type of external data. This allowed us to compare the model fit to the different data types. The threshold for choosing one model over another was a 5-point difference in the DIC  27  on the RCT data. Details regarding choice of initial values,  56  convergence checks, burn-in period, and posterior sampling are given in Web Appendix D.
Introducing External Data from SEER on Conditional Survival We assumed that the RCT control arm population 1-year conditional survival at time t conditional on being alive at time (t-1), CS 0,RCT (tjt À 1), is no different to the matched SEER population conditional survival, CS SEER (tjt À 1), from 6 years onwards until 26 years (last time point available in the SEER database). The methods for calculating the numbers of person alive and at risk in the SEER population between time t and t -1, respectively r SEER (tjt À 1) and n SEER (tjt À 1), are shown in the Web Appendix B, along with the data itself. Assuming a binomial likelihood for 1-year conditional survival probabilities from the SEER population, we implemented this by specifying: r SEER (t t À 1 j );Binomial(CS SEER (t t À 1 j ),n SEER (t t À 1 j )) where CS SEER (t t À 1 j ) is constrained to be equal to CS 0,RCT (t t À 1 j ) so that CS SEER (t t À 1 j ) 5 CS 0,RCT (t t À 1 j ), 6 t 26 years ð2Þ Introducing Both General Population Survival Data and Conditional Survival Data When adding the SEER data on conditional survival (Equation [2]), we observed that the 1-year conditional estimated survival curve for the RCT population crossed that from the general population at 33 years.