dataesr/scientific-paragraphs-categorization huggingface dataset:
Search for relevant studies We searched ClinicalTrials. gov on April 14, 2014, by using "completed" for recruitment, "observational studies" for study type, and date of first registration between October 1, 2007, and December 31, 2011, and "has an outcome measure designated as a safety issue" in the safety issue field of ClinicalTrials. gov. We chose October 2007 because modifications were made at this time to the design-specific data elements used for registering observational studies on ClinicalTrials. gov. These changes were strongly influenced by protocol-related items in the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement  [13]. Observational studies are defined by ClinicalTrials. gov as studies where "investigators assess health outcomes in groups of participants according to a protocol or research plan.
Identification of relevant observational studies Among the studies retrieved, we identified all observational prospective studies assessing interventions that had a primary completion date (i. e. the date when the final patient was examined or received an intervention for the purposes of final collection of data for the primary outcome) between October 1, 2007, and December 31, 2011. We defined interventions as all pharmacologic and non-pharmacologic treatments (pharmaceutical drugs, surgery, education, rehabilitation, etc. ) aimed at improving the participants' health. We chose this time period to be able to investigate public availability (i. e. at least 24 months after the study primary completion). We excluded studies with a primary completion date after December 2011, studies assessing genetics, predictors, or risk factors; not assessing interventions, pharmacodynamics, and pharmacokinetics of drugs; phase 0, I, II, I/II and II/III studies; studies with healthy volunteers, and retrospective and randomized studies.
Extraction of data from ClinicalTrials. gov We downloaded from ClinicalTrials. gov the following data concerning the characteristics of the studies: clinical trial number (NCT), title, study design (defined with the observational model: cohort, case-control, case-only, case crossover or other; and time perspective: prospective, cross-sectional or other), enrollment (i. e. sample size), first received date, primary completion date, results first received date, condition and intervention under study, outcome measures, locations of recruitment and funding source. The funding source was classified at Clini-calTrials. gov as "NIH" (National Institutes of Health), "US federal", "industry", and other non-industry organizations (universities, hospitals, foundations, and other government and other non-industry organizations).
We secondarily categorized funding sources as non-industry (i. e. funded by NIH, US federal, other non-industry organizations) or industry (i. e. partially or totally funded by industry). One researcher classified the following information from the full ClinicalTrials. gov record: medical field, type of intervention (drug, device, procedure/surgery, other), location of recruitment (Europe, North America, South America, Africa, Asia, Oceania) and study purpose as safety, efficacy or both. As a quality assurance procedure, a second researcher independently verified 50 % of the data.
Publication of study results in journals For each observational study identified, we systematically searched for a publication reporting the study results (search date June 2014). First, we examined the "publication" field at ClinicalTrials. gov to search for a citation for an article that described the study results. If no citation was reported, we searched MEDLINE via PubMed by using the ClinicalTrials. gov identification number (NCT). If no publication was identified, we searched MEDLINE via PubMed by using keywords for the intervention under study and the condition. A researcher screened all citations retrieved up to the primary completion date registered at ClinicalTrials. gov and selected all citations corresponding to the selected study. A second researcher independently performed the search on PubMed for all the studies for which no publication was identified; any discrepancies were discussed until obtaining consensus.
Finally, if no publication was identified, we contacted the sponsor or the principal investigator. We searched in the "additional information" field of ClinicalTrials. gov for a link to the sponsor website for contacting the sponsor. If no link was available, we recorded the principal investigator's name from the "contacts and locations" fields of ClinicalTrials. gov and searched PubMed and Google to identify their email address. The email reminded the recipient of the study NCT number and inquired about the study publication, presentation at a congress, and plans for future publication (Additional file 1: Appendix 1). We systematically sent two reminders. If no answer was received, the study findings were considered unpublished. When the study results were reported as an abstract or poster presented at a scientific meeting, we considered that the results were not available.
In fact, previous evidence showed that the quality of abstracts presented at meetings is suboptimal, frequently including results that are not the final results  [19, 20]. To determine whether the publication(s) corresponded to the registered observational study, we retrieved the full-text article for all citations selected and assessed from the abstract and the full text, if needed, whether a combination of information, including description of interventions and conditions, population, location, responsible party, number of participants, primary outcome measures, and primary funding sponsor partly or completely matched the information at ClinicalTrials. gov. We only selected articles that reported the results of the study. All cases were assessed by a second independent researcher, and disagreements were resolved by consensus.
Search of results posted on ClinicalTrials. gov and the sponsor website For each observational study identified, we assessed whether unpublished results were publicly available from other sources. For this purpose, (1) we searched whether results were posted at ClinicalTrials. gov and (2) systematically searched the sponsor website to identify whether the results were available. For this, we searched Clinical-Trials. gov for a link to the sponsor website. If no link was available, we searched Google using the sponsor name to identify the sponsor website. Then, we searched the website for a section dedicated to access to study results and used the study NCT number to find the study results.
Main outcomes The main outcomes were the time to the first publication in journals, and the time to the first public availability of the study results. The time to the first publication in journals was the time (in months) that elapsed between the primary completion date of the study and the publication. The study primary completion date was obtained from ClinicalTrials. gov. The publication date was the first date an article was made available online ahead of print (i. e. epub date indexed on PubMed) or published in a paper-printed version. The time to the first public availability of the study results was the time (in months) between the primary completion of the study and the first public availability of the study results by publication or posting on Clini-calTrials. gov or the sponsor website.
Completeness of reporting of outcomes designated as safety issues For all studies with results publicly available (via publication or posting on ClinicalTrials. gov or the sponsor website), we evaluated the completeness of reporting of all primary outcome measure(s) designated as a "safety issue" and when not available, all secondary outcome measure(s) designated as a safety issue as recorded from ClinicalTrials. gov. Then, for each outcome recorded, we systematically searched in the report available (i. e. publication, and for unpublished studies, results posted on ClinicalTrials. gov or the sponsor website) whether the results were adequately reported (i. e. reported with the number of events per arm), partially reported (i. e. reported with the number of events pooled or only mentioned), or not reported.
Publication of study results The median time between the study primary completion date and our search was 49. 0 (Q1-Q3, 41. 0-62. 0) months; for all studies, at least 30 months had elapsed since the study primary completion date. Among the 489 studies, we identified a publication via the citation reported at ClinicalTrials. gov for 75 and via a systematic search of PubMed for 99. For the 314 remaining studies without a publication identified, we obtained an email address and contacted the sponsor/principal investigator of 241 studies, of which 52 responded and 15 provided an article (Additional file 1: Appendix 2). Overall, results of only 189 (39 %) studies were published at least 30 months after study primary completion, corresponding to a cumulative target of enrollment of 785,437 participants (65 % of the total target number of participants).
Posting of study results on ClinicalTrials. gov and on the sponsor website When searching the results section of ClinicalTrials. gov, we obtained the results for 31 % (n = 94) of our unpublished studies (i. e. 147,593 cumulative target participants). When searching sponsor websites, we obtained the results for 64 (21 %) of unpublished studies (i. e. 195,291 cumulative target participants), of which 39 could be obtained through a link to the sponsor website available in Clinical-Trials. gov. All study results obtained from sponsor websites were sponsored by Bayer (n = 16), GSK (n = 12), or Novo Nordisk (n = 36).
Overall public availability of study results Figure  3  summarizes the public availability of results of the selected studies in terms of publication or posting on ClinicalTrials. gov or the sponsor website. Overall, 347 studies (71 %) had results publicly available at least 30 months after study primary completion, corresponding to a cumulative target of enrollment of 1,128,321 participants (i. e. 93 % of the total target number of participants). The median (Q1-Q3) sample size of studies with publication available was 165 (60-882), studies with unpublished data available through ClinicalTrials. gov, 503 (154-1307) studies with unpublished data available through the sponsor website 1001 (393-2366), and studies with no results publicly available 130 (50-556). The cumulative proportion of studies with results publicly available over time overall and stratified by funding source is shown in Fig.
Completeness of reporting (Table 2) Of the 347 studies with results publicly available, 243 reported at least one primary outcome measure designated as a "safety issue". We evaluated the completeness of reporting of 242 studies (one excluded because not published in English): all these outcomes were adequately reported in 86 % (114/133) when available in a publication, 91 % (62/68) when available on Clinical-Trials. gov, and 80 % (33/41) when available on the sponsor website. Of the 96 studies that did not have a primary outcome measure designated as a "safety issue", 66 (69 %) adequately reported all secondary outcomes designated as a safety issue; 60 % when available in a publication, 92 % when available on ClinicalTrials. gov, and 61 % when available on the sponsor website.
Discussion We evaluated the public availability of study results in a cohort of 481 observational studies evaluating an intervention with safety outcome(s) registered at Clinical-Trials. gov and completed for more than 30 months. Only 39 % (n = 189) had results published, corresponding to 65 % of the total target number of participants. The Fig. 3  Availability of reports at least 30 months after study completion cumulative percentage of studies with results published at 12 and 24 months after primary completion were 8. 2 % (95 % CI, 5. 7-10. 6) and 21. 3 % (17. 6-24. 9), respectively. However, when searching other data sources (results posted on ClinicalTrials. gov and sponsor websites), we obtained the results for about half of the studies with unpublished results (n = 158, 53 %) corresponding to 93 % of the total target number of participants.
Further, the median sample size of unpublished results posted on ClinicalTrials. gov and sponsor websites was high. To our knowledge, this is the first large study evaluating the public availability of the results of observational studies registered at ClinicalTrials. gov in terms of publication or posting on ClinicalTrials. gov or the sponsor's website. Most evidence of the lack of availability of research results has focused on the publication of clinical trials and the posting of results on ClinicalTrials. gov [5, 6, [21] [22] [23] [24]. Large cohorts of registered clinical trials showed that the results of only 46-63 % are published  [21, 23]. In the field of diagnostic studies, results for 54 % of studies completed for at least 18 months were published  [25]. Our study has several important implications; they clearly illustrate the need for a change in policy with a request to also prospectively register observational studies.
In many situations, observational studies are the only data available because RCTs are not appropriate or feasible. The number of published meta-analyses including observational studies in health has increased substantially and these meta-analyses could be used to inform clinical decision-making and public health policy  [18, 26]. Registration of observational studies is debated, as shown by recent editorials published in major medical journals  [17, [27] [28] [29] [30] [31] [32]. However, much of the rationale for the prospective registration of clinical trials  [18, 33]  also applies to the registration of observational studies. Although registration at ClinicalTrials. gov does not guarantee that trial results will be published in a timely manner  [22, [34] [35] [36] , it allows knowing the existence of the study, searching for unpublished data, exploring publication bias, outcome reporting bias, and fidelity to the protocol  [37, 38].
Our results also highlight the need to reconsider the strategy used to identify research findings. In fact, searching the results section at ClinicalTrials. gov as well as the sponsor website increased by two-fold the number of studies with results available and allowed access to the data of 93 % of the total target number of participants. It is consequently very important that systematic reviewers search for these data in trials registries and the sponsors websites. Previous studies comparing the results posted and published for clinical trials showed that results are more completely reported in registries than in publications  [21, 23]  and that discrepancies between the ClinicalTrials. gov results database and matching publications are common  [38]. However, it is unclear which source is more accurate  [38].
Publication of research findings in peer-reviewed journals is considered essential for disseminating research results. However, the publication process is long, complicated and supposes an important investment from the sponsor and investigator. Lack of or delayed publication could be related to the lack of incentives to disseminate negative results, time constraints, limited resources, changing interests, or difficulties and failure to have the results published  [34]. The sponsor may prefer posting results on a website than investing in publication in a peer-reviewed journal. However, we question why sponsors post results on their website and not on Clinical-Trials. gov. In fact, ClinicalTrials. gov performs a quality control, which is not the case with the sponsor website. Finally, ClinicalTrials.
gov offers researchers the opportunity to provide access to their data if they decide not to publish them in a peer-reviewed journal. For clinical trials, the posting of results is a requirement. The World Health Organization is calling for a strict timeline for public disclosure of clinical trial results and published a new Statement on the Public Disclosure of Clinical Trial Results, which specifies that study results be reported at least 30 months after a study is completed. This requirement should also be extended to observational studies. Our study has some potential limitations. First, as registration of observational studies is not mandatory, we could explore the public availability of the trial results only for observational studies registered at Clini-calTrials. gov. However, the publication rate and public availability of results is not likely to be higher for observational studies that were not registered.
Second, our search was performed only in Medline via PubMed and we may have missed some publications. However, Medline is the largest database of biomedical journals and is the source that nearly all physicians and policymakers use to access clinical trial findings. Further, a recent study showed that searching Embase has a modest impact on the results of systematic reviews  [39]  and most studies evaluating RCT publication did not search Embase  [21, 34]. Additionally, we systematically contacted the sponsor or investigator to check whether the study was published. Finally, we used the data recorded at ClinicalTrials. gov, but this information is not always accurate, and ClinicalTrials. gov added a database of summary results allowing for reporting results of observational studies according to the STROBE statement only in September 2008.
Statistical analysis involved use of SAS v9. 4 (SAS Inst. Inc. , Cary, NC) and R software (v3. 1. 2) (http://www. R-project. org, the R Foundation for Statistical Computing, Vienna, Austria). Search strategy n = 1243 citations Exclusion using ClinicalTrials. gov filters n = 506 Studies with primary completion date n = 262 after12. 31. 11 Retrospectives studies n = 163 Phase 0, 1, or 2 studies n = 24 Studies without primary completion date n = 57 reported Assessment of the study record at Clinicaltrial. gov n = 737 Trials excluded after reading the full record at Clinicaltrials. gov n = 248 Randomized controlled trials n = 39 Studies not assessing a medical Intervention n = 110 Studies evaluating a genetic, predictor or risk n = 28 factors, or pharmacodynamics and pharmacokinetics of drugs Retrospectives studies n = 9 Phase 0, 1, or 2 studies n = 8 Studies with healthy volunteers n = 12 Completion date after October 2007 n = 39 Studies included in the analysis Other n = 3 n = 489 Fig.
Procedures We performed a gene expression microarray on resected CRLM in the derivation cohort to assess individual gene expression, as previously described. (19)  Briefly, fresh frozen tumor specimens were obtained from archived tissue collected at the time of surgery from patients who underwent liver resection at MSKCC. Corresponding clinical information was obtained from a prospectively maintained hepatic resection database and supplemented by medical record review. After the above described exclusions, 187 frozen tissue samples were found to have at least 70% viable tumor cells based on histologic verification under hemotoxylin and eosin staining. These samples were macrodissected, RNA was extracted using Trizol (Invitrogen, Carlsbad, CA), quality analyzed using an Agilent Bioanalyzer (Agilent Technology, Palo Alto, CA), and included for microarray analysis if the RNA integrity number (RIN) was ≥ 7 in both cohorts.
Extracted total RNA was reverse-transcribed by a previously published method and the resulting complimentary DNA (cDNA) template was applied to gene expression analysis. (19)  The target cDNAs were hybridized to the Illumina Human HT-12 Gene Chip containing a total of 47,231 annotated gene probe sets (Illumina, San Diego, CA). Arrays were scanned by using standard Illumina protocols and scanners. Microarray data are available in the ArrayExpress database under accession number E-MTAB-1951. For the validation cohort, after the above described exclusions, frozen tumor samples were obtained and macrodissected as above, RNA extracted, cRNA synthesized and labeled, and mRNA generated and amplified as previously described  (19). Gene expression was determined using the Human Array-Ready Oligo set (version 2.
0, Qiagen, Limburg, Netherlands). (20)  All data and protocols are available in Array Express under accession number E-TABM-1112. All microarray data were log transformed and quantile normalized. Principal Component Analysis (PCA) is a mathematical technique commonly used to visualize datasets whose samples are characterized by a large number of variables. PCA reduces the dimensionality of the data while retaining most of the variation in the dataset by identifying directions along which the data variation is maximal, called principal components. This allows for each sample to be represented by a limited number of principal components, allowing for visual plotting of samples, and hence identifying similarities and differences. ( 21 ) Supervised PCA (SPCA) is similar to conventional PCA however uses only a subset of the predictors selected based on their high association with outcome, to overcome over fitting of highly correlated variables.
Statistical methods We compared categorical variables using the χ 2 test and continuous variables using a twosided t-test. We performed univariate analysis using the log-rank test and multivariate analysis using the Cox regression model. Gene expression data were analyzed using R statistical software (version 3. 0). We performed all other statistical analysis using StataSE (version 13. 1, TX, USA) and Prism (version 6, CA, USA). P<0. 05 was considered statistically significant.
(3, 36)  However, certain subgroups of patients may experience benefit. In the validation cohort, in patients who did not receive neoadjuvant chemotherapy, adjuvant chemotherapy was associated with improved OS in the low MRS group (HR 8. 5, 95% CI 1. 4-118, P=0. 02) but not the high MRS group (P=0. 38). Although this result is based on a small sample size and must be explored in a larger, prospective dataset, it suggests that the MRS may have clinical applications akin to multigene assays in breast cancer by identifying subsets of patients that derive greater benefit from chemotherapy, and shifting risk stratification from a TNM to a multigene system. (18)  It may also enhance our ability to molecularly stratify and select patients for clinical trial inclusion to assess the benefit of novel systemic therapies.
This study is retrospective, limited to resected patients, and fails to capture high risk, resectable candidates who progress to unresectability on initial systemic treatment. Conversely, it does not capture patients who have a complete pathologic response to systemic treatment, as seen in up to 37% of patients. (40)  The study was also performed on banked frozen tissue and omitted patients who had poor quality RNA, or no banked tissue, providing additional sources of selection bias. It is unknown if the MRS remains prognostic when expression is measured on tissue embedded in paraffin, and this is currently under investigation. Finally, this study is limited by small sample size and mandates validation in larger prospective datasets. In summary, we report a novel, externally validated prognostic gene expression signature for OS in resected CLRM that is superior to existing CRSs.