dataesr/scientific-paragraphs-categorization huggingface dataset:
Quantitative RT-PCR analyses Reverse transcription was performed using Superscript II reverse transcriptase (Invitrogen) with random hexamers as primers. RNA samples isolated from at least three independent experiments were tested for each condition. Real-time PCRs were run on a LightCycler system (Roche) using the FastStart DNA MasterPLUS SYBRGreen I kit (Roche) according to the manufacturer's instructions. For gene expression normalization, six reference genes were selected from the RNA-Seq data of the current study, on the basis of their similar levels of expression in both culture conditions (exponential and stationary growth phases) and M. truncatula nodules. The expression level of these genes was then examined by qRT-PCR in wild-type and rpoE2 mutant strains grown at 28 and 408C, and expression data were computed using the NormFinder application.
Overall, each variable y i , representing possible annotations for nucleotide i, may take 11 different states. Such states cannot appear arbitrarily in the genome sequence. For example, a CDS must start and end at specific codons. The CRF model can capture gene structures described as simple automaton. The automaton used in EuGene-P is described in Fig. 2  where i is the frame of the corresponding codon in the sequence) enable to, respectively, start or end a CDS inside a transcript and possibly inside another CDS in a different frame. Finally, the conditional probability distribution that relates the evidence in x and possible annotations in y must be described. In CRF, this is done through a set of features. Every type of experimental or statistical evidence is represented by one (or more) feature.
A feature is a small mathematical function that uses some available evidence to vote in favour of (or against) the prediction of specific elements. For example, a 'protein similarities' feature would vote in favour of CDS prediction in the regions that have similarities with known proteins. A precise definition of the different features available in EuGene-P is given in Materials and Methods. Once the set of features used for gene finding is fixed, the CRF model can be trained. This training process computes a multiplicative factor for each feature that determines a feature-specific confidence. The prediction is then in charge of finding the annotation that has maximum conditional probability. This is the prediction that accumulates most support from all features. Overall, the mathematical model and associated software provide a qualitative improvement in terms of its abilities in predicting TSSs, untranslated transcribed regions, overlapping CDSs, ncRNA genes and antisense genes.
Generation of high-quality Sinorhizobium meliloti 2011 genome and transcriptome sequencing data The genome sequence of the streptomycin-resistant derivative of S. meliloti strain 2011 was generated using a combination of 454 (Roche) and Solexa (Illumina) technologies that provided a total coverage of 190 genome equivalents. The assembly of the complete genome sequence was guided by the S. meliloti 1021 sequence that was determined previously. 31 he comparison of these two DNA sequences revealed 463 polymorphisms, including 332 SNPs, 119 Indels and 12 large deletions or insertions (. 10 bp; Supplementary Table  S3 ). In addition to these differences, a 3564-nt region was specifically present in the chromosome of Sm2011 but not in Sm1021. This insertion, located between SMc03253 and SMc03254, was checked and confirmed by PCR amplification.
These results suggest that presumably only 10% of the 463 polymorphisms are real (most being errors in the Sm1021 sequence). To obtain a global view of the transcriptome of Sm2011, RNAs were prepared from bacteria grown in three very different physiological conditions to cover a large number of expressed genes. These include RNAs extracted from bacteria grown in liquid cultures (in both exponential and stationary growth phases) and from 10-day-old nodules in which bacteria were differentiated in nitrogen-fixing bacteroids. 52 For each condition, three biological replicates were performed to assess data reproducibility RNA samples were depleted in both ribosomal RNAs and the highly abundant tRNA-Ala using a S. meliloti-specific capture set of oligonucleotides and were sequenced using the stranded Illumina protocol.
53, 54 his protocol, based on ligation of adapters directly to the 3 0 and 5 0 ends of the RNA molecules, has the advantage of preserving the information about the transcript orientation. The RNA-Seq libraries generated in this study are listed in Table  1. The resulting sequences were mapped onto the S. meliloti genome sequence. RNA-Seq data appeared to be highly reproducible as shown in Supplementary Fig. S1  (Pearson correlation values varied from 0. 899 to 0. 998 between biological replicates). Of the 6308 S. meliloti annotated CDSs (see below), the expression of 5717 (90%) was detected in at least one experimental condition [raw expression level summed in the six libraries (short and long) of one condition was above 50 reads]. The number of mapped reads per nucleotide (summed values from triplicates) was visualized using the Apollo interface.
Annotation of the Sinorhizobium meliloti 2011 genome using EuGene-P EuGene-P inherits from EuGene its ability to integrate a variety of data. Selecting the most significant or informative sources of evidence is highly beneficial for the quality of the final annotation. We decided to use: (i) Similarities with known protein sequences modelled as a dedicated feature that votes for the prediction of coding regions in the corresponding coding frame (see Supplementary Materials and Methods). To identify similarities, we used the SwissProt database as a reliable general source of information for protein similarities. In addition, we used the proteome of the Sm1021 (set of all the protein sequences obtained by translating all CDS of the Sm1021 annotation) as a more specific source of information.
39,63 -66 The TSSs of. 100 promoters known or supposed to be controlled by either one of these sigma factors were experimentally mapped in various studies (Table  4 ). The TSSs annotated from the transcript 5 0 ends mapped in the present study are in good agreement with these data, as 72% of the experimentally mapped TSSs match (+5 nt) our annotated TSSs (Table  4  and Supplementary Table  S8 ). Several authors used the consensus promoter sequences deduced from these experimentally determined TSSs, combined or not with microarray or Affimetrix data, to predict. 200 additional putative targets of these sigma factors (Table  4 ). The good congruence of these predictions with our data (74%) further strengthens our annotation (Table  4  and Supplementary Table  S8 ). Note, however, that the number of correctly annotated TSSs was found to be positively correlated with the number of reads.
The Sm2011 oriented RNA sequencing also showed a complex landscape of expression on both strands. Such complexity would have been completely hidden by nonoriented sequencing, possibly leading to biased b Data extracted from  67, 85, 86  (RpoD)  65  (RpoH1/H2),  39, 40, 70  (RpoE2),  66  (RpoE1/E4)  87 -94   Oriented RNA-Seq data give an opportunity to define a new generation of integrative prokaryotic genome annotation tools. In the area of prokaryotic genome annotation, existing NGS-related studies  11  have focussed on the possibly increased level of sequencing errors associated with such technologies. Here, we showed that the quality of the Sm2011 genomic sequence obtained by NGS is comparable with, if not better than, the Sm1021 genomic sequence previously generated by Sanger sequencing.
5S RNA (SRP, ffs), 6S RNA (ssrS), tmRNA (ssrA), the ribozyme RNase P (rnpB) and incA that mediates plasmid incompatibility phenotypes,  75  36 ncRNAs belong to known families described in the Rfam database, whereas the remaining predicted ncRNAs could not be assigned to a given class. A lot of work thus remains to be done to validate the existence of predicted ncRNAs and to elucidate their function in S. meliloti. Interestingly, 454-sequencing of small ncRNAs of A. tumefaciens, a bacterium phylogenetically close to S. meliloti, recently revealed the presence of numerous small RNAs on all four replicons. 69 The number of ncRNAs in S. meliloti would be even higher if widespread repeated elements like the RIME, MOTIF and Sm-1 to Sm-5 repeats  59 -61  , that appeared to be highly transcribed elements, were taken into account.
Acknowledgements: We thank  Benedicte Bastiat  for constructing the Sm2011 rpoE2 mutant, and  Pierre Dupuy  for help with qRT-PCR experiments. We thank  Svetlana Yurgel  and  Michael Kahn  ( University of Washington ) for exchanging the sequence data on putative frameshifts between the two S. meliloti strains, Sm2011 and Sm1021. Supplementary data: Supplementary Data are available at www. dnaresearch. oxfordjournals. org.
Funding This work was supported by the  Agence Nationale de la Recherche  under grant  ANR-08-GENO-106  " SYMbiMICS ". This research was done in the  Laboratoire des Interactions Plantes-Microorganismes , part of the  Laboratoire d'Excellence (LABEX)  entitled  TULIP  ( ANR-10-LABX-41 ). F. Jardinaud was supported by the  Institut National Polytechnique de Toulouse.
Back Close Full against observations in not available yet due to the absence of sufficiently large observational datasets. This is, however, likely to change in the near future owing to the rapid development of radar based technology  (Raumonen et al. , 2013). In the mean time, inter-model comparison  (Fontes et al. , 2010)  and stand level validation  (Kramer et al. , 2008)  increased our confidence that the model simulates a realistic canopy structure under different management strategies. In this study, ForGEM was set up to simulate forest stands composed of different species and subject to different thinning regimes over one rotation (Table  1 ). These simulations provided us with key variables to describe the forest canopy structure, i. e. , tree height, crown volume as ellipsoids, leaf mass, leaf area index (LAI) and the spatial position of all trees within the stand.
Back Close Full plicit position of trees and their crown dimensions. This information was retrieved from the ForGEM simulations mentioned above. To calculate the albedo from the effective LAI the following vegetation radiative properties are required: the effective leaf single scattering albedo (for both visible (VIS) and near-infrared (NIR) wavelengths), the effective preferred scattering direction of vegetation scattering (VIS and NIR), and the so-called true background albedo which is the albedo of the surface below the dominant tree canopy (VIS and NIR). All parameters were taken from the Joint Research Centre Two-stream Inversion Package (JRC-TIP)  (Pinty et al. , 2011a (Pinty et al. , , 2011b)) , software which inverts the two-stream model  (Pinty et al. , 2006)  to best fit the MODIS broadband visible and near-infrared white-sky surface albedo from 2001 to 2010 at 0.
Set-up of the species-thinning experiment The above model chain was applied to pine, beech and oak single-species forest. For each species we simulated four different thinning strategies, going from unmanaged to intensively thinned forest (see Table  1 ). All simulations were based on the growing conditions that approximate the highest site class of Dutch yield tables  (Jansen et al. , 1996) , using observed climate over the period 1975-2010 for central Netherlands from the MARS database (accessed online 30 April 2013: http://www. marsop. info/marsop3/). The simulations start with about 5000 trees per hectare aged between 10 and 15 years and forest regeneration was considered absent within a rotation. For each species and thinning type, canopy albedo was calculated across Europe for a transect along the 9 • E line of longitude stretching from 40 • to 60 • N, in discrete steps of.
From this database years were only selected when outgoing and incoming shortwave radiation was recorded. Albedo was calculated as the ratio of downward and upward radiation as observed with two-way pyranometers. The overall expected instrumental accuracy is in the range 4-7 % in clear sky and 1-4 % in overcast conditions  (Cescatti et al. , 2012). The radiation measurements cover the wavelengths from 0. 21 to 2. 80 µm. The albedo at EE-Jär is measured as top-of-canopy bidirectional reflectance factor (BRF) with a UAV spectrometer  (Kuusk, 2011). The measured BRF was carried out at different dates in July and August in 2012 and transformed with the help of the 6S atmosphere radiation transfer model  (Vermote et al. , 1997)  and the FRT forest reflectance model  (Kuusk and Nilson, 2000)  into visible and near-infrared albedo for the solar zenith angle 39.
Breast cancer is the most common malignancy in women and is responsible for hundreds of thousands of deaths annually. As with most cancers, it is a heterogeneous disease and different breast cancer subtypes are treated differently. Understanding the difference in prognosis for breast cancer based on its molecular and phenotypic features is one avenue for improving treatment by matching the proper treatment with molecular subtypes of the disease. In this work, we employed a competition-based approach to modeling breast cancer prognosis using large datasets containing genomic and clinical information and an online real-time leaderboard program used to speed feedback to the modeling team and to encourage each modeler to work towards achieving a higher ranked submission. We find that machine learning methods combined with molecular features selected based on expert prior knowledge can improve survival predictions compared to current best-in-class methodologies and that ensemble models trained across multiple user submissions systematically outperform individual models within the ensemble.
The MAQC-II project  [24]  employed blinded evaluations and standardized datasets in the context of a large consortium-based research study to assess modeling factors related to prediction accuracy across 13 different phenotypic endpoints. Efforts such as CASP  [25] , DREAM  [26] , and CAFA  [27]  have created communities around key scientific challenges in structural biology, systems biology, and protein function prediction, respectively. In all cases it has been observed that the best crowd-sourced models usually outperform state-of-the-art off-the-shelf methods. Despite their success in achieving models with improved performance, existing resources do not provide a general solution for hosting open-access crowd-sourced collaborative competitions due to two primary factors. First, most systems provide participants with a training dataset and require them to submit a vector of predictions for evaluation in the held-out dataset  [20, 22, 24, 26] , often requiring (only) the winning team to submit a description of their method and sometimes source code to verify reproducibility.
While this achieves the goal of objectively assessing models, we believe it fails to achieve an equally important goal of developing a transparent community resource where participants work openly to collaboratively share and evolve models. We overcome this problem by developing a system where participants submit models as re-runnable source code by implementing a simple programmatic API consisting of a train and predict method. Second, some existing systems are designed primarily to leverage crowd-sourcing to develop models for a commercial partner  [22, 23]  who pays to run the competition and provides a prize to the developer of the best-performing model. Although we support this approach as a creative and powerful method for advancing commercial applications, such a system imposes limitations on the ability of participants to share models openly as well as intellectual property restrictions on the use of models.
This research group, called the Federation, was set up as a mechanism for advancing collaborative research projects designed to demonstrate the benefit of teamoriented science. The rest of our group consisted of the organizers of the DREAM project, the Oslo team from the Norwegian Breast Cancer study, and leaders of the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC), who provided a novel dataset consisting of nearly 2,000 breast cancer samples with median 10-year follow-up, detailed clinical information, and genome-wide gene expression and copy number profiling data. In order to create an independent dataset for assessing model consistency, the Oslo team generated novel copy number data on an additional 102 samples (the MicMa cohort), which was combined with gene expression and clinical data for the same samples that was previously put in the public domain by the same research group  [4, 28].
The initial study using the METABRIC data focused on unsupervised molecular sub-class discovery  [29]. Although some of the reported sub-classes do correlate with survival, the goal of this initial work was not to build prognostic models. Indeed, the models developed in the current study provide more accurate survival predictions than those trained using molecular sub-classes reported in the original work. Therefore, the current study represents the first large-scale attempt to assess prognostic models based on a dataset of this scale and quality of clinical information. The contributions of this work are two-fold. First, we conducted a detailed post-hoc analysis of all submitted models to determine model characteristics related to prognostic accuracy. Second, we report the development of a novel computational system for hosting community-based collaborative competitions, providing a generalizable framework for participants to build and evaluate transparent, re-runnable, and extensible models.
Further, we suggest elements of study design, dataset characteristics, and evaluation criteria used to assess whether the results of a competition-style research study improve on standard approaches. We stress that the transparency enabled by making source code available and providing objective pre-defined scoring criteria allow researchers in future studies to verify reproducibility, improve on our findings, and assess their generalizability in future applications. Thus the results and computational system developed in this work serve as a pilot study for an open community-based competition on prognostic models of breast cancer survival. More generally, we believe this study will serve as the basis for additional competitionbased research projects in the future, with the goal of promoting increased transparency and objectivity in genomics research (and other applications) and providing an open framework to collaboratively evolve complex models leading to patient benefit, beyond the sum of the individual efforts, by leveraging the wisdom of crowds.
Competition dataset characteristics We used the METABRIC dataset as the basis of evaluating prognostic models in this study. This dataset contains a total of nearly 2,000 breast cancer samples. 980 of these samples (excluding those with missing survival information) were available for the duration of the collaborative competition phase of this study. An additional 988 samples became available after we had concluded our evaluation in the initial dataset and, fortunately, served as a large additional dataset for assessing the consistency of our findings. For each sample, the dataset contains median 10 year follow-up, 16 clinical covariates (Table  1 ), and genome-wide gene expression and copy number profiling data, normalized as described in  [29] , resulting in 48,803 gene expression features and 31,685 copy number features summarized at the gene level (see Methods).
Initial analysis was performed to confirm that the data employed in the competition were consistent with previously published datasets and to identify potential confounding factors such as internal subclasses. Data-driven, unsupervised hierarchical clustering of gene expression levels revealed the heterogeneity of the data and suggested that multiple subclasses do exist (not shown)  [29]. However, for the current analysis we decided to focus on the well established separation into basal, luminal, and HER2 positive subclasses, as previously defined  [2, 30]. These subclasses are known to closely match clinical data in the following way: most triple-negative samples belong to the basal subclass; most ER positive samples belong to the luminal subclass; and most ER negative HER2 positive samples belong to the HER2 subclass.
To ensure that this holds in the current dataset, the 50 genes that best separate the molecular subclasses in the Perou dataset  [31]  (PAM50) were used for hierarchical clustering of the METABRIC data and compared with a similar clustering of the Perou dataset (Figure  1A ). The results of the supervised clustering reveal similar subclasses with similar gene expression signatures as those presented by Perou et al, and were also consistent with the clinical definitions as presented above. Finally, the 3 subclasses show a distinct separation in their Kaplan-Meier overall survival plots for the three subtypes defined by the clinical data, where the HER2 subclass has the worst prognosis, followed by the basal subclass, and the luminal subclass has the best prognosis, as expected (Figure  1B ).
3. We constructed a series of ensemble learning algorithms by computing concordance index scores after averaging the rank predictions of subsets of models. Models trained using ensemble strategies included: a. 15 ensemble models combining the learning algorithms for each model category. b. 4 ensemble models combining the model categories for each learning algorithm. c. 1 ensemble model combining all model categories and learning algorithms. This experiment design resulted in a total of 60 models based on combinations of modeling strategies from the uncontrolled. Hierarchical clustering on the gene expression data of the PAM50 genes in both datasets reveals a similar gene expression pattern that separates into several subclasses. Although several classes are apparent, they are consistent with sample assignment into basal-like, Her2-enriched and luminal subclasses in the Perou data.
Similarly, in the METABRIC data the subclasses are consistent with the available clinical data for triplenegative, ER and PR status, and HER2 positive. (B) Kaplan-Meier plot for subclasses. The METABRIC test dataset was separated into 3 major subclasses according to clinical features. The subclasses were determined by the clinical features: triple negative (red); ER or PR positive status (blue); and HER2 positive with ER and PR negative status (green). The survival curve was estimated using a standard Kaplan-Meier curve, and shows the expected differences in overall survival between the subclasses. (C,D) Kaplan-Meier curve by grade and histology. The test dataset was separated by tumor grade (subplot C; grade 1 -red, grade 2 -green, grade 3-blue), or by histology (subplot D; Infilitrating Lobular -red, Infiltrating Ductal -yellow, Medullary -green, Mixed Histology -blue, or Mucinous -purple).
The survival curves were estimated using a standard Kaplan-Meier curve, and show the expected differences in overall survival for the clinical features. doi:10. 1371/journal. pcbi. 1003047. g001 experiment (Table  S4 ), plus 20 models using ensemble strategies. This controlled experimental design allowed us to assess the effect of different modeling choices while holding other factors constant. Following an approach suggested in the MAQC-II study  [24] , we designed negative and positive control experiments to infer bounds on model performance in prediction problems for which models should perform poorly and well, respectively. As a negative control, we randomly permuted the sample labels of the survival data, for both the training and test datasets, and computed the concordance index of each model trained and tested on the permuted data.