dataesr/scientific-paragraphs-categorization huggingface dataset:
5/619. 1), compared with 17 features with significantly increased concentrations. The high proportion (35%) of features with decreased concentrations among the discriminating signals may therefore explain why two of them have been included in the SVM classification rule. Finally, to demonstrate that our approach can be applied to other omics data, we analyzed the reference transcriptomics dataset resulting from the microarray analysis of bone marrow samples from 72 leukemia patients  (Golub et al. , 1999). Preliminary univariate hypothesis testing indicated that 1154 out of the 7129 features were significant for median difference between the lymphoblastic and myeloid groups. In addition, the two groups were clearly visible on the score plots from PLS-DA, but not PCA (data not shown). Our algorithm identified signatures from 1 to 2 gene probes (Figures  3C, 4C ).
Random Forest and SVM (but not PLS-DA) models trained on the signatures had superior or equal prediction accuracies than the classifiers trained on the full dataset (Table  1 ). The four selected features ranked 1st, 5th, 7th, and 12th in the list of significant variables by univariate hypothesis testing, ordered by increasing p-values. Three of them, cystatin C, zyxin, and CD33, were also part of the 50 gene signature selected by Golub and colleagues on the basis of a filter metric derived from the Student's statistic  (Golub et al. , 1999). The accuracies of the models trained on the final S signatures are in bold. na: for the diaplasma dataset, no feature was selected as significant in the last iteration with the SVM classifier; ns: the Q2Y value of the PLS-DA classifier on the full spikedApples dataset is not significant indicating that the model before feature selection is overfitted.
Stability of the Signatures and Sensitivity/Specificity of the Selection The influence of bootstraping on the stability of the S and S+A signatures was assessed for each of the 4 datasets by increasing the number of bootstraps from 5 to 200, and looking for differences in stability by using repeated measure ANOVA. No significant difference was observed above 20 bootstraps for PLS-DA and SVM, and 10 bootstraps for Random Forest. The number of 50 bootstraps was thus selected as the default value in biosigner, and used in all computations. To assess the sensitivity and the specificity of the methodology, datasets containing known discriminant variables were simulated (see Supplementary Material). To avoid making hypotheses about the structure of an "omics" dataset (e. g. , noise, intensity distribution, etc.
), we started from the real metabolomics and transcriptomics datasets and removed all features which were significant by univariate hypothesis testing  (Wehrens and Franceschi, 2012). We then increased the discriminant capacity of one of the features by multiplying the intensities in one of the sample groups by a factor. The factor was chosen so that the target feature was still not detected at a False Discovery Rate of 0. 05. By applying this methodology to our transcriptomics and metabolomics datasets, we observed that, despite the high ratio of variables to samples (up to 100 for the leukemia dataset), the target feature was detected with a high sensitivity (from 60% up to 100% in the union of the three classifier signatures) and a high specificity (more than 98%; see Supplementary Material).
Comparison with Alternative Feature Selection Methods We compared our algorithm with 6 alternative approaches for feature selection, namely VIP filtering (at the 1 and 1. 5 thresholds;  Mehmood et al. , 2012) , recursive feature elimination wrapper (RFE;  Guyon et al. , 2002) , and 4 embedded methods (prediction analysis of microarray, PAM, also called nearest shrunken centroids,  Tibshirani et al. , 2002 , sparse PLS-DA, Chun and Keles, 2010 , Lasso, Tibshirani, 1996 , and Elastic Net, Zou and Hastie, 2005). In particular, to achieve a comprehensive comparison between the biosigner and RFE wrapper approaches, we applied the RFE methodology not only to SVM (as initially described,  Guyon et al. , 2002)  but also to PLS-DA and Random Forest. For each algorithm, the accuracy of the final model, the size and stability of the signature, as well as the performance (harmonic mean between accuracy and stability) were computed for each of the 4 datasets, and the running time on a laptop computer was recorded (Table  2 ).
The best performances were achieved by using the biosigner algorithm: whereas higher prediction accuracies could be obtained by regularized methods such as Elastic Net, the signatures from biosigner were usually more stable. Surprisingly, the stability of the Random Forest and SVM biosigner signatures on the spikedApples dataset (and also of the SVM signature on the diaplasma dataset) was low. The variability observed with the spikedApples dataset may be due to the small number of samples (10 in each group) in addition to the very small proportion of discriminating signals. The median and interquartile metric values for the sacurine, leukemia and diaplasma datasets are plotted on Figure  5. We see that biosigner selects restricted signatures, which are usually of high stability and provide high prediction accuracy.
Implementations of the biosigner Methodology into an R/Bioconductor Package and a Galaxy/Workflow4Metabolomics Module To share the algorithm and its source code with the bioinformatics community, a biosigner R package (http:// bioconductor. org/packages/biosigner) was published on the Bioconductor repository  (Gentleman et al. , 2004). Furthermore, to provide a graphical interface for the experimenter community, we developed a Galaxy module which was integrated into the Worfklow4Metabolomics (W4M) online infrastructure for computational metabolomics (http://workflow4metabolomics. org;  Giacomoni et al. , 2015). The full history (i. e. , workflow and associated input and output data) of the statistical analysis of the diaplasma dataset described in this study is publicly available on W4M, with the W4M00003 reference number.
Since permutations occur only in the test subsets, the number of models to be built is limited and the algorithm is fast (only a few minutes on a laptop for datasets of several thousands of variables). biosigner returns (i) the final S signature, (ii) the tiers (A to E) containing the features discarded during one of the previous selection rounds, and iii) the models trained on the S (and S+A) signatures to be used for future predictions. The term significance is usually associated in the statistical literature to a hypothesis test and to a so called null hypothesis. Here, we do not assess the significance of the model itself, but we rather estimate the influence of a given subset of features on the prediction accuracy of the model. However, since the number of selected features in the final signature is restricted (i.
e. , smaller than the number of samples), the risk of overfitting is limited. Also, since the training and testing subsets are not set apart during the full procedure (because of the resampling between the selection rounds), the returned performance of the final model may be slightly over-optimistic. External validation on a new dataset is therefore required to refine the estimation of the model accuracy  (Esbensen and Geladi, 2010). By wrapping the biosigner algorithm around 3 binary classifiers with specific mathematical backgrounds (PLS-DA, Random Forest, and SVM), we observed on three published metabolomics and transcriptomics datasets that the signatures had some degree of similarity (e. g. , at least one of the features in the tiers S+A was common to at least two classifiers), but also included classifier-specific features.
For example, with the spikedApples dataset, SVM selects features with opposite variations compared with PLS-DA and Random Forest. The complementarity between the signatures is in agreement with several recent studies reporting classifier-specific results depending on the structure of the dataset (correlation between The metrics obtained with the sacurine (sacu), spikedApples (spik), leukemia (leuk), and diaplasma (diap) datasets are shown (performance is the harmonic mean between the accuracy and the stability). For each metric and each dataset, the method(s) with optimum value is/are in bold. Elast. Net, Elastic Net; PAM, prediction analysis of microarrays (also called nearest shrunken centroids); RF, Random Forest; RFE, recursive feature elimination; sPLS, sparse PLS-DA; VIP filter, filtering by the variable of importance in projection metric from PLS-DA.
2 : the median and interquartile range for the sacurine, leukemia, and diaplasma datasets are indicated as colored symbols and grays arrows, respectively. PAM: prediction analysis of microarrays (also called nearest shrunken centroids); RF, Random Forest; RFE, recursive feature elimination; filter VIP, filtering by the variable importance in projection metric from PLS-DA. features, noise, proportion of zeros intensities;  Christin et al. , 2013; Tarca et al. , 2013; Determan, 2015). The discrepancies come from the specific weights each classifier assigns the variables (or the samples in the case of SVM). In fact, if the same ranking metric is used for all classifiers in our algorithm, signatures become more similar (see Supplementary Material). The use of classifier-specific metrics in biosigner (i.
e. , VIP for PLS-DA, variable importance for Random Forest, and squared weights for SVM) should therefore increase the chances of discovering distinct features of interest. The signatures obtained with the three datasets were short (up to 3 features), and the restriction to these signatures usually resulted in higher prediction accuracies of the classifiers (except for the PLS-DA model on the leukemia dataset). Importantly, restricting to small signatures also helped avoiding the risk of overfitting: for instance, the high performance of PLS-DA model of the spikedApples full dataset was not significant, contrary to the model built on the single feature signature. The selected molecules were also shown to be in good agreement with the reported results: the 2 features from the spikedApples PLS-DA and Random Forest signatures were related to the spiked compounds  (Franceschi et al.
, 2012) , and the 3 gene probes selected by PLS-DA and SVM on leukemia dataset were already included in the published signature  (Golub et al. , 1999). Interestingly, the probe selected by Random Forest, myeloperoxidase, is a cytochemical marker for the diagnosis (and also potentially the prognosis) of acute myeloid leukemia (AML;  Matsuo et al. , 2003). Although myeloperoxidase would also have been included in the published signature if the full dataset had been used for training (as in this study) instead of the 38 sample subset  (Golub et al. , 1999) , this feature would have been ranked only in the 24th position. Altogether, the above results show that biosigner selects relevant signatures providing high and significant prediction accuracy of the classifiers. Comparison with alternative feature selection strategies revealed that biosigner specifically selected stable signatures.
In contrast, Lasso and Elastic net classifiers showed higher accuracy but the signatures were more prone to instability, in agreement with a previous analysis of 4 transcriptomics datasets  (Haury et al. , 2011) : such methods can therefore be of high interest when classification accuracy (instead of feature selection) is the primary goal of the study. For feature selection, however, stability is critical since the subsequent validation steps leading to the diagnostic product will focus on the selected features only. Surprisingly, the stabilities of the biosigner SVM signatures were low for two of the datasets (diaplasma and spikedApples). This suggests that the combination of backward selection and feature ranking with the SVM square weights may result with some datasets in the elimination of relevant features during the first selection rounds.
The biosigner signatures were of restricted size. Such small signatures are pivotal for diagnosis purposes, where only a limited number of candidates are expected to enter the validation phase. The size of the signatures may be a consequence of the stringency of our selection algorithm, but also of the structure of the datasets analyzed in this study, where a very few variables are sufficient to efficiently discriminate between the two sample groups. It should be noted that the criterion for feature selection focuses on the added value of the tested features for model performance. Hence, additional features with equal relevance for prediction may not appear in the final S signature. As an example, only a fraction of the features related to the spiked compounds in the spikedApples dataset is selected.
Matching (e. g. , by age), however, would have resulted in a very restricted subset of only 14 patients, in which no type 2 vs. type 1 significant feature could be found by univariate testing nor biosigner feature selection. We therefore used in this study the full diaplasma dataset instead (63 samples and 5501 features). Two features were selected either in the PLS-DA or Random Forest signatures. Importantly, when these features were modeled by a combination of the 3 covariates (diabetic type, age, and body mass index), only the type effect was found significant by analysis of variance, thus emphasizing the putative value of these markers in the classification of diabetes. Interestingly, one of the features from the S+A PLS-DA signature (497. 275/487) matched with an isotope of a taurochenodeoxycholic acid fragment (according to m/z ratio and retention time): diabetes-associated changes in bile acid metabolism have been reported  (Prawitt et al.
, 2011) , and variation of the taurochenodeoxycholic acid concentration has very recently been described in urine of type 2 patients  (Taylor et al. , 2014). The biosigner signature, which requires further validation by MS/MS experiments and confirmation in another cohort study where patients and controls are matched by age, may therefore highlight new candidates for diabetes screening and diagnosis. The biosigner algorithm is available as an R/Bioconductor package. As no clinical metabolomics dataset is currently available on Bioconductor to the best of our knowledge, we included the diaplasma LC-HRMS dataset into the package: this dataset should be useful for the benchmarking of new statistical and annotation algorithms. We also developed a Galaxy module which was integrated into the Workflow4metabolomics online infrastructure for computational metabolomics (W4M;  Giacomoni et al.
, 2015). Galaxy is a powerful open-source workflow manager enabling users to build their own workflow by selecting the tools and the parameter values via a graphical interface  (Goecks et al. , 2010). Workflows and associated data inputs and outputs can be saved and shared, allowing fine tuning of parameters and reproducible research  (Boekel et al. , 2015). The W4M infrastructure is therefore of high interest for experimenters to build, run, and reference reproducible LC-MS, GC-MS, and NMR workflows, for developers to compare and diffuse their tools, and also for teachers to organize hands-on sessions (since no software installation is required). In conclusion, the biosigner algorithm and the associated software tools should be of high value for biologists, practitioners, and biostatisticians, to identify robust biomarker signatures from large omics datasets for the development of new diagnostics.
Within the workflow, the input data are checked for compliance with the CF and Climate Model Output Rewriter (CMOR, http://pcmdi. github. io/cmor-site/tables. html) standards required by the tool (see Sect. 3) via a set of dedicated reformatting routines, which are also able to fix the most common errors in the input data (e. g. wrong coordinates, undefined or missing values, non-compliant units). It is additionally possible to define new variables using variablespecific scripts, for example to calculate the total column ozone from a 3-D ozone field (tro3), temperature (ta), and surface pressure (ps). The diagnostic and graphic routines are written in a modular and flexible way so that they can be customized by the user via diagnostic-specific settings in the configuration file (cfg-file) and variable-specific settings (in the directory variable_defs/) without changing the source code.
Runoff Evaluation of precipitation is a challenge due to potentially large errors and uncertainty in observed precipitation data  (Biemans et al. , 2009; Legates and Willmott, 1990 ). An alternative or additional option to the direct evaluation of precipitation over land (such as e. g. included in the global precipitation evaluation in Sect. 4. 1. 2) is the evaluation of river runoff that can in principle be measured with comparatively small errors for most rivers. Routine measurements are performed for many large rivers, generating a large global database (e. g. available at the Global Runoff Data Centre (GRDC,  Dümenil Gates et al. , 2000) ). The length of available time series, however, varies between the rivers, with large data gaps especially in recent years for many rivers. The evaluation of runoff against river gauge data can provide a useful independent measure of the simulated hydrological cycle.
, 2009)  from the network of eddy-covariance flux towers  (FLUXNET, Beer et al. , 2010). The leaf area index data set used for evaluation (LAI3g) is derived from the Global Inventory Modeling and Mapping Studies group (GIMMS) AVHRR normalized difference vegetation index (NDVI-017b) data  (Zhu et al. , 2013). Finally, cSoil and cVeg are assessed as mean annual values over different large subdomains using the Harmonised World soil Database  (HWSD, Fischer et al. , 2008)  and the Olson-based vegetation carbon data set  (Gibbs, 2006; Olson et al. , 1985).
In addition, diagnostics that evaluate tropospheric ozone and its precursors (nitrogen oxides (vmrnox), ethylene (vmrc2h4), ethane (vmrc2h6), propene (vmrc3h6), propane (vmrc3h8) and acetone (vmrch3coch3)) are compared to the observational data of  Emmons et al. (2000). A diagnostic to compare tropospheric column ozone from the CMIP5 historical simulations to Aura MLS/OMI observations  (Ziemke et al. , 2011)  is also included and shown as an example in Fig. 24. This diagnostic also remaps the data to the coarsest grid using local area averaging in order to calculate differences. For the stratosphere, total column ozone (toz) diagnostics are implemented. As an example, Fig. 25  shows the CMIP5 total column ozone time series compared to the NIWA combined total column ozone database  (Bodeker et al.
The obvious solution is to download these data sets from elsewhere in the ESGF, and store them locally whilst the analysis is carried out. The indexed search facility provided by the ESGF makes it easy to identify the download URL of such "remote" data sets, and a prototype of the ESMValTool (not included in v1. 0) has been developed that performs this search automatically using esgf-pyclient (https://pypi. python. org/pypi/ esgf-pyclient). If the search is successful, the prototype provides the user with the URL of each file in the data set, and the user (or system administrator) is then responsible for performing the download. The workflow of this prototype is illustrated in Fig. 27. It is possible that the fully automated downloading of remote ESGF data sets may be provided by a future version of the ESMValTool, but for now it is preferable for a human to manage the process due to the large size of the files involved.
Acknowledgements. The development of the ESMValTool (v1. 0) was funded by the  European Commission 's  7th Framework Programme , under Grant Agreement number  282672 , the " Earth system Model Bias Reduction and assessing Abrupt Climate change (EMBRACE) " project and the  DLR  " Earth System Model Validation (ESMVal) " and " Klimarelevanz von atmosphärischen Spurengasen, Aerosolen und Wolken: Auf dem Weg zu EarthCARE und MERLIN (KliSAW) " projects. In addition, financial support for the development of ESMValTool (v1. 0) was provided by  ESA's Climate Change Initiative Climate Modelling User Group (CMUG). We acknowledge the  World Climate Research Program 's (WCRP's) Working Group on Coupled Modelling (WGCM), which is responsible for CMIP, and we thank the climate modelling groups for producing and making available their model output.
This resulted in average GB1  15   Finally, we used in-cell PCS and RDC data as input for GPS-Rosetta, a program that integrates 1 PCSs from multiple paramagnetic centers into unified distance constraints in structure 2 calculation routines. 23 Following the fragment-based rationale used by Rosetta, we generated 3 input libraries of 3-and 9-residue fragments of known protein structures, excluding the structure 4 of GB1 and homologous folds. Using these fragments, we generated 10,000 GB1 structures, out 5 of which we collected the 100 lowest-energy models and compared their conformations to 6 experimentally determined GB1 structures, i. e. X-ray crystallography (PDB code: 2QMT 35 ) and 7 solution NMR (PDB code: 1GB1 36 , 2PLP 37 ). We found poor convergence of individual models, 8 with a median backbone Cα RMSD of 1.
PCS and RDC data can jointly be obtained from single 2D NMR 9 experiments on in-cell NMR samples of low intracellular protein concentrations, measured at 10 moderate magnetic field strengths, which makes them easily accessible and highly useful. The  11   presented approach can further be used for determining glycan and nucleic acid structures, 2,41 as 12 well as to probe ligand interactions. 42 In addition, the high rigidity of the DOTA-M7Py tag 13 renders it a useful tool for in-cell EPR studies. 43, 44 Given that the intracellular delivery of 14 paramagnetically tagged proteins into cultured mammalian cells by electroporation is 15 straightforward, 9 combined PCS and RDC measurements in live cells also hold great promise for 16 future structure determination efforts in intact mammalian specimens.
is available free of charge on the ACS Publications website. It 20 contains Material and Methods, Supporting Figures and Tables, in pdf format. 21 Physical Chemistry Letters § Department of Structural Biology, Institute of Integrative Biology of the Cell (I2BC) -UMR 1 9198, CNRS/CEA/Paris-Saclay University, CEA Saclay Bât. 144, 91191 Gif-sur-Yvette, France. 2 Funding Sources 3 This work was supported by the Agence Nationale pour la Recherche, grant ANR-14-ACHN-4 0015-01 (FXT) and the Fondation Claude et Giuliana, Vaduz, Liechtenstein (TM). PS is 5 supported by an ERC Consolidator Grant #647474 NeuroInCellNMR. 6 Notes 7.
Mobile Browsers Responsive Compatibility A study conducted in late 2013 shows that 50% of mobile phone users use mobile as their primary Internet source  (Roudaki, 2015). However, the development of mobile websites and especially for LCMS lags behind mobile devices. We have created courses for some informatics modules using authoring tools accessible in LCMS Moodle and Dokeos and we have had multiple requests from students for options that they could run on Smartphones to fit the content on the screen size to obtain better legibility. To proof the content portability for mobile browsers, we have processed by several tests. We put a representative selection of mobile browsers through a series of web pages under mentored projects. For this experience a common type of web pages were designed by students, not especially to work on small screens.