dataesr/scientific-paragraphs-categorization huggingface dataset:
Finally, studies with a before/after/ comparator/intervention (BACI) design combine the two approaches, making both a temporal and spatial comparison. Actually, in a BACI design, data are collected both in a control site and before and after intervention in the study site. These types of study designs have different levels of quality. BA designs with a sampling collection on the long term will likely be more valuable than BA designs with a sampling collection on the short term, since they may account for lasting effects of the intervention and seasonal variation. Moreover, BACI study designs will probably be more reliable than BA and CI designs. Similarly, studies that detail potential effect modifiers will probably be more valuable than studies that do not describe the local environment. Detailed reasoning concerning critical appraisal will be displayed in a transparent manner.
Systematic map database The studies included after critical appraisal will be mapped in a database. Table  5  displays the coding tool that will be used for the systematic map database. Key variables of interest and coding options within these variables were identified by the review team and subject experts. These key variables include the following type of information: bibliographic reference, geographical localization of the study sites, subject population, type of exposure/ intervention, methodological design, setting/context and outcome measures. Studies may be coded with multiple keywords within each coding variable where appropriate. For instance, a study conducted in several regions will be coded with multiple keywords for the "Study region" variable. Each included article will first be coded by one reviewer.
Data extraction strategy Once the systematic map will be produced, a synthesis might be realized on more specific questions. In this case, data will be extracted from included studies as follows. Outcome means, measures of variation (standard deviation, standard error, confidence intervals, etc. ) and sample sizes will be extracted from tables and graphs, using image analysis software when necessary. If only raw data are provided, summary statistics will be calculated. If relevant data are difficult to extract accurately from graphs or if they are assumed not to be published, authors might be asked to provide primary data. Data on potential effect modifiers will also be extracted from the included articles.
Data synthesis and presentation The mapping phase will provide descriptive statistics of the regions, taxa and LTIs studied. A narrative synthesis will describe the quality of the results of all the included studies, along with the findings of studies of sufficient quality. Tables will be produced to summarize these results with respect to each of the specific questions. Meta-analyses of effect sizes will be conducted if questions, designs and data formalizations of the included studies are sufficiently homogeneous and if the susceptibility to bias of these studies is low enough. The questions and modalities of the meta-analysis might only be specified once the papers have been read and assessed in order to properly weight the results and choose the metrics for net-effect calculations. Publication bias analysis will also be carried out where possible with the Egger test or the fail safe number.
Acknowledgements The authors thank the CILB for funding this research project, the  ITTECOP research incentive program  for their logistical help and the  FRB  for their methodological support on systematic reviews and the organization of the steering committee meetings. The authors also thank  Leyli Borner  and  Isabelle Le Viol  (MNHN),  Jean Carsignol  and  Eric Guinard  (Centre d'études et d'expertise sur les risques, l'environnement, la mobilité et l'aménagement),  Violette Le Féon  ( Institut national de la recherche agronomique ),  Jean-François Godeau  (Life Elia.
A number of approaches have been developed to call somatic variation in high-throughput sequencing data. Here, we present an adaptive approach to calling somatic variations. Our approach trains a deep feed-forward neural network with semi-simulated data. Semi-simulated datasets are constructed by planting somatic mutations in real datasets where no mutations are expected. Using semi-simulated data makes it possible to train the models with millions of training examples, a usual requirement for successfully training deep learning models. We initially focus on calling variations in RNA-Seq data. We derive semi-simulated datasets from real RNA-Seq data, which offer a good representation of the data the models will be applied to. We test the models on independent semi-simulated data as well as pure simulations.
On independent semi-simulated data, models achieve an AUC of 0. 973. When tested on semi-simulated exome DNA datasets, we find that the models trained on RNA-Seq data remain predictive (sens 0. 4 & spec 0. 9 at cutoff of P >= 0. 9), albeit with lower overall performance (AUC=0. 737). Interestingly, while the models generalize across assay, training on RNA-Seq data lowers the confidence for a group of mutations. Haloplex exome specific training was also performed, demonstrating that the approach can produce probabilistic models tuned for specific assays and protocols. We found that the method adapts to the characteristics of experimental protocol. We further illustrate these points by training a model for a trio somatic experimental design when germline DNA of both parents is available in addition to data about the individual.
These methods are widely recognized as being sub-optimal, but are nevertheless widely applied to new assays. We believe that these methods are used in these instances because: (1) they do not require a strong knowledge of probabilistic models, (2) they are easy to implement in software and (3) domain experts can look at results and contribute suggestions to improve the next iteration of the approach and (4) they enable ranking interesting candidates in the early days of an assay to demonstrate its biological interest. At these stages, it is more important to identify a few strong candidates than to obtain optimal sensitivity and specificity across a wide range of datasets. In contrast, probabilistic methods are developed when an assay becomes popular and more data is produced that requires more sensitive or specific ranking and filtering tools.
Edited bases may therefore appear as mutations in RNA when DNA is not mutated. The main contribution of this manuscript is to present a new paradigm to develop approaches to rank and filter genomic sites. In this paradigm, we use semi-simulated datasets to train a probabilistic model. We evaluate the performance of models trained in this way for calling mutations in RNA-Seq and exome data, discuss the portability of the model from one assay to the other, and describe the versatility of the paradigm by training models for an experimental design where DNA from parents of a patient is available. In contrast to existing approaches, we propose that this new method can be used to quickly adapt a probabilistic model to specific experimental and analysis protocols.
A new paradigm to develop probabilistic models The key idea of this study is presented in Figure  1  where we describe how we train probabilistic models using semi-simulated datasets. This approach does not rely on pre-existing training sets. Instead, we create semi-simulated datasets by planting signal in real datasets and then train probabilisitic models to recover the signal from the noise found in these datasets. We chose deep learning to implement the probabilistic models of this approach because deep neural networks (technically, networks are called "deep" if they have three or more layers) can approximate arbitrary functions and have produced state of the art performance on a wide range of machine learning problems  Angermueller et al. [2016]. Figure 1. Overview of the approach. A minimum of two samples are necessary to produce a semi-simulated dataset for training.
These samples are chosen from data measured from the same individual such that genotypes should match at the majority of sites measured. One sample is arbitrarily assigned the germline role, and the second sample is assigned the somatic role. These samples are provided to the somatic call simulator (whose design can be informed by expert input), which will produce mutated examples using non-mutated examples provided in the input samples (see Methods). Mutations are always added in the sample labeled "somatic". This process yields a training dataset. The same process is repeated with independent pairs of input samples (typically from distinct subjects) to yield a validation and a test set. The training set is used to train a feedforward neural network until performance measured on a small part of the validation set starts to decrease (early stopping).
Semi-simulated training sets To circumvent this problem, we train deep learning models with semi-simulated datasets. Semi-simulated datasets are constructed by simulating signal into samples where no signal is otherwise expected. For instance, in this project, we use two RNA-Seq samples from different types of immune cells from the same subject. Few if any somatic mutations are expected for most sites of the genome in these cells 1. We simulate somatic variations in one of the two samples by moving bases and associated features of the real data to another genotype (see Material and Methods for details). The resulting semi-simulated dataset should accurately reflect the properties of the real data used as input, and for instance, it is expected to capture characteristics of the library preparation and sequencing assay used to obtain the data.
Such errors are possible if the genotype of germline samples differ at some sites (D) This negative example was wrongly classified as a mutation. Upon further investigation of the simulator code, we identified a software bug. During the feature encoding phase, the order of bases was not consistent from sample to sample. Fixing this bug resolved such errors. (D) In a first attempt at simulation, we trained with datasets restricted to have at least 10 counts across the samples at a site. This prevented the model from learning that such sites are more difficult to predict and resulted in over-estimates of model probability. Removing filters from the construction of the training set (to include sites irrespective of their coverage) resolved this issue. 1 ). The increase in performance of the model was expected because an exome assay is specifically optimized to produce data suitable to call genotypes, whereas RNASeq assays are optimized to measure gene expression.
The reliability diagram also confirms that the model performs less reliably across assays. Taken together these data suggest that large performance gains can be obtained by training a model to the specific assay that it will be used for, adapting the model to the assay. While we have not measured this effect at this time, we expect that smaller performance advantages may result from adapting a model to The same model trained on Pair Exome data predicts accurately a subset of high-confidence sites (1), with high predicted probability of mutation, but has degraded performance on other sites (2). This is confirmed by the RC, which shows sub-optimal reliability. specific data analysis protocols.
Training new models Models for new assays can be trained by constructing a semi-simulated dataset using data obtained with the specific assay or analysis protocol. The samples used to train the model should contain no somatic variation (or a small expected number of variations when completely germline samples cannot be obtained). Detailed steps are documented with the software (see https://github. com/CampagneLaboratory/ variationanalysis), but briefly, the semi-simulated dataset is produced by converting the samples.
8/13 Additional validations are needed Additional validations will be needed to firmly establish that the models trained in this way are predictive on real datasets. Because of regulatory limitations on data sharing of genotype data, which have hampered our ability to obtain suitable validation data, we chose to distribute the software and the models to make it possible for others to conduct independent evaluations on private datasets. We hope to be able to test the models on the ICGC GoldSet Alioto et al. [2015]  (request initiated with ICGC in July 2016, approved Sept 2016, we have yet to obtain access to the data files for the ICGC GoldSet which cannot be located through the ICGC portal. An email to the ICGC support mailing list to request access through the ENA, as per ICGC instructions on the ENA web site has remained unanswered as of end of Sept 2016).
Training for new Assays and Experimental Designs Should our approach perform competitively on real datasets, one of its key advantage will be that training can be performed for arbitrary combinations of new assays, experimental designs, and analysis protocol. For instance, training for a new assay only requires a dataset where no somatic variations are expected (such as biological or technical replicates from the same individual). An assay-specific model can be trained following simple documented steps using the software that we have developed for this study. Training for a different analysis protocol (e. g. , combination of read pre-processing and aligner) can be performed similarly. Training for a new experimental design requires additional feature engineering, which must be implemented in new software.
Semi-simulated datasets Many previous studies have taken advantage of machine learning approaches to train probabilistic methods using annotated real datasets. This work differs in that we create the training sets using real datasets where no difference exists, and plant signal artificially in the real data background. If no datasets can be found with no expected difference, training could be also be performed with a dataset where only a minority of the sites are mutated. This would result in under-estimates of the probability of mutation at a site, but this bias should be small if the proportion of mutated sites in the sample is low with respect to the number of planted mutations introduced in the sample.
Of the expected mutation rate We showed that our approach has strong reliability when applied on the same type of assay as that used in the training set. The model outputs the probability that a site is mutated, given the prior distribution of mutated sites in the training set and the reliability diagrams indicate that the forecast probability produced by the model is close to optimal (i. e. , lying very close to the diagonal on the reliability diagram). When the proportion of mutated sites in a dataset differs from that used in the training set, as is often the case in practice, it will be necessary to apply Bayes Theorem to adjust for the difference in priors. Doing so will require an estimate of the rate of mutation in the dataset where predictions must be made. Importantly, such adjustments will not change rank order, and for this reason we expect that the model probability output by the software is suitable for ranking somatic mutation in new samples.
Subject characteristics and recruitment Data from four subjects was used for developing the RNA-Seq somatic caller. RNA was obtained from sorted cells for control subjects who participated to the CFS/ME study. Any minor subject(s) (<18 years old in New York State) who did not provide written acknowledgment of parental permission to participate in the study where excluded. The study was reviewed and approved by the Weill Cornell Medical College Institutional Review Board (protocol 1302013563. "Immune cell gene expression and predictive models in CFS"), and patients and controls gave written informed consent after the study protocol was fully explained. All consented to blood draw and to the availability of the stored samples for additional bioassays and analyses. Data from 12 subjects (three subjects for paired exome caller and nine subjects for trio design: three subjects and their parents) was used to develop the trio somatic caller.
Training, Validation and Test Datasets The RNA-seq training, validation and test sets were created by serializing alignments to the. sbi/. sbip format with the Goby framework. The. sbi/. sbip format is a protocol buffer format developed with methods described in  Campagne et al. [2013] , which serializes information about aligned bases for one or more samples. The record of information in the. sbi/. sbip format is a single genomic site. For the somatic models, each record stored information about a sequenced position in two different samples (germline or somatic), or three samples (mother, father, and somatic). The protobuf schema for the. sbi/. sbip format is distributed in the Goby 3. 0 repository (https://github. com/CampagneLaboratory/ goby3/goby-distribution/protobuf/BaseInformationRecords.
RNA-Seq Datasets We used B, NK, and T-cell samples from two control subjects (CFS/ME study). For each position, a different record represented a different permutation of two cell-types for a subject, so we had 12 records for each position. We introduced mutated records with the mutation simulator, as described previously, and then randomized the order of the records within the training set. The validation and test sets were created in the same way, using only NK and B cells data from one subject for each set.
For every given position, mappers which produced features about just one sample were reapplied to each sample in the assay with the results concatenated. The last mapper, which generated a comparative feature between two samples, was applied once for pair design (germline/somatic), and twice for trio design (father/child,mother/child). Mappers are implemented in the variationanalysis project available at GitHub https://github. com/CampagneLaboratory/variationanalysis.
to develop models to rank and filter somatic mutations in RNA-Seq data. Briefly, we trained a model with data from two individuals consisting of pairs of control samples (B vs T, T vs B, NK vs B, B vs NK,. ) where mutations were planted to create a semi-simulated dataset (see Methods for details). The model was trained with early stopping (see Methods). Performance was measured with the AUC: the area under the Receiver Operator Characteristic (ROC) curve. At the end of training, performance measured on the validation set was 0. 974. A test of the model on an independent test set gave an AUC of 0. 953.
Figure3. Model Refinement Cycle (A) Example of incorrect predictions on a dataset. The table is ordered by model probability (that the site is a somatic mutation) and filtered to show only false positive predictions. Manual inspection of such a table helps identify the most extreme errors made by a model. (B) A low signal to noise ratio, as in this case, is an expected source of false positives and may indicate that a model needs further training. (C) This negative example is indistinguishable from a positive example (it looks just like the simulator planted a mutation, when this was not the case). Such errors are possible if the genotype of germline samples differ at some sites (D) This negative example was wrongly classified as a mutation. Upon further investigation of the simulator code, we identified a software bug.
During the feature encoding phase, the order of bases was not consistent from sample to sample. Fixing this bug resolved such errors. (D) In a first attempt at simulation, we trained with datasets restricted to have at least 10 counts across the samples at a site. This prevented the model from learning that such sites are more difficult to predict and resulted in over-estimates of model probability. Removing filters from the construction of the training set (to include sites irrespective of their coverage) resolved this issue.
Torracinta and Campagne [2016]r exome datasets, we only created records of positions for one sample pairing (DNA extracted from the subject's blood and skin). The training, validation, and test sets each drew data from an independent subject. Mutated records were introduced in training, validation and test sets. The records of each set were shuffled with the Randomizer2 tool provided inTorracinta and Campagne [2016]. For trio exome datasets, we created records of positions for one sample trio (DNA from father's blood, mother's blood, and subject's blood). The training, validation, and test sets drew data from independent trios. Mutated records were introduced in training, validation and test sets. The records of each set were shuffled.
Objective: The aim of this study is to characterize the patterns and trends in the editorial process and features of the first decade of Pharmacy Practice, with the final goal of initiating a benchmarking process to enhance the quality of the journal. Methods: Metadata of all of the articles published from 2006 issue #3 to 2016 issue #2 were extracted from PubMed and complemented by a manual data extraction process on the full-text articles. Citations of these articles were retrieved from Web of Science (WOS), Scopus, and Google Scholar on August 15, 2016. The references from all of the articles published by Pharmacy Practice in 2015 were also extracted. International collaboration was explored with a network analysis. Results: A total of 40 issues were published in this timespan, including 349 articles, 91.
METHODS Articles published in the first decade of Pharmacy Practice were included for analysis (2006-2016). Metadata were compiled by importing a file with all of the articles indexed in PubMed using the MEDLINE format. This file contained all authors (full and abbreviated name and complete affiliation) and full references for each article (publication year, volume, issue, and pages), and the publication type was obtained through a manual search of the journal's table of contents. Editorial process dates (received, accepted, published) were extracted from PubMed and then corroborated from submission records. The editorial process duration for original research articles was computed as three different time periods: acceptance delay (time from submission to acceptance), publication delay (time from acceptance to publication); and editorial process delay (time from submission to publication).
Citations for each published article were obtained on August 15, 2016, from three different sources: the Web of Science (apps. webofknowledge. com/), Scopus (www. scopus. com), and Google Scholar (scholar. google. com/). To evaluate the patterns in the referencing practices observed in Pharmacy Practice and its potential influence in other journals' 2015 Impact Factor, all bibliographic references in articles published in 2015 were manually compiled. The citation half-life was computed as the median value of the distribution of the cited articles' publication years. Authors were retrieved from PubMed records, and their affiliations were retrieved from a manual search of original articles. For each article, the countries of the authors' affiliation were noted, and a collaboration network was created using the Gephi software (gephi.
8 A few variations in publication times were observed in Pharmacy Practice over the years. The time to acceptance increased significantly in the last several years, which may be associated with changes in the review process. The peer review process is a key element of scientific publishing. In 2013, Pharmacy Practice modified its process of selecting potential peer reviewers for a manuscript. Instead of using a closed database of individuals who offered to be reviewers, reviewers were selected from PubMed among authors of similar articles. By using this process, Pharmacy Practice ensures that the reviewers have participated in research with similar characteristics to the study that they are asked to evaluate. 9 However, this selection process consumes more time due to lower task acceptance, which may delay the entire editorial process.