dataesr/scientific-paragraphs-categorization huggingface dataset:
Reads mapping and transcript model reconstruction of canine RNA-seq samples The processing of RNA-seq reads from the mapping of the reads to the transcript model reconstruction was performed using standard bioinformatic pipeline  (48). Such genomeguided transcript model reconstruction has already been validated as for instance in the canine genome  (49). Briefly, the mapping of the canine RNA-seq reads was done using the STAR v2. 5. 0a program (50) while Cufflinks v2. 2. 1  (11)  was used to reconstruct transcript models for each sample separately using the dog genome annotation by EnsEMBL and by the Broad (49) as a guide. Finally, the cuffmerge tool, from the Cufflinks package, was used to compute a single consensus file with all reconstructed transcript models (all command lines and parameters used for each tool are also provided in Supplementary Data).
Using the gold-standard GENCODE human learning set (HL) for training, we evaluated the performance of FEELnc on 5000 lncRNAs and 5000 mRNAs of the test data set with respect to the five ORF types and multi k-mer combinations. Remarkably, we observed that 'type 1' and 'type 3' ORFs, which extract the longest ORF even in the absence of stop codon, consistently display better achievements (mean MCC = 0. 816) than 'type 0' and 'type 2' ORFs (mean MCC = 0. 67 and 0. 68, respectively) whichever combination of k-mers is considered (Supplementary Figure  S2 ). In addition, the multi k-mer strategy improves the performance of the program with a MCC performance starting at 0. 80 when only using 6-mers but reaching 0. 85 with a combination of {1, 2, 3, 6, 9, 12}-mers (with a fixed ORF type 3, Supplementary Figure  S2 ).
Fourth, we observed similar high performance with mouse GENCODE data set ML and MT (see materials) composed of 5000 mRNAs and 2000 lncRNAs where FEELnc achieves 0. 938 in sensitivity, 0. 941 in specificity and an MCC of 0. 856. Finally, among the manually curated list of 35 'well-characterized' lncR-NAs from Chen et al. (52) , FEELnc correctly classifies 33 (95%) of them as non-coding. The two discordant lncRNAs are PWRN1, which has a CPS (0. 374) just above the optimized cut-off (0. 372), and FIRRE, which exhibits the highest CPS (0. 58 when the median CPS for the 35 lncRNAs is 0. 088). This high coding potential score for FIRRE lncRNA can be explained by a large and complete ORF (546 nt, i. e. 58% of the total RNA sequence) and a high level of sequence similarity with the FAM195A protein-coding gene.
The third FEELnc module (FEELnc classifier ) formalizes the annotation of lncRNAs based on neighboring genes in order to predict lncRNA functions and RNA partners (see classes in methods). To illustrate the outcome of the FEELnc classifier , we applied it on the human EnsEMBL v83 annotation composed of 24 659 lncRNAs ('lincRNA' and 'antisense' biotypes) and 79 901 mRNA transcripts ('protein coding' biotype). For instance, FEELnc classifier anno-PAGE 7 OF 12 Nucleic Acids Research, 2017, Vol. 45, No. 8 e57 tates 5544 lncRNAs as 'intergenic antisense upstream' which correspond to divergent lincRNAs (i. e. transcribed in head to head orientation with the RNA partner) among which 28. 3% (n = 1572) are less than 1kb from their mRNA partner Transcription Start Sites (TSSs). This class directly pinpoints to lincRNAs potentially sharing a bi-directional promoter with their mRNA partners  (53, 54).
Benchmarking FEELnc and existing tools We next compared the performance of FEELnc with five state-of-the-art programs either alignment-free (CPAT, CNCI and PLEK) or alignment-based (PhyloCSF and CPC). Similarly to FEELnc, we used the balanced human learning data set (HL) composed of 5000 lncRNAs and 5000 mRNAs (see Methods) to construct the models for CPAT and PLEK (denoted CPAT train and PLEK train ). We also used default pre-built models for PLEK and CPAT although some of the transcripts from the human GEN-CODE data set test file (HT) could have been used for building these models. For all programs, performance metrics were calculated according to the HT data set. This showed that FEELnc had the highest classification power (AUC, i. e. Area Under the Curve value = 0. 97) compared to the others tools as illustrated by the ROC curves (Figure  3A ).
Accordingly, FEELnc displays the highest sensitivity (0. 923) and the second highest specificity (0. 915) among all tools while PLEK displays the highest specificity (0. 985) and precision (0. 981). In general, alignment-based methods have lower classification metrics than alignment-free programs as they usually depend on the quality of the input crossspecies alignments or the completeness of species-specific protein databases. Finally, FEELnc also shows the highest classification accuracy (0. 919), F-score (0. 919) and MCC values (0. 838) indicating that it performs well on the human GENCODE data set in comparison to other tools (Table  1 ). We further investigated the performance on the mouse data sets composed of 2000 lncRNAs and 5000 mRNAs in order to replicate the analysis in another organism using an unbalanced data set.
For this benchmark, we included the same programs except PhyloCSF due to the labor-intensive task to extract input cross-species multiple alignments. Again, FEELnc displays the highest classification accuracy, F-score and MCC (Table  2 ) while CPC shows the best specificity (0. 992) and precision (0. 996) despite a weak sensitivity (0. 744). As in human, the CPAT program performs well even if we consider both the re-trained and prebuilt models. Interestingly, CPAT used with the trained mouse model has only slightly better performance than used with the human prebuilt model. This suggests that withinspecies training achieves relatively few performance gains compared to cross-species training with a closely related model species. Finally, we assessed the computational time of each program including the time required for computing the model for training-based tools on a linux RedHat station (Intel(R) Xeon(R) CPU X5550 @ 2.
All tests were further assessed on the catalogues of lncRNAs annotated in the NONCODE database. We also compared the performance with CNCI knowing that NONCODE uses both CNCI and matching protein-coding coordinates from RefSeq database to remove all ncRNAs annotated as protein-coding. In Supple-Figure  3. FEELnc performance against coding potential tools and with shuffle, intergenic and cross-species approaches. (A) ROC curve analysis of FEELnc versus coding potential tools based on GENCODE human data set (HT). (B) Empirical cumulative distribution of FEELnc codpot feature scores with the true set of human lncRNAs ('lncRNA') in comparison with the 'shuffle' and 'intergenic' methods. (C) FEELnc codpot MCC values tested on human HT set and trained using human mRNAs and species-specific NONCODE lncRNAs (cross-species).
The x-axis represents the time of speciation between human and NONCODE species as given in  (69). Species abbreviations are the following: Atha: Arabidopsis; Btau: Cow; Cele: Nematode; Dmel: Fly; Drer: Zebrafish; Ggal: Chicken; Ggor: Gorilla; Hsap: Human; Mdom: Opossum; Mmul: Rhesus; Mmus: Mouse; Oana: Platypus; Pabe: Orangutan; Ptro: Chimpanzee; Rnor: Rat. S3 , we showed that FEELnc without annotated lncRNAs achieved good classification metrics in many species (MCCs ranging from 0. 70 for rat to 0. 903 for cow) and even better than CNCI in five diverse species (nematode, fly, gorilla, orangutan and rhesus macaque). As a third approach to model non-coding sequences, we sought to use cross-species lncRNAs for learning FEELnc model parameters. To this end, NONCODE lncRNAs from 13 diverse organisms were used to serve as a proxy for modeling human lncRNAs, and evaluation metrics were then performed on the HT data set.
As expected, FEELnc performance is negatively correlated with respect to the evolutionary distance between human and NONCODE species (Spearman rho = -0. 85; P-value = 5. 6e-05) with MCC values of 0. 374 when using Caenorhabditis elegans lncRNAs to 0. 823 with chimpanzee lncRNAs (Figure  3C  and Supplementary Table  S4 ). This probably reflects the variability in term of lncRNA sequence conservation (and thus in non-coding k-mer frequencies) between human and NON-CODE species  (35, 52). Interestingly, the shuffle strategy showed a MCC of 0. 748, which corresponds to the performance obtained with species that diverged about 100 million years ago. This indicates that it constitutes an interesting approach when no lncRNAs from closely related species are available.
Application to identify an extended catalog of canine lncRNAs Within the framework of the LUPA consortium  (29, 57) , we performed 20 whole transcriptome sequencing experiments of 16 canine tissues (Supplementary Table  S1 ). After QC, ∼1, 3 billions reads were mapped onto the CanFam3 genome assembly using the STAR mapper  (50). Cufflinks  (11)  was further used to reconstruct the transcript models in each tissue separately guided by a consensus reference annotation given by the Broad  (49)  and EnsEMBL v83  (22)  annotations (called CanFam3. 1). Then, the cuffmerge tool (Cufflinks package) merged the tissue samples files into a single GTF file containing 211 794 transcript models of more than 50 000 gene models. In order to annotate new transcribed loci, we used the FEELnc filter module to flag all transcripts which overlap exons (in sense) from the reference annotation, single exonic intergenic transcripts and transcripts with a size below 200 nt.
A total of 5523 remaining candidate transcripts were then analyzed by the FEELnc codpot module. The 'two cut-offs' option was run with a minimal specificity threshold fixed at 0. 93 in both biotypes. These cut-offs allow leveraging the number of ambiguous transcripts (TUCps) while optimizing classification specificity (see Materials and Methods). This analysis identified 3822 novel lncRNA transcripts, 477 new mRNA transcripts and 884 TUCps. Because we also aimed at developing a comprehensive annotation of novel transcript isoforms, we used FEELnc codpot , with the exact same parameters as before, on the subset of assembled models overlapping (but not included in) the current CanFam3. 1 annotation (n = 69 602) (See Supplementary Methods for details). In addition, since a novel transcript isoform could merge two or more genes from the reference annotation, we defined rules to avoid the merging of transcripts having different biotypes by removing these incompatible transcripts (see Supplementary Methods for details).
Also, the number of isoforms per gene locus has been expanded to ∼2. 2 and ∼7. 2 for lncRNAs and mRNAs, respectively. Third, using STAR and RSEM (58) programs to quantify transcripts expression levels, we found that 86% of novel lncRNAs have a TPM (transcript per million) value higher than 0. 5 in at least one of the 20 tissues highlighting a robust set of new lncRNAs. Finally, among the novel canine lncRNA genes, ∼15% are also found as non protein-coding genes in the human GENCODE annotation by using the EnsEMBL compara EPO alignments  (59). The CanFam3. 1-plus now annotates, for instance, three CAncer Susceptibility Candidates lincRNAs (CASC lin-cRNAs) such as the CASC9 lincRNA located on canine chr29:23,554,585-23,605,371 and involved in esophageal squamous cell carcinoma  (60). Other examples include the well-described MALAT1 cancer-associated lincRNA  (61)  which was considered as an unclassified non-coding transcript in CanFam3.
1 and the IFNG-AS antisense lncRNA involved in T-cell differentiation  (62). Finally, by employing the FEELnc classifier on all CanFam3. 1-plus lncRNA transcripts (see Supplementary Table  S4 ), we annotated 8209 lincRNAs. Among them, 1279 are located at a distance smaller than 5 kb and transcribed in a divergent orientation from their neighbor mRNA which could suggest potential canine bi-directional promoters  (54). For genic lncRNAs, FEELnc classifier identified 5085 antisense exonic lncRNAs as possible candidates for sense-antisense regulation by sequence complementarity  (55). The CanFam3. 1-plus annotation constitutes a new resource that will help identifying lncRNA candidates for understanding genotype to phenotype relationships.
DISCUSSION In this study, we designed a new program to identify and annotate lncRNAs called FEELnc for FlExible Extraction of Long non-coding RNAs. Using the gold-standard GENCODE annotation in human and mouse  (27) , we showed that FEELnc performs well to discriminate long non-coding versus protein-coding RNAs. Most probably, FEELnc includes predictors (multi k-mer frequencies and ORF coverage) that are general enough to cap-e57 Nucleic Acids Research, 2017, Vol. 45, No. 8 PAGE 10 OF 12 ture all lncRNAs classes whereas alignment-based methods will be biased toward misclassifying species-specific transcripts or coding transcripts that are not referenced in peptide databases. The integration of a Random Forest-based model in FEELnc contributes to the good achievements of the tool because of the intrinsic properties of randomly subsampled features thereby encompassing diverse lncRNA characteristics.
For instance, the shuffle strategy only requires species-specific protein-coding sequences and it is thus suitable for analyses without a reference genome assembly, which is still the case for many non-model organisms. Third, we expect the FEELnc classifier module to be of great interest to researchers in order to automatically annotate novel lncRNAs thus directly providing candidate pairs of lncRNA and mRNA partners to be investigated for experimental validations. For non-model organisms, we have shown that FEELnc performs similarly to CNCI although the benchmark was done on NONCODE lncRNAs that were a priori filtered by the CNCI tool. As for human and mouse where the manually curated GENCODE annotation is considered as a standard, this stressed the importance of also defining gold-standard sets of lncRNAs/mRNAs to correctly evaluate programs for 'non-model' organisms.
This is envisaged within the framework of collaborative projects such as FAANG, the Functional Annotation of Animal Genomes project  (30). Although the purpose of FEELnc is not to assemble transcripts from RNA-seq data, the program relies on the correct modeling of transcripts. For weakly expressed mRNAs, the corresponding reconstructed models might not be fulllength and thus may also introduce a bias for the FEELnc sequence length predictors. However, the benchmarking on shortened sequences (Supplementary Figure  S5 ) showed that FEELnc performs well even if 25% of either transcriptend sequences are removed. With the availability of very long reads from third generation sequencing technologies  (64) , the issue will not so much consist in annotating fulllength transcripts but rather in taking into account the high error rate inherent to these technologies, which could lead to the misannotation of the correct ORF.
Using a specific FEELnc option, preliminary tests on degraded transcript sequences showed that it raised good evaluation metrics (Sn = 0. 87, Prec = 0. 78) when the computation of multi k-mer frequencies is done on the entire transcript sequence (i. e in-dependently of the ORF annotation) and the ORF coverage is removed from the predictors (see Supplementary Table  S7 ). Finally, we illustrated the usefulness of FEELnc on the dog transcriptome for which 20 RNA-seq samples were sequenced in the frame of the LUPA consortium. The biological relevance of this expanded canine resource can be illustrated by the increased transcript and CDS sizes as well as higher exon numbers. Although this improved annotation will facilitate the identification of genotype to phenotype associations, it will still need further investigations given the plethora of expressed biotypes beside lncRNAs and mRNAs.
This highlights the potential use of FEELnc to annotate pseudogene-derived lncR-NAs  (65, 66)  for which the CPS score should be in between lncRNAs and mRNAs. Similarly, enhancer RNAs defined as non-coding transcripts derived from enhancer elements  (67, 68)  should harbor small ORFs and specific patterns of k-mers corresponding to their transcription factor binding sites, which would then be caught by FEELnc predictors. Altogether, FEELnc provides a standardized and exhaustive protocol to identify and annotate lncRNAs.
AVAILABILITY FEELnc is implemented in Perl and R (and KmerInShort in C++) and is available throw a github: https://github. com/ tderrien/FEELnc. A UCSC track hub for the extended canine annotation CanFam3. 1-plus is available using the following URL: http://tools. genouest. org/data/tderrien/canFam3. 1p/ annotation/trackhub/canfam3. 1p trackhub/hub. txt with transcripts from the original CanFam3. 1 annotation in blue, new isoforms from CanFam3. 1 genes in green and transcripts from CanFam3. 1-plus genes in red. A guideline is available in the Supplementary Data for running FEELnc analyses with or without a reference genome.
ACKNOWLEDGEMENTS The authors thank the  Cani-DNA CRB  for providing RNA samples for the project (http://dog-genetics. genouest. org), and the biobank at  University of Copenhagen. They are grateful to referring veterinarians, all dog owners and breeders for sample collection and also colleagues from the  European LUPA consortium  (www. eurolupa. org),especially  Anne-Sophie Lequarré  and  Marilou Ramos  for coordinating the program. We also thank  FEELnc users  and more particularly  Kevin Muret ,  Sandrine Lagarrigue ,  Mark Cock  and  Sarah Djebali. Sequencing was performed at the  Broad Institute's Genomics Platform. Finally, we are grateful to  Rory Johnson  and  Jacques Nicolas  for useful discussions and the  GenOuest Bioinformatics core facility  (www. genouest. org)for storing sequencing data, hosting the  Cani-DNA  website and for the use of the bioinformatic cluster to analyze the data.
FUNDING The 7th PCRD 'Health programs' LUPA consortium [FP7, GA: 201370];  French National Research Agency  in the frame of the ' Investing for the Future' program  [ ANR-11-INBS-0003  to The ' Cani-DNA ' biobank, which is part of the  CRB-Anim  infrastructure] (https://www. crb-anim. fr); European Young Investigator Award  from  European Science Foundation  as well as Consolidator Award from the  European Research Council  [to K. L. T. ].
Metagenomics data analyses from independent studies can only be compared if the analysis workflows are described in a harmonized way. In this overview, we have mapped the landscape of data standards available for the description of essential steps in metagenomics: (i) material sampling, (ii) material sequencing, (iii) data analysis, and (iv) data archiving and publishing. Taking examples from marine research, we summarize essential variables used to describe material sampling processes and sequencing procedures in a metagenomics experiment. These aspects of metagenomics dataset generation have been to some extent addressed by the scientific community, but greater awareness and adoption is still needed. We emphasize the lack of standards relating to reporting how metagenomics datasets are analysed and how the metagenomics data analysis outputs should be archived and published.
However, meaningful interpretation across studies (either through aggregation and interpretation of existing published analyses or through meta-analysis of published experimental data using a uniform method) is challenging. A number of reasons exist for this: (i) each "omic" analysis workflow is a complex process, consisting of disparate and diverse tasks, ranging from sample collection and processing to data generation and analysis, where each task has many parameters that can affect analysis outputs (e. g. , it has been shown that a major factor explaining correlations within metagenomics datasets can be DNA preparation and sequencing)  [1] ; (ii) each variable is frequently recorded in a nonstandardized way, or not recorded at all; (iii) presentation formats of the produced omics data are not unified; (iv) omics experimental data and related analysis outputs are either dispersed in several public repositories or not archived at all.
Here, we review the workflow for metagenomics data generation and analysis. Where possible, we specify essential parameters in the workflow and advise on standardized systematic reporting of these as variables. We build on the expertise of major public genomic and metagenomic resources: the European Nucleotide Archive (ENA)  [2]  and European Molecular Biology Laboratory (EMBL)-European Bioinformatics Institute Metagenomics (EMG)  [3]  at the EMBL European Bioinformatics Institute in the UK; Metagenomic Rapid Annotations using Subsystems Technology (MG-RAST)  [4]  at the Argonne National Laboratory in the USA; and the extensive knowledge bases in metagenomics available at research centers of excellence, the Universitetet i Tromsø in Norway, Genoscope in France, SB-Roscoff in France, and Consiglio Nazionale delle Ricerche in Italy.
Overview of the Metagenomics Data Model The introduction of new generation sequencing technologies has enabled even small research groups to generate large-scale sequencing data. The resultant DNA sequences and associated information are typically captured in several interconnected objects (Fig. 2 ), which represent the following concepts: r Study: information about the scope of a sequencing effort that groups together all data of the project; r Sample: information about provenance and characteristics of the sequenced samples. r Analysis: a set of outputs computed from primary sequenc- ing results, including sequence assemblies and functional and taxonomic annotations. Information associated with DNA sequence is frequently referred to as "metadata. " This includes all information described in the Study, Sample, Experiment, and Run data objects, spanning sampling context, description of sample processing, experimental design, library creation, sequencer configuration, and provenance information required for attribution and credit to comply with best scientific practice for publication in the academic literature and to inform processes around Access and Benefit Sharing.
Primary data represent, in this context, primary "raw" experimental sequence reads produced by sequencing machines. (On occasion, some basic data processing, such as quality control (filtering out of poor-quality reads, clipping of low-quality regions, etc. ), is applied to "raw" primary data, and these processed data are retained as primary; while it is preferable to retain true "raw" primary data, perhaps in addition to these processed data, it is important to apply broadly accepted processing methods and to describe these methods as part of the metadata record. ) Following this, for some metagenomics studies, the primary data are analysed directly (e. g. , 16S or 18S rRNA gene amplicon studies), while in others, they are assembled into contigs before undergoing further analysis. Regardless of the approach, the output of any computational analysis process (including assembly) on the primary data is here referred to as derived data.
We discuss derived data in more detail below, but the more harmonized the formats and validations for data and metadata objects, the more easily the generated data can be shared, discovered, re-used, and repurposed. Each metagenomics initiative has a scope, aim, and 1 or more (human) contributors in each step of the workflow, who may be distributed over a wide geographical area. It is essential to capture contextual information regarding the contributors as this supports appropriate attribution and credit and clarifies the responsible parties for each step of the workflow. Contributors to (i) material sampling, (ii) primary data generation, and (iii) derived data generation should always be clearly presented in data records. Minimum metadata checklists frequently do not specifically capture data generating or contributing institutions.