dataesr/scientific-paragraphs-categorization huggingface dataset:
TSDF method was not designed for dynamic scenes; for this reason, we reconstruct the scene separately on each frame. This results in a mesh that is not completely stable in time. The Poisson method, however, can fill in missing areas of the scene. This might be a good feature if the holes in the model are small, but would cause problems in our case. Large filled-in areas would be inaccurate and tend to flicker. While there are plenty of RGB-D camera datasets available  [18] , almost none of them use multiple sensors simultaneously in a dynamic scene. Thus, we used both a dataset created by Kuster et al. [33]  and our own recorded data. Figure  12  shows them in use. Scene A, courtesy of Kuster et al. [33] , uses Asus Xtion cameras based on structured light technology. Scene B, created by us, uses Microsoft Kinect 2 cameras based on time-of-flight technology.
Background: Although ultrahigh-throughput RNA-Sequencing has become the dominant technology for genomewide transcriptional profiling, the vast majority of RNA-Seq studies typically profile only tens of samples, and most analytical pipelines are optimized for these smaller studies. However, projects are generating ever-larger data sets comprising RNA-Seq data from hundreds or thousands of samples, often collected at multiple centers and from diverse tissues. These complex data sets present significant analytical challenges due to batch and tissue effects, but provide the opportunity to revisit the assumptions and methods that we use to preprocess, normalize, and filter RNA-Seq data critical first steps for any subsequent analysis. Results: We find that analysis of large RNA-Seq data sets requires both careful quality control and the need to account for sparsity due to the heterogeneity intrinsic in multi-group studies.
Background RNA-Seq experiments using ultrahigh-throughput sequencing-by-synthesis technologies were first performed in 2008 and have since been used for large-scale transcriptome analysis and transcript discovery in mammalian genomes  [1] [2] [3]. Although hundreds of published studies have used this technology to assay gene expression, the majority of studies consist of relatively small numbers of samples. There are many widely used methods for normalization and analysis of expression data from modest numbers of relatively homogeneous samples  [4] [5] [6]. The workflow for RNA-Seq typically includes basic quality control on the raw reads and alignment of those reads to a particular reference database to extract sequence read counts for each feature-gene, exon, or transcript-being assayed  [7].
The resulting features-by-samples matrix is then filtered, normalized and analyzed to identify features that are differentially expressed between phenotypes or conditions. Functional enrichment analysis is then performed on these features  [7]. There are now many large cohort studies, including the Genotype-Tissue Expression project (GTEx) and The Cancer Genome Atlas (TCGA) that have generated transcriptomic data on large populations and across multiple tissues or conditions to study patterns of gene expression  [8, 9]. The GTEx project is collecting genome-wide germline SNP data and gene expression data from an array of different tissues on a large cohort of research subjects. GTEx release version 6. 0 sampled over 550 donors with phenotypic information representing 9590 RNA-Seq assays performed on 54 conditions (51 tissues and three derived cell lines).
Established methods for RNA-Seq analysis can be used to make direct comparisons of gene expression profiles between phenotypic groups within a tissue. However, they are not well suited for comparisons across multiple, diverse tissues, of which each exhibit a combination of commonly expressed and tissuespecific genes. This characteristic is a feature that confounds most normalization methods, which generally assume the majority of expressed transcripts are common across samples. Widely-used normalization methods make assumptions that are valid only in fairly consistent samples and assume that most genes are not differentially expressed, that housekeeping genes are expressed at equivalent rates, or that the expression distributions vary only slightly due to technology  [4] [5] [6]. In large heterogeneous data sets, such as GTEx, these biological assumptions are violated.
Implementation YARN, shown in Fig. 1 , is instantiated as a Bioconductor (BioC version 3. 4+) R package. YARN is built on top of the Biobase Bioconductor package that defines the ExpressionSet class, a S4 object class structure. Using this class structure, multiple helper functions were designed to help 1) filter poor quality samples -(checkMi-sAnnotation), 2) merge samples derived from similar sources (in our case, different sampling regions of the "same" tissue) for increased power (checkTissuesTo-Merge), 3) filter genes while preserving tissue or group specificity -(filterLowGenes, filterGenes, filterMissing-Genes), 4) normalize while accounting for global differences in tissue distribution (normalizeTissueAware), and 5) visualize the structure of the data (plotDensity, plotHeatmap, plotCMDS).
Finally, the data were normalized using a global count distribution method to support cross-tissue comparison while minimizing within-group variability learned that this individual had undergone sexreassignment surgery providing evidence that this quality check had appropriately flagged an individual who was genetically male. The PCA plot in the first step of Fig. 1  and the collected set in Additional file 2: Figure  S1  were produced using the functions checkMisAnnotation and plotCMDS in the YARN package. While the majority of variation in the GTEx data was present in the first two components and clearly showed separation between the sexes, as a rule of thumb one should check components until 90% of the variation has been captured in the PCs. The plotCMDS function is structured to return as many components as requested for pairwise scatterplots, and users can adjust the number of PCs to capture the desired level of variation.
We tested these filtering strategies and compared the results to unfiltered data by assessing differential expression between whole blood (n = 444) and lung (n = 360), two tissues with relatively large numbers of samples, and for which we expect to find many differentially expressed genes (Additional file 1). Following filtering, we normalized the data using qsmooth and used voom, from Bioconductor R package limma  [20] , to identify differentially expressed genes. We found the smallest fraction of differentially expressed genes in the unfiltered data set (54%). The tissue-agnostic filtering identified the largest fraction (80%), but many of the differentially expressed genes were noncoding genes. The tissue-aware filtered data yielded an intermediate fraction of differentially expressed genes (69%), but the greatest number of differentially expressed protein coding genes.
Consequently, we chose to use tissue-aware filtering as it provides for identification of tissue-specific, differentially expressed genes (Additional file 1). Using this filtering with the GTEx data reduced the number of mapped genes from 55,003 to 30,333 genes that were advanced to the next step in the pipeline. Figure  3  shows examples of genes related to tissuespecific function or disease that would have been lost using the tissue-agnostic approach that are retained by the tissue-aware filtering. MUC7 (Fig. 3a ) is overexpressed in the minor salivary gland and has been associated with asthma. REG3A (Fig. 3b ) is overexpressed in pancreas and small intestine and has been associated with cystic fibrosis and pancreatitis. AHSG (Fig. 3c ) is overexpressed in the liver and has been associated with uremia and liver cirrhosis.
When the distributional shapes are dissimilar across tissues, the reference is not representative of any particular tissue and scaling of quantiles is dependent on the largest tissue's distribution. In GTEx, we wanted to use a single normalization method for all tissues. Here, with a very diverse set of tissues, the assumptions underlying quantile normalization clearly break down (Additional file 4: Figure  S3 ). The qsmooth  [27]  normalization method is a generalization of quantile normalization that normalizes all samples together but relaxes the assumption that the statistical count distribution should be similar across all samples and instead assumes only that it is similar in each phenotypic group (as one might expect for different tissues in GTEx). We used qsmooth to normalize the GTEx expression data where phenotypic groups were determined using the 38 "merged" tissues that resulted from our quality control assessment.
We compared the effects of "full" quantile normalization to the "tissue-specific" strategy implemented in qsmooth. We observed much larger root mean squared errors (RMSE) using an all-sample reference ("full" quantile normalization) than we saw using qsmooth's tissuespecific references (Fig. 4 ). The root mean square error estimates the divergence of transcriptome distributions from the assumed transcriptome reference distribution. The more the RMSE varies by tissue, the larger the number of tissue-specific counts. Figure  4  suggests that global quantile normalization disproportionately weights and biases tissue-specific transcripts based on other tissues' proportion of zeros in the distribution and tissue sample size (Additional file 1, Additional file 7: Figure  S5 ). Both qsmooth (smooth quantile normalization) and full quantile normalization (over every specific tissue) are implemented in YARN's normalizeTissueAware function.
Discussion Large-scale transcriptional studies, such as GTEx, present unique opportunities to compare expression in a relatively large population and across a large number of tissues. However, as with all analyses of gene expression, it requires careful quality assessment, gene filtering, and normalization if meaningful conclusions are to drawn from the data. We developed a simple and robust software pipeline, YARN, to allow us to perform quality control assessment of the metadata associated with a large, heterogeneous data sets such as the collection of RNA-Seq assays that are available as part of the GTEx v6 release. YARN was designed to process RNA-Seq data to allow comparisons between diverse conditions and consists of four basic steps: quality assessment filtering to remove questionable samples, comparison of "related" sample groups to merge them or split them into separate groups, filtering genes that have too few counts while preserving tissue-specific genes, and normalizing the data.
For each step, YARN contains multiple options that allow user to adapt the pipeline for their use. In our analysis of GTEx v6 data, we began by using PCoA to filter samples based on misidentification by sex. We then used PCoA to compare samples from the same general body site so as to merge those that were indistinguishable. Next, we used a tissue-aware filtering method to retain genes that were expressed in one or a small number of tissues, while eliminating those in too few samples to perform a reliable normalization. Finally, we used qsmooth to perform a tissue-aware normalization (Additional file 1). This pipeline allowed us to identify one individual who was misidentified by sex, to reduce the 53 sampling site conditions to 38 non-overlapping tissues, eliminated 24,670 genes for which there was insufficient data to perform a reasonable normalization or subsequent analysis, and to produce normalized data for 30,333 genes in 9435 samples distributed across 38 tissues.
Conclusions YARN is a flexible software pipeline designed to address a problem that is becoming increasingly challenging-that of normalizing increasingly large, complex, heterogeneous data sets, often consisting of many samples representing many different physical states, perturbations, or phenotype groups. YARN is implemented as a Bioconductor package and is available under the open source GPL v3 license at http://www. bioconductor. org/ packages/yarn. The workflow includes numerous quantitative options for filtering as well as tools for visual inspection of data to allow users to understand the distributional and other characteristics of the data. The Bioconductor vignette includes sample skin data from GTEx that can be used to work through as an example analysis. Example code to reproduce the figures in this manuscript is available through GitHub at: https://github.
Funding This work was supported by grants from the  US National institutes of Health , including grants from the  National Heart, Lung, and Blood Institute  ( 5P01HL105339 ,  5R01HL111759 ,  5P01HL114501 ,  K25HL133599 ), the  National Cancer Institute  ( 5P50CA127003 ,  1R35CA197449 ,  1U01CA190234 ,  5P30CA006516 ,  P50CA165962 ), the  National Institute of Allergy and Infectious Disease  ( 5R01AI099204 ), and the  Charles A. King Postdoctoral Research Fellowship Program ,  Sara Elizabeth O'Brien ,  Bank of America ,  N. A. , Co-Trustees. Additional funding was provided through a grant from the  NVIDIA foundation.
Availability and requirements Project name: Yet Another RNA Normalization software pipeline (YARN). Project home page: http://bioconductor. org/packages/ yarn Operating system(s): Platform independent. Programming language: R. Other requirements: Dependencies: Biobase. Imports: biomaRt, downloader, edgeR, gplots, graphics, limma, matrixStats, preprocessCore, readr, RColorBrewer, stats, quantro. Suggests: knitr, rmarkdown, testthat (> = 0. 8). License: GPLv3. Any restrictions to use by non-academics: None. Availability of data and material YARN is implemented as a Bioconductor package and is available under the open source GPL v3 license at http://www. bioconductor. org/packages/yarn. Example code to reproduce the figures in this manuscript is available through GitHub at: https://github. com/QuackenbushLab/normFigures.
MSPminer relies on a new robust measure of proportionality coupled with an empirical classifier to group and distinguish not only species core genes but accessory genes also. Applied to a large scale metagenomic dataset, MSPminer successfully delineates in a few hours the gene repertoires of 1 661 microbial species with similar specificity and higher sensitivity than existing tools. The taxonomic annotation of MSPs reveals microorganisms hitherto unknown and brings coherence in the nomenclature of the species of the human gut microbiota. The provided MSPs can be readily used for taxonomic profiling and biomarkers discovery in human gut metagenomic samples. In addition, MSPminer can be applied on gene count tables from other ecosystems to perform similar analyses.
Second, although species of public health interest (e. g. Escherichia coli, Salmonella enterica or Clostridium difficile) are represented by hundreds or even thousands of strains in genome databases, only few strains are available for the great majority of commensal species. Consequently, accessory genes associated with microbial phenotypic traits may be missing in gene repertoires constructed from reference genomes. Metagenomic assembly where overlapping reads are merged into longer sequences called contigs is a powerful reference-free technique for overcoming the limitations of reference-based methods. However, assembly remains a computationally challenging task and despite the many dedicated tools proposed, the process only recovers incomplete genomes scattered in multiple contigs  (Sczyrba et al.
In addition, it is not suitable for detecting clusters shared between several species. To overcome these limitations, we have developed MSPminer, the first tool that discovers, delineates and structures Metagenomic Species Pangenomes (MSPs) from large-scale shotgun metagenomics datasets without referring to genomes from isolated strains. MSPminer presents several significant improvements over existing methods. First, it relies on a robust measure of proportionality for the detection of co-abundant but not necessarily co-occurring genes as expected for non-core genes. Second, genes grouped in a MSP are empirically classified as core, accessory and shared genes. To illustrate its usefulness, we applied MSPminer to the largest publicly available gene abundance table which is composed of 9. 9M genes quantified in 1 267 human stool samples  (Li et al.
Simulated dataset For evaluation purposes, we generated abundance tables simulating the counts of genes from a single virtual species. The pan-genome of this species consisted in 1 000 core genes detected in all strains and 6 000 accessory genes present only in some of them. Gene lengths were randomly drawn between 100 bp and 5 000 bp. The prevalence of accessory genes was randomly drawn between 2. 5% and 99. 5%. In a first simulation, 200 samples containing each a single strain of the species were generated. The sequencing coverage of a strain in a sample was drawn from a uniform law (min=0. 6, max=20) and read length was set to 100 bp. In a given sample, the theoretical number of reads mapped on a gene was calculated according to the gene length, the strain coverage and the presence or not of the gene in the strain.
Comparison to the Canopy clustering algorithm MSPminer was compared to the Canopy clustering algorithm  (Nielsen et al. , 2014)  which is the only gene binning tool publicly available. Both tools were applied to the metagenomic dataset described above using default parameters (Supplementary Methods). In total, MSPminer grouped 17. 8% more genes than Canopy (3 288 928 vs 2 704 552) although MSPminer had a more stringent gene selection criterion (6 971 229 vs 7 304 439 genes processed). Both tools had a very high precision (mean > 98%) but MSPminer brought a significant gain in sensitivity (median: 77% vs 62%) (Supplementary Table  4 ). Remarkably, Canopy produced more objects with at least 150 genes than MSPminer (2 010 CAGs vs 1 661 MSPs) as it splits some species (e. g. E. coli) into multiple clusters.
Taxonomy 642 MSPs (38. 7%) could be annotated at species level, 315 (19. 0%) at genus level, 525 (31. 6%) at a higher taxonomic level from family to superkingdom and the remaining 179 (10. 8%) could not be annotated, indicating that a majority of MSPs correspond to species not represented in GenBank (Supplementary Fig. 5  and Supplementary Table  3. C). Among the annotated MSPs, one corresponded to Homo sapiens, 4 were unicellular eukaryotes of the genus Blastocystis, 8 were Archaea and the remaining 99% were Bacteria represented predominantly by the phyla Firmicutes (1 016 MSPs), Bacteroidetes (263 MSPs), Proteobacteria (94 MSPs) and Actinobacteria (46 MSPs). Among the 642 MSPs annotated at species level, 304 corresponded to well-defined species and 338 matched genomes with imprecise taxonomy  (i.
Parameters impacting the quality of the MSPs The quality of the MSPs is impacted by the upstream steps required for generating the count matrix, as well as by the biological and ecological characteristics of the dataset. At the sequencing level, the number of reads (sequencing depth) generated for each sample impacts the detection and coverage of subdominant species, while read length affects the quality of the assembly and the ability to assign a read to a gene without ambiguity. At the bioinformatics level, assembly, gene prediction, gene redundancy removal, mapping and counting require expertise to select the most appropriate strategies, tools and parameters. Indeed, assemblers returning chimeric contigs which combine sequences from highly related species, inaccurate predictors generating truncated or merged genes, redundancy removal with a common threshold for all genes (95% of nucleotide identity) lead to genes of variable quality in catalogues.
Reactive systems can be modeled with various kinds of automata, such as Input Output Symbolic Transition Systems (IOSTS). Symbolic execution (SE) applied to IOSTS allows computing constraints associated to IOSTS path executions (path conditions). In this context, generating test cases amounts to finding numerical input values satisfying such constraints using solvers. This paper explores the case where IOSTS models contain functions which are outside of the scope of such solvers. We propose to use function summaries which are logical formulas built from concrete values describing some representative input/output data tuples of the function. We define algorithmic strategies to solve path conditions including such functions based on techniques using and enriching function summaries. Our method has been implemented within the Diversity tool and has been applied to several examples.
A function summary is a logical formula that can be computed from a partial knowledge of the function, represented as a table containing some representative tuples of inputs/output data. Those tuples are obtained by (concretely) executing the function on a set of inputs, either produced randomly or given by the programmer. They may also result from pre-existing unit test campaigns. Path conditions are then computed based on a joint analysis of the guards occurring in executed transitions and of the function summaries associated to external functions called in those transitions. A drawback of such an approach is of course that the tables might be too incomplete to provide input/output data to follow a given model path, meaning that their corresponding summaries are too restrictive to follow the path.
Implementation DIVERSITY tool: DIVERSITY  [3, 14] , is a multi-purpose and customizable platform for formal analysis based on symbolic execution (test generation, proof, deadlock search, etc. ) that is on its way to becoming an Eclipse open-source project  5. DIVERSITY generates a symbolic tree (for a fixed maximal height) by simulating the system specification with input symbols rather than concrete values for data. Test inputs are computed by solving the path conditions. For that purpose, DIVERSITY integrates solvers such as Z3 6  , and CV C4  7. We have implemented the SolveT ab algorithm 1 and the heuristics described in Section 4 as additional Formal Analysis Module (FAM) for checking satisfiability of path conditions within DIVERSITY. By default, constraints involving external functions are left uninterpreted (i.
Complex phenotypes, such as lipid accumulation, result from cooperativity between regulators and the integration of multiscale information. However, the elucidation of such regulatory programs by experimental approaches may be challenging, particularly in context-specific conditions. In particular, we know very little about the regulators of lipid accumulation in the oleaginous yeast of industrial interest Yarrowia lipolytica. This lack of knowledge limits the development of this yeast as an industrial platform, due to the time-consuming and costly laboratory efforts required to design strains with the desired phenotypes. In this study, we aimed to identify context-specific regulators and mechanisms, to guide explorations of the regulation of lipid accumulation in Y. lipolytica. Using gene regulatory network inference, and considering the expression of 6539 genes over 26 time points from GSE35447 for biolipid production and a list of 151 transcription factors, we reconstructed a gene regulatory network comprising 111 transcription factors, 4451 target genes and 17048 regulatory interactions (YL-GRN-1) supported by evidence of protein-protein interactions.