dataesr/scientific-paragraphs-categorization huggingface dataset:
Because of these exotic features the proposed algorithm was implemented in Matlab. It is an instance of sequential quadratic programming (SQP)  [22, Chapter 18]  thus possesses all the convergence properties of that class of algorithms. An objective function f and the vector of optimization variables p ∈ R m are introduced. The variables may be subject to n e linear equality constraints: ∀i ∈ 1, n e , E i (p) = 0 ( 44 ) And n c convex inequality constraints: ∀ j ∈ 1, n c , C j (p) ≤ 0 (45) For a starting point p 0 , the function's value f (p 0 ), the gradient ∇ f (p 0 ) and the Hessian ∇ 2 f (p 0 ) are computed. A surrogate function f approximating f around p 0 is introduced: f (p) ≈ f (p) = f (p 0 ) + ∇ f (p 0 ) (p -p 0 ) + 1 2 (p -p 0 ) T ∇ 2 f (p 0 ) (p -p 0 ) (46) If the Hessian is positive definite, a line search method is used  [22, Chapter 3].
A typical workflow includes successive steps: (i) OTU (Operational Taxonomic Unit) picking (dereplication, denoising, chimera filtering and clustering)  [6] , (ii) OTU quantification in each sample and (iii) OTU annotating with respect to a reference taxonomic database. This process may require substantial computational resources depending on both the number of samples involved and the sequencing depth. Several methods are currently available to complete these tasks, such as Mothur  [7] , Usearch  [8] , DADA2  [9]  or Vsearch  [10]. The popular application Qiime  [11]  simplifies these tasks (i to iii) and visualizations by providing a python-integrated environment. Schematically, once data processing is over, both a contingency table and a taxonomic table are obtained. They contain the abundance of OTUs in the different samples and the taxonomic annotations of OTUs, respectively.
SHAMAN is an all-inclusive approach to estimate the composition and abundance of OTUs, based on raw sequencing data, and to perform statistical analysis of processed files. First, the user can submit raw data in FASTQ format and define the parameters of the bioinformatic workflow. The output returns, a BIOM file for each database used as a reference for annotation, a phylogenetic tree in Newick format as well as FASTA-formatted sequences of all OTUs that were identified. The second step consists in performing statistical analysis. The user has to provide a "target" file that associates each sample with one or several explanatory variables. These variables are automatically detected in the target file. An automatic filtering of the contingency matrix of OTUs may be activated in order to remove features with low frequency.
Material and method SHAMAN is implemented in R using the shiny-dashboard framework. The application is divided into three main components (Fig. 1 ): a bioinformatic workflow to process the raw FASTQ-formatted sequences, a statistical workflow to normalize and further analyse data, as well as a visualization platform. Users may run each component of the workflow independently or run the whole process from raw FASTQ data to visualization. SHAMAN provides, for each component, scores and figures that summarize the quality of data processing. When installed with Docker, SHAMAN has low computing resource requirements with a minimum 1Ghz processor, 1Gbyte of ram memory and 3. 4Gbytes of disk.
Bioinformatic workflow in SHAMAN The metataxonomic pipeline implemented in SHAMAN relies on the Galaxy platform  [26]. All calculations are remotely done on galaxy. pasteur. fr; this process is transparent to the user. The pipeline flows in the following manner: 1 Optional filtering of reads. It is worth noticing that previous studies, e. g. carried out on mosquito microbiota  [27] , showed that some non-annotated OTUs turned to be sequences of the host organism. To overcome such issues, the user can optionally filter out reads that align with the host genome and the PhiX174 genome Fig. 1  SHAMAN workflow. SHAMAN can start from raw reads or from processed data. In this last case, it needs at least three tables, the count matrix, the annotation table and the metadata which can either be provided into three different files or by using the BIOM format.
The user can then select the variables of interest and add some batch effects. Then, contrast vectors can easily be defined and interactive visualizations are available (used as a control in Illumina sequencers). The latter task is performed with Bowtie2 v2. 2. 6  [28]. 2 Quality of reads is checked with AlienTrimmer  [29]  v0. 4. 0, a software for trimming off contaminant sequences and clipping. 3 Paired-end reads are then merged with Pear  [30]  v0. 9. 10. 1. 4 OTU picking, taxonomic annotation and OTU quantification are performed using Vsearch  [10]  v2. 3. 4. 0, a software that was shown to be both accurate and efficient  [6, 31]. The OTU picking process consists in five steps, i. e. , dereplication, singleton removal, chimera detection, clustering and alignment. It follows the approach and default parameters previously described by the Uparse pipeline  [8].
Input amplicons are aligned against the set of detected OTUs to create a contingency table that contains the number of amplicons assigned to each OTU. This step aims at refining OTU counts by including singletons that correspond to sequences with a reasonable amount of sequencing errors (i. e. , <3%). 5 The taxonomic annotation of OTUs is performed based on various databases, i. e. , with SILVA  [32]  rev. 132 SSU (for 16S, 18S rRNA genes) and LSU (for 23S and 28S rRNA genes), Greengenes  [33]  (for 16S, 18S rRNA genes) and Underhill rev. 1. 6. 1  [34] , Unite rev. 8. 0  [35]  and Findley  [36]  for ITS rRNA sequences. These databases are kept up-to-date every two month with biomaj. pasteur. fr. 6 OTU annotations are filtered according to their identity with the reference  [37]. Phylum annotations are kept when the identity between the OTU sequence and the reference sequence is ≥ 75%, ≥ 78.
5% for classes, ≥ 82% for orders, ≥ 86. 5% for families, ≥ 94. 5% for genera and ≥ 98% for species. In addition, a taxonomic inference based on a naive Bayesian approach, RDP classifier  [38]  v2. 12, is systematically provided. By default, RDP annotations are included whenever the annotation probability is ≥ 0. 5. All the above-mentioned thresholds may be tuned by the user. 7 A phylogenetic analysis of OTUs is provided: multiple alignments are obtained with Mafft  [39]  v7. 273. 1, filtering of regions that are insufficiently conserved is made using BMGE  [40]  v1. 12 and finally, FastTree  [41]  v2. 1. 9 is used to infer the phylogenetic tree. Based on the latter tree, a Unifrac distance  [42]  may be computed in SHAMAN to compare microbial communities. The outcomes of the overall workflow are stored in several files: a BIOM file (per reference database), a phylogenetic tree as well as a summary file specifying the number of elements passing the different steps of the workflow.
Statistical workflow in SHAMAN The statistical analysis in SHAMAN is based on DESeq2 which is a method to model OTU counts with a negative binomial distribution. It is known as one of the most accurate approach to detect differentially abundant bacteria in metagenomic data  [17, 18]. Relying on robust estimates of variation in OTUs, the DESeq2 method has suitable performances even with datasets characterized by a relatively low number of observations per group (together with a high number of OTUs). This method typically requires the following input files: a contingency table, a taxonomic table and a target file describing the experimental design. These data are processed to generate a meta-table that assign to each OTU, a taxonomic annotation and a raw count per sample.
Normalization Normalization of the raw counts is one of the key issues when analyzing microbiome experiments. The uniformity of the sequencing depth is affected by sample preparation and dye effects  [43]. Normalizing data is therefore expected to increase the accuracy of comparisons. It is done by adjusting the abundance of OTUs across samples. Four different normalization methods are currently implemented in SHAMAN. For the sake of consistency, all of these methods are applied at the OTU level. A first method is the relative log expression (RLE) normalization and is implemented in the DESeq2 package. It consists in calculating a size factor for each sample, i. e. , a multiplication factor that increases or decreases the OTU counts in samples. It is defined as the median ratio between a given count and the geometric mean of the corresponding OTU.
The size factor of sample j is denoted by s j. s (1) j = median i c ij k∈S i c ik 1/n i , ( 1 ) s (2) j = w. median i c ij k∈S i c ik 1/n i , ( 2 ) where S i stands for the subset of samples with non null values for the feature j and n i is the size of this subset. The function w. median corresponds to a weighted median. An alternative normalization technique is the total counts  [45]  which is convenient for highly unbalanced OTU distribution across samples. Using a simulation-based approach, we addressed the question of the performance of the non-null and the weighted non-null normalization techniques when the matrix sparsity and the number of observations increase. We compared these new methods to those normally performed with DESeq2 and Phyloseq. To do so, we normalized 500 simulated matrices using the function makeExampleDESeqDataSet of DESeq2 with varied sparsity levels (i.
Both a guided mode and an expert mode are available in SHAMAN. In the guided mode, the user specifies the groups to be compared using a dropdown menu. This mode is only available for DESeq2 v1. 6. 3 which is implemented in the DESeq2shaman package (https://github. com/aghozlane/DESeq2shaman). In advanced comparisons, the user may define a contrast vector by specifying coefficients (e. g. , -1, 0, 1) assigned to each variable.
User case datasets To illustrate how SHAMAN operates, we analysed two datasets. The first dataset originates from a mock sequencing we performed on purpose (a standard practice when assessing metagenomics methods). The second dataset is publicly available and originates from a published study (the afribiota dataset,  [50] ). The latter dataset was collected to perform a typical differential analysis (i. e. , a very common approach in metagenomics). In both analyses, we submitted the raw FASTQ files and provided a target file containing sample information (needed for statistical analysis).
Zymo mock dataset The mock sequencing (EBI ENA code PRJEB33737) of the ZymoBIOMICS TM Microbial Community DNA was performed with an Illumina MiSeq resulting in 12 samples of 257,000 ±85,000 (mean ± SD) sequences of 300-base-long paired-end reads. The composition of the Zymo mock community is known and is composed with 8 phylogenetically distant bacterial strains, 3 of which are gram-negative and 5 of which are gram-positive. DNA of two yeast strains that are normally present in this community were not amplified. Genomic DNA from each bacterial strain was mixed in equimolar proportions (https:// www. zymoresearch. com/zymobiomics-community-standard). We compared the impact of both the number of amplification cycles (25 and 30 cycles) and the amount of DNA loaded in the flow cell (0. 5ng and 1ng), on the observed microbial abundance.
Afribiota dataset The second dataset included 541 samples of microbial communities in stunted children aged 2-5y living in sub-Saharan Africa (EBI ENA code PRJEB27868)  [50]. Three groups (nutritional status) of individuals were considered: NN=non stunted, MCM=moderately stunted, MCS=severely stunted. samples originated from the small intestine fluids (gastric and duodenal) and feces. The authors performed the bioinformatic treatment with QIIME framework and the statistical analysis with several R packages including Phyloseq for the normalization and DESeq2 for the differential analysis. Using SHAMAN, raw reads were filtered against Human HG38 and PhiX174 genomes. A total of 2386 OTUs were calculated and 76% were annotated with SILVA database at genus level. The sparsity rate of the contingency table was high with 0.
Benchmarking Last, we compared the running time performance of SHAMAN with five other web applications for metataxonomy studies (ASaiM, FROGS, MetaDEGalaxy and Qiita). For each web interface, we submitted the raw sequencing reads of the Zymo mock dataset and estimated the time to generate a BIOM containing the contingency matrix and OTU annotation. Qiita and MetaDEGalaxy were used remotely, FROGS was installed on galaxy. pasteur. fr and ASAIM was installed on Mac book pro with a core i7 6cpu with 32Gbytes of ram.
User cases Data analysis in SHAMAN may easily be done by users who are not familiar with command-line analyses. In this paper we analyzed two datasets. In the analysis of the Zymo mock dataset the 8 bacterial strains present in the samples were suitably detected (Fig. 2 ). We then defined a statistical model to test the effects of the two varying experimental factors: the amount of DNA and the number of amplification cycles and the interaction between th ese variables. The statistical comparison showed a stronger impact of the number of amplification cycles as compared to the amount of DNA. While we found no differential features between 0. 5 ng and 1 ng DNA for each group of number of cycle (25 and 30 cycles), the comparison of the number of amplification cycles within each group of DNA amount revealed a significant impact on the observed abundance of mock bacteria (Tables  S1, S2 ).
These results are in agreement with previous studies that presented the PCR-induced bias on similar mixtures  [51, 52]. Regarding the analysis of the second dataset with SHAMAN, overall our results were highly consistent with those of Vonaesch et al. [50]. We detected a significant change in the community composition between gastric and duodenal samples as compared to feces samples at Genus level (Fig. 3a ) (PERMANOVA, P=0. 001). The most abundant genera were reported in Fig. S3. α-Diversity was not affected by stunting (Fig. 3b ). We looked for a distinct signature of stunting in the feces. In the volcano plot (Fig. 3c ), we represent genera with differential abundance between stunt samples as compared to non-stunt (complete list available in Table  S3 ). Twelve microbial taxa, corresponding to members of the oropharyngeal core microbiota, were over-represented in feces samples of stunted children as compared to non-stunted children.
Any results in SHAMAN may be cross-checked with a quantification or an annotation performed at an earlier stage. Comparison of computation time revealed that SHAMAN was faster than the other five web application to process the data of Zymo mock dataset. FROGS and QIITA are also convenient solutions for data processing since the whole OTU processing was performed in few hours. In both case, they provided an accurate description of the community as the 8 main communities composing the mock were correctly detected. On Table  1  Mapping SHAMAN among the other web interface for metataxonomic analysis the other side, the computation time obtained with ASaiM and MetaDEGalaxy appeared much longer as we could not obtain a BIOM file after several days of calculation. Results obtained with Frogs and Qiita are reported on figshare (10.
Conclusions SHAMAN enables users to lead most of the classical metagenomics approaches. It also makes use of statistical analyses to provide support to each data visualization. The possibility to deploy SHAMAN locally constitutes an important feature when the data cannot be submitted on servers for privacy issues or because of insufficient internet access. SHAMAN also simplifies the access to open computational facilities, making a careful use of the dedicated server, galaxy. pasteur. fr. Currently SHAMAN is limited to metataxonomic analyses. In a close future, we plan to extend our application to whole genome analysis, notably by using of microbial gene catalogs. Several catalogs are currently available to study human, mouse, cow, as well as ocean microbial diversity. A perspective will be to combine these results with metataxonomic data, and to perform integrative analyses.
Availability and requirements Project name: SHAMAN Project home page: http://shaman. pasteur. fr, https://github. com/aghozlane/shaman Operating system: Platform independent Programming language: R Other requirements: Python 3 License: GNU GPL V3 Any restrictions to use by non-academics: No Availability of data and materials Sequence reads of Zymo Mock have been deposited in the European Nucleotide Archive, https://www. ebi. ac. uk/ena/ accession no. PRJEB33737. The datasets generated, analysed during the current study and the simulation script of the supplementary figure 2 are available in the repository: 10. 6084/m9. figshare. 11815860.
To our knowledge, neither this tool, nor any other existing tool, supports S. cerevisiae population genomic datasets. Here we developed and describe CRISpy-pop as a python-based (Van Rossum and Drake 2009) web application for the design of CRISPR/Cas9 sgRNAs for genetic modifications on populations of strains. CRISpy-pop incorporates popular diverse strain sets of S. cerevisiae from recent population genomic studies  (Peter et al. 2018; Sardi et al. 2018 ) and uses the existing tools sgRNA Scorer 2. 0 and Cas-OFFInder to assess the strain coverages of sgRNAs, predict their activities, and determine their off-target potentials. As a proof of principle, here we use CRISpy-pop to design ade2 knockout mutants and accurately predict which strains can be targeted by which sgRNAs. CRISpy-pop fills a needed niche in functional and population genomic research.
Results and Discussion Population-level variation in sgRNA target sites S. cerevisiae is a useful genetic model system and bioengineering chassis due to its wellstudied genome and ease of genetic manipulation. With growing population genomic datasets, functional investigations with CRISPR/Cas9 tools can now be extended beyond traditional laboratory strains, but variation in sgRNA target sites can still limit portability. To explore variation within the 1011-strain set, we calculated the total number of genomes that could be targeted by each sgRNA (Figure  2 ). The total number of sgRNAs designed was 658,304. Only 73,495 of the sgRNAs have perfect matches in all 1011 genomes, while 584,809 sgRNAs are predicted to target some other fraction of the genomes. Thus, randomly picking a sgRNA designed against the S288C reference genome would be unlikely to target all strains of potential interest.
sgRNA selection using CRISpy-pop To validate the strain coverage function of CRISpy-pop, we designed multiple sgRNAs targeting the gene ADE2 with varying predicted strain coverage to create ade2 knockout mutants in the Sardi et al. strain set (Figure  3 ). This strain set was chosen because we had access to the strains, but the 1011-strain population genomic data dataset  (Peter et al. 2018)  was also searched to compare relative strain coverage predictions for selected sgRNAs. Specifically, we selected two sgRNAs predicted to target all 167 strains (high-coverage sgRNAs) and two sgRNAs predicted to target only 42 of the 167 strains (low-coverage sgRNAs). The two high-coverage sgRNAs, H1 and H2, had activity scores of 1. 341 and 0. 426, respectively. The two low-coverage sgRNAs, L1 and L2, had activity scores of 2.
569 and 2. 050, respectively. None of the sgRNAs selected had any offsite matches or human hits. To determine whether the high-coverage guides also had high strain coverage within the previously published 1011-strain population genomic dataset we reran the search with the same criteria on this dataset. H1 and H2 were also predicted to cut the vast majority of the 1011-strain set, targeting 905 and 910 genomes, respectively.
Conclusions In summary, CRISpy-pop is a powerful and flexible design tool for planning and executing CRISPR/Cas9-driven genetic modifications on individual strains or large panels of strains. CRISpy-pop can continue be expanded to support new genomes as more data become available. The ability to target different PAM sites allows potential to use or screen for other Cas systems. It correctly predicts which strains can be targeted by which sgRNAs, as well as the activities of sgRNAs. Offsite targets, including a biosafety feature that scans for potential human genome binding, can be easily avoided with CRISpy-pop. This unique combination of features and its user-friendly web interface make CRISpy-pop ideal for designing experiments in diverse populations used for genetic engineering.
Acknowledgments We thank  Kevin S. Myers  for beta testing and feedback,  Maria Sardi  for providing datasets, and the  University of Wisconsin Biotechnology Center DNA Sequencing Facility  for providing Sanger-sequencing facilities and services. This material is based upon work supported in part by the  Great Lakes Bioenergy Research Center, U. S. Department of Energy, Office of Science, Office of Biological and Environmental Research  under Award Number  DE-SC0018409 ; the  National Science Foundation  under Grant No. DEB-1442148 ;  USDA National Institute of Food and Agriculture (Hatch Project   1020204 ). C. T. H. is a  Pew Scholar  in the  Biomedical Sciences and a H. I. Romnes Faculty Fellow , supported by the  Pew Charitable Trusts and Office of the Vice Chancellor for Research and Graduate Education  with funding from the  Wisconsin Alumni Research Foundation , respectively.
Compositional analyses have long been used to determine the geological sources of artefacts. Geochemical "fingerprinting" of artefacts and sources is the most effective way to reconstruct strategies of raw material and artefact procurement, exchange or interaction systems, and mobility patterns during prehistory. The efficacy and popularity of geochemical sourcing has led to many projects using various analytical techniques to produce independent datasets. In order to facilitate access to this growing body of data and to promote comparability and reproducibility in provenance studies, we designed Pofatu, the first online and open-access database to present geochemical compositions and contextual information for archaeological sources and artefacts in a form that can be readily accessed by the scientific community.
Furthermore, accessing large geochemical datasets of archaeological artefacts will lead to more robust and large-scope modelling of prehistoric exchange systems  [18] [19] [20]. However, the current lack of appropriate global data management platform makes it difficult to access and reference relevant archaeological datasets and often induces duplication of individual endeavors. In this data descriptor, we introduce the Pofatu Database, a curated and open-access database of geochemical data on archaeological materials and sources supported by comprehensive contextual information about individual samples and artefacts, including about the archaeological provenance, and a thorough description of analytical procedures. The goals of the database are (i) to provide easy access to published compositional data of archaeological sources and artefacts, (ii) to assemble contextual archaeological information for each individual sample, (iii) to facilitate reuse of existing data and encourage the appropriate crediting of original data sources, and (iv) to ensure reproducibility and comparability by documenting instrumental details, analytical procedures and reference materials used for calibration purposes or quality control.