dataesr/scientific-paragraphs-categorization huggingface dataset:
The discovery of more taxa not in reference databases may help to reduce the large fraction of DNA that cannot currently be classified. Our data do not support the presence of any viruses in the core microbiome. However, this cannot be excluded and should be thoroughly addressed in the future with more in-depth sequencing, sampling/extraction techniques, or long-read technologies. Many of the identified taxa are frequently implicated as infectious agents in a clinical setting including specific Staphylococcus, Streptococcus, Corynebacterium, Klebsiella, and Enterobacter species. However, there is no indication that the species identified in the urban environments are pathogenic, and further in-depth studies are necessary to determine the clinical impact of urban microbiomes. This includes microbial culture studies, explicitly searching for virulence factors and performing strain-level characterization to determine biological functions carried by specific populations.
e. , fluoroquinolones) are sensitive to small mutations, and methodological improvements may refine our results. A companion study to this paper by  Chng et al. (2020)  has examined the spread of AMRs in hospital settings. Further research is needed to explore AMR genes fully in the urban environment, especially in medical environments, including cultural studies that directly measure the phenotype of resistance. In summary, this study presents the first genetic atlas of urban and mass-transit metagenomics from across the world. By facilitating large-scale epidemiological comparisons, it is a first critical step toward quantifying the distribution, types, and dynamics of environmental microbiomes, providing requisite data for tracking changes in ecology or virulence. As more datasets emerge from rural and suburban areas with livestock and farms, sewage from cities  (Fresia et al.
Data and code availability Materials, Methods, and Open-Source Code To make our study fully reproducible, we released an open-source version-controlled pipeline called the MetaSUB Core Analysis Pipeline (CAP)  (Danko and Mason, 2020). This pipeline includes all steps from extracting data from raw sequence FASTQ files to producing refined results like taxonomic and functional profiles. Every tool in the CAP is open source with a permissive license. The CAP is available as a docker container for easier installation in some instances, and all databases used in the CAP are available for public download. The CAP is versioned and includes all necessary databases, allowing researchers to replicate results and figures. The MetaSUB dataset and CAP are built and organized for full accessibility to other researchers.
This is consistent with the concept of Open Science. Specifically, we built our study with the FAIR principles in mind: Findable, Accessible, Interoperable, and Reusable. To make our results more reproducible and accessible, we have developed a program to merge the CAP's output into a condensed data-packet. This data packet contains results as a series of Tidy-style data tables with descriptions. The advantage of this set-up is that result tables for an entire dataset can be parsed with a single command in most high level analysis languages like Python and R. This package also contains Python utilities for parsing and analyzing data packets which streamline most of the boilerplate tasks of data analysis. All development of the CAP and data packet builder (Capalyzer) package is open source and permissively licensed.
In addition to general-purpose data analysis tools, essentially all analysis in this paper is available as a series of Jupyter notebooks. These notebooks allow researchers to reproduce our results, build upon our results in different contexts, and better understand precisely how we arrived at our conclusions. By providing the exact source used to generate our analyses and figures, users can quickly incorporate new data or correct any bugs. For less technical purposes, we also provide web-based interactive visualizations of our dataset (typically broken into city-specific groups). These visualizations are intended to provide a quick reference for major results as well as an exploratory platform for generating novel hypotheses and serendipitous discovery. The web platform used, MetaGenScope, is open source, permissively licensed, and can be run on a moderately powerful machine (though its output relies on results from the MetaSUB CAP).
Our hope is that by making our dataset open and easily accessible to other researchers the scientific community can more rapidly generate and test hypotheses. One of the core goals of the MetaSUB consortium is to build a dataset that benefits public health. As the project develops, we want to make our data easy to use and access for clinicians and public health officials who may not have computational or microbiological expertise. We intend to continue to build tooling that supports these goals. released to the CAMDA community in the form of an annual challenge addressing the issue of geographically locating samples: 'The MetaSUB Inter-City Challenge' in 2017 and 'The MetaSUB Forensics Challenge' in 2018 and 2019. In the latter challenge the MetaSUB data has been complemented by data from EMP  (Thompson et al.
, 2017)  and other studies  (Delgado-Baquerizo et al. , 2018; Hsu et al. , 2016). This Open Science approach of CAMDA has generated multiple interesting results and concepts relating to urban microbiomics, resulting in several publications https://biologydirect. biomedcentral. com/articles/collections/camdaproc as well as perspective manuscript about moving toward metagenomics in the intelligence community  (Mason-Buck et al. , 2020).
We reasoned that uncontaminated samples may contain the same species as negative controls but were less likely to contain identical strains. Only 137 case samples were observed to have any sequence with high similarity to an assembled sequence from a negative control (8,000 base pairs minimum of 99. 99% identity). The identified sequences were principally from Bradyrhizobium and Cutibacterium. Since these genera are core taxa (See a core urban microbiome centers global diversity) observed in nearly every sample but high similarity was only identified in a few samples, we elected not to remove species from these genera from case samples. Comparison of taxonomic and k-mer based metrics to establish database quality We generated 31-mer profiles for raw reads using Jellyfish. All k-mers that occurred at least twice in a given sample were retained.
We also generated MASH sketches from the non-human reads of each sample with 10 million unique minimizers per sketch. We calculated the Shannon's entropy of k-mers by sampling 31-mers from a uniform 10,000 reads per sample. We found clear correlations between k-mer based Jaccard distance (MASH) and taxonomic Jaccard distance (Figure  S2A ). We also compared alpha diversity metrics (Figure  S2B ): Shannon entropy of k-mers, and Shannon entropy of taxonomic profiles. As with pairwise distances these metrics were correlated though noise was present. This noise may reflect sub-species taxonomic variation in our samples. Evaluation of unmapped DNA to establish aligner performance A large proportion of the reads in our samples were not mapped to any reference sequence. There are three major reasons why a fragment of DNA would not be classified in our analysis 1) The DNA originated from a non-human and non-microbial species which would not be present in the databases we used for classification 2) Our classifier (KrakenUniq) failed to classify a DNA fragment that was in the database due to slight mismatch 3) The DNA fragment is not represented in any existing database.
Explanations (1) and (  2 ) are essentially drawbacks of the database and computational model used, and we can quantify them by mapping reads using a more sensitive aligner to a larger database, such as BLASTn  (Altschul et al. , 1990) , or ensemble methods for analysis  (McIntyre et al. , 2017). To estimate the proportion of reads which could be assigned, we took 10k read subsets from each sample and mapped these to a set of large database using BLASTn (see a core urban microbiome centers global diversity for details). This resulted in 34. 6% reads which could not be mapped to any external database compared to 41. 3% of reads mapped using our approach with KrakenUniq. We note that our approach to estimate the fraction of reads that could be classified using BLASTn does not account for hits to low quality taxa which would ultimately be discarded in our pipeline, and so represents a worstcase comparison.
Generating taxonomic profiles for samples We generated taxonomic profiles by processing non-human reads with  KrakenUniq (v0. 3. 2 Breitwieser et al. (2018) ) using a database based on all draft and reference genomes in NCBI/RefSeq Microbial (bacteria/archaea, fungi and virus) ca. March 2017. KrakenUniq was selected because its high performance, as it has been demonstrated to be comparable or having higher sensitivity than the best tools identified in a recent benchmarking study  (McIntyre et al. (2017) ) on the same comparative dataset. In addition, KrakenUniq allows for tunable specificity and identifies k-mers that are unique to particular taxa in a database. Reads are broken into k-mers and searched against this database. Finally, the taxonomic makeup of a sample is given by identifying the taxa with the greatest leaf to ancestor weight.
Contextualizing samples We compared our samples to metagenomic samples from the Human Microbiome Project and a metagenomic study of European soil samples using MASH  (Ondov et al. , 2016) , a fast k-mer based comparison tool. We built MASH sketches from all samples with 10 million unique k-mers to ensure a sensitive and accurate comparison. We used MASH's built-in Jaccard distance function to generate distances between our samples and HMP samples. We then took the distribution of distances to soil and to each particular human commensal community as a proxy for the actual similarity of our samples to the site. We used the Microbe Directory  (Shaaban et al. , 2018)  to annotate taxonomic calls. The Microbe Directory is a hand curated, machine readable, database of functional annotations for 5,000 microbial species.
Analysis of Antimicrobial Resistance Genes We generated profiles of antimicrobial resistance genes using  MegaRes (v1. 0. 1, Lakin et al. (2017) ). To generate profiles from Mega-Res, we mapped non-human reads to the MegaRes database using Bowtie2 (v2. 3. 0, very-sensitive presets,  Langmead and Salzberg (2013) ). Subsequently, alignments were analyzed using ResistomeAnalyzer (commit 15a52dd https://github. com/cdeanj/ resistomeanalyzer) and normalized by total reads per sample and gene length to give RPKMs. MegaRes includes an ontology grouping resistance genes into gene classes, AMR mechanisms, and gene groups. AMR detection remains a difficult problem and we note that detection of a homologous sequence to a known AMR gene does not necessarily imply an equivalent resistance in our samples.
Currently, the gold standard for detecting AMR is via culturing. Known AMR genes can come from gene families with homologous regions of sequence. To reduce spurious mapping from gene homology we used BLASTn to align all MegaRes AMR genes against themselves. We considered any connected group of genes with assemblies greater than 5kb. Then, the UViGs were clustered with the content of the IMG/VR system (a total of over 730k viral sequences including isolate viruses, prophages, and UViGs from all kind of habitats). The clustering step relied on a sequence-based classification framework (based on 95% sequence identity across 85% of the shortest sequence length) followed by the markov clustering (mcl). This approach yielded 2,009 viral clusters (ranging from 2-611 members) and 9,605 singletons (or viral clusters of 1 member), sequences that failed to cluster with any sequence from the dataset or the references from IMG/VR, resulting in a total of 11,614 vOTUs.
Identifying Host Virus Interactions We used two computational methods to reveal putative host-virus connections  (Paez-Espino et al. , 2016a). (1) For the 686 vOTUs that clustered with viral sequences from the IMG/VR system, we projected the known host information to all the members of the group (total of 2,064 MetaSUB UViGs). (  2 ) We used bacterial/archaeal CRISPR-Cas spacer matches (from the IMG/M 1. 1 million isolate spacer database) to the UViGs (allowing only for 1 SNP over the whole spacer length) to assigned a host to 1,915 MetaSUB vOTUs. Additionally, we also used a database of over 20 million CRISPR-Cas spacers identified from metagenomic contigs from the IMG/M system with taxonomy assigned. Since some of these spacers may derive from short contigs these results should be interpreted with caution.
CRISPR Array Detection and Annotation Using CRISPRCasFinder  (Couvin et al. , 2018)  the MetaSUB database was investigated to predict CRISPR arrays and annotate them with their corresponding predicted type based on CRISPR-Cas genes in their vicinity. CRISPRCasFinder was run with default parameters, ''-so'' and ''-cas'' options to identify cas genes. The precision and recall of the virus detection was 99. 6% and 37. 5% respectively, as previously reported by  (Paez-Espino et al. , 2016). CRISPR-Cas types were assigned to arrays based on detected cas genes within a 10 kilobases vicinity. Cases where CRISPRCas-Finder associated several cas genes of contradicting CRISPR-Cas types with the same CRISPR array were regarded as unclear annotation. This procedure yielded 838,532 predicted CRISPR arrays (with additional CRISPR arrays predicted with default parameters for PILER-CR), of which, 3,245 CRISPR arrays had unambiguous annotation, resulting in 43,656 unique spacers queried against genomic databases using BLASTN.
Matching CRISPR Spacers to Organism Databases In order to associate detected spacers within defined groups (plasmids, prophages, viruses) four different genomic databases were aggregated to be searched with BLASTN. The aggregated database consisted of IMG/VR, PHASTER, and PLSDB alongside bacterial and archaeal genomic sequences from the National Center for Biotechnology Information (NCBI). All database downloads were made on the 28th January 2020. Detected and annotated spacers were searched against the databases mentioned above using BLASTN with the following additional arguments, which correspond to the default parameters of CRISPRTarget: word_size = 7, evalue = 1, gapopen = 10, gapextend = 2, penalty = À1, reward = 1.
ACKNOWLEDGMENTS We thank these organizations, people, and grants for their support: the  Tri-I Program in Computational Biology and Medicine (CBM)  funded by  NIH  grant  1T32GM083937 ;  GitHub ;  Philip Blood  and the  Extreme Science and Engineering Discovery Environment (XSEDE) , supported by  NSF  grant number  ACI-1548562  and  NSF  award number  ACI-1445606 ; the  WCM SCU and Epigenomics and Genomics Core Facilities ; the  Vallee Foundation ; the  WorldQuant Foundation ; Igor Tulchinsky; The  Pershing Square Sohn Cancer Research Alliance ;  NASA  ( NNX14AH50G ,  NNX17AB26G ), the  NIH  ( R01AI151059 ,  R25EB020393 ,  R21AI129851 ,  R35GM138152 ,  U01DA053941 );  STARR Foundation  ( I13-0052 );  LLS  ( MCL7001-18 ,  LLS 9238-16 ,  LLS-MCL7001-18 ); the  NSF  ( 1840275 ); the  Bill and Melinda Gates Foundation  ( OPP1151054 ); the  Alfred P.
Since 2017, MetaSUB has partnered with the Critical Assessment of Massive Data Analysis (CAMDA) camda. info, a whole conference track at the Intelligent Systems for Molecular Biology (ISMB) Conference. At this venue, a subset of the MetaSUB data was Continued REAGENT or RESOURCE SOURCE IDENTIFIER Figure Generation Code This paper https://github. com/MetaSUB/main_paper_figures Other Copan Liquid Amies Elution Swab Copan Diagnostic Cat. :480C Isohelix Swabs Isohelix Cat. :MS-02 2D Thermo Scientific Matrix Thermo Scientific 3741-WP1D-BR ZR BashingBead Lysis Tubes (0. 1 & 0. 5 mm) Zymo Research Cat# S6012-50.
We also vary the size of the initial set of data to simulate some noise (missing states in the traces). In the beginning, 100% of the data is used, then we will decrease by two percent the percentage of data used at each simulation, gradually moving from 0% of missing data to 90%. For each data range, we run two hundred times the three algorithms, taking randomly the amount of data needed in our dataset and computing the average of the results. However, the same percentage of data is used for positive and negative examples.
The experiments being done on an old CPU i7 two cores at 3. 3GHz and using Python as the programming language. We cannot see C4. 5 in Figures  14  and 15 , this is because C4. 5 take much time compared to the two other algorithms to learn the concept (build the decision tree) 1. 9 to 0. 6 s on average for GridWorld and 1. 5 to 0. 5 s for CubeWorld, which makes it difficult to see the difference between MGI and LFST. One might argue that the learning time is not essential to learn concepts since it is just learned one time, though we believe that if it must be part of some real-time system, it is better to have a fast algorithm. We do think we made the right choice using our algorithm LFST for this problem since it performs as well if not better than traditional concept-learning algorithms, and compared to C4.
We will then have for each state of the world (observed previously in the collected data) the different actions that the agent did when it was in this state and the states of the world that it reached after these actions. So, we can infer a potential action model of the agent and the dynamic of the environment based on the observed transitions. Unfortunately, LFIT is not very robust against the lack of data, and the noise, especially the data need to cover as many transitions as possible between all the different possible states of the world. It is not very convenient because usually, the agent we observe is not acting randomly, and so, many possible transitions will never be observed. We also plan to use SMILE  [36]  another method that we can use to learn the action model of the agent. This method uses the same kind of data as LFIT, so here also the modification of the data collection process to use SMILE, and our method is not too cumbersome.
For instance, if we use 80% of the data, we will randomly pick 80% of the positive examples and 80% of the negative ones. For GridWorld environment, we have done the experiments with a set of 128 positive examples and a set of 1791 negative ones, which have been extracted from a set of 128 successful traces. For CubeWorld environment, we have done the experiments with a set of 100 positive examples and a set of 559 negative ones, which have been extracted from a set of 100 successful traces. In 421World, we have done the experiments with a set of 200 positive examples and 882 negative examples; this results from 200 successful traces. The python code to generate data from these three environments is available here (https://github. com/glorthioir/LFST_data. git (accessed on 29 April 2021)).
Measurement areas and sets We used data collected in the framework of the project ACCEDERA: Improving the accuracy of personal RF-EMF measurements and characterization of exposure levels in different environments across countries  (SwissTPH, 2020). We collected RF-EMF measurements in five European countries (Belgium, France, Spain, Switzerland and the Netherlands). Measurements were conducted in several cities (Paris and Amiens, Barcelona and Tarragona, Brussels and Ghent, Zurich and Basel, Utrecht and surrounding) and in different types of microenvironments, such as downtown shopping areas, decentral and central residential areas, playgrounds, parks, train stations, bus stops, university libraries and so on (supplemental Table  ST1 ). The idea of including these different types of microenvironments was to collect average exposures in such "typical" environments that may display a broad range of exposure situations.
Data collection Trained field research assistants performed measurements in various indoor and outdoor microenvironments (supplemental Table  ST1 ) in Belgium, Spain, France, Netherlands and Switzerland. Measurements were collected using the BWDM in parallel with an ExpoM and either an EmeSpy or a second ExpoM exposimeter, worn on opposite sides of the torso. Measurements were collected every 1-4 s depending on the measurement device, while walking pre-defined routes to capture different types of microenvironments. Each microenvironmental measurement was aimed to be measured for about 15 min and assistants were asked to switch off their mobile phones. All microenvironments were a priori defined, and timing and characteristics were recorded with an application on a mobile phone set to flight mode.
All the aforementioned quantities were calculated using power densities in mW/m 2. Because of the higher proportion of measurement values below the LoQ for uplink, DECT and WiFi signals, we also recalculated Spearman correlation coefficients including all measurements that fell below the LoQ. We also produced Bland-Altman plots on logtransformed values  (Bland and Altman, 1986). Finally, we used the BWDM measurements that had been collected with a sampling interval of one measurement per second. We artificially generated larger sampling intervals between two and 90 s, as similar intervals have been previously applied in some epidemiological studies  (Bolte et al. , 2019; Frei et al. , 2009). Equally spaced sampling intervals were generated, with random starting points within the first 1. 5 min of the BWDM data files, and geometric mean (total) exposure of the whole data set was calculated for all generated sampling intervals.
Comparison of artificially increased logging intervals using total exposure as measured by BWDM Scatter plot of ratios of geometric mean levels as measured with the BWDM. We compared artificially generated increasing logging intervals from 2 to 90 s and compared to taking a measurement every 1 s. Shown here are ratios of geometric means. Ratios were calculated separately for each countrys' data set. The red line shows the fitted regression line and the grey area the corresponding confidence interval. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article. ).
Acknowledgement Grant Sponsors: This research was funded by the  National Research Program of the French Agency for Food, Environmental and Occupational Health and Safety (ANSES) , grant No  2015-2-RF-07 , and the equipment was supported by funding from the  European Commission Seventh Framework Programme  under grant agreement number  FP7 603794  (GERoNiMO). Our funders had no involvement in the study design; data collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. A. T. is a postdoctoral fellow of the  Research Foundation Flanders (FWO)  under grant agreement no. 12U1417N. ISGlobal also acknowledges support from the  Spanish Ministry of Science, Innovation and Universities  through the " Centro de Excelencia Severo Ochoa 2019-2023" Program  ( CEX2018-000806-S ), support from the  Generalitat de Catalunya  through the  CERCA Program  and support from the  Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat of Catalonia through AGAUR (the Catalan Agency for Management of University and Research Grants)  (Project  2017 SGR 1487 ).