dataesr/scientific-paragraphs-categorization huggingface dataset:
Sequence Trimming and Bioinformatics Pipelines The preliminary processing of bacterial and archaeal 16S rRNA gene libraries was performed using mothur v. 1. 34. 4  (Schloss et al. , 2009)  to join the paired ends. To reduce the file size for bacterial 16S rRNA gene sequences in order to accommodate the size limit of the 32-bit version of USEARCH  (Edgar, 2010) , we discarded singletons using the "unique. seqs" and "split. abund" commands with a cutoff of 1. This method replaces the equivalent procedure within USEARCH that is recommended by the Brazilian Microbiome Project (BMP) bacterial 16S pipeline  (Pylro et al. , 2014a,b). The database for aligning and classifying the bacterial and archaeal reads was downloaded from GreenGenes (gg_13_8_otus). Paired-end fungal reads were also joined similarly into contigs using mothur as described above, and the reads were then processed using the BMP fungal ITS pipeline  (Pylro et al.
, 2014a,b)  using QIIME  (Caporaso et al. , 2010) ,  USEARCH, ITSx (Bengtsson-Palme et al. , 2013) , FASTA formatter (http://hannonlab. cshl. edu/fastx_toolkit/index. html), and the scripts written by the members of the BMP, with the UNITE ITS database (sh_qiime_release_s_01. 08. 2015)  (Kõljalg et al. , 2013). The identity threshold for the OTUs was set at 97%. Since the database was poorly representative of archaea, we constructed an archaeal phylogenetic tree, based on the OTUs of the members of the archaeal core microbiomes, which we had computed previously using the "compute_core_microbiome. py" script with Phylogeny. fr (Dereeper et al. , 2008). The phylogenetic tree improved the identification of OTUs using reference sequences selected from GenBank. More details on bioinformatic processing are available in Supplementary Information.
Data Processing and Statistical Analyses We used the QIIME script "core_diversity_analyses. py" to estimate α-diversity (Chao1, Simpson's reciprocal, and evenness indices). The effect of increased fertilization or seeding density, crop identity, and sample biotope (roots or rhizosphere soil) on the indices was tested by analysis of variance (ANOVA) with Tukey's post-hoc analyses using the "HSD. test" function of the "agricolae" package in R (https://www. r-project. org). The effect of crops and canola treatments on microbial community structure was assessed by permutational multivariate analysis of variance (PERMANOVA) using the "adonis" function of the "vegan" package in R. Two groups of comparisons were conducted: (1) the effect of crops (canola, wheat, and pea) on root and rhizosphere communities of bacteria, fungi, and archaea, and (2) the effect of canola treatments (Can_RE, Can_HF, and Can_HD) on root and rhizosphere microbial communities.
To assess the clustering of samples in each group of comparisons, principal coordinates analysis (PCoA) plots were drawn using the "vegan" package in R based on the OTU matrices generated by the QIIME script "core_diversity_analyses. py" (see Supplementary Information for details). In order to identify potentially important microorganisms in our system, we used the concept of core microbiome, as defined by  Vandenkoornhuyse et al. (2015). The core microbiome associated with each crop and the eco microbiomes associated with each treatment were identified using the QIIME script "compute_core_microbiome. py". Only the OTUs that formed more than 1% of the assemblage for at least one of the combinations of treatment and location were used. The criteria for selection and screening OTUs are detailed in Supplementary Information.
To determine whether OTUs belong to core or eco microbiomes, we compared them across crops and canola treatments, without considering location, using the QIIME script "group_significance. py" to assess their association with canola. Kruskal-Wallis tests and Benjamini-Hochberg FDR (false-discovery-rate-corrected) P-values  (Goeman and Solari, 2014)  were used to evaluate the significance of the treatment effect. When the frequency of an OTU exceeded a threshold in all samples from the three canola treatments and locations, the OTU was considered part of the canola core microbiome. If an OTU was present in one or two treatments but not in all, the OTU was considered part of the eco microbiome. The remaining OTUs constituted the unshared fraction and were not considered further. Details on the definition of the core and eco microbiomes are given in Supplementary Information.
Bacterial, fungal, and archaeal OTUs of the core and eco microbiomes were analyzed across two different subgroups, (1) roots and (2) rhizospheres, and contrasted between all canola plots with reference crops (canola, wheat, and pea) and between canola treatments (Can_RE, Can_HF, and CAN_HD). To estimate the similarity of the core microbiomes of canola, wheat, and pea, and the similarity of the core microbiomes of canola as influenced by seeding rate and fertilization, we computed the Sørensen indices using the QIIME script "beta_diversity. py" with the "binary_sorensen_dice" option. We used canola crops as the baseline for comparison when comparing the crops and Can_RE as the baseline for comparison when comparing the canola treatments. The values of the Sørensen index were tested by ANOVA with Tukey's HSD posthoc analyses using the "HSD.
test" function of the "agricolae" package in R. The links between the bacterial and fungal core microbiomes from roots and from rhizosphere soil were separately analyzed with co-inertia analyses (CoIAs) using the "ade4" package in R. The input OTU tables were Hellinger transformed using the 'deconstand' function of the "vegan" package. The transformed matrices were used as input data for the principal component analysis (PCA) using the "dudi. pca" function from "ade4. " Outputs of "dudi. pca" were run with the function "coinertia" from "ade4" to generate the co-intertia analysis. The results were tested using the Monte Carlo method to obtain the simulated P value. Redundancy analysis (RDA) was used to correlate post-spray weed counts and emergence counts with the bacterial and fungal root and rhizosphere core microbiomes and plant yields in order to assess their association with these field variables.
The matrices were Hellinger-transformed using the "decostand" function followed by the "rda" function of the "vegan" package in R. The significance of the RDA model was tested by ANOVA and the R 2 values were generated by the "RsquareAdj" function in R. The significance of the whole model, the axes, and each factor were also tested by ANOVA. To identify the microorganisms correlated with yields, the association of each member of the core microbiomes of canola roots and rhizospheres with yield was tested using Spearman's correlation. The matrices were standardized using the "decostand" function of the "vegan" package in R, followed by the "rcorr" function in the "Hmisc" package. Only the members with significant correlations were included in the results.
Taxonomic Affiliations of the Bacterial, Fungal, and Archaeal OTUs After the reads from different data sets were filtered for quality, the bacterial gene data set allowed us to retrieve a total of 3,230,956 sequences (ranging from 3,574 to 72,331 reads across the 120 samples; Table  S2 ) that were assigned to 6,376 OTUs after subsampling to 3,574 reads per sample. For the fungal data set, 1,112,137 sequences (ranging from 97 to 59,199 reads across 120 samples) were obtained and assigned to 679 OTUs after subsampling to 123 reads while discarding the sample with <100 reads. In the archaeal data set, we obtained a total of 2,936,314 sequences (ranging from 329 to 203,921 reads across 120 samples) that were assigned to 49 OTUs after subsampling to 329 reads per sample. Sixteen bacterial phyla with four classes from the phylum Proteobacteria were identified.
OTU-F9 belonging to O. brassicae was the only fungal OTU classified as core for canola, with abundances that varied significantly between treatments (up to 83%) in roots but not in the rhizosphere (Table  S5 ). OTU-F9 was also detected in the microbiomes of wheat and pea roots, but in much lower abundances than in canola roots (Table  S5 ), while it was not detected in the rhizosphere of these reference crops. In comparison with the canola fungal core microbiomes, both reference crops harbored a higher fungal diversity (Table  5 ). Three OTUs were shared between wheat and pea roots and were absent from canola roots. Among them, OTU-F16 was first classified as O. brassicae using the UNITE database (Table  5 ), but since OTU-F16 differed from OTU-F9 (from canola roots) (lengths of 102 vs. 132 bp; identity of 86% within 78% coverage), they were furthered compared.
A BLASTn search in the NCBI database found them to be 100% identical with 100% coverage to the sequences AB205208 and AB205213, respectively classified as O. virulentus and O. brassicae in a taxonomic study of Olpidium spp. (Sasaya and Koganezawa, 2006 ) that sought to differentiate the non-virus-carrying and virus-carrying species within this genus. Our classification results were therefore updated accordingly. The OTU belonging to O. virulentus, which is the virus-carrying Olpidium species, then appeared as part of the microbiome of wheat and pea roots but was not detected in canola. Among the 15 OTUs retained as part of the fungal core microbiomes of the canola, wheat, and pea rhizospheres (Table  5 ), 2 OTUs were found associated with all crops, 5 OTUs were shared between canola and pea, 2 OTUs were shared between canola and wheat, 7 OTUs were shared between wheat and pea, and 5 OTUs were associated with the rhizosphere of one crop only.
Three fungal members of the rhizosphere microbiome were found to be significantly correlated with yield: Monographella cucumerina (negatively correlated), Fusicolla sp. (positively correlated), and F. merismoides (positively correlated). In roots, five bacteria were positively correlated with yield: Amycolatopsis sp. , Stenotrophomonas sp. , S. proteamaculans, Arthrobacter sp. , and Pedobacter sp. The remaining bacteria were negatively correlated with yield. Amycolatopsis sp. and Stenotrophomonas sp. were members of the eco microbiome associated with   The symbol "-" indicates that the OTU does not meet the criteria for the core/eco microbiome. f When followed by the symbol "∧", the classification was done by BLASTn manually with the NCBI database. g The Sørensen index compares the assemblages between wheat and pea with canola.
Sequence data We used the UniProt database (Release 2016_03)  (79)  and the National Center for Biotechnology Information (NCBI) Reference Sequence (RefSeq) (61) Viral Section (Release 75) database to collect Rpb1 and Rpb2 protein sequences from cellular organisms and NCLDVs. We additionally used the GenomeNet/Virus-Host Database  (49)  to retrieve the nucleotide sequences of Rpb1 and Rpb2. Marine metagenomic sequence data were obtained from CAMERA (75) and the Tara Oceans project  (32, 76)  (Table  S1 ). In addition, we obtained metagenomic data for other non-marine environments from CAMERA and KEGG/MGENES  (35). Collectively, we used metagenomic data derived from 58 projects (Table  S1 ). Based on metagenomic data, we initially prepared files for amino acid sequences for open reading frames (ORFs) that were longer than or equal to 150 codons.
The total number of ORFs was 149,645,996: marine metagenomes (101,856,227 ORFs, 68%), other aquatic environmental metagenomes (8,385,210 ORFs, 5. 6%), mammalassociated microbial metagenomes  (38, 341, 510  ORFs, 26%), and other metagenomes (1,063,049 ORFs, 0. 7%). Most of the analyses presented in the present study focused on data from marine metagenomes, mainly derived from two large scale oceanic microbiome projects: Tara Oceans and the Global Ocean Sampling project  (68). Data from other environments were used to confirm marine data.
Reference sequence alignments and phylogenetic trees We identified Rpb1 and Rpb2 homologs in the UniProt and RefSeq databases using HMMER/HMMSEARCH (version 3. 1; E-value<1×10 -5 ) (20) based on profile hidden Markov models that we built from alignments of Rpb1 (COG0086) and Rpb2 homologs (COG0085)  (30). We used CD-HIT version 4. 6 to reduce the redundancy of the collected known Rpb1/2 sequences  (47). The resulting non-redundant sequences were aligned using MAFFT v7. 215  (36)  with default parameters and alignment columns containing gaps were trimmed using trimAl v1. 2rev59  (8). We referred to the resulting reference sequence alignments for Rpb1 and Rpb2 as RAln-Rpb1 and RAln-Rpb2, respectively. We also generated reference sequence alignments solely composed of sequences from Megaviridae, Bacteria, and Archaea, and referred to the alignments as RAln-MBA-Rpb1 and RAln-MBA-Rpb2.
Reference trees and taxonomic classification of metagenomic sequences We identified 59,938 Rpb2 and 40,534 Rpb1 homologs in the UniProt/RefSeq sequence databases using profile HMMs derived from COG0085 (Rpb2) and COG0086 (Rpb1). Viral Rpb2/Rpb1 sequences identified by this search all originated from NCLDVs. Among these sequences, 511 Rpb2 and 575 Rpb1 sequences were selected as reference sequences after reducing redundancy by clustering and discarding unusually long and short sequences. Based on the reference sequences, we built reference phylogenetic trees for Rpb2 (Fig. 2A ) and Rpb1 (Fig. 2B ). The reference trees were generally consistent with the classification of prokaryotes and viruses as well as eukaryotic paralogs. Using profile HMMs built from these reference sequences, 248,101 and 252,609 sequences were obtained from metagenomes as candidates of environmental Rpb2 and Rpb1, respectively.
Acknowledgements This work was supported by The  Canon Foundation  (No. 203143100025 ),  JSPS/KAKENHI  (Nos. 16KT0020 ,  17H03850 , and  26430184 ),  Scientific Research on Innovative Areas  from the  Ministry of Education, Culture, Science, Sports and Technology (MEXT) of Japan  (Nos. 16H06429 ,  16K21723 , and  16H06437 ), and the  Collaborative Research Program of the Institute for Chemical Research, Kyoto University  (No. 2016-28 ,  2017-25 ). PH was supported by the  OCEANOMICS "Investissements d'Avenir" program  of the  French government  (No. ANR-11-BTBR-0008 ). Computational work was completed at the  SuperComputer System, Institute for Chemical Research, Kyoto University. We thank the  Tara Oceans consortium , staff, and sponsors who supported the  Tara Oceans Expedition  (http://www.
Whole genome sequencing (WGS) of important foodborne pathogens is a technology under development, but is already employed in routine surveillance by public health agencies and is being increasingly exploited in tracing transmission routes and identifying contamination events (source tracking) that take place in the farm-to-fork continuum. Furthermore, data generated from WGS, complemented by other -omics data, have the potential to be integrated into and strengthen microbiological risk assessment. In this paper, we discuss the contribution of WGS in diverse areas important to food safety and public health. Additionally, an outlook of future WGS applications, which should contribute to our understanding of the ecology and physiology of foodborne microorganisms, is presented.
To further promote application of WGS, the ECDC will oversee and coordinate efforts to overcome current challenges and is aiming to establish WGS as the method of choice for foodborne pathogen typing and surveillance within the next five years  (ECDC, 2016a). WGS typically includes several steps: obtaining a pure culture of the organism of interest, DNA extraction, library preparation and amplification, and DNA sequencing, followed by possible alignment, data analysis, and finally biological interpretation. Today, WGS is not necessarily followed by annotation, data analysis or functional (biological) interpretation. It is generally limited to obtaining and ideally depositing a file with the entire genome sequence in a database such as the National Center for Biotechnology Information (NCBI).
Furthermore, in most instances WGS data are in the form of contigs, and in the absence of a closed genome sequence the available data are expected to contain sequence gaps. As WGS evolves from a research tool to a routine surveillance instrument, different pipelines, software and platforms are used to perform sequencing data analysis. Many of the pipelines are operational as research tools and under continuous development; for instance, FDA CFSAN provides periodic updates on the single nucleotide polymorphism Pipeline (https://github. com/CFSAN-Biostatistics/snp-pipeline; Davis et al. , 2015). An overview of different bioinformatics approaches to WGS data analysis is provided by  Franz et al. (2016). Standardization, i. e. validation and articulation of key quality requirements and criteria, is needed to be able to compare results produced by different pipelines, software and platforms.
, 2011). WGS has the potential to overcome challenges related to discriminatory power. The integration of WGS in surveillance programs has already started to significantly contribute to the investigation of foodborne outbreaks involving bacterial agents. In the United States, the FDA and the CDC are intensively using this technology in outbreak investigation by comparing isolates from patients to those from food or food production environments. WGS data are uploaded to the open-access database GenomeTrakr. The GenomeTrakr network is currently comprising of numerous laboratories in the US and elsewhere, leading to a dramatic increase in the number of sequences in the database  (Allard et al. , 2016). In the European Union, the ECDC is also putting in place a network of public health laboratories employing WGS, with at least 19 countries enrolled in surveillance programs and outbreak investigations.
A common European database for WGS data and metadata is under development  (ECDC, 2016b). In the context of outbreak investigations, GenomeTrakr and equivalent databases will make it increasingly possible to accurately establish links between sequences from outbreaks and from food or environmental sources. Furthermore, the high discriminatory power of WGS allows for detection of diffuse outbreaks by linking infrequent cases, which would otherwise have been considered sporadic cases without a common source. Such capacities are expected to help to contain outbreaks due to earlier outbreak detection and intervention at the source(s). Recent examples are given in "Applications of WGS in Food Safety management" (FAO, 2016) and are increasingly encountered in the literature  (Butcher et al. , 2016; Dallman et al.
, 2016; Holmes et al. , 2015). The development of such databases combined with epidemiological investigations can enhance the possibility to share sequences among countries and thus identify and monitor global outbreaks  (Allard et al. , 2016). The expansion of the databases will allow more links between present and past outbreaks or contamination events, thus potentially identifying strains persistent through time in certain production facilities or other ecosystems. Thus, WGS data may be employed to retrospectively investigate unsolved outbreaks or contamination events. However, this type of analysis can only be supported by complete metadata providing the relevant context. The metadata should include information, such as where strains were isolated, from which kind of sample, country of origin and date of isolation.
monocytogenes can occur at different deli environments at different locations, with no evident clear links in transmission. WGS data provided evidence for repeated introduction of L. monocytogenes from an external source to multiple retail establishments. Similarly,  Fagerlund et al. (2016)  reported nearly identical L. monocytogenes MLST Sequence Type 8 isolates in Danish and Norwegian salmon processing plants spanning a period of more than 10 years. In another case, nearly identical L. monocytogenes ST 8 isolates were found in two poultry processing plants in Norway  (Fagerlund et al. , 2016). These examples emphasized that linking disease back to isolates from processing environments can be challenging as almost identical isolates can be detected in more than one food processing plant. To resolve situations of uncertainty it is important that metadata are carefully collected and evaluated in combination with the WGS data.
Similar efforts are underway in order to associate genotypes of foodborne pathogens with tolerance to common food stresses, such as cold, salt or acid  (Hingston et al. , 2017). As the cost of sequencing continues to decrease and public databases for WGS data for foodborne pathogens continue to develop (e. g. GenomeTrakr), the amount of WGS data is dramatically increasing, and is expected to continue to do so in the coming years. The potential use of these 'big data' in hazard identification will enable risk assessors and risk managers to move from a 'taxonomic' understanding of the hazard to one based on genotype-phenotype associations and on a much deeper understanding of virulence at the strain level. An example of effective integration of bacterial population genomics with clinical and epidemiological data was recently presented with L.
monocytogenes, with certain hypervirulent clones found to be over-represented among human disease cases, while hypovirulent ones were over-represented in foods  (Maury et al. , 2016). Interestingly, this analysis also revealed that commonly employed reference strains of L. monocytogenes actually belonged to hypovirulent clones  (Maury et al. , 2016). In addition to public WGS databases, further information about relevant factors for hazard identification needs to be developed. Especially needed are data to inform gene family profiling, as well as the establishment and agreement on relevant reference genomes for specific pathogens that reflect strains actually involved in human illness. As a systematic approach to support decision-making, MRA requires that all stakeholders in the food safety continuum have confidence in the robustness of the data, approaches and scientific principles underpinning the output of the risk assessment.
In addition to its use in surveillance and outbreak investigation, WGS-based information can also provide other subtyping data of relevance to the target microorganism, such as in silico determination of serotype and sequence type (ST) (via in silico multi-locus sequence typing [MLST]). In fact, such determinations are not only more rapid and accurate but ultimately more cost-effective via WGS-based analysis  (Kwong et al. , 2016). Besides the application of WGS on pure cultures of specific isolates, it is starting also to be used to study microbial populations. Whole genomes of microorganisms can be fully or partially reconstructed from metagenomics data  (Cocolin et al. this issue). Metagenomics is a culture-independent application of NGS whereby the community DNA from a sample is sequenced, including DNA from all microbes in the sample.
Conclusions and future perspectives The potential of WGS as a typing tool to be used in surveillance and outbreak investigation and to contribute in risk assessment is indisputable. Scientists in academia, the regulatory sector and the food industry increasingly recognize WGS as the method of choice for basic research applications and epidemiological investigations. Nevertheless, complete replacement of other typing techniques by WGS will need adequate attention to overcome existing challenges and limitations. These include expenses related to initial investment and running costs, human resources, laboratory and bioinformatics infrastructure but also the integration of WGS data into risk assessment and decision-making processes. A gradual phasing out of existing methodologies is foreseen as technical capacity is building up in, and becoming harmonized among, different countries.