dataesr/scientific-paragraphs-categorization huggingface dataset:
Modeling pipeline overview Table  1  summarizes the entire data pre-processing, analysis and modeling pipeline developed in this study, and highlights in particular the differences between PLSR and CNN models. The following sections provide full details of each individual task. To ensure reproducibility, the source code of the modeling pipeline and data to replicate this work are available at the following link: https://fr. anckalbi. net/mirzai. Python version 3. 6, NumPy (https://numpy. org), Scikit-Learn (https:// scikit-learn. org) and PyTorch (https://pytorch. org) have been used for the data pre-processing, analysis and modeling parts as well as Captum (https://captum. ai/) to address the interpretability needs. A Google Colab Pro computing environment has been used with the following specification: 1 GPU Tesla P100 16GB, 2 Intel(R) Xeon(R) CPU @ 2.
Data selection, preprocessing The ability to predict and characterize K ex is key to optimize the remediation of radioactive contamination in agriculture. For this study, we focused on a selection of the Mid-Infrared Spectral database made available to the Joint FAO/IAEA Centre of Nuclear Techniques in Food and Agriculture by the Kellogg Soil Survey Laboratory (KSSL) from the United States Department of Agriculture (USDA-NRCS). The Mid-Infrared spectra were acquired on air-dried and ground soil samples using a Bruker Vertex 70 FTIR spectrometer with an HTS-XT highthroughput accessory  (Dangal et al. , 2019). In total 50,494 samples were identified and selected from the dataset, on which K ex was measured (as of January 2022). The mid-infrared spectra (from wavenumbers 4000 to 600 cm -1 at resolution of 2-1764 wavenumbers in total) were systematically centered and normalized, for both the baseline (PLSR) and CNN models, using the Standard Normal Variate transform (SNV)  (Barnes et al.
, 1989). When used to train the baseline model, a Savitsky-Golay transformation (window length of 11, polynomial order of 1 and first derivative) was further applied and the CO 2 spectral region was removed to avoid influence from this artifact independent from the soil analysis (from wavenumbers 2269 to 2389 cm -1 ). The CNN will safely ignore the CO 2 spectral region and learn the first derivative (finite difference) operator if it is deemed necessary to K ex prediction, hence, both pre-treatments were not applied in that case. In the dataset involved in this study, K ex content in the soil has been measured using the NH 4 OAc pH 7 extraction method (in cmol(+). kg -1 ). Table  2  provides the summary statistics of the empirical distribution of these measurements and their log 10 -transformation before any data selection (50,494 measurements in total).
For each of these different 20 random splits, both models were exposed to the same train, validation and test (unseen) data. The mean performance as well as their standard deviation were reported. To assess the ability of the PLSR and CNN models to leverage an increasing data regime, PLSR and CNN models were also trained on random subsamples of increasing sizes of the data and their performance on the test sets portrayed through the so-called "Learning curve" graph  (Viering and Loog, 2021). The assessment of the models was further performed in two different contexts that we call in this study "global" and "local". In the global setting, the models were trained on all data and performance was assessed on the entire test set and by Soil Taxonomy Orders. To provide further insights into the CNN training process, the validation loss by Soil Taxonomy Orders was also monitored as the number of epochs increased during training.
In the local setting, the models were trained by Soil Taxonomy Orders separately, and tested on the Soil Taxonomy Orders they were trained on. Only performance on Mollisols, Vertisols and Gelisols are reported (i) to highlight the role of distinctive data regimes (from high to low), and (ii) to assess the ability of the CNN model to "re-use" or transfer spectral features learned across different Soil Taxonomy Orders, in particularly between well and non-represented ones in the MIRS KSSL dataset.
Metrics To evaluate both the PSLR and the CNN models on training, validation and test datasets, the following metrics were used: (i) coefficient of determination (R 2 ) and (ii) Lin's concordance correlation coefficient (LCCC). LCCC measures the linear correlation between the predicted and true values but penalized by a bias term allowing to assess the agreement between the observed and predicted values with respect to the 1:1 line  (Lawrence and Lin, 1989). These two metrics were calculated on the log 10 -transformed version of the true and predicted values of K ex. In addition, the Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE) were calculated on the original scale of the target variable to facilitate interpretation and provide two complementary perspectives on the performance of the models.
Interpretability In this study, we implemented the GradientShap  (Lundberg and Lee, 2017; Sundararajan et al. , 2017; Smilkov et al. , 2017; Kokhlikyan et al. , 2020)  model interpretability algorithms to identify the spectral features regarded as important by the learning algorithm for the prediction task being performed. The importance of spectral features can be computed model-wide or for an individual or group of instances with marginal additional computational burden. In our effort to relate this information to domain knowledge, feature importance was calculated for the overall CNN models based on (i) the full dataset, used in this study, and (ii) the Soil Taxonomy Orders separately. The correlation of the GradientShap values of the spectral features for predicting K ex content by Soil Taxonomy Orders was also assessed, allowing understanding their similarities.
Exchangeable potassium prediction models in a growing data regime The learning curves and model performance of both PLSR and CNN models for the prediction of exchangeable potassium (log 10 (K ex )), with increasing number of training examples and using all Soil Taxonomy Orders, are shown in Fig. 3. Up to a dataset of 5000 samples, both PLSR and CNN models perform poorly with an R 2 below 0. 65 and a high level of variance across different random splits. However, a data regime of 10,000 samples appears to be an inflection point. While the PLSR model levels off at approximately an R 2 of 0. 64, the performance of the CNN keeps increasing up to an R 2 of about 0. 79 as the size of the dataset increases.
Moreover, when evaluated by Soil Taxonomy Orders, the average MAPE ranges from 26. 9% for Vertisols to 47. 3% for Vertisols. On Mollisols, the most represented order in the KSSL dataset, the average MAPE is 27. 4%. The variability of the metrics increases as sample size of the test set decreases as it is the case for Gelisols with a standard deviation of 9. 9% for its MAPE. The average bias values for CNN predictions evaluated on all data is 0. 0047 cmol(+). kg -. When evaluated on individual Soil Taxonomy Orders, bias values range from -0. 04 to 0. 02 cmol(+). kg -  for less represented soil orders such as Gelisols or Histosols. As for the PLSR, the average bias values predictions evaluated on all data is -0. 0002 cmol(+). kg -1. When evaluated on individual Soil Taxonomy Orders, bias values range from -0.
A detailed analysis of the correlation levels is conducted below. To only consider the general trends at this stage, a high level of correlation is observed between the GradientShap values calculated for Spodosols, Entisols and the Undefined category. The GradientShap values calculated on Mollisols, the most represented Soil Taxonomy Orders in the KSSL database, are highly correlated positively with Vertisols and negatively with Spodosols and Entisols. On the other hand, the GradientShap values computed for Andisols, Aridisols and Ultisols show little correlation with other orders and appear to be distinctive. Finally, the GradientShap values of Gelisols and Histosols correlate strongly with each other but show a low correlation with the other orders, suggesting a distinctive cluster.
In addition, the PLSR model appears to be dominated by majority of measurements available in the dataset within the 0. 3 to 1 cmol(+). kg -  range (-0. 5 to 0 when log 10 -transformed). In contrast, the CNN model has clearly superior predictive power in this range, despite the lack of representativeness of this type of data in the training set. As mentioned, the early stopping technique was used to train our models. In the case of the CNN, the training was stopped after 3 epochs without validation loss decrease to avoid overfitting. However, Fig. 5  showing the validation curves by Soil Taxonomy Orders allows for a more in-depth and differentiated analysis. Two groups of validation curves were identified, one group of curves showing a steady decrease throughout the training period and another group whose curves often level off or start increasing again by the end of the training period.
Our interpretation is that the overall validation loss decrease was mainly driven by Mollisols and Alfisols orders, the two most represented orders in the KSSL dataset. More training epochs would have conducted most likely to even better performance on those orders. However, the group including Gelisols, Spodosols and Ultisols validation losses stopped decreasing after 100 epochs (halfway through the training period). We are overfitting those soil orders. Hence, a more conservative approach would have been to stop training at 100 epochs.
The different scenarios will then be interpreted in light of domain expertise. In general, the pattern of higher level of performance as the data regime increases is consistent with the summary of Table  4. However there are exceptions. For instance, the case of Vertisols is instructive, where, despite the low data regime, very good level of performance is reported. The analysis of the correlation of its GradientShap values shows that it is (negatively) highly correlated with Mollisols and Entisols, both soil orders very well represented in the KSSL database and with Spodosols. The 2:1 type clay minerals (e. g. , micas, vermiculites, and smectites) determine soil's ability to retain potassium in most cases  (Sawhney, 1970; Sharpley, 1989; Parfitt, 1992; Coulombe et al. , 1996). In general, Vertisols have high clay content with relatively simple clay mineralogy dominated by smectite  (Coulombe et al.
93). In the United States, smectites are also common in Mollisols  (Fenton, 1983; Nettleton and Peterson, 1983). The similar smectitic nature of Vertisols and Mollisols may have also driven the unexpected high level of performance for Vertisols, suggesting a reuse of spectral features learned on Mollisols, very well represented in the KSSL database. In contrast, the level of performance for Histosols and Gelisols is the lowest. Their GradientShap values are highly correlated with each other (0. 95) but not with the other orders. Indeed, they show unique spectral features importance in the 3000-2800 cm -1 region, related to high organic matter content that control the cation exchange capacity of the soil and, hence K ex  (Le Guillou et al. , 2015). Both their low data regime and unique GradientShap values might explain the poor performance.
Acknowledgements This work was carried out in the context of the IAEA funded Coordinated Research Project ( CRP D1. 50. 19 ) titled " Remediation of Radioactive Contaminated Agricultural Land ", under IAEA Technical Contract n°23685. This work is a direct outcome from the collaboration between  IAEA  and  The Food and Agriculture Organization (FAO)  of the  United Nations. We thank the support from the  Global Soil Partnership of the FAO and its Global Soil Laboratory Network initiative on soil spectroscopy (GLOSOLAN-Spec ) which is actively supporting countries to develop their capacities on soil spectroscopy. We also thank  Richard Ferguson  from  Kellogg Soil Survey Laboratory  for providing access to the  USDA  MIR soil spectra library and the required training sessions for its operation.
Distinct core genome-based MLST schemes have been proposed for high resolution typing of the foodborne pathogen Listeria monocytogenes, ranging from 1013 to 1827 loci  [12, [23] [24] [25] , including an open-source reference cgMLST scheme of 1748 gene loci that is used worldwide  [26, 28, 28, 30, 31, 31]  and curated in the open-source Bacterial Isolate Genome Sequence database (BIGSdb)  [26]. Several parameters such as genetic background of tested strains  [32] , successive platings  [33, 38] , replicates of DNA extraction and sequencing  [35] , targeted depth of coverage  [24, 28, 36] , estimated depth and breadth of coverage  [13, 38, 42]  and assembly quality  [39] , may impact alleles called, compromising cgMLST profiles reproducibility and the definition of outbreak clusters. We therefore aimed to identify in vitro and in silico parameters impacting the precision of cgMLST profiles from six cgMLST complete workflows while assessing clustering concordance using the cut-off of 7 alleles mismatches  [24].
Results The experimental plan set-up for this study (Fig. 1A ) allowed us to build an accurate dataset of paired-end reads controlling the depth of coverage (Fig. 1B-i ), statistically identify parameters explaining the cgMLST precision among a large set of in vitro and in silico parameters (Fig. 1B -ii), and illustrate graphically those parameters explaining the cgMLST precision (Fig. 1B-iii ). Here, we focused on the precision (i. e. identical alleles against reference circular genomes (IAAR)) and completeness (i. e. identified alleles against schema (IAAS)) of cgMLST profiles, rather than accuracy, because allele differences were observed when comparing cgMLST profiles of reference circular genomes from compared cgMLST workflows (i. e. BIGSdb, INNUENDO, GENPAT, SeqSphere, BioNumerics and MentaLiST) (Fig.
Benchmarking dataset of downsampled reads Paired-end reads used for downsampling (n = 42) contained enough reads (3. 77 ± 0. 71 × 10 6 ) to prepare a dataset of downsampled paired-end reads to process with the selected cgMLST workflows. This dataset presented the highest expected read depth of coverage (i. e. 100X), as well as high and stable average Phred quality scores (34. 64 ± 0. 07) and percentages of Phred quality scores higher than 30 (93. 00 ± 1. 29%). No single nucleotide variant (SNV) was detected during Confindr-based exogenous DNA contamination screening  [41]  in the dataset used for downsampling. Regardless of the targeted read depth of coverage (Dr) defined according to kmer depth (Dk) with BBNorm (ranging from 10X to 100X)  [42] , the breadth of coverage of the downsampled paired-end reads estimated with BBMap (n = 420)  [42]  was very high (> 99.
0 × 10 -16 ) Fig. 1  Experimental plan (A) and data analyses (B) aiming at controlling a dataset of downsampled paired-end reads (i: n = 420 paired-end reads), as well as identifying statistically (ii) and confirming significant parameters (iii) explaining the Listeria monocytogenes cgMLST precision (identical alleles against circular reference genomes: n = 2520 cgMLST typing) among in vitro (i. e. tested reference genomes, successive platings, as well as replicates of DNA extraction and sequencing) and in silico (i. e. targeted depth of coverage, read depth and breadth of coverage, assembly metrics, cgMLST workflows, identified alleles against schema) parameters (n = 57 parameters of interest) for BioNumerics (Additional file 5F), while poorly correlated linearly with the other parameters for BIGSdb (Additional file 5B: p > 9.
Further studies may be needed to assess precision at higher coverage, yet 100X is enough high for L. monocytogenes cgMLST typing. Indeed, bacterial genomes sequences deposited in public databases (e. g. RefSeq, independently of the considered assembly surveillance project) are mostly generated at ≤100X sequencing depth (range: 30-150X)  [66]. Our main goal here was to provide guidance concerning the "standalone" solutions that can be adopted today for assembly and allele calling following developers' recommendations. Our results suggests that the assembly pipelines may impact the cgMLST precision to a greater extent than the allele calling pipelines. This hypothesis should be further confirmed assessing the impact of allele callers on cgMLST precision pipeline. However, results    [46]. To foster interoperability between the tested cgMLST solutions, the impact of different allele calling settings on cgMLST precision and nonidentical calls (i.
2. 2)  [49]  internal module based on Bowtie2 (version 2. 2. 9)  [73]  and Samtools (version 1. 3. 1) [74]. In total, 420 paired-end reads were produced to assess the impacts on cgMLST precision of in silico parameters (Fig. 1 ). Following de novo assembly steps recommended by developers detailed below, the 420 paired-end reads were processed through the six cgMLST workflows of interest (n = 6 × 10 x [3 × 2 + 3 × 2 × 3 × 2] = 2520 cgMLST results) (Fig. 1 ). Then, de novo assembly metrics of the assembly-based cgMLST workflows were assessed with Quast (version 5. 0. 2)  [75]  and combined with MultiQC (version 1. 9)  [76]  (n = 5 × 10 x [3 × 2 + 3 × 2 × 3 × 2] = 2100 quality results assessing 48 assembly metrics) (Fig. 1 ).
BIGSdb Paired-end reads were de novo assembled using fq2dna version 21. 06 (https:// gitlab. paste ur. fr/ GIPhy/ fq2dna; strategy B; default settings). The corresponding fq2dna pipeline consists of trimming and clipping of low-quality reads and adapters with AlienTrimmer (version 2. 0)  [77] , sequencing error correction with Musket (version 1. 1)  [78] , paired-end read merging with FLASh (version 1. 2. 11)  [79] , coverage homogenization with ROCK (version 1. 9. 3; https:// gitlab. paste ur. fr/ vlegr and/ ROCK)  [81, 82, 90] , and de novo assembly with SPAdes (version 3. 15. 0) [57]. In brief, the paired-end reads were first preprocessed through deduplication, clipping, trimming (Phred score threshold: 15, minimum read length: 50 bp) and error correction. Second, two distinct sequence datasets were created for each paired-reads by merging or not the pre-processed paired-end reads.
Third, the coverage depth of the two read datasets (i. e. merged or not) was homogenized to 60X (i. e. digital normalization procedure), and each of the two resulting subsets of paired-end reads was used to infer a de novo genome assembly. The most precise between the two assemblies was selected by maximizing the number of genes completely contained within assembled contigs (E-size)  [83]. Finally, the selected assembly was used together with its corresponding paired-end reads to infer a genome coverage profile (GCP) (i. e. distribution of the number of assembled bases per sequencing depth value)  [84]. Based on the coverage profile, sufficiently long (> 1000 bp) and significantly covered scaffold sequences were finally selected. Contigs smaller than 300 bp were ignored. Draft assemblies were uploaded in a dedicated project in BIGSdb-Lm (https:// bigsdb.
INNUENDO As proposed by the cross-sectoral platform for the integration of genomics in the surveillance of food-borne pathogens (INNUENDO), the cgMLST workflow INNU-ENDO was based on de novo assembly and allele calling using INNUca (version 4. 2. 2)  [49]  and chewBBACA (version 2. 6. 0; default setting)  [14]  pipelines, respectively. More precisely, the INNUca assembler performs successively read control with FastQC (version 0. 11. 5) [79] , trimming with Trimmomatic (version 0. 36; clipping 3:30:10:6; sliding window 5:20; leading 3; trailing 3; minimum length 55)  [58] , coverage estimation with the internal module based on Bowtie2 (version 2. 2. 9)  [73]  and Samtools (version 1.
MentaLiST Working directly with the raw paired-end reads, MentaL-iST does not require prior genome assembly (i. e. de novo assembly or reference genome mapping)  [19]. In brief, the workflow MentaLiST (version 1. 0. 0) implements the principle of kmer counting  [54]  and data compression to decrease dataset sizes and execution duration based on the construction of a coloured de Bruijn graph  [87]. After assessment of all kmers present on the schema of alleles for each locus stored as a kmer hash map, all alleles that contain kmers from reads of a given sample will receive one vote, and the called alleles are those with the most votes for each locus  [19]. The argument "--fasta" of MentaLiST was used to perform cgMLST of the three ATCC reference assemblies used in the present study. The default parameters of MentaLiST were applied in the present study considering multiple possible alleles (+) and partially covered alleles (-) as identical calls, as well as missing loci (0 or 0?) and new allele (N) as nonidentical calls.
The PCA and GLM were selected because of their abilities to manage together categorical and numerical parameters. The in vitro parameters of interest include 4 categorical parameters (i. e. tested reference genomes, successive platings, as well as replicates of DNA extraction and sequencing). The in silico ones include 2 categorical (i. e. cgMLST workflows and targeted read/kmer depth of coverage) and 51 numerical parameters (i. e. read depth of coverage, read breadth of coverage, 48 assembly metrics and number of identified alleles against schema) (Additional file 3). The R-scripts dedicated to statistical analyses are available in GitHub (https:// github. com/ Nicol as-Radom ski/ Downs ample dReads and https:// github. com/ Nicol as-Radom ski/ cgMLS Tcomp arison).
Principal component analyses The exploratory PCAs aimed at increasing interpretability and minimizing information loss at the same time, by reducing the dimensional of the large dataset of numerical parameters through projection of data points on the first few principal components  [90]. Two different PCAs were performed in the present study. The first PCA assessed the behavior of the response variable (i. e. the parameter to explain: IAAR) together with the explanatory variables corresponding to in silico numerical parameters of interest estimated through all assembly-based and assembly-free cgMLST workflows (i. e. the parameters potentially explaining the response variable: DEPTH, BREADTH and IAAS). The PCA was repeated excluding the assembly-free workflow MentaLiST (i. e. DEPTH, BREADTH, assembly metrics and IAAS) to additionally evaluate the impact of 48 assembly metrics on cgMLST precision for a total of 52 numerical parameters (i.
e. DEPTH, BREADTH, 48 assembly metrics, IAAS and IAAR). For readability of the illustrations, these numerical parameters were grouped together according to PCA outcomes and only one parameter from each group was represented (Additional file 4). These PCAs were systematically performed in comparison to the in vitro and in silico categorical parameters of interest (i. e. tested reference genomes, successive platings, as well as replicates of DNA extraction and sequencing, targeted read/kmer depth of coverage and cgMLST workflows). These PCAs were performed with the ggplot2-based biplot R library  [91]  called "ggbiplot" (https:// github. com/ vqv/ ggbip lot) requiring R libraries "usethis" and "devtools"  [72].
MST-based clustering The cgMLST clustering was represented through MSTs according to parameters explaining the cgMLST precision. The cgMLST profiles from each workflow (Additional file 2) were used to build MSTs using Bionumerics software (version 7. 6. 3). Missing alleles calls were ignored in the MST differences calculations. The MST clusters containing at least two genomes and allele differences ≤7, were highlighted in grey.