dataesr/scientific-paragraphs-categorization huggingface dataset:
the choice of a k-mer length, the key requirement is that k-mers should be sufficiently long so that most of them will be specific to each genome, thereby capturing genuine abundance patterns of individual genomes. In our experiments, the k-mer length  (31)  was chosen to be close to the value used by the authors of ref  [11]  to analyze their largest (terabase-sized) dataset. Some limited experiments with varying k-mer length values were carried out on smaller subsets of the data to check that small variations in k-mer size did not result in disproportionate differences in clustering outputs. In choosing the k-mer length, we were also guided by the observations in ref  [21]  that k-mer similarity between genomes at different k approximates various degrees of taxonomic similarity, with k=31 appearing to correspond to species-level similarity.
g. the 1 norm) and C is a constraint set for the dictionary (positivity constraints can be added to α as well). The following optimization scheme was used (FL stands for fused lasso): spiked reads (up to 4000) distributed among the samples in LSA's test dataset was sufficient to yield contigs covering a large fraction of the plasmid genome upon assembly, we derived a new dataset only containing 0 to 100 paired-reads (14 samples contained 100 paired-reads while 4 were entirely devoid of plasmid reads) and used the latter for this experiment. This dataset is available on the website associated to this publication's material. After checking that no kilobase-sized contig could be assembled in any of the samples -thus precluding the application of contig binning-the dataset was processed by our pre-assembly pipeline using the following settings: a k-mer length of 30 and a hash size of 22 were used to build the k-mer abundance matrix, the latter was decomposed by a svd and the columns of the eigen-kmer matrix were clustered using a cosine similarity threshold of 0.
Second experiment for comparing assembly-first versus bin-first strategies The raw sequence data from 18 (randomly chosen) individuals of the LLDeep cohort were either assembled individually (i. e. on a sample by sample basis) with metaSPAdes (v3. 13. 0) followed by contig binning across samples with the MetaBat2 adaptive algorithm  [19] , or the raw reads were clustered using our read-level binning pipeline, followed by metaSPAdes assembly of the resulting partitions/bins. The raw reads were first mapped to the assembled contigs using bwa-mem  [22]  using default parameters. MetaBat2 was then invoked in the following way: first, the jgi summarize bam contig depths script was called to compute contig abundance statistics from the read mapping bam files, with the default options (minimum percent identity for a mapped read: 0.
97, minimum contig length: 1000, minimum contig depth: 1). The metabat2 program was then called using the default parameters (minCV 1. 0, minCVSum 1. 0, maxP 95%, minS 60, and maxEdges 200) and the previously generated coverage statistics file, leading to the generation of 225 bins covering 694000907 bases. For the comparison, our sparse coding pipeline was then executed under the same settings as in the full cohort analysis (hash size and k-mer size equal to 30 and 31 respectively, and default parameters for the dictionary learning and sparse decomposition of the abundance matrix), with the exception of the number of components that was matched to the number of bins (225) generated by MetaBat2. To generate Figure  1 , the complete genomes retrieved using both approaches were aligned (using nucmer  [26]  with default parameters) to individual assemblies from all the samples, and the number of distinct contig hits (≥99% identity and ≥ 2500 bp) was recorded.
Comparison of read binning algorithms The virtual cohort dataset described above and in ref  [15]  was used to compare the clustering accuracies of the original LSA  [11]  and sparse coding methods, as well as the performance of directly clustering the columns of the abundance matrix using a k-means algorithm as a baseline. The reads to genome memberships being comprehensively known in these controlled genome mixtures, clustering accuracy metrics (precision, recall and F-measure) could be quantified as in ref  [27]  (Table  1 ). Briefly, each bin is first mapped to its most abundant (in terms of number of reads) genome (note that if each bin is mapped to a single genome, a given genome can be mapped to multiple bins). Precision is defined as the ratio of reads originating from the mapped genome to all the bin's reads.
Evaluation of read enrichment levels To assess whether we could identify genomes segregating at consistently low abundance levels in real-life datasets, we characterized the abundance of a dozen of MAGs reconstructed from the LifeLinesDeep cohort analysis by directly mapping the raw reads from the original samples against them. Given the large size of the cohort (and the significant amount of computer resources associated with this analysis), and given that our objective was to establish whether consistently rare genomes can be identified by the method or not, this analysis was performed on a limited number of genomes. Relative enrichment levels were estimated by mapping the original reads (after removal of duplicated reads) to the genome-resolved partitions using bwamem  [22]  with default parameters.
Comparison of genome-resolved partitions to reference genomes To assess the novelty of the genomes assembled from individual partitions produced by our pipeline through the analysis of the LifeLines-Deep cohort, we screened them against two reference libraries. First, the genomes were compared to the Kraken2 (v1) database (https://ccb. jhu. edu/software/kraken2/) built from NCBI's refseq bacteria, archaea, and viral libraries (in August 2018), using the Kraken2 classifier  [39]  and a confidence score threshold of 0. 2. Second, the same genomes were compared against the Human Gastrointestinal Bacteria Genome Collection  [13]  (HGG, encompassing more than 100 GB of sequence data) using the nucmer aligner  [26]  with default parameters. A genome was marked as already known if it shared at least ten distinct 99% identity alignments of length ≥5 kb to any HGG entry.
Binning implementation Code for the pipeline used to perform the analysis of the LifeLinesDeep cohort can be cloned from https://gitlab. com/kyrgyzov/lsa_slurm, while a lightweight implementation of key algorithms (including sparse NMF) can be downloaded from https://github. com/vincentprost/LSA_NMF; they draw on the code base of the LSA tool (ref  [11]  and https://github. com/ brian-cleary/LatentStrainAnalysis), and on the SPAMS (SPArse Modeling Software) library that can be downloaded from http://spams-devel. gforge. inria. fr/. The analysis of the metagenomes from the LifeLines-DEEP cohort was carried out on a Bullion S6130 octo module server equipped with 2 Intel Xeon Haswell E7-4890 v3 CPU (18 cores) per module, 8 TB of RAM and 35 TB storage. Most of the tasks being embarassingly parallelizable, they were run through a Slurm workload manager.
The use of efficient online matrix factorization techniques  [25]  leads to limited memory footprints. Even though we leveraged here a powerful computer infrastructure to carry out the analysis of the large cohort dataset (10 terabytes of data), our pipeline is routinely executed on commodity hardware for smaller projects. Table  4 : Cluster assignments of reads from a target genome versus background (unrelated) reads. Nearly all the 2800 reads from the target genome segregating at low levels in the samples (100 paired-reads per sample in 14 samples, none in the remaining samples) are binned in a single partition using our bin-first pipeline, leading to the complete genome after assembly. No kilobase-sized contig could be assembled from any individual sample, making the assembly-first protocol inoperable (see main text).
In the case of B. burgdorferi, several hundred strain types have been characterized using the MLST scheme developed in  [11] , while only 111 fully sequenced B. burgdorferi genomes 1 are currently available in the NCBI databases. MLST strain types thus provide a finer-grained picture of the strain diversity of this pathogen, which motivates the need for developing novel diversity estimation methods that combine NGS data and the wealth of strain types already characterized by  MLST. In principle, this problem is a special instance of estimating the diversity and abundance of microbial strains from metagenomics data, a problem for which several accurate methods have recently been developed (e. g. [12] [13] [14] ). De novo methods, such as DESMAN  [12] , cannot take advantage of known reference strains or alleles and are likely to be confounded by the high similarity observed between strain types.
In the second stage, it assigns combinations of these alleles, with corresponding proportions, to each sample, while maximizing the usage of known strains and minimizing the number of novel strains, a parsimony-based approach that has been shown to perform well in related contexts  [15]. We evaluate our approach on simulated samples and find that it is accurate in identifying both the fractional allele composition at each housekeeping gene, as well as the complete strain types present in each sample. We then apply it to a dataset of 24 real tick samples containing B. burgdorferi extracted via whole-genome capture, and find a substantial amount of diversity, as well as a number of new strains. In conclusion, our work provides a robust and reproducible pipeline for accurate strain typing via MLST from WGS data even in the presence of substantial within-host heterogeneity.
Methods Terminology. An MLST scheme is composed of a set of loci together with a database of known alleles for each locus  [16]. An allele distribution for a given locus is a set of alleles for this locus together with a proportion assigned to each allele; the proportions must be non-negative and add up to 1. A strain type is an assignment of a specific allele to each gene of the MLST scheme. A strain type distribution is a set of strain types together with a proportion assigned to each strain type; the proportions must once again be non-negative and add up to 1. A sample is a WGS dataset obtained from a single host that contains the sequence data from one or several pathogen strains present in the host (see Fig. 1 ).
Data. In the present work we use the traditional B. burgdorferi MLST scheme  [11]  composed of 8 housekeeping genes having a combined total of 1726 known alleles. For each locus, the various known alleles differ from one another primarily by single nucleotide polymorphisms (SNPs), with small indels also appearing in 4 out of the 8 genes. The number of known strain types is 753. Problems and contribution overview. The problems we address in this work take as input (1) an MLST scheme together with databases of known alleles and strain types and (2) WGS data for a set of samples that are mapped using a short-read mapper of choice onto the database of known alleles for the provided MLST scheme. It then proceeds in two stages, each addressing a specific problems: • The Allele Diversity Problem.
For a given sample and a given locus of the MLST scheme, given the mappings of DNA reads onto the known alleles for this locus, detect the alleles present in the sample and the corresponding allele distribution. • The Strain Diversity Problem. Given a set of samples and an allele distribution for each locus at each sample, compute a strain type distribution per sample that requires the smallest number of novel strain Fig. 1  A dataset with two samples and an MLST scheme of three loci (genes clpA, clpX, nifS). The strain type distributions require 5 different strains as the strain (clpA_1,clpX_1, nifS_7) appears in both distributions types among all considered samples, which are as similar as possible to known strains.
Implementation All scripts are written in Python 2. 7. Both ILPs are formulated and solved using the Python API of IBM's CPLEX 12. 6. 3. 0. For the ADP, each sample and each locus may require a different number of variables in the ILP. To evaluate the practical resources requirements of our ILP, we choose the sample SRR2034336, which has the largest number of reads among our samples. The average number of variables across each gene for this sample is 20,112, the maximum RAM usage is ∼1. 5GB, and the time taken for all 8 genes is ∼33 min on a 4 CPUs Intel ® Xeon ® machine. The total time taken for each sample is presented in Additional file 1. For the MILP solving the SDP on all 30 samples, there are a total of 21,885 variables, with 10,682 strain type variables, 10,795 proportion variables and 408 error variables.
ADP simulation. For each locus of the Borrelia MLST scheme, we drew a random number k ∈[ 2, 7], selected a random allele from the database and selected k -1 other alleles, each at edit distance at most d (a given parameter) from the first one chosen. Next, we randomly assigned proportions to each selected allele, which sum up to 1, then generated reads with coverage c. To align the simulated reads to the alleles of the database, we used Bowtie v0. 12. 7  [19]. We used parameters c ∈ {30, 100, 300} and d ∈ {5, 10, 15, 20, 25} and we ran 40 simulations for each combination of these parameters. For this experiment, we compared our results with the results obtained with Kallisto  [20] , a recent method for isoform abundance estimation that has also been applied to metagenomics.
For all experiments, we assigned random proportions to the chosen strains. Full pipeline simulation. We generated strain type distributions as in the SDP simulations above, then generated reads as in the ADP simulations. The generated reads were then fed to the ADP solver, and the ADP results were provided as input to the SDP solver. We compared our pipeline with strainEST  [13] , a recent method to estimate the strain composition and abundance in metagenomics datasets. However, strainEST does not predict novel strain types. Hence, to complement EvoMod1, 2, 2e and 2n, we added an additional simulation where we randomly pick k = {1, 2} existing strains and assign them random proportions. The statistics described above rely on a stringent measure of accuracy in calling alleles, strain types or proportions.
Simulated data We describe several sets of experiments based on simulated data. In the first one we evaluate our method for the ADP problem and compare it with Kallisto. In the second experiment, we evaluate our method for the SDP, using simulated allele frequencies, i. e. perfect input to the SDP, and 4 different evolutionary models explaining the diversity within a sample, from a simple model based on within-host mutations to a complex model based on co-infection and recombination. We then repeat the same experiment using simulated short reads, to evaluate our pipeline on ADP + SDP. Finally, we compare our method to strainEST using simulated datasets with no novel strains (the ideal case for strainEST) and then datasets simulated using evolutionary modes identical to the ones in the previous experiment.
Conclusion We presented an optimization-based pipeline for estimating the within-host strain diversity of a pathogen from WGS data analyzed in the MLST framework. This is a specific instance of estimating the diversity of a bacterial pathogen from metagenomics data, focusing on withinhost diversity and taking advantage of the availability of a large database of known MLST strain types. Our approach is composed of two main steps, each of a different nature; the first step detects the alleles present in a sample from the sequence data, while the second step estimates the strain diversity based on the output of the first one. In both steps we follow a parsimonious approach that aims at explaining the input using as few alleles or novel strains as possible. The main contribution of our work is the formulation and the solution of the Strain Diversity Problem for a group of samples.
We can observe in our results on real data several groups of very closely related strain types, sometimes differing by a single SNP, which likely results from this issue. At the moment, our approach to this problem is to post-process the result of our pipeline to identify clusters of closely related strains, but other more principled approaches should be explored. Notwithstanding the aforementioned issues, our experiments suggest a strikingly high diversity in our dataset of 24 tick samples. This is not altogether surprising since the library of known strains might be limited, and within-host (or, more precisely, within-vector) evolution might result in the presence of a number of strains that only differ by a small number of SNPs in one or two loci of the MLST scheme. Our work is, to our knowledge, the first comprehensive approach to the problem of reference-based detection of pathogen diversity in a collection of related samples that considers novel strain types.
The Strain Diversity Problem (SDP) can be defined as follows. It takes as input four elements: (1) the set G ij = {g ij1 , g ij2 ,. } of all alleles selected for locus j in sample i (2) the set P ij = {p ij1 , p ij2 ,. } of proportions of these alleles, (3) a database of known strain types, (4) an error bound ∈[ 0, 1]. From now on, we assume that there are loci and m samples.
Availability of data and materials The programs and simulated datasets generated and/or analysed during the current study are available at https://github. com/WGS-TB/MLST. The real data are available from the NCBI Sequence Read Archive https://www. ncbi. nlm. nih. gov/sra under accession numbers SRR2034333, SRR2034339, SRR2034344, SRR2034349, SRR2034355, SRR2034334, SRR2034340, SRR2034345, SRR2034350, SRR2034356, SRR2034335, SRR2034341, SRR2034346, SRR2034351, SRR2034360, SRR2034336, SRR2034342, SRR2034347, SRR2034352, SRR2034362, SRR2034338, SRR2034343, SRR2034348, SRR2034353.
The Strain Diversity Problem (SDP) can be defined as follows. It takes as input four elements: (1) the set G ij = {g ij1 , g ij2 ,. } of all alleles selected for locus j in sample i (2) the set P ij = {p ij1 , p ij2 ,. } of proportions of these alleles, (3) a database of known strain types, (4) an error bound ∈[ 0, 1]. From now on, we assume that there are loci and m samples. 1, or 10%). The Strain Diversity Problem: problem definition and tractability.
(2011). DNA of each sample (200 ng) was digested with ApeKI (New England Biolabs, Hitchin, UK). Digestion took place at 75 °C for 2 h and then at 65 °C for 20 min to inactivate the enzyme. The ligation reaction was completed in the same plate as the digestion, again using T4 DNA ligase (New England Biolabs) at 22 °C for 1 h, and the ligase was inactivated prior to pooling the samples by holding it at 65 °C for 20 min. Ligated samples were pooled and PCR-amplified in a single tube. Complexity was further reduced using PCR primers with one selective base (A) as performed by  Sonah et al. (2013). Single-end sequencing was performed on a single lane of an Illumina HiSeq2000. The Illumina Hiseq 2000 sequencing raw data are available in the NCBI SRA (Sequence Read Archive), under the accession numbers SRP109295 for the 21 mandarin, pummelo, orange, grapefruit and clementine sequences already published in  Oueslati et al.
Thus, regions with heterozygosity values >0. 2 were assumed to be introgressed and were removed. The patterns of similarity between each accession and the centroid of the four horticultural groups were also examined. The regions with an increase in heterozygosity were associated with a decrease in similarity to their representative horticultural group and an increase in similarity to the horticultural group involved in the introgression. An example is given for chromosome 2 of the 'King' mandarin (Supplementary Data Fig. S4 ). A heterozygous introgression was clearly identified at the end of the chromosome. Heterozygosity increased with a decrease in similarity, starting at 25 Mb and continuing to the end of the chromosome. Similarity analysis was particularly useful to identify homozygous introgressions as described by  Oueslati et al.
maxima introgressions and one, 'Shekwasha' mandarin, has a small introgression of C. micrantha. No C. maxima introgressions were detected in 'Shekwasha', 'Cleopatra' and 'Sunki' mandarins. Limited introgressions were identified in 'Szibat' mandarin (1. 49 %), 'Ladu' mandarin (1. 72 %), 'Nan Feng Mi Chu' mandarin (1. 74 %) and 'Se Hui Gan' mandarin (1. 39 %). 'Satsuma' and 'King' mandarins were distinguished from all the other introgressed mandarins by their higher rate of C. maxima introgressions (22. 6 and 19. 5 %, respectively). Our results for newly studied varieties confirm that most edible mandarins are introgressed by C. maxima fragments as previously detected from WGS  (Wu et al. , 2014 (Wu et al. , , 2018)) , 454 amplicon sequencing data  (Curk et al. , 2015)  and GBS data  (Oueslati et al.
It was suggested that ffCNNs mainly focus on local, texture-like features, while humans harness global shape computations (1, 14-18; but see  19) ). In this context, it was shown that changing local features of an object, such as its texture or edges, leads ffCNNs to misclassify  (14, 15) , while humans can still easily classify the object based on its global shape. There are no widely accepted diagnostic tools to specifically characterize global computations in neural networks. Models are usually compared on computer vision benchmarks, such as ImageNet  (20) , or with neural responses in the visual system  (21, 22). One drawback of these approaches is that the datasets are hard to control. Psychophysical results can be used to fill this gap and create wellcontrolled challenges for visual models, tailored to target specific aspects of vision  (23).
In this respect, we have shown that CapsNets generalize more similarly to humans than ffCNNs and standard recurrent networks in the context of global (un)crowding. Relatedly, CapsNets may also be more resilient to adversarial attacks  (57, 58) , generalize better to new viewpoints  (59) , and outperform ffCNNs on difficult visual tasks with scarce training data such as tumor detection  (60) , also suggesting potentially more humanlike generalization than ffCNNs. One limitation in our experiments is that we explicitly taught the CapsNets which configurations to group together by selecting which groups of shapes were present during training (e. g. , only groups of identical shapes in experiment 1). Effectively, this gave the network adequate priors to produce uncrowding with the appropriate configurations (i.
e. , only identical, but not different flankers). Hence, our results show that, given adequate priors, CapsNets explain uncrowding. We have shown that ffCNNs and CNNs with lateral or top-down recurrent connections do not produce uncrowding, even when they are trained identically on groups of identical shapes and showed learning on the training data comparable to the CapsNets (furthermore, we showed previously that ffCNNs trained on large datasets who are often used as general models of vision do not show uncrowding either; 18). This shows that merely training networks on groups of identical shapes is not sufficient to explain uncrowding. It is the recurrent segmentation in CapsNets that is crucial. Humans do not start from zero and therefore do not need to be trained in order to perform crowding tasks.