dataesr/scientific-paragraphs-categorization huggingface dataset:
RFLP data detecting IS6110 insertion sequence and/or that of CRISPR locus (long called "Direct Repeat" or "DR locus", and the method, "Spoligotyping") have initiated MTC infraspecies MTC taxonomies strictly based on genotyping data  [21, 22, 23]. The taxonomy based on CRISPR locus, relying on the presence/absence of specific spacers, became soon the most extensively used and a worldwide database referred to as SpolDB and later SITVITWEB is including an increasing number of sublineages since 1999  [24, 25, 26]. The CRISPR locus was found to have carried 68 spacers in the most recent common ancestor to all MTC species except M. canettii  [27, 28]  and to have subsequently evolved by the loss of spacers or the integration of IS6110 sequences  [29]. Recurrent "signatures", i. e. the absence of specific spacers, easily detected by experts, led to the naming of LAM, CAS, S, X, etc.
The automation of CRISPR data use for classification has led to several web interfaces: spoligoforest to infer transmission chains SPOTCLUST and TB-Lineage for labeling new isolates  [29, 37, 38] , with TB-Lineage using a simplified taxonomy indicating large lineages as defined by Gagneux et al. [38, 39]. From 2000 on, tuberculosis taxonomy was complexified by the advent of large deletions  [40, 41]  and minisatellites termed MIRU-VNTR (mycobacterial interspersed repetitive units, variable number tandem repeats)  [42, 43, 44, 45, 46, 47]. The large acquisition of MIRU-VNTR data soon suggested that at least some specific labels provided using spoligotype patterns could be flawed  [48]. An independent database and the corresponding taxonomy was set up to classify isolates according to the standardized combination of 24 MIRU-VNTR patterns  [49] , and/ or Regions of deletion: MIRU-VNTRPlus  [47, 50].
tuberculosis complex isolates to taxonomic groups at different depths, but little time was by now invested in characterizing the concordance between the corresponding taxonomies. As a consequence, TB epidemiologists are often puzzled when trying to characterize their isolates. There is a clear need for using large studies and powerful algorithms on large datasets for establishing consensual infra-specific MTC taxonomy. Machine learning is a statistics science identifying relevant information in large datasets even when some data are missing. It involves pattern recognition in prototypes i. e. the identification of formal rules correlated to a characteristic of interest  [64]. Once patterns have been identified, assignation of unknown individuals is easy and very fast. Such method has been applied previously on spoligotyping data and helped identifying informative spacers to recognize experts-based groups  [65].
Weka is a work bench set up in 1997 and implementing state of the art machine learning algorithms. This ability proved critical for reducing assignation errors  [66]. It is very popular and was used in studies as diverse as epilepsy characterization using imaging data  [67]  and methylated DNA patterns linked to genetic diseases  [68]. Its swiftness enables to implement it in parallel on different type of data, so that complete annotation of very large dataset can be reached in a timeframe of one minute. In this work, we first completed the genotyping of a large dataset of 3,454 human M. tuberculosis isolates from the National Reference Center of Netherlands (RIVM) collected between 2004 and 2008  [69]. This data was used to further describe TB diversity and transmission dynamics in Netherlands and to clarify the potential of spoligotyping in molecular epidemiology.
We then used this large database as a reference for human M. tuberculosis worldwide diversity after complementing it with genotypes from Lineages 5 and 6 particularly underrepresented in the RIVM dataset. We annotated all these genotypes according to existing classification tools to search for stable correspondences between the underlying taxonomies and proposed a new consensus where the correspondences are made explicit. We finally used Weka software to learn in parallel classification procedures, handling Spoligotype or MIRU-VNTR data, reproducing original and new taxonomies, and made the most successful procedure available on-line. Altogether, our work successfully clarifies the correspondence between the existing M. tuberculosis complex taxonomies. This consensus can be retrieved for any sufficiently informed new genotype (best when including at least Spoligotype + 15 MIRU-VNTR) using our new web interface, TBminer.
Assignation to sublineages using available classifications Files of 500 24-VNTR and spoligotyping genotypes complying with all specified requirements were loaded onto TB-lineage and MIRU-VNTRPlus websites. In MIRU-VNTRPlus, default settings were changed to assign isolates to the closest inside the curated database as soon as the distance is of 0. 5 maximum (default = 0. 17), so that most isolates could be classified (n = 3382). TB-lineage could classify 3283 isolates (i. e. 97%). To assign genotypes according to SITVITWEB taxonomy, we first made use of an Excel version of SpolDB4 uncovering the 2881 first SITs implemented in SITVITWEB. The assignations were slightly corrected by taking into account recent knowledge on relatedness among Euro-American sublineages. For instance, CAMEROON genotypes previously referred to as LAM10_CAM and subsequently found not to be related to LAM were simply labeled CAM, TURKISH isolates previously referred to as LAM7_TUR and subsequently found not to be related to LAM were simply labeled TUR, H4 subsequently found not to be related to Haarlem were renamed URAL, and more precisely URAL1 when spacer 2 was present and URAL2 when spacer 2 was absent.
Classification learning using Weka A curated database enlarged to include more M. africanum isolates was imported into Weka. More specifically, curation removed all isolates for which at least one taxonomy was not able to provide an assignation or when one VNTR had a zero value (indeed absence of results was at some point in this long-lasting study recorded as 0 which may have lead to erroneous profiles); this concerned all lineages (data not shown) and almost all MIRU-VNTR so that it is not likely to have introduced any bias in the database. M. africanum isolates added to the RIVM curated dataset came from a Nigerian study. Altogether, the database counted 2,904 isolates including Weka was used to train a classifier using different Machine learning algorithms and a Vote procedure. Machine algorithms handle characteristics such as genotype patterns at different loci, and a selected feature to be predicted.
Every genotypic characteristic handled independently is called an "attribute". Different algorithms exist. A short description of their different characteristics is as follows: 1) J48 algorithm is the Weka version of the C4. 5 algorithm. J48 is used to learn decision trees using quantitative values of attributes. First, the pair (attribute, value) that optimizes a criterion (entropy, gini index) is used to split the data in two sample. Then, for each sample, if it is pure (only one lineage represented) or if it contains less than a predefine number of data, the tree growing is stopped, else another split is determined based on the same algorithm; 2) JRip algorithm (with Rip standing for Repeated Incremental Pruning, and J for Java, the programming language used to implement Weka) an algorithm already implemented on tuberculosis data for TB-lineage  [38]  consists in identifying an ordered list of complex rules using quantitative values, beginning with less prevalent class; if the first set of rules is fulfilled, the isolate is assigned to the corresponding lineage, otherwise the next set of rules is examined; a default assignation is proposed at the end of the list if no set of rules has been fulfilled; 3) Naive Bayes algorithm is based on the assumption that attributes are independent.
All these methods can undergo meta-bagging procedures to reduce the impact of overfit to the data. Overfitting occurs when lots of data are used to set up the classifiers so that some irrelevant features are included in the decision trees or rules. Meta-bagging consists in randomly selecting features and data used to induce the classifiers, a step that actually reduces the chances for learning irrelevant features. All algorithms can be combined and contribute to a Vote step during which the most frequent assignation is selected. Evaluation was performed for each algorithm using stratified cross-validation. Stratified cross-validation consists in partitioning the training dataset in a specified number of folds k, learn on k-1 folds and test the algorithm of the remaining fold. The number of folds was varied between 3 and n-1 (leave-one-out procedure, n corresponding to the total number of items).
When the number of folds is low, the computed accuracy is more likely to match that obtained on an independent dataset as the testing set is quite large, however, the rules or trees inferred may be less precise as they have been set up with smaller training data. The "Vote-10" algorithm (using all 5 original algorithms as well as all their 5 meta-bagging derivates) was applied on the complete training dataset to build up the Lineage prediction classifier available on TBminer website.
Datasets for independent evaluation of Lineage Prediction tool The first independent dataset was that of MIRU-VNTRPlus database uncovering 186 isolates from both human and animal tuberculosis. The second independent dataset was built by complete genotyping as described in Yasmin et al. [53]. Only isolates with "complete" spoligotype and 24-MIRU-VNTR genotypes i. e. lacking at most one VNTR (n = 225) were kept. Isolates were mainly from Punjab. Altogether, only 38 datapoints were missing out of 15,075, mostly from VNTR Qub26 (n = 10), Qub11b (n = 7) and ETR-A (n = 6).
Characteristics of the RIVM dataset and contribution of spoligotyping to molecular epidemiology We characterized by spoligotyping 3,454 DNA samples retrieved from as many clinical isolates of the RIVM collection  (2004) (2005) (2006) (2007) (2008). Genotypes obtained by 24-MIRU-VNTR and IS6110 RFLP profiles were already available  [69]. Ninety-nine percent of the patterns were successfully obtained (99. 4%; n = 3432). According to TB incidence estimates in the Netherlands for this period (n = 1330 per year as estimated in 2004), this represents around 65% of all TB cases  [75]. Assignation of genotypic profiles to described lineages was performed independently for each isolate and for each taxonomy using available on-line tools: 1) SITVITWEB database using spoligotype patterns and upgraded by the expert eye for new profiles, 2) MIRU-VNTR-Plus assignation tool implementing similarities on 24 MIRU-VNTR patterns, 3) TB-Lineage using both spoligotype patterns and 24 MIRU-VNTR information, 4) Borile et al.
Altogether, this RIVM dataset is typical of that of a Western country (predominance of Euro-American lineage) with worldwide immigration providing larger diversity. Euro-American isolates infra-lineage diversity (67. 9% of the total dataset) can be described using the MIRU-VNTRPlus classification. Most were labeled Haarlem (n = 777; 22. 5% of total dataset), second, LAM (n = 548; 15. 9%) and third, Cameroon (n = 260; 7. 5%). We assessed the clustering level in this database and set up an online tool to automate it, available at info-demo. lirmm. fr/TBminer/ (Clusters Analysis option). We used 24-VNTR profiles to define clusters as it stands as the current Gold standard for molecular epidemiology since 2006  [49, 69, 76]. We detected 109 to 295 clustered isolates each year (S2 Table ). When considering the global 5-years set, the number of clustered isolates reached 1,362 which was higher than the sum of clustered isolates for each separate year.
All spoligotype patterns of isolates belonging to the same 24-VNTR cluster were similar. The isolates carrying different spoligotype signatures often were collected on different years. Inside these 24-VNTR clusters, spoligotype patterns harboring more deletions were rather posterior to patterns with fewer deletions (n = 21), albeit with several exceptions (n = 7; Table  2 ). This confirms that both rare mutations in spoligotype pattern without changes in the 24-VNTR set may occur, and that convergence in 24-VNTR data for related isolates is not so rare. On the whole dataset, adding spoligotyping to 24-VNTR typing refined clustering by 10. 1% (S1 Fig), likely identifying most convergence events among 24-VNTR clusters that could be detected using Whole Genome data  [59]. All major lineages except Beijing were represented among these large clusters and no significant difference with the global representation of lineages could be detected (Fisher exact test n1 = 9, n2 = 2, p = 0.
Set-up of a correspondence table between the different M. tuberculosis taxonomies and proposition of a consensual "expert" taxonomy We took advantage of the large resulting database to examine the correspondence between the existing MTC taxonomies. Preliminarily, we removed all isolates for which no assignation existed for at least one of the taxonomy. Dataset was subsequently enriched in Afri1 and Afri2 isolates by including 52 additional isolates from a published study  [80]. The resulting dataset counted 2,904 genotype profiles with classifications by TB-Lineage, Miru-VntrPlus, SITVIT-WEB and Borile (S3 Table ). When browsing this material, we identified good concordance for 22 sublineages between the TB-lineage, MIRU-VNTRPlus, Borile and SITVITWEB classification tools. In addition, we could find a good correspondence with the SNP-based classification set up by Coll et al.
LAM3_S SITVITWEB sublineage could not find any clear correspondence in the other taxonomies; we therefore propose to temporarily abandon this sublineage and simply label the corresponding isolates "L4" as they clearly belong to the Euro-American lineage also known as lineage 4. We similarly propose to temporarily abandon MIRU-VNTRPlus Ghana sublineage. Our interpretation of such absence of correspondence for some sublineages with other taxonomies is that they represent an insufficient number of isolates to be relevant in a worldwide classification. Similarly, these sublineages are not described in the SNP-inspired taxonomy of Coll et al. [62, 63]. We measured the concordance of every existing taxonomies with the newly proposed "expert" one on all isolates of our database. This consensual "expert" taxonomy reached almost perfect concordance with that of TB-lineage as retrieved using 24VNTR and spoligotype data (99.
Rapid machine learning algorithms to achieve fine and specific classification using spoligotyping and/or MIRU-VNTR data Applying different classification tools on the same dataset helps understanding the real diversity in a sample, but this procedure is very time-consuming. We took advantage of our large amount of data and on machine learning to set-up a fast webtool combining all existing taxonomies including the newly proposed consensual "expert" one. We aimed at providing a classifier using spoligotype pattern, 24-VNTR profile and/or 15-MIRU profile. The 15-MIRU panel is a subset of the 24-VNTR loci with high discriminatory power. It has been validated alone or rather in combination with spoligotyping as an alternative to the 24-VNTR typing scheme for epidemiological studies  [49]. We used spoligotype data to predict spoligotype-inspired taxonomies (SITVITWEB, Borile et al.
The training dataset included all the 2,904 isolates described above. Accuracies of the different algorithms were stable when assessed with different levels of stratified cross-validation (S4 Table ) and were impacted both by the number of lineages in the taxonomy to be inferred and by the input genotypic data (Table  3 ): TB-lineage with only 7 lineages was the most easy to predict, with 99. 8% accuracy using the Vote-10 classifier. Spoligotype-related predictions reached higher accuracies potentially due to the simplest nature of this locus (complexity of 2 43 as compared to more than 5 24 i. e. a difference in complexity of more than 1,000). 15-MIRU data allowed for higher performances than 24-VNTR data when using the finest algorithms such as Vote-10, showing that phylogenetic information can be identified even among the most variable MIRU-VNTR loci.
Lineage Prediction available on-line on TBminer website A Lineage prediction tool on the website TBminer was set up to make the Vote-10 classifier available to all users. Underlying rules and trees were derived using the complete training dataset. The web interface allows to upload any genotypic data including CRISPR spacer data (1 to 43 standard spacer set), and 24 standard MIRU-VNTR numbers of repeats. When including at least 43-spacers spoligotype data or 15-MIRU, TBminer Lineage Prediction webtool provides an output file with 7 new columns, one column for each of the 7 predictions mentioned in Table  3  (Pred1 to Pred5bis), as well as 2 consensus columns clarifying whether the predictions using spoligotype data and MIRU-VNTR data are concordant (Pred6 and Pred6bis; Fig  3 ).
Precise performances of Lineage Prediction tool on the training set For TB-lineage taxonomy, sensitivity and specificity of Pred1 reached 100% for all sublineages (data not shown). For MIRU-VNTRPlus taxonomy (Pred2 and 2bis), median sensitivity of sublineage prediction was 98. 4% and 98. 1% when performed on 24VNTR and 15MIRU respectively. Both predictions had very high sensitivity for all sublineages, the minimum corresponding in both cases to Ghana sublineage (90% and 85% respectively). The median specificities were 99. 6% and 98. 7% respectively with minima for UgandaII sublineage (93% and 90% respectively, data not shown). For SITVITWEB-expert taxonomy that includes 50 sublineages, Pred4 predicted most sublineages with a sensitivity of 100% (n = 44) and the minimum sensitivity was 94. 1% for T4_CEU1.
Validation of the automatic assignations on independent datasets We tested the performance of our webtool to predict the assignations (as provided by all existing taxonomic tools and the newly proposed "expert one") of the isolates included in the reference database of MIRU-VNTRPlus. This database includes not only human but also animal isolates. We first classified all isolates using the standard tools (SITVITWEB adjusted using the expert eye, MIRU-VNTRPlus, TB-lineage) and inferred their expert assignation as described above (Table  1 ). We then ran our webtool to retrieve all assignations predicted by the learnt classifier obtained with Weka. We observed that the prediction of MIRU-VNTRPlus assignations (Pred2 tool) had an accuracy of 100% when assigning human isolates from lineages 1 to 6 (Fig  4A ).
It failed in predicting animal sublineages other than M. bovis as expected due to the absence of such isolates in the training dataset. Most of them (n = 14 out of 21; 67%) were assigned to the closely related lineage Lineage6_Afri1(WestAfrican2). Altogether, diversity picture of the whole sample as provided by Pred2 tool in Lineage Prediction module of TBminer was very similar to that of MIRU-VNTRPlus tool, overestimating only the prevalence of West African 2 lineage and being unable to identify peculiar animal isolates as well as M. canettii (Fig  4A ). The performance of the consensus tool (consensus between spoligo and MIRU-VNTR data as appearing in column 6_Consensus_Pred1_2_3_4) was characterized by measuring its ability to reproduce the "expert" assignation. The consensus tool proved able to identify more precise labels than the expert taxonomy for most Lineage_1 isolates and a significant proportion from Lineage_0 (Bovis-BCG) and Lineage_4 (such as H1 as compared to H3) (Fig  4B ).
It provided a less precise assignation for less than 20% of the Lineage 4 (possibly due to a lower rate of false positive) and was unable to recognize both animal isolates other than M. bovis and M. canettii isolates. Altogether, this tool provides precise assignations for typical human isolates and is able to detect when no known/implemented lineage corresponds to the uploaded genotype. We performed the same approach on a representative dataset of Pakistanis clinical isolates for which complete genotypes (at most one VNTR lacking) were available (n = 225). This dataset is interesting because it uncovers human isolates of an origin very different from that of the dataset used to set up TBminer Lineage Prediction webtool. Our rationale was that, if good performances could be observed with this dataset, this would mean that TBminer Lineage Prediction tool is robust to the geographical origin of the isolates.
For instance, we identified isolates carrying a Beijing spoligotype but a VNTR profile characteristic of CAS isolates. As recent studies have identified Pseudo-Beijing isolates in the Middle East region, these specific isolates might well represent new cases of this sublineage identified as a bona-fide Lineage3-CAS  [84]. Interestingly, the consensus based on 15-MIRU and spoligotype (6bis) performed almost as well as that using 24-VNTR (91. 6% instead of 92% concordance), with the only discordance reported for 24-VNTR being moved to the unassigned group (No consensus) (data not shown). We proceeded to a third validation test using the SNP-classification provided by Abadia et al. [56]. In that study, several SIT were shown to be associated to phylogenetic SNPs classifying them differently than SITVITWEB.
Discussion We characterized M. tuberculosis complex diversity in the Netherlands using the 3 most widely-used genotyping techniques, 24-MIRU-VNTR and spoligotyping and IS6110-RFLP on a set of clinical isolates collected over 5 years  (2004) (2005) (2006) (2007) (2008). To analyze this diversity we set up an automatic Cluster analysis tool available on a new website dedicated to molecular epidemiology and classification for human tuberculosis, called TBminer. Combined with a few additional genotypes from Lineages 5 and 6 (M. africanum lineages), the resulting database allowed us to identify correspondences between the existing taxonomies based on spoligotype or MIR-U-VNTR data, leading to the setting up of a consensual "expert" taxonomy. We then set up a tool based on machine learning for identifying the alternative assignations of new isolates according to these different taxonomies.
The redefinitions of groups were confirmed by the use of SNPs  [62, 63]. SNP subclassification tends thus to become the new standard. As its cost when using complete SNP panels is still high, a combination of a global panel and a restricted panel using geographically-specific SNPs could become future practice for correct and precise taxonomical assignation of any local TB clinical isolate. Confronting VNTR markers with assignation based on spoligotype patterns can still provide helpful information for classifying new isolates. The large database set up in this study allowed us to build a new classification tool efficient when used on both 15-MIRU-VNTR and spoligotype patterns. This on-line induction algorithm proved fairly robust. We believe that until Whole Genome Sequencing decreases to below 40 euros per isolate, the continuous use of 15-MIRU-VNTR typing in combination with spoligotyping is relevant to both infer main epidemiological events and get a clear picture of the circulating diversity.
As our tool greatly speeds up the bioinformatical analysis of produced genotyping patterns, it may be of great use to epidemiologists. A limitation to our tool is that it was built exclusively on widespread human M. tuberculosis lineages, excluding animal sublineages such as M. caprae, M. microtii, M. pinnipedii, as well as M. canettii and the recently described Lineage7 isolates  [16]. M. caprae and M. microtii genotypes from MIRU-VNTRPlus database were confidently classified as Lineage 0 or Lineage 6 by our tool. Most M. canettii genotypes (21-VNTR data) as available in Blouin et al. [18]  found no concordant assignation by Pred2, Pred2bis, Pred5 and Pred5bis in our Lineage prediction tool (n = 60; 75%), 15% were erroneously assigned to L4 Lineage, 4% to EAI_Lineage 1 and 1% to Animal lineage_Lineage 0 (data not shown).