dataesr/scientific-paragraphs-categorization huggingface dataset:
The design of prediction techniques for MTBC lineages is a critical task in TB molecular epidemiology because each lineage has its specificity and peculiarity regarding public health aspects such as virulence or antibiotic drug resistance. Therefore, knowledge concerning the lineages involved in a given outbreak may bring potential clues in TB diagnostics (by providing a better understanding of the disease) and in the development of improved treatments. Easily interpretable rules or methods allowing to differentiate lineages are constantly needed for a better understanding of TB epidemiological diversity in the given regions of the world. Data-driven methods could facilitate the accurate and rapid prediction of lineages based on an updated nomenclature. The automation of lineages prediction is also helpful for the curation/improvement of existing or novel biological databases.
In this paper, we focus on the classification of MTBC genotypic lineages. In particular, we propose two approaches that aim to extract interpretable models: (i) the decision tree (DT) with data transformation and (ii) the search for binary mask combinations with an evolutionary algorithm (EA). Our objective was to evaluate the performances of our two approaches in the prediction task of MTBC lineages. For this purpose, the performances of our methodology were compared to the three main classification approaches that stand as references in the field. The experiments conducted have thus allowed highlighting the good performances of our approaches, which are of great significance for the current database development and improvement.
Related works Knowledge Discovery from Databases (KDD) is the 'nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data'  (11). Over the last 30 years, advanced prediction methods and tools borrowed to the KDD field have contributed to defining new kinds of prediction models, namely data-driven models. They consist of searching for correlations between predictive historical variables and output variables. When the output variables are discrete, the problems considered are referred to as 'classification problems'. Among the current techniques used to tackle these problems, artificial neural networks (ANNs), DTs, instance-based learning or correlation analysis are the most common. They have proved to perform quite accurately in comparison to knowledge-driven solutions with the advantage of requiring less knowledge for their implementation.
In the following section, we present the main parts of the EA that we have designed to address our classification problem. Several software tools allow to predict MTBC genotyping families from spoligotyping and/or MIRU-VNTR patterns; examples include the following online resources: MIRU-VNTRplus  (17) , StackTB  (18) , TB-Lineage and other TB-Insight's tools  (19)  and TBminer  (20) ; however, Downloaded from https://academic. oup. com/database/article/doi/10. 1093/database/baaa108/6035122 by guest on 02 January 2021 most of the existing tools are only available as online tools limiting the amount of data that can be analysed. Some available software tools are using more or less complex algorithmic approaches that are not easily understandable for neophytes (or persons with few or no computing skills).
Researchers can also use the SITVIT2 database and its associated tool SpolSimilaritySearch  (24)  to query the database or search for similar spoligotype patterns from the database using regular expressions. The binary rules described in SITVIT2 (defined here as expert rules) were used with regular expressions taking into account 43 spoligotyping spacers, where the character 'n' (or number 1) represents a mandatory presence of spacer, the character 'o' (or number 0) represents an absence of spacer and the character '. ' represents either an absence or a presence of spacer. When a given spoligotyping pattern is classified ambiguously and corresponds to different families, the newly defined priority rules consist in first selecting the families that would be the most specific (basing on our past experience in spoligotyping patterns analysis).
Methodology Datasets were extracted from the SITVIT2 database. A subset of 7668 (4594 for the training set + 3074 for the testing set) unique profiles were randomly generated from strains belonging to 14 representative MTBC spoligotype families as defined in SITVIT2  (6). Predictions from NEW-1 (previously misnamed as Ural-2 in SITVIT2 database) were intentionally included in the Ural family for the analyses. Predictions from Borile AP and TB-Lineage were collected using TBminer (which provides a greater capacity of input file size). The current solution for classifying MTBC spoligotype families in SITVIT databases is based on spoligotyping binary rules. Spoligotyping-based binary rules exist since previous versions of SITVIT databases. These binary rules have been updated in the more recent versions of the database in the function of current and ongoing findings concerning the MTBC families.
These binary rules were generally used to classify TB strains in the function of their spoligotyping patterns. Users can either (i) use these rules directly for manual annotation/curation of their data (using other metadata when necessary); (ii) query the SITVIT2 database to check if their data were already detected elsewhere or (iii) use other online tools such as StackTB, TBminer, TB-Lineage or MIRU-VNTRplus to predict lineages from their data. In the improved approach using expert binary rules, RuleTB  (18)  refined rules were used to predict lineages from 24-loci MIRU-VNTR data. Data on 24-loci MIRU-VNTR from our datasets were not sufficient to conduct a comparative study as was done for spoligotypes. This is why RuleTB was used as an intermediate method (more pending data are available on 24-loci MIRU-VNTR).
Objective function The role of the objective function, often called fitness function in the EA vocabulary, is to assign a quality value to an individual to implement the 'survival of the fittest' aspect of the evolutionary process. The choice of this function is extremely important as individuals with good values will be favoured during the process. In a classification problem, the most straightforward choice for this function is the precision of the classifier on the learning dataset, i. e. the percentage of its right answers, which gives an overall performance measure. However, as we use an OVA strategy, the relabelled datasets on which each binary classifier works are necessarily quite imbalanced, with the 'family' class being much less represented as the 'Other' class. In this case, the search for individuals with good precision values would lead to classifiers with better proper negative rates.
It is the reason why we use the 'Matthews Correlation Coefficient (MCC)' (35) defined as follows: MCC = TP × TN -FP × FN (TP + FP) × (TP + FN) × (TN + FP) × (TN + FN) where TP is the quantity of true positive, TN is the quantity of true negative, FP is the quantity of false positive and FN is the quantity of false negative. It offers the advantage to be particularly well suited for imbalanced datasets.
Comparison approach Our objective in this paper is to evaluate the performances of our two approaches in the prediction of MTBC spoligotype families: (i) the DT with the reduction of spacers and (ii) the binary masks with the EA. To evaluate the efficiency of these approaches, we compare the performances with the three main classification approaches that are references in the field: 'Expert rules', 'TB-Lineage' and 'Borile AP', described in 'Related works' section. Thus, the comparison process we adopt performs in five steps as described in Figure  3. We start from a spoligotype file that contains multiple samples of spacers for each MTBC spoligotype family. We then separate the dataset into two parts: 60% of the samples are used for generating a training set while the remaining 40% are used for testing.
The training set is used for modelling with DT and EA as described previously. Finally, the models learned are applied to the testing set to compare the results with 'Expert rules', 'TB-Lineage' and 'Borile AP'. The quality of the different models is compared by using several performance indicators such as the true positive (TP) rate, the false positive (FP) rate, the precision and the average error rate. We compare the performances on each of the 14 MTBC spoligotype families addressed and we also focus on the average of the results to study and compare the overall performances of the models. To go further, we compare the execution time to evaluate the ability of models to analyse large datasets. Finally, we show some examples of the rules used by our models to perform classification.
Since our approach is a binary classification method by nature, we have followed a 'OVA' strategy to integrate them into a single multiclass classifier. Each of these classifiers takes as input specific relabelled datasets where only one family is kept and all the other families are changed to 'Other'. When a new example has to be classified, it is given successively as input to each binary classifier. The final answer corresponds to that of the classifier that detects a family. A priority table obtained through an optimization technique based on the answers of each classifier on the learning dataset is used to make a single prediction when two or more binary classifiers detect different families.
In particular, even when the spacers are reduced to eight attributes the model retains very good performance, with a precision of about 98%. Finally, if the EA is not in the top three, we can still note the good performance of the binary masks approach with a precision of about 97%. To further understand these results, we have also compared the performance of models in the classification of each MTBC spoligotype family. Table  5  details the 'precision' of models for 14 families in the dataset. Comprehensively, good results of the models can also be seen in the classification of different families. Indeed, except the PINI family for which the results are rather bad, the methods compared are largely concordant since they present precision rates varying from 70% to 90% for the other MTBC spoligotype families.
Runtime evolution In the third part of our study, we focused on calculation times. Figure  5  describes the evolution of runtimes in seconds for different sizes of test datasets. For these tests, we assume that the model has already been learned and we only take into account the time needed to apply the model on test dataset. Runtime tests have been conducted on the following environment: Intel Core i7, 2. 4 GHz, 32Go Ram, Linux Ubuntu 19. 10. Because of the input file size limit of TB-Lineage and TBminer (Borile AP) online tools, the comparison of calculation times has not been performed for these two approaches, since a lot of aspects (such as internet connection quality, performance of their web server, etc. ) could have biased the results. As a result, we compare runtimes for Expert rules and the two approaches we propose-DT (with 8 and 43 attributes) and EA.
The computation time of all compared approaches increases linearly with the size of the dataset. However, when the dataset is large, the time required by Expert rules is significantly higher compared to other approaches. For instance, with 5 00 000 lines, Expert rules performs the classification in >120 seconds while <1 second is required for EA. We can also observe the impact of reducing the number of attributes on DTs. These results on the computation time allow putting in perspective the performances observed in Table  4. Indeed, if the binary masks approach does not offer the best performances, it allows us to perform the classification very quickly. Consequently, binary masks based approach is undoubtedly the best compromise between speed of runtime and performance, since it still has an overall precision of 97%.
Towards interpretable models Finally, in the last part of our study, we focus on the interpretability of the models. Some examples of extracted models for (a) DT and (b) binary masks are shown in Fig. 6. Note that only a subset of the underlying rules of these models is displayed on this graph. As you can observe, one of the main advantages of the two approaches we propose is their ability to provide simple classification rules that can Downloaded from https://academic. oup. com/database/article/doi/10. 1093/database/baaa108/6035122 by guest on 02 January 2021.
SpolLineages tool To facilitate user analysis, we have decided to distribute SpolLineages as a freely available user-friendly commandline tool (https://github. com/dcouvin/SpolLineages) and as a web resource (http://www. pasteur-guadeloupe. fr:80 81/SpolLineages). SpolLineages scripts are mainly written in Java and C programming languages. The online programme (using JavaServer Pages, HTML and JavaScript) allows users with no or few computing skills to easily perform their analysis based on spoligotyping and/or 24-loci MIRU-VNTR data. Users can either upload their CSV/TSV (separated by commas, tabs or semicolons) analysis file containing spoligotyping and/or 24-loci MIRU-VNTR data or directly enter their data in the provided text area. The output result file contains predictions of MTBC spoligotype families using our methods.
Conclusion and future directions To conclude, we propose novel algorithmic approaches allowing quick and precise prediction of MTBC genotypic families from spoligotyping data, using a DT, an EA or classical binary rules. These approaches are helpful for a better understanding and analysis of genotypic variability and evolution of MTBC. SpolLineages software tool (available online or via the command line) includes these methods and allows users to easily search for MTBC genotypic families provided their corresponding spoligotyping data are given. Indeed, this tool provides interpretable rules and a user-friendly interface, which could be helpful for the scientific community. In addition, data reduction methodologies were used to speed up calculations. Furthermore, the approaches presented here are of great importance for database improvement and development.
"OSCILLOSCOPUS; EU project eraNET-NEURON 2009 "BEYONDVIS"; FP6 (EU VIth Frame Program); IHU-A-ICM Translationnel 2011; NIHNINDS-CDI, R21 NS062317 Recovering Localized Brain Damage by tDCS; PHRC Regional "NEGLECT" 2009; SFB 936/A1; TRR 169/A2 attention orienting networks, consolidated by further studies, may help develop and customize brain stimulation approaches for the rehabilitation of visuospatial neglect.
Different technological approaches have significantly contributed to build a framework from which to investigate the functional roles of white matter tracts and model the impact of disconnection phenomena. The most relevant of all has been diffusion neuroimaging, which opened the door to in vivo studies of white matter bundles in the human brain  (Basser, Mattiello, & LeBihan, 1994). Thanks to this technology, tractography by diffusion-tensor imaging (DTI) has now become a widely used method for the "virtual dissection" and characterization of white matter connectivity in healthy participants or neuropsychiatric patients  (Basser, Pajevic, Pierpaoli, Duda, & Aldroubi, 2000; Le Bihan et al. , 2001). Most recently, reliable probabilistic atlases of human white matter tracts developed with DTI, used in conjunction with structural T1 MRI sequences and lesion masks allow the identification and quantification of altered bundles without the need of diffusion datasets  (Foulon et al.
, 2018). Finally, innovative mathematical models integrating alterations of neurophysiological (electroencephalography  [EEG]  and magnetoencephalography  [MEG] ) or hemodynamic (functional MRI  [fMRI] ) datasets from specific gray matter regions, and behavioral/clinical information have proven instrumental to unearth causal evidence supporting interactions among brain areas and their functional links. However, the features of "disconnection syndromes" addressed with all these approaches are considered for many conceptually different from exploring events of the original notion defined by  Geschwind (1965) , hence, they must only be interpreted cautiously as part of the latter framework. In a recent study  (Toba et al. , 2017) , we used a computational approach based on multiperturbation Shapley value analysis (MSA)  (Keinan, Sandbank, Hilgetag, Meilijson, & Ruppin, 2004 ; see also  Kaufman, Serfaty, Deouell, Ruppin, & Soroker, 2009 , for an application of this method in stroke patients with neglect) to identify in stroke datasets gray matter nodes with causal bearing on attentional orienting.
| Patient recruitment demographics and consent form In the present study, we analyzed existing MRI and behavioral datasets of a cohort of 25 right-handed stroke patients (17 men, mean age 55. 96 years, SD 10. 63, range 35-79) previously explored with different goals  (Toba et al. , 2017) , evaluated by the PHRC Regional NEGLECT trial (a multicentric double blind clinical trial evaluating efficacy and safety of a repetitive transcranial magnetic stimulation [rTMS] regime for 2 weeks to improve visuospatial neglect in chronic stroke patients, with a 6 months follow-up), and databased by the Centre for Cognitive Anatomy (CAC) at the Paris Brain Institute, Pitié-Salpêtrière Hospital, Paris, France. Patients presented for the first time a stroke in the right hemisphere and were neuropsychologically evaluated at least 2 months following the stroke event after verifying stability of neglect symptoms in two consecutive measures.
Consequently, we opted for adding to our analyses the ATP, the OR, the APS, the CA and the CP. In sum, the rationale behind the a priori selection of a coalition of players practiced in the present study was based on empirical data  Using the MRIcro software  (Rorden & Brett, 2000)  and a graphic tablet (A6 WACOM Intuos), lesion masks were manually delineated and segmented by expert personnel (trained in clinical neuroimaging and neuroanatomy) on each relevant section of the raw 3D T1 MRI volume for each patient. 3D T1 images and lesion masks were reoriented in order to match the standard MNI 152 atlas. We then used the N4ITK tool implemented in ANTS  (Tustison et al. , 2010)  and completed intensity bias correction, ignoring the segmented lesion. A binary mask of the brain was computed by registering the MNI 152 atlas to the T1 native space.
| Data preparation for MSA The dataset used for MSA computations was derived from line bisection, bells cancellation, and letter cancellation outcomes. For each patient, the graded measure of relative white matter lesion size (percentage of disconnection) for each selected bundle and for the RoB was associated with the binarized performance score representing two binary outcomes of each respective test (0 indicating "normal" performance and 1 signaling "pathological" performance). Specific clinical standardized cut-off levels were used in order to binarize behavioral scores. As mentioned above (Section 2. battery were used in order to binarize behavioral scores: scores larger than two in absolute value in the bells and letter cancellation (for ages above 50 years), and scores higher than +6.
5 mm or smaller than -7. 3 mm in line bisection  (Rousseaux et al. , 2001). Since binary scores represent the severity of neurologic deficit and MSA requires a score representing behavioral ability, we used the inverse of each score as an indicator of functional performance (1current score). The original dataset, composed of ~25 graded stroke lesion configurations (describing relative lesion size) for the 11 areas and the RoB (i. e. , a total of 11 + 1 = 12 white matter bundles or players) and their associated performance scores were far from representing the full set of all possible combinations of binary states, as is typically the case for opportunistic samples (original-graded dataset). Indeed, due to the large number of white matter "players" (11 + 1 = 12) and the small number of patients (~25), we had to apply the "estimated" MSA method which predicts binary performance scores for any perturbation configuration generated by 1,000 sampled permutations.
A machine-learning model for binary classification, and a support vector machine, SVM (Matlab, Mathworks), were trained on the available input original-graded dataset (training dataset) of ~25 stroke clinical cases, each characterized by a unique graded lesion pattern, spanning from 0 to 1, ("0" indicating complete damage of a white matter tract and "1" indicating its complete preservation or sparing from damage), and the corresponding binary scores for each neuropsychological test (training labels). We first assessed the statistical power of the binary predictor by computing a classification (prediction) accuracy applying a "leave-oneout" cross-validation on the original-graded dataset for every test, The parameter representing the box-constraint factor for the SVM was set to default value (c = 1).
A SVM with linear kernel was used for the line bisection, whereas a SVM with polynomial kernel was implemented for the bells cancellation and a SVM with quadratic kernel for the letter cancellation. This choice was made in order to maximize classification accuracy for each individual neuropsychological test after a sensitivity analysis of the SVM's kernel parameter. The result of this process yielded prediction classification accuracies of 91%, 75%, and 68% for line bisection, bells cancellation, and letter cancellation, respectively. These levels proved considerably higher than their respective statistical chance levels (52%, 55%, and 50%) computed as the classification accuracies (i. e. , leaving out in turn each single case), but using randomly permutated instead of predicted scores. More specifically, we left out on each turn, each single case from the original dataset and compared the actual value to the corresponding randomly permutated score obtained by shuffling, instead of using the predicted score.
The bootstrapping was used to ensure the robustness of the obtained contributions and to estimate the associated SE. Essentially, from the available dataset, we chose 1,000 random samples with replacement, with the size of the original dataset. We then performed the estimated MSA on each of these 1,000 samples. Finally, the contributions and SEs were averaged across the pool of 1,000 samples. Note that the F I G U R E 2 Legend on next column. F I G U R E 2 Patterns of relative lesion sizes of white matter regions and associated binary neuropsychological scores across patients. Panels represent the relative lesion size (in % of white matter damaged voxels with respect to the total number of voxels associated to each tract) for each of the 11 white matter bundles considered in the analyses, and an additional white matter ROI representing the rest of the brain (RoB), that is, a compound of any white matter area not included in the 11 bundles of our study.
For all the tests, pathological score values were widely spread across patients, who showed or not lesions of different levels of severity encompassing the white matter bundles considered in our analyses. We then computed pairwise Pearson correlation coefficients using Matlab (Mathworks, MA; Figure  3 ) for the relative regional lesion patterns (i. e. , correlations between all pairs of relative lesion sizes, specifically sorted for each task to present the highest values close to the diagonal) for the selected coalitions of white matter bundles across patients (n = 23, n = 24, and n = 19), which were evaluated separately for each of the three neuropsychological tests (line bisection, bells cancellation, and letter cancellation). These analyses assessed the level of covariance in lesion patterns across white matter bundles, caused by their dependence on a common source of blood supply (i.