dataesr/scientific-paragraphs-categorization huggingface dataset:
Results: We present the comparative study of two multilocus GWAS strategies, in the random forest-based framework. The first method, T-Trees, was designed by Botta and collaborators (Botta et al. , PLoS ONE 9(4):e93379,  2014). We designed the other method, which is an innovative hybrid method combining T-Trees with the modeling of linkage disequilibrium. Linkage disequilibrium is modeled through a collection of tree-shaped Bayesian networks with latent variables, following our former works (Mourad et al. , BMC Bioinformatics 12(1):16, 2011). We compared the two methods, both on simulated and real data. For dominant and additive genetic models, in either of the conditions simulated, the hybrid approach always slightly performs better than T-Trees. We assessed predictive powers through the standard ROC technique on 14 real datasets.
For 10 of the 14 datasets analyzed, the already high predicted power observed for T-Trees (0. 910-0. 946) can still be increased by up to 0. 030. We also assessed whether the distributions of SNPs' scores obtained from T-Trees and the hybrid approach differed. Finally, we thoroughly analyzed the intersections of top 100 SNPs output by any two or the three methods amongst T-Trees, the hybrid approach, and the single-SNP method. Conclusions: The sophistication of T-Trees through finer linkage disequilibrium modeling is shown beneficial. The distributions of SNPs' scores generated by T-Trees and the hybrid approach are shown statistically different, which suggests complementary of the methods. In particular, for 12 of the 14 real datasets, the distribution tail of highest SNPs' scores shows larger values for the hybrid approach.
The second randomization source both aims at decreasing complexity for large datasets, and diminishing the variance. Two of the three methods compared in the present study, T-Trees and the hybrid FLTM/T-Trees approach, are variants of random forests. For further reference, Algorithm 3 outlines the simple generic sketch that governs the growing of an ensemble of tree-based models, in the random forest context. It has to be noted that a novel set of K variables is sampled at each node, to compute the cut-point at this node. It follows that the instantiations of generic Algorithm 1 (growTree) into growDecisionTree (Algorithm 2), and growRFTree adapted to the random forest framework (Appendix, Algorithm 7), only differ in the cut-point identifications. Table  1 (A) and 1(B) show the difference between growDecisionTree and growRFTree.
Simulated data To simulate realistic genotypic data and an association between one of these SNPs and the disease status, we relied on one of the most widely-used software programs, HAPGEN (http://mathgen. stats. ox. ac. uk/ genetics_software/hapgen/hapgen2. html) [41]. To control the effect size of the causal SNPs, three ingredients were combined: severity of the disease expressed as genotype relative risks (GRRs) for various genetic models (GMs), minor allele frequency (MAF) of the causal SNP. The genetic model was specified among additive, dominant or recessive. Three genotype relative risks were considered (1. For each replicate, we simulated 10 causal SNPs. Standard quality control for genotypic data was carried out: we removed SNPs with MAF less than 0. 05 and SNPs deviant from Hardy-Weinberg Equilibrium with a p-value below 0.
Implementation The T-Trees (sequential) software written in C++ was provided by Botta. A single run is highly time-consuming for GWASs in the orders of magnitude we have to deal with. For example, on a processor INTEL Xeon 3. 3 GHz, running T-Trees on chromosome 1 for Crohn's disease requires around 3 days. In these conditions, a 10-fold cross-validation (to be further described) would roughly require a month. On the other hand, around 5 GB are necessary to run T-Trees with the parameter values recommended by Botta  [36] , which restrains the number of executions in parallel. The only lever of action left was to speed up T-Trees software through parallelization. We parallelized Botta's code using the OpenMP application programming interface for parallel programming (http:// www. openmp. org/).
Nonetheless, the empirical complexity is known to be lower. DBSCAN requires two parameters: R, the maximum radius of the neighborhood to be considered to grow a cluster, and N min , the minimum number of neighbors required within a cluster. Details about the DBSCAN algorithm are available in  [47]  (page 526, http://www-users. cs. umn. edu/~kumar/dmbook/ch8. pdf). Finally, we wrote scripts (in Python) to automatize the comparison of the results provided by the three GWAS strategies: single-SNP GWAS, T-Trees, and hybrid FLTM/ T-Trees approach.
Validation of the software parallelization Parallelization is known to potentially entail biases in results. We therefore took great care to check whether this was the case for the parallelization of T-Trees software. We recall that T-Trees is stochastic, with four sources of randomization. Thus, we controled randomization through fixing random grains and we compared the sequential and parallelized versions of T-Trees on five datasets. The comparison shows that importance measures are practically indentical, with some rare minor discrepancies (results not shown). In any case, SNP ranking is conserved from sequential to parallelized version.
Parameter setting T-Trees and the hybrid FLTM / T-Trees approach share 7 parameters (T, S n , S t , K, s n , s t , k). We tuned them after the experiments and remarks reported in  [36]. In T-Trees, the size of the blocks of SNPs was set to 20. The FLTM learning algorithm requires 6 parameters (α, β, card max , τ , nb{-}EM{-}restarts, δ). All 13 parameters are depicted in Table  3. The values of α, β, τ and nb{-}EM{-}restarts were set after the indications in  [32]. The value of card max was set according to our own experimental feedback. The value for δ was chosen to control the running time in the case of high dimensional data. In addition, the clustering procedure DBSCAN plugged into FLTM learning requires 2 parameters (R, N min ). To avoid questionable empirical setting of the R parameter, for each of the 14 datasets analyzed, we ran the FLTM learning algorithm for a wide range of possible values of R.
We compared the AUCs obtained for T-Trees and the hybrid FLTM / T-Trees approach on 14 datasets, following a 10-fold cross validation scheme. In this scheme, the initial dataset D is split into 10 smaller datasets of equal size {D i , 1 ≤ i ≤ 10}. The model is trained on nine tenths of the data (training set) and tested on the remaining tenth (testing set). This process is iterated ten times. The i th iteration involves testing set D i. It is important to note that in the hybrid FLTM / T-Trees approach, the LD map provided by FLTM is the same for all 10 iterations.
for abbreviations and details As the trends for the additive and dominant genetic are comparable for each of the 9 conditions simulated (GRR, MAF), we only focus here on the additive and recessive models Additional file 1 in Supplementary data provides the results of this thorough analysis. Table  1  in Additional file 1 displays the minimum, maximum, average and standard deviation values obtained for both T-Trees and hybrid approaches. The extract shown in Table  6  is representative of the trend observed over the 14 datasets. For T-Trees, importance measures vary from non-detectable The bold face characters highlight the highest AUC observed between T-Trees and the hybrid approach to orders of magnitude of 10 -2 for 7 datasets and 10 -1 for the 7 other datasets. As regards the hybrid approach, importance measures vary from non-detectable to orders of magnitude 10 -2 and 10 -1 respectively for 6 and 8 datasets.
Except for one dataset, (CD, Chr20), the maximal importance values are in the same order of magnitude for both methods. Besides, for T-Trees and the hybrid approach, averages across SNPs are always in the same order of magnitude (10 -5 , exceptionally 10 -4 for 3 datasets ((BD, Chr21), (RA, Chr19) and (T2D, Chr21)). The orders of magnitude for the standard deviations are in the range [10 -4 -10 -3 ]. Again, the trend observed is similarity in the orders of magnitude for T-Trees and the hybrid approach, except for 3 datasets ((BD, Chr03), (BD, Chr21) and (CD, Chr20)). However, complementary statistics shed more light on the distributions compared. From Table  2  in Additional file 1 (Supplementary data), we observe that the correlation coefficient varies from 0. 13 to 0. 98. Moreover, the Wilcoxon test rejects the null hypothesis of similarity for the two distributions, in all 14 cases.
An extract of this table is shown in Table  7. Since, according to the Wilcoxon test, the mean ranks between the distributions differ, the conclusion to draw is that, in particular, the top SNPs are likely to differ between the two methods compared. Tables 3 to 16 in Additional file 1 (Supplementary data) help decipher the origin of the discrepancies between the T-Trees and hybrid approaches' distributions. For each dataset (pathology, chromosome), the corresponding table compares the 25%, 50%, 75% and 100% quantiles represents the unique case of discrepencies between the orders of magnitude of the maxima (in favor of the hybrid approach). Dataset (CD, Chr20) also one of the three cases showing a discrepancy in the standard deviations' orders of magnitude obtained for T-Trees and the hybrid approach.
Importantly, a focus is also set on the 6 quantiles that correspond to the top 300, top 200, top 100, top 50, top 20 and top 10 SNPs output by each method. A first observation is that up to the 50% quantile, for both methods, the importance measures vary in the range of from nondetectable (i. e. below 10 -8 ) to the order of magnitude 10 -5 at most (exceptionally 10 -4 at most for T-Trees and datasets (BD, Chr21) and (RA, Chr19)). We also observe a constant trend: up to 75% quantile included, the hybrid's method quantiles are always lower than T-Tree's quantiles. In contrast, for 12 datasets out of 14, the hybrid's method quantiles are higher than T-Trees' quantiles as from some threshold quantile. To fix ideas, we show in Table  8  the quantiles relative to dataset (RA, Chr19). In this case, more "important" SNPs are likely to be found in the hybrid's top 300 SNPs than in the T-Trees' top 300s (quantile hybrid = 1.
5 × 10 -2 versus quantile T-Trees = 6. 0 × 10 -3 ). As biologists expect a short list of prioritized SNPs to be further analyzed, it is important to study whether one of the two methods yields higher importance values than the other for the top SNPs. Table  9  subsumes the tendencies observed from Tables  3 to 16  in Additional file 1 (Supplementary data). Hintertho we knew that the two distributions of the variable importances differed between T-Trees and the hybrid approach. The conclusion to draw now is that, except for 2 datasets out of 14, the hybrid method outputs top ranked SNPs with relatively higher importances than in T-Trees, which is our focus in GWASs. Therefore, the search space seems to be more efficiently explored. Potentially, the hybrid approach allows us to group the SNPs in clusters in a manner more efficient to target the putative associated SNPs.
It is likely that the multivariate split produced from a cluster of SNPs tagging the putative influential SNP, or best capturing association with the phenotype, will be the optimal split (Algorithm 5, line 11), for any such cluster draw (at any meta-node of the T meta-trees). In T-Trees, a block is also likely to be drawn at any meta-node of the T meta-trees. However, if the SNPs that best tag the influential SNP, or best capture association with the phenotype, are spread over two or more blocks, none of these blocks is likely to produce an optimal multivariate split. Prediction performance measured through the AUC of a ROC curve relies on data describing both true and predicted statuses for the individuals. We now examine whether, for a given dataset, a method with a higher AUC than the other method (Table  5 ) would also be the approach showing inflated top SNPs' importance quantiles (Table  9 ).
Indeed, in 10 cases out of 14, we observe consistency between Tables  5  and 9. Inconsistencies are pinpointed for datasets (CAD, Chr05), (CAD, Chr06), (RA, Chr06) and (T2D, Chr21). First, for 6 out of 10 cases for which the AUC is the highest for the hybrid approach, we observe that the highest importances distribution tail for the hybrid approach is over that of T-Trees as from top200 or top300 quantile. In contrast, The bold face characters highlight the highest value observed between T-Trees and the hybrid approach when the hybrid approach distribution is T-Trees' as only from top10 quantile, we observe a higher AUC for T-Trees (inconsistencies for (CAD, Chr05) and (RA, Chr06)). In the intermediary top quantiles (top100), the situation is mitigated, with dataset (CAD, Chr06) showing the lowest AUC for the hybrid approach (inconsistency) whereas it is the contrary for dataset (T1D, Chr13).
To attempt to explain the fourth inconsistency, (T2D, Chr21), we compare this case with that of (T1D, Chr02). In the second case, T-Trees' importance values are inflated as from top25% quantile (except for top300 quantile) and T-Trees' AUC is the highest. The inconsistency of dataset (T2D, Chr21) might be explained by the fact that even though T-Trees's importance values are also inflated as from top25% quantile, top300 and top200 quantiles are missing. However, the explanation is not so simple as it would well explain the inconsistency for (CAD, Chr06) (both inflated distribution as from top100 quantile and lowest AUC for the hybrid approach) but not the consistency of (T1D, Chr13) (inflated distribution as from top100 quantile for the hybrid approach and highest AUC for the hybrid approach).
With this latter unique restriction, for the datasets analyzed, we are inclined to draw two conclusions: the method with the highest importance distribution tail would also show the highest predictive power; the importances of the variables in the rank intervals top300 and top200 might be crucial for the method's predictive power. Table  9  Quantile from which the quantiles for one method are always greater than those of the other method, regarding the distributions of variable importances, across the SNPs, for T-Trees and the hybrid FLTM / T-Trees approach. For example, for (T1D, Chr13) dataset, the hybrid quantiles are always greater than the T-Trees quantiles as from top100 quantile. Moreover, the hybrid quantiles are an order of magnitude higher as from top20 quantile An additional conclusion is that both methods can be used to pinpoint various top SNPs, in a complementary way.
Analysis of the sets jointly identified by any two among the three, or the three methods compared on real data We reported the SNPs jointly identified in the top 100s by any two or all three methods studied : single-SNP GWAS versus T-Trees, single-SNP GWAS versus hybrid approach, T-Trees versus hybrid approach, and the three methods. Additional file 2 (Supplementary data) describes these results for each of the 14 pairs (pathology, chromosome) analyzed. Figure  8  plots the cardinalities of the 3 pairwise intersections and of the three-way intersection for the 14 datasets analyzed. The Venn diagrams in Additional file 3 (Supplementary data) provide an intuitive insight of the trends observed across the 14 datasets analyzed. For an illustration, Fig. 9  focuses on the Venn diagrams of (BD, Chr03), (CD, Chr01) and (T2D, Chr10).
(T2D, Chr10) is the dataset observed with the highest number of top 100 SNPs common to the three methods  (23) , over the 14 datasets. Together with datasets (CD, Chr20) and (T2D, Chr10), dataset (BD, Chr03) shows the largest number of top 100 SNPs common to single-SNP GWAS and T-Trees  (31). The largest number of top 100 SNPs common to Single-SNP GWAS and the hybrid approach (30) is observed for (CD, Chr01). Finally, (T2D, Chr10) is again the dataset for which the number of top 100 SNPs common to T-Trees and the hybrid approach is the highest (60). The reader interested in a more thorough analysis of the number of top 100 SNPs common to any or the three methods is reported to the end of Additional file 3. It was expected that the number of top 100 SNPs common to each T-Tree-based strategy and the Single-SNP GWAS would be smaller than the number of SNPs shared with the other T-Tree-based strategy.
The number of such SNPs varies in the range  [8, 23]  over the 14 datasets analyzed. Table  1  in Additional file 4 (Supplementary data) describes the characteristics of the SNPs jointly identified in the top 100s by each of the three methods. Additional file 4 then discusses these results in details. In the sequel, A summary of Additional file 4 is provided in Table  10. We observe that each time a top*1 SNP is detected (that is for datasets (BD,Chr03), (CAD,Chr05), (CD,Chr01), (RA,Chr06), (RA,Chr19) and (T2D,Chr21)), it is significantly associated with the disease, according to the Single-SNP GWAS strategy. Besides, 62 other top*m SNPs are also characterized with a significant p-value. In addition, Table  10  allows to prioritize top*100 SNPs for further biological investigation: among the 211 top*100 SNPs detected over the 14 datasets, we identified 72 top*25 SNPs including 38 top*10 SNPs.
Thus, an additional insight of our study is the interest to select top*m SNPs to prioritize a list of SNPs. Finally, Table  11  displays the number of top 100 SNPs common to T-Trees and the hybrid approach. For 8 datasets over 14, the correlation coefficient of the corresponding variable importances is greater than 0. 93. Regarding the 6 other datasets, the correlation coefficient varies between 0. 58 and 0. 87.
e. a window-based version), FLTM was also tested on still larger datasets as in  [40]  (e. g. describing 100,000 variables  [32] ). For information, the WTCCC dataset (Crohn's disease, chromosome 1) describes 37,267 SNPs. It is important to emphasize that the FLTM algorithm used in the present paper does not coerce the latent trees' structure to binary structure, does not impose a user-defined shared cardinality for all latent variables, and does not require contiguity (on the genome) for the variables to be clustered. A flexible and thus faithful modeling of linkage disequilibrium comes at a cost in the present (yet optimized) version of FLTM: high memory consumption and high running time not allowing scalability to genome scale.
Conclusions and perspectives In this paper, we put forth an hybrid approach combining two machine learning models, T-Trees and FLTM, to enhance genome-wide association studies. We compared the performances of the integrated approach and of T-Trees based on simulated realistic genetical data. The integrated approach was shown to perform slightly better than T-Trees for the additive and dominant genetic models. There is no subtantial advantage shown for the recessive model, except in few cases. We then performed a comparative analysis of the predictive powers and  derived from FLTM is shown beneficial: on the datasets analyzed, the already high predicted power observed for T-Trees is increased in the majority. The distributions of SNPs' scores generated by T-Trees and the hybrid approach are shown statistically different.
In particular, in a vast majority of cases, the hybrid method outputs top ranked SNPs with relatively higher importances than in T-Trees. Thus are pinpointed more interesting SNPs than in T-Trees, to be provided as a short list of prioritized SNPs, for a further analysis by biologists. Not only did we show that both methods can be used to pinpoint various top SNPs, in a complementary way. Complementarity can also be used to reinforce confidence in top SNPs. Therefore, we analyzed the pairwise and three-way intersections of SNPs ranked in the top 100s, for the standard single-SNP GWAS, T-Trees and the hybrid method. In particular, among the 211 top 100 SNPs jointly detected by the three methods, over the 14 datasets analyzed, we identified 72 and 38 SNPs respectively present in the top25s and top10s for each method.
In future work, we will extend the comparative study of T-Trees and the hybrid approach to more GWAS datasets. In particular, we will examine whether a method with a higher predictive power than the other method also tends to show inflated top SNPs' importance quantiles. This paper was fully dedicated to the thorough comparison of T-Trees and the FLTM / T-Trees approach. In the future, we plan the design of a vast study, to compare the hybrid approach to a panel of other approaches derived from machine learning such as logistic and penalized regressions, gradient boosting machines, ensemble methods, artificial neural networks, support vector machines and Bayesian network-based analysis. Besides, since the FLTM model is the key to the improvement brought by the hybrid approach over T-Trees, several directions focused on FLTM learning need be explored for future work.
2, 1. 5 or 1. 8). The range of the MAF at the causal SNP was specified within one of the three intervals [0. 05-0. 15], [0. 15-0. 25] or [0. 25-0. 35]. The disease prevalence (percentage of cases observed in a population) specified to HAPGEN was set to 0. 01. These choices are justified as standards used for simulations in association genetics. HAPGEN was run on a reference haplotype set of the HapMap phase II coming from U. S. residents of northern and western European ancestry (CEU). Datasets of 20000 SNPs were generated for 2000 cases and 2000 controls. Each condition (GM, GRR, MAF) was replicated 30 times.
LDMap ← runFLTM(V , D) /* set of disjoint clusters, partitio-*/ 2: /* ning V, and modeling linkage disequilibrium (LD) */ 3: F ← ∅ 4: for i in 1 to T 5: D i ← bootstrapping(D) 6:T i ← growMetaTree(V , c, D i , S n , S t , LDMap, K, s n , s t , k) 7: F ← F ∪ T i 8: endfor1 diabetes (T1D) and Type 2 diabetes (T2D). The data from the two cohort controls provided by the WTCCC was also included. For each pathology, we carried out a comparative study on two datasets corresponding to two chromosomes. The NHGRI-EBI Catalog of published genome-wide association studies (https://www. ebi. ac. uk/gwas/) allowed us to select these two chromosomes: for each pathology, we retained the chromosomes respectively showing Algorithm 5 Meta-tree learning in the hybrid FLTM / T-Trees framework -Detailed scheme. Notation: given a cluster c of variables in V, M[ [c ] ] denotes the matrix constructed by concatenating the columns M[v], with v ∈ c (see line 8).
SNPs' scores distributions, for T-Trees and the hybrid FLTM / T-Trees approach, on real datasets. The sophistication of T-Trees through finer linkage disequilibrium modeling Table 10 Summary of the top 100 SNPs common to single-SNP GWAS, T-Trees and hybrid FLTM / T-Trees approaches. A SNP jointly identified in the top n SNPs by each of the three GWAS strategies is called a top*n SNP.
1 : 1 LDMap ← runFLTM(V , D) /* set of disjoint clusters, partitio-*/ 2: /* ning V, and modeling linkage disequilibrium (LD) */ 3: F ← ∅ 4: for i in 1 to T 5: D i ← bootstrapping(D) 6:T i ← growMetaTree(V , c, D i , S n , S t , LDMap, K, s n , s t , k) 7: F ← F ∪ T i 8: endfor1 diabetes (T1D) and Type 2 diabetes (T2D). The data from the two cohort controls provided by the WTCCC was also included. For each pathology, we carried out a comparative study on two datasets corresponding to two chromosomes. The NHGRI-EBI Catalog of published genome-wide association studies (https://www. ebi. ac. uk/gwas/) allowed us to select these two chromosomes: for each pathology, we retained the chromosomes respectively showing Algorithm 5 Meta-tree learning in the hybrid FLTM / T-Trees framework -Detailed scheme.