dataesr/scientific-paragraphs-categorization huggingface dataset:
In practice, the features that are contributing negatively should be moved to meet the conditions. In this experiment, we use the churn dataset DSX Local Telco Churn demo used by IBM in demo product. 4 We consider 100 random instances from the dataset which do not satisfy the rules (described in the supplementary) and we apply SMACE, SHAP, and LIME, to extract features that contribute negatively. Negative features are then "removed" oneby-one in order of importance until the rule is satisfied and a retention offer is made. To remove a feature, we replace it with locally perturbed samples. The average decision made A mobile phone company wants to evaluate whether it is worth proposing a retention offer to a customer. The decision is 1 for a customer who meets the conditions, and 0 otherwise.
Nonetheless, it has been argued that TBE must be used with care due to sampling issues, especially in datasets with a high number of closely related taxa. In this study, we conduct multiple experiments by varying taxon sampling and comparing FBP and TBE support values on different phylogenetic depths, using empirical datasets. Our results show that the main critique of TBE stands in extreme cases with shallow branches and highly unbalanced sampling among clades, but that TBE is still robust in most cases, while FBP is inescapably negatively impacted by high taxon sampling. We suggest guidelines and good practices in TBE (and FBP) computing and interpretation. [Felsenstein's bootstrap; phylogenetic trees; support robustness; taxon sampling; transfer bootstrap.
" Despite those important limitations,  Soltis and Soltis (2003)  note that "many systematists have adopted Hillis and Bull's '70%' value as an indication of support," an observation still true today, even though the "95%" and the more arbitrary "80%" cut-offs can often be seen in the literature. Aside from conceptual matters, two main criticisms against FBP are recurring in the literature, especially for "large" datasets. By large datasets, we mean in this study matrices where the number of taxa is large, while the number of sites remains moderate, usually corresponding to a single gene or phylogenetic marker. The first limitation is technical: re-estimation (usually by ML) of pseudo-trees is computationally demanding on large datasets and can be impractical on very large datasets. A rapid bootstrap support (RBS;  Stamatakis et al.
Other criticisms discuss the problem of "large" datasets in the sense of "large number of sites" (e. g. ,  Sharma and Kumar 2021) , but this will not be covered here as it is not the subject of this study. The commonly used alternatives to Felsenstein Bootstrap Support (FBP) can be divided into 2 main classes. The first one is the Posterior Probability (PP) of Bayesian phylogenetics  (Rannala and Yang 1996) , calculated as the proportion of all sampled trees in the MCMC chain (post burn-in) in which the branch of interest is found. Due to their parametric nature and implementation, PP is often considered liberal, as opposed to the more conservative FBP. Abundant literature exists on comparing BP and PP (e. g. ,  Douady et al. 2003) , but expanding on that matter goes beyond the frame of this article.
Local supports are another class of alternative supports, responding to a need for computational speed in a context of ever-growing datasets. These supports are obtained by locally rearranging the tree topology around the branch of interest, using Nearest Neighbor Interchange. Some of the most popular ones include the approximate likelihood-ratio test (aLRT;  Anisimova and Gascuel 2006 ) and the non-parametric Shimodaira-Hasegawa-like version (aLRT SH-like;  Guindon et al. 2010). However, these approaches only provide a local view of the support. It is also important to note that none of these two classes explicitly addresses the problem or rogue taxa: Bayesian PP is expected to behave similarly to FBP with a tree that contains rogues; the local supports are little affected by the presence of a few rogues, but they are also unable to detect them and measure their impact on the overall tree due to their local nature.
" TBE also has solid mathematical ground and is guaranteed to converge in probability to 0 when the size of the tree grows and there is no signal in the data  (Dávila Felipe et al. 2019). TBE was proposed in a context of the ever-growing number of sequences, in particular in the epidemiology field. It has become recurrent in the literature to find extremely large datasets (in terms of number of sequences): bacteria/archaea (10,575 tips;  Zhu et al. 2019), mushrooms (5284 tips; Varga et al. 2019), fishes (31,526 tips; Rabosky et al. 2018), diatoms (19,197 tips; Lewitus et al. 2018), angiosperms (36,101 tips; Janssens et al. 2020) , and especially viruses-like HIV (9147 tips,  Lemoine et al. 2018 ) and of course SARS-CoV-2 (>15 million tips;  Turakhia et al. 2022). In datasets of this size, it is expected that some of the taxa will behave like rogues, thus lowering FBP values.
In fact, it is not rare to find FBP scores below 20% (or even at 0%), especially for deep branches, even when those have a strong phylogenetic signal. Yet these deep branches are usually the primary focus of large-scale studies. Lemoine et al. (2018)  showed that TBE was in fact able to support deep branches that have strong phylogenetic signals without being affected by a few rogues. TBE is available on the BOOSTER platform (https://booster. pasteur. fr/) and in software programs like Gotree  (Lemoine and Gascuel 2021) , PhyML  (Guindon et al. 2010) , Seaview  (Gouy et al. 2021) , IQ-TREE 2  (Minh et al. 2020) , and RAxML-NG  (Kozlov et al. 2019). Since its publication 5 years ago, TBE has been cited more than 400 times (Google Scholar, June 2023), but has generated little debate. However, a recent review of the history and development of branch support measures  (Simon 2022 ) contained a stimulating critique communicated by Nick Goldman, that "TBE must be used with care due to sampling issues.
For example, if many closely related taxa are added to the tree TBE values will increase across the entire tree. This is because the measure is based on counting the number of sequences sampled, not taking into account their variation. " This critique raises a series of important questions related to sampling variation (or taxon sampling) and sampling disequilibrium in datasets with large numbers of taxa. In this study, we will explore the impact on both FBP and TBE of taxon sampling, rogue taxa, and sampling disequilibrium in large datasets, using theoretical examples and empirical datasets. We provide a series of guidelines in the "Discussion" section for good practices and interpretation when estimating FBP and TBE support values on large datasets.
Impact of Duplicated Sequences on TBE Scores In his personal communication to  Simon (2022) , Nick Goldman suggests that, for a given phylogenetic signal, TBE supports will be increased if we add many closely related taxa to the tree. A model that closely matches this hypothesis is presented hereafter. Let P|Q be a bipartition of reference tree T, where P and Q have sampling size p and q, respectively, 𝑝 ≤ 𝑞 and 𝑝 + 𝑞 = 𝑛, where n corresponds to the total number of taxa (p is again the depth of bipartition/branch P|Q). To keep the same phylogenetic signal while increasing the sampling, we simply add duplicated taxa (i. e. , strictly identical sequences). The number of taxa will be multiplied by a factor k, where k represents the number of duplicated sequences, 𝑘 = 1 corresponds to the original dataset, and p and q are transformed into kp and kq when duplicates are added.
TBE remains mostly stable no matter the depth of the reference branch (bipartition) when the number of rogue taxa is reasonable (i. e. , 1-10 among 1000 taxa). On the contrary, the FBP support will always be considered low if there is even a single rogue when the topological depth of the reference branch is high (i. e. , a large clade). These behaviors are clearly visible in our analyses of empirical datasets. In light of the two models presented above, there is reason to be concerned for both branch supports: for FBP, with deep branches and large clades in case of rogue taxa; for TBE, with shallow branches and small clades in case of sampling issues. However, real data are much more complex than these two theoretical examples, and thus we will further explore the properties of the two branch supports on a series of large biological datasets that supposedly have more or less sampling disequilibrium and rogue taxa.
Overview Our dataset-picking strategy consists in taking large empirical datasets from the literature, for which we expect some degree of heterogeneity in phylogenetic signal and sampling. This would imply that the genetic locus is expected to return good phylogenetic signal for some clades, but will fail to fully resolve a large portion of the phylogeny due to a lack of (informative) sites. For example, barcode markers like the ∼660 bp segment of cytochrome c oxidase I (COI) gene in most metazoans, or 16S ribosomal ribonucleic acid (16S rRNA; ∼1500 bp) in prokaryotes, are typically adequate phylogenetic markers to get a rough idea of the relationships between taxa, but are not sufficient to build reliable phylogenies, especially with thousands of taxa as we are considering here. Another striking example of a weak phylogenetic signal is that of SARS-CoV-2, for which we have complete genome sequences (∼29,500 bp), but very few sites are informative, which prevents any complete resolution of the tree  (Morel et al.
2021). Moreover, unbalanced sampling is expected, with some "charismatic" clades being over-represented (e. g. , Primates and Cetaceans in Mammals), with many more taxa and sequences available in the databases and published phylogenies, and some others under-represented (e. g. , Rodents; see our numbers below for Mammals). We selected the two empirical datasets of the original TBE study  (Lemoine et al. 2018 ) consisting of a COI Mammals dataset (1449 sequences; ∼266 aa) and an HIV dataset (9147 pol sequences; ∼1036 bp). Additionally, we used the full SARS-CoV-2 dataset from  Zhukova et al. (2021)  comprising 11,316 genomes (∼29,500 bp). Finally, we selected 10 aligned nucleotide barcode datasets from  Delsuc and Ranwez (2020)  comprising between 1000 and 2000 mitochondrial COI gene sequences (∼660 bp).
All of these published datasets were already aligned and had in common to be large (>1000 sequences), with heterogeneous phylogenetic signal and unbalanced taxonomic sampling. Our methodology consisted in studying the evolution of FBP and TBE supports, starting from a reference tree with N taxa and unbalanced sampling, to reach a reduced and re-balanced target tree with n (<< N) taxa. In addition, along this path, we studied progressively more and more balanced intermediate trees. The sampling of these intermediate trees was defined by: 𝑛 𝑓 = 𝑁 -𝑓 * (𝑁 -𝑛), with 0 ≤ 𝑓 ≤ 1. For example, to get the midpoint between N and n, we choose f = 0. 5, while f = 0 and f = 1 correspond to the starting and target trees, respectively. When we wanted to target the sampling of specific clades (e. g. , HIV subtypes), then the representativeness of each clade was multiplied by n to obtain the required number 𝑛 𝑋 of sequences to be retained for clade X.
Let 𝑁 𝑋 be the initial size of clade X, the intermediate clade samplings were then defined by 𝑛 𝑋,𝑓 = 𝑁 𝑋 -𝑓 * (𝑁 𝑋 -𝑛 𝑋 ). The target samplings (𝑛 𝑋 ) in the reduced tree were defined by different criteria based on the specifics of the dataset: representativeness for Mammals, worldwide prevalence for HIV subtypes, sampling density per country for SARS-CoV-2, and uniform sampling per delimited species for the barcode datasets. The objective was to assess the stability and robustness of each branch-support measure throughout the subsampling. We selected a set of relevant clades for which one expects a strong phylogenetic signal. For instance, long-branch clades like Cetacea or Marsupialia in the Mammals dataset are expected to be recovered even with a simple COI marker. The same applies to HIV subtypes and the pol gene.
For such clades, with strong phylogenetic signal, a stable and robust branch support should give a high score, with more or less the same value, whatever the sampling size and balance. To get supports for the target and intermediate trees, the ideal approach would be re-estimating the reference and bootstrap trees at each sampling point. However, this procedure can be quite computationally intensive, especially for very large datasets with multiple replicates. Another much faster approach is to prune the leaves of the starting reference and bootstrap trees, and calculate FBP/TBE supports on the reduced trees. The "pruned" approach is the one we favored to reduce computational costs, but in parallel, we also used the "re-estimated" approach on the target sampling to check if the results matched those obtained with the "pruned" approach.
At each intermediate point, FBP/TBE values are computed from the pruned reference and bootstrap trees. Once target sampling is achieved, we re-estimate reference and bootstrap trees from sampling n and compute again FBP/TBE values for comparison with the "pruned" approach. To implement this workflow, we extensively used Gotree/Goalign (Lemoine and Gascuel 2021), a toolkit that implements more than 120 user-friendly commands dedicated to multiple sequence alignment and phylogenetic tree manipulations (e. g. , tree pruning, collapsing branches, MSA de-duplicating). The list of Gotree/Goalign commands used for FBP/TBE support exploration is provided in the Supplementary Appendix. Prior to focusing on the selected clades, we briefly analyzed the overall average support in the Mammals, HIV, and SARS-CoV-2 datasets, for different samplings and depths (Supplementary Fig.
SF1 ). Our results mainly show that FBP and TBE average supports remain stable (with a few exceptions) throughout the successive subsamplings ("pruned" experiments), even when the number of taxa is drastically reduced. Furthermore, we observe that the difference in average scores between the "pruned" and "re-estimated" experiments is low, thus validating our overall strategy (Fig. 3 ). Average TBE supports are on average higher than those of FBP are, especially for deep branches (as expected, see above). Nevertheless, average FBP and TBE supports are low, even for shallow branches (<0. 6 for FBP and <0. 8 for TBE), as expected with these datasets. However, all these are average supports, and the results for the biologically important clades tell a very different story, as shown in the following.
Mammals Dataset The Mammals dataset consists of 1449 aligned COI protein sequences used in  Lemoine et al. (2018). A few clades, almost without contradiction with the NCBI taxonomy (w ≈ m ≈ 0; for details, see  Lemoine et al. 2018) , had been highlighted to illustrate the differences between the FBP and TBE supports (Cetacea, Elephantidae, Geomyidae, "Insectivora," Marsupialia, Monotremata, Mustelinae, Perognathinae, and Simians). We selected these clades but discarded those where the number of available species was insufficient for our experiments (i. e. , Elephantidae and Monotremata). As expected, these clades have been unevenly sampled across the mammalian diversity, either over-sampled or under-sampled. For example, the Cetacea sampling FIGURE 3. Summary of the overall sampling strategy.
The HIV Dataset Like the Mammals dataset, the HIV dataset was used in  Lemoine et al. (2018)  to highlight the differences between FBP and TBE. It consists of 9147 HIV-1 group M pol sequences representing the 9 subtypes, and includes 50 recombinants detected using jpHMM  (Schultz et al. 2009) , that is, sequences that contain DNA from at least two different subtypes in the pol region. In this experiment, we achieved a representative sampling based on the current prevalence of each subtype in the worldwide population. The prevalence values of each subtype were obtained from  Cassan et al. (2016, Fig. 3  and Supplementary Table  ST2 ). Then, we applied the same approach as used with the Mammals dataset. We divided 2 by the prevalence of the most under-represented subtype (i. e. , subtype K, with 𝑅 𝑋 = 0.
This additional experiment allowed us to follow the evolution of the average FBP/TBE supports at different depths while reducing the sampling, instead of focusing only on HIV-1 group M subtypes that are known to be well supported in most datasets. With the initial sampling,  Lemoine et al. (2018)  showed that all 9 subtypes were found in the reference tree and highly supported by TBE (although the average TBE support across the tree is relatively low; Supplementary Fig. SF1 ). On the opposite, only the sparsely represented subtypes (i. e. , F, H, J, and K) were supported by FBP. A prominent example is the FBP/TBE supports of the B subtype, a clade comprising 3559 sequences and which is almost perfect in the reference tree since it contains all B sequences plus 2 B-recombinants (as detected by jpHMM).
One could thus distinguish two forms of taxon instability (with the whole spectrum of intermediate cases): single rogue terminal taxa prone to instability (e. g. , recombinants) and taxa belonging to globally unstable clades. Our results again illustrate the low robustness of FBP to rogue taxa and show how robust TBE is, whether or not some rogues are present (as predicted by our theoretical model with rogues, see above). One could argue that TBE might be over-supporting those clades, but our experiment on the overall support values across the tree shows another story (Supplementary Fig. SF1 ). The overall FBP/TBE support in the phylogeny is low for all ranges of branch depth (by the standards of each metric, that is, less than 0. 4 for FBP and less than 0. 6 for TBE) and shows little change as trees are increasingly subsampled, except for cherries and deep branches where a trend emerges with this HIV dataset.
Indeed, average FBP/TBE support for cherries (p = 2) slowly decreases as we remove more and more sequences. Conversely, average TBE supports for deep branches (p > 9) tend to increase. These overall trends contradict the results with selected clades/subtypes (Fig. 5 ), but are slight (less than 10% point difference between initial and target samplings) and are not found in the analysis of the SARS-CoV-2 dataset that follows.
The SARS-Cov-2 Dataset We retrieved the SARS-CoV-2 dataset from  Zhukova et al. (2021)  and processed the sequences through the Nextclade Web 2. 3. 0 interface  (Aksamentov et al. 2021) ; https://clades. nextstrain. org) to assign each sequence to a Nextstrain "clade. " Some Nextstrain "clades" are paraphyletic, as they do not include the sequences of new child clades. When a "clade" was paraphyletic (e. g. , clade 20B), we added all child clades in the sampling to make it monophyletic (i. e. , 20D in the case of 20B). The result is a reference classification composed of 5 clades: 19B, 20A (= 20A+20B+20C+20D Nextstrain "clades"), FIGURE 5. Results on the HIV dataset. We removed taxa within each HIV subtype based on prevalence. For each subtype (clade), the first value on the x-axis corresponds to the initial number of taxa, and the last value to the number of taxa in the target sampling.
We also provide intermediate values, corresponding to proportions f = 0. 2, 0. 4, 0. 6, and 0. 8 (see text for details). Trees were re-estimated on the target sampling (reest. ). The number of wrong (𝑤) and missing (𝑚) taxa is indicated next to each subtype identifier. To obtain the subsamples, taxa were randomly removed. The ribbon shows the maximum and minimum values over 30 replicates, the thick line is the mean value. 20C, 20B (= 20B + 20D Nextstrain "clades"), and 20D. Sampling re-equilibrium had already been achieved by  Zhukova et al. (2021) , starting from the complete 11,316sequence tree to 5 subsampled and re-balanced trees of size ∼2000. The target sampling for these 5 datasets was defined to reflect the number of cases reported by country over time during the early months of the pandemic.
For the SARS-CoV-2 dataset, we ran multiple tests to determine what would be the best approach to infer the tree on the 8541 genomes-after removing duplicate genomes from the initial 11,316 genomes dataset of  Zhukova et al. (2021). We looked at the best trade-off for running time and tree accuracy between FastTree, IQ-TREE 2  (Minh et al. 2020) , IQ-TREE 2 in fast mode, and RAxML-NG (results not shown). Tree accuracy was evaluated by counting the number of taxa to transfer from the estimated tree to recover the Nextstrain clades from our reference taxonomy. The fastest approaches were by far FastTree and IQ-TREE 2-fast, but the total number of taxa to transfer for FastTree (309) was much higher than for IQ-TREE (82). The total number of taxa to transfer was not much different from other, more time consuming, methods; hence, IQ-TREE 2-fast was selected as the best approach for the SARS-CoV-2 dataset.
Branches shorter than 0. 000017 (= 0. 5/29,726, where 29,726 is the number of alignment sites) were collapsed (resulting in ∼84% of internal branches collapsed, as expected due the very low number of mutations in this dataset). We successively randomly pruned taxa by batches of ∼1500 in the starting tree to retrieve the sampling of the 5 trees of size ∼2000 from  Zhukova et al. (2021) , using 20 replicates each time, for a total of 20 * 5 = 100 replicates. We extracted the FBP/TBE supports for the 5 selected clades (Fig. 6 ) and the average of these FIGURE 6. Results on the SARS-CoV-2 dataset. We randomly removed taxa from the original SARS-CoV-2 tree (11,316 tips) in order to achieve a better sampling equilibrium on trees with about 2000 taxa. Trees were re-estimated on target sampling (reest.
). The number of wrongly (𝑤) and missing (𝑚) taxa is given next to each clade identifier, and we provide the original and target samplings for each clade (e. g. , "77 to 3" with 20D). The ribbon indicates the maximum and minimum values, and the thick line corresponds to the mean value across all replicates. supports for different branch depth intervals (Supplementary Fig. SF1 ), from the pruned and estimated trees. As with the other datasets, we retrieved the selected clades in the estimated trees by minimizing the number of wrong (w) and missing (m) taxa (see above and the Supplementary Appendix for details). The difference with the previously analyzed dataset (HIV) is that the overall phylogenetic signal is scarce, with clades supported by only one or two mutations (https://nextstrain. org; Supplementary Table  ST3 ).
As for the previous datasets, most Nextstrain clades are well supported by TBE and not by FBP, even when the clade is well retrieved in the reference tree (e. g. , 20D). Interestingly, support values are little affected by the number of wrongly placed and missing sequences (w + m), and the FBP/TBE scores remain stable across most clades upon subsampling (Fig. 6 ). For example, clade 20C is not well supported by TBE (∼0. 62). Indeed, with 2193 sequences in this clade, a score of 0. 62 means that on average (2193 -1) × 0. 38 = 833, sequences need to be transferred to recover that clade in the bootstrap trees; this number of 833 is much greater than the number of wrong/missing sequences in the reference clade (=30, Fig. 6 ). As opposed to HIV, results with SARS-CoV-2 seem to indicate a global instability of a large number of taxa.