dataesr/scientific-paragraphs-categorization huggingface dataset:
However, recent studies have established that summary methods can have reduced accuracy in the presence of gene tree estimation error, and also that many biological datasets have substantial gene tree estimation error, so that summary methods may not be highly accurate in biologically realistic conditions. Mirarab et al. (Science 2014) presented the "statistical binning" technique to improve gene tree estimation in multi-locus analyses, and showed that it improved the accuracy of MP-EST, one of the most popular coalescent-based summary methods. Statistical binning, which uses a simple heuristic to evaluate "combinability" and then uses the larger sets of genes to re-calculate gene trees, has good empirical performance, but using statistical binning within a phylogenomic pipeline does not have the desirable property of being statistically consistent.
Introduction The estimation of phylogenetic trees, whether of individual loci (so called "gene trees") or at the genomelevel (species trees), is a basic step in many biological analyses  [1]. However, estimating gene trees and species trees with high accuracy is difficult for many reasons, including computational issues (nearly all problems are NP-hard) and dataset issues. For example, while highly accurate gene trees can be computed for some loci, when a locus has limited phylogenetic signal (e. g. , its sequences are too short, or it evolves too slowly), its gene tree may only be estimated with partial accuracy. Species tree estimation is also difficult, because different loci can have different phylogenetic trees, a phenomenon that occurs due to several different biological processes. In particular, many groups of species evolve with rapid speciation events, a process that is likely to produce conflict between gene trees and species trees due to incomplete lineage sorting (ILS)  [2] [3] [4] [5].
Examples of coalescent-based summary methods include MP-EST  [15] , ASTRAL  [16, 17] , STAR  [13]  and NJst  [18]. Coalescent-based analyses of biological datasets typically use this kind of pipeline, since they can be computationally more efficient than other types of coalescent-based analyses (for example, methods like *BEAST  [19]  that co-estimate the gene trees and species tree). Thus, we focus the discussion in this study on phylogenomic pipelines that have the following basic structure: • Step 1: a gene tree is estimated for each locus • Step 2: the gene trees are combined into a species tree using a coalescent-based summary method. While many studies have explored the statistical properties of coalescent-based summary methods given true gene trees, here we focus on their use when the input is a set of sequence alignments for multiple loci.
In  [23] , we developed a technique we called "Statistical Binning" to improve species tree estimation using phylogenomic pipelines based on coalescent-based summary methods. Statistical binning partitions the genes into sets based on a heuristic to evaluate "combinability", concatenates the gene sequence alignments within each set into a "supergene alignment", and then estimates a "supergene tree" on the supergene alignment using a fully partitioned maximum likelihood analysis. The newly estimated supergene trees are then used by the preferred coalescent-based summary method to compute a species tree on the dataset. As shown in  [23] , statistical binning improved the estimation of gene trees and gene tree distributions, and this resulted in improved estimates of the species tree topology and branch lengths when species trees were computed using MP-EST with multi-locus bootstrapping (MLBS).
Furthermore, when used with statistical binning, MP-EST was almost always at least as accurate as concatenation (more accurate than concatenation when the ILS level is high, and only less accurate than concatenation for very low levels of ILS). Finally, MP-EST used with statistical binning was used to compute a species tree on the avian phylogenomic dataset, and this "MP-EST*" tree was nearly identical to the concatenation analysis we obtained; the MP-EST* and concatenation trees were presented in  [26]  as the two major hypotheses for the avian phylogeny. Thus,  Mirarab et al. proposed a new type of phylogenomic species tree estimation pipeline that has four steps instead of two (where the extra two steps are partitioning the genes into bins based on perceived "incompatibility", and computing supergene trees for each bin using a fully partitioned maximum likelihood analysis).
This pipeline, which we referred to as "statistical binning", showed very promising results when used with MP-EST. However, we did not address the theoretical properties of these pipelines, we only examined model trees with 37 or more species, and we only analyzed one coalescent-based summary method, MP-EST. In this paper, we report on an extended evaluation of statistical binning. Specifically, • We provide a proof of statistical inconsistency under GTR+MSC for pipelines based on the original protocol for statistical binning presented in  [23]. • We describe a variant of statistical binning that we call "weighted statistical binning", and provide a proof of statistical consistency under GTR+MSC for pipelines based on weighted statistical binning. • We evaluate the impact of statistical binning (both weighted statistical binning and the original unweighted statistical binning technique) on biological and simulated datasets under the GTR+MSC model.
We examine pipelines using two coalescent-based summary methods, ASTRAL and MP-EST. We include results on simulated and biological datasets studied in  [23] , and also on additional simulated datasets with 10 and 15 taxa. The study shows that weighted and unweighted statistical binning have very similar results across most datasets, and also that both ASTRAL and MP-EST tend to improve in accuracy when used with binning. However, there was one condition (characterized by a very high level of ILS, low average bootstrap support for the gene trees, and only ten species) in which statistical binning reduced accuracy for both MP-EST and ASTRAL. Thus, this study shows that binning is often beneficial, but also that there are some conditions under which binning can increase rather than decrease species tree error.
Experimental Study Datasets. We use the avian and mammalian simulated datasets studied in  [23]  (each based on MP-EST analyses of biological datasets, and having at least 37 taxa) and two other collections of simulated datasets with 10 and 15 taxa. The simulated datasets range from moderately low ILS (the lowest ILS mammalian condition) to extremely high ILS conditions (the higher ILS 10-taxon and 15-taxon model conditions), and range in terms of average gene tree bootstrap support (from very low to moderately high). Thus, the simulated datasets provide a range of conditions in which we explore the impact of statistical binning. We also analyzed two biological datasets (a 48-species avian dataset and a 37-species mammalian dataset) studied in  [23]. We used biologically-based simulated datasets that were studied in  [23] , and are based on species trees estimated using MP-EST on the avian dataset of  [26]  and the mammalian dataset of  [31].
In the avian simulation, the markers vary in sequence length (250bp, 500bp, 1000bp, and 1500bp) in order to produce bootstrap support values similar to those we observed in the biological dataset. In the mammalian simulation, we again explored the impact of phylogenetic signal by varying the sequence length (250bp, 500bp, and 1000bp) for the markers. In both cases, three levels of ILS are simulated by multiplying or dividing all internal branch lengths in the model species tree by two, and we also explore various numbers of genes. The mammalian datasets range in ILS level from relatively low (18% average distance between true gene trees and the species tree) for the 2X branch length level to relatively high (54% average distance between true gene trees and the species tree) for the 0. 5X branch length level, and the average bootstrap support on the estimated gene trees ranges from low (43%) for the shorter (250bp) sequences to moderately high (79%) for the longest (1000bp) sequences.
The avian datasets have higher ILS levels than the mammalian datasets, and range from moderate (35% average distance between true gene trees and the species tree) for the 2X branch length condition to high (59% average distance between true gene trees and the species tree) for the 0. 5X branch length condition. The estimated gene trees range in average bootstrap support from very low (27%) for the shortest (250bp) sequences to moderate (60%) for the longest (1500bp) sequences. We also used a 15-taxon model species tree with a caterpillar-like (also known as a pectinate, or ladder-like) topology, which has 12 short internal branches (0. 1 in coalescence units) in succession, a condition that gives rise to high levels of ILS  [8, 32]. Ultrametric gene trees were simulated down this tree using the multi-species coalescent process (see Methods).
Unlike the biologically-based model conditions, no transformations of branch lengths were used, and therefore, gene trees follow a strict molecular clock. Sequence data were simulated down each gene tree, and we built four model conditions by trimming gene data to 100 or 1000 sites, and by using 100 or 1000 genes. This dataset is very homogeneous since all 10 replicates we simulated are based on the same species tree, and gene trees differ in topology and branch length only due to the coalescence process. The 15-taxon datasets have very high ILS levels (82% average topological distance between true gene trees and the species tree), and so represent a rather extreme condition. The gene trees estimated on the shorter sequences (100bp) had only 35% average bootstrap support, and the combination of very high ILS and very low average bootstrap support represents a very challenging condition.
Gene trees estimated on the longer sequences have better average bootstrap support (70%), and so represent a somewhat easier condition. We also generated 10-taxon simulated datasets using simPhy  [33]. In this simulation protocol, we simulated a different species tree for each replicate, and simulated 200 gene trees for each species tree using the multi-species coalescent process. We simulated two model conditions, one with very high ILS and another with somewhat lower (but still high) ILS. The simPhy procedure uses a host of various distributions to make the gene trees heterogeneous in various aspects, such as sequence lengths, deviation of branch lengths from the strict molecular clock, and rate variation across different genes. We used these gene trees to simulate sequence data with 100 sites using Indelible  [34].
Therefore, our 10-taxon datasets are very heterogeneous: different replicates have different species trees, and within each replicate, various genes have different rates of evolution. The ILS levels of the 10-taxon datasets range from moderately high (40% average distance from true gene trees to the species tree) for the "lower ILS" condition to extremely high (84% average distance) for the "higher ILS" condition. The average bootstrap support on the estimated gene trees ranged from 37% for the higher ILS condition to 45% for the lower ILS condition, and so both have very poor average bootstrap support. Thus, the 10-taxon and the 15-taxon datasets with short sequences represent the hardest model conditions in that they have very high ILS and very low average bootstrap support. The simulated datasets we studied varied in many respects (sequence length per locus, whether the sequence evolution is ultrametric or not, and the ILS level).
Methods. We computed coalescent-based species trees using summary methods with MLBS gene trees in three ways: without binning, with weighted statistical binning and with unweighted statistical binning. Our main focus is on MP-EST, but we explore results with ASTRAL on a subset of the data. ASTRAL estimates species trees given unrooted gene trees, and can analyze very large datasets (such as the plant transcriptome dataset with approximately 100 species and 600 loci  [35] ); hence, ASTRAL can analyze larger datasets than MP-EST, and so understanding the impact of binning on ASTRAL's accuracy is of practical importance. We perform statistical binning using both weighted and unweighted pipelines and using two support thresholds (B): 50% and 75%. Due to the extremely large computational effort involved, on our two large biologically-based simulated datasets, we explore one threshold for most of our results; we follow the protocol used in  [23]  and set B = 50% for the avian datasets, and B = 75% for the mammalian datasets.
However, we also study the impact of B on one model condition for avian and mammalian datasets. We compute gene trees and concatenation species trees using RAxML  [36]  maximum likelihood. For estimating supergene trees, we use fully partitioned RAxML analyses (using the -M option to vary branch lengths across genes) for smaller (10-and 15-taxon) simulated datasets and for all biological analyses. However, since partitioned analyses are expensive, we use unpartitioned analyses to compute supergene trees for our studies on the avian and mammalian simulated datasets (because these studies are very extensive). We compare results using coalescent-based summary methods to concatenation, also using unpartitioned maximum likelihood. Note that the binned methods and the concatenation analysis would potentially become more accurate if fully partitioned analyses were employed.
Measurements. For the simulated datasets, we explore species tree accuracy with respect to the true (model) species tree topology (the missing branch rate, or false negative rate (FN)) and branch lengths, and also examine the branch support of both true positive and false positive branches. We also explore the error in the estimated gene trees and gene tree distribution estimated using binning (weighted and unweighted), compared to unbinned analyses. We analyze these simulated datasets using weighted statistical binning with MP-EST and ASTRAL, to determine if there are differences between weighted and unweighted statistical binning. Since ASTRAL does not produce branch lengths, we only use MP-EST to evaluate branch length estimation. In addition, we examine the bootstrap support on the branches of estimated species trees produced using MP-EST, as false positive edges that have low support are not as deleterious as false positive edges with high support.
The bootstrap support of estimated species trees was not studied in  [23] , and so this study provides the first analysis of bootstrap support for MP-EST on these datasets, as well as of the impact of binning on bootstrap support values. These aspects of phylogenomic estimation are important for different reasons. Species tree topologies indicate which species are more closely related to each other than to others, and so estimating accurate species tree topologies is the most important aspect of phylogenomic estimation. However, the improvement in species tree (coalescent-unit) branch length estimation is also biologically relevant, since these lengths are related to effective population sizes and generation times of ancestral species, and are also used to estimate the amount of ILS in the data.
Bootstrap support is important, since low support branches are often ignored, but high support branches are generally assumed to be correct; hence, understanding whether a method returns high support for false positive branches (indicating incorrect relations within a tree) is particularly important. Improvements in estimating the gene tree distribution matter because the accuracy of summary methods depends on an input that captures the correct gene tree distribution. For the biological datasets, we compare estimated species trees to the literature for each dataset, focusing on whether the estimated species tree violates known subgroups for the phylogeny.
Biologically-based simulated datasets Gene tree error and gene tree distribution error on avian simulated datasets We evaluated the impact of statistical binning on gene tree estimation error for the 1X (default ILS) model condition, with sequence lengths varying from 250bp to 1500bp. At the shorter sequence lengths, gene tree estimation error was reduced substantially (from 79% to 57% for 250bp, and from 69% to 57% for 500bp) (S1 Table ). Gene tree estimation error was reduced slightly at 1000bp (from 55% to 51%) and even less at 1500bp (from 46% to 45%). Hence, when gene tree estimation error is high due to insufficient sequence length, then binning reduces gene tree estimation error, but binning has little impact on gene tree estimation error when the sequences are long enough. We measure the error in estimated gene tree distributions using the deviation of triplet frequencies from the triplet frequency distribution computed using the true gene trees (see Materials and Methods).
Also, the improvement over unbinned analyses was highest for the lowest ILS level (2X species tree branch lengths), but was high even for the highest ILS level we explored. Species tree estimation error on avian simulated datasets Fig. 3  shows results for species tree topology estimation error for analyses of avian genes of different length under the default ILS level using MP-EST and ASTRAL, for varying number of 500bp genes with default ILS using MP-EST, and for 1000 genes of 500bp with varying ILS using MP-EST. Weighted and unweighted statistical binning are essentially identical for both MP-EST and ASTRAL (no statistically significant differences were observed according to a two-way ANOVA test; see Tables  2  and 3 ), and both reduce species tree estimation error compared to unbinned analyses (differences were always statistically significant with p < 0.
001; see Tables  2  and 3 ). We performed ANOVA to test the significance of the choice of methods (unbinned, weighted binned, unweighted binned, WSB-50: weighted statistical binning using 50% bootstrap support threshold and WSB-75: weighted binning using 75% bootstrap support threshold). For weighted vs. unweighted, we compared 50% bootstrap support threshold for avian, 75% for mammalian, and both 50% and 75% for 15and 10-taxon datasets. All p-values are corrected for multiple hypothesis testing using the FDR correction (n = 16). "n. a. " stands for "not available". The largest improvements are for the shortest gene sequences, where error is reduced from 23% to 14% using MP-EST and from 19% to 13% using ASTRAL. The difference between binned and unbinned analyses is lower for 1000bp sequences, and there are no noteworthy differences for 1500bp sequences (sequence length has a statistically significant impact; see Tables  4  and 5 ).
When the number of genes is changed (see Fig. 3(c )), the impact of binning on MP-EST ranges from neutral to highly positive, and the largest improvements are for datasets with large numbers of genes (impact of the number of genes is We performed ANOVA to test the significance of whether there is an interaction between the choice of the method (unbinned, weighted binned, unweighted binned, WSB-50: weighted statistical binning using 50% bootstrap support threshold and WSB-75: weighted statistical binning using 75% bootstrap support threshold) and the variable changed in each dataset. For weighted vs. unweighted, we compared 50% bootstrap support threshold for avian, 75% for mammalian, and both 50% and 75% for 15-and 10-taxon datasets. All p-values are corrected for multiple hypothesis testing using the FDR correction (n = 21).
Bootstrap support on avian simulated datasets We explore bootstrap support of trees estimated on simulated avian datasets, as follows. We assign relative quality to each edge in an estimated tree, taking bootstrap support into account. The highest quality edges are the true positive branches with the highest bootstrap support, and the lowest quality edges are the false positive branches with the highest bootstrap support, and all other edges fall in between. We order all the edges by their quality, so that the true positive branches come first (with the high support branches before low support branches), followed by the false positive branches (with the low support branches before the high support branches). Given this ordering, we create figures in which the x-axis indicates the edge quality (from very high to very low, as you move from left to right), and the y-axis indicates the fraction of the edges having at least the quality indicated by the x-axis.
Comparisons to concatenation on avian simulated datasets On the shortest 250bp sequences, concatenation matches the accuracy of weighted and unweighted binned MP-EST methods (Fig. 3(a) ) and is slightly less accurate than both binned ASTRAL trees (Fig. 3(b) ). As sequence length increases, both types of binning using either ASTRAL or MP-EST become more accurate than concatenation. Unbinned analyses are less accurate than concatenation for shorter sequences and more accurate for longer sequences (the transition point depends on whether ASTRAL or MP-EST is used). Both binned analyses are more accurate than concatenation and unbinned analyses at all ILS levels (Fig. 3(d) ). Thus, compared to concatenation, binned analyses have their largest advantage on longer gene sequences, higher ILS levels, and higher number of genes.
Results on mammalian datasets Results on simulated mammalian datasets are similar to analyses of avian datasets. In nearly every condition, both weighted and unweighted binning show very similar results (see Fig. 6 ) and have no statistically significant differences using either ASTRAL or MP-EST (see Tables  2  and 3 ). As before, we evaluated the impact of statistical binning on gene tree estimation error under the 1X (default ILS) model condition with varying sequence lengths (S1 Table ), and observed that binning substantially reduces gene tree estimation error for short sequences (250bp and 500bp) but had little impact on longer sequences (1000bp). Binning improves gene tree distributions, generally with very large improvements, and the improvements decrease with the sequence length and ILS level (S3 Fig.
). Binning also improves species tree topology estimation (Fig. 6  and Tables  2  and 3 ). The impact appears to depend on the sequence length (binning seems more beneficial for shorter sequences and neutral for longer sequences) and number of genes (binning can dramatically improve species tree topologies given a large number of genes, but can be neutral or even detrimental for a small number of genes), and the choice of summary method (binning helps both ASTRAL and MP-EST, but helps MP-EST more). ILS level also seems to impact relative accuracy (Tables  4  and 5 ), so that binning seems most helpful for low ILS levels, and less helpful for high ILS levels (S4 Fig. ). However, the effects of number of genes, sequence length, and the ILS level were not statistically significant for this dataset (Tables  4  and 5 ).
Impact of support threshold B on avian and mammalian simulated datasets In addition to varying model conditions, we use a single avian and a single mammalian model condition to study the impact of the support threshold B on binning (Fig. 7 ). We use a mixed model condition with 200 genes of 500bp and 200 genes of 1000bp for the mammalian dataset, and a model condition with 1000 genes of 500bp for the avian dataset (both with default 1X ILS level). On the avian dataset, binning is always beneficial, but the impact is larger with B = 50% compared to B = 75% (Fig. 7 (a)). For example, unbinned MP-EST has 19% error, and using B = 50% reduces the error to 11%, and using B = 75% reduces the error to 13%. On the mammalian mixed data, binning is beneficial in all cases (see Fig. 7(b )); however, the extent of the impact depends substantially on both the threshold and the summary method.