The number of replications needed for accurate estimation of the bootstrap P value in phylogenetic studies.
Authors: S B Hedges
Journal: Molecular Biology and Evolution (1992)

The bootstrap is a statistical method for obtaining a nonparametric estimate of error (Efron 1979, 1982). Felsenstein ( 1985) was the first to apply this method to phylogeny estimation, and his approach is now widely used. Taxa are held constant, and the characters (for sequence data, nucleotide or amino acid sites) are resampled randomly with replacement. A phylogeny is constructed from each replication of the data, and the frequency of appearance of particular phylogenetic groups (groups of alleles or taxa) among all of the trees constructed by this resampling is the bootstrap confidence limit, or bootstrap P value (BP). The BPS of different nodes within a tree can be used to assess the relative stability of those phylogenetic groups or, if applied strictly, to test their statistical significance (e.g., at the 95% or 99% level). The application of bootstrapping to phylogeny estimation is a tradeoff between the maximum number of replications that can be performed by the researcher in a reasonable amount of time and the minimum number of replications needed for accurate estimation of the BP. The purpose of the present report is to explore the variance (and hence the accuracy) of the phylogenetic BP and to establish guidelines for efficient bootstrap sampling. BP is the proportion of trees containing a particular phylogenetic group. It therefore follows the binomial distribution, which has a variance of o2 = [P( 1 P)/n], where P is the BP and II is the number of replications. Although Li and Gouy ( 1990) recently suggested that more replications are needed for larger numbers of taxa, the accuracy of the BP is a function only of P and n. If the interval containing 95% of the samples (+ 1.96 standard deviations) is used as a measure of accuracy, then the application of the above formula shows that 1,825 replications [ =0.95 X 0.05( 1.96/ 0.0 1 )2] are needed to attain + 1% accuracy at a BP of 0.95 (fig. 1). This is more than an order of magnitude higher than the number of replications (50-100) normally used in phylogenetic analyses. Based on this, a practical guideline for efficient and accurate bootstrap sampling can be made: If one wishes the expectation to be that the 95% confidence range is + 1% of the BP, then one must perform 2,000 bootstrap replications (if BP = 0.95) or 400 replications (if BP = 0.99) in phylogenetic analyses, unless the computational time is prohibitive; additional replications are unnecessary, and fewer replications may sacrifice statistical accuracy. Moreover, statistical testing at the 95% level cannot be made using ~73 replications, even if the group is supported by a BP of 1 .OO. This is because the inaccuracy at a mean BP of 0.95 is greater than f5% (fig. 1) when ~73 replications are used. In other words, a BP > 1 .OO could be obtained when the actual (mean) BP is not significant ( cO.95). Thus, the 50 replications used by Felsenstein ( 1985) in his original example and the 20100 replications used in many subsequent studies (e.g., see Ovenden et al. 1987; Thomas et al. 1989; Jansen et al. 1990; Meyer and Wilson 1990; Douglas et al. 199 1; Irwin et al. 199 1) would appear to be far too few for the intended purpose of statistical testing at the 95% level.