MITOCHONDRIAL-GENE TREES VERSUS NUCLEAR-GENE TREES, A REPLY TO HOELZER.

With regard to lineage sorting, I argued that the mitochondrial-haplotype (mt-haplotype) tree is more likely to be congruent with the species tree than is a nuclear-gene tree. The probability of congruence depends on the length of internodes between bifurcations or speciation events and the effective size of the ancestral populations in the internodes of the species tree. For a three-species tree, when the internode is just long enough that the probability of congruence between the mt-haplotype tree and the species tree is 0.95, the probability of congruence for a nuclear-autosomal-gene tree is only 0.62. In this case, resolution of multiple nucleargene trees would probably result in a mixture of topologies, and approximately 16 nuclear-gene trees would need to be resolved before the majority topology would have a probability of 0.95 of being the species tree. The duration of common ancestry represented by the length of the species-tree internode is the same for the mt-genome as for a nuclear autosome, but the effective population size for the mt-genome is one-fourth that of a nuclear autosome. In general, the smaller the effective population size, the higher the probability of congruence between gene tree and species tree; thus, the mt-haplotype tree is more likely to be congruent with the species tree. My specific argument is based on a number of assumptions. Among these are that nucleotide substitutions are neutral, the species population is panmictic, and that progeny are procreated by equal numbers of males and females; that is, variance in reproductive success is the same for males as females. Hoelzer provides commentary on conditions that "contradict" my analysis. The two scenarios he examines, (1) polygyny and (2) female philopatry, both reduce effective population size of nuclear genes relative to that of the mt-genome, and thus, of course, I agree with his conclusion that there are circumstances where a nuclear-gene tree might provide a better inference of the species tree than the mt-haplotype tree. But, in answering Hoelzer's rhetorically posed question, "Is mtDNA a useful source of data for the estimation of species trees?" I would answer usually rather than "sometimes." Is this a trivial distinction? I think not. Words like "sometimes" and "usually" are ambiguous, and thus to reduce, but not eliminate this ambiguity, I say that there are very few circumstances where the mt-haplotype tree would not be the best bet as the true species tree. Before considering Hoelzer's scenarios, I point out again that my arguments apply only to the problem of lineage sorting and presuppose that the mt-haplotype tree or nuclear-gene trees have been correctly resolved. There is probably a broad range of circumstances where a given amount of nucleargene sequence might actually lead to a more highly resolved gene tree than does a comparable amount of mt-gene sequence. This could result, for example, from the base composition bias of mitochondrial DNA (mtDNA) or from the more extreme asymmetry of its transition probability matrix (e.g., the probabilities of C to T changes and vice versa are unusually high at synonymous positions of mtDNA). But, under most plausible circumstances of natural history, given that the gene trees have been correctly resolved, the mt-haplotype tree would provide a more reliable inference of the species tree than a nuclear gene-indeed, than several nuclear-gene trees. Actually, I am moderately surprised that conditions must be as extreme as those calculated by Hoelzer before nuclear-gene trees provide an advantage. Polygyny is the simpler of the two scenarios considered by Hoelzer. Hoelzer calculated that a nuclear-autosome tree would have probability of congruence with the species tree equal to that of the mt-haplotype tree when seven females procreate for each male. Even for very polygynous species, this seems extreme, and the effect of apparent polygyny would be mitigated by phenomena such as cleptogamy and continuous generations where males unable to reproduce in one breeding season later succeed. As implied by Hoelzer, few quantitative data are available for empirically assessing the effect of mating systems on effective population size, but his point is well taken that the mating system should be considered in choosing genes for phylogenetic inference. Hoelzer's second scenario, based on an island model of habitat occupation, is more complex to assess. For low levels of dispersal between islands, mt-haplotype frequencies will vary more among islands (higher Fst) than will nuclear autosomes because of the smaller effective population size experienced by the mt-genome; this differential would be enhanced if females were less inclined to disperse than males. The net effect is to increase the effective population size of the mt-genome, when the species population as a whole is considered, because alternate haplotypes become locally fixed, resulting in the species population being more polymorphic than expected if there were specieswide panmixis. Hoelzer argues, given that the effective population size of the mt-genome for the species is increased relative to that of a nuclear autosome, the probability of congruence of the mt-haplotype tree with the species tree should diminish relative to that of a nuclear autosome. This argument can be subtly misleading. The key concept is that the effective size of the species population as a whole is increased. But the effect on congruence of gene trees and species trees would depend on how speciation occurs; that is, whether the incipient species arises in a geographically localized area or from the whole of the ancestral species population.