Is recombination a problem for species-tree analyses?

As the field of phylogenetics transitions into phylogenomics it has spurred a shift in the general paradigm for data analysis whereby specific dataset attributes can now be considered in a model-based framework. Much effort has been put into modeling the effects of nucleotide substitution within a genealogy (i.e., modeling mutations; Felsenstein 2005) and the sorting of genes between lineages (i.e., coalescence; Knowles and Kubatko 2010), both of which are inherent properties of all species histories. Improvements in accuracy of speciestree estimation related to using models that account for these two sources of uncertainty have been well documented (Carstens and Knowles 2007; Edwards et al. 2007; Kubatko and Degnan 2007), but these are clearly not the only sources of gene-tree species-tree discordance. As the field transitions towards using multilocus data, the role of other inherent dataset properties that may be sources of uncertainty, such as intralocus recombination, needs to be examined and quantified. As all species-tree methods in current use rely on the simplifying assumption that recombination occurs between but not within loci, ignoring the presence of recombination represents a widespread and ubiquitous violation of species-tree models. Recombination within a locus may be a greater problem for species-tree methodologies than for concatenation because species-tree methods more accurately model the patterns arising from coalescent stochasticity. Concatenation assumes that all genes share a common underlying tree, a model violated by most multilocus data (Knowles and Carstens 2007; Cranston 2010; Linnen 2010; Wiens et al. 2010). The failure to recognize recombination events may result in a mismatch between estimated gene trees and the true genealogies (Fig. 1). This discord is necessarily smaller relative to the discord of both ignoring recombination and making the assumption that all loci would have the same genealogy (i.e., ignoring coalescent variance), as when data are concatenated. Nevertheless, because coalescent methods for species-tree estimation explicitly model each gene tree separately, the relative contribution of recombination to the uncertainty of the estimated species tree may be substantial. Challenges posed by recombination may also be greatest for species-tree methods that rely on estimating parameters related to the scaled mutation rate (Liu 2008; Liu et al. 2008; Heled and Drummond 2010) because recombinationintroduced heterogeneity may interfere with branch length estimates or for species-tree methods that do not fully account for gene-tree uncertainty (Maddison and Knowles 2006; Kubatko et al. 2009). Lastly, recombination may interact with other aspects of the speciation history, such as time to divergence, population size, the length of time between speciation events, and sampling effort (McCormack et al. 2009). For example, recombination events occurring in recently speciated groups may not have accumulated a sufficient number of mutations for the effects to be problematic (see Fig. 1). Unlike coalescent events, which are treelike and depend on waiting time to coalescence, accounting for both coalescence and recombination requires an ancestral recombination graph with a mathematically complex series of reticulations (Hein et al. 2005; Wakeley 2009). Because of the complexity introduced by recombination, no species-tree reconstruction method (in current use) deals with it explicitly. Options do exist for simulating the process of coalescence with recombination as it affects a genealogy, though, and these may be useful for understanding the effects of recombination in species-tree estimation. Hudson’s (1983) formulation models coalescence and recombination as 2 competing exponentially distributed events that differ in their relationship to population size. The rate of coalescence is proportional to the square of the number of ancestors, whereas recombination rate increases linearly with ancestral population size (Hein et al. 2005), meaning that coalescence will always be more prevalent in a data set than recombination. Furthermore, not all recombination events are likely to have the same effect on gene-tree quality. Looking backward through time, recombination events can be divided into 3 categories relative to coalescence Advance Access publication on January 2, 2012 Syst. Biol. 61(4):691–701, 2012