Original: V ARIATIONAL B AYESIAN P HYLOGENETIC I NFERENCE
Revised: V ARIATIONAL B AYESIAN P HYLOGENETIC I NFERENCE

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Bayesian phylogenetic inference is currently done via Markov chain Monte Carlo with simple mechanisms for proposing new states, which hinders exploration efﬁciency and often requires long runs to deliver accurate posterior estimates.
Revised: Bayesian phylogenetic inference is currently done via Markov chain Monte Carlo with simple mechanisms for proposing new states, which hinders exploration efﬁciency and often requires long runs to deliver accurate posterior estimates.

Original: In this paper we present an alternative approach: a variational framework for Bayesian phylogenetic analysis.
Revised: In this paper we present an alternative approach: a variational framework for Bayesian phylogenetic analysis.

Original: We approximate the true posterior using an expressive graphical model for tree distributions, called a subsplit Bayesian network, together with appropriate branch length distributions.
Revised: We approximate the true posterior using an expressive graphical model for tree distributions, called a subsplit Bayesian network, together with appropriate branch length distributions.

Original: We train the variational approximation via stochastic gradient ascent and adopt multi-sample based gradient estimators for different latent variables separately to handle the composite latent space of phylogenetic models.
Revised: We train the variational approximation via stochastic gradient ascent and adopt multi-sample based gradient estimators for different latent variables separately to handle the composite latent space of phylogenetic models.

Original: We show that our structured variational approximations are ﬂexible enough to provide comparable posterior estimation to MCMC, while requiring less computation due to a more efﬁcient tree exploration mechanism enabled by variational inference.
Revised: We show that our structured variational approximations are ﬂexible enough to provide comparable posterior estimation to MCMC, while requiring less computation due to a more efﬁcient tree exploration mechanism enabled by variational inference.

Original: Moreover, the variational approximations can be readily used for further statistical analysis such as marginal likelihood estimation for model comparison via importance sampling.
Revised: Moreover, the variational approximations can be readily used for further statistical analysis such as marginal likelihood estimation for model comparison via importance sampling.

Original: Experiments on both synthetic data and real data Bayesian phylogenetic inference problems demonstrate the effectiveness and efﬁciency of our methods.
Revised: Experiments on both synthetic data and real data Bayesian phylogenetic inference problems demonstrate the effectiveness and efﬁciency of our methods.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Bayesian phylogenetic inference is an essential tool in modern evolutionary biology.
Revised: Bayesian phylogenetic inference is an essential tool in modern evolutionary biology.

Original: Given an alignment of nucleotide or amino acid sequences and appropriate prior distributions, Bayesian methods provide principled ways to assess the phylogenetic uncertainty by positing and approximating a posterior distribution on phylogenetic trees (Huelsenbeck et al., 2001).
Revised: Given an alignment of nucleotide or amino acid sequences and appropriate prior distributions, Bayesian methods provide principled ways to assess the phylogenetic uncertainty by positing and approximating a posterior distribution on phylogenetic trees (Huelsenbeck et al., 2001).

Original: In addition to uncertainty quantiﬁcation, Bayesian methods enable integrating out tree uncertainty in order to get more conﬁdent estimates of parameters of interest, such as factors in the transmission of Ebolavirus (Dudas et al., 2017).
Revised: In addition to uncertainty quantiﬁcation, Bayesian methods enable integrating out tree uncertainty in order to get more conﬁdent estimates of parameters of interest, such as factors in the transmission of Ebolavirus (Dudas et al., 2017).

Original: Bayesian methods also allow complex substitution models (Lartillot & Philippe, 2004), which are important in elucidating deep phylogenetic relationships (Feuda et al., 2017).
Revised: Bayesian methods also allow complex substitution models (Lartillot & Philippe, 2004), which are important in elucidating deep phylogenetic relationships (Feuda et al., 2017).

Original: Ever since its introduction to the phylogenetic community in the 1990s, Bayesian phylogenetic inference has been dominated by random-walk Markov chain Monte Carlo (MCMC) approaches (Yang & Rannala, 1997; Mau et al., 1999; Huelsenbeck & Ronquist, 2001).
Revised: Ever since its introduction to the phylogenetic community in the 1990s, Bayesian phylogenetic inference has been dominated by random-walk Markov chain Monte Carlo (MCMC) approaches (Yang & Rannala, 1997; Mau et al., 1999; Huelsenbeck & Ronquist, 2001).

Original: However, this approach is fundamentally limited by the complexities of tree space.
Revised: However, this approach is fundamentally limited by the complexities of tree space.

Original: A typical MCMC method for phylogenetic inference involves two steps in each iteration: ﬁrst, a new tree is proposed by randomly perturbing the current tree, and second, the tree is accepted or rejected according to the Metropolis-Hastings acceptance probability.
Revised: A typical MCMC method for phylogenetic inference involves two steps in each iteration: ﬁrst, a new tree is proposed by randomly perturbing the current tree, and second, the tree is accepted or rejected according to the Metropolis-Hastings acceptance probability.

Original: Any such random walk algorithm faces obstacles in the phylogenetic case, in which the high-posterior trees are a tiny fraction of the combinatorially exploding number of trees.
Revised: Any such random walk algorithm faces obstacles in the phylogenetic case, in which the high-posterior trees are a tiny fraction of the combinatorially exploding number of trees.

Original: Thus, major modiﬁcations of trees are likely to be rejected, restricting MCMC tree movement to local modiﬁcations that may have difﬁculty moving between multiple peaks in the posterior distribution (Whidden & Matsen IV, 2015).
Revised: Thus, major modiﬁcations of trees are likely to be rejected, restricting MCMC tree movement to local modiﬁcations that may have difﬁculty moving between multiple peaks in the posterior distribution (Whidden & Matsen IV, 2015).

Original: Although recent MCMC methods for distributions on Euclidean space use intelligent proposal mechanisms such as Hamiltonian Monte Carlo (Neal, 2011), it is not straightforward to extend such algorithms to the composite structure of tree space, which includes both tree topology (discrete object) and branch lengths (continuous positive vector)
Revised: Although recent MCMC methods for distributions on Euclidean space use intelligent proposal mechanisms such as Hamiltonian Monte Carlo (Neal, 2011), it is not straightforward to extend such algorithms to the composite structure of tree space, which includes both tree topology (discrete object) and branch lengths (continuous positive vector)

Original: (Dinh et al., 2017).
Revised: (Dinh et al., 2017).

Original: Variational inference (VI) is an alternative approximate inference method for Bayesian analysis which is gaining in popularity (Jordan et al., 1999; Wainwright & Jordan, 2008; Blei et al., 2017).
Revised: Variational inference (VI) is an alternative approximate inference method for Bayesian analysis which is gaining in popularity (Jordan et al., 1999; Wainwright & Jordan, 2008; Blei et al., 2017).

Original: Unlike MCMC methods that sample from the posterior, VI seeks the best candidate from a family of tractable distributions that minimizes a statistical distance measure to the target posterior, usually the Kullback-Leibler (KL) divergence.
Revised: Unlike MCMC methods that sample from the posterior, VI selects the best candidate from a family of tractable distributions to minimize a statistical distance measure to the target posterior, usually the Kullback-Leibler (KL) divergence.

Original: By reformulating the inference problem into an optimization problem, VI tends to be faster and easier to scale to large data (via stochastic gradient descent) (Blei et al., 2017).
Revised: By reformulating the inference problem into an optimization problem, VI tends to be faster and easier to scale to large data (via stochastic gradient descent) (Blei et al., 2017).

Original: However, VI can also introduce a large bias if the variational distribution is insufﬁciently ﬂexible.
Revised: However, VI can also introduce a large bias if the variational distribution is insufﬁciently ﬂexible.

Original: The success of variational methods, therefore,largely relies on the existence of appropriate tractable variational distributions and efﬁcient training procedures.
Revised: The success of variational methods, therefore, relies on having appropriate tractable variational distributions and efﬁcient training procedures.

Original: To our knowledge, there have been no previous variational formulations of Bayesian phylogenetic inference.
Revised: To our knowledge, there have been no previous variational formulations of Bayesian phylogenetic inference.

Original: This has been due to the lack of an appropriate family of  distributions on phylogenetic trees that can be used as an approximating distribution.
Revised: This has been due to the lack of an appropriate family of approximating distributions on phylogenetic trees.

Original: However the prospects for variational inference have changed recently with the introduction of subsplit Bayesian networks (SBNs) (Zhang & Matsen IV, 2018), which provide a family of ﬂexible distributions on tree topologies (i.e. trees without branch lengths).
Revised: However the prospects for variational inference have changed recently with the introduction of subsplit Bayesian networks (SBNs) (Zhang & Matsen IV, 2018), which provide a family of ﬂexible distributions on tree topologies (i.e. trees without branch lengths).

Original: SBNs build on previous work (H¨ohna & Drummond, 2012; Larget, 2013), but in contrast to these previous efforts, SBNs are sufﬁciently ﬂexible for real Bayesian phylogenetic posteriors (Zhang & Matsen IV, 2018).
Revised: SBNs build on previous work (H¨ohna & Drummond, 2012; Larget, 2013), but in contrast to these previous efforts, SBNs are sufﬁciently ﬂexible for real Bayesian phylogenetic posteriors (Zhang & Matsen IV, 2018).

Original: In this paper, we develop a general variational inference framework for Bayesian phylogenetics.
Revised: In this paper, we develop a general variational inference framework for Bayesian phylogenetics.

Original: We show that SBNs, when combined with appropriate approximations for the branch length distribution, can provide ﬂexible variational approximations over the joint latent space of phylogenetic trees with branch lengths.
Revised: We show that SBNs, when combined with appropriate approximations for the branch length distribution, can provide ﬂexible variational approximations over the joint latent space of phylogenetic trees with branch lengths.

Original: We use recently-proposed unbiased gradient estimators for the discrete and continuous components separately to enable efﬁcient stochastic gradient ascent.
Revised: We use recently-proposed unbiased gradient estimators for the discrete and continuous components separately to enable efﬁcient stochastic gradient ascent.

Original: We also leverage the similarity of local structures among trees to reduce the complexity of the variational parameterization for the branch length distributions and provide an extension to better capture the between-tree variation.
Revised: We also leverage the similarity of local structures among trees to reduce the complexity of the variational parameterization for the branch length distributions and provide an extension to better capture the between-tree variation.

Original: Finally, we demonstrate the effectiveness and efﬁciency of our methods on both synthetic data and a benchmark of challenging real data Bayesian phylogenetic inference problems.
Revised: Finally, we demonstrate the effectiveness and efﬁciency of our methods on both synthetic data and a benchmark of challenging real data Bayesian phylogenetic inference problems.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: Phylogenetic Posterior
Revised: Phylogenetic Posterior

Original: A phylogenetic tree is described by a tree topology τ and associated nonnegative branch lengths q .
Revised: A phylogenetic tree is described by a tree topology τ and associated nonnegative branch lengths q .

Original: The tree topology τ is used to model the evolutionary diversiﬁcation of the species.
Revised: The tree topology τ represents the evolutionary diversiﬁcation of the species.

Original: It is a bifurcating tree with N leaves, each of which has a label corresponding to one of the observed species.
Revised: It is a bifurcating tree with N leaves, each of which has a label corresponding to one of the observed species.

Original: The internal nodes of τ represent the unobserved characters (e.g. DNA bases) of the ancestral species.
Revised: The internal nodes of τ represent the unobserved characters (e.g. DNA bases) of the ancestral species.

Original: A continuous-time Markov model is often used to describe the transition probabilities of the characters along the branches of the tree.
Revised: A continuous-time Markov model is often used to describe the transition probabilities of the characters along the branches of the tree.

Original: A Bayesian approach to phylogenetic analysis requires a likelihood model for sequence evolution through a phylogenetic tree, and prior distributions on the tree topologies and the branch lengths.

Original: Let Y =
Revised: Let Y =

Original: { Y 1 , Y 2 , . . .
Revised: { Y 1 , Y 2 , . . .

Original: , Y M } ∈ Ω N × M be the observed sequences (with characters in Ω ) of length M over N species.
Revised: , Y M } ∈ Ω N × M be the observed sequences (with characters in Ω ) of length M over N species.

Original: The probability of each site observation Y i is deﬁned as the marginal distribution over the leaves
Revised: The probability of each site observation Y i is deﬁned as the marginal distribution over the leaves

Original: [Equation]
Revised: [Equation]

Original: where ρ is the root node (or any internal node if the tree is unrooted and the Markov model is time reversible), a i ranges over all extensions of Y i to the internal nodes with a iu being the assigned character of node u , E ( τ ) denotes the set of edges of τ , P ij ( t ) denotes the transition probability from character i to character j across an edge of length t and η is the stationary distribution of the Markov model.
Revised: where ρ is the root node (or any internal node if the tree is unrooted and the Markov model is time reversible), a i ranges over all extensions of Y i to the internal nodes with a iu being the assigned character of node u , E ( τ ) denotes the set of edges of τ , P ij ( t ) denotes the transition probability from character i to character j across an edge of length t and η is the stationary distribution of the Markov model.

Original: Assuming different sites are identically distributed and evolve independently, the likelihood of observing the  sequence set Y has the form
Revised: Assuming different sites are identically distributed and evolve independently, the likelihood of observing the entire sequence set Y is p ( Y | τ, q ) =

Original: [Equation]
Revised: (cid:81)

Original:  The phylogenetic likelihood  in equation 1 can be evaluated efﬁciently through the pruning algorithm (Felsenstein, 2003), also known as the sum-product algorithm in probabilistic graphical models (Strimmer & Moulton, 2000; Koller & Friedman, 2009; H¨ohna et al., 2014).
Revised: Mi =1 p ( Y i | τ, q ) . The phylogenetic likelihood for each site in equation 1 can be evaluated efﬁciently through the pruning algorithm (Felsenstein, 2003), also known as the sum-product algorithm in probabilistic graphical models (Strimmer & Moulton, 2000; Koller & Friedman, 2009; H¨ohna et al., 2014).

Original: Given a proper prior distribution with density p ( τ, q ) imposed on the tree topologies and the branch lengths, the  posterior p ( τ, q | Y ) is proportional to the joint density
Revised: Given a proper prior distribution with density p ( τ, q ) imposed on the tree topologies and the branch lengths, the phylogenetic posterior p ( τ, q | Y ) is proportional to the joint density

Original: [Equation]
Revised: [Equation]

Original: where p ( Y ) is the intractable normalizing constant.
Revised: where p ( Y ) is the intractable normalizing constant.

Revised: Subsplit Bayesian Networks We now review subsplit Bayesian networks (Zhang & Matsen IV, 2018) and the ﬂexible distributions on tree topologies they provide.

Revised: Let X be the set of leaf labels.

Original: Species A:
Revised: Species A:

Original: Species B:
Revised: Species B:

Original: Species C:
Revised: Species C:

Original: Species D:
Revised: Species D:

Original: ATGAAC ··· ATGCAC ···
Revised: ATGAAC ··· ATGCAC ···

Original: ATGCAT ··· ATGCAT ···
Revised: ATGCAT ··· ATGCAT ···

Original: D
Revised: D

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: ABCD
Revised: ABCD

Original: ABC
Revised: ABC

Original: D
Revised: D

Original: A
Revised: A

Original: BC
Revised: BC

Original: D
Revised: D

Original: D
Revised: D

Original: ABCD
Revised: ABCD

Original: AB
Revised: AB

Original: CD
Revised: CD

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D a ss i gn a ss
Revised: D a ss i gn a ss

Original: i g n
Revised: i g n

Original: S S
Revised: S S

Original: S S
Revised: S S

Original: S S
Revised: S S

Original: S
Revised: S

Original: Subsplit Bayesian Networks We now review subsplit Bayesian networks (Zhang & Matsen IV, 2018), powerful probabilistic models that can provide a rich family of ﬂexible distributions on tree topologies.

Original: Let X be the set of leaf labels.

Original: We call a nonempty subset of X a clade .
Revised: We call a nonempty subset of X a clade .

Original: Let (cid:31) be a total order on clades (e.g., lexicographical order).
Revised: Let (cid:31) be a total order on clades (e.g., lexicographical order).

Original: A subsplit ( W, Z ) of a clade X is an ordered pair of disjoint subclades of X such that W ∪ Z = X
Revised: A subsplit ( W, Z ) of a clade X is an ordered pair of disjoint subclades of X such that W ∪ Z = X

Original: and W (cid:31) Z .
Revised: and W (cid:31) Z .

Original: A subsplit Bayesian network B X on a leaf set X of size N is a Bayesian network whose nodes take on subsplitvalues or singleton clade values that represent the local topological structure of trees (Figure 1).
Revised: A subsplit Bayesian network B X on a leaf set X of size N is a Bayesian network whose nodes take on subsplit or singleton clade values that represent the local topological structure of trees (Figure 1).

Original: Following the splitting processes (see the solid dark subgraphs in Figure 1, middle right panel), rooted trees have unique subsplit decompositions and hence can be uniquely represented as compatible SBN assignments (Figure 1).
Revised: Following the splitting processes (see the solid dark subgraphs in Figure 1, middle right), rooted trees have unique subsplit decompositions and hence can be uniquely represented as compatible SBN assignments.

Original: Given the subsplit decomposition of a rooted tree τ = { s 1 , s 2 , . . . } , where s 1 is the root subsplit and { s i } i> 1 are otherlocal subsplits, the SBN tree probability takes the following form
Revised: Given the subsplit decomposition of a rooted tree τ = { s 1 , s 2 , . . . } , where s 1 is the root subsplit and { s i } i> 1 are other subsplits, the SBN tree probability is

Original: [Equation]
Revised: [Equation]

Original: where S i denotes the subsplit- or singleton-clade-valued random variables at node i and π i is the index set of the parents of S i .
Revised: where S i denotes the subsplit- or singleton-clade-valued random variables at node i and π i is the index set of the parents of S i .

Original: The Bayesian network formulation of SBNs enjoys many beneﬁts: i) ﬂexibility .
Revised: The Bayesian network formulation of SBNs enjoys many beneﬁts: i) ﬂexibility .

Original: The expressiveness of SBNs is freely adjustable by changing the dependency structures between nodes, allowing for a wide range of ﬂexible distributions; ii) normality .
Revised: The expressiveness of SBNs is freely adjustable by changing the dependency structures between nodes, allowing for a wide range of ﬂexible distributions; ii) normality .

Original: SBN-induced distributions are all naturally normalized if the associated conditional probability tables (CPTs) are consistent, which is a common property of Bayesian networks.
Revised: SBN-induced distributions are all naturally normalized if the associated conditional probability tables (CPTs) are consistent, which is a common property of Bayesian networks.

Original: The SBN framework also generalizes to unrooted trees, which are the most common type of trees in phylogenetics.
Revised: The SBN framework also generalizes to unrooted trees, which are the most common type of trees in phylogenetics.

Original: Concretely, unrooted trees can be viewed as rooted trees with unobserved roots.
Revised: Concretely, unrooted trees can be viewed as rooted trees with unobserved roots.

Original: Marginalizing out the unobserved root node S 1 , we have the SBN probability estimates for unrooted trees
Revised: Marginalizing out the unobserved root node S 1 , we have the SBN probability estimates for unrooted trees

Original: [Equation]
Revised: [Equation]

Original: where ∼ means all root subsplits that are compatible with τ .
Revised: where ∼ means all root subsplits that are compatible with τ .

Original: To reduce model complexity and encourage generalization, the same set of CPTs for parent-child subsplit pairs is shared across the SBN network, regardless of their locations.
Revised: To reduce model complexity and encourage generalization, the same set of CPTs for parent-child subsplit pairs is shared across the SBN network, regardless of their locations.

Original: Similar to weight sharing used in convolutional networks (LeCun et al., 1998) for detecting translationally-invariant structure of images (e.g., edges, corners), this heuristic parameter sharing used in SBNs is for identifying conditional splitting patterns of phylogenetic trees.
Revised: Similar to weight sharing used in convolutional networks (LeCun et al., 1998) for detecting translationally-invariant structure of images (e.g., edges, corners), this heuristic parameter sharing used in SBNs is for identifying conditional splitting patterns of phylogenetic trees.

Original: Prior to this work, SBNs were introduced as an extension of previous work (H¨ohna & Drummond, 2012; Larget, 2013) to further improve the posterior estimation of phylogenetic trees based on MCMC samples.

Original: See Zhang & Matsen IV (2018) for more detailed discussion on SBNs.
Revised: See Zhang & Matsen IV (2018) for more detailed discussion on SBNs.

Original:  V ARIATIONAL P HYLOGENETIC I NFERENCE VIA SBN S
Revised:  V ARIATIONAL P HYLOGENETIC I NFERENCE VIA SBN S

Original: The ﬂexible and tractable tree topology distributions provided by SBNsalso serve as an essential building block to perform variational inference (Jordan et al., 1999) for phylogenetics.
Revised: The ﬂexible and tractable tree topology distributions provided by SBNs serve as an essential building block to perform variational inference (Jordan et al., 1999) for phylogenetics.

Original: Suppose that we have a family of approximate distributions Q φ ( τ ) (e.g., SBNs) over phylogenetic tree topologies, where φ denotes the corresponding variational parameters (e.g., CPTs for SBNs).
Revised: Suppose that we have a family of approximate distributions Q φ ( τ ) (e.g., SBNs) over phylogenetic tree topologies, where φ denotes the corresponding variational parameters (e.g., CPTs for SBNs).

Original: For each tree τ , we posit another family of densities Q ψ ( q | τ ) over the branch lengths, where ψ is the branch length variational parameters.
Revised: For each tree τ , we posit another family of densities Q ψ ( q | τ ) over the branch lengths, where ψ is the branch length variational parameters.

Original: We then combine these distributions and use the product
Revised: We then combine these distributions and use the product

Original: [Equation]
Revised: [Equation]

Original: as our variational approximation.
Revised: as our variational approximation.

Original: Inference now amounts to ﬁnding the member of this family that minimizes the Kullback-Leibler (KL) divergence to the exact posterior,
Revised: Inference now amounts to ﬁnding the member of this family that minimizes the Kullback-Leibler (KL) divergence to the exact posterior,

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to maximizing the evidence lower bound (ELBO),
Revised: which is equivalent to maximizing the evidence lower bound (ELBO),

Original: [Equation]
Revised: [Equation]

Original: As the ELBO is based on a single-sample estimate of the evidence, it heavily penalizes samples that fail to explain the observed sequences.
Revised: As the ELBO is based on a single-sample estimate of the evidence, it heavily penalizes samples that fail to explain the observed sequences.

Original: As a result, the variational approximation tends to cover only the high-probability areas of the true posterior.
Revised: As a result, the variational approximation tends to cover only the high-probability areas of the true posterior.

Original: This effect can be minimized by averaging over
Revised: This effect can be minimized by averaging over

Original: K > 1 samples when estimating the evidence (Burda et al., 2016; Mnih & Rezende, 2016), which leads to tighter lower bounds
Revised: K > 1 samples when estimating the evidence (Burda et al., 2016; Mnih & Rezende, 2016), which leads to tighter lower bounds

Original: [Equation]
Revised: [Equation]

Original: where Q φ , ψ ( τ 1: K , q 1: K ) ≡ (cid:81)
Revised: where Q φ , ψ ( τ 1: K , q 1: K ) ≡ (cid:81)

Original: Ki
Revised: Ki

Original: =1 Q φ , ψ ( τ i , q i ) ; the tightness of the lower bounds improves as the number of samples K increases (Burda et al., 2016).
Revised: =1 Q φ , ψ ( τ i , q i ) ; the tightness of the lower bounds improves as the number of samples K increases (Burda et al., 2016).

Original: We will use multi-sample lower bounds in the sequel and refer to them as lower bounds for short.
Revised: We will use multi-sample lower bounds in the sequel and refer to them as lower bounds for short.

Original: 3. V ARIATIONAL P ARAMETERIZATION
Revised: 3. V ARIATIONAL P ARAMETERIZATION

Original: The CPTs in SBNs are, in general, associated with all possible parent-child subsplit pairs.
Revised: The CPTs in SBNs are, in general, associated with all possible parent-child subsplit pairs.

Original: Therefore,  a full parameterization requires a combinatorially increasing number of parameters.
Revised: Therefore, in principle a full parameterization requires an exponentially increasing number of parameters.

Original: In practice, however, we can ﬁnd a sufﬁciently large subsplit support of CPTs (i.e. where the associated conditional probabilities are allowed to be nonzero) that covers favorable subsplit pairs from trees in the high-probability areas of the true posterior.
Revised: In practice, however, we can ﬁnd a sufﬁciently large subsplit support of CPTs (i.e. where the associated conditional probabilities are allowed to be nonzero) that covers favorable subsplit pairs from trees in the high-probability areas of the true posterior.

Original: Compared to estimating the true posterior probability of the trees that often involves an expensive long MCMC run, locating the high-probability areas or ﬁnding favorable subsplit pairs could be much cheaper and can be done via either initial MCMC runs, fast bootstrap methods or heuristic search based on compatibility or a distance measure induced by the input sequences.

Original: In this paper, we will mostly focus on the variational approach and assume the support of CPTs is available, although in our experiments we ﬁnd that a simple bootstrap-based approach does provide a reasonable CPT support estimate for real data.
Revised: In this paper, we will mostly focus on the variational approach and assume the support of CPTs is available, although in our experiments we ﬁnd that a simple bootstrap-based approach does provide a reasonable CPT support estimate for real data.

Original: We leave the development of more sophisticated methods for ﬁnding the support of CPTs to future work.
Revised: We leave the development of more sophisticated methods for ﬁnding the support of CPTs to future work.

Original: Now denote the set of root subsplits in the support as S r and the set of parent-child subsplit pairs in the support as S ch | pa .
Revised: Now denote the set of root subsplits in the support as S r and the set of parent-child subsplit pairs in the support as S ch | pa .

Original: The CPTs are deﬁned according to the following equations
Revised: The CPTs are deﬁned according to the following equations

Original: [Equation]
Revised: [Equation]

Original: where S ·| t denotes the set of child subsplits for parent subsplit t .
Revised: where S ·| t denotes the set of child subsplits for parent subsplit t .

Original: We use the Log-normal distribution Lognormal( µ, σ 2 ) as our variational approximation for branch lengths to accommodate their non-negative nature in phylogenetic models.
Revised: We use the Log-normal distribution Lognormal( µ, σ 2 ) as our variational approximation for branch lengths to accommodate their non-negative nature in phylogenetic models.

Original: Instead of a naive parameterization for each edge on each tree (which would require a large number of parameters when the high-probability areas of the posterior are diffuse), we use a split -based structured parameterization that exploits the similarity of  local structures among trees to reduce the number of parameters.
Revised: Instead of a naive parameterization for each edge on each tree (which would require a large number of parameters when the high-probability areas of the posterior are diffuse), we use an amortized set of parameters over the shared local structures among trees.

Original: A split is  a bipartition ( X 1 , X 2 ) of the leaf labels X (i.e. X 1 ∪ X 2 ={+
+}
Revised: A simple choice of such local structures is the split, a bipartition ( X 1 , X 2 ) of the leaf labels X (i.e. X 1 ∪ X {+
+}

Original:  X , X 1 ∩ X 2 = ∅ ) and each edgee of a phylogenetic tree τ represents a split, the bipartition that consists of the leaf labels from both side of the edge.
Revised: = X , X 1 ∩ X 2 = ∅ ), and each edge of a phylogenetic tree naturally corresponds to a split, the bipartition that consists of the leaf labels from both sides of the edge.

Original: Note that a split can be viewed as a root subsplit.
Revised: Note that a split can be viewed as a root subsplit.

Original: We then assign µ ( · , · ) , σ ( · , · ) for each split ( · , · ) in S r .
Revised: We then assign µ ( · , · ) , σ ( · , · ) for each split ( · , · ) in S r .

Original: We denote the corresponding split of edge e of tree τ as e/τ .
Revised: We denote the corresponding split of edge e of tree τ as e/τ .

Original: A Simple Independent Approximation Given a phylogenetic tree τ , we use a simple model that assumes the branch lengths for the edges of the tree are independently distributed.
Revised: A Simple Independent Approximation Given a phylogenetic tree τ , we start with a simple model that assumes the branch lengths for the edges of the tree are independently distributed.

Original: The approximate density Q ψ ( q | τ ) , therefore, has the form
Revised: The approximate density Q ψ ( q | τ ) , therefore, has the form

Original: [Equation]
Revised: [Equation]

Original: where p Lognormal ( ·| µ, σ ) is the probability density function of Lognormal( µ, σ 2 ) .
Revised: e ψ µ ( W,Z ) W W Z

Revised: Z

Original: [Figure]
Revised: W ψ µ ( W 1 , W 2 | W , Z ) Z ψ µ ( Z 1 , Z 2 | W , Z ) + µ ( e,τ )

Original: Capturing Between-Tree Branch Length Variation
Revised: Capturing Between-Tree Branch Length Variation

Original: The above approximation equation 4 implicitly assumes that the branch lengths in different trees have the same distribution if they correspond to the same split, which fails to account for between-tree variation.
Revised: The above approximation equation 4 implicitly assumes that the branch lengths in different trees have the same distribution if they correspond to the same split, which fails to account for between-tree variation.

Original: To capture the between-tree variation, one can use a more sophisticated parameterization that allows other tree-dependent terms for the variational parameters µ and σ .
Revised: To capture this variation, one can use a more sophisticated parameterization that allows other tree-dependent terms for the variational parameters µ and σ .

Original: Speciﬁcally, we use additional local structure associated with each edge as follows:
Revised: Speciﬁcally, we use additional local structure associated with each edge as follows:

Original: Deﬁnition 1 (primary subsplit pair)
Revised: Deﬁnition 1 (primary subsplit pair)

Original: Let e be an edge of a phylogenetic tree τ which represents a split e/
Revised: Let e be an edge of a phylogenetic tree τ which represents a split e/

Original: τ =
Revised: τ =

Original: ( W, Z ) .
Revised: ( W, Z ) .

Original: Assume that at least one of W or Z , say W , contains more than one leaf label and denote its subsplit as ( W 1 , W 2 ) .
Revised: Assume that at least one of W or Z , say W , contains more than one leaf label and denote its subsplit as ( W 1 , W 2 ) .

Original: We call the parent-child subsplit pair ( W 1 , W 2 ) | ( W, Z ) a primary subsplit pair .
Revised: We call the parent-child subsplit pair ( W 1 , W 2 ) | ( W, Z ) a primary subsplit pair .

Original: We assign additional parameters for each primary subsplit pair.
Revised: We assign additional parameters for each primary subsplit pair.

Original: Denoting the primary subsplit pair(s) of edge e in tree τ as e//τ , we then simply sum all variational parameters associated with e to form the mean and variance parameters for the corresponding branch length (Figure 2):
Revised: Denoting the primary subsplit pair(s) of edge e in tree τ as e//τ , we then simply sum all variational parameters associated with e to form the mean and variance parameters for the corresponding branch length (Figure 2):

Original: [Equation]
Revised: [Equation]

Original: This modiﬁes the density in equation 4 by adding contributions from primary subsplit pairs and hence allows for more ﬂexible between-tree approximations.
Revised: This modiﬁes the density in equation 4 by adding contributions from primary subsplit pairs and hence allows for more ﬂexible between-tree approximations.

Original: Note that the above structured parameterizations of branch length distributions also enable joint learning across tree topologies.
Revised: Note that the above structured parameterizations of branch length distributions also enable joint learning across tree topologies.

Original: 3. S TOCHASTIC G RADIENT E STIMATORS AND T HE VBPI A LGORITHM
Revised: 3. S TOCHASTIC G RADIENT E STIMATORS AND T HE VBPI A LGORITHM

Original: In practice, the lower bound is usually maximized via stochastic gradient ascent (SGA).
Revised: In practice, the lower bound is usually maximized via stochastic gradient ascent (SGA).

Original: However, the naive stochastic gradient estimator obtained by differentiating the lower bound has very large variance and is impractical for our purpose.
Revised: However, the naive stochastic gradient estimator obtained by differentiating the lower bound has very large variance and is impractical for our purpose.

Original: Fortunately, various variance reduction techniques have been introduced in recent years including the control variate (Paisley et al., 2012; Ranganath et al., 2014; Mnih & Gregor, 2014; Mnih & Rezende, 2016) for general latent variables and the reparameterization trick (Kingma & Welling, 2014) for continuous latent variables.
Revised: Fortunately, various variance reduction techniques have been introduced in recent years including the control variate (Paisley et al., 2012; Ranganath et al., 2014; Mnih & Gregor, 2014; Mnih & Rezende, 2016) for general latent variables and the reparameterization trick (Kingma & Welling, 2014) for continuous latent variables.

Original: In the following, we apply these techniques to different components of our latent variables and derive efﬁcient gradient estimators with much lower variance, respectively.
Revised: In the following, we apply these techniques to different components of our latent variables and derive efﬁcient gradient estimators with much lower variance, respectively.

Original: In addition, we also consider a stable gradient estimator based on an alternative variational objective.
Revised: In addition, we also consider a stable gradient estimator based on an alternative variational objective.

Revised: See Appendix A for derivations.

Original: The VIMCO Estimator Let f φ , ψ ( τ, q ) =
Revised: The VIMCO Estimator Let f φ , ψ ( τ, q ) =

Original: p ( Y | τ, q ) p ( τ, q ) Q φ ( τ ) Q ψ ( q | τ ) .
Revised: p ( Y | τ, q ) p ( τ, q ) Q φ ( τ ) Q ψ ( q | τ ) .

Original: The stochastic lower bound with K samples is ˆ L K ( φ , ψ ) = log (cid:16) 1 K (cid:80) 
Revised: The stochastic lower bound with K samples is ˆ L K ( φ , ψ ) = log [Equation] 1 K (cid:80) K i =

Original: Ki =1 f φ , ψ ( τ i , q i ) (cid:17) .
Revised: 1 f φ , ψ ( τ i , q i ) (cid:17) .

Original: Mnih & Rezende (2016) propose a localized learning signal strategy that signiﬁcantly reduces the variance of the naive gradient estimator by utilizing the independence between the multiple samples and the regularity of the learning signal, which estimates the gradient as follows(derivation in Appendix A)
Revised: Mnih & Rezende (2016) propose a localized learning signal strategy that signiﬁcantly reduces the variance of the naive gradient estimator by utilizing the independence between the multiple samples and the regularity of the learning signal, which estimates the gradient as follows

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Revised: [Equation]

Revised: 

Original: [Equation]
Revised: [Equation]

Original: is the per-sample local learning signal, with ˆ f φ , ψ ( τ − j , q − j ) being some estimate of f φ , ψ ( τ j , q j ) for sample j using the rest of samples (e.g., the geometric mean), and ˜ w j = 
Revised: is the per-sample local learning signal, with ˆ f φ , ψ ( τ − j , q − j ) being some estimate of f φ , ψ ( τ j , q j ) for sample j using the rest of samples (e.g., the geometric mean), and ˜ w j = f [Equation] ( τ j , q j ) [Equation]

Revised: Ki

Original: [Equation]
Revised: [Equation]

Original:  ) is the self-normalized importance weight.
Revised: f [Equation] , ψ ( τ i , q i ) is the self-normalized importance weight.

Original: This gives the following VIMCO estimator
Revised: This gives the following VIMCO estimator

Original: [Equation]
Revised: [Equation]

Original: The Reparameterization Trick Theabove VIMCO estimator also works for the branch length gradient.
Revised: The Reparameterization Trick The VIMCO estimator also works for the branch length gradient.

Original: However, as branch lengths are continuous latent variables, we can usean alternative approach, the reparameterization trick, to estimate the gradient.
Revised: However, as branch lengths are continuous latent variables, we can use the reparameterization trick to estimate the gradient.

Original: Note that the Log-normal distribution has a simple reparameterization: q ∼ Lognormal( µ, σ 2 ) ⇔ q = exp( µ + σ(cid:15) ) , (cid:15) ∼ N (0 , 1) , we can rewrite the lower bound as follows
Revised: Because the Log-normal distribution has a simple reparameterization, q ∼ Lognormal( µ, σ 2 ) ⇔ q = exp( µ + σ(cid:15) ) , (cid:15) ∼ N (0 , 1) , we can rewrite the lower bound:

Original: [Equation]
Revised: [Equation]

Original: where g ψ ( (cid:15) | τ ) =
Revised: where g ψ ( (cid:15) | τ ) =

Original: exp( µ ψ ,τ + σ ψ ,τ (cid:12) (cid:15) ) .
Revised: exp( µ ψ ,τ + σ ψ ,τ (cid:12) (cid:15) ) .

Original: Then the gradient of the lower bound w.r.t.
Revised: Then the gradient of the lower bound w.r.t.

Original: ψ is(see
Revised: ψ is

Original: Appendix A for derivation)

Original: [Equation]
Revised: [Equation]

Original: where ˜ w j =
Revised: where ˜ w j =

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =1 f φ , ψ ( τ i [Equation] ψ (

Original: [Equation]
Revised: [Equation]

Original:  |
Revised: i |

Original: [Equation]
Revised: [Equation]

Original:  )) is the same normalized importance weight as in equation equation 5.
Revised: i )) is the same normalized importance weight as in equation equation 5.

Original: Therefore, we can form the Monte Carlo estimator of the gradient∇ ψ L K ( [Equation] , ψ [Equation]
Revised: Therefore, we can form the Monte Carlo estimator of the gradient

Original: g

Original: [Equation]
Revised: [Equation]

Original: (cid:15)

Original: [Equation]

Original: with τ j iid ∼ Q φ ( τ

Original: [Equation]
Revised: [Equation]

Original: , (cid:15) j iid ∼ p (cid:15) ( (cid:15) ) .

Original: Self-normalized Importance Sampling Estimator
Revised: Self-normalized Importance Sampling Estimator

Original: In addition to the standard variational formulation equation 2, one can reformulate the optimization problem by minimizing the reversed KL divergence, which is equivalent to maximizing the likelihood of the variational approximation[Equation] , where [Equation]
Revised: In addition to the standard variational formulation equation 2, one can reformulate the optimization problem by minimizing the reversed KL divergence, which is equivalent to maximizing the likelihood of the variational approximation

Original: We can use an importance sampling estimator to compute the gradient of the objective ∇ φ ˜

Original: [Equation] ) ∇ [Equation]
Revised: Q φ ∗ , ψ ∗ ( τ

Original: log

Original: [Equation]
Revised: [Equation]

Original: ˜ w j ∇ φ log Q φ ( τ j )with τ j , q j iid ∼ Q φ[Equation] ψ
Revised: q ) , where φ

Original: [Equation]
Revised: [Equation]

Original: τ [Equation] q
Revised: We can use an importance sampling estimator to compute the gradient of the objective ∇

Original: [Equation]
Revised: [Equation]

Original: with the same importance weights ˜ w j as in equation 5 (a detailed derivation can be found in Appendix A).
Revised: with the same importance weights ˜ w j as in equation 5.

Original: This can be viewed as a multi-sample generalization of the wake-sleep algorithm (Hinton et al., 1995) and was ﬁrst used in the reweighted wake-sleep algorithm (Bornschein & Bengio, 2015) for training deep generative models.
Revised: This can be viewed as a multi-sample generalization of the wake-sleep algorithm (Hinton et al., 1995) and was ﬁrst used in the reweighted wake-sleep algorithm (Bornschein & Bengio, 2015) for training deep generative models.

Original: We therefore call the gradient estimator in equation 10 the RWS estimator.
Revised: We therefore call the gradient estimator in equation 10 the RWS estimator.

Original: Like the VIMCO estimator, the RWS estimator also provides gradients for branch lengths.
Revised: Like the VIMCO estimator, the RWS estimator also provides gradients for branch lengths.

Original: However, we ﬁnd in practice that equation 8 that uses the reparameterization trick is more useful and often leads to faster convergence, although it uses a different optimization objective.
Revised: However, we ﬁnd in practice that equation 8 that uses the reparameterization trick is more useful and often leads to faster convergence, although it uses a different optimization objective.

Original: A better understanding of this phenomenon would be an interesting subject of future research.
Revised: A better understanding of this phenomenon would be an interesting subject of future research.

Original: All stochastic gradient estimators introduced above can be used in conjunction with stochastic optimization methods such as SGA or some of its adaptive variants (e.g. Adam (Kingma & Ba, 2015)) to maximize the lower bounds.
Revised: All stochastic gradient estimators introduced above can be used in conjunction with stochastic optimization methods such as SGA or some of its adaptive variants (e.g. Adam Kingma & Ba, 2015) to maximize the lower bounds.

Original: See algorithm 1 in Appendix B for a basic variational Bayesian phylogenetic inference (VBPI) approach.[Figure]
Revised: See algorithm 1 in Appendix B for a basic variational Bayesian phylogenetic inference (VBPI) approach.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Throughout this section we evaluate the effectiveness and efﬁciency of our variational framework for inference over phylogenetic trees.
Revised: Throughout this section we evaluate the effectiveness and efﬁciency of our variational framework for inference over phylogenetic trees.

Original: The simplest SBN (the one with a full and complete binary tree structure) is used for the phylogenetic tree topology variational distribution; we have found it to provide sufﬁciently accurate approximation.
Revised: The simplest SBN (the one with a full and complete binary tree structure) is used for the phylogenetic tree topology variational distribution; we have found it to provide sufﬁciently accurate approximation.

Original: For real datasets [Figure] we estimate the CPT supports from ultrafast maximum likelihood phylogenetic bootstrap trees using UFBoot (Minh et al., 2013)
Revised: For real datasets, we estimate the CPT supports from ultrafast maximum likelihood phylogenetic bootstrap trees using UFBoot (Minh et al., 2013), which is a fast approximate bootstrap method based on efﬁcient heuristics.

Revised: We compare the performance of the VIMCO estimator and the RWS estimator with different variational parameterizations for the branch length distributions, while varying the number of samples in the training objective to

Original: [Figure]
Revised: [Figure]

Original: which is a fast approximate bootstrap method based on efﬁcient heuristics. We compare the performance of the VIMCO estimator and the RWS estimator with different variational parameterizations for the branch length distributions, while varying the number of samples in the training objective to see how these affect the quality of the variational approximations.
Revised: see how these affect the quality of the variational approximations.

Original: For VIMCO, we use Adam for stochastic gradient ascent with learning rate 0 .
Revised: For VIMCO, we use Adam for stochastic gradient ascent with learning rate 0 .

Original: 001 (Kingma & Ba, 2015).
Revised: 001 (Kingma & Ba, 2015).

Original: For RWS, we also use AMSGrad (Sashank et al., 2018), a recent variant of Adam, when Adam is unstable.
Revised: For RWS, we also use AMSGrad (Sashank et al., 2018), a recent variant of Adam, when Adam is unstable.

Original: Results were collected after 200,000 parameter updates.
Revised: Results were collected after 200,000 parameter updates.

Original: The KL divergences reported are over the discrete collection of phylogenetic tree structures, from trained SBN distribution to the ground truth, and a low KL divergence means a high quality approximation of the distribution of trees.
Revised: The KL divergences reported are over the discrete collection of phylogenetic tree structures, from trained SBN distribution to the ground truth, and a low KL divergence means a high quality approximation of the distribution of trees.

Original: 4. S IMULATED S CENARIOS
Revised: 4. S IMULATED S CENARIOS

Original: To empirically investigate the representative power of SBNs to approximate distributions on phylogenetic trees under the variational framework, we ﬁrst conduct experiments on a simulated setup.
Revised: To empirically investigate the representative power of SBNs to approximate distributions on phylogenetic trees under the variational framework, we ﬁrst conduct experiments on a simulated setup.

Original: We use the space of unrooted phylogenetic trees with 8 leaves, which contains 10395 unique trees in total.
Revised: We use the space of unrooted phylogenetic trees with 8 leaves, which contains 10395 unique trees in total.

Original: Given an arbitrary order of trees, we generate a target distribution p 0 ( τ ) by drawing a sample from the symmetric Dirichlet distributions Dir( β 1 ) of order 10395, where β is the concentration parameter.
Revised: Given an arbitrary order of trees, we generate a target distribution p 0 ( τ ) by drawing a sample from the symmetric Dirichlet distributions Dir( β 1 ) of order 10395, where β is the concentration parameter.

Original: The target distribution becomes more diffuse as β increases; we used β = 0 .
Revised: The target distribution becomes more diffuse as β increases; we used β = 0 .

Original: 008 to provide enough information for inference while allowing for adequate diffusion in the target.
Revised: 008 to provide enough information for inference while allowing for adequate diffusion in the target.

Original: Note that there are no branch lengths in this simulated model and the lower bound is
Revised: Note that there are no branch lengths in this simulated model and the lower bound is

Original: [Equation]
Revised: [Equation]

Original: with the exact evidence being log(1) = 0 .
Revised: with the exact evidence being log(1) = 0 .

Original: We then use both the VIMCO and RWS estimators to optimize the above lower bound based on 20 and 50 samples ( K ).
Revised: We then use both the VIMCO and RWS estimators to optimize the above lower bound based on 20 and 50 samples ( K ).

Original: We use a slightly larger learning rate (0.002) in AMSGrad for RWS.
Revised: We use a slightly larger learning rate (0.002) in AMSGrad for RWS.

Original: In addition to the lower bound estimate, KL divergence from the variational approximation to the target distribution is also provided to measure the performance of different methods.

Original: Figure 3 shows the empirical performance of different methods.
Revised: Figure 3 shows the empirical performance of different methods.

Original: From the left plot, we see that the lower bounds converge rapidly and the gaps between lower bounds and the exact model evidence are close to zero, demonstrating the expressive power of SBNs on phylogenetic tree probability estimations.
Revised: From the left plot, we see that the lower bounds converge rapidly and the gaps between lower bounds and the exact model evidence are close to zero, demonstrating the expressive power of SBNs on phylogenetic tree probability estimations.

Original: The evolution of KL divergences (middle plot) is consistent with the lower bounds.
Revised: The evolution of KL divergences (middle plot) is consistent with the lower bounds.

Original: All methods beneﬁt from using more samples, with VIMCO performing better in the end and RWS[Figure] learning slightly faster at the beginning.
Revised: All methods beneﬁt from using more samples, with VIMCO performing better in the end and RWS learning slightly faster at the beginning.

Original: The slower start of VIMCO is partly due to the regularization term in the lower bounds, which turns out to be beneﬁcial for the overall performance since the regularization encourages the diversity of the variational approximation and leads to more sufﬁcient exploration in the starting phase, similar to the exploring starts (ES) strategy in reinforcement learning (Sutton & Barto, 1998).
Revised: The slower start of VIMCO is partly due to the regularization term in the lower bounds, which turns out to be beneﬁcial for the overall performance since the regularization encourages the diversity of the variational approximation and leads to more sufﬁcient exploration in the starting phase, similar to the exploring starts (ES) strategy in reinforcement learning (Sutton & Barto, 1998).

Original: The right plot compares the variational approximations obtained by VIMCO and RWS, both with 50 samples, to the ground truth p 0 ( τ ) .
Revised: The right plot compares the variational approximations obtained by VIMCO and RWS, both with 50 samples, to the ground truth p 0 ( τ ) .

Original: We see that VIMCO slightly underestimates trees in high-probability areas as a result of the regularization effect.
Revised: We see that VIMCO slightly underestimates trees in high-probability areas as a result of the regularization effect.

Original: While RWS provides better approximations for trees in high-probability areas, it tends to underestimate trees  in low-probability areas which deteriorates the overall performance.
Revised: While RWS provides better approximations for trees in high-probability areas, it tends to underestimate trees [Figure] in low-probability areas which deteriorates the overall performance.

Original: We expect the biases in both approaches to be alleviated with more samples.
Revised: We expect the biases in both approaches to be alleviated with more samples.

Original: 4. R EAL D ATA P HYLOGENETIC P OSTERIOR E STIMATION
Revised: 4. R EAL D ATA P HYLOGENETIC P OSTERIOR E STIMATION

Original: In the second set of experiments we evaluate the proposed variational Bayesian phylogenetic inference (VBPI) algorithms at estimating unrooted phylogenetic tree posteriors on 8 real datasets commonly used to benchmark phylogenetic MCMC methods (Lakner et al., 2008; H ¨ ohna & Drummond, 2012; Larget, 2013; Whidden & Matsen IV, 2015)
Revised: In the second set of experiments we evaluate the proposed variational Bayesian phylogenetic inference (VBPI) algorithms at estimating unrooted phylogenetic tree posteriors on 8 real datasets commonly used to benchmark phylogenetic MCMC methods (Lakner et al., 2008; H ¨ ohna & Drummond, 2012; Larget, 2013; Whidden & Matsen IV, 2015)

Original: (Table 1).
Revised: (Table 1).

Original: We concentrate on the most challenging part of the phylogenetic model: joint learning of the tree topologies and the branch lengths.
Revised: We concentrate on the most challenging part of the phylogenetic model: joint learning of the tree topologies and the branch lengths.

Original: We assume a uniform prior on the tree topology, an i.i.d.
Revised: We assume a uniform prior on the tree topology, an i.i.d.

Original: exponential prior ( Exp(10) ) for the branch lengths and the simple Jukes & Cantor (1969) substitution model.
Revised: exponential prior ( Exp(10) ) for the branch lengths and the simple Jukes & Cantor (1969) substitution model.

Original: We consider two different variational parameterizations for the branch length distributions as introduced in section 3.1.
Revised: We consider two different variational parameterizations for the branch length distributions as introduced in section 3.1.

Original: In the ﬁrst case, we use the simple split-based parameterization that assigns parameters to the splits associated with the edges of the trees.
Revised: In the ﬁrst case, we use the simple split-based parameterization that assigns parameters to the splits associated with the edges of the trees.

Original: In the second case, we assign additional parameters for the primary subsplit pairs (PSP) to better capture the between-tree variation.
Revised: In the second case, we assign additional parameters for the primary subsplit pairs (PSP) to better capture the between-tree variation.

Original: We form our ground truth posterior from an extremely long MCMC run of 10 billion iterations (sampled each 1000 iterations with the ﬁrst 25% discarded as burn-in) using MrBayes (Ronquist et al., 2012), and gather the support of CPTs from 10 replicates of 10000 ultrafast maximum likelihood bootstrap trees (Minh et al., 2013).
Revised: We form our ground truth posterior from an extremely long MCMC run of 10 billion iterations (sampled each 1000 iterations with the ﬁrst 25% discarded as burn-in) using MrBayes (Ronquist et al., 2012), and gather the support of CPTs from 10 replicates of 10000 ultrafast maximum likelihood bootstrap trees (Minh et al., 2013).

Original: Following Rezende & Mohamed (2015), we use a simple annealed version of the lower bound which was found to provide better results.
Revised: Following Rezende & Mohamed (2015), we use a simple annealed version of the lower bound which was found to provide better results.

Original: The modiﬁed bound is:
Revised: The modiﬁed bound is:

Original: [Equation]
Revised: [Equation]

Original: where β t ∈ [0 , 1] is an inverse temperature that follows a schedule β t = min(0 .
Revised: where β t ∈ [0 , 1] is an inverse temperature that follows a schedule β t = min(0 .

Original: 001 , t/ 100000) , going from 0 .
Revised: 001 , t/ 100000) , going from 0 .

Original: 001 to 1 after 100000 iterations.
Revised: 001 to 1 after 100000 iterations.

Original: We  train the variational approximations with VIMCO and RWS using 10 and 20 sample objectives and use Adam with learning rate 0 . 001 for both methods.
Revised: We use Adam with learning rate 0 . 001 to train the variational approximations using VIMCO and RWS estimators with 10 and 20 samples.

Original: [Table] Figure 4 (left and middle plots) shows the resulting KL divergence to the ground truth on DS as a function of the number of parameter updates.
Revised: Figure 4 (left and middle plots) shows the resulting KL divergence to the ground truth on DS1 as a function of the number of parameter updates.

Original: The results for methods that adopt the simple split-based parameterization of variational branch length distributions are shown in the left plot.
Revised: The results for methods that adopt the simple splitbased parameterization of variational branch length distributions are shown in the left plot.

Original: We see that the performance of all methods improves signiﬁcantly as the number of samples is increased.
Revised: We see that the performance of all methods improves signiﬁcantly as the number of samples is increased.

Original: As before, RWS learns faster at the beginning while VIMCO performs better in the end.

Original: The middle plot, containing the results using PSP for variational parameterization, clearly indicates that a better modeling of between-tree variation of the branch length distributions is beneﬁcial for all method / number of samples combinations.
Revised: The middle plot, containing the results using PSP for variational parameterization, clearly indicates that a better modeling of between-tree variation of the branch length distributions is beneﬁcial for all method / number of samples combinations.

Original: Speciﬁcally, PSP enables more ﬂexible branch length distributions across trees which makes the learning task much easier, evidenced by the considerably smaller gaps between the methods.
Revised: Speciﬁcally, PSP enables more ﬂexible branch length distributions across trees which makes the learning task much easier, as shown by the considerably smaller gaps between the methods.

Original: To benchmark the learning efﬁciency of VBPI, we also compare to MrBayes 3.2.5 (Ronquist et al., 2012), a standard MCMC implementation.
Revised: To benchmark the learning efﬁciency of VBPI, we also compare to MrBayes 3.2.5 (Ronquist et al., 2012), a standard MCMC implementation.

Original: We run MrBayes with 4 chains and 10 runs for two million iterations, sampling every 100 iterations.
Revised: We run MrBayes with 4 chains and 10 runs for two million iterations, sampling every 100 iterations.

Original: For each run, we compute the KL divergence to the ground truth every 50000 iterations with the ﬁrst 25% discarded as burn-in.
Revised: For each run, we compute the KL divergence to the ground truth every 50000 iterations with the ﬁrst 25% discarded [Table]

Original:  For a relatively fair comparison (in terms of the number of likelihood evaluations), we compare 10 (i.e. 2 · 20/4) times the number of MCMC iterations with the number of 20-sample objective VBPI iterations.
Revised: as burn-in. For a relatively fair comparison (in terms of the number of likelihood evaluations), we compare 10 (i.e. 2 · 20/4) times the number of MCMC iterations with the number of 20-sample objective VBPI iterations.

Original: 2 Although MCMC converges faster at the start, we see that VBPI methods (especially those with PSP) quickly surpass MCMC and arrive at good approximations with much less computation.
Revised: 2 Although MCMC converges faster at the start, we see that VBPI methods (especially those with PSP) quickly surpass MCMC and arrive at good approximations with much less computation.

Original: This is because VBPI iteratively updates the approximate distribution of trees (e.g., SBNs) which in turn allows guided exploration in thephylogenetic tree  space.
Revised: This is because VBPI iteratively updates the approximate distribution of trees (e.g., SBNs) which in turn allows guided exploration in the tree topology space.

Original: As a commonly used summarization of tree samples, we also compare the  majority-rule consensus trees obtained from VBPI and the ground truth MCMC run (Figure 5 in Appendix D), and ﬁnd they are identical.
Revised: VBPI also provides the same majority-rule consensus tree as the ground truth MCMC run (Figure 5 in Appendix D).

Original: The variational approximations provided by VBPI can be readily used to perform importance sampling for phylogenetic inference (see more details in Appendix C).
Revised: The variational approximations provided by VBPI can be readily used to perform importance sampling for phylogenetic inference (more details in Appendix C).

Original: Thanks to the structured variational parameterization for the branch length distributions, VBPI can provide variational approximations on branch lengths for any tree that is covered by the subsplit support. We, therefore, can use them to estimate the marginal likelihood of trees via importance sampling.
Revised: The right plot of Figure 4 compares

Original: The right plot of ﬁgure 4 compares VBPI using VIMCO with 20 samples and PSP to the state-of-the-art generalized stepping-stone (GSS) (Fan et al., 2011) algorithm for estimating the marginal likelihood of trees in the 95% credible set of DS1.
Revised: VBPI using VIMCO with 20 samples and PSP to the state-of-the-art generalized stepping-stone (GSS) (Fan et al., 2011) algorithm for estimating the marginal likelihood of trees in the 95% credible set of DS1.

Original: For GSS, we use 50 power posteriors and for each power posterior we run 1,000,000 MCMC iterations, sampling every 1000 iterations with the ﬁrst 10% discarded as burn-in.
Revised: For GSS, we use 50 power posteriors and for each power posterior we run 1,000, MCMC iterations, sampling every 1000 iterations with the ﬁrst 10% discarded as burn-in.

Original: The reference distribution for GSS was obtained from an independent Gamma approximation using the maximum a posterior estimate.
Revised: The reference distribution for GSS was obtained from an independent Gamma approximation using the maximum a posterior estimate.

Original: We see that the estimates given by VBPI closely match GSS estimates and have small variance in general.

Original: Similarly, we can use VBPI to estimate the marginal likelihood of the data (i.e., the model evidence) via importance sampling.

Original: Table 1 shows the  marginal likelihood estimates using different VIMCO approximations and one of the state-ofthe-art methods, the stepping-stone (SS) algorithm (Xie et al., 2011), which unlike GSS is able to also integrate out tree topologies.
Revised: Table 1 shows the estimates of the marginal likelihood of the data (i.e., model evidence) using different VIMCO approximations and one of the state-of-the-art methods, the stepping-stone (SS) algorithm (Xie et al., 2011).

Original: For each data set, all methods provide estimates for the same marginal likelihood, with better approximation leading to lower variance.
Revised: For each data set, all methods provide estimates for the same marginal likelihood, with better approximation leading to lower variance.

Original: We see that VBPI using 1000 samples is already competitive with SS using 100000 samples and provides estimates with much less variance (hence more reproducible and reliable).
Revised: We see that VBPI using 1000 samples is already competitive with SS using 100000 samples and provides estimates with much less variance (hence more reproducible and reliable).

Original: Again, the extra ﬂexibility enabled by PSP alleviates the demand for larger number of samples used in the training objective, making it possible to achieve high quality variational approximations with less samples.
Revised: Again, the extra ﬂexibility enabled by PSP alleviates the demand for larger number of samples used in the training objective, making it possible to achieve high quality variational approximations with less samples.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: In this work we introduced VBPI, a general variational framework for Bayesian phylogenetic inference.
Revised: In this work we introduced VBPI, a general variational framework for Bayesian phylogenetic inference.

Original: By combining subsplit Bayesian networks, a recent framework that provides ﬂexible distributions of trees, and efﬁcient structured parameterizations for branch length distributions, VBPI exhibits guided exploration (enabled by SBNs) in tree space and provides competitive performance to MCMC methods with less computation.
Revised: By combining subsplit Bayesian networks, a recent framework that provides ﬂexible distributions of trees, and efﬁcient structured parameterizations for branch length distributions, VBPI exhibits guided exploration (enabled by SBNs) in tree space and provides competitive performance to MCMC methods with less computation.

Original: Moreover, variational approximations provided by VBPI can be readily used for further statistical analysis such as marginal likelihood estimation for model comparison via importance sampling, which, compared to MCMC based methods, dramatically reduces the cost at test time.
Revised: Moreover, variational approximations provided by VBPI can be readily used for further statistical analysis such as marginal likelihood estimation for model comparison via importance sampling, which, compared to MCMC based methods, dramatically reduces the cost at test time.

Original: We report promising numerical results demonstrating the effectiveness and efﬁciency of VBPI on a benchmark of real data Bayesian phylogenetic inference problems.
Revised: We report promising numerical results demonstrating the effectiveness and efﬁciency of VBPI on a benchmark of real data Bayesian phylogenetic inference problems.

Original: When the data are weak and posteriors are diffuse, support estimation of CPTs becomes challenging.
Revised: When the data are weak and posteriors are diffuse, support estimation of CPTs becomes challenging.

Original: However, compared to classical MCMC approaches in phylogenetics that need to traverse the enormous support of posteriors on complete trees to accurately evaluate the posterior probabilities, the
Revised: However, compared to classical MCMC approaches in phylogenetics that need to traverse the enormous support of posteriors on complete trees to accurately evaluate the posterior probabilities, the

Original: SBN parameterization in VBPI has a natural advantage in that it alleviates this issue by factorizing the uncertainty of complete tree topologies into local structures.
Revised: SBN parameterization in VBPI has a natural advantage in that it alleviates this issue by factorizing the uncertainty of complete tree topologies into local structures.

Original: Many topics remain for future work: constructing more ﬂexible approximations for the branch length distributions (e.g., using normalizing ﬂow (Rezende & Mohamed, 2015) for within-tree approximation and deep networks for the modeling of between-tree variation), deeper investigation of support estimation approaches in different data regimes, and efﬁcient training algorithms for general variational inference on discrete / structured latent variables.
Revised: Many topics remain for future work: constructing more ﬂexible approximations for the branch length distributions (e g., using normalizing ﬂow (Rezende & Mohamed, 2015) for within-tree approximation and deep networks for the modeling of between-tree variation), deeper investigation of support estimation approaches in different data regimes and efﬁcient training algorithms for general variational inference on discrete / structured latent variables A CKNOWLEDGMENTS

Revised: This work supported by National Science Foundation grant CISE-1564137, as well as National

Revised: Institutes of Health grant R01-GM113246.

Revised: The research of Frederick Matsen was supported in part by a Faculty Scholar grant from the Howard Hughes Medical Institute and the Simons Foundation.

Original: A G RADIENT D ERIVATION FOR T HE M ULTI - SAMPLE O BJECTIVES
Revised: A G RADIENT D ERIVATION FOR T HE M ULTI - SAMPLE O BJECTIVES

Original: In this section we will derive the gradient for the multi-sample objectives introduced in section 3.
Revised: In this section we will derive the gradient for the multi-sample objectives introduced in section 3.

Original: We start with the lower bound
Revised: We start with the lower bound

Original: [Equation]
Revised: [Equation]

Original: Using the product rule and noting that ∇ φ log f φ, ψ ( τ j , q j ) =
Revised: Using the product rule and noting that ∇ φ log f φ

Original: −∇
Revised: [Equation]

Original: φ log Q φ ( τ j) ,
Revised: ψ ( τ j

Original: [Equation]
Revised: [Equation]

Revised: q j )

Original: = [Equation]
Revised: = −∇

Original: φ , ψ ( [Equation] [Equation] K ,
Revised: φ log Q φ ( τ j )

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: 

Original: [Equation]
Revised: [Equation]

Original: This gives the naive gradient of the lower bound w.r.t.
Revised: This gives the naive gradient of the lower bound w.r

Revised: [Equation]

Original:  φ .
Revised: t. φ .

Original: Using the reparameterization trick
Revised: Using the reparameterization trick

Original: [Equation]
Revised: [Equation]

Original: the lower bound has the form
Revised: the lower bound has the form

Original: [Equation]
Revised: [Equation]

Original:  Since ψ is not involved in the distribution with respect to which we take expectation,
Revised:   Since ψ is not involved in the distribution with respect to which we take expectation,

Original: [Equation]
Revised: [Equation]

Original: Next, we derive the gradient of the multi-sample likelihood objective used in RWS
Revised: Next, we derive the gradient of the multi-sample likelihood objective used in RWS

Original: [Equation]
Revised: [Equation]

Original: Again, p ( τ, q | Y ) is independent of φ , ψ , and we have
Revised: Again, p ( τ, q | Y ) is independent of φ , ψ , and we have

Original: [Equation]
Revised: [Equation]

Original: The second to last step uses self-normalized importance sampling with K samples.
Revised: The second to last step uses self-normalized importance sampling with K samples.

Original: ∇ ψ ˜ L ( φ , ψ ) can be computed in a similar way.
Revised: ∇ ψ ˜ L ( φ , ψ ) can be computed in a similar way.

Original: B T HE V ARIATIONAL B AYESIAN P HYLOGENETIC I NFERENCE A LGORITHM
Revised: B T HE V ARIATIONAL B AYESIAN P HYLOGENETIC I NFERENCE A LGORITHM

Revised: Algorithm

Original: 1: φ , ψ ← Initialize parameters 2: while not converged do 3: τ 1 , . .
Revised: 1: φ , ψ ← Initialize parameters 2: while not converged do 3: τ 1 , . .

Original: , τ K ← Random samples from the current approximating tree distribution Q φ ( τ ) 4: (cid:15) 1 , . .
Revised: , τ K ← Random samples from the current approximating tree distribution Q φ ( τ ) 4: (cid:15) 1 , . .

Original: , (cid:15) K ← Random samples from the multivariate standard normal distribution N ( 0 , I ) 5: g ← ∇ φ , ψ L K ( φ , ψ ; τ 1: K , (cid:15) 1: K ) (Use any gradient estimator from section 3.2) 6: φ , ψ ← Update parameters using gradients g
Revised: , (cid:15) K ← Random samples from the multivariate standard normal distribution N ( 0 , I ) 5: g ← ∇ φ , ψ L K ( φ , ψ ; τ 1: K , (cid:15) 1: K ) (Use any gradient estimator from section 3.2) 6: φ , ψ ← Update parameters using gradients g

Original: (e.g. SGA) 
Revised: (e.g. SGA) 7: end while 8: return φ , ψ

Original: C I MPORTANCE S AMPLING FOR P HYLOGENETIC INFERENCE VIA V ARIATIONAL A PPROXIMATIONS
Revised: C I MPORTANCE S AMPLING FOR P HYLOGENETIC INFERENCE VIA V ARIATIONAL A PPROXIMATIONS

Original: In this section, we provide a detailed importance sampling procedure for marginal likelihood estimation for phylogenetic inference based on the variational approximations provided by VBPI.
Revised: In this section, we provide a detailed importance sampling procedure for marginal likelihood estimation for phylogenetic inference based on the variational approximations provided by VBPI.

Original: C. E STIMATING M ARGINAL L IKELIHOOD OF T REES
Revised: C. E STIMATING M ARGINAL L IKELIHOOD OF T REES

Original: For each tree τ that is covered by the subsplit support,
Revised: For each tree τ that is covered by the subsplit support,

Original: [Equation]
Revised: [Equation]

Original: can provide accurate approximation to the posterior of branch lengths on τ , where the mean and variance parameters µ ( e, τ ) , σ ( e, τ ) are gathered from the structured variational parameters ψ as introduced in section 3.1.
Revised: can provide accurate approximation to the posterior of branch lengths on τ , where the mean and variance parameters µ ( e, τ ) , σ ( e, τ ) are gathered from the structured variational parameters ψ as introduced in section 3.1.

Original: Therefore, we can estimate the marginal likelihood of τ using importance sampling with Q ψ ( q | τ ) being the importance distribution as follows[Equation] M
Revised: Therefore, we can estimate the marginal likelihood of τ using importance sampling with Q ψ ( q | τ ) being the importance distribution as follows

Original: [Equation]
Revised: [Equation]

Original: Q [Equation] q 
Revised: with q j iid ∼ Q

Original: [Equation]
Revised: [Equation]

Original:  | τ
Revised: ( q | τ

Original: [Equation]
Revised: [Equation]

Original: with q j iid ∼ Q ψ ( q | τ )

Original: C. E STIMATING M ODEL E VIDENCE
Revised: C. E STIMATING M ODEL E VIDENCE

Original: Similarly, we can estimate the marginal likelihood of the data as follows
Revised: Similarly, we can estimate the marginal likelihood of the data as follows

Original: [Equation]
Revised: [Equation]

Original: with τ j , q j iid ∼ Q [Equation] ,  q )
Revised: with τ j , q j iid ∼ Q φ , ψ ( τ, q )

Original: [Equation]
Revised: [Equation]

Original: In our experiments, we use K = 1000 .
Revised: In our experiments, we use K = 1000 .

Original: When taking a log transformation, the above Monte Carlo estimate is no longer unbiased (for the evidence log p ( Y ) ).
Revised: When taking a log transformation, the above Monte Carlo estimate is no longer unbiased (for the evidence log p ( Y ) ).

Original: Instead, it can be viewed as one sample
Revised: Instead, it can be viewed as one sample

Original: Monte Carlo estimate of the lower bound
Revised: Monte Carlo estimate of the lower bound

Original: [Equation]
Revised: [Equation]

Original: whose tightness improves as the number of samples K increases.
Revised: whose tightness improves as the number of samples K increases.

Original: Therefore, with a sufﬁciently large
Revised: Therefore, with a sufﬁciently large

Original: K , we can use the lower bound estimate as a proxy for Bayesian model selection.
Revised: K , we can use the lower bound estimate as a proxy for Bayesian model selection.

Original: D C ONSENSUS T REE C OMPARISON ON DS 
Revised: D C ONSENSUS T REE C OMPARISON ON DS

Original: Alligator mississippiensis
Revised: Alligator mississippiensis

Original: Trachemys scripta
Revised: Trachemys scripta

Original: Ambystoma mexicanum
Revised: Ambystoma mexicanum

Original: Siren intermedia
Revised: Siren intermedia

Original: Typhlonectes natans
Revised: Typhlonectes natans

Original: Amphiuma tridactylum
Revised: Amphiuma tridactylum

Original: Grandisonia alternans
Revised: Grandisonia alternans

Original: Hypogeophis rostratus
Revised: Hypogeophis rostratus

Original: Ichthyophis bannanicus
Revised: Ichthyophis bannanicus

Original: Plethodon yonhalossee
Revised: Plethodon yonhalossee

Original: Scaphiopus holbrooki
Revised: Scaphiopus holbrooki

Original: Discoglossus pictus
Revised: Discoglossus pictus

Original: Bufo valliceps
Revised: Bufo valliceps

Original: Hyla cinerea
Revised: Hyla cinerea

Original: Eleutherodactylus cuneatus
Revised: Eleutherodactylus cuneatus

Original: Nesomantis thomasseti
Revised: Nesomantis thomasseti

Original: Gastrophryne carolinensis
Revised: Gastrophryne carolinensis

Original: Xenopus laevis
Revised: Xenopus laevis

Original: Latimeria chalumnae
Revised: Latimeria chalumnae

Original: Homo sapiens
Revised: Homo sapiens

Original: Mus musculus
Revised: Mus musculus

Original: Rattus norvegicus
Revised: Rattus norvegicus

Original: Oryctolagus cuniculus
Revised: Oryctolagus cuniculus

Original: Gallus gallus
Revised: Gallus gallus

Original: Turdus migratorius
Revised: Turdus migratorius

Original: Heterodon platyrhinos
Revised: Heterodon platyrhinos

Original: Sceloporus undulatus
Revised: Sceloporus undulatus

Original: Alligator mississippiensis
Revised: Alligator mississippiensis

Original: Trachemys scripta
Revised: Trachemys scripta

Original: Sceloporus undulatus
Revised: Sceloporus undulatus

Original: Heterodon platyrhinos
Revised: Heterodon platyrhinos

Original: Turdus migratorius
Revised: Turdus migratorius

Original: Gallus gallus
Revised: Gallus gallus

Original: Oryctolagus cuniculus
Revised: Oryctolagus cuniculus

Original: Homo sapiens
Revised: Homo sapiens

Original: Rattus norvegicus
Revised: Rattus norvegicus

Original: Mus musculus
Revised: Mus musculus

Original: Latimeria chalumnae
Revised: Latimeria chalumnae

Original: Xenopus laevis
Revised: Xenopus laevis

Original: Hyla cinerea
Revised: Hyla cinerea

Original: Bufo valliceps
Revised: Bufo valliceps

Original: Gastrophryne carolinensis
Revised: Gastrophryne carolinensis

Original: Nesomantis thomasseti
Revised: Nesomantis thomasseti

Original: Eleutherodactylus cuneatus
Revised: Eleutherodactylus cuneatus

Original: Typhlonectes natans
Revised: Typhlonectes natans

Original: Siren intermedia
Revised: Siren intermedia

Original: Ambystoma mexicanum
Revised: Ambystoma mexicanum

Original: Discoglossus pictus
Revised: Discoglossus pictus

Original: Ichthyophis bannanicus
Revised: Ichthyophis bannanicus

Original: Amphiuma tridactylum
Revised: Amphiuma tridactylum

Original: Hypogeophis rostratus
Revised: Hypogeophis rostratus

Original: Grandisonia alternans
Revised: Grandisonia alternans

Original: Scaphiopus holbrooki
Revised: Scaphiopus holbrooki

Original: Plethodon yonhalossee
Revised: Plethodon yonhalossee