PhyloGen: Language Model-Enhanced Phylogenetic Inference via Graph Structure Generation
Authors: ChenRui Duan
Zelin Zang
Siyuan Li
Yongjie Xu
Stan Z. Li
Venue: NeurIPS 2024 poster (2024)
Keywords: Phylogenetic Inference
Genome Language Model
Transformer
Graph Structure Generation
DNA
Large Language Models
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Phylogenetic trees elucidate evolutionary relationships among species, but phylogenetic inference remains challenging due to the complexity of combining continuous (branch lengths) and discrete parameters (tree topology). 
    Traditional Markov Chain Monte Carlo methods face slow convergence and computational burdens. Existing Variational Inference methods, which require pre-generated topologies and typically treat tree structures and branch lengths independently, may overlook critical sequence features, limiting their accuracy and flexibility.
    We propose PhyloGen, a novel method leveraging a pre-trained genomic language model to generate and optimize phylogenetic trees without dependence on evolutionary models or aligned sequence constraints. PhyloGen views phylogenetic inference as a conditionally constrained tree structure generation problem, jointly optimizing tree topology and branch lengths through three core modules: (i) Feature Extraction, (ii) PhyloTree Construction, and (iii) PhyloTree Structure Modeling. 
    Meanwhile, we introduce a Scoring Function to guide the model towards a more stable gradient descent.
    We demonstrate the effectiveness and robustness of PhyloGen on eight real-world benchmark datasets. Visualization results confirm PhyloGen provides deeper insights into phylogenetic relationships.

PhyloGen: Language Model-Enhanced Phylogenetic Inference via Graph Structure Generation
Chenrui Duan 1.2+ Zelin Zang2* Siyuan Li Yongjie Xu Stan Z. Li
'Zhejiang University, College of Computer Science and Technology; Westlake University
duanchenrui @westlake edu. cn;
Izangzelin; lisiyuan; xuyongj ie; stan 2q .1 @westlake edu. "Equal contribution
'Corresponding author
Abstract
Phylogenetic trees elucidate evolutionary relationships among species, but phyloge- netic inference remains challenging due to the complexity of combining continuous (branch lengths) and discrete parameters (tree topology). Traditional Markov Chain Monte Carlo methods face slow convergence and computational burdens. Existing Variational Inference methods, which require pre- pre- generated topologies and typi- cally treat tree structures and branch lengths independently, may overlook critical sequence features, limiting their accuracy and flexibility. We propose PhyloGen, novel method leveraging a pre-trained genomic language model to generate and optimize phylogenetic trees without dependence evolutionary models aligned sequence constraints. PhyloGen views phylogenetic inference as a conditionally constrained tree structure generation problem, jointly optimizing tree topology and branch lengths through three core modules: (i) Feature Extraction, (ii) Phy- loTree Construction, and (iii) PhyloTree Structure Modeling. Meanwhile, we introduce a Scoring Function to guide the model towards a more stable gradient descent. We demonstrate the effectiveness and robustness of PhyloGen on eight real-world benchmark datasets. Visualization results confirm PhyloGen provides deeper insights into phylogenetic relationships.
Introduction
Phylogenetic trees 42 (or evolutionary trees) are tree-structured graphs representing kinship rela- tionships between species [II] where each leaf node represents distinct species and internal nodes represent evolutionary bifurcations. The tree's topology reflects evolutionary relationships based on their genetic characteristics, and branch lengths indicate evolutionary distances. Phylogenetics study is the foundation of evolutionary synthetic biology and is critical for tracking the evolutionary trajec- tories of species, analyzing the transmission pathways of newly discovered viruses, and providing meaningful insights for clinical applications 25
13
33
Background. Despite DNA sequencing technologies relationships based molecular properties (DNA, RNA, and proteins), phylogenetic inference remains challenging due to its complex parameter space, which encompasses both continuous (branch lengths) and discrete (tree topology) components. Traditional MCMC-based methods like MrBayes [29] and RevBayes [9, despite their ability to extensively explore the huge tree space, are hindered by slow convergence rates. Additionally, the number of possible tree topologies for species grows factorially as (2n 55!! for n 2 3, posing huge computational challenges. In contrast, VI-based methods leverage approximate inference to provide more efficient estimations. Depending data type and research objectives, these methods can be further categorized into tree representation learning and tree structure generation. Tree Representation Learning, shown in Fig. aa).
32 allowing us construct evolutionary
the
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Seq.
Raw
MEGA
Raw
MEGA
Raw Seq.
f
Not Need Align
Rely on Pre-generates Candidate Tree
MCMC Method
Independen peneendent Modules
Piey (Go-Modules)
Feature Extraction Module
(Heavy Computation) Y
Topological
Blen
Prediction
Prediction
Tree Construction Module
GNN/SBN-based Sampling Module
(GNNISBN)
(JOIGTR)
Structure Module
(No Blen)
Estimation
(with Bien)
Cemeraled Tree
(with Blen)
Generated Tree
Tree
(8) Tree Representation Learning
(b) Existing Tree Structure Generation
(c) PhyloGen (Ours)
Figure I: Comparison of Phylo Tree Inference Methods. (a) The inputs are aligned sequences, and topologies are learned from existing tree structures using methods like SBNs, which rely on MCMC-based methods for pre-generated candidate trees without considering branch lengths directly, (b) The inputs are aligned sequences, and then tree structures and branch lengths are directly inferred by variational inference and biological modules. These methods optimize tree topology and branch lengths separately. (c) The inputs raw sequences processed by pre-trained language model to generate species representations. Then, an initial topology is generated through a tree construction module, and the topology and branch lengths co-optimized by the tree structure modeling module.
intends to learn topological representations from existing tree structures. For instance, Subsplit Bayesian Networks (SBNs) 45) focus on probabilistic representations from given tree structures without considering branch lengths. Variational Bayesian Phylogenetic Inference (VBPI) [46) and its
extensions VBPI-GNN 44 utilize the topological probabilities provided by SBNs and jointly model tree structures in latent space by deriving branch lengths through variational approximations. However, these methods require pre-gent rreggenerated topology, based on which better topological representations are then learned. Tree Structure Generation, as shown in Fig. [b), aim to infer tree structures directly from sequences. PhyloGFN [48] integrates VI and reinforcement learning to construct tree topologies, simplifying branch lengths into discrete intervals, thus necessitating posterior data for inference. VaiPhy [16] introduces the SLANTIS sampling strategy and basic biological models (e.g.. Jukes-Cand Cantor(JC) model (24)) for topology and branch length estimation. ARTree [36] builds tree topologies recursively using a graph autoregressive model. GeoPhy (22 models tree topologies in a continuous geometric space. However, these methods require input sequences to be aligned to equal lengths, and tree topology and branch lengths are optimized separately. They also ignore sequence features that are critical for accurate phylogenetic analysis.
Our Method. As depicted in Fig. [c). we propose novel approach based on a pre-trained genome language model. Our model does not rely on evolutionary models or the requirement to align input sequences equal lengths and fully exploits the prior knowledge embedded in biological sequences. PhyloGen models phylogenetic tree inference as problem, aiming generate and optimize the tree topology and branch lengths jointly. We map species sequences into continuous geometric space and perform end-to-end variational inference without restricting topological candidates. To ensure the oppology-invariane of phylogenetic trees, we incorporate distance constraints in the latent space to maintain translational rotation invariance. Our approach demonstrates effectiveness and efficiency on the eight real-world benchmark datasets and verifies its robustness through data augmentation and noise addition [T]. In addition, we propose a new scoring function to guide the model towards a more stable and faster gradient descent.
tree structure generation
Related Works
MCMC-based methods, such as MrBayes 22 and RevBayes 9,, have been widely used for phylogenetic inference due their ability explore vast tree spaces.
VI-based methods offer more efficient alternative to MCMC by leveraging approximate inference techniques. These methods can be categorized into two main approaches: structure representation and structure generation. Tree Representation Learning. This approach focuses on extracting information from existing tree structures. SBNs [5 capture the relationships between existing

---

L
Scoring Fundtion
C1. Topology Learning
Rx(=)r(=))
Qs(r(e)1=)
Rew Seq.
Qa(=)
Enc
MLP
E
Pre-tratned
+
Bußuer Joqueee
C2. Blens Learning Qs r((2)
()
Tapo
Feature Extraction
PhyloTree Construction
PUs cxp(A)
melocular sequence
genamic ambedding
Dual-Pass Tra.
DGCNN
Blens Reparameterization
Iopological embedding
refined lopological embedding
C. PhyloTree Structure Modeling
Framework of PhyloGen. A. Feature Extraction module extracts genome embeddings E from raw sequences Y using a pre-trained language model. Phylo'Tree Construction module uses E to compute topological parameters, which generate an initial tree structure r via the Neighbor- Joining algorithm. C. PhyloTrree Structure Modeling module jointly model and B, through the topology learning component (TreeEncoder R and TreeDecoder Q) and the branch length (Blens) learning component (dual-pass traversal, DGCNN network, Blens reparameterization).
Figure 2:
subsplits without addressing branch lengths. VBPI 46 and its variants VBPI-NF f and VBPI- GNN [44] These methods introduce a two-pass approach to learn node representations, including branch lengths. VBPI employs variational approximations handle branch lengths, allowing for joint modeling of the tree's latent space. Tree Structure Generation. This approach aims to infer tree structures directly from sequence data. PhyloGFN [48] utilizes GFlowNet [10 20 211, a combination of VI and reinforcement learning, which requiring posterior data for accurate inference. VaiPhy []6] incorporates a sampling strategy and basic biological models to estimate topology and branch lengths. ARTree (36] employs graph autoregressive model to build detailed tree topologies. GeoPhy [22] models tree topologies within continuous geometric space, offering a different approach to the distribution of tree topologies. For details related work, please Appendix B
Methods
The phylogenetic tree is denoted as (r, B,), where is an unrooted binary tree topology reflecting evolutionary relationships among N species. B, denotes the non-negative evolutionary distances of each branch. The tree consists of N leaf nodes, each corresponding to a species, and several internal nodes. PhyloGen aims to generate tree topology and branch lengths directly from the raw sequences.
Notation.
Framework. We model the phylogenetic tree inference problem a tree structure generation task under conditional constraints, consisting of three main modules as shown in Fig. 2 Feature Extraction module extracts genome embeddings from raw sequences via pre-trained language model. PhyloTree Construction module uses these embeddings to generate an initial tree structure using tree construction algorithm, introducing the latent variable 2" represent the topology. PhyloTree Structural Modeling module iteratively refines the tree structure and branch lengths through topology learning and branch length learning components. By integrating these modules, we transform the complex dynamics of evolutionary history into tree-based learning framework, facilitating deeper understanding of phylogenetic relationships.
A. Feature Extraction We utilize DNABERT2 ular (DNA) sequences Y = {3: 7. into genomic embeddings E = [e These embeddings discern complex patterns and long-range dependencies, serving as the basis for generating the latent variables =", which dynamically inform both the topology and the branch lengths. By introducing DNABERT2, we redefine the construction of phylogenetic trees continuous optimization problem within a biologically meaningful embedding space.
genome language model, to transform molec-
3

---

B. PhyloTree Construction To construct the phylogenetic tree, genomic embeddings E are input into an MLP network to derive the parameters j and a of the latent space, representing topologiaa embeddings =". This latent space effectively captures the evolutionary relationships that guide the subsequent tree structure generation process. The latent variable is then generated using the reparameterization trick (15): 2" = p a Ô e ~ (0, T). Then, distance matrix D is computed using =": D(i,5) - -
o @ 2ÿ. where $ means an XOR operation reflecting the nucleotide mismatches. This distance matrix is fed into the Neighbor-Joining (NJ) algorithm [31 to generate the initial tree topology r(²*). The tree's topology is structured using a parent-child relationships list [Z R f Pl where P(i] denotes the parent of node i, and [[i] and R(t] denote the left and right child node. This representation highlights the tree's hierarchical nature, optimizing its structure based on the evolutionary patterns in the data.
N
C. PhyloTree Structure Modeling The purpose of the tree structure modeling module is to jointly optimize both the phylogenetic tree's discrete topology and continuous branch lengths.
C.1. Topology Learning. The first component involves a TreeEncoder R()( and TreeDe- coder ((T(2)]z) to learn the tree topology. Concretely, the encoder R receives an initial tree topology r(=") and genomic embeddings F. from DNABERT2 inputs, conditions the latent state to refine topological embeddings in a continuous space. We introduce variational inference to enhance the quality of the tree structure and adapt data uncertainty and complexity by minimizing the Kullback- Leibler (KL) divergence. Additionally, the encoder R acts as regularization mechanism, reducing overfitting and enhancing gradient stability. To refine the topology, the decoder Q samples from the probability distribution parameterized by the encoder's 's latent state. This process optimizes model parameters by minimizing the KL divergence between the true distribution ((r()) (=)) B,) and the variational distribution Q(T(z)) g(((2), B.). To facilitate backpropagation through stochastic nodes, the latent variable is sampled using the reparamcterization trick: 2 = + E. where = ~ (0, I) introduces controlled stochasticity while maintaining differeentiability, ensuring that 2 effectively captures reñiedd topological embeddings.
C.2. Branch Length (Blens) Learning. The second component utilizes the inferred topology and genomic embeddings from DNABERT2 to generate and adjust branch lengths.
Stepl: Node Feature via Linear-Time Dual-Pass Traversal. We utilize a linear time O(n) dual-pass traversal method, combining postorder (bottom-up) and preorder (top-down) strategies, guaranteeing each node is processed only once.
Postorder Traversal (bottom-up) up) aggregates information from leaf nodes toward the root node.
G max(1, K
Σ Cg)
jEch(()
where is the scaling factor for node i, influenced by the node's connectivity, and K 3 due to binary tree properties.
ft = fj +
(2)
where fi incorporates contributions from children nodes fy and its own initial feature e; from pre-trained genome model. Initially, G 0, f: CG.
Preorder Traversal (top-down) propagates information from the root node toward the leaf nodes, enhancing each node's 's feature zn:
I; = Epli] + Six
where Ex] is the feature of node i's parent.
Step2: Feature Enhancement with Dynamic Graph Convolution (DGCNN) We use the edge convolutional layer of DGCNN (35] to enhance node features. This approach captures both local interactions (between neighboring nodes) and global structural dependencies (reflecting the entire tree structure) within the phylogenetic tree. For layer L, inputs {a! E R F and outputs (of-+) E FR Me have feature dimensions F 768 and F 100. The transformations are:
L+1 (GCNN(E
maj = h<<=/,,
where h. IRF x RF + BF is an MLP-shared asymmetric edge function.

---

Topology Invaryynnvve Property. Despite the non- non-uniqueness of 2D coordinates due to infinite possible translational rotations, the distances between species remain constant. We define this as topologiaal invariance: dy = || 2j 1212, where 2; and 2j are coordinates in the hidden space, maintaining accurate topological relationships. The function h(z, incorporates global and local features through: ,,,,,,,)) T,) h F - ).)
Aggregation of Features. The node features aggregated using MAX operation across all edges:
K
L+1
Σ m k=
mii @... @ m/k:
where @ denotes the MAX aggregation function, focusing on the most significant features.
Step3: Reparametrization of Branch Length. Node features x. obtained from edge convolution layers are further fed into the M network, parameterizing mean and log- vagraaie for branch lengths:
nr, log(o²) = (
(1)) Concretely, the M network is defined as:
h (1) (x)), (1) MM ( :j EENOUU (
(5)
where h; are node features processed through an additional MLP1 to capture inter-node interactions. Branch lengths are updated by reparametrization b e + exp(or) * rvs), where rvs ~ (O, f). ensuring differentiability and capturing the probabilistic nature of estimates. Throughout, the prior P(B,) is assumed exponential, while the posterior ((r,). learned via graph neural network, reflects inferred tree topologies and node characteristics.
3.1 Scoring Function
To address the convergence challenges often associated with the ELBO in VAE models, we incorporate a scor- ing function S, implemented via MLP network. This function assesses each leaf node in the latent space = and provides additional gradient information, facilitating more efficient learning and convergence.
ELBO Metric and Scoring Function Steps
-7000
-7100
Smoothed
-7300
7400
7500
ELBO
During training, S and ELBO form joint optimization ob- jective, optimizing gradient directions improve overall performance. [ng.Bcomparrss the convergence behaviors and stability of S and ELBO throughout the training pro- cess. The horizontal axis represents the training steps, and the vertical axis represents the two metric values. The closer the S curve is to the ELBO curve, the more it proves that S can effectively evaluate the model performance and maintain a consistent optimization trend with ELBO. Different configurations of S, including those with Fully Connected layers (w/ FC), with two layers MLP (w/ MLP-2), and with three layers MLP (w/ MLP-3), demonstrate similar trends, closely following the ELBO curve. After an initial period of rapid change, all metrics stabilize and exhibit minor fluctuations, demonstrating robustness in convergence. What's 's more, the number of layers in MLP has less impact performance.
-7600
Scoring Function w/ MLP-2 Scoring Function
Scoring Function
250
500
750
1500 1750 2000 Steps
1000
1250
Figure 3: Scoring Function over Training Steps on DSI. Closer curves mean better.
Comparison of ELBO and
3.2 Learning Objectives
As discussed in Appendix Background A. our primary goal is to maximize the expected marginal likelihood of the observed species sequence Y via max log p( (T))) B)).. The posterior distribu- tion as p(T(z), (2), B, YY) is difficult to infer directly, so we utilize variational inference to approximate it as ((((2), B, JY). The detailed deviation is in Appendix D.2
To minimize the KL divergence between the true prior and the approximate posterior distributions, start from the joint probability distribution: p(Y, 722), B,) = p(Y]r(2), (-))((B,lrr))))d(
where (2), B+) represents the conditional probability of the observed data Y. We assume the tree topology (2) and the branch lengths B, are conditionally independent.
We introduce variational distribution ((((=), B+) q( (B)I((=))g(((()) to approximate the true posterior p(((2), (Y). The ELBO loss initially be written as:
5

---

(()) E, [log PY, ((2), B+)] B2 (T(),, B+)].
(6)
To improve training variance and gradient stability, we introduce a regularization term [(z[T(=*))
and reformulate the ELBO loss as:
(QQ, R) Bo(a) (o(B) log p(Y (()r
(7)
Q(B.Ir(2))Q(2*)
For better performance and reduced variance, we adopt a multi-sample approach [23
K
log p(Y, B TT2 7(==)))(7(22)) T(2**))
Smulti-sample (Q, R) K
Σ k=1
(8)
Q T(=*))(((z*)
where =* and B are samples from the variational distributions (==") and Q (z)+ respectively. K' represents the number of Monte Carlo samples from the variational distribution to compute the empirical average, providing more robust approximation of the ELBO. Typically, K is chosen to be moderate to balance between computational efficiency and estimation accuracy [18]. And the multi-sample ELBO reduces to the standard ELBO formula when K=1.
The total loss Liotal is then defined as:
Litaal - Lmmlti sample (Q. Rt) + L(S) + LKL,
where LKL = (LLL gence, and C(S) E f(z) accounts for the scoring function loss of the embeddings.
- (g (B,),, minimizing the KL diver-
3.3 End-to-End Learning via Stochastic Gradient Descent
We derive the gradients of model parameters as follows:
P(Y,B,l=(2),
V&L = [Bq(==)) V log Qo Qe(=") (0(="))q(8,rr)))ll4
Q(BrIT(=))
(10)
log +(+())))((er((+))))))+))
where H[Qe(=")) is the entropy of Qo(=*). The derivation of the model parameters Φi
P(Y, B, T()))
VoL Vollqo (BB log Q(B T(=)))
(11)
The derivation of the model parameters th:
V = ==o=eRo("") V log (R)((((()"))
(12)
Experiments
In this section, we conduct experiments demonstrate the effectiveness our proposed PhyloGen. We aim to answer seven research questions as follows:
RQ1: How effective is PhyloGen in generating tree structures under the benchmark datasets?
RQ2: How diverse are the tree topologies generated by PhyloGen?
RQ3:
How consistent is the PhyloGen-generaane
tree structure compared to the MrBayes method?
RQ4:
How robust is PhyloGen to species sequences?
RQS: How does each PhyloGen's module affect its performance?
RQ6: What evolutionary relationships between species does PhyloGen learn?
RQ7: How do key hyper-parameters affect PhyloGen's performance?
4.1
Experiment Setup
Tasks and Datasets. We evaluate the performance of PhyloGen the Variational Bayesian Phyloge- netic Inference task with Evidence Lower Bound (ELBO) and Marginal Log Likelihood (MI..) as metrics eight benchmark datasets (see Appendix C

---

Baselines. We compared PhyloGen against three categories of methods: MCMC-based methods like MrBayes and SBN. Structure Representation methods, including VBPI and VBPI-GNN, use pre-generated tree topologies in training that have the potential to achieve high likelihoods, and thus, their training and inference are restricted to a small space of tree topologies and thus are not directly comparable. Structure Generation methods, which are Phylo'Tree Structure Generation tasks that perform approximate Bayesian inference without pre-selecting topologies. For additional experimental details, including training details, baselines, architectures, and hyperparameters, the interested reader is referred to Appendix L
4.2 Performance evaluation across eight benchmark datasets (RQ1)
Table 1: Comparison of the MLL (f) with different approaches in eight benchmark datasets. VBPI and VBPI-GNN use pre-generated tree topologies in training and thus are not directly comparable. Boldface for the highest result, underline for the second highest from traditional methods, and underline for the second highest from tree structure generation methods.
Methods
Dataset
DS1
DS2
DS3
DS4
DSS
DS6
DS7
DS8
#Taxa (N)
27
29
36
41
50
50
59
64
-7108.42
-26367.57
-33735.44 44
-13330.44
-8214.51
-6724.07 07
-37332.76
-8649.88
MCMC-based
MrBayes
(0.18)
(0.48)
(0.50)
(0:54)
(0,28)
(0.86)
(2.42)
(1.75)
-7108.41
-23367771
-33735.09
-13329.94
-8214.62
-6724.37
-37331.97
-660.64
SBN
(0.15)
(0.08)
(0.09)
(0.20)
(0.40)
(0.43)
(0.28)
(0.50)
-7108.42
-26367.72
-33735.10
-13329.94
-8214.61
-6724.34
-37332.03
8650.63
Stracture Representation
VBPT
(0.10)
(0.12)
(0.11)
(0.31)
(0.67)
(0.68)
(0.43)
(0.55)
-7108.41
-26367.73
-33735.12
-13329.94
-8214.64
-6724.37
-37332.04
8650.65
VBPI-GNN
(0.14)
(0.07)
(0.09)
(0.19)
(0,38)
(0.40)
(0.12)
(0.45)
-108.41
-2636771
-37.35.09
-13329.94
$214.59
-6724.37
-37331.95
8650.61
ARTree
(0.19)
(0.07)
(0.09)
(0.17)
(0.34)
(0.46)
(0.27)
(0.48)
Structure Generation
phi-C'MCC
-7290.36
-30568.49 49
-35798.06
-13582.24
-3367.51
-7013.83
NA
9209.18
(7.23)
(31.34)
(6.62)
(35.08)
(8.87)
(16,99)
(18.03)
-7111.555
-26379448
333757.79
-133342.71
-8240.87
-6735.14
-37377,80 86
-663.51
GooPby
(0.07)
(1 (11)60)
(8.07)
(1.61)
(9,80)
(2.64)
(29.48)
(6.85)
-7116.09
-26368.54
-33735.85
-13337.42
-8233.89
-6735.9
-37358.96
-660.48
GeoPhy LOO(3)+
(10.67)
(0.12)
(0.12)
(1.32)
(6.63)
(1.13)
(13.06)
(0.78)
-7108.95
26368.9
.3735.6
11333183
.2215.15
6730.68
337359.96
8654.76
PhyloGEN
(0.06)
(0.28)
(0.35)
(0.19)
(0,20)
(0:54)
(1.14)
(0.19)
-6910.02
-66257.09
.34811.77
.13063..15
-7928.4
63330.2
-6638.42
.811104
Ours
(0.07)
(0.06)
(0.10)
(1.34)
(0.23)
(0.31)
(12.03)
(0.96)
Table 2: Comparison of ELBO (f) on eight datasets. GcoPhy is not provided in the original paper and is tested by us. Boldface for the highest result, underline for the second highest result.
Dataset
DS1
DS2
DS3
DS4
DS3
DS6
DS7
DS8
Methods
#Taxa (N)
27
41
50
50
MCMC-based
SBN
77100.24
-26368.88 88
.3336.222
.1333183
-8217880
6628.66
-37334.85
8655.05
(0.03)
(0.03)
(0.02)
(0.02)
(0.04)
(0.04)
(0.03)
(0.04)
-7116.67
266134.84
-33766.72
-13389.4 36
8220.91
-6769.41
-37882.96
8654,39
GeoPhy
Structure Generation
(1.71)
(0.10)
(0.15)
(3.45)
(2.64)
(6,25)
(1.97)
(0.97)
-7110.09
-26368.78
-33735.25
-13330.27
-8215.34
-6725.33
-37332.54 54
-6611.73
ARTree
(0.04)
(0.07)
(0.08)
(0.05)
(0.04)
(0.06)
(0.13)
(0.05)
-7005.98 (0.06)
-26362.75
-33430.94
.13113.03
-8053.23
-6324.9
-36838.42
-8409.06
Ours
(0.12)
(0.34)
(3.67)
(2.58)
(1.26)
(1.97)
(1.07)
Comparison of ELBO Across Models for DS1
Comparison of MLL Across Mode's for DS1
-6500
-6000
-7000
-7000
Mean ELBO
-8000
Mcan MLL
-8000
-9000
-9000
-10000
Ours ARTive GeoPhly
-10000
ART
Ours
GooPhy
-11000
-11000
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Steps
Steps
Figure 4: Comparison of ELBO and MLL Metrics for DSI Dataset with Different Baselines.
We compare the MLL and ELBO metrics of various phylogenetic inference methods on eight benchmark datasets. Tab. D and Tab. 2 show that methods that utilize Tree Structure Generation have wider applicability than Structure Representation methods, which are restricted to limited pre- generated topologies. Our method PhyloGen outperforms other methods, achieving the highest MLL
7

---

Dals
Normalizea Irequency
Normalized Frequency
Normalized Frequency
0.-
6.3
283
Bipartiioon
Bipartition
Bipartition Rank
Figure 5: Comparative Bipartition Frequency Distribution in Tree Topologies for DSI, DS2, and DS3 datasets. The closer the two curves are, the better, which suggests that our method is highly consistent with the gold standard MrBayes approach.
and ELBO values on all datasets. The left plot of Fig. Iillustrates our model's high stability and rapid convergence in ELBO metrics DSI, which significantly outperforms the competition. ARTree performance improves the later stages but exhibits large fluctuation. GeoPhy performs the worst, with consistently the lowest and more fluctuating ELBO values. The right plot of Fig. Fig.-Ademmm demonstrates our model's advantages in the MLL metric, where it rapidly achieves and maintains high-performance levels. In contrast, ARTree and GeoPhy have lower MLL values, especially GeoPhy, which has the weakest performance throughout.
4.3 Tree Topological Diversity Analysis (RQ2)
To evaluate the topological diversity of trees gen- erated by PhyloGen on DSI, I, we use three metrics:
Table 3: Diversity of tree topologies.
Simpson's 's Diversity Index [5). Top Frequency, and Top 95% Frequency, as detailed Tab.
Statistics
MrBayes
GeolPhy
Ours
Diversity Index (1)
0.36
0.89
Top Frequency
0.27
0.80
0.008
higher Diversity Index, which approaches 1, sug- gests broad diversity among generated tree topolo-
Top 95% Frequency (1)
42
11
149
gies. The lower Top Frequency suggests balanced distribution, preventing single tree structures from being overly dominant. Furthermore, the presence of 149 distinct topologies within the Top 95% Frequency underscores PhyloGen's ability to generate diverse range of topologies.
4.4 Bipartition Frequency Distribution (RQ3)
Fig. 5 shows the bipartition frequency distributions of trees inferred by PhyloGen for datasets DS1, DS2, and DS3. The horizontal axis indicates the ranking of the bipartitions in the tree topology, and the vertical axis indicates the normalized frequency of occurrence of the corresponding bipartitions. The similarity of our method's curves to those of MrBayes underscores its accuracy, demonstrating that PhyloGen consistently captures evolutionary patterns with reliability comparable to the gold standard. This indicates robust validation of PhyloGen's phylogenetic inference capabilities. More detailed information provided in Appendix
Table 4: Model Robustness Assessment. Performance is evaluated by ELBO (1) and MLL (t). (A) represents the absolute difference change after node additions and deletions, with positive values indicating improved performance and negative values indicating decline. Time records the total computation duration.
Metrie
PhyloGEN
GeoPhy
GeoPhy LO0(3)+
Ours w/o KL
Ours w/o S
Ours
serting2sscting!
ELBO
NA
-7721.82 (-100)
-7729.28 (-107)
-6725.49
-6713.01(+15)
-6711.47(+14)
MLL (A)
6705.55 (-93)
7440.38 (-198)
-7599.85 (:357)
-654..51 (+18)
-6547.32 (+20)
-6542.75 (+22)
Time
18h28min
6h16min
15h23min
6h43min
7h42min
6h32min
(LB
NA
-1802.07 (-88)
-11676.29 (-76)
-10678 24(43)
10655.02 (42)
-10674,28
MLL (A)
-25665.76 (-233)
-11763.40 (-131)
-11630.16 (-98)
-10422.12 (+6)
-10654.53 (+12)
-10432.71(+5)
Time
24h15min
12h24min
7h54min
8h6min
6h37min

---

4.5 Robustness Assessment (RQ4)
To assess our model's 's robustness, we test its adaptability to data changes by modifying the number of nodes in the DS1 dataset, which initially contained 27 species sequences. Specifically, we conduct two experiments: Setting I: randomly deleting 4 nodes to simulate the impact of data incompleteness and potential information loss, and Setting 2: randomly adding 4 nodes to simulate an increase in data size. As shown Tab. g our model and its variants exhibit significant stability and adaptability under both cases. Changes in ELBO and MILL metrics are represented by A values, where positive changes indicate improved performance and negative changes indicate decreased performance. Specifically, smaller positive increases after node deletion (Setting 1) and node addition (Setting 2) emphasize the model's s ability to adapt to changes in data structure effectively. In contrast, larger negative decreases highlight challenges when adjusting to increased data complexity. Furthermore, our model exhibits considerable computational efficiency, outperforming baselines in runtime, a critical advantage for handling the complexities and variabilities of bioinformatics datasets.
4.6 Ablation Study (RQ5)
Tab. [ shows the performance of our model compared with the removal of the KL loss and the Scoring Function S, respectively. Ours performs best on ELBO and MLL metrics, with a significant decrease in performance after the removal of the KL loss, suggesting that the KL loss plays a key role in regularizing the model and avoiding overfitting. While removing the S module had a large
Table 5: Ablation Study of PhyloGen.
Methods
ELBO (t)
MLL (t)
Ours
-7005.98
-6910.02
Ours w/o KL
-7017.57
-6917.34
Ours w/o S
-7011.94
-6919.39
impact on MIL,, the ELBO impact was relatively small, indicating that the impact of the S module is more complex and may be related to specific feature extraction functions. Our future work will explore further enhancements to the S module and investigate other regularization techniques to refine the model's performance.
Fig. 6 shows that our method (PGen) achieves the highest MLL value, indicating optimal fit and stability throughout the training process. Adjust- ments to the model structure, particularly without layer normalisation (PGen w/o LN) and reduc- ing the hidden dimensions (Pden-H re- sult in lower MIL,, but convergence remains stable. These results highlight the model's 's sen- sitivity hyperparameters and affirm its robust- ness under different configurations. Replacing our designed distance matrix D with both Euclidean ) and cosine (PGen-Cosine) dis- tance matrices would greatly affect the effective- ness of MLL. These methods do not capture complex evolutionary relationships as effectively as distance matrices derived from the potential space z. The model with one-hot encoding as the leaf node representation (PGen-One- One-Hot)) Hot) has the lowest MLL, which indicates the importance of our feature extraction module.
Co coariisnn of MLL for Different Contiguration: DS1
6050
-7050
-8000
Mcan
-3000
-10000
11000
Pilte
Hot
120000
Stops
Figure 6: Ablation Study on DSI Dataset.
4.7 Case Study of PhyloTree Structure (RQ6)
p01t
Fig. 7 shows a phylogenetic tree constructed from DS1 dataset, where each leaf node represents a specific species, and the text next to the node indicates the species name. The branch lengths reflect the genetic distances, with shorter branches indicating recent evolutionary history and longer branches indicating greater genetic differences. The phylogenetic tree shown in Fig. places Siren intermedia and Trachemys scripta, both aquatic organisms, on ad- jacent branches, reflecting our model could capture their adaptive evolutionary information to aquatic environments. Meanwhile, the reptilian species Heterodon platyrhinos and Trachemys scripta are also on neighbouring branches, suggesting relatively recent com-
pant playhhise boit
Umfaanttppii Jonoor
part
Figure 7: Plot of PhyloTrees.

---

mon ancestor compared to other amphibian and reptilian species. Notably, our method does not require padding the sequences unified length, which effectively reflects actual sequence variation. For detailed species sequence and topology comparison, please refer to Appendix E.6
5 Conclusion
Contributions In this study, we introduced PhyloGen, novel approach leveraging pre-trained genomic language models to enhance phylogenetic tree inference through graph structure generation. By addressing the limitations of traditional MCMC and existing VI methods, PhyloGen jointly optimizes tree topology and branch lengths without relying on evolutionary models or equal-length sequence constraints. PhyloGen views phylogenetic tree inference
conditionally constrained tree structure generation problem, jointly optimizing tree topology and branch lengths through three core modules: (i) Feature Extraction, (ii) PhyloTree Construction, and (iii) PhyloTree Structure Modeling. These modules map species sequences into a continuous geometric space. refine the tree topology and branch lengths, and maintain topological invariance. Our method demonstrated superior performance and robustness across multiple real-world datasets, providing deeper insights into phylogenetic relationships.
Limitations and Future Works While our model demonstrates outstanding performance on stan- dard benchmarks, it may benefit from using more expressive distributions or incorporating prior constraints to better capture complex dependencies and interactions in the latent space. Additionally, although the Neighbor-Joinigg algorithm effective for iterative tree construction, it is computa- tionally intensive. We are exploring efficient data structures and parallel processing techniques to address this bottleneck. Furthermore, our model has primarily been applied to genomic data, and further research is needed to extend its applicability to diverse biological data, such protein and single-cell data.
Acknowledgement
This work was supported by National Science and Technology Major Project (No. 20222D0115101), National Natural Science Foundation of China Project (No. U21A20427), Project (No. WU2022A009) from the Center of Synthetic Biology and Integrated Bioengineering of Westlake University and Integrated Bioengineering of Westlake University and Project (No. WU2023C019) from the Westlake University Industries of the Future Research Funding. We thank the Al Station of Westlake University for the support of GPUs.
References
[1] Y. Chen, L. Wu, and M. Zaki. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. Advances in neural information processing systems, 13:19444119326, 2020.
[2] P. Diaconis. Sequential importance sampling for estimating the number of perfect matchings in bipartite graphs: An ongoing conversation with laci. Building Bridges II: Mathematics of Lászlö Lovász, pages 223-233, 2019.
[3] L. Franceschi, M. Niepert, M. Pontil, and X. He. Learning discrete structures for graph neural networks. In International conference on machine learning, pages 1972-1982, PMLR, 2019.
[4] J. R. Garey, T. J. Near, M. R. Nonnemacher, and S. A. Nadler. Molecular evidence for acanthocephala as subtaxon of rotifera. Journal of Molecular Evolution, 43:287-292, 1996.
[5] F. He and S. Hu. Hubbell's fundamental biodiversity parameter and the simpson diversity index. Ecology Letters, 8(4):386-30,, 2005.
[6] S. B. Hedges, K. D. Moberg, and I.. R. Maxson. Tetrapod phylogeny inferred from 18s and 28s ribosomal ma sequences and a review of the evidence for amniote relationships. Molecular Biology and Evolution, 7(6):607-633, 1990.
10

---

[7] D. A. Henk, A. Weir, and M. Blackwell. Laboulbeniopsis termitarius, an ectoparasite of termites newly recognized as a member of the laboulbeniomycetes. Mycologia, 95(4)5561-564, 2003.
[8] S. Höhna and A. J. Drummond. Guided tree topology proposals for bayesian phylogenetic inference. Systematic biology, 61(1):1-11, 2012.
[9] S. Höhna, M. J. Landis, T. A. Heath, B. Boussau, N. Lartillot, B. R. Moore, J. P. Huelsenbeck, and F. Ronquist. Revbayes: Bayesian phylogenetic inference using graphical models and an interactive model-specification language. Systematic biology, 65(4):726-736, 2016.
[10] J. Hu, N. Malkin, M. Jain, K. E. Everett, A. Graikos, and Y. Bengio. Gflownet-em for learning compositional latent variable models. In International Conference on Machine Learning, pages 13528-13549. PMLR, 2023.
[11] J. P. Huelsenbeck, F. Ronquist, R. Nielsen, and Bollback. Bayesian inference of phylogeny and its impact on evolutionary biology. science, 294(5550):23102233 2001.
[12] W. Jin, Y. Ma, X. Liu, X. Tang, S. Wang. and J. Tang. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference knowledge discovery & data mining, pages 66-74, 2020.
[13] P. Kapli, Z. Yang, and M. J. Telford. Phylogenetic tree building in the genomic age. Nature Reviews Genetics, 21(7):428- 444, 2020.
[14] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6900 2014.
[15] D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint 1r1:iv1312..11111 2013.
[16] H. Koptagel, O. Kviman, H. Melin, N. Safinianaini, and J. Lagergren. Vaiphy: a variational inference based algorithm for phylogeny. Advances in Neural Information Processing Systems, 35:14758-14777 2022.
[17] C. Lakner, P. Van Der Mark, J. P. Huelsenbeck, B. Larget, and F. Ronquist. Efficiency of markov chain monte carlo tree proposals in bayesian phylogenetics. Systematic biology, 57(1):86-103, 2008.
[18] R. Liao, Y. Li, Song, S. Wang, W. Hamilton, D. K. Duvenaud, R. Urtasun, and R. Zemel. Effi cient graph generation with graph recurrent attention networks. Advances in neural information processing systems, 32, 2019.
[19] Y. Liu, Y. Zheng, D. Zhang, H. Chen, H. Peng, and S. Pan. Towards unsupervised deep graph structure learning. In Proceedings of the ACM Web Conference 2022, pages 1392-1403, 2022.
[20] N. Malkin, M. Jain, E. Bengio, C. Sun, and Y. Bengio. Trajectory balance: Improved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955-5967, 2022.
[21] N. Malkin, Lahlou, T. Deleu, X. Ji, E. Hu, K. Everett, D. Zhang, and Y. Bengio. Gflownets and variational inference. arXiv preprint arXiv:2210.00580 2022.
[22] T. Mimori and M. Hamada. Geophy: differentiable phylogenetic inference via geometric gradients of tree topologies. Advances in Neural Information Processing Systems, 36, 2024.
[23] A. Mnih and D. Rezende. Variational inference for monte carlo objectives. In International Conference on Machine Learning, pages 2188-2196. PMLR, 2016.
[24] H. N. Munro. Mammalian protein metabolism, volume 4. Elsevier, 2012.
[25] M. Nei and S. Kumar. Molecular evolution and phylogenetics. Oxford university press, 2000.
[26] M. Newman. Networks. Oxford university press, 2018.

---

[27] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, perforreformacce deep learning library. Advances in neural information processing systems, 32, 2019.
[28] F. P. Preparata and M. I. Shamos. Computational geometry; introduction. Springer Science
& Business Media, 2012.
[29] F. Ronquist, M. Teslenko, P. Van Der Mark, D. L. Ayres, A. Darling, S. Höhna, B. Larget, L. Liu, M. A. Suchard, and J. P. Huelsenbeck. Mrbayes 3.2: efficient bayesian phylogenetic inference and model choice across a large model space. Systematic biology, 61(3):539-542 2012.
[30] A. Y. Rossman, J. M. McKemy, R. A. SerullSchut,heis, and H.-J. Schroers. Molecular studies of the bionectriaceae using large subumit rdna sequences. Mycologia, 93(1):100-110, 2001.
[31] N. Saitou and M. Nei. The neighbor-joining method: method for reconstructing phyloge- netic trees. Molecular biology and evolution, 4(4)) 1987.
[32] J. Shendure, S. Balasubramanian, G. M. Church, W. Gilbert, J. Rogers, J. A. Schloss, and R. H. Waterston. Dna sequencing at 40: past, present and future. Nature, 550(7676):345-333,, 2017.
[33] J. L., Steenwyk, Y. Li, X. Zhou, X.-X. Shen, and A. Rokas. Incongruence in the phylogenomies era. Nature Reviews Genetics, 24(12)834-850, 2023.
[34] S. Tavaré. Some probabilistic and statistical problems on the analysis of dna sequence. Lecture of Mathematics for Life Science, 17:57, 1986.
[35] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and M. Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog), 38(5):1-12, 2019.
[36] T. Xie and C. Zhang, Artree: A deep autoregressive model for phylogenetic inference. Advances in Neural Information Processing Systems, 36, 2024.
[37] W. Xie, P. O. Lewis, Y. Fan, I.. Kuo, and M.-H. Chen. Improving marginal likelihood estimation for bayesian phylogenetic model selection. Systematic biology, 60(2):150-160, 2011.
(38) Z. Yang and A. D. Yoder. Comparison of likelihood and bayesian methods for estimating divergence times using multiple gene loci and calibration points, with application to radiation of cute-looking mouse lemur species. Systematic biology. 52(5):705-716, 2003.
[39] A. D. Yoder and Z. Yang. Divergence dates for malagasy lemurs estimated from multiple gene loci: geological and evolutionary context. Molecular Ecology, 13(4):757-773, 2004.
[40] Z. Zang, S. Li, D. Wu, G. Wang, K. Wang, L. Shang, B. Sun, H. Li, and S. Z. Li. Dlme: Deep local-flatness manifold embedding. In European Conference Computer Vision, pages 576-592. Springer, 2022.
[41] Z. Zang, H. Luo, K. Wang, P. Zhang, F. Wang, S. Z. Li, and Y. You. Diffaug: Enhance unsuper- vised contrastive learning with -nmainnknnwwedge-eee free diffusion- -ased data augmentation. In Forty-first International Conference on Machine Learning.
[42] 7. Zang, Y. Xu, C. Duan, J. Wu, S. 7. Li, and Z. Lei. A review of artificial intelligence based biological-tree construction: Priorities, methods, applications and trends. arXiv preprint arXiv:2410.04811 2024.
[43] C. Zhang. Improved variational bayesian phylogenetic inference with normalizing flows. Advances in neural information processing systems, 33:18760-18771, 2020.
[44] C. Zhang. Learnable topological features for phylogenetic inference via graph neural networks. ArXiv, 2023.
[45] C. Zhang and F. A. Matsen IV. Generalizing tree probability estimation via bayesian networks. Advances neural information processing systems, 31, 2018
[46] C. Zhang and F. A. Matsen IV. Variational bayesian phylogenetic inference. In International Conference Learning Representations, 2018
12

---

[47] N. Zhang and M. Blackwell. Molecular phylogeny of dogwood anthracnose fungus (discula destructiva) and the diaporthales. Mycologia, 93(2):355-365 2001.
[48] M. Zhou, Z. Yan, E. Layne, N. Malkin, D. Zhang, M. Jain, M. Blanchette, and Bengio. Phy- logfn: Phylogenetic inference with generative flow networks. arXiv preprint arXiv:2310.08774, 2023.
[49] Z. Zhou, Ji, W. Li, P. Dutta, R. Davuluri, and H. Liu. Dnabert-2: Efficient foundation model and benchmark for multi-species genome. arXiv preprint arXiv:2306. 55006, 2023.
13

---

A Background
A.1 Graph Structure Generation (GSG)
Let 9 = (A, )) denote graph, where A E RNNN is the adjacency matrix and E NxF the node feature matrix with 24 IR f being the embedding of node uy Given a feature matrix X, the larget of GSG is to directly learn a graph structure A by jointly optimizing the graph structure and Graph Neural Networks (GNN)
Feature Extraction To better model the similarity of node pairs, a feature extractor is usually diw-emmnniionll embedding
needed to map node features from high-dimensional input space
space [40].
Graph Construction A similarity metric function is generally used to compute the similarity between embedding pairs as edge weights. There are several ways to construct a sparse adjacency matrix from fully connected similarity matrix. For instance, node pairs whose similarity exceeds some predetermined threshold. In addition, we can connect the k nearest nodes to one node, thus constructing a k-nearest neighbor (kNN) (2
create graph that selects only connected
Graph Structure modeling The core of GSG is the structure learner, which models edge connectivity to refine the preliminary graph [1]]. Metric-base and neural network approaches are generally employed to learn edge connectivity through parameterized networks receive node representations
and regenerate the adjacency matrix that optimizes the graph structure, which is able to reveal the real connectivity relationship between nodes better and can be widely used in various downstream
tasks 226
A.2 Variational Inference (VI)
Variational Autoencoders (VAE) 15 is deep generative models that learn the distribution of input data by encoding it into latent space. In this process. the encoder maps each input to latent space defined by parameters: mean μ and variance a.. Latent variables are then sampled from this distribution for data generation.
VI is employed within VAE to handle the computational challenges of estimating marginal likelihoods of observed data. This involves computing the log of the marginal likelihood:
Σ log / Po Z)dz
N
max log logppogp)
(x(
(13)
where (X( Z) represents the joint distribution of the observable data e.g. Gaussian distribution, (x]au, σ)) and its latent encoding Z under the model parameter 0.
Since the direct estimation of marginal likelihoods is typically infeasible, VI introduces variational distribution 4s (z) to approximate the true posterior. The goal of VI is to maximize the Evidence Lower Bound (ELBO), formulated as:
ELBO- Bgg(=1s) (log pa (xs [c]=]] KL (=)x((
(14)
The first term is the reconstruction log- lig-liiel,hood, log (z(22) can be considered as decoder, i.c.. the log-likelihood between the reconstructed data and the original data given the potential representation. The second term, the KL divergence, quantifies the difference between the variational posterior (=|xx) and the latent prior p(2). Usually, VAE utilizes a reparameterization trick for gradient backpropagation through non non-differentiable sampling operations. Once trained, VAEs can generate new data by directly sampling from the latent space and processing through the decoder.
B Related Work
MCMC-based methods, such as MrBayes (2 and RevBayes (9, have been widely used for phylogenetic inference due to their ability to explore vast tree spaces. However, these methods are limited by the high-dimensional search space and the combinatorial explosion of tree topologies, which makes estimating posterior probabilities using simple sample relative frequencies (SRF) problematic [8]. The accuracy of these methods is compromised in unsampled tree spaces, and they
14

---

tend to be unstable when estimating low-probability trees, often requiring impractically large sample sizes achieve reliable results.
VI-based methods offer a more efficient alternative to MCMC by leveraging approximate inference techniques. These methods can be categorized into main approaches: structure representation and structure generation.
Tree Representation Learning. This approach focuses extracting information from existing tree structures. Subsplit Bayesian Networks (SBN) [45]: Given a collection trees, SBNs capture the relationships between existing subsplits, providing probabilistic representations of various tree shapes under given data. However, SBNs do not directly address branch lengths.
Variational Bayesian Phylogenetic Inference (VBPI) 46 and its variants VBPI-NF and VBPI- GNN
44 These methods introduce a two-pass approach to learn node representations, including branch lengths. VBPI employs variational approximations to handle branch lengths, allowing for joint modeling of the tree's latent space.
Tree Structure Generation. This approach aims to infer tree structures directly from sequence data. combination of VI and reinforcement learning, PhyloGFN constructs tree topologies by simplifying branch lengths into discrete intervals, requiring posterior data for accurate inference. VaiPhy [6
PhyloGFN 48 utilizes GelowNet 10
20 21
incorporates the SLANTIS sampling strategy 2 and basic biological models (e.g., Jukes Cantor model) to estimate topology and branch lengths. ARTree [36] employs a graph autoregressive model to build detailed tree topologies. Branch length estimation is conducted independently through log- ogg-robabblittes Once the topology is determined, classical evolutionary models (e.g., Cakeor (24 or GTR (eneralized time reversible) (44
are used to estimate branch lengths via maximum likelihood or Bayesian methods. GeoPhy [22
models tree topologies within continuous geometric space, offering a different approach to the distribution of tree topologies. ARTree and GeoPhy differ primarily in their approach to modeling tree topologies. While ARTree uses a discrete autoregressive model, GeoPhy leverages continuous geometric representations, providing more flexible framework for capturing the evolutionary relationships among species.
C Datasets
Our model PhyloGen performs phylogenetic inference on biological sequence datasets of 27 to 64 species compiled in [17]. Notably, our approach does not require sequences to be of equal length, overcoming common limitation in traditional phylogenetie analysis. In Tab. 6 summarize the statistics of benchmark datasets.
Table 6: Statistics of the benchmark datasets from DS1 to DS8..
Dataset
# Species
Sites
Reference
DS1
27
1949
6
DS2
29
2520
DS3
36
1812
38
DS4
41
1137
DS5
50
378
DS6
50
1133
DS7
59
1824
39
DS8
64
1008
30
D Methods
D.1 Scoring Function
To address the convergence challenges often associated with the ELBO in VAE models, we incorporate a scoring function S, implemented via an MI.P network. This function assesses each leaf node the latent space and provides additional gradient information, facilitating more efficient learning and
15

---

Similarity between ELBO Metric and Scoring Function
Nomaazzdd Difference (85)
60
Scoring Function w/ FC Scoring Function Scoring Runction MID-3
0.6 Scaled Cosine Similanity
1.5
Figure 8: Analysis of the Cosine Similarities between Scoring Function and ELBO.
During training. S and ELBO form a joint optimization objective, optimizing gradient directions to improve overall performance.
convergence.
Fig. compares the convergence behaviors and stability of S and ELBO throughout the training process. The horizontal axis represents the training steps, and the vertical axis represents the two proves that S can effectively
metric values. The closer the curve is the ELBO curve, the evaluate the model performance and maintain a consistent optimization trend with ELBO. Different configurations of S. including those with Fully Connected layers (w/ FC), with layers MLP (w/ MLP-2), and with three layers MLP (w/ MLP-3), demonstrate similar trends, closely following the ELBO curve. After an initial period of rapid change, all metries stabilize and exhibit minor fluctuations, demonstrating robustness in convergence. To further evaluate the performance of each configuration, we computed their scaled cosine similarity is calculated as: Scaled Cosine Similarity
ELBO. Scaled cosine similarity where x and y are vectors of normalized performance metries for S and ELBO, respectively. As depieted in Figure X the horizontal axis measures the similarity ranging from 0 to 1, with values closer to indicating a closer alignment with ELBO. The vertical axis displays the normalized difference percentage, highlighting how each configuration deviates from ELBO. All configurations maintain high similarity values, generally above 0.8, suggesting robust performance across different architectures. Given these results, the Fully Connected (w/ FC) configuration is selected for its closest to the ELBO curve and has better performance consistency. What's more, the number of layers in MLP has less impact on performance, as verified in Fig. and Fig. 3
x-y
| yll
D.2 Gradient Derivation for Objective Function
It is important emphasise that longer distinguish between 2" and in the derivation for simplicity and uniformly use the latent variable 2 to denote the topology.
In variational inference, we typically start from the joint probability distribution:
p(Y 0) = p(Y))))(())
(15)
where 9 could be any of parameters or latent variables, and is the observed data.
For a tree model, assume that includes the tree topology r(z) and the branch lengths B,. Thus, we have:
p(Y, (2), B,) ) = B
(16)
where (2)) B-) represents the conditional probability of the observed data Y, given the tree structure and branch lengths, while p(T(2), B,) is the joint prior probability of the tree structure and branch lengths.
Further assuming that the tree topology T (2) and the branch lengths B, are conditionally independent, the prior be decomposed as:
p(T(2), B-) p(B-(r(=))p(((())
(17)
Combining the above formula 16 and 17 we can rewrite the joint probability as:
p(Y,r(z),4 B,) (.))=
(18)
16

---

In variational inference. we introduce variational distribution ((T(2),B,) to approximate the true posterior distribution (z](2), B, B) and often assume:
((<()), B,) ((B.)r
(19)
Thus, the ELBO then be written
[ = By (lggp(Y, B+)] - By [log (T)) B,)]
(20)
Inserting the joint distribution and variational distribution formulas, obtain:
((Q) = E, Bbg + logg(((>))] -E, [log ((B,1T(2)) + log (T))(
B,)
(21)
This can be simplified by integrating the logarithmic terms:
((Q) By log B.
(22)
((B-IT(3))
This step introduces the key P and Q ratio. We are essentially comparing the variational distribution and the conditional probability model for discrepancies.
The conditional distribution C()(T(=)) is introduced to enhance the model's capacity for capturing complex dependencies within the data. This distribution is aimed at refining the approximation of latent variables given specific conditions. The revised ELBO, incorporating C, becomes:
C(Q, R) (Q(=)Q((,IT())) log p(Y, (.Y.BBBrr)))o(((t)))(()rr)))
(23)
Q(B,IT=)))(22)
This formulation explicitly considers the influence of (==)+(=)), reinforcing the variational frame- work's ability to more accurately model the posterior distributions conditioned on complex hierarchi- cal data structures.
Since 7(2) and B, are sampled from their respective distributions and B, explicitly depends on the former:
L(Q, R.) = (QQ==) EQ(B,IT(=)) log p(Y, (()
(24)
Q(B.IT()))((3)
Begin the derivation of the model parameters
p(Y, (.TBB//r()))((((()))) log Q(B.(7(3))((2)
VoL VelEuu()) (Q(B))
(25)
voL=v (ʃ Q0(=)) (o(R)))
log (()))
dz
(26)
(()x(((2))((2)
log P(Y, B, (.T
Vol Qa(=)
(Q(B,,I(E)))
dz (27)
((B,1T()))((2)
p(V, B, (())
= =Eo((2)
Voog Q0(2)
[Q(BLIT()))
log
(28)
((B,)T()))((z)
where H[Qo[z)] EEo log Qe())] the differential entropy. The derivative of H is voH(Qo(=)] (V) llog Qo(=)).
Then we use the chain rule apply the derivative operation the desired logarithmic term yields:
[[H(((()))] = - Egg(t)) V log Q=(=)) and Vo log Qo(z) = Qa(3) we rewrite the VoL as:
v.Qo(z)
P(Y, P.Y.B.
VoL EQu(z) V log (= (2) log
(29)
Q(B+(T()))Q(2))
P(Y, B,,,B,II(2)) log P(T(E))
-Eqo(s) ( log Q 1og
VH(
Q(B+IT(=))
(30)
17

---

Begin the derivation of the model parameters di
P(Y, B, 7()))
VoL V (=) (Q((B.Ir(a))
log Q -(=)))
(31)
We apply the reparameterization technique express variables involving stochasticity combi- nation of deterministic transformation and a noise distribution that does not depend on model parameters. With this transformation, the gradients of the model parameters can be passed directly through subsequent computations without being interrupted by the random sampling process. Next, assume that
be expressed as result of deterministic transformation ho(=)).
Accordingly, B, is no longer sampled directly from a simple normal distribution but is instead obtained by transforming it through the function ht),), which combines the independent noise variable ε (usually from a simple distribution such as the standard normal distribution) and r()) to generate B7.
B, = (S (en),((()),)))))) (z)), p(e)
(32)
Such treatment allows the model to take into account the fact that complex dependency structures into account, allowing the computation of gradients be carried in this reparameterized way. which is important for reflecting the actual complexity in biological data.
The expectation can be rewritten as:
(Q,(B, pe)) [log P(Y, (()) (x)))
(33)
Q(ssEE 7(2)) (()))
Substituting this into the derivation formula gives:
P(Y,B, (()))
V&C VolEqntt) log
(34)
Q(B, r(=)))
(Z)r)) Begin the derivation of the model parameters gh:
V Ful C Eqo(=)]Ve log (27(21((2))
(35)
D.3 Algorithm
PhyloGen's algorithm flow is as shown in Algorithm
Algorithm Phylogenetic Tree Generation
1: Input: Genomic sequences Y, Model parameters
2: Output: Optimized phylogenetic tree (r, B-)
3: Preprocessing: Encode Y into genomic embeddings using DNABERT2.
4: Initialize: Generate initial latent variable z..
5: Construct Tree: Compute distance matrix D from 2" and use NJ algorithm.
6: Optimize Tree Structure and Branch Lengths.
7: procedure GR TRRADINTUPPATE.: F,, T(2") ), 0, Φ, 1/)
8: Initialize model parameters 0, Φ,
9: while not converged do
Sample
Ryp (2*12) using the reparameterization trick.
11: Sample 2 ~ Qa(zlY) using the reparameterization trick.
Compute the loss Ltotal
(Eq. 9
13:
Compute gradients using backpropagation and Update parameters using Adam optimizer:
0 VeL
(Eg. 70
15:
Φ
- v V
(Eg. 17
1/
- 3/- V.L.
(Eg. 72
17:
end while
return Updated 0, d, 1/b
19:
end procedure
20: return the generated phylogenetic tree (r, Br)
18

---

E
Experiment
E.1
Training Details
We focus on the most challenging aspect of the phylogenetic tree inference task: the joint learning of tree topologies and branch lengths. For this, employ uniform prior for the tree topology and an independent and identically distributed (i.i.d.) exponential prior (Exp( (10) for the branch lengths. We evaluate all methods across eight real datasets (DS1-8) frequently used to benchmark phylogenetic tree inference methods. These datasets include sequences from 27 to 64 eukaryote species, each comprising 378 to 2520 sites. Notably, our approach does not require sequences to be of equal length, thus overcoming a common limitation in traditional phylogeneti analysis. For our Monte Carlo simulations, select K 2 samples and apply an annealed unnormalized posterior during each i-th iteration, where A, = min [ .0, 0.001 + i/H} acts as the inverse temperature. This parameter starts 0.001 and gradually increases to over H iterations, effectively simulating cooling schedule commonly used in annealing algorithms, similar to the approach in [45 with an initial temperature of 0.000, which gradually decreases over 100,000 steps.
During the model training process, we utilize stochastic gradient descent to process a total of one million Monte Carlo samples, employing K samples at each training step. The stepping-stone (SS) algorithm 377 in MrBayes is viewed as the gold-standard value. All models were implemented in Pytorch [7 with the Adam optimizer [14], The MIL estimate derived by sampling the importance of 1000 samples, with the larger mean value being better. The learning rate is initially set to le-4 and is reduced by factor of 0.75 every 200,000 training steps. Momentum is
0.9 to prevent the optimization process from becoming trapped in local minima. Utilizing the Stepl .R scheduler, the current learning rate is multiplied by 0.75 every 200,000 steps ensure steady progression, detailed in Tab. and Tab, g
Table 7: Training Settings of PhyloGen.
Table 8: Common Hyperparameters for PhyloGen.
Training Configuration
TopoNet
Optimizer
Adam optimizer
Hidden Dim. 256
Learning rate
le-4
# Layer
2
Schedule
Step Learning Rate
Output Dim,
Weight Decay
0.0
TreeEncoder
momentum
0.9
Hidden Dim.
256
min
le-6
# Layer 2
base_Ir max Ir
le-4
TreeDecoder
0.001
Hidden Dim.
256
scheduler.game gamma
0.75
# Layer 2
annealing init
0.001
DGCNN
annealing steps
100,000
# Layer 2
E.2 Baselines
We mainly compare PhyloGen with three types of methods, including two MCMC-based methods (i.c., MrBayes, SBN), two tree-structure representation learning methods (i.c.. VBPI, VBPI-GNN), and five tree-structure generation methods. It should be noted that the results of all bascline methods not included in the MLL tables, as some of the baseline methods are not provided with source code, and the results of the MLL metrics are not shown in the original paper.
E.3 Bipartition Frequency Distribution (RQ3)
bipartition frequency comparison plot is used in evolutionary analysis show the bipartition frequencies observed in a sample of tree topologies. Each bipartition represents node in the tree and defines the division of two sets of taxonomic units (e.g., species) on either side of that node. The frequency of these bipartitions in the posterior tree topology distribution reflects how often each topology appeared during the MCMC sampling process, which in turn reveals the confidence level of the different topological features.
19

---

We sample 1000 Monte Carlo trees from the posterior distribution Q(T), calculate the frequeney of each bipartition within these samples, and then compare these frequencies with the bipartition frequencies obtained using MrBayes. These frequencies are then compared with those obtained using the MrBayes method to visualize the consistency and differences between the two methods in terms of tree topology inference through graphs. Fig. [] shows the bipartition frequency distributions of trees inferred by PhyloGen for the DSI, DS2, and DS3 datasets. The horizontal axis indicates the ranking of the bipartitions in the tree topology. and the vertical axis indicates the normalized frequency of occurrence the corresponding bipartitions, reflecting the prevalence of various topological features observed during the MCMC sampling process. The results in the figure show that method's variation curves are consistent with those of MrBayes, which indicates that our method has a high degree of consistency in tree topology inference and is able capture underlying evolutionary patterns with a confidence level similar to the established gold standard MrBayes method.
E.4
Ablation Study (RQ5)
Tab. 5 shows the performance of our model compared with the removal of the KL loss and the Scoring Function S, respectively. ours performs best on both the ELBO and MLL. metrics, with significant decrease in performance after the removal of the KI. loss, suggesting that the KI. loss plays a key role in regularizing the model and avoiding overfitting. While removing the S module had a large impact on MLL, the ELBO impact was relatively small, indicating that the impact of the module is more complex and may be related to specific feature extraction functions. Our future work will explore further enhancements to the module and investigate other regularization techniques to refine the model's 's performance.
Fig. presents a comparative analysis of the MLL metric across six configurations our model on the DS1 dataset. The standard configuration ("Ours') demonstrates superior performance with the highest MLL. value, indicating optimal fit and stability throughout the training process. Variants without Layer Normalization or with reduced hidden dimension show significant decreases in MLL metric. Furthermore, models using other distance functions for tree construction Saaerioonsssiilllllll.l..mm Euclidean and Cosine distances-alose with those using one-hot encoding for leaf node representation demonstrate lower MLL values. Notably, the one-hot approach works the worst, suggesting that both the method of computing the distance matrix and the choice of feature representation for leaf nodes critically influence the model's 's accuracy. Additionally, the distance matrix (calculated from the latent space 2) that we specifically designed about evolutionary relationships that a simple Euclidean or Cosine distance matrix would not be able to capture, resulting in MLL values lower than standard configuration.
input to the NJ algorithm effectively captures key information
E.5 Hyper-Parameter Analysis (RQ7)
Table 9: Hyperparameter Analysis of PhyloGen Performance in Various Parameter Configurations.
Parameter Setting
Cfg.
ELBO (1)
MLL (1)
2
-7111.95
-6908.65
3
-7015.65
-6914.82
Output Dimension (emd)
4
-7012.20
-6911.02
(hid=256)
8
-7005.98
-6910.02
16
-7023.67
-6928.89
64
-7116.75
-6863.84
Hidden Dimension (hid)
128
-7021.16
-6917.80
(Layer Norm, emd=8)
256
-7005.98
-6910.02
512
-7023.96
-6915.18
64
-7123.71
-6912.71
Hidden Dimension
128
-7024.32
-6919.42
(No Layer Norm, emd-8)
256
-7020.07
-6920.05
512
-7028.93
-6921.01
The first part of Tab. demoonttrates the effect of different output dimensions on the model perfor- mance at fixed hidden layer dimension (hd=256). It can be seen that the ELBO and MLL metrics
20

---

show different trends when the output dimension is increased from 2 to 16, especially when the output dimension is 8. The ELBO and MLL values are optimal, suggesting that higher output dimensions may contribute to the model performance, but there are some fluctuations. The second and third parts of Tab. 9 compare the effects of different hiding dimensions on model performance with and without using layer regularization. When using layer regularization, the model reaches optimal values for both ELBO and MLL at a hidden layer dimension of 256. The overall decrease in performance when layer regularization is not used shows that layer normalization plays an important role in model stability and performance.
E.6 Visualizations
undornchi
LaStrsccnnre soliss
cond
Lusino Lutmmok Shallama
"Ammphum priccylin
Coooopiaanaa pictan
Moollluu galka
June crydolaggnnnnoo punicaioe
Ceaphopuur nollrocd
"Paa prancuss scoi
"espagaspiou.co coataat
cetrig
Figure 9: Visualization of phylogenetic trees. The left side shows phylogenetic tree constructed from the sequences of the DS1 dataset. Each leaf node represents a specific species, and the text next to the node indicates the species name. On the right side, the colored sequences represent fragments of the species' protein sequences, with different colored blocks corresponding to different amino acid residues. is worth noting that this method of sequence presentation does not require uniform sequence length or padding procedure and can effectively reflect actual sequence variations. The scale bar in the lower left corner indicates the ratio between branch length and evolutionary distance. These sequence fragments visualize the key sequence features which the construction of the phylogenetic tree depends.
F Broader Impacts
We recognise the importance of addressing the societal impact of work. Phylogenetic inference methods such as the one we propose have the potential to greatly improve our understanding of the evolution, origin, and transmission mechanisms of viruses and bacteria. Such understanding could have far-reaching societal implications, especially the fields of public health and disease control.
21

---

-iren_ Intermedia
Siren Intermedia
Grandisonia, _alternans
Grandisonia alternans s
-Trachemys _crippt
Trachemys_ sscrsssssataaaa
LLatieriia_che chalumnnae
Latimeria chalumnae
Latimeria_callumnae
Heterodon, centerodon_platyhhine
platyrhinos
Heterodon_platye platyrhinos
Nesomantis ttoomasseet
Nesomantis_thommann
Plethodon yonhalossee
Plethodon_ yonhalossee
Homoosapiens sapiens
|AApphumaa tridactylum
Amphiuma_ trridaccylu
lichthyophis bannanicus
Ichthyophis bannanicus
Typhlonectes_ _atans
Typhlonectes_natee _natans
Discoglossus pictus
Discoglossus _accuuuuup___
carolinensis Hyla cinerea
eeesroohhyne__aarliinensas
Gastrophryne. caroliinnsssrlllnnsissssss
Tyla_cinerea
+Gallus gaallss
Gallus gallus
Oryctolagus cuniculus
Oryctolagus_cumme cuniculus
Eleutherodactylus KuneatusEleethherodacy//ee cururr
cuneatus
Scaphiopus holbrooki
Scaphiopus _hobbrookhhoool
Sceloporus undulatus
Sceloporus_undule undulatus
Alligator__mine mississippiensis
Alligator_miae paatto__misiissppiensss
|Muss_ masscuuuuuuus
Mus_ musculus
Rattus norvegicus
Rattus norvegicus
Ambystoma mexicanum
babystomaa_mexianuuee
Hypogeophis rostratus rostrratus
sypogeophis_rostraie
Bufo valliceps
jufo_valliceps
2.5
Figure 10: Enhanced visualization of phylogenetic relationships depicted through coloured heatmap integrated with a phylogenetic tree. The evolutionary tree, structured using Newick format data, illustrates the hierarchical relationships among species. Nodes are distinctly coloured to represent the frequency of bipartition support derived from posterior probability analysis: high support (>0.2) is indicated with blue, medium support (>0.1) with green, and low support with red.
Shorerode cahel ans
Chennrdkk zoe bre plancrade cuive ken Enioiarac bóc. precereel ase baworeeh planarale pobteccats
Steree poreidees
(70
Celdeo (T))
buranqprieeea wew.ereeeeeeeeer dese
instrteenntte gosutooendeeee Inhden
(1)
Belalver-kn ooikeoor Belseriinoanee a Woodiemmaalee besseees ven joeseetbe wwweeereet
Kippy ouz.
Kaoure D338
(12)
pnockteee Histnede
Anallnad (16)
F1u Jouthete Certrstacbettrn busienteen bestontean pedeaaaiieeaaeeee
Pas4ppn jelerratttetee aothinrssesees peresees peoterbe
www.yeaaatttiieee..o. (20
Eleecismeii Hrrdés
(1]
ciiiititinn (20
speenn (12)
Chooken
Scrog
Schopommnss
55y00 STIZ
Figure 11: Phylogenetic tree visualization. This figure shows the phylogenetic relationships of bacterial taxa constructed based comparative genomies analysis. Each node in the tree represents a species sample, and the labels on the right side show the detailed classification the samples according to the taxonomic hierarchy, from broad taxonomic classes (c.g., "Bacteria") to more specific classes (e.g., "Genus" and "Species"). "species").
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction do contain the claims, innovations and contributions mentioned in the paper.
22

---

Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss a comparison of the metrics, design experiments to verify the robustness of the model to noisy data, and state the assumptions.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texx system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We detail all the formulae derived in the paper in the main text and in the appendix for the derivation part of the equation.
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
23

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We detail dataset information, network architecture, various hyperparameters, and algorithms in the main text and appendices.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- spen-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We are not providing the source code now and will release the full code if it is acceptable.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
24

---

Please see the NeurIPS code and data submission guidelines (tttps://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nip. ptlpse.ccpublicc/uuueeee Swweaee//doeSSmmieeeeee ssloncliccyii for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We detail the network architecture of the model, the various hyperparameter settings and the algorithmic pseudocode in the main text and in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In the experimental section, we provide standard deviations for the ELBO and MILL metrics and, in the appendix, we mention the choice of parameters and describe how we ran the various methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
25

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the time required to run the datasets in the robustness experiment analysis.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppp///.earpps..cppppp.///taaa
Answer: [Yes]
Justification: We have carefully read the NeurIPS Code of Ethics ensure compliance in all areas.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We illustrate in the Appendix, in the Broader Impacts section, about the societal impact of our work, which has the potential greatly improve our understanding of the evolution, origins, and transmission mechanisms of viruses and bacteria, particularly in the areas of public health and discase control.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
26

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work has no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite all research works.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
27

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We are not providing the code and will release all code if the paper is accepted.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work does not involve crowdsoureing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "v2NuF5fgbl", "reviewer": "Reviewer_QAeZ", "summary": "This paper presents PhyloGen, a novel approach for phylogenetic tree inference using pre-trained genomic language models and graph structure generation. PhyloGen aims to jointly optimize tree topology and branch lengths without relying on evolutionary models or equal-length sequence constraints. The method demonstrates superior performance and robustness across multiple real-world datasets compared to existing approaches.", "questions": "How does PhyloGen perform on larger datasets with hundreds or thousands of species?\n\nHave you explored using other types of pre-trained language models besides DNABERT2?\n\nHow sensitive is the method to hyperparameter choices?\n\nWhat are the main computational bottlenecks of the approach?\n\nHave you tested PhyloGen on simulated datasets where the true phylogeny is known?", "limitations": "While some robustness tests were performed (node additions and deletions), the paper doesn't explore all possible types of data noise or perturbations.\n\nThe paper doesn't deeply explore the interpretability of the model's decisions or the learned representations.\n\nThe paper doesn't extensively discuss how well the method generalizes to different types of genetic data or organisms not represented in the test datasets.\n\nThe method relies on pre-trained genomic language models (specifically DNABERT2), but doesn't explore how performance might vary with different pre-trained models.", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Novel approach combining pre-trained genomic language models with graph structure generation for phylogenetic inference\n\nJoint optimization of tree topology and branch lengths without typical constraints\n\nSuperior performance on benchmark datasets compared to existing methods\n\nRobust to data changes and noise\n\nProvides deeper insights into phylogenetic relationships\n\nComputationally efficient compared to baselines", "weaknesses": "Limited discussion of potential limitations or failure cases\n\nLack of comparison to some recent methods in the field\n\nSource code not provided", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness:**\n\n**W1: Limitations:** \n\nPlease see the General Response.\n\n**W2: Comparison with Recent Methods:**\n\nWe appreciate the reviewer's concern regarding the comparison with recent methods. We believe that our manuscript already includes comparisons with several of the latest approaches, specifically GeoPhy (NIPS2023), PHyloGFN (ICLR 2024), ARTree (NIPS2023), and VBPI-GNN (ICLR2023), which have been influential in recent literature. However, we acknowledge the rapidly evolving nature of the field and invite the reviewer to suggest any additional methods that should be considered for comparison. \n\n**W3: Code:** \n\nPlease see the General Response.\n\n**Questions:**\n\n**Q1: Performance on Larger Datasets:**\n\nThank you for your question. We want to emphasize that PhyloGen has been tested on the same datasets as all recent methods, ensuring a fair evaluation. PhyloGen is designed to scale, yet there are currently no datasets available that contain hundreds to thousands of species. This is primarily because most phylogenetic analyses don’t require simultaneous processing of such extensive species counts; instead, research focuses on constructing subtrees for specific subsets. Phylogenetic methods fall into two categories: distance-based (UPGMA and NJ) and character-based(Maximum Parsimony (MP), Maximum Likelihood (ML), and Bayesian methods). The former is fast but limited, while the latter, although computationally intensive, offers greater precision. Review studies [1] suggest Bayesian is the most accurate, followed by ML and MP. Although PhyloGen offers significant speed improvements while maintaining accuracy, its performance on large datasets is still constrained by the inherent time complexity of these methods.\n\n[1] Hall, Barry G. \"Comparison of the accuracies of several phylogenetic methods using protein and DNA sequences.\" *Molecular biology and evolution* 22.3 (2005): 792-802.\n\n**Q2 and L4: Pre-trained Language Models:**\n\nThank you for your question. We selected the genome-specific foundation model DNABERT2 for our phylogenetic inference research. Although models like HyenaDNA and Nucleotide Transformer (NT) excel in long-sequence modeling, they are less apt for our specific needs. Upon your suggestion, we tested these models:\n\n| Method | MLL |\n| --- | --- |\n| DNABERT2 | -6910.02 |\n| HyenaDNA | -6918.63 |\n| NT | -6921.81 |\n\nDNABERT2 outperformed others, likely due to its specific optimization for genomic data. Our revised manuscript will detail these findings to justify our choice and showcase our method's flexibility.\n\n**Q3: Hyperparameter Sensitivity:**\n\nThank you for your question. Our Experiment Analysis (see Section E.5 in Appendix A, Table 9) demonstrates PhyloGen's robustness across various hyperparameter settings, including Output Dimension, Hidden Dimension, and Layer Normalization. We also evaluated the impact of changing hidden dimensions with and without layer regularization. The results show that PhyloGen maintains robustness to hyperparameter choices.\n\n**Q4: Computational Bottlenecks:**\n\nWe've detailed this in General Response's Limitations.\n\n**Q5: Simulated Datasets Tests:**\n\nThank you for your insightful question. In phylogenetic analysis, \"true phylogeny\" often relies on theoretical assumptions derived from widely accepted biological software. As discussed in our introduction, these traditional methods require sequence alignment, which is time-consuming and computationally intensive. Additionally, our manuscript includes \"PhyloTree Case Studies\" (RQ6) and visualizations in Sec. E.6 demonstrates PhyloGen's ability to cluster biologically relevant species and highlight evolutionary relationships effectively. While defining true phylogenies in simulated datasets presents challenges, natural tree structures offer a more valid basis for inference. Furthermore, this setup is rare in standard references and baselines. We appreciate your interest, as it helps clarify our method's scope and potential enhancements. Future work will explore this area more thoroughly to validate PhyloGen's effectiveness.\n\n**Limitations:**\n\n**L1: Robustness:**\n\nThank you for your comments. We've extended our tests to include scenarios such as genetic mutations. Here are the results from dataset DS1 under different settings: \n\n| Setting | Description | ELBO |\n| --- | --- | --- |\n| Setting 3 | Replaced sequences with alternatives from the same species | -7008.23 |\n| Setting 4 | Applied mutation rate of 5% | -7012.02 |\n| Setting 5 | Applied mutation rate of 10% | -7021.17 |\n\nThe results show that the ELBO metric for Setting 3 is similar to the original result while Setting 4’s result is closer to the original than Setting 5. This indicates that our model maintains robustness under different types of data noise and perturbations, particularly in handling genetic mutations.\n\n\n**L2: Interpretability:** \n\n\nThank you for your feedback. Our manuscript includes case studies in RQ6 and visualizations in Sec. E.6, demonstrating PhyloGen's application to real-world genetic data. These provide intuitive understanding and help enhance overall interpretability. Specifically, Fig. 7 illustrates the clustering and phylogenetic tree structure, effectively showcasing the representation results. Compared to similar studies, our approach provides a more extensive and comprehensive analysis, setting a high standard for interpretability within the field. We appreciate your feedback and will continue to refine this aspect.\n\n**L3: Data Generalizability:**\n\nThank you for your question. Indeed, the benchmark datasets used by PhyloGen include a wide variety of organisms covering marine animals, plants, bacteria, fungi and eukaryotes. This ensures that our model has been tested in various biological environments, demonstrating its ability to generalize to different types of genetic data and organisms not specifically represented in the test dataset.\n\n**L4:Pre-trained Models:** \n\nPlease see our response to Q2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZF7JWS6Rls", "reviewer": "Reviewer_XKPL", "summary": "The authors propose a new method, PhyloGEN, for phylogenetic inference. The method is able to perform end-to-end variational inference in order to jointly optimize the tree topology and the branch lengths. To achieve this, the authors propose using a pre-trained genomic language model to extract genome embeddings, and the embeddings are then used to form an initial topology which is refined iteratively using the proposed topology and branch length learning modules.", "questions": "- Could the authors explain that if MrBayes is considered the gold standard, and the proposed method is validated by comparing it against MrBayes in RQ2 and RQ3, why is the proposed method able to achieve such a big increase in MLL and ELBO while the other methods yield relatively similar results? Relatedly, do the resulting topologies look significantly different in comparison to the other methods?\n\n- The proposed method has low standard deviations in Table 2 for multiple datasets. Is this due to the optimization being robust with respect to the random seed or could it not indicate that $Q(z)$ does not support multiple topologies? Do the MrBayes posteriors support multiple topologies?\n\n- How does the Bipartition Frequency Distribution look for other methods like GeoPhy and ARTree?\n\n- How does the runtime of MrBayes compare to the runtime of the proposed method?\n\n- Can the authors hypothesize why a simple fully connected $S$ would work so well (Figure 3)?\n\n- In the construction of the distance matrix, could the authors explain why using XOR makes sense since aren't the $z_i^*$ continuous? Moreover, would it make sense to try out other similar algorithms to neighbor-joining to initialize the topology?\n\n- I would suggest that the authors move section 3.3 (with the derived gradients) to the Appendix, leaving space for a more detailed description of the various design choices.", "limitations": "The authors do not explicitly discuss the limitations of their work. As I am not an expert in phylogenetic inference, it is difficult to judge the assumptions made in the paper. Are there conditions that are required to be met for using a pre-trained language model for extracting embeddings, and is the assumption of the tree topology and branch lengths being conditionally independent reasonable? It would be helpful for the authors to elaborate on what they think the possible limitations of their own work are.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper makes a novel contribution to an important research topic. Unlike previous variational inference methods, the method does not require pre-generated topologies.\n\n- The incorporation of a pre-trained genomic language model to the method is an interesting contribution and is something that will likely be increasingly explored in the future.\n\n- The method is compared against a number of recent phylogenetic inference methods, and it achieves superior performance in terms of MLL and ELBO on standard benchmark datasets compared to the previous methods. In addition, experiments are performed to investigate the robustness of the proposed method and the diversity of the generated topologies, indicating that the method is consistent with the \"gold standard\" MrBayes approach. The authors also provide an ablation study to understand the impact of the various design choices in their method.", "weaknesses": "- The methods section (Section 3) is difficult to read in that variables are introduced without leading sentences, some notation is undefined, and design choices are introduced without clear motivation.\n  - What is the motivation for $f_i$ and how is it defined in (2) if there are multiple children $f_j$?\n  - The function $h$ is used before its definition.\n  - Which function $h$ is the latter $h$ in the definition $h(x_i, x_j) = h(x_i, x_i - x_j, d_{ij}^2)$?\n  - Why is max aggregation a good choice?\n  - The scoring function $S$ is not described in enough detail. What does it output and how does it provide additional gradient information?\n\n- Most experiments, in particular RQ2, RQ4, RQ5, RQ6, and RQ7 are performed on only one data set (DS1), making it difficult to draw meaningful conclusions. I would also consider the runtime an important metric, but it is measured only in conjunction with the robustness assessment on only one data set.\n\n- The authors do not provide code for their method or experiments (although the code is promised to be released later). Without the code, considering the lack of detail in the methods section, it would be difficult to reproduce the proposed method.\n\n- Minor:\n  - The red and yellow colors are opposite to what they should be in the caption of Table 1.\n  - There is a subsection numbered 3.1 in Section 4.\n  - RQ7 is only referred to in the Appendix, and not in the main text.\n  - Typos: \"TreeEnocoder\" (Figure 2 caption), \"Metrtic\" (Figure 3 title)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weaknesses：**\n\n**W1: Clarity in Methods:**\n\nWe acknowledge the reviewers' concerns about the presentation of the methods section. In response, we have made a thorough revision:\n\n1. **$f_i$** enriches the node feature $i$ by integrating contributions from its children and its inherent traits. In Eq. 2, $f_j$ represents the features of child nodes, and for multiple children, we aggregate their contributions directly in Eq. 1.\n2. And 3. **h Clarification:** We acknowledge the confusion caused by using the same symbol h for two conceptually different functions in the manuscript. The first $h(x_i, x_j)$ is an abstract representation used to describe a distance measure between two nodes. The second $h(x_i, x_i - x_j, d_{ij}^2)$ is a specific implementation of the first, calculating differences in the latent space between nodes x_i and x_j, evaluated using the formula $\\sum (\\| x_i - x_j \\|^2 - d_{uv})^2$.\n3. **Max Aggregation** selects the most significant features from parent and child nodes, ensuring the most prominent features are retained during propagation.\n4. **Scoring Function S** maps each leaf node in the latent space to a score reflecting its contribution to the model's overall performance. Joint optimization with the ELBO objective ensures that parameter updates during training align with the intended direction.\n\n**W2: Ablation:** \n\nWe chose DS1 because it is representative and commonly used in the field, ensuring comparability with other recent studies. Due to time constraints, we have included ablation results for some datasets in the attached file. We can add more if you feel it’s necessary. For a complete comparison of runtime, please refer to Tab. R2.\n\n**W3: Code:** \n\nPlease see the General Response.\n\n**M1, M2, M3, M4:** \n\nThank you for your correction. We have corrected the caption and subsection numbers accordingly. We will also explicitly reference RQ7 in the experimental section to better connect it with the appendix details.\n\n**Questions:**\n\n**Q1: Superior Performance Over MrBayes:**\n\nWe appreciate your insightful inquiry. PhyloGen outperforms MrBayes and other methods on MLL and ELBO due to enhanced genetic sequence representations via a pre-trained genomic language model. This approach allows for a deeper utilization of genetic information, improving the accuracy of phylogenetic tree construction. For topology comparisons, we use metrics such as Simpson's Diversity Index and Frequency of the Most Frequent Topology. Results across multiple datasets, detailed in Tab. R1 shows PhyloGen's ability to generate diverse and balanced topologies, revealing subtle evolutionary relationships that traditional methods may miss while maintaining biological accuracy.\n\n**Q2: Low Stds:**\n\nTab. 2's low stds demonstrate PhyloGen's robustness against random seed variations, maintaining low variance despite supporting diverse topologies. Our variance levels are comparable compared to recent methods like ARTree and PhyloGFN. In contrast, MrBayes, which uses a sampling-based approach, exhibits higher variance across varied topologies but does not match PhyloGen's performance even when considering maximum variance. Furthermore, MrBayes employs MCMC algorithms for estimating posterior distributions of phylogenetic trees, necessitating lengthy MCMC chains to achieve convergence. While parallel computing and evolutionary model optimization can mitigate some computational demands, they do not overcome the inherent efficiency and variance control limitations.\n\n**Q3: Bipartition Frequency Distribution:**\n\nThank you for your inquiry. We have included a comparison of bipartition frequency distributions for GeoPhy, as detailed in Fig. R2. However, direct comparisons with ARTree are not feasible since its autoregressive generation method does not produce Newick files.\n\n**Q4: Runtime Comparison:**\n\nThank you for your suggestion. We detailed this in Tab. R2. \n\n**Q5: Efficacy of a Simple FC Layer:**\n\nThe simple FC layer's effectiveness can be attributed to the high-quality features provided by DNABERT2, which allow the basic architectures to handle complex tasks efficiently. Additionally, minimal parameters and appropriate regularization ensure model stability and convergence.\n\n**Q6: XOR in Distance Matrix:**\n\nUsing XOR to construct the distance matrix effectively captures discrete nucleotide mismatches without requiring sequence alignment, directly measuring genetic variations. This approach aligns with recent methods like GeoPhy and VaiPhy. Additionally, we explored the UPGMA algorithm on DS1, and the results are summarized in Fig. R1 and the following table:\n\n|  | ELBO |\n| --- | --- |\n| NJ | -7005.98 |\n| UPGMA | -7115.95 |\n\nUPGMA assumes uniform evolution rates and is straightforward but limited to consistent-rate scenarios. In contrast, NJ doesn’t assume constant rates, offering more accurate reflections of evolutionary branching in uneven-rate organisms. Overall, NJ proves more effective in handling complex and uneven evolutionary data.\n\n**Q7: Suggestion:**\n\nThank you for your suggestions; we will change them in the revised version.\n\n**Limitations:**\n\nThank you for your inquiry. Our first use of pre-trained language models to extract embeddings in this domain is validated by our experimental results and not contested in the literature, and we expect that this could spark further discussion in the field. Regarding the dependence on tree topology and branch lengths, our assumptions align with recent methods such as ARTree, GeoPhy, and PhyloGFN, which are widely accepted. A detailed discussion of limitations can be found in the General Response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2qP77c83Y8", "reviewer": "Reviewer_4WGb", "summary": "The paper propose phylogenetic tree inference by modeling it as a problem of conditional-constrained tree structure generation. Its goal is to jointly generate and optimize the tree topology and branch lengths. By mapping species sequences into a continuous geometric space, PhyloGen performs end-to-end variational inference without limiting the possible topologies. To maintain the topology-invariance of phylogenetic trees, distance constraints are applied in the latent space to preserve translational and rotational invariance. They have shown the proposed model  is  effective  across eight real-world benchmark datasets.", "questions": "1. Regarding the statement, \"Existing Variational Inference methods, which require pre-generated topologies and typi6 cally treat tree structures and branch lengths independently, may overlook critical sequence features, limiting their accuracy and flexibility\", can you elaborate on what kind of features do you refer to that the method could overlook?\n\n2. I am not sure fromt he distance matix to the tree topology, the  Neighbor-Joining (NJ) algorithm can be differentiated to be able to learm end to end?\n\n3. I think the section C.1. Topology Learning is written somehow confusing. Especially from line 121 to line 124. Why the distribution  p(\\tao(z)) is represented as the distribution of y conditioned on tree structure? same as the q(\\tao(z), B_\\tao) is not clear what distribution it represents.  \n\n4. I am not sure in this kind of tasks, if the ELBO is a good metric to reflect the inference power of the model.\n\n5. The VAE model basically takes the embedding of the genomics sequences, and the initial topology of the tree \\tao, then try to reconstruct the tree (without the leaf length)? I was thinking what happens if we remove the learning of the first component, make the phylo tree construction constant using directly the embeddings from LLM?\n\n6. I am not sure if I understood how the scoring function is implemented, f(z), z is the embedding of the genomic sequence, f applied on the leaf nodes? or all the nodes, and why minimize it?", "limitations": "1. While PhyloGen outperforms many existing methods, it still requires substantial computational resources, particularly during the training phase. The complexity of jointly optimizing tree topology and branch lengths, along with maintaining topological invariance, sounds computationally heavy, for instance:\n1.1  Monte Carlo simulations to sample from the posterior distributions of tree topologies and branch lengths.\n1.2 The tree construction module computes distance matrices from latent variables and uses algorithms like Neighbor-Joining (NJ) to generate initial tree structures. \n1.3 The utilization of  a pre-trained genome language model (DNABERT2) to transform DNA sequences into genomic embeddings, \n\n2, Looking at the table 5, I am wondering how it works in inference time, would peformance drops when we have less or more specious than the one in the training data the model was trained on? Also how it translates to other specious? moreover, did you train one model per dataset, or is there a way to train all together?", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper addresses very interesting problem and propose novel approches.", "weaknesses": "I am not sure if the formulation equation 7, adding the term R mathematically make sense, as orignal ELBO in equation 6, is the lower bound, now adding the term log of R which is negative, and trying to maximize equation 7 would result in minimizing R?  Also in equation 7, q(\\tao (z)) seems missing, or is there a kl between R and Q from module B?\nI think the paper presentation needs to be improved. The figure 2 is very helpful but the text  and some notations are a bit confusing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness: Formulation Clarify:** \n\nThank you for your valuable feedback. Regarding the introduction of the R term in Eq. 7, we state the following:\n\n**Introduction of R:** R represents the posterior probability in the second part of the variational network. Its inclusion aims to enhance model expressiveness by capturing the underlying structure and parameter dependencies more effectively.\n\n**Expression of $q(\\tau(z))$:** Detailed derivations in **Appendix, Equations 19-31** clarify that $q(\\tau(z))$ is indeed Q(z), indicating that our model includes the KL divergence between R and Q, which is a crucial component. We will improve the presentation and explanation of these concepts in the revised manuscript to make them clearer. \n\nWe will also enhance the overall presentation of the paper to reduce any confusion.\n\n**Question：**\n\n**Q1: Overlooked Sequence Features**\n\nThank you for your question. The key sequence features we refer to involve the semantic aspects of genetic information, which are often overlooked by traditional distance-based methods. These methods typically rely on simplified one-hot encoding, which fails to capture the complex interactions and biological functions within sequences. In contrast, our approach utilizes pre-trained language models to extract richer representations from the sequence context, effectively capturing complex relationships and semantic information. This enhances the accuracy and flexibility of phylogenetic inference.\n\n**Q2: Differentiability of NJ Algorithm**\n\nThank you for your question. You are correct that the Neighbor-Joining (NJ) algorithm is not differentiable. However, in our approach, NJ is only used to generate the initial tree topology and does not require differentiation. The overall model is trained end-to-end, with differentiation occurring in other parts of the model, not within the NJ algorithm.\n\n**Q3: Clarification on Section C.1:**\n\nThank you for pointing out this issue. The notation $p(\\tau(z))$ in the main text is incorrect and should be corrected to match the accurate representation in the **appendix**, specifically on **line 540**. Additionally, $q(\\tau(z), B_\\tau)$ corresponds to the distribution detailed in Eq. 19. We will ensure these notations are consistent and clear in the revised manuscript.\n\n**Q4: ELBO Metric:**\n\nThank you for your insightful comment. You are correct that ELBO alone may not fully capture the model’s inference ability, which is why we introduced the Scoring function. Additionally, it is important to note that our baselines and recent methods also use ELBO and MLL, so we must use these metrics for fair comparison.\n\n**Q5: LLM Embeddings and Tree Construction:**\n\nThank you for your question. The embeddings generated by the LLM are primarily for feature extraction, not for direct tree construction. The initial topology and the subsequent adjustments made by the first module are essential to ensure the embeddings are constrained to a Gaussian distribution via the encoder, as assumed in our model. Without this step, the embeddings might not converge properly, and the phylogenetic tree construction could be less accurate.\n\n**Q6: Scoring Function:**\n\nThank you for your question. The scoring function maps each leaf node in the latent space to a score, representing their contribution to the model's overall performance. The function is jointly optimized with the ELBO objective to ensure the directionality of parameter updates during training. We minimize the scoring function to enforce low-rank representations in the matrix, which helps construct a well-structured initial tree, leading to faster convergence and improved accuracy.\n\n**Limitations:**\n\n**L1: Computational Resources**\n\n**1.1 Monte Carlo:** PhyloGen does not use traditional MCMC to sample from posterior distributions of tree topologies and branch lengths. Instead, we employ a Variational Bayesian approach [1], which significantly reduces computational demands by optimizing the variational lower bound to approximate the posterior, avoiding the inefficiencies and high sample requirements of MCMC in high-dimensional tree spaces.\n\n**1.2 NJ Algorithm:** NJ is used to construct the initial tree structure and requires some computational resources. It is executed during each iteration's initialization phase. While it adds to the computational load, this step ensures accuracy and stability, providing benefits that outweigh the computational costs and are not achieved by other methods.\n\n**1.3 DNABERT2:** This step is highly efficient. For instance, processing 100 sequences takes only 1.76s, and our benchmark datasets are much smaller, ensuring that embedding computation isn’t a bottleneck.\n\n[1] Zhang C, Matsen IV F A. Variational Bayesian Phylogenetic Inference[C]//ICLR (Poster). 2019.\n\n**L2: Training and Inference Performance:**\n\nYou are correct in noting that we trained one model per dataset, similar to recent methods in the field. During inference, the model's performance is limited to the current dataset on which it was trained. We are exploring fully end-to-end approaches to address these limitations and improve generalization across different species and varying numbers of species.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y7f0Mm1HAJ", "reviewer": "Reviewer_MKto", "summary": "The authors introduce PhyloGen, a new method that uses a pre-trained genomic language model to generate phylogenetic trees without relying on evolutionary models or aligned sequences. PhyloGen treats phylogenetic inference as a conditionally constrained tree structure generation problem, jointly optimizing tree topology and branch lengths through three modules: Feature Extraction, PhyloTree Construction, and PhyloTree Structure Modeling. A Scoring Function guides the model towards stable gradient descent. The method achieves state-of-the-art performance when evaluated on benchmark datasets.", "questions": "1. Could you clarify whether the computation of the marginal log likelihood involves an evolutionary model like [1]? Does the evolutionary model utilize aligned sequences? If so, does this contradict the statement in the article that sequence alignment is not required?\n\n2. I noticed that the author provides a comparison of run times. Could you provide details about the GPU used in the experiments, as well as the memory utilization?\n\n\n[1] M. Zhou, Z. Yan, E. Layne, N. Malkin, D. Zhang, M. Jain, M. Blanchette, and Y. Bengio. Phy-logfn: Phylogenetic inference with generative flow networks. arXiv preprint arXiv:2310.08774, 2023.", "limitations": "I have no concerns on the potential societal impact of their work.", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The framework is novel in that it leverages a pre-trained genomic language model  instead of relying on aligned sequences.\n2. The method achieves state-of-the-art performance evaluated on real-world datasets.", "weaknesses": "1. Ablation Experiments Clarity\n\nThe ablation experiments are unclear. My main concern is the lack of clarity regarding why the proposed method significantly outperforms other methods. Specifically:\n\n(1) Table 5: This table shows the average metrics across the eight datasets. However, Table 1 and Table 2 only shows the metrics for each dataset individually without any average metrics. It’s unclear what the performance gap is between the ablation model and the baseline methods when KL or S are removed.\n\n(2) Figure 6: The ablation studies in this figure only show the metrics for the DS1 dataset. Please include the average metrics across all eight datasets.\n\nTo improve the clarity of the ablation experiments, I strongly suggest making the evaluation settings consistent in Table 1, Table 2,Table 5, and Figure 6. At the very least, include the average metrics across the eight datasets. Additionally, provide more explanations based on the complete ablation experiments to clarify why the proposed method significantly outperforms other methods.\n\n2.  Claim on Aligned Sequences vs. Unaligned Sequences\n\nThe claim that unaligned sequences better reflect actual sequence variation than aligned sequences is questionable. Generally, aligned sequences with direct site-to-site comparisons are more effective in comparative sequence analysis. The authors need to provide more direct experimental evidence to support the advantage of using unaligned sequences.\n\n3. Writing Issues\n\nThere are many writing issues. For instance,\n\na. line 141. \"m_ij\" -> \"h\"\n\nb. In the caption of table1, \"VBPI-GNNuse\" -> \"VBPI-GNN use\"\n\nc. line 213, \"3.1 Experiment Setup\"  ->  \"4.1 Experiment Setup\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weakness:**\n\n**W1: Table 5 Clarification:** Tab. 5 presents results for the DS1 dataset, not an average across eight datasets. The impact of removing KL or S has already been shown in this table.\n\n**Figure 6 Explanation:** Fig. 6 shows ablation results for the DS1 dataset, chosen for its representativeness, as many recent studies also use this dataset. This is a common practice in the field. Due to time constraints, we have added ablation results for some other datasets in the attached file. We can continue to add results if you feel it is necessary.\n\n**Performance Explanations:** The complete ablation experiments are shown in Tab. R1 indicates a significant performance decline when KL or S is removed, underscoring their importance in model regularization and overfitting prevention. Replacing our distance matrix with Euclidean or cosine distances significantly lowers MLL, indicating these conventional distance methods fail to capture complex evolutionary patterns. Using one-hot encoding exhibited the lowest MLL, highlighting the superiority of our feature extraction module and the pre-trained language model in representing complex biological data.\n\n**W2: Aligned vs. Unaligned Sequences**\n\nThank you for your feedback. There might be some misunderstanding regarding the concept of \"aligned sequences\". In our manuscript, we refer to equal-length sequences, not multiple sequence alignments (MSA). This definition is consistent with recent methods like PhyloGFN and GeoPhy, which refer to \"pre-aligned sequences\" as uniform-length sequences, as seen in statements like \"These datasets feature pre-aligned sequences\" or \"Let Y be a set of aligned sequences with length M from the species.\". Our method analyzes raw sequences directly, avoiding biases introduced by alignment algorithms, thus preserving the original sequence diversity and more accurately reflecting sequence variation and evolutionary relationships.\n\n**W3: Writing Issues**\n\nThank you for highlighting the writing issues. We will thoroughly review and improve the manuscript for clarity.\n\n**Questions:**\n\n**Q1: MLL:**\n\nThank you for your question. Let me clarify these concepts:\n\n**PhyloGFN:**\n\n1. **Evolutionary Model:** PhyloGFN uses an evolutionary model to compute the marginal log likelihood (mll) through trajectory sampling and importance sampling. This involves generating trajectory data, calculating the forward and backward path log probabilities, and computing the log scores for each generated tree.\n2. **Aligned Sequences:** PhyloGFN does not rely on traditional multiple sequence alignment (MSA). Instead, it uses sequences of equal length, which do not require MSA for alignment.\n\n**Our Method:**\n\n1. **Evolutionary Model:** Our mll computation does not involve PhyloGFN's evolutionary model. We use a mutation model based on mutation probabilities, not the trajectory and importance sampling methods used in PhyloGFN.\n2. **Aligned Sequences:** Our method does not require either MSA or equal-length sequences. PhylGen directly analyzes raw sequence data, avoiding the computational overhead and potential errors introduced by sequence alignment. Thus, it more accurately reflects sequence variations and evolutionary relationships.\n3. **Consistency with Claims:** Our method aligns with the statement that sequence alignment is unnecessary. By handling unaligned, variable-length raw sequences, PhylGen more accurately reflects sequence variation and evolutionary relationships. This makes our method distinct from others that rely on equal-length sequences, like PhyloGFN and GeoPhy, and consistent with the claims made in our article.\n\n**Q2: Memory Usage**\n\nThank you for your question. We used an NVIDIA A100-SXM4-80GB GPU for our experiments. However, the choice of hardware is not crucial for running our training algorithm. Detailed memory usage can be found in the General Response and the Tab. R2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**General Response:**

We are very grateful to the four reviewers for their insightful comments, which have significantly improved the quality and clarity of our manuscript. 

**Summary:**

We are pleased that the reviewers recognized the highlights of our work, including the novel framework combining pre-trained genomic language models with graph structure generation for phylogenetic inference. This approach, which received praise from **all reviewers**, is expected to significantly impact the community and attract further exploration **(QAeZ, XKPL, 4WGb, MKto)**. 

Our method achieves joint optimization of tree topology and branch lengths without typical constraints and does not require pre-generated topologies **(QAeZ, XKPL)**. It demonstrates state-of-the-art performance on real-world datasets **(QAeZ, XKPL, MKto)**. Extensive case studies and ablation experiments validate the robustness of our method and the diversity of the generated topologies **(QAeZ, XKPL)**. Additionally, our approach shows higher computational efficiency than baseline methods **(QAeZ)**. These results highlight our method's practicality and underscore its broad theoretical and applied potential **(QAeZ, XKPL)**.

We thank all reviewers for their insightful comments and for addressing some common concerns regarding the limitations and performance of our model:

**1. Limitations:**

While our model demonstrates outstanding performance on standard benchmarks, it may benefit from using more expressive Q(z) distributions or incorporating prior constraints to better capture complex dependencies and interactions in the latent space. Additionally, although the Neighbor-Joining (NJ) algorithm is effective for iterative tree construction, it is computationally intensive. We are exploring efficient data structures and parallel processing techniques to address this bottleneck. Furthermore, our model has primarily been applied to genomic data, and further research is needed to extend its applicability to diverse biological data, such as protein and single-cell data.

**2. Runtime Comparison:**

Despite these limitations, our model has performed exceptionally well in most evaluations and offers significant advantages in runtime and memory usage on the DS1 dataset:

|  | MrBayes | GeoPhy | PhyloGFN| ARTree | Ours |
| --- | --- | --- | --- | --- | --- |
| Runtime (H) | 22h46m | 8h10m | 20h40m | 62h21min | 6h53min |
| Memory (MB) | — | 1450.93 | 2341.80 | 2040.50 | 1051.11 |


**3. Source Code Availability:** 

Due to time constraints, we have not yet fully organized all the code. The current version could be more organized, but we continue to refine it to meet the reviewers' needs. We have sent an anonymized link to the AC for reference.

**4. Thorough Ablation Study:**

We chose DS1 for our primary experiments because it is representative and commonly used in the field, ensuring comparability with other recent studies. Our original experiment design is both adequate and reasonable. In response to further curiosity and to provide additional details, we have included ablation results for additional datasets in the attached file. These are not to compensate for an insufficient design but to demonstrate the robustness of our method more comprehensively. If necessary, we can further supplement with more datasets. 

Thank you again for your continued attention and valuable insights.

Meta-review: The reviewers find the novelty of the method and its performance as notable strengths. (Minor) weaknesses include: the code being not yet mature enough to be released (it was only made available to the Area Chairs), missing comparisons to recent methods and otherwise limited empirical experiments, and some need for improvement in the presentation. 

Overall, the recommendation is unambiguously in favor of accepting.

(As a final detailed comment: The terminology around "(pre-) aligned" vs "unaligned" sequences should be explicitly defined. To someone familiar with phylogenetic terminology, aligned sequences suggest multiple sequence alignment, which, correctly done, should in principle be more informative about phylogenetic relationships than unaligned sequences as one reviewer points out.)
Decision: Accept (poster)