Title: Evolutionary Reasoning Does Not Arise in Standard Usage of Protein Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Protein Language Models

Reasoning

Sequence Modeling

Evolution
Abstract: Protein language models (PLMs) are often assumed to capture evolutionary information by training on large protein sequence datasets. Yet it remains unclear whether PLMs can reason about evolution—that is, infer evolutionary relationships between sequences. We test this capability by evaluating whether standard PLM usage, frozen or fine-tuned embeddings with distance-based comparison, supports evolutionary reasoning. Existing PLMs consistently fail to recover phylogenetic structure, despite strong performance on sequence-level tasks such as masked-token and contact prediction. We present Phyla, a hybrid state-space and transformer model that jointly processes multiple sequences and is trained using a tree-based objective across 3,000 phylogenies spanning diverse protein families. Phyla outperforms the next-best PLM by 9\% on tree reconstruction and 23\% on taxonomic clustering while remaining alignment- and guide-tree-free. Although classical alignment pipelines achieve higher absolute accuracy, Phyla narrows the gap and achieves markedly lower end-to-end runtime. Applied to real data, Phyla reconstructs biologically accurate clades in the tree of life and resolves genome-scale relationships among Mycobacterium tuberculosis isolates. These findings suggest that, under standard usage, evolutionary reasoning does not reliably emerge from large-scale sequence modeling. Instead, Phyla shows that models trained with phylogenetic supervision can reason about evolution more effectively, offering a biologically grounded path toward evolutionary foundation models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis model proposed an evaluation benchmark to exam the existing PLMs on phylogenetic reconstruction and a new model PHYLA for evolutionary reasoning. The new model rely on a hybrid state-space and attention-based architecture to capture both long-range dependency and local context. It can process multiple sequences in parallel to discover inter and intra sequence relationship. The model is trained on a tree loss function to reflect evolutionary distance and phylogenetic structure in the embedding. \nCompared to existing PLMs, who often failed on phylogenetic reconstruction, PHYLA achieved good accuracy and efficiency on phylogenetic inference on benchmark datasets.\nThis method is effective on evolutionary inference without sequence alignment and guide tree.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper reveals an important question that the evolutionary reasoning is different from sequence modeling. Evaluation on existing PLMs shows that the limitation of large language models on specific biological tasks. The impact is huge for comparative studies.\nThe design of model combines state-space module with sparsified attention mechanism. Effectively capture both intra and inter sequence relationships. The tree loss enables the model to learn evolutionary relationships directly from sequences, which is quite novel and biologically significant.   \nThe model can process multiple sequence semutaniousy, infer phylogenetic tree without multiple sequence alignment and guide tree, greatly improved inference efficiency. \nThe experiment is comprehensive, covers multiple tasks, include multiple benchmark datasets.\nSome weaknesses include that this study does not include traditional phylogenetic tree inference, such as RAxML, Fasttree, etc. It is unclear about the model robustness and effectiveness on datasets with high sequence diversity. It would be helpful to include simulated datasets with artificially designed sequence diversity to simulate sequences with different evolutionary distance.\n\nQUESTIONS:\n1. It would be interesting to discuss about why PLMs can’t even beat hamming distance on phylogenetic inference. That means PLMs does not capture basic evolutionary information from the sequences. \n2. How does the model performance compare to traditional phylogenetic inference methods? On both accuracy, running time and memory usage.\n\nLIMITATIONS:\nOverall, this paper is a solid study on a biologically significant problem, phylogenetic inference. This study not only identified an astonishing fact that the existing PLMs are not applicable on evolutionary studies, but also proposed a solid solution to resolve this problem by introducing both intra and inter sequence inference blocks, tree loss to capture tree information directly from the model. \nThe limitation of this work mainly include leak of traditional phylogenetic inference baselines, model robustness analysis, and theoretical analysis on the model effectiveness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work's main contribution is a novel tree-based loss function for (pre-)training PLMs, which is crucially distinct from MLM. The work trains a new multi-sequence model called PHYLA using this loss. Thanks to this loss, PHYLA's latent space better reflects true tree evolutionary distances. Using PHYLA's latent space improves performance on phylogenetic tree reconstruction, compared to using the embeddings of traditional PLMs.\n\nSTRENGTHS AND WEAKNESSES:\nStregths:\n1) The tree-based loss function is interesting and clearly distinct from MLM. It reminds me of contrastive losses which try to  to place closely related objects closer in latent space. These kinds of losses have seen success in other areas such as computer vision.\n\nWeaknesses:\n1) Firstly, the way in which PLMs are used in the tree reconstruction benchmark is by just taking the Euclidean distance in embedding space. This fact deferred to the appendix (line 607) and is quite important: none of these PLMs (e.g. ESM) have been trained to recapitulate true tree distance, so in a sense it is expected that they will not do well on phylogenetic tree reconstruction -- at least not as well as PHYLA, since PHYLA has been trained to recover quartets. There could be non-linear effects on the latent space of traditional PLMs, but if you trained a network to \"correct\" these embeddings such that they recapitulate tree distance (using, for example, your quartet-based tree loss, or by other means such as matching to true tree distances), the corrected embeddings might do just as well as PHYLA on tree reconstruction. I thus find the presented results are unfairly harsh with current PLMs.\n2) Secondly -- and also deferred to the appendix (line 585) -- is the fact that the model is trained using (normalized) Hamming distances instead of true tree distances, or more reallistically, distances obtained from reconstructed tree using software such as FastTree. The manuscript says that \"To create trees we would have to run an algorithm on top of the MSA which could introduce biases while training\". As someone with experience in statistical phylogenetics, I find this statement alarming and reversed: I would argue that using Hamming distances is biased whereas using tree reconstruction software such as FastTree or IQTree will give more accurate and less biased results. For example, Hamming distances disregard the charge of the residues, and in general are a non-linear transformation of true tree distance. Instead, CTMC-based software such as FastTree uses amino acid substitution rate matrices which account for residue charge and attempt to reconstruct true tree distance. Therefore, it appears to me that PHYLA is essentially learning to \"imitate\" the Hamming distance quartet resolution criteria, which seems an uninteresting task (compared to MLM, for example) via a continuous relaxation.\n3) While PHYLA does not require aligned sequences, the fact that it was trained exclusively on MSAs (Section A.1.2: training details) makes me wonder why it would even generalize to unaligned sequences.\n4) I didn't find any discussion on how the training set (OpenProteinSet) overlaps with the test set (TreeBase and TreeFam). If there is overlap, then there is test set leakage, and since PHYLA is tarined to explicitly recover quartets it would be unsurprising that it did well on the tree reconstruction task.\n\nQUESTIONS:\n1) In the appendix, line 586, why do you state that tree reconstruction could introduce biases during training? You are using the Hamming distance, is that not \"more biased\"? To my understanding, this essentially contradicts the whole body of work on statistical phylogenetics.\n2) Why do you use the Euclidean distance on the embedding space of traditional PLMs? Might fitting a network to \"correct\" potential \"nonlinearities\" in these embeddings make them perform well on the phylogenetic tree reconstruction task? This would put into question the claim that traditional PLMs are not well positioned to perform evolutionary reasoning.\n3) Is it possible that there is test set leakage in the phylogenetic tree reconstruction task? I.e., how do OpenProteinSet and TreeBase/TreeFam overlap?\n4) If PHYLA was trained on aligned sequences, why is there any expectation that it would generalize to unaligned sequences?\n\nLIMITATIONS:\nAs mentioned in my review, many important facts -- specifically, limitations -- are deferred to the appendix and not clearly communicated and discussed upfront in the main text.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work argues that current Protein Language Models (PLMs) are incapable of performing proper phylogenetic analysis. The authors address this limitation by proposing an approach that incorporates inter-sequence elements (in addition to the usual intra-sequence elements) in both the model architecture and the training objective. The authors also propose a benchmark to evaluate the level of phylogenetic analysis that different PLMs are capable of. The proposed approach is compared against different PLMs and the Hamming Distance in this benchmark and functional prediction using the ProteinGym benchmark.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths *\n\n- The idea of informing protein embedding learning by including context from other proteins, using inter-protein modules in this work, is interesting and is shown to enhance PHYLA performance in the presented ablation studies \n\n- Introduction of an interesting benchmark to assess the performance of PLMs at phylogenetic analysis. \n\n*Weaknesses*\n\n- It is interesting to evaluate different widely used PLMs for performing phylogenetic analysis. However, it is unclear how these analyses were made for the different models from reading the manuscript. I suspect that the experiments were based on the distances between protein representations, and if that is the case, there is no reason to assume that those distances should reflect phylogenetic structures.  \n\n- The authors attempt to make a distinction between evolutionary modeling and evolutionary reasoning. I believe that a more formal definition of the latter should be included, given that one of the main objectives of this manuscript is to show that PHYLA, in contrast to other PLMs, is capable of evolutionary reasoning. \n\n- Fairness of valuation is lacking: no comparison with SOTA Phylogenetic models was provided, the experiment's results on clustering and tree reconstruction compare a trained Phyla model with PLMs on a zero-shot task.  \n\n- It's unclear to me if Protein Language Models should be evaluated on their ability to perform phylogenetic analysis without further fine-tuning. \n\n- Lack of reference/explanation on some statements. For example, lines 27 to 30 state that PLMs are thought of as being capable of evolutionary reasoning without offering any reference.\n\n- A clearer explanation of the model architecture is needed. Currently, Figure 2 seems to be the best explanation of the model and should be accompanied by a textual description. As it stands now, this work cannot be reproduced. \n\n- No information on Training data in the main Manuscript. Supplementary material is the first place where it can be found that the training is informed by distances that are derived from an MSA dataset. \n\n- Line 129 claims that PHYLA does not depend on MSA; however, the supplementary material says that it was trained on high-quality multiple sequence alignments (MSAs).\n\nQUESTIONS:\n- The authors state that PHYLA is the first PLM designed to process multiple sequences concurrently. In the training schema, we see that the loss depends on sequence quartets. Does that mean that PHYLA process always involves 4 proteins? Or does it process more than that? \n\n- The final embedding of a protein depends on the context of other protein sequences. If this is the case, protein embeddings and distances between those embeddings are not deterministic. How are the test sequences fed to the model to obtain the representations and the distances used in the experiments to assess PHYLA? \n\n- Are all the predictions of the competing PLMs obtained similarly to PHYLA? If that is the case, PHYLA was explicitly trained to reflect phylogenetic trees by means of the representation distances. Given that function prediction depends on tree reconstruction, this is a clear disadvantage for the competitors in this task. \n\n- How does PHYLA perform on other standard PLM tasks? (Enzyme Commission Number Prediction, Gene Ontology Term Prediction, Remote homology detection)\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper challenges the assumption that protein language models (PLMs), which are trained on large protein sequence datasets, can inherently perform evolutionary reasoning. While PLMs excel at tasks like masked token prediction, they fail to infer phylogenetic relationships between sequences. To address this, the authors introduce a new model called PHYLA, which combines state-space and transformer architectures and is trained using a tree-based objective across thousands of phylogenies. PHYLA significantly outperforms existing models in reconstructing evolutionary trees and clustering sequences by taxonomy. Unlike traditional PLMs, PHYLA is explicitly designed to reason across multiple sequences, capturing evolutionary structure without relying on sequence alignment. The study concludes that true evolutionary reasoning requires specialized training and architecture, not just large-scale sequence modeling.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper makes a compelling case that current protein language models (PLMs) conflate evolutionary modeling with evolutionary reasoning, and it clearly defines the difference.\n\n2. Introduces the first benchmark specifically designed to evaluate evolutionary reasoning over protein sequences, enabling rigorous assessment.\n\n3. Innovative Architecture: Proposes PHYLA, a hybrid state-space and transformer model that processes multiple sequences jointly—unlike traditional PLMs.\n\n4. PHYLA outperforms much larger models (e.g., ESM3, ProGen2) in phylogenetic tree reconstruction and taxonomic clustering, despite having far fewer parameters (24M).\n\n5. Demonstrates that PHYLA’s embeddings align with known biological and functional structures, including real-world applications like reconstructing the tree of life.\n\n6. Achieves high accuracy without relying on multiple sequence alignment or guide trees, and is computationally efficient compared to traditional methods.\n\nWeaknesses\n1. While PHYLA excels at evolutionary reasoning, its performance on broader protein modeling tasks (e.g., generative tasks) is not deeply explored.\n\n2. The evolutionary reasoning benchmark, though novel, may not capture the full complexity of evolutionary dynamics across all protein families.\n\n3. The paper does not deeply address how interpretable PHYLA’s embeddings are in terms of biological insight beyond tree topology.\n\n4. PHYLA’s success relies on curated phylogenetic trees for training, which may limit its applicability in domains lacking such data.\n\n5.  While outperforming PLMs in evolutionary reasoning, PHYLA’s performance on standard PLM tasks like masked token prediction is not benchmarked.\n\nQUESTIONS:\nrefer to Strengths And Weaknesses\n\nLIMITATIONS:\nrefer to Strengths And Weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}