Lost in Tokenization: Context as the Key to Unlocking Biomolecular Understanding in Scientific LLMs
Authors: Kai Zhuang
zhang jiawei
Yumou Liu
Hanqun Cao
Chunbin Gu
Mengdi Liu
Zhangyang Gao
Zitong Jerry Wang
Xuanhe Zhou
Pheng-Ann Heng
Lijun Wu
Conghui He
Cheng Tan
Venue: ICLR 2026 Poster (2026)
Keywords: Biomolecular learning
Protein sequence
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Scientific Large Language Models (Sci-LLMs) have emerged as a promising frontier for accelerating biological discovery. However, these models face a fundamental challenge when processing raw biomolecular sequences: the tokenization dilemma. Whether treating sequences as a specialized language, risking the loss of functional motif information, or as a separate modality, introducing formidable alignment challenges, current strategies fundamentally limit their reasoning capacity. We challenge this sequence-centric paradigm by positing that a more effective strategy is to provide Sci-LLMs with high-level structured context derived from established bioinformatics tools, thereby bypassing the need to interpret low-level noisy sequence data directly. Through a systematic comparison of leading Sci-LLMs on biological reasoning tasks, we tested three input modes: sequence-only, context-only, and a combination of both. Our findings are striking: the context-only approach consistently and substantially outperforms all other modes. Even more revealing, the inclusion of the raw sequence alongside its high-level context consistently degrades performance, indicating that raw sequences act as informational noise, even for models with specialized tokenization schemes. These results suggest that the primary strength of existing Sci-LLMs lies not in their nascent ability to interpret biomolecular syntax from scratch, but in their profound capacity for reasoning over structured, human-readable knowledge. Therefore, we argue for reframing Sci-LLMs not as sequence decoders, but as powerful reasoning engines over expert knowledge. This work lays the foundation for a new class of hybrid scientific AI agents, repositioning the developmental focus from direct sequence interpretation towards high-level knowledge synthesis.

Published as a conference paper at ICLR 2026
LOST IN TOKENIZATION: CONTEXT AS THE KEY TO UNLOCKING BIOMOLECULAR UNDERSTANDING IN SCIENTIFIC LLMS
Kai Zhuangg.2.3. Jiawei "hang";Yumin Liu Hanquu Cao Chunbin Gu Mengdi Liu Zhangyang Gao Zitong Jerry Wang Xuanhe Zhou?, Pheng-Ann Heng Lijun Wu!, Conghui He Cheng Tan-
SSanggaai Artificial Intelligence Laboratory WWestlake University SShnghai Innovation Institute *Shanghai Jiaotong University "The Chinese University of Hong Kong
"Institute of Computing Technology, Chinese Academy of Sciences
ABSTRACT
Scientific Large Language Models (Sci-LLMs) have emerged as a promising fron- tier for accelerating biological discovery. However, these models face funda- mental challenge when processing raw biomolecular sequences: the tokeniza- tion dilemma. Whether treating sequences the loss of functional motif information, or as a separate modality, introducing formidable alignment challenges, current strategies fundamentally limit their rea- soning capacity. We challenge this sequence-centre paradigm by positing that more effective strategy is provide Sci-LLMs with high-level structured con- text derived from established bioinformatics tools, thereby bypassing the need to interpret low-level noisy sequence data directly. Through systematic compar- ison of leading Sci-LLMs on biological reasoning tasks, we tested three input modes: sequence-only, context-only, and combination of both. Our findings are striking: the context-only approach consistently and substantially outperforms all other modes. Even more revealing, the inclusion of the raw sequence alongside its high-level context consistently degrades performance, indicating that raw se- quences act as informational noise, even for models with specialized tokenization schemes. These results suggest that the primary strength of existing Sci-LLMs lies not in their nascent ability to interpret biomolecular syntax from scratch, but in their profound capacity for reasoning over structured, human-readable knowl- edge. Therefore, we argue for reframing Sci-LLMs not as sequence decoders, but powerful reasoning engines tion for a new class of hy brid scientific AI agents, repositioning the developmental focus from direct sequence interpretation towards high-level knowledge synthesis.
specialized language. risking
expert knowledge. This work lays the founda-
INTRODUCTION
The convergence of artificial intelligence and the life sciences has given rise to new class of pow- erful tools: Scientific Large Language Models (Sci-LLMs), Built on Transformer architectures (c.g.
BERT, GPT) that have revolutionized natural language processing (Ferruz et al. (022). these models hold immense promise for accelerating biological discovery (Karpatne et al. (025).. From predicting protein function (Brandes al. 2023) to designing novel therapeutics (Hie et al. 2024). Sci-LLMs such as Intern-SI (Bai et al. 2025). Evolla (Zhou et al. 2025). and NatureLM (Xia et al.) (0225)
are being developed to interpret the complex "language of life encoded in DNA, RNA, and protein sequences (Taylor et al. 2022). Early efforts have demonstrated their potential, sparking visions of an Al-driven future for scientific research. This burgeoning field has largely coalesced around two primary strategies for integrating biomolecular data (Hwang et al. 2024). The first "sequence- as-language" approach treats sequences as specialized form of language, extending the model's
"Equal contribution.
1 "orresponning author.

---

Published as a conference paper at ICLR 2026
vocabulary include individual amino acids or nucleotides and pre-training it on vast corpora of se- approach, inspired by multimodal learn- distinct modality, employing specialized encoder (c.g.,
quence and text data. The second segquence-asmmodeee
pre-trained biological foundation model like ESM (Lin et al., 2023) and Evo (Fallahpour et al., (025) ) to gen- erate rich embeddings that are then aligned with and injected into the language model's input space, allowing LLMs to reason over high-level features of the sequence provided by the encoder, rather
ing, treats sequences
than the raw sequence itself (Abdine al. 2024 Liu et al. 2024b de Almeida et al. 2025).
While both paradigms have shown progress, they share a fundamental, yet often overlooked, vul- nerability that we term the tokenization dilemma. In the crcquence-as-languee kenization process is often too granular (Rao et al. [2000] Brandes et al., 2023). By breaking down sequences into their atomic components- single amino acids or nucleotides it destroys the very structures that carry biological meaning: functional motifs, domains, and regulatory elements (Fer- ruz et al., 2022). The model is consequently forced into the complicated task of re-learning these fundamental "words" of biology from stream of disconnected "letters," process that is both inef- ficient and struggles with generalization. Conversely, the "sequence-as-modalii"" paradigm, while preserving structural information within its high- figh-fidelity embeddings, introduces formidable align- ment challenge (Hu et al. 2025). The hidden space learned by a bioinformatics encoder is governed by the principles of evolution and biophysics, world of alpha-helices and selective pressure. The hidden space of an LLM,, however, is shaped by human language. Bridging this profound semantic gap between the two modalities is non-trivial task, and imperfect alignment can introduce ambi- guity or even misinterpretation. limiting the model's ability to ground its reasoning accurately in the underlying biological reality, We are, essence, asking these models to perform a task for which they are ill-equipped: they are becoming lost in tokenization.
paradigm, the to-
(a) sasiencceas-Language
(b) Scquence-as-Modalit
(c) Context-driven
ACTGTAA...
MKTTM.
Text
Domain Domain-specific
MEME
Specialized encoder
BLAST
Tokenizer
Plam
& GA
T
High information density!
Functional motifs are destroyed?
Modality gap exists?
Natively aligned language!
Figure 1: Paradigms for integrating biomolecular sequences into Sci-LLMss (a) The sequence-as- language approach, tokenization fragments sequences into atomic symbols, potentially destroying functional motifs. (b) The grquence-as-modalee approach preserves structure via specialized en- coders but suffers from semantic misalignment with natural language. (c) The context-driven ap- proach leverages bioinformatics tools to provide high-density, natively aligned textual context.
In this work, we challenge the prevailing sequence-centric view and propose an alternative, more effective paradigm overcome the tokenization dilemma. We hypothesize that rather than forcing LLMs to directly decipher the noisy, low-level syntax of raw biomolecular sequences, we should leverage their core strength: reasoning over high-level, structured knowledge. Decades of accumu- lated biological wisdom are embedded in expert tools and databases from BLAST for sequence homology to Pfam for conserved domains and Gene Ontology for functional terms. As shown in Figure D we posit that these resources can be transformed into information-rice -ich textual context for the LLM. This "context"' presented as human-readable text, is not only information-dern hav- ing already distilled functional insights from the raw sequence, but is also natively aligned with the LLM's linguistic domain, entirely circumventing the tokenization dilemma.
We conduct systematic empirical study across representative set of state-of-ther Sci-LLMs. Surprisingly, we observe that adding the raw sequence to an already informative context often de- grades performance: the sequence acts form of "informational noise" that confuses otherwise
2

---

Published as a conference paper at ICLR 2026
well-informee model. When both sequence and context given, the sequence introduces mislead- ing signals that reduce accuracy, suggesting that the true power of current Sci-LLMs lies not their abiliity to serve de novo sequence interpreters, but sophisticated reasoning engines over inte- grated domain knowledge. Models that are fed high-level biological context can make insightful connections and generalizations whereas those fed only raw sequences struggle to draw any infer- ence until they essentially "learn biology" from scratch.
2 RELATED WORK
2.1 FOUNDATION MODELS IN BIOLOGICAL REPRESENTATION
Foundation models for biological sequences have made rapid strides in representation learning. In the protein domain, large language models like ProtBERT (Elnaggar al.. 202T and the ESM ries (Lin et al. [023] Hayes et al. 2025) are trained on massive sequence corpora, capturing signals of evolutionary conservation, structural motifs, and residue co-variation that enable downstream generalization. On the nucleotide side, models such as DNABERT (Ji et al. 2021 and the more recent Nucleotide Transformer (Dalla-Torre et al.] 2025) apply k-mer tokenization or other sub- word strategies to genome-scale data, achieving high accuracy in identifying promoters, splice sites, and transcription factor binding locations. Multi-species genome models like DNABERT-2 (Zhou et al., 2024) further improve efficiency by replacing k-mers with Byte-Pair Encoding to accom- modate longer input sequences. Meanwhile, specialized transformer architectures have extended context lengths to capture distal regulatory interactions and boost gene expression prediction (Avsec) jet al., 2021] Nguyen et al.] 2023; 2024). Despite their powerful representational capacity, these bio-sequence foundation models largely act as "black boxes". Their internal embeddings are high- dimensional and not straightforwardly mapped to numan-interpreeabbe biological units like motifs, domains, or pathways, making difficult extract mechanistic insight.
2.2 SCIENTIFIC LARGE LANGUAGE MODELS
Large language models tailored to scientific domains (Sci-LLMs) have rapidly advanced, extending the success of general LLMs into tasks like protein or molecule design, genomic analysis, and scien- tific reasoning. Galactica Taylor et al. 2022), a 120-billion partmmeee model trained on a corpus of papers and knowledge bases, was introduced to store and reason over scientific knowledge. Domain- focused sequence models have also emerged: NatureLM (Xia al., (025). for example, a unified sequence-based model pre-trained across proteins, nucleic acids and small molecules. Likewise, Intern-S1 (Bai et al.] 2025) is a recent large multimodal MoE model (28B activated parameters) with specialized tokenization and encoders for different scientific modalities. In this work, we focus on biomolecular understanding as a representative scientific challenge: information is inherently encoded in sequences (genes proteins), which can be expressed in textual form as distinct modality, making it an ideal testbed for probing how well Sci-LLMs integrate domain knowledge and whether they truly understand biological sequences.
2.3 EXISTING STRATEGIES IN BRIDGING SEQUENCES AND LANGUAGE
Sci-LLMs have adopted several strategies to bridge low-level biomolecular sequences with higher- level reasoning and knowledge, One common approach is treating sequences as specialized lan- guage. Models like NatureLM (Xia et al.) 2025) and Intern-S1 (Bai al., 2025) ingest raw or tokenized sequences directly as input, training on vast datasets of sequences annotated with text so that the model learns joint representations. Another emerging strategy is treating sequences as a separate modality. For example, EvoLLaMA (Liu et al. 2024b) incorporates a protein structure en- coder and a sequence encoder alongside an L.MM to enable multimodal protein question-answering. and Evolla (Zhou et al. 2025) employs SaProt (Su et al. (2024a) as the structure encoder. BioRea- son (Fallahpour et al. 2025) similarly couples a frozen DNA foundation model Evo (Nguyen et al.) 2024) with language model Qwen3 (Yang et al.) 2025). so that genomic sequences are converted into contextual embeddings which the LLM can reason over in natural language. A third line of work explores agent-based or tool-augmented approaches. Rather than having single model di- rectly analyze sequences, the LLM is equipped with the ability to call external tools or databases as needed. Notable examples include GeneAgent (Wang et al. 2025), which self-verifies for gene-set
3

---

Published as a conference paper at ICLR 2026
analysis using domain databases, and ChemCrow (Bran al. 2023), which multi-step chemistry tasks by invoking suite of expert tools. While all these strategies have pushed the frontier of scientific AI (Huang al.] 2025), it remains unclear how much of the success in Sci-LLMs comes from genuine reasoning over raw sequences. In this work, we adopt a deliberately context-driven
agent plan
the model with only high-level, structured annotations of the sequence. By comparing this setup to one where the model sees the raw sequence, we can assess how and when sequence information truly adds value.
PRELIMINARIES
3.1 THE BIOMOLECULAR UNDERSTANDING TASK
Let S be the space of all possible biomolecular sequences (e.g., protein, RNA, DNA, and small molecules), Q be the space of natural language questions about sequence, and A be the space of plausible natural language answers. The general task is to learn a function f S x Q - A that maps sequence S and question
Q to factually correct and relevant answer E A.
A Scientific LLM, denoted as M,, aims to approximate this function by learning set of optimal parameters 0. The generation of an answer be expressed
a= MM q; 0)
The fundamental distinction between the paradigms we investigate lies in how the sequence 8 and question y represented and processed by the model M.
3.2
DOQUENCE-AS-LANGUUEE
This approach, utilized by models such as NatureLM (Xia et al. 2025) and Intern-SI (Bai et al. 2025 treats biomolecular sequence as specialized string of text. Let Teeg be a tokenizer that maps a sequence into series of tokens from biological vocabulary, Voia. and let Trest be standard tokenizer for a natural language question q with vocabulary Vient. The model operates on an extended vocabulary Vezt = Vient (J Viio. The input to the LLM,, X Kimput. is formed by the concatenation the tokenized question and sequence:
Xinpot [9:
The model M then processes this unified token sequence autoregressively generate the answer at
[a]
P(als,g) = II
P(ax agck, Xinpat: 0)
(3)
1=1
the weak representation comes from the low-level tokenization atomizes the sequence, destroying the hierarchical structures of functional motifs. The model receives a high-dimensional but low-information-densitt signal, from which must re-learn the fundamental grammar of biology, notoriously difficult and data-intensive task.
introduces the first horn of the tokenization dilemma:
3.3 SEQUENCE-AS- SOUUENCE-ASSMODALITYY
Inspired by successes in vision-language modeling, this paradign--emlloeed by models like Evolla [Zhou et al.) 2025) and BioReason (Fallabpour et al. 2025) the biomolecular se- quence distinct, non-textual modality. A specialized, pre-trained biomolecular encoder, Eoso S + RLxd first transforms the sequence into sequence of rich, contextualized em- beddings. An alignment module, Aulign. then projects these biological embeddings into the LLM's semantic space, creating an aligned sequence representation Ealigned seq E R kxxx xd The final input to LLM is structured combination of the embedded text and the aligned sequence embeddings:
inppat [Ti=a(q); Eatigned
While this approach preserves the sequence's structural integrity, it introduces the second horn of the tokenization dilemma: the challenge of semantic misalignment. The semantic space of Esio is governed by the principles of biophysics and evolution, whereas the LLM's space is structured by human linguistics and logic. The alignment module Aatigo must learn to bridge this profound semantic gap. Any imperfection in this translation can inject ambiguity noise.

---

Published as a conference paper at ICLR 2026
4 THE CONTEXT-DRIVEN APPROACH
In this work, we propose and investigate third paradigm that circumvents the tokenization dilemma entirely. This approach posits that the most effective way to leverage an LLM is to provide it with what it processes best: high-quality, human-readable text.
We define a set of established bioinformatics tools as a function C - Teottext where Teonext the space of structured, human-readable textual descriptions. This function transforms raw sequence 8 into a high-level context = C(s). The model's input deliberately omits the raw sequence 8:
X input (Teex(g) (e)((c)
The model approximates the answer's probability by conditioning only high-level knowledge:
|a]
p) P(alc,g)
II Plaxlaeck Xinpus 0)
This paradigm reframes the task from one of low-level sequence interpretation to one of high-level knowledge synthesis. The context is information-dent and natively aligned with the LLM's nat- ural language space, shifting the model's role from low-level sequence interpretation to high-level knowledge synthesis and reasoning.
Specifically, we design a pipeline to generate and structure the context for any given protein se- quence. First, we generate comprehensive functional profile by executing multi-source toolchain. InterProScan (Jones et al. 2014) is used to identify conserved domains and motifs based on the se- quence's 's intrinsic features, while BLASTp (Altschul al.) 1990) retrieves annotations from close homologs in the Swiss-Prot database (Boeckmann et al. 2003). For novel orphan sequences lacking hits from these tools, we use the tri-modal retrieval model ProTrek (Su al. (222b)) fallback to generate a basic semantic description. The outputs from these tools are then integrated into a final context using an empirically-driven hierarchical strategy. The details are the Appendix
Structured Prompt for Context- Drivenn Reasoning
You senior systems biologist. Analyze the input information to answer the given question.
Question: User'' Question Text
Conserved Domains (from Pfam)
[POR EACH Pfam entry Ffam]:
[the description detected conserved domains/motise
Functional Annotations (from Homology via BLASTp)
GO terms associated with the hcmelog:
{the GO terms of the homolog) Fallback Semantic Analysis (from ProTrek): [ONLY if no homology or domain data is available]
[FOR EACH ProTrek entry In Protrek];
[the description of Protrek)
Answer: (answer)
central concern in fair evaluation is the prevention of information leakage. Our context-driven approach is explicitly designed to avoid label leakage along two complementary axes:
Intrinsic analysis rather than identity lookup. We employ InterProScan to detect conserved domains and motifs intrinsic to the query sequence. This constitutes an ab initio. feature- based analysis grounded in domain knowledge bases, not in annotation records of the query protein itself. Consequently, even for genuinely novel proteins, recognizable elements such as a kinase domain be identified without ground-truth labels.
5

---

Published as a conference paper at ICLR 2026
Homology-basee inference rather than direct annotation matching. When using BLASTp. restrict our context-driven approach to reading GO annotations from the homologous sequences, record. This reflects standard bioinformatics practice:
rather than from the query protein's predicting the function of unknown sequences by analogy to characterized homologs rather than simply retrieving pre-annotated answers.
THE TOKENIZATION DILEMMA IN PRACTICE
5.1 THE PRIMACY CONTEXT OVER SEQUENCE
Following a standardized protocol inspired by Evolla (Zhou et al. (025). our benchmark focuses on three fundamental aspects of protein biology: molecular function, metabolic pathway involvement, and subcellular localization. For each protein in our test set, we generated queries corresponding to these categories (e.g., "What is the function of this protein?"). To ensure a set of factually grounded and verifiable ground truths, question was only included if its corresponding annotation field was explicitly present in the source database entry, from which the answer was directly excerpted. Per- formance was quantified using an automated pipeline, leveraging general- expert judge, a metric we term the LLM-Score. A detailed description of the dataset construction, evaluation protocol, and prompt design is provided in Appendices B and C formance of both specialized Sci-LLMs and leading general
L.M as an
We evaluate the per- general-purpose purpose LLMs across three distinct
input configurations: (i) Sequence-Only where the model receives only the raw protein sequence; (ii) Sequence Context, a combined input; (iii) Context-Only. where the model receives only the high-level context. The results presented in Table
Table 1: Comparison of performance across specialized Sci-LLMs and general- purpose LLMs on protein QÅ benchmark. v indicates that the corresponding input modality provided to the model. Results are reported on three task-specific subsets subsets-Function (Func.), Pathway (Path.), and Subcellular Location (Sub. Loc.) well the overall average (All). The best score for each model is underlined, and the overall best performance across all models highlighted in bold.
Model
Sequence
Context
Func.
Path.
Sub. Loc.
All
Specialized Sci-LLMs
Intern-S1
20.57
26.56
69.75
43.33
Intern-$1
74.18
98.85
93.00
84.03
Intern-S1
76.22
97.60
95.60
86.15
Evolla
40.23
72.71
79.76
$9.93
Evolla
57.46
84,69
83.05
70.53
Evolla
65.77
83.33
81,88
74.02
NatureLM
3.58
5.52
10.45
6.82
NatureLM
/
42.33
64.25
32.30
38.86
NatureLM
44.77
51.35
32.51
39.50
General LLMs
Deepseek-v3
24.54
74.72
40.77
Deepseek-v3
77.40
91.35
94.75
86.03
Deepseck-v3
75.79
93.96
93.65
84.99
Gemini2.5 Pro
10.40
13.85
77.58
41.25
Gemini2.5 Pro
79.12
94.17
94.65
86.98
Gemini2.5 Pro
79.17
98.65
94,56
87.19
GPT-5
19.64
17.08
64.15
39.83
GPT-5
79.89
89.48
71.30
76.45
GPT-5
77.25
85.73
73.05
75.76
Qwen3-235B-A22B
13.67
19.90
37.17
39.51
Qwen3-235B-A22. A22B
76.62
96.35
94.78
85,90
Owen3-235B-A22)
75.63
92.19
94.28
84.99
Takeaway: Raw biomolecular sequences, when provided alone, offer limited utility and, when combined with context, consistently act as informational noise.
Our findings demonstrate that the Context-Only approach dramatically superior, confirming our hypothesis: I.LMs excel when they leverage their core strength of reasoning over structured

---

Published as a conference paper at ICLR 2026
knowledge. Even more revealing is the consistent performance degradation observed in the Se- quence + Context configuration. The inclusion of the raw sequence alongside its high-level summary resulted in lower score. For instance, Evolla's 's score dropped from 74.02 to 70.53, and Intern-S1's Sl's from 86.15 15 to 84.03. This counter-intuitive result provides evidence that raw sequences, in their current tokenized form, not merely redundant but actively detrimental, acting as source of noise. The models become, as we posited, "lost in tokenization". This phenomenon underscores the profound limitations of existing sequence tokenization paradigms.
5.2 DECONSTRUCTING THE DILEMMA I: THE WEAK REPRESENTATION
We visualize the embeddings of the outputs, where ground-truth classes established by clus- tering homologous proteins using MMseqs2 at a 50% sequence identity threshold. For each model, extracted the final-layer embeddings for their outputs. We employed t-SNE (Maaten & Hin- ton 2008) to project them into 2D space. The quality of the resulting functional separation was then quantified by performing clustering on the high- high-dimensional embeddings and calculating the Adjusted Rand Index (ARI) against the MMseqs2 ground-truth truth clusters. For our context- ap- proach, we generated embeddings from the structured context itself using the text embedding model Owen-embedding (Zhang et al. 2025). The results are visualized in Figure
ARI: 0.809
ARI:
ARI: 0.492
ARI: 0.958
0.690
(a) Evolla 10B
(b) A1 8B
(c) NatureLM
(d) Ours
Figure 2: The visualization of representation spaces.
Takeaway: Simple context provides vastly superior functional representation of proteins compared to both sepueme-tto-nnuaggemdddii strategies.
The visualizations confirm the weak representation horn of the tokenization dilemma. The sequence- as-language models, NatureLM (c) and Intern-S1 (b), exhibit highly disorganized latent spaces, quantitatively confirmed by their low ARI scores of 0.492 and 0.690, respectively. Evolla (a), which employs the sequence-as-mmodaiiy paradigm, demonstrates a significantly more structured representation, highlighting the benefit of using specialized sequence encoder. However, both paradigms are dramatically outperformed by our context-driven approach (d). The representation derived purely from the textual context achieves near-perfect functional separation.
5.3 DECONSTRUCTING THE DILEMMA II: THE SEMANTIC MISALIGNMENT
While the sequence-as-modalie paradigm, exemplified by Evolla, overcomes the weak represen- tation problem, it introduces more subtle yet equally critical challenge: semantic misalign- ment. The specialized encoder and the generalist LLM operate fundamentally different semantic worlds-one one governed by biophysics, the other by linguistics. We performed a layer- wise represen- tational analysis of the Evolla- 10B model, tracing the informational journey of a protein sequence from its biological embedding to its final interpretation by the language model. As shown in Fig- ureß] the initial SaProt encoder generates a well-structured latent space. As the Q-Former works to translate these biological embeddings for the LLM, the functional clarity begins to blur.
The degradation of functional representation stems not from the initial protein encoding, but from the subsequent semantic alignment to the language model.
Takeaway:
7

---

Published as a conference paper at ICLR 2026
ARE: 0.945
ARI: 0,916
ARE: 0.809
Evolla Encoder (SaProt)
Evolla Alignment (Q-Former)
Evolla Decoder (last embedding)
Figure 3:
Visualization of representation spaces at different stages within the Evolla-100 model.
5.4 DEGRADING PHENOMENON ACROSS TIME
We curated a dataset by randomly sampling about 100 proteins for each year from 1995 2024 based on the first publication year. The relationship between a protein's 's first publication year and the models' LLM-Scores is visualized in Figure
First Publication Year vs Score Seatter Plot
Score Distribution by Year
100
100
80
80
Score
Score
40
40-
20-
20-0
Trend line (slope: -0.618)
00000000000
Ours
100
100
Score
Score
40
40
20-
Trend line (slope: 41.223)
(b) Evolla
100
100
Score
60
Score
60
40-
20
Trend line (slope: 0 0.065)
0-
1995
2000
2005
2010
2015
2020
2025
1995
2000
2005
2010
2015
2026
2024
First Publication Year
(c) Intern-SI
First Publication Year
Figure 4: Analysis of model performance versus protein's 's first publication year.
For this analysis, context-driven approach employed DeepSeek-V3 V3 (Liu et al. 2024a) its base LLM to ensure a fair comparison against models with similar training data cut-off dates. Our context-driven approach (a), while maintaining the highest overall performance. exhibits slight negative trend over time due to the diminishing availability of rich, homologous information in the knowledge bases. For very recent proteins, homology- characterized relatives, leading to a sparser context and thus slightly less precise answers. The paquencc-as-mddalit model, Evolla (b), displays much more pronounced degradation. Its per- formance well-studied proteins from the 1990s and early 2000s is strong, but it deteriorates significantly for proteins discovered in the last decade. data, sourced from Swiss-Prot Release (202303), has temporal bias. Therefore, part of this decline can be attributed to its lack of exposure to the most recent protein data. However, this training bias alone does not fully account for the steepness of the collapse. The trend suggests deeper issue: Evolla's 's encoder appears to rely heavily on the dense web of evolutionary information available for older, larger protein families. When faced with recent, potentially more unique proteins that lack this
tools like BLAST find fewer well-
crucial to note that Evolla's training
8

---

Published as a conference paper at ICLR 2026
deep evolutionary context- problem exacerbated by its training data cutoff the encoder's ability to generate meaningful representations weakens considerably. The sequence-as-languaag model, Intern-S1 (c), shows performance profile that is almost entirely flat and consistently low across the entire 30-year period. This lack of temporal trend, combined with its overall poor performance, indicates fundamental failure extract meaningful biological signals from the raw sequence.
Takeaway: Our context-driven approach demonstrates superior generalization: (i) Robust- ness to sequence novelty: Unlike Sci-LLMs which suffer collapsing on proteins dissimilar training data, our context maintains high accuracy regardless of sequence identity. (ii) Temporal stability: Our approach's 's performance degrades far more gracefully over time on recently discovered proteins compared to other paradigms.
The above dual robustness confirms that reasoning over stable, high-level knowledge is a more robust foundation for Al in biology than relying on the difficult task of raw sequence interpretation.
5.5 QUANTITATIVE ANALYSIS OF REAL- WORLD EFFICIENCY
We conducted a detailed, three-way analysis to quantify the trade-offs between computational cost, time, and performance. We compare: (i) A general LLM baseline Deepseek-v3, v3,, which feeds the raw sequence to the DeepSeek-Ve V3 API and yields a performance score of 40.77, (ii) A specialized Sci-LLM Evolla, which requires high-end GPU and achieves performance score of 59.93, and (iii) our context-driven method, which uses bioinformatics tools on a CPU plus the DeepSeek- V3 API achieve a stateoof-the-at art performance score of 84.99. We conducted the analysis across both single-sequence and Batch-processing scenarios. The results, based on AWS on-demann pricing, are summarized in Table 2
with the cost estimation detailed in Appendix M
For individual queries, our method is not only dramatically more effective but also approximately 23 times cheaper and 1.3 times faster than the specialized end-to-end model. The true efficiency of our pipeline is most evident in high-throughput research. In realistic, large-scale scenario, our method is nearly 30 times cheaper and, critically, 154 times faster than Evolla on a per-sequence basis. While the direct API bascline is cheaper, its poor performance renders it unsuitable for scien- tific applications. In summary, the small cost of running established bioinformatics tools is a highly effective investment, unlocking 2x performance gain over the LLM baseline while simultane- ously outperforming and costing less than specialized end-to-end models, especially at scale.
Table 2: Comparative analysis of inference efficiency.
Method
Mode
Input to LLM
Avg. Time
Avg. Cost
Deepseck-v3
Single
Raw sequence
~30s
$0.0005
Evolla
Single
Raw sequence
90s
$0.0690
Our Method
Single
Context
70s
$0.0030
Evolla
Batch Batch
Raw sequence
20s
$0.0152
Our Method
Context
~0.13s
$0.0005
5.6
WET-LAB VALIDATION ON NOVEL SEQUENCES
We curated a set of novel functional protein sequences obtained from wet-lab experiments. They were unpublished at the time of our analysis and absent from major databases, including Swiss-Prot, thereby representing a true test of performance on unseen data. The task was formulated as a binary classification problem for two distinct protein families: Rhodopsin and PETase. For each sequence, the LLM was prompted to predict its classification. As shown in Figure 5 our method achieves 100% accuracy for Rhodopsin and 97.3% accuracy for PETase. While Evolla (Figure 6 attains reasonable 80.0% accuracy on Rhodopsin, it fails catastrophically on PETase. may caused by its training data bias. A detailed description of the wet-lab validation is shown in Appendix

---

Published as a conference paper at ICLR 2026
Rhodopsin
PETase
97.30%
Confidence Score
1.0
Confidence Score
1.0
080080000
0.8
0.4
0.4
0.2
0.2
Incorrect (1)
12
15
18
21
12
16
20
24
28
32
36
Figure 5: Sample-level performance of our context-driven method + Deepseek-v3,
Rhodopsin
PETase
63.78%
Confidence Score
1.0
X
Confidence Score
1.0
000000000000000o
XXXXX
0.8
0.8
0.4
0.2
0.2
(31)
Incorrect (19)
Incorrect (6)
0.0
0.0
12
15
18
21
12
16
20
24
32
36
Figure 6: Sample-level performance of Evolla.
6
CONCLUSION AND LIMITATION
In this work, we confronted fundamental chal- lenge at the heart of modern Sci-L.LMs: the tokenization dilemma. We demonstrated that current paradigms, whether treating biomolec- ular sequences as specialized language a distinct modality, are fundamentally handi- capped by issues of weak representation and semantic misalignment. Our central contribu- tion is the validation of third paradígm that resolves this dilemma. By shifting the focus from low-level sequence interpretation high- level knowledge synthesis, our context-driven approach entirely circumvents the tokenization problem, illustrated in the conceptual land- scape of F Notably, our approach also computationally efficient, it leverages gen- eralist LLMs without the costly retraining re- quired by domain-specifi Sci-LLMs.
Representation vs Semantic Alignment
Context-driven
Strong representation
Natively aligned Low cost
Our
Semantic Alignment
Sequence-as -aatdee-ss-LLnuugge
NatureLM
Intern-St SI
Sequende Modality
Evetia 10B
Representation (evaluated ARI)
While our findings are compelling, we ac- knowledge several limitations. For truly novel orphan proteins from unexplored regions of the protein universe, method's performance may be constrained. Furthermore, our cur- rent analysis has primarily focused proteins; although we provide some preliminary explo- ration Appendix G a more comprehensive treatment remains open for future research.
Figure 7: The trade-off landscape of representa- tion vs. semantie alignment. The x-axis quantifies the quality of the biological representation (mea- sured by ARI), while the y-axis conceptually rep- resents the degree of semantic alignment with nat- ural language. The area of each circle propor- tional to the computational cost, with larger cir- cles indicating higher computational expenses.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Hadi Abdine, Michail Chatzianastasis, Costas Bouyioukos, and Michalis Vazirgiannis. Prot2text: Multimodal protein's s function generation with gnns and transformers. In Proceedings of AAAI Conference Artificial Intelligence, volume 38, pp. 10757-10765, 2024.
Stephen Altschul, Warren Gish, Webb Miller, Eugene W Myers, and David Lipman. Basic local alignment search tool. Journal of molecular biology, 215(3):403-10,, 1990.
Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan Davis, Kara Dolinski, Selina Dwight, Janan Eppig, al. Gene ontology: tool for the unification of biology. Nature genetics, 25(1):25-29, 2000.
Ziga Avsec, Vikram Agarwal, Daniel Visentin, Joseph R Ledsam, Agnieszka Brabska-Barwinska Kyle R Taylor, Yannis Assacl, John Jumper, Pushmeet Kohli, and David R Kelley. Effective gene expression prediction from sequence by integrating long-range interactions. Nature methods, 18 116):1196-1203, 2021.
Lei Bai, Zhongrui Cai, Maosong Cao, Weihan Cao, Chiyu Chen, Haojiong Chen, Kai Chen, Pengcheng Chen, Ying Chen, Yongkang Chen, et al. Intern-sl: A scientific multimodal foun- dation model. arXiv preprint arXiv:2508.1570 2025.
Brigitte Boeckmann, Amos Bairoch, Rolf Apweiler, Marie-Claude Blatter, Anne Estreicher, Elis- abeth Gasteiger, Maria J Martin, Karine Michoud, Claire O' D'Doooa,,, Isabelle Phan, et al. The swiss-prot protein knowledgebase and its supplement trembl in 2003. Nucleic acids research, 31 (1):365-370, 2003.
Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew White, and Philippe Schwaller. Chemcrow: Augmenting large-language models with chemistry tools. arXiv preprint 05355 2023.
Nadav Brandes, Grant Goldman, Charlotte H Wang, Chun Jimmie Yc, and Vasilis Ntranos. Genome- wide prediction of disease variant effects with a deep protein language model. Nature Genetics, 55(9):1512- 1522, 2023.
Hugo Dalla- Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P de Almeida, Hassan Sirelkhatim, et al. Nucleotide transformer: building and evaluating robust foundation models for human genomics. Nature Methods, 22(2):287-299 2025.
Bernardo P de Almeida, Guillaume Richard, Hugo Dalla-Torre. Christopher Blum, Lorenz Hexemer, Priyanka Pandey, Stefan Laurent, Chandana Rajesh, Marie Lopez, Alexandre Laterre, et al. A multimodal conversational agent for dna, ma and protein tasks. Nature Machine Intelligence, pp- 1-14, 2025.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang. Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: Toward un- derstanding the language of life through self-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 44(10):7112-711 2021.
Adibvafa Fallahpour, Andrew Magnuson, Purav Gupta, Shihao Ma, Jack Naimer, Arnav Shah, Haonan Duan, Omar Ibrahim, Hani Goodarzi, Chris J. Maddison, and Bo Wang. Bioreason: Incentivizing multimodal biological reasoning within a dna-llm model, 2025. URL https:
//arxiv. //arxxvv.rrg/ab//5555.. 23579
Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. In The Twelfth International Conference on Learning Representations, 2024.
Noelia Ferruz, Steffen Schmidt, and Birte Höcker. Protgpt2 is a deep unsupervised language model for protein design. Nature communications, 13(1):4348, 2022.
11

---

Published as a conference paper at ICLR 2026
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with language model. Science, 38766736):850-855555 858, 2025.
Brian Hie, Varun R Shanker, Duo Xu, Theodora UJ Bruun, Payton Weidenbacher, Shaogeng Tang, Wesley Wu, John E Pak, and Peter S Kim. Efficient evolution of human antibodies from general protein language models. Nature biotechnology, 42(2):275-283, 2024.
Ming Hu, Chenglong Ma. Wei Li, Wanghan Xu, Jiamin Wu, Jucheng Hu, Tianbin Li, Guohang Zhuang. Jiaqi Liu, Yingzhou Lu, et al. A survey of scientific large language models: From data foundations to agent frontiers. arXiv preprint arXiv:2508.2 148, 2025.
Kexin Huang, Serena Zhang, Hanchen Wang, Yuanhao Qu, Yingzhou Lu, Yusuf Roohani, Ryan Li, Lin Qiu, Gavin Li, Junze Zhang, et al. Biomni: A general-purposs biomedical ai agent. biorxiv, 2025.
Yunha Hwang, Andre L Commman, Elizabeth H Kellogg, Sergey Ovchinnikov, and Peter R Girguis. Genomic language model predicts protein co-regulation and function. Nature communications, 15(1):2880, 2024.
Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformatics, (7(15):2112-2122 2021.
Philip Jones, David Binns, Hsin-Ya Yu Chang, Matthew Fraser, Weizhong Li, Craig McAnulla, Hamish McWilliam, John Maslen, Alex Mitchell, Gift Nuka, et al. Interproscan 5: genome-scale protein function classification. Bioinformatics, 10(9):1236-1240) 2014.
Anuj Karpatne, Aryan Deshwal, Xiaowei Jia, Wei Ding, Michael Steinbach, Aidong Zhang, and Vipin Kumar. Ai-enabled scientific revolution in the age of generative ai: second nsf workshop report. npj Artificial Intelligence, 1(1):18, 2025.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123-111 2023.
Aixin Liu, Bei Feng. Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Nuowei Liu, Changzhi Sun, Tao Ji, Junfeng Tian, Jianxin Tang, Yuanbin Wu, and Man Lan. Evol- lama: Enhancing lims' understanding of proteins via multimodal structure and sequence repre- sentations. arXiv preprint 2rXiv:2412.1100/ 2024b.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, ((NNov):57792600) 2008
Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Michael Wornow, Callum Sycch,Sykes, Stefano Massaroli, Aman Patel, Clayton Rabideau, Yoshua Bengio, et al. Hyenadna: Long-rang genomic sequence modeling at single nucleotide resolution. Advances in neural information processing systems, 36:43177-43200 2023.
Eric Nguyen, Michael Poli, Matthew G Durrant, Brian Kang, Dhruva Katrekar, David B Li, Liam Bartie, Armin W Thomas, Samuel H King, Garyk Brixi, al. Sequence modeling and design from molecular to genome scale with evo. Science, 38666723)eadd09336 2024.
Roshan Rao, Joshua Meier, Tom Sercu, Sergey Ovchinnikov, and Alexander Rives. Transformer protein language models are unsupervised structure learners. Biorxiv, pp. 2020-12, 2020.
Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure- aware vocabulary. In The Twelfth International Conference on Learning Representations, 2024a.
12

---

Published as a conference paper at ICLR 2026
Jin Su, Xibin Zhou, Xuting Zhang, and Fajie Yuan. Protrek: Navigating the protein universe through tri-modal contrastive learning. bioRxiv, pp. 2024-05, 2024b.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: large language model for science. arXiv preprini vrXiv:2211.09083 2022.
Zhizheng Wang. Qiao Jin, Chih-Hsuan Wei, Shubo Tian, Po-Ting Lai, Qingqing Zhu, Chi-Ping Day, Christina Ross, Robert Leaman, and Zhiyong Lu. Geneagent: self self-verificatioo language agent for gene-set analysis using domain databases. Nature Methods, pp. 1-9, 2025.
Yingce Xia, Peiran Jin, Shufang Xie, Liang He, Chuan Cao, Renqian Luo, Guoqing Liu, Yue Wang, Zequn Liu, Yuan-Jyue Chen, et al. Naturelm: Deciphering the language of nature for scientific discovery. arXiv e-prints, pp. arXiv-2502, 2025.
An Yang. Anfeng Li, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388. 2025.
Tianhao Yu, Haiyang Cui, Jianan Canal Li, Yunan Luo, Guangde Jiang, and Huimin Zhao. Enzyme function prediction using contrastive learning. Science, 1358--9)::33813666 2023.
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang. Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:5506.05776 2025.
Xibin Zhou, Chenchen Han, Yingqi Zhang, Jin Su, Kai Zhuang, Shiyu Jiang, Zichen Yuan, Wei Zheng, Fengyuan Dai. Yuyang Zhou, et al. Decoding the molecular language of proteins with evolla. bioRxiv, pp. 2025 01, 2025.
Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu. Dnabert-2: Efficient foundation model and benchmark for multi-species genomes. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
13

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
2
Related Work
2.1
Foundation Models in Biological Representation
2.2
Scientific Large Language Models
2.3
Existing Strategies in Bridging Sequences and Language
3
Preliminaries
The Biomolecular Understanding Task
3.2
Sequence-a--Lanuaage
3.3
Scquence-us-Modality
4
The Context-Driven Approach
5
The Tokenization Dilemma in Practice
5.1
The Primacy of Context over Sequence
5.2
Deconstructing the Dilemma The Weak Representation
5.3
Deconstructing the Dilemma II: The Semantic Misalignment
5.4
Degrading Phenomenon Across Time
5.5
Quantitative Analysis of Real- World Efficiency
9
5.6
Wet-lab Validation on Novel Sequences
Conclusion and Limitation
10
Context details
15
Context Generation
15
A Context Construction
16
Context-basee Inference
16
Dataset details
16
Protein Dataset
16
B.2 DNA Dataset
17
Evaluation Metric
18
LLM-Score for General Protein QA Tasks
18
Hierarchical Metries for EC Number Prediction
18
D Quantitative Benchmark: EC Number Prediction
19
Ablation Study
20
14

---

Published as a conference paper at ICLR 2026
Impact of Semantic Alignment on Mutation Sensitivity
21
Generalizability Across Biomolecular Types
23
Performance on Protein Function Predication
23
G.2
Performance DNA Mutation Prediction
23
H
Independence from Clustering Metries
24
QA examples
25
Comparative Analysis on Protein A6LHQ9
25
L11.1
Case 1: Context-Only (Success)
25
I.1.2
Case 2: Sequence-Only (Total Failure)
26
I.13
Case 3: Sequence Context (Noise Effect
28
Showcase of Our Method's Success Cases
29
I.2.1
Example 1: Function Prediction
................................
29
1.2.2
Example Pathway Prediction
I.2.3
Example Subcellular Location Prediction
33
Limitations
33
U.1
Wild-Type Protein Example
................................
J.2
Mutated Protein Example
................................
J.3 Summary of the Limitation
35
K Comparison of Model Performance on the Same Profein and Question
35
K.I Analysis of Model Differences)
38
L The Use of Large Language Models (LLMs)
38
M Detailed Cost Estimation
38
N Wet-lab Validation Samples
39
N.1 Prompt
39
A
CONTEXT DETAILS
Our context-driven consists of three stages: (1) generation of a multi-source evidence profile, (2)
hierarchical construction of textual context, and (3) context-based inference by LIMss.
A.
CONTEXT GENERATION
For any given biomolecular sequence (we use proteins as the running example), we first generate a comprehensive functional profile using a deliberately crafted, multi-source toolchain.
Input: A single protein FASTA sequence.
Toolbox Execution:
15

---

Published as a conference paper at ICLR 2026
1. Feature-intrinsic domain analysis: We first scan the input sequence with Inter- ProScan (Jones et al., 2014) to identify conserved domains and key motifs by integrating multiple signature libraries (e.g.. Pfam, PROSITE, SMART). This step is analysis grounded in intrinsic sequence features: even for a completely novel, unannotated protein, InterProScan can recognize known modular features. We extract textual descriptions of detected domains together with any directly linked Gene Ontology (GO) annotations (Ashburner et al. 2000)
2. Homology-base functional inference. In parallel, we run BLASTp (Altschul et al. 1990) against curated reference database (c.g.. Swiss-Prot) retrieve close homologs. We transfer GO annotations from the most similar sequences to the query. Critically, we never use the query's own (possibly unknown) labels.
3. Information integration. We merge GO evidence obtained from InterProScan (feature- intrinsic) and from BLASTp (homology-baseee to form a comprehensive functional profile.
4. Fallback mechanism. In rare cases where neither BLASTp nor InterProScan yields infor- mative signals ("orphan" sequences), we invoke ProTrek (Su et al.] 2024b) to synthesize a concise, model-based textual description that serves as minimal context.
A.2 CONTEXT CONSTRUCTION
The raw outputs from these tools can be redundant, conflicting, or noisy, especially for novel pro- teins. naive combination of all outputs is suboptimal. Therefore, based on rigorous empirical evaluation (see Ablation Study), we developed a hierarchical strategy designed to gracefully handle the spectrum of protein novelty:
1. Prioritizing High- High-Confidence Homology: Our analysis revealed that for generating list of candidate GO terms, the single most reliable source is the annotation of the top homolog found by BLAST. This strategy maximizes precision while maintaining high recall (see Table 3
2. Integrating Domain Information: Pfam motifs identified by InterProScan are added as separate, complementary source of evidence, providing structural and functional context.
3. Semantic Evidence: Our experiments showed that ProTree'ss 's semantic hits, while powerful, could introduce noise when combined with high-quality GO/Pfam data. Therefore, ProTrek's output is used fallback-it only added the context when primary sources like GO and Pfam are sparse or absent.
This empirically-drive driven, hierarchical process culminates final textual context engineered to be as factually dense and noise-free free as possible, ready for the final inference stage.
A.3 CONTEXT-BASED INFERENCE
The final stage transforms the structured biological evidence into a query that the LLM can process. The constructed context and the original user question are formatted into unified prompt using predefined template as shown in Figure The LLM's role is to act as a knowledge synthesizer. It processes the prompt, which contains series of factual statements derived from the context. This final step leverages the LLM's core strength in natural language understanding and reasoning, entirely bypassing the need for it to interpret the complex, low-level syntax of the raw biomolecular sequence itself.
B DATASET DETAILS
B. PROTEIN DATASET
To ensure comprehensive and fair evaluation of our model against Evolla, employed multi- faceted dataset strategy. This approach incorporates not only the benchmark datasets used in the original Evolla study but also dataset we have meticulously reconstructed to address specific limi- tations in their evaluation methodology. Our assessment is primarily based on the following datasets:
1. Our Benchmark Dataset
We reconstructed part of the Evolla dataset to create more rigorous and objective benchmark.
16

---

Published as a conference paper at ICLR 2026
This subset, containing proteins with less than 30% sequence identity to the training set, is critical for assessing a model's 's generalization capabilities on novel proteins. Our reconstruction guided by the following principles:
Standardized Questions: We replaced open-ended queries with a fixed set of three tar- geted question templates: "What is the function of this protein?", "What is the pathway of this protein?", and "What is the subcellular location of this protein?"
Conditional Question Generation: To ensure every question has verifiable answer, only generated a specific question if the corresponding field ("Function," " "Pathway," or "Subcellular location") was explicitly present in the protein's source database entry.
Authentic Ground Truth: Crucially, our ground truth answers are direct excerpts from the protein's 's database entry. Unlike the original dataset, did not use an LLM generate answers, thereby ensuring the objectivity and factual accuracy of the GT and creating more reliable scoring standard.
The results for our benchmark dataset are presented in Table I where we compare the perfor- mance of different methods using sequence-only context-only, and sequence+context inputs.
2. Enzyme Commission (EC) Number Dataset
To evaluate the model's performance on a specific, structured bioinformatics task, we also uti- lized the EC Number dataset from the Evolla study. This task requires the model to accurately predict the functional class of enzymes, which notation task. Testing on this dataset allows us to gauge the model's capabilities in handling classification-orrineedd protein function prediction, and is presented in Figure S
standardized and important functional an-
3. Time-Split Dataset
To investigate the model's performance over time and the impact of sequence novelty, we cu- rated dataset by randomly sampling about 100 proteins for each year from 1995 to 2024 based on the first publication year. This allows us to analyze the relationship between a protein's 's first publication year and the model's MLo-S, shown in the "Degrading Phenomenon Across Time" section (Figure 4 The time-split dataset is valuable for understanding how well the model generalizes older versus more recent proteins, and whether its performance degrades as the data becomes more novel, less represented in training, or more temporally distant from the model's training data cutoff.
4. Mol-Instruction Dataset
To further evaluate the model's ability to handle diverse protein tasks, we tested on the Mol- Instruction benchmark (Fang et al. 2024). This benchmark provides datasets for assessing molecular understanding across range of tasks. We specifically evaluated the model on three functionally distinct protein subsets: Catalytic Activity, General Function, and Protein Func- tion. The performance on these datasets, as illustrated the "Performance on Protein Function Prediction" section (Figure 10]. highlights the model's ability to accurately predict protein function even across different protein familics, demonstrating the robustness and flexibility of our context-driven approach.
B.2 DNA DATASET
To assess whether our context-driven paradigm extends beyond proteomics, we evaluated its perfor- mance on a DNA-based mechanistic reasoning task. We utilized the KEGG Disease Pathway dataset curated by BioReason (Fallahpour et al., 2025), which provides unique benchmark for connecting genomic variants to disease phenotypes through multi-step biological pathways. Each entry in the dataset consists of a reference and variant DNA sequence, the associated KEGG pathway defini- tion, and the ground-truth disease outcome. The task requires the model to reason from the mutation and its functional context to predict the correct discase.
We designed three experimental setups to investigate the impact of different data configurations on the model's performance. The first setup included only pathway-related contextual information (context-only). The second setup incorporated both the pathway context and the raw DNA sequence data (context and sequence). The third setup focused solely the DNA sequence itself (sequence- only). These configurations allowed us to evaluate the effect of using context, sequence, or both on the model's ability predict mutations.
17

---

Published as a conference paper at ICLR 2026
The KEGG dataset's comprehensive pathway data, paired with precise mutation annotations, pro- vided a solid foundation for designing these experiments. By varying the inclusion of sequence and context information, aimed to assess the model's 's ability to predict the effects of DNA mutations based on both pathway context and raw sequence data.
C EVALUATION METRIC
To conduct a comprehensive and multi-dimensional assessment of our model's performance, we designed specific evaluation metrics tailored to each of our distinct tasks.
C.I LLM-SCORE FOR GENERAL. PROTFIN QA TASKS
For the open-ended protein (e.g., BLEU, ROUGE) are inadequate for assessing the semantic accuracy and factual consistency of generated answers. To address this, we adopted an automated evaluation methodology leveraging LLM adjudicator, which term the LLM-Sccre. The core principle of this metric is to a powerful, independent third-party LLM (in this case, a DeepSeek-V3 (Liu et al. 20224) model) to score the quality of model's generated is as follows:
task, traditional metrics based on lexical overlap
against the ground truth. The evaluation process
1. Prompt Construction: We embed the generated answer and the ground truth answer into carefully designed prompt template. This prompt instructs the adjudicator LLM to act as an expert biologist and perform a holistic evaluation based on factual accuracy. The exact prompt is shown below.
LL.M-Score Adjudicator Prompt
As an expert biologist, you are assigned to check one paragraph is aligned with facts or not. You will receive some facts, and one paragraph. Score the paragraph between 0 to 100. The score should be the format of ("score"; score]
Here's s the facts:
[Ground Truth Text from Database]
Here's s the paragraph: [Generated Answer from Model to be Scored]
2. Score Generation:
The adjudicator LLM processes the prompt and returns a numerical score scale from 0 to 100, where higher score indicates that the generated answer is of higher quality and more closely aligned with the ground truth.
3. Score Extraction and Aggregation: A robust parsing function extracts the numerical score from the LLM's ss textual response. The model's final performance on the dataset is reported as the average LLM-Score across all test samples.
This approach moves beyond surface-level text matching to provide a deeper, more semantically aware assessment of the model's ability to comprehend and articulate biological knowledge.
C.2 HIERARCHICAL METRICS FOR EC NUMBER PREDICTION
The Enzyme Commission (EC) number is four-level hierarchical classification system (c.g., A proficient model should rewarded not only for predicting the exact four-digit code
A. B. B.C. C. C.D. D). but also for correctly identifying the broader functional classes higher levels of the hierarchy. Therefore, an exact-match accuracy metric at a single level is insufficient.
To capture this, implemented more nuanced, hierarchical evaluation scheme. We calculate F1-Score at each of the four functional levels.
The methodology is follows:
18

---

Published as a conference paper at ICLR 2026
Hierarchical Matching: To evaluate performance level-N. all predicted and ground truth EC numbers are truncated to their first N digits for comparison. For example, at level-3, a prediction of 1. 3.5 is considered a match for ground truth of ...3..4
Multi-Label Formulation: As single protein can be associated with multiple EC numbers, the task is treated as a multi-label classification problem.
Micro-Averaging: We compute the total number of True Positives (TP), False Positives (FP), and False Negatives (FN) by aggregating their counts over the entire test set. Global Precision, Recall, and F1-Score are then calculated from these aggregate sums.
TTP; E
Precision
ΣTP, ΣTP, + ENN
Recallmicro
(8)
Fl micro = 2 Precisionmiero Recallmicro
(9)
Precisionmicro + Recallmicro
This suite of metrics provides a comprehensive view of the model's predictive accuracy varying degrees of granularity and effectively handles the multi-label nature of the data, offering a more equitable measure of performance.
D QUANTITATIVE BENCHMARK: EC NUMBER PREDICTION
To further validate our central thesis on the primacy of context over sequence, we introduce quan- titative benchmark: EC number prediction. This task provides objective, verifiable measure of a model's ability to understand a protein's 's precise biochemical function. The hierarchical nature of EC numbers allows us evaluate performance four increasing levels of specificity (from 1-digit to 4-digit matches), with the F1-Score serving as our primary metric.
models, which include both general- purpose LLMs and CLEAN (Yu et al.) (223) (a model specifically trained for this task), and "Context-Driven" models, which leverage the contextual information as described in the main text.
We compare two categories of models: "Sequeece-
The comparative performance is visualized in Figure 8
The results presented in Figure 8 are unequivocal and offer several key insights:
In the Sequence- setting, both general- general-purpose purpose LLM DeepSeck-Ve V3 and Sci-LLM Intern-S1 perform poorly. Their Fl-scores plummet as the quired specificity increases, demonstrating their inability to decipher complex enzymatic func- tion from raw sequence data alone. This reinforces "lost in tokenization" hypothesis.
Failure of Sequence-Only LLMs:
Context Outperforms Specialization: CLEAN, model specifically trained sequences for EC prediction, establishes a respectable baseline. However, every model in the Context-Drivee category outperforms CLEAN across the first three levels of precision (1-digit, 2-digit, and 3- digit). This demonstrates that providing high-level context to general model is more effective than training specialized model sequence data for these levels.
Robustness of the Context-Driven Approach: While all models show natural decline in performance as the task becomes harder (from 1-digit to 4-digit prediction). the context- driven models exhibit much more graceful degradation. Gemini-2.5 Pro, using only context, achieves the highest Fl-score of 0.406 on the most challenging 4-digit prediction task, a score comparable to the specialized CLEAN model's performance (0.408).
In summary, this quantitative benchmark provides strong, direct evidence that high-level biological context is far more effective and reliable representation of protein function for LLMs than the raw amino acid sequence. It enables general- purppose models to excel at highly specific bioinformatics tasks without needing task-specific architectures fine-tuning.
19

---

Published as a conference paper at ICLR 2026
Sequence-Only
Context-Driven
1.0
Matched Digits
F1-Score
I-digii march
2-dig march
3 digit march
digit march
CLEAN
Inter-SS1
Deepseak-v3
Intern-St
Intem-81 (Coe-Only)
Deepseek-v3 (Cre-Only)
Gemini-2.5 Pro
(Seq+Ctx)
(Cxc-Only)
Figure 8: Performance on EC Number Prediction (F1-Score). The plot is divided into Sequence- Only models (left) and Context-Driven models (right). A clear and dramatic performance gap is vis- ible between the two groups. Context-driven approaches significantly outperform even specialized sequence-based models like CLEAN, especially at higher levels (3 and 4 digits).
E ABLATION STUDY
To dissect the contribution of each component within our framework, we conducted a comprehen- sive ablation study. The experiments were performed evaluating generalization, as it contains proteins with less than 40% sequence identity to the Evolla training set. We systematically evaluated the performance by providing different combinations of contextual information- Pfam, GO, and ProTrek-2 to the DeepSeek model. The results, summa- rized in Table reveal the individual and synergistic effects of these components.
benchmark dataset, which is ideal for
Table Ablation study of context components on our benchmark dataset. Scores reflect the model's performance when provided with different combinations of contextual information. Our final, con- ditional approach yields the best result.
Context Components Provided
LLM Score
Single Components
Pfam only
74.90
GO only
84.02
ProTrek only
66.44
Pairwise Combinations
Pfam + GO
84,60
Pfam + ProTrek
77.00
GO + ProTTrek
77.78
Full Combinations
Pfam + GO + ProTrek (Unconditional)
81.56
Pfam + GO Pro (Conditional)
84.99
20

---

Published as a conference paper at ICLR 2026
Analysis of Individual Components The results from single-componer experiments establish clear hierarchy of information value. Gene Ontology (GO) anmotations emerge as the most powerful single source of context, achieving high score of 84.02 strong signal, scoring 74.90. In contrast, ProTTrek alone is the least informative component, with a score of 66.44, suggesting its raw output may be noisy or less directly useful for functional queries.
its own. Pfam provides moderately
Synergistic and Antagonistic Effects in Combinations Combining Pfam and GO yields a score of 84.60, a slight improvement over GO alone, indicating positive, synergistic relationship where Pfam provides complementary information. However, critical observation arises when combining components with ProTrek. Both Pfam + ProTrek (77.00) and 'GO + ProTrek' (77.78) perform worse than their stronger counterparts (Pfam and GO, respectively) alone. This trend is amplified when all three are combined unconditionally ('Pfam + GO + ProTrek'), resulting in a score of 81.56, which is substantially lower than 'Pfam GO'. This strongly suggests that naively adding ProTrek'ss s information introduces noise that dilutes the high-quality signals from Pfam and GO, ultimately degrading the model's performance.
Justification for the Conditional Strategy Based on this insight, we implemented our final, con- ditional strategy: ProTrek information is only included as a fallback when both Pfam and GO annota- tions are unavailable for given protein. This intelligent inclusion prevents ProTrek from interfering with higher-quality data while still providing a baseline of information for sparsely annotated pro- teins. As shown in the final row of Table 3 this conditional approach achieves the highest score of 84.99. It effectively captures the synergy of Pfam and GO while mitigating the negative, noisy impact of ProTrek, thus justifying its selection our final methodology.
F IMPACT OF SEMANTIC ALIGNMENT ON MUTATION SENSITIVITY
A critical capability for any protein model is the ability to detect and represent the effects of small quence variations, such as point mutations. To investigate how the internal mechanisms of sequence- as-modality models like Evolla handle such changes, we conducted an analysis on the feature repre- sentations before and after its Q-Former alignment module.
differing sites) into sample protein sequence. We then extracted the resulting feature embeddings at two key stages: (1) directly from the SaProt protein encoder, and (2) after they had been processed by the Q-Former. The difference between the pre-mutation (wild-type) and post-mutation embeddings at each stage was then visualized and quantified. The results are presented in Figure 9
We introduced series of mutations (from
High Sensitivity at the Protein Encoder Stage As expected, the SaProt protein encoder is highly sensitive to sequence mutations. The visualizations (Figure top row) show clear, localized changes in the feature map corresponding the mutation sites. Quantitatively, while the cosine similarities between pre- and post-mutation embeddings remain high (0.980-0.995), the Euclidean distances are substantial (ranging from 13.3 to 26.2). This confirms that the encoder accurately captures the perturbation caused by the mutation, altering the feature vector in a meaningful way. This sensitivity the foundation required for any downstream mutation effect analysis.
Loss of Sensitivity After Q-Former Alignment A starkly differrnt picture emerges after the features pass through the Q-Former. The difference heatmaps (Figure 9 bottom row) are almost entirely uniform, indicating negligible change between the wild-type and mutated representations. This visual observation is confirmed by the quantitative metrics: the cosine similarities are nearly perfect (approaching 1.0, e.g., 0.9999), and the Euclidean distances (5.8-9.9) significantly smaller than those observed from the encoder.
Implications for Downstream Tasks This analysis reveals a critical limitation of the sequence-as- modality paradigm employed by models like Evolla. The Q-Former, in its role of compressing and aligning the detailed protein features into a fixed set of tokens for the language model, effectively "smooths out" or discards the fine-gre -raiird information related to single point mutations. While this may be sufficient for generating high-level functional descriptions, it renders the final representation insensitive to the subtle yet critical differences that underpin tasks like mutation effect prediction,
21

---

Published as a conference paper at ICLR 2026
Orig. (Pro-Mut)
3: AFeal
Foxt Dim
Fosn Dim
Fost Dio
Soq Pos
Seq Pos
Soq Pos
Seq Pos
Soq Pos
Orij. Fest Dist
Mut ADDr
Mun 2: ADirt
Mot 3: ADis
Mir d: ADiet
Prequecy
Frequmiy
Value
Villl
(a) Evolla SaProt
Oris.
Dmn
Saq Poe
Sog Poe
Saq Pae
Sog Pres
Sog Pos
Orig, Feat. Dis.
Mu. 1:DDist
Mut 2. Albii
Mus A.D
Mur 4: Aliit
12105
Mosi coom
D100
HWW
Frequetcy
4000
2004
2000
Festure Value
3NNaloo
Vallo
(b) Evolla Q-Former
Figure 9: Effect of mutations on internal representations of Evolla. The top row shows feature differences from the SaProt encoder, and the bottom row from the Q-Former. Heatmaps visualize the difference vector ('mutated original"). SaProt's representation is clearly perturbed by muta- tions, showing localized and significant changes. In contrast, the Q-Former's 's output shows almost no change, indicating that the alignment process erases the fine-grained signal of the mutation.
22

---

Published as a conference paper at ICLR 2026
disease variant analysis, and protein engineering. This inherent loss of information at the alignment stage explains why such architectures are fundamentally ill-suited for these precision tasks.
G GENERALIZABILITY ACROSS BIOMOLECULAR TYPES
To demonstrate the broad applicability and robustness of our context-driven methodology, we eval- uated its performance standard benchmarks beyond our primary QA dataset. This tests the ap- proach on different tasks and different biomolecular types.
G.1 PERFORMANCE ON PROTEIN FUNCTION PREDICATION
We first evaluate our approach on the protein classification tasks from the Mol-Instruction bench- mark (Fang al. 2024). This benchmark contains curated datasets for assessing molecular under- standing. We specifically tested on three functionally distinct protein subsets: Catalytic Activity, General Function, and Protein Function. The performance shown Figure
Catalytic Activity
General Function
Protein Function
Sumptac 1600 Valvt: Temm (UN (S
Sumpies 2615 Volle: 2611 (1oc
Sampplo Sa9d Vaiet 3464 (100 0%g) Moa::
Marc 10.222
Sd 22.84)
100
Milr 000 Mae 100 000
Min 0.000) Maw 100.000
Mie o Mae 100 500
Pereertug Pooerttge (56)
Petcesttggs (56)
0.20
21-40
41-50
61-80
$1-100
06:00
21:40
4.-50
61.90
51.100
0.20
21-40
41:00
61.30
81-100
Performance Mol-Instruction Protein Classification. Score distributions of our context-driven method on three sub-datasets. The results show consistently high performance across all categories, with mean scores above 85 and the vast majority of answers (> 84%) falling into the highest score bracket (81 100), demonstrating robust generalization.
Figure 10:
As illustrated Figure 10 our method achieves excellent performance across all three predica- tion tasks. The mean score were exceptionally high: 91.2 for Protein Function, 90.2 for Catalytic Activity and 85.5 for General Function. The score distributions are heavily skewed towards the maximum, with over 84% of answers in all three tasks receiving score in the 81-100 range. This demonstrates that our method not only effective on our QA benchmark but also generalizes ro- bustly to standard, multi-category protein function predication tasks, validating its broad utility.
G.2 PERFORMANCE ON DNA MUTATION PREDICTION
We evaluated powerful generalist LLMs, GPT-4o and Qwen3-4B, across three standard input configurations: Context-Only, Sequence + Context, and Sequence-Only. To provide a com- prehensive view of performance, we measured Precision, and Recall, accounting for potential class imbalances in the dataset. The results, pre- sented in Figure I1, strongly corroborate our findings from the protein- tasks and confirm the paradigm's 's generalizability. For both models, the Context-Only configuration consistently achieved the highest scores across all four evaluation metries. Crucially, the "informational noise" effect of raw sequences persists in the DNA domain. The Sequence Context configuration consistently un- derperformed the Context-Oni Only setup, indicating that the models were again "lost in tokenization," struggling to integrate the low-level signal from the DNA sequence with the clear, high-level context. The Sequence-Onle approach yielded the poorest results, confirming that atomic tokeniza- tion of nucleotide sequences is insufficient for complex biological reasoning.
only classification accuracy but also F1 Score,
23

---

Published as a conference paper at ICLR 2026
1.0
Accuracy
context
1.0
F1 Score
context
context-seg
context+seg
seq
soq
0.8
0.8
Accuracy
0.6
F1 Score
0.6-
0,4
0.4
0.2
0.2
0.0
0.0
gpt-4o
qwen3-4b
gpt-4o
qwen3-4b
Model
Model
1.0
Precision
context
1.0
Recall
context
context-seq
context+seq
seq
seq
Procision
Recall
0.4
0.2
0.2-
0.0
gpt-4o
qwen3-4b
gpt-4o
qwen3-4b
Model
Model
Model performance on the KEGG DNA dataset under different input configurations. Both Qwen3-4B and GPT-4o were evaluated across context-only, sequence-only, and combined inputs, with both models showing consistent improvements when using context- baseed inputs com- pared to sequence-basee approaches.
Figure 11:
H INDEPENDENCE FROM CLUSTERING METRICS
To verify that our conclusions are robust and not contingent on single definition protein families, we evaluated all embeddings against two distinct ground-truth labeling schemes: UniClust50 and UniClust30. These standards group proteins 50% and stricter 30% sequence identity threshold, respectively, providing different granularities for functional classification. While both are generated using the MMseqs2 algorithm, they represent conceptually different criteria for defining protein homology. We performed the same hierarchical clustering analysis for each representation against both sets of labels. The resulting ARI scores are presented Table
As shown in Table the performance hierarchy and the specific ARI scores remain identical across both labeling schemes. While the identical results suggest that our test set's structure stable across these two identity thresholds, the key insight is the unwavering consistency of our central observations. Specifically, three conclusions hold firm regardless of the ground-truth definition:
Our context-driven approach consistently achieves the highest functional separation (ARI 0.958).
The "semantic misalignment' within Evolla consistently demonstrated by the progressive drop in ART from 0.945 0.809.
24

---

Published as a conference paper at ICLR 2026
Table 4: Performance (ART) against ground-truth labels from UniClust50 and UniClust30.
Model / Representation Stage
ARI (vs. UniClust50)
ARI (vs. UniClust30)
Ours
0.958
0.958
Evolla Encoder (SaProt)
0,945
0,945
Evolla Alignment (Q-Former)
0.916
0,916
Evolla Decoder (Final)
0.809
0.809
Intern-S1 8B MatureLM
0.690
0.690
0.492
0,492
The "weak representation' of Sqquence-to-languae models (Intern-S1 and NatureLM) is con- sistently evident from their significantly lower scores.
This stability across different conceptual standards for protein families validates our findings, con- firming that tokenization dilemma and the superiority of our context-driven solution are funda- mental phenomena, not artifacts of particular evaluation metric.
I QA EXAMPLES
To provide concrete illustrations of the performance differences reported in the main paper, this section presents qualitative examples from our benchmark. We analyze the outputs for a single protein (UniProt ID:: A6L across the three input modalities: Context-Only. Sequence-Only, and Sequence + Context. These case studies offer tangible insights into why our context-driven approach excels and how raw sequence inputs can be detrimental.
1.1 COMPARATIVE ANALYSIS ON PROTEIN A6LHQ9
The following examples use the same protein and question ("What is the function of this protein?") demonstrate how the input modality directly impacts the model's reasoning and final output. The ground truth is: "Putative component of the fimbrium tip. Fimbriae are filamentous appendages on the cell surface that mediate cell adhesion and biofilm formation".
1.1.1 CASE 1: CONTEXT-ONLY (SUCCESS)
In the Context- Only setting, the model receives curated information from Pfam. The model demon- strates strong ability to synthesize this structured knowledge into a comprehensive and accurate answer, achieving a perfect score.
Protein A6LHQ9: Context-Only
You are a senior systems biologist. Analyze the input information to answer the given question.
Question:
What is the function this protein?
context Provided:
Conserved Domains (from Pfam)
Pfam PF06321: This family consists of several Porphyromonas gingivalis major fimbrial subunit protein (FimA) sequences. Fimbrise of Forphyromenas gingivalis, a periodontopathhogen, play an important role in its adhesion to and invasion of host cells. The fimA genes encoding fimbrillin (FimA), subunit protein of fimbriae, have been classified into five types, types to v, based on nucleotide sequences. has been found that type
25

---

Published as a conference paper at ICLR 2026
FimA can bind to epithelial cells most efficiently througa specific host receptors [Icite::00000040011 Human dental plaque is a multispecies microbial biofilm that is associ with two common oral diseases, dental caries and periodontal disease. There communication system between P. gingivalis and S. cristatus that involces the Arc-À enzyme [[cite:PUB0066920]]
inter-species contact-dependent
Pfam PF22449: This domain found at the C-terminal end of the putative fimbrium tip subunit FimlF from Parabacteroides distasonis, which contain an N-terminal domain (Ipfam:PFF63211) and slightly larger C-terminal domain (this entry) with transthyretin-like feld that contains seven core beta-strands arranged in two beta-sheets an extra conserved 'appendage' of two amphipathic beta-strands [[cite:PUB00080111]]
Answer:
The protein in question, which incorporates the domains PF06321 (fimbrial subunit FimA) and PF22449 (C-terminal
transthyretin-like domain), functiens as structural component of bacterial fimbriae with primary role in adhesion to host cells and surfaces. Here's detailed breakdown:
PF06321 (FimA-like domain): Found in fimbrial subunits of
Porphyromonas gingivalis (a periodontal pathogen), this domain mediates host cell adhesion and invasion. Type FimA variant within this family) binds efficiently to epithelial cells via specific hest receptors, highlighting its role in pathogenicity and bicfilm formation (e.g., in dental plaque).
PF22449 (transthyretin-like domain): Present at the C-terminus of Simbrial tip subunits (e.g., FimlF in Parabacteroides distasonis), this domain stabilizes the fimbrial structure through its transthyretin-like fold (seven beta-strands forming two beta-sheets, plus an amphipathic "aapendagee"); itansthyretin-llke fold may enhance structural integrity enable additional interactions, such as binding to host molecules or other bacterial proteins.
The
Overall Function; The protein acts as a fimbrial structural subunit, critical for assembling ing adhesive fimbriae. These appencages facilitate pacterial attachment to host tissues (e.g., oral epithelial cells) and surfaces, enabling colonization, biofilm formation, and inter-species interactions (e.g., with Streptococcus cristatus via the Arc-A system) The combination of FF06321 and PF22449 domains likely optimizes both adhesion efficiency and structural stability, contributing to the pathogenicity of periodontal bacteria. Key References:
PF06321: Binding to host receptors (PU800010404). PF22449: structural role in fimbriae (PU800080711). communication (PUB00069820).
inter-species
Ground Truth:
Putative component of the fimbrium tip. Fimbriae are filamentous appendages on the cell surface that mediate cell adhesion and biofilm formation.
Result:
LLM-Score: 100
I.1.2 CASE 2: SEQUENCE-ONLY (TOTAL FAILURE)
When provided with only the amino acid sequence, the model fails completely. It hallucinates a function (Chitin-Binding Binding Module) that entirely incorrect, likely by misinterpreting spurious patterns in the tokenized sequence. This is stark example of the "lost in tokenization" phenomenon.
26

---

Published as a conference paper at ICLR 2026
Protein A6LHQ9: Sequence-Only
You are a senior systems biologist. Analyze the input information to answer the given question.
Question:
What is the function of this protein?
Sequence Provided:
Answer:
Based on the amino acid sequence provided, this protein is very likely a Chitin-Binding Module (specifically, a CBM12 domain).
Here's breakdown of the evidence:
Signal Peptide: The sequence starts with MRFWLLLIVALLGGELSTSS,, which is a classic signal peptide. This indicates the protein is secreted from the cell.
CBM12 Sigmature Motifs: The sequence contains the highly characteristic motifs of the Carbohydrate- Module family 12 (CBM12);
DGDG (appears twice: DGEG and DGDG).
NGAN tappears: NGAN).
TGS (appears: TGS)
QGN (appears: QGN)
Multiple conserved cysteine residues (C) that likely form stabilizing disulfide bonds.
Function of CBM12:
Primary Function:
Carbchydrate binding, specifically draacttylglucoosmine (GIcNAc) residues. GlcNAc the building block of chitin (a major component of fungal cell walls, insect exoskeletons, and crustacean shells) and is also found in various glycoproteins and glycolipids (like hyaluronan).
Role in Proteins: CBM12 domains are typically found attached to enzymes (like chitinases, chitobicsidases,, dracetylgglucomminidasss
structural proteins. Their role is to target the catalytic demain to its specific substrate (chitin or other GieNac-containing polysaccharides) and increase the enzyme's efficiency by bringing it into close proximity with the substrate.
Binding Specificity: CBM12 modules generally bind to short stretches of G1cNAc residues, including chitobiose and
(G1 and chitetricse GNN---G show weaker binding to isclated GleNAc monomers. They do not bind cellulose.
In summary, the primary function of this protein is to act as a Chitin-Binding Module (CBM12). Its job is to specificall) recognize and bind to s-acetylglucossminne (GLeMAc)-containi) carbohyarates, particularly chitin, facilitating the action of associated enzymes (like chitinases) or contributing to the structural integrity/function of proteins interacting with chitin or related polysaccharides.
Typical Context: Proteins containing CBM12 domains very common in fungi, where they play crucial roles cell wall
27

---

Published as a conference paper at ICLR 2026
remoceling, nutrient acquisition (degrading chitin from other organisms), and pathogenicity. They are also found in bacteria and some other organisms.
Ground Truth:
Putative cemponent of the fimbrium tip. Fimbriae are filamentous appencages on the cell surface that mediate cell adhesion and biofilm formation.
Result: LIM-Scorc:
1.1.3 CASE 3: SEQUENCE CONTEXT (NoISE EFFECT)
In the combined setting. the model produces a correct but slightly less focused answer than the Context-Only version. The inclusion of the sequence acts informational noise, causing minor degradation answer quality and a corresponding drop in the score. This supports our quantitative finding that the sequence is merely unhelpful but be actively detrimental.
Protein A6LHQ9: Sequence Context
You are a senior systems biologist. Analyze the input information to answer the given question.
Question:
Mhat is the function of this protein?
Context Provided:
[Same
Case 1] Case 21
Sequence Provided: Same
Answer:
The protein in question, based en the sequence and associated motifs (PF06321 and PF22449), functions as a fimbrial subunit protein involved in bacterial adhesion and host-microbe interactions. Here's detailed breakdown of its role: Fimbrial Structure (N-terminal) correspends to the major fimbrial subunit Pima of Porphyromonas gingivalis. This domain is critical for hair-like
Assembly: -The PF06321 domain
forming the structural backbone of fimbriae, which appendages on the bacterial surface. The PF22449 domain (C-terminal) contains a transthyretin-like fold with a conserved "aappendage'' of amphipathic beta-strands. This domain likely stabilizes the fimbrial tip structure and may mediate specific interactions with host cells or other microbial species.
Fimbriae in gingivalis (a periodontal pathogen) are essential for hest cell adhesion and invasion. The type 11 FimA variant (relevant to PF06321) binds efficiently to epithelial cells via specific host receptors, facilitating colonization and immune evasion. The cransthyretin-like fold in PF22449 may enhance binding specificity or structural resilience, cnabling the fimbriac to withstand mechanical stress in the oral environment.
Adhesion and Invasion:
Biofilm Formation and Interspecies Communication: contribute to the formation of multispecies biofilms (e.g., dental plaque), which are linked to oral diseases like periodontitis. The protein may mediate interbacterial adhesion, as seen in the interaction between gingivalis
Finbriae
28

---

Published as a conference paper at ICLR 2026
and Streptococeus cristatus via the Arc-A enzyne system. The amphipathic beta-strand ''appendage" in PF22449 could play role in direct contact-dependent signaling or physical interactions with other microbes. item Pathogenicity: enabling adhesion to hest tissues and other bacteria, this protein promotes the establishment of polymicrobial infections, contributing to the progression of periodental disease and potentially systemic inflammation.
By
Conclusion: The protein is fimbrial subunit that assembles into achesive structures critical for bacterial attachment host cells, biofilm formation, and interspecies interactions in the oral nicrobiome. Its dual-domain architecture (PF06321 + PF22449) ensures both structural integrity and functional specificity in these processes.
Ground Truth:
Futative component of the fimbrium tip. Fimbriac are filamentous appendages en the cell surface that mediate cell adhesion and biofilm formation.
Result: LIM-Score: 95
The comparative analysis of protein A6LHQ9 clearly illustrates the core tenets of our findings: text the primary driver accurate biological reasoning, while raw sequence data can lead to catastrophic failures introduce performance-dgrrading noise.
1.2 SHOWCASE OF OUR METHOD'S SUCCESS CASES
Having established the superiority of the context-driven paradigm, we now showcase its robustness and versatility across the three primary query types in our benchmark: molecular function, metabolic pathway, and subcellular localization. The following examples demonstrate the model's ability to consistently generate high-quality, accurate, and detailed answers for each category.
1.2.1 EXAMPLE 1: FUNCTION PREDICTION
The example of PDB ID: PODJ91
You senior systems biologist. Analyze the input information answer the given question.
Question:
What is the function of this protein?
Conserved Domains (from Pfam):
PF03549: Intimin and its translocated intimin receptor (Tir) are bacterial proteins that mediate adhesion between mammalian colls and attaching and effacing (A/E) pathogens. unique and essential feature of A/E bacterial pathogens is the formation of actin-rich pedestals beneath the intimately adherent bacteria and localised destruction of the intestinal brush border. The bacterial outer membrane adhesin, intimin, is necessary for the production of the A/E Lesion and diarrnoea. The A/E bacteria translocate their Tir, into the membrane mammalian cells using the type III secretion system. The translocated Tir triggers additional host signalling events and actin nucleation, which are essential for lesion formation [[cIte:PUB000663]] This family represents the intimin-binding domain (Tir IBD) which needed
receptor for intimin,
29

---

Published as a conference paper at ICLR 2026
bind intimin and support the predicted topology for Tir, with both N- and C-terminal regions in the mammalian cell cytosol [[cite:PUB00142844]
PF07489: Intimin and its translocated intimin receptor (Tir) are bacterial proteins that mediate adhesicn between manmalian cells and attaching and effacing (A/E) pathogens. A unique and essential feature of A/E bacterial pathogens is the formation actin-rich pedestals beneath the intimately adherent bacteria and Localised destruction of the intestinal brush border. The bacterial outer membrane adhesin, intimin, is necessary for the production of the A/E lesion and diarrhoea. The A/E bacteria the
translocate their own receptor for intimin, Tir, membrane of mammalian cells using the type III secretion system. The translocated Tir triggers additional hest signalling events and actin nucleation, which are essential for lesion formation [[sste:FU30006623]] C-terminal domain which has been reported bind uninfected host cells and beta-1 integrins although the role of intimin binding to integrins is unclear. This intimin c-terminal domain has also been shown [[cite:PUR001122441]]
This family represents the Tir
sufficient for Tir recognition
PF07490: Intimin and its translocated intimin receptor (Tir) are bacterial proteins that mediate adhesicn between manmalian cells and attaching and effacing (A/E) pathogens. A unique and essential feature of A/E bacterial pathogens is the formation of actin-rish pedestals beneath the intimately adherent bacteria and localised destruction of the intestinal brush border. The bacterial outer membrane adhesin, intimin, production of the A/E lesion and diarrhoea. The A/E bacteria translocate their own receptor for intimin, Tir, into the membrane of mammalian mimaliian cells using the type 111 secretion system. The translocated Tir triggers additional hest signalling events and actin nucleation, which are essential for lesion formation [[ci=e:PUB000066231] This family represents the Tir N-terminal domain which is involved in Tir stability and Tir secretion
necessary for the
Functional Annotations (from Homology via BLASTp):
GO terms associated with the hemolog:
GO tcrml: 0005515- definitien: Binding to a protein.
Answer:
As a senior systems biologist, my analysis the provided data loads to the following conclusicn about the protein's function. Summary of Function: Based on the provided information, this protein is the Translocated Intimin Receptor (Tir), key multi-domain effector protein from attaching and effacing (A/E) bacterial pathogens like enteropathogenic g. coli (EPEC). Its primary function is to act as a molecular bridge that subverts host cell processes to facilitate bacterial adhesion and pathogeessis. After peing injected directly into the host cell by the bacterium, Tir inserts into the host cell membrane and functions as high-affinity receptor for the bacterial outer membrane adhesin, intimin. This specific interaction is the central event that triggers cascade of host signaling events, leading to dramatic cytoskeletal rearrangements and the formation of an "actin pedestal'' structure beneath the bacterium.
Detailed Functional Analysis by Domain: The protein's overall function is accomplished through the coordinated action of its distinct domains:
Tir Intimin-Binding Domain (IBD) (PF03549) This is the core interaction domain. Its speci fic function is to bind directly
30

---

Published as a conference paper at ICLR 2026
to intimin. This binding event essential for the intimate attachment of the bacterium to the host cell and for establishing the correct topology of the Tir pretein within the host membrane. 2. Tir C-terminal Domain (PF07489) This domain is also crucial for host interaction. It is sufficient for recognizing and binding intimin, reinforcing the function of the IRD. Additicnally, it has secondary role in binding to host cell beta-1 integrins, suggesting it may modulate or interfere with native hest cell adhesion and signaling pathways, although the precise role this interaction noted as unelcar.
3, Tir N-terminal Domain (PF07490) This domain's function is related to the protein's own lifecycle and deployment. It is involved in ensuring the stability of the Tir protein and is necessary for its secretion from the bacterium into the host cel1 via the Type III secretion system.
The associated Gene Ontology term, G0:0005515 (protein binding), serves as high-level functional annotation that is strengly supported by the specific binding activities described for both the IBD (PF03549) and the c-terminal domain (PF07489).
Role in the Biological System: From à systems perspective, Tir The pathogen
is a masterful example of pathogenic manipulation. does not rely on pre-existing hest receptor; instead, it injects its as follows:
custom receptor into the host. The process is
1. The A/E pathogen uses its Type III Secretion System to inject Tir into manmalian intestinal cell. 2. The Tir protein, guided by its N- and C-terminal demains, inserts into the host cell membrane, exposing
central intimin-Dinding domain the exterior. 3. Intimin on the bacterial surface binds with high affinity to the translocated Tir. 4. This binding event triggers hest signaling pathways, leading to the recruitment and nucleation of actin filaments directly beneath the bacterium. 5. This results
formaticn of an "actin-rich pedestal'' and the destruction of the local intestinal microvilli (the A/E lesion), which is the hallmark of this pathogenic mechanism and is critical for causing disease.
In summary, Tir is multi-functienal protein that central hub in bacterial pathogenesis. stability and secretion, inserts into the host membrane, and orchestrates the specific binding and signaling events required to hijack the host's cytoskeleton for the bacteriun's benefit.
ensures its own
Ground Truth:
celtifunciionn protein that is required for efficient pedestal formation in hest epithelial cells during infectien. The racelllul region acts as receptor for bacterial intimin, allowing the bacterium to attach tightly to the hest-cell surface. simultaneously, the intracellula region initiates a signaling ing cascade in the host cell, which leads to actin po1 imerization and formation of actin pedestals the sites of bacterial adhesion (By similarity)
1.2.2 EXAMPLE 2: PATHWAY PREDICTION
The example of PDB ID: P78748
You are a senior systems biologist. Analyze the input information to answer the given question.
Question:
31

---

Published as a conference paper at ICLR 2026
What the pathway this protein?
Conserved Domains (from Pfam)
PF00215:
This family includes Orotidine 5' -phospheeee decarboxylase enzymes [ec:4.1.1.23] that involved in the Sinal step pyrimidine biosynthesis. The family also includes enzymes such as texuicse--6ppooppatee synthase. This family appears to be distantly related to [pfam:Pr0083]]
Functional Annotations (from Homology via BLASTP):
terms associated with the homelog:
GO 0044205: The chemical reactions and pathways resulting in the formation of UMP, uridine monophosphate, starting with
the synthesis of (G)-dibydroorotate from bicarbonate, UMP biosynthesis may either occur via reduction by quinone, NAD+ or oxygen.
0005829: The part of the cytoplasm that does not contain organelles but which does contain other particulate matter, such as protein complexes.
GO 0004590: Catalysis of the reaction: H orotidine 5' -phosphtte = C02 UMP.
GO 0006207: The chemical reactions and pathways resulting in the formation of pyrimidine nucleobases, 1, 1,3-diazine, organic nitrogenous bases, beginning with the synthesis of pyrimidine ring from simpler precursors.
Answer:
Based systems-level analysis of the provided data, the protein is key enzyme the novo pyrimidine biosynthesis pathway.
Here is a breakdown of the evidence:
1. Broad Pathway Identification: The motif (PF10215) and term 60:0006207 both explicitly place the protein within "pyrimidine biosynthesis."' This is the overarching metabolic network responsible for creating pyrimidine nucleobases.
Specific Sub-Pathway and Function: The protein's role is further refined to the UMP biosynthesis sub-pathway (G0:0044205). The motif describes process.
2.
function as the "final step' this
3. Molecular Mechanism: Its prccise enzymatic function is Orotidine -p-poshe decarboxylase activity (G0:0004590), catalyzing the conversion of orctidine 5' 5'-phopphate into UMP. This confirm rms its role as the terminal enzyme in the synthesis of UMP, a foundational pyrimidine nucleotide.
4. Cellular Location: The pathway is localized to the cytosol (G0:0005829).
Ios protein an Orotidine sphoshhtte
decarboxylase that executes the final, decisive step of the UMP biosynthesis pathway. This pathway is a critical component of the larger pyrimidine biosynthetic network, and it occurs within the cytescl of the cell.
Ground Truth:
Pyrimidine netabolism JMP bicsynthesis via de novo pathway; UMP from orotate: step 2/2
32

---

Published as a conference paper at ICLR 2026
1.2.3 EXAMPLE 3: SUBCELLULAR LOCATION PREDICTION
The example of PDB ID: P63171
You are a senior systems biologist. Analyze the input information to answer the given question.
Question:
What is the subcellular locaticn of this protein?
Conserved Domains (from Pfam)
1ro3645:1ctexx11 is dynein light chain. It has been shown that Ictex-1 can bind to the cytoplasmic tail of rhodopsin. c-terminal rhocopsin mutations responsible for retinitis pigmentosa inhibit this interaction.
Answer:
Based the analysis of the previded information, the subcellular location Tetex-1 is the cytoplasm. Here the reasoning:
1. Component of the Dynein Moter Complex: The protein is identified as "dynein light chain." Dynein is motor protein complex that functions within the cytoplasm, moving cellular cargo along microtubule tracks.
Specific Binding Partner: The text explicitly states that Tctex-1 binds to the cytoplasmic tail of rhodopsin. For this interaction to occur, l'ctex-1 must be located in the cytoplasm where it access this specific domain of rhodopsin.
Ground Truth:
Golgi apparatus, Cytoplasm, cytoskeleton, spindle
J LIMITATIONS
While context-driven approach demonstrates impressive performance in various bioinformatics tasks, there are still significant challenges, particularly in predicting the effects of protein mutations. One of the primary difficulties that InterProScan and BLAST- key bioinformatics tools that we use for generating functional context-are are not sensitive to subtle mutations at single or few amino acid positions. This because both tools rely on sequence-lever comparisons, and mutations that alter few amino acids may not result in any meaningful changes in the overall sequence context, making the corresponding Pfam domains GO terms identical before and after mutation.
Thus, when we apply our method to predicting the effects of mutations, we encounter situation where the context generated for the wild-type and mutated proteins essentially the same. This leads to the limitation that our approach, at present, cannot effectively predict changes in the pro- tein's function or characteristics due to small mutations.
Below are two examples demonstrating this limitation: one shows the context for wild-type protein, and the other for a mutated version of the same protein. The only difference between the two sequences is the mutation at two amino acid positions, which have highlighted in red. For clarity and case of comparison, we have provided only the Pfam domain and GO annotations (numbers) rather than the complete context, which would otherwise be too long display for these examples.
33

---

Published as a conference paper at ICLR 2026
J.1
WILD-TYPE PROTEIN EXAMPLE
WT Wild-Type
Conserved Domains (from Pfam) РF00732
PF05199
Functional Annotations (from Homology via BLASTp)
GO 0005737
GO 0005576
GO 0046562
0050660
0044550
Sequence:
J.2 MUTATED PROTEIN EXAMPLE
MUTI MMtttted
Conserved Domains (from Pfam)
[same as WT]
Functional Annotations (from Homology via BLASTp)
[same WT
Sequence:
G TIASHUPPPUUBBRTTYYRRRRR/TTATTATT H ESGSYES
DRGP WABTIIIUWWNNOOSSSSSSTTTTTTTTAAIA
In both cases, the context generated for the wild-type and mutated proteins is identical, as Inter- ProScan and BLAST fail to identify significant differences caused by mutations at amino acid po- sitions. This limitation underscores the challenge of predicting the effects of mutations based on sequence context alone, as even small changes in the sequence can have profound effects on protein function that captured by these tools.
34

---

Published as a conference paper at ICLR 2026
J.3 SUMMARY OF THE LIMITATION
The core limitation of our current method is its inability to distinguish the functional consequences of amino acid mutations, due to the lack of sensitivity of InterProScan and BLAST to small sequence changes. This highlights the complexity of mutation prediction tasks, where even subtle changes to the sequence can lead to vastly different biological outcomes.
K COMPARISON OF MODEL PERFORMANCE ON THE SAME PROTEIN AND QUESTION
In this section, we compare the performance different models on the task predicting the function of the same protein, specifically protein A7YWC4, using the question: "What is the function of this protein" The ground truth for this protein as follows:
Ground Truth
Ground Truth: Essential for mitochondrial network organization, mitochondrial metabolism and cell growth at the organism and cellular level. May play important role in mitochondrial protein synthesis, mitochondrial DNA replication, and contribute nuclcoid stability. Also involved mitochondrial-mcdiann
antiviral immunity and the mitochondrial DNA damage response, promot- ing signaling between damaged genomes and the mitochondrial membrane, which leads to activa- tion of the integrated stress response (ISR).
Below, we compare the answers generated by our method (using context), Interns Evolla, and NatureLM.
Our Method (Context- -Diiven,, Gemini2.5Pro)
Answer:
As a senior systems biologist, my analysis the provided information leads to the following con- clusion about the function of this protein.
Executive Summary This protein is ATAD3 (ATPase family AAA domain-containing protein functional mitochondrial protein acts as a central and function. Its core activity is derived from its AAA ATPase domain, which uses the energy from ATP hydrolysis to perform chaperone-lie functions, likely remodeling protein complexes. This molecular activity is essential for maintaining the mitochondrial network's architecture, regulating mitochondrial metabolism, and ultimately, supporting cellular growth.
is multi- organizing mitochondrial structure
Detailed Functional Analysis
The protein's 's function can he broken down by integrating the motif data with the relevant Gene Ontology (GO) terms.
1. Protein Identity and Core Mechanism
Identity: The presence
PF12037 domain specifically identifies this protein as ATAD3,
Mechanism: The protein contains PF00004 (AAA ATPase) domain. This domain confers key molecular functions:
ATP binding (GO:0005524): As ATPase, it must first bind ATP.
ATPase activity (C0:0016887); It hydrolyzes ATP ADP to release energy. This energy powers primary role a chaperone-like protein that assists in the "assembly, operation, or disassembly of protein complexes."
Biological Processes
The energy-dependent chaperone activity of ATAD3 is leveraged to control several critical cellular processes:
Mitochondrion organization (GO:0007005): This a primary function. The PF12037 motif description explicitly states that ATAD3 is "essential for mitochondrial network organisation." This involves managing the shape, size, and distribution of mitochondria within the cell.
35

---

Published as a conference paper at ICLR 2026
Regulation growth (GO:0001558); motif information directly links ATAD3's func- tion to being "essential for... cell growth at organism and cellular level." Healthy mitochondrial function is prerequisite for cell proliferation.
Mitochondrial Metabolism and Protein Synthesis: The PF12037 description notes its im- portance in "mitochondrial metabolis" and a potential role in "mitochondrial protein synthe- sis." This is consistent with its organizational role, proper structure required for efficient metabolic function and positioning ribosomes. association with the mitochondrial nu- cleoid (GO:0042645), where mitochondrial DNA resides, would be critical for these processes.
Cellular Component (Localization)
The protein's 's functions are tied to its specific location within the cell:
Mitochondrion (GO:0005739): The protein fundamentally mitochondrial.
Mitochondrial Membranes: The PF12037 domain "interacts with the inner surface of the mitochondrial membrane." This positions the protein at a critical interface. Given its role organizing the entire organelle, is highly likely to also associate with the mitochondrial inner membrane (GO:0005743).
Membrane Contact Sites (GO:0044233); ATAD3's 's role in organizing the mitochondrial net- work and its localization at the membrane interface strongly suggest its presence endoplasmic membrane contact sites. These sites crucial for inter-organt organeee communication lipid exchange, which are vital mitochondrial biogenesis and dynamics.
Integrated Synopsis
ATAD3 functions as molecular machine anchored to the mitochondrial membranes. uses the energy from ATP hydrolysis to remodel protein structures, thereby orchestrating the complex archi- tecture of the mitochondrial network. By controlling the physical organization of the mitochondria, including its association with the nuclcoid and contact sites with the ER, ATAD3 governs essen- tial downstream processes. This structural regulation is indispensable for efficient mitochondrial metabolism, protein synthesis, and, consequently, Any disruption in ATAD3 function would lead to collapse of the mitochondrial network, metabolic failure, and impaired cell growth. Score: 100
overall growth and viability the cell.
Interns1 (Sequence Input)
Answer:
Essential mitochondrial network organization, mitochondrial metabolism and cell growth organism and cellular level. May play an important role in mitochondrial protein synthesis. May also participate in mitochondrial DNA replication. May bind to mitochondrial DNA D-loops and contribute to nucleoid stability. Required for enhanced channeling of cholesterol for hormone- dependent steroidogenesis. Involved in mitochondrial-median involved the mitochondrial DNA damage response by promoting signaling between damaged genomes and the mitochondrial membrane, leading to activation of the integrated stress response (ISR).
antiviral innate immunity. Also
LLM Answer: The protein described the provided amino acid sequence is likely an ATPase involved in energy energy-dependent cellular processes. Here's a detailed breakdown of inferred func- tion:
1. ATPase Domain:
The sequence contains Walker A motif (GXXXXGKT), a hallmark of AiP-binding sites, and a potential Walker B motif (DExxG), which is critical for ATP hydrolysis. These motifs indicate the protein uses ATP hydrolysis to drive conformational changes mechanical work.
2. Structural Features:
Coiled-coil regions: Suggest oligomerization or
interactions, common AT-
Pases that form multimeric complexes (e.g.. hexamers).
N-terminal proline/glycine-rich region: May serve as a flexible linker or binding site for regulatory proteins or substrates.
3. Functional Inference:
36

---

Published as a conference paper at ICLR 2026
The protein likely involved energy-dependent processes such as:
Protein remodeligddisaeggrgattoo (e.g., chaperones like Hsp104).
Vesicle trafficking (c.g., NSF, which disassembles SNARE complexes).
DNA repair/replication (e.g., helicases that unwind DNA). Proteolysis (e.g., subunits of ATP-dependent proteases like Lon or Clp).
4. Key Motifs and Comparisons:
The Walker A/B motifs align with A ATPases, family involved diverse functions like protein degradation, membrane fusion, and DNA metabolism.
The absence of protcase-speciffc motifs (c.g., Ser-Lys dyad in Lon) or helicase signatures (c.g., DExD in DEAD-box proteins) makes these less likely, though not excluded.
The protein most likely A ATPase functioning in protein complex remodel- ing or vesicle trafficking, with roles in processes such as SNARE complex disassembly (like NSF) stress response (like Hsp104). Further experimental validation (e.g.. domain mapping, interac- tion studies) would refine its specific rolc.
Conclusion:
Score:
Evolla (Sequence Input)
Answer:
This protein, which is associated with the mitochondrial membrane in the organism Oryza sativa (rice), serves multifaceted role primarily within the mitochondria. It known as the Mitochondrial import receptor for the endoplasmic reticulum (MIA40), and its functions are intricately tied the protein import quality control processes within the mitochondria.
The primary function of this protein is to act as a receptor for the mitochondrial presequence translo- case (MPT). which is crucial for the import of proteins into the mitochondrial matrix. This process is essential for the proper functioning of the mitochondria, as it ensures that the necessary proteins are correctly targeted Additionally, the protein involved in the translocation proteins into the mitochondrial matrix. It is likely to recognize mitochondrial presequences that are less than 10 amino acids long, which are critical for the proper targeting and import of these proteins. This suggests specialized role in the import of subset of mitochondrial proteins.
translocated
the organelle.
protein also plays role regulating the mitochondrial permeability transition pore (mPTP), which is a non-selective pore that can lead to mitochondrial swelling and cell death if activated. The regulation of this pore is vital for maintaining mitochondrial homeostasis and preventing apoptosis. In terms of its interaction with other proteins, the protein forms homohexamers, and it is part of complex that includes the MICOS complex subunit MIC10. This complex involved in the regulation of the mitochondrial membrane potential the opening the mPTP. The protein' 's interaction with the mPTP is also noted to be indirect, suggesting a complex regulatory network within the mitochondria.
Moreover, the protein interacts with the mitochondrial presequence translocase, which hetero- hexamer composed of subunits TIM10, TIM22, and TIM23. This interaction is essential the import of proteins into the mitochondrial matrix.
The protein's 's involvement in various cellular processes is further highlighted by its association with the biological process of protein import into the mitochondrial matrix, well as its role in the regulation of mitochondrial membrane potential and the opening of the mPTP. These functions integral to maintaining mitochondrial homeostasis and preventing cell death, making the protein a key player in cellular survival and metabolic regulation. Score: 30
M (Sequence Input)
Answer:
Upon analysis of the given protein sequence, it appears that the primary function of this protein RNA binding, and it could be implicated in the mRNA transport.
Score: 30
37

---

Published as a conference paper at ICLR 2026
K.1 ANALYSIS OF MODEL DIFFERENCES
Our Method provided the most accurate and complete functional analysis, correctly identifying the protein ATAD3 and detailing its involvement in mitochondrial network organization, metabolism, and cell growth, which closely aligns with the ground truth. The score of 100 reflects the compre- hensive nature of the answer and the high alignment with the expected protein function.
Internsl, relying solely on sequence input, inferred the protein to be an ATPase, which is reason- able prediction given the presence specific ATP-binding motifs. However, failed identify the specific protein (ATAD3) and did not connect its functions to the mitochondrial network organiza- tion, leading to much lower score of 30.
Evolla also struggled with correct protein identification, suggesting mitochondrial import recep- tor for the endoplasmic reticulum (MIA40), which does not match the true function of ATAD3. This error resulted in score of 20.
MatureLM provided very generic answer, linking the protein to RNA binding and mRNA transport, which is not at all related to the actual function of ATAD3. This misinterpretation also earned score of 20.
In conclusion, while all models gave some plausible biological functions, none of them fully cap- tured the detailed and specific roles ATAD3 within the mitochondrial network, as outlined in the ground truth. Our method, leveraging context, was able to provide the most accurate and thorough analysis of the protein's ss function, demonstrating the advantage of context-driven approaches over sequence-based models in protein functional prediction tasks.
L THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this work, we employed a Large Language Model (LLM) as an assistive tool to enhance the quality of our writing. Specifically, the LLM was utilized identify and correct grammatical errors, as well as to refine the phrasing and style of the text to improve clarity and readability. Additionally, the LLM assisted formatting the prompts within the "QA examples" section ensure consistency and alignment with our intended experimental setup.
In accordance with the ICLR policies LLM usage, we disclose our the LLM in this section and affirm that the assistance provided by the LLM did not compromise the originality or integrity of our work. We take full responsibility for the content of this manuscript and have ensured that all information is accurate and appropriately referenced.
M DETAILED COST ESTIMATION
To ensure transparency, provide the basis for cost estimations using publicly available demand pricing and API costs.
Settings
CPU Instance (Single): An economical instance such as c6a. c6a.xlarge xlarge) $0.153 hour.
CPU Instance (Batch): powerful instance like c6a. 24xlarge $2.7 (each 1.35) / hour.
GPU Instance (Single A A100): A single A100 instance at $2.75 / hour.
LLM API Cost: A single call to DeepSeek-Vy V3 is estimated at $0.000446.
Single-Sequence Inference
Our Method Cost: ==$0.0030
1. CPU Cost: ($0.153/ 3600 seconds) & 60 seconds x $0.00255.
2. API Cost: $0.000446.
3. Total: $0.00255 $0.000446 $0.002996 $0.0030.
38

---

Published as a conference paper at ICLR 2026
Evolla Cost: $0006690
1. GPU Cost: ($2.75 3600 seconds) 90 seconds ~ $0.06875 = $0.0690.
Batch Processing Inference The per-sequence averages for batch processing were derived by modeling large-scale run to capture amortization and throughput effects.
Our Method: $00.00054 per sequence
1. Throughput Model: 2 powerful CPU machines process 1.12M sequences in 40 hours.
2. CPU Cost: (2 machines * 40 hours * $1.35/hourr) 1.12M seq ss $0.000096/seq.
3. API Cost: $0.000446/seq.
4. Total: $0.000096 $0.000446 $0.00054/seq.
Evolla: $0.01552 per sequence
1. Throughput Model: 1 A100 machine (with 20x concurrency) processes 180 sequences
per hour.
2. GPU Cost: (1 hour $2.75/hour) 180 seq = $0.0152/seq.
N
WET-LAB VALIDATION SAMPLES
N.1
PROMPT
Rhodopsin
PETase
Acouracy:
Confidnce Score
1.0
Confidence Score
1.0
XXX
Incorrect (28)
0.8
0.8
X00000000X
0.4
0.2
0.2
Incorrect (4)
0.0
0.0
12
15 18
12
16
20
24
32
36
Figure 12: Sample-level performance classification accuracy of Intern-S1.
To evaluate the models' performance on the novel protein classification task. we designed stan- dardized prompt that frames the problem as a zero-shot binary classification. For each protein family under investigation (i.c.. Rhodopsin or PETase), the model was tasked with determining whether the provided input protein belonged to the specified family to a generic "Other" category. For the structure-aware model, Evolla, we first generated high-quality three-dimensional structure predic- tions for each novel sequence using the state-of-the-art AlphaFold3. It should be noted that the MatureLM model consistently failed to produce coherent or valid classifications across samples in preliminary tests; it was therefore excluded from the final comparative analysis maintain focus on viable models. Intern-S1 results are shown in Figure
Prompt for Novel Protein Sequence Classification
What is the category of this protein?
(1) Rhodopsin/PETTase (2) Other
Please provide your answer the following structured format: Category: [1, 2] Confidence score: [a score between and 1.0]
39


Reviews:
[{"review_id": "mP2ddusOo8", "reviewer": "Reviewer_oqHE", "summary": "This paper tackles how Sci-LLMs handle biomolecular sequences. It argues current methods are stuck in a \"tokenization dilemma\": either they treat sequences as language, breaking up important motifs, or as a separate modality, which creates an alignment gap. The authors propose a \"context-driven\" approach, skipping raw sequences entirely. Instead, they use bioinformatics tools (BLAST, Pfam) to create a text summary for the LLM . Their experiments show this context-only method works best, and that adding the raw sequence back in actually hurts performance, acting like noise.", "questions": "Given the issue with mutations, do you have ideas for how this context-driven method could be adapted for those tasks? Maybe by using different tools that are sensitive to mutations to generate the context?\n\nYou mention your method is efficient because it avoids retraining, but running tools like InterProScan and BLAST for every query isn't free. How does the real-world inference time/cost of your pipeline compare to running a big, end-to-end model?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper's \"tokenization dilemma\" concept is a really clear and smart way to frame a major hurdle for Sci-LLMs. The main idea—that feeding LLMs text context from tools like BLAST is better than giving them the raw sequence—is surprising but backed up well by the experiments. The finding that raw sequences just add \"noise\" and make things worse is a big deal. The visualizations (like in Figure 3) showing how alignment fails are also very convincing . This work is important because it questions the push for end-to-end models and offers a practical, hybrid alternative.", "weaknesses": "The main drawback, which the authors rightly point out, is that this method can't handle mutation effect prediction. The bio-tools (BLAST, etc.) used to create the context just aren't sensitive to tiny, single-point changes, so the context for a normal protein and its mutant look the same . This is a major limitation, as it rules out a big area of computational biology. Also, the claims about it working on DNA are mostly tucked away in the appendix, not fully explored in the main paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DHv82GdNeg", "reviewer": "Reviewer_CSFM", "summary": "The paper proposes a “paradigm shift” for how Scientific Large Language Models (Sci-LLMs) are trained, leveraging context-centric approaches driven by high-level structured knowledge from bioinformatics tools (e.g., GeneOntology, ProTrek, BLASTp, etc.).  The solution addresses two key tokenization “dilemmas” that have posed challenges on the Sci-LLM space: sequence-as-language and sequence-as-modality.  This approach accounts for multiple levels of language used to describe biomolecular phenomena – from human-encoded knowledge to genetics/evolutionary-encoded knowledge. Strikingly, the context-only approach largely outperforms joint context + raw sequences, suggesting that raw sequences contribute more to information noise.  The contribution suggests that Sci-LLMs don’t necessarily require solving complex biological “language” from scratch but can leverage decades of accumulated biological knowledge contained within structured databases.", "questions": "1.\tHow do you address the circular reasoning between the strengths of the approach (incorporating high-quality expert annotations) and using this approach to predict those annotations where they do not yet exist?  Could tool-calling agents solve this rather than building directly into the LLM?  What are the tradeoffs?\n2.\tAlong this line of questioning, does the core contribution put a focus on the LLMs, or are you simply demonstrating that tradition bioinformatics pipelines already solve most of the problems around understanding protein function?\n3.\tHave these results been validated against human expert annotators?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.\tOverall: The paper and aims to address a novel challenge in the Sci-LLM space, making a case that Sci-LLMs are better served as “reasoning engines over expert knowledge”, rather than pure sequence decoders. While this is noted and there is some evidence that this is the case, it does raise some circular logic around the quality of the annotations derived from the bioinformatics knowledgebases (addressed below in the weaknesses).\n2.\tGeneralizability: The solution in generalizable, with applications ranging from known proteins to “novel” proteins, as well as different biomolecular types.\n3.\tPracticality: The solution as it is described is practical, as it allows to more easily keep models up to date with new biological knowledge with lower development costs.  (Although it could be argued that most of the effort is derived from maintaining the bioinformatics knowledgebases).", "weaknesses": "1.\tCircular Logic: The approach works well when high-quality annotations exist, yet the solution also exists to propose annotations to fill in knowledge gaps. This counter-intuitively raises a bit of a “Catch 22” scenario.\n2.\tCore Argument: The basis of the manuscript suggests that there is in fact valuable information encoded within the evolutionary language through sequence tokens, yet the results suggest the opposite – and that human context exclusively drives the value.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3XgS2SWa7e", "reviewer": "Reviewer_J4os", "summary": "This paper argues that Scientific Large Language Models face a \"tokenization dilemma,\" struggling to interpret raw biomolecular sequences, which are either broken down into meaningless components or difficult to align with natural language. Through systematic experiments, the authors demonstrate that a \"context-only\" approach, where models are given high-level, human-readable knowledge from bioinformatics tools (like BLAST or Pfam) , consistently and substantially outperforms models given the raw sequence.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Pros:\n- The authors proposed a new “context-only” method, which achieved significantly \n- The context-driven approach achieve good performance.", "weaknesses": "Cons:\n- Context-only approach sounds interesting. However, compared with raw biomolecular sequences input, an inevitable con of this approach would be significant information loss (by discarding too many detailed information).\n- The capability of this approach is capped by the bioinformatics tools being used, e.g., InterProScan and BLAST.\n- As the context-only model relies majority on prior, it may not be a good tool for exploring “novel” findings (which may be out of distribution a bit).\n- Why in Table 1, QWEN series of models are not considered, while in Figure 2, for “ours” model, the author choose to use Qwen-embedding. What about the embedding visualization for specialized language models [1] like ESM series\n\n\n[1] Zheng, Y., Koh, H. Y., Ju, J., Yang, M., May, L. T., Webb, G. I., ... & Church, G. (2025). Large language models for drug discovery and development. Patterns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b9E8GWVKK2", "reviewer": "Reviewer_2DYC", "summary": "This paper identifies and investigates a fundamental challenge in Scientific Large Language Models (Sci-LLMs) for biomolecular understanding, which the authors term the \"tokenization dilemma.\" They argue that existing paradigms—\"sequence-as-language\" (tokenizing sequences into atomic units) and \"sequence-as-modality\" (encoding sequences via specialized encoders)—suffer from weak representation and semantic misalignment, respectively. As a solution, the authors propose a \"context-driven\" paradigm, which bypasses raw sequence input. Instead, it leverages established bioinformatics tools (e.g., InterProScan, BLASTp) to generate high-level, human-readable textual context (e.g., functional domains, GO terms) that is natively aligned with the LLM's linguistic space.  The authors evaluated three input modes: sequence-only, context-only, and a combination of both. Through extensive empirical evaluation on protein QA, EC number prediction, and DNA mutation tasks, the authors demonstrate that the context-only approach consistently and substantially outperforms all other modes. They find that adding raw sequence information to context often degrades performance, acting as \"informational noise.\"", "questions": "- Given the high performance of general-purpose LLMs like Deepseek-v3, Gemini2.5 Pro and GPT-5, what steps did you take to ensure that the ground-truth annotations for your test proteins were not present in these models' pre-training data? Could the results be partly explained by memorization rather than reasoning?\n- Your approach depends on external tools. Can you provide a qualitative analysis or failure case study for proteins where InterProScan and BLASTp return no or incorrect hits? How does the performance of your method degrade in such \"orphan\" scenarios, and what are the potential remedies?\n- The paper convincingly shows that context is superior for *retrieving* known functional annotations. However, do you believe your paradigm can be extended to tasks that require *discovering* novel functions or reasoning about structure-sequence relationships that are not yet captured in existing databases?\n- You note your method is computationally efficient as it avoids Sci-LLM retraining. However, running InterProScan and BLASTp for every query in a real-time application could be costly and slow. Could you comment on the latency and scalability of the full context-generation pipeline compared to a single forward pass of a sequence-as-language and a sequence-as-modality model?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper clearly articulates the \"tokenization dilemma\" as a critical, yet overlooked, bottleneck in Sci-LLMs. The conceptual framing of the two existing paradigms and their respective weaknesses is compelling and well-supported by prior work.\n- The central claim—that raw sequences can be detrimental when combined with high-level context—is counter-intuitive and strongly supported by systematic experiments across multiple models (Intern-S1, Evolla, NatureLM, GPT-4o, etc.) and tasks (protein function, pathway, localization, EC prediction). The consistent performance drop in \"Sequence + Context\" settings is a powerful result.\n- The authors evaluate their method on a wide range of benchmarks, including their own reconstructed dataset, temporal splits, and sequence identity-based splits (Easy/Medium/Hard). The inclusion of DNA-based tasks also demonstrates generalizability beyond proteomics.\n- The paper goes beyond mere performance comparisons. The layer-wise analysis of Evolla (Section 5.3, Appendix F) convincingly shows how semantic alignment (via Q-Former) erases fine-grained mutation signals, providing a mechanistic explanation for the limitations of the sequence-as-modality approach.", "weaknesses": "- The context-driven approach relies heavily on the quality and coverage of external tools (InterProScan, BLAST). While an ablation study is provided (Appendix E), it does not fully explore the performance ceiling—what happens when these tools fail completely on highly novel proteins? The method's performance is inherently tied to the underlying databases' completeness and timeliness.\n- The paper equates \"biomolecular understanding\" primarily with high-level functional annotation (GO terms, pathways). It does not assess whether the model gains *mechanistic* or *structural* insights that might require raw sequence analysis (e.g., predicting the effect of a point mutation). The limitation section (Appendix J) correctly notes this but underscores a fundamental constraint of the proposed paradigm.\n- The strong performance of general LLMs (Gemini, GPT) in the context-only setting raises questions about potential memorization of public protein annotations from their vast pre-training corpora. While the authors take care to prevent label leakage in their *context generation*, they do not explicitly audit whether the test proteins' annotations were already in the LLMs' training data.\n- The primary metric (LLM-Score) relies on another LLM (DeepSeek-V3) to judge answer quality. While this is a reasonable approach for open-ended QA, it introduces potential biases and lacks the objectivity of exact-match metrics used in tasks like EC prediction.\n- Code is not provided in the current submission, providing it would be helpful to make work reproducible.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)