Protein as a Second Language for LLMs

## Abstract

Abstract Deciphering the function of unseen protein sequences is a fundamental challenge with broad scientific impact, yet most existing methods depend on task-specific adapters or large-scale supervised fine-tuning. We introduce the “ Protein-as-Second-Language ” framework, which reformulates amino-acid sequences as sentences in a novel symbolic language that large language models can interpret through contextual exemplars. Our approach adaptively constructs sequence–question–answer triples that reveal functional cues in a zero-shot setting, without any further training. To support this process we curate a bilingual corpus of 79,926 protein–QA instances spanning attribute prediction, descriptive understanding, and extended reasoning. Empirically, our method delivers consistent gains across diverse open-source LLMs and GPT-4o, achieving up to 17.2% ROUGE-L improvement (average +7%) and even surpassing fine-tuned protein-specific language models. These results highlight that generic LLMs, when guided with protein-as-language cues, can outperform domain-specialized models, offering a scalable pathway for protein understanding in foundation models.

## 1 Introduction

Proteins are indispensable molecular machines of life, driving key functions such as maintaining cell structure and enabling cell communication.
Their three-dimensional architectures, catalytic activities, interaction networks, and evolutionary trajectories are all encoded within a linear sequence composed of twenty amino-acid characters (22, 55).
Therefore, the core of understanding protein function lies in accurately “reading” and “translating” the biological meaning contained within these amino-acid sequences (8, 23).
However, this task is fraught with challenges. Although the amino acid sequence is formally like a language—possessing a fixed character set (over 20 genetically encoded amino acids) and potential grammatical rules (physicochemical laws)—the mapping relationship from the one-dimensional sequence to the three-dimensional structure and function is extremely complex and highly context-dependent (39, 49).
Consequently, the central challenge of “what cellular function does an unknown amino acid sequence encode?” still lacks a comprehensive solution.

To address this challenge, research efforts on protein understanding can be broadly categorized into two dominant paradigms:
protein representation learning and protein–language alignment modeling.
Protein representation learning sees amino-acid sequences as a standalone modality like language and visual, acquires universal protein representations through self-supervised pre-training on large-scale amino-acid sequences, and then attaches lightweight decoders to predict structure or function (59, 3, 25, 42, 6, 52, 51). While this paradigm excels in the universality of its embeddings and in mining deep sequential patterns, these embeddings still rely on additional “interpreters”, *i.e.*, post-processing adapters, to be converted into human-understandable explanations.
Protein–language alignment modeling, in contrast, co-trains on paired protein sequences and their textual descriptions, establishing a bidirectional mapping within a shared latent space that enables end-to-end text-based question answering (56, 37, 15, 1, 48, 54). Although this route bypasses downstream adapters, it is intrinsically bound to large-scale paired data and often requires re-fine-tuning whenever the output format or downstream objective shifts.
In summary, both of these approaches face bottlenecks of large training data requirements, high computational costs, and limited generalization ability.

##### Protein as Second Language.

Reflecting on the human cognitive process, we observe that humans exhibit remarkable efficiency and generalization ability when learning a brand-new symbolic system (i.e., a new language). The key lies in their ability to rely on and transfer their existing native language knowledge system (13, 20).
Given the aforementioned “linguistic” properties of protein sequences—possessing a compositional structure and contextual semantics—and our goal of understanding their function using natural language, we propose a novel perspective: to treat protein sequences as a symbolic system that can be learned and interpreted by large language models (LLMs) as a “second language”.

Analogous to how humans acquire a second language, *i.e.*, by encountering new words in context and inferring their meaning and usage, we propose a protein language learning framework in which an LLM acquires protein semantics and reasoning ability through context-driven exposure that grounds sequence patterns in functional and structural examples. This framework adaptively constructs learning contexts for a given protein understanding goal, enabling rapid acquisition of target protein knowledge without additional training or sacrificing generalization.
To support effective learning, we constructed a “bilingual” dataset of 79,926 protein-sequence–question–answer triples covering functional, descriptive, and extended-information queries.
Across Protein2Text (58), Mol-Instructions(11) and ProtDescribe-QA (19), our framework raises the average ROUGE-L by 7% across diverse open-source models and GPT-4o, with a maximum gain of 17.2%, without any task-specific fine-tuning.
Our contributions are as follows:

- •
We introduce the “Protein-as-Second-Language” conceptual framework, which recasts amino-acid sequences as a second language that can be acquired via in-context learning, enabling efficient and generalized protein understanding.
- •
We construct a protein-natural language bilingual dataset that spans four task families: attribute-based QA, True or False QA, descriptive-text QA, and extended-information QA, to support effective protein language learning and benchmarking.
- •
We present a protein language learning framework that adaptively constructs learning contexts for protein understanding, yielding significant gains for both open-source models and GPT-4o, enabling them to outperform domain-specialized models without additional training.

## 2 Related Work

### 2.1 Language Models in Protein

Protein representation learning with protein language models (PLMs) extends the Transformer to amino-acid strings, producing dense embeddings for property prediction (16, 3, 10, 17, 4, 6, 7) or generative design (32, 34, 29, 12).
Because these models are trained exclusively on amino acid sequences, their outputs remain latent vectors that external classifiers must translate into human-readable function.
To obviate this indirection, protein–language alignment modeling has emerged, which jointly connects sequences with textual descriptions via (i) contrastive objectives mapping proteins and sentences into a shared space (57, 51), (ii) bioknowledge-augmented pre-training on curated protein–text corpora (12, 44, 29, 36, 60, 26), or (iii) multi-modal LLMs that graft protein encoders onto frozen language backbones (27, 1, 48, 7, 31, 53).
While effective, these approaches entail costly retraining or gradient updates and risk catastrophic forgetting when scaled to larger LLMs (21, 50), prompting a shift toward parameter-efficient adaptation.

### 2.2 Protein QA Datesets

Datasets that couple proteins with natural-language annotations have become the empirical bedrock for developing protein–text hybrid systems. At present, two complementary families of corpora dominate the landscape. The first centers on protein captioning: given an amino-acid sequence alone, the objective is to generate a concise textual description. Representative instances include the richly annotated Swiss-Prot collection (2), the ProteinKG resource (59) and ProtDescribe (58). The second family targets protein question answering: here, both a sequence and a natural-language query are supplied, and the model is required to synthesize an answer grounded in the provided protein. Curated examples span Mol-Instructions (11), UniProtQA (28), ProteinLMBench (40), VenusX (43) and Protein2Text-QA (19).

## 3 Protein as Second Language

We introduce “Protein-as-Second-Language”, a framework that treats amino-acid sequences as a new symbolic system to be learned much like humans acquire a foreign language. Just as learners infer the meaning of unfamiliar words by repeatedly encountering them in context, we construct a protein–natural language bilingual dataset (Sec. [3.1](https://arxiv.org/html/2510.11188v1#S3.SS1)) and design an adaptive context construction mechanism (Sec. [3.2](https://arxiv.org/html/2510.11188v1#S3.SS2)) to provide such contextual exposure. In this way, our framework enables LLMs to acquire protein semantics through exemplars rather than through extensive re-training.

### 3.1 Bilingual Dataset Construction

We curate our bilingual dataset in three steps (Figure [1](https://arxiv.org/html/2510.11188v1#S3.F1)). Starting from 573,661 Swiss-Prot (2) entries with gene ontology (GO) annotations, we avoid directly converting all annotations, as this would introduce heavy redundancy; instead, we construct a balanced sample. Specifically, (i) we prune the GO-directed acyclic graph (GO-DAG) to obtain representative functional categories and group proteins accordingly (Sec. [3.1.1](https://arxiv.org/html/2510.11188v1#S3.SS1.SSS1)), (ii) perform bilingual deduplication by clustering sequences within each protein group and sampling proteins with diverse functional annotation (Sec. [3.1.2](https://arxiv.org/html/2510.11188v1#S3.SS1.SSS2)), and (iii) use DeepSeek-R1 (14) to generate attribute, knowledge, descriptive, and true/false QA pairs, yielding 79,926 high-quality protein–QA triples (Sec. [3.1.3](https://arxiv.org/html/2510.11188v1#S3.SS1.SSS3)).

Figure: Figure 1: The overview of data construction of our bilingual protein–QA dataset.
Refer to caption: x1.png

#### 3.1.1 Function-Based Grouping

To enable representative sampling across functional categories, the dataset is partitioned according to the GO hierarchy. Directly using the raw directed acyclic graph (DAG) risks over-fragmentation from overly fine, sparsely populated terms, and excessive generalization near the root. To address this, we adapt a pruning strategy inspired by decision tree simplification (33), where complexity is managed through a penalty to avoid overfitting. This strategy aims to retain an optimal set of GO terms as functional grouping nodes. It balances granularity and coverage, ensuring that the retained nodes represent biologically diverse yet statistically well-supported categories for downstream sampling.

The pruning process is driven by two main criteria:
(1) A node is retained if it meets the minimum support threshold, which ensures that the node has a sufficient number of associated proteins, and does not exhibit significant child imbalance.
(2) If the child-imbalance ratio is high, meaning the protein distribution among a node’s child terms is uneven, the parent node is retained, even if the child nodes fail to meet the minimum support threshold.

##### Minimum Support Threshold

A node is retained only if the number of associated proteins meets a depth-adjusted threshold $m(d)$, which adapts based on the node’s depth in the GO hierarchy. The threshold is calculated as:

$$ $m(d)=\lambda\cdot C_{tot}\cdot(1+\beta d)$ (1) $$

where $C_{tot}$ is the total protein count, $d$ is the node depth, and $\lambda$ and $\beta$ are constants. This dynamic threshold is designed to prevents deep nodes from splitting infinitely due to overly small absolute values.

##### Child-Imbalance Ratio

The child-imbalance ratio is applied to assess whether the child nodes of a given term are too imbalanced. The imbalance ratio $\rho(v)$ is computed as the ratio of the largest to the smallest protein count among the child nodes:

$$ $p(u)=\frac{\displaystyle\max_{u\in C^{+}(v)}C(u)}{\displaystyle\min_{u\in C^{+}(v)}C(u)}$ (2) $$

where $C^{+}(v)$ represents the set of valid child nodes with non-zero protein counts. If the imbalance ratio $\rho(v)$ exceeds a specified threshold $\tau(d)$, the parent node $v$ is retained to preserve the biological diversity. This threshold is adjusted dynamically with the depth $d$ to allow for greater flexibility at deeper levels of the hierarchy:

$$ $\tau(d)=\tau_{0}\cdot\alpha^{d}$ (3) $$

where $\tau_{0}$ is the base threshold, and $\alpha$ is a scaling factor.

By applying these two criteria, the pruning process is carried out recursively, allowing the algorithm to adaptively prune the GO DAG and identify the most relevant, biologically diverse functional groups.

#### 3.1.2 Bilingual deduplication

After grouping by GO term, proteins within the same node often exhibit high similarity, as they represent homologous proteins.
To address this, we use MMseqs2 (41) for sequence clustering within each GO node, applying a 70% amino acid sequence similarity threshold. From each cluster, a single representative sequence is selected. This threshold efficiently removes redundant sequences with minimal functional variation while preserving functional diversity.

While sequence similarity-based redundancy removal effectively reduces sequence-level redundancy, it does not necessarily capture functional divergence. Specifically, sequence similarity below 70% does not imply functional divergence, and substantial functional redundancy may still exist within the set (9). To address this, we focus on annotation semantic similarity, quantifying the functional relationships between proteins based on their GO annotations. Inspired by the simGIC method (38) for calculating GO terms semantic similarity, we calculate the Protein Functional Information Content $\mathrm{IC}_{\text{protein function}}$ for each protein, which is the sum of the Information Content (IC) of all associated GO terms and their ancestral terms. The IC of each GO term is calculated based on its frequency in the dataset, using the total protein set after sequence redundancy removal. The $\mathrm{IC}_{\text{protein function}}$ value for each Protein ID is computed as:

$$ $\text{IC}_{\text{protein function}}=\sum_{g\in\text{GO terms of }p}\text{IC}(g)+\sum_{g^{\prime}\in\text{ancestors of GO terms of }p}\text{IC}(g^{\prime}).$ (4) $$

This provides a quantitative measure of each protein’s functional information, capturing both direct and indirect annotations. For each GO term, proteins are sampled based on their unique $\mathrm{IC}_{\text{protein function}}$ values (rounded to 3 decimal places). To ensure balanced species representation, a species quota strategy is applied based on the proportions of Eukaryota, Bacteria, Archaea, and Viruses in the dataset after sequence redundancy removal. This ensures an unbiased species distribution in the final sample.
The bilingual deduplication process reduces redundancy in two aspects, amino acid sequence and annotation semantics, ensuring a balanced and diverse protein corpus.

#### 3.1.3 LLM-based QA Construction

To transform curated protein annotations into natural-language question–answer pairs, we prompt the DeepSeek-R1 (14) model to generate biologically grounded QA texts that reflect both functional attributes and contextual knowledge (the prompts used for each QA type are provided in Appendix [D](https://arxiv.org/html/2510.11188v1#A4)). The resulting QA corpus covers four complementary types:
1 Attribute-based QA captures factual properties directly associated with a protein, such as molecular function, cellular component, or family.
2 Knowledge-based QA comprises concise, annotation-driven questions and answers that involve in multiple biological aspects of a protein, such as expression, localization, mechanism, and interactions.
3 Descriptive Text QA produces longer natural-language explanations that integrate multiple annotations into coherent functional summaries.
4 True or False QA consists of single statements that integrate multiple biological aspects of a protein, accompanied by a True/False answer and a brief explanation.

These four types yield a rich and varied bilingual dataset, ensuring that models are exposed to both concise factual knowledge and more detailed contextual explanations, supporting their ability to understand and reason about protein functions.

### 3.2 Bilingual Contextual Learning

Figure: Figure 2: Process of Query-Adaptive Context Construction.
Refer to caption: x2.png

In practical scenarios, questions concerning protein sequences are often highly flexible and complex: they require not only analogous proteins with similar sequence patterns to capture potential structural or functional signals, but also complementary descriptive knowledge and QA pairs to provide semantic grounding.
As shown in Figure [2](https://arxiv.org/html/2510.11188v1#S3.F2), we propose an adaptive context construction mechanism, for bilingual contextual learning, designed to selectively build bilingual learning contexts for each query.
Instead of brute-force mixing of amino acid sequences and descriptive texts, the mechanism follows the principle of second language acquisition—exposing learners to new words in context so that meaning and usage can be inferred (18).
By analogy, LLMs acquire protein semantics and reasoning ability through context-driven exposure that grounds sequence patterns in functional and structural exemplars.

The mechanism operates in three stages. First, the adaptive context provider selects candidate contexts from the protein–natural language bilingual dataset, guided by two criteria:
(i) amino acid sequence homology between candidate proteins and the query sequence, computed with MMseqs2 (41), and (ii) similarity between the descriptive texts or QA pairs of candidate proteins and the query question.
Second, the contextual integration module structures the selected examples into a coherent context. Finally, the constructed bilingual context is combined with the query and presented to the LLM as in-context examples, enabling analogy-based reasoning and evidence integration to produce biologically meaningful responses.

## 4 Experiments

### 4.1 setup

##### Evaluation Datasets

We comprehensively evaluated our method using 3 text-based protein understanding datasets: 1 ProtDescribe (58) comprises 553,052 high-quality protein–text pairs extracted from Swiss-Prot. Each instance pairs an amino-acid sequence with a single textual description obtained by concatenating four annotation fields in a fixed order: protein name, function, subcellular location, and similarity. The resulting descriptions average 40–60 tokens.
2 Protein2Text-QA (19) comprises 209,847 open-ended question–answer pairs covering 5,574 unique proteins. Each instance consists of an amino-acid sequence, a free-form question, and a concise answer; all QAs are automatically generated from PubMed abstracts/discussion/introduction sections and presented as conversational natural-language text without fixed templates.
3 Mol-Instructions (11) comprises 2.04 M instruction instances divided into three major sections: molecule-oriented, protein-oriented, and biomolecular-text. The protein-oriented section alone contributes 505 K instructions covering diverse tasks. Each sample is formatted as a natural-language “instruction–input–output” triplet: the input is a UniProt amino-acid sequence, and the output is a free-text answer tailored to the specific task.

##### Models

All experiments are conducted under identical prompting protocols. We first evaluate the proposed adaptive context construction method on frozen LLMs, including Qwen2.5-3B (46), Mistral-7B-Instruct-v0.3 (5), Qwen3-14B (47), Kimi-k2 (45), and GPT-4o (35), to test few-shot and compositional reasoning capabilities, thereby mimicking the dynamics of second language acquisition. In addition, we also evaluate fine-tuned protein-oriented LLMs, such as BioT5-plus-base (37) and ProLLaMA (30), which have been explicitly trained on large-scale protein corpora. These models serve as a baseline for comparison, allowing us to examine the performance gains of our method in general-purpose frozen LLMs relative to specialized protein LLMs.

##### Metrics

We evaluate model outputs using both an automatic metric (ROUGE-L (24)) and human evaluation. ROUGE-L (24), though widely used for text generation, primarily measures lexical overlap and may not fully capture semantic correctness in protein-related QA. To address this limitation, five evaluators rated the quality of generated answers on a 0–5 scale, where 0 denotes garbled and unreadable content, intermediate scores reflect increasing levels of informativeness and accuracy, and 5 represents fully correct outputs (detailed scoring rubrics are provided in Appendix [A](https://arxiv.org/html/2510.11188v1#A1)). This combined evaluation provides a more reliable assessment of factual accuracy and overall comprehensibility.

### 4.2 Quality of Dataset

Figure [3](https://arxiv.org/html/2510.11188v1#S4.F3) (a-f) provides a multidimensional analysis of the protein sequences included in our dataset. The collection spans a wide range of sequence lengths, from short peptides to large multi-domain proteins, and covers proteins from 4,135 species across diverse evolutionary lineages. At the family level, the dataset comprises 63,749 families and 1,115 superfamilies, ensuring representation of both well-studied proteins and rare functional groups. Additional annotations capture domain composition, catalytic activity classes, and gene ontology categories, collectively highlighting the long-tail distribution across sequence space and functional categories.
This diversity ensures broad biological coverage while posing realistic challenges in inferring functions for proteins, particularly for infrequent families and underexplored functions.

Figure [3](https://arxiv.org/html/2510.11188v1#S4.F3) (g,h) summarizes the distribution of tasks and token composition within the dataset. The corpus encompasses four distinct protein-QA types, with sample counts ranging from 11,693 (attribute-based QA) to 32,444 (true/false QA), thereby providing balanced coverage across multiple functional perspectives.
In terms of token composition, amino-acid sequences constitute nearly 70 % of the corpus, reflecting the sequence-centric nature of protein understanding tasks and highlighting the need for models to align symbolic sequence information with natural-language context effectively.

Figure: Figure 3: Dataset statistics. Left: Multidimensional analysis of protein amino-acid sequences, including length, domain composition, and catalytic activity. Right: Sample sizes for the four protein-QA types and the ratio of textual to amino-acid sequence tokens.
Refer to caption: x3.png

### 4.3 Main Results

##### Accuracy gains from context-driven exposure

Table [1](https://arxiv.org/html/2510.11188v1#S4.T1) presents that our method consistently improves performance on three text-based protein understanding datasets.
Our method raises the average ROUGE-L by 7% across diverse open-source models and GPT-4o (35), with a maximum gain of 17.2%, demonstrating that context-driven exposure allows LLMs to acquire protein semantics and reason about function directly from sequence and textual context without any parameter updates.
Larger models benefit more, suggesting that greater capacity enhances the ability to leverage contextual cues, consistent with learning protein meaning through in-context analogy and reasoning.
In contrast, fine-tuned protein LLMs such as ProLLaMA-7B (30) do not surpass frozen LLMs augmented with our method, likely due to limited training coverage and task-specific rigidity. This underscores the our method as a lightweight alternative that enables general-purpose LLMs to potentially exceed the performance of domain-adapted models.

**Table 1: Comparison of different approaches in protein question answering “Fun.”, “Des.”, “Dom.”, and “Cat.” denote the 4 protein-oriented tasks in the Mol-Instructions dataset (11): protein function prediction (Fun.), general textual description generation (Des.), domain/motif recognition (Dom.), and catalytic activity prediction (Cat.). $\Delta$ Gain shows the percentage performance increase. $\diamondsuit$ indicates LLMs augmented with our adaptive context construction method. Metric: ROUGE-L.**
| Model | ProtDescribe | Protein2Text- | Mol-Instructions |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| QA | Func. | Desc. | Dom. | Cat. | Avg. |  |  |
| \rowcolorgray!20<br>Fine-tuned LLM |  |  |  |  |  |  |  |
| BioT5+ (37) | 9.97 | 6.96 | 2.92 | 6.22 | 2.37 | 2.87 | 3.60 |
| ProLLaMA-7B (30) | 12.77 | 10.09 | 16.89 | 15.34 | 15.85 | 19.32 | 16.85 |
| \rowcolorgray!20<br>Frozen LLM |  |  |  |  |  |  |  |
| Qwen2.5-3B (46) | 18.45 | 23.21 | 18.91 | 17.18 | 18.01 | 20.05 | 18.54 |
| \rowcolor[HTML]E6F7FF<br>Qwen2.5-3B (46) $\diamondsuit$ | 27.32 | 28.66 | 22.05 | 22.23 | 25.14 | 15.96 | 21.35 |
| $\Delta$ Gain | +8.87 | +5.45 |  |  |  |  | +2.81 |
| Mistral-7B-Instruct-v0.3 (5) | 15.02 | 20.97 | 17.05 | 18.59 | 14.95 | 18.07 | 17.17 |
| \rowcolor[HTML]E6F7FF<br>Mistral-7B-Instruct-v0.3 (5) $\diamondsuit$ | 29.39 | 28.59 | 15.77 | 22.72 | 17.46 | 21.20 | 19.29 |
| $\Delta$ Gain | +14.37 | +7.62 |  |  |  |  | +2.12 |
| Qwen3-14B (47) | 23.20 | 21.02 | 15.80 | 12.75 | 15.81 | 14.06 | 14.61 |
| \rowcolor[HTML]E6F7FF<br>Qwen3-14B (47) $\diamondsuit$ | 35.53 | 25.93 | 20.17 | 17.37 | 18.47 | 23.25 | 19.82 |
| $\Delta$ Gain | +12.33 | +4.91 |  |  |  |  | +5.21 |
| kimi-k2 (45) | 26.74 | 17.33 | 12.60 | 12.36 | 10.32 | 15.97 | 12.81 |
| \rowcolor[HTML]E6F7FF<br>kimi-k2 (45) $\diamondsuit$ | 35.91 | 21.04 | 14.47 | 14.97 | 15.68 | 17.02 | 15.54 |
| $\Delta$ Gain | +9.17 | +3.71 |  |  |  |  | +2.72 |
| GPT-4o (35) | 18.29 | 20.84 | 16.89 | 14.50 | 16.74 | 20.00 | 17.03 |
| \rowcolor[HTML]E6F7FF<br>GPT-4o (35) $\diamondsuit$ | 35.53 | 26.86 | 20.24 | 19.23 | 17.46 | 22.61 | 19.89 |
| $\Delta$ Gain | +17.22 | +6.02 |  |  |  |  | +2.85 |

Human evaluation further demonstrates that exposing models to curated protein–language contexts improves the perceived quality of outputs (Figure [4](https://arxiv.org/html/2510.11188v1#S4.F4)).
Across all rated instances, inter-rater consistency was substantial (Krippendorff’s $\alpha$ = 0.72%), ensuring reliable annotations; the detailed rubric is given in Appendix [A](https://arxiv.org/html/2510.11188v1#A1).
Models receiving context-driven exposures achieve higher or comparable ratings on most tasks (left panel), with the clearest improvements observed on Protein2Text-QA (19) and several Mol-Instructions (11) subtasks.
Furthermore, pairwise win/lose analyses (right panel) show that outputs generated with context-driven exposure are preferred in the majority of comparisons, with win rates systematically exceeding loss rates.

Figure: Figure 4: Comparison of human evaluation results. Left: Absolute human rating scores (0–5) for zero-shot model outputs (dark bars) and model outputs with adaptive context exposure (light bars) on three datasets. Right: Pairwise win/lose proportions comparing outputs with and without adaptive context exposure. Each comparison is based on 8 randomly selected cases per subset (48 cases in total across six subsets).
Refer to caption: x4.png

##### Varying exemplar number ( k k )

Figure [5](https://arxiv.org/html/2510.11188v1#S4.F5) illustrates how model performance varies with the number of exemplars ($k$) provided in context across different datasets. Performance generally improves as $k$ increases, but only up to a task-dependent optimum; beyond this point, additional exemplars offer little benefit or even introduce noise.
The optimal $k$ differs by task. For ProtDescribe (58), which involves fixed attribute-centric questions, a larger set of bilingual exemplars from related proteins helps the model capture recurring patterns, with performance peaking at $k=10$–$11$. In contrast, Protein2Text-QA (19) requires open-ended and integrative reasoning, where only a small number of highly relevant exemplars are beneficial; here, performance peaks earlier at $k=3$–$4$.
In our experiments, we therefore adopt the task-specific optimal settings: $k=11$ for ProtDescribe (58), $k=4$ for Protein2Text-QA (19), and $k=4$ for Mol-Instructions (11).

Figure: Figure 5: Effect of varying exemplar number ($k$) on model performance. We explored $k\in[1,12]$ as the search space; the upper bound was set after a coarse scan up to $k=50$ showed performance saturation around 2-12 exemplars. Metric: ROUGE-L.
Refer to caption: x5.png

##### Ablation on dual-criterion context selection

Table [2](https://arxiv.org/html/2510.11188v1#S4.T2) shows that using both sequence homology and text/QA similarity (Dual) outperforms either criterion alone, providing complementary signals that maximize the effectiveness of context-driven exemplar selection. On average across three datasets, using only sequence homology reduces performance by 5.2%, and using only text/QA similarity reduces performance by 2.8% compared to Dual, though all variants still outperform zero-shot models.

**Table 2: Ablation on dual-criterion context selection. Columns show model performance when using both sequence homology and text/QA similarity (Dual), sequence homology only (SeqOnly), or text/QA similarity only (QAOnly) to guide exemplar selection. Metric: ROUGE-L.**
| Model | ProtDescribe | Protein2Text-QA | Mol-Instructions |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dual | SeqOnly | QAOnly | Dual | SeqOnly | QAOnly | Dual | SeqOnly | QAOnly |  |
| Qwen2.5-3B (46) | 27.32 | 20.10 | 25.24 | 28.66 | 27.76 | 25.73 | 21.35 | 20.16 | 19.68 |
| Mistral-7B-Instruct-v0.3 (5) | 29.39 | 19.46 | 25.24 | 28.59 | 21.85 | 22.68 | 19.12 | 14.31 | 17.40 |
| Qwen3-14B (47) | 35.53 | 22.90 | 30.52 | 25.93 | 23.26 | 25.87 | 19.82 | 15.07 | 17.58 |
| kimi-k2 (45) | 35.91 | 28.58 | 32.43 | 21.04 | 17.10 | 18.06 | 15.54 | 12.79 | 12.09 |
| GPT-4o (35) | 35.51 | 27.72 | 32.59 | 26.86 | 23.90 | 26.45 | 19.89 | 17.46 | 17.08 |

##### Case studies and qualitative evaluation

Figure [6](https://arxiv.org/html/2510.11188v1#S4.F6) illustrates that context-driven exposure produces concise, function-specific descriptions consistent with UniProt annotations. In the two examples shown, the model correctly identifies “intrinsically disordered regions”, and “[4Fe-4S] RNA methyltransferase activity”, whereas zero-shot outputs remain generic.

Figure: Figure 6: Qualitative examples of protein question answering. We present two examples with answers generated by GPT-4o (35) along with the target ground truth. The green color highlights accurate keywords, while the red color indicates prediction errors.
Refer to caption: x6.png

## 5 Conclusion

We have proposed the “Protein-as-Second-Language” framework, which leverages adaptive context construction to enhance bilingual protein understanding by dynamically integrating sequence homology and textual similarity. Additionally, we introduced a protein-natural language bilingual dataset, specifically designed to support this framework and facilitate the bridging of protein sequences with functional descriptions. Our approach has successfully enhanced large language models’ ability to acquire protein semantics and reasoning capabilities without the need for task-specific parameter updates. Experiments on multiple protein-language datasets demonstrate that our framework consistently outperforms zero-shot baselines, highlighting the effectiveness of context-driven learning in bridging protein sequences with functional descriptions.

## 6 Ethics statement

This work complies with ethical standards and established research practices. All protein data were sourced from publicly available databases, with no proprietary or confidential information involved. Quality assurance and safety checks were applied to minimize harmful or inappropriate content. We acknowledge the broader risks of combining LLMs with biomolecular knowledge, including potential misuse for harmful purposes, and therefore emphasize responsible use guided by fairness, transparency, and accountability. Any harmful or unsafe applications of this dataset are strictly prohibited.

## 7 Reproducibility statement

We provide detailed descriptions of the protein–natural language bilingual dataset (Sec. [3.1](https://arxiv.org/html/2510.11188v1#S3.SS1), Appendix [C](https://arxiv.org/html/2510.11188v1#A3)), the adaptive context construction mechanism (Sec. [3.2](https://arxiv.org/html/2510.11188v1#S3.SS2)). Data processing steps and QA generation prompts for all four question types are included in Sec. [3.1](https://arxiv.org/html/2510.11188v1#S3.SS1) and Appendix [D](https://arxiv.org/html/2510.11188v1#A4). Code implementing the framework and instructions for reproducing experiments on both frozen and protein-adapted LLMs will be provided as supplementary material upon acceptance.

## References

- Abdine et al. (2024)
Hadi Abdine, Michail Chatzianastasis, Costas Bouyioukos, and Michalis Vazirgiannis.
Prot2text: Multimodal protein’s function generation with gnns and transformers.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 38, pp. 10757–10765, 2024.
- Bairoch & Apweiler (2000)
Amos Bairoch and Rolf Apweiler.
The swiss-prot protein sequence database and its supplement trembl in 2000.
*Nucleic acids research*, 28(1):45–48, 2000.
- Brandes et al. (2022)
Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial.
Proteinbert: a universal deep-learning model of protein sequence and function.
*Bioinformatics*, 38(8):2102–2110, 2022.
- Cao & Shen (2021)
Yue Cao and Yang Shen.
Tale: Transformer-based protein function annotation with joint sequence–label embedding.
*Bioinformatics*, 37(18):2825–2833, 2021.
- Chaplot (2023)
Devendra Singh Chaplot.
Albert q. jiang, alexandre sablayrolles, arthur mensch, chris bamford, devendra singh chaplot, diego de las casas, florian bressand, gianna lengyel, guillaume lample, lucile saulnier, lélio renard lavaud, marie-anne lachaux, pierre stock, teven le scao, thibaut lavril, thomas wang, timothée lacroix, william el sayed.
*arXiv preprint arXiv:2310.06825*, 3, 2023.
- Chen et al. (2024a)
Bo Chen, Xingyi Cheng, Pan Li, Yangli-ao Geng, Jing Gong, Shen Li, Zhilei Bei, Xu Tan, Boyan Wang, Xin Zeng, et al.
xtrimopglm: unified 100b-scale pre-trained transformer for deciphering the language of protein.
*arXiv preprint arXiv:2401.06199*, 2024a.
- Chen et al. (2024b)
Zhiyuan Chen, Tianhao Chen, Chenggang Xie, Yang Xue, Xiaonan Zhang, Jingbo Zhou, and Xiaomin Fang.
Unifying sequences, structures, and descriptions for any-to-any protein generation with the large multimodal model helixprotx.
*arXiv preprint arXiv:2407.09274*, 2024b.
- Clark & Radivojac (2011)
Wyatt T Clark and Predrag Radivojac.
Analysis of protein function and its prediction from amino acid sequence.
*Proteins: Structure, Function, and Bioinformatics*, 79(7):2086–2096, 2011.
- Devos & Valencia (2000)
Damien Devos and Alfonso Valencia.
Practical limits of function prediction.
*Proteins: Structure, Function, and Bioinformatics*, 41(1):98–107, 2000.
- Elnaggar et al. (2021)
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al.
Prottrans: Toward understanding the language of life through self-supervised learning.
*IEEE transactions on pattern analysis and machine intelligence*, 44(10):7112–7127, 2021.
- Fang et al. (2024)
Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen.
Mol-instructions: A large-scale biomolecular instruction dataset for large language models.
In *The Twelfth International Conference on Learning Representations*. OpenReview.net, 2024.
URL [https://openreview.net/pdf?id=Tlsdsb6l9n](https://openreview.net/pdf?id=Tlsdsb6l9n).
- Ferruz et al. (2022)
Noelia Ferruz, Steffen Schmidt, and Birte Höcker.
Protgpt2 is a deep unsupervised language model for protein design.
*Nature communications*, 13(1):4348, 2022.
- Gass et al. (2020)
Susan M Gass, Jennifer Behney, and Luke Plonsky.
*Second language acquisition: An introductory course*.
Routledge, 2020.
- Guo et al. (2025)
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*, 2025.
- Guo et al. (2023)
Han Guo, Mingjia Huo, Ruiyi Zhang, and Pengtao Xie.
Proteinchat: Towards achieving chatgpt-like functionalities on protein 3d structures.
*Authorea Preprints*, 2023.
- Hayes et al. (2025)
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al.
Simulating 500 million years of evolution with a language model.
*Science*, pp. eads0018, 2025.
- Hu et al. (2024)
Bozhen Hu, Cheng Tan, Yongjie Xu, Zhangyang Gao, Jun Xia, Lirong Wu, and Stan Z. Li.
Protgo: Function-guided protein modeling for unified representation learning.
In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), *Advances in Neural Information Processing Systems*, volume 37, pp. 88581–88604. Curran Associates, Inc., 2024.
URL [https://proceedings.neurips.cc/paper_files/paper/2024/file/a1722a6bd1023c026a3d6a570fb3af75-Paper-Conference.pdf](https://proceedings.neurips.cc/paper_files/paper/2024/file/a1722a6bd1023c026a3d6a570fb3af75-Paper-Conference.pdf).
- Huckin & Coady (1999)
Thomas Huckin and James Coady.
Incidental vocabulary acquisition in a second language: A review.
*Studies in second language acquisition*, 21(2):181–193, 1999.
- Jararweh et al. (2025)
Ala Jararweh, Oladimeji Macaulay, David Arredondo, Yue Hu, Luis Tafoya, Kushal Virupakshappa, and Avinash Sahu.
Protein2text: Resampling mechanism to translate protein sequences into human-interpretable text.
In *NAACL 2025 - Industry Track*, 2025.
- Jarvis & Pavlenko (2008)
Scott Jarvis and Aneta Pavlenko.
*Crosslinguistic influence in language and cognition*.
Routledge, 2008.
- Kirkpatrick et al. (2017)
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al.
Overcoming catastrophic forgetting in neural networks.
*Proceedings of the national academy of sciences*, 114(13):3521–3526, 2017.
- Kitadai & Maruyama (2018)
Norio Kitadai and Shigenori Maruyama.
Origins of building blocks of life: A review.
*Geoscience Frontiers*, 9(4):1117–1153, 2018.
- Koonin & Galperin (2002)
Eugene Koonin and Michael Y Galperin.
*Sequence—Evolution—Function: Computational Approaches in Comparative Genomics*.
Springer Science & Business Media, 2002.
- Lin & Hovy (2002)
Chin-Yew Lin and Eduard Hovy.
Manual and automatic evaluation of summaries.
In *Proceedings of the ACL-02 workshop on automatic summarization*, pp. 45–51, 2002.
- Lin et al. (2023)
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al.
Evolutionary-scale prediction of atomic-level protein structure with a language model.
*Science*, 379(6637):1123–1130, 2023.
- Liu et al. (2024a)
Nuowei Liu, Changzhi Sun, Tao Ji, Junfeng Tian, Jianxin Tang, Yuanbin Wu, and Man Lan.
Evollama: Enhancing llms’ understanding of proteins via multimodal structure and sequence representations.
*arXiv preprint arXiv:2412.11618*, 2024a.
- Liu et al. (2024b)
Zhiyuan Liu, An Zhang, Hao Fei, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, and Tat-Seng Chua.
ProtT3: Protein-to-text generation for text-based protein understanding.
In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 5949–5966, Bangkok, Thailand, 2024b. Association for Computational Linguistics.
doi: 10.18653/v1/2024.acl-long.324.
URL [https://aclanthology.org/2024.acl-long.324/](https://aclanthology.org/2024.acl-long.324/).
- Luo et al. (2024)
Yizhen Luo, Jiahuan Zhang, Siqi Fan, Kai Yang, Massimo Hong, Yushuai Wu, Mu Qiao, and Zaiqing Nie.
Biomedgpt: An open multimodal large language model for biomedicine.
*IEEE Journal of Biomedical and Health Informatics*, 2024.
- Lv et al. (2024)
Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Li Yuan, and Yonghong Tian.
Prollama: A protein large language model for multi-task protein language processing.
*arXiv e-prints*, pp. arXiv–2402, 2024.
- Lv et al. (2025)
Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Li Yuan, and Yonghong Tian.
Prollama: A protein large language model for multi-task protein language processing.
*IEEE Transactions on Artificial Intelligence*, 2025.
- Ma et al. (2025)
Zicheng Ma, Chuanliu Fan, Zhicong Wang, Zhenyu Chen, Xiaohan Lin, Yanheng Li, Shihao Feng, Jun Zhang, Ziqiang Cao, and Yi Qin Gao.
Prottex: Structure-in-context reasoning and editing of proteins with large language models.
*arXiv preprint arXiv:2503.08179*, 2025.
- Madani et al. (2023)
Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos, Caiming Xiong, Zachary Z Sun, Richard Socher, et al.
Large language models generate functional protein sequences across diverse families.
*Nature echnology*, 41(8):1099–1106, 2023.
- Mondon (1984)
Camille Mondon.
Classification and regression trees, 1984.
- Nijkamp et al. (2023)
Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani.
Progen2: exploring the boundaries of protein language models.
*Cell systems*, 14(11):968–978, 2023.
- OpenAI et al. (2024)
OpenAI et al.
Gpt-4 technical report, 2024.
- Pei et al. (2023a)
Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan.
BioT5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations.
In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pp. 1102–1123, Singapore, December 2023a. Association for Computational Linguistics.
doi: 10.18653/v1/2023.emnlp-main.70.
URL [https://aclanthology.org/2023.emnlp-main.70/](https://aclanthology.org/2023.emnlp-main.70/).
- Pei et al. (2023b)
Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan.
Biot5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations.
*arXiv preprint arXiv:2310.07276*, 2023b.
- Pesquita et al. (2008)
Catia Pesquita, Daniel Faria, Hugo Bastos, António EN Ferreira, André O Falcão, and Francisco M Couto.
Metrics for go based protein semantic similarity: a systematic evaluation.
*BMC bioinformatics*, 9(Suppl 5):S4, 2008.
- Rost et al. (1998)
Burkhard Rost et al.
Protein structure prediction in 1d, 2d, and 3d.
*Encyclopedia of Computational Chemistry*, pp. 2242–2255, 1998.
- Shen et al. (2024)
Yiqing Shen, Zan Chen, Michail Mamalakis, Luhan He, Haiyang Xia, Tianbin Li, Yanzhou Su, Junjun He, and Yu Guang Wang.
A fine-tuning dataset and benchmark for large language models for protein understanding.
In *2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)*, pp. 2390–2395. IEEE, 2024.
- Steinegger & Söding (2017)
Martin Steinegger and Johannes Söding.
Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.
*Nature biotechnology*, 35(11):1026–1028, 2017.
- Su et al. (2023)
Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan.
Saprot: Protein language modeling with structure-aware vocabulary.
*BioRxiv*, pp. 2023–10, 2023.
- Tan et al. (2025)
Yang Tan, Wenrui Gou, Bozitao Zhong, Liang Hong, Huiqun Yu, and Bingxin Zhou.
Venusx: Unlocking fine-grained functional understanding of proteins, 2025.
URL [https://arxiv.org/abs/2505.11812](https://arxiv.org/abs/2505.11812).
- Taylor et al. (2022)
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic.
Galactica: A large language model for science.
*ArXiv preprint*, abs/2211.09085, 2022.
URL [https://arxiv.org/abs/2211.09085](https://arxiv.org/abs/2211.09085).
- Team et al. (2025)
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al.
Kimi k2: Open agentic intelligence.
*arXiv preprint arXiv:2507.20534*, 2025.
- Team (2024)
Qwen Team.
Qwen2.5: A party of foundation models, September 2024.
URL [https://qwenlm.github.io/blog/qwen2.5/](https://qwenlm.github.io/blog/qwen2.5/).
- Team (2025)
Qwen Team.
Qwen3 technical report, 2025.
URL [https://arxiv.org/abs/2505.09388](https://arxiv.org/abs/2505.09388).
- Wang et al. (2024)
Chao Wang, Hehe Fan, Ruijie Quan, and Yi Yang.
Protchatgpt: Towards understanding proteins with large language models.
*arXiv preprint arXiv:2402.09649*, 2024.
- Wang & Pollock (2005)
Zhengyuan O Wang and David D Pollock.
Context dependence and coevolution among amino acid residues in proteins.
In *Methods in enzymology*, volume 395, pp. 779–790. Elsevier, 2005.
- Wu et al. (2025)
Juntong Wu, Zijing Liu, He Cao, Hao Li, Bin Feng, Zishan Shu, Ke Yu, Li Yuan, and Yu Li.
Rethinking text-based protein understanding: Retrieval or llm?
*arXiv preprint arXiv:2505.20354*, 2025.
- Wu et al. (2024a)
Kevin E Wu, Howard Chang, and James Zou.
Proteinclip: enhancing protein language models with natural language.
*bioRxiv*, pp. 2024–05, 2024a.
- Wu et al. (2024b)
Kevin E Wu, Kathryn Yost, Bence Daniel, Julia Belk, Yu Xia, Takeshi Egawa, Ansuman Satpathy, Howard Chang, and James Zou.
Tcr-bert: learning the grammar of t-cell receptors for flexible antigen-binding analyses.
In *Machine Learning in Computational Biology*, pp. 194–229. PMLR, 2024b.
- Xiang et al. (2024)
Wenkai Xiang, Zhaoping Xiong, Huan Chen, Jiacheng Xiong, Wei Zhang, Zunyun Fu, Mingyue Zheng, Bing Liu, and Qian Shi.
Fapm: functional annotation of proteins using multimodal models beyond structural modeling.
*Bioinformatics*, 40(12):btae680, 2024.
- Xiao et al. (2024)
Yijia Xiao, Edward Sun, Yiqiao Jin, Qifan Wang, and Wei Wang.
Proteingpt: Multimodal llm for protein property prediction and structure understanding.
*arXiv preprint arXiv:2408.11363*, 2024.
- Xiao et al. (2025)
Yijia Xiao, Wanjia Zhao, Junkai Zhang, Yiqiao Jin, Han Zhang, Zhicheng Ren, Renliang Sun, Haixin Wang, Guancheng Wan, Pan Lu, et al.
Protein large language models: A comprehensive survey.
*arXiv preprint arXiv:2502.17504*, 2025.
- Xu & Wang (2022)
Hanwen Xu and Sheng Wang.
Protranslator: zero-shot protein function prediction using textual description.
In *International conference on research in computational molecular biology*, pp. 279–294. Springer, 2022.
- Xu et al. (2023a)
Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang.
Protst: Multi-modality learning of protein sequences and biomedical texts.
In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), *International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA*, volume 202 of *Proceedings of Machine Learning Research*, pp. 38749–38767. PMLR, 2023a.
URL [https://proceedings.mlr.press/v202/xu23t.html](https://proceedings.mlr.press/v202/xu23t.html).
- Xu et al. (2023b)
Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang.
Protst: Multi-modality learning of protein sequences and biomedical texts.
In *International Conference on Machine Learning*, pp. 38749–38767. PMLR, 2023b.
- Zhang et al. (2022)
Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Qiang Zhang, Jiazhang Lian, and Huajun Chen.
Ontoprotein: Protein pretraining with gene ontology embedding.
In *The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022*. OpenReview.net, 2022.
URL [https://openreview.net/forum?id=yfe1VMYAXa4](https://openreview.net/forum?id=yfe1VMYAXa4).
- Zhuo et al. (2024)
Le Zhuo, Zewen Chi, Minghao Xu, Heyan Huang, Jianan Zhao, Heqi Zheng, Conghui He, Xian-Ling Mao, and Wentao Zhang.
ProtLLM: An interleaved protein-language LLM with protein-as-word pre-training.
In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 8950–8963, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
doi: 10.18653/v1/2024.acl-long.484.
URL [https://aclanthology.org/2024.acl-long.484/](https://aclanthology.org/2024.acl-long.484/).

## Appendix A Evaluation Metrics

We use the automatic metric ROUGE-L (24) to assess the quality of the generated text by comparing it with reference answers. In addition, we incorporate manual checking into the evaluation pipeline and compute a human-rating score. Five evaluators with biological-research experience were asked to rate each generated answer on a 0–5 scale (the integer score corresponds to the category number minus one). All evaluators have at least two years of research experience in biology. The six ordinal categories they used are:

- 1.
Garbled – the text is incomprehensible and lacks any readability.
- 2.
Inaccurate – the text is readable but entirely incorrect and devoid of meaningful information.
- 3.
Partially informative – the text offers some reference value, yet its factual correctness is poor.
- 4.
Moderately accurate – roughly half of the information is correct, but several errors remain.
- 5.
Mostly accurate – the content is almost entirely correct, with only minor omissions or errors.
- 6.
Completely correct – the content is accurate in its entirety, without any mistakes.

## Appendix B ADDITIONAL RESULTS

##### Evaluation on real-world protein scenarios

To examine the applicability of our framework beyond benchmark datasets, we evaluated it on biologically relevant queries involving uncharacterized Homo sapiens proteins. For each case, a current biologically relevant question of research interest was paired with the corresponding protein amino acid sequence and input to representative LLMs guided by our framework. As illustrated in Figure [7](https://arxiv.org/html/2510.11188v1#A2.F7), the models produced plausible hypotheses aligned with biological knowledge. These results demonstrate that our framework can extend to real-world scenarios, offering interpretable preliminary insights into proteins lacking experimental annotation and potentially guiding future biological investigations.

Figure: Figure 7: Exploratory evaluation of our framework on real-world protein scenarios.
Refer to caption: x7.png

## Appendix C Bilingual Dataset: Details and Examples

All protein entries are derived from the manually curated SwissProt (2) section of UniProt, which provides high-quality annotations of protein sequences and functional descriptions. After deduplication, we prompted LLMs to generate four types of bilingual QA pairs from these entries. To assess data quality, we randomly sampled 500 pairs from the full set of 79,926 automatically constructed examples. Each sampled pair was manually reviewed by domain experts along three dimensions: semantic fidelity, biological plausibility, and translation fluency. The review showed a 95% pass rate, confirming that the dataset maintains high linguistic accuracy and biological reliability. Examples of four bilingual QA types are provided in Figure [8](https://arxiv.org/html/2510.11188v1#A3.F8) and Figure [9](https://arxiv.org/html/2510.11188v1#A3.F9).

Figure: Figure 8: Illustrative examples of the four bilingual QA types (Part 1).
Refer to caption: x8.png

Figure: Figure 9: Illustrative examples of the four bilingual QA types (Part 2).
Refer to caption: x9.png

## Appendix D llm statement

We acknowledge the use of LLMs in this work. Specifically, DeepSeek-R1 (14) was employed for two purposes: (i) polishing the English presentation of the manuscript, and (ii) generating bilingual dataset entries from curated protein annotations, where the prompts were carefully designed to ensure scientific accuracy and linguistic quality. Below we provide the exact prompts used for each bilingual QA type in the dataset construction process.

Prompt for Attribute-based Answer generation is following:

Prompt for True or False QA generation is following:

Prompt for Descriptive Text generation is following:

Prompt for Knowledge-based QA generation is following:

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 6
presentation: 2
questions: - How does the method perform on out-of-distribution or rare protein families, especially those absent from the QA corpus?
soundness: 3
strengths: - Introduces the idea of treating protein sequences as a "second language" for LLMs, bridging symbolic biological and natural language reasoning.
- The bilingual corpus is diverse (79k QA pairs across 4 types), functionally rich, and biologically balanced.
- Works across frozen LLMs (3B–14B) and improves both open-source and proprietary models in zero-shot settings.
summary: Protein-as-Second-Language (PSL) framework is a method for protein function understanding using large language models (LLMs) without fine-tuning. The approach reformulates amino acid sequences as symbolic language and uses adaptive, bilingual context construction (sequence + natural language) to enable zero-shot reasoning. A curated dataset of ~80k protein–QA pairs spanning functional, descriptive, and reasoning tasks supports the method.
weaknesses: - No wet-lab or structure-level validation is presented; success is only measured by text-based QA metrics (e.g., ROUGE-L).

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: Please address my two major concerns in the “Weaknesses” section first. I will reassess after the rebuttal. Other miscellaneous questions are as follows: 

* Were the quality and correctness of the augmented QA from DeepSeek-R1 verified somehow? Teacher LLMs are known to hallucinate, especially on complex scientific topics like biology. 

* Are GO annotations an effective criterion for grouping and redundancy reduction? As far as I understand, a group of proteins might be very different from each other even though they share similar GO annotations. This is especially the case when some proteins are not very well annotated and have very few GO annotations. 

* The automatic evaluation solely relying on the ROUGE-L score is most likely not sufficient. Other metrics like accuracy (particularly for True or False QA), BLEU score, and BERTScore might provide a better understanding of the improvement in performance. 

* Besides, a human evaluation is conducted in the paper, but the domain knowledge or expertise of these evaluators is not mentioned (maybe I missed it), making the reliability of this evaluation questionable. 

* The term “zero-shot” used multiple times in the paper is a bit misleading, because retrieved similar proteins in the prompt may provide contextual information. The framework described in the paper is more likely a “few-shot” one. 

* Numbers in Figure 4 (left) are not consistent with the bars. I would recommend a thorough double-check of all the numerical results presented in the paper.
soundness: 2
strengths: * This work proposes a framework that eliminates the need to train or fine-tune a task-specific LLM for protein sequence analysis. 

* The paper employs dual criteria to retrieve similar proteins for augmenting the prompt, considering both sequence and text similarity.
summary: This work introduces a novel question-answering (QA) dataset focused on protein expression, localization, mechanism, and interaction. The authors also propose a retrieval-based framework that enables pretrained, generic large language models (LLMs) to analyze unknown protein sequences using an in-context learning approach. By including similar proteins and their corresponding descriptions in the prompt, the paper reports an average 7% improvement in the ROUGE-L score on QA tasks for the target unknown protein.
weaknesses: * The overall novelty of the paper appears limited.  Prior works, particularly ProtEx from Google DeepMind, already explored the possibility of in-context learning for biological entity analysis. This submission however, neither mentions nor compares with them. 

* The similarity-based retrieval likely limits the generalization ability of the model on unseen protein sequences that are very different from existing ones in the database. A 70% threshold is applied on sequence similarity when constructing the dataset and retrieving for the prompt, but novel or orphan proteins often share <40% identity to the existing database.

---

confidence: 5
contribution: 3
initial_rating: 4
presentation: 3
questions: 1.Could the authors clarify whether the constructed bilingual corpus overlaps with the evaluation datasets? Since both the corpus and the benchmarks seem to originate from Swiss-Prot or related sources, it would be important to specify how potential data leakage was prevented to ensure fair evaluation.

2.The paper transforms Swiss-Prot annotations into multiple QA formats rather than using the full annotations directly. What is the motivation for this choice, and would incorporating broader and more complete biological knowledge lead to more stable contextual enhancement?

3.The proposed framework involves a retrieval step for each query to build adaptive bilingual contexts. Could the authors discuss the computational overhead introduced by this process and its impact on inference time and scalability compared with fine-tuned models?

4.The exemplar selection process is described as combining both sequence homology (via MMseqs2) and semantic similarity between QA pairs. Could the authors elaborate on how these two signals are integrated into the final retrieval score or ranking criterion?

5.The method assumes that proteins with high MMseqs2 similarity share similar contexts. Have the authors considered using alternative similarity measures, such as embedding-based similarity from ESM or structure-based similarity from ProTrek, and could they provide related ablation results?

6.In the comparison with ProLLaMA and BioT5+, were these models fine-tuned on any part of the proposed corpus (like RAFT), or were their publicly released parameters used directly for inference? Please clarify whether additional training or adaptation was performed to ensure a fair comparison.
soundness: 3
strengths: This paper introduces a conceptually novel and computationally efficient framework that enables large language models to understand protein sequences through bilingual contextual reasoning without any fine-tuning. In addition to the framework, the authors construct a large-scale bilingual protein–text corpus containing 79,926 sequence–question–answer pairs, which serves as the foundation for retrieval-based in-context learning and systematic evaluation.
summary: This paper proposes Protein-as-Second-Language (PSL), a training-free framework that enables large language models to interpret protein sequences as a “second language.” Instead of fine-tuning, PSL performs retrieval-based in-context learning by constructing bilingual contexts that pair amino-acid sequences with natural-language descriptions. The authors build a 79K protein–QA corpus via Gene Ontology–based functional grouping, MMseqs2 clustering with semantic deduplication, and automatic QA generation using DeepSeek-R1 across four question types. During inference, PSL selects relevant examples based on sequence homology and semantic similarity, forming adaptive prompts for frozen LLMs (GPT-4o, Qwen, Mistral). Across three benchmarks (ProtDescribe, Protein2Text-QA, Mol-Instructions), PSL achieves up to 17.2% ROUGE-L improvement, outperforming domain-specific models like ProLLaMA-7B and BioT5+, and reframes protein understanding as retrieval-driven bilingual reasoning rather than supervised fine-tuning.
weaknesses: 1. The bilingual corpus is constructed using Swiss-Prot as the primary data source, while the evaluation datasets are also derived from or highly overlap with Swiss-Prot. The paper does not provide sufficient details on how potential data leakage or overlap was prevented, which raises concerns about the fairness of evaluation.

2. Each inference involves a retrieval step to construct query-specific contexts, but the computational overhead and latency introduced by this process are not analyzed. The practical efficiency of the framework therefore remains unclear.

3. The method assumes that proteins with high MMseqs2 similarity share similar functional or semantic contexts. However, this assumption may not always hold, especially for multi-domain proteins. A more critical discussion or ablation on this assumption would strengthen the justification.

4. The experimental comparison includes only two domain-specific baselines, ProLLaMA-7B and BioT5+, which may not be sufficient to establish broad effectiveness. Including more diverse or fine-tuned protein LLMs could improve the reliability of the conclusion.

5. The framework appears to treat protein sequences and text jointly as input without a dedicated modality projector or alignment module. While this simplifies the design, it may not fully exploit cross-modal complementarities, and more structured feature integration could further enhance performance.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. Can the authors formalize the context construction mechanism, especially the mathematical definitions of exemplar ranking, weighting, or aggregation (e.g., is there a score function for context selection, or are choices made heuristically)? Please clarify with explicit pseudocode or equations.
2. What is the full breakdown of annotation QA pass/fail in the 5% discarded portion of the dataset? Are there any systematic biases or edge cases in the rejected data?
3. Did the authors attempt to benchmark model inference/prompt assembly time versus fine-tuned protein LLMs, or estimate resource requirements (memory/latency) for large context window assembly in practical use?
4. Could the authors compare directly with analogy-driven or multi-hop retrieval LLM frameworks (such as ANALOGYKB, BeamAggR) and efficient adaptation methods (S-LoRA, MoDeGPT)?
5. Can the authors clarify the apparent error in the human-in-the-loop Krippendorff's $\alpha$ (is 0.72% a typo?), and provide more detail on the distribution of human evaluations by task/model?
soundness: 2
strengths: 1. The paper cleverly reframes protein sequences as a "second language," allowing general-purpose LLMs to build protein-function mappings in a zero-shot regime. This paradigm bridges symbolic biological and natural languages, bypassing the need for task-specific fine-tuning or retraining and creatively leverages LLMs' in-context learning strengths.
2. A substantial and well-constructed bilingual (protein-natural language) dataset is curated, addressing redundancy at both sequence and annotation levels. Figures 1 and 3 illustrate a careful, step-wise reduction of redundancy with diverse coverage across species, protein families, superfamilies, and ontology categories.
3. The method is evaluated on multiple datasets (ProtDescribe, Protein2Text-QA, Mol-Instructions) with frozen, general-purpose LLMs (Qwen, GPT-4o, Mistral, Kimi) and compared against strong protein-LLM baselines (BioT5-plus, ProLLaMA). Main results (Table 1) and multiple figures demonstrate robust performance gains in both automatic (ROUGE-L) and human evaluations, frequently surpassing or matching fine-tuned specialized models.
summary: This paper introduces the "Protein-as-Second-Language" framework, which aims to enable large language models (LLMs) to interpret protein (amino acid) sequences as if they were acquiring a second symbolic language. By curating a bilingual dataset of almost 80k protein-question-answer triples and implementing an adaptive context construction mechanism, the approach supplies protein sequence–question–answer exemplars as in-context learning cues for frozen LLMs. Extensive experiments on multiple protein-text QA benchmarks demonstrate that LLMs guided by this framework substantially outperform their zero-shot counterparts, and in some cases, even domain-specialized, fine-tuned protein LLMs.
weaknesses: Weaknesses
1. The methodology’s underpinnings, particularly the bilingual context construction mechanism (Section 3.2), lack precise mathematical formalization. While Equation-based thresholds (for sequence and annotation deduplication, Section 3.1.1/3.1.2) are provided, the procedure for query-to-context matching (how candidate exemplars are scored and integrated, and any ranking or aggregation formulae) is only described at a high level. For instance, does selection use hard thresholds or similarity-weighted composition? What is the precise mathematical form of the query-context similarity metric for textual and sequence components? Without a formal definition of, for example, the joint scoring function or aggregation, it is difficult for others to re-implement or to affirm the reproducibility or generalizability of the context construction process. Explicit equations or algorithms are expected for a submission at this technical level.
2. While the empirical evaluation is extensive by the standards of current protein-language model research, critical baselines are missing:
   - Direct comparison with leading analogy/reasoning-augmented LLM paradigms, such as those leveraging knowledge graphs (e.g., ANALOGYKB[1]), hierarchical retrieval (e.g., BeamAggR[2]), or reinforcement-learning-based self-correction (e.g., SeRL[3], AlphaEdit[4], Self-Correct RL[5]), is not present.
   - There is also a missed opportunity to benchmark efficiency: the computational cost of in-context "second language" understanding (which requires substantial prompt assembly with many exemplars) is never compared directly to the one-time fine-tuning cost of protein LLMs, or to parameter-efficient adaptation approaches (e.g., S-LoRA[6], MoDeGPT[7]). Without this, claims of scalability remain qualitative.
3. The central claim—LLMs can generalize protein understanding more efficiently through contextual exemplars—is only validated empirically under a restricted set of Q/A regimes. There is no attempt to analyze theoretical properties: e.g., under what assumptions does the contextual analogy paradigm guarantee generalization or compositional reasoning for proteins? What are failure modes if distributional shifts exist (out-of-distribution proteins/annotations not covered by exemplars)? The paper could benefit from at least some basic analysis or a discussion of the limitations.

references
[1] Yuan, S., Chen, J., Sun, C. (2024): "ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base"
[2] Chu, Z., Chen, J., Chen, Q. (2024): "BeamAggR: Beam Aggregation Reasoning over Multi-source Knowledge for Multi-hop Question Answering"
[3] Fang, W., Liu, S., Zhou, Y. (2025): "SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data"
[4] Fang, J., Jiang, H., Wang, K. (2025): "AlphaEdit: Null-Space Constrained Model Editing for Language Models"
[5] Kumar, A., Zhuang, V., Agarwal, R. (2025): "Training Language Models to Self-Correct via Reinforcement Learning"
[6] Wu, Y., Piao, H., Huang, L. (2025): "S-LoRA: Scalable Low-Rank Adaptation for Class Incremental Learning"
[7] Lin, C., Gao, S., Smith, J. S. (2025): "MoDeGPT: Modular Decomposition for Large Language Model Compression"