Venue: NAACL 2025

# Protein2Text: Resampling Mechanism to Translate Protein Sequences

into Human-Interpretable Text

Ala Jararweb\({}^{*,1,2}\), Oladimeji Macaulay\({}^{*,2}\), David Arredondo\({}^{2}\), Yue Hu\({}^{2}\), Luis Tafoya\({}^{2}\),

**Kushal Virupakshappa\({}^{2}\), Avinash Sahu\({}^{1,2}\)**

\({}^{1}\)Department of Computer Science, The University of New Mexico

\({}^{2}\)Comprehensive Cancer Center, The University of New Mexico {ajarrarweh,asahu}@salud.unm.edu

###### Abstract

Proteins play critical roles in biological systems, yet 99.7% of over 227 million known protein sequences remain uncharacterized due to the limitations of experimental methods. To assist experimentals in narrowing down hypotheses and accelerating protein characterization, we present Protein2Text, a multimodal large language model that interprets protein sequences and generates informative text to address open-ended questions about protein functions and attributes. By integrating a resampling mechanism within an adapted LLaVA framework, our model effectively maps protein sequences into a language-compatible space, enhancing its capability to handle diverse and complex queries. Trained on a newly curated dataset derived from PubMed articles and rigorously evaluated using four comprehensive benchmarks--including in-domain and cross-domain evaluations--Protein2Text outperforms several existing models in open-ended question-answering tasks. Our work also highlights the limitations of current evaluation metrics applied to template-based approaches, which may lead to misleading results, emphasizing the need for unbiased assessment methods. Our model weights, evaluation datasets, and evaluation scripts are publicly available at [https://github.com/alaaji27/Protein2Text.git](https://github.com/alaaji27/Protein2Text.git).

## 1 Introduction

Proteins are essential to nearly all biological processes. Understanding protein functions is crucial for unraveling disease mechanisms, predicting the effects of genetic mutations in conditions like cancer, and discovering targeted and personalized therapeutics Liu et al. (2020); Quazi (2022); Wu et al. (2023b). Despite the characterization of 460,000 proteins in UniProt Consortium (2022), a staggering 99.7% of the 227 million protein sequences remain poorly characterized (Consortium, 2022; Coudert et al., 2022). This vast number of uncharacterized proteins poses a significant bottleneck in biomedical research, impeding the full realization of the potential envisioned with the sequencing of the human genome. Experimental methods for protein characterization are inherently time-consuming and costly, making it impractical to scale to millions of proteins. Therefore, there is an urgent need for computational methods to complement and accelerate traditional experimental approaches.

For the first time, Large Language Models (LLMs) are offering an alternative to these challenges. For example, encoder-based models like ESM-2 and OntoProtein leverage masked language modeling on protein sequences to generate embeddings that capture structural and functional information Lin et al. (2022),Dang et al. (2023, 2022). Similarly, to predict gene/protein structural and functional information, several approaches use other modalities such as text Jararweh et al. (2024) and expression Du et al. (2019); Cui et al. (2024). Decoder-based models such as AlphaFold predict 3D structures from amino acid sequences John Jumper and Hassabis (2021). Moreover, multimodal LLMs have been developed to bridge the gap between biological sequences and natural language, translating complex protein data into accessible human language Luo et al. (2023); Fang et al. (2024). Bimodal Protein Language Models (PLMs), including Protein-Chat and ProtChatGPT Guo et al. (2023); Wang et al. (2024), attempt to co-embed protein sequences with natural language using projection mechanisms.

However, existing PLMs face limitations. A critical gap is the lack of rigorous quantitative evaluation on question-answering (QA) tasks, which are vital for practical utility. Many PLMs depend on template-based QA datasets, transforming structured data into unstructured text using

fixed templates (Guo et al., 2023; Xiao et al., 2024a; Luo et al., 2023). This methodology limits the models' ability to generalize to new unseen queries and diminishes their adaptability to diverse instructions. Consequently, template-based QA datasets hinder model expressiveness, and often -as we also demonstrate - overfit to specific patterns and lack the conversational flexibility necessary for addressing complex research questions (see Table 14) (Liu et al., 2024).

Therefore, we present a novel multimodal reasoning model that modifies the LLAVA Liu et al. (2023) framework to adopt for the protein domain. Our model provides real-time, interactive analysis of protein properties and handles complex, open-ended questions, empowering researchers to gain actionable insights for laboratory research. Trained on a newly curated dataset derived from published literature on proteins in PubMed articles, our model benefits from a rich and diverse corpus surpassing template-based methods' limitations. We also compiled four comprehensive evaluation datasets to benchmark our model against existing PLMs rigorously. By releasing these evaluation datasets and model weights, we aim to promote a thorough assessment of protein LLMs across a wide range of tasks and specialized datasets.

## 2 Related Work

The sequential nature of protein primary structure lends itself to language modeling for protein characterization. For example, encoder-based LLMs trained on protein amino acid sequences have been adopted to generate a representation space that captures sequence structures Lin et al. (2022b, a; Elnaggar et al., 2021); Zhang et al. (2022). Generative LLMs have also been proposed for a variety of protein generation tasks such as generating 3D structure John Jumper and Hassabis (2021), and novel protein sequences Madani et al. (2020); Nijkamp et al. (2022); Lv et al. (2024). LLMs that incorporate natural language and protein as one modality (i.e. considering protein as text modality) have been proposed. For example, Galactic models are general-purpose LLMs that are trained on scientific corpora to perform different reasoning tasks including protein captioning. Several studies attempt to integrate text with protein modalities such as DNA/RNA sequences Richard et al. (2024), 3D structure Guo et al. (2023); Wang et al. (2024), and amino acid sequences Xiao et al. (2024b; Luo et al., 2023). Similarly, multi-modality projection similar to vision-language alignment Alayrac et al. (2022); Liu et al. (2023), has been applied to align between protein and natural text where protein is considered as single modality Guo et al. (2023); Wang et al. (2024); Liu et al. (2024); Luo et al. (2023); Fang et al. (2024). See Appendix D for detailed discussion on related work.

## 3 Protein2Text

Protein EncoderOur approach is based on LLaVA Liu et al. (2023) which integrates images and text via instruction tuning. We adopt LLaVA to protein amino acid sequences by replacing the image encoder with a protein encoder (Figure 1b). We use ESM-2 Lin et al. (2022b) a transformer-based encoder that has 33 transformer layers and a total of 652 million parameters. Every sequence (\({\cal P}\)) is encoded to a multidimensional token embedding using ESM2 (\(\phi_{esm}\)) where every character is considered a token. Formally:

\[{\bf Z}_{v}=\phi_{esm}({\cal P})\]

where \({\bf Z}_{v}\in\mathbb{R}^{d\times T_{1}}\) represents the embedding of the protein tokens where \(d\) is the dimension size and \(T_{1}\) is the number of tokens.

LLMEncoderSimultaneously, the instruction/question \({\bf X}_{q}\), given as natural language input, is tokenized and embedded using LLaMA-3, \(\phi_{LLM}\):

\[{\bf H}_{q}=\phi_{LLM}({\bf X}_{q})\]

where \({\bf H}_{q}\in\mathbb{R}^{k\times T_{2}}\) represents the token embeddings of the instruction, with \(k\) being the embedding dimension and \(T_{2}\) the number of tokens.

Perceiver ResamplerIn LLaVA, images are divided into a fixed number of patches, yielding a fixed number of image tokens without losing information. However, protein sequences have different sizes, and truncating them to a fixed size might remove potentially critical information. To this end, we extend their architecture by adding a protein resampler Jaegle et al. (2021); Carion et al. (2020); Alayrac et al. (2022). The resampler finds a fixed number of latent tokens from varying-size protein sequences (Figure 1b). This reduces the computational complexity of the cross-attention in the LLM and prevents long protein tokens from 

exhausting the model's maximum length. Given the protein token embeddings (\({\bf Z}_{v}\)), the resampler generates \({\bf H}_{v}\in\mathbb{R}^{d\times S}\), where \(S\) is the number of latent tokens that compress the information in the original tokens:

\[{\bf H}_{v}=\phi_{Resampler}({\bf Z}_{v})\]

**Protein2Text Projector.** To align the protein and the text modalities (Figure 1a), we project the dimensions of protein latent tokens (\(d\)) into the language embedding space (\(k\)) via the projector:

\[{\bf H}^{\prime}_{v}=W\cdot{\bf H}_{v}\]

where \(W\) is the set of trainable parameters and \({\bf H}^{\prime}_{v}\in\mathbb{R}^{k\times S}\). The projected tokens are then concatenated to the text tokens, producing \({\bf H}\in\mathbb{R}^{k\times(S+T_{2})}\). \({\bf H}\) is then fed to the LLM decoder (\(f_{\phi}\)) to generate the response.

Dataset CollectionWe collect four different datasets tailored to distinct requirements. First, the pretraining dataset spans 394,000 protein amino acid sequences and function descriptions collected from UniProt (Consortium, 2022). This dataset is entirely used to train the resampler and the projector during the pretraining stage.

Next, we generate a comprehensive question and answering dataset (i.e. **Protein2Text-QA**) to fine-tune the model parameters. The dataset spans approximately 210,000 pairs of QA. We utilize research carried out on proteins from published articles in the PubMed Central (PMC) database Consortium (2015) to create questions and answers. Articles mentioning the protein names are located and fed to the LLaMA3.1 model to generate a series of QA pairs, such that they focus only on the protein name given.

The test set and zero-shot set are then sampled from the Protein2Text-QA dataset. The proteins in the test set can be found in the pre-training dataset but not in the fine-tuning dataset. On the other hand, the zero-shot set is sampled such that the protein sequences and their variants are not mentioned in both pre-training and fine-tuning sets. The variants were also filtered out to eliminate data leakage (Bushuiev et al., 2024) since some protein variants might have different sequences but similar/same function (Brett et al., 2002; Schluter et al., 2009). Finally, we generate two cross-domain datasets to evaluate the model on questions not mentioned in the abstracts. First, the **DiscussionQA** which spans QAs extracted from discussion sections, and the **IntroductionQA** which spans QAs extracted from introduction sections. The collection process of training and evaluation datasets, and detailed statistics, generation pipelines, preprocessing, and sample QAs are further discussed in Appendix A.

Training.The model training process consists of two stages: pretraining and fine-tuning. During pretraining, we freeze the protein encoder and the LLM while the parameters for the re-sampler and projector are trained. Next, we perform fine-tuning, where we train the entire model except the protein encoder parameters. In this stage, the LLM is trained using Low-Rank Adaptation (LoRA) (Hu et al., 2021). Finally, we assess the performance by designing four evalua

Figure 1: **Protein2Text Architecture Overview.** a) Protein2Text generates descriptive text from amino acid sequences by combining pre-trained protein encoder and language models. b) The protein tokens are compressed into latent tokens using the resampler and projected to the language space using the projector.

 

tion datasets tailored to distinct requirements such as baseline benchmarking, zero-shot ability, and cross-domain evaluations. Further details about training details, hyperparameters, baselines, and benchmarks are discussed in Appendices C.1, C.2, E, and F respectively.

## 4 Experiments

### Protein2Text-QA Evaluation

Experiment.We evaluated the performance of Protein2Text against two categories of large language models (LLMs): general-purpose LLMs and protein-specific LLMs. For general-purpose LLMs, such as GPT4o-mini [3] and LLaMA3.1 [5], the evaluation focused on assessing the degree of potential data leakage within the question prompts. We hypothesized that if the answers were embedded in the question prompts, general-purpose LLMs would likely respond correctly [7]. In the second category, we benchmarked Protein2Text against multimodal LLMs tailored for protein-related tasks, including Mol-Instruction [6], BioMedGPT [8], and ProtT3 [9], all of which are open-source tools. We evaluated the performance using lexical metrics such as BLEU [3], ROUGE [10], and METEOR [1], and semantic similarity metrics such as BERT similarity [12], and BiomBERT similarity [13]. Further details on baseline models and scores can be found in Appendices E and G.

Findings.Table 1 summarizes the performance of models on the Protein2Text QA test set. General-purpose LLMs exhibited poor performance due to their inability to interpret protein sequences (see Table 6), indicating minimal data leakage from the prompts. In contrast, protein-specific LLMs like BioMedGPT and Mol-Instruction showed competitive performance likely because they are also trained on PubMed data. BioMedGPT achieved higher semantic similarity scores but lower lexical scores compared to Mol-Instruction, suggesting its answers were semantically relevant but not necessarily accurate (Table 6). ProfT3, trained on template-based benchmarks or short QA (1-3 words), struggled with out-of-domain instructions, unlike Protein2Text, Mol-Instruction, and BioMedGPT.

Protein2Text consistently outperformed baselines across both semantic and lexical metrics. To explore potential enhancements, we implemented a GCA mechanism [11, 12, 13] at the top of the resampler architecture. Surprisingly, adding GCA resulted in reduced performance; therefore, was excluded in the final Protien2Text. Further investigation is needed to determine whether this decrease is due to the increased number of parameters requiring larger training data or if GCA is ill-suited for this problem. Details on parameter counts and the GCA ablation study are provided in Table 9 and Table 8, respectively.

### Cross-domain Evaluations

Experiment.We assess Protein2Text's generalizability to new domains. Here, we evaluate the performance on the zero-shot QA where proteins and their variants in this set are hidden during the entire training pipeline. Similarly, we assess the performance where the domain of the extracted QA is different such as the introduction (**IntroductionQA**) and discussion (**IntroductionQA**) sections. We focus on the PLM baselines throughout this experiment due to their superior performance compared to general-purpose LLMs.

Findings.First, the baselines showed similar performance in the Zero-shotQA (Table 2) compared to their performance in the test set (Table 1). Even though our model matches and often outperforms the baselines, the performance expectedly dropped compared to the test set. Since proteins and their variants were hidden during the alignment stage, novel sequence domains might have been introduced, hindering the resampler compression. The baselines showing similar performance could also indicate that these proteins might have been seen by these models during their training. Next, we evaluate the performance on the **IntroductionQA** as demonstrated in Table 3. Our model outperforms the baselines across lexical and semantic metrics; however, we see a slight decrease in metrics performance compared to the test set. This is likely because introduction sections usually present new information that was not necessarily mentioned in the abstract [3]. For the **DiscussionQA**, however, we found the performance of QAs from abstracts is similar 

to the performance of those extracted from the discussion sections as shown in Table 4, suggesting that discussion and abstract sections are more semantically aligned.

### ProteinKG25 Benchmark Evaluation

The ProteinKG25 dataset, originally designed as a knowledge base for protein attributes, was adapted into a question-answering (QA) format using templated questions by the authors of ProtT3 Liu et al. (2024) (see Appendix F). ProtT3 was fine-tuned specifically on this templated dataset. We evaluated our Protein2Text model on this benchmark in a zero-shot manner, without any additional fine-tuning.

As anticipated, ProtT3 achieved high-performance metrics on lexical evaluation scores (Table 13). However, we observed that in template-based scenarios, these metrics might not fully capture a model's ability to predict embedded protein attributes in the template. Models trained on templates can replicate the template structure, leading to high lexical similarity scores, even if the critical details within the responses are incorrect. Using the empty template as the prediction and ignoring attributes in the blanks achieved high lexical scores (Table 13). In contrast, models like Protein2Text, which are not trained on these templates, may generate responses that deviate from the template format, resulting in lower performance despite potentially providing accurate and informative answers.

To investigate this further, we focused on the task of predicting protein subcellular localization, a classification problem present in the ProteinKG25 dataset. We specifically prompted the models to predict protein localization among three classes and assessed their outputs using standard classification accuracy.

Our results indicated that while the template-trained models achieved high lexical similarity metrics (Table 13), they exhibited lower classification accuracy on the protein localization task (Figure 2a). This suggests that these models, despite effectively reproducing the template structure, may not reliably predict the correct protein attributes. In contrast, Protein2Text demonstrated higher classification accuracy in this task, indicating a better ability to generalize and accurately predict protein localization in a zero-shot setting. Furthermore, we observed that the LitGene-based encoder predictor, which was specifically fine-tuned for protein localization, achieved the highest accuracy among the models evaluated. It suggests that decoder-based models like Protein2Text would benefit from further enhancements, such as larger or more diverse training datasets or architectural improvements, to close the performance gap, as GPT-4 and other general-purpose LLMs have matched supervised models for general NLP tasks.

\begin{table}
\begin{tabular}{l c c c c c c c c} \hline
* & **BLEU*
* & **BLEU*
* & **ROUGE*
* & **ROUGE*
* & **METEOR*
* & **BERT*
* & **BiomedBERT*
* \\  & **2*
* & **4*
* & **1*
* & **2*
* & **L*
* & **Score*
* & **Score*
* \\ \hline BioMedGPT
* & **0.075*
* & 0.0347 & 0.159 & 0.0536 & 0.1429 & 0.139 & 0.750 & 0.905 \\ Mol-Instructions
* & 0.067 & **0.038*
* & 0.193 & 0.0953 & 0.172 & 0.282 & 0.744 & 0.880 \\ Proft3
* & \(7\times 10^{-6}\) & \(9\times 10^{-7}\) & 0.062 & 0.001 & 0.061 & 0.017 & 0.769 & 0.843 \\ Protein2Text & 0.043 & 0.0248 & **0.265*
* & **0.148*
* & **0.239*
* & **0.326*
* & **0.815*
* & **0.897*
* \\ \hline \end{tabular}
\end{table}
Table 2: **Zero-shot analysis on unseen proteins.*
* Proteins and their variants, in this analysis, were held out during the two stages of Protein2Text training. However, it is not guaranteed that these proteins were also hidden during the training of the baselines (i.e. denoted by *).
 

We extended our evaluation to protein solubility prediction tasks and observed similar trends. The template-trained models again showed high lexical similarity scores but lower classification accuracy compared to Protein2Text and the fine-tuned encoder-based model (Figure 1(b)). These findings reinforce the notion that while template-based models excel in reproducing specific formats, they may not always capture the underlying protein attributes accurately.

### Ablation Study

Since wide-range ablation studies are prohibitive and time-consuming in LLMs due to their training time, we focus on more targeted ablation such as the extension of our model beyond LLaVA, the resampler. To assess the effect of the resampler, we remove it from the model. In this case, the <CLS> token from the protein encoder is used as the sole token representing the protein sequence, resulting in a single token projection. We compare this to the proposed model, in which the resampler creates 128 tokens, distilled from the embeddings of the entire protein sequence including the <CLS> token. The resampler uses roughly twofold the number of trainable parameters compared to the projector-only model (Table 9). We find a significant increase in performance across all metrics when moving from a single token projection to the latent 128 tokens generated by the resampler (Table 5).

### Qualitative Analysis

The results in Table 6 and 15 show substantial differences in the ability of various baseline models to answer questions related to proteins accurately and contextually. Protein2Text outputs precise and detailed responses that align closely with the truth of the ground in most questions. For example, it effectively identifies the primary function of P29474 and correctly notes the mitochondrial localization of A0A1W2PPH1. In contrast, models like ProtT3 and Mol-Instructions often produce incomplete or speculative answers, demonstrating limited generalization. Also, Protein2Text performed well in handling nuanced queries, such as the impact of nitrosylation on protein activity or changes in protein abundance at high altitudes. Nonetheless, despite its overall accuracy, Protein2Text occasionally fails and confidently generates incorrect answers (Table 15).

## 5 Conclusion

We introduced Protein2Text, a multimodal large language model designed to interpret protein sequences and generate descriptive text, facil

\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Model** & **BLEU** & **BLEU** & **ROUGE** & **ROUGE** & **METEOR** & **BERT** & **BiomedBERT** \\  & **2** & **4** & **1** & **2** & **L** & **Score** & **Score** \\ \hline BioMedGPT & 0.0577 & 0.0272 & 0.1724 & 0.0601 & 0.1506 & 0.1316 & 0.7344 & 0.9057 \\ Mol-Instructions & 0.0795 & 0.0475 & 0.2135 & 0.1159 & 0.1892 & 0.0475 & 0.743 & 0.8771 \\ ProtT3 & \(2\times 10^{-6}\) & \(3\times 10^{-7}\) & 0.05407 & 0.00166 & 0.05276 & 0.015878 & 0.7465 & 0.8387 \\ Protein2Text & **0.143** & **0.089** & **0.346** & **0.212** & **0.311** & **0.392** & **0.895** & **0.943** \\ \hline \hline \end{tabular}
\end{table}
Table 4: **Evaluating Protein2Text on the DiscussionQA set.** The DiscussionQA set is constructed from discussion sections of PubMed articles.

\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Model** & **BLEU** & **BLEU** & **ROUGE** & **ROUGE** & **METEOR** & **BERT** & **BiomedBERT** \\  & **2** & **4** & **1** & **2** & **L** & **Score** & **Score** \\ \hline BioMedGPT & 0.068 & 0.032 & 0.172 & 0.059 & 0.152 & 0.133 & 0.754 & 0.907 \\ ProT3 & \(5\times 10^{-6}\) & \(6\times 10^{-159}\) & 0.054 & 0.001 & 0.052 & 0.0167 & 0.748 & 0.840 \\ Mol-Instructions & 0.072 & 0.042 & 0.196 & 0.099 & 0.17079 & 0.287 & 0.733 & 0.877 \\ Protein2Text & **0.130** & **0.078** & **0.318** & **0.181** & **0.279** & **0.366** & **0.882** & **0.939** \\ \hline \hline \end{tabular}
\end{table}
Table 3: **Model evaluation on the IntroductioQA set.** The QA dataset is constructed from article introductions.

Figure 2: Evaluation on protein attribute prediction tasks: a) Subcellular localization and b) Protein solubility.

 

itating complex and open-ended question-and-answer tasks related to protein functions and attributes. By employing a resampling mechanism to embed protein sequences into a human-interpretable space compatible with language models, Protein2Text demonstrated strong performance on various benchmarks, outperforming general-purpose and several protein-specific multimodal LLMs, particularly in open-ended QA tasks. The model showed robustness across different types of textual inputs derived from scientific literature in both fine-tuned and zero-shot settings. To enable rigorous benchmarking, we compiled four new datasets to evaluate in-domain and cross-domain capabilities. Our analyses also revealed limitations of current metrics when dealing with template-based datasets like ProteinKG25, indicating that standard lexical similarity metrics may not fully capture a model's ability to predict specific protein attributes and highlighting the need for cautious interpretation of these metrics. Incorporating task-specific fine-tuning or architectural adjustments may help bridge the gap between decoder-based models like Protein2Text and specialized encoder-based models in certain applications.

By providing a framework capable of interpreting protein sequences and generating informative text, our work demonstrates the potential to use multimodal language models for protein analysis, which may assist researchers in exploring protein functions and attributes. We hope that releasing our evaluation datasets and model weights will encourage further research and development in this area, ultimately contributing to advancements in computational biology and bioinformatics. Protein2text is not immune to occasional hallucinations of incorrect answers, which represents an important avenue for future work.

## 6 Acknowledgements

This project was partially supported by R00CA248953 and the UNM Comprehensive Cancer Center Support Grant NCI P30CA118100. This research also used resources of the Oak

\begin{table}
\begin{tabular}{p{71.1pt}|p{34.1pt} p{34.1pt}} \hline \hline
**Protein2Text** & **Ground Truth** \\ \hline
**ID:**\(P29474\) & This protein **Q1**: What is the primary information of thesis of this protein in the audio? & **Protein2Text:** The primary function of this protein is related to the metabolism of nitric oxide. It is involved in the regulation of nitric oxide levels in the body, which is essential for various physiological processes. \\  & In this section, it is missing to determine the exact function of this protein in the body.

 

Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE- AC05-00OR22725.

## 7 Ethical Considerations

AI has a major impact on the scientific, health, and social fields. We encourage responsible evaluation of LLMs to eliminate potential biases that could affect future applications. We also encourage the responsible usage of resource and utilizing Low-Rank fine-tuning mechanisms when applicable, aiming to alleviate environmental risk. Protein2Text is an AI agent that is meant to positively contribute to the current progress by advancing state-of-the-art results and providing new evaluation benchmarks. However, our evaluation indicates that the model occasionally outputs incorrect answers confidently when uncertainty is warranted. As such, Protein2Text should be used as a complementary tool, with its outputs critically assessed by experts who understand the model's limitations.

## References

* A. J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooeoi, M. Monteiro, J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Barreira, O. Vinyals, A. Zisserman, and K. Simonyan (2022)Flamingo: a visual language model for few-shot learning. Preprint arXiv:2204.14198. Cited by: SS3.1.
* S. Banerjee and A. Lavie (2005)METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, Michigan, pp. 65-72. External Links: Link Cited by: SS1.
* D. Bertt, H. Poysil, J. Valcarcel, and W. Berk (2002)Alternative splicing and genome complexity. Nature Genetics30 (1), pp. eppu 2001 Dec 17. External Links: ISSN 0037-64X, Document, Link Cited by: SS1.
* T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020)Language models are few-shot learners. arXiv preprint arXiv:2005.14165. Cited by: SS1.
* A. Bushnuiev, B. Bushnuiev, J. Sedlar, T. Pluskal, J. Damborsky, S. Mazurenko, and S. Isvic (2024)Revealing data leakage in protein interaction benchmarks. PreprintarXiv:2404.10457. Cited by: SS1.
* R. Cadene, C. Danectte, B. Ben-younes, M. Cord, and D. Parikh (2020)Rubi: reducing unimodal biases in visual question answering. PreprintarXiv:1906.10169. Cited by: SS1.
* H. Cao, X. Liu, Y. Lu, and Z. Yu Li (2023)Instructmol: multi-modal integration for building a versatile and reliable molecular assistant in drug discovery. PreprintarXiv:2311.16208. Cited by: SS1.
* N. Carion, F. Massa, G. Synnaeve, U. Usunier, and S. Zagoryko (2020)End-to-end object detection with transformers. PreprintarXiv:2005.12872. Cited by: SS1.
* K. Bretonnel Cohen, L. Johnson, V. Verspoor, and E. Hunter (2010)The structural and content aspects of abstracts versus bodies of full text journal articles are different. BMC Bioinformatics11 (1), pp. 492. Cited by: SS1.
* E. PMC Consortium (2015)Europe pmC: a fulltext literature database for the life sciences and platform for innovation. Nucleic acids research43 (D1), pp. D1042-D1048. Cited by: SS1.
* G. Ontology Consortium, A. Askander, J. Balhoff, S. Carbon, J.M. Cherny, H. Drabkin, D. Ebert, M. Feurermann, P. Gaudet, N.L. Hill, L. Lee, H. Mi, S. Oxson, C. Mungal, A. Murguanguan, Y. Mohayama, T. Sternberg, P.P. Thomas, V. Auken, J.J. Kimsey, D. Chisholm RL, P.F. Poy, M. McAly, M.Nunes, F.Tosatto, M.S. Gliglio, M. Nadendla, G. Antonazzo, H. Attrill, S. Santos, S. Margold, V. Strelets, C. Tabone, J. Thurmond, P.Z. P. Ahmed, S.A. Smithung, D.L. Buitrago, M.H. Erdol MN, A.G. Mac, K. Kadhum, A.K. Krcc, L.M. Michalak, A. Pesala, A. Pritazawa, A. Saverimuttu SCC, S. R. Thurlow, H. Ke, L.M. Koc, C.L. Logie, S. Blake, K. Christie, L.Corbani, M.M. Dehakin, H.H. Drip, L.I. Ni, D.O. Smith, C.Cuzick, A. Seager, J. Cooper, L.Elser, J.Jaiswal, P. Gupta, P.Jaiwani, S.Lera-Ramirez, M.Rutherford, K. Wood, D.P. Jones, D.L.Dwinnell, M.R. Hayman, G.Kalduntsik ML, K. AE, L. SJF, T. MA, V.M. Wang, S.J.D'Etustachio, L.A. Azelsen, A. Hyka-Nouspikel, A.M. Morat, A.J. Assander SA, Cherry JM, E.SR, K.M. Miyasato 

SR, Nash RS, Skrzypek MS, Weng S, Wong ED, Bakker E, Berardini TP, Reiser L, Auchinolcs A, Axelsen K, Argoud-Puy G, Blatter MC, Boutet EE, Breuz LA, Bridge A, Casals-Casas C, Coudert E, Estreicher AA, Livia Famiglietti M, Fuermann M, Gos A, Gruaz-Gumowski N, Hulo C, Hyka-Nouspikel N, Jungo F, Le Mercier P, Lieberherr D, Masson P, Morgat A, Pedruzzi I, Pourcel L, Roux S, Rivoire C, Sundaram S, Bateman A, Bowler-Barnett E, Bye-A-Jee H, Denny P, Ignatcheko A, Ishtiata R, Lock A, Lussi Y, Magrane M, Martin MJ, Orchard S, Raposo P, Speretta E, Tyagi N, Warner K, Zaru R, Diehl AD, Lee R, Chan J, Diamantakis S, Raciti D, Zarowieck M, Fisher M, James-Zorn C, Ponferrada V, Zorn A, Ramachandran S, Ruzicka L, and Westerfield M. 2023. The gene ontology knowledgebase in 2023. Genetics, 224(1):ijayd031.
* The UniProt Consortium [2022] The UniProt Consortium. 2022. UniProt: the Universal Protein Knowledgebase in 2023. Nucleic Acids Research, 51(D1):D523-D531.
* Coudert et al. [2022] Elisabeth Coudert, Sebastien Gehant, Edouard de Castro, Monica Pozzato, Delphine Baratin, Teresa Netto, Christian J A Sigrist, Nicole Redaschi, Alan Bridge, and The UniProt Consortium. 2022. Annotation of biologically relevant ligands in UniProtKB using ChEBI. Bioinformatics, 39(1):btc793.
* Cui et al. [2024] Haotian Cui, Chloe Wang, Hassaan Maan, Kuang Pang, Fengning Luo, Nan Duan, and Bo Wang. 2024. scgpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, 21(8):1470-1480.
* Das et al. [2022]Sowmen Das, Md. Saiful Islam, and Md. Ruhul Amin. 2022. Gca-net : Utilizing gated context attention for improving image forgery localization and detection. Preprint, arXiv:2112.04298.
* Devlin et al. [2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. Preprint, arXiv:1810.04805.
* Du et al. [2019] Jingcheng Du, Peilin Jia, YuLin Dai, Cui Tao, Zhongming Zhao, and Degui Zhi. 2019. Gene2vec: distributed representation of genes based on co-expression. BMC Genomics, 20(1):82.
* Dubey et al. [2024] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. _arXiv preprint arXiv:2407.21783_.
* Elnaggar et al. [2021] Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rihawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steingeger, Debsindhu Bhownik, and Burkhard Rost. 2021. Protrtrans: Towards cracking the language of life's code through self-supervised deep learning and high performance computing. Preprint, arXiv:2007.06225.
* Fang et al. [2024] Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Hujun Chen. 2024. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. Preprint, arXiv:2306.08018.
* National Center for Biotechnology Information [2024] National Center for Biotechnology Information. 2024. Pubmed. [https://pubmed.ncbi.nlm.nih.gov](https://pubmed.ncbi.nlm.nih.gov). Accessed: 2024-11-26.
* Gu et al. [2021] Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare, 3(1):1-23.
* Guo et al. [2023] Han Guo, Mingjia Huo, Ruiyi Zhang, and Pengtao Xie. 2023. Proteinchat: Towards achieving chatqpt-like functionalities on protein 3d structures.
* Hu et al. [2021] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.
* Huang et al. [2020] Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2020. Clinicalbert: Modeling clinical notes and predicting hospital readmission. Preprint, arXiv:1904.05342.
* Jaegle et al. [2021] Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, and Joao Carreira. 2021. Perceiver: General perception with iterative attention. Preprint, arXiv:2103.03206.
* Jararweh et al. [2024] Ala Jararweh, Oladimeji Macaulay, David Arredondo, Olufunmilola M Oyebarniji, Yue Hui, Luis Taofya, Yanfu Zhang, Kwhal Virupakshappa, and Avinash Sahu. 2024. Litgene: a transformer-based model that uses contrastive learning to integrate textual information into gene representations. bioRxiv.
* Jia et al. [2024] Xiaogang Jia, Songlei Jian, Yusong Tan, Yonggang Che, Wei Chen, and Zhengfa Liang. 2024. Gated cross-attention network for depth completion. Preprint, arXiv:2309.16301.
* Trizel et al. [2021] Alexander Pritzel Tim Green Michael Figurnov Olaf Ronneberger Kathryn Tunyasuvunakosool Russ Bates Augustin Zidek Anna Potapenko1 Alex Bridgland Clemens Meyer Simon A. A. Kohl Andrew J. Ballard Andrew Cowie Bernardino Romera-Paredes Stanislav Nikolov Rishub Jain Jonas Adler Trevor Back Stig Petersen David Reiman Ellen Clancy Michalai Zielinski Martin Steingger Michalina Padcholska Tamas Berghammer Sebastian Bodenstein David Silver Oriol Vinyals Andrew W. Senior Koray Kavukcuoglu Pushmeet Kohl John Jumper, Richard Evans and Demis Hassabis. 2021. Highly accurate protein structure prediction with alphafold.

 

Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A Shoemaker, Paul A Thiessen, Bo Yu, Leonid Zaslavsky, Jian Zhang, and Evan E Bolton. 2022. Pubchem 2023 update. _Nucleic Acids Research_, 51(D1):D1373-D1380.
* Lee et al. [2019] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019. Biobert: a pre-trained biomedical language representation model for biomedical text mining. _Bioinformatics_, 36(4):1234-1240.
* Lewis et al. [2020] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. _arXiv preprint arXiv:1910.13461_.
* Li et al. [2023] Junnan Li, Dongxu Li, Chong Xiong, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pretraining with frozen image encoders and large language models. _arXiv preprint arXiv:2301.12597_.
* Lin [2004] Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In _Text Summarization Branches Out_, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
* Lin et al. [2022a] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, and Alexander Rives. 2022a. Language models of protein sequences at the scale of evolution enable accurate structure prediction. _bioRxiv_.
* Lin et al. [2022b] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, and Alexander Rives. 2022b. Evolutionary-scale prediction of atomic level protein structure with a language model. _bioRxiv_.
* Liu et al. [2023a] Haotian Liu, Chunyuan Lin, Fangyun Zeng, and et al. 2023a. Llava: Large language and vision assistant. _arXiv preprint arXiv:2304.08485_.
* Liu et al. [2020] Leilei Liu, Xianglei Zhu, Yi Ma, Haiyu Piao, Yaodong Yang, Xiaotain Hao, Yue Fu, Li Wang, and Jiajie Peng. 2020. Combining sequence and network information to enhance protein-protein interaction prediction. _BMC Bioinformatics_, 21(16):537.
* Liu et al. [2023b] Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. 2023b. Multimodal molecule structure-text model for text-based retrieval and editing. _Nature Machine Intelligence_, 5(12):1447-1457.
* Liu et al. [2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. _Preprint_, arXiv:1907.11692.
* Liu et al. [2024] Zhiyuan Liu, An Zhang, Hao Fei, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, and Tat-Seng Chua. 2024. Prott3: Protein-to-text generation for text-based protein understanding. _Preprint_, arXiv:2405.12564.
* Luo et al. [2023] Yizhen Luo, Jiathan Zhang, Siqi Fan, Kai Yang, Yushuai Wu, Mu Qiao, and Zaiqing Nie. 2023. Biomedgpt: Open multimodal generative pretrained transformer for biomedicine. _Preprint_, arXiv:2308.09442.
* Lv et al. [2024] Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chuan Chen, Li Yuan, and Yonghong Tian. 2024. Prollama: A protein language model for multi-task protein language processing. _Preprint_, arXiv:2402.16445.
* Madani et al. [2020] Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R. Eguchi, Po-Ssu Huang, and Richard Socher. 2020. Progen: Language modeling for protein generation. _Preprint_, arXiv:2004.03497.
* Muennighoff et al. [2023] Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Haniley Schoelokopf, Xiangru Tang, Dragomir Radev, Alham Fiki Arij Aki, Khalid Almubarak, Samuel Albanie, Zaid Aylafeau, Albert Webson, Edward Raff, and Colin Raffel. 2023. Crosslingual generalization through multitask finetuning. _Preprint_, arXiv:2211.01786.
* Nijkamp et al. [2022] Erik Nijkamp, Jeffrey Ruffolo, Eli N. Weinstein, Nikhil Naik, and Ali Madani. 2022. Progen2: Exploring the boundaries of protein language models. _Preprint_, arXiv:2206.13517.
* OtpenAI et al. [2023] OpenAI et al. 2023. Gpt-4 technical report. _arXiv preprint arXiv:2303.08774_.
* Ouyang et al. [2022] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhnii Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. _Preprint_, arXiv:2203.02155.
* Papineni et al. [2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
* Quazi [2022] Sameer Quazi. 2022. Artificial intelligence and machine learning in precision and genomic medicine. _Medical Oncology_, 39(8):120 

* Alec Radford et al. (2022)Alec Radford, Jong Wook Kim, Chris Hallacy, and et al. 2022. Whisper: Openai's speech recognition model. OpenAI.
* Radford et al. (2019)Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. _OpenAI_. Accessed: 2024-11-15.
* Richard et al. (2024)Guillaume Richard, Bernardo P. de Almeida, Hugo Dalla-Torre, Christopher Blum, Lorenz Hexemer, Priyanka Pandey, Stefan Laurent, Marie Lopez, Alexandre Laterre, Maren Lang, Ugur Sahin, Karim Beguir, and Thomas Pierrot. 2024. Chatnt: A multimodal conversational agent for dna, rna and protein tasks. _bioRxiv_.
* Schluter et al. (2009)Hartmut Schluter, Rolf Apweiler, Hermann-Georg Holzhutter, and Peter R. Jungblatt. 2009. Finding one's way in proteomics: a protein species nomenclature. _Chemistry Central Journal_, 3:11.
* Schuler et al. (1996)Gregory D Schuler, Jonathan A Epstein, Hitomi Ohkawa, and Jonathan A Kans. 1996. [10] entrez: Molecular biology database and retrieval system. In _Methods in enzymology_, volume 266, pages 141-162. Elsevier.
* Tjandra et al. (2017)Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura. 2017. Attention-based wav2text with feature transfer learning. _Preprint_, arXiv:1709.07814.
* Touvron et al. (2023a)Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. Llama: Open and efficient foundation language models. _arXiv preprint arXiv:2302.13971_.
* Touvron et al. (2023b)Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. 2023b. Llama 2: Open foundation and fine-tuned chat models. _arXiv preprint arXiv:2307.09288_.
* Wang et al. (2024)Chao Wang, Hehe Fan, Ruijie Quan, and Yi Yang. 2024. Protchatgpt: Towards understanding proteins with large language models. _Preprint_, arXiv:2402.09649.
* Wei et al. (2022)Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned language models are zero-shot learners. _Preprint_, arXiv:2109.01652.
* Wu et al. (2023a)Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023a. Prmc-llama: Towards building open-source language models for medicine. _Preprint_, arXiv:2304.14454.
* Wu et al. (2023b)Zhourun Wu, Mingyue Guo, Xiaopeng Jin, Junjie Chen, and Bin Liu. 2023b. CFAGO: cross-fusion of network and attributes based on attention mechanism for protein function prediction. _Bioinformatics_, 39(3):brad123.
* Xiao et al. (2024a)Yijia Xiao, Edward Sun, Yiqiao Jin, Qifan Wang, and Wei Wang. 2024a. Proteingrptt: Multimodal llm for protein property prediction and structure understanding. _Preprint_, arXiv:2408.11363.
* Xiao et al. (2024b)Yijia Xiao, Edward Sun, Yiqiao Jin, Qifan Wang, and Wei Wang. 2024b. Proteignpt: Multimodal llm for protein property prediction and structure understanding. _Preprint_, arXiv:2408.11363.
* Yang et al. (2020)Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2020. XLnet: Generalized autoregressive pretraining for language understanding. _Preprint_, arXiv:1906.08237.
* Zhang et al. (2022)Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Jiazhang Lian, Qiang Zhang, and Huajun Chen. 2022. Ontoprotein: Protein pretraining with gene ontology embedding. _Preprint_, arXiv:2201.11147.
* Zhang et al. (2024)Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2024. Instruction tuning for large language models: A survey. _Preprint_, arXiv:2308.10792.
* Zhang et al. (2023)Zuobai Zhang, Chuanrui Wang, Minghao Xu, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. 2023. A systematic study of joint representation learning on protein sequences and structures. _Preprint_, arXiv:2303.06275.
* Zhao et al. (2024)Zihan Zhao, Da Ma, Lu Chen, Liangtai Sun, Zihao Li, Yi Xia, Bo Chen, Hongshen Xu, Zichen Zhu, Su Zhu, Shuai Fan, Guodong Shen, Kai Yu, and Xin Chen. 2024. Chemdrm: A large language foundation model for chemistry. _Preprint_, arXiv:2401.14818.
* Zheng et al. (2023)Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llvm-as-a- judge with mt-bench and chatbot arena. _Preprint_, arXiv:2306.05685.

## Appendix A Dataset Collection

### Pretraining Dataset

The pretraining dataset consists of protein sequences and their corresponding descriptive information obtained from UniProt 1 (Consortium, 2022). We have removed proteins with sequences that are longer than 2000 due to insufficient resources. These sequences when fed through the model, consume GPU VRAM and cause CUDA error even with a batch size of 1. We have not 

performed any truncation to eliminate introducing noise to the model. We consider one specific prompt and its variant paraphrases such as _"Discuss the molecular function of this protein", "Determine the function of this protein sequence"_, or _"Summarize the functional role of this protein sequence"_. The question, the function description (as the answer), and the protein sequence are used to create the dataset. Similar to image, instruction, and response in LLaVA Liu et al. (2023). This dataset is entirely used to train the resampler and the projector during the pretraining stage.

An example of the dataset is presented in Table 11, illustrating the structure and content of the data entries. Table 10 provides statistical details about the dataset, including the number of unique proteins, their variants, and the average lengths of sequences and descriptions. Variants--proteins derived from the same gene family--were carefully managed to ensure no data leakage, as all splits were performed based on unique proteins.

### Finetuning Dataset: Protein2TextQA

The finetuning dataset (Protein2Text-QA) collection process involved two major steps: retrieving relevant abstracts from the literature and generating corresponding question-answer (QA) pairs using LLAMA3.

#### a.2.1 Abstract Retrieval

To collect protein-related abstracts, we used a systematic query approach with the PubMed Central (PMC) database (for Biotechnology Information, 2024). The queries targeted abstracts containing specific protein-related keywords. For each keyword, we performed a search using the Entrez library Schuler et al. (1996), which interfaced with the PMC API. The search results returned lists of relevant PMC IDs, which were then used to fetch the abstracts. To ensure relevance, only abstracts

\begin{table}
\begin{tabular}{l} \hline
**Protein2Text-QA** \\ \hline Q1: What is the primary function of this protein in brain development? \\ A1: It promotes neural progenitor cell survival and neurogenesis. \\ \hline Q2: What happens to the brain when this protein is depleted? \\ A2: The brain exhibits dysplasia with robust induction of caspase 9-dependent apoptosis. \\ \hline Q3: How does this protein influence cell survival and death in the developing brain? \\ A3: It regulates target genes that promote cell survival and neurogenesis. \\ \hline Q4: What signaling pathways affect the activities of this protein? \\ A4: TGFb2 and NF3baB signaling pathways influence its activities. \\ \hline Q5: What complex does this protein facilitate the genomic occupancy of? \\ A5: It facilitates the genomic occupancy of Polycomb complex PRC2. \\ \hline Q6: What is the general function of this protein? \\ A6: This protein is involved in inhibiting transforming growth factor-3b2 (TGF-3b2) signaling, which is a process that helps regulate cell growth and division. \\ \hline \end{tabular}
\end{table}
Table 7: **Sample of our Protein2Text-QA Data.** The data is extracted for the protein _”Smad nuclear-interacting protein 1”_ with ID: _“Q8TAD8”_.

Figure 3: The pipeline to collect **Protein2Text-QA**. The prompt used to query the LLaMA3.1-Instruct model is comprised of three components: the role, the abstract extracted from PubMed (for Biotechnology Information, 2024), and the protein name to extract QA for.

 

explicitly mentioning the queried proteins were included.

Once retrieved, the abstracts were processed to remove redundant text (e.g., headings such as _Abstract_, _Methods_, and _Conclusion_) and cleaned of formatting inconsistencies. This preprocessing ensured that the text was suitable for input into the question generation pipeline.

#### a.2.2 Generating the QAs using LLaMA3

Figure 3 demonstrates the QA collection process pipeline. The cleaned abstracts, protein names, and the role were fed into **LLaMA3.1-8B-Instruct**[45] to generate a conversation-style output. The model is prompted to generate a conversation between a chatbot and a human where the questions and answers are conditioned on the protein name mentioned in the prompt. The prompt instructed the LLaMA model to focus only on general protein functions and attributes explicitly mentioned while processing the abstract, ignoring other proteins. We limit the number of retrieved QA to up to 10 QA pairs per abstract.

The QA data are further preprocessed and tokenized to remove unnecessary questions that mention phrases such as _"no information found"_, _"answer not in the abstract"_, and _"not mentioned in the study"_. We attempt to make the questions general and related to the proteins instead of being related to the abstract. Table 7 shows a sample question and answers generated by LLaMA for the protein with ID _"Q8TAD8"_.

An example of the finetuning dataset is presented in Table 7, which highlights the structure of the QA pairs. The data extraction and question-generation pipeline, as implemented with LLaMA3 [46], is demonstrated in Figure 3. The overall statistics of the finetuning dataset, including the number of QA pairs, unique proteins, and sequence lengths, are summarized in Table 10.

### Evaluation Datasets

The evaluation datasets comprised four distinct subsets: Protein2Text QA test set, Zero-shot QA, DiscussionQA, and IntroductionQA. Each subset was curated to assess the model's performance.

First, the Protein2TextQA test set was randomly chosen from the entire Protein2TextQA without consideration of family or variant relationships. The protein sequences in the test set can be found in the pretraining dataset but not in the fine-tuning dataset.

Second, to generate the **Zero-shot QA** set, proteins and all their variants--defined as those from the same gene family--were entirely excluded from the training set. These proteins were included only in the test set, ensuring the model had no prior exposure to them during training. This dataset evaluates the model's ability to generalize to entirely unseen sequences.

The **Discussion QA** subset was derived using the same list of proteins from the test set subset. However, the QA pairs were generated from the _Discussion_ sections of the corresponding research articles instead of the Abstracts. This subset tests the model's ability to handle context-specific questions derived from a different section of scientific texts. Similarly, the **Introduction QA** subset utilized the same list of proteins as the test set subset, but the QA pairs were generated from the _Introduction_ sections of the articles. We were not able to extract introductions and abstracts for all of the articles, and we only considered proteins where we could find an introduction or discussion section that mentions them.

## Appendix B Gated Cross Attention (GCA)

Gated-Cross Attention (GCA) [19, 18, 17] attempts to find sampled media tokens that are influenced by the text tokens. For example, Alayrac et al. [18] used GCA to allow the text modality to attend to the vision modality through a gating mechanism that controls the influence of the vision modality on text features. Here, we attempt to do the same approach but we allow the protein embeddings to attend to the text embeddings, aiming to create refined protein embeddings.

In our setup, the GCA operates after the projector. That is, it takes as input the projected protein embeddings (\(\mathbf{H}^{\prime}_{v}\)) and the instruction embeddings (\(\mathbf{H}_{q}\)) and outputs text-informed protein embeddings (\(\mathbf{H}^{\prime\prime}_{v}\)).

The final text-informed protein embeddings (\(\mathbf{H}^{\prime}_{v}\)/) are then concatenated to the original instruction embeddings and fed to the LLM decoder to generate the response.

\[\mathbf{H}^{\prime\prime}_{v}=\phi_{GCA}(\mathbf{H}^{\prime}_{v}, \mathbf{H}_{q})\] \[\mathbf{H}=[\mathbf{H}^{\prime\prime}_{v};\mathbf{H}_{q}]\] 

The final set of tokens (**H**) is fed to the LLM decoder to obtain the language response.

\[{\text{response}=f_{\phi}(\textbf{H}).}\]

## Appendix C Training

### Training Procedure

The training consists of two main stages: pre-training and fine-tuning. Throughout the experiments in the manuscript, we use LLAMA3.1-Instruct model as the language decoder and facebook/esm2_t33_650M_UR50D as the protein encoder, unless otherwise specified. Every training stage is tailored to specific input, output, and training procedures. We now provide an overview of training details for every stage.

Pretraining.During pretraining, the model is expected to align the protein and the text modalities. Thus, we utilize protein sequences and their descriptions. During this stage only, the resampler and the projector are trained, aiming to learn the alignment between protein sequences and text. The dataset collected for this stage spans paraphrases on the question "Describe the function of the protein?". A sample of the dataset is shown in Table 11. We pre-train the model for one epoch following the LLaVA Liu et al. (2023a) approach. The number of trainable parameters for the stage is 42 million (Table 9).

Finetuning.We next train the model to predict answers to a wide range of prompts where the prompt and the sequence are fed as input, and the response as the output. During this stage, the re-sampler, the projector, and the LLM are trained. We utilize LoRA (Low-Rank Adaptation) to train the model Hu et al. (2021). LoRA freezes the pre-trained linear layers of the LLM architecture and learns a decomposition of two matrices of the frozen weights. The number of trainable parameters for this stage is 232 million parameters (190 million for LoRA adapters). The dataset used to train the model is a QA dataset. Refer to Appendix A and Table 7 for the dataset collection and an example conversation from the dataset respectively.

### Hyperparameters

Since performing a parameter search to find the best-performing parameters is computationally intensive and exhaustive for LLMs Benington et al. (2023), we rely on different factors to identify parameters. First, we inspect model parameters mentioned in previous studies in the same domain Gu et al. (2021); Liu et al. (2024); Fang et al. (2024); Lin et al. (2022b) or similar domains Liu et al. (2023a); Alayrac et al. (2022). Second, we track our training logs using Wandb to ensure the loss decreases for any respective ablation study.

Third, we also focus on targeted ablation studies to find the main parameters such as model sizes (i.e. ESM2-650 vs ESM2-3B). For example, Table 5 demonstrates reported ablation studies. We found that increasing model parameter

\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Stage**} & \multicolumn{3}{c}{**Number of Trainable Parameters**} \\ \hline Protein2Text & Pretraining & 22M & \\ _w/o resampler_ & Fine-tuning & 190M & \\ \hline Protein2Text & Pretraining & 42M & \\ _w/ Resampler_ & Fine-tuning & 232M & \\ \hline Protein2Text & Pretraining & 76M & \\ _w/ Resampler + GCA_ & Fine-tuning & 307M & \\ \hline \hline \end{tabular}
\end{table}
Table 8: Adding Gated-Cross Attention (GCA) on top of the resampler shows no performance improvement.

\begin{table}
\begin{tabular}{p{42.7pt} p{42.7pt} p{42.7pt} p{42.7pt} p{42.7pt} p{42.7pt} p{42.7pt} p{42.7pt}} \hline \hline
**Protein2Text** & **BLEU** & **BLEU** & **ROUGE** & **ROUGE** & **ROUGE** & **METEOR** & **BERT** & **Biomed-** \\  & **2** & **4** & **1** & **2** & **L** & & **Score** & **BERT Score** \\ \hline \hline _-w/ resampler-w/ GCA_ & **0.144** & **0.083** & **0.322** & **0.180** & **0.288** & **0.378** & **0.891** & **0.942** \\ -w/ GCA_ & 0.1017 & 0.0596 & 0.3054 & 0.170 & 0.278 & 0.358 & 0.863 & 0.929 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Number of parameters in various model architectures. Protein2Text w/o resampler refers to using only the projector (i.e. pure LLaVA model with changing the encoder).

 

size reduces the model performance and suggests the need for more data samples. Table 9 indicates the number of parameters for the main ablations performed. We found that increasing the number of latent tokens generated by the resampler from 128 to 256 worsened the performance of the model. Also, we saw adding gated cross-attention Alayrac et al. (2022); Das et al. (2022); Jia et al. (2024) increases the number of parameters but decreases the performance. Refer to Section B for description about adding GCA, Table 9 for number number of parameters, and Table 8 for GCA results.

Model training and inferencing were mainly performed on 2 NVIDIA H100 PCIe GPUs of 80GB VRAM. The estimated training time is roughly dependent on the number of parameters, the batch sizes, and other configurations such as gradient checkpointing, LoRA parameters, and resampler configurations. However, the estimated training time for the pretraining stage varies from 8 to 13 hours while the fine-tuning stage varies from 12-20 hours. The time estimations are based on the parameters found in Table 9. The table also highlights the best-performing model parameters of the experiments in this manuscript.

## Appendix D Expanded Discussion on Related Work

Instruction TuningLarge Language Models (LLMs) have demonstrated significant capabilities in human understanding tasks, such as GPT models Radford et al. (2019); Brown et al. (2020); OpenAI et al. (2023) and LLaMA models Touvron et al. (2023); Dubey et al. (2024). When LLMs were first introduced, they were mainly trained on next token prediction Touvron et al. (2023); Radford et al. (2019); Lewis et al. (2020); Liu et al. (2019); Yang et al. (2020). Instruction tuning has

\begin{table}
\begin{tabular}{l|c c c c|c c c} \hline \hline  & \multicolumn{4}{c|}{**Number of**} & \multicolumn{4}{c}{**Avg. Length**} \\ \cline{2-7}
**Split** & **QA Pairs** & **Sequences** & **Proteins** & **PMC IDs** & **Queries** & **Answers** & **Sequences** \\ \hline Pretraining & 393,849 & 393,849 & 70,854 & 0 & 8.8 & 42.1 & 378.4 \\ Fine-tuning & 209,847 & 5,556 & 5,574 & 29,198 & 12.3 & 12.9 & 511.0 \\ Test QA & 38,585 & 1044 & 1044 & 5,880 & 12.3 & 12.9 & 499.1 \\ Zero-shot & 14,107 & 348 & 348 & 2,164 & 12.2 & 13.0 & 433.9 \\ DiscussionQA & 2,629 & 180 & 180 & 263 & 12.9 & 17.3 & 385.6 \\ IntroductionQA & 1269 & 51 & 51 & 111 & 13.3 & 16.4 & 401.6 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Main statistics of the datasets used for the experiments in the study. Unique proteins can have different variants, and every variant has its sequence. These variants usually share a function similar to that of the dominant protein. We split based on unique proteins to eliminate data leakage. For the average length section, questions and answers are measured with words while sequences are measured in characters.

\begin{table}
\begin{tabular}{p{113.8pt} p{227.6pt}} \hline \hline
**PostAD8**: & & \\ Smad nuclear-interacting protein 1 & Required for pre-mRNA splicing as a component of the spliceosome, involved in the splicing of U12-type introns in pre-mRNAs (Probable). Down-regulates NF-kappa-B signaling by competing with RELA for CREBBP/EP300 binding. Involved in the microRNA (miRNA) biogenesis. May be involved in cyclin-D1/CCND1 mRNA stability through the SNARP complex which associates with both the 3’ end of the CCND1 gene and its mRNA. & \\ ATP synthase gamma chain & Produces ATP from ADP in the presence of a proton gradient across the membrane. The gamma chain is believed to be important in regulating ATPase activity and the flow of protons through the CF(0) complex.

 

been proposed to align the training objective with the user objective by enhancing the model's ability to follow instructions (Zhang et al., 2024). Several models trained via instruction tuning have been proposed for a variety of tasks such as summarization (Basyal and Sanghvi, 2023), question answering (Ouyang et al., 2022; Muennighoff et al., 2023; Zheng et al., 2023), and zero-shot capabilities (Zheng et al., 2023; Ouyang et al., 2022; OpenAI et al., 2023; Wei et al., 2022; Dubey et al., 2024).

Multimodal LLMs.Multimodal LLMs have also been extensively applied to perform cross-modal tasks beyond the text modalities. For instance, several studies have been proposed to integrate vision and language (Liu et al., 2023a; Alayrac et al., 2022; Li et al., 2023), and integrate audio and language (Radford et al., 2022; Tjandra et al., 2017). Building on these efforts, LLMs have also witnessed prosperous adaptation to scientific and biomedical domains such as biomedical text understanding (Jararweh et al., 2024; Lee et al., 2019), biomedical QA (Wu et al., 2023a; Luo et al., 2023), clinical reasoning tasks (Huang et al., 2020), and molecular structure understanding (Zhao et al., 2024; Fang et al., 2024; Cao et al., 2023; Liu et al., 2023b).

Protein-related LLMs.The sequential nature of protein primary structure lends itself to language modeling for protein characterization. For

\begin{table}
\begin{tabular}{l c} \hline \hline
**Hyperparameter** & **Pre-training** & **Fine-tuning** \\ \hline \hline _Training_ & \\ Number of Epochs & 1 & 5 \\ Per-device Batch Size & 10 & 5 \\ Learning Rate & \(2\times 10^{-3}\) & \(8\times 10^{-6}\) \\ Max Sequence Length & 2048 tokens \\ Precision & bf16 (Mixed Precision) \\ Optimizer & AdamW \\ Gradient Accumulation Steps & 1 step \\ Warmup Ratio & 0.03 \\ \hline _Protein Encoder_ & \\ Model & ESM2-650M \\ Output Tokens & All (i.e. no truncation) \\ Feature Layer & -2 (i.e. second to last) \\ \hline \hline _Language Model_ & \\ Model & LLaMA-3.1-8B-Instruct \\ LoRA Rank & 64 \\ LoRA Alpha & 16 \\ Context Length & 2048 \\ \hline _Projector_ & \\ Number of Layers & 2 layers \\ Activation & GELU \\ Hidden Dimensions & 4096 \\ \hline _Perceiver Resampler_ & \\ Number of Attention layers & 4096 \\ Attention Heads & 8 \\ Dimension of Attention Heads & 4 \\ Multiplication Factor of Hidden State & 2 \\ Number of Latent Tokens & 128 \\ \hline \end{tabular}
\end{table}
Table 12: An overview of the hyperparameters used to train the two stages of Protein2Text. If one parameter is mentioned across the two columns, the same value is used in the two training stages.

 

example, encoder-based LLMs trained on protein amino acid sequences have been adopted to generate a representation space that captures sequence structures Lin et al. (2022); Zhang et al. (2022); Elnaggar et al. (2021). Similarly, generative LLMs have also been proposed for a variety of protein generation tasks such as generating 3D structure John Jumper and Hassabis (2021), and novel protein sequences Madani et al. (2020); Nijkamp et al. (2022); Lv et al. (2024). LLMs that incorporate natural language and protein as one modality (i.e. considering protein as text modality) have been proposed. For example, Galactica models are general-purpose LLMs that are trained on scientific corpora to perform different reasoning tasks including protein captioning. Leveraging advances in multimodal LLMs, several studies attempt to integrate text with protein modalities such as DNA/RNA sequences Richard et al. (2024), 3D structure Guo et al. (2023); Wang et al. (2024), and amino acid sequences Xiao et al. (2024); Luo et al. (2023). Similarly, multi-modality projection similar to vision-language alignment Alayrac et al. (2022); Liu et al. (2023), has been applied to align between protein and natural text where protein is considered as single modality Guo et al. (2023); Wang et al. (2024); Liu et al. (2024); Luo et al. (2023); Fang et al. (2024).

## Appendix E Baselines

We compare our model to different baselines throughout the manuscript. We mainly focus on two types of baselines: general-purpose LLMs and protein-specific LLMs. The general-purpose LLMs were used as a measure of data leakage, identifying the amount of information leaked from the prompt into the generated answer. Second, we assess protein-specific LLMs that use protein sequences and a text prompt as input. We now provide a high overview of the baselines and the prompting mechanism.

GPT4o-miniOpenAI et al. (2023). The model is a variant of the GPT4 family with a reduced number of parameters. We used the OpenAI API to generate responses in this manuscript where we feed the prompt and the sequence as input. We set the role to _"You are an expert assistant for protein-related inquiries"_. The average response time is 30 seconds per query. We launched multiple processes per day for multiple days until the maximum number of tokens quota was reached.

LLaaM3.1-8B-Instruct[5]. LLaMA3.1-8B-Instret 2 is a general multilingual model trained using instruction tuning to perform reasoning tasks. We utilize the same prompt structure used to query GPT4o-mini to extract responses from the model. We use the released model checkpoints from HuggingFace to extract responses. The average request time is 30 seconds per prompt on an 80GB H100.

BioMedGPTLuo et al. (2023). BioMedGPT is a multimodal LLM that integrates molecular structures, protein sequences, and natural language text. The model aligns the three modalities to perform cross-modal tasks about proteins and molecular compounds. The model utilizes LLaMA2 Touvron et al. (2023) as the LLM base model. The training data was extracted from different sources such as PubMed Central (PMC), PubChem Kim et al. (2022), and UniProt Consortium (2022). We utilize the weights and default parameters released by the authors to perform inferencing. The inference time is 0.09 seconds per query on an 80GB H100.

Footnote 2: [https://huggingface.co/blog/llama31?utm](https://huggingface.co/blog/llama31?utm) source=chatgpt.com

Mol-Instruction[6]. Similarly, Mol-Instruction is a multimodal LLM that integrates text, molecular compounds, and protein sequences. The model utilizes GPT3.5 to generate a QA dataset about proteins and compounds from PubMed articles. We utilize the LoRA weights published by the authors and the LLaMA-2-7b-chat-hf model from HuggingFace to perform inferencing. We utilize the default parameters as found

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **BLEU-2** & **BLEU-4** & **ROUGE-1** & **ROUGE-2** & **ROUGE-L** & **METEOR** \\ \hline ProtT3- _Fine-tuned_ & **0.765** & **0.688** & **0.783** & **0.705** & **0.714** & **0.768** \\ Template predictor & 0.243 & 0.219 & 0.667 & 0.621 & 0.667 & 0.498 \\ Protein2Text- _Zero-shot_ & 0.277 & 0.217 & 0.447 & 0.345 & 0.383 & 0.396 \\ \hline \hline \end{tabular}
\end{table}
Table 13: Performances on the ProteinKG25 Zhang et al. (2022); Liu et al. (2024) benchmark. Template predictor refers to predicting the QA template as the response for all questions in the ProteinKG25 test set.

 

in the released evaluation script. The approximate inferencing time is \(18.17\) seconds per query on an 80GB A100.

ProtT3[Liu et al., 2024]. ProtT3 utilizes multi-modal projection to align between protein amino acid sequences and natural language text. The model is trained in two stages: protein-text retrieval and protein-text generation. During the first stage, contrastive learning objectives are utilized to extract protein features that match the description. Then, the LLM model is trained using LoRA to perform generative tasks. The authors release three different checkpoints for different tasks. We utilize the checkpoint released by the author for the QA task. The response time is 0.14 seconds per query on an 80GB H100.

LitGene[Jararweb et al., 2024]. LitGene is an encoder-based model that refines protein/gene embeddings by integrating textual descriptions and Gene Ontology (GO) terms. The model is designed for classification and retrieval tasks based on protein/gene embeddings. In this study, we use LitGene as a benchmark to evaluate our model ability in classification tasks. The results demonstrated in Figure 2 are based on benchmarks from the LitGene paper. We use their reported mean values on these benchmarks as a baseline for our model predictions.

## Appendix F Benchmarks

ProteinKG25.The ProteinKG25 benchmark is a template-based dataset designed for protein captioning. The dataset is originally a gene ontology knowledge graph that consists of protein sequences, descriptions, and protein attributes [Zhang et al., 2022; Consortium et al., 2023]. The authors of ProtT3 [Liu et al., 2024] synthesized a QA dataset based on the knowledge graph and used it for benchmarking. Table 14 shows a sample of the dataset, highlighting the template used to design the QA dataset from gene attributes.

Solubility.The solubility benchmark is a classification-based dataset that classifies whether a protein is soluble or insoluble. The dataset was collected by Jaranwei et al. jaranwei2024authors and used to benchmark their LLM-based model on the solubility task. The dataset originally consisted of protein descriptions and their respective classes. We further process the dataset into a QA format by extracting the sequences and adding the prompt _"Is this protein Soluble or Membrane?"_. The answer to this prompt would be the protein class: _"Soluble"_ or _"Insoluble"_.

Localization.Similarly, the localization dataset is another classification benchmark from the LitGene paper Jararweh et al. jaranwei2024authors. The dataset is designed to classify the subcellular localization of proteins and spans the classes: _"Nucleus"_, _"Cytoplasm"_, and _"Cell Membrane"_. To design a QA dataset from this dataset, we extract protein sequences using the UniProt IDs and add the prompt: _"Is this protein localized in Nucleus, Cytoplasm, or Cell Membrane?"_. The answer to this prompt would be the protein label from the LitGene localization benchmark.

## Appendix G Evaluation Metrics

BLEU Scores[Papineni et al., 2002]. BLEU (Bilingual Evaluation Understudy) score relies on n-grams matching to calculate the performance of the generated text. The BLEU score is a precision-based metric that quantifies the number of n-grams in the generated text that are also mentioned in the ground-truth text. **BLEU-2** denotes bigrams matching while **BLEU-4** denotes 4-grams matching.

ROUGE Scores[Lin, 2004].Similarly, the ROUGE (Recall-Oriented Understudy for Gigting Evaluation) score also focuses on sequence matching. As opposed to BLEU, the ROUGE score is recall-based which calculates the amount of n-grams from the ground truth that are captured by the generated response. **ROUGE-1** denotes unigram matching, **ROUGE-2** bigrams matching, and **ROUGE-L** denotes longest common subsequence matching.

METEOR Score[Banerjee and Lavie, 2005]. METEOR (Metric for Evaluation of Translation with Explicit ORdering) weights recall and precision while performing n-gram matching. METEOR also captures high-level semantic similarity by applying stemming and synonym matching.

BERT Similarity Score[Devlin et al., 2019]. We also utilize a similarity score generated using BERT. We embed the generated and ground truth sentences using the bert-base-uncased model, then we extract the embedding for the CLS token. The cosine similarity between the two sentences is then computed 

BiomedBERT Similarity ScoreGu et al. (2021). We also utilize BiomedBERT, a variant of the BERT model but trained on text from the the biomedical domain. We use the CLS embedding of the generated and the ground truth texts to calculate the cosine similarity.

## Appendix H Reproducibility and Availability

We release the model weights, evaluation scripts, and evaluation datasets at [https://github.com/alaaj27/Protein2Text.git](https://github.com/alaaj27/Protein2Text.git). Our efforts aim to enable results reproducibility and facilitate more rigorous evaluations of PLMs' generative capabilities across various tasks and different domains.

 



\begin{table}
\begin{tabular}{l|p{284.5pt}|p{170.9pt}}
**Protein ID/ Question** & **Ground Truth** & **Model Outputs** \\ \hline
**ID:** P1704I & The nucleus. & **Protein2Text:** The protein localizes to the nucleus. \\
**Q1**: Where does the protein localize? & & **LLaMA3:** The sequence you provided appears to be a long protein sequence, and it’s difficult to pinpoint the localization without knowing the specific protein or its function. \\  & & **ProT3:** NUCLEAR PROTEIN \\  & & **BioMedGPT:** Nucleus \\  & & **Mol-Instructions:** Based on the input provided, the protein localizes to the endoplasmic reticulum. \\ \end{tabular} 
\begin{tabular}{c c} \hline
**ID**: Q99489 & Nirosylation affects the activity of this protein in a limited extent. \\ \end{tabular} 
\begin{tabular}{c c} \hline
**DQ2**: What happens to the activity of this protein which it is unfyated? \\ \end{tabular} 
\begin{tabular}{c c} \hline
**ID**: G3V24I & No, the abundance of this proenvironment characterized by an elevated abundance of this protein? \\ \end{tabular} 
\begin{tabular}{c c} \hline
**QD3**: Is high-altitude & No, the abundance of this protein was decreased at high altitude into protein? \\ \end{tabular} 
\begin{tabular}{c c} \hline
**DD**: C7JQQ2 & No, this protein is a protein as a canonical enzyme that is a canonical enzyme similar to some other proteins. \\ \end{tabular} 
\begin{tabular}{c c} \hline
**DQ4**: Is this protein in a more aminoonical enzymeyme similar to some other proteins? & No, this protein is a canonical enzyme but a canon+ cannot+ have+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch+anch++anch+anch+anch+anch++anch+anch+anch+anch++anch+anch+anch++anch+anch+anch++anch+anch+anch++anch+anch++anch+anch++anch+anch++anch+anch++anch+anch++anch+anch++anch+anch++anch++anch+anch++anch++anch+anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch++anch+++anch++anch++anch++anch+++anch++anch+++anch++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch+++anch++anch+++anch+++anch++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch++anch+++anch+++anch+++anch+++anch++anch+++anch+++anch+++anch++anch+++anch+++anch++anch+++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch++anch+++anch+++anch++anch+++anch++anch+++anch++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch+++anch++++anch+++anch+++anch+++anch+++anch++++anch+++anch++++anch+++anch++++anch+++anch++++anch++++anch++++anch