P rotein2 T ext: Resampling Mechanism to Translate Protein Sequences into Human-Interpretable Text (2025)

Abstract: Proteins play critical roles in biological systems, yet 99.7% of over 227 million known protein sequences remain uncharacterized due to the limitations of experimental methods. To assist experimentalists in narrowing down hypotheses and accelerating protein characterization, we present Protein2Text, a multimodal large language model that interprets protein sequences and generates informative text to address open-ended questions about protein functions and attributes. By integrating a resampling mechanism within an adapted LLaVA framework, our model effectively maps protein sequences into a language-compatible space, enhancing its capability to handle diverse and complex queries. Trained on a newly curated dataset derived from PubMed articles and rigorously evaluated using four comprehensive benchmarks—including in-domain and cross-domain evaluations—Protein2Text outperforms several existing models in open-ended question-answering tasks. Our work also highlights the limitations of current evaluation metrics applied to template-based approaches, which may lead to misleading results, emphasizing the need for unbiased assessment methods. Our model weights, evaluation datasets, and evaluation scripts are publicly available at https://github.com/alaaj27/Protein2Text.git.

## introduction proteins are essential to nearly all biological processes. understanding protein functions is crucial for unraveling disease mechanisms, predicting the effects of genetic mutations in conditions like cancer, and discovering targeted and personalized therapeutics @xcite @xcite . despite the characterization of 460,000 proteins in uniprot @xcite , a staggering 99.7% of the 227 million protein sequences remain poorly characterized @xcite . this vast number of uncharacterized proteins poses a significant bottleneck in biomedical research, impeding the full realization of the potential envisioned with the sequencing of the human genome. experimental methods for protein characterization are inherently time-consuming and costly, making it impractical to scale to millions of proteins. therefore, there is an urgent need for computational methods to complement and accelerate traditional experimental approaches. for the first time, large language models (llms) are offering an alternative to these challenges. for example, encoder-based models like esm-2 and ontoprotein leverage masked language modeling on protein sequences to generate embeddings that capture structural and functional information @xcite @xcite . similarly, to predict gene/protein structural and functional information, several approaches use other modalities such as text @xcite and expression @xcite . decoder-based models such as alphafold predict 3d structures from amino acid sequences @xcite . moreover, multimodal llms have been developed to bridge the gap between biological sequences and natural language, translating complex protein data into accessible human language @xcite . bimodal protein language models (plms), including protein-chat and protchatgpt @xcite , attempt to co-embed protein sequences with natural language using projection mechanisms. however, existing plms face limitations. a critical gap is the lack of rigorous quantitative evaluation on question-answering (qa) tasks, which are vital for practical utility. many plms depend on template-based qa datasets, transforming structured data into unstructured text using fixed templates @xcite @xcite . this methodology limits the models' ability to generalize to new, unseen queries and diminishes their adaptability to diverse instructions. consequently, template-based qa datasets hinder model expressiveness, and often -as we also demonstrate -overfit to specific patterns and lack the conversational flexibility necessary for addressing complex research questions (see therefore, we present a novel multimodal reasoning model that modifies the llava @xcite framework to adopt for the protein domain. our model provides real-time, interactive analysis of protein properties and handles complex, open-ended questions, empowering researchers to gain actionable insights for laboratory research. trained on a newly curated dataset derived from published literature on proteins in pubmed articles, our model benefits from a rich and diverse corpus surpassing template-based methods' limitations. we also compiled four comprehensive evaluation datasets to benchmark our model against existing plms rigorously. by releasing these evaluation datasets and model weights, we aim to promote a thorough assessment of protein llms across a wide range of tasks and specialized datasets. ## related work the sequential nature of protein primary structure lends itself to language modeling for protein characterization. for example, encoder-based llms trained on protein amino acid sequences have been adopted to generate a representation space that captures sequence structures @xcite @xcite . generative llms have also been proposed for a variety of protein generation tasks such as generating 3d structure @xcite , and novel protein sequences @xcite @xcite . llms that incorporate natural language and protein as one modality (i.e. considering protein as text modality) have been proposed. for example, galactica models are general-purpose llms that are trained on scientific corpora to perform different reasoning tasks including protein captioning. several studies attempt to integrate text with protein modalities such as dna/rna sequences @xcite , 3d structure @xcite , and amino acid sequences @xcite . similarly, multi-modality projection similar to visionlanguage alignment @xcite , has been applied to align between protein and natural text where protein is considered as single modality @xcite @xcite @xcite . see appendix d for detailed discussion on related work. ## protein2text protein encoder. our approach is based on llava @xcite which integrates images and text via instruction tuning. we adopt llava to protein amino acid sequences by replacing the image encoder with a protein encoder (figure 1b ). we use esm-2 @xcite a transformerbased encoder that has 33 transformer layers and a total of 652 million parameters. every sequence (p) is encoded to a multidimensional token embedding using esm2 (ϕ esm ) where every character is considered a token. formally: llm encoder. simultaneously, the instruction/question x q , given as natural language input, is tokenized and embedded using llama-3, ϕ llm : perceiver resampler. in llava, images are divided into a fixed number of patches, yielding a fixed number of image tokens without losing information. however, protein sequences have different sizes, and truncating them to a fixed size might remove potentially critical information. to this end, we extend their architecture by adding a protein resampler @xcite @xcite . the resampler finds a fixed number of latent tokens from varying-size protein sequences (figure 1b ). this reduces the computational complexity of the cross-attention in the llm and prevents long protein tokens from exhausting the model's maximum length. given the protein token embeddings (z v ), the resampler generates h @xmath0, where s is the number of latent tokens that compress the information in the original tokens: protein2text projector. to align the protein and the text modalities (figure 1a ), we project the dimensions of protein latent tokens (d) into the language embedding space (k) via the projector: ## conclusion we introduced protein2text, a multimodal large language model designed to interpret protein sequences and generate descriptive text, facilprotein2text: the primary function of this protein is related to the metabolism of nitric oxide. it is involved in the regulation of nitric oxide levels in the body, which is essential for various physiological processes. llama3: based on the provided sequence, it appears to be a protein sequence. however, without further information or context, it is challenging to determine the exact function of this protein in the body. prott3: immune system biomedgpt: may be involved in the regulation of cell proliferation. may be involved in the regulation of cell cycle progression. may be involved in the regulation.