Protein Large Language Models: A Comprehensive Survey (2025)

Abstract: Protein-specific large language models (ProteinLLMs) are revolutionizing protein science by enabling more efficient protein structure prediction, function annotation, and design. While existing surveys focus on specific aspects or applications, this work provides the first comprehensive overview of ProteinLLMs, covering their architectures, training datasets, evaluation metrics, and diverse applications. Through a systematic analysis of over 100 articles, we propose a structured taxonomy of state-of-the-art ProteinLLMs, analyze how they leverage large-scale protein sequence data for improved accuracy, and explore their potential in advancing protein engineering and biomedical research. Additionally, we discuss key challenges and future directions, positioning ProteinLLMs as essential tools for scientific discovery in protein science. Resources are maintained at https://github.com/Yijia-Xiao/Protein-LLM-Survey.

## introduction proteins are essential biological molecules, driving functions such as catalyzing biochemical reactions, maintaining cell structure, and enabling cellular communication. understanding their sequencestructure-function relationships is central to biological research. however, traditional experimental methods, including x-ray crystallography, nmr spectroscopy, and cryo-electron microscopy, are time-consuming and labor-intensive, posing bottlenecks for large-scale applications. recent advancements in language modeling have revolutionized computational biology, offering powerful tools for protein analysis. protein large language models (protein llms) share several foundational similarities with llms: 1) training objectives and learning paradigms, both llms and protein llms are trained in a self-supervised manner on large-scale datasets using objectives such as masked language modeling @xcite , auto-regressive modeling @xcite , or sentence permutation @xcite , learning to predict missing or next elements in sequences from the vocabulary. while llms predict missing words or phrases within textual data @xcite @xcite , protein llms predict amino acids or subsequences within protein sequences. 2) pretraining data. protein llms adopt a data-driven paradigm to learn directly from largescale protein datasets @xcite . the datasets for training protein llms consist of vast collections of protein sequences, analogous to the textual corpora used for llms. this eliminates the need for explicit feature engineering, allowing protein llms to learn intricate patterns, such as structural motifs, evolutionary relationships, and functional insights, similar to how llms capture semantic and syntactic structures in language. this paradigm shift has led to the emergence of highly effective models that can predict protein folding, annotate biological functions, and even design novel proteins with desired characteristics. however, adapting language modeling techniques to protein modeling introduces unique challenges. unlike natural language, protein sequences encode structural and biochemical constraints that are nonlinear and spatially dependent, making direct adaptation of nlp techniques suboptimal. long-range dependencies in proteins arise from 3d folding rather than sentence-level syntax, and the scarcity of annotated biological data further limits the effectiveness of supervised fine-tuning. additionally, interpretability and robustness are especially critical in biological domains, requiring models to incorporate domain-specific priors and rigorous evaluation standards. beyond their predictive capabilities, protein llms also provide interactive interfaces that allow users to upload protein sequences or structural files (e.g., pdb format), pose questions, and interact with the model in a conversational manner @xcite , proving deeper insights into protein structure, function, and design. recent surveys have covered the intersection of large language models and biological sciences. some works have reviewed protein language models in terms of model design, pretraining, and downstream applications @xcite , while others have discussed scientific llms more broadly, spanning biology, chemistry, and bioinformatics @xcite . we present the first dedicated survey of protein llms, analyzing their unique architectures, training methodologies, and practical applications in protein research. while previous studies have explored the applications of various computational methods for protein research @xcite or discussed the role of language models in general scientific domains such as biomedicine @xcite and chemistry @xcite , this survey focuses specifically on protein llms-a rapidly evolving area at the intersection of computational biology and nlp. the key contributions are as follows: ## llm methods for protein engineering, generation and translation protein engineering and generation aims to design protein sequences with desired attributes (e.g. structures and properties). given the desired attributes t and reference protein sequence s (optional), the model is expected to output a protein sequence s ′ with desired attributes. key tasks include: i. protein engineering: f θ : (s, t ) → s ′ modifies protein s toward the desired attributes t , yielding the engineered protein s ′ . ii. protein generation: f θ : (t, r) → p generates proteins with attributes t by sampling from the protein space using random seeds r. iii. protein translation: f θ : (p, t ) → p ′ translates a protein p into an alternative representation p ′ based on the target translation parameters t . ## datasets datasets are crucial for training and evaluating protein llms. they are categorized into pretraining datasets, comprising unlabeled protein sequences for self-supervised learning, and benchmark datasets, which contain labeled sequences for supervised fine-tuning and evaluation on specific biological tasks. ## conclusion and future work this survey provides an overview of protein large language models, highlighting architectures, datasets, evaluation, and applications. these works represent significant advancements in protein science and offer innovative approaches to protein analysis and design. several challenges remain. protein dynamics. alphafold @xcite has been shown to provide accurate static 3d structures. however, proteins are naturally dynamic molecules with various conformations @xcite . although several works incorporate 3d structures into llms, the conformational dynamics of proteins have not yet been considered. since conformational dynamics are highly related to the transporter functions of proteins, it would benefit the model to include protein dynamics. combination with single-cell data. recently, single-cell proteomics sequencing technology @xcite @xcite has attracted extensive attention in biology, which can help us understand the pathways in specific cells. since llms have shown effectiveness in understanding both proteins and single-cell data, they can be extended to learn from single-cell proteomics data in the future. interpretability. model interpretability is crucial for building trustworthy protein llms @xcite . unlike traditional models (e.g., decision trees, svms), protein llms are often opaque, making it difficult to identify which amino acids or structural features drive predictions-hindering adoption in biology and chemistry. progress has been made through attention-based attribution @xcite , latent space analysis @xcite , and recent methods like interplm @xcite , which use sparse autoencoders to uncover biologically meaningful representations.