Large Language Models in Bioinformatics: A Survey (2025)

Abstract: Large Language Models (LLMs) are revolutionizing bioinformatics, enabling advanced analysis of DNA, RNA, proteins, and single-cell data. This survey provides a systematic review of recent advancements, focusing on genomic sequence modeling, RNA structure prediction, protein function inference, and single-cell transcriptomics. Meanwhile, we also discuss several key challenges, including data scarcity, computational complexity, and cross-omics integration, and explore future directions such as multimodal learning, hybrid AI models, and clinical applications. By offering a comprehensive perspective, this paper underscores the transformative potential of LLMs in driving innovations in bioinformatics and precision medicine.

## introduction bioinformatics is an interdisciplinary field that combines biology, computer science, and information technology to analyze and interpret complex biological data @xcite @xcite . recently, large language models (llms) have demonstrated remarkable progress in the domain of natural language processing (nlp), with applications that span a wide array of tasks @xcite @xcite . however, the nature of biological data and the associated tasks differ significantly from text data, presenting unique challenges. the accurate and precise handling of biomedical data to effectively form features and embeddings suitable for llms is an ongoing challenge that necessitates innovative solutions @xcite @xcite . within the biological domain, tasks exhibit a high degree of variability and specificity. these include the functional prediction and generation of * equal contribution dna sequences @xcite , the prediction of rna structure and function @xcite , the prediction and design of protein structures @xcite @xcite , and the analysis of single-cell data @xcite , which encompasses dimensionality reduction, clustering, cell annotation, and developmental trajectory analysis. there is a growing interest among researchers in harnessing the power of llms for bioinformatics and computational biology, yielding significant results @xcite @xcite . as illustrated in figure 1 , the development, training, and application of large models in bioinformatics are increasing at a rapid pace. despite this, the diverse methodologies focused on these various tasks have not been systematically summarized and analyzed, presenting an opportunity for comprehensive review and synthesis. organization of this survey: this paper provides a comprehensive review of recent advancements in llms for bioinformatics. we begin with preliminary concepts ( §2), covering key model architectures and their relevance to biological data. next, an overview of representative llms applied in bioinformatics is presented in ## preliminaries llms have demonstrated remarkable advancements across various ai applications, including bioinformatics, where they enable sophisticated sequence modeling, structure prediction, and functional annotation @xcite . mechanistically, the initial design of each llm typically follows three main architectural paradigms: encoder-only, decoder-only and encoder-decoder models. each of these architectures has distinct advantages and is suited for different types of bioinformatics tasks. therefore, in this section, we provide a concise overview of these architectures and corresponding relevance to their applications in bioinformatics. ## dna and genomics: learn and generate the research landscape of llms in genomics is witnessing a surge in development, particularly in their application across a spectrum of genomic tasks. these models not only improve the analytical capabilities of dna sequence analysis, but also accurately predict the impact of genetic mutations, identify key regulatory sequences, and thus advance the understanding of genomic functions. moreover, the generation of biologically functional gene sequences using llms is an area worth exploring, with potential implications for synthetic biology and gene therapy. the integration of llms in genomics has opened new avenues for research, offering insights into the complex interplay between genetic sequences and their biological functions. for instance, in the prediction of gene regulatory elements, llms have been shown to outperform traditional machine learning algorithms, providing a more nuanced understanding of gene regulation and expression @xcite . additionally, their application in the prediction of the effects of genetic mutations has led to a better comprehension of disease mechanisms and potential therapeutic targets. dnabert is a pre-trained bidirectional encoder representation, which can capture global and transferrable understanding of genomic dna sequences based on up and downstream nucleotide contexts. it can be fined tuned to many other sequence analyses tasks such as predict proximal and core promoter regions, identify transcription factor binding sites, recognize canonical and non-canonical splice sites and identify functional genetic variants @xcite . dnabert-2 is a high-performance foundation model based on the transformer architecture, designed for multi-species genome analysis @xcite . by incorporating innovative tokenization methods, efficient attention mechanisms, and a comprehensive evaluation benchmark, it provides a powerful tool for understanding and interpreting genomic data across diverse species @xcite . genebert is a large-scale pre-trained model that integrates diverse genomic data modalities across multiple cell types in a self-supervised manner. by leveraging multi-modal genome data during pre-training, genebert captures complex biological patterns and relationships, enabling robust performance on a wide range of downstream tasks. these tasks include promoter prediction, transcription factor binding site prediction, disease risk estimation, and rna splicing analysis. its ability to generalize across various genomic contexts and tasks makes genebert a powerful tool for advancing genome research and precision medicine @xcite . grover is a deep learning model designed to capture the structural and contextual features of dna sequences by simultaneously learning tokenlevel characteristics and broader sequence contexts. it demonstrates superior performance in both nextk-mer prediction and fine-tuning tasks, such as promoter identification and dna-protein binding prediction. grover's ability to effectively model dna language structure makes it a valuable tool for advancing genomic research and understanding regulatory mechanisms @xcite . megadna is a groundbreaking long-context generative model designed for genomic sequence analysis and de novo sequence generation. this model leverages a multi-scale transformer architecture to process and generate dna sequences at single-nucleotide resolution, enabling unprecedented capabilities in genomics research @xcite . in the realm of gene sequence generation, llms hold promise for the creation of sequences with specific biological functions. this capability could facilitate the design of genes for targeted therapies, the enhancement of crop traits, and the development of biomanufacturing processes. the potential of llms in this domain is vast, and ongoing research is expected to unveil novel applications and techniques. nucleotide transformer is a state-of-the-art foundation model designed for genomic sequence analysis, leveraging large-scale pre-training on diverse dna and rna sequences to capture complex biological patterns. developed to address the challenges of understanding genomic language, this model integrates advanced deep learning techniques with extensive genomic datasets, enabling robust performance across a wide range of downstream tasks @xcite . evo is a groundbreaking genomic foundation model designed to predict and generate biological sequences-spanning dna, rna, and proteins-from molecular to genome scales. developed by researchers at the arc institute and stanford university, evo leverages advanced deep learning architectures and large-scale pre-training to achieve unprecedented capabilities in biological sequence analysis and design @xcite . rna molecules play critical roles in biological systems, functioning as catalytic ribozymes, metabolite-sensing riboswitches, and epigenetically regulatory long noncoding rnas @xcite . these diverse functions are enabled by the ability of single-stranded rna to fold into intricate secondary and tertiary structures. accurate prediction of rna structures is therefore essential for understanding their biological mechanisms and therapeutic potential. however, rna structure prediction remains challenging due to the complexity and dynamic nature of rna folding. rna molecules exhibit conformational flexibility, long-range interactions, and non-canonical base pairing, making their 3d structures difficult to model. additionally, the scarcity of high-quality experimental rna structure data limits the training and development of llms for this purpose. recent advances in computational methods, including deep learning and physics-based simula-tions, have significantly improved rna structure prediction. here, we conclude some rna-related llms recently to better understand developments and limitations in rna structure predction. ## protein: prediction and design recently, llms have emerged as transformative computational tools in protein research, demonstrating remarkable potential to advance both fundamental comprehension and applied engineering of protein structures and functions. in this section, we introduce representative protein-related llms, as categorized in ## scrna: development and challenge single-cell sequencing technology, an evolution of transcriptome sequencing, enables the examination of gene expression and transcription levels at the individual cell level @xcite . this technology is pivotal for understanding various biological processes at the cellular level, such as disease progression, therapeutic efficacy, and resistance. it can identify relevant cell subpopulations and molecules, holding profound significance for the advancement of bioinformatics. however, the effective and accurate processing and analysis of vast single-cell data sets present a challenge for bioinformaticians. traditional analysis methods and pipelines are based on tools like seurat and scanpy @xcite . with the advancement of deep learning, numerous studies have integrated frameworks such as deep learning and large language models with the processing and analysis of singlecell data, achieving significant progress and demonstrating the immense potential of combining these approaches to faster development in the field of single-cell research @xcite . geneformer is a foundational deep learning model based on the transformer architecture, specifically designed for analyzing scrna-seq data. pre-trained in a massive corpus of 29.9 million single-cell human transcriptomes, geneformer uses self-supervised learning to capture gene-gene interactions and regulatory dynamics in various cell types and tissues @xcite . gptcelltype is an innovative r software package designed to automate cell type annotation in scrna-seq analysis by leveraging the capabilities of gpt-4, a state-of-the-art llms. this tool represents a significant advancement in the field of bioinformatics, offering a cost-effective and efficient alternative to traditional manual or semi-automated annotation methods @xcite . foundation models like scbert @xcite , and scgpt @xcite have transformed single-cell data analysis through selfsupervised pretraining on millions of transcriptomes, enabling robust transfer learning for tasks such as cell annotation, perturbation prediction, and batch correction. architectural innovations (e.g., performer encoders, rank-based encoding) address computational challenges of high-dimensional data, while task-specific designs improve generalizability and interpretability via attention mechanisms. however, limitations persist, including trade-offs between computational efficiency and expression resolution, biases from imbalanced pretraining data, and the "black-box" nature of transformer architectures. high computational costs further hinder accessibility. future efforts should prioritize multimodal integration of transcriptomic, epigenetic, and spatial data, knowledge-guided architectures incorporating biological networks, efficient fewshot learning frameworks, and lightweight adaptations for greater usability. addressing these challenges will enhance biological interpretability and bridge ai capabilities with actionable insights, advancing precision medicine and systems biology. ## conclusions and future directions in conclusion, this paper comprehensively examined the applications of llms across dna, rna, protein, and single-cell data analysis, highlighting key research contributions and emerging methodologies. despite advances, llm applications in bioinformatics remain evolving, requiring key challenges to address for full potential. therefore, we here discuss the current limitations and outline promising future directions for advancing llmdriven bioinformatics research.