Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models (2025)

Abstract: Large language models (LLMs) have shown remarkable capabilities in general domains, but their application to multi-omics biology remains underexplored. To address this gap, we introduce Biology-Instructions, the first large-scale instruction-tuning dataset for multi-omics biological sequences, including DNA, RNA, proteins, and multi-molecules. This dataset bridges LLMs and complex biological sequence-related tasks, enhancing their versatility and reasoning while maintaining conversational fluency. We also highlight significant limitations of current state-of-the-art LLMs on multi-omics tasks without specialized training. To overcome this, we propose ChatMultiOmics, a strong baseline with a novel three-stage training pipeline, demonstrating superior biological understanding through Biology-Instructions. Both resources are publicly available, paving the way for better integration of LLMs in multi-omics analysis. The Biology-Instructions is publicly available at: https://github.com/hhnqqq/Biology-Instructions.

## introduction understanding the intricate and dynamic activities across diverse omics layers in living organisms is of paramount importance. this includes studying dna regulatory elements that control gene expression @xcite , rna regulation @xcite ) that influences protein synthesis, and the functional properties of proteins themselves @xcite . these molecular processes critically affect the development of diseases and the synthesis of drugs within organisms. recent bert-like encoder-only models @xcite have achieved significant advances in natural language understanding tasks. when applied to genome or protein understanding tasks, these models @xcite are capable of capturing complex intrinsic relationships within biological sequences, achieving high accuracy in tasks such as promoter prediction. however, their reliance on specific classification or regression heads to predict a single task at a time limits their versatility, and their repeated fine-tuning sessions with different prediction heads to address multiple tasks further complicate the training, inference, and deployment processes. in contrast, powerful general-purpose large language models (llms), such as gpt-4 @xcite and gemini @xcite , trained on vast corpora of natural language data encompassing broad human knowledge, have demonstrated remarkable potential in domain-specific applications. these decoder-only models frame every task as a text completion problem through the next-token prediction objective, enabling them to unify diverse biological sequencerelated tasks by leveraging natural language as an intermediary representation-while preserving their inherent conversational capabilities. therefore, utilizing llms combined with unified training and dataset construction techniques can make it possible to replace bert-like models with a complicated fine-tuning pipeline. recently, several studies have explored leveraging llms for tasks related to biological sequences through instruction tuning, such as chatnt @xcite and prollama @xcite . while these models show promising performance, they are typically trained on instruction-tuning datasets that employ only basic linguistic patterns, thereby underutilizing the full linguistic capabilities inherent in pre-trained llms. moreover, these models mainly focus on singleomics data-either for proteins or dna-thereby limiting their capacity to deliver comprehensive multi-omics insights as a unified foundational language model. inspired by multimodal llms like minigpt-4 @xcite , we see an opportunity to extend this approach to biology. in biology, where molecular interactions are fundamentally grounded in the central dogma @xcite , integrating multi-omics data holds immense potential for generating mutually reinforcing insights. our study attempts to answer a key question: can instruction-tuned language models, already proficient in understanding human language, be effectively adapted to comprehend biological sequences and excel in solving biologically critical tasks? the motivation behind this inquiry lies in the intrinsic parallels between biological sequence data and human language-both are discrete, sequential, abundant, and rich in encoded information. these shared characteristics suggest that, with appropriate adaptation, instruction-tuned llms could unlock transformative capabilities in biology. to properly investigate the gap between human language and biological sequence understanding, we introduce biology-instructions, the first large-scale, multi-omics biology sequence-related instruction-tuning benchmark supporting 21 distinct tasks spanning dna, rna, proteins, and multi-molecular prediction tasks for a comprehensive assessment of biological understanding. using biology-instructions, we conduct a comprehensive evaluation of various open-source and closed-source llms, revealing that most models, including the state-of-the-art model such as gpt-4o, perform at near-random levels on biological sequencerelated understanding tasks without prior specialized training. this suggests the lack of inherent biological sequence knowledge in llms and highlights the critical need for effective strategies to integrate biological knowledge into these models. furthermore, we attempt to activate the biological multi-omics sequence understanding ability of llms with the constructed instruction data. we discover that solely performing instruction tuning on biology-instructions cannot yield satisfactory results. to address this gap, we propose a three-stage training pipeline: (1) train the model on unsupervised dna, rna, and protein sequences; (2) train the model on the question-answer pairs of biology-instructions; (3) train the model on reasoning data. the first stage serves as a warm-up to enhance the model's ability to understand biological sequences. in the second stage, the model follows natural language instructions to interpret biological sequences. in the third stage, the model leverages the implicitly learned knowledge base to perform reasoning and deepen its understanding of biological sequences. we incorporate reasoning data that begins with biological sequence analysis and culminates in conclusions derived from prior analysis and logical inference, ensuring that models retain comprehensive conversational capabilities while simultaneously acquiring a deeper understanding of biological sequences and tasks. we have implemented this training pipeline on llama3.1-8b-instruct @xcite using biology-instructions, resulting in significant performance improvements shown in figure 1. our findings and experiences are thoroughly documented. the contributions of this study can be summarized as: we present the first dataset specifically designed for multi-omics instructionfollowing, which includes reasoning instruction data and multi-sequence, multimolecule instruction data. this dataset aims to improve the ability of llms to comprehend and analyze biological sequences. to ensure the quality of biology-instructions, we conduct a comprehensive multi-stage quality control process, as detailed in appendix d. our results reveal that even sota llms can not solve biological sequence-related tasks. ## model as shown in figure 4 , we train a model based on llama3.1-8b-instruct (dubey et al., 2024) named chatmultiomics using multi-omics pre-training data and biology-instructions. in general, we perform a three-stage training paradigm to enhance the interactive, biology-related chat performance of the final biological assistant. please refer to appendix b for more training details. ## conclusion we introduce biology-instructions, the first large-scale, multi-omics instruction-tuning dataset focused on biological sequences. biology-instructions bridges the gap between large language models (llms) and complex biological tasks by encompassing 21 diverse tasks involving dna, rna, proteins, and multi-molecule interactions, spanning both single-sequence analysis and interaction-based reasoning. by incorporating reasoning-oriented instructions, biology-instructions enables llms to handle complex biological tasks with greater versatility while preserving conversational fluency. our evaluation reveals that sota llms such as gpt-4 struggle with biological sequence-related tasks without specialized training. using biology-instructions for instruction tuning, we demonstrate significant improvements, proving its value in enhancing llms for multi-omics sequence analysis. we also develop chatmultiomics, along with a three-stage training pipeline: continued pre-training, massive instruction tuning, and reasoning instruction tuning. this pipeline yields significant performance improvements, offering an effective and scalable approach for training biological llms.