LM 2 P rotein: A Structure-to-Token Protein Large Language Model (2025)

Abstract: Proteins are critical for various molecular functions, relying on their precise tertiary structures. This structure-sequence relationship is complex and degenerate, meaning multiple sequences can fold into a similar structure. The challenges in protein prediction, design, and modification increase with sequence complexity, while research on RNA-protein interactions, especially RNA-binding proteins (RBPs), is gaining importance. Large-scale pre-trained language models (LLMs) have shown promising results in handling biological sequences by treating them as natural language, though integrating spatial structures remains complex due to the need for specialized visual and 3D modeling approaches. We introduce a method to integrate protein 3D structural data within a sequence processing framework, converting 3D coordinates into discrete structure tokens using a VQ-VAE-like network. This simplifies the handling of 3D data, avoiding complex pipelines and facilitating a unified sequence-to-sequence processing model. Our approach demonstrates strong performance across a range of tasks, achieving high sequence recovery in inverse folding and protein-conditioned RNA design. These outstanding results demonstrate significant potential for application in complex biological systems research.

## introduction proteins play crucial roles in molecular biology and biochemistry, from molecular catalysis to signal transduction, with diverse functions relying on their precisely folded three-dimensional structures @xcite @xcite . however, there is no strict one-toone mapping between protein sequences and their 3d structures @xcite , and increasing sequence length and complexity poses significant challenges for prediction, * equal contribution. llms demonstrate powerful capabilities across tasks @xcite @xcite , leveraging self-supervised learning on large corpora to capture long-range dependencies and complex patterns. this approach extends to biological sequences such as proteins and nucleic acids. however, spatial structures often require additional computer vision or 3d pipelines, increasing complexity, resource usage, and potential mismatches in multimodal integration. to efficiently incorporate 3d structural information into a unified sequence framework, we propose discretizing protein 3d coordinates into "structural tokens" via a vq-vae-like network. this mapping from continuous 3d coordinates to discrete indices removes the need for elaborate visual or 3d pipelines, instead treating protein structures as sequential tokens. consequently, various tasks-sequence-to-structure, structure-tosequence, and protein-nucleic acid co-design-can be unified under a sequence-to-sequence (seq2seq) paradigm, maintaining autoregressive modeling strengths and leveraging llm capabilities in capturing long-range dependencies and intricate pattern recognition. the main contributions of this paper can be summarized as follows: (1) we propose a unified structure representation method: converting protein 3d coordinates into structure tokens through the discretization of vq-vae, enabling the autoregressive sequence modeling paradigm of traditional llms to be directly applied to protein structure learning. ## conclusion we introduced lm2protein, a unified framework that processes discrete structural tokens within a llm to bridge protein sequence and 3d geometry. this simplified approach achieves high sequence recovery in protein-conditioned rna design. however, it also reveals a critical challenge: the model captures local sequence motifs but struggles with global folding constraints. future work will focus on integrating biophysical priors to enhance structural accuracy and extending the framework to other molecular systems, paving the way for a universal model for in silico molecular engineering.