GLP rotein: Global-and-Local Structure Aware Protein Representation Learning (2025)

Abstract: Proteins are central to biological systems, participating as building blocks across all forms of life. Despite advancements in understanding protein functions through protein sequence analysis, there remains potential for further exploration in integrating protein structural information. We argue that the structural information of proteins is not only limited to their 3D information but also encompasses information from amino acid molecules (local information) to protein-protein structure similarity (global information). To address this, we propose GLProtein, the first framework in protein pre-training that incorporates both global structural similarity and local amino acid details to enhance prediction accuracy and functional insights. GLProtein innovatively combines protein-masked modelling with triplet structure similarity scoring, protein 3D distance encoding and substructure-based amino acid molecule encoding. Experimental results demonstrate that GLProtein outperforms previous methods in several bioinformatics tasks, including predicting protein-protein interactions, contact prediction, and so on.

## introduction proteins are fundamental to virtually every biological process, serving as the building blocks for cells and organs and acting as catalysts, messengers, and structural elements in all life forms. understanding the structure and function of proteins is crucial for advances in health, agriculture, and environmental science, making protein research a cornerstone of biotechnology and medicinal science @xcite @xcite . recognizing the critical role of proteins in various scientific fields, many efforts have been made to design computational methods to further understand these crucial molecules @xcite . particularly, protein representation learning, * corresponding author as one significant part, involves capturing the complex features and relationships within proteins in a condensed form that can be utilized for various computational tasks and analyses. it is crucial for enhancing the understanding of protein structures and functions, improving predictive modelling in bioinformatics, facilitating the drug discovery process, and advancing our knowledge of biological systems through interpretable and efficient representations of proteins @xcite @xcite . in recent years, the success of language models in natural language processing (nlp) has paved the way for innovative approaches in bioinformatics areas, such as protein modeling @xcite , protein generation @xcite , and protein-protein interaction prediction @xcite . to be specific, by treating protein sequences as linguistic strings, these models have demonstrated remarkable effectiveness in predicting protein function based on sequence data alone. technically, as shown in figure 1 , protein sequences (e.g., the amino acid sequence 'mltahv...') are treated as sentences in natural language and amino acids (e.g., 'm', 'l', and 't') resemble words. thus, leveraging the powerful bert architecture originally developed for natural language, protbert @xcite adeptly adapts the bert @xcite masked language modelling framework to the field of bioinformatics. this analogy allows protbert to employ the technique of predicting randomly masked elements in sequences, thereby learning to identify complex patterns and dependencies among amino acids. similar to protbert, esm @xcite @xcite extends this paradigm by employing a more refined transformer-based architecture, focusing on capturing the evolutionary relationships and functional dynamics within protein sequences. in other words, most existing protein modelling methods aim to perform protein representation learning by encoding the protein's sequence information for various downstream applications, such as amino acid contact prediction @xcite , protein homology detection @xcite , protein stability prediction @xcite , protein-protein interaction identification @xcite , etc. despite the aforementioned successes, most existing protein language modelling methods suffer from intrinsic limitations. specifically, most of their focuses have primarily been on the amino acid sequence, often neglecting the crucial aspects of protein structure. proteins possess the ability to fold into diverse 3d shapes, interacting with various proteins and small molecules in biologically significant ways @xcite @xcite . since protein's structure determines function @xcite , utilizing protein 3d structure information effectively is crucial for protein language modelling, in which many studies have demonstrated the potential of pre-training on experimentally determined protein structures @xcite @xcite . nevertheless, these methods focus only on the structure within proteins and ignore the global similarities between proteins. we emphasize that the information on protein structure is not only limited to its structure (i.e., conformation) in 3d space but also includes information ranging from local amino acid molecules to the global structural similarity between proteins, as shown in figure 1 . local information involves the detailed properties and orientations of individual amino acids, which can affect protein stability and biochemical activity (renaud et al., 2021). these specifics are vital as they demonstrate how modifications or mutations at the amino acid level can alter the overall structure and functionality of the protein @xcite . furthermore, protein structure similarities provide information on evolutionary relationships and functional classes, which are crucial for understanding how structurally similar proteins of different species can perform similar or complementary functions within biological systems @xcite . for example, as shown in figure 2 , the bacterial ice-binding protein ffibp and the diatom adhesion protein catrailin_4 exhibit no detectable sequence similarity despite their functional similarities @xcite . their predicted structures exhibit a remarkable similarity (tm-@xmath0.6), with both proteins adopting a beta-helical fold comprised of two units linked by an alpha helix. this structural topology is characteristic of ice-binding proteins. such comparisons are key to predicting the functions of newly discovered proteins based on known structures, thereby enhancing our grasp of complex biological processes and interactions @xcite @xcite . however, most existing approaches have ineffectively incorporated amino acid molecule information and protein structural similarities into protein representation learning. to eliminate these limitations, we propose a novel protein pre-training framework glprotein with global-and-local protein structure information for protein representation learning. our major contributions are summarized as follows: ## methodology in this section, we aim to introduce our proposed framework (glprotein) as a novel solution to learn global and local protein structure information for protein representation learning. we develop glprotein that incorporates both global and local protein structure information into protein representation learning. the framework of glprotein, shown in figure 3 , consists of three components: protein language modelling (section 3.1), global structure information modelling (section 3.2), and local structure information modelling (section 3.3). ## experiments in this section, we evaluate the generalization ability of the learned protein representation by finetuning the pre-trained model across a diverse array of downstream applications, including amino acid contact prediction, protein homology detection, protein stability prediction, protein-protein interaction identification, protein-protein binding affinity prediction and semantic similarity inference. pretraining datasets. swiss-prot @xcite offers a comprehensive and manually curated protein sequence database that includes nearly 600k protein sequences. we use it as pertaining dataset. additionally, we use alphafolddb to obtain the protein 3d coordinate datasets. implementation details. we conducted some experiments and compared glprotein with baselines regarding pre-training and inference time in contact prediction tasks, as shown in appendix ## conclusion and future work in this work, we propose glprotein, a general protein language model with global and local protein structure information. glprotien outperforms the previous protein representation learning model on most downstream applications, demonstrating the performance superiority of glprotein. in the future, we aim to further enhance glprotein's capabilities by exploring novel avenues for incorporating multi-modal data sources, refining the model's interpretability, and extending its applicability to a wider array of biological contexts.