Enhancing Graph Contrastive Learning for Protein Graphs from Perspective of Invariance
Authors: Yusong Wang
Shiyin Tan
Jialun Shen
Yicheng Xu
Haobo Song
Qi Xu
Prayag Tiwari
Mingkun Xu
Venue: ICML 2025 poster (2025)
Keywords: Protein Representation Learning
Graph Contrastive Learning
Graph Augmentation
3D geometry modeling
Structure Invariance
Primary area: applications->everything_else
TL;DR: We introduce biology-aware graph augmentations for GCL-based protein representation learning, integrating 2D functional community and 3D structure invariance to enhance protein representation learning.

Abstract: Graph Contrastive Learning (GCL) improves Graph Neural Network (GNN)-based protein representation learning by enhancing its generalization and robustness. Existing GCL approaches for protein representation learning rely on 2D topology, where graph augmentation is solely based on topological features, ignoring the intrinsic biological properties of proteins. Besides, 3D structure-based protein graph augmentation remains unexplored, despite proteins inherently exhibiting 3D structures. To bridge this gap, we propose novel biology-aware graph augmentation strategies from the perspective of invariance and integrate them into the protein GCL framework. Specifically, we introduce Functional Community Invariance (FCI)-based graph augmentation, which employs spectral constraints to preserve topology-driven community structures while incorporating residue-level chemical similarity as edge weights to guide edge sampling and maintain functional communities. Furthermore, we propose 3D Protein Structure Invariance (3-PSI)-based graph augmentation, leveraging dihedral angle perturbations and secondary structure rotations to retain critical 3D structural information of proteins while diversifying graph views.
Extensive experiments on four different protein-related tasks demonstrate the superiority of our proposed GCL protein representation learning framework.

Enhancing Graph Contrastive Learning for Protein Graphs from Perspective of Invariance
Yusong Wang" * 12 Shiyin Tan 2 Jialun Shen * Yicheng Xu²
Haobo Song? Qi Xu Prayag Tiwari
Mingkun Xu
Abstract
Perturbed
plane
Graph Contrastive Learning (GCL) improves Graph Neural Network (GNN)-based protein rep- resentation learning by enhancing its generaliza- tion and robustness. Existing GCL approaches for protein representation learning rely 2D topology, where graph augmentation is solely based on topological features, ignoring the in- trinsic biological properties of proteins. Besides, 3D structure-based protein graph augmentation remains unexplored, despite proteins inherently exhibiting 3D structures. To bridge this gap, we propose novel biology-awas aware graph augmentation strategies from the perspective of invariance and integrate them into the protein GCL framework. Specifically, we introduce Functional Commu- nity Invariance (FCI)-based graph augmentation. which employs spectral constraints to preserve topology-driven community structures while in- corporating residue-level chemical similarity as edge weights to guide edge sampling and main- tain functional communities. Furthermore, we propose 3D Protein Structure Invariance (3-PSI)- based graph augmentation, leveraging dihedral angle perturbations and secondary structure rota- tions retain critical 3D structural information of proteins while diversifying graph views. Exten- sive experiments four different protein-related tasks demonstrate the superiority of our proposed GCL protein representation learning framework.
Peptide
Brasse
plane
R
(a)
(b)
(c)
Figure 1. Visualization limitations in existing graph augmen- tation approaches for protein representation learning. Figure (a) shows a functional domain where nodes and edges are crroncously removed during graph augmentation. Figure (b) depicts how tradi- tional 3D augmentation with coordinate pernturbation distorts the peptide plane and critical bond relationships. Figure (c) compares the original protein structure (left) and its augmented views (right) generated by SWISS-MODEL, protein homology modeling tool. High similarity limits diversity.
1. Introduction
Graph Neural Networks (GNNs) (Scarselli et al., 2009) enable efficient protein representation learning by model- ing proteins graphs, where residues represent nodes and bonds represent edges. GNN encoders map each protein into vector representation, which is then processed by a pre- diction module to predict specific properties (Zhang et al., 2023; Wang et al., 2023b). Graph Contrastive Learning (GCL) enhances GNN-based protein representation learn- ing in two ways: first, it improves the generalization and robustness of GNNs through diverse protein graph augmen- tation strategies; second, GCL effectively captures structural patterns across different protein conformations, leading to more discriminative protein representations (Li et al., 2022; Wang et al., 2022a; Tan et al., 2024).
"Equal contribution GGuangdong Institute of Intelligence Sci- ence and Technology, Zhuhai, China School of Engineering, Insti- tute Science Tokyo, Tokyo, GGraduate School Medicine, The University of Tokyo. Tokyo, Japan "School of Computer Sci- ence and Technology. Dalian University of Technology, Dalian, China "Shooo of Information Technology, Halmstad University, Halmstad, Sweden. Correspondence to: Mingkun Xut <xum- ingkun gdiist.cn?..
Current GCL approaches for protein representation learning focus on 2D topology-based graph augmentation (Suresh et al., 2021). They seek minimize redundant infor- mation while preserving essential information across aug- mented views, assuming critical protein structures remain unchanged (Suresh et al., 2021). For instance, Wei et al. (2023) identify important edges between residues in a graph by measuring their semantic impacts based on the gradients of mutual information. Liu al. (2022a) explores graph
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Jivariance-Enhande Protein GCL
augmentation in the spectral space to preserve 2D protein topology through eigen-va seee-vvuue-bseed transformations. How- ever, only focusing topological structures tends to neglect the inherent biological significance of proteins, which di- lutes or even discards drructral-funcciioall associations tied to protein functionality. As Figure (a) shows, such graph augmentation disrupts a functional domain which is essen- tial for protein function (Heinemann et al., 2021).
preserve biological integrity within proteins.
We develop unified protein GCL framework that inte- grates both graph augmentation strategies to facilitate protein representation learning.
Experimental results on four downstream protein- related tasks highlight the superior performance of our proposed GCL framework, demonstrating its effective- ness in learning protein representations.
Furthermore, while integrating 3D structural information can enhance protein representation learning (Liu et al., 2022b), 3D structure-based graph augmentation remains unexplored. Although several 3D augmentation strategies exist, they face challenges when applied to GCL for protein representation learning: (1) applying traditional 3D augmen- tation (e.g., random coordinate perturbations, anisotropic scaling) (Hermosilla et al., 2021; Wang et al., 2023b) risks protein structure disruption. As Figure 1 (b) shows, the pep- tide plane is distorted by random coordinate perturbations, affecting protein representation learning. (2) using protein homology modeling tools like MODELLER (Eswar et al., 2006) and generate conformations (Li al., 2022: Gao et al., 2023) reduces structural diversity. As Figure (c) shows, gener- ated structures are nearly identical to original structures. It could also introduce inaccurate protein structure (see Ap- pendix A). Applying GCL with these compromised aug- mentation graphs can result in learning representations of biologically irrelevant proteins, undermining the model's performance on downstream tasks.
2. Related Work
Protein representation learning. Protein representation learning methods can be broadly divided into two categories: sequence- eqquence-based and structure- based approaches (Quan et al., 2024). Sequence- bequence-based methods primarily utilize word em- bedding techniques to capture the semantic relationships be- tween residues (Asgari & Mofrad, 2015; Yang et al., 2018; Zhuo et al., 2024) and one one-dimensional convolutional neu- ral networks extract local sequence patterns and motifs (Hou et al., 2017; Tsubaki et al., 2018; Kulmanov & Hoehn- dorf, 2019). For example, ProIBERT-BFD (Elnaggar et al., 2020a) trained a BERT model on a large corpus of pro- tein sequences and demonstrated superior performance by capturing sequential dependencies and evolutionary rela- tionships between residues. Structure-based approaches, predominantly based on GNN, more effectively lever- age structural information, such as spatial relationships, than sequence- based methods, enabling a more compre- hensive understanding of protein structures (Zhang et al., 2023; Wang et al., 2023b; Jamasb et al., 2024). For in- stance, GraphTrans (Liu et al., 2022b) developed a graph transformer architecture that processes protein structures as spatial graphs achieve ttate-of-the-art (SOTA) perfor- mance in protein property prediction tasks. Our research focuses on the latter.
(Waterhouse et al., 2018) to
To address aforementioned issues, incorporate biolog- ical properties of proteins to guide graph augmentations, thereby helping the GCL model learn accurate protein rep- resentations. Specifically, we propose two biology-aware graph augmentation strategies from the perspective of (i) 2D topelogy-based Functional Community Invariance (FCI) and (ii) 3D Protein Structure Invariance (3-PSI) and inte- grate them into unified GCL framework for protein repre- sentation learning. FCI ensures that functional communities, ie., clusters of spatially and chemically similar residues col- laborating to enable specific functions, remain intact. It con- siders both topological structures and chemical properties, ensuring the preservation of proteins' functional commu- nities during graph augmentation. To generate augmented graphs while preserving 3D-related information of protein, 3-PSI uses two distinct coordinate perturbation strategies: (1) rotations of backbone dihedral angles, and (2) rotations of secondary structures (a-helices and B-sheets), preventing the disruption of peptide planes and secondary structures during graph augmentation.
Graph Contrastive Learning. GCL has been widely adopted for enhancing GNN-based protein representation learning (Li et al., 2022; Tan et al., 2024; Wang et al., 2025). Current approaches focus on 2D topology-based graph aug- mentations, altering graph structures through node and edge modifications. Early methods, such as random node/edge dropping (Zhu et al., 2020; You et al., 2020; 2021) of- ten disrupted semantic integrity of proteins by removing key residues and bonds. Recent methods enhance protein semantic preservation via learnable distributions, mutual- information gradients, adversarial training, and spectral con- straints (Yin et al., 2021; Wei al., 2023; Suresh al., 2021; Liu et al., 2022a; Lin et al., 2023a). However, their focus on topology overlooks the biological significance of functional communities, missing key protein features.
Main contributions of this research are summarized as:
Based on the perspective of invariance, we propose two novel biology-aware graph augmentation strategies to
Despite proteins' inherent 3D nature, 3D structure- based protein graph augmentations remain unexplored. Existing
2

---

Setariance-Enhancee Protein GCL
3D augmentation strategies face challenges when applied to GCL for protein representation learning: traditional 3D augmentation methods, such as coordinate perturbations and anisotropic scaling (Hermosilla et al., 2021; Wang et al., 2023b; Feng et al., 2024), often disrupt peptide planes and protein hierarchical structures, yielding biologically im- plausible proteins; protein homology modeling tools (e.g., MODELLER (Eswar et al., 2006) and SWISS-MODEL (Waterhouse et al., 2018)) generate structural variants augmented views (Li et al., 2022) with limited diversity or inaccuracies, undermining the model's performance.
where [g] is the number of samples, is the graph embed- ding after encoder and readout functions, 2 is the embedding of augmented graph, and sim(;,) is the similarity function of two embeddings. LGCL quantifies the disagreement be- tween graphs.
Graph Spectrum. The spectral decomposition of Lnorn is defined as m = LAp(A) UAU where the diagonal matrix A = eig (Lap (A)) = diag (A1...., An) consists of real eigenvalues { 1,.... n,, known as the graph spectrum. And U [u],.... Un E B=*x are the corre- sponding orthonormal eigenvectors known as the spectral bases (Gene et al., 2013).
In this work, we incorporate inherently invariant properties of proteins into the augmented graphs and propose two graph augmentation strategies: FCI and 3-PSI, improving protein representation learning.
Key Protein-Related Terms. The hierarchical organization of protein structure emerges from the interplay of several fundamental elements. At its core, amino acid residues serve as the basic building blocks, with their variable side chains determining the chemical properties. These residues are connected by peptide bonds, forming rigid peptide planes that include the C N bond and adjacent atoms (Co.. C, N, and O). The spatial arrangement of these planes is governed by dihedral angles (ø and $)), which define the backbone's 's main conformation and give rise distinct local conformations along the polypeptide chain. These local conformations are stabilized by hydrogen bonds leading to the formation of common secondary structures such as a-helices and 8-sheets. The combination of these ele- ments determines the overall protein conformation, the dimee-dimeniiona shape that is essential for protein folding and function, thus influencing classification. The details are presented at Appendix B.
3. Preliminary
We employ GNN for its flexibility in integrating protein topology and geometry.
Notations. Let G = (V,E,P) be a protein graph with nodes (residues, each represented by its central carbon atom C.) and m edges, where V = {w denotes the set of nodes, and & {eur} represents the set of edges. P = iP:li=...... denotes the set of position ma- trices, where each P; E R4x33 x3 represents the position matrix for node i. The protein graph construction, in- cluding node features, edge features and rotational invari- ance, are detailed in Appendix C.I. The graph can be represented as adjacency matrix A E 10, where Ai.j 1 if an edge exists between node and otherwise Aij = 0. The normalized Laplacian matrix is defined as Laona = Lap(A) = I, - D D-1/2AD AD-1/2 where L, is an identity matrix, D is a degree matrix, D represents the degree of node i, and D.j = i + j. The complement of adjacency matrix is represented as A Af, = 1 an edge does not exist between node i and j. otherwise Ai,- O.
4. Methodology
In this section, we introduce two novel graph augmentation methods based on 2D topology-based Functional Commu- nity Invariance (FCI) in Sec 4.1 and 3D Protein Structure Invariance (3-PSI) in Sec 4.2. The overview of both aug- mentation methods is illustrated in Figure 2. Subsequently, we integrate both proposed augmentation methods into a unified GCL framework for protein representation learning in Sec 4.3. The overall architecture is illustrated in Figure 3.
Problem definition. A protein is represented as a quadru- plet G = (V,E,P.)) ) is the set of labels. The objective of protein classification is to learn mapping f: E) V.
Graph Contrastive Learning. GCL trains an encoder to maximize the mutual information between an original graph and its augmented view generated through graph augmen- tation. Specifically, given an encoder Enn(-),, readout function Readout(-) and an augmentation function t(G), the training objective of GCL is expressed as:
4.1. Functional Community Invariance
Within proteins, there are specific regions where residues cluster together to form communities. These clustered residues have similar chemical properties, collaborating to enable specific functions, such as residues clustered around active sites catalyzing enzymatic reactions (Mehta & Beck, 2014). Breaking such clusters during graph augmentation fails to preserve proteins' functionality and thus loses the key information for protein prediction. To address this, we introduce 2D topology- -bsed Functional Community
min LOCL (t (G), G, Enc, Readout) Fac
g
exp(sim(2,,, (2,, 2nn))
log
9
En'oe
2n)))
3

---

Geariance-Enhaneee Protein GCL
Protein 3D conformation
FCI view
Adjacency
Edge
Dihedral angle rotation
3-PSI view
R
Ori. view
Training
Peptide
Functional Community
Prob.
A ED AA
plana
(2) a-hellces f-sheets rotation
Sampling by Bern(A, e)
Backbons
edges
a-helix
Edge Dropping
Edge Adding
B-sheet
(G)
coW--constructed
edges
Hydrogen bond
Augmented graph
(a) Functional Community Invariance (FCI)
(b) 3D Protein Structure Invariance (3-PSI)
Figure 2. This is the overview of our proposed two invariance-based augmentation strategies. (a): FCI takes the adjacency matrix and sidc-chain similarity as input, optimizing the probability matrix in the spectral space to generate augmentations that preserve functional communities. (b): 3-PSI has two structure-invariann perturb residue coordinates while preserving peptide plane, and (2) a-helices and 9-sheets rotation that maintains secondary structures. Middle: The original protein structure with backbone edges GNN-constructed edges.
augmentation strategies: (1) Dihedral angle rotation that adjusts & and angles
Invariance (FCT) to graph augmentation GCL, which con- siders both topological structures and chemical properties (Figure 2 (a)), ensuring the preservation of proteins' func- tional communities during augmentation. In the following paragraphs, we formalize the FCI augmentation an opti- mization problem.
Ey= My - 11. respectively. Here, U, represents the i- th row vector of denoting the i-th node embedding the spectral space.
Therefore, to preserve communities, we drop edges that cause large spectral changes and add edges that cause small spectral changes, i.e., dropping edges between communities and adding edges within communities. The loss function of Community Invariance is:
Augmentation Process. We first formulate the graph aug- mentation process as sampling from a probability matrix according to a Bernoulli distribution,
((G) = Bern(A, t), where (-r E {0, 1) mxn is the sampling function from Bernoulli distribution, A 10 is the probability ma- trix that represents the edge perturbation rates, and controls the augmentation strength (the sampled number is less than m). By designing proper probability matrix A, we can ensure that key components of the graph are preserved during the augmentation process.
max A AM
La A ED A AAA = (AD(A ED - LAA(A EA),
(2)
LED( FD) |eig(Lap(A A RD)) eig(Lap(A)))
(3)
(A FA) |eeig(Lap(A A FAJ) eig(Lap(A))]I..
(4)
where denotes element-wise product and A is to guide edge addition operations. To achieve CI, we split the prob- ability matrix A to A ED and A for edge dropping and edge adding, respectively.
CI-based Augmentation. To ensure the preservation of community structure, we introduce Community Invariance bCl)-based augmentation by controlling the spectral changes. According to the upper and lower bounds of spectral changes (Theorem 4. )), maximizing spectral changes involves max- imizing their upper bound, which is achieved by flipping edges with largest distance spectral space. And nodes with larger spectral distances typically belong to different communities (proven in Appendix F.6).
FCI-based Augmentation. However, simply preserving communities without considering chemical properties may lead to incorrect assignment of residues with similar func- tional roles to different communities. To address this issue, we incorporate side-chains into CI-based augmentation, as they reflect the chemical properties of residues (Guy, 1985; Yan & Jernigan, 2005). Specifically, larity between the side-chains of two residues to measure their chemical similarities and use it as the edge weight. We refer to this similarity as chemical similarity, which com- puted using side-chain embeddings Vse. These embeddings use the sine and cosine values of the first four torsion angles
calculate the simi-
Theorem 4.1. (Bounds of Spectral changes) For single edge perturbation Aij. it induces absolute spectral changes given by 2ym1 AAyy Σn (U.,, ), + (Ag
() + U. It is upper bounded by ||U 1
Σカ Ay - and lower bounded by U, 1,

---

Divariance-Enhannee Protein GCL
Xy as elements, following Wang et al. (2023b):
in native states (Mishra & Jha, 2022), and reduce disrup- tions the primary and secondary structures of proteins, preserving the integrity of proteins as much as possible.
Wij- = Vse.i" Vie.j
Vsc - (on(xx))co((x)11))) (5)
Dihedral Angle Rotation. The protein backbone consists of repeating N - C. - C units, with its folding largely gov- erned by the backbone dihedral angles & and va (Jha et al., 2005). Directly perturbing absolute atomic coordinates dis- rupts the integrity of peptide planes and eventually disrupts hydrogen bond networks (Liwo et al., 1993; Nisius & Grze- siek, 2012). To mitigate this issue, we propose 3-PSIDiag. which applies guided and constrained rotations to dihedral angles, preserving the relative integrity of peptide planes. Specifically, we introduce small perturbations Aø and Ap:
where X and X1.2 denote the i-th torsion angles of residues T3 and rj, respectively.
Lemma 4.2. For weighted graphs, the absolute spec- tral changes Eymz Adyy during edge dropping are up- per bounded by (wis] (IIU, - -IIU, Here, Waj is the weight of the dropped edge Gij-
- Day -11)
By introducing chemical similarity (weights) for edges, clus- tering communities can better group residues that share similar chemical properties, leading to more biologically meaningful functional communities. According to Lemma 4.2, maximizing the spectral change during edge dropping results in the removal of edges with large edge weights. To ensure that edges with high chemical similarity are pre- served while edges crossing functional communities are removed, Function Community Invariance (FCI) is com- puted as:
djuuww = ((old) + +AØ, (i(eww) (h(old) + A (8)
while ensuring that (ci(new), ()(eew) remains within the al- lowed Ramachandran region (Ramachandran et al., 1963), thereby maintaining the energetic feasibility of the perturbed protein structure. After perturbing the dihedral angles, the coordinates of residues are adjusted correspondingly. It min- imizes disruption to hydrogen bonds and secondary structure interfaces, avoiding generating unrealistic protein structures.
max CECI ED AED AEA CED (
EA = (A - LEA(ARA) )A),
(6) a-helices and f-sheets Rotation. Proteins have well- defined secondary structures, primarily a-helices and B- sheets, which maintain protein stability and facilitate es- sential functions (Stanger et al., 2001). Existing 3D aug- mentation methods often disrupt the integrity of secondary structures, resulting in the learning of biologically irrelevant protein representations. To address this issue, we introduce 3- 3-PSIAlpha, a controlled rotation method for a-helices and 8-sheets to preserve essential secondary structures through- out the augmentation process. Specifically, we first identify the secondary structures in a protein (i.e., a-helices and | sheets) using the DSSP algorithm (Kabsch & Sander, 1983). Each secondary structure is approximated as a planar region, with its corresponding normal vector denoted as n,, We define the rotation axis of two adjacent secondary structures as the cross product of their normal vectors:
A AD
W
(7)
|cig(Lapp( W- - W WwW.WOO............ 1 - cigg(ap((W()]
where the element of W is chemical similarity defined in Eq. 5, the first term of LED is for preservation of high chemical similarity and the second term is for removal of cross-community edges. LEA remains the same as in CI since chemical similarity cannot affect edge adding. After solving Eq. 6, the augmented view is sampled by: ((G) = A Bern( A e) + A Bern(A EA c).
4.2. 3D Protein Structure Invariance
Exploring diverse 3D protein structures benefits models in understanding protein conformations and enhances the learning of protein representation. However, existing 3D augmentation methods applied for protein graph augmenta- tion, such as ordinate perturbations and anisotropic scaling, disrupt essential structural features, including peptide planes and key secondary structures like a-helices and B-sheets. Consequently, these disruptions in protein structures lead the model learn inaccurate 3D spatial relationships between residues, degrading its performance on downstream tasks. To alleviate this problem, we propose 3-PSI, novel graph augmentation method that incorporates two types of 3D protein structure invariance. As illustrated in Figure 2 (b), 3-PSI includes Dihedral Angle Rotation (3-PSIDiag) and a-helices and 3-sheets Rotation (3 (3-PSIAlpha). These strate- gies make use of the flexible and dynamic nature of proteins
(9) where x denotes the cross product operation, ensuring that a is perpendicular to both planes. A small random rotation angle 0 is then sampled from uniform distribution to de- termine the extent of the opening closing motion around a: 0 ~ U1(0mmax: mmax). This controlled rotation ensures that secondary structures remain intact while introducing realistic conformational variations. A 3D rotation about the axis a by the angle 0 can be described by the standard Rodrigues rotation matrix Ra,0- For unit axis à ||||
n2 X nj
Ra is given by:
R = I cos 0 + (1 - cos a)āa + [a] x sin 0 (10)
5

---

Enwariance-nhhanee Protein GCL
where denotes the skew-symmetrri matrix, as given in Appendix C.3. Next, randomly select n. secondary struc- tures and rotate them around by the angle 0. This ensures that the secondary structure is preserved while still keep meaningful 3D structure perturbations for augmentation.
and cellular component (CC) ontology term prediction. De- tails of tasks and datasets are provided in Appendix D.1.
Evaluation Metrics. FOLD and Reaction are single-label classification tasks, where accuracy serves as the evaluation metric. GO and EC are multi-label classification tasks, evaluated using the protein-centric maximum F-score Fmoss as detailed in Appendix D.2.
4x
FCI view
Classification
encoder
Pooling
Contrastive learning
Comparison Methods. We evaluate different combinations of all the proposed augmentation strategies and conduct ab- lation studies on each strategy to validate their effectiveness. For a comprehensive comparison, we select various SOTA methods from different categories as baselines. Protein- specific methods: GVP (Jing et al., 2021), GraphQA (Bal- dassarre et al., 2020), 3DCNN (Derevyanko al., 2018), IEConv (Hermosilla & Ropinski, 2022), GearNet (Zhang et al., 2023), ProNet (Wang et al., 2023b), CDConv (Fan et al., 2023); 2D topology-based GCL: GraphCI. (You et al., 2020), Auto-GCL (Yin et al., 2021), MolCLR (Wang et al., 2022b), T-MGCL (Guan & Zhang, 2023) GCL-Span (Lin et al., 2023a), GCS (Wei et al., 2023), CI-GCL (Tan et al., 2024); 3D structure-based GCL: protein homology modeling tool-based augmentation (Model. Tool) (Water- house et al., 2018). and traditional 3D augmentation (Trad. 3D) (Wang et al., 2023b). To our best knowledge, few GCL methods have been specifically designed for proteins. There- fore, we have additionally included two GCL approaches designed for molecules and leveraged their methodological concepts. We adopt a hierarchical Graph Convolutional Network (GCN) serving as the backbonn (encoder) for all implemented GCL methods. Details of the baselines and training setup are in Appendix D.3 and C.2, respectively.
GNN
FC
Ori view
Protein 3D conformation
3-PSI view
Figure 3. The architecture begins with three views of input graphs: the original view and two augmented views (FCI and 3-PSI).
4.3. Optimization
We follow the GCL framework introduced in Sec 3, where the original view and augmentation views (FCI and 3-PSI) are passed through the GNN encoder, as illustrated in Fig- ure 3. Following Fan et al. (2023)'s work, each view un- dergoes four hierarchical pooling operations, with all views sharing the same GNN encoder at each hierarchical level. The output from the final GNN encoder represents each view and is used for contrastive learning. The original view also serves as input for downstream tasks. The GNN en- coders are optimized using combination of GCL loss and classification loss, defined as follows:
LALL = LCLS+ + (FCD) + LGCL
(3--SS)))
(11)
where LeLS is the classification loss!, L.cer is the GCL loss based on different augmentation methods, and A is the weight to balance the GCL and classification loss. 3-PSI augmentation is selected either by 3-PSIDiag or 3-PSIAlpha. Details of the GNN encoder can be found in Appendix C.4.
5.2. Quantitative Results
We summarize the performance comparison of different combinations of FCI and 3-PSI augmentations across four tasks in Table 9. Extra quantitative results are provided in Appendix F. "Backbone" is learning without any graph augmentation and only supervised by classification loss.
5. Experiments
Compared to 2D topology- based GCL approaches, our pro- posed FCI consistently enhances performance across all tasks. Notably, it surpasses the best 2D topology- GCL method, achieving 4.4% relative gain in GOCC. These results suggest that relying solely on 2D topological information may be insufficient for comprehensive protein representation learning. By preserving functional commu- nity information in augmented graphs, FCI enhances protein representation learning. leading to improved performance.
5.1. Experimental Settings
Tasks. Building upon the evaluation protocols of Fan et al. (2023), we assess the effectiveness of our approach across four protein-related tasks: Protein Fold Classification (FOLD), Enzyme Reaction Classification (Reaction). Gene Ontology Term Prediction (GO), and Enzyme Commission (EC) number prediction. For FOLD, we evaluate perfor- mance under three scenarios: fold, superfamily, and family classification. For GO, we assess performance across three sub-tasks: biological process (BP), molecular function (MF),
Compared to 3D graph augmentation methods, our proposed 3-PSIDiag and 3-PSIalpha yield consistent improvements
FFllowing (Fan et al., 2023), we use NLL loss for single-lahel protein classification tasks (RC and FOLD) and BCE loss for multi-label protein classification tasks (GO and FC).
"Since combining both 3-PSIDiag and 3-PSIAlpha would dis- rupt the protein structure, we use them separately.
6

---

Eivariance-Enhance Protein GCL
Table This table shows the comparison of protein-speeific methods, graph contrastive learning approaches (hoth 2D) topology-based and 3D structure- based) and our proposed GCL approach on various protein-related tasks. We report the maximum of Fl score (Fmaz) for Enzyme Classification (EC) and Gene Ontology (GO) prediction tasks; and Top-1 accuracy (%) for Fold Reaction classification tasks. The performance of CDConv is from our reproduction, while other baseline results are from Fan et al. (2023).
GO
FOLD
Method
EC
Reaction
BP
MF
CC
Fold
Super.
Fam.
Avg.
GVP
0.489
0.326
0.426
0.420
16.0
22.5
82.8
40.4
65.5
Protein Protein-specile specific
3DCNN
0.077
0.240
0.147
0.305
31.6
45.4
92.5
56.5
72.2
GraphQA
0.509
0.308
0.329
0.413
23.7
32.5
84.4
46.9
60.8
|EConv
0,421
0.624
0,431
47.6
70.2
99.2
72.3
87.2
0.810
0.400
0.581
0.430
48.3
70.3
99.5
72.7
85.3
ProNet
52.7
70.3
99.3
74.1
86.4
CDConv
0.870
0.450
0.652
0.475
56.9
76.7
99.5
77.0
88.6
2D Topo. GCL
GraphCL
0.860
0,432
0.644
0.432
56.9
77.8
99.2
77.9
86.7
Auto-GCL
0.852
0.436
0.641
0,417
53.6
72.1
99.1
74.9
86.1
MolCI.
0.869
0.443
0.623
0.455
54.0
77.9
99.5
77.1
87.7
T-MGCL
0.843
0,422
0.628
0.468
55.0
77.6
99.5
77.3
85.3
GCL-Span
0.859
0.434
0.650
0,419
58.2
79.1
99.4
78.9
86.1
GCS
0.864
0.440
0.645
0.437
54.7
75.8
99.2
76.5
87.1
CI-GCL
0.870
0.440
0.651
0.453
57.5
79.2
99.5
78.7
86.6
3D
Trad. 3D Aug.
0.870
0.432
0.645
0.420
56.6
78.5
99.4
78.1
86.5
Auto. Tool Aug.
0.863
0.439
0.643
0.435
56.
77.7
99.4
77.4
86.3
Ours (Ablation)
Backbone
0.853
0.437
0.642
0.432
56.5
76.9
99.3
77.8
86.3
FCI
0.878
0.446
0.656
0,473
58.7
79.4
99.5
79.2
87.2
+ 3-PSSIDiag
0.873
0.449
0.654
0.468
58.1
78.6
99.5
78.7
86.9
+3-PSIAlpha
0.874
0.444
0.652
0.453
56.7
80.4
99.5
78.8
88.3
33-PSIDiag + FCI
0.883
0.461
0.662
0.484
58.9
81.3
99.7
80.0
87.8
+ +3-PSIAlpha + FCI
0.885
0.454
0.659
0.477
59.8
80.8
99.5
80.0
89.0
across all tasks. In particular, 3-PSIDiag achieves relative improvements of 11.4% in GOCC while 3-PSIalpha shows relative improvements of 2. in Reaction compared with best-perforning baselines. These improvements stem from the ability of 3-PSIDiag and 3-PSIAlpha to preserve es- sential protein structural information in augmented graphs, unlike traditional 3D augmentation methods that risk struc- tural disruption lack conformational diversity.
structure-based graph augmentations, using the optimal framework setting for each task (e.g., 3-PSIAlpha FCI for EC, 3-PSIDiag + FCI for GO). The complete results are provided in Appendix F.6.
2D Augmentation Strength Analysis. For 2D augmenta- tion comparison, we select GraphCL as the baseline, which randomly drops and adds edges during augmentation. As illustrated in Figure 4, we vary the augmentation strength from 0.1 to 0.4, as performance degradation becomes significant beyond this range. Our approach consistently outperforms random augmentation, showing superior perfor- mance and stability across different strengths. Furthermore, the results suggest that an optimal e critical for generating effective graph views GCL, as it balances augmentation diversity with the preservation of essential graph properties. The default for different tasks is provided in Appendix C.2.
Our combined graph augmentation approaches (3-PSIDiag + FCI and 3-PSIAlpha FCI) achieves substantial improve- ments over all three categories of baselines. These com- prehensive results highlight the synergistic effect of inte- grating both augmentation strategies, as they complement each other in preserving biological properties of proteins. By maintaining both functional communities and protein 3D structural integrity, our approach generates graphs with enriched protein properties, enabling models to learn better protein representations and enhancing the performance.
3D Rotation Strength Analysis. While our proposed a- helices and 8-sheets rotation preserves protein secondary structures, we vary the number of rotation operations from to 5 across different tasks, as the number of rotations directly influences augmentation strength. As illustrated in Figure 5, we observe that FOLD and GOBP achieve
5.3. Effect of Augmentation Strength
In this section, we evaluate the effect of augmentation strength of our framework in 2D topology-based and 3D

---

Separiance-Enhancee Protein GCL
GraphCL
Ours
GCL. As presented in Figure 6, our method consistently outperforms baseline methods across all tasks, particularly at higher proportions, demonstrating strong robustness un- der high-noise conditions. While all methods experience performance degradation as the proportion increases, our approach exhibits less decline, maintaining ~ 5% better performance- pacicce--prtccuaarry in the Reaction task compare to the strongest baseline, Trad. 3D Aug.
0.90
90
89
0.88
88
0.85
87
86
0.84
85
0.1
0.2
0.3
0.4
0.1
0.2
0.3
0.4
(a) e of EC
(b) e of Reaction
Figure 4. Performance comparison between GraphCL and our approach under different graph augmentation strengths (0.1-0.4).
Trad3D
SDCov
CI-GCL
Ours
0.90
90
85
optimal performance with 2 rotations; Reaction and GOCC perform best with 3 rotations. These findings highlight that optimal augmentation strength (2-3 rotations) provides balance between diversity and fidelity of protein structure. Additionally, the augmentation strength should be selected based on the specific requirements of each task, as too few rotations may provide sufficient diversity, while rotating excessively can potentially distort the protein structure. The default number of rotations for different tasks is provided in Appendix C.2.
0.85
Accuracy
80
0.80
75
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4 0.5
(a) Proportion of EC
(b)
Proportion of Reaction
Figure 6. Robustness analysis of models under varying proportion (0.1-0.5) of residues within protein structure. (a) EC, (b) RC
Recognizing that the resolution of protein data can present challenge and affect precision, it is crucial to validate the robustness of our approach. Therefore, we provide a specific robustness test designed to evaluate the performance of our method confronted with input protein data of varying resolutions. We follow the resolution classification proposed by Rupp (2009) and divide the test set into low-resolution (23 A) and high-resolution (<3 Å) structures for three tasks (we cannot retrieve resolution of data in Fold task). Results are displayed in Table 2. The superior performance of our proposed method on low-resolution structures indicates that the model does not overfit to fine-grained coordinate noise but instead learns invariant features critical for function prediction. This behavior demonstrates strong robustness and generalization across inputs of varying quality.
GOBP
GOCC
Reaction
FOLD
0.50
Accuracy (%)
90
0.48
85
0.46
80
0.44
0.42
Number of Rotations
Number of Rotations
Figure 5. Performance analysis under varying rotational strengths of oeheices and B-sheets across protein-related tasks.
5.4. Qualitative Evaluation
In this section, we analyze the robustness of our proposed method, the sensitivity of parameter, and the visualization of representation.
Table 2. Evaluation of robustness to input resolution across various protein function prediction tasks.
Resolution
EC
GO-BP
GO-MF
GO-CC
Reaction
Robustness Analysis. In practical application, the pro- tein structures of test data may undergo structural changes in response to environmental factors such as pH, temper- ature, or the presence of specific ions (Wang et al., 2008). This variability poses a challenge to the model's robustness against structural fluctuations during testing. To system- atically evaluate such robustness of our proposed method, we randomly select a proportion (10% to 50%) of residues within each protein sample in the test set and apply rota- tional transformations to these residues along with their connected segments, mimicking intrinsic structural fluc- tuations in proteins. Higher proportion of transformed residues represents more significant structural deviations from the initial structure. We compare our method with three strong baselines: Trad. 3D Aug. CDConv and CI-
Low High
0.903
0.485
0.648
0,504
0.921
0.888
0.465
0.687
0.463
0.894
Sensitivity Study. To evaluate the influence of weighting parameter À on model performance, we conducted a sensi- tivity analysis, the results of which are presented in Table 3. The results demonstrate that model performance exhibits only slight fluctuations across different values, with X=1 generally yielding superior performance on the majority of metrics. This low sensitivity to À indicates that the model is relatively robust to this hyperparameter, suggesting a straightforward optimization process. The complete results are presented in Table 10.
Visualization Representation Learning. We use UMAP
8

---

Ceaariance-nnhaneee Protein GCL
most of the training time is typically consumed by the GNN encoder, FCI demonstrates good scalability. For 3-PSI, its Table Performance of the proposed methad using various values for the hyperparameter À in the loss fünction. Optimal scores are highlighted bold.
preprocessing is performed once per protein structure to de- termine dihedral angles and secondary structures with time complexity of O(N,), where N, is the number of residues. During training, 3-PSI's 3D augmentation perturbs residues, also incurring an ON,, time complexity. The memory footprint for this augmentation scales as O(B N,). with B denoting the batch size. Despite the linear time complex- ity for augmentation, these operations benefit significantly from NumPy's s SIMD) vectorization, leading to fast practi- cal runtime. The resulting performance improvements are considered a justifiable trade-off for additional training time.
EC
GO-BP
GO-MF
GO-CC
0.2
0.882
0,451
0.656
0.467
Value
0.6
0.879
0.443
0.658
0.479
1.0
0.885
0.461
0.662
0.484
1.4
0,880
0,457
0.649
0,489
1.8
0.877
0.448
0.653
0.475
projection (McInnes et al., 2018) to visualize learned rep- resentations for the Reaction task, comparing our method with the best-perfomiing baseline (Trad.3D Aug.) to assess representation quality. To demonstrate results clearly, we randomly select 6 classes from Reaction task, each contain- ing more than 20 samples. As visualized in Figure 7, 3- PSIAIpha+FCI exhibits more distinct and compact clusters, with clearer boundaries, In contrast, Trad.3D Aug. shows more scattered distribution patterns with noticeable overlap between classes. Quantitatively, 3-PSIAIpha+FCI achieves both tighter intra-class clustering (distance: 19.90 vs 25.21) and better inter-class separation (distance: 43.33 vs indicating its superior protein representation learning.
5.6. Exploration on Ligand Binding Affinity
Since protein structure and functionality are preserved, this motivates an assessment of the model's performance on ligand binding affinity datasets. Here, we briefly explore our best setting model on ligand binding affinity task on PDBbind dataset (Liu et al., 2014). We select representa- tive baseline foloprot-Superpixx. (Somnath al., 2021) for comparison and results are represented at Table 4. Results demonstrate that our 3-PSIAlpha + FCI model achieves competitive performance. This superiority underscores the robustness and effectiveness of our approach in learning meaningful protein representations that can be successfully applied predict complex biomolecular interactions.
Table 4. Model performance comparison on ligand binding affinity task. Model performance comparison. Metrics are abbreviated as follows: R. (RMSE: lower is better), P. (Pearson: higher is better), and S. (Spearman: higher is better). The first three metrics are evaluated at 30% sequence identity, while the last three are at 60%.
Average Intra-class Distance: 25.21 Average Inter-class Distance: 42.11 Trad, 3D Aug.
Average Intra-class Distance: 19.90
Average Inter-class Distance: 43.33
3-PSIAlpha + FCI
Method
R.j.
P.
S.1
R.4
P.t
S.1
1.491
0.491
0.482
1.416
0.724
0.715
P-PSAAlpha + FCI
1.462
0.515
0.510
1.383
0.753
0.749
Figure 7. UMAP projection of learned protein representations with quantified intra-class and inter-class distances, comparing 3- 3-PSIAlpha+FCI and Trad.3D Aug. approaches.
6. Conclusion
In this work, address two key limitations in current GCL approaches for protein representation learning: the lack of biologically meaningful 2D topology- based augmentation and the lack of exploring in 3D protein structural graph augmentation. Specifically, we propose two biology-aware graph augmentation strategies from an invariance perspec- tive: 2D topology- basse Functional Community Invariance (FCI) and 3D Protein Structure Invariance (3-PSI), integrat- ing them into a unified GCL framework for protein represen- tation learning. Extensive experiments on four widely used protein benchmarks demonstrate that these biology-aware graph augmentation strategies consistently enhance perfor- mance across multiple protein prediction tasks, validating their effectiveness in protein representation learning.
5.5. Analysis on Complexity
Here we provide the scalability and complexity analysis of our proposed approach. For FCI, preprocessing centers requires computing probability matrix once, which guides edge perturbations during training. This involves spectral de- composition using the Lanczos algorithm, which efficiently reduces the complexity from O(N3) to ON, where N, is the average number of residues per graph and K is the number of selected eigenvalues. With this matrix precom- puted, FCT's augmentation during training (which samples edges based on this matrix) can achieve a highly efficient O(1) time complexity. The augmentation operations are accelerated by NumPy's SIMD vectorization. Given that

---

Protein GCL
Impact Statement
ment of protein folds. Bioinformatics, 34:4046-4053
2018. URL https: apii. swwanti cachona.. htg//ooppu1D:21554220
This work advances protein representation learning by intro- ducing biologically-awant graph augmentation methods that preserve important structural and functional properties of proteins. We elaborate on the broader impacts of our work from the following two aspects: (1) Better preservation of protein integrity during graph augmentation, reducing the risk of learning biologically implausible representations: (2) A framework that bridges topology-basee based and structure- based protein analysis, potentially enabling new insights into protein related tasks. This paper presents work whose goal is to advance the field of Machine Learning. There many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Derewenda, Z., Lee, L., and Derewenda, U. The oc- currence of c-h...o hydrogen bonds in proteins. Jour- nal of molecular biology, 252 2:248-66 1995. doi: 10. 10.1006/JMBI.J 1995.0992.
Doncheva, N., Klein, K., Domingues, S., and Albrecht, M. Analyzing and visualizing residue networks protein structures. Trends in biochemical sciences, 36 4:179-82, 2011. doi: 10.1016/f.ibb.2201 201.00.00......
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., BHOWMIK, D., and Rost, B. Prottrans: Towards cracking the language life's code through self-supervised deep learning and high performance com- puting. bioRxiv, 2020a. doi: 10. 121.22022..0.2219995555
Acknowledgments
This work was supported part by Guangdong S&T Pro- gram (00098900))))02) and National Natural Science Foundation of China (NSFC) under Grant No.62204140 We gratefully acknowledge all current related works in this field, which were crucial for completing our research. We also thank Keyu Mao for the discussions on structure re- trieval approaches.
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang. Y.. Jones, L., Gibbs, T., Fehér, T. B., Angerer, C., Steinegger, M., Bhowmik, D., and Rost, B. Prottrans: Towards cracking the lan- guage of life's code through self-supervised deep learning and high performance computing. bioRxiv, 2020b. URL https: / /api. semanticscho org/ rtg//orrpuID:222495555
References
Asgari, and Mofrad, M. R. K. Continuous dis- tributed representation of biological sequences for deep proteomies and genomics. PLoS ONE, 10, 2015. URL https:ttt//c/c...tcc.tt http://ppp.emaatisscccceaee htt//orrpuSDD:2500000
Eswar, N., Webb, B. M., Martí-Renom, M. A., Mad- husudhan, M. S., Eramian, D., Shen, M.-Y., Pieper, U., and Sali, A. Comparative protein structure modeling using modeller. Current Protocols in Bioinformatics, 15, 2006. URI https: /app:///ppia// semant insaantcccchr ccschola lrr. org/CorrpsID::5555
Baldassarre, F., Hurtado, D. M. Elofsson, A., and Azizpour, H. Graphqa: protein model quality assessment using graph convolutional networks. Bioinformatics, 37:360 - 366, 2020. URL https: //api. www.ssieeeannnnnneeeee www.ccccow.......rrrroooooo CorgCsir: 210848493.
Fan, H., Wang, Z., Yang, Y., and Kankanhalli, M. Continuous-discres convolution for geometry- sequence modeling in proteins. In The Eleventh In- ternational Conference Learning Representations, 2023. URL https: /tL.ttts:://ppeeee//// /openrevie net /formm?rm id-P52- 219XJ7.
Berkholz, D., Shapovalov, M., Dunbrack, R. L., and Karplus, P. Conformation dependence of backbone geometry in proteins. Structure, 17 10:1316-25, 2009. doi: 10. 1016// str.2009.08.012.
Feng, S., Ni, Y., Li, M., Huang, Y., Ma, Z.-M., Ma, W.-Y., and Lan, Y. Unicorn: a unified contrastive learning ap- proach for multi-vien molecular representation learning. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024.
Betancourt, M. and Skolnick, J. Local propensities and sta- tistical potentials of backbone dihedral angles in proteins. Journal of molecular biology, 342 2:635 49, 2004. doi: 1MB.2004.0060911.
Gao, B., Qiang, B., Tan, H., Jia, Y., Ren, M., Lu, M., Liu, J., Ma, W.-.., and Lan, Y. Drugelip: Contra- sive protein-molecule representation learning for virtual screening. In Neural Information Processing Systems, 2023. URL https: https::/ppp.ssmmmtttissss.. /tt///CopuSD::2680000 www.stiimman.o.o.....e..
Chothia, C. and Lesk, A. M. The relation between the divergence of sequence and structure in proteins. The EMBO journal, 5(4):823-226 1986.
Derevyanko, G., Grudinin. S., Bengio, Y., and Lamoureux, G. Deep convolutional networks for quality assess-
10

---

Anwariance-Enhande Protein GCL
Gene, G., Van Loan, C. F., and al. Matrix compulations.
2017. URL https: //api semant csshollrr (ts/CooppuTT::491455033
JHU press, 2013.
Glaser, F, Steinberg, D., Vakser, I., and Ben-Tal, N. Residue frequencies and pairing preferences at pro-
Hou, J., Adhikari, B., and Cheng, J. Deepsf: Deep convolutional neural network for mapping protein se- quences to folds. In Proceedings of the 2018 ACM International Conference on Bioinformatics, Compula- tional Biology, and Health Informatics, BCB '18, pp. 565, New York, NY, USA, 2018. Association for Com- puting Machinery. ISBN 9781450357944. doi: 10. 1145/323354732233166 URL https: htt//ooooorrgo 10. D0.1145/3233544.7 32337771...2
tein-protein interfaces. Proteins: Structure, 43, 2001. doi: 1010021107700144200000053333333 AID--PROT102133..0000211
Gligorijevié, V., Renfrew, P. D., Kosciólek, T., Leman, J. K., Berenberg, D., Vatanen, T., Chandler, C., Tay- lor, B. C., Fisk, I. Vlamakis, H., Xavier, R. J., Knight, R., Cho, K., and Bonneau, R. Structure- based protein function prediction using graph con- volutional networks. Nature Communications, 12, 2021. URL https://apl: https//app.smmmtiiccooo. semanticschola.. org/CorpusIDI 235218243
Huang, Y., Li, S., Su, J., Wu, L.., Zhang, O., Lin, H., Qi, J., Liu, Z. Gao, Z., Liu, Y., Zheng, J., and Li, S. Z. Protein 3d graph structure learn- ing for robust structure-based protein property predic- tion. In AAAI Conference on Artificial Intelligence, 2023. URL Stttp:://ap) org/ /tt////ot:::2/2295000
Goh, C., Milburn, D., and Gerstein, M. Conformational
inmmanticschoo
changes associated with protein- pootein-protein interactions. Cur- rent opinion in structural biology, 14 1:104-9, 2004. doi: 10.1016/..BB..0044 2205.00.0.0....
Hus, J.-C., Salmon, L., Bouvignics, G., Lotze, J., Blackledge, M., and Brüschweiler, R. 16-fold de- generacy of peptide plane orientations from residual dipolar couplings: analytical treatment and implica- tions for protein structure determination. Journal of the American Chemical Sociery, 130 47:15927-37, 2008. URL httph:tt///// /tppp/p..../ semant cscholar. org/ /tt///oo:SSS:::5555..
Guan, X. and Zhang, D. T-mgel: Molecule graph contrastive learning based on transformer for molecular property prediction. IEEE/ACM Transactions on Computational Biology and Bioinformaties, 20(6))3851-338 -3665,, 2023. doi: 10.1 N1110/TCBB.2003333222
Guy, H. R. Amino acid side-chain partition energies and distribution of residues in soluble proteins. Biophysi- cal journal, 47 1:61-70, 1985. URL https: //api. semanticschon wtt.oorg////ooooooo : 42820515.
Jamasb, A. R., Morehead, A., Joshi, C. K., Zhang, Z., Didi, K., Mathis, S. V., Harris, C., Tang, J., Cheng, J., Lio, P., and Blundell, T. L. Evaluating representation learn- ing on the protein structure universe. In The Twelftt International Conference on Learning Representations, 2024. URL https: http:///pppeeeeeeeeeeeeeee net id=sTYuRVrdK3.
Heinemann, P. M., Armbruster, D., and Hauer, B. Active- site loop variations adjust activity and selectivity of the cumene dioxygenase. Nature Communications, 12, 2021. URL https: http:://app..emmatttcccooool www.ntisschoooee.........ccco 231952335
net//orunt forum?
Janin, J., Wodak, S., Levitt, M., and Maigret, B. Con- formation amino acid side-chains proteins. Jour- nal of molecular biology, 125 3:357-86, 1978. doi: 10. (00006000228866))90000000
Hermosilla, P. and Ropinski, T. Contrastive representation
learning for 3d protein structures. ArXiv, 0itp/2205.15675 2022. URL. https: /aap: seninteemntt.. cccholar. htg//oorpuuID::4921000
Jha, A. K., Colubri, A., Zaman, M., Koide, S., Sosnick, T., and Freed, K. Helix, sheet, and polyproline ii frequencies and strong nearest neighbor effects in restricted coil library. Biochemistry, 44 28:9691-703 2005. doi: 10. 1021/B10474822.
Hermosilla, P., Schäfer, M., Lang, M., Fackelmann, G., Vázquez, P.-P., Kozlikova, B., Krone, M., Ritschel, T., and Ropinski, T. Intrinsic-extrinsi convolution and pooling for learning on 3d protein structures. In In- ternational Conference on Learning Representations, fermm
Jing, B., Eismann, S., Suriana, P., Townshend, R. J. L., and Dror, R. Learning from protein structure with ge- ometric vector perceptrons. In International Confer- ence on Learning Representations, 2021. URL https: / //openreview.com forummooummom http://tetttooout.//.oo.oSSSSS.
2021. URL id=1 id=10mSURCpwY.
Hogben, Handbook of linear algebra. CRC press, 2013.
Hou, J., Adhikari, B., and Cheng, J. Deepsf: deep convolutional neural network for mapping protein se- quences to folds. Bioinformatics, 34:1295 1303,
Jones, D. C. Protein secondary structure prediction based on position-specine scoring matrices. Journal of molecular

---

Invariance--nhanned Protein GCL
biology, 292 2:195-202, 1999. doi: 10. 10.1006/1MB111999, 3091.
Liu, Z., Li, Y., Han, L.,, Li, Liu, J., Zhao, Z., Nie. W., Liu, Y., and Wang, R. Pdb-wide collection of binding data: current status of the pdbbind database. Bioinformatics, 31(3):405-41, 10 2014. ISSN 1367-4803. doi: 10.1093/ bioinformatics/ttu66.. URL https:///://// hu93//ioinformaticc//oooo //ooi. org/10.
Kabsch, W. and Sander, C. A. Dictionary of protein secondary structure: Pattern recognition of hydrogen- bonded and geometrical features. Biopolymers, 22, 1983. URL https: /ttp://ppi.aat org/CoorpusD::2918557 sww..i.setttticcooo..io..
Liwo, A., Pincus, M., Wawak, R., Rackovsky, S., and Scheraga, H. Calculation of protein backbone geom- etry from alpha-carbon coordinates based on peptide- group dipole alignment. Protein Science, 2, 1993. doi: 10. 11002/PRO.556002000
Kulmanov, M. and Hoehndorf, R. Deepgoplus: im- proved protein function prediction from sequence. Bioinformatics, 36:422 429, 2019. URL https: //api.
146065857.
Maier, J., Martinez, C., Kasavajhala, K., Wickstrom, L., Hauser, K., and Simmerling, C. ff14sb: Improving the accuracy of protein side chain and backbone parameters from f199sb. Journal of chemical theory and computation, 11 8:3696 2015. doi: 10. 10.102//acsjectc550022..
Li, S., Zhou, J., Xu, T., Dou, D., and Xiong. H. Geomgel: Geometric graph contrastive learning for molecular prop- erty prediction. In Proceedings of the Thirty-Six AAAI Conference on Artificial Intelligence, pp. 2022.
McInnes, L., Healy, J., Saul, N., and Großberger, L. Umap: Uniform manifold approximation and projection. J. Open Source Softw, 3:861, 2018. URL httpt://///// /api. httg//orrpusDD. 53244422232
Liao, Y.-L., Wood, B., Das A., and Smidt*, T. EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degre Representations. In Interna- tional Conference on Learning Representations (ICLR), 2024. URL https::////. http//pppereeeeeeeeeeeeeeeee net. forum? 1d=mCO8k2mrZD.
S semantis www.nniccccolarr..
Mehta, I. D. and Beck, B. Modeling proteins residue interaction networks to understand stracture-function relationship. Biophysical Journal, 106, 2014. doi: B..01//..BI.201...11
Lin, L., Chen, J., and Wang, H. Spectral augmentation for self-supervised learning on graphs. In The Eleventh International Conference on Learning Representations, www.rrewwwwwwwweeeeeeeeeeeeeeeee / forum?
Meng, E. C., Goddard, T. D., Pettersen, E. F., Couch, G. S., Pearson, Z. J., Morris, J. H., and Ferrin, T. E. UCSF ChimeraX: Tools for structure building and analysis. Pro-
2023a. URL https: http///wwwwe...ec/////// id-DjzBCrMBJJ
tein Science, 32(1 (el1))))4722, 2023.
Lin, Z. Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., dos Santos Costa, A., Faral-Zarandi, M., Sercu, T., Can- dido, S., and Rives, A. Evolutionary-scale prediction of atomic-level protein structure with language model. Science, 379(6637): 123 1130, 2023b. doi: 10.1 126/ science.ade2574. URL
Mishra, P. and Jha, S. K. The native state conformational heterogeneity in the energy landscape of protein folding. Biophysical Chemistry, 283: 2022.
Murzin, A. G., Brenner, S. E., Hubbard, T., and
Chothia, C. Scop: A structural classification of proteins database for the investigation of sequences and structures. Journal of Molecular Biology. 247(4):536 540, 1995. ISSN 0022-2836. doi:
org/doi/abo / 10.
ade2574.
Liu, N., Wang. X., Bo, D., Shi, C., and Pei, J. Revisiting graph contrastive learning from the perspective of graph spectrum. In Proceedings of the 36th International Con- ference on Neural Information Processing Systems, NIPS "2,, Red Hook, NY, USA, 2022a. Curran Associates Inc. ISBN 97817138777 1188810088
http://k...og///0000000002222222 httt::::/wwwwscceeceeeeee URL (tleene//rttcole///////222228880
Nisius, L.. and Grzesiek, S. Key stabilizing elements of protein structure identified through pressure and tempera- ture perturbation of its hydrogen bond network. Nature chemistry, 4 9:71 2012. doi: 10. 1.1108/nchem.113 1336.
Liu, S., Wang, H., Liu, W., Lasenby, J., Guo, H., and Tang, J. Pre-training molecular graph representation with 3d geometry. In International Conference on Learning Rep- resentations, 2022b. URL https: openreview. net/ forum? d=xQUe1 1
Pothen, A., Simon, H. D., and Liou, K.-P. Partitioning sparse matrices with eigenvectors of graphs. SIAM journal on matrix analysis and applications, (1(3):330-43 1990.
12

---

Invariance-Enhaneed Protein GCL
Quan, R., Wang, W., Ma, E,, Fan, H., and Yang.
volume 34, pp. 25244-25255. Curran Associates, Inc.,
Y. Clustering for protein representation learn-
2021. URL httpt:ttt///ccccc wttttss:/pproceeedeeeet .ne
ing. 2024 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pp. 319-329, 2024. URL. https: wwt.ssstmatttcttooaioiiii.i. org/CozpusID: 20801:26888888 9344.
cc cc/ aB4220ffeccllee10ec7773333333333 pdf.
Stanger, H. E., Syud, F. A., Espinosa, J. F,, Giriat, I., Muir, T. W., and Gellman, H. Length-dependent stability and strand length limits in antiparallel f-sheet secondary structure. Proceedings of the National Academy of Sci- ences of the United States of America, 98:12015 12020, 2001. URL https: /ppi. wew.emiiicccoo........ org Ct:ppssD::1504499000
Ramachandran, G. N., Ramakrishnan, C., and Sasisekha- ran, V. Stereochemistry of polypeptide chain con- figurations. Journal of molecular biology, 7:95-9, 1963. URL https: semattematt.ppee...eaaaeeeeee..e icscholar. org/CozpusID: 39001118
Rives, A., Goyal, S., Meier, J., Guo, D., Ott, M., Zitnick, C. Ma,, J., and Fergus, R. Biological structure and func- tion emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences of the United States of America, 118, 2019. URL https:/ http://aap....aatiie.ees www.etiiiccooooeee....e. org/Cozoe hetp/orppSID:::00 CCo-puSID:1551223:.. 62335.
Sticke, D. F., Presta, L.. Dill, K., and Rose, G. Hydrogen bonding in globular proteins. Journal of molecular biol- ogy, 226 1111-455, 1992. doi: 10.1016/0022-28880 2836((22) 91058-W W.
Suresh, S., Li, P., Hao, C., and Neville, J. Adversar- ial graph augmentation
improve graph contrastive learning. In Neural Information Processing Systems, 2021. URL https: / /app..pppp////// www.nticscoolle......o.. org/ / CorpuSID:23539100
Rost, B. Review: protein secondary structure prediction continues to rise. Journal of structural biology, 134 2-3: 204-18, 2001.
doi: 10. 10.1006/1SBII20)) 4336.
Tan, S., Li, D., Jiang, R., Zhang, Y., and Okumura, M. Community-invarian graph contrastive learning. In Forty-first International Conference on Machine Learn- ing. ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.ne., 2024. WRRLhttps://opeeeiiiiee pett/forumidd-dsk.gee
Roterman, The geometrical analysis of peptide backbone structure and its local deformations. Biochimie, 77 3:204- 16, 1995, URL https: wtt.ppisssmmaannscccsaeoeeeoeoee org/CorpusID 2 23130770.
Rupp, B.
Biomolecular crystallography: Princi- ples. practice, and application to structural biology. 2009, URL https: //api. http:///app..aaaaaaaaeeeemmaaaaa www.ssiscchhosooeee.... org/CorpusID: 92657569.
Tsubaki, M., Tomii, K., and Sese, J. Compound- protein interaction prediction with end-to-end learning of neural networks for graphs and sequences. Bioin- formatics, 35:309-318, 2018. URL https://api. semanticscholar org/ http:/ooopII::::100022222
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-80, 2009. doi: 200.0
van Kempen, M., Kim, S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, L., Söding, J., and Steineg- ger, M. Fast and accurate protein structure search with foldseek. Nature Biotechnology, 42:243 246, 2022. URL https://aap www.nticscnooooeeo.o.oo org/ /CCC/oosSD:
Schulze, J. O., Saladino, G., Busschots, K., Neimanis, S., Süss, E., Odadzic, D., Zeuzem, S., Hindié, V., Her- brand, A. K., Lisa, M.-N., Alzari, P. M., Gervasio, F. L., and Biondi, R. M. Bidirectional allosteric communi- cation between the atp-binding site and the regulatory pif pocket in pdk1 protein kinase. Cell chemical biol- ogy, 23 10:1 1935, 2016. URL https::/ /aapi www.:oaarrrrrrrooooooooeeoeeeeee wwwprr////ooa:aaaoooooo 12656411222
Wang, D., Pourmirzaei, M., Abbas, U. L., Zeng, S., Man- shour, N., Esmaili, F,, Poudel, B., Jiang, Y., Shao, Q. Chen, J., and Xu, D. S-plm: Structure-awart protein language model via contrastive learning be- tween sequence and structure. Advanced Science, 12, 2023a. URL https: //api. semanticscholes org http/CoopuIID:260733555
Shi, and Malik, J. Normalized cuts and image segmen- tation. In Conference on Computer Vision and Pattern Recognition, pp. 731-737. IEEE Computer Society, 1997.
Wang, L.,, Liu, H., Liu, Y., Kurtin, J., and Ji, S. Learn- ing hierarchical protein representations via complete 3d graph networks. In The Eleventh International Confer- ence on Learning Representations, 2023b. URL https: / //opnreviees eww net forum? d= 9XLkk.
Somnath, V. R., Bunne, C., and Krause, A. Multi-scale rep- resentation learning on proteins. In Ranzato, M., Beygelz- imer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems,
13

---

Eivariance-Enhance Protein GCL
Wang, Q. Shui, B., Kotlikoff, M. I,, and Sondermann, H. Structural basis for calcium sensing by geamp2. Structure, 16 12:1817-27, 2008. URL https: //api. www.stteeannttnnaeeeeee 205258893.
Wood, M. J. and Hirst, J. Protein secondary structure predic- tion with dihedral angles. Proteins: Structure, 59, 2005. doi: 10. 00.1002/pot..24455
org/CorpusID:
Yan, A. and Jernigan, R. L. How do side chains orient globally in protein structures? Proteins: Structure, 61, 2005. URL htwp:: org/CorpusID: :022::56:::222225522 20224659.
Wang. Y., Wang. J., Cao, Z.,, and Barati Farimani, A. Molec- ular contrastive learning of representations via graph neu- ral networks. Nature Machine Intelligence, 4(3):279-277, 2022a.
Yang, K. K., Wu, Z., Bedbrook, C. N., Arnold, F. H., and Wren. J. Leamed protein embeddings for machine learning. Bioinformatics. 34:2642-2648, 2018. URL https: /tttps::/ane //apii www.sticscaoolle.....o.o Prg//oorpusID:26347550
Wang, Y., Wang. J., Cao. Z., and Farimani, A. B. Mol- clr: Molecular contrastive learning of representations via graph neural networks. nature machine intelligence, 2022b.
Yin, Y., Wang, Q. Huang, S., Xiong, H., and Zhang, X. Autogel: Automated graph contrastive learning via learnable view generators. In AAAI Confer- ence on Artificial Intelligence, 2021. URL https: aepp.ssemaniccccollaae org wowp///orrrriiir 237581111.
Wang, Y., Fang, X., Yin, H., Li, D., Li, G., Xu, Q.. Xu, Y., Zhong, S., and Xu, M. Big-fusion: Brain-inspired global-local context fusion framework for multimodal emotion recognition in conversations. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 39, pp. 1574-1582, 2025.
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., and Shen, Y. Graph contrastive learning with augmen- tations. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, vol- ume 33, pp. 5812-5823, Curran Associates, Inc., 2020. URL https: https::/pproeeed.ngg cc cc/pspper_iilee/ppprr///////// http:///pprooeeddnneennee.....s neurips. R pdf.
Wang, Z., Combs, S. A., Brand, R., Rebollar, M. C., Xu, P. Price, G. D., Golovach, N., Salawu, E. O., Wise, P., and Clark, P. M. Lm-gvp: A generalizable deep learning framework for protein prop- erty prediction from sequence and structure. bioRxiv, 2021. URL https:/ aapaapiii.. saapiissmmaticschho..oo TTg//oorpuTD::2333 0356.
C., Ponnapalli,
Waterhouse, A. M., Bertoni, M., Bienert, S., Studer, G., Tauriello, G., Gumienny, R., Heer, F. T., de Beer, T. A. P., Rempfer, C., Bordoli, I.., Lepore, R., and Schwede, T. SWISS-MODEL: homology modelling of protein structures and complexes. Nucleic Acids Re- search, 46:W296 W303, 2018. URL httpps///pppp semanticschole www.laarrrrr/oooeee ortp:///o////1110400000000
You, Y., Chen, T. Shen, Y., and Wang. Z. Graph contrastive learning automated. In International Conference on Machine Learning, 2021. URL https: //api. seapi..smmnttcschhee 235422690.
orq/CorpusID:
Zhang, Z., Xu, M., Jamasb, A. R., Chenthamarakshan, V., Lozano, A., Das, P., and Tang, J. Protein representa- tion learning by geometric structure pretraining. In The Eleventh International Conference on Learning Represen- tations, 2023. URL https: / openreview / net/ forum ? i d= 30CCBt3tg
Webb, E. C. Enzyme nomenclature 1992. recommen- dations of the nomenclature committee of the inter- national union of biochemistry and molecular biology on the nomenclature and classification of enzymes. 1992. URL https: /atp//appi.p// http::s:/ap....mmaatttaee.. semanticscholar....... Crg/CorppsID: 83964422.
Zhou, G., Gao, Z., Ding, Q., Zheng, H., Xu, H., Wei, Z., Zhang, L., and Kc, G. Uni-mol: A universal 3d molecular representation learning framework. In In- lernational Conference on Learning Representations, 2023. URL /ttps::/api..... sewanticscholl org (oopcoaI:
Wei, C., Wang, Y., Bai, B., Ni, K., Brady, D., and Fang, L. Boosting graph contrastive learning via graph con- trastive saliency. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Pro- ceedings of the 40th International Conference on Ma- chine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 36839-366666 36855. PMLR, 23-29 Jul 2023. URL https: http///ceednnngge mlr. press// v2.02/ w 12c html.
Zhu, Y., Xu, Y., Yu, F,, Liu, Q. Wu, S., and Wang. L. Deep Graph Contrastive Representation Learning In ICML Workshop on Graph Representation Learning and Be- yond, 2020. URL http: / aarii.iii.. www.iv.rrg///..oo. 04131.
wttp/br/g///o/ooooo/oes/o//////e abs/2006.
14

---

Invariance-Enhaneed Protein GCL
Zhuo, L., Chi, Z., Xu, M., Huang, H., Zhao, J., Zheng, H., He, C., Mao, X.-I.., and Zhang, W. ProtLLM: An interleaved protein-language LLM with protein-as-wont pre-training. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 8950-8963, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.186653/112224.allloloogg 484. URL https:// aclanthology otg/2044...../0000... acl acl-long....o..o..oo.oooooo..... 484.
15

---

Envariance-Enhaneed Protein GCL
A. Augmentation with Homology Modeling
Homology modeling a computational technique that predicts protein 3D structures by identifying and leveraging structural templates from homologous proteins with known conformations (Chothia & Lesk, 1986). For our implementation, we employ SWISS-MODEL (Waterhouse et al., 2018) for homology modeling and visualize the results using ChimeraX (Meng al., 2023). The effectiveness of homology modeling varies significantly with template availability. In the upper panel of Figure 8 ("Matched' case), we demonstrate cases where high-identity templates exist (demonstrated with Protein IDs: ADD6 and 9PCY). Here, we can compare multiple predicted structures against their known original structures, revealing high structural similarity regardless of which template is used. This consistency, while validating the prediction accuracy, also indicates limited structural diversity in the augmented outcomes. The lower panel illustrates a more challenging scenario ("Ummatched" case) using artificially constructed protein sequences that simulate novel proteins without known structures. For each sequence, we designate the prediction from the highest-identity template as reference structure for comparison. Due to the lack of high- high-identity templates, these predictions, substantial variations in both local secondary structures and global folds, demonstrating the inherent uncertainty in structure prediction when suitable templates are scarce. It also raises concerns about whether such structural uncertainty would impact the reliability and effectiveness of graph augmentation.
when compared to their respective references, exhibit
Matched
Original (1AD6)
Predicted (2r7g.1.A)
Predicted (3pom.1.A)
Predicted (P06400.1.A)
Original (9PCY)
Predicted (P00292.1.A)
Predicted (1ag6.1.A)
Predicted (1byp.1.A)
Unmatched
Reference (7rgs.1.A)
Predicted (2dlx.1.A)
Predicted (3m1c.1.B)
Predicted (1jj7.1.A)
Reference (5ell.1.A)
Predicted (2ag8.1.5 1.B)
Predicted (3hrk.1.B)
Predicted (7pli.2.C)
Figure 8. Structural variations in SWISS-MODEL predictions. Upper panel: Known proteins (LAD6 and 9PCY) serve as references to evaluate multiple predictions using different templates (template IDs parentheses), showing high consistency across predictions. Lower panel: For each artificial sequence, the prediction using the highest-identity template is labeled as reference, with subsequent predictions demonstrating structural diversity arising from template scarcity.
16

---

Invariance-Enhaneedd Protein GCL
B. Key Terms Related to Protein
Amino Acid Residue: An amino acid residue refers to a single amino acid unit within polypeptide chain, formed after the loss of water during peptide bond formation. Residues are the basic units of protein sequences and structures, playing a key role in all four tasks: FOLD, Reaction, GO, and EC, as their properties and arrangements determine protein function and classification (Glaser al., 2001; Doncheva et al., 2011).
Dihedral Angle: A dihedral angle (or torsion angle) describes the rotation around a bond in protein backbone, such the phi (6) and psi (v²) angies. These angles define the spatial arrangement of the polypeptide chain. Dihedral angles determine protein folding patterns and are crucial for predicting protein structure in tasks like FOLD and GO (Betancourt & Skolnick, 2004; Wood & Hirst, 2005).
Hydrogen Bond: A hydrogen bond is a weak electrostatic interaction between a hydrogen atom (bound to an electronegative atom like nitrogen or oxygen) and another electronegative atom. In proteins, hydrogen bonds stabilize secondary structures like a-helices and B-shects. Hydrogen bonds are essential for maintaining protein structure and function, directly impacting tasks like FOLD and Reaction (Sticke et al., 1992; Derewenda et al., 1995).
Peptide Plane: The peptide plane refers to the rigid, planar structure formed by the peptide bond (C-N) and its adjacent atoms (Co. C, N, and O) in a protein backbone. This planar structure is a fundamental unit of protein conformation. The peptide plane's 's rigidity influences protein folding and secondary structure formation, which are critical for tasks like FOLD and GO (Roterman, 1995; Hus et al., 2008).
Protein conformation refers to the three-dimensional shape or spatial arrangement of a protein, determined by the rotation of bonds and interactions between amino acids. Protein conformation is critical for all four tasks (FOLD, Reaction, GO, and EC) as it determines structural, functional, and interaction properties of proteins (Goh et al., 2004; Berkholz et al., 2009).
Protein Conformation:
The secondary structure of a protein refers local, repetitive conformations of the polypeptide chain, primarily a-helices and B-sheets, stabilized by hydrogen bonds. Secondary structures are the building blocks of protein folds and are directly relevant to tasks like FOLD and EC, they influence enzyme active sites and catalytic mechanisms (Jones, 1999; Rost, 2001).
Secondary Structure:
Side Chain: The side chain (or R-group) is the variable part of an amino acid that extends from the backbone. Side chains determine the chemical properties and interactions of amino acids. Side chains are critical for protein function, including enzyme catalysis and binding, making them essential for tasks like Reaction and EC (Janin et al., 1978; Maier et al., 2015).
C. Details of Implementation
C.1. Graph Construction
The node feature is derived from the Torch Embedding function 3 of residue types. Additionally, we incorporate the side chain embeddings Vsa for each residue to enrich biological information in GNN representations. To achieve rotational invariance, we adopt the relative spatial encoding scheme from Fan et al. (2023) to construct edge features. For edge features between nodes and j. we combine sequence information from the original dataset with three spatial components: the normalized relative position vector, its projection in the local coordinate system, and the pairwise distance, ensuring protein structure representation remains invariant under global rotations.
C.2. Details of Training Setup
Our approach is implemented using the PyTorch framework 1.0 and the PyTorch-Geometrc library 2.6.0. The CUDA version is 12.4. It is trained on RTX 3090 GPUs. We utilize the SGD optimizer with a momentum of 0.9, learning rate of 1e-2, and weight decay of 5e-4. The initial graph construction radius is set to 4. All results are averaged over 5 runs with different random seeds. The training batch sizes for each task are follows: FOLD (16), Reaction (32), GOCC (32), GOMF (24), GOBP (24), and EC (32). We simply set the weight of GCL loss to lambda=1 in our objective function throughout our experiments. For the augmentation strength of FCI, we set e = 0.2 for GO, EC, and Reaction, and e = 0.3 for FOLD. For the augmentation strength of 3-PSIAIpha, we set 2 rotations for EC, GOBP, and FOLD, and 3 rotations for GOCC and
hhtp:://pppcccoooooooo///seeeeee http:////eeee.aoeccooooooooccc.a Embedding..tnt
17

---

Invariance-Enhancee Protein GCL
Reaction. For 3-PSIDiag, we perturb all dihedral angles.
C.3. Details of Skew-symmetric Matrix
Here the detail of the skew-symmetric matrix a x in Sec 4.2:
0
n2
fly
0
-y 0
C.4. Details of The Backbone
We implement an Edge-aware Graph Convolutional Network (EdgeGCN) our GNN encoder. It incorporates edge features into the message-passing framework. The encoder consists of I. stacked EdgeGCN layers, where each layer performs the following operations:
(1+1) h
Σ h W..y JEN(S)
(12)
where ()) denotes the node features of node at layer I, (()) represents the neighborhood of node i, eij is the edge feature vector between nodes and j. and We is a learnable edge transformation matrix. The operator @ denotes element-wise multiplication.
We introduce structure-aware graph pooling mechanism from (Fan al., 2023) that hierarchically reduces graph resolution while preserving critical structural information in protein graphs. The pooling operation is guided by sequence indices to ensure biologically meaningful node clustering. The pooling process first establishes clustering assignments by halving sequence indices, effectively grouping adjacent nodes in the protein chain. Within each formed cluster, the pooling mechanism aggregates different node attributes through specialized operations: node features and 3D coordinates are averaged to maintain spatial information, orientation vectors are averaged and re-normalized to preserve valid directional information, and sequence information is preserved through maximum pooling. This pooling strategy reduces the graph resolution by approximately half each level while preserving the essential structural relationships, enabling the network build hierarchical representations multiple scales.
D. Details of Datasets, Evaluation, and Baselines
D.1. Details of Tasks and Datasets
We use the same dataset settings as the previous work (Fan et al., 2023), and the details are shown as follows:
Protein Fold Classification. Protein fold classification crucial for understanding the relationship between protein structure and evolution. Fold classes capture secondary structure compositions, orientations, and connection orders. Following Hermosilla al. (2021), conduct fold classification using the SCOPe 1.75 dataset (Hou al., 2018), comprising 16.712 proteins across 1,195 fold classes. The 3D coordinates derived from the SCOPe 1.75 database (Murzin et al., 1995). The dataset provides three evaluation scenarios: Fold: Excludes proteins from the same superfamily during training; Superfamily: Excludes proteins from the same family during training; Family: Includes proteins from the same family during training. Mean accuracy is used as the evaluation metric.
Enzyme Reaction Classification. This task involves classifying enzyme-catalyzed reactions based on all four levels of the Enzyme Commission (EC) number (Webb, 1992), representing protein function classification problem. We utilize the dataset by Hermosilla al. (2021), which includes 384 four-level Enzyme Commission classes and comprises 29,215/2.562/5.66 proteins for training/validationnless respectively. Mean accuracy is the evaluation metric.
Gene Ontology Term Prediction. This multi-label classification task predicts protein functions through Gene Ontology terms, organized into three hierarchical ontologies: biological process (BP, 1,943 classes), molecular function (MF, 489 classes), and cellular component (CC, 320 classes). Using the dataset from (Gligorijević et al., 2021), we train/valliate/test on 29, 00008898/333224444 415 proteins, respectively. The Fmax metric (Fan et al., 2023) is used for evaluation.
18

---

Protein GCL
Enzyme Commission Number Prediction. This multi-label task predicts three-level and four-level EC numbers across 538 classes. Using the dataset from Gligorijević et al. (2021), train/validateftest 15,550/1,729/1, ,919 proteins, respectively. The Fmoz metric is applied for evaluation. For GO term and EC number prediction, we adhere the multi-cutoff splits from Gligorijevćć et al. (2021), ensuring the test set only includes PDB chains with sequence identity I 95% to the training set, consistent with Zhang et al. (2023); Fan et al. (2023).
D.2. Details of Evaluation Matrix Fmaz
Multi-label classification can be interpreted as set of binary classification tasks. The protein-centric maximum F-Score (Fman), as defined by Gligorijević et al. (2021), is used to evaluate the accuracy of multi-label classification. Given a decision threshold E [0, 1]. let pi denote the predicted probability for the j-th class of the i-th protein, by {0, 1] be the corresponding binary class label, and J be the total number of classes. The precision and recall for the i-th protein are calculated follows, following the approach described in (Fan et al., 2023):
Cy 2 X) 2) Σj=( (of 2 A)
Ej= (rf 2)
precision,(A)
recall,
Σj=1 1/
The average precision and recall across all proteins are then defined as:
precision, (A)
recall(A) En (Acall;(A)
precision(A)
شز= (r) A)) 21
Li=1
Finally, Fmax is calculated as the maximum F-Score across all thresholds [0, 1]:
2 precision(A) x recall(A) )
Fmmxx max X
precision(A) + recall(A)
D.3. Details of Baselines
For comprehensive performance evaluation, we compare our approach with the following SOTA baselines, categorized into protein-specific methods, 2D topology-based GCL methods and 3D structure-based GCL:
Protein-specific methods:
GVP (Jing et al., 2021) specifically handles scalar and geometric vector features in protein structures, incorporating geometric relationships between amino acids. The method introduces a novel neural network layer that maintains rotational and translational equivariance through specialized vector operations.
GraphQA (Baldassarre al., 2020) focuses on protein structure quality assessment using graph- representations of proteins. It uniquely integrates both local and global structural features through hierarchical graph representation.
3DCNN (Derevyanko et al., 2018) uses 3D convolutional neural networks to analyze protein structures directly in their dire-dimmnsional form. The method employs a voxelization strategy to convert continuous protein structures into discrete 3D grids.
IBConv (Hermosilla Ropinski, 2022) uses contrastive learning with a specific focus on maintaining geometric invariance and equivariance in protein structure analysis. introduces specialized convolution operations that preserve geometric symmetries while processing protein structures.
GearNet (Zhang et al., 2023) is specifically designed for protein structure analysis incorporating geometric information. It features multi-scale pessage-passing mechanism that captures both short-range and long-range protein interactions.
ProNet (Wang et al., 2023b) is a specialized protein structure learning framework. The framework stands out for its multi-view representation learning approach that jointly considers backbone geometry, side-chain orientations, and residue interactions.
19

---

Invariance-Enhaneed Protein GCL
CDConv (Fan al., 2023) bridges continuous and discrete representations in protein structure analysis. It features an adaptive sampling strategy that efficiently processes both continuous sequences and discrete atom coordinates.
2D topelogy-based GCL methods:
GraphCI. (You et al., 2020) a graph contrastive learning framework that applies various augmentation techniques including node dropping, edge perturbation, attribute masking, and subgraph sampling.
Auto-GCL (Yin et al., 2021) automatically learns optimal augmentation strategies for graph contrastive learning through bi-level optimization framework.
GCL-Span (Lin et al., 2023a) uses spectral analysis for graph contrastive learning, leveraging graph spectral filtering to create diverse views.
GCS (Wei et al., 2023) focuses on effective sampling strategies with an adaptive mechanism for informative negative samples.
CI-GCL (Tan et al., 2024) incorporates causal inference principles, employing intervention mechanisms to generate counterfactual graph views.
3D structure-based GCL methods:
Traditional 3D augmentation (Wang et al., 2023b), which employs traditional transformations like random coordinate perturbations.
Protein homology modeling tool (Waterhouse et al., 2018) leverages established protein modeling algorithms to generate physically plausible structural variants.
E. Proofs
E.1. The Proof of Theorem 4.1 in the Draft
Definition E.1. (Eigenvalue Perturbation) Assume matrix A', the altered portion is represented by AA = A'- A, and the changed degree is denoted as AD. According to matrix perturbation theory (Hogben, 2013), the change in amplitude for the y-th eigenvalue can be represented as:
Ady = > uT, - AD,JADu, ADu, +
O(IIAA])
(13)
Lemma E.2. If we only flip one edge (1,3) on adjacency matrix A, the change of y-th eigenvalue can be write as
Ady = Acij (2ugi tly Xy (12) + 2
(14)
where uyi is the i-th entry of y-th eigenvector uy and Aeij = (1 - 2Aj)) indicates the edge flip, i.e +1
Proof. Let AA be a matrix with only 2 non-zero elements, namely AA; = AAA = 2A, corresponding to a single edge flip (i,j). and AD the respective change in the degree matrix, i.e. A' = + AA and D' D AD.
Denote with e; the vector of all zeros and single one at position i. Then, we have AA Acj(c.,]] tc,,,) and AD = Acj (e:eT + e.
Based on eigenvalue perturbation formula (13) by removing the high-order term O(I]AA]), we have:
A ~ u, (AA- A,AD)u,
(15)
Substituting AA and AD, we conclude Eq. 14.
Theorem E.3. For unweighted graph, the constraint on the lowest k eigenvalues of the normalized Laplacian matrix Lyorn ensures the preservation of the community structure of nodes.
Proof. Firstly, we sepurate AA AA+ AA- where AA+ and A indicate which edge added and deleted, respectively. To analyze the change of eigenvalues in spectral space corresponding to the perturbation of edges in spatial
20

---

Ennariance-Enhaneed Protein GCL
space, we first consider the situation that only augments one edge for both edge-dropping (Acr -1) and edge-adding
(Arij = 1):
! In the case of edge dropping (Acij = -1 in Lemma E.2), we have
A = -2y Uuy + Xy (ugs + (33)) (uyi ) (2y 1) (ug, mn))
(16)
(17)
If we only drop the edge (i,j) that makes a large change in the eigenvalues. We have the objective function as
arg max (1.1)
A (Uyi unj)² + (Ny
1) (u + =my))
(18)
Any (Aii)
(Uyi
Uns) -2ys) 2 + y=1
Ay 11 (u 2
(19)
n
UU U3.113+ + Σ
12g 1| [1) + m
(20)
y=1
S U, + Σ
|> (2
622 + ym
(21)
y=1
n
UU Uy.112 + Σ
(22)
Notice that uy is the y-th column of U, so Uyi = Up and U.. = [uuss U1i. m From the first item in Eq. 35, we prefer to select the nodes with larger distances in the eigenvector spaces, i.e. two nodes belonging to different communities (The relationship between eigenvector and community structure is proved in Theorem E.6).
In the case of edge adding (Acij in Lemma E.2), we have
A 24y Uyi - by ( (ugs + (zz))
(23)
(uyi Uyy) )² (2y 1) (12) + uzz))
(24)
If we only add the edge (i that makes a small change in the eigenvalues. We have the objective function as
arg min (L.jAc +1)
A = (Uyi unj)² + (2, - 1) (u + m
(25)
(Ugi 1² Σ (Λy)l (une + +znd)
(26)
U
U
Σ 1(1 X²) (u]) + +u2z + + (gn))
(27)
y=1
n
U U, 2
Σ
(1
Xn)
(28)
y=1
From the first item in Eq. 28, we prefer to select nodes with smaller distances in the eigenvector spaces, i.e. nodes belonging to one community (Theorem F.6).
Previously, we have proven that the constraint on the lowest eigenvalues of L ensures the preservation of community structure when we only augment one edge. Next, we will demonstrate that the perturbation of more than one edge still aligns with this theory.
21

---

Envariance-Enhanee Protein GCL
Suppose augment edges, similar Lemma 14, replace AA Σ
[ teje,) and AD
(i.j)E(m edges)
Σ ( odges)
( + +,je,), ) we Substituting AA and AD of Eq. (31), we get:
4 Σ
Act (2tgss Uyi Xy (u + m))))
(29)
(E.)) (m edges)
By replacing Ac; (2ugi Uyi b (uge + )) with Eqs. (35, 28), we could easily see that the community preserving
theory satisfied.
E.2. The Proof of Lemma 4.2 in the Draft
Lemma E.4. If we only flip one edge (i, on adjacency matrix with edge weight W, the change of y-th eigenvalue can be write as
-Was (2ugi Uyi by (uge + uzg)) 2ugy Uyy by (ug, + ugg)
if edge dropping if edge adding
(30)
where Uyt is the i-th entry of y-th eigenvector uy.
Proof. Let AA be a matrix with only 2 non-zero elements, namely AA AAA WWs corresponding to single edge dropping (i,j), and AAj AA = corresponding degree matrix, iee. A' A AA and D' = D + AD.
single edge adding (i. AD is the respective change in the
Denote with e; the vector of all zeros and a single one at position i. For edge dropping, we have A = (e,e teje," and AD- We, + e,e]). ). For edge adding, have A (e.ee + e;e,) and AD+ (eve, t
Based on eigenvalue perturbation formula (13) by removing the high-order term O(I)AA]), we have:
A u.J (AA - AD)u,
(31)
Substituting AA and AD, for edge dropping and edge adding respectively, we conclude:
Wij (Zugi Uyy by (ug, + + ) if edge dropping 2uyy Ugy by (ug, + )
Ady
(32)
if edge adding
Lemma E.5. For weighted graphs, maximizing the spectral change during edge dropping ensures that edges between communities and those with large edge weights more likely be removed.
Proof. For weighted graph (Lemma E.4), we have
A = Wij (Nyi 2² + Wiy(Ay - 1) (ne + n
(33)
If we only drop the edge (i,j) that makes large change in the eigenvalues. We have the objective function (similar to the proof in Theorem E.3) as:
arg max (i5)
| W
(1ys (
unz))² + (2g 1) (1 2
(34)
y=1
< WWol [|U [ U Σ My - -11 11)
(35)
In this case, edge (i,j) across different communities ( - Uy. is large) and with large Wayl are more likely to be removed.
22

---

Invariance-Enhaneed Protein GCL
E.3. The relation between community and spectrum
Theorem E.6. The spectral decomposition can indicate relaxed solution of graph vertex partition. Given graph G with Laplacian matrix L, and the spectral decomposition is indicated as D LDD UAU", where D is the degree matrix and U, A are eigenvectors, eigenvalues correspondingly. The second smallest eigenvalue and its corresponding eigenvector indicate bipartition of the graph.
Proof. Given partition of nodes of graph (split V into two disjoint sets SA and SB), let be indicator vector for the partition, = 1 if node i is in SA, and 1, otherwise. Let d; be the degree of node i, and Waj is the weight of edge (i,j). Based on (Shi & Malik, 1997), a normalized cut could be write as
(S, Ex, Wiy
Earc0,0,20 W0y-,z
(36)
Σz.50
Ez,0
The optimal partition computed by minimizing the normalized cut: miny Ncut(x).. By setting y = (1 x) -b(1 - x),
E,,20 d
k = d:
and b = 1-k- we could rewrite as
min Neut(x) min Ly
(37)
y Dy
with the condition yi E bb, and y D1 D1 0. According to the Rayleigh quotient (Gene et al., 2013), we can minimize Eq. 37 by solving the generalized eigenvalue system, y is relaxed to take on real values.
Ly = A
(38)
By writing z = D we could transform the Eq. 38 to a standard eigensystem:
D LD tz λz
(39)
Because the Laplacian matrix is positive semidefinite (Pothen al., 1990), can easily verify that 2p D is the smallest eigenvector of Eq. 39 with eigenvalue 0. And correspondingly. yo = 1 is the smallest eigenvector with an eigenvalue of in the general cigensystem 38, but do not satisfy the condition y D1 = 0. According to the Lemma E.7, we could know that the second smallest eigenvector is the solution of Eq. 37, because z zo y D1 = 0 satisfy the condition in Eq.37. 37.
Therefore, the second smallest eigenvalue and its corresponding eigenvector indicate bipartition of the graph. While the next cut must be perpendicular others, which is the third smallest eigenvalue corresponding eigenvector 2 Z z Zo. Recursively, the k-dimensional eigenvectors represent the community structure of graph some extent.
Lemma E.7. simple fact about the Rayleigh quotient (Gene et al., 2013): Let A real symmetric matrix. Under the constraint that x is orthogonal to the j - smallest eigenvectors X1, X2, X;-1, the quotient x x'Ax Ax is minimized by the next smallest eigenvector X, and its minimum value is the corresponding eigenvalue A
x
F. More Detailed Experiments
F.1. Go and EC Prediction under Different Sequence Cutoffs
For GO and EC, we follow the evaluation protocol established by (Gligorijević al., 2021), where proteins in the test set are divided based on their sequence similarity the training set, using cutoffs of 30%, 40%, 50%, 70%, and 95%. This systematic evaluation across different similarity thresholds helps assess model generalization ability and robustness. Table E.3 presents comprehensive results comparing our methods with existing approaches. Our complete framework demonstrates consistent improvements across all four protein-related tasks compared to existing methods. For instance, compared to the top-performing method CDCony, the combination of 3-PSIAIpha and FCI demonstrates superior performance, achieving a 0.0185 improvement at the 95% similarity cutoff EC. The ablation studies further validate the effectiveness of our design choices, showing that each component contributes to the overall performance improvement. Notably, the performance
23

---

Shaariance-nhancee Protein GCL
Table 5. This table shows the comparison of general protein approaches, graph contrastive lcarning approaches (both 2D copology-bassee 3D structure- based) our proposed GCL approach on Go under different sequence cutoffs. We report maximum FI score (Fiaz) for Enzyme Classification (EC) and Gene Ontology (GO) prediction tasks; and Top-1 accuracy (%) for Fold and Reaction classification tasks. The performance CDConv is from our reproduction, while other baseline results are from (Fan et al., 2023).
GO-CC
GO-MF
Cutoff
30%
40%
50%
70%
95%
30%
40%
50%
70%
95%
CNN
0.258
0.257
0.260
0.263
0.387
0.238
0.243
0.256
0.292
0.354
ResNet
0.277
0.273
0.280
0.278
0.304
0.282
0.288
0.308
0.347
0.405
Protein-speciite
LSTM
0.263
0.264
0,269
0.270
0.283
0.223
0.229
0,245
0.276
0.321
Transformer
0.378
0,382
0.388
0.395
0.405
0.184
0.187
0.195
0.204
0.211
GearNet
0.381
0,385
0,393
0.398
0.414
0.382
0.397
0.425
0,474
0.503
GearNet-Edge
0.394
0,394
0,401
0.408
0.450
0.444
0.461
0,490
0.537
0.580
CDConv
0.429
0.433
0.442
0.447
0.475
0.535
0.551
0.573
0.620
0.652
2D Topo. GCL
GraphCL
0.393
0.378
0,400
0,405
0.432
0.516
0.530
0,544
0,610
0.644
Auto-GCL
0.373
0.374
0.399
0.394
0.417
0.510
0.531
0.556
0.604
0.641
GCL--ppan
0.381
0.383
0.394
0.400
0.419
0.515
0.527
0,551
0.620
0.650
GCS
0.389
0.382
0.395
0.404
0.437
0.518
0.536
0.566
0.613
0.645
CI-GCL
0.357
0.373
0.391
0.410
0.453
0.519
0.533
0.561
0.614
0.651
3D
Trad. 3D Aug.
0.385
0.384
0.387
0.394
0.420
0.521
0.543
0.547
0.615
0.645
Auto. Tool Aug.
0.367
0.376
0,380
0.405
0.435
0.515
0.540
0.535
0.612
0.643
Ours (Ablation)
Backbone
0.391
0.397
0,404
0.428
0.432
0.509
0.531
0.558
0.604
0.642
+ FCI
0.439
0.441
0.456
0,454
0.473
0.532
0.549
0,575
0.625
0.656
+ 3-PSSDDiag
0.438
0.437
0.445
0.456
0.468
0.532
0.552
0.578
0.620
0.654
3-PSTAlpha
0.425
0.429
0.431
0.437
0.453
0.525
0.544
0.572
0.618
0.652
33-PSIDiag + FCT
0.459
0.462
0.467
0.468
0.484
0.542
0.559
0.585
0.629
0.662
+ 3-PSSAAlpha FCT
0.442
0.450
0.475
0.455
0.477
0.533
0.551
0.578
0.622
0.659
GO-BP
EC
Cutoff
30%
40%
50%
70%
95%
30%
40%
50%
70%
95%
CNN
0.197
0.195
0.197
0.211
0.244
0.366
0,361
0.372
0.429
0.545
ResNet
0.230
0.230
0.234
0.249
0.280
0.409
0.412
0.450
0.526
0.605
Protein-specile
LSTM
0.194
0.192
0.195
0.205
0,225
0.247
0,249
0.270
0.333
0.425
Transformer
0.267
0.265
0.262
0.262
0.264
0.167
0.173
0.175
0.197
0.238
GearNet
0.309
0.309
0.315
0.336
0.356
0.557
0.570
0.615
0.693
0.730
GearNet-Fdge
0.345
0.347
0.354
0.378
0.403
0.625
0.646
0.694
0.757
0.810
CDConv
0.380
0.387
0.402
0.431
0.450
0.721
0.748
0.782
0.825
0,870
2D Topo. GCL
GraphCL
0.382
0.386
0.410
0.411
0.432
0.724
0.743
0.771
0.829
0,860
Auto-GCL
0.365
0.373
0.385
0.408
0.436
0.714
0.746
0.783
0.834
0.852
GCL-Span
0.387
0.395
0.404
0.408
0.434
0.739
0.774
0.804
0.847
0.859
GCS
0.387
0.386
0.407
0.414
0.440
0.724
0.756
0.792
0.832
0.864
CI-GCL
0.392
0.391
0.411
0.420
0.440
0.749
0.783
0.810
0.844
0,870
3D
Trad. 3D Aug.
0.378
0.389
0.388
0.405
0.432
0.747
0.760
0.796
0.843
0,870
Auto. Tool Aug.
0.368
0.375
0.383
0.397
0.439
0.729
0.761
0.788
0.839
0.863
Ours (Ablation)
Backbone
0.351
0.371
0.385
0.408
0.437
0.710
0.737
0.768
0.818
0.853
FCI
0.385
0.380
0.398
0.415
0.446
0.743
0.771
0.807
0.850
0.878
+ +
0.377
0.385
0.394
0.412
0.449
0.744
0.769
0.806
0.852
0.873
3--PSIAlpha
0.381
0.380
0.390
0.413
0.444
0.738
0.765
0.803
0.848
0.874
+ 3-PSSIDiag + FC1
0.391
0.399
0.410
0.436
0.461
0.745
0.783
0.815
0.858
0.883
3-PPSAAlpha FCI
0.378
0.388
0.399
0.425
0.454
0.749
0.773
0.808
0.859
0.885
24

---

Invariance-Enhaneed Protein GCL
advantage of methods is more pronounced at lower sequence similarity cutoffs (30-50%), which represents a more challenging and realistic scenario in protein function prediction, demonstrating remarkable robustness. This suggests that supervised graph contrastive learning framework better captures fundamental protein structural features, leading to more robust and generalizable predictions.
F22 Compare with Protein Language Models
In recent years, protein representation learning has witnessed significant advances through the development of large-scale protein language models and self-supervised learning approaches (Hermosilla & Ropinski, 2022: Zhang et al., 2023). These methods typically rely on extensive pre-training with massive datasets to learn protein representations. For example, ESM-Ib (Rives et al., 2019) utilizes UniRei50 (24M sequences), while ProtBERT- BFD (Elnaggar et al., 2020a) leverages the even larger BFD dataset IB sequences). The underlying assumption is that utilizing large-scale protein data during pre-training enables these models to capture fundamental protein properties and patterns. A parallel line of research has explored self-supervised learning specifically for 3I protein structures. These approaches aim learn effective representations by designing various pretext tasks. For instance, Hermosilla & Ropinski (2022) proposed a contrastive learning framework that maximizes similarity between sub-structures from the same protein while minimizing similarity between sub- strbstruuttres from different proteins. Zhang et al. (2023) extended this idea by incorporating multiple self-supervised tasks, including: maximizing agreement between different augmented views of the same protein, predicting residue types, and learning to estimate geometric properties.
However, some fundamental questions remain: (1) how can we select self self-super tasks, which is sufficient learn truly effective protein representations? (2) these methods typically require extensive computational resources for training large- scale datasets with data cleaning, preprocessing, and significant effort in designing and validating multiple self-supervised tasks. Different from these works, our work concentrates on designing effective supervised graph contrastive learning paradigm that enables graph neural networks to better capture protein structural information. Also, this invariant paradigm may improve the baseline for the pre-training or self-supervised learning works. Here, we compare our approach with various SOTA methods including protein language models and self-supervised learning approaches: DeepFRI (Gligorijević et al., 2021), ESM-Ib Ib (Rives al., 2019), ProtBERT-BFD (Elnaggar et al., 2020b), LM-GVP (Wang et al., 2021), residue-level IEConv (Hermosilla & Ropinski, 2022), and GearNet-based methods (Zhang al., 2023). As demonstrated in Table F:2 and Table F.2, our anvariance-based supervised GCL framework achieves comparable superior performance across multiple tasks, notably outperforming these methods on fold classification and enzyme reaction prediction, despite not using any pre-training or self-supervised learning strategies.
Table 6. Accuracy of protein fold classification between different protein language models and our proposed GCL approach. Baseline results are from (Fan et al., 2023).
Method
Pre-training Dataset (Size)
Fold Classification Superfamily
Fold
Family
DecpFRI
Pfam (10M)
15.3
20.6
73.2
ESM-1b
UniRef50 (24M)
26.8
60.1
97.8
Pro(BERT-BFD
BFD (2.1B)
26.6
55.8
97.6
IEConv (residue level)
PDB (476K)
50.3
80.6
99.7
GearNet-Edge-IICConv with Multiview Contrast
AlphaFoldDB (805K)
54.1
80.5
99.9
GearNet-Edde--ECConv with Residue Type Prediction
AlphaFoldDB (805K)
48.8
71.0
99.4
Gead/eet-Fdgee-IlCConn with Distance Prediction
AlphaFoldDB (805K)
50.9
73.5
99.4
GeanNet-Edgee-EECon with Angle Prediction Gea-Net-Edgee-ECConv with Dihedral Prediction
AlphaFoldDB (805K)
56.5
76.3
99.6
AlphaFoldDB (805K)
51.8
77.8
99.6
3-PSIDiag + FCI
58.9
81.3
99.7
3-PSIAlpha + FCI
59.8
80.8
99.5
F.3. Functional Community Preservation Analysis
In this section, we evaluate how well different 2D topology-baid based graph augmentation strategies preserve functional communities within protein graphs. Specifically, we focus on protein pockets (i.e., spatially contiguous sets of residues),
25

---

Ewaariance-nnaanee Protein GCL
Table Accuracy of enzyme catalytic reaction classification and max of gene ontology term prediction and enzyme commission number
prediction between different protein language models and our proposed GCL approach. Baseline results are from (Fan et al., 2023).
Method
Pre-training Dataset
Enzyme
Gene Ontology
Enzyme
(Size)
BP
MF
CC
Commission
DeepFRI
Pfam (10M)
63.3
0.399
0.46
0.460
0.631
ESM-Ib
UniRef50 (24M)
83.1
0.476
0.657
0.498
0.864
ProtBERT-BFD
BFD (2.1B)
72.2
0.279
0.456
0.408
0.838
LM-GVP
UniRef100 (216M)
0.417
0.545 0.527
0.664
IEConv (residue level)
PDB (476K)
88.1
0.468
0.661 0.516
GearNet-Edge with Multiview Contrast
AlphaFoldDB (805K)
87.5
0.490 0.654 0,488
0.874
GearNet-Edge with Residue Type Prediction
AlphaFoldDB (805K)
86.6
0.430
0.604 0.465
0.843
GearNet-Edge with Distance Prediction
AlphaFoldDB (805K)
87.5
0.448
0.616 0.464
0.839
GearNet-Edge with Angle Prediction
AlphaFoldDB (805K)
86.8
0.458
0.625 0.473
0.853
GearNet-Fädge with Dihedral Prediction
AlphaFoldDB (805K)
87.0
0.458
0.626 0.465
0.859
3-PSIDiag + FCI 3-PSIAlpha FCI
87.8
0.461
0.662
0.484
0.883
89.0
0.452
0.659 0.477
0.885
which naturally align with the radius-based graph construction used in GNNs. Protein pockets directly govern essential protein functions, such substrate binding, catalysis, and allosteric regulation (Schulze et al., 2016). Although various functional annotation methods exist (such as domain partitioning or coevolutionary residue coupling), these approaches do not necessarily translate into local adjacency in graph neural network. By contrast, pockets exhibit spatially contiguous arrangement of residues that naturally aligns with the radius-based graph construction employed by GNNs to some extent. Thus, pockets offer a practical evaluation, unlike other globally defined or purely evolutionary methods that are not readily translated into graph-level representations.
For quantitative evaluation, we first identify functional edges by detecting residues in pockets using fpocket 4. These edges connecting residues within pockets are considered critical and should be preserved during augmentation. Then evaluate how well our augmented graphs maintain these essential functional connections by calculating the preservation rate:
Let Etune be the set of functional edges (connections between residues within identified pockets) in the original graph, and Eaug be the edges in the augmented graph. The preservation rate rj is calculated as:
Efunc
Eaugl
&funel x 100%
(40)
where Egume Eaugl represents the number of functional edges preserved after augmentation, and | is the total number of functional edges in the original graph.
As shown in Table 8, our FCI method achieves higher preservation rates across all tasks compared to other methods, demonstrating its effectiveness in maintaining the integrity of functional communities during graph augmentation.
Table Preservation rates (%) functional communities across three graph augmentation methods (Auto-GCL, CI-GCL, FCI) on four mrotein-related tasks: GO, EC, FOLD, and Reaction.
Method
GO
EC
FOLD
Reaction
Auto-GCL
75.29
78.93
70.88
73.43
CI-GCL
91.87
94.21
76.88
88.05
FCI
96.69
98.02
90.87
97.45
"hats://fooketssswrccfooeeee
26

---

Invariance-Enhancee Protein GCL
F.4. Compare with Protein Learning Methods
In this section, we further compare with our proposed method with protein structure learning models (PSL): Uni-mol (Zhou et al., 2023), P3G (Huang et al., 2023), ESM-2 (Lin et al., 2023b), S-PLM (Wang al., 2023a); equivariant graph neural networks (EG) EquiformerV2 (Liao et al., 2024); and structure retrieval approaches (SR) Foldseek (van Kempen et al., 2022). We report results of S-PLM and P3G from their original papers, and ESM-2 from [1]. Uni-mol and EquiformerV2 are reproduced. For the GO task, we evaluated Foldseek on a subset of 150 samples as exploration. Despite this, our method achieves comparable performance, even though Foldseek relies on larger external protein database (c.g.. RCSB and PDB) for retrieval-baas based prediction. While our GO-BP result is lower, we outperform Foldseek on GO-MF and GO-CC, Trained from scratch on a smaller dataset, our model shows strong generalization and effectiie representation learning. Uni-mol and EquiformerV2 underperform due their models being tailored to molecules rather than proteins. Compared S-PLM, method achieves competitive results the EC and GO datasets, while consistently outperforming S-PLM on FOLD and Reaction.
Table Evaluation of proposed method against several categories of protein learning models: protein structure learning (PSL). structure retrieval (SR), and equivariant graph networks (EG). Best scores are highlighted in bold.
GO
FOLD
Method
EC
Reaction
BP
MFF
CC
Fold
Super.
Fam.
Avg.
Uni-mol
0.721
0.347
0.441
0.397
31.4
61.0
90.5
60.9
74.2
PSL
P3G
0.784
0.379
0.548
0,448
ESM-2
0.861
0,460
0.663
0,427
38.5
81.5
99.2
73.0
S-PLM
0.888
0.495
0.685
0.484
37.7
78.0
98.8
71.5
86.71
EG SR
Foldseek
0.582
0.570
0.472
90.60
EquiformerV2
0.751
0.351
0.480
0.375
29.9
65.2
88.0
61.0
76.42
Ours
0.885
0.461
0.662
0.484
59.8
81.3
99.7
80.3
89.00
F.5. Sensitive Study of
Complete results sensitive study are presented at Table 10.
Table Performance of the proposed method using various values for the hyperparameter the loss function. Optimal scores are highlighted in bold.
GO
FOLD
X
EC
Reaction
BP
MF
CC
Fold
Super.
Fam.
Avg.
0.2
0.882
0.451
0.656
0.467
56.5
80.2
99.8
78.8
86.2
Value
0.6
0.879
0.443
0.658
0.479
58.2
80.7
99.6
79.5
87.5
1.0
0.885
0.461
0.662
0.484
58.9
81.3
99.7
79.9
89.0
1.4
0.880
0.457
0.649
0.489
59.1
80.6
99.7
79.8
87.8
1.8
0.877
0.448
0.653
0.475
58.5
80.2
99.8
79.5
88.3
F.6. Complete Results in the Draft
Figure shows results of 2D augmentation strength analysis on all protein-related tasks. For both FOLD and GO, we report the mean performance their subtasks.
Figure 10 shows results of robustness analysis GOME, EC, FOLD and Reaction.
Figure 1l shows results of 3D augmentation strength analysis on all tasks.
27

---

Seaariance-nnhanee Protein GCL
GraphCL
Ours
0.55
0.90
0.88
Fmax
0.51
0.86
0.1
0.3
0.4
0.2
0.3
(a) of GO
(b) of EC
81
Accuracy (%)
79
Accurat
77
87
75
0.3
0.4
0.2
(c) of FOLD
(d) æ of Reaction
Figure 9. Performance comparison between GraphCL and our method under different graph augmentation strengths (0.1-0.4). For the GO, we report average performance three subtasks.
Trad3D
3DCov
CI-GCL
Ours
0.68
0.90
0.64
Fmax
0.85
0.60
0.56
0.80
0.52
0.75
(a) Proportion of GOMF
(b) Proportion of EC
80
90
Accuracy (9)
88
75
70
65
(c) Proportion of FOLD
(d) Proportion of Reaction
Figure 10. Robustness analysis of models under varying perturbation ratios for protein structure. This figure shows model performance (Fmaz GOMF and tasks; accuracy FOLD and RC tasks) across different noise ratios (0.1-0.5) on four distinct prediction tasks: (a) GOME, (b) EC, (c) FOLD, and (d) RC.
EC
GO
GOBP
GOCC
Reaction
FOLD
0.9
90
0.8
Accuracy (%)
85
0.6
0.5
Number Rotations
Number of Rotations
Figure 11. Performance analysis of the model under varying a-helices and f-sheets rotational fold strengths across all tasks.
28


Reviews:
[{"review_id": "UNNFLdFBoS", "reviewer": "Reviewer_exBg", "summary": "This paper investigates methods to improve Graph Contrastive Learning for protein representation learning by incorporating biologically-aware graph augmentation strategies. The authors propose two novel augmentation strategies: Functional Community Invariance (FCI) and 3D Protein Structure Invariance (3-PSI), which are integrated into a unified GCL framework for protein representation learning. Extensive experiments on four protein-related tasks show that their approach consistently improves classification accuracy and robustness over existing 2D topology-based and traditional 3D augmentation methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely appreciate the reviewer exBg's thoughtful and valuable comments.**\n***\n> **_Q1_** *The derivation of both the upper and lower bounds ... requires clarification. & The weighted upper bound may need to be revised.*\n\n**A1**  We appreciate your valuable feedback on the proof. The proof may be unclear due to the use of different symbols for eigenvalues (k in main text, y in appendix). We will unify them to enhance clarity. \nWe clarify the derivation of upper and lower-bounds as follows. When sampling an edge for augmentation, we first determine the change in the y-th eigenvalue of the adjacency matrix by applying the eigenvalue perturbation theory (Lemma E.2, Line 988). Next, we derive the upper and lower bounds by applying the triangle inequality when dropping or adding an edge (see derivations in Theorem E.3, Line 1034 and Line1056, respectively).\n\nRegarding the weighted upper bound, after careful verification, we confirm that no revision is necessary. The change in the eigenvalue for weighted graphs is scaled by the edge weight Wij (Line 1079). By following the same derivation steps as in the unweighted case, we then obtain the upper bound for weighted graphs (Line 1113). \n\nTo improve clarity, we will revise the main text to better point readers to the corresponding derivations and proofs in the appendix for the upper and lower bounds.\n***\n> **_Q2_** *The random rotation angle θ in Section 4.2 (line 246) requires clarification.*\n\n**A2** We appreciate your attention to the selection of the θ. Our goal is to enhance the diversity of protein structures through augmentation while preserving their functionality. Therefore, we prefer to start with relatively small angles. The choice of θ is further guided by our experimental results. We conducted experiments on two tasks with varying θ values. The results show that θ = 10° yields the best performance. Too small θ may lead to insufficient conformational diversity, while a larger θ risks distorting the original protein structure—supporting our hypothesis.\n\n| θ | FOLD-fold | FOLD-Super. | FOLD-Family | Reaction |\n|-------|-----------|-------------|-------------|----------|\n| 5     | 59.5      | 80.5        | 99.7        | 88.8     |\n| **10**| **59.8**  | **81.3**    | **99.7**    | **89.0** |\n| 15    | 59.1      | 80.4        | 99.7        | 88.1     |\n| 20    | 58.6      | 79.9        | 99.4        | 87.4     |\n| 30    | 57.2      | 78.2        | 99.3        | 85.6     |\n| 50    | 56.4      | 77.6        | 98.9        | 83.8     |\n***\n> **_Q3_** *2D topology-based methods for protein graph topology as baselines for comparison.*\n\n**A3** Thank you for your valuable suggestion. To the best of our knowledge, few GCL methods have been specifically designed for proteins. Therefore, we have additionally included two relevant GCL approaches designed for molecules in the following table. \n\n| Model  |  EC   | GO-BP | GO-MF | GO-CC | FOLD-fold | FOLD-Super. | FOLD-Family | Reaction |\n|--------|-------|-------|-------|-------|-----------|------------|------------|----------|\n|MolCLR[1]|0.869|0.443| 0.623 | 0.455 | 54.0| 77.9       | 99.5       | 87.7     |\n|T-MGCL[2]|0.843|0.422| 0.628 | 0.468 | 55.0      | 77.6       | 99.5       | 85.3     |\n|Ours|**0.885**|**0.461**|**0.662**|**0.484**|**59.8**|**81.3**|**99.7**|**89.0**|\n***\n> **_Q4_** *Some heuristic choices ... require tuning for new datasets.*\n \n**A4** \nThank you for your attention. Experiments in Section 5.3 indicate that variations in these hyperparameters do not lead to significant changes in performance, suggesting that our proposed augmentation strategy is relatively robust to their selection. As illustrated in our experiment setting part Appendix C.2., a shared hyper-parameter configuration such as ϵ = 0.2 for GO, EC, and Reaction, and θ = 2 for EC, GOBP, and FOLD, works well across different datasets and tasks. These choices can serve as useful tuning basics for new datasets. \n***\n> **_Q5_** *Other factors may also have a substantial impact on protein structure and function...*\n\n**A5** We sincerely appreciate the reviewer’s insightful comments. \nWe acknowledge that protein structure and function can be influenced by various factors. We primarily use dihedral angles, α-helices and β-sheets rotation to maintain the essential protein structure and improve the protein representation learning. Our experiments have proven the importance of using these factors and keeping invariance in graph augmentation. In future work, we will explore more factors to further strengthen invariance to improve protein representation.\n***\n**_Reference_**\n\n[1] Molecular Contrastive Learning of Representations via Graph Neural Networks\n\n[2] T-MGCL: Molecule Graph Contrastive Learning Based on Transformer for Molecular Property Prediction\n***\n__If your concerns have been addressed, could you kindly consider raising your score? We greatly appreciate your comments and support.__", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence except for the bounds of spectral changes in Theorem 4.1 (line 201 and line 202). The derivation of both the upper and lower bounds appears unclear and requires further clarification.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense for the problem except for the selection of the random rotation angle θ in Section 4.2 (line 246). There appears to be no theoretical justification for the choice of the random rotation angle, which requires further clarification.", "theoretical_claims": "I have verified the accuracy of the claim stated in lines 90–96:\n\"To generate augmented graphs while preserving the 3D-related information of proteins, 3-PSI employs two distinct coordinate perturbation strategies: (1) rotations of backbone dihedral angles and (2) rotations of secondary structures (α-helices and β-sheets), ensuring that peptide planes and secondary structures remain intact during graph augmentation.\"\nIt has been well-established that dihedral angles, α-helices, and β-sheets significantly influence a protein's structure and, consequently, its function. However, it is worth noting that other factors may also have a substantial impact on protein structure and function, which this paper has not considered.", "experimental_designs_or_analyses": "I have carefully examined the experimental designs and analyses. Overall, the experiments in this paper are highly convincing. However, when comparing 2D topology-based methods, the selected baseline methods are all graph contrastive learning approaches that do not specifically account for the topological structure of protein graphs. Intuitively, these methods may significantly impact the original protein structure. If there are 2D topology-based methods designed specifically for protein graph topology, it would be beneficial to use them as baselines for comparison.", "relation_to_broader_scientific_literature": "1.\tThis paper extends spectral graph contrastive learning by incorporating functional community constraints, building on prior work like GraphCL, GCS, and CI-GCL.\n2.\tThis paper bridges the gap between 2D and 3D protein graph learning, incorporating structural constraints often overlooked in self-supervised learning.\n3.\tThis paper improves generative augmentation techniques, refining methods used in homology modeling tools (e.g., SWISS-MODEL, MODELLER) by introducing rotation-aware 3D augmentation.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "--Strengths: This paper introduces novel biologically-aware augmentations that explicitly consider both functional and structural integrity as well as improves robustness against structural perturbations, which is crucial for real-world protein modeling. It presents a novel approach to protein modeling, which could significantly inspire protein research and contribute to advancements in drug discovery.\n\n--Weaknesses: Some heuristic choices in augmentation design: The degree of perturbation (ϵ) and number of rotations (θ) remain task-specific hyperparameters, which might require tuning for new datasets.", "other_comments_or_suggestions": "There is an issue in section 4.1 (lemma 4.2, line 209): the weighted upper bound may need to be revised.", "questions_for_authors": "1.\tThere appears to be no theoretical justification for the choice of the random rotation angle θ in Section 4.2 (line 246), which requires further clarification.\n2.\tThe derivation of both the upper and lower bounds in Theorem 4.1 (line 201 and line 202) appears unclear and requires further clarification.\n3.\tFor 3D-related information of proteins, besides dihedral angles, α-helices, and β-sheets, are there any other factors that may also have a substantial impact on protein structure and function?\n4.\tAre there 2D topology-based methods designed specifically for protein graph topology? I am convinced that it would be beneficial to use them as baselines for 2D topology-based methods comparison.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vNNCQP1Q5N", "reviewer": "Reviewer_s9oD", "summary": "This paper introduces novel biology-aware graph augmentation strategies for protein representation learning within a Graph Contrastive Learning (GCL) framework. The authors identify limitations in existing GCL approaches that either focus exclusively on 2D topology (neglecting intrinsic biological properties) or lack effective 3D structure-based augmentation methods. To address these shortcomings, they develop two complementary strategies: (1) Functional Community Invariance, which preserves topology-driven community structures while incorporating residue-level chemical similarity, and (2) 3D Protein Structure Invariance, which employs dihedral angle perturbations and secondary structure rotations to maintain critical 3D structural information. Experiments across four protein-related tasks demonstrate consistent improvements over existing GCL methods and protein-specific models. The paper offers a valuable solution for GCL in the context of protein structure learning. If the authors could further enhance their main experiments by incorporating comparisons with relevant models, I would be inclined to increase my overall rating accordingly.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We thank reviewer s9oD for the constructive feedback and valuable suggestions.**\n***\n> **__Q1__** *Incorporating comparisons with relevant models.*\n\n**A1** We have conducted extensive experiments with the other suggested baselines shown as follows.\n\n|Model|EC|GO-BP|GO-MF|GO-CC|FOLD-fold|FOLD-Super.|FOLD-Family|Reaction|\n|-|-|-|-|-|-|-|-|-|\n|Foldseek|-|**0.582**|0.570|0.472|-|-|-|**90.60**|\n|Uni-mol|0.721|0.347|0.441|0.397|31.35|60.97| 90.51|74.20|\n|S-PLM|**0.888**|0.495|**0.685**|**0.484**|37.74|77.95|98.82| 86.71|\n|P3G [r3]|0.784|0.379|0.548|0.448|-|-|-|-|\n|ESM-2| 0.861|0.460|0.663|0.427|38.50|**81.50**|99.20|-|\n|EquiformerV2|0.751|0.351|0.480|0.375| 29.88|65.18|88.02|76.42|\n|**Ours**|0.885|0.461|0.662 |**0.484**|**59.80** |81.30|**99.70**|89.00|\n\nWe report results of S-PLM and P3G from their original papers, and ESM-2 from [1]. Uni-mol and EquiformerV2 are reproduced.\nFor the GO task, we evaluated Foldseek on a subset of 150 samples due to time constraints and manual extraction of ground-truth labels. Despite this, our method achieves comparable performance, even though Foldseek relies on a **larger external protein database** (e.g., RCSB and PDB) for retrieval-based prediction. While our GO-BP result is lower, we outperform Foldseek on GO-MF and GO-CC. Trained from scratch on a smaller dataset, our model shows strong generalization and effective representation learning.\nUni-mol and EquiformerV2 underperform due to their models being tailored to molecules rather than proteins. Compared to S-PLM, our method achieves competitive results on the EC and GO datasets, while consistently outperforming S-PLM on FOLD and Reaction.\n***\n> **__Q2__** *How does the ... using predicted protein structures? How sensitive is the 3-PSI to the initial quality...? Would lower-resolution structures impact performance?*\n\n**A2** We appreciate your valuable questions. Our approach is designed agnostic to data source and currently being evaluated on experimental structures following previous works.\n\nRegarding the sensitivity of 3-PSI to the initial quality of protein structures, we design two experiments: (1) low resolution (2) partial PDB data available.\n\n(1) Low resolution: We follow the resolution classification proposed by [2] and divide the test set into low-resolution (≥3 Å) and high-resolution (<3 Å) structures for three tasks (we cannot retrieve resolution of data in Fold task). Result shows that 3-PSI maintains strong performance across both resolutions, suggesting that 3-PSI is robust to input resolution.\n\n(2) Partial PDB data available: We simulate incomplete protein structure. Here we remove 20% of residues and train the model. The result is shown in the same table. Here we can find that our model is robust and not sensitive to the initial quality of protein structure.\n\nOverall, our method shows robustness to diverse input qualities, which indicate that our method has the potential of being generalized to predicted protein structures. We will conduct experiments on predicted structures in our future work.\n\n||EC|GO-BP|GO-MF|GO-CC|Reaction|\n|-|-|-|-|-|-|\n|**Resolution test**|||||||||\n|Low|0.903|0.485|0.648|0.504|0.921|\n|High|0.888|0.465|0.687|0.463|0.894|\n|**Incomplete pdb test**|||||||||\n|Incomplete|0.871|0.453|0.649|0.457|87.6|\n|Complete|0.885|0.461|0.662|0.484|89.0|\n***\n> **__Q3__** *Complexity analysis and potential of being adopted to more complexed encoders.*\n\n**A3** Our method consists of two stages: preprocessing and training.\nEach protein structure undergoes preprocessing once. For 3-PSI, we iterate over residues to determine the current dihedral angles and secondary structures in $O(n)$ time, where $n$ is the number of residues. For FCI, we search the probability matrix for edge perturbation using the Lanczos algorithm [3] for spectral decomposition, with a time complexity of $O(n^2K)$, where $K$ is the number of selected eigenvalues.\nDuring training, each 3-PSI augmentation perturbs all residues with $O(n)$ time complexity, while FCI samples edges based on the possibility matrix with $O(n^2)$ complexity; however, both operations benefit significantly from NumPy’s SIMD vectorization, resulting in fast practical runtimes. The running time is detailed in A2 of Reviewer UCwy.\n\nOur method essentially provides an augmentation strategy and is not tightly coupled to specific encoder architecture. It can be extended to more complexed encoders.\nHere we adopt a Graph Transformer encoder and report the performance:\n\n|Model| EC|GO-BP|GO-MF|GO-CC|FOLD-fold|FOLD-Super.|FOLD-Family|Reaction|\n|-|-|-|-|-|-|-|-|-|\n|graph transformer|0.874|0.470|0.652|0.485|58.8|80.0|99.7|87.8|\n***\n**Reference**\n\n[1] Endowing Protein Language Models with Structural Knowledge\n\n[2] Biomolecular Crystallography: Principles, Practice, and Application to Structural Biology\n\n[3] The Lanczos Algorithm with Selective Orthogonalization\n***\n**Given these clarifications, could you kindly consider raising your score? We greatly appreciate your support.**", "claims_and_evidence": "The claims are generally well-supported by evidence. The authors provide: \n\n1. Theoretical motivation for both augmentation strategies.\n\n2. Quantitative results showing performance improvements across multiple tasks.\n\n3. Ablation studies demonstrating the contribution of each component.\n\n4. Qualitative analyses showing preservation of functional communities and visualization of learned representations. \n\nThe performance improvements are modest but consistent.", "methods_and_evaluation_criteria": "The proposed methods are appropriate for the problem. The authors evaluate on standard protein-related benchmarks with comprehensive comparison across various baselines, including protein-specific methods, 2D topology-based GCL, and 3D structure-based GCL methods. However:\n\n1. Since protein structures are assumed to be known in advance before model training, their annotations are easily obtainable. Alternatively, structure retrieval approaches like Foldseek [r1] could be used to find similar structures with known function annotations. This should be considered as a baseline in the comparison.\n\n2. The evaluation would benefit from comparison with state-of-the-art structure learning models [r2,r3,r4,r5] and equivariant graph neural networks [r6] as encoders for protein-specific baselines.\n\n3. The paper lacks complexity analysis, making it unclear whether 3-PSIAlpha + FCI could be easily adopted into mainstream transformer-based foundation models.\n\n[r1] Kempen et al. Fast and accurate protein structure search with Foldseek. Nature biotechnology 2024.\n\n[r2] Zhou et al. Uni-mol: A universal 3d molecular representation learning framework. ICLR 2023.\n\n[r3] Huang et al. Protein 3D Graph Structure Learning for Robust Structure-Based Protein Property Prediction. AAAI 2024.  \n\n[r4] Lin et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 2023.\n\n[r5] Wang et al. S‐PLM: Structure‐Aware Protein Language Model via Contrastive Learning Between Sequence and Structure. Advanced Science 2025.\n\n[r6] Liao et al. EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations. ICLR 2024.", "theoretical_claims": "The derivation of the Functional Community Invariance approach is sound, establishing clear connections between spectral constraints and community preservation. The authors effectively develop theoretical links between graph spectra and community structures, providing solid motivation for their approach.", "experimental_designs_or_analyses": "The experimental setup is comprehensive. The authors:\n\n1. Compare against multiple baseline approaches.\n\n2. Perform ablation studies to validate each key component.\n\n3. Analyze augmentation strength effects, evaluate robustness against structural perturbations.\n\n4. Provide qualitative analyses through visualization and functional community preservation. \n\nThe performance improvements are modest but consistent across tasks and evaluation settings. The robustness analysis is particularly valuable, showing that their approach maintains better performance under structural perturbations.", "relation_to_broader_scientific_literature": "This paper builds upon both graph contrastive learning and protein-specific approaches. However, the comparison with equivariant graph neural networks and protein foundation models could be more extensive. Recent works like ESM-2 [r1], EquiformerV2 [r2], and other models have shown strong performance on protein-related tasks but are not thoroughly compared against.\n\n[r1] Lin et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 2023.\n\n[r2] Liao et al. EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations. ICLR 2024.", "essential_references_not_discussed": "[r1] Kempen et al. Fast and accurate protein structure search with Foldseek. Nature biotechnology 2024.\n\n[r2] Zhou et al. Uni-mol: A universal 3d molecular representation learning framework. ICLR 2023.\n\n[r3] Huang et al. Protein 3D Graph Structure Learning for Robust Structure-Based Protein Property Prediction. AAAI 2024.  \n\n[r4] Lin et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 2023.\n\n[r5] Wang et al. S‐PLM: Structure‐Aware Protein Language Model via Contrastive Learning Between Sequence and Structure. Advanced Science 2025.\n\n[r6] Liao et al. EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations. ICLR 2024.", "other_strengths_and_weaknesses": "The paper would benefit from deeper comparison with more recent models.", "other_comments_or_suggestions": "Figure 3, Page 5, 'SSI view' -> '3-PSI view'", "questions_for_authors": "1. How does the approach perform when using predicted protein structures (e.g., from AlphaFold) rather than experimental structures? How sensitive is the 3-PSI method to the initial quality of protein structures? Would lower-resolution structures significantly impact performance? This would significantly expand the applicability to proteins without solved structures.\n\n2. Considering the complexity of edge add/drop, can the proposed GCL approaches easily be adopted to more complex encoders?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jEq1CVmVjC", "reviewer": "Reviewer_ZGC5", "summary": "This paper improves on the Graph Contrastive Learning (GCL) by introducing two graph augmentation techniques: Functional Community Invariance (FCI) and 3D Protein Structure Invariance (3-PSI). These augmentation techniques are designed to preserve the functional and structural integrity of proteins. The authors used end-to-end training on 4 datasets for classification tasks, integrating contrastive learning loss with classification errors. Experimental results show improvements in classification accuracy and F-1 score, and ablation studies showcase effective preservation of protein structures.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We are grateful to reviewer ZGC5 for insightful reviews.**\n***\n> **__Q1__** *Did not describe the GNN encoder. In the main text, the authors...in Appendix C4...why GNN is chosen.*\n\n**A1** We sincerely apologize for lack of clarity. We employ GNNs because GNNs can flexibly integrate a protein’s topological and geometric information. Additionally, nodes and edges in the graph can incorporate physicochemical information such as residue distances; then we implement EdgeGCN which fuses edge features into the message-passing framework.\nFor Fan et al's work, we employ their hierarchical pooling strategy rather than the encoder. When encoding protein graphs, the hierarchical pooling aggregates nodes layer by layer without losing critical structural information, thereby enabling the GNNs to summarize protein structure at a higher level. Thus, we fuse these components as our encoder. We will provide details in the main text.\n***\n> **__Q2__** *The model...end-to-end fashion...& Such training lead to bias...& a probable way is to train...*\n\n**A2** We'd like to clarify that our goal is to learn effective protein representations. To this end, we adopt supervised contrastive learning to incorporate label information, which further facilitates representation learning on top of the contrastive framework. Following your suggestion, we conducted additional experiments under a **pure self-supervised** setting. The results show competitive performance, but incorporating supervision further improves results across datasets.\n\n||EC|GO-BP|GO-MF|GO-CC|FOLD-Fold|FOLD-Super.|FOLD-Family|Reaction|\n|-|-|-|-|-|-|-|-|-|\n|pure self-supervised|0.863|0.443|0.655|0.467|56.9|78.0|99.4|87.2|\n|+ supervised|0.885|0.461|0.662|0.484|59.8|81.3|99.7|89.0|\n***\n> **__Q3__** *To demonstrate the effectiveness...use the benchmark models and compare.*\n\n**A3** We additionally experimented with two models: CDConv and ProNet with the following results:\n \n||EC|GO-BP|GO-MF|GO-CC|FOLD-fold|FOLD-Super.|FOLD-Family|Reaction|\n|-|-|-|-|-|-|-|-|-|\n|CDConv|87.9(+0.9)|0.445(-0.005)|0.664(+0.012)|0.479(+0.004)|58.1(+1.2)|81.6(+4.9)|99.8(+0.3)|87.8(-0.8)|\n|Pronet|-|-|-|-|51.8(-0.9)|72.5(+2.3)|99.5(+0.2)|87.1(+0.7)|\n\nwhere +/- shows performance improvement/reduction.\nAfter applying our augmentation strategy, performance improved across the majority of tasks, showing its effectiveness.\n***\n> **__Q4__** *How different λ impact model performance.*\n\n**A4**  We provide an ablation study as follows:\n\n|λ|EC|GO-BP|GO-MF|GO-CC|FOLD-Fold|FOLD-Super.|FOLD-Family|Reaction|\n|-|-|-|-|-|-|-|-|-|\n|0.2|0.882|0.451|0.656|0.467|56.5|80.2|**99.8**|86.2|\n|0.6|0.879|0.443|0.658|0.479|58.2|80.7| 99.6|87.5|\n|1.0|**0.885**|**0.461**|**0.662**|0.484|58.9|**81.3**|99.7|**89.0**|\n|1.4|0.880|0.457|0.649|**0.489**|**59.1**|80.6|99.7|87.8|\n|1.8|0.877|0.448|0.653|0.475|58.5|80.2|99.8|88.3|\n\nThe results show that performance fluctuates slightly across different λ values, with λ=1 shows better performances on majority of metrics. It indicates model is not highly sensitive to λ, suggesting an ease of optimization.\n***\n> **__Q5__** *$L_{\\mathrm{FCL}}$ is defined but $L_{\\mathrm{3-PSI}}$ is not clearly defined.*\n\n**A5** 3-PSI is used solely for data augmentation based on protein geometry and does not introduce learnable parameters, hence no standalone loss is required. Moreover, the loss defined in Eq.~(11), $L_{\\text{GCL}}^{(3\\text{-PSI})}$, specifically represents the GCL loss calculated using 3-PSI augmentation. \n***\n> **__Q6__** *It is interesting to...ligand binding affinity*\n\n**A6** Thanks for your insightful suggestion. We briefly present the results with our best setting on ligand binding affinity task on PDBbind dataset. \nWe select a representative baseline [1] for comparison. \n\n||RMSE↓|Pearson↑|Spearman↑|RMSE↓|Pearson↑|Spearman↑|\n|-|-|-|-|-|-|-|\n|Holoprot-Superpixel|1.491|0.491|0.482|1.416|0.724|0.715|\n|FCI+3PSI-alpha|1.462|0.515|0.510|1.383|0.753|0.749|\n\nResults show that our model achieves competitive performance. The first three metrics are evaluated at **30%** sequence identity, while the last three are at **60%**.\n***\n> **__Q7__** *The spectral decomposition...high computational cost & The scalability...not provided.*\n\n**A7** The original spectral decomposition has a time complexity of $O(n^3)$ and it is reduced to $O(n^2K)$ by Lanczos algorithm [2], where $n$ is the average number of residues per graph and $K$ is the number of selected eigenvalues. The probability matrix for FCI only needs to be precomputed once for each task. The majority of the training time is attributed to the GNN encoder. Thus, our method is scalable. The running time is detailed in A2 of Reviewer UCwy.\n***\n**Reference**\n\n[1] Multi-Scale Representation Learning on Proteins\n\n[2] The Lanczos Algorithm with Selective Orthogonalization\n***\n__If your concerns have been addressed, could you kindly raise the score? We greatly appreciate your comments and support.__", "claims_and_evidence": "The authors claim that that existing augmentation techniques could lead to incorrect protein structure or disrupt protein functionalities. Such claim is valid and is supported by the design of FCI and 3-PSI augmentation techniques. FCI preserves functional community by controlling spectral changes and incorporating chemical similarity, and 3-PSI preserves the secondary and tertiary structures through controlled rotations of dihedral angles and secondary structures.", "methods_and_evaluation_criteria": "The design of the augmentation techniques make sense, as they are rooted in biological principles to ensure that the augmentations are meaningful and realistic. However, the authors did not describe the GNN encoder used in detail, simply pointing to the work of Fan et al. (2023). The model was also trained on an end-to-end fashion for each classification task, which means the learnt representation would change for each downstream task. I have two concerns/suggestions: 1) To fully demonstrate the effectiveness of the proposed augmentation techniques, the authors could use the benchmark models (eg. ProNet, CDConv, GraphCL, GearNet, etc.) and compare performance before and after the augmentation methods are used. 2) The model is trained with equal weight to classification loss and CL loss (λ=1). There is no ablation study on how different λ impact model performance. 3) To demonstrate robustness of the learnt representation, a probable way is to train the GCL model using the proposed augmentation methods and use the learnt representation on downstream tasks. \nRegarding evaluation, the proposed metrics (accuracy and F-1 score) is suitable for classification tasks.", "theoretical_claims": "Yes, the equations and theorems in the main text are correct. One question, L_FCL is defined but L_(3-PSI) is not clearly defined.", "experimental_designs_or_analyses": "The authors used 4 datasets for classification tasks only. The model performance and ablation studies on these datasets are valid. However, since the protein structure and functionality is preserved, it is interesting to investigate the model performance on dataset such as ligand binding affinity (eg. PDBbind).", "relation_to_broader_scientific_literature": "The manuscript points out one important limitation in the literature, that the graph augmentation is solely based on topological features, ignoring the intrinsic biological properties of proteins. As one of the most important large molecules to study, how GNN models can be tailored to effectively learn protein representation is an important problem. The proposed augmentation techniques are carefully designed following biological principles and should have broader impact to future work.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strength:\n* The manuscript provides a novel approach to integrate domain-specific knowledge in contrastive learning framework.\n* The paper is clearly written and well-structured.\n\nWeakness:\n* The spectral decomposition might involve high computational cost. The scalability of the proposed approach is not provided.\n* The sensitivity of model performance regarding some parameters (eg. λ) is not provided.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. Regarding the GNN encoder structure. In the main text, the authors mentioned that they follow the work of Fan et al. (2023), which is CDConv. However, in Appendix C4, the author mentioned using EdgeGCN with pooling mechanism from Fan’s work. The GNN architecture should be described in the main text, why it is chosen. \n\n2. The augmentation techniques are innovative and biological-aware. However, it would be possible that the augmentation technique works well with the chosen GNN architecture. The authors should eliminate such possibility by applying it on other GNN models, and showcase performance improvements despite the choice of backbone models. \n\n3. Why is the end-to-end training technique chosen for each classification task? Such training loss could lead to representation bias for each task, rather than learning a robust protein representation for multiple downstream tasks. \n\n4. How is the choice of λ in the loss function impact performance?\n\n5. What is the definition of L_(3-PSI)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q8e14paM1i", "reviewer": "Reviewer_UCwy", "summary": "This paper proposes a novel framework for protein representation learning by introducing two biologically informed graph-augmentation strategies for contrastive learning. Specifically, it combines:\n\n\t1.\tFunctional Community Invariance (FCI), which preserves crucial residue clusters (communities) involved in protein functionality when augmenting the 2D graph connectivity.\n\n\t2.\t3D Protein Structure Invariance (3-PSI), which perturbs three-dimensional protein conformations in a biologically plausible manner by manipulating backbone dihedral angles or rotating secondary structures (α-helices and β-sheets) without destroying essential structural motifs.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank reviewer UCwy for detailed reading and meaningful feedback.**\n***\n> **_Q1_** *Not against large-scale pretrained language models.*\n\n**A1** In Appendix F.2, we have provided comparisons with protein LMs like ESM-1b. For extended experiments, like comparison to ESM-2, you can refer to Reviewer s9oD, A1.\n***\n> **_Q2_** *Scalability and complexity analysis.*\n\n**A2** Time complexity for 3D augmentation is $O(n)$ for 3-PSI and $O(n^2)$ for FCI, as detailed in A3 in Reviewer s9oD. \nFor 3D augmentation, each residue is transformed by a 3×3 rotation matrix. The memory complexity scales as $O(Bn)$, where $B$ is batch size and $n$ is number of residues per sample. \nFor large dataset, we can focus augmentation on specific regions of interest (e.g., binding sites) rather than the entire protein to reduce the cost.\nHere we provide running time.\nFor 3-PSI, it costs 22.38s and 18.91s each epoch on 3PSI-Diag and 3PSI-Alpha on Fold task.\nRandom augmentation (Trad. 3D Aug. in paper) costs 3.56s.\nOverall, Diag and Alpha constitute about 18.0% and 15.3% of total training time (124s per epoch). Considering the performance improvements, the additional training time is a justifiable trade-off.\n***\n> **_Q3_** *Only sequences are available...& Only partial PDB data is available...*\n\n**A3** We provide possible solutions: for no available experimental structure or PDB data missing, our model can use a predicted structure generated by homology modeling tools such as SWISS-MODEL[1]; for structures that are uncertain with PDB data missing, we can select those from homology modeling tools with higher confidence scores, perform augmentation on each, and then fuse the resulting features. \nAs for incomplete PDB data, 3-PSI can also work when a moderate amount of residue remains and you can refer to experiment results in Reviewer s9oD, A2 (incomplete input PDB). If the primary PDB data is unavailable, we need to employ homology modeling tools to reconstruct the structure.\n***\n> **_Q4_** *Several hyperparameters might be non-trivial to tune optimally & guidelines might needed*\n\n**A4**\nFor hyperparameter selection, we follow a common paradigm. Values too small may not provide sufficient augmentation of protein conformation, while larger values may distort it.\nFor example, when selecting ε for edge dropping, it is recommended to start with a small value (e.g., 0.1) and gradually adjust it up to 0.4 to avoid damaging the graph structure. For dense graphs, a slightly higher ε can be used, vice versa. \nFor ϵ=0.2 and rotation number=2 can serve as a useful starting point for tuning on new datasets.\n***\n> **_Q5_** *Giving an example on potential practical use...why preserving functional communities...*\n\n**A5** We have provided an experiment of ligand binding affinity task and you can refer to Reviewer ZGC5, A6. Our method achieves competitive performance as it preserves functional communities and protein 3D structure in augmentation, enabling model to learn meaningful representation like structural information.\n***\n> **_Q6_** *Some readers may appreciate a reference of other community detection algorithms...*\n\n**A6** We will discuss these methods in the main text. Spectral methods can effectively capture edge-level interactions, providing a strong theoretical foundation for maintaining invariance. Therefore, we chose it. We plan to extend FCI to support other related methods.\n***\n> **_Q7_** *Adding lines about E(3)-equivariance...3-PSI and equivariant GNNs are complementary.*\n\n**A7** 3-PSI focuses on augmenting proteins while preserving key structural properties to generate more biologically reasonable augmentations. In contrast, SE(3) emphasizes the invariance of network inputs to global rotations and translations, ensuring the model's robustness. They are complementary.\nWe will discuss them.\n***\n> **_Q8_** *Is torsion angle-based side-chain similarity robust? Does external chemical used...?*\n\n**A8** Previous work [2] shows that side‐chain torsion angles are effective features of residues. Also, it represents chemical properties of residues. Thus, we compute side‐chain similarity by torsion angles. While our implementation does not incorporate external chemical knowledge or force‐field parameters, we are exploring them to further enhance its robustness, particularly for chemically diverse and modified amino acids.\n***\n> **_Q9_** *Data Splitting Protocols*\n\n**A9** Following previous work [3], we used SCOP 1.75, filtered with a pairwise identity of less than 95%, and employed a three-level homology reduction dataset as training. \n***\n**Reference**\n\n[1] Generative models for graph-based protein design\n\n[2] Learning Hierarchical Protein Representations via Complete 3D Graph Networks\n\n[3] DeepSF: deep convolutional neural network for mapping protein sequences to folds\n***\n**If your concerns have been addressed, could you kindly consider raising your score? We greatly appreciate your comments and support.**", "claims_and_evidence": "1. Claim: Biology-Aware Augmentations Improve Representation Quality\n\t•\tThe authors propose Functional Community Invariance (FCI) to preserve biologically relevant residue clusters. They measure how often pockets (functional communities) remain intact after augmentation and compare results with standard or random graph augmentations. This directly supports the claim that their method better retains key functional components.\n\t•\tThey introduce 3D Protein Structure Invariance (3-PSI) to avoid unrealistic disruptions to three-dimensional protein structures (e.g., by preserving secondary structure integrity or peptide planes). \n\n\n2. Claim: The Proposed Method Achieves Superior Performance Across Multiple Tasks\n\t•\tThe authors conduct evaluations on four tasks: Protein Fold Classification, Enzyme Reaction Classification, Gene Ontology (GO) Prediction, and Enzyme Commission (EC) Number Prediction. Within fold classification and GO, they further break down sub-tasks (e.g., family, superfamily, etc.), generating a broad evidence base.\n\t•\tResults Tables show consistent gains over both:\n\t•\t2D-only GCL baselines (e.g., GraphCL, CI-GCL), and\n\t•\t3D-augmentation baselines (e.g., random coordinate perturbations, homology modeling).\n\t•\tAblation studies (e.g., using FCI alone, 3-PSI alone, or their combination) confirm that combining the two invariance strategies yields the best improvements.\n\n3. Claim: The Proposed Approach is Robust to Noisy Structural Perturbations\n\t•\tIn the robustness experiment, the authors randomly rotate backbone segments in test proteins, simulating environmental or measurement-induced structural variations. They then plot model accuracy/Fmax at proportions from 10% to 50% of residues disturbed.\n\t•\tResults show that their method’s performance degrades more slowly than traditional 3D augmentations and certain baselines, indicating robustness to structural noise.\n\n4. Claim: FCI and 3-PSI are Synergistic\n\t•\tAblation: FCI or 3-PSI alone improve performance, but together they outperform either approach individually. The results tables consistently show the “FCI + 3-PSI” variant is highest-performing.\n\n\nThe paper’s primary claims are largely well-supported by clear empirical results, multiple tasks, ablation studies, and robust analyses. Minor open questions around comparisons to cutting-edge pretrained models or large-scale data do not undermine the validity of the demonstrated improvements within the scope of graph-based protein learning.", "methods_and_evaluation_criteria": "Method: The paper’s primary contributions—Functional Community Invariance (FCI) and 3D Protein Structure Invariance (3-PSI)—directly address known gaps in protein contrastive learning. Most existing approaches use 2D topology manipulations that may disrupt functionally critical residues or rely on overly simplistic 3D perturbations that distort key structural motifs. By incorporating community preservation (via spectral constraints and side-chain similarity) and 3D backbone constraints (via dihedral angles or secondary structure rotations), the paper’s techniques remain biologically plausible. This stronger biological grounding is exactly what protein graph methods need to capture higher fidelity embeddings.\n\nBenchmark Datasets: The authors run experiments on four major tasks: fold classification, enzyme reaction classification, gene ontology (GO) prediction, and enzyme commission (EC) number prediction. These are widely used benchmarks in structure-based protein modeling research. For fold classification, they even subdivide it (family, superfamily, fold), which is aligned with SCOP classification and demonstrates how well the method distinguishes proteins at varying levels of structural/sequence similarity. GO and EC tasks capture functional annotation challenges, which are biologically relevant real-world scenarios.\n\nPotential Improvements: One area that might strengthen the paper is a scalability discussion—e.g., how the computational overhead of generating more sophisticated 3D augmentations compares with simpler random approaches when moving to large datasets. Another is comparing or combining these graph-based methods with large-scale protein language models. However, the existing baselines and metrics remain representative of graph-based contrastive learning and typical protein structure tasks.", "theoretical_claims": "The paper’s main theoretical arguments center on characterizing how individual edge perturbations affect the graph spectrum. Specifically:\nTheorem 4.1 (Bounds of Spectral Changes)\nClaim: When a single edge is flipped (added or removed), the change in the eigenvalues of the normalized Laplacian can be upper-bounded and lower-bounded by expressions involving the spectral embeddings (the eigenvectors) of the unperturbed graph.\nAssessment: This result aligns with known techniques in spectral graph theory, where the Davis–Kahan or Weyl inequalities often provide bounds on eigenvalue/eigenvector perturbations. The text’s statement that spectral changes are related to the l_2 distance between the two node eigenvector embeddings also matches typical graph-spectral intuitions: if two nodes lie “far apart” in the spectral embedding, flipping edges between them tends to yield a larger spectral impact. No immediate inconsistencies stand out; the proof outline (in references to standard matrix perturbation results) appears coherent and consistent with existing spectral bounding approaches.\n\nLemma 4.2 (Weighted Graphs)\nClaim: The absolute spectral change when dropping an edge of weight ￼ can be upper-bounded by the product of \\(\\lvert w_{ij}\\rvert\\) and terms involving eigenvectors/eigenvalues.\nAssessment: This follows a similar logic to Theorem 4.1 but accounts for weighted adjacency. The bound’s proportionality to \\(\\lvert w_{ij}\\rvert\\) is intuitively correct—heavier edges should create more significant perturbations when flipped. This is reminiscent of standard matrix perturbation arguments. The statement seems plausible, and no obvious red flags arise in the bounding steps as described. The full formal proofs are outlined briefly in the text (and presumably with additional detail in an appendix). From the information provided and familiarity with spectral graph theory, these claims appear mathematically consistent and do not contradict well-known eigenvalue/eigenvector perturbation theories. The derivations rely on expansions of the difference between Laplacian eigenvalues and standard bounding approaches (e.g., norm-based inequalities). While the paper does not reproduce the entire step-by-step derivation in the main body (likely for brevity), the reasoning is logically sound on inspection.", "experimental_designs_or_analyses": "1. Benchmark Tasks and Datasets:\nThe authors evaluate on four widely used tasks in protein representation learning (fold classification, enzyme reaction classification, GO term prediction, and EC number prediction), each with recognized standard datasets.\n\n2. Comparisons with Baselines:\nBaseline Variety: The authors compare with multiple 2D topology-based GCL approaches (GraphCL, GCS, etc.) and 3D structure-based augmentations (random coordinate perturbation, homology modeling tools). Hyperparameters: The paper mentions the range of augmentation strengths (e.g., the proportion of edges changed in 2D, or number of dihedral/secondary structure rotations in 3D). They present ablation curves that show how these hyperparameters affect performance. This is helpful to confirm that gains are robust to different augmentation intensities.\n\n3. Ablation Studies:\nFCI vs. 3-PSI vs. Combined: The experiments systematically compare using Functional Community Invariance (FCI) alone, 3D Protein Structure Invariance (3-PSI) alone, or combining them. Observing that the combined approach generally yields the best performance supports the synergy claim. Sensitivity to Augmentation Strength: Varying the fraction of edges dropped/added (2D) or the number of secondary-structure rotations/dihedral perturbations (3D) is a direct test of how robust the method is to over- or under-augmentation. This analysis appears well-motivated and thorough.\n\n4. Robustness Checks:\nStructural Perturbation at Test Time: The authors apply random rotations to subsets of residues at test time, mimicking protein conformation changes or partial misalignments. They track performance as the proportion of rotated residues increases. This design is a realistic measure of robustness, given that experimental structural data can be noisy.\n\n5. Qualitative Analyses:\nFunctional Community Preservation: They measure the fraction of intact protein pockets under each augmentation method. This is a direct reflection of how well the augmentation strategy avoids disrupting crucial functional sites.\n\n6. Potential Limitations or Omissions\n\t1.\tComparison to Protein Language Models: The authors do not evaluate the proposed method in conjunction with or against large-scale pretrained language models (like ESM or ProtT5). While not necessarily invalidating the experiment design, it’s a potential extension.\n\t2.\tScalability: They do not provide time or memory complexity analyses for 3D augmentations, though that would be helpful for large-scale applications.", "relation_to_broader_scientific_literature": "1.\tProtein Representation Learning\n\t•\tOver the past few years, protein representation learning has emerged as a critical area at the intersection of machine learning, structural bioinformatics, and computational biology. Classic approaches include:\n\t•\tSequence-based methods (e.g., language-model–type architectures such as ESM, ProtT5, and ProtBERT), focusing on large-scale protein sequence corpora. These methods often capture powerful evolutionary context but may overlook critical 3D structural cues.\n\t•\tStructure-based methods (e.g., GNNs like GVP, GearNet, and IEConv), which treat proteins as graphs of residues to incorporate geometric constraints. These approaches have proven effective in tasks such as fold classification, function prediction, and protein–protein interaction studies.\n\n2.\tGraph Contrastive Learning (GCL)\n\t•\tIn the broader machine learning literature, contrastive learning has become a standard technique for self-supervised representation learning on images, text, and graphs. Recent advances on graphs (e.g., GraphCL, GCS, Auto-GCL, and CI-GCL) demonstrate how data augmentations can push GNNs toward more generalizable embeddings. However, these methods often rely on simplistic graph transformations (like random edge dropping or adding) and rarely leverage specific biological or domain constraints.\n\n3.\tBiologically Grounded Augmentations\n\t•\tIn protein research, random manipulations of graph structures can cause the loss of essential structural or functional information (e.g., removing edges around catalytic sites). The idea of building biology-aware augmentations partly echoes earlier efforts in structure-based drug design and protein–ligand modeling, where physically plausible perturbations are used to sample conformational states.\n\t•\tThis paper’s Functional Community Invariance (FCI) connects with the notion of residue interaction networks and community detection (common in network analyses of protein structures), ensuring that biologically relevant clusters (pockets, communities, etc.) remain intact. Prior literature on “residue co-evolution” and “community detection in protein structures” has highlighted the importance of pockets and functional domains for accurate protein functional interpretation.", "essential_references_not_discussed": "1. Rives A, Meier J, Sercu T, Goyal S, Lin Z, Liu J, Guo D, Ott M, Zitnick CL, Ma J, Fergus R. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences. 2021 Apr 13;118(15):e2016239118.\n2. Jumper J, Evans R, Pritzel A, Green T, Figurnov M, Ronneberger O, Tunyasuvunakool K, Bates R, Žídek A, Potapenko A, Bridgland A. Highly accurate protein structure prediction with AlphaFold. nature. 2021 Aug;596(7873):583-9.\n3. Batzner S, Musaelian A, Sun L, Geiger M, Mailoa JP, Kornbluth M, Molinari N, Smidt TE, Kozinsky B. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications. 2022 May 4;13(1):2453.", "other_strengths_and_weaknesses": "Additional Strengths\n\n\t1.\tOriginality through Domain-Driven Adaptations\n\t•\tThe paper demonstrates a creative fusion of well-known spectral graph theory methods with domain-specific protein structural constraints. While graph contrastive learning itself is not new, the authors’ emphasis on preserving functional communities and realistic 3D transformations elevates the approach beyond standard random augmentations.\n\t•\tBy explicitly integrating knowledge about side-chain similarity, functional pockets, dihedral angles, and secondary structure motifs, they bring a strong biological grounding to the contrastive learning framework, which is less common in generic GCL.\n\t2.\tClarity in Presentation\n\t•\tThe paper largely maintains clear structuring: definitions, methodology (with sub-sections for 2D vs. 3D invariance), experiment descriptions, and ablation studies.\n\t•\tMany figures and tables (e.g., pocket-preservation, UMAP visualizations, sensitivity analyses) effectively illustrate key points. This visual clarity helps readers grasp the impact of each augmentation technique.\n\t3.\tRobust Methodological Design\n\t•\tThe authors run multiple ablations (e.g., FCI alone, 3-PSI alone, combined) and vary hyperparameters to show where the model sees performance gains. Their robustness experiment specifically showcases how the model handles noisy protein structures—mirroring real-world conditions where data can be imperfect or proteins adopt alternative conformations.\n\nAdditional Weaknesses\n\n\t1.\tLimited Discussion of Scalability\n\t•\tPerforming complex 3D transformations (e.g., systematically rotating dihedral angles, secondary structures) might increase computational overhead compared to simpler node/edge manipulations. The paper does not deeply analyze how this scales to massive protein databases, which can be a concern for large-scale or high-throughput applications.\n\t2.\tComparisons with Non-Graph Methods\n\t•\tThe paper positions itself within the ecosystem of graph-based approaches and does a thorough job comparing with various GCL baselines. However, given the explosive growth of large language models for proteins (e.g., ESM), a more explicit recognition or mention of these non-graph methods would help situate the work in the broader landscape of protein representation learning. Without such discussion, some readers might overlook how these approaches could complement or compete with purely sequence-based techniques.\n\t3.\tDependency on High-Quality 3D Structures\n\t•\tWhile the approach benefits from realistic structural perturbations, it inherently assumes the existence of reasonably accurate 3D protein models (or experimental structures). In cases where only sequences are available (or structural data is uncertain), the method may not be directly applicable. The paper does not discuss how to handle partial or noisy structural data beyond artificially simulating noise.\n\t4.\tHyperparameter Tuning Complexity\n\t•\tThe method includes several hyperparameters for controlling augmentation strength (fraction of edges dropped, number of secondary structures to rotate, etc.), which might be non-trivial to tune optimally. Although the authors present ablation curves, there is still a risk that real-world users would need extensive trial and error to find the “sweet spot” for a specific application.", "other_comments_or_suggestions": "1.\tHighlighting Real-World Applications\n\t•\tThe paper demonstrates strong performance on widely used benchmarks, but giving a short paragraph or example on potential practical use cases (e.g., ligand-binding site analysis, enzyme engineering) could clarify why preserving functional communities and realistic 3D geometry might be game-changing in a practical setting.\n\n2.\tDiscussion on Parameter Sensitivity\n\t•\tThere is a helpful analysis of varying augmentation strengths. It might be useful to explicitly mention some rough guidelines for how a user might select these parameters in practice (e.g., typical values of ϵ for edge dropping or typical rotation angles for dihedral modifications).\n\t\n3.\tFurther Exploration of Community Detection\n\t•\tThe FCI portion uses spectral theory to preserve functional communities. Some readers may appreciate a reference or brief mention of other community detection algorithms (like Louvain or Infomap) or the well-known “normalized cuts” approach to show possible variations or confirm that spectral clustering is a robust choice.\n\n4.\tComparisons to Equivariant Models\n\t•\tAlthough the authors mention 3D geometry-based GNN approaches, adding one or two lines about E(3)-equivariance or SE(3)-equivariance could clarify that 3-PSI and equivariant GNNs are complementary. They solve different problems (augmentations vs. architecture design) but both preserve geometry in different ways.", "questions_for_authors": "1. Question: When only partial PDB data is available or some residues are missing coordinates (e.g., unresolved loop regions in cryo-EM structures), how do you apply 3-PSI augmentations? Is there a fallback procedure or do you skip those proteins?\n\n2. Computational Overhead of 3D Augmentations: How does the runtime for dihedral/secondary structure rotations compare with simpler graph augmentations in large-scale experiments? Did you measure any significant slowdown?\n\n3. Criteria for Side-Chain Similarity: In Functional Community Invariance (FCI), side-chain similarity is computed via torsion angles. How robust is this measure for chemically diverse or modified amino acids (e.g., PTMs), and do you incorporate any external chemical knowledge or force-field parameters (e.g., from Amber or CHARMM)?\n\n4. Data Splitting Protocols for Fold Classification: Which SCOP (or SCOPe) version did you use, and what was your sequence identity threshold to ensure minimal overlap between training and test sets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes two augmentation strategies for protein graphs: (i) Functional Community Invariance, and (ii) 3D Protein Structure Invariance. The two augmentation strategies are integrated into a Graph Contrastive Learning framework for protein representation learning.

Overall, all reviewers are positive about this work. They generally agree that the proposed augmentations are novel and well motivated. They also agree that the paper is well written and structured. Still, several concerns were raised which revolved around: missing baselines, the scalability of the proposed approach and the lack of complexity analysis, the large number of hyperparameters that need to be tuned, and the dependence of the approach on the availability of 3D structures. Most of these concerns were addressed in the rebuttal, and all four reviewers are leaning towards acceptance. Hence, I recommend the acceptance of this paper. The authors should make sure to update the manuscript to include the reviewers' suggestions.
Decision: Accept (poster)