Towards Stable Representations for Protein Interface Prediction

Authors: ziqi  gao
zijing  liu
yu  li
jia  li
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 09:24:55

Abstract: The knowledge of protein interactions is crucial but challenging for drug discovery applications. This work focuses on protein interface prediction, which aims to determine whether a pair of residues from different proteins interact. Existing data-driven methods have made significant progress in effectively learning protein structures. Nevertheless, they overlook the conformational changes (i.e., flexibility) within proteins upon binding, leading to poor generalization ability. In this paper, we regard the protein flexibility as an attack on the trained model and aim to defend against it for improved generalization. To fulfill this purpose, we propose ATProt, an adversarial training framework for protein representations to robustly defend against the attack of protein flexibility. ATProt can theoretically guarantee protein representation stability under complicated protein flexibility. Experiments on various benchmarks demonstrate that ATProt consistently improves the performance for protein interface prediction. Moreover, our method demonstrates broad applicability, performing the best even when provided with testing structures from structure prediction models like ESMFold and AlphaFold2.

Main paper: 
 related work: 
 Protein interface prediction. Protein interface prediction (PIP), a well-studied problem, focuses on determining whether there is an interaction between amino acids from two different proteins.Recently, a series of methods based on protein [8, 19, 17, 20, 14] or amino acid representation [39, 46] learning have achieved significant success. NEA [15] pioneers the use of protein graphs to address PIP, where protein structure information is represented and aggregated, followed by the dense layers. SASNet [43] considers embedding the hierarchical structures of proteins, integrating atomic and amino acid information into a 4D-grid data, and employs 3D CNN for learning. To further enhance performance, more fine-grained structure information modeling, specifically surface geometry [46, 39] , is introduced to effectively learn amino acid representations. Existing methods have effectively represented proteins from various perspectives of protein information. However, we have observed that protein flexibility, which is overlooked by most methods, poses significant performance bottlenecks for them. We focus on this key issue of mitigating the bound-unbound mismatch in protein structures to improve model generalization.Modelling protein flexibility. Recently, pioneer works in biology confirm that protein-protein interaction (PPI) conforms to the "induced fit" theory [27, 38] . Specifically, proteins undergo structure changes due to residue-level forces, and they adjust structures to achieve the best binding state. More importantly, proteins with PPI typically undergo larger structure changes at the interface compared to non-interface regions [10, 11, 52, 12] , which will exacerbate the generalization challenge of the PIP task. Modelling flexibility directly is challenging, whether using traditional computational or deep learning (DL) approaches. Traditional methods often rely on finding the lowest energy state [42, 51] or introducing an induced fit model (specifically, the elastic network model) [12, 3] to guide structure deformations. The optimization space in these methods is vast, making them very time-consuming. DL-based methods [10] struggle to achieve satisfactory accuracy in learning the distribution mapping of bound-unbound states due to limited training data (i.e., 253 complexes in the DB5.5). As directly predicting bound structures is challenging, in the context of the PIP task, we choose to eliminate the influence of different versions of the same protein structure on the task. 
preliminaries: 
 In this section, (1) we present the definition of the protein interface prediction (PIP). Then, (2) we verify the importance of representation stability through empirical and mathematical views. Finally, (3) we investigate to ensure protein representation stability within the adversarial training framework.Problem definition. We are given as input two proteins P 1 and P 2 , consisting of M and N residues, respectively. The proteins are represented as their residue sequences and 3D structures, which are composed of α-carbon atom locations of all residues. The goal of the PIP is to classify all possible pairs of residues from separate proteins. More formally, the set of data is {(P 1 i , P 2 j ), y ij } 1≤i≤M,1≤j≤N , where P 1 i represents the i-th residue in protein P 1 and y ij ∈ {0, 1}. We verify the importance of stable protein representations from both empirical and theoretical perspectives. For clarity, we use the notations X b 1 , X b 2 to represent the native bound structures of proteins P 1 and P 2 , while X t 1 , X t 2 represent their structures used for testing. After using a protein graph encoder, we have their representations, denoted asH b 1 , H b 2 , H t 1 , H t 2 .We denote the protein representation perturbation as ∥δ 1 ∥ p + ∥δ 1 ∥ p , whereδ 1 = H t 1 -H b 1 , δ 2 = H t 2 -H b 2 .From an empirical perspective, in Figure 3 , we quantify the test results of the NEA method [15] under flexibility. We gradually increase the structure change of test samples and calculate the representation perturbation. We note that the test performance is negatively correlated with the representation perturbation caused by flexibility. Moreover, testing with bound structures yields the best results.From a theoretical perspective, we draw the consistent conclusion that stable representations lead to improved PIP results. Following [15] , we model PIP as a pairwise classification problem. Specifically, we concatenate the i-th and j-th rows of H t 1 and H t 2 into a vector embedding, which is then sent to the PIP classifier f . In this case, the following proposition describes the impact of representation perturbations on PIP results. Proposition 3.1. For two proteins with M and N residues respectively, the classification results obtained using bound and unbound structures are the same. This is true ifN ∥δ 1 ∥ p p + M ∥δ 2 ∥ p p < A(f, p), where A(f, p) can be a constant depending only on the PIP classifier f and the norm ∥•∥ p .We detail A(f, p) and prove Proposition 3.1 in Appendix B.1. This proposition tells that stable protein representations (i.e., smaller ∥δ 1 ∥ p and ∥δ 2 ∥ p ) under flexibility are necessary for achieving high performance. Thus, to effectively address the structure mismatch in PIP, an intuitive idea is to perform adversarial training for protein representation learning.Here, we introduce the concept of adversarial training (AT) [1, 6, 18, 5, 7] and establish a connection between it and the stability of protein representations. We consider a classification task with a given dataset D = {(x i , y i )} n i=1 , consisting of K classes. We assume that the entire prediction pipeline includes a representation model (e.g., encoder) and a classifier. The concept of adversarial training (AT) requires the entire pipeline to perform well not only on D but also on the worst-case distribution near D, as determined by a specific distance metric. More concretely, the AT that we primarily focus on in this paper is the ℓ p -robustness. For a given p value and a finite ϵ > 0, AT aims to train a pipeline that can correctly classifies (x + δ, y) for any ∥δ∥ p ≤ ϵ, where (x, y) belongs to D. Among all AT methods, Lipschitz neural networks belong to a common and effective category. Specifically, an encoder function is considered to have Lipschitz continuity if a slight perturbation to the input of the encoder does not significantly change its output.Formally, the definition of Lipschitz continuity is given by: Definition 3.1. (Lipschitz continuity in adversarial training) An encoder function ENC is said to be C-Lipschitz continuous w.r.t. norm ∥•∥ if for any two versions of inputsEQUATIONLipschitz continuity explains the requirements of AT for a general representation learning encoder. In the context of protein graph representation, this can be modified to become the definition below. Definition 3.2. (Lipschitz continuity for protein representations) A protein representation encoder Ψ(•) has C-Lipschitz continuity w.r.t. norm ∥•∥ if for any two versions of structure inputs X t , X b and the invariant residue feature input F ,EQUATIONAs an expressive representation, graph structured data is widely used for representing input proteins [17, 22, 19] , with residues acting as nodes and physical interactions as edges. Let G = (V, E) be an undirected graph with nodes V = {1, ..., N }, edges E ⊂ V 2 , graph signal F ∈ R N ×d and a graph shift operator L ∈ R N ×N (i.e., node connectivity). We consider any variant of the spectral GNN (e.g., GCN [26, 30] , ChebNets [13] , BWGNN [41, 40] ) that follows the concept of learning filter coefficients for graph convolution. By constructing the filter h(L) := K k=0 θ k L k (θ k are learnable parameters), the protein graph representation can be defined asH = K k=0 θ k L k F := h(L)F .Here, we extend the Definition 3.2 to the scenario of using GNN models. To achieve this, we assume L is perturbed to become L due to the protein flexibility, and introduce the key factor (GNN filter stability constant C h ), for achieving GNN-based stable protein representations. Definition 3.3. (GNN filter stability constant) Given a graph spectral filter h : [0, 2] → [0, 1], it is defined as Lipschitz with constant C h > 0 if for any pair of points λ 1 , λ 2 : EQUATION) Node Feature Edge Feature 𝑣 ! ! 𝑣 " ! 𝑣 # ! 𝑣 $ ! 𝑣 " " 𝑣 ! " 𝑣 $ " Stability-regularized GNN ⨀ 𝜽 𝟎 𝜽 𝟏 𝜽 𝟐 𝜽 𝑲 Bernstein Basis Learnable Coefficients Filter ℎ($) 𝑏 ! " 𝑏 # " 𝑏 $ " … 𝑏 " " Graph Convolution … … Cross Attention Cross Attention 𝐻 " 𝐻 # PIP Classifier ℒ %&' ℒ % = max &'!,#,…" 𝜃 & -𝜃 &*# Protein 1 Protein 2 𝐌𝐀𝐒 ≤ 𝐾ℒ %EQUATIONwhere A(Ψ) is a constant determined by the model (e.g., layer number and feature dimension).Based on Eq. 3 and Eq. 4, the lower bound of the left term in Eq. 4 is solely determined by the maximum of C h (denoted as C * h ). To be more intuitive, C * h is equal to the maximum absolute slope (MAS) of the filter h (the straightforward proof is in the Appendix B.2). Therefore, we conclude the core idea for designing our ATProt framework as follows:We can enhance the stability of protein representations by decreasing the MAS value of h. 
method: 
 Overview. We propose ATProt, an end-to-end framework (illustrated in Fig. 4 ) to boost PIP from a view of representation stability. Specifically, our model inputs two proteins whose structures can be provided in various sources (e.g., native bound, native unbound, AlphaFold2, ESMFold). ATProt incorporates a protein graph representation encoder and a targeted differentiable regularization scheme to theoretically guarantee the representation stability. Our ATProt framework can be implemented with at least four protein graph encoders, and we provide specific examples of these cases. Lastly, for the PIP task, we apply a cross-attention module to facilitate communication between the protein representations and use a simple linear classifier for final prediction.Protein representation. We represent a protein as an undirected weighted graph G = (V, E). Each node v i ∈ V representing one residue has a d-dimensional feature vector F i ∈ R d (i.e., residue type) and a 3D coordinate X i ∈ R 3 (i.e., the α-carbon atom location). Edges E = {(i, j)} are constructed with a k-nearest-neighbor (k-NN) graph using Euclidean distance. To distinguish the edges, we follow [17] to construct the SE(3)-invariant edge features {e i,j : ∀(i, j) ∈ E}.The goal of ATProt. According to Definition 3.2, we aim to achieve representation stability, which means that the perturbation in the representation caused by protein flexibility is constrained. In addition, it is also important to ensure the commonly encapsulated SE(3)-invariance, which means that the representation of the protein is not affected by its rotation or translation. Formally, we represent a single protein P consisting of N residues, with its residue-level feature matrix F ∈ R N ×d and residue-level structure X ∈ R N ×3 . We wish our model Ψ(•) to ensure the following property.Given Z, H = Ψ(F , X); Z ′ , H ′ = Ψ(F , QX + g + ∆X), we have H -H ′ ≤ C • ∥∆X∥ , (C-Lipschitz continuity) ∀Q ∈ SO(3), ∀g ∈ R 3 , ∀∆X ∈ R 3×N , ∃C ∈ R.(5)The Fourier transform is a powerful tool in the representation of both structured [50, 26, 49, 28] and unstructured data [9, 21, 31] . Our main focus is on employing spectral graph encoders for representing protein graphs. These encoders adhere to the principle of utilizing a graph spectral filter h(λ)= K k=0 θ k b k (λ)to learn effective protein representations. Here, {b k } K k=0 is the predefined filter basis and {θ k } K k=0 is the learnable parameters. For graph convolution, the filter h(•) will be applied to the whole Laplacian matrix L, which is calculated from the protein structure data. Specifically, to construct L, we first apply a Multi-Layer Perceptron (MLP) to reduce the dimensionality of E to 1, and incorporate the results into the edge weight matrix W :EQUATIONThen the Laplacian matrix L is obtained byL = I -D -1/2 W D -1/2, where D is the degree matrix, i.e., D = diag( j W 1,j , ..., j W N,j ).The protein graph representation can be defined as:EQUATIONwhere the result of h(L) represents the graph spectral response.In this paper, we introduce four types of top graph encoders (i.e., Simple GCN [49] , Chebynet [13] , Low-pass filter [34] , and BernNet [23] ) along with their corresponding stability regularizations. Case 4.1. (Simple GCN encoder [49] ) The Simple GCN encoder (SGC) utilizes a spectral filter in the monomial function form:EQUATION) Since the spectrum λ lies in [-1, 1], it is clear that the maximum absolute slope (MAS) that h(λ) can reach is K. Therefore, the stability regularization of the SGC encoder does not involve any loss function. We can directly constrain the size of the order K. Case 4.2. (Chebynet encoder [13] ) The Chebynet encoder utilizes a spectral filter in the polynomial function form:EQUATION)The stability regularization of Chebynet can be implemented by the loss function L S = K k=1 k|λ k |. Case 4.3. (Low-pass filter encoder [34] ) The Low-pass filter (LPF) encoder utilizes a spectral filter in the low-pass filter function form:EQUATION)The stability regularization of LPF can be implemented by the loss function L S = θ. Case 4.4. (BernNet encoder [23] ) The BernNet encoder utilizes Bernstein basis, the state-of-the-art graph spectral basis, to construct the graph spectral filter:EQUATION)According to Theorem 3.1, the BernNet encoder easily satisfies the representation C-Lipstchiz continuity in Eq. 4 provided that the filter h(λ) always has a finite MAS. However, in Eq. 11, the analytical relationship between MAS and {θ k } K k=0 is intractable, suggesting the difficulty of constraining C with a gradient descent manner. Thus, our next goal is to discover a differentiable method for constraining the minimum of C in the training process.The stability of the BernNet is still unclear in these four cases, prompting us to investigate its regularization form. To the best of our knowledge, it is the first investigation of the Lipstchiz continuity for Bernstein-based spectral filters.For clarity, we rewrite the Eq. 4 asH -H ′ ≤ C • ∥∆X∥ = C h • A(Ψ) • ∥∆X∥ • ∥F ∥,where, referring to Eq. 3, A(Ψ) is determined by the model architecture hyperparameters and remains constant. We say the overall model Ψ is of C-Lipstchiz continuity and the learned spectral filterh(λ) = K k=0 θ k b K k (λ) is of C h -Lipstchiz continuity.We aim to constrain the minimum of constant C h (denoted as C * h ) with {θ k } K k=0 by discovering the underlying relationship between them. Finally, we propose an auxiliary differentiable regularization of {θ k } K k=0 to constrain C * h to a controllable bound, for any ∆X. Theorem 4.1. Given an arbitrary polynomial function f (λ) on λ ∈ [0, 2] and suppose its K-order Bernstein polynomial is denoted asEQUATIONWe introduce Theorem 4.1, which describes the stability relationship between the filter h(λ) and an auxiliary function f (λ). It tells that with Berstein basis, the MSA of h(λ) will never exceed that of the auxiliary function f (λ). Due to fEQUATIONλ) can be any 2-D curve passing through all points of {( 2k K , θ k )} K k=0 . Therefore, the MAS of f (λ) is analytically tractable, and we can subsequently provide the bounds for the MAS of h(λ). We have the following proposition, which is accompanied by a detailed derivation in Appendix B.3. Proposition 4.1. Suppose h(λ) is approximated with Bernstein basis, i.e., h(λ)= K k=0 θ k b K k (λ).h , it can be upper bounded byEQUATION)To summarize, Bernstein basis applied to our ATProt guarantees a finite MAS, and more importantly, we can further enhance the stability of the BernNet encoder with the following objective:EQUATION)Given proteins P 1 , P 2 with their initial feature F 1 , F 2 and coordinates X 1 , X 2 , ATProt produces their stable representations under respective structure perturbations.EQUATION)We apply a cross-attention layer (shown in Appendix D) to enable communication between proteins and obtain their final representations H ′ 1 ,H ′ 2 . Next, we aim to predict whether pairs of inter-protein residues belong to the interface, which involves performing pairwise binary classification. Concretely, for each training pair of proteins, we have a set of10N I labeled pairs (([H ′ 1 ] i , [H ′ 2 ] i ), y i ) 10N I i=1, where y i ∈ {0, 1}, N I is the number of positive residue pairs and 9N I negative ones are downsampled. We take the element-wise product of two residue representations and feed it to another MLP with the Sigmoid function to compute the probability p i . Weighted cross-entropy loss is used for training.EQUATION)5 Experiments 
experimental setup: 
 Datasets and processing. We evaluate our method on the complexes from Docking Benchmark 5.5 (DB5.5) [45] , a gold standard dataset with high-quality, and Database of Interacting Protein Structures (DIPS) [43] , which collects 41,876 complexes mined from PDB [4] . DB5.5 only contains 253 complex structures manually curated by domain experts, which cover both native unbound and bound structures. In comparison, DIPS has a significantly larger data size, but it only includes bound structures of proteins. The two datasets are randomly divided into training, validation, and testing sets with the following sizes: 203/25/25 (DB5.5) and 39,937/974/965 (DIPS).For both datasets, we test using various versions of protein structures as inputs for PIP. To accomplish this, we prepared three versions of the testing set for DB5.5, including native unbound structures, structures produced by AlphaFold2, and structures produced by ESMFold. We use Native-Bound, Native-Unbound, ESMFold, AlphaFold2 to denote these four settings respectively.As for DIPS, since it does not have native unbound structures, we only used ESMFold to prepare unbound structure inputs for its testing set. AlphaFold2 is not considered due to its high computational cost for the entire testing set of DIPS.Baselines. We compare our ATProt method with state-of-the-art conventional machine learning method BIPSPI [36] , the CNN-based methods Siamese Atomic Surfacelet Network (SASNet) [43] , Diffusion-Convolutional Neural Networks (DCNN) [2] , differentiable molecular surface interaction fingerprinting (dMaSIF) [39] , and a set of GNN-based methods Deep Tensor Neural Networks (DTNN) [37] , and NEA [15] .Setup and metrics. We consider three experimental setups: (1) performing training and testing both on the DB5.5; (2) performing training and testing both on the DIPS; and (3) performing pre-training on the DIPS, fine-tuning on the DB5.5 and testing on the DB5.5. For our proposed ATProt method, we consider graph encoders of Simple GCN, Chebynet, Low-pass filter and BernNet (denoted as ATProt-SGC, ATProt-Cheby, ATProt-LPF, and ATProt-Bern, respectively).For each complex in the testing set, assuming that the two proteins have M and N residues, respectively, we test all its M × N binary classification samples and calculate the Area Under the ROC Curve (AUC) value. Following [15] , we report the median AUC score (MedAUC) across all complexes as the final evaluation metric.Table 1 : Training and testing on the DB5.5. Mean and standard deviation values of the MedAUC scores of all baselines, computed from three random seeds. The best performance is in bold and the second best one is underlined. 'SR' means the proposed stable regularization L S .BIPSPI [36] 0.937 (0.008) 0.911 (0.017) 0.896 (0.008) 0.887 (0.013) SASNet [43] 0.902 (0.007) 0.876 (0.017) 0.887 (0.025) 0.881 (0.020) dMaSIF [39] 0.928 (0.005) 0.912 (0.009) 0.906 (0.003) 0.892 (0.012) DTNN [37] 0.912 (0.005) 0.886 (0.007) 0.883 (0.010) 0.878 (0.021) NEA [15] 0 Results. Table 1 , 2, and Table 3 (shown in Appendix) show the model performance for PIP. We find that our method is competitive and outperforms the majority of baseline methods with native bound structures. Under this Native-Bound setting, although ATProt is slightly less effective than BIPSPI, it demonstrates the ability to learn sufficiently powerful protein representations (especially with BernNet and Chebynet). Notably, all the baselines fail significantly when using non-bound structures as inputs, whereas our method maintains similar performance compared to that under Native-Bound.As can be seen from the results of 'ATProt-BernNet w/o SR', the robustness to structure perturbation is attributed to the proposed SR (stable regularization) strategy. Overall, the experiments on DB5.5 demonstrate that our method does not rely on native bound or unbound structure data for inference, but can directly utilize structure prediction software to achieve satisfactory PIP results. From Table 2 , 3, we note that although DIPS has a larger data scale than DB5.5, it is difficult to achieve better results. However, pre-training on DIPS can improve most of the testing results on DB5.5.In summary, all three tables show consistent performance degradation for all baselines with the non-bound structures. In contrast, our method can defend against this bound-unbound mismatch and achieve performance similar to, or even better than that with Native-Bound.Benefits of representation stability. Figure 5 shows the dimensionality reduction visualization of residue representations with t-SNE [44] . We observe that ATProt leads to a "clustering" effect in the representations, which is the result of stability regularization. Importantly, this ultimately generates clearer classification boundary in visualization and quantification (i.e., the Silhouette Score [35] ).  
conclusion: 
 In this paper, we present ATProt, an end-to-end learning framework for protein interface prediction (PIP). By highlighting the importance of protein representation stability for the PIP task, we tailor the stability regularization for four types of spectral graph encoders, theoretically ensuring that the ATProt framework exhibits Lipschitz continuity properties. Our method demonstrates competitive empirical performance compared to leading deep learning-based baselines, especially when dealing with significant bound-unbound structure gaps. Importantly, ATProt demonstrates the ability to perform inference in native-structure free scenarios.Limitations. To present a clear and focused issue, we do not incorporate adversarial training-based classifiers into our framework. However, it is expected that their inclusion could further enhance the performance of ATProt, as they have shown promising results in the field of computer vision.According to the Lipschitz constant of f , we haveEQUATIONConsidering ( 3) and ( 4), we have the initial conclusion as followsEQUATIONThen we consider the complete residue pair representation set {xEQUATIONWe denote the perturbations on protein representations H 1 and H 2 as δ 1 and δ 2 , respectively. δ (ij) is actually the concatenation of the i-th row of δ 1 and the i-th row of δ 2 , i.e., δ (ij) = AGG(δ 1 , δ 2 ).Thus the left term of ( 6) is equal to N ∥δ 1 ∥ p p + M ∥δ 2 ∥ p p . Finally, we conclude the proof: Proof. We prove this based on the definition of graph Lipschitz filter, which is continuously differentiable due to the polynomial approximation.EQUATIONLet C h be the Lipschitz constant of h, and let u(λ) = |h ′ (λ)|. We want to show that C h is minimized when C h = sup λ u(λ).Suppose there exist λ 1 and λ 2 such that |λ 1 -λ 2 | > 0 and |h(λ 1 ) -h(λ 2 )| > C h |λ 1 -λ 2 |. Then, by the mean value theorem, there exists λ 3 between λ 1 and λ 2 such that |h ′ (λ 3 )| > C h . But this contradicts the assumption that C h is the Lipschitz constant of h. Therefore, C h must be greater than or equal to sup λ u(λ).To show that C h is minimized when C h = sup λ u(λ), suppose there exists a Lipschitz constant C ′ h such that C ′ h < sup λ u(λ). Then, for any λ 1 and λ 2 , we have|h(λ 1 ) -h(λ 2 )| ≤ C ′ h |λ 1 -λ 2 | < sup λ u(λ)|λ 1 -λ 2 | ≤ |h ′ (λ 3 )||λ 1 -λ 2 | (24)where λ 3 is some point between λ 1 and λ 2 . But this contradicts the definition of u(λ) as the maximum of |h ′ (λ)|. Therefore, C h must be equal to sup λ u(λ).Thus, we have shown that C h is minimized when C h = sup λ u(λ), as desired. For readability, we denote the minimum of C h as C * h and sup λ u(λ) as the MAS (maximum absolute slope) of h, respectively. 
: 
 Protein-protein interactions are important for understanding biological processes, and for the design of novel therapies [48, 16] and drugs [47] . The protein interface refers to the surface region of a protein where the interaction occurs. It therefore holds the key to revealing the specific interaction mechanism and understanding protein functions. In this work, we tackle the problem of protein Figure 2 : The impact of flexibility on results with the DB5.5 dataset [45] . (A) The testing results of two baselines (SASNet [43] , NEA [15] ) and our method. 'B-U' represents the popular formulation, i.e., training with bound structures and testing with unbound ones. 'B-B' refers to the formulation where both training and testing are conducted with bound structures. (B) Loss trends for three method.interface prediction (PIP) (shown in Figure 1 (A)): predicting whether two residues, each from an individual protein, interact with each other, given the separate structures of two proteins.For years, data-driven methods based on deep learning (DL) have made significant progress in response to this critical task by effectively learning protein structures using geometric graph neural networks (GNN) [15, 33] , 3D CNN [43] , etc. Limited by the difficulty in accessing protein structure data, they typically follow a formulation of training on bound (after binding) structures and testing on unbound (before binding) ones, as depicted in Figure 1 (B). For training, large-scale datasets like the Database of Interacting Protein Structures (DIPS) [43] typically consist of only bound structures, which are directly extracted from the PDB database [4] . In contrast, in the practical inference scenario, the model cannot access the bound structures but can only be provided with unbound ones [43, 15, 33] .In this paper, we empirically find that this training (bound)-testing (unbound) formulation leaves significant room for performance improvement. In Figure 2 , exploratory experiments show that prevailing PIP methods are sensitive to flexibility. Utilizing the bound version structures for testing can greatly boost their performance. Based on these findings, we aim to answer the question in this paper-how to handle the mismatch between bound and unbound structures for PIP?Since it is usually impractical to access the protein bound structures for testing, the most intuitive solution is to explicitly learn the mapping relationship from unbound to bound structures of proteins. However, this is challenging due to the following two factors: (1) The amount of pairwise unbound and bound structure data for proteins is extremely limited [10] (to our knowledge, only DB5.5 [45] ).(2) A protein's bound structure is not unique and depends on its binding partner, so diverse training data is necessary. To address this issue, we take a different route. We consider any potentially complicated flexibility in a protein as an attack [29, 24] , which can harm the testing performance of a model trained on bound structures. Therefore, our core idea is to enable protein representations with adversarial robustness, which can defend against the attacks of protein flexibility. In simple terms, for a protein with both unbound and bound versions, the model outputs similar (stable) representations.In this work, we take an important step forward in mitigating the impact of flexibility on PIP. We propose ATProt, an end-to-end adversarial training (AT) framework for protein representations, to effectively defend against protein flexibility in PIP. Inspired by the recent protein graph representation methods [17, 53, 15] , our model comprises graph-based feature extractors (encoders) for protein graphs. Our ATProt framework does not require computationally expensive data augmentation and can be smoothly applied to most existing protein graph encoders. Specifically, we implement differentiable AT regularizations for various protein representation encoders. Importantly, we introduce a novel and expressive graph encoder for protein representations and propose its theoretical regularization form for the first time. ATProt can produce stable representations for the same protein with different structure versions (e.g., bound, unbound, and model-generated ones). Extensive experiments on several protein interaction benchmarks verify that our ATProt method consistently outperforms advanced PIP methods. The results demonstrate the effectiveness of the AT regularization. Furthermore, ATProt maintains excellent performance even when tested with structures generated by AlphaFold2 [25] and ESMFold [32] , allowing for user-friendly inference without the need for native structures. 
 
 Appendix: We start the analysis with the unique stability-preservation property of the Bernstein basis shown in Theorem 4.1. Simply put, during Bernstein polynomial approximation, the outcome polynomial h(λ) is at least as stable as the target function f (λ). This is crucial as it tells that the MAS of an exact function f (λ) can always serve as an upper bound for C * h . The relationship between f (λ) and {θ k } K k=0 . Setting the polynomial order to K, our model learns a set of K + 1 weights {θ k } K k=0 that acts as the coefficients for polynomial approximation, such that f ( 2k K ) = θ k , for all k ∈ [0, K] ∩ Z. Put differently, any 2-D curve passing through all points of (0/K, θ 0 ), (2/K, θ 1 ).., (2K/K, θ K ) can be regarded as a possible version of f (λ) for polynomial approximation. Consequently, given any set of {θ k } K k=0 , the stability of h(λ) is not worse than the most stable version among all possible f (λ).Formally, we naturally extend the Theorem 4.1 to the following proposition, which provides a theoretical upper bound for C * h of h(λ), directly using the learnable coefficients {θ k } K k=0 . As h is at least as stable as the most stable f , the MAS of h can be upper bounded by the minimum MAS among all possible f functions, which is infinitely close to the MAS of the broken line passing through points {(k/K, θ k )} K k=0 . Figure 6 helps to better understand Proposition 4.1. For example, let us set K to 1, any curve passing through (0, θ 0 ) and (2, θ 1 ) can be a version of f . Out of all possible f versions, the line segment directly connecting two points has the minimum MAS (i.e., |θ0-θ1|).We have introduced in the main text the relationship between f (λ) and {θ} K k=0 . Based on this, the proof of Proposition 3.2 is equivalent to proving the following lemma. Lemma B.1. Given K + 1 points (0, θ 0 ), ( 2K , θ 1 ), . . . , ( 2K K , θ K ), suppose there are infinitely many functions f (λ) that pass through these K + 1 points. The maximum absolute slope (MAS) of any version of a function f (λ) is not less than the MAS of the piecewise linear function passing through these K + 1 points.Proof. Let f broken (λ) be the piecewise linear function passing through the given K + 1 points. The slope of f broken (λ) between two consecutive points ( 2i K , θ i ) and ( 2(i+1) K , θ i+1 ) is given by:EQUATIONThe MAS of f broken (λ) is the maximum of the absolute values of these slopes: MAS broken = max 0≤i≤K-1EQUATIONNow, consider any function f (λ) that passes through the K + 1 points. Since f (λ) is differentiable, by the mean value theorem, for each interval 2i K , 2(i+1), there exists a point λ i such that:EQUATIONSince f (λ) passes through all the given points, we have:EQUATIONThus, the MAS of any function f (λ) satisfying the conditions is greater than or equal to the MAS of the piecewise linear function f broken (λ) passing through the points.Table 3 shows the results on DIPS. Kindly note that we train and test all methods both on the DIPS dataset. EQUATIONEQUATIONThe training process takes around 0.5 hours with 1 Nvidia 4090 GPUs with 24GB RAM. The hyper-parameters used in this paper are listed in the following table. 


Decision: The paper proposes ATProt, an adversarial training framework for protein interface prediction, designed to enhance model robustness against protein flexibility by treating it as an attack, with experiments showing improved performance across various benchmarks. Reviewers appreciated the novel approach of treating protein flexibility as an adversarial attack, the theoretical grounding and soundness of the framework, and its demonstrated robustness across different datasets. Reviewers criticized the lack of clarity in the problem formulation, the limited novelty in the model architecture, inadequate ablation studies, missing up-to-date baselines, and the need for more detailed experimental settings and visualizations to substantiate better the claims made in the paper. The authors addressed many of these issues during the discussion period, and subsequently, the reviewers unanimously recommended that the paper be accepted. Therefore, we accept the paper, but we urge the authors to carefully incorporate all the additional updates and results in the final revision.

Reviews: review: paper_summary: paper_summary: This paper considers the generalization issue caused by the conformational changes of two proteins before and after binding in the PIP task. The authors view protein flexibility as an attack on the model and aim to defend against it for better generalization. Hence, an adversarial training framework for protein representation is proposed, termed ATProt. ATProt is theoretically proven to ensure protein representation stability under complex protein flexibility. Experiments on the DB5.5 and DIPS datasets further validate the effectiveness of ATProt, especially when using unbound structures as input.
strength_weakness: strength_weakness: * Although the idea of ATProt is interesting and effective, the model's architecture adopts a combination of existing methods, BernNet and Cross-attention.
* The ablation experiment targeting 'SR' was conducted only within the 'ATProt-BearNet' combination.
* Lacking some visualization experiments. No examples are presented to show which residue pairs are predicted more accurately before and after incorporating ATProt.
* In Supplementary.E, some hyperparameters are missing, such as the number of epochs and the number of Cross-attention layers.
questions: questions: * What about the experimental results for the removal of 'SR' in the other three encoders? Clearly demonstrating the performance improvement brought by 'SR' in the other three encoders would better validate the effectiveness of the framework.
* Considering visualizing the changes in interface prediction before and after adding 'SR' could more intuitively validate the proposed framework..
* Showcasing the hyperparameter tuning process for “the weight of loss $L_{s}$ with BernNet” would be beneficial, as it is the core idea of ATProt.
* Providing the number of epochs and the number of cross-attention layers would help readers better reproduce the results.
limitations: limitations: The authors have thoughtfully addressed the limitations of their study, particularly highlighting the absence of adversarial training-based classifiers in their framework.
paper_summary: paper_summary: This paper first identifies a commonly overlooked issue in protein docking: protein flexibility, and proposes an improved method to address it. This approach utilizes an adversarial training framework to maximize Lipschitz continuity. Experimental results demonstrate the effectiveness of this method.
strength_weakness: strength_weakness: The writing in one part of the paper is not very clear. I hope the authors can adjust it. The paper discusses a method of adversarial training to ensure Lipschitz continuity, but the specific training objectives of adversarial training are not explicitly stated here. Adversarial training typically refers to having two competing objectives, which are contradictory. However, I did not see this aspect described in the paper.
questions: questions: None
limitations: limitations: Excessive emphasis on ensuring Lipschitz continuity may limit the expressive power of the network ($L_S$). The article is meaningful and interesting. However, excessive restriction on the network's expressive ability could potentially prevent this method from becoming the ultimate solution. Therefore, I believe this article does not reach the level of spotlight or oral presentation.
paper_summary: paper_summary: This work focuses on protein interface prediction, i.e., determining whether a pair of residues from different proteins interact. It notices the conformational change within the protein upon binding and regards the flexibility as an attack on the trained model. An adversarial training framework named ATProt is introduced to defend against this kind of attack with a theoretical guarantee. Experiments demonstrate consistent improvements. However, I am not completely certain about many unignorable points, such as the experimental setting, the missing of up-to-date baselines, and etc. I am absolutely fond of the idea of being robust to conformational changes and give a score between 4 and 5 (leave the decision to other reviewers and AC).
strength_weakness: strength_weakness: (1) The problem formulation is kind of confusing. The authors target classifying all possible pairs of residues from separate proteins. I suppose this may not be the regular version of protein interface prediction (PIP). The standard one would be to classify whether each residue on the ligand and receptor is on the interface or not. The PIP task in this paper is more challenging than the standard PIP task, and is very close to (but easier than) the protein-protein docking problem. Because once the pairwise interactions are recognized, the docking pose is nearly known. I recommend the authors reconsider the problem definition and formulate it more carefully. 

(2) No adequate ablation study. I believe a more proper ablation analysis is needed to showcase the contribution of each component. For instance, it would be interesting to see ATProt-SGC/CHEBY/LPF's performance without the stability regularization. 

(3) More baselines are required. I notice that all baselines are pretty old, containing NEA (2017), dMasif (2021), BIPSPI (2019), SASNet (2019), DTNN (2017), and DCNN (2016). It is not convincing enough to select these old-school algorithms as competitors. There has been a surge of interest in developing DL models for interface identification. Please see below. Moreover, it is necessary to also evaluate ESM-Fold and Alphafold-2 (if possibly, AF3) in this problem, because they can also immediately output the complex structures (so the interface is given) from sequences. These structure prediction models completely do not suffer from structural perturbation. 

[A] Epitope 3D: a machine learning method for conformational b-cell epitope prediction. Brief in Bioinformatics, 2022. 

[B] ScanNet: an interpretable geometric deep learning model for structure-based protein binding site prediction, Nature Methods, 2022. 

[C] PesTo: parameter-free geometric deep learning for accurate prediction of protein binding interfaces. Nature Communications, 2023. 

[D] SEMA-2.0: Antigen b-cell conformational epitope prediction using deep transfer learning. Frontiers in Immunology, 2022 

[E] Pair-EGRETt: enhancing the prediction of protein-protein interaction sites through graph attention networks and protein language models. ICLR Workshop, 2023. 

(4) I really appreciate the authors' efforts in attacking the structure change's bad effect in DL models. However, there is an unavoidable issue that I want to discuss with the authors and AC. If our concentration is to bridge the gap between training (bound) and test (native/unbound) datasets, why does not the author directly train the model on unbound or predicted structures? To be specific, we can use AF2/ESMFold to obtain unbound structures and leverage them as the training data. Moreover, it is also feasible to rely on Rosetta, PyMol, OpenMM, etc. to relax the bound structures with some force fields and approximate the unbound circumstance. In the AI4S community, I believe the solution is more flexible and we should consider more simple but effective methods to solve the problem. There have already been some works [A] that mention this representation shift between real structures and predicted structures. 

[A] Protein 3D Graph Structure Learning for Robust Structure-Based Protein Property Prediction. AAAI 2024.
questions: questions: (1) In Figure 3, how do the authors increase the structure change of test samples? I agree with Equ. 5 to use \Delta X to represent the structure perturbation. However, I suppose there are some constraint in this \Delta X. Notably, the protein flexibility is not entirely random. The atoms' movement need to follow some physical rules. See [A] for illustration. 

[A] Fractional denoising for 3d molecular pre-training. ICML 2023. 

(2) As far as I am concerned, dMasif is a surface point cloud-based mechanism, and it only outputs the prediction of surface points instead of residues. How do the authors make the comparison compatible? 

(3) The conformational change is fancy without doubt and many studies have been trying to do relevant analysis. For instance, ProtMD [A] pretrains the DL model on MD data to learn the protein flexibility with continuous time domain. Can the authors do an additional experiment to see whether this category of pretraining is effective to mitigate the negative effect of conformation change for PIP? 

[A] Pre-training Protein Geometric Models via Molecular Dynamics Trajectories. Advanced Science, 2022. 


(4) It looks weird to me that the pretraining on DIPS brings a negative impact on dMasif but positive impacts for all the others. After pretraining, dMasif drops from 0.928 to 0.922 for native-bound, from 0.912 to 0.903 for native-unbound, etc. Can the authors explain this counterintuitive phenomenon?
limitations: limitations: Yes, the limitation is addressed.

Comments: title: Thank you for raising the score and helping us improve.
Update
comment: Dear AC,

I have reviewed the authors' response. I have no further questions and stand by my original assessment. I believe the paper has reached the standard for acceptance, but it does not meet the criteria for spotlight or oral presentation.
Dear RLcQ,

Have you had the chance to read the authors’ rebuttal, and does it address the concerns you raised, and has it influenced your evaluation of the paper?

Best regards, AC
Dear kBXn,

Have you had the chance to read the authors’ rebuttal, and does it address the concerns you raised, and has it influenced your evaluation of the paper?

Best regards,
AC
Thank you for raising the score. We greatly appreciate your recognition and assistance of our work. 

Based on your constructive suggestions, we commit to including all the added baselines in the final version manuscript. Additionally, we will introduce a discussion on AF2 data augmentation (providing experimental results) and the use of MD for relaxation.

***About dMaSIF***

We previously misunderstood your question. Please allow us to explain again. Regarding dMaSIF, we did perform additional processing to make it compatible with the PIP task. In the dMaSIF paper, the predicted elements of the interface are **not residues but points** simulated by the dMaSIF method. This can be found in its source code and paper, for example: *"For interaction prediction, we compute dot products between the feature vectors of both proteins to use them as interaction scores between pairs of points."* This setting is obviously different from the mainstream approach of PIP, as it calculates the link probability between points rather than residues. Additionally, when we reproduced dMaSIF under its default setting, the result was only around 0.85, indicating significantly lower performance.

**Therefore, we made modifications by referencing the processing manner in the ScanNet paper.**

In the ScanNet paper, to allow for a fair comparison with MaSIF-site, they converted MaSIF's predictions from the surface point level to the residue level. This is described in the ```Baseline methods/masif-site``` section of ScanNet. The approach is to first assign each point to the nearest atom and the corresponding residue. Then for each residue, take the highest probability among all its points as the final result. However, things are slightly different because we need to calculate the residue-residue link probability rather than classify each residue in the binding site prediction task. 

**Therefore, we made some minor adjustments:**

We first assign each point to the nearest atom and the corresponding residue. Then for each **residue-residue pair**, take the highest probability among all its point pairs as the final result. For an illustrative example, let's assume two proteins have 1 and 2 residues respectively, labeled as {1} and {1',2'}. dMaSIF generates a total of 6 surface points for these 3 residues. The correspondence between points and residues is: {a,b$\rightarrow$1}, {c,d$\rightarrow$1'}, {e,f$\rightarrow$2'}. dMaSIF can outputs the linking probabilities between points: {a,c:0.2},{a,d:0.4},{a,e:0.6},{a,f:0.8},{b,c:0.1},{b,d:0.3},{b,e:0.5},{b,f:0.7}. Finally, for residue pair {1,1'}, we pick the highest value among {a,c:0.2},{a,d:0.4},{b,c:0.1},{b,d:0.3} (i.e., 0.4) as the result. For residue pair {1,2'}, we pick the highest value among {a,e:0.6},{a,f:0.8},{b,e:0.5},{b,f:0.7} (i.e., 0.8) as the result. We tried alternative approaches for ```pick the highest value```, such as taking the average of all relevant probabilities or the average of the top k probabilities, but they were not as effective as simply using the highest value.

Thus, we can reformulate dMaSIF into our PIP setting. Under this setting, its performance can match or even surpass that of several baselines.
Thanks for the authors' efforts in answering my question. I am glad to see that ATProt-Bern outperforms all essential baselines and stands the No. 1 in this binding site prediction task. I am also happy to know that my suggestion helps the author clarify more clearly the concepts and definition of interface v.s. binding site.  

I understand that a conference paper cannot totally solve the binding site prediction task and there is still a long way to go, e.g., considering more training data (like using Alphafold Database), using MD simulations to relax the structures, designing domain adaptation and generalization algorithms to adapt the methods to predicted structures, etc. Based on this fact, I believe ATProt-Bern is worthy of publication and should be regarded as an adequate contribution to this important challenge. 

I have raised my score to 6 and recommend the authors to incorporate the updates to the final revisions, such as the ablation studies, and the new baseline (PesTo, ProtMD).

Last but not least, I am still confused by this reply: "[Q2. About dMaSIF] The main point of dMaSIF is to generate point cloud representations. However, the dMaSIF paper also contributes a geometric convolutional model, enabling it to address structure-related downstream tasks. Importantly, the PIP task is implemented in the dMaSIF paper."

As far as I am concerned, the quasi-geodesic convolutional model is performed in the point cloud instead of the structure. My question is how the author mapped the prediction of dMaSIF on the surface point to the residues?
Also, please add the ablation study and baselines (e.g. EBMdock) as reviewer RNrt mentioned.
Let me add that I believe the method proposed by the author is a form of regularization rather than an adversarial training approach. If I am correct, please adjust the relevant writing accordingly.