Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
Authors: Shiwei Liu
Tian Zhu
Milong Ren
Yu Chungong
Dongbo Bu
Haicang Zhang
Venue: NeurIPS 2023 poster (2023)
Keywords: Riemannian Diffusion Probabilistic Model
Mutation
Protein-protein binding
TL;DR: We predict mutational effects on protein-protein binding using a side-chain diffusion probabilistic model.

Abstract: Many crucial biological processes rely on networks of protein-protein interactions. Predicting the effect of amino acid mutations on protein-protein binding is important in protein engineering, including therapeutic discovery. However, the scarcity of annotated experimental data on binding energy poses a significant challenge for developing computational approaches, particularly deep learning-based methods. In this work, we propose SidechainDiff, a novel representation learning-based approach that leverages unlabelled experimental protein structures. SidechainDiff utilizes a Riemannian diffusion model to learn the generative process of side-chain conformations and can also give the structural context representations of mutations on the protein-protein interface. Leveraging the learned representations, we achieve state-of-the-art performance in predicting the mutational effects on protein-protein binding. Furthermore, SidechainDiff is the first diffusion-based generative model for side-chains, distinguishing it from prior efforts that have predominantly focused on the generation of protein backbone structures.

Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
Shiwei L
Tian Zhu
Milong Ren
Chungong Yu!.2
Dongho B22.
Haicang Zhang1.21
1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100080, China. University Chinese Academy of Sciences, Beijing 100080, China. 3. Central China Research Institute for Artificial Intelligence Technologies, Zhengzhou 450046, China
Abstract
Many crucial biological processes rely on networks of protein-protee interactions. Predicting the effect of amino acid mutations on protein-protein binding is vital in protein engineering and therapeutic discovery. However, the scarcity of annotated experimental data on binding energy poses significant challenge for developing computational approaches, particularly deep learning based methods. In this work, we propose SidechainDiff, a representation learning based approach that leverages unlabelled experimental protein structures. SidechainDiff utilizes Riemannian diffusion model to learn the generative process of side-chain conformations and can also give the structural context representations of mutations on the protein- protein interface. Leveraging the learned representations, art performance in predicting the mutational effects on protein-proters binding. Furthermore, SidechainDiff is the first diffusion-based generative model for side- chains, distinguishing it from prior efforts that have predominantly focused on generating protein backbone structures.
achieve state-of-the-
1 Introduction
Many crucial biological processes, including membrane transport and cell signaling, are likely regulated by intricate networks protein-protein interactions rather than single proteins acting independently [Calakos et al. 1994 Jones and Thorton 1996 Levskaya al. 2009 Schwikowski ct al. 2000]. One representative example is the interaction between the spike protein of the SARS- CoV-2 virus and its receptor protein ACE2 on the surface of human cells, which is crucial for the virus to invade target cells [Lan et al. 2020). Meanwhile, specific human antibodies, an essential type of protein in the immune system, prevent the virus entry by competitively binding to the spike protein [Shan et al. 2022).
In protein engineering and therapeutic discovery, it is crucial step induce amino acid mutations on the protein-prott protein interface modulate binding affinity O'Nee and Hoess 1995]. For example, to enhance the efficacy of an antibody against virus, one practical approach is to introduce amino acid mutations and filter the resulting antibody sequences to increase binding affinity and specificity to the target viral protein [Msson et al. 2021]. However, the variant space that can be explored
'These authors contributed equally to this work. Corresponneecc should be addressed to H.Z. (zhanghacaan@@ict.cc
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

using experimental assays is very limited, and developing an effective high- high-throughput screening can require a significant experimental effort.
Computational methods have been developed to predict the mutational effect on binding affinityy measured by the change in binding free energy (i.e., AAG). Traditional computational approaches mostly used physical energy features, such as van der Waals and solvation energy. in combination
with statistical models predict AAG [Schymkowitz et al. 2005 Meireles et al. 2010 Alford
et al. 2017 Barlow et al. 2018). The limited model capacity and bias inherent in luman-engineered
features in these methods hinder their ability to characterize the complex mutational landscape of
binding energy. Despite recent breakthroughs in protein modeling with deep learning (Jumper et al.
2021 Back et al. [02T] Watson et al., 2023]. developing deep learning- based models to predict binding remains challenging due to the scarcity of labeled
mutational effects on protein- 2019
experimental data (aakauskaiiě et al.
Recent studies have investigated various self-supervised learning strategies on protein structures and
sequences [Liu et al. 202T Meier al. 2021 Hsu et al. 2022 Luo al. 2023 ease the data scarcity issue. Among them, GcoPPI Liu et al. 2021 and RDE Luo et al. 2023 have focused on optimizing protein side-chain conformations during pre-training, as the side-chain conformations on the protein-protein interface play a critical role in determining binding energy. In protein-protein interactions, the side-chain conformations of amino acids the interface may exhibit significant variations when comparing the same receptor with different ligands. These side-chain conformations can be more effectively characterized using probability density. Notably, RDE explores a flow- based model to estimate the uncertainty in side-chain conformations and leverages the learned representations achieve sfate-of-the-ard performance in predicting AAG. However, flow-based models possess inherent limitations as they require specialized architectures to construct accurate bijective transformations in probability density [Papamakarios et al. 2021] which results increased costs associated with model design and implementation.
To address the above limitations, we propose SidechainDiff, representation learning framework for the protein-protein interfaces. It employs a Riemannian diffusion model to learn the generative process of side-chain conformations and the representation of the structural context of amino acids. To the best of our knowledge, SidechainDiff is the first diffusion probabilistic model for side-chain modeling, whereas previous methods have only focused on protein backbone structures. Furthermore, we leverage the learned representations and neural networks to predict AAG.
2 Related Work
2.1 Protein side-chain conformation prediction
Accurate side-chain modeling is essential in understanding the biological functions of proteins. There two primary categories in predicting protein side-chain conformations: end-to-end full-atom structure prediction methods and side-chain packing. AlphaFold and RoseTTAFold (Jumper et al.] Baek et al., 202T are two representative methods that simultaneously generate side-chain conformations and backbone structures. In scenarios like structure refinement and protein design, side-packing becomes pivotal. The objective is to determine the conformations of protein side-chains while having their backbone structures already defined. Traditional methods, including Rosetta [Leman al., 2020 and SCWRL4 (Krivov al. 2009 operate by minimizing the energy function across detinnd rotamer library. Recent methods for side-chain packing often employ deep learning models, such as 3D convolution networks and graph attention networks [McPartlon and Xu
202T
2023 Misiura et al. 2022 Xu et al. 2022
Our model distinguishes itself from previous methods for side-chain modelling in two key aspects. First, it's capable of generating a distribution of side-chain conformations rather than a single conformation. Second, we emphasize side-chain modeling specifically for mutation sites within the protein-protein interface, leveraging the known structural context of these sites.
2.2 Prediction of mutational effects on protein-protein binding
Methods for predicting AAG can be broadly classified into three categories: biophysical methods, statistical methods, and deep learning-basee approaches. Biophysical methods offer a robust means of clucidating the molecular mechanisms governing protein- protein interactions and the impact of

---

mutations on these interactions [Schymkowitz et al. 2005 Alford et al. 2017 Steinbrecher et al. 2017]. These methods directly integrate information on protein structures and key biophysical properties, such as solvent accessibility, electrostatic potential, and hydrogen bonding patterns. Statistical methods tailor statistical models for the protein properties such as evolutionary conservation
and geometric characteristics [[Li et al. 2016
Geng et al. 2019 Zhang et al. 2020
Deep learning-based methods can be categorized into sequence-based methods and structure-base methods. Sequence-based methods primarily either focus on the evolutionary history, multiple sequence alignments (MSAs) in most cases, of the target protein (Hopf et al. 2017 Riesselman et 2018 Frazer et al., 202T] or act as protein language models (PLMs) trained on large amounts of protein sequences MMeerr et al. 2021 Notin al. 2022). Structure- based methods can be categorized into end-to-end methods and pre- training- based methods. The and-to-end methods extract features from protein complexes and directly train neural network model on them [Shan et al., 2022]. To mitigate overfitting caused by data sparsity, an alternative approach is to learn representations by pre-training feature extraction network on unlabeled structures [Liu et al., 2021 Luo et al.. 2023)
Among them, RDE-Network (Luo et al. 2026 utilizes normalizing flows in torus space to estimate the density of amino acid side-chain conformations and leverages the learned representation to predict AAG.
2.3 Diffusion probabilistic models
Diffusion Probabilistic Models (DPMs) are generative models to transform a sample from a tractable noise distribution towards a desired data distribution with a gradual denoising process [Sohl-Dickstein
et al.
2013 Kingma et al., 202T Dhariwal and Nichol, 202T]. DPMs have achieved impressive
results in generating various data types, including images [Ho et al. 2020 Nichol and Dhariwal
2021].
waveforms, and discrete data like text (Hoogeboom et al. 2021 DPMs-based autoencoders
have also been proposed to facilitate representation learning for image data PPreehakul et al.) 2022
Zhang al.
2022
Inspired by these progresses, DPMs have also been explored in protein modeling, including de novo
protein design [Watson et al. 2023 Ingraham et al. Yim et al., [023], motif-scaffolding [Trippe et al.
2023]. and molecular dynamics Arts et al. 2023 Wu and Li 2023 While the existing methods utilize DPMs for generating protein backbones, research gap remains in modeling side-chain conformations, which play a critical role in protein-proted binding. The studies by Ding et al.] [2022) and Corso et al. 2023] employ diffusion models for generating torsion angles in the context of small molecular design. In contrast, our research is centered on modeling the torsional angles of protein side-chains. Furthermore, our approach distinguishes itself from their models in terms of how construct the diffusion process.
3 Methods
Our methods have two main components. We first propose SidechainDiff (Figure D.. which is a diffusion probabilistic model for protein side-chains and can also give the structural context representations of mutations on the protein-protein interface. We then propose DiffAffinity that utilize the learned representations to predict A AAG We organize this section as follows: Section 3.T presents the preliminaries and notations that are used throughout the paper and formally defines the problem. Section 3.2 and 3.3 describe SidechainDiff and DiffAffiiity, respectively. Section 3.4 describes key details in model training.
3.1 Preliminaries and notations
single-chain protein structure is composed of multiple residues (amino acids). Each amino acid shares common central carbon atom, called the alpha (a) carbon. The alpha carbon is bonded to an amino group (-NH2), a carboxyl group (-COOH), and a hydrogen atom, forming the amino acid's backbone. Each amino acid has a distinct atom or group bonded to the alpha carbon, known as the side-chain. A residue's 's side-chain structure usually termed side-chain conformation. The side-chain conformation varies across different amino acids, and it depends on the type of amino acid and the overall protein structure. Our main work focuses on building a generative process of side-chain conformations given the protein backbone structures.
3

---

side chain
chain
Latent Space
Rotamer Qu.2.2130
Diffusion Process
Ground Truth
Ane byp Mowdraangs amgo Ddemtatoo
Process
Destance Map
Backbone Frame
Predicted
Rotamer
(1 2.2.)
Encoder
T' space
Figure 1: The overall architecture of Sidechain
Multiple protein chains bind with each other and form protein complex. Here we only consider protein complexes with two chains just for notation simplicity and note that model works for protein complexes with more chains. For a protein complex with n residues, we denote the two chains as chain A and B, and the set of its residues (1,...,,) n). The properities of the i-th amino acid include the amino acid type a; E (1,..., 20), the orientation R, E SS(3), the 3D coordinates (k) belongs [0, 2mr)
X; 3 and the side-chain torsion angles X y where X;
The bond lengths and bond angles in side-chains are usually considered fixed and given backbone structures, can solely use Xi reconstruct the coordinates of each atom in side-chain. Thus, side-chain conformation also called a rotamer, which is parameterized with X The number of torsional angles ranges from 0 to 4, depending on the residue type. Here, place rotamer in 4-dimensional torus space T (S)) where denotes a unit circle.
3.2 SidechainDiff: side-chain diffusion probabilistic model
SidechainDiff utilizes conditional Riemannian diffusion model in T to build the generative process of the four rotamers and estimate their joint distributions (Figure [ The generative process conditioned on the structural context of mutations on the protein- -pretinn interface, which is encoded using (E(33--invarle) neural network. The learned conditional vectors serve as a natural representation of the structural context.
Conditional diffusion probabilistic model on T1 We adopt the continuous score-based framework on compact Riemannian manifolds (De Bortoli al. 2022 to construct the generative process for rotamers within T Our primary adaptation involves conditioning our diffusion model on vectors that are concurrently learned via an encoder network.
Let the random variable denote the states rotamers. And let (X,)eeoory and (Y,) (Xr-<)ee(o.r) denote the diffusion process and associated reverse diffusion process in T respectively. The stochastic differential equation (SDE) and reverse SDE [De Bortoli et al. 2022 Song al. 2021 can be defined as follows:
dx, = M.
(I)
dY, V P M
Here, B.M represents the Brownian motion on T which approximated with Reodesic Random
Walk (GRW) on T (Algorithm I). The score V log pogprre(Y Y, ()) in Equation 2 is estimated with

---

score network (x t, Z). We introduce the conditional vector Z in the score. parameterized with an encoder network that takes the structural context of mutation sites as inputs. We will delve into these components separately in the following subsections.
Geodesic Random Walk in T When approximating the Brownian motion the compact Riemannian manifolds using GRWs, a crucial step is defining the projection map and the exponential map that establishes the connection between the ambient Euclidean space and the associated tangent space, as discussed in previous studies (Jorgensen, 1975 De Bortoli et al. [022] In the context of T which is constructed as the quadruple product of St within the ambient Euclidean space RR,,, is
adequate to focus on modeling Brownian motion on S', defined as follows:
K = ( 0 )
dxg K dB,,
(3)
Here, B, represents the Brownian motion on the real line the ambient Fuclidean space IR,,, and K denotes the diffusion coefficient.
Similarly, the projection map Projy; within the tangent space Ty, (T*) and the exponential map Exp T can be derived by applying the Cartesian product of the maps designed for St. To be more precise, these two maps are defined as follows:
cxp,(V) cos([Ivvl))) (Ilvv))) sin(llv])) [|v|
(4)
proj =
||al|2 n.
(5)
Here, E and IR?
Algorithm 1 Geodesic Random Walk (GRW) during the sampling phase
Require: T.N,Yo YJ', fa;, x, I: y = T/N
Step-size
Enca- (fas, X.)
b Encode the structural context into hidden representation
3: for h E (O..... ...... - 1} do
4: R&41 NO).dd)
D. Standard Normal noise in ambient space (I=²)4 Projection in the Tangent Space Tv, (T*) Compute the Euler-Maruyana, step tangent space Move along the geodesic defined by W and Y? on T
5: Ra+1 Projx, (R++1)
6: W8+1 -7s0 (Ysa(YY,,og D) + R
7: Y Expy? (W++1)
8: end for
9: return (YNIN
Score network The architecture of the score network s t, Z) is implemented as a three-layer Multilayer Perceptron (MLP) with 512 units in each layer. takes as inputs the conditional vector from the encoder network representing the structural context of the mutations, the sampled rotamers, and the timestep
Conditional encoder network The encoder network (Figure ST takes as inputs both single features, consisting of amino acid type, backbone dihedral angles, and local atom coordinates for each amino acid, as well as pair features such as pair distances and relative positions between two amino acids. An SE3-invariant IPA network (Jumper al. 2021 is utilized obtain the conditional vector Z for the structural context of the mutations. Further information regarding the encoder can be found in App. A.I
Training objectives trained simultaneously. We adopt the implicit loss (s.) in De Bortoli et al. 2022). defined as follows:
The parameters 0 and á in the score network and conditional encoder network
1 (1111ss0Xxx.0 3 Z))
2im (st)
(6)
3.3 DifTAffinity: mutational effect predictor
For the mutational effect predictor, DiffAffinity, we imitate the structure of the mutated protein by setting the side-chain conformation empty set and altering the amino acids at the mutation
5

---

sites. Subsequently, we use the pre- tre-trained conditional encoder Enc,- from SidechainDiff to encode wild-type and mutant structures into distinct hidden representations. These hidden representations. following additional IPA-like transformer network and max aggregation, are fed into a Multilayer Perceptron (MIP) of only one layer for the prediction of AAG. To assess the quality of our hidden representation, we also utilize the linear combination of SidechainDiff's hidden representations to predict AAG. More model details can found in App. A.2
3.4 Model training
We train SidechainDiff with the refined experimental protein structures on the database of Protein Data Bank-REDO Doosten et al. follow the same protocol in RDE Luo et al. 2023 to preprocess the data and split the train and test sets. We train DiffAffinity with the SKEMPI2 dataset (Jankauskaité et al. 2019]. Due to limited samples, perform strategy of three-fold cross-validation train the model and benchmark the performance. Specifically, we split the SKEMPI2 dataset into three folds based on structure, ensuring that each fold contains unique protein complexes not present in the other folds. Two of the folds are utilized for training and validation, whereas the remaining fold is reserved for testing purposes. More training settings are shown in App
2014]. To fairly benchmark the performance of SidechainDiff, we
4 Results
First, we assess DiffAffinity's performance in predicting AAG using the SKEMPI2 dataset (Section and the latest SARS-CoV-2 dataset (Section 4.2p. Subsequently, we demonstrate DiffAffinity's effectiveness in optimizing antibodies, with a focus on SARS-CoV-2 as an illustrative case (Section
Additionally, we evaluate the prediction accuracy of SidechainDiff in side-chain packing.
4.1 Prediction of the mutational effects on binding affinity
DiffAffinity leverages the learned representation from SidechainDiff to predict mutational effects on potein-prooeein binding. Here, we benchmark the performance on the widely used SKEMPI2 dataset.
Baseline models As mentioned in section 2.2 the models for mutational effects prediction can be categorized into energy-based, sequence-basee based, unsupervised, end-to-end, and pre-trained models. We have selected several representative models from each category. The selected models include FoldX [Schymkowitz et al. 2005). Rosetta [Alford et al. 2017]. flex ddG (Barlow et al. 2018).
ESM-1v Meier et a 2021 IF [Hsu et al. 2022 ESM2 Lin et al., 2023). ESM2", DDGPred
Shan et al. 2022 and RDE-Network
Luo et al.
2023 More details baselines are shown App.
A.4
For ablation studies, have also developed two variants named DiffAffinit** and SiffAfffinit-LLinee DiffAffinity** employs the same model architecture as DiffAffinity but without the learned representa- tions from SidechainDiff and serves as an end-to-end prediction method. And DiffAffinity-Line is a direct linear projection of SidechainDiff's hidden representations to AAG.
Evaluation metrics The performance of model is evaluated using variety of metrics. These include traditional prediction accuracy metrics such as root mean squared error (RMSE) and mean absolute error (MAE). Given that the scalar values of experimental AAG are not standardized, we also employ Pearson and Spearman correlation coefficients to evaluate the models' predictive capability. To assess our model's ability to identify positive negative mutations affecting protein- protein binding, we use the AUROC and AUPRC metrics. Mutations are classified based on the sign of AAG, where positive values indicate positive impacts on binding and negative values indicate negative impacts. AUROC measures the overall discriminative power, while AUPRC accounts for imbalanced datasets.
Following the nettnetwork (Luu et al. 2023 we also evaluate the prediction performance for each complex separately. We group mutations by complex structure, excluding groups with fewer than ten mutation data points, and derive two metrics: average per-structure Pearson correlation coefficient per- structure Pearson and average per-structure Spearman correlation coefficient per-structure Spearman.
Experimental results Our methods, DiffAffinity and DiffAffiniyy* outperform all baseline models across almost all metrics and achieve state-of-the-an performance on the SKEMPI2 dataset (Table I

---

Particularly for multi-ponn point mutations, DiffAffinity notably outperforms all other methods, suggesting that our model effectively captures the crucial relationship between mutational effects and side-chain conformations, regardless of the number of mutations. DiffAffinity surpasses the performance of the
and ESM2*, indicating that the latent representations derived from our diffusion probabilistic model, SidechainDiff,, are more beneficial than those from flow-based generative models and other self-supervised pre-training representations with the same network architecture for the same task. More analysis of the three pre-training representations can be found in App
Furthermore, DiffAffiiity-Linee shows that even straightforward linear projection of latent repre- sentations can achieve performance comparable to FoldX and ESM-IF and outperform GSM-1v. The reduced performance observed with DiffAffinit**, when the SidechainDiff encoder is absent, further emphasizes the effectiveness of the latent representations extracted from SidechainDiff.
In subsequent sections (Section 4.2 and
employ the three models trained different splits of the SKEMPI2 dataset without any fine-tuning on downstream tasks related to predicting binding affinity. This approach allows us assess our model's performance diverse tasks.
Table I: Evaluation of prediction the SKEMPI2 dataset
Overall
Per-Structure Spearman
Method
Mutations
Pearson
Spearman
RMSE
MAE
AUROC
AUPRC
Pearson
all
0.319
0.416
1.959
1.357
0.671
0.839
0.376
0.375
FoldX
single
0.315
0.361
1.651
1.146
0.657
0.839
0.382
0.360
multiple
0.256
0.418
2.608
1.926
0.704
0.841
0.333
0.340
all
0.311
0.346
1.617
1.131
0.656
0.810
0.328
0.298
Rosetta
single
0.325
0.367
1.183
0.987
0.674
0.834
0.351
0.418
multiple
0.199
0.230
2.658
2.024
0.621
0.798
0.191
0.083
0.402
0.427
1.587
1.102
0.675
0.866
0.414
0.386
flex ddG
single
0.425
0.431
1.457
0.997
0.677
0.874
0.433
0.435
multiple
0.398
0.419
1.765
1.326
0.669
0.854
0.401
0.363
all
0.192
0.157
1.961
1.368
0.541
0.735
0.007
00.12
ESM-Iv
single
0.191
0.157
1.723
1.192
0.549
0.770
0.042
0.027
multiple
0.192
0.175
2.759
2.119
0.542
0.678
-0.000
0.128
all 0.319
0.281
1,886
1.286
0,590
0.768
0.224
0.202
ESM-IF
single
0.296
0.287
1.673
1.137
0.605
0.776
0.391
0.364
multiple
0.326
0.335
2.645
1.956
0.637
0.754
0.202
0.149
all 0.133
0.138
2,048
1.460
0.547
0.738
0,044
0.039
ESM2
single 0.100
0.120
1.730
1.210
0.541
0.734
0.019
0.036
multiple
0.170
0.163
2.658
2.021
0.566
0.746
0.010
0.010
all 0.623
0.498
1.615
1.179
0.721
0.887
0.362
0.316
ESM2*
single
0.625
0.468
1.357
0.986
0.707
0.879
0.391
0.342
multiple
0.603
0.529
2.15
1.67
0.758
0.909
0.333
0.304
0.630
0.400
1.313
0.995
0.696
0.892
0.356
0.321
DDGPred
single
0.652
0.359
1.309
0.936
0.656
0.884
0.351
0.318
multiple
0.591
0.503
2.181
1.670
0.759
0.913
0.373
0.385
all
0.632
0.527
1.601
1.142
0.731
0.887
0.415
0.376
RDE-Net
single
0.637
0.491
1.341
0.961
0.720
0.885
0.413
0.385
multiple
0.601
0.567
2.157
1.631
0.768
0.898
0.390
0.360
all 0.326
0.305
1.954
1.399
0.642
0.857
0.222
0.222
DA-Linear'
single
0.318
0.293
1.649
1.175
0.651
0.854
0.209
0.202
multiple
0.277
0.288
2.593
1.961
0.629
0.867
0.193
0.195
all 0.646
0.538
1.578
1.113
0.742
0.741
0.415
0.392
DiffAffinit ssingle
0.657
0.523
1.312
0.931
0.742
0.741
0.417
0.396
multiple
0.613
0.542
2.133
1.606
0.750
0.750
0.407
0.379
all 0.669
0.556
1.535
L.093
0.744
0.896
0. 0.422
0.397
DiffAffinity single
0.672
0.523
1.288
0.923
0.733
0.887
0.429
0.409
multiple
0.650
0.602
2.051
1.540
0.784
0.921
0.414
0.387
-nn--nn is short of DiffA -inear just to save space in the table.

---

4.2 Prediction of mutationnl effects on binding affinity of SARS-CoV-2 RBD
SARS-CoV-2 initiates infection by binding to ACE2 protein on host cells via the viral spike protein. The receptor binding domain (RBD) of the spike proteins exhibits high-affinity binding to ACE2.
The technique of deep mutational scanning has been employed to experimentally quantity the effects of all single-point mutations on the binding affinity of the ancestral Wuhan-Hur. RBD (PDB ID: 6M0J) [Starr et al. 2022]. These measurements have guided the survey SARS SARS-CoV-2 evolution and identified several significant mutation sites on RBD that substantially contribute to the binding affinity. A total of 15 significant mutation sites like NE50 have been identified (see App. A.5 We predicted all 285 possible single-point mutations for these sites and calculated the Pearson correlation coefficient between the experimental and predicted AAG.
Our results show that Dfffnifinity outperforms all baseline methods (Table 2) highlighting its potential facilitate biologists understanding the evolution of SARS-CoV-2. Furthermore. we note that DiffAffinity exhibits a substantial improvement over DiffAffinit** indicating the effectiveness of the learned representations from SidechainDiff.
4.3 Optimization of human antibodies against SARS-CoV-2
The receptor-binding domain (RBD) the SARS-CoV-2 virus spike protein (PDB ID: 7FAE) plays a pivotal role in the binding process with the host's ACE2 protein. It serves as a prime target for neutralizing antibodies against SARS-CoV-2 [Shan al. 2022]. In our experiment, we predict all 494 possible mutations at 26 sites within the CDR region of the antibody heavy chain. We expect that predicting mutational effects on binding affinity can facilitate in identifying top favorable mutations that can enhance the neutralization efficacy of antibodies.
These mutations ranked in ascending order according their AAG values, with the most favorable (lowest A AAG mutations positioned at the top. To enhance neutralization efficacy, we aim accurately identify and rank the five most favorable mutations. DiffAffinity successfully identifies four of these mutations within the top 20% of the ranking and two within the top 10% (Table 33. Diff'ffiiity consistently outperforms all the baseline methods, indicating its superior ability predict the effects of mutations. This highlights DiffAffinity potential as robust tool for optimizing human antibodies.
Table 2: Pearson correla- tion coefficient in SARS- COV-2 binding affinity
Table Rankings of the five favorable mutations on the human anti- body against SARS-CoV-2 receptor- binding domain (RBD) by various competitive methods
Method
Pearson
Method
TH31W
AH53F
NH57L
RH103M
LH104F
FoldX 0.385
FoldX
4.25%
14.57%
2.43%
27.13%
63.77%
RDE-Net
0.438
RDE-Net
5.06%
12.15%
55.47%
50.61%
9.51%
DiffAffinity*
0.295
DiffAffinity*
7.29%
0.81%
19.03%
84.21%
28.54%
DifTAffinity
0.466
DifTAffinity
7.28%
3.64%
18.82%
81.78%
10.93%
4.4 Prediction of side-chain conformations
To assess the generative capacity of SidechainDiff straightforwardly, we employ it for predicting side-chain conformations of specific amino acids given their structural context. Our approach involves sampling rotamers from the distribution modeled by SidechainDiff and selecting the rotamer with the highest likelihood from a pool of 100 samples. For this task, we draw upon the test dataset from the PDB-REDO test split.
For rigorous evaluation, we compare SidechainDiff with both energy-based methods, including Rosetta [eman et al. 2020 and SCWRL4 [Kivvv et al. 2009]. as well as the deep learning basedd methods, including RDE, AttnPacke [MPPartlon and Xu 2023 and DLPacker [Mistura al.) 2022]
All methods are run under the same settings to ensure a fair comparison. We evaluate the performance
'N500Y denotes the substitution of asparagine (N) at position 501 on chain E with tyrosine (Y), where N and Y represent the single-letter codes for the amino acids asparagine and tyrosine, respectively.

---

using the Mean Absolute Error (MAE) and steric clash number of the predicted rotamers McPartlon and Xu 2023
In terms of MAE for rotamer prediction, Sidechain outperforms energy-based methods and achieves comparable results deep beanning-asse methods such RDE and AttnPacker (Table and Table[STp. Notably, our method surpasses all the other methods in terms of the steric clash number. It's noteworthy that while methods like AttnPacker focus on predicting the torsion angles of static side-chain conformations, SidechainDiff emphasizes the distribution of side-chain conformations under various physical constraints. We measure the physical constraints of mutation sites using the contact number, i.e., the number of neighboring amino acids within an 8.4 radius. Generally, a higher contact number indicates more constrained structural context. We observe that for mutation sites with higher constraints, SidechainDiff tends to produce more accurate side-chain conformations (Table 5 We further illustrate cases showing the trade-off between accuracy and diversity achieved by SidechainDiff under various physical constraints (Appendix and Figure S5a (5d).
Table 4: Evaluation of predicted side-chain conformations
Table Accuracy of side-chain conformations stratified based on the contact number
Method
(1)
x (2)
X
(3)
X (4)
Average
Clash number
Proportion
100%
82.6%
28.2%
12.6%
Contact number
Average
SCWRL4
24.33
32.84
47.12
56.15
28.21
118.44
X
Rosetta
23.98
32.14
48.49
58.78
28.09
113.31
RDE
17.13"
28.47"
43.54
58.62
21.99"
34.41
1 ~ 7
28.70
DLPacker
21.40"
29.27'
50.77
74.64
26.42"
62.45
8 ~ 10
22.39
AttnPacker
19.65
25.17
47.61
55.08
22.35
57.31
11 19
15.43
DifTAffinity
18.00
27.41"
43.97
56.65
22.06
27.70
5 Conclusions
We present SidechainDiff, Riemannian diffusion basedd generative model for protein side-chains. excels in both generating side-chain conformations and facilitating representation learning. Utilizing the learned representations, our method achieves of-te-of-the-art performance in predicting AAG across various test datasets, including the SKEMPI2 dataset and the latest SARS- L-RS-CoV-2 dataset. Moreover, we demonstrate effectiveness in antibody screening and its superior side-chain packing accuracy.
In future research, we can consider integrating the side-chain clash loss (Jumper et al. 202 refine our generative process. Additionally, we can explore building an end-to-end diffusion- based generative model for protein full-atom structures by integrating our model with existing backbone structure generation methods [Watson et al., 2023 Yim et al. 2023]. The main limitation of our model is the lack of consideration of backbone structure changes induced by mutations. Addressing the limitation could lead to enhanced performance.
Acknowledgements
We would like to thank the National Key Research and Development Program of China (2000YFA0970000), and the National Natural Science Foundation of China (32370657, 32271297, 82130055, 62072435), and the Project of Youth Innovation Promotion Association CAS to H.Z. for providing financial support for this study and publication charges. The numerical calculations in this study were supported by ICT Computer-X center.
References
Rebecca F Alford, Andrew Leaver-Fay, Jeliazko R Jeliazkov, et al. The Rosetta All-Atom Energy Function for Macromolecular Modeling and Design. Journal of Chemical Theory and Computation, 1336)3303,,3048, 2017.

---

Marloes Arts, Victor Garcia Satorras, Chin-Wei Huang, et al. Two for One: Diffusion Models and Force Fields for Poarse-Graine, Molecular Dynamics. Journal of Chemical Theory and Computation, 19(18):1511-615, 2023.
Minkyung Back, Frank DiMaio, Ivan Anishchenko, al. Accurate prediction of protein structures and interactions using three-track neural network. Science, 373(6557)8871-87777 2021.
Kyle A Barlow, Shane Ó Conchúir, Samuel Thompson, et al. Flex ddG: Rosetta Ensemble-Base Estimation Changes in Protein-Proe Protein Binding Affinity upon Mutation. The Journal of Physical 2018.
Chemistry B, I
Nicole Calakos, Mark K Bennett, Karen E Peterson, al. Protein-protein interactions contributing to the specificity of intracellular vesicular trafficking. Science, (63(5150):1146))) 149, 1994.
Gabriele Corso, Hannes Stärk, Bowen Jing, et al. DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking. In International Conference on Learning Representations, 2023.
Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, et al. Riemannian Score-Based Generative Modelling. In Advances in Neural Information Processing Systems, volume 35, pages 1406-2442, 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems, volume 34, pages 8780-8794. 2021.
Jonathan Frazer, Pascal Notin, Mafalda Dias, et al. Disease variant prediction with deep generative models of evolutionary data. Nature, 599(7883):91-95, 2021.
Cunliang Geng, Anna Vangone, Gert E Folkers, et al. iSEE: Interface structure, evolution, and energy-based machine learning predictor of binding affinity changes upon mutations. Proteins: Structure, Function, and Bioinformatics, 87(2):1 10-119, 2019.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840 6851, 2020.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, et al. Argmax Flows and Multinomial Diffussin: Learning Categorical Distributions. In Advances in Neural Information Processing Systems, volume 34, pages 12454-1222 12465, 2021.
Thomas Hopf, John B Ingraham, Frank Poelwijk, et al. Mutation effects predicted from sequence co-variation. Nature Biotechnology, 35(2): 128-):2281135 2017.
Chloe Hsu, Robert Verkuil, Jason Liu, et al. Learning inverse folding from millions of predicted structures. In Proceedings of the 39th International Conference on Machine Learning, volume 162, pages 1970, 2022.
John Ingraham, Max Baranov, Zak Costello, et al. Illuminating protein space with programmable generative model. bioRxiv. doi 10.1001/202.1220111116688
Justina Jankauskaité, Brian Jiménez-García, Justas Dapkūnas, et al. Skempi 2.0: an updated benchmark of changes in protein binding energy, kinetics and thermodynamies upon mutation. Bioinformatics, 35(3):462-469 2019.
Bowen Jing, Gabriele Corso, Jeffrey Chang, et al. Torsional Diffusson for Molecular Conformer Generation. In Advances in Neural Information Processing Systems, volume 35, pages 24240- 24253, 2022.
Susan Jones and Janet M Thorton.. Principles of protein-protehe interactions. Proceedings of the National Academy of Sciences, 93(1):13-20, 1996.
Robbie P. Joosten, Fei Long, Garib N. Murshudov, et al. The PDB REDO server for macromolecular structure model optimization. IUCrJ, 1(4):213-220, 2014.
Erik Jørgensen. The central limit problem for geodesic random walks. Zeitschrifi für Wahrschein- lichkeitstheorie und verwandte Gebiete, 32(1-2):1164, 1975.
10

---

John Jumper, Richard Evans, Alexander Pritzel, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):533-58)) 2021.
Diederik Kingma, Tim Salimans, Ben Poole, et al. Variational Diffusion Models. In Advances in Neural Information Processing Systems, volume 34, pages 21 21696-21707, 2021.
Georgii Krivov, Maxim Shapovalov, and Roland Dunbrack. Improved prediction of protein side-chain conformations with SCWRL4. Proteins, 77:778-981 95, 12 2009.
Jun Lan, Jiwan Ge. Jinfang Yu, et al. Structure of the SARS-Co spike receptor- receptor-binding domain bound to the ACE2 receptor. Nature, 581(7807:215-2220 2020.
Julia Koehler Leman, Brian D. Weitzner, Steven M. Lewis, et Macromolecular modeling and
design in Rosetta: recent methods and frameworks. Nature Methods, 17:665-6800 2020.
Anselm Levskaya. Orion D Weiner, Wendell A Lim, et al. Spatiotemporal control of cell signalling using a swiht-swi protein interaction. Nature, 461((266)997-1001,, 2009.
Minghui Li, Franco L Simonetti, Alexander Goncearenco, et al. Mutabind estimates and interprets the effects of sequence variants on protein-protein interactions. Nucleic Acids Research, 44(W1); W494-W501, 2016.
Zeming Lin, Halil Akin, Roshan Rao, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123-1133) 2023.
Xianggen Liu, Yunan Luo, Pengyong Li, et al. Deep geometric representations for modeling effects of mutations protein-protein binding affinity. PLoS Computational Biology, 1008)440099284 2021.
Shitong Luo, Yufeng Su, Zuofan Wu, et al. Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations Protein-Protedd Interaction. In International Conference Learning Representations, 2023.
Derek M Mason, Simon Friedensohn, Cédric R Weber, et al. Optimization of therapeutic antibodies by predicting antigen specificity from antibody sequence via deep learning. Nature Biomedical Engineering. 5(6):600-612, 2021.
Matthew McPartlon and Jinbo Xu. An end-to-end deep learning method for protein side-chain packing and inverse folding. Proceedings of the National Academy of Sciences, 120(23):e22164438822 2023.
Joshua Meier, Roshan Rao, Robert Verkuil, et al. Language models enable zero-shot prediction of the effects of mutations on protein function. In Advances in Neural Information Processing Systems, volume 34, pages 29287-29303, 2021.
Lidio MC Meireles, Alexander S Di; 1/2mling, and Carlos J Camacho. ANCHOR: web server and interaction binding pockets for drug discovery. Nucleic
database for analysis of Acids Research, 38: 14407-W411 2010.
Mikita Misiura, Raghav Shroff, Ross Thyer, et al. DLPacker: Deep learning for prediction of amino acid side chain conformations in proteins. Proteins: Structure, Function, and Bioinformatics, 90 (6):1278-1290, 2022.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 2162-8171, 2021.
Pascal Notin. Mafalda Dias, Jonathan Frazer, et al. Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time Retrieval. Proceedings of the 39th International Conference on Machine Learning, volume 162, pages 16990-17017, 2022.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, et al. Normalizing Flows for Probabilistic Modeling and Inference. The Journal of Machine Learning Research, 22(1):2617- 2680, 2021.

---

Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, et al. Diffusion Autoencoders: Toward Meaningful and Decodable Representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 10619-10629, 2022.
Adam J Riesselman, John B Ingraham, and Debora S Marks. Deep generative models of genetic variation capture the effects mutations. Nature Methods, 15(10):816- -222222, 2018.
Benno Schwikowski, Peter Uetz, and Stanley Fields. A network of protein-prooe interactions in yeast. Nature Biotechnology, 18(12):1257-12611 2000.
Joost Schymkowitz, Jesper Borg, Francois Stricher, al. The FoldX web server: an online force field. Nucleic Acids Research, 33:W382-W388, 2005.
Sisi Shan, Shitong Luo, Ziqing Yang. et al. Deep learning guided optimization of human antibody against SARS-CoV-2 variants with broad neutralization. Proceedings of the National Academy of Sciences, (19(11):212295411 2022.
Jascha Sohl-Dicksteii Eric Weiss, Niru Maheswaranathan, et al. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pages 2256-2266 2015.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, et al. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representa- fions, 2021.
Tyler N. Starr, Allison J. Greaney, William W. Hannon, et Shifting mutational constraints in the SARS-CoV-2 receptor-binding domain during viral evolution. Science, 377(6604)::420 4244 2022.
Thomas Steinbrecher, Robert Abel, Anthony Clark, et al. Free Energy Perturbation Calculations of the Thermodynamics of Protein Side-Chain Mutations. Journal of Molecular Biology, 429(7): 923-929, 2017.
Karyn T O'Neill and Ronald H Hoess. Phage display: protein engineering by directed evolution. Current Opinion in Structural Biology, 5(4):443 449, 1995.
Brian L. Trippe, Jason Yim, Doug Tischer, et al. Diffusion Probabilistic Modeling of Protein Backbones in 3D for the motif-scaffolding problem. In International Conference Learning Representations, 2023.
Joseph L Watson, David Juergens, Nathaniel R Bennett, et al. De novo design of protein structure and function with REdiffusion. Nature, 620979766 2023.
Fang Wu and Stan Li. DIFFMD: A Geometric Diffusion Model for Molecular Dynamics Sim- ulations. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 5321-5329, 2023.
Gang Xu, Qinghua Wang, and Jianpeng Ma. OPUS-Rote gradient-based protein side-chain modeling framework assisted by deep learning-bar based predictors. Briefings in Bioinformatics, 23 (1):bbab529, 2022.
Jason Yim, Brian L. Trippe, Valentin De Bortoli, et al. SE(3) Diffusion Model with Application to Protein Backbone Generation. Proceedings of the 40th International Conference on Machine Learning, volume 39, page 1672, 2023.
Ning Zhang, Yuting Chen, Haoyu Lu, et al. MutaBind2: Predicting the Impacts of Single and Multiple Mutations on Protein-Proted Interactions. Iscience, 23(3):100939, 2020.
Zijian Zhang, Zhou Zhao, and Zhijie Lin. Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, volume 35, pages 12-17-2113, 2022.
12


Reviews:
[{"review_id": "47dLvhfaPX", "reviewer": "Reviewer_wzHr", "summary": "The paper studies the problem of mutational effect prediction on protein-protein binding. To address the data scarcity issue, the authors propose to first pre-train their model on PDB using diffusion models to fit the protein side-chain distribution. Then, the learned representations are used for predicting the mutational effects on SKEMPIv2. The method achieves good results on the benchmark in the author’s evaluation setting.", "questions": "Major points:\n1. A significant portion of Section 3.2 appears to be directly copied from the Riemannian diffusion model paper [1], including Propositions 1 and 2. Please discuss why the two propositions are important to side-chain problems and mention in the main paper that **these contributions are derived from previous works**. Also, the section should be rewritten with appropriate paraphrasing to clarify the changes made to adapt the model for the new task.\n2. The authors propose to use a three-fold cross-validation for evaluation, but several details regarding the dataset split need to be addressed. (a) How is the dataset split? Is it split by proteins or mutations? Is it split based on sequence similarity or structure similarity? (b) The claim that DDGPred does not provide training scripts, and the authors copied the results from the published work is not a valid excuse. The training script of DDGPred is very easy to implement. Reproducing the results with the same dataset splits is essential to ensure fair comparison, as the performance can vary significantly with different splits due to limited data. Cross-validation should be repeated multiple times to assess the significance of the improvement.\n3. Important baselines are missing in the benchmark. First, more traditional methods should be included in comparison, e.g. FlexDDG [2]. These methods hardly suffer from the overfitting problem. Second, the performance of using pre-trained representations with a linear predictor is low and the improvement of DiffAffinity over DiffAffinity* is small. This raises the doubt on the effects of pre-training. Please include a baseline using other pre-trained representations, e.g., ESM2. Two baselines should be considered (a) ESM-2 with a 2-layer MLP predictor and (b) Feeding ESM-2 representations to the DiffAffinity architecture.\n4. The reported metrics are problematic. First, AUROC is more suitable for balanced binary classification tasks, and the number of positive and negative samples should be reported for binary classification tasks. If the data is imbalanced, AUPRC should also be reported. Second, treating mutations with ddG around 0 as either positive or negative is not appropriate, considering the measurement errors and the fact that most mutations in SKEMPIv2 are neutral. Additionally, discarding proteins with fewer than 10 mutations when reporting per-structure Pearson raises concerns about the sensitivity of the reported metric to this threshold. The authors should clarify the number of proteins and mutations left for evaluation and provide justifications for choosing 10 as the threshold.\n5. The evaluation in Sec. 4.3 is not convincing. Assigning high ranks to only five experimentally tested mutations is insufficient to draw conclusions about the effectiveness of the proposed method, especially considering the limited number of mutations tested experimentally in the original paper [3]. The authors should consider a more comprehensive evaluation by testing a larger set of possible mutations.\n6. In Sec. 4.4, the authors only compare their method with three baselines and ignore recent advancements in side-chain packing problem [4, 5]. Since modeling side-chain conformation is a major contribution of the paper, more experimental evaluation with a broader set of baselines should be included to demonstrate the method's effectiveness.\n\nMinor points:\n1. Line 121: Please give a definition of backbone atoms and side-chain atoms. This will be more friendly to audience without knowledge about proteins.\n2. Eq. (3) in Sec. 3.2: $U(X_t)$ is used without any definition.\n3. Line 148: Please discuss how the Brownian motion on $\\mathrm{T}^4$ is defined in the context of side-chain packing problem.\n4. Eq. (4) in Sec. 3.2: The SDE here is Variance-Exploding SDE, while the concepts introduced before are based on Variance-Preserving SDE. Please correct me if I am wrong.\n5. Line 165: Please discuss why the perturbation kernel $p_{t|0}$ is difficult to obtain.\n6. Line 171: “equation 6 -> Equation 6” Please make the references to equations consistent across the paper.\n\nOverall, I think the paper is not ready to publish in its current state. However, I firmly believe that the paper holds great potential for making a valuable contribution to the field if the authors dedicate more efforts towards improving the experimental design and writing. By addressing the weaknesses and incorporating the suggested changes, the paper can reach a higher standard of quality and become a valuable addition to the existing literature. With the necessary revisions, I am confident that the paper can be transformed into a strong and impactful publication.\n\n[1] De Bortoli, Valentin, et al. \"Riemannian score-based generative modelling.\" Advances in Neural Information Processing Systems 35 (2022): 2406-2422.\n\n[2] Barlow, Kyle A., et al. \"Flex ddG: Rosetta ensemble-based estimation of changes in protein–protein binding affinity upon mutation.\" The Journal of Physical Chemistry B 122.21 (2018): 5389-5399.\n\n[3] Shan, Sisi, et al. \"Deep learning guided optimization of human antibody against SARS-CoV-2 variants with broad neutralization.\" Proceedings of the National Academy of Sciences 119.11 (2022): e2122954119.\n\n[4] Misiura, Mikita, et al. \"DLPacker: deep learning for prediction of amino acid side chain conformations in proteins.\" Proteins: Structure, Function, and Bioinformatics 90.6 (2022): 1278-1290.\n\n[5] McPartlon, Matthew, and Jinbo Xu. \"An end-to-end deep learning method for protein side-chain packing and inverse folding.\" Proceedings of the National Academy of Sciences 120.23 (2023): e2216438120.", "limitations": "The authors should add a separate paragraph to dicuss the potential limitations.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. Mutational effects on protein-protein binding is an important and difficult problem in the protein community. It suffers from the data scarcity issue and pre-training seems like a good solution to the problem.\n2. The paper introduces diffusion-based models to fit the protein side-chain distribution, which, to the best of my knowledge, is among the first papers to explore this approach.", "weaknesses": "1. The paper's writing and clarity are poor, making it hard to follow. Many sections seem to be directly copied from previous works. The authors introduce many unnecessary notations and equations, which makes the method even more difficult to understand.\n2. The novelty of the paper is limited, as it primarily extends existing works without introducing problem-specific designs. The substitution of flow-based models with diffusion models for side-chain distribution modeling is not sufficiently innovative.\n3. The related work section lacks a discussion of existing methods on protein side-chain packing, which would provide a more comprehensive context for readers.\n4. The experiment design contains significant flaws that undermine the persuasiveness and support of the evaluation presented. Notably, many important baselines and details are missing from the experiment section, which weakens the arguments made by the authors. (See questions)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for constructive comments on our work. Below are our responses to the reviewer's concerns.\n\n__Q1:__\nWe realize that too many notations similar to the paper of the Riemannian Diffusion Model may be misleading. We agree with the reviewer in this aspect. Following the reviewer’s suggestion, we have rewritten Section 3.2.\n\nIn the initial submission, we included these two propositions and explicitly cited their work just for the completeness of our paper. To avoid misleading, we now have put these propositions in the supplementary and clearly state their reference.\n\nTo clarify the change for our model, the primary modifications of the original Riemannian diffusion model for our task are presented in **Q2 in the \"global\" response**.\n\n__Q2:__\n1. We split the dataset by proteins based on structure similarity. Specifically, we have split the SKEMPI2 dataset into three folds based on structure, ensuring that each fold contains unique protein complexes not present in the other folds. Two of the folds are utilized for training and validation, whereas the remaining fold is reserved for testing purposes. \n2. Following the reviewer’s suggestion, we have retrained the DDGPred model using the same hyperparameters as outlined in the reference. The result indicates that the performance of DDGPred is roughly comparable with the reported in their reference (please see **Table a in the PDF**).\n3. Following the reviewer's suggestion, we repeated the training of our model 5 times in cross-validation experiments. The results demonstrate that our methods are robust and stable in the test (Please see the **Table b in the PDF**).\n\n__Q3:__\n1. In the previous submission, we selected the most representative and publicly available benchmarks from different method categories. As the reviewer pointed out, we have checked FlexDDG according to their reference. But FlexDDG was not accessible, and no server was available for evaluation\n2. Following the reviewer's suggestion, we performed comparisons between DiffAffinity and two baselines. The new results can be found in **Table a in the PDF**. For an in-depth analysis of SidechainDiff, we extend to a visualization analysis of the baseline results using PCA (Please see **Q4 in the \"global\" response and in Figure a in the PDF**). \n\n__Q4:__\n1. Following the reviewer's suggestion, we report the new AUPRC metrics behind. The AUPRC results demonstrate that our model outperforms all the baselines, indicating its effectiveness in handling imbalanced datasets.\n    | Method | FoldX | ESM-1v | ESM-IF| DDGPred| RDE-Net | Linear| DiffAffinity*| DiffAffinity|\n    |-|-|-|-|-|-|-|-|-|\n    |AUPRC|0.839|0.735|0.768|0.892|0.887|0.741|0.857|0.896|\n2. Following the reviewer's suggestion, we have reported the results considering only mutations with $\\Delta \\Delta G$ numeric values above 1 or below -1. Our model exhibits consistent performance with previous results (please see **Table e in the PDF**). \n3. Following the RDE setting, we choose 10 as per-structure thresholds. We clearly show that our model consistently outperforms all the baselines, regardless of the chosen threshold ( please see **Table d in the PDF**).\n\n__Q5:__\n1. In addition to the task in Sec. 4.3, we also evaluated our method on another downstream task, including 285 mutations (as described in Sec. 4.2).  Our method achieves a Pearson correlation of 0.466, higher than other methods.\n2. Following the reviewer’s suggestion, we further evaluate our method on the HER2-antibody data including 491 mutations with experimentally validated binary labels[4]. Our methods also achieve better performance than RDE. \n| Method | AUPRC |\n|----|----|\n|ESM2*\t|0.739|\n|RDE|0.755|\n|DiffAffinity| 0.768|\n\n__Q6:__\nFollowing the reviewer's suggestion, we have included AttnPaker and DLPaker for comparison. The analysis of the updated results can be found in **Q5 in the \"global\" response and Table c in the PDF**.   Our method achieves better performance than AttnPaker and DLPacker in terms of both average MSE of side-chain torsion angles and side chain clash number.\n\n__Q7: Minor points__\n1. We have now added a definition of backbone atoms and side-chain atoms in the previous submission.\n2. Thank you for the guidance. We have removed the irrelevant notation $U(X_t)$ from Eq. (3) as it does not pertain to our tasks.\n3. In the context of the side-chain packing problem, we represent the rotamer (side-chain conformation) in $\\mathbb{T}^4 = (\\mathbb{S}^1)^4$. Consequently, we initially consider the Brownian motion in $\\mathbb{S}^1$ and then extend it directly to $\\mathbb{T}^4$.The diffusion equation for the density of Brownian particles $\\mathbf{Y}$ at point $x$ and time $t$ on the unit circle is given by:\n$\\mathrm{d}\\mathbf{Y} = -\\frac{1}{2}\\mathbf{Y}\\cdot\\mathrm{d}t + \\mathbf{K}\\cdot\\mathbf{Y}\\cdot\\mathrm{d}\\mathbf{B}_t$ \n\n   $\\quad K_{11}=K_{22}=0,K_{12}=-1,K_{21}=1$. Here, $\\mathbf{B}_t$ represents the Brownian motion on the real line. By applying the diffusion process to each dimension and taking the Cartesian product of the results, we obtain the Brownian Motion in the 4-dimensional  torus space $\\mathbb{T}^4$. \n\n4. Yes, the SDE in our paper is Variance-Exploding SDE.\n \n5. We have now provided more details about the perturbation kernel in **Q6 in the \"global\" response**.\n\n6. We have corrected the writing errors in our paper.\n\n__References__\n\n[1] Shan et al. Deep learning guided optimization of human antibody against SARS-CoV-2 variants with broad neutralization. Proceeding of the National Academy of Sciences (2022)\n\n[2] Luo et al. Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction. ICLR (2023)\n\n[3] Starr et al. Shifting mutational constraint in the SARS-CoV-2 receptor-binding domain during viral evolution. Science (2022)\n\n[4] Masonet et al. Optimization of therapeutic antibodies by predicting antigen specificity from antibody sequence via deep learning. Nat Biomed Eng (2021)"}, {"review_id": "YGGYBGlDtf", "reviewer": "Reviewer_U583", "summary": "The paper introduces DiffAffinity, a novel method for predicting the effects of mutations on protein-protein interactions. The authors leverage a diffusion model to learn representations that aid in predicting changes in binding affinity (∆∆G). They compare DiffAffinity's performance with existing methods such as FoldX and RDE-Net. The paper also explores the application of DiffAffinity in optimizing human antibodies against SARS-CoV-2.", "questions": "1. Could the authors elucidate why the representation learned from the diffusion-based model benefits binding affinity prediction? More elaboration on this point would be beneficial.\n2. Is there a performance comparison between the model using diffusion-based representation and the model using other self-supervised representations (e.g., ESM2)? It would be insightful to see the model performance when replacing the diffusion-based representation with other representations.\n3. The score network receives timestep t as input. When computing the hidden representation for the downstream task of affinity prediction, what's the value of t, and why was it chosen?\n4. After obtaining the hidden representation from the diffusion-based encoder, an additional module predicts the binding affinity from the hidden representation. Is this additional module also an IPA-like transformer? The appendix seems to lack the specific model architecture of this module.\n5. In Table 1, the model appears to have achieved state-of-the-art performance without the representation learned from the diffusion model. Does this suggest that the performance gain primarily stems from the model architecture itself?\n6. Could the authors elaborate on the computational efficiency of DiffAffinity? How does it scale with the size of the protein or the number of mutations?", "limitations": "The proposed method only considers side chain conformation changes and does not take backbone changes into account. This limitation might hinder its practical applications.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper innovatively proposes a diffusion-based approach for predicting sidechain conformations at the protein-protein interface.\nThe authors provide a thorough comparison of DiffAffinity with existing methods, demonstrating its superior performance.\n2. The authors provide a thorough comparison of DiffAffinity with existing methods, demonstrating its superior performance.", "weaknesses": "1. The paper's main claim—that the hidden representation learned from the diffusion model can enhance binding affinity prediction—is not clearly elaborated.\n2. The proposed method does not account for changes in the backbone, which could limit its effectiveness in cases where mutations cause such changes.\n3. The paper could benefit from a discussion on the computational efficiency of DiffAffinity compared to other methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's approval and constructive comments on our work. In response, we have carefully addressed their concerns and provided clarification on various issues as follows:\n\n__Q1:__\nThe affinity of the complex is derived from the dynamic conformation of the protein rather than its precise fixed structure. We employ a diffusion model to characterize the distribution of side-chain conformations, which allows for a more accurate simulation of the true physical interactions of the side chains. \n\nTo elucidate the benefits of the representations learned by the diffusion model for affinity prediction, we incorporated an experiment visualizing these representations after dimensionality reduction using PCA (**Q4 in the \"global\" response and Figure a in the PDF**). In comparison with the representations from the flow model (RDE) and the pretrained language model (ESM-3B), we found that the representations based on the Sidechain Diff can more directly discern the effects of mutations on affinity.\n\n__Q2:__\nWe conducted additional comparisons between our DiffAffinity model and two other baselines: (a) ESM2 with a 2-layer MLP predictor, and (b) feeding ESM2 representations to the DiffAffinity architecture. Importantly, our DiffAffinity model consistently outperformed these baselines in all datasets mentioned.\nThe results of these comparisons can be found in **Table a in the PDF**. The findings from these comparisons align with the conclusions drawn from addressing Q1, both indicating that the hidden representation obtained from SidechainDiff is more suitable for mutation-related tasks. These results further validate the efficacy and superiority of our DiffAffinity model in handling mutation-related challenges.\n\n__Q3:__\nWe thank the reviewer for pointing out the potential confusion caused by the usage of the score network, in the DiffAffinity model, we exclusively use the conditional encoder from SidechainDiff, where the inputs are wild-type sequences and wild-type protein structures to obtain hidden representations. These representations are then fed into DiffAffinity. Consequently, we do not utilize the score network from SidechainDiff, and there is no need to consider the timestep $t$ for the score network in this context. \n\nIn the sampling procedure, we employ the score network to generate samples from the reverse process. The specifics of the sampling procedure and the setting of parameter $t$ can be found in **Algorithm 1 of Section 3.2**. \n\n__Q4:__\nYes, the additional module is an IPA-like transformer. The DiffAffinity model utilizes the hidden representation from SidechainDiff, along with sequence and protein structure information, as input to an IPA-like transformer. This transformer generates the final representation, and the difference between the final representations for the wild-type and mutation is used to predict the $\\Delta \\Delta G$ value. We have now added a detailed description in **Section A.2 of Appendix**. \n\n__Q5:__\nIn our experiments, we observed that DiffAffinity* without any pre-training features outperformed RDE-Network and other methods. Unfortunately, the original paper [1] does not provide the results of RDE-Network without RDE hidden representation, preventing us from directly verifying whether our results align with the original findings. Nevertheless, in downstream tasks of SARS-CoV-2 RBD and human antibodies, we observed a significant improvement in the performance of DiffAffinity over DiffAffinity*. This suggests that the pre-training features from SidechainDiff can indeed enhance the model's generalization ability and the End-to-End model DiffAffinity* may overfit the SKEMPI2 dataset. Meanwhile, we keep the same hyperparameter and model architecture with RDE-Network and ESM2* in the second stage of DiffAffinity. Our model shows superior performance than RDE-Network and ESM2* in downstream tasks which also indicates our hidden representation is more suitable for mutation-related tasks.We extend our analysis about pre-training features in **Q4 in the \"global\" response**.\n\n__Q6:__\nThanks for your attention to computational efficiency, according to our preprocess procedure, we first crop all protein complexes into patches containing 128 residues by first choosing a seed residue, and then choose 127 nearest neighbors based on C-beta distances. Therefore, the computational efficiency of DiffAffinity is not affected by the size of the protein or the number of mutations from the model design. We also experiment to verify this conclusion.\n| Number of mutations | Time |\n|----|----|\n| 1 | 2.33s|\n| 2~5 | 2.33s|\n| >6 | 2.33s|\n\n__Q7:__\nWe have now acknowledged the limitations of our model in the **Conclusion section** in the revision.\n\n\n__Reference__\n\n[1] Luo, Shitong, et al. \"Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction.\" bioRxiv (2023): 2023-02.\n"}, {"review_id": "sOsJkGavTO", "reviewer": "Reviewer_Z2H5", "summary": "The paper introduces a new representation learning-based approach called SidechainDiff that predicts how amino acid mutations influence protein-protein binding using Riemannian diffusion model. It's very important problem in the field of structural biology, for example, in evaluation of antibody variants impact to antigen binding. Moreover, SidechainDiff is the first approach that focuses on side chains compared to previous methods that work with protein backbones. Overall, this paper is a valuable contribution to both computational biology and artificial intelligence. ", "questions": "1. Will SidechainDiff be an open source approach? If so, it would be very useful to prepare a GitHub repository with the code and instruction of how to use the code.\n2. Can the results of SidechainDiff be biased towards SARS-CoV-2 RBD? Have you checked the results on other antibody-antigen systems? Will results remain the same? ", "limitations": "The limitations of SidechainDiff are not clearly written, please provide detailed discussion", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Originality: The paper provides a new solution for a well-known task of the prediction of amino acid mutation impact into binding. Intoduced methods are new (SidechainDiff and DiffAffinity), though authors used an already known concept of diffusion models. Current work focuses on side chain generation while previous studies provides protein backbone generation. The related studies are adequately cited.\nQuality: The submission is technically sound, appropriate, and complete. It provides a clear and detailed explanation of the all steps of the work. In addition, the authors compare DiffAffinity with other models providing a baseline of its performance. The authors provide insights into the further steps of the work and potential applications. \nClarity: The submission is written clearly and organized. It provides all the necessary details for the reader. \nSignificance: Due to the lack of experimentally determined structures there is a need for tools that are able to predict the effect of amino acid mutations to the binding without this information. This paper provides such a tool.", "weaknesses": "The predicted mutational effect was checked only on one type of antibody-antigen system (SARS-CoV-2 RBD). It's not clear if the model will show the same performance  in other antibody-antigen systems or how the performance will change. Also, it's not discuss how the protein characteristics (such as length, mutation position, etc.) can influence the performance and predictions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's constructive comments on our work. In response, we have carefully addressed their concerns and provided clarification on various issues as follows:\n\n__Q1:__\nTo obey the rules of the double-blind reviewing policy, we initially don't provide a GitHub URL. But SidechainDiff will be released soon after the peer review finishes, and we will provide the training, prediction, and instruction of SidechainDiff.\n\n__Q2:__\nDiffAffinity is also practical in other antigen and antibody systems. In response to the question, we introduce additional tasks in other antibody-antigen systems. We aim to predict deep-sequences libraries of the therapeutic antibody trastuzumab for specificity to human epidermal growth factor receptor 2 (HER2) with experimentally validated binary labels. For this purpose, we utilize DiffAffinity, RDE, and ESM2* to discriminate between antigen-binding or non-binding mutations in the CDR H3 region of the 1N8Z antibody protein, where the total number of mutations is 491 [1]. The mutations are classified based on the sign of $\\Delta \\Delta G$, where positive values indicate antigen-binding and negative values indicate non-binding interactions. To address the imbalanced dataset, we employ the AUPRC metric. The results of this task are presented in the table below.\n| Method | AUPRC |\n|----|----|\n|ESM2*|0.739|\n|RDE|0.755|\n|DiffAffinity| 0.768|\n\nCompared with other deep learning methods, our model DiffAffinity also achieves the best performance in this task which indicates our model can easily handle the different datasets.\n\n__Q3:__\nWe acknowledge that one of the disadvantages of our model is its inability to capture the changes in backbone structure caused by mutations.\n\n__Reference__\n\n[1] Mason, et al. Optimization of therapeutic antibodies by predicting antigen specificity from antibody sequence via deep learning. Nat Biomed Eng (2021)"}, {"review_id": "TAbRHJeVZf", "reviewer": "Reviewer_HbdY", "summary": "The paper proposes SidechainDiff a diffusion model over the torsion angles of the protein sidechains and uses the learned representations of this model to predict the mutational effects of protein-protein binding. ", "questions": "See weaknesses section.", "limitations": "See weaknesses section.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper shows a wide variety of promising experimental results ranging from standard benchmarks of binding affinity and side-chain conformation to case studies related to SARS-CoV-2 RBD and antibodies. I cannot comment on their reproducibility because the code was not provided.\n", "weaknesses": "While the experimental results are strong, however for the paper to be ready for publication, the manuscript should be improved in many regards: comparison to the literature, presentation of the method and certain baseline comparisons. \n\nComparison to the literature:\n\n1. The paper lacks references to existing published methods that have developed Riemannian diffusion models over the hyperdimentional torus to model molecular torsion angles (e.g. [1] and [2]). In these regards, the authors should clarify the relationship between the diffusion process presented here and in those works. To my understanding, the diffusion processes are identical which raises the question of why the authors refer to the perturbation kernel as intractable and resort to the use of implicit score matching. [1] suggests that the perturbation kernel can be sampled analytically (with a truncated infinite series). Are the perturbation kernels the same? If so how does the explicit score matching approach of [1] compare to the one developed?\n2. Related to the point before how are the samples X_t and the scores s_t obtained in equation (7)?\n3. Except from the diffusion component, the overall approach of the paper seems analogous to RDE [3]. I believe that given the similarity it would be useful to clarify the differences between the methods. For example line 256 claims the two methods use the same architecture. Is this true and if so does doing the pretraining with the flow of RDE-Network get worse performance than no pretraining at all (DiffAffinity*)?\n\nPresentation of the method:\n\n4. For the network architecture hardly any details are provided in the main text and even in the appendix these are not very clear. The authors should add some details of the architecture to the main text and clarify the presentation in the appendix: e.g. how is the score of the chi angles predicted? Is it an MLP on top of the hidden representation?\n\nBaseline comparisons:\n\n5. The authors should add details about the hyperparameter and model tuning process. E.g. were any hyperparameters tuned for the retrospective studies on SARS-Cov-2?\n6. Given the high-dimensional distribution with complex interdepencies between the angles of the different sidechains, looking simply at the MAE of individual torsion angles seems inadequate. This is especially relevant when some angles have reported MAE above 40 degrees, e.g. what would be the MAE performance of a simple baseline of predicting the median (on the circle) for every residue type and angle? More generally, I believe the authors should also provide further metrics, for example what is the steric clash rate in each of the approches?\n7. Some claims in the paper seem not well justified and should be adjusted. E.g. line 305 (and similar 322) “[SidechainDiff] achieves comparable performance to RDE, suggesting superior capability in generating side-chain conformations”, if they are comparable then the method is not superior.\n\nMinor:\n\n8. The text presents many orthographic errors (e.g. to name just a few at lines 161, 224, 549) I suggest the authors to use publicly available programs to review the text and correct these.\n\n[1] Jing, Bowen, et al. \"Torsional diffusion for molecular conformer generation.\" *NeurIPS 2022*.\n\n[2] Corso, Gabriele, et al. \"Diffdock: Diffusion steps, twists, and turns for molecular docking.\" ICLR 2023.\n\n[3] Luo, Shitong, et al. \"Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction.\" ICLR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's constructive feedback. We have carefully addressed each comment and made the necessary revisions accordingly.\n\n__Q1:__\n1. Thanks for your advice, We have incorporated references to existing published methods [1, 2] in the related work section. \n2. Comprehensive analysis between our method and their methods [1, 2] can be found in **Q6 in the \"global\" response**.\n\n3. Due to time constraints, we were unable to perform a comprehensive comparison between the explicit score matching approach of [1, 2] and the implicit score matching approach used in our paper. In our setting, ISM loss doesn't need to calculate the ground truth score function in torus space $\\mathbb{T}^{4}$ which is consistent with our method, so we choose ism loss as our loss function. We acknowledge that the ism loss has its disadvantages, particularly in its reliance on the calculation of score network divergence, which can be challenging.\n\n__Q2:__\n We use a random walk sample procedure to get $X_t$ which means pertubated $X_0$ . The forward diffusion procedure can be expressed as : $dX_t = d\\mathbf{B}_t$. To be specialized, the forward diffusion procedure involves the following steps:\n\n 1. Sample a point $r$ from a standard normal distribution in $(\\mathbb{R}^2)^4$.\n 2. Use the exponential map to project $r$ onto the tangent space of $X_0$.\n 3. Scale the tangent vector by a factor of $\\sqrt{t}$.\n 4. Project the scaled tangent vector back to the unit circle using the projection map.\n 5. Obtain the point $X_t$ on the unit circle.\n\n We can formalize this sampling procedure on the 4-dimensional torus space $\\mathbb{T}^4$ using the following equation:\n\n $X_t = Proj_{X_0}(\\sqrt{t} Exp_{X_0}(r)), r\\sim \\mathcal{N}(0,I_{(\\mathbb{R}^2)^4}).$\n\nIn Equation 7, We apologize that $s_t$ is a typo. we mean $s_\\theta$ acturally.  $s_\\theta$ is score network. \n\n__Q3:__\n1. We thank the reviewer for pointing out the error in that model architecture. We adopted the same architecture to the RDE-Network for the $\\Delta\\Delta G$ stage. However, it is important to note that we employed a completely different model to handle the side-chain conformation. We use the same architecture to predict binding affinity but use different hidden representations from pre-training modules. Our performance is better than RDE-Network, which indicates that our pre-training module is more suitable for mutation-related tasks.\n\n2. Yes, DiffAffinity* has better performance than RDE-Network in SKEMPI2 dataset. In our experiments, we observed that DiffAffinity* without any pre-training features outperformed RDE-Network. Unfortunately, the original paper [2] does not provide the results of RDE-Network without RDE hidden representation, preventing us from directly verifying whether our results align with the original findings. Nevertheless, in downstream tasks of SARS-CoV-2 RBD and human antibodies, we observed a significant improvement in the performance of RDE-Network over DiffAffinity*. This suggests that the pre-training features from RDE could indeed enhance the model's generalization ability and the end-to-end model DiffAffinity* might overfit the SKEMPI2 dataset.\n\n A comprehensive analysis of our hidden representation obtained from SidechainDiff and the comparation with that from RDE and ESM2 can be found **in Q4 in the \"global\" response and in Figure a in the PDF**.\n \n__Q4:__\nThe architecture of the score network $s_{\\theta}(\\mathbf{X}_t,t,\\mathbf{Z})$ is implemented using a multi-layer perceptron (MLP) with 3 layers, each containing 512 units. In DiffAffinity, we concat the hidden representation from SidechainDiff and sequence information as the sequential input of the IPA-like transformer.\nWe have incorporated additional details regarding our network architecture in both the main text and the appendix.\n\n__Q5:__\nWe appreciate the point raised by the reviewer on the issue of model hyperparameters. We do not employ any fine-tuning techniques in SARS-CoV-2 RBD mutational effects prediction and optimization of human antibodies against SARS-CoV-2 tasks. Instead, we utilize the models trained on the SKEMPI2 dataset and directly use their predictions for the $\\Delta\\Delta G$ of mutations in downstream tasks. \n\n__Q6:__\n1. We have incorporated more metircs and methods [3, 4] of side-chain conformation **in Q6 in the \"global\" response and in Table c in the PDF**.\n2. Additionally, we experimented to predict the median (on the circle) for each residue type and angle, and the results are presented in the table below. \n   \n    | Method | $\\chi_1$ | $\\chi_2$ | $\\chi_3$ | $\\chi_4$ |\n    |--------|---------|---------|---------|---------|\n    | Random | 89 | 98 | 68 | 114.64 |\n\n__Q7:__\nWe have made the necessary corrections to rectify all the writing mistakes in our paper. We sincerely appreciate your valuable advice and guidance\n\n__Reference__\n\n[1] Jing, Bowen, et al. \"Torsional diffusion for molecular conformer generation.\" Advances in Neural Information Processing Systems 35 (2022): 24240-24253.\n\n[2] Corso, Gabriele, et al. \"Diffdock: Diffusion steps, twists, and turns for molecular docking.\" ICLR 2023.\n\n[3] Luo, Shitong, et al. \"Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction.\" bioRxiv (2023): 2023-02.\n\n[4] McPartlon, Matthew, and Jinbo Xu. \"An end-to-end deep learning method for protein side-chain packing and inverse folding.\" Proceedings of the National Academy of Sciences 120.23 (2023): e2216438120.\n\n[5] Misiura, Mikita, et al. \"DLPacker: deep learning for prediction of amino acid side chain conformations in proteins.\" Proteins: Structure, Function, and Bioinformatics 90.6 (2022): 1278-1290."}]

Author rebuttal:
In our “global” response, to address reviewers’ main concerns,  we elucidate our motivation, delineate the distinctions between our approach and previous methods (including the original Riemannian diffusion model and torsion diffusion model), and present new results (including comparison with ESM2, visualization analysis of the learned representation, and expanded method comparisons for side-chain conformations)

__Q1: Motivation__

Given limited annotated experimental data, we utilize representation learning on unlabeled data to improve mutational effect prediction in protein-protein binding. Recognizing the inherent flexibility of side-chain conformations, we introduce a conditional diffusion model to capture the dynamic nature of side-chain conformations, rather than relying on a fixed conformation. Our downstream task results and visualization analysis (**Figure a in the PDF**) of learned representations demonstrate its effectiveness.

__Q2: Difference with Riemannian Diffusion Model__

Our primary modifications to adapt the original Riemannian diffusion model for our task are as follows:
1. We extend the original Riemannian diffusion model to a conditional diffusion model, where we jointly learn the conditional vector and the diffusion process. Specifically, we encode the structural context of the mutation as a conditional vector and learn the generative process of its side-chain conformations in a 4-dimensional torus space.

2. We explore the model's capacity for representation learning. Side-chain conformations are inherently flexible, with the degree of flexibility dependent on their structural context. Therefore, we hypothesize that a representation learning approach, aimed at learning the distribution and generative process of side-chains rather than a point estimation, can lead to improved performance in downstream tasks.


__Q3: Additional Benchmark of ESM2__

We included pre-trained language model baselines in SKEMPI2 dataset. We conducted comparative experiments with ESM2-3B + 2-layer MLP and ESM2-3B + DiffAffinity (**Table a in the PDF**). Notably, DiffAffinity achieved state-of-the-art results on almost all benchmarks within the SKEMPI2 and SARS-CoV-2 datasets.

__Q4: Pre-training representation Analysis__

To assess the pre-trained representative capacity of SidechainDiff based on the diffusion model (**Figure a (a) in the PDF**), we compared it with other methods RDE (**Figure a (b) in the PDF**) and ESM2-3B (**Figure a (c) in the PDF**).
We calculated the difference of hidden representations between wild-type and mutant proteins obtained from pre-training methods and applied PCA to reduce the dimensions of the representations from SKEMPI2. We visualized the distribution of the representations and colored them based on their $\Delta\Delta G$ values.
Compared to other pre-trained methods, the representations from our model can more effectively discern affinity changes caused by mutations in the latent space.


__Q5: Side-chain Conformation Results__

We provided a sidechain conformation comparison with baselines based on side-chain packing methods, such as AttnPacker [1] and DLPacker [2]. Our model achieves comparable results with these methods which aim to predict side-chain conformations accurately.
The results can be found in **Table c in the PDF**. We also discuss the diversity of side-chain conformation generated by SidechainDiff in Append. D.3.2

We also use the steric clash number metric [1] to evaluate the quality of the side-chain conformations generated. Even if Attnpacker adds additional steric clash loss, DiffAffinity also achieves SOTA results in this metric across all representative methods. 

__Q6: Difference with existing torsion angle diffusion model__

In this paper [4], both methods view the torus space $\mathbb{T}^4 \cong [0,2\pi)^4$ and set the exponential map on the torus as $\mathrm{exp}_{x}(y) = x + y\ \mathrm{mod}\ 2\pi$. 

In contrast, our model views the torus space $\mathbb{T}^4 \cong (\mathbb{S}^1)^4$ and sets the exponential map on the unit circle $\mathbb{S}^1$ as $\mathrm{exp}_{\mathbf{\mu}}(\mathbf{v}) = \cos(\Vert \mathbf{v} \Vert)\mathbf{\mu}+\sin(\Vert \mathbf{v} \Vert)\frac{\mathbf{v}}{\Vert \mathbf{v} \Vert}$, where $\mathbf{\mu} \in \mathbb{S}^1$ and $\mathbf{v} \in \mathbb{R}^1$. To handle the 4-dimensional torus space $\mathbb{T}^4$, we utilize projection mapping and exponential mapping on the unit circles of each dimension. By applying these mappings to each dimension and taking the Cartesian product of the results, we obtain the exponential mapping $\mathrm{Exp}$ and projection mapping $\mathrm{Proj}$ on the 4-dimensional torus space $\mathbb{T}^4$.

The perturbation kernel is impossible to get an exact solution but requires numeric approximation. The perturbation kernel $p_{t|0}$ in the torus space $\mathbb{T}^4 \cong [0,2\pi)^4$ is proposed as:

$p_{t|0}(x'|x) \propto \sum_{d\in \mathbb{Z}^m}\mathrm{exp}(-\frac{\Vert x-x' + 2\pi d\Vert^2}{2\sigma^2(t)})$. 

While the two exponential maps are equivalent to Riemannian exponential maps on the torus and the diffusion process remains the same in the 4-dimensional torus space $\mathbb{T}^4 \cong (\mathbb{S}^1)^4$, using the perturbation kernel $p_{t|0}$ in our paper requires complex derivation based on our specific exponential map. With implicit score matching loss, we are not required to approximate the perturbation kernel numerically. 

__Reference__

[1] Matthew et al. An end-to-end deep learning method for protein side-chain packing and inverse folding. Proceedings of the National Academy of Sciences (2023)

[2] Mikita, et al. DLPacker: deep learning for prediction of amino acid side chain conformations in proteins. Proteins: Structure, Function, and Bioinformatics  (2022)

[3] Lin et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science (2023)

[4]  Bowen et al. Torsional diffusion for molecular conformer generation. NeurIPS (2022)

Meta-review: This paper proposed a new diffusion model called SidechainDiff to learn the conformational distributions of protein side chains. The diffusion models also learns a contextual representation of mutations on the ppi-interface. Experimental results on existing benchmarks prove the effectiveness of the proposed approach. 


The reviewers in general like the proposed diffusion model, which is new and original. The paper would benefit from (1) more discussion on existing methods on side-chain packing and comparison in the experiments; (2) Testing on more antigen than the SARS-COV-2 target.
Decision: Accept (poster)