Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction
Authors: Shitong Luo
Yufeng Su
Zuofan Wu
Chenpeng Su
Jian Peng
Jianzhu Ma
Venue: ICLR 2023 poster (2023)
Keywords: effect of mutations
protein-protein interaction
unsupervised learning

Abstract: Protein-protein interactions are crucial to many biological processes, and predicting the effect of amino acid mutations on binding is important for protein engineering. While data-driven approaches using deep learning have shown promise, the scarcity of annotated experimental data remains a major challenge. In this work, we propose a new approach that predicts mutational effects on binding using the change in conformational flexibility of the protein-protein interface. Our approach, named Rotamer Density Estimator (RDE), employs a flow-based generative model to estimate the probability distribution of protein side-chain conformations and uses entropy to measure flexibility. RDE is trained solely on protein structures and does not require the supervision of experimental values of changes in binding affinities. Furthermore, the unsupervised representations extracted by RDE can be used for downstream neural network predictions with even greater accuracy. Our method outperforms empirical energy functions and other machine learning-based approaches.

Published as a conference paper at ICLR 2023
ROTAMER DENSITY ESTIMATOR IS AN UNSUPER- VISED LEARNER OF THE EFFECT OF MUTATIONS ON PROTEIN-PROTEN INTERACTION
Shitong Luo"r, Yufeng Su Zuofan Wu', Chenpeng Su², Jian Peng Jianzhu Ma
Helixon Research
2 University of Illinois Aibana-Champae
3 Institute for AI Industry Research, Tsinghua University
mnsst@ellixon com, luost 60gmail.....mm wwa.gaal..mm com j jianpenge edu, ma maj.anzzuettsinhuuaae edu. en
ABSTRACT
Protein Protein-protein interactions are crucial to many biological processes, and predict- ing the effect of amino acid mutations on binding important for protein engi- neering. While data-driven approaches using deep learning have shown promise, the scarcity of annotated experimental data remains major challenge. this work, we propose new approach that predicts mutational effects binding us- ing the change in conformational flexibility of the protein-protein interface. Our approach, named Rotamer Density Estimator (RDE), employs flow-based gen- crative model estimate the probability distribution of protein side-chain confor- mations and uses entropy to measure flexibility. RDE is trained solely protein structures and does not require the supervision of experimental values of changes in binding affinities. Furthermore, the unsupervised representations extracted by RDE can be used for downstream neural network predictions with even greater accuracy. Our method outperforms empirical energy functions and other machine learning- baaeing-basedd approaches.
INTRODUCTION
Proteins rarely act alone and usually interact with other proteins to perform a diverse range of bio- logical functions (Alberts & Miake-Lye, 1992; Kastritis & Bonvin, 2013). For example, antibodies, a type of immune system protein, recognize and bind to proteins on pathogens' surfaces, eliciting immune responses by interacting with the receptor protein of immune cells (Lu al., 2018). Given the importance of protein-protein interactions in many biological processes, developing methods to modulate these interactions is critical. A common strategy modulate protein-protell interactions is to mutate amino acids on the interface: some mutations enhance the strength of binding, while others weaken or even disrupt the interaction (Gram et al., 1992; Barderas et al., 2008). Biologists may choose to increase or decrease binding strength depending their specific goals. For example, enhancing the effect of neutralizing antibody against virus usually requires increasing the bind- ing strength between the antibody and the viral protein. However, the combinatorial space of amino acid mutations large, so it is not always feasible or affordable to conduct wet-lab assays to test all viable mutations. Therefore, computational approaches are needed to guide the identification of desirable mutations by predicting their mutational effects on binding strength, typically measured by the change in binding free energy (AAG).
Traditional computational approaches are mainly based on biophysics and statistics (Schymkowitz al., 2005; Park al., 2016; Alford et al., 2017). Although these methods have dominated the field for years, they have several limitations. Biophysics-based methods face a trade-off between efficiency and accuracy since they rely on sampling from energy functions. Statistical methods more efficient, but their capacity is limited by the descriptors considered in the model. Furthermore, both biophysics and statistics-base methods heavily rely on human knowledge, preventing it to
"Equal contribution.

---

Published as a conference paper at ICLR 2023
benefit from the growing availability of protein structures. As result, predicting the effects of mutations on protein-protein binding remains an open problem.
Recently, deep learning has shown significant promise in modeling proteins, making data-driven approaches more attractive than ever (Rives et al., 2019; Jumper et al., 2021). However, developing deep learning-based based models to predict mutational effects due to the scarcity of experimental data. Only a few thousand protein mutations, annotated with changes in binding affinity, are publicly available (Geng et al., 2019b), making supervised learning challenging due to the potential for overfitting with insufficient training data. Another difficulty the absence of the structure of mutated protein- proteiroote complexes. Mutating amino acids on protein complex leads to changes mainly in sidechain conformations (Najmanovich et al., 2000; Gaudreault et al., 2012), which contribute the change in binding free energy. However, the exact conformational changes upon mutation are unknown.
protein-proteii binding is challenging
In this work, we draw inspiration from the thermody- namic principle that protein-protein binding usually leads to entropy loss on the binding interface, which can used to determine binding affinity (Brady & Sharp, 1997;
Unbound state
B
Cole & Warwicker, 2002; Kastritis & Bonvin, 2013). When two proteins bind, the residues located at the in- terface tend to become less flexible (i.e. having lower en- tropy) due to the physical and geometric constraints im- posed by the binding partner (Figure 1). A higher amount of entropy loss corresponds to a stronger binding affinity, Therefore, by comparing the entropy losses of wild-type and mutated protein complexes, we can estimate the ef- fect of mutations on binding affinity. Please refer to Sec- tion in the appendix for a detailed discussion.
Flexible / Higher Entropy
Rigid / Lower Entropy
A-B Complex
Bound state
Based on this principle, we introduce a novel approach to predict the impact of amino acid mutations on protein- protein interaction. The core of our method is Rotamer Density Estimator (RDE), a conditional generative model that estimates the density of amino acid sidechain conformations (rotamers). We the entropy of the estimated density as a metric of conformational flexibility. By subtracting the entropy of the separated proteins from the entropy of the complex, we obtain estimation of binding affinity. Finally, we can assess the effect of mutations by comparing the estimated binding affinities wild- type and mutant protein complexes. In addition to directly comparing entropy, also employ neural networks to predict AAG from the representations learned by RDE.
The conformational flexibil- ity of the interface generally decreases upon binding.
Figure 1:
Our method is attempt address the aforementioned challenges. Rotamer Density Estimator is trained solely on protein structures, not requiring other labels, making it an unsupervised learner of the mutation effect on protein-pree protein interaction. This feature mitigates the challenge posed by the scarcity of annotated mutation data. Moreover, our method does not require the mutated protein structure as input. Instead, treats mutated structures as latent variables, which are approximated by RDE, Our method outperforms both empirical energy functions and machine learning models for predicting AAG. Additionally, as a generative model for rotamers, RDE accurately predicts sidechain conformations.
RELATED WORK
2.1 MUTATIONAL EFFECT PREDICTION FOR PROTEIN- PROTEIN-PROTEIN INTERACTION
Traditional approaches to predicting the effect of mutation on protein binding can be roughly divided into two classes: biophysical and statistical methods. Biophysical methods utilize energy functions to model inter-atomic interactions. These methods sample conformations of the mutated protein complex and predict changes in binding free energy (Schymkowitz et al., 2005; Park et al., 2016; Alford et al., 2017: Steinbrecher et al., 2017). Statistical methods rely feature engineering, which uses descriptors summarizing geometric, physical, evolutionary, and motif properties of proteins to predict mutational effects (Geng al., 2019a; Zhang et al., 2020). Traditional methods face the
2

---

Published as a conference paper at ICLR 2023
trade-off between speed and accuracy. Their performance depends heavily on human expertise, which limits their pace to improve with the fast-growing of available protein structures.
Recently, deep learning- based approaches have emerged. We group them into three categories: end- to-end models, pre-training-ber baasin models, and unsupervised models. End-to-end models directly predict the difference in binding free energy by taking both mutant and wild-type protein struc- models attempt to address data scarcity by
tures as input (Shan et al., 2022). Pre-training- pre-training feature extraction network (Liu al., 2021; Yang et al., 2022; Zhang et al., 2022). However, most pre-training tasks are not designed to capture the foundation of protein-protein inter- actions. Unsupervised models adopt the mask-predict paradigm to protein 3D structures, partially masking amino acid types on given protein backbone, and recovering the masked information using neural networks (Wang et al., 2018; Shroff et al., 2020; Jing et al., 2020; Yang et al., 2022; Hsu et al., 2022). These models can serve as unsupervised predictors of the mutational effects on binding, as the difference in the probability of amino acid types before and after mutation correlates mildly to the change in binding free energy.
2.2 MUTATIONAL EFFECT PREDICTION FOR SINGLE PROTEINS
The prediction of mutational effects for single proteins can be achieved using either structure-based or cauuence-based (evolution-base -base) approaches. Structure- methods can be categorized into biophysical, statistical, and deep learning-basee based methods, which aim to predict the thermal stability or fitness of the protein rather than the binding free energy between proteins (Schymkowitz et al., 2005; Park al., 2016; Alford et al., 2017; Lei et al., 2023). Sequence-based methods rely on the mining of evolutionary history, done by performing statistics on multiple sequence alignments (MSAs) constructed from large-scale sequence databases (Hopf al., 2017; Riesselman al., 2018; Rao et al., 2021; Luo et al., 2021; Frazer et al., 2021), or leveraging protein language models (PLMs) (Meier et al., 2021; Notin et al., 2022).
However, it is important to note that sequence-based methods are not suitable for predicting muta- tional effects on general protein- protein interactions due to the lack of evolutionary information in many cases. Protein-protein interactions typically involve two or more chains, which may belong to different species or may not experience inter-chain co-evolution. As such, it infeasible to pre- dict mutational effects via mining sequence databases using existing powerful tools such as MSAs or PLMs. Thus, effective ways for predicting mutational effects on on structure-basee approaches rather than sequences alone. We present detailed discussion and supporting experimental results in Section C.1 of the appendix.
interaction rely
METHOD
3.1 OVERVIEW AND PRELIMINARIES
Overview Our method comprises three main components. The first Rotamer Density Estimator (RDE), which is a conditional normalizing flow that models the probability density of sidechain conformations (rotamers) based on the amino acid type and backbone structures (Section 3.2). The second component is algorithm that estimates the entropy of the distribution parameterized by the normalizing flow (Section 3.3). Lastly, we describe how we use the entropy of the protein-protein interfaces in both the mutated and wild-type states, both bound and unbound, to estimate the change in binding free energy (AAG). We also detail how we use neural networks to achieve more accurate predictions of AAG using the unsupervised representations from RDE (Section 3.4).
Definitions and Notations A protein-protere complex is a multi-chain protein structure that can be divided into two groups. Each group contains at least one protein chain and each chain consists of multiple (amino acid) residues. For protein complex containing residues, we number them from 1 to n. The two groups of the complex can be represented by two disjoint sets of indices A, B C (1...n). A residue is characterized by its type, position, orientation, and sidechain conformation. We denote the type. position, and orientation of the i-th (i E .I....n) residue as a, E {(1..220 20), P, R33 and O; SO(3) respectively. The sidechain conformation of the residue is called rotamer. As the conformational degree of freedom of the sidechain is defined by rotatable bonds, a rotamer can be parameterized by torsional angles w.r.t. the rotatable bonds. The number of torsional angles varies between 0 to 4 depending on the residue type. For residue with d torsional angles, we denote the k-th (I: {1 ..) torsional angle by x; (l)
E [0, 2m). Collectively, all the torsional angles
3

---

Published as a conference paper at ICLR 2023
k=1- Using the language of geometry, an angle can be represented by a point on the unit circle $* A vector consisting of d angular values resides on the product of d unit circle, known as the d-dimensional torus T = (S
denoted by vector X;
(x: (k)
In this work, our first goal is to model the conditional probability density of rotamers, given the type, position, orientation, and prior rotamer of itself and other residues: p(x: {aj. Oj, X }=1) The prior rotamers Xj are often inaccurate or unknown. For example, if we mutate some residues, the rotamers of the mutated residues are unknown, and the rotamers of residues nearby the mutated ones are inaccurate because they are affected by the mutation. The probability density is deñned over the d-dimensional torus TD =S')D, and we describe below the flow-based architecture to model the density.
3.2 ROTAMER DENSITY ESTIMATOR
Rotamer Density Estimator (RDE) designed to estimate the conditional distribution of the rotamer of the i-th residue, given the information of itself and other residues: p(x: {aj. Pj. Oj, X; f-=1) In this section, we first introduce the encoder network that produces hidden representations for each residue, taking into account its environment (at, Pt, O., x=) Next, we present a conditional normalizing flow defined on s' for rotamers with only 1 torsional angle. Based on the S flow, we further extend the flows to d-dimensional torus T (D > 1) for rotamers with more than torsional angle.
The encoder network starts with two multi-layer perceptrons (MLPs) that gen- erate embeddings for each individual single residue and each pair of residues respectively. The MLP for single residues encodes the residue type, backbone dihedral angles, and local atom coordinates into a vector e; (i = 1. .1..my n). The other MLP for residue pairs encodes the distance and the rela- tive position between two residues. We denote a pair embedding vector as Zij (i.j = 1...n). n). To transform the single embeddings e and pair embeddings Σij into hidden representations h, we use a self-attention- based network that is invariant to rotation and translation (Jumper et al., 2021). The hidden representation h, aims to capture both the information of the i-th residue itself and its struc- tural environment. It serves as an encoding of the condition {a,, Pj. O,, Xز for the probability density with respect to Xs.
Encoder Network
Conditional Flow on S To model the distribution of a rotamer with torsional angle, we utilize conditional normalizing flows on S² (Rezende et al., 2020). A normalizing flow is bijective func- tion, and to construct one on S1, we parameterize it by an angle E 0,.r). and define a bijective function on [0, 2m] that is equivalent to bijective over S!. A common method for constructing bijective function is to ensure strict monotonicity, which we adopt by constructing a strictly mono-
tonically increasing function on [0, 2x.. denoted by f [0, 2x + [0, 2nr.. Notably, due to the periodicity of angular values, and 2n are the same point, so to preserve continuity at both ends, we must ensure that f(0)= = 0, f(2m)= quadratic spline flow (Durkan et al., 2019; Rezende et al., 2020), a piece-wise function that contains K pieces delimited by K + knots. Each piece takes the form:
2n, and (0) f2(2T). To achieve this, we use the rational
(yk+1 3/k) [<x) + 56 (1 (x(x)) (x))
(2/ Mk.411 6x, yk
(1)
8% [8k+1 + 8g 2sk] (1 (x)) (xx)
where 84 = 1k+1 yk and E(x) Ik+1 - Fk
1 Tb (w E [Tx, xk+1));
(2)
Øk+1 - Dkk where the spline is parameterized by the coordinates and derivatives of the K + knots denoted by Ik, yes and dR (k 1...K
K + 1). To fulfill the requirement of monotonicity, continuity, and periodicity, we impose the following constraints on the knots: i. 0= =0 x < x2 <... < Zx = 2n, ii. 0 y1 y/2 < ... y 2v, iii. òz > 0(k 1... + 11, and iv. 81 8x+1- These parameters are produced by transforming the hidden representation h; of residues using neural networks. so the probability distribution defined by the bijective is conditional on the residue and its environment. Therefore, we may also denote the spline by f(x(h.).
We choose the uniform distribution on [0, 2TJ as the base distribution, represented by pz(2) (2 E [0. 2T)). The mapping from the target distribution to the base distribution is denoted by J. According to the offnng----vvriablll formula, the target rotamer density of the i-th residue given by:
log h.) = logps (f(=)) If (x)h::) = log 2n log
(3)
4

---

Published as a conference paper at ICLR 2023
(A)
(B)
XI
Encoder Network
XP
X2
Amino Acids
Embeddings
Coupling Layer
Rotamer Distribution On
336
Knots
IN
(Ata (Fk)
Sum
[6 [6....4x]
Uniform Distribution
Spline-Based Bijective
Distribution
Figure 2: (A) The overall architecture of Rotamer Density Estimator (RDE) for estimating distri- butions of rotamers with torsional angle. (B) Invertible coupling layers alternating between different dimensions enable modeling distributions of rotamers with multiple torsional angles.
The derivative f" can be computed analytically according to Eq. See Figure 2A for an illustration of the model.
In practice, we stack multiple bijectives to enable more complex transformation. The derivative of efficiently
the composite can be computed efficiently using the chain rule. At inference time, compute the inverse mapping f (Rezende et al., 2020): To find the solution of f f- (v), the first step locate the unique bin that contains y. Assuming y belongs the k-th bin, finding its corre- sponding æ amounts to finding the root of the quadratic equation fx (3r|24,,441, 1k, 0k, the interval [TA T1 for which a closed-form solution exists.
Conditional Flow on T Rotamers with D torsional angles can be viewed as points on the D- dimensional torus which is the product of D circles S i.e. TD = S x ... x $ To model the distribution on TD, we adopt the coupling layer technique to model the joint distribution (Dinh et al., 2016). Each coupling layer updates one dimension using the bijective for S1, keeping the other dimensions fixed, and uses the of the residue as the conditioner to parameterize the bijective (Figure 2B):
dimensions along with the hidden representation
[ ( xz,
j j= mod i mod D
ga ga( (x))))))
(4)
where d is the dimension to update. The coupling layer Ja preserves invertibility and has closed- form inversion. The determinant of its Jacobian is equal to the derivative of (ır,læn), h.). We stack multiple spline-based coupling layers to ensure that all the dimensions updated at least once, resulting in a stack of L coupling layers: = 91 o 92 ..... o gr. (L D).
We choose the uniform distribution the base distribution, denoted by P=(2) = ) and let map the target distribution to the base distribution. The log-likelihood is computed using the -hange-of-variable rule: logp(a: log logg=((()))) log /ee V=g(z))
(5)
= log 2x + log ((aaaylaaa h.)
Training Objective To train RDE, we minimize the negative log-likelihood of native (ground truth) rotamers:
min R E S) (x
where S {aj. P.. O,, 1 is protein structure sampled from the dataset. Since the model is designed for mutation analysis, the prior rotamers should emulate the rotamers after some residues mutated. Thus, mask the rotamers of random portion of residues and add noise to the tamers of the neighbors of the masked ones. The negative likeeiioddood training objective function evaluated on the masked residues and random part of the perturbed residues.
5

---

Published as a conference paper at ICLR 2023
3.3 ROTAMER ENTROPY ESTIMATION
RDE models the distribution of possible rotamers of residue, making the entropy of the rotamer distribution a natural measure of conformational flexibility (Brady & Sharp, 1997):
((S,i) = Ex>ppp (- log lo)
(7)
To estimate the entropy, we use a stochastic method:
First, we sample a set of rotamers from the distribution using the inverted flows 1,q.1,4) Then, we compute the negative log probability of the samples and take their average as an estimate of the entropy. Computing these steps is efficient thanks to the ability to compute the exact likelihood of normalizing flows.
For residues without rotatable sidechains (alanine, glycine, and proline), we consider their con- formational entropy be constant. Nonetheless, residues to evaluate their impact on conformational flexibility.
still estimate the entropy of neighboring
3.4 MUTATIONAT. EFFECT (AAG) PREDICTION
Let consider protein-protein complex WLR = fas. 0i, Xi consisting of N residues, where M of them belong the first group long to the second group R = {M + 1, -....,NN N]. We refer to group L as the ligand group since it contains mutations, and group as the receptor group. In the unbound state, we rep- resent the two separated structures by Wr. and Wn respectively. If we mutate m residues in group L, numbered by {1..... m) C L, the mutated structure in the bound state denoted by MLR {ā., Pt, O,, )) (a, Pt, O. x. Jummutated. where ā represents the mutated residue type and Ø indicates that the rotamers of the mutated residues are unknown. The two groups in the un- bound state are denoted by Mr and Mr respectively. Note that MR -W if no mutations are in the receptor group.
I......); and the remaining N - M be-
Linear Predictor
The entropy loss upon binding of a protein complex S (S = W.M is defindd as the difference of entropy between the bound and the unbound states (Kastritis & Bonvin, 2013). We approximate the entropy loss AHs using a linear model and our estimated entropy:
AHs
ST.
bound
Σ SEL
(H E(a,)) + S
SR
bound Σ
(H(S E(a)))
iER
Hiren bound ligand
Hatored bound receptor
unbnd
Σ
(H(SL + E(a:)) + SR
unhnd
Σ
(H(S + E(a)))
(s- W.MM). (8)
St
EE..
ER
Frunted unhound ligand
Frusbed unhound receptor
where WSJ > - J L, R) is the coefficient that controls the contribution of entropy to the binding free energy, and E()) is the entropy bias for 20 different amino acid types. The ther- modynamic definition states that the core component in binding AAG is the difference in entropy loss between the wild-type and mutant structure. Therefore, we estimate AAG by:
=
AAGpred AHM - All +
=(w hound H + bound H (w WMR
unhad H + unhnd WMR H
(9)
(woyrdd Hhhoadd + WWR
bound Hboomd + unhad Huuhhnd
unbod -WWR Hunbad b.
Note that as we assume there are no mutations in the receptor group (MR WR). H and Humbmd cancel out each other and do not contribute AAG.
To calibrate the parameters in Eq.9, we use the block coordinate descent method, which alternates between optimizing the coefficients (wsJ. b) and the entropy biases (E) using the mean squared error (MSE) loss.
Neural Network Predictor Each residue's 's hidden representation h, used to parameterize the nor- malizing flows contains sufficient information about the rotamer distribution. To extract binding in- formation from these representations in more flexible way, employed neural networks. Specif- ically, we utilized a network that shares the same architecture as the encoder to transform the repre- sentation h, and applied max-pooling obtain a global structure representation. We then subtracted

---

Published as a conference paper at ICLR 2023
the representation of the wild-type structure from the mutant representation and fed it into another MLP to predict AAG. To enforce anti- symmmttyy we swapped the wild-type and mutant to predict -AAG, and computed (AÁG ( (-AAG)) /2 as the final prediction. The network was trained using the MSE loss. During training, we freeze the weights of RDE and do not back-propagate gradients through h, fully exploit the unsupervised representations learned by RDE.
3.5 MODEL TRAINING
The dataset for training RDE is derived from PDB-REDO (Joosten et al., 2014), which is a database containing refined X-ray structures in PDB. The protein chains are clustered based on 50% sequence identity, leading 38,413 chain clusters, which are randomly divided into the training, validation, and test sets by 15%%..5%/455%. respectively. During training, the data loader randomly selects a cluster and then randomly chooses a chain from the cluster to ensure balanced sampling. We crop structures into patches containing 128 residues by first choosing a seed residue, and then selecting its 127 nearest neighbors based on C-beta distances. To simulate mutations, we masked the rotamers of 10% of residues in the patch, and we added noise to the rotamers of residues whose C-beta distance to the closest masked residue was less than 8A.
The SKEMPI2 database (Jankauskaité et al., 2019) is used to train the models for AAG prediction described in Section 3.4. We split the dataset into 3 folds by structure, each containing unique protein complexes that do not appear in other folds. Two folds are used for training and validation, and the remaining fold is used for testing. This approach yields different sets parameters and ensures that every data point in SKEMPI2 is tested once.
4 RESULTS
4.1 PREDICTION OF THE EFFECT OF MUTATIONS ON BINDING
Baselines We evaluate the performance of our two AAG predictors, RDE-Linear and RDE- Network, against several categories of bascline methods. The first category comprises traditional empirical energy functions, including Rosetta Cartesian ddG (Park et al., 2016; Alford et al., 2017: Leman et al., 2020) and FoldX (Delgado et al., 2019). The second category consists of methods, represented by ESM-1v (Meier et al., 2021), PSSM (position- specific scoring matrix), MSA Transformer (Rao et al., 2021), and Tranception (Notin et al., 2022). The third category includes end-to-end learning models, such as DDGPred (Shan et al., 2022), and a model that shares the same encoder architecture as RDE but uses an MLP to di- rectly predict AAG' (End-to-End). The fourth category consists of learning methods, including ESM-IF (Hsu et al., 2022) and Masked Inverse Folding (MIF) (Yang al., 2022). Similar to our RDE-Network, this class of methods pre-train network structures and use the pretrained representations to predict A AAGs. The baseline MIF network also uses the same encoder architecture as RDE for comparison. There are two variants for AAG prediction: MIF-Alogit, which uses the difference in log-probability of amino acid types to predict AAG, and MIF-Network, which predicts AAG from the learned representations using the same network ar- chitecture as RDE-Network. Finally, given that our method is based on conformational flexibility, we train a network to predict the B-factor of residues and use predicted B-factors in place of entropy to predict AAG.
sequence/evolution-beee
Metries We use five metrics to evaluate the accuracy of AAG prediction: Pearson correlation coefficient, Spearman's 's rank correlation coefficient, minimized RMSE (root mean squared error), minimized MAE (mean absolute error), and AUROC (area under the receiver operating charac- teristic). To calculate AUROC, mutations are classified based on the sign of AAG. In practical applications, the correlation for one specific protein complex is often of greater interest. There- fore, we group mutations by structure, discard groups with less than 10 mutation data points and calculate correlations for each structure separately. This leads to two additional metrics: average per-structure Pearson correlation coefficient and average per-structure Spearman correlation co- efficient.
Results According to Table 1, our RDE-Network outperforms all the baselines. Notably, it demon- strates significant improvement in per-structure correlations, indicating its greater reliability for practical applications. The superior performance of RDE-Network over MIF-Network suggests that representations derived from fitting rotamer densities more effective than those from masked in-
7

---

Published as a conference paper at ICLR 2023
Table Evaluation of AAG prediction on the SKEMPI2 dataset. RDE-Network outperforms base- line methods. Most notably, RDE-Netword significantly improves per-structure correlations, which are more relevant to practical applications.
Per- PtrrStruc
Overall RMSE
Category
Method
Pearson
Spearman
Pearson
Spearman
MAE
AUROC
ESM-1v
0.0073
-0.0118
0.1921
0.1572
1.9609
1.3683
0.5414
Sequence
PSSM
0.0826
0.0822
0.0159
0.0666
1.9978
1.3895
0.5260
Based
MSA Transf.
0,1031
0.0868
0.1173
0.1313
1.9835
1,3816
0,5768
Tranception
0.1348
0.1236
0.1141
0.1402
2.0382
1.3883
0,5885
Energy
Rosetta
0.3284
0.2988
0.3113
0.3468
1.6173
1.1311
0.6562
Function
FoldX
0.3789
0.3693
0.3120
0.4071
1.9080
1.3089
0.6582
DDGPred
0.3750
0,3407
0.6580
0,4687
1.4998
1.0821
0.6992
Supervised
End-to-Ene End
0,3873
0.3587
0.6373
0.4882
1.6198
1.1761
0.7172
B-factor
0.2042
0.1686
0.2390
0.2625
2.0411
1,4402
0.6044
Unsup. / Semisup.
ESM-IF
0,2241
0,2019
0.3194
0,2806
1.8860
1.2857
0,5899
MIF-A MIF-Alogit
0.1585
0.1166
0.2918
0.2192
1.9092
1.3301
0,5749
MIF-Net.
0.3965
0.3509
0.6523
0,5134
1,5932
1.1469
0.7329
RDE-Linear
0.2903
0.2632
0.4185
0.3514
1.7832
1,2159
0.6059
Ours
RDE-Net.
0.4448
0.4010
0.6447
0.5584
1.5799
1.1123
0.7454
Per Strunture Spearman
Peanson: 0,4158
10
Pearson: 0 0.6540
1-
RDE- Linear AdG
Spearman: 0.3614
RDE-NNetwork A.GG
5
Spearmant D. 3.6422
2
-10
Rosetta
FoldK
RDE Linest
DDGPred
End-o-EEd End
MIF Network
Ret-Newwork
-10
0
10
-10
10
Experimental
Experimental
Figure 3: Left: The distribution of per-structure Spearman correlation coefficients. relation between experimental AAGs and AAGs predicted by RDE-Linear. Right: Correlation between experimental AAGs and AAGs predicted by RDE-Network.
Middle: Cor-
verse folding, as RDE captures atomic interactions well by modeling the conformation of sidechain atoms.
RDE-Linear achieves comparable performance to Rosetta and outperforms some unsupervised learning baselines. While does not surpass most baseline methods over the entire SKEMP12 dataset, we observe that its performance is better when considering only single-point mutations (Table in the appendix). This might be attributed to the fact that simple linear models cannot cap- ture well the non-linear relationship dominating multi-poin mutations. Nevertheless, RDE-Linear demonstrates that using the basic statistics of the estimated rotamer density alone can predict AAG, which lays the foundation for the more accurate RDE-Network.
Sequence-basee models do not accurately predict AAG for protein-protein binding, as discussed in Section 2.2. Figure 3 shows the distribution of per-complex correlation coefficients. Please refer to Section C of the appendix for more results and discussion.
4.2 OPTIMIZATION OF HUMAN ANTIBODIES AGAINST SARS-COV-2
In Shan et al. (2022), the authors report five single-point mutations on human antibody against SARS-CoV-2 that enhance neutralization effectiveness. These mutations are among the 494 possible single-point mutations on the heavy chain CDR region of the antibody. We use the most competitive methods benchmarked in Section 4.1 to predict AAGs for all the single-point mutations and rank them in ascending order (lowest AAG in the top). The effectiveness by the number of favorable mutations ranked in the top place. As shown in Table 2, RDE-Netwon and DDGPred successfully identify three mutations (Ranking < 10%), with RDE-Network ranking them higher.
predictor is determined
8

---

Published as a conference paper at ICLR 2023
Table 2: Rankings of the five favorable mutations on the human antibody against SARS- 2-V- by various compet- itive methods. RDE-Network ranks 3 of the 5 mutations in the top place (<10%).
Table 3: Linear regression shows that the relevant terms estimated by RDE correlate significantly to AAG.
Var.
Sign
Coef.
P-value
Signif.
Method
TH31W
AF5.FF
NH151
RH1 0.3M
11 Mas H MR
0,5122
<0.001
Rosetta
10.73%
76.72%
93.93%
11.34%
27.94%
0.1808
0.005
FoldX
13.56%
6.88%
5.67%
16.60%
66.19%
0,2849
<0.001
DDGPred
68.22%
2.63%
12.35%
8.30%
8,50%
W.
0.4939
<0.001
End-to-End
29.96%
2.02%
14.17%
52.43%
17.21%
0.2515
<0.001
MeF-Net.
24.49%
4.05%
6.48%
80.36%
36.23%
WW
0.2471
<0.001
0.0325
0.230
RDF-Net.
1.62%
2.02%
20.65%
61.54%
5.47%
Bias
0.1888
<0.001
444
4.3 ANALYSIS OF THE ENTROPY ESTIMATED BY ROTAMER DENSITY ESTIMATOR
Statistical Significance To demonstrate statistically significant relationship between the entropy estimated by RDE and experimental AAG values, we conduct linear regression analysis using the RDE-Linear model defined in Eq. 9, The linear model consists of seven coefficients and one bias bound WWR "WL ..L" w MMR w/
MR wWRk) ) and b. Note that WMR and "WR are merged into u.unhod as the receptor is not mutated. We perform linear re- gression on the SKEMPI2 mngle-mutation dataset and present the regression coefficients, bias, and P-values in Table 3. According to the statistics, all entropy terms, except the entropy of unbound receptor Humbhd (coefficient w whih ) show a significant relationship with experimental AAGs. The coefficients of the significant terms are all positive and roughly similar. The entropy of the un- bound receptor H has no contribution because the receptor alone does not involve in the muta- tion. These results agree with the thermodynamic definition of the change in binding free energy: AAG = AG AGW (G G.M. GY)) (GLR G GW) ) where GM and GW cancel each other out as the receptor is unmutated, therefore indicating that our model captures well the thermodynamics underlying protein- -eriiinn- interactions. For a detailed discussion about the thermodynamic background, please refer to Section B in the appendix.
term: worrd hound unbed
unhad
"A
unbed
unbed
Correlation Between Estimated Entropy and B-factors B-factor is an experimental measure- ment that quantifies the conformational flexibility. We calculate the average b-factor of sidechain atoms of residues in the test split of the PDB-REDO dataset. Then, we estimate the conformational entropy for each residue in the test split using RDE. The average Pearson correlation coefficient between these two quantities is 0.4637, and the average Spearman coefficient is 0.4282. Detailed results are presented in Table 8 in the appendix. In summary, this indicates that there a correlation between the entropy estimated by RDE and experimentally determined conformational flexibility measured by B-factor.
4.4 PREDICTION OF SIDECHAIN CONFORMATIONS
RDE is generative model for protein sidechain structures, which can predict sidechain conforma- tions by sampling from the estimated distribution. We use RDE to sample sidechain torsional angles (rotamers) for structures with 10% sidechains re- moved in our test split of PDB-REDO, For each residue, 10 rotamers are sampled independently, and the one with the highest probability is selected as the final prediction. We compare RDE with two baseline methods Rosetta (fixbb) (Leman et al., 2020) and SCWRI4 .4 (Krivov et al., 2009). Our results shown in Table 4 demonstrate that RDE outperforms the baselines on all four torsional angles in terms of angular errors. For a detailed per-amino-acid accuracy, please refer to Table 9 in the appendix.
Table 4: Mean absolute error of the predicted sidechain torsional angles.
X1
X2
X3
X1
SCWRL4
24.33
32.84
47,42
56.15
Rosetta
23.98
32.
48,49*
58.78
RDB
16.02
28.75"
45.38
53.26
CONCLUSIONS
In this work, we introduce Rotamer Density Estimator (RDE) which estimates the distribution of rotamers for protein sidechains. We demonstrate that RDE leads to improved accuracy in predicting binding AAG compared to other methods. One limitation of RDE is the inability to model back- bone flexibility directly which is important future direction for extending the proposed model. Nonetheless, our work highlights the potential of using machine learning techniques to improve mutational effect prediction for protein-protein interaction.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
Supported by National Key R&D Program China No. 2021 YF01100000000.
REFERENCES
Bruce Alberts and Ryn Myake-Iyee Unscrambling the puzzle of biological machines: the importance of the details. Cell, 68(3):415 420, 1992.
Rebecca F Alford, Andrew Leaver-Fay. Jeliazko R Jeliazkov, Matthew J O O'Meara, Frank P DiMaio, Hahnbeom Park, Maxim V Shapovalov, Douglas Renfrew, Vikram K Mulligan, Kalli Kappel, et al. The rosetta all-atom energy function for macromolecular modeling and design. Journal of chemical theory and computation, 13(6):3031- -3048, 2017.
Rodrigo Barderas, Johan Desmet, Peter Timmerman, Rob Meloen, and J Ignacio Casal. Affinity maturation of antibodies assisted by in silico modeling, Proceedings of the National Academy of Sciences, 105(26);9029-9000 2008
Michael Bower, Fred Cohen, and Roland L Dunbrack Jr. Prediction of protein side-chain ro- tamers from a backbone-dependenn rotamer library: new homology modeling tool. Journal of molecular biology, 267(5):1268 1282, 1997.
Patrick Brady and Kim Sharp. Entropy in protein folding and in protein protein-protein interactions.
Current opinion in structural biology, 7(2):215-221, 1997.
Christian Cole and Jim Warwicker. Side-chain conformational entropy at protein-protein interfaces. Protein Science, 11(12)2880-2870, 2002.
Javier Delgado, Leandro G Radusky, Damiano Cianferoni, and Luis Serrano. Foldx 5.0: working with ma, small molecules and a new graphical interface. Bioinformatics, 35(20):4188444 -16499966614,,,,, 2019.
Laurent Dinh, Jascha Dick-ticc and Samy Bengio. Density estimation using real nvp. arXiv preprint 18050036,,.08803, 2016.
Roland Dunbrack Jr. Rotamer libraries the 21st century. Current opinion structural biology, 12(4):431-44)) 2002.
Roland L Dunbrack Jr and Martin Karplus. Backbone-dependent rotamer library for proteins appli- cation side-chain prediction. Journal of molecular biology, 230(2):543-574, 1993.
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows. Ad-
vances in neural information processing systems, 32, 2019.
Jonathan Frazer, Pascal Notin, Mafalda Dias, Aidan Gomez, Joseph K Min, Kelly Brock, Yarin Gal, and Debora S Marks. Disease variant prediction with deep generative models of evolutionary data. Nature, 599(7883):91-95, 2021.
Francis Gaudreault, Matthieu Chartier, and Rafael Najmanovich. Side-chain rotamer changes upon ligand binding: common, crucial, correlate with entropy and rearrange hydrogen bonding. Bioin- formatics, 28( 28(18):443--4430 2012.
Cunliang Geng, Anna Vangone, Gert E Folkers, Li C Xue, and Alexandre MJJ Bonvin. isee: Inter- face structure, evolution, and easrd machine learning predictor of binding affinity changes upon mutations. Proteins: Structure, Function, and Bioinformatics, 87(2): 10-119,,, 2019a.
Cunliang Geng. Li C Xue, Jorge Roel-Touris, and Alexandre MJJ Bonvin. Finding the 56g spot: Are predictors of binding affinity changes upon mutations in protein-protein interactions ready for it? Wiley Interdisciplinary Reviews: Computational Molecular Science, 9()):e 1410, 2019h.
Hermann Gram, Lori-Anne Marconi, CF Barbas 3rd, Thomas A Collet, Richard A Lerner, and Angray S Kang. In vitro selection and affinity maturation of antibodies from naive combinatorial immunoglobulin library. Proceedings of the National Academy of Sciences, 99(3)3576-33500 1992.
10

---

Published as a conference paper at ICLR 2023
Thomas A Hopf, John B Ingraham, Frank Poelwijk, Charlotta PI Schärfe, Michael Springer, Chris Sander, and Debora S Marks. Mutation effects predicted from sequence co-variation. Nature biotechnology, 35(2):128- 2017.
Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie. Tom Sercu, Adam Lerer, and Alexan- der Rives. Learning inverse folding from millions of predicted structures. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 8946-8970. PMLR, 17-23 Jul 2022.
Justina Jankauskaité, Brian Linénez-Garcin Justas Dapkūnas, Juan Ferúáddz-Recio, and lain H Moal. Skempi 2.0: an updated benchmark of changes in pootein-protein binding energy, kinetics and thermodynamies upon mutation. Bioinformatics, 35(3):462 469, 2019.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael JL Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. arXiv preprint arXiv:2009.01411 2020.
L Steven Johnson, Sean R Eddy, and Elon Portugaly. Hidden markov model speed heuristic and iterative hmm search procedure. BMC bioinformatics, 11(1):1-8, 2010.
Robbie Joosten, Fei Long, Garib N Murshudov, and Anastassis Perrakis. The pdb redo server for macromolecular structure model optimization. JUCNJ, 1(4):213-220, 2014.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger,
Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-58) 2021.
Panagiotis L Kastritis and Alexandre MJJ Bonvin. On the binding affinity of macromolecular in- teractions: daring to ask why proteins interact. Journal of The Royal Society Interface, 10(79): 20120835, 2013.
Georgii G Krivov, Maxim V Shapovalov, and Roland L Dunbrack Jr. Improved prediction of protein side-chain conformations with scwrl4. Proteins: Structure, Function, and Bioinformatics, 77(4); 778-77 795, 2009.
Ruipeng Lei, Andrea Hernandez Garcia, Timothy JC Tan, Qi Wen Teo, Yiquan Wang, Xiwen Zhang, Shitong Luo, Satish K Nair, Jian Peng, and Nicholas Wu. Mutational fitness landscape of human influenza h3n2 neuraminidase. Cell Reports, 42(1): 1 1951, 2023.
Julia Koehler Leman, Brian D Weitzner, Steven Lewis, Jared Gooff-Bryfogle, Nawsad Alam, Re- becca F Alford, Melanie Aprahamian, David Baker, Kyle A Barlow, Patrick Barth, al. Macro- molecular modeling and design in rosetta: recent methods and frameworks. Nature methods, 17 (7):665-680, 2020.
Xianggen Liu, Yunan Luo, Pengyong Li. Sen Song, and Jian Peng. Deep geometric representa- tions for modeling effects mutations on protein-protein binding affinity. PLoS computational biology, 17(8) 1009284, 2021.
Lenette Lu, Todd Suscovich, Sarah Fortune, and Galit Alter. Beyond binding: antibody effector functions in infectious diseases. Nature Reviews Immunology, 18(1):46-61, 2018.
Yunan Luo, Guangde Jiang, Tianhao Yu, Yang Liu, Lam Vo, Hantian Ding, Yufeng Su, Wesley Wei Qian, Huimin Zhao, and Jian Peng. Ecnet an evolutionary context-integrated deep learning framework for protein engineering. Nature communications, 12(1):1-14, 2021.
Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu, and Alex Rives. Language models enable zero-shot prediction of the effects of mutations on protein function. Advances in Neurul Information Processing Systems, 34:29287-29303, 2021.
Rafael Najmanovich, Josef Kuttner, Vladimir Sobolev, and Marvin Edelman. Side-chain flexibility in proteins upon ligand binding. Proteins: Structure, Function, and Bioinformatics, 39(3):261- 268, 2000.
11

---

Published as a conference paper at ICLR 2023
Pascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena Hurtado, Aidan N Gomez, Debora Marks, and Yarin Gal. Tranception: protein fitness prediction with autoregressive transform- and inference-time retrieval. In International Conference Machine Learning, pp. 16990- 17017. PMLR, 2022.
Hahnbeom Park, Philip Bradley, Per Greisen Jr. Yuan Liu, Vikram Khipple Mulligan. David E Kim, David Baker, and Frank DiMaio. Simultaneous optimization of biomolecular energy functions on features from small molecules and macromolecules. Journal of chemical theory and computation, (2(12):621-62222 2016.
Roshan Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Picter Abbeel, Tom Sercu, and Alexander Rives. Msa transformer. In International Conference on Machine Learning, pp- 8844-8856. PMLR, 2021.
Danilo Jimenez Rezende, George Papamakarios, Sébastien Racaniere, Michael Albergo, Gurtej Kanwar, Phiala Shanahan, and Kyle Cranmer. Normalizing flows on tori and spheres. In In- ternational Conference on Machine Learning, pp.
PMLR, 2020.
Adam J Riesselman, John B Ingraham, and Debora S Marks. Deep generative models of genetic variation capture the effects mutations. Nature methods, 15(10):8 81--0)88,622,,,0 2018.
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. bioRxiv, 2019. doi: 10. 10.1101/62280.
Joost Schymkowitz, Jesper Borg, Francois Stricher, Robby Nys, Frederic Rousseau, and Luis Ser- rano. The foldx web server: an online force field. Nucleic acids research, E3(suppl_2::W382 W388, 2005.
Sisi Shan, Shitong Luo, Ziqing Yang, Junxian Hong. Yufeng Su, Fan Ding, Lili Fu, Chenyu Li, Peng Chen, Jianzhu Ma, et al. Deep learning guided optimization of human antibody against sars-cov-2 variants with broad neutralization. Proceedings of the National Academy of Sciences, 119(11):
e21229541 19, 2022.
Maxim V Shapovalov and Roland L Dunbrack smoothed backbone-dependent rotamer library for proteins derived from adaptive kernel density estimates and regressions. Structure, 19(6): 844-858, 2011.
Raghav Shroff, Austin W Cole, Daniel Diaz, Barrett R Morrow, Isaac Donnell, Ankur Anna- pareddy, Jimmy Gollihar, Andrew D Ellington, and Ross Thyer. Discovery of novel gain-of- function mutations guided by structure-based deep learning. ACS synthetic biology, (11):29277- 2935, 2020.
Thomas Steinbrecher, Robert Abel, Anthony Clark, and Richard Friesner. Free energy perturba- tion calculations of the thermodynamics of protein side-chain mutations. Journal of Molecular Biology, 429(():923 929, 2017.
Martin Steinegger, Markus Meier, Milot Mirdita, Harald Vöhringer, Stephan J Haunsberger, and Johannes Söding. Hh-suite3 for fast remote homology detection and deep protein annotation. BMC bioinformatics, 20(1):1-15, 2019.
Baris Suzek, Hongzhan Huang, Peter McGarvey, Raja Mazumder, and Cathy H Wu. Uniref: comprehensive and non-redundant uniprot reference clusters. Bioinformatics, 12810))1282-1 -2288, 2007.
Jingxue Wang, Huali Cao, John ZH Zhang, and Yifei Qi. Computational protein design with deep learning neural networks. Scientific reports, 8(1):199,0013 2018.
Kevin K Yang, Niccolò Zanichelli, and Hugh Yeh. Masked inverse folding with sequence transfer for protein representation learning. bioRxiv, 2022.
12

---

Published as a conference paper at ICLR 2023
Ning Zhang, Yuting Chen, Haoyu Lu, Feiyang Zhao, Roberto Vera Alvarez, Alexander Goncearenco, Anna R Panchenko, and Minghui Li. Mutabind2: predicting the impacts of sin- pedtein-protein interactions. Iscience, 23(3):100939, 2020.
gle and multiple mutations
Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining. arXiv preprint 003:0:2 2022.
13

---

Published as a conference paper at ICLR 2023
A IMPLEMENTATION DETAILS
A.1 NETWORK ARCHITECTURE
The encoder of the rotamer density estimator is a stack of attention layers invariant the rotation and translation (Jumper et al., 2021) of the input protein structure. Let h denote the embedding of the i-th amino acid output by the previous attention layer. The logit of attention weights between residue i and residue j is defined as:
aig = (Q(h)). k(h)) + G(z.) + H ((O) (a) x{))x).
(10)
where Q, K, G. and H are MLP networks that transform the embeddings into queries, keys, pairwise bias, and distance bias, respectively. The attention weight is computed by taking the softmax on the dimension: Wij softmax, (a.5). In practice, use multiple attention heads. Each attention head has different attention weights. The vector to update the representation of residue i from residue is computed by:
v = V (hf, Zuj 10, (a)))))))) ++))))
(11)
Finally, the sum of J weighted by (wij} is used to update the representation of residue j using residual connection and layer normalization, similar to the standard transformer architecture.
The rotamer density estimator has encoder layers. Node features and pairwise features have 128 channels and 64 channels respectively. The normalizing flow has blocks and each spline has 65 knots, dividing [0, 2x] into 64 bins.
A.2 TRAINING
The rotamer density estimator is trained using the Adam optimizer for 200K iterations. The initial learning rate is 0.0001. The learning rate decays by 0.8 if the validation loss does not decrease the last 5 validation steps (the model is validated every 1000 iterations), until the learning rate reaches 0.000001. The batch size is 64. It takes 8h56m in total on a single A100 GPU.
To emulate mutations, the rotamers of 10% of amino acids are masked. Noise is added to the rotamers of amino acids whose C-beta distance to the closest masked amino acid is less than 8.0A. The noise added to X angles consists of two components. The first component is Gaussian noise centered at 0 wrapped into [-x, nT. standard deviation dependent on the C-beta distance: du =
B, + 1, where i is the index the nearest masked amino of the j-th amino acid that the noise added to, and Bij is the C-beta distance between them. The second component is random flipping (adding the angle). Every X angle in the 8Á neighborhood has 25% chance of being flipped. Our noise model is totally empirical. There are other ways to perturb rotamers, for example, using rotamer libraries (Dunbrack Jr & Karplus, 1993; Bower et al., 1997; Dunbrack Jr, 2002; Shapovalov & Dunbrack Jr, 2011). We leave the problem of finding an optimal noise model that emulates mutations in future work.
A.3 BASELINES
Baselines that require training and calibration using the SKEMPI2 dataset (DDGPred, End-to-End, B-factor, MIF-Alogit, MIF-Network) are trained independently using the 3 different splits of the dataset as described in Section 3.5. This is to ensure that every data point in the SKEMPI2 dataset tested on once. Below are descriptions of the implementation baseline methods.
Rosetta (Alford et al., 2017; Leman et al., 2020) The version we used is 2021.16, and the scoring function ref2015_cart first pre-processed using the relax application. The mutant structure is built by cartesian.ddg, The binding free energies both wild-type and mutant structures are predicted by interface. .ereeeeeerrggeee rgy (dG_sepane Sew.rated/SSAASAAAAAS.... 100).. Finally, the binding AAG is calculated by substracting the binding energy of the wild-type structure from the binding energy of the mutant.
Every protein structures in the SKEMP12 dataset
FoldX (Delgado et al., 2019) Structures first relaxed by the RepairPDS command. Mutant structures are built with the Bui 1 command based on the repaired structure. The change in binding free energy AAG is calculated by subtracting the wild-type energy from the mutant energy.
14

---

Published as a conference paper at ICLR 2023
ESM-1v (Meier et al., 2021) We use the implementation provided in the ESM open-source reposi- tory. Protein language models can only predict the effect of mutations for single protein sequences. Therefore, ignore cases where mutations the mutated protein chain from the SEQRES entry of the PDB file. We use masked- rg na s mode to score both wild-type and mutant sequences and use their difference the estimation of AAG.
multiple sequences. We extract the sequence of
PSSM We construct MSAs from the Uniref90 database (Suzek al., 2007) for chains with mutation annotations in the SKEMPI dataset. We use Jackhmmer (Johnson et al., 2010) version 3.3.1 follow- ing the setting in Meier al. (2021). The MSAs are filtered using HHfilter (Steinegger et al., 2019) with coverage 75 and sequence identity 90. This HHfilte parameter is reported to have the best performance for MSA Transformer according to Meier et al. (2021). We calculate position-specific scoring matrices (PSSM) and use the change in probability as the prediction of AAG.
MSA Transformer (Rao et al., 2021) We use the implementation provided in the ESM open-source repository. We input the MSAs constructed during the evaluation of PSSM to MSA Transformer. We used masked beeeeddmaggii marging na 1s mode to score both wild-type and mutant sequences and use their difference the prediction of AAG.
(Notin et al., 2022) We use the implementation provided in the Tranception open- source repository. We predict mutation effects using the large model checkpoint. Previously built MSAs (not filtered by HHfilter) are used for inference-time retrieval.
Tranception
DDGPred (Shan et al., 2022) We use the implementation accompanying the paper by Shan et al. (2022). Since this model requires predicted sidechain structures of the mutant, we use mutant struc- tures packed during our evaluation of Rosetta to train the model and run prediction.
End-to-End The end-to-end model shares the same encoder architecture as the rotamer density estimator. The difference is that in the RDE normalizing flows follow the encoder model rotamer distributions, but in the end-to-end model, the embeddings are directly fed to an MLP to predict AAG.
B-factor This model predicts per-atom b-factors for proteins. It has the same encoder architecture the RDE. Following the encoder is MLP that predicts vector for each amino acid where each dimension is the predicted b-factor of different atoms in the amino acid. The amino acid-level b-factor calculated by averaging the atom-level b-factors. The predicted b-factors are used a measurement of conformational flexibility. They are used to predict AAG using the linear model same as RDE-Linear defined Eq.9.
ESM-IF (Hsu et al., 2022) ESM-IF can score protein sequences using the log-likelihood. The scoring function implementation is provided in the ESM repository. We enable the --multi cha in backbone flag to let the model see the whole protein-protein complex. We subtract the log-likelihood of the wild-type from the mutant to predict AAG.
MIF Architecture The masked inverse folding (MIF) network uses the same encoder architecture as RDE. Following the encoder a per-amino-acio 20-category classifier that predicts the type of masked amino acids. We use the same PDB-REDO train-test split to train the model. At training time, we randomly crop a patch consisting of 128 residues, and randomly mask 10% amino acids. The model learns to recover the type of masked amino acids with the standard cross entropy loss.
MIF-Alogit
To score mutations, we first mask the type of mutated amino acids. Then, we use the log probability of the amino acid type as the score. Analogously, we have the score of the wild- type bound ligand, wild-type bound receptor, wild-type unbound ligand, unbound receptor, mutated bound ligand, mutated bound receptor, and mutated unbound ligand. Therefore, we use the linear model identical to RDE-Linear (Eq.9) to predict AAG from the scores.
MIF-Network This is similar to RDE-Network. The difference is that we use the pre-trained en- coder of MIF rather than the encoder of RDE. We also freeze the MIF encoder as we aim to utilize the unsupervised representations.
A.4 SOURCE CODE
Available at https://gtthubb.cm///o////
15

---

Published as a conference paper at ICLR 2023
B BACKGROUND
This section introduces the thermodynamic principle underlying the design RDE, which connects entropy and binding affinity.
The Gibbs free energy of association is the physical quantity used to measure the binding affinity between groups of protein:
AG = AH - TAS.
(12)
In this equation, A is the change in enthalpy upon the formation of the complex, which is generally assumed to be negligible as covalent bonds formed or broken upon protein-protein binding. is the temperature parameter, and A, is the change in entropy upon binding (Kastritis & Bonvin, 2013). By ignoring HT and expanding A.S, we can rewrite AG, AG, ( Sa), where S. is the entropy of the proteins in the bound state (complex), and Sj is the entropy in the unbound state (separated). To predict binding affinity, we need to calculate the entropy of the separated proteins (Sa) and the entropy of the protein complex (Sa).
The entropy S is defined by the Boltzmann expression:
S= - / p(x) log p(æ)dx -kBlEww log p(x),
(13)
where p(a:) is the distribution of conformation æ and kg is the Boltzmann constant (Brady & Sharp, 1997). To calculate the entropy of the protein complex Sa, we need to evaluate the integral with respect to the distribution of the complex conformation denoted by Pa. To calculate the entropy of the separated proteins Sd, we can factorize the probability density Pa = Pligana (Fiigund) "Preceptor (Freveptor) and then evaluate- n (Eze--pgnn log Pligand (z) Eapgpesppe log Preceptor (z)). Substituting Sa and S. in AG, with these expressions, we can obtain the formula:
AG = kB' (Eaa-rrana log Pligand( + Egeeerenee log Proceptor (x) - Ex-p, la)), (14)
which indicates that we can predict binding affinity by estimating the entropy of the conformation distributions of the protein complex Pa and each separated protein Pligand: Precoptor-
We assume that sidechain conformation changes are the major determinant of protein-prose protein bind- keep the protein backbone fixed and model only the distribution of sidechain confor- mations (rotamers) (Najmanovich et al., 2000; Cole & Warwicker, 2002). This assumption leads to the core component of this work, Rotamer Density Estimator (RDE), which approximates Pas Pligand. and Preceptor enabling us to estimate AG, by evaluating the entropy of these distributions.
ing,
Finally, to evaluate the effect of mutations, we apply RDE estimate G2 for both the wild-type and the mutant. We then calculate the difference between the Ga values of the mutant and wild-type, yielding the quantity AAG:
AAG AG mutant
Aaiidd-yyee
(15)
We refer the reader to Brady & Sharp (1997) and Kastritis & Bonvin (2013) for a comprehensive treatment of the relationship between binding affinity and entropy.
16

---

Published as a conference paper at ICLR 2023
C ADDITIONAL RESULTS
C.I FURTHER ANALYSIS ON THE PERFORMANCE OF SEQUENCE- BASEE METHODS
As discussed in Section 2.2, sequence-base baseec cevolution-based methods unsuitable for predicting protein- proniin interactions due to the lack of co-evolutionary information between the two proteins. This is supported by the results in Table 1, which indicate that sequence-bar based methods are inaceu- rate in predicting AAG. We analyzed two classes of PPIs to better understand the performance of sequence-based methods on PPIs.
Antibody-antigen binding is typical class of PPI that lacks co-evolutionary information. The vari- ability of the binding interface antibodies (complementarity determining region, CDR) means that there is evolutionary history in the region, making it infeasible to mine the preference of mutations from sequence databases. Additionally, in most cases, antigens do not evolve to increase binding specific antibodies, sequence databases provide little information about mutational ef- fects on antibody-antige binding. We evaluated per-structure Spearman correlation coefficients of MSA Transformer and RDE-Network The average Spearman score of MSA Transformer is 0.0744 and the average score of RDE-Network 0.4284. Figure 4 shows the results. where the z-axis and y-axis are the per-structure Spearman co- efficients of MSA Transformer and RDE-Network. respectively. Orange crosses represent antibody- antigen complexes, and blue dots represent other complexes. The results indicate that when we have little co-evolutionary information such as in antibody-antigen binding, structure-based meth- ods, represented by our RDE, perform better than evolution-based methods, represented by MSA Transformer.
antibody-antigen complexes from the SKEMPI datasets.
When the proteins a complex come from the same organism, evolution-bane methods are more likely to be effective. These proteins usually function together in the organism, they evolve together. Mutations that enhance the complexation may be more favorable, and this preference might be reflected in evolutionary history. We inspected 10 complexes performed the best in terms of Spearman coefficients and found that of them consist of proteins from single organism (Table 5). However, when evaluating MSA Transformer organism complexes, its Spearman score is 0.1651, which is still low. The reason may be that even if the proteins in complex come from the same organism, the member protein might also bind to other proteins to be functional. In this case, it is more challenging to predict its binding in a specific complex according to its general evolutionary history.
which MSA Transformer
all the single-
Figure 4: Per-structure Spearman correlation coef- ficients of the prediction by MSA Transformer and RDE-Network. Orange crosses represent antibody- antigen complexes and blue dots represent other types of complexes. Axes are cropped to [0, 1].
Table 5: Protein complexes on which MSA Transformer performs the best in terms of Spearman coefficients.
MSA Transf. Spearman
Number of Organisms
Complex
0.8
4OFY A
0.7454
RDEeNetwork (Speamaa)
2J0T
0.6990
I
1AK4_A.D
0.6578
2
0.6
2SIC.E.I
0.5915
1
IBRS A D
0.5755
0.4
3SE3 B A
0.5213
TEMV_A_E
0,5082
IB4I
0.4972
1Z7X W X
0.4647
3SE4 B C
0.4268
0.0
MSA Transformer (Spearman)
17

---

Published as a conference paper at ICLR 2023
C.2 SINGLE-MUTATION AND MULTI-MUTATION
Table 6: Evaluation of AAG predictors on the
subset of SKEMPI2.
Per-Struct. Pearson
Per-Struct. Spearman
Overall Pearson
Overall Spearman
RMSE
MAE (kcal/mol)
AUROC
Method
(kcal/mol)
ESM-Iv
0.0422
0.0273
0.1914
0.1572
1.7226
1.1917
0.5492
PSSM
0.1215
0.1229
0.1224
0.0997
1.7420
1.2055
0.5659
MSA Transf.
0.1415
0.1293
0.1755
0.1749
1.7294
1.1942
0.5917
Tranception
0.1912
0.1816
0.1871
0.1987
1.7455
1.1708
0.6089
Rosetta
0.3284
0.2988
0,3113
0.3468
1.6173
1,1311
0,6562
FoldX
0.3908
0.3640
0.3560
0.3511
1.5576
1.0713
0.6478
DDGPred
0.371
0.3427
0.6515
0.4390
1.3285
0.9618
0.6858
End-to-End
0,3818
0.3426
0.6605
0.4594
1.3148
0.9569
0.7019
B B-factor
0.1884
0.1661
0.1748
0.2054
1,7242
1.1889
0.6100
ESM-IF
0.2308
0,2090
0,2957
0.2866
1.6728
1.1372
0.6051
MIF-Alogit
0.1616
0.1231
0.2548
0.1927
1.6928
1.1671
0.5630
MIF-Net.
0.3952
0.3479
0.6667
0.4802
1,3052
0.9411
0.7175
RDE-Lineat 0.3192
0.2837
0.3796
0.3394
1,5997
1,0805
0.6027
RDE-Net.
0.4687
0.4333
0.6421
0.5271
1.3333
0.9392
0.7367
Table 7: Evaluation of AAG predictors on the multi-mutation subset of SKEMP12.
Per-Struct.
Per-Struct. Spearman
Overall Pearson
Overall Spearman
RMSE
MAE (kcal/mol)
AUROC
Method
Pearson
(kcal/mol)
ESM-Iv
-0.0599
-0.1284
0.19
0.1749
2.7586
2.1
0.5415
PSSM
-0.0174
-0.0504
-0.1126
-0.0458
2.7937
2.1499
0.4442
MSA Transf.
-0.0097
-0,0400
0,0067
0,0030
2.8115
2.1591
0.4870
Tranception
-0.0688
-0.0120
-0.0185
-0.0184
2,9280
2.2359
0.4874
Rosetta
0.1915
0.0836
0.1991
0.2303
2.6581
2.0246
0.6207
FoldX
0.2801
0.2771
0.2347
0.4137
2.5290
1.8639
0.6828
DDGPred
0.3912
0.3896
0.5938
0.5150
2.1813
1.6699
0.7590
End-to-End
0.4178
0.4034
0.5858
0.4942
2.1971
1.7087
0.7532
B-factor
0.2078
0.1850
0.2009
0.2445
2.6557
2.0186
0.5876
ESM-IF
0.2016
0.1491
0.3260
0.3353
2,6446
1.9555
0.6373
MIF-Alogit
0.1053
0.0783
0.3358
0.2886
2.5361
1.8967
0.6066
MIF-Net.
0.3968
0.3789
0.6139
0.5370
2.1399
1.6422
0.7735
RDE-Lineer
0.1763
0.2056
0.4583
0.4247
2.4460
1.8128
0.6573
RDE-Net.
0.4233
0.3926
0.6288
0.5900
2.0980
1.5747
0.7749
REC-SENN 3000
60
600
-60
-60
16
Diviimeta KAG
Egeeieemta000
Dysiinetd AAS
Correlation between experimen- tal AAGs and AAGs predicted by RDE- Linear and RDE-Network SKEMPI2 single- mutation subset.
Figure 5:
Figure 6: Correlation between experimen- tal AAGs and AAGs predicted by RDE- Linear and RDE-Network SKEMP12 multi-
mutation subset.
18

---

Published as a conference paper at ICLR 2023
C.3 CORRELATION WITH B B-FACTOR
Table 8: Correlation between average sidechain B-factors and entropy estimated by RDE. Scatter plots of each amino acid type are in Figure
Type
Pearson
Spearman
ARG
0.4683
0.4904
ASN
0.4329
0.3826
ASP
0.4680
0.4408
CYS
0.4099
0,4099
GLN
0.4691
0.4463
GLU
0.4683
0.4861
HIS
0.3743
0.3261
ILE
0.5288
0.4677
LEU
0.5281
0.4638
LYS
0.5479
0.5777
MET
0.5026
0,4758
PHE
0.3615
0.2667
SER
0.5368
0.5247
THR
0.5945
0.5263
TRP
0.3263
0.2910
TYR
0.3588
0,2493
VAL
0.5068
0.4536
Average
0.4637
0.4282
aatype LEU
Batype GLN
ARC
aatype VAL
aetype TYA
entropy
8stype PHE
aatype - SER
aetype = ASP
aetyge = GLU
eatype - THR
entrony
ASN
ILE
aelyge LYS
TRP
MET
entropy
-
0
100
200
300
0
100
200
300
0
100
200
300
bfactor
eslype = CYS
aslype = IIIS
5
entropy
0
bfactor
afactor
Figure 7: Scatter plots of average sidechain B-factors and estimated entropy.
19

---

Published as a conference paper at ICLR 2023
C.4
SIDECHAIN CONFORMATION PREDICTION
Table 9: Mean absolute error of the predicted sidechain torsional angles.
Type
X
SCWRL4
Rosetta
RDE
Type
X
SCWRIA
Rosetta
RDE
29.30
30.50
20.90
LEU
1
13.97
14.15
9.41
28.68
36.12
28.72
2
23.76
27.61
19.32
ARG
57.89
60.73
56.83
31.32
33.88
21.64
60.35
63.62
56.76
2
30.95
33.15
32.90
LYS
21.70
19.39
17.26
3
38.90
42.07
37.70
ASN
44.00
43.41
41.36
51.94 4
53.94
49.76
25.75
22.62
17.54
1
26.36
26.07
16.87
ASP
23.90
21.26
21.15
MET
2
38.52
36.09
27.09
3
55.11
58.77
50.21
CYS
24.83
25.90
12.74
1
12.30
13.08
9.22
33.16
31.53
22.78
PHE
2
12.40
12.35
9.45
GLN
46.33
33.96
35.16
53.72
56.52
52.80
SER
1
47.19
46.83
25.66
35,45
34.69
26.14
THR
28.05
22.67
16.85
GLU
38.23
38.43
34.60
1
14.52
18.64
8.50
31.50
30.85
29.35
TRP
2
31.74
31.44
24.42
HIS
70.62
23.15
61.97
19.12
69.69
17.88
TYR
2
1
11.37
11.39
14.56
14.45
9.25
8.35
ILE
13.92
14.65
8.86
VAL
1
21.31
19.41
10.82
26.43
27.54
21.54
20


Reviews:
[{"review_id": "sX2B9SS0Ub", "reviewer": "Reviewer_GvK9", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on quantifying the effects of mutations on protein-protein interactions. It does so by first building a (conditional) normalizing flow model to estimate the probability of sidechain conformations (rotamers) conditioning on the type, position, orientation, and prior rotamer of itself and other amino acids. Building on the intuition that higher binding affinity between two proteins implies higher rigidity (lower entropy), it then estimates the change in binding free energy due to the mutation as the change in delta entropy (bound - unbound) between mutated sequence and wild type. Experiments on the SKEMPI2 dataset illustrate the strengths of the method over current baselines.", "summary_of_the_review": "A compelling approach to quantify the effects of mutations in PPI, which does not rely on labels for rotamer density estimates and does not need structure input to make predictions. The approach achieves strong empirical performance relative to baselines. I would be willing to recommend acceptance more enthusiastically if the concerns discussed above are addressed during rebuttal. \n\n------------------------------------------------------------------------------------------------------------\n[UPDATE POST REBUTTAL]\nAuthors' responses adequately addressed my feedback during rebuttal. Updating my score accordingly.", "strength_and_weaknesses": "**Strengths**\n- Very clear write up, with a well-thought-through experimental design\n- From a methodology standpoint, the unsupervised modeling approach via RDE avoids potential issues due to label scarcity / bias. Furthermore, the choice of normalizing flow allows efficient sampling of rotamers when estimating the different entropy terms since it enables exact likelihood estimation\n- The method achieves very strong performance relative to other baselines, without relying on structure data as an input\n\n**Weaknesses**\n- While it seems intuitively sensible, the claim that sequence-based models fail to predict ∆∆G for protein-protein binding does not seem particularly well backed-up experimentally. To achieve good performance with ESM-1v, one must both: a) use an ensemble of 5 ESM1v transformers b) fine-tune each model on a set of homologous sequences. When both conditions are not met (and it seems it is not the case here), a single non fine-tuned ESM-1v performs worse than a site independent model (see Tables 1 and 2 of [1], or table 2 of [2]). More performing baselines include Tranception [2], EVE [3] or MSA Transformer [4].\n- Table 1: it seems that DDGPred does best in terms of both RMSE and MAE (bolding issue?) \n\n----------------------------------------------------------------------------------------\n[1] Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021). Language models enable zero-shot prediction of the effects of mutations on protein function. NeurIPS.\n\n[2] Notin, P., Dias, M., Frazer, J., Marchena-Hurtado, J., Gomez, A.N., Marks, D.S., & Gal, Y. (2022). Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time Retrieval. ICML.\n\n[3] Frazer, J., Notin, P., Dias, M., Gomez, A.N., Min, J.K., Brock, K.P., Gal, Y., & Marks, D.S. (2021). Disease variant prediction with deep generative models of evolutionary data. Nature.\n\n[4] Rao, R., Liu, J., Verkuil, R., Meier, J., Canny, J.F., Abbeel, P., Sercu, T., & Rives, A. (2021). MSA Transformer. ICML.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "8lqJPZD69x-", "reviewer": "Reviewer_R6Cf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper develops a flow-based generative approach to model the probability distribution of side-chain conformations (called RDE) in protein-protein interactions and then estimates changes in entropy by measuring the changes in side-chain flexibility. The paper uses a conditional generative model built upon normalizing flows for estimating densities. It uses a neural network to extract delta delta G from the RDE representation.", "summary_of_the_review": "Overall the paper tackles an important problem and some of the empirical results are very string however in terms of methodology it is not a significant contribution to machine learning.", "strength_and_weaknesses": "Strength.\n1- The paper tackles an important problem. Predicting protein-protein interactions in a low-data regime is a very important problem.\n\n2- No direct homology and evolutionary information is required in the model and thus the method is applicable to find the effect of mutations on protein-protein interactions with a lack of homology information. \n\n3- Empirical results in Table 1 are convincing (although I have some questions about the baselines).\n\n4- The flow-based generative model works on exact likelihood.\n\n5- Using side-chain conformational change as a proxy for delta delta G seems to be novel.\n\nWeakness.\n1- The approach only takes into account flexibility in the side chains and not the backbone.\n\n2- If mutations cause change beyond side-chain conformational variations it is not clear how the method can handle those mutations.\n\n3- The significance of the contribution to the ML community is limited. The work tackles a specific problem in protein-protein interactions and mostly uses a combination of standard DL modules (flow based generative models, MLP, attention with rotation/translation invariance) as building blocks. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "8For8AHl2-", "reviewer": "Reviewer_NQyq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed a new method (Rotamer Density Estimator): a flow-based generative model to estimate the probability distribution of conformation. The proposed Rotamer Density Estimator was evaluated on mutational effects for ddG (by using entropy of the probability distribution as the measure of flexibility). ", "summary_of_the_review": "Original approach for density estimation of protein conformations with limited improvement in empirical results.", "strength_and_weaknesses": "Strengths:\n+ Interesting approach for modeling the probability distribution of conformations with flow models. \n+ The problem of modeling the distribution of conformation itself is of wide interest. (I would encourage the author to also explore more applications.)\n+ Empirical results are based on a common benchmark dataset (SKEMPI).\n+ Relatively clear presentation of the method and the results.\n\nWeaknesses:\n+ On the mutation effect prediction task evaluated in the paper, several machine learning baselines are missing (e.g. GVP, ESM-IF1).\n+ Misleading bolding in the tables (when the proposed method is not the best, the best method is not bolded)\n+ No confidence intervals for Table 1 and Table 2\n+ Does the training data (crystallography) faithfully capture the distribution of conformations? Empirical proof or theoretical reasoning would be helpful.\n+ Is there any rigorous biophysical justification for the entropy correlating with ddG? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster