R OTAMER D ENSITY E STIMATORS ARE U NSUPERVISED L EARNERS OF THE E FFECT OF M UTATIONS ON P ROTEIN -P ROTEIN I NTERACTION
A BSTRACT
Protein -protein interactions are crucial to many biological processes, and predicting the effect of amino acid mutations on binding is important for protein engineering.
While data-driven approaches using deep learning have shown promise the scarcity of annotated experimental data remains a major challenge.
In this work, we propose a new approach that predicts mutational effects on binding using the change in conformational flexibility of the protein-protein interface.
Our approach, named Rotamer Density Estimator (RDE), employs a flow-based generative model to estimate the probability distribution of protein side-chain conformations and uses entropy to measure flexibility.
RDE is trained solely on protein structures and does not require the supervision of experimental values of changes in binding affinities.
Furthermore, the unsupervised representations extracted by RDE can be used for downstream neural network predictions with even greater accuracy.
Our method outperforms empirical energy functions and other machine learning-based approaches
I NTRODUCTION
Proteins rarely act alone and usually interact with other proteins to perform a diverse range of biological functions (Alberts & Miake-Lye, 1992; Kastritis & Bonvin, 2013).
For example, antibodies, a type of immune system protein, recognize and bind to proteins on pathogens’ surfaces, eliciting immune responses by interacting with the receptor protein of immune cells
(Lu et al., 2018).
Given the importance of protein-protein interactions in many biological processes, developing methods to modulate these interactions is critical.
A common strategy to modulate protein-protein interactions is to mutate amino acids on the interface: some mutations enhance the strength of binding, while others weaken or even disrupt the interaction (Gram et al., 1992; Barderas et al., 2008).
Biologists may choose to increase or decrease binding strength depending on their specific goals.
For example, enhancing the effect of a neutralizing antibody against a virus usually requires increasing the binding strength between the antibody and the viral protein.
However, the combinatorial space of amino acid mutations is large, so it is not always feasible or affordable to conduct wet-lab assays to test all viable mutations.
Therefore, computational approaches are needed to guide the identification of desirable mutations by predicting their mutational effects on binding strength, typically measured by the change in binding free energy ( ∆∆ G ).
Traditional computational approaches are mainly based on biophysics and statistics (Schymkowitz et al., 2005;
Park et al., 2016; Alford et al., 2017).
Although these methods have dominated the field for years, they have several limitations.
Biophysics-based methods face a trade-off between efficiency and accuracy since they rely on sampling from energy functions.
Statistical methods are more efficient, but their capacity is limited by the descriptors considered in the model.
Furthermore, both biophysics and statistics-based methods heavily rely on human knowledge, preventing it to ∗ Equal contribution.
benefit from the growing availability of protein structures.
As a result, predicting the effects of mutations on protein-protein binding remains an open problem.
Recently, deep learning has shown significant promise in modeling proteins, making data-driven approaches more attractive than ever (Rives et al., 2019; Jumper et al., 2021).
However, developing deep learning-based models to predict mutational effects on protein-protein binding is challenging due to the scarcity of experimental data.
Only a few thousand protein mutations, annotated with changes in binding affinity, are publicly available (Geng et al., 2019b), making supervised learning challenging due to the potential for overfitting with insufficient training data.
Another difficulty is the absence of the structure of mutated protein-protein complexes.
Mutating amino acids on a protein complex leads to changes mainly in sidechain conformations (Najmanovich et al., 2000;
Gaudreault et al., 2012), which contribute to the change in binding free energy. However, the exact conformational changes upon mutation are unknown.
[Figure]
In this work, we draw inspiration from the thermodynamic principle that protein-protein binding usually leads to entropy loss on the binding interface, which can be used to determine binding affinity (Brady & Sharp, 1997;
Cole & Warwicker, 2002; Kastritis & Bonvin, 2013).
When two proteins bind, the residues located at the interface tend to become less flexible (i.e. having lower entropy) due to the physical and geometric constraints imposed by the binding partner (Figure 1).
A higher amount of entropy loss corresponds to a stronger binding affinity.
Therefore, by comparing the entropy losses of wild-type and mutated protein complexes, we can estimate the effect of mutations on binding affinity.
Based on this principle, we introduce a novel approach to predict the impact of amino acid mutations on proteinprotein interaction. The core of our method is the Rotamer Density Estimator (RDE), a conditional generative model that estimates the density of amino acid sidechain conformations (rotamers).
We use the entropy of the estimated density as a metric of conformational flexibility. By subtracting the entropy of the separated proteins from the entropy of the complex, we obtain an estimation of binding affinity.
Finally, we can assess the effect of mutations by comparing the estimated binding affinities of wild-type and mutant protein complexes.
In addition to directly comparing entropy, we also employ neural networks to predict ∆∆ G from the representations learned by RDE.
Our method is an attempt to address the aforementioned challenges.
The Rotamer Density Estimator is trained solely on protein structures, not requiring other labels, making it an unsupervised learner of the mutation effect on protein-protein interaction. This feature mitigates the challenge posed by the scarcity of annotated mutation data.
Moreover, our method does not require the mutated protein structure as input.
Instead, it treats mutated structures as latent variables, which are approximated by RDE.
Our method outperforms both empirical energy functions and machine learning models for predicting ∆∆ G .
Additionally, as a generative model for rotamers, the RDE accurately predicts sidechain conformations.
R ELATED W ORK 2. M UTATIONAL E FFECT P REDICTION FOR P ROTEIN -P ROTEIN I NTERACTION
Traditional approaches to predicting the effect of mutation on protein binding can be roughly divided into two classes: biophysical and statistical methods.
Biophysical methods utilize energy functions to model inter-atomic interactions.
These methods sample conformations of the mutated protein complex and predict changes in binding free energy (Schymkowitz et al., 2005;
Park et al., 2016; Alford et al., 2017; Steinbrecher et al., 2017).
Statistical methods rely on feature engineering, which uses descriptors summarizing geometric, physical, evolutionary, and motif properties of proteins to predict mutational effects (Geng et al., 2019a; Zhang et al., 2020).
Traditional methods face the trade-off between speed and accuracy.
Their performance depends heavily on human expertise, which limits their pace to improve with the fast-growing of available protein structures.
Recently, deep learning-based approaches have emerged.
We group them into three categories: endto-end models, pre-training-based models, and unsupervised models.
End-to-end models directly predict the difference in binding free energy by taking both mutant and wild-type protein structures as input (Shan et al., 2022).
Pre-training-based models attempt to address data scarcity by pre-training a feature extraction network (Liu et al., 2021; Yang et al., 2022; Zhang et al., 2022).
However, most pre-training tasks are not designed to capture the foundation of protein-protein interactions.
Unsupervised models adopt the mask-predict paradigm to protein 3D structures, partially masking amino acid types on a given protein backbone, and recovering the masked information using neural networks (Wang et al., 2018; Shroff et al., 2020;
Jing et al., 2020; Yang et al., 2022; Hsu et al., 2022).
These models can serve as unsupervised predictors of the mutational effects on binding, as the difference in the probability of amino acid types before and after mutation correlates mildly to the change in binding free energy.
2. M UTATIONAL E FFECT P REDICTION FOR S INGLE P ROTEINS
The prediction of mutational effects for single proteins can be achieved using either structure-based or sequence-based (evolution-based) approaches.
Structure-based methods can be categorized into biophysical, statistical, and deep learning-based methods, which aim to predict the thermal stability or fitness of the protein rather than the binding free energy between proteins (Schymkowitz et al., 2005;
Park et al., 2016; Alford et al., 2017; Lei et al., 2023).
Sequence-based methods rely on the mining of evolutionary history, done by performing statistics on multiple sequence alignments (MSAs) constructed from large-scale sequence databases (Hopf et al., 2017; Riesselman et al., 2018;
Rao et al., 2021; Luo et al., 2021; Frazer et al., 2021), or leveraging protein language models (PLMs) (Meier et al., 2021; Notin et al., 2022).
However, it is important to note that sequence-based methods are not suitable for predicting mutational effects on general protein-protein interactions due to the lack of evolutionary information in many cases.
Protein-protein interactions typically involve two or more chains, which may belong to different species or may not experience inter-chain co-evolution.
As such, it is infeasible to predict mutational effects via mining sequence databases using existing powerful tools such as MSAs or PLMs.
Thus, effective ways for predicting mutational effects on protein-protein interaction rely on structure-based approaches rather than sequences alone.
M ETHOD 3. O VERVIEW AND P RELIMINARIES
Overview Our method comprises three main components.
The first is the Rotamer Density Estimator (RDE), which is a conditional normalizing flow that models the probability density of sidechain conformations (rotamers) based on the amino acid type and backbone structures (Section 3.2).
The second component is an algorithm that estimates the entropy of the distribution parameterized by the normalizing flow (Section 3.3).
Lastly, we describe how we use the entropy of the protein-protein interfaces in both the mutated and wild-type states, both bound and unbound, to estimate the change in binding free energy ( ∆∆ G ). We also detail how we use neural networks to achieve more accurate predictions of ∆∆ G using the unsupervised representations from the RDE (Section 3.4).
Definitions and Notations A protein-protein complex is a multi-chain protein structure that can be divided into two groups.
Each group contains at least one protein chain and each chain consists of multiple (amino acid) residues.
For a protein complex containing n residues, we number them from 1 to n .
The two groups of the complex can be represented by two disjoint sets of indices A, B ⊂ { 1 . . .
n } .
A residue is characterized by its type, position, orientation, and sidechain conformation.
We denote the type, position, and orientation of the i -th ( i ∈ { 1 . . .
n } ) residue as a i ∈ { 1 . . .
20 } , p i ∈ R 3 , and O i ∈ SO (3) respectively.
The sidechain conformation of the residue is called rotamer .
As the conformational degree of freedom of the sidechain is defined by rotatable bonds, a rotamer can be parameterized by torsional angles w.r.t.
the rotatable bonds.
The number of torsional angles varies between 0 to 4 depending on the residue type.
For a residue with d torsional angles, we denote the k -th ( k ∈ { 1 . . .
4 } ) torsional angle by χ ( k ) i ∈ [0 , 2 π ) .
Collectively, all the torsional angles are denoted by a vector χ i =
( χ ( k ) i )
dk
=1 .
Using the language of geometry, an angle can be represented by a point on the unit circle S 1 .
A vector consisting of d angular values resides on the product of d unit circle, known as the d -dimensional torus T D =
( S 1 ) D .
In this work, our first goal is to model the conditional probability density of rotamers, given the type, position, orientation, and prior rotamer of itself and other residues: p ( χ i | { a j , p j , O j , ˜ χ j } nj =1 ) .
The prior rotamers ˜ χ
j are often inaccurate or unknown.
For example, if we mutate some residues, the rotamers of the mutated residues are unknown, and the rotamers of residues nearby the mutated ones are inaccurate because they are affected by the mutation.
The probability density is defined over the d -dimensional torus T D =
( S 1 ) D , and we describe below the flow-based architecture to model the density.
3. R OTAMER D ENSITY E STIMATOR
The Rotamer Density Estimator (RDE) is designed to estimate the conditional distribution of the rotamer of the i -th residue, given the information of itself and other residues: p ( χ i | { a j , p j , O j , ˜ χ j } nj =1 ) .
In this section, we first introduce the encoder network that produces hidden representations for each residue, taking into account its environment { a i , p i , O i , ˜ χ i } n i =1 .
Next, we present a conditional normalizing flow defined on S 1 for rotamers with only 1 torsional angle.
Based on the S 1 flow, we further extend the flows to d -dimensional torus T D ( D > 1 ) for rotamers with more than 1 torsional angle.
Encoder Network
The encoder network starts with two multi-layer perceptrons (MLPs) that generate embeddings for each individual single residue and each pair of residues respectively.
The MLP for single residues encodes the residue type, backbone dihedral angles, and local atom coordinates into a vector e i ( i = 1 . . .
n ).
The other MLP for residue pairs encodes the distance and the relative position between two residues. We denote a pair embedding vector as z ij ( i, j = 1 . . .
n ).
To transform the single embeddings e i and pair embeddings z ij into hidden representations h i , we use a self-attention-based network that is invariant to rotation and translation (Jumper et al., 2021).
The hidden representation h i aims to capture both the information of the i -th residue itself and its structural environment.
It serves as an encoding of the condition { a j , p j , O j , ˜ χ j } nj =1 for the probability density with respect to χ i .
Conditional Flow on S
To model the distribution of a rotamer with 1 torsional angle, we utilize conditional normalizing flows on S 1 (Rezende et al., 2020).
A normalizing flow is a bijective function, and to construct one on S 1 , we parameterize it by an angle θ ∈ [0 , 2 π ] , and define a bijective function on [0 , 2 π ] that is equivalent to a bijective over S 1 .
A common method for constructing a bijective function is to ensure strict monotonicity, which we adopt by constructing a strictly monotonically increasing function on [0 , 2 π ] , denoted by f : [0 , 2 π ] → [0 , 2 π ] .
Notably, due to the periodicity of angular values, 0 and 2 π are the same point, so to preserve continuity at both ends, we must ensure
that f (0) = 0 , f (2 π ) = 2 π , and f ′ (0) =
f ′ (2 π ) .
To achieve this, we use the rational quadratic spline flow (Durkan et al., 2019; Rezende et al., 2020), a piece-wise function that contains
K pieces delimited by K + 1 knots. Each piece takes the form: [Equation]
[Equation]
where s k =
y k
+1 − y k x k +1 − x k , and ξ ( x ) = x − x k x k +1 − x k
[Equation]
x ∈
[Equation]
x k , x k +1 ]) , (2) where the spline is parameterized by the coordinates and derivatives of the K + 1 knots denoted by x k , y k , and δ k ( k = 1 . . .
K + 1 ).
To fulfill the requirement of monotonicity, continuity, and periodicity, {+we impose the following constraints on the knots: i. 
+}
[-+Hbmt5+o0iyWD2aS0EDgoWQRI9hYqdUjI9b3+6WyV/XmQOvEz0kZcjT6pZ/eICapoNIQjrXu+l5iggwrwwinU7eXappgMsZD2rVUYkF1kM2vnaKKVQYoipV90qC5+rcjw0LriQhtpcBmpFe9mfif101NdBNkTCapoZIsFkUpRyZGs6+jAVOUGD6xBBPF7K2IjLDCxNiAXLeCSKpNLFA+b3lxKKauzclfTWWdtC6r/lW1dl8r1708sSKcwTlcgA/XUIc7aEATCDzCM7zCm/PivDsfzueitODkPaewBOfrF6h/nDg=</latexit>  we impose the following constraints on the knots: i. 
-]
= x
1 < x 2 < . . .
< x K +1 = 2 π , ii. {+
+}
= y 1 < y 2 < . . .
<
y K +
= 2 π , iii.
δ k > 0( k = 1 . . .
K + 1) , and iv.
δ 1 =
δ K +1 .
These parameters are produced by transforming the hidden representation h i of residues using neural networks, so the probability distribution defined by the bijective is conditional on the residue and its environment.
Therefore, we may also denote the spline by f ( x | h i ) .
We choose the uniform distribution on [0 , 2 π ] as the base distribution, represented by p z ( z ) = 12 π ( z ∈ [0 , 2 π ] ).
The mapping from the target distribution to the base distribution is denoted by f .
According to the change-of-variable formula, the target rotamer density of the i -th residue is given [Table] N [Table]
… … (A) (B)
[Table]
Rotamer
[Table]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Table]
+Hbmt5+o0iyWD2aS0EDgoWQRI9hYqdUjI9b3+6WyV/XmQOvEz0kZcjT6pZ/eICapoNIQjrXu+l5iggwrwwinU7eXappgMsZD2rVUYkF1kM2vnaKKVQYoipV90qC5+rcjw0LriQhtpcBmpFe9mfif101NdBNkTCapoZIsFkUpRyZGs6+jAVOUGD6xBBPF7K2IjLDCxNiAXLeCSKpNLFA+b3lxKKauzclfTWWdtC6r/lW1dl8r1708sSKcwTlcgA/XUIc7aEATCDzCM7zCm/PivDsfzueitODkPaewBOfrF6h/nDg=</latexit>  by:
[Equation]
The derivative f ′ can be computed analytically according to Eq.1.
See Figure 2A for an illustration of the model.
In practice, we stack multiple bijectives to enable more complex transformation.
The derivative of the composite can be computed efficiently using the chain rule.
At inference time, we can efficiently compute the inverse mapping f − 1 ( y ) (Rezende et al., 2020):
To find the solution of f − 1 ( y ) , the first step is to locate the unique bin that contains y .
Assuming y belongs to the k -th bin, finding its corresponding x amounts to finding the root of the quadratic equation f k ( x | x k,k +1 , y k,k +1 , δ k,k +1 )
=
y in the interval [ x k , x k +1 ] , for which a closed-form solution exists.
Conditional Flow on T D
Rotamers with D torsional angles can be viewed as points on the Ddimensional torus which is the product of D circles S 1 , i.e. T D = S 1 × · · · × S 1 .
To model the distribution on
T D , we adopt the coupling layer technique to model the joint distribution (Dinh et al., 2016).
Each coupling layer updates one dimension using the bijective for S 1 , keeping the other D − 1 dimensions fixed, and uses the D − 1 dimensions along with the hidden representation of the residue as the conditioner to parameterize the bijective (Figure 2B):
[Equation]
where d is the dimension to update.
The coupling layer g d preserves invertibility and has closedform inversion.
The determinant of its Jacobian is equal to the derivative of f ( x j | x \ j , h i ) .
We stack multiple spline-based coupling layers to ensure that all the dimensions are updated at least once, resulting in a stack of L coupling layers: g = g 1 ◦ g 2 ◦ · · · ◦ g L ( L ≥ D ).
We choose the uniform distribution as the base distribution, denoted by p z
( z ) =
( 12 π ) D and let g map the target distribution to the base distribution.
The log-likelihood is computed using the change-of-variable rule:
[Equation]
Training Objective To train the rotamer density estimator, we minimize the negative loglikelihood of native (ground truth) rotamers:
[Equation]
[Equation]
where S = { a j , p j , O j , ˜ χ j } Nj =1 is a protein structure sampled from the dataset.
Since the model is designed for mutation analysis, the prior rotamers should emulate the rotamers after some residues are mutated.
Thus, we mask the rotamers of a random portion of residues and add noise to the rotamers of the neighbors of the masked ones.
The negative log-likelihood training objective function is evaluated on the masked residues and a random part of the perturbed residues.
3. R OTAMER E NTROPY E STIMATION
The rotamer density estimator models the distribution of possible rotamers of residue, making the entropy of the rotamer distribution a natural measure of conformational flexibility (Brady & Sharp, 1997):
[Equation]
To estimate the entropy, we use a stochastic method: First, we sample a set of rotamers from the distribution using the inverted flows (Eq.1, 4).
Then, we compute the negative log probability of the samples and take their average as an estimate of the entropy.
Computing these steps is efficient thanks to the ability to compute the exact likelihood of normalizing flows.
For residues without rotatable sidechains (alanine, glycine, and proline), we consider their conformational entropy to be constant.
Nonetheless, we can still estimate the entropy of neighboring residues to evaluate their impact on conformational flexibility.
3. M UTATIONAL E FFECT ( ∆∆ G ) P REDICTION
Let us consider a protein-protein complex W LR = { a i , p i , O i , ˜ χ i } consisting of N residues, where M of them belong to the first group L = { 1 , . . .
, M } , and the remaining N − M be- long to the second group R =
{ M + 1 , . . . , N } .
We refer to group L as the ligand group since it contains mutations, and group R as the receptor group . In the unbound state, we represent the two separated structures by W L and W R respectively.
If we mutate m residues in group L , numbered by { 1 , . . .
, m } ⊂ L , the mutated structure in the bound state is denoted by M [Equation]
, p
[Equation] , O [Equation] , ∅ } mi =1 ∪{ a i , p i , O i , ˜ χ i } unmutated , where ˜ a i represents the mutated residue type and ∅ indicates that the rotamers of the mutated residues are unknown.
The two groups in the unbound state are denoted by M L and M R respectively.
Note that M R = W R if no mutations are in the receptor group.
Linear Predictor
The entropy loss upon binding of a protein complex S ( S = W , M ) is defined as the difference of entropy between the bound and the unbound states (Kastritis & Bonvin, 2013).
We approximate the entropy loss ∆ H S using a linear model and our estimated entropy:
[Equation]
where w S J > 0 ( S = W , M ; J = L, R ) is the coefficient that controls the contribution of entropy to the binding free energy, and E ( · ) is the entropy bias for 20 different amino acid types.
The thermodynamic definition states that the core component in binding ∆∆ G is the difference in entropy loss between the wild-type and mutant structure.
Therefore, we estimate ∆∆ G by:
[Equation]
Note that as we assume there are no mutations in the receptor group ( M R = W R ), H unbnd M R and H unbnd W R cancel out each other and do not contribute to ∆∆ G .
In some cases where mutations are on both groups, we consider the set of mutated residues as the ligand group and all the remaining as the receptor group.
To calibrate the parameters in Eq.9, we use the block coordinate descent method, which alternates between optimizing the coefficients ( w S J , b ) and the entropy biases ( E ) using the mean squared error (MSE) loss. [Table]
Neural Network Predictor
Each residue’s hidden representation h i used to parameterize the normalizing flows contains sufficient information about the rotamer distribution.
To extract binding information from these representations in a more flexible way, we employed neural networks.
Specifically, we utilized a network that shares the same architecture as the encoder to transform the representation h i and applied max-pooling to obtain a global structure representation.
We then subtracted the representation of the wild-type structure from the mutant representation and fed it into another MLP to predict ∆∆ G .
To enforce anti-symmetry, we swapped the wild-type and mutant to predict − ∆∆ G , and computed (∆∆ G − ( − ∆∆ G )) / 2 as the final prediction.
The network was trained using the MSE loss.
During training, we freeze the RDE weights and do not back-propagate gradients through h i to fully exploit the unsupervised representations learned by RDE.
3. M ODEL T RAINING
The dataset for training the rotamer density estimator is derived from PDB-REDO (Joosten et al., 2014), which is a database containing refined X-ray structures in PDB.
The protein chains are clustered based on 50% sequence identity, leading to 38,413 chain clusters, which are randomly divided into the training, validation, and test sets by 95%/0.5%/4.5% respectively.
During training, the data loader randomly selects a cluster and then randomly chooses a chain from the cluster to ensure balanced sampling.
We crop structures into patches containing 128 residues by first choosing a seed residue, and then selecting its 127 nearest neighbors based on C-beta distances.
To simulate mutations, we masked the rotamers of 10% of residues in the patch, and we added noise to the rotamers of residues whose C-beta distance to the closest masked residue was less than 8 ˚A.
The SKEMPI2 database (Jankauskait˙e et al., 2019) is used to train the models for ∆∆ G prediction described in Section 3.4.
We split the dataset into 3 folds by structure, each containing unique protein complexes that do not appear in other folds.
Two folds are used for training and validation, and the remaining fold is used for testing.
This approach yields 3 different sets of parameters and ensures that every data point in SKEMPI2 is tested once.
R ESULTS 4. P REDICTION OF THE E FFECT OF M UTATIONS ON B INDING
Baselines We evaluate the performance of our two ∆∆ G predictors, RDE-Linear and RDENetwork , against several categories of baseline methods.
The first category comprises traditional empirical energy functions, including Rosetta Cartesian ddG (Park et al., 2016; Alford et al., 2017; Leman et al., 2020) and FoldX (Delgado et al., 2019).
The second category consists of sequence/evolution-based methods, represented by ESM-1v (Meier et al., 2021), PSSM (positionspecific scoring matrix),
MSA Transformer (Rao et al., 2021), and Tranception (Notin et al., 2022).
The third category includes end-to-end learning models, such as DDGPred (Shan et al., 2022), and a model that shares the same encoder architecture as RDE but uses an MLP to di- [Figure] rectly predict ∆∆ G
( End-to-End ).
The fourth category consists of unsupervised/semi-supervised learning methods, including ESM-IF (Hsu et al., 2022) and Masked Inverse Folding (MIF)
(Yang et al., 2022).
Similar to our RDE-Network, this class of methods pre-train a network on structures and use the pretrained representations to predict ∆∆ G s.
The baseline MIF network also uses the same encoder architecture as RDE for comparison.
There are two variants for ∆∆ G prediction: MIF- ∆ logit , which uses the difference in log-probability of amino acid types to predict ∆∆ G , and
MIF-Network , which predicts ∆∆ G from the learned representations using the same network architecture as RDE-Network.
Finally, given that our method is based on conformational flexibility, we train a network to predict the B-factor of residues and use predicted B-factors in place of entropy to predict ∆∆ G .
Metrics We use five metrics to evaluate the accuracy of ∆∆ G prediction: Pearson correlation coefficient, Spearman ’s rank correlation coefficient, minimized RMSE (root mean squared error), minimized MAE (mean absolute error), and AUROC (area under the receiver operating characteristic).
To calculate AUROC, mutations are classified based on the sign of ∆∆ G .
In practical applications, the correlation for one specific protein complex is often of greater interest.
Therefore, we group mutations by structure, discard groups with less than 10 mutation data points and calculate correlations for each structure separately.
This leads to two additional metrics: average per-structure Pearson correlation coefficient and average per-structure Spearman correlation coefficient.
Results According to Table 1, our RDE-Network outperforms all the baselines.
Notably, it demonstrates a significant improvement in per-structure correlations, indicating its greater reliability for practical applications.
The superior performance of RDE-Network over MIF-Network suggests that representations derived from fitting rotamer densities are more effective than those from masked inverse folding, as RDE captures atomic interactions well by modeling the conformation of sidechain atoms.
RDE-Linear achieves comparable performance to Rosetta and outperforms some unsupervised learning baselines.
While it does not surpass most baseline methods over the entire SKEMPI dataset, we observe that its performance is better when considering only single-point mutations (Table 6 in the appendix).
This might be attributed to the fact that simple linear models cannot capture well the non-linear relationship dominating multi-point mutations.
Nevertheless, RDE-Linear demonstrates that using the basic statistics of the estimated rotamer density alone can predict ∆∆ G , which lays the foundation for the more accurate RDE-Network.
Sequence-based models do not accurately predict ∆∆ G for protein-protein binding, as discussed in Section 2.2.
Figure 3 shows the distribution of per-complex correlation coefficients.
Please refer to Section B of the appendix for more results and discussion.
4. O PTIMIZATION OF H UMAN A NTIBODIES A GAINST SARS-C O V-
In Shan et al.
, the authors report five single-point mutations on a human antibody against SARS-CoV-2 that enhance neutralization effectiveness.
These mutations are among the 494 possible single-point mutations on the heavy chain CDR region of the antibody.
We use the most competitive methods benchmarked in Section 4.1 to predict ∆∆ G s for all the single-point mutations and rank them in ascending order (lowest ∆∆ G in the top).
The effectiveness of a predictor is determined by the number of favorable mutations ranked in the top place.
As shown in Table 2, RDE-Network and DDGPred successfully identify three mutations (Ranking ≤ 10%), with RDE-Network ranking them higher.
[Table] [Table]
4. A NALYSIS OF THE E NTROPY E STIMATED BY THE R OTAMER D ENSITY E STIMATOR
Statistical Significance To demonstrate a statistically significant relationship between the entropy estimated by RDE and experimental ∆∆ G values, we conduct linear regression analysis using the RDE-Linear model defined in Eq.
The linear model consists of seven coefficients and one bias term: w bound W L , w bound W R , w unbnd W L , w bound M L , w bound M R , w unbnd M L , w unbnd R = ( w unbnd M R − w unbnd W R ) , and b .
Note that w unbnd M R and w unbnd W R are merged into w unbnd R , as the receptor is not mutated.
We perform linear regression on the SKEMPI2 single-mutation dataset and present the regression coefficients, bias, and P-values in Table 3.
According to the statistics, all entropy terms, except the entropy of unbound receptor H unbnd R (coefficient w unbnd R ), show a significant relationship with experimental ∆∆ G s.
The coefficients of the significant terms are all positive and roughly similar.
The entropy of the unbound receptor H unbnd R has no contribution because the receptor alone does not involve in the mutation.
These results agree with the thermodynamic definition of the change in binding free energy:
∆∆
[Equation]
) , where G M R and G W R cancel each other out as the receptor is unmutated, therefore indicating that our model captures well the thermodynamics underlying protein-protein interactions.
Correlation Between Estimated Entropy and B-factors B-factor is an experimental measurement that quantifies the conformational flexibility.
We calculate the average b-factor of sidechain atoms of residues in the test split of the PDB-REDO dataset.
Then, we estimate the conformational entropy for each residue in the test split using the RDE.
The average Pearson correlation coefficient between these two quantities is 0.4637, and the average Spearman coefficient is 0.4282.
Detailed results are presented in Table 8 in the appendix.
In summary, this indicates that there is a correlation between the entropy estimated by the RDE and experimentally determined conformational flexibility measured by B-factor.
4. P REDICTION OF S IDECHAIN C ONFORMATIONS
[Table]
RDE is a generative model for protein sidechain structures, which can predict sidechain conformations by sampling from the estimated distribution.
We use RDE to sample sidechain torsional angles (rotamers) for structures with 10% sidechains removed in our test split of PDB-REDO.
For each residue, 10 rotamers are sampled independently, and the one with the highest probability is selected as the final prediction.
We compare RDE with two baseline methods Rosetta (fixbb) (Leman et al., 2020) and SCWRL4 (Krivov et al., 2009).
Our results shown in Table 4 demonstrate that the RDE outperforms the baselines on all four torsional angles in terms of angular errors.
For a detailed per-amino-acid accuracy, please refer to Table 9 in the appendix.
C ONCLUSIONS
In this work, we introduce the Rotamer Density Estimator (RDE) which estimates the distribution of rotamers for protein sidechains.
We demonstrate that RDE leads to improved accuracy in predicting binding ∆∆ G compared to other methods.
One limitation of RDE is the inability to model backbone flexibility directly which is an important future direction for extending the proposed model. Nonetheless, our work highlights the potential of using machine learning techniques to improve mutational effect prediction for protein-protein interaction.
A I MPLEMENTATION D ETAILS A. N ETWORK A RCHITECTURE
The encoder of the rotamer density estimator is a stack of attention layers invariant to the rotation and translation (Jumper et al., 2021) of the input protein structure.
Let h ℓi denote the embedding of the i -th amino acid output by the previous attention layer.
The logit of attention weights between residue i and residue j is defined as:
[Equation]
where Q , K , G , and H are MLP networks that transform the embeddings into queries, keys, pairwise bias, and distance bias, respectively.
The attention weight is computed by taking the softmax on the j dimension: w ij = softmax j ( a ij ) .
In practice, we use multiple attention heads.
Each attention head has different attention weights.
The vector to update the representation of residue i from residue j is computed by:
[Equation]
Finally, the sum of { v ij } j weighted by { w ij } j is used to update the representation of residue j using residual connection and layer normalization, similar to the standard transformer architecture.
The rotamer density estimator has 6 encoder layers.
Node features and pairwise features havechannels and 64 channels respectively.
The normalizing flow has 8 blocks and each spline hasknots, dividing [0 , 2 π ] into 64 bins.
A. T RAINING
The rotamer density estimator is trained using the Adam optimizer for 200K iterations.
The learning rate is 0.0001.
The learning rate decays by 0.8 if the validation loss does not decrease in the last 5 validation steps (the model is validated every 1000 iterations), until the learning rate reaches 0.000001.
The batch size is 64.
It takes 8h56m in total on a single A100 GPU.
To emulate mutations, the rotamers of 10% of amino acids are masked.
Noise is added to the rotamers of amino acids whose C-beta distance to the closest masked amino acid is less than 8.0 ˚A.
The noise added to χ angles consists of two components.
The first component is Gaussian noise centered at 0 wrapped into [ − π, π ] .
It standard deviation is dependent on the C-beta distance: σ ij = − 116 β ij + 1 , where i is the index of the nearest masked amino of the j -th amino acid that the noise is added to, and β ij is the C-beta distance between them.
The second component is random flipping (adding π to the angle).
Every χ angle in the 8 ˚A neighborhood has 25% chance of being flipped.
Our noise model is totally empirical.
There are other ways to perturb rotamers, for example, using rotamer libraries (Dunbrack Jr & Karplus, 1993; Bower et al., 1997; Dunbrack Jr, 2002; Shapovalov & Dunbrack Jr, 2011).
We leave the problem of finding an optimal noise model that emulates mutations in future work.
A. B ASELINES
Baselines that require training and calibration using the SKEMPI2 dataset (DDGPred, End-to-End,
B-factor, MIF- ∆ logit, MIF-Network) are trained independently using the 3 different splits of the dataset as described in Section 3.5.
This is to ensure that every data point in the SKEMPI2 dataset is tested on once.
Below are descriptions of the implementation of baseline methods.
Rosetta (Alford et al., 2017; Leman et al., 2020)
The version we used is 2021.16, and the scoring function is ref2015 cart .
Every protein structures in the SKEMPI2 dataset are first pre-processed using the relax application.
The mutant structure is built by cartesian ddg .
The binding free energies of both wild-type and mutant structures are predicted by interface energy (dG separated/dSASAx100).
Finally, the binding ∆∆ G is calculated by substracting the binding energy of the wild-type structure from the binding energy of the mutant.
FoldX (Delgado et al., 2019)
Structures are first relaxed by the RepairPDB command.
Mutant structures are built with the BuildModel command based on the repaired structure.
The change in binding free energy ∆∆ G is calculated by subtracting the wild-type energy from the mutant energy.
ESM-1v (Meier et al., 2021)
We use the implementation provided in the ESM open-source repository.
Protein language models can only predict the effect of mutations for single protein sequences.
Therefore, we ignore cases where mutations are on multiple sequences.
We extract the sequence of the mutated protein chain from the SEQRES entry of the PDB file.
We use masked-marginals mode to score both wild-type and mutant sequences and use their difference as the estimation of ∆∆ G .
PSSM We construct MSAs from the Uniref90 database (Suzek et al., 2007) for chains with mutation annotations in the SKEMPI dataset.
We use Jackhmmer (Johnson et al., 2010) version 3.3.1 following the setting in Meier et al.
The MSAs are filtered using HHfilter (Steinegger et al., 2019) with coverage 75 and sequence identity 90.
This HHfilter parameter is reported to have the best performance for MSA Transformer according to Meier et al.
We calculate position-specific scoring matrices (PSSM) and use the change in probability as the prediction of ∆∆ G .
MSA Transformer (Rao et al., 2021)
We use the implementation provided in the ESM open-source repository.
We input the MSAs constructed during the evaluation of PSSM to MSA Transformer.
We used masked-marginals mode to score both wild-type and mutant sequences and use their difference as the prediction of ∆∆ G .
Tranception (Notin et al., 2022)
We use the implementation provided in the Tranception opensource repository.
We predict mutation effects using the large model checkpoint.
Previously built
MSAs (not filtered by HHfilter) are used for inference-time retrieval.
DDGPred (Shan et al., 2022)
We use the implementation accompanying the paper by Shan et al.
Since this model requires predicted sidechain structures of the mutant, we use mutant structures packed during our evaluation of Rosetta to train the model and run prediction.
End-to-End The end-to-end model shares the same encoder architecture as the rotamer density estimator.
The difference is that in the RDE normalizing flows follow the encoder to model rotamer distributions, but in the end-to-end model, the embeddings are directly fed to an MLP to predict ∆∆ G .
B-factor This model predicts per-atom b-factors for proteins.
It has the same encoder architecture as the RDE.
Following the encoder is an MLP that predicts a vector for each amino acid where each dimension is the predicted b-factor of different atoms in the amino acid.
The amino acid-level b-factor is calculated by averaging the atom-level b-factors.
The predicted b-factors are used as a measurement of conformational flexibility.
They are used to predict ∆∆ G using the linear model same as RDE-Linear defined in Eq.9.
ESM-IF (Hsu et al., 2022)
ESM-IF can score protein sequences using the log-likelihood.
The scoring function implementation is provided in the ESM repository.
We enable the --multichain backbone flag to let the model see the whole protein-protein complex.
We subtract the log-likelihood of the wild-type from the mutant to predict ∆∆ G .
MIF
Architecture The masked inverse folding (MIF) network uses the same encoder architecture as RDE.
Following the encoder is a per-amino-acid 20-category classifier that predicts the type of masked amino acids.
We use the same PDB-REDO train-test split to train the model.
At training time, we randomly crop a patch consisting of 128 residues, and randomly mask 10% amino acids.
The model learns to recover the type of masked amino acids with the standard cross entropy loss.
MIF- ∆ logit To score mutations, we first mask the type of mutated amino acids.
Then, we use the log probability of the amino acid type as the score.
Analogously, we have the score of the wildtype bound ligand, wild-type bound receptor, wild-type unbound ligand, unbound receptor, mutated bound ligand, mutated bound receptor, and mutated unbound ligand.
Therefore, we use the linear model identical to RDE-Linear (Eq.9) to predict ∆∆ G from the scores.
MIF-Network This is similar to RDE-Network.
The difference is that we use the pre-trained encoder of MIF rather than the encoder of RDE.
We also freeze the MIF encoder as we aim to utilize the unsupervised representations.
A. S OURCE C ODE
Available at https://github.com/luost26/RDE-PPI .
B A DDITIONAL R ESULTS B. F URTHER A NALYSIS ON THE P ERFORMANCE OF S EQUENCE -B ASED M ETHODS
As discussed in Section 2.2, sequence-based (evolution-based) methods are not suitable for proteinprotein interactions due to missing co-evolutionary information between two proteins.
This is supported by the results in Table 1, which show that sequence-based methods do not accurately predict ∆∆ G .
Below we analyze two classes of PPIs to provide a better understanding of the performance of sequence-based methods on PPIs.
Antibody-antigen binding is a typical class of PPI that lacks co-evolutionary information.
The first reason is that the binding interface of antibodies (complementarity determining region) is highly variable (Weitzner et al., 2015).
There is no evolutionary history in the region.
Therefore, it is infeasible to mine the preference of mutations from sequence databases.
Another reason is that antigens do not evolve to increase binding to antibodies.
The evolution of antigens is typically independent of specific antibodies.
Hence, in most cases, sequence databases can provide little information about mutational effects on antibody-antigen binding.
We select antibody-antigen complexes from the
SKEMPI datasets and evaluate per-structure Spearman correlation coefficients of the prediction by
MSA Transformer and RDE-Network on these antibody-antigen complexes.
The average Spearman score of MSA Transformer is 0.0744 and the average score of RDE-Network is 0.4284.
Further, we plot the results in Figure 4, where x -axis
and y -axis are per-structure Spearman coefficients of MSA
Transformer and RDE-Network respectively.
Orange crosses represent antibody-antigen complexes and blue dots represent other complexes.
The results show that under circumstances where we have little co-evolutionary information such as antibody-antigen binding, structure-based methods represented by our RDE perform better than evolution-based methods represented by MSA Transformer.
When the proteins in a complex come from the same organism, evolution-based methods are more likely to be effective.
Proteins in such complex usually function as a whole in the organism, so they evolve together.
Mutations that enhance the complexation might be more favorable and this preference might be reflected in evolutionary history.
We inspect 10 complexes on which MSA
Transformer performs the best in terms of Spearman coefficients, and find that 9 of them consist of proteins from a single organism (Table 5).
However, when evaluating MSA Transform on all the single-organism complexes, its Spearman score is 0.1651, which is still not high though better than antibody-antigen cases.
The reason might be that even if the proteins in a complex come from the same organism, the member protein might also bind to other proteins to be functional.
In this case, it is more difficult to predict its binding in a specific complex according to its general evolutionary history.
[Figure]
[Figure] p [Figure] m
[Figure]
n )
[Table]
B. S INGLE -M UTATION AND M ULTI -M UTATION
[Table]
[Table]
End-to-End 0.
B-factor 0.
ESM-IF 0.
MIF- ∆ logit 0.
MIF-Net.
RDE-Linear 0.
RDE-Net.
[Figure]
C ORRELATION WITH B-F ACTOR
[Table]
[Table]
ONFORMATION
[Table]
REDICTION
[Table]