Title: Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
Venue: NeurIPS 2023
Accepted: yes
Keywords: Riemannian Diffusion Probabilistic Model

Mutation

Protein-protein binding
Abstract: Many crucial biological processes rely on networks of protein-protein interactions. Predicting the effect of amino acid mutations on protein-protein binding is important in protein engineering, including therapeutic discovery. However, the scarcity of annotated experimental data on binding energy poses a significant challenge for developing computational approaches, particularly deep learning-based methods. In this work, we propose SidechainDiff, a novel representation learning-based approach that leverages unlabelled experimental protein structures. SidechainDiff utilizes a Riemannian diffusion model to learn the generative process of side-chain conformations and can also give the structural context representations of mutations on the protein-protein interface. Leveraging the learned representations, we achieve state-of-the-art performance in predicting the mutational effects on protein-protein binding. Furthermore, SidechainDiff is the first diffusion-based generative model for side-chains, distinguishing it from prior efforts that have predominantly focused on the generation of protein backbone structures.

Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model

Abstract

Many crucial biological processes rely on networks of protein-protein interactions. Predicting the effect of amino acid mutations on protein-protein binding is vital in protein engineering and therapeutic discovery. However, the scarcity of annotated experimental data on binding energy poses a significant challenge for developing computational approaches, particularly deep learning-based methods. In this work, we propose SidechainDiff, a representation learning-based approach that leverages unlabelled experimental protein structures. SidechainDiff utilizes a Riemannian diffusion model to learn the generative process of side-chain conformations and can also give the structural context representations of mutations on the proteinprotein interface. Leveraging the learned representations, we achieve state-of-theart performance in predicting the mutational effects on protein-protein binding. Furthermore, SidechainDiff is the first diffusion-based generative model for sidechains, distinguishing it from prior efforts that have predominantly focused on generating protein backbone structures.

1 Introduction

Many crucial biological processes, including membrane transport and cell signaling, are likely regulated by intricate networks of protein-protein interactions rather than single proteins acting independently [Calakos et al., 1994, Jones and Thornton, 1996, Levskaya et al., 2009, Schwikowski et al., 2000]. One representative example is the interaction between the spike protein of the SARSCoV-2 virus and its receptor protein ACE2 on the surface of human cells, which is crucial for the virus to invade target cells [Lan et al., 2020]. Meanwhile, specific human antibodies, an essential type of protein in the immune system, can prevent the virus entry by competitively binding to the spike protein [Shan et al., 2022].

In protein engineering and therapeutic discovery, it is a crucial step to induce amino acid mutations on the protein-protein interface to modulate binding affinity [T O'Neil and Hoess, 1995]. For example, to enhance the efficacy of an antibody against a virus, one practical approach is to introduce amino acid mutations and filter the resulting antibody sequences to increase binding affinity and specificity to the target viral protein [Mason et al., 2021]. However, the variant space that can be explored

using experimental assays is very limited, and developing an effective high-throughput screening can require a significant experimental effort.

Computational methods have been developed to predict the mutational effect on binding affinity measured by the change in binding free energy (i.e., ∆∆ G ). Traditional computational approaches mostly used physical energy features, such as van der Waals and solvation energy, in combination with statistical models to predict ∆∆ G [Schymkowitz et al., 2005, Meireles et al., 2010, Alford et al., 2017, Barlow et al., 2018]. The limited model capacity and bias inherent in human-engineered features in these methods hinder their ability to characterize the complex mutational landscape of binding energy. Despite recent breakthroughs in protein modeling with deep learning [Jumper et al., 2021, Baek et al., 2021, Watson et al., 2023], developing deep learning-based models to predict mutational effects on protein-protein binding remains challenging due to the scarcity of labeled experimental data [Jankauskait˙ e et al., 2019].

Recent studies have investigated various self-supervised learning strategies on protein structures and sequences [Liu et al., 2021, Meier et al., 2021, Hsu et al., 2022, Luo et al., 2023] to ease the data scarcity issue. Among them, GeoPPI [Liu et al., 2021] and RDE [Luo et al., 2023] have focused on optimizing protein side-chain conformations during pre-training, as the side-chain conformations on the protein-protein interface play a critical role in determining binding energy. In protein-protein interactions, the side-chain conformations of amino acids at the interface may exhibit significant variations when comparing the same receptor with different ligands. These side-chain conformations can be more effectively characterized using probability density. Notably, RDE explores a flowbased model to estimate the uncertainty in side-chain conformations and leverages the learned representations to achieve state-of-the-art performance in predicting ∆∆ G . However, flow-based models possess inherent limitations as they require specialized architectures to construct accurate bijective transformations in probability density [Papamakarios et al., 2021], which results in increased costs associated with model design and implementation.

To address the above limitations, we propose SidechainDiff, a representation learning framework for the protein-protein interfaces. It employs a Riemannian diffusion model to learn the generative process of side-chain conformations and the representation of the structural context of amino acids. To the best of our knowledge, SidechainDiff is the first diffusion probabilistic model for side-chain modeling, whereas previous methods have only focused on protein backbone structures. Furthermore, we leverage the learned representations and neural networks to predict ∆∆ G .

2 Related Work

2.1 Protein side-chain conformation prediction

Accurate side-chain modeling is essential in understanding the biological functions of proteins. There are two primary categories in predicting protein side-chain conformations: end-to-end full-atom structure prediction methods and side-chain packing. AlphaFold and RoseTTAFold [Jumper et al., 2021, Baek et al., 2021] are two representative methods that simultaneously generate side-chain conformations and backbone structures. In scenarios like structure refinement and protein design, side-packing becomes pivotal. The objective is to determine the conformations of protein side-chains while having their backbone structures already defined. Traditional methods, including Rosetta [Leman et al., 2020] and SCWRL4 [Krivov et al., 2009] operate by minimizing the energy function across a pre-defined rotamer library. Recent methods for side-chain packing often employ deep learning models, such as 3D convolution networks and graph attention networks [McPartlon and Xu, 2023, Misiura et al., 2022, Xu et al., 2022].

Our model distinguishes itself from previous methods for side-chain modelling in two key aspects. First, it's capable of generating a distribution of side-chain conformations rather than a single conformation. Second, we emphasize side-chain modeling specifically for mutation sites within the protein-protein interface, leveraging the known structural context of these sites.

2.2 Prediction of mutational effects on protein-protein binding

Methods for predicting ∆∆ G can be broadly classified into three categories: biophysical methods, statistical methods, and deep learning-based approaches. Biophysical methods offer a robust means of elucidating the molecular mechanisms governing protein-protein interactions and the impact of

mutations on these interactions [Schymkowitz et al., 2005, Alford et al., 2017, Steinbrecher et al., 2017]. These methods directly integrate information on protein structures and key biophysical properties, such as solvent accessibility, electrostatic potential, and hydrogen bonding patterns. Statistical methods tailor statistical models for the protein properties such as evolutionary conservation and geometric characteristics [Li et al., 2016, Geng et al., 2019, Zhang et al., 2020].

Deep learning-based methods can be categorized into sequence-based methods and structure-based methods. Sequence-based methods primarily either focus on the evolutionary history, multiple sequence alignments (MSAs) in most cases, of the target protein [Hopf et al., 2017, Riesselman et al., 2018, Frazer et al., 2021] or act as protein language models (PLMs) trained on large amounts of protein sequences [Meier et al., 2021, Notin et al., 2022]. Structure-based methods can be categorized into end-to-end methods and pre-training-based methods. The end-to-end methods extract features from protein complexes and directly train a neural network model on them [Shan et al., 2022]. To mitigate overfitting caused by data sparsity, an alternative approach is to learn representations by pre-training a feature extraction network on unlabeled structures [Liu et al., 2021, Luo et al., 2023]. Among them, RDE-Network [Luo et al., 2023] utilizes normalizing flows in torus space to estimate the density of amino acid side-chain conformations and leverages the learned representation to predict ∆∆ G .

2.3 Diffusion probabilistic models

Diffusion Probabilistic Models (DPMs) are generative models to transform a sample from a tractable noise distribution towards a desired data distribution with a gradual denoising process [Sohl-Dickstein et al., 2015, Kingma et al., 2021, Dhariwal and Nichol, 2021]. DPMs have achieved impressive results in generating various data types, including images [Ho et al., 2020, Nichol and Dhariwal, 2021], waveforms, and discrete data like text [Hoogeboom et al., 2021]. DPMs-based autoencoders have also been proposed to facilitate representation learning for image data [Preechakul et al., 2022, Zhang et al., 2022].

Inspired by these progresses, DPMs have also been explored in protein modeling, including de novo protein design [Watson et al., 2023, Ingraham et al., Yim et al., 2023], motif-scaffolding [Trippe et al., 2023], and molecular dynamics [Arts et al., 2023, Wu and Li, 2023]. While the existing methods utilize DPMs for generating protein backbones, a research gap remains in modeling side-chain conformations, which play a critical role in protein-protein binding. The studies by Jing et al. [2022] and Corso et al. [2023] employ diffusion models for generating torsion angles in the context of small molecular design. In contrast, our research is centered on modeling the torsional angles of protein side-chains. Furthermore, our approach distinguishes itself from their models in terms of how we construct the diffusion process.

3 Methods

Our methods have two main components. We first propose SidechainDiff (Figure 1), which is a diffusion probabilistic model for protein side-chains and can also give the structural context representations of mutations on the protein-protein interface. We then propose DiffAffinity that utilize the learned representations to predict ∆∆ G . We organize this section as follows: Section 3.1 presents the preliminaries and notations that are used throughout the paper and formally defines the problem. Section 3.2 and 3.3 describe SidechainDiff and DiffAffinity, respectively. Section 3.4 describes key details in model training.

3.1 Preliminaries and notations

A single-chain protein structure is composed of multiple residues (amino acids). Each amino acid shares a common central carbon atom, called the alpha ( α ) carbon. The alpha carbon is bonded to an amino group (-NH2), a carboxyl group (-COOH), and a hydrogen atom, forming the amino acid's backbone. Each amino acid has a distinct atom or group bonded to the alpha carbon, known as the side-chain. A residue's side-chain structure is usually termed a side-chain conformation. The side-chain conformation varies across different amino acids, and it depends on the type of amino acid and the overall protein structure. Our main work focuses on building a generative process of side-chain conformations given the protein backbone structures.

GLYPH<c=1,font=/AAAAAO+Helvetica>GLYPH<c=2,font=/AAAAAO+Helvetica>GLYPH<c=3,font=/AAAAAO+Helvetica>GLYPH<c=4,font=/AAAAAO+Helvetica>GLYPH<c=5,font=/AAAAAO+Helvetica>GLYPH<c=3,font=/AAAAAO+Helvetica>GLYPH<c=6,font=/AAAAAO+Helvetica>GLYPH<c=7,font=/AAAAAO+Helvetica>GLYPH<c=8,font=/AAAAAO+Helvetica>GLYPH<c=2,font=/AAAAAO+Helvetica>GLYPH<c=9,font=/AAAAAO+Helvetica>GLYPH<c=4,font=/AAAAAO+Helvetica>

GLYPH<c=1,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=3,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>GLYPH<c=5,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=6,font=/AAAAAP+Helvetica-Bold>GLYPH<c=7,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=9,font=/AAAAAP+Helvetica-Bold>GLYPH<c=9,font=/AAAAAP+Helvetica-Bold>GLYPH<c=10,font=/AAAAAP+Helvetica-Bold>GLYPH<c=5,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=13,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=14,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=5,font=/AAAAAP+Helvetica-Bold>GLYPH<c=5,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=1,font=/AAAAAQ+TimesNewRomanPS-ItalicMT>

GLYPH<c=1,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=15,font=/AAAAAP+Helvetica-Bold>GLYPH<c=16,font=/AAAAAP+Helvetica-Bold>GLYPH<c=17,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=1,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=15,font=/AAAAAP+Helvetica-Bold>GLYPH<c=16,font=/AAAAAP+Helvetica-Bold>GLYPH<c=17,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=1,font=/AAAAAQ+TimesNewRomanPS-ItalicMT>

GLYPH<c=1,font=/AAAAAR+TimesNewRomanPSMT>

GLYPH<c=1,font=/AAAAAQ+TimesNewRomanPS-ItalicMT>

GLYPH<c=1,font=/AAAAAR+TimesNewRomanPSMT>

GLYPH<c=1,font=/AAAAAQ+TimesNewRomanPS-ItalicMT>

GLYPH<c=18,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>GLYPH<c=14,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=19,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=7,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=9,font=/AAAAAP+Helvetica-Bold>GLYPH<c=9,font=/AAAAAP+Helvetica-Bold>GLYPH<c=10,font=/AAAAAP+Helvetica-Bold>GLYPH<c=5,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>GLYPH<c=6,font=/AAAAAP+Helvetica-Bold>GLYPH<c=13,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=14,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=5,font=/AAAAAP+Helvetica-Bold>GLYPH<c=5,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=1,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=2,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=1,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=2,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=1,font=/AAAAAS+MicrosoftYaHei-Bold>

GLYPH<c=1,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=2,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=1,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=2,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=1,font=/AAAAAS+MicrosoftYaHei-Bold>

GLYPH<c=1,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=2,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=3,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=4,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=5,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=6,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=7,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=8,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=2,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=4,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=9,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=10,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=11,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=2,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=12,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=6,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=13,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=14,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=9,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=12,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=6,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=20,font=/AAAAAP+Helvetica-Bold>GLYPH<c=17,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=6,font=/AAAAAP+Helvetica-Bold>GLYPH<c=16,font=/AAAAAP+Helvetica-Bold>GLYPH<c=14,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=19,font=/AAAAAP+Helvetica-Bold>GLYPH<c=6,font=/AAAAAP+Helvetica-Bold>GLYPH<c=15,font=/AAAAAP+Helvetica-Bold>GLYPH<c=21,font=/AAAAAP+Helvetica-Bold>GLYPH<c=22,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=7,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=23,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=19,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>GLYPH<c=16,font=/AAAAAP+Helvetica-Bold>GLYPH<c=24,font=/AAAAAP+Helvetica-Bold>GLYPH<c=6,font=/AAAAAP+Helvetica-Bold>GLYPH<c=16,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>GLYPH<c=25,font=/AAAAAP+Helvetica-Bold>GLYPH<c=24,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=26,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>GLYPH<c=19,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>GLYPH<c=16,font=/AAAAAP+Helvetica-Bold>GLYPH<c=15,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=27,font=/AAAAAP+Helvetica-Bold>GLYPH<c=4,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=2,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>GLYPH<c=15,font=/AAAAAP+Helvetica-Bold>GLYPH<c=16,font=/AAAAAP+Helvetica-Bold>GLYPH<c=15,font=/AAAAAP+Helvetica-Bold>GLYPH<c=8,font=/AAAAAP+Helvetica-Bold>GLYPH<c=11,font=/AAAAAP+Helvetica-Bold>GLYPH<c=12,font=/AAAAAP+Helvetica-Bold>

GLYPH<c=3,font=/AAAAAS+MicrosoftYaHei-Bold>

GLYPH<c=3,font=/AAAAAS+MicrosoftYaHei-Bold>

GLYPH<c=3,font=/AAAAAS+MicrosoftYaHei-Bold>

GLYPH<c=3,font=/AAAAAS+MicrosoftYaHei-Bold>

GLYPH<c=15,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=13,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=16,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=9,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=17,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=5,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=14,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=12,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=7,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=18,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=17,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=19,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=20,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=21,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=15,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=7,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=22,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=12,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=17,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=9,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=4,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=2,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=12,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=16,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=23,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=17,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=14,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=24,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=25,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=3,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=5,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=12,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=22,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=2,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=17,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=26,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=12,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=14,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=10,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=17,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=13,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=5,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=7,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=23,font=/AAAAAT+MicrosoftYaHei>

GLYPH<c=4,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=5,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=6,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=7,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=8,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=9,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=10,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=11,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=5,font=/AAAAAS+MicrosoftYaHei-Bold>GLYPH<c=12,font=/AAAAAS+MicrosoftYaHei-Bold>

GLYPH<c=14,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=10,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=17,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=13,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=5,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=7,font=/AAAAAT+MicrosoftYaHei>GLYPH<c=27,font=/AAAAAT+MicrosoftYaHei>

(x1.X2.X3.X.)

T4s

space

Multiple protein chains bind with each other and form a protein complex. Here we only consider protein complexes with two chains just for notation simplicity and note that our model works for protein complexes with more chains. For a protein complex with n residues, we denote the two chains as chain A and B , and the set of its residues as { 1 , ..., n } . The properities of the i -th amino acid include the amino acid type a i ∈ { 1 , . . . , 20 } , the orientation R i ∈ SO (3) , the 3D coordinates x i ∈ R 3 , and the side-chain torsion angles χ i = ( χ ( k ) i ) 4 k =1 where χ ( k ) i belongs to [0 , 2 π ) .

The bond lengths and bond angles in side-chains are usually considered fixed and given backbone structures, we can solely use χ i to reconstruct the coordinates of each atom in a side-chain. Thus, a side-chain conformation is also called a rotamer, which is parameterized with χ i . The number of torsional angles ranges from 0 to 4, depending on the residue type. Here, we place a rotamer in a 4 -dimensional torus space T 4 = ( S 1 ) 4 where S 1 denotes a unit circle.

3.2 SidechainDiff: side-chain diffusion probabilistic model

SidechainDiff utilizes a conditional Riemannian diffusion model in T 4 to build the generative process of the four rotamers and estimate their joint distributions (Figure 1). The generative process is conditioned on the structural context of mutations on the protein-protein interface, which is encoded using a SE(3)-invariant neural network. The learned conditional vectors serve as a natural representation of the structural context.

Conditional diffusion probabilistic model on T 4 We adopt the continuous score-based framework on compact Riemannian manifolds [De Bortoli et al., 2022] to construct the generative process for rotamers within T 4 . Our primary adaptation involves conditioning our diffusion model on vectors that are concurrently learned via an encoder network.

Let the random variable X denote the states of rotamers. And let ( X t ) t ∈ [0 ,T ] and ( Y t ) t ∈ [0 ,T ] = ( X T -t ) t ∈ [0 ,T ] denote the diffusion process and associated reverse diffusion process in T 4 , respectively. The stochastic differential equation (SDE) and reverse SDE [De Bortoli et al., 2022, Song et al., 2021] can be defined as follows:

Here, B M t represents the Brownian motion on T 4 , which is approximated with a Reodesic Random Walk (GRW) on T 4 (Algorithm 1). The score ∇ log p T -t ( Y t | Z ) in Equation 2 is estimated with a

score network s θ ( X , t, Z ) . We introduce the conditional vector Z in the score, parameterized with an encoder network that takes the structural context of mutation sites as inputs. We will delve into these components separately in the following subsections.

Geodesic Random Walk in T 4 When approximating the Brownian motion on the compact Riemannian manifolds using GRWs, a crucial step is defining the projection map and the exponential map that establishes the connection between the ambient Euclidean space and the associated tangent space, as discussed in previous studies [Jørgensen, 1975, De Bortoli et al., 2022]. In the context of T 4 , which is constructed as the quadruple product of S 1 within the ambient Euclidean space R 2 , it is adequate to focus on modeling Brownian motion on S 1 , defined as follows:

Here, B t represents the Brownian motion on the real line in the ambient Euclidean space R 2 , and K denotes the diffusion coefficient.

Similarly, the projection map Proj Y γ k within the tangent space T Y k ( T 4 ) and the exponential map Exp on T 4 can be derived by applying the Cartesian product of the maps designed for S 1 . To be more precise, these two maps are defined as follows:

Here, µ ∈ S 1 , v ∈ T µ ( S 1 ) , and z ∈ R 2 .

Algorithm 1 Geodesic Random Walk (GRW) during the sampling phase

Score network The architecture of the score network s θ ( X , t, Z ) is implemented as a three-layer Multilayer Perceptron (MLP) with 512 units in each layer. It takes as inputs the conditional vector from the encoder network representing the structural context of the mutations, the sampled rotamers, and the timestep t .

Conditional encoder network The encoder network (Figure S1) takes as inputs both single features, consisting of amino acid type, backbone dihedral angles, and local atom coordinates for each amino acid, as well as pair features such as pair distances and relative positions between two amino acids. An SE3-invariant IPA network [Jumper et al., 2021] is utilized to obtain the conditional vector Z for the structural context of the mutations. Further information regarding the encoder can be found in App. A.1.

Training objectives The parameters θ and φ in the score network and conditional encoder network are trained simultaneously. We adopt the implicit loss l im t ( s t ) in De Bortoli et al. [2022], defined as follows:

3.3 DiffAffinity: mutational effect predictor

For the mutational effect predictor, DiffAffinity, we imitate the structure of the mutated protein by setting the side-chain conformation as an empty set and altering the amino acids at the mutation

sites. Subsequently, we use the pre-trained conditional encoder Enc φ ∗ from SidechainDiff to encode wild-type and mutant structures into distinct hidden representations. These hidden representations, following additional IPA-like transformer network and max aggregation, are fed into a Multilayer Perceptron (MLP) of only one layer for the prediction of ∆∆ G . To assess the quality of our hidden representation, we also utilize the linear combination of SidechainDiff's hidden representations to predict ∆∆ G . More model details can be found in App. A.2.

3.4 Model training

We train SidechainDiff with the refined experimental protein structures on the database of Protein Data Bank-REDO [Joosten et al., 2014]. To fairly benchmark the performance of SidechainDiff, we follow the same protocol in RDE [Luo et al., 2023] to preprocess the data and split the train and test sets. We train DiffAffinity with the SKEMPI2 dataset [Jankauskait˙ e et al., 2019]. Due to limited samples, we perform a strategy of three-fold cross-validation to train the model and benchmark the performance. Specifically, we split the SKEMPI2 dataset into three folds based on structure, ensuring that each fold contains unique protein complexes not present in the other folds. Two of the folds are utilized for training and validation, whereas the remaining fold is reserved for testing purposes. More training settings are shown in App.A.

4 Results

First, we assess DiffAffinity's performance in predicting ∆∆ G using the SKEMPI2 dataset (Section 4.1) and the latest SARS-CoV-2 dataset (Section 4.2). Subsequently, we demonstrate DiffAffinity's effectiveness in optimizing antibodies, with a focus on SARS-CoV-2 as an illustrative case (Section 4.3). Additionally, we evaluate the prediction accuracy of SidechainDiff in side-chain packing.

4.1 Prediction of the mutational effects on binding affinity

DiffAffinity leverages the learned representation from SidechainDiff to predict mutational effects on protein-protein binding. Here, we benchmark the performance on the widely used SKEMPI2 dataset.

Baseline models As mentioned in section 2.2, the models for mutational effects prediction can be categorized into energy-based, sequence-based, unsupervised, end-to-end, and pre-trained models. We have selected several representative models from each category. The selected models include FoldX [Schymkowitz et al., 2005], Rosetta [Alford et al., 2017], flex ddG [Barlow et al., 2018], ESM-1v [Meier et al., 2021], ESM-IF [Hsu et al., 2022], ESM2[Lin et al., 2023], ESM2*, DDGPred [Shan et al., 2022], and RDE-Network [Luo et al., 2023]. More details of baselines are shown in App. A.4

For ablation studies, we have also developed two variants named DiffAffinity* and DiffAffinity-Linear. DiffAffinity* employs the same model architecture as DiffAffinity but without the learned representations from SidechainDiff and it serves as an end-to-end prediction method. And DiffAffinity-Linear is a direct linear projection of SidechainDiff's hidden representations to ∆∆ G .

Evaluation metrics The performance of our model is evaluated using a variety of metrics. These include traditional prediction accuracy metrics such as root mean squared error (RMSE) and mean absolute error (MAE). Given that the scalar values of experimental ∆∆ G are not standardized, we also employ Pearson and Spearman correlation coefficients to evaluate the models' predictive capability. To assess our model's ability to identify positive or negative mutations affecting proteinprotein binding, we use the AUROC and AUPRC metrics. Mutations are classified based on the sign of ∆∆ G , where positive values indicate positive impacts on binding and negative values indicate negative impacts. AUROC measures the overall discriminative power, while AUPRC accounts for imbalanced datasets.

Following the RDE-network ([Luo et al., 2023]), we also evaluate the prediction performance for each complex separately. We group mutations by complex structure, excluding groups with fewer than ten mutation data points, and derive two metrics: average per-structure Pearson correlation coefficient perstructure Pearson and average per-structure Spearman correlation coefficient per-structure Spearman.

Experimental results Our methods, DiffAffinity and DiffAffinity*, outperform all baseline models across almost all metrics and achieve state-of-the-art performance on the SKEMPI2 dataset (Table 1).

Particularly for multi-point mutations, DiffAffinity notably outperforms all other methods, suggesting that our model effectively captures the crucial relationship between mutational effects and side-chain conformations, regardless of the number of mutations. DiffAffinity surpasses the performance of the RDE-Network and ESM2*, indicating that the latent representations derived from our diffusion probabilistic model, SidechainDiff, are more beneficial than those from flow-based generative models and other self-supervised pre-training representations with the same network architecture for the same task. More analysis of the three pre-training representations can be found in App C.2.

Furthermore, DiffAffinity-Linear shows that even a straightforward linear projection of latent representations can achieve performance comparable to FoldX and ESM-IF and outperform ESM-1v. The reduced performance observed with DiffAffinity*, when the SidechainDiff encoder is absent, further emphasizes the effectiveness of the latent representations extracted from SidechainDiff.

In subsequent sections (Section 4.2 and 4.3), we employ the three models trained on different splits of the SKEMPI2 dataset without any fine-tuning on downstream tasks related to predicting binding affinity. This approach allows us to assess our model's performance in diverse tasks.

4.2 Prediction of mutational effects on binding affinity of SARS-CoV-2 RBD

SARS-CoV-2 initiates infection by binding to ACE2 protein on host cells via the viral spike protein. The receptor binding domain (RBD) of the spike proteins exhibits high-affinity binding to ACE2.

The technique of deep mutational scanning has been employed to experimentally quantify the effects of all single-point mutations on the binding affinity of the ancestral Wuhan-Hu-1 RBD (PDB ID: 6M0J) [Starr et al., 2022]. These measurements have guided the survey of SARS-CoV-2 evolution and identified several significant mutation sites on RBD that substantially contribute to the binding affinity. A total of 15 significant mutation sites like NE501 3 have been identified (see App. A.5) . We predicted all 285 possible single-point mutations for these sites and calculated the Pearson correlation coefficient between the experimental and predicted ∆∆ G .

Our results show that DiffAffinity outperforms all baseline methods (Table 2), highlighting its potential to facilitate biologists in understanding the evolution of SARS-CoV-2. Furthermore, we note that DiffAffinity exhibits a substantial improvement over DiffAffinity*, indicating the effectiveness of the learned representations from SidechainDiff.

4.3 Optimization of human antibodies against SARS-CoV-2

The receptor-binding domain (RBD) of the SARS-CoV-2 virus spike protein (PDB ID: 7FAE) plays a pivotal role in the binding process with the host's ACE2 protein. It serves as a prime target for neutralizing antibodies against SARS-CoV-2 [Shan et al., 2022]. In our experiment, we predict all 494 possible mutations at 26 sites within the CDR region of the antibody heavy chain. We expect that predicting mutational effects on binding affinity can facilitate in identifying top favorable mutations that can enhance the neutralization efficacy of antibodies.

These mutations are ranked in ascending order according to their ∆∆ G values, with the most favorable (lowest ∆∆ G ) mutations positioned at the top. To enhance neutralization efficacy, we aim to accurately identify and rank the five most favorable mutations. DiffAffinity successfully identifies four of these mutations within the top 20% of the ranking and two within the top 10% (Table 3). DiffAffinity consistently outperforms all the baseline methods, indicating its superior ability to predict the effects of mutations. This highlights DiffAffinity's potential as a robust tool for optimizing human antibodies.

4.4 Prediction of side-chain conformations

To assess the generative capacity of SidechainDiff straightforwardly, we employ it for predicting side-chain conformations of specific amino acids given their structural context. Our approach involves sampling rotamers from the distribution modeled by SidechainDiff and selecting the rotamer with the highest likelihood from a pool of 100 samples. For this task, we draw upon the test dataset from the PDB-REDO test split.

For a rigorous evaluation, we compare SidechainDiff with both energy-based methods, including Rosetta [Leman et al., 2020] and SCWRL4 [Krivov et al., 2009], as well as the deep learning-based methods, including RDE, AttnPacker [McPartlon and Xu, 2023], and DLPacker [Misiura et al., 2022]. All methods are run under the same settings to ensure a fair comparison. We evaluate the performance

using the Mean Absolute Error (MAE) and steric clash number of the predicted rotamers [McPartlon and Xu, 2023].

In terms of MAE for rotamer prediction, SidechainDiff outperforms energy-based methods and achieves comparable results to deep learning-based methods such as RDE and AttnPacker (Table 4 and Table S1). Notably, our method surpasses all the other methods in terms of the steric clash number. It's noteworthy that while methods like AttnPacker focus on predicting the torsion angles of static side-chain conformations, SidechainDiff emphasizes the distribution of side-chain conformations under various physical constraints. We measure the physical constraints of mutation sites using the contact number, i.e., the number of neighboring amino acids within an 8 ˚ A radius. Generally, a higher contact number indicates a more constrained structural context. We observe that for mutation sites with higher constraints, SidechainDiff tends to produce more accurate side-chain conformations (Table 5). We further illustrate cases showing the trade-off between accuracy and diversity achieved by SidechainDiff under various physical constraints (Appendix D and Figure S5a-S5d).

5 Conclusions

We present SidechainDiff, a Riemannian diffusion-based generative model for protein side-chains. It excels in both generating side-chain conformations and facilitating representation learning. Utilizing the learned representations, our method achieves state-of-the-art performance in predicting ∆∆ G across various test datasets, including the SKEMPI2 dataset and the latest SARS-CoV-2 dataset. Moreover, we demonstrate its effectiveness in antibody screening and its superior side-chain packing accuracy.

In future research, we can consider integrating the side-chain clash loss [Jumper et al., 2021] to refine our generative process. Additionally, we can explore building an end-to-end diffusion-based generative model for protein full-atom structures by integrating our model with existing backbone structure generation methods [Watson et al., 2023, Yim et al., 2023]. The main limitation of our model is the lack of consideration of backbone structure changes induced by mutations. Addressing the limitation could lead to enhanced performance.

References

Rebecca F Alford, Andrew Leaver-Fay, Jeliazko R Jeliazkov, et al. The Rosetta All-Atom Energy Function for Macromolecular Modeling and Design. Journal of Chemical Theory and Computation , 13(6):3031-3048, 2017.

Marloes Arts, Victor Garcia Satorras, Chin-Wei Huang, et al. Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics. Journal of Chemical Theory and Computation , 19(18):6151-6159, 2023.

Minkyung Baek, Frank DiMaio, Ivan Anishchenko, et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science , 373(6557):871-876, 2021.

Kyle A Barlow, Shane Ó Conchúir, Samuel Thompson, et al. Flex ddG: Rosetta Ensemble-Based Estimation of Changes in Protein-Protein Binding Affinity upon Mutation. The Journal of Physical Chemistry B , 122(21):5389-5399, 2018.

Nicole Calakos, Mark K Bennett, Karen E Peterson, et al. Protein-protein interactions contributing to the specificity of intracellular vesicular trafficking. Science , 263(5150):1146-1149, 1994.

Gabriele Corso, Hannes Stärk, Bowen Jing, et al. DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking. In International Conference on Learning Representations , 2023.

Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, et al. Riemannian Score-Based Generative Modelling. In Advances in Neural Information Processing Systems , volume 35, pages 2406-2422, 2022.

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems , volume 34, pages 8780-8794, 2021.

Jonathan Frazer, Pascal Notin, Mafalda Dias, et al. Disease variant prediction with deep generative models of evolutionary data. Nature , 599(7883):91-95, 2021.

Cunliang Geng, Anna Vangone, Gert E Folkers, et al. iSEE: Interface structure, evolution, and energy-based machine learning predictor of binding affinity changes upon mutations. Proteins: Structure, Function, and Bioinformatics , 87(2):110-119, 2019.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems , volume 33, pages 6840-6851, 2020.

Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, et al. Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions. In Advances in Neural Information Processing Systems , volume 34, pages 12454-12465, 2021.

Thomas A Hopf, John B Ingraham, Frank J Poelwijk, et al. Mutation effects predicted from sequence co-variation. Nature Biotechnology , 35(2):128-135, 2017.

Chloe Hsu, Robert Verkuil, Jason Liu, et al. Learning inverse folding from millions of predicted structures. In Proceedings of the 39th International Conference on Machine Learning , volume 162, pages 8946-8970, 2022.

John Ingraham, Max Baranov, Zak Costello, et al. Illuminating protein space with a programmable generative model. bioRxiv . doi:10.1101/2022.12.01.518682.

Justina Jankauskait˙ e, Brian Jiménez-García, Justas Dapk¯ unas, et al. Skempi 2.0: an updated benchmark of changes in protein-protein binding energy, kinetics and thermodynamics upon mutation. Bioinformatics , 35(3):462-469, 2019.

Bowen Jing, Gabriele Corso, Jeffrey Chang, et al. Torsional Diffusion for Molecular Conformer Generation. In Advances in Neural Information Processing Systems , volume 35, pages 2424024253, 2022.

Susan Jones and Janet M Thornton. Principles of protein-protein interactions. Proceedings of the National Academy of Sciences , 93(1):13-20, 1996.

Robbie P. Joosten, Fei Long, Garib N. Murshudov, et al. The PDB_REDO server for macromolecular structure model optimization. IUCrJ , 1(4):213-220, 2014.

Erik Jørgensen. The central limit problem for geodesic random walks. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete , 32(1-2):1-64, 1975.

John Jumper, Richard Evans, Alexander Pritzel, et al. Highly accurate protein structure prediction with AlphaFold. Nature , 596(7873):583-589, 2021.

Diederik Kingma, Tim Salimans, Ben Poole, et al. Variational Diffusion Models. In Advances in Neural Information Processing Systems , volume 34, pages 21696-21707, 2021.

Georgii Krivov, Maxim Shapovalov, and Roland Dunbrack. Improved prediction of protein side-chain conformations with SCWRL4. Proteins , 77:778-95, 12 2009.

Jun Lan, Jiwan Ge, Jinfang Yu, et al. Structure of the SARS-CoV-2 spike receptor-binding domain bound to the ACE2 receptor. Nature , 581(7807):215-220, 2020.

Julia Koehler Leman, Brian D. Weitzner, Steven M. Lewis, et al. Macromolecular modeling and design in Rosetta: recent methods and frameworks. Nature Methods , 17:665-680, 2020.

Anselm Levskaya, Orion D Weiner, Wendell A Lim, et al. Spatiotemporal control of cell signalling using a light-switchable protein interaction. Nature , 461(7266):997-1001, 2009.

Minghui Li, Franco L Simonetti, Alexander Goncearenco, et al. Mutabind estimates and interprets the effects of sequence variants on protein-protein interactions. Nucleic Acids Research , 44(W1): W494-W501, 2016.

Zeming Lin, Halil Akin, Roshan Rao, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science , 379(6637):1123-1130, 2023.

Xianggen Liu, Yunan Luo, Pengyong Li, et al. Deep geometric representations for modeling effects of mutations on protein-protein binding affinity. PLoS Computational Biology , 17(8):e1009284, 2021.

Shitong Luo, Yufeng Su, Zuofan Wu, et al. Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction. In International Conference on Learning Representations , 2023.

Derek M Mason, Simon Friedensohn, Cédric R Weber, et al. Optimization of therapeutic antibodies by predicting antigen specificity from antibody sequence via deep learning. Nature Biomedical Engineering , 5(6):600-612, 2021.

Matthew McPartlon and Jinbo Xu. An end-to-end deep learning method for protein side-chain packing and inverse folding. Proceedings of the National Academy of Sciences , 120(23):e2216438120, 2023.

Joshua Meier, Roshan Rao, Robert Verkuil, et al. Language models enable zero-shot prediction of the effects of mutations on protein function. In Advances in Neural Information Processing Systems , volume 34, pages 29287-29303, 2021.

Lidio MC Meireles, Alexander S Dï¿ 1 / 2 mling, and Carlos J Camacho. ANCHOR: a web server and database for analysis of protein-protein interaction binding pockets for drug discovery. Nucleic Acids Research , 38:W407-W411, 2010.

Mikita Misiura, Raghav Shroff, Ross Thyer, et al. DLPacker: Deep learning for prediction of amino acid side chain conformations in proteins. Proteins: Structure, Function, and Bioinformatics , 90 (6):1278-1290, 2022.

Alexander Quinn Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models. In Proceedings of the 38th International Conference on Machine Learning , volume 139, pages 8162-8171, 2021.

Pascal Notin, Mafalda Dias, Jonathan Frazer, et al. Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time Retrieval. In Proceedings of the 39th International Conference on Machine Learning , volume 162, pages 16990-17017, 2022.

George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, et al. Normalizing Flows for Probabilistic Modeling and Inference. The Journal of Machine Learning Research , 22(1):26172680, 2021.

Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, et al. Diffusion Autoencoders: Toward a Meaningful and Decodable Representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10619-10629, 2022.

Adam J Riesselman, John B Ingraham, and Debora S Marks. Deep generative models of genetic variation capture the effects of mutations. Nature Methods , 15(10):816-822, 2018.

Benno Schwikowski, Peter Uetz, and Stanley Fields. A network of protein-protein interactions in yeast. Nature Biotechnology , 18(12):1257-1261, 2000.

Joost Schymkowitz, Jesper Borg, Francois Stricher, et al. The FoldX web server: an online force field. Nucleic Acids Research , 33:W382-W388, 2005.

Sisi Shan, Shitong Luo, Ziqing Yang, et al. Deep learning guided optimization of human antibody against SARS-CoV-2 variants with broad neutralization. Proceedings of the National Academy of Sciences , 119(11):e2122954119, 2022.

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, et al. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning , volume 37, pages 2256-2265, 2015.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, et al. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations , 2021.

Tyler N. Starr, Allison J. Greaney, William W. Hannon, et al. Shifting mutational constraints in the SARS-CoV-2 receptor-binding domain during viral evolution. Science , 377(6604):420-424, 2022.

Thomas Steinbrecher, Robert Abel, Anthony Clark, et al. Free Energy Perturbation Calculations of the Thermodynamics of Protein Side-Chain Mutations. Journal of Molecular Biology , 429(7): 923-929, 2017.

Karyn T O'Neil and Ronald H Hoess. Phage display: protein engineering by directed evolution. Current Opinion in Structural Biology , 5(4):443-449, 1995.

Brian L. Trippe, Jason Yim, Doug Tischer, et al. Diffusion Probabilistic Modeling of Protein Backbones in 3D for the motif-scaffolding problem. In International Conference on Learning Representations , 2023.

Joseph L Watson, David Juergens, Nathaniel R Bennett, et al. De novo design of protein structure and function with RFdiffusion. Nature , 620(7976):1089-1100, 2023.

Fang Wu and Stan Z Li. DIFFMD: A Geometric Diffusion Model for Molecular Dynamics Simulations. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 5321-5329, 2023.

Gang Xu, Qinghua Wang, and Jianpeng Ma. OPUS-Rota4: a gradient-based protein side-chain modeling framework assisted by deep learning-based predictors. Briefings in Bioinformatics , 23 (1):bbab529, 2022.

Jason Yim, Brian L. Trippe, Valentin De Bortoli, et al. SE(3) Diffusion Model with Application to Protein Backbone Generation. In Proceedings of the 40th International Conference on Machine Learning , volume 39, page 1672, 2023.

Ning Zhang, Yuting Chen, Haoyu Lu, et al. MutaBind2: Predicting the Impacts of Single and Multiple Mutations on Protein-Protein Interactions. Iscience , 23(3):100939, 2020.

Zijian Zhang, Zhou Zhao, and Zhijie Lin. Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems , volume 35, pages 22117-22130, 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes SidechainDiff a diffusion model over the torsion angles of the protein sidechains and uses the learned representations of this model to predict the mutational effects of protein-protein binding.\n\nSTRENGTHS:\nThe paper shows a wide variety of promising experimental results ranging from standard benchmarks of binding affinity and side-chain conformation to case studies related to SARS-CoV-2 RBD and antibodies. I cannot comment on their reproducibility because the code was not provided.\n\nWEAKNESSES:\nWhile the experimental results are strong, however for the paper to be ready for publication, the manuscript should be improved in many regards: comparison to the literature, presentation of the method and certain baseline comparisons. \n\nComparison to the literature:\n\n1. The paper lacks references to existing published methods that have developed Riemannian diffusion models over the hyperdimentional torus to model molecular torsion angles (e.g. [1] and [2]). In these regards, the authors should clarify the relationship between the diffusion process presented here and in those works. To my understanding, the diffusion processes are identical which raises the question of why the authors refer to the perturbation kernel as intractable and resort to the use of implicit score matching. [1] suggests that the perturbation kernel can be sampled analytically (with a truncated infinite series). Are the perturbation kernels the same? If so how does the explicit score matching approach of [1] compare to the one developed?\n2. Related to the point before how are the samples X_t and the scores s_t obtained in equation (7)?\n3. Except from the diffusion component, the overall approach of the paper seems analogous to RDE [3]. I believe that given the similarity it would be useful to clarify the differences between the methods. For example line 256 claims the two methods use the same architecture. Is this true and if so does doing the pretraining with the flow of RDE-Network get worse performance than no pretraining at all (DiffAffinity*)?\n\nPresentation of the method:\n\n4. For the network architecture hardly any details are provided in the main text and even in the appendix these are not very clear. The authors should add some details of the architecture to the main text and clarify the presentation in the appendix: e.g. how is the score of the chi angles predicted? Is it an MLP on top of the hidden representation?\n\nBaseline comparisons:\n\n5. The authors should add details about the hyperparameter and model tuning process. E.g. were any hyperparameters tuned for the retrospective studies on SARS-Cov-2?\n6. Given the high-dimensional distribution with complex interdepencies between the angles of the different sidechains, looking simply at the MAE of individual torsion angles seems inadequate. This is especially relevant when some angles have reported MAE above 40 degrees, e.g. what would be the MAE performance of a simple baseline of predicting the median (on the circle) for every residue type and angle? More generally, I believe the authors should also provide further metrics, for example what is the steric clash rate in each of the approches?\n7. Some claims in the paper seem not well justified and should be adjusted. E.g. line 305 (and similar 322) “[SidechainDiff] achieves comparable performance to RDE, suggesting superior capability in generating side-chain conformations”, if they are comparable then the method is not superior.\n\nMinor:\n\n8. The text presents many orthographic errors (e.g. to name just a few at lines 161, 224, 549) I suggest the authors to use publicly available programs to review the text and correct these.\n\n[1] Jing, Bowen, et al. \"Torsional diffusion for molecular conformer generation.\" *NeurIPS 2022*.\n\n[2] Corso, Gabriele, et al. \"Diffdock: Diffusion steps, twists, and turns for molecular docking.\" ICLR 2023.\n\n[3] Luo, Shitong, et al. \"Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction.\" ICLR 2023.\n\nQUESTIONS:\nSee weaknesses section.\n\nLIMITATIONS:\nSee weaknesses section.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new representation learning-based approach called SidechainDiff that predicts how amino acid mutations influence protein-protein binding using Riemannian diffusion model. It's very important problem in the field of structural biology, for example, in evaluation of antibody variants impact to antigen binding. Moreover, SidechainDiff is the first approach that focuses on side chains compared to previous methods that work with protein backbones. Overall, this paper is a valuable contribution to both computational biology and artificial intelligence.\n\nSTRENGTHS:\nOriginality: The paper provides a new solution for a well-known task of the prediction of amino acid mutation impact into binding. Intoduced methods are new (SidechainDiff and DiffAffinity), though authors used an already known concept of diffusion models. Current work focuses on side chain generation while previous studies provides protein backbone generation. The related studies are adequately cited.\nQuality: The submission is technically sound, appropriate, and complete. It provides a clear and detailed explanation of the all steps of the work. In addition, the authors compare DiffAffinity with other models providing a baseline of its performance. The authors provide insights into the further steps of the work and potential applications. \nClarity: The submission is written clearly and organized. It provides all the necessary details for the reader. \nSignificance: Due to the lack of experimentally determined structures there is a need for tools that are able to predict the effect of amino acid mutations to the binding without this information. This paper provides such a tool.\n\nWEAKNESSES:\nThe predicted mutational effect was checked only on one type of antibody-antigen system (SARS-CoV-2 RBD). It's not clear if the model will show the same performance  in other antibody-antigen systems or how the performance will change. Also, it's not discuss how the protein characteristics (such as length, mutation position, etc.) can influence the performance and predictions.\n\nQUESTIONS:\n1. Will SidechainDiff be an open source approach? If so, it would be very useful to prepare a GitHub repository with the code and instruction of how to use the code.\n2. Can the results of SidechainDiff be biased towards SARS-CoV-2 RBD? Have you checked the results on other antibody-antigen systems? Will results remain the same?\n\nLIMITATIONS:\nThe limitations of SidechainDiff are not clearly written, please provide detailed discussion", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces DiffAffinity, a novel method for predicting the effects of mutations on protein-protein interactions. The authors leverage a diffusion model to learn representations that aid in predicting changes in binding affinity (∆∆G). They compare DiffAffinity's performance with existing methods such as FoldX and RDE-Net. The paper also explores the application of DiffAffinity in optimizing human antibodies against SARS-CoV-2.\n\nSTRENGTHS:\n1. The paper innovatively proposes a diffusion-based approach for predicting sidechain conformations at the protein-protein interface.\nThe authors provide a thorough comparison of DiffAffinity with existing methods, demonstrating its superior performance.\n2. The authors provide a thorough comparison of DiffAffinity with existing methods, demonstrating its superior performance.\n\nWEAKNESSES:\n1. The paper's main claim—that the hidden representation learned from the diffusion model can enhance binding affinity prediction—is not clearly elaborated.\n2. The proposed method does not account for changes in the backbone, which could limit its effectiveness in cases where mutations cause such changes.\n3. The paper could benefit from a discussion on the computational efficiency of DiffAffinity compared to other methods.\n\nQUESTIONS:\n1. Could the authors elucidate why the representation learned from the diffusion-based model benefits binding affinity prediction? More elaboration on this point would be beneficial.\n2. Is there a performance comparison between the model using diffusion-based representation and the model using other self-supervised representations (e.g., ESM2)? It would be insightful to see the model performance when replacing the diffusion-based representation with other representations.\n3. The score network receives timestep t as input. When computing the hidden representation for the downstream task of affinity prediction, what's the value of t, and why was it chosen?\n4. After obtaining the hidden representation from the diffusion-based encoder, an additional module predicts the binding affinity from the hidden representation. Is this additional module also an IPA-like transformer? The appendix seems to lack the specific model architecture of this module.\n5. In Table 1, the model appears to have achieved state-of-the-art performance without the representation learned from the diffusion model. Does this suggest that the performance gain primarily stems from the model architecture itself?\n6. Could the authors elaborate on the computational efficiency of DiffAffinity? How does it scale with the size of the protein or the number of mutations?\n\nLIMITATIONS:\nThe proposed method only considers side chain conformation changes and does not take backbone changes into account. This limitation might hinder its practical applications.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of mutational effect prediction on protein-protein binding. To address the data scarcity issue, the authors propose to first pre-train their model on PDB using diffusion models to fit the protein side-chain distribution. Then, the learned representations are used for predicting the mutational effects on SKEMPIv2. The method achieves good results on the benchmark in the author’s evaluation setting.\n\nSTRENGTHS:\n1. Mutational effects on protein-protein binding is an important and difficult problem in the protein community. It suffers from the data scarcity issue and pre-training seems like a good solution to the problem.\n2. The paper introduces diffusion-based models to fit the protein side-chain distribution, which, to the best of my knowledge, is among the first papers to explore this approach.\n\nWEAKNESSES:\n1. The paper's writing and clarity are poor, making it hard to follow. Many sections seem to be directly copied from previous works. The authors introduce many unnecessary notations and equations, which makes the method even more difficult to understand.\n2. The novelty of the paper is limited, as it primarily extends existing works without introducing problem-specific designs. The substitution of flow-based models with diffusion models for side-chain distribution modeling is not sufficiently innovative.\n3. The related work section lacks a discussion of existing methods on protein side-chain packing, which would provide a more comprehensive context for readers.\n4. The experiment design contains significant flaws that undermine the persuasiveness and support of the evaluation presented. Notably, many important baselines and details are missing from the experiment section, which weakens the arguments made by the authors. (See questions)\n\nQUESTIONS:\nMajor points:\n1. A significant portion of Section 3.2 appears to be directly copied from the Riemannian diffusion model paper [1], including Propositions 1 and 2. Please discuss why the two propositions are important to side-chain problems and mention in the main paper that **these contributions are derived from previous works**. Also, the section should be rewritten with appropriate paraphrasing to clarify the changes made to adapt the model for the new task.\n2. The authors propose to use a three-fold cross-validation for evaluation, but several details regarding the dataset split need to be addressed. (a) How is the dataset split? Is it split by proteins or mutations? Is it split based on sequence similarity or structure similarity? (b) The claim that DDGPred does not provide training scripts, and the authors copied the results from the published work is not a valid excuse. The training script of DDGPred is very easy to implement. Reproducing the results with the same dataset splits is essential to ensure fair comparison, as the performance can vary significantly with different splits due to limited data. Cross-validation should be repeated multiple times to assess the significance of the improvement.\n3. Important baselines are missing in the benchmark. First, more traditional methods should be included in comparison, e.g. FlexDDG [2]. These methods hardly suffer from the overfitting problem. Second, the performance of using pre-trained representations with a linear predictor is low and the improvement of DiffAffinity over DiffAffinity* is small. This raises the doubt on the effects of pre-training. Please include a baseline using other pre-trained representations, e.g., ESM2. Two baselines should be considered (a) ESM-2 with a 2-layer MLP predictor and (b) Feeding ESM-2 representations to the DiffAffinity architecture.\n4. The reported metrics are problematic. First, AUROC is more suitable for balanced binary classification tasks, and the number of positive and negative samples should be reported for binary classification tasks. If the data is imbalanced, AUPRC should also be reported. Second, treating mutations with ddG around 0 as either positive or negative is not appropriate, considering the measurement errors and the fact that most mutations in SKEMPIv2 are neutral. Additionally, discarding proteins with fewer than 10 mutations when reporting per-structure Pearson raises concerns about the sensitivity of the reported metric to this threshold. The authors should clarify the number of proteins and mutations left for evaluation and provide justifications for choosing 10 as the threshold.\n5. The evaluation in Sec. 4.3 is not convincing. Assigning high ranks to only five experimentally tested mutations is insufficient to draw conclusions about the effectiveness of the proposed method, especially considering the limited number of mutations tested experimentally in the original paper [3]. The authors should consider a more comprehensive evaluation by testing a larger set of possible mutations.\n6. In Sec. 4.4, the authors only compare their method with three baselines and ignore recent advancements in side-chain packing problem [4, 5]. Since modeling side-chain conformation is a major contribution of the paper, more experimental evaluation with a broader set of baselines should be included to demonstrate the method's effectiveness.\n\nMinor points:\n1. Line 121: Please give a definition of backbone atoms and side-chain atoms. This will be more friendly to audience without knowledge about proteins.\n2. Eq. (3) in Sec. 3.2: $U(X_t)$ is used without any definition.\n3. Line 148: Please discuss how the Brownian motion on $\\mathrm{T}^4$ is defined in the context of side-chain packing problem.\n4. Eq. (4) in Sec. 3.2: The SDE here is Variance-Exploding SDE, while the concepts introduced before are based on Variance-Preserving SDE. Please correct me if I am wrong.\n5. Line 165: Please discuss why the perturbation kernel $p_{t|0}$ is difficult to obtain.\n6. Line 171: “equation 6 -> Equation 6” Please make the references to equations consistent across the paper.\n\nOverall, I think the paper is not ready to publish in its current state. However, I firmly believe that the paper holds great potential for making a valuable contribution to the field if the authors dedicate more efforts towards improving the experimental design and writing. By addressing the weaknesses and incorporating the suggested changes, the paper can reach a higher standard of quality and become a valuable addition to the existing literature. With the necessary revisions, I am confident that the paper can be transformed into a strong and impactful publication.\n\n[1] De Bortoli, Valentin, et al. \"Riemannian score-based generative modelling.\" Advances in Neural Information Processing Systems 35 (2022): 2406-2422.\n\n[2] Barlow, Kyle A., et al. \"Flex ddG: Rosetta ensemble-based estimation of changes in protein–protein binding affinity upon mutation.\" The Journal of Physical Chemistry B 122.21 (2018): 5389-5399.\n\n[3] Shan, Sisi, et al. \"Deep learning guided optimization of human antibody against SARS-CoV-2 variants with broad neutralization.\" Proceedings of the National Academy of Sciences 119.11 (2022): e2122954119.\n\n[4] Misiura, Mikita, et al. \"DLPacker: deep learning for prediction of amino acid side chain conformations in proteins.\" Proteins: Structure, Function, and Bioinformatics 90.6 (2022): 1278-1290.\n\n[5] McPartlon, Matthew, and Jinbo Xu. \"An end-to-end deep learning method for protein side-chain packing and inverse folding.\" Proceedings of the National Academy of Sciences 120.23 (2023): e2216438120.\n\nLIMITATIONS:\nThe authors should add a separate paragraph to dicuss the potential limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-25"}