Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures
Authors: Shitong Luo
Yufeng Su
Xingang Peng
Sheng Wang
Jian Peng
Jianzhu Ma
Venue: NeurIPS 2022 Accept (2022)
Keywords: antibody design
diffusion probabilistic model
protein structure

Abstract: Antibodies are immune system proteins that protect the host by binding to specific antigens such as viruses and bacteria. The binding between antibodies and antigens is mainly determined by the complementarity-determining regions (CDR) of the antibodies. In this work, we develop a deep generative model that jointly models sequences and structures of CDRs based on diffusion probabilistic models and equivariant neural networks. Our method is the first deep learning-based method that generates antibodies explicitly targeting specific antigen structures and is one of the earliest diffusion probabilistic models for protein structures. The model is a "Swiss Army Knife" capable of sequence-structure co-design, sequence design for given backbone structures, and antibody optimization. We conduct extensive experiments to evaluate the quality of both sequences and structures of designed antibodies. We find that our model could yield competitive results in binding affinity measured by biophysical energy functions and other protein design metrics.

Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures
Shitong Luo"*, Yufeng Su Xingang Peng", Sheng Wang', Jian Peng".?, Jianzhu Ma
Helixon Research
University of Illinois Urbana- Chhama-Champaign
3 School of Intelligence Science and Technology, Peking University
Paul G. Allen School of Computer Science, University of Washington Institute for AI Industry Research, Tsinghua University luost@hell ixon. com, 11l.st260mmiil........ com
swangdes. washington edu, aaaaanggeillnnnn wwwpnng@llllnnioo edu, wwwajiinnuutsinggee edu,
Abstract
Antibodies are immune system proteins that protect the host by binding to specific antigens such as viruses and bacteria. The binding between antibodies and antigens is mainly determined by the complementarity -ettatty-deeermningg regions (CDR) of the antibodies. this work, we develop a deep generative model that jointly models sequences and structures of CDRs based on diffusion probabilistic models and equivariant neural networks. Our method the first deep learning- based method that generates antibodies explicitly targeting specific antigen structures and is of the earliest diffusion probabilistic models for protein structures. The model is "Swiss Army Knife" capable of sequence-structure co-design, sequence design for given backbone structures, and antibody optimization. We conduct extensive experiments to evaluate the quality of both sequences and structures of designed antibodies. We find that our model could yield competitive results in binding affinity measured by biophysical energy functions and other protein design metrics.
Introduction
Antibodies are important immune proteins generated during an immune response to recognize and neutralize the pathogen [Janeway et al., 2001]. As illustrated in Figure la, an antibody contains two heavy chains and two light chains, and their overall structure is similar. Six variable regions determine the specificity of an antibody to the antigens. They are called the Complementarity Determining Regions (CDRs), denoted as H1, H2, H3, L1, L2, and L.3. Therefore, the most important step for developing effective therapeutic antibodies is to design CDRs that bind to the specific antigen [Presta, 1992, Akbar et al., 2022a].
Similar to other protein design tasks, the search space of CDRs is vast. A CDR sequence with L amino acids has up to 20" possible protein sequences. It not feasible to test all the possible sequences using experimental approaches, so computational methods are needed. Traditional computational approaches rely on sampling protein sequences and structures from complex biophysical energy functions [Pantazes and Maranas, 2010, Lapidoth et al., 2015, Adolf-Bryfogle et al., 2018, Warszawski et al., 2019]. They are generally time-consuming and are prone to get trapped in local optima. Recently, various deep generative models have been developed to design antibodies [Saka et al., 2021, Akbar et al., 2022b, Jin et al., 2022]. Compared to conventional algorithms, deep generative models
Egqual contribution.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

(a)
Antigen
(b)
Same Co. Distance Between Amino Acids
Antigen
Heavy Chain
Interacting
Light Chain
CDRH
COR-L
Interaction
CDR
Non Non-interacting
CDR
Antibody
Figure 1: (a) intboddy-nttigen complex structure and CDR structure. (b) The orientations of amino acids (represented by triangles) determine their side-chain orientations, which are key to inter-amino- acid interactions.
could directly capture higher-order interactions among amino acids on antibodies and antigens and generate antibodies more efficiently [Akbar et al., 2022a]. Recently, Jin et al. proposed generative snneture-sequence co-design. Their model addresses two important computational challenges: First is how to model the intrinsic relation between CDR sequences and 3D structures, and second is how to model the distribution of CDRs conditional the rest of the antibody sequence. However, there is still a large gap to fill before generative models become practical for antibody design.
model for antibody
Here, we identify another three challenges for antibody sequence-strute structure co-design. First, the model should be explicitly conditional on the 3D struciures of the antigen and generate CDRs that fit the antigen structure in the 3D space. This is indispensable for the model to generalize to new antigens. Second, the interactions between amino acids are mainly determined by side-chains which are groups of atoms stretching out from the protein backbone (Figure 1b) [Liljas et al., 2016]. Therefore, the model should be able to consider both the position and orientation of amino acids. Third, in drug discovery, pharmacologists collect multiple initial antibodies either from humanized mice or patients [Presta, 1992, Barlow et al., 2018, Warszawski et al., 2019]. Therefore, instead of de novo design, the model should be applicable to another realistic scenario: optimizing a particular antibody to increase the binding affinity the antigen. To the best of our knowledge, no previous machine learning model satisfies all of the above design principles.
To address these challenges, we propose a diffusiin-basee generative model [Sohl-Dickstein al., 2015, Song and Ermon, 2019, Ho et al., 2020, Yang et al., 2022] capable of jointly sampling antibody CDR sequences and struc- tures. More importantly, the joint distribution of CDR sequence and its structure is directly conditional on anti- gen structures. Given a protein complex consisting of antigen and an antibody framework as input' (as illustrated in Figure 2), first initialize the CDR with arbitrary sequence, positions, and orientations. The diffusion model first aggregates information from the antigen and the tibody framework. Then, it iteratively updates the amino acid type, position, and orientation of each amino acid on CDRs. In the last step, we reconstruct the CDR structure at the atom level using side-chain packing algorithms based on the predicted orientations [Alford al., 2017]. From the perspective of model capability, one of the most important reasons for us to choose the diffusion -based model over other generative models such as generative adversarial networks [Goodfellow et al., 2014] and variational auto-encoders [Kingma and Welling, 2013] is that it generates CDR candidates iteratively in the interfere and impose constraints on the sampling process support a broader range of design tasks.
Antigen
Structure
Generative Model
CDR Designs
BOROUSAPRI AASAASAAAAAAAA
Antibody Framework
BOLO YOAYL MENYAYG
Figure 2: The task in this work is to de- sign CDRs for given antigen structure and an antibody framework.
space so that we can
We summarize our contributions as follows:
We propose the first deep learning models to perform antibody sequence-structuue design by considering the 3D structures of the antigen.
In our model, we not only design protein sequences and coordinates but also side-chain orientations (represented as S3) element) of each amino acid. It is the first deep learning
'The structure of the anttibddy framework can be obtained either from existing untigen-antibody structure by docking initial antibody to the target antigen.

---

model that could achieve atomic-resolution antibody design and is equivariant to rotation and translation.
We show that our model can be applied to wide range of antibody design tasks, including sequence-struuturr co-design, fix-backbone CDR design, and antibody optimization.
2
Related Work
Computational Antibody Design Conventional computational approaches are mainly based on sampling algorithms over hand-crafted and statistical energy functions and iteratively modify protein sequences and structures AAdolf-Bryfogll
al., 2018, Lapidoth al., 2015, Warszawski et al., 2019, Pantazes and Maranas, 2010, Ruffolo et al., 2021]. 1.. These methods are inefficient and prone to getting stuck local optima due to the rough energy landscape. In recent years, deep learning methods have shown potential in antibody design by using language models to generate protein sequences [Alley et al., 2019, Shin et al., 2021, Saka et al., 2021, Akbar al., 2022b]. Although much more efficient, the sequence- saaee methods can only generate new antibodies based on previously observed antibodies but can hardly generate antibodies for specific antigen structures.
Jin et al. proposed the first CDR sequence-structuur co-design deep generative model which focuses on designing antibodies to neutralize SARS-CoV-2. relies on an additional antigen-specific predictor to predict the neutralization of the designed antibodies, which is not generalizable to arbitrary antigens. In comparison to their model, we explicitly model the 3D structure of an antigen, opening the door to generalizing the prediction to unseen antigens with solved 3D structures. Another advantage of our model is that we consider not only backbone atom coordinates but also the orientation of amino acids. The orientation is critical to protein-protedd protein interactions as most of the atoms interacting between antibodies and antigens are in the side-chain [Liljas al., 2016] (as illustrated in Figure lb). Lastly, the model proposed by Jin et al. is not equivariant by construction, which is fundamental in molecular modeling.
Protein Structure Prediction Protein structure prediction algorithms take protein sequences and Multiple Sequence Alignments (MSAs) as input and translate them to 3D structures [Jumper et al., 2021, Back et al., 2021, Yang et al., 2020]. Accurate protein structure prediction models predict not only the position of amino acids but also their orientation [Jumper et al., 2021, Yang al., 2020]. The orientation of amino acids determines the direction in which its side chain stretches, so is indispensable for reconstructing full-atom structures. AlphaFold2 [Jumper et al., 2021] predicts per-amino-aue orientations in an iterative fashion, similar to our proposed model. However, it is not generative, unable to efficiently sample diverse structures for protein design. Recently, based on prior protein structure prediction algorithms, methods for predicting antibody CDR structures have emerged [Ruffolo al., 2022b,a], but they are not able to design CDR sequences.
Diffusion-Based Generative Models Diffusion probabilistic models learn to generate data via denoising samples from a prior distribution [Sohl-Dickstein et al., 2015, Song and Ermon, 2019, Ho et al., 2020]. Recently, progress has been made in developing equivariant diffusion models for molecular 3D structures [Shi et al., 2021, Hoogeboom et al., 2022, Jing et al., 2022, Xu et al., 2022]. Atoms in molecule do have natural orientations, so the generation process differs from generating protein structures. Diffusion models have also been extended to non-Euclidean data, such as data in the Riemannian manifolds [Leach et al., 2022, De Bortoli et al., 2022]. These models are relevant to modeling orientations which are represented by elements in SO(3). In addition, diffusion models can also be used generate discrete categorical data [Hoogeboom et al., 2021, Austin et al., 2021]. Concurrently with this work, various diffusion probabilistic models have been developed for proteins [Anand and Achim, 2022, Trippe al., 2022, Wu al., 2022].
3 Methods
This section is organized as follows: Section 3.1 introduces notations used throughout the paper and formally states the problem. Section 3.2 formulates the diffusion process for modeling antibodies. Section 3.3 introduces details about the neural network parameterization for the diffusion processes. Section 3.4 presents sampling algorithms for various antibody design tasks.
3

---

3.1 Definitions and Notations
An amino acid in a protein complex can be represented by its type, C. atom coordinate, and the orientation, denoted as E MACDEFGHIKLMIQQRSTWYY 2, E R O2 E SO(3), respectively. Here i 1.....A N, and N is the number of amino acids the protein complex'.
In this work, we assume the antigen structure and the antibody framework is given (Figure 2), and we focus designing CDRs on the antibody framework. Assume the CDR to be generated has amino acids with index from 1+1 to (+mm They are denoted as R = {(}, x,, O₃) j = I + I...., + m Formally, our goal jointly model the distribution R given the structure of the antibody-antigen complex C = {(5,, Fi O,) i (I! I,..., m}}.
3.2 Diffusion Processes
diffusion probabilistic model defines two Markov chains of diffusion processes. The forward diffusion process gradually adds noise to the data until the data distribution approximately reaches the prior distribution. The generative diffusion process starts from the prior distribution and iteratively transforms it to the desired distribution. Training the model relies on the forward diffusion process to simulate the noisy data. Let (s).. x Of) denote the intermediate state of amino acid j time step tt
R' = {s}. x5. O [+m D)) represents the sequence and structure sampled at step t. t = 0 represents the state of real data (observed sequences and structures of CDRs) and t T represents samples from the prior distribution. Forward diffusion goes from 0 to T, and generative diffusion proceeds in the opposite way. The diffusion processes for amino acid types sy. coordinates x5. and orientations O are defined as follows:
Multinomial Diffusion for Amino Acid Types The forward diffusion process for amino acid types is based on the multinomial distribution defined as follows [Hoogeboom et al., 2021]:
al
Multinomial
(1- (ppee) (mehot(s/-1)
Plype
20 1
(1)
where onehot represents a function that converts amino acid type to 20-dimensional one-hot vector and 1 is an all-one vector. Syye the probability of resampling another amino acid over 20 types uniformly. When + T, A is set close to and the distribution is closer to the uniform distribution. The following probability density provides an efficient way to perturb sy for timestep during training [Hoogeboom al., 2021]:
1
{()) Multinomial aiype onehot(s)) (1 aispe)
.1)
(2)
20
where ayyp = Dtype)
The generative diffusion process is defined as:
P C)
Multinomial (F(R',C)Li))
where F(-)li) is a neural network model taking the structure context (antigen and antibody framework) and the CDR state from the previous step as input and predicts the probability the amino acid type for the j-th amino acid on the CDR. Note that, different from the forward diffusion process, the generative diffusion process must rely on the structure context C and the CDR state the previous step including positions and orientations. The main difference between these two processes is that the forward diffusion process adds noise to data so it is irrelevant to data or contexts but the generative diffusion process depends the given condition and full observation of the previous step. The generative diffusion process needs to approximate the posterior ((sf-"1s; / sg) derived from Eq. I and Eq.2 to denoise. Therefore, the objective of training the generative diffusion process for amino acid types is to minimize the expected KL divergence between Eq.3 and the posterior distribution:
Lype ERinp Σ, DKL. (p ::)
NNote that protein complex contains more than one chain, so N is not the length of one protein but is the sum of the lengths of all chains the complex.

---

Prior
R'
Sample From ((=)
.. (R(,C)
CDR
Typex
Amina Acid Types
Neural Network Parameterization
Arnina Acid Types
Sample for (1)) Steps
Sample Steps
Ca Positions
Positions
Sidechain Packing
uHj
Orientations
Orientations
O
0
Figure Illustration of the generative diffusion process. At each step, the network takes the current CDR state input and parameterizes the distribution of the CDR's 's sequences, positions, and orientations for the next step. In the end, full-atom structures are constructed by the side-chain packing algorithm.
Diffusion for C. Coordinates the coordinate an atom could be an arbitrary value, we scale and shift the coordinates of the whole structure such that the distribution of atom coordinates roughly match the standard normal distribution. We define the forward diffusion for the normalized C. coordinate x, as follows:
x = N (x) (x)
BP x :puss
(5)
(x) x9) = N (x) a0
x). (1
apps)
where f controls the rate diffusion and its value increases from to 1 as time step goes from O to and ã II= (1 Bg Using the reparameterization trick proposed by Ho al., the generative diffusion process is defined as:
p (xx R (x5, =N (xx (x M4 (R :) :poss))
8pss
Mp (R',C)
x
G(R', (RR'
(8)
apps
pos
Here, G(-)[ is neural network that predicts the standard Gaussian noise c, (N, added to
60 poc xf (scaled coordinate of amino acid j) based the reparameterization of Eq.6: x)
/ E. The objective function training the generative process is the expected
pos
MSE between G and cj. which is simplified from aligning distribution to the posterior q(x), x5. x [Ho et al., 2020]:
pos E
(9)
SO(3) Denoising for Amino Acid Orientations We empirically formulate an iterative perturb- denoise scheme for learning and generating amino acid orientations represented by SO(3) elements [Leach et al., 2022]. Note that we do not use the term diffusion because the formulation does not strictly follow the framework of diffusion probabilistic models though the overall principle is the same. Similar to the typical diffusion process, the distribution orientations perturbed steps defined as, according to Leach et al. [2022]:
(0) ScaleRot (Vaten 0.) ,1)),1100), arec
a (O', 10) =IGso
(10)
IGso(3) denotes the isotropic Gaussian distribution on SO(3) parameterized by mean rotation and a scalar variance [Leach al., 2022, Matthies et al., 1970, Nikolayev and Savyolov, 1970]. ScaleRot modifies the rotation matrix by scaling its rotation angle with the rotation axis fixed [Gallier and Xu,
5

---

denote the rotated and translated structure. Note that F, G. and H are not single ML.Ps. Each of 2003]. Cai = II (1-5))) where Pot is the variance increases with the step t. The conditional distribution used for the generation process of orientations defined as:
(0 R², C)) (Gso(3)
(0)- H(R', C)D,,,
(11)
where [[] is neural network that denoises the orientation and outputs the denoised orientation matrix of amino acid j. Training the conditional distribution requires aligning the predicted orientation from (-) the real orientation. Hence, we formulate the training object that minimizes the expected
discrepancy measured by the inner product between the real and the predicted orientation matrices:
1 m Σ (0))
=
I
(12)
F
where 0 = =)] is the predicted orientation for amino acid j.
The Overall Training Objective By summing Eq.4, 9, and 12 and taking the expectation w.r.t. 4 obtain the final training objective function:
L = Uniifeeee().....) [Lype + L + Lan]
(13)
To train the model, we first sample a time step and then sample noisy states {s/, xf. Of 1+
j=1+1 p
by adding noise the training sample using the diffusion process defined by Eq.2, 6, and 10. We compute the loss using the noisy data and backpropagate the loss to update model parameters.
3.3 Parameterization with Neural Networks
In this section, we briefly introduce the neural network architectures used in different components of the diffusion process. The purpose of the networks to encode the CDR state at a time step f along and then denoises with the context structure: {s}, x), Of 1- U {s (1..,(1...) (....N))((.......... the CDR amino acid types (F), positions (G), and orientations (H).
First, we adopt Multiple Layer Perceptrons (MLPs) to generate embeddings for single and pairs of amino acids. The single amino-acid embedding MLP creates vector e; for amino acid which encodes the information of amino acid types, torsional angles, and 3D coordinates of all the heavy atoms. The pairwise embedding MLP encodes the Euclidean distances and dihedral angles between amino acid and j to feature vectors Zij. We adopt IPA [Jumper et al., 2021], an orientation-aware roto-translation invariant network transform e, and Zij into hidden representations ha, which aims to represent the amino acid itself and its environment. Next, the representations are fed to three different MLPs to denoise the amino acid types, 3D positions, and orientations of the CDR, respectively.
In particular, the MLP for denoising amino acid types outputs 20-dimensional vector representing the posterior probabilities. The MLP for denoising C. coordinates predicts the scaled change of the coordinate in terms of the current orientation of the amino acid. As the coordinate deviation is calculated in the local frame, we left-multiply it by the orientation matrix and transform it back to the global frame. Formally, this can be expressed as éj O MLP (hz). Predicting coordinate deviations in the local frame and projecting it to the global frame ensures the equivariance of the prediction, as when the entire 3D structure rotates by particular angle, the coordinate deviations also rotate by the same angle. The MLP for denoising orientations first predicts a so(3) vector [Gallier and Xu, 2003]. The vector is converted to a rotation matrix M, E SO(3) right-multiplied to the orientation to produce a new mean orientation for the next generative step: 0--1 + M2M The proposed networks are equivariant to the rotation and translation of the overall structure:
Proposition 1. For any proper rotation matrix R E SO(3) and any 3D vector r E R (rigid transformation (R, r) E SE(3)), F, G and H satisfy the following equivariance properties:
F(RR* + r, RC + r) = (((R,.C.
(14)
G(RR' + +r, RC + +r) = RG(RR,C) ,C),
(15)
H(RR( + r. RC + rr = -RH(R'. ,C),
(16)
where RR'+T := Xt,, RO 111=1+1 and RC+r := {ss, x2 + r, RO N
them includes the shared encoder and specific MLP.

---

3.4 Sampling Algorithms
The sampling algorithm first samples amino acid types from the uniform distribution over 20 classes: sf Unif Ca positions from the standard normal distribution: x ~ (O, Is), and orientations from the uniform distribution over SO(3): o Uniform(SO normalize the coordinates of the structure in the same way as training such that Ca positions in the CDR roughly follow the standard normal distribution. Next, we iteratively sample sequences and structures from the generative diffusion kernel by denoising amino acid types, C. coordinates, and orientations until
Note that we
To build a full atom 3D structure, we construct the coordinates of N, C., C, O, and side-chain C (except glycine that does not have C₃)) according to their ideal local coordinates relative to the C/ position and orientation of each amino acid [Engh and Huber, 2012]. Based on the five reconstructed atoms, the rest of the side-chain atoms constructed using the side-chain packing function implemented in Rosetta [Alford et al., 2017]. In the end, we adopt the AMBER99 force field [Lindorff-Larser et al., 2010] in OpenMM [Eastman et al., 2017] to refine the full atom structure.
In addition to the joint design of sequences and structures, we can constrain partial states for other design tasks. For example, by fixing the backbone structure (positions and orientations) and sampling only sequences, we can fix-backbone sequence design. Another usage to optimize an existing antibody. Specifically, we first add noise to the existing antibody for / steps and denoise the perturbed antibody sequence starting from the /-th step of the generative diffusion process.
4 Experiments
We present the application of our model, named DifTAb', in three antibody design tasks: sequence- structure co-design (Section 4)1,, antibody sequence design based on antibody backbones (Section 4.2), and antibody optimization (Section 4.3). In Section 4.4, we show how to use our model without known antibody frameworks bound to the antigen.
4.1 Sequence-Structure Co-design
The dataset for training the model is derived from the SAbDab database[Dunber et al., 2014]. We first remove structures whose resolution is worse than 4A and discard antibodies targeting non non-protein antigens. We cluster antibodies in the database according CDR-H3 sequences at 50% sequence identity. We manually select five clusters as the test set, containing 19 antibody- antigen complexes in total. The test set includes antigens from several well-known pathogens including SARS-CoV-2, MERS, influenza, and so on. Structures in the remaining clusters are used for training.
To evaluate the performance, we remove the original CDR from the antibody-antiget complex in the test set and sample both the sequence and structure of the removed region. We set the length of the CDR to be identical to the length of the original CDR for simplicity. In practice, one can enumerate different lengths of CDRs. We compare our model to GosettaaAntioddyesspp (RAbD) [Adolf-Bryfogle et al., 20 101 antibody design software based on Rosetta energy functions. For each model, we draw 100 samples for each CDR. Both the original structures and designed structures from different methods are refined by OpenMM and Rosetta.
We use the following metrics evaluate designed antibodies: (1) IMP:
the percentage of designed CDRs with lower (better) binding energy (AG) than the original CDR. The binding energy is calculated by InterfaceAnalyzerr
the Rosetta software package [Alford et al., 2017]. (2) RMSD: is the C. tal--mman-qqurrr deviation (RMSD) between the generated structure and the original structure with only antibody frameworks aligned. (3) AAR: the amino acid recovery rate measured by the sequence identity between the reference CDR sequences and the generated sequences [Adolf- Bryfogle et al., 2018]. Note that different from Jin et al. [2022], we do not use neutralization prediction models because they are sequence-based and are specified to a limited class of antigens, which deviates from our goal of developing a general antibody design model.
Table shows that our model (DiffAb) recovers CDR sequences more accurately than RAbD (higher AAR). The RMSDs of CDRs generated by DiffAb are higher in CDR-H3, which indicates that our generated samples are more diverse structurally. The IMP score of DiffAb is on par with RAbD in
*Code and data available https: com/Iuosttt////ffffa.
7

---

Table Evaluation of the generated antibody CDRs (sequence co-design) by RAbD and our DiffAb model.
CDR
Method
AAR
RMSD
IMP
CDR
Method
AAR
RMSD
IMP
RAbD DilTAb
22.85%
2.261
43.88%
RAbD DifTAb
34.27%
1.204A
46,81%
LI
65.75%
1.188Ä
53.63%
56.67%
1.388A
45.58%
RAbD
25.50%
1.641À
53.50%
RAbD 26.30%
1.767A
56.94%
H2
L2
DiffAb
49.31%
1.076A
29.84%
DiffAb
59.32%
1.373A
49.95%
RAbD
14%
2.900Å
23.25%
RAbD
20.73% 46.47%
1.624A
55.63%
H3
L3
DifTAb
26.78%
3.597A
23.63%
DiflAb
1.627À
47.32%
Referance
Sample 1
Sample 2
Sample
-20
2 RMSD
AO - 45.45 RVSID 4.26A DISVESSAY
DO 12.7 AMSO SJRA AITWWADROPPWEED
40 8.00 RIASI + A.A
45 - TEURS TRGIGIYE VBRANISKDE
LIVERYEN LINEEEENTSAPPPN TWYYPMOY
Figure Examples of CDR-H3 designed by the sequence-structun co-design method and the distribution of their interaction energy and RMSD. The aatigen-antibodyy template is derived from PDB:7chf, where the antigen SARS-CoV-2 RBD. Sample has better complementarity to the antigen while Sample 3 fits the antigen worse. This could explain their difference in the binding energy (AG).
CDR-H3, and lower in other CDRs. However, it should be noted that RAbD optimizes the Rosetta energy function, which is also used for evaluation. Our model achieves reasonably good binding energy without explicit supervision signal from Rosetta energy functions. Figure presents three generated examples of CDR-H3 targeting SARS-CoV-2 RBD. Sample has the lowest binding energy and it can be observed that it has better complementarity to the antigen. The binding energy of Sample higher than the original one and visually, the shape of the CDR does not the antigen well.
4.2 Fix-Backbone Sequence Design and Structure Prediction
In this setting, the backbone structure CDRs is given and we only need to design the CDR sequence, which transforms the task into constrained sampling problem. Fix-backbone design is common setting in the area of protein design [Ingraham et al., 2019, Hsu et al., 2022, Anishchenko et al., 2021, Strokach et al., 2020, Tischer et al., 2020]. For this task, consider FixBB, cosetta-based sequence design software given CDR backbone structure, as the baseline. We use the AAR metric introduced in Section 4.1 to evaluate the designed CDRs.
As shown in Table 2, our model achieves better AAR all the CDRs. This shows that our model is also powerful in modeling the conditional probability of sequences given backbone structures. Admittedly, the training data is clustered only by CDR-H3 sequences, so the model might have seen other CDRs in the test set during training, leading to even higher AAR. However, we believe this is not an issue as CDRs other than H3 are generally conserved and contribute less to the specificity Xu and Davis [2000].
Our model can predict CDR structures by fixing the sequence. Table 3 shows that it accurately predicts the structure of CDR H1, H2, L1, L2, and L3 (RMSD < 1. The accuracy of CDR-H3 prediction is lower due to the high variability. Figure 5a separately shows the accuracy of different CDR-H3 lengths. The prediction is generally more accurate for shorter ones. When the CDR-H3 contains more than 10 amino acids, the prediction accuracy drops.

---

Table 2: Comparison of FixBB and DiffAb terms of amino acid recovery (AAR) in the fix backbone CDR design task. DiffAb achieves higher AAR. The AAR of DiffAb on CDR-H3 lower than other CDRs since H3 is much more versatile.
Table 3: The accuracy of CDR structures predicted by DiffAb in RMSD.
CDR
Method
AAR
CDR
Method
AAR
CDR
RMSD
FixBB DifTAb
37.14%
FixBB DifTAb
33.80%
HI 0.901À
H1
L1
87.83%
86.63%
H2 1.0444
FixBB 43.08%
FixBB 28.54%
H3 3,246A
H2
L2
DiffAb
79.70%
DiffAb
88.91%
LI
1.365A
FixBB DiffAb
30.74%
FixBB DiffAb
17.92%
L2
1.321À
H3
59.48%
L3
78.69%
L3
1.492À
(a)
(b)
Forward Diffusion
SuqiD/N) RASSSA (MP(N)
steps
CDR
26
Optimized
0.0
steps
01
11
16
Backward Diffusion
20
40
60
Length
Number Stepo
Figure 5: (a) RMSD of predicted CDR-H3 structures grouped by lengths. (b) The antibody optimiza- tion algorithm first perturbs the initial CDR for steps using the forward diffusion process and then denoises it by the backward diffusion process into the optimized CDR. (c) IMP, RMSD, and SeqlI) of the CDRs optimized with different numbers of steps. Dashed lines represent the results of de novo design. When 4, the optimized CDRs reach an IMP score close to de novo CDRs but remain structurally similar to the original one.
4.3 Antibody Optimization
We use our model to optimize existing antibodies which is another common pharmaceutical applica- tion. To optimize an antibody, we first perturb the CDR sequence and structure for t steps using the forward diffusion process. Then, we denoise the se- quences starting from the (T - t)-th step (t steps remaining) of the generative diffusion process and obtain set of optimized antibodies. This process illustrated in Figure 5b. We optimize CDR-HI of the antibodies in the test with various val- ues. For each antibody and I, we perturb the CDR independently 100 times and collect 100 optimized CDRs different from the original CDR. We report the percentage of optimized antibodies with improved binding energy (IMP). RMSD, and sequence identity (SeqID) of the optimized CDR in comparison the original antibody. We also compare the optimized antibodies with the de novo T 100) designed antibodies introduced in Section 4.1. As shown in Table and Figure 5c, the optimization method could produce antibodies with improved binding energy measured by the Rosetta energy function. In contrast to redesigning CDRs, optimization improves binding energy while keeping the optimized CDR similar to the original one, which is desired in many practical applications.
Table 4: Evaluation of optimized CDR-H3s with different numbers of optimization steps. In contrast to redesigning the CDR, the opti- mization method can improve binding energy while keeping the optimized CDR similar to the original one. Figure 5c shows the line plot of the results.
t
IMP
RMSD
SeqID
18.52%
1.194Ä
92.42%
16.67%
1.252Á
91.61%
4
23.29%
1.290Å
91.16%
8
22.01% %
1.447Ä
88.78%
16
18.02%
1.759À
78.43%
32
16.43%
2.623À
40.58%
64
15.47%
3.380A
27.30%
23.63%
3.597
26.78%

---

Sumple
<0
-00
-20
20
40
Template (POB: BQHF)
(POB: 7WVN)
Docking
Sumple
Sumple:3
Sample 5
KINGDAED
AntioodAAntigen Framework
Figure A human antibody framework docked to SARS-CoV-2 Omieron RBD using HDOCK. CDR-H3s are designed based on the docking structure.
4.4 Design Without Bound Antibody Frameworks
In the last experiment, we consider designing antibodies without a known binding pose against the antigen, more general and challenging setting. We show that this challenging task could be achieved with docking software. Specifically, we create an antibody template from an existing antibody structure by removing its CDR-H3. This is because CDR-H3 is the most variable and accounts for most of the specificity, while other CDRs are much more conserved [Xu and Davis, 2000]. Next, we use HDOCK [Yan et al., 2017] dock the antibody template to the target antigen produce the antibody-antigen complex. In this way, the problem reduces to the original problem so we can adapt our model design the CDR-H3 sequence and structure and re-design other CDRs. We demonstrate using this method to design antibodies for the SARS-CoV-3 Omicron RDB structure (PDB: 7wvn, residue A322-A590, the structure from human antibody against influenza (PDB: 3qhf). Figure 6 shows the docking structure, five designed CDR-H3s and the binding energy distribution. It is hard confidently conclude that the generated antibodies are effective without reference antibody. However, according to the binding energy distribution, we can still say the generated antibodies are at least reasonable.
bound any antibodies). The antibody template is derived
Conclusions and Limitations
In this work, we propose diffusion-based generative model for antibody design. Our model is capable of a wide range of antibody design tasks and can achieve competitive performance. One main limitation of this work is that it relies on an antibody framework bound to the target antigen. Therefore, we leave for future work design an effective model for generating antibodies without bound structures. Another limitation is that it remains unclear whether the generated antibodies can be produced in the wet lab and actually binds to the target. More efforts are needed to design biologically effective antibody.
Acknowledgments and Disclosure of Funding
Supported by National Key R&D Program China No. YFF
References
Jared Adolf-Brylogle, Oleks Kalyuzhniy, Michael Kubitz, Brian D Weitzner, Xiaozhen Hu, Yumiko Adachi, William R Schief, and Roland Dunbrack Jr. Rosettaatiboodydsiin (rabd); A general framework for computational antibody design. PLoS computational biology, 14(4):e10011 12, 2018.
Rahmad Akbar, Habib Bashour, Puneet Rawat, Philippe Robert, Eva Smorodina, Tudor-Stefan Robert Frank, Brij Bhushan Mchta, Mai Ha Vu, et al. Progress and
Cotet, Karine
10

---

challenges for the machine learning- based design of fit-for-uurpose monoclonal antibodies. In Mabs, volume 14, page 2008790. Taylor & Francis, 2022a.
Rahmad Akbar, Philippe A Robert, Cédric Weber, Michael Widrich, Robert Frank, Milena Pavlović, Lonneke Scheffer, Maria Cheriggovskay,, Igor Snapkov, Andrei Slabodkin, et al. In silico proof of principle of machine learning-basee antibody design at unconstrained scale. Mabs, volume 14, page 2031482, Taylor & Francis, 2022b.
Rebecca F Alford, Andrew Leaver-Fay, Jeliazko R Jeliazkov, Matthew O'Meara, Frank P DiMaio, Hahnbeom Park, Maxim V Shapovalov, Douglas Renfrew, Vikram Mulligan, Kalli Kappel, et al. The rosetta all-atom energy function for macromolecular modeling and design. Journal of chemical theory and computation, 13(6):3031-33) 3048, 2017.
Ethan C Alley, Grigory Khimulya, Surojit Biswas, Mohammed AlQuraishi, and George M Church. Unified rational protein engineering with sequence-based deep representation learning, Nature methods, 16(12):1315-322 2019.
Namrata Anand and Tudor Achim. Protein structure and sequence generation with equivariant denoising diffusion probabilistic models. arXiv preprint arXiv:2205.150193 2022.
Ivan Anishchenko, Samuel Pellock, Tamuka M Chidyausiku, Theresa A Ramelot, Sergey Ovehin- nikov, Jingzhou Hao, Khushboo Bafna, Christoffer Norn, Alex Kang. Asim K Bera, et al. De novo protein design by deep network hallucination. Nature, 600(7889):547-522 2021.
Jacob Austin, Daniel Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34: 7999999993, 2021.
Minkyung Baek, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov, Gyu Rie Lee, Jue Wang. Qian Cong, Lisa N Kinch, R Dustin Schaeffer, et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science, 373(6557) 2021.
Kyle A Barlow, Shane O Conchuir, Samuel Thompson, Pooja Suresh, James E Lucas, Markus Heinonen, and Tanja Kortemme. Flex ddg: Rosetta ensemble-based estimation of changes in protein-protein binding affinity upon mutation. The Journal of Physical Chemistry B, 122(21): 5389-5399, 2018.
Sidhartha Chaudhury, Sergey Lyskov, and Jeffrey J Gray. Pyrosetta: script-based interface for implementing molecular modeling algorithms using rosetta. Bioinformatics, 26(5))688-66) 2010.
Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet. Riemannian score-based generative modeling. arXiv preprint wrxiv:2202.027600 2022.
James Dunbar, Konrad Krawezyk, Jinwoo Leem, Terry Baker, Angelika Fuchs, Guy Georges, Jiye Shi, and Charlotte M Deane. Sabdab: the structural antibody database. Nucleic acids research, 42 (DI::DI40-DD11 2014.
Peter Eastman, Jason Swails, John D Chodera, Robert T McGibbon, Yutong Zhao, Kyle A Beauchamp, Lee-Ping Wang, Andrew C Simmonett, Matthew Harrigan, Chaya D Stern, et al. Openmm Rapid development of high performance algorithms for molecular dynamics. PLoS computational biology, 13(7);e 1e 0056599, 2017.
RA Engh and R Huber. Structure quality and target parameters. 2012.
Jean Gallier and Dianna Xu. Computing exponentials of skew-symmetric matrices and logarithms of orthogonal matrices. International Journal of Robotics and Automation, 18(1):10-20, 2003.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.

---

Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34, 2021.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. arXiv preprint arXiv:2203.17000 2022.
Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. bioRxiv, 2022.
John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph- based protein design. Advances in Neural Information Processing Systems, 32, 2019.
Charles A Janeway, Paul Travers, Mark Walport, and Donald Capra. Immunobiology. Taylor & Francis Group UK: Garland Science, 2001.
Wengong Jin, Jeremy Wohlwend, Regina Barzilay, and Tommi S. Jaakkola. Iterative refinement graph neural network for antibody sequence-structure co-design. In International Conference on Learning Representations, 2022.
Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. arXiv preprini arXiv:2206.017229 2022.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger,
Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-58) 2021.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes.
arXiv preprint
arXiv:1312.61 14, 2013.
Gideon D Lapidoth, Dror Baran, Gabriele M Pszolla, Christoffer Norn, Assaf Alon, Michael D Tyka, and Sarel Fleishman. Abdesign: algorithm for combinatorial backbone design guided by natural conformations and sequences. Proteins: Structure, Function, and Bioinformatics, 83(8): 1385- 2015.
Adam Leach, Sebastian M Schmon, Matteo Degiacomi, and Chris G Willcocks. Denoising diffusion probabilistic models (3) for rotational alignment. In ICLR 2022 Workshop Geometrical and Topological Representation Learning, 2022.
Anders Liljas, Lars Liljas, Goran Lindblom, Poul Nissen, Morten Kjeldgaard, and Miriam-rose Ash. Textbook of structural biology, volume 8. World Scientific, 2016.
Kresten Lindorff-Larsed Stefano Piana, Kim Palmo, Paul Maragakis, John Klepeis, Ron O Dror, and David E Shaw. Improved side-chain torsion potentials for the amber ff99sb protein force field. Proteins: Structure, Function, and Bioinformatics, 78(8):1950-199 1958, 2010.
Matthies, Muller, and GW Vinel. On the normal distribution in the orientation space. Textures and Microstructures, 10, 1970.
Dmitry I Nikolayev and Tatjana Savyolov. Normal distribution the rotation group so (3). Textures and Microstructures, 29, 1970.
RJ Pantazes and Costas D Maranas. Optedr: a general computational method for the design of antibody complementarity determining regions for targeted epitope binding. Protein Engineering, Design & Selection, 23(1 (4491))849-858 2010.
Leonard G Presta. Antibody engineering. Current Opinion in Structural Biology, 2(4):593 1992.
Jeffrey A Ruffolo, Jeffrey J Gray, and Jeremias Sulam. Deciphering antibody affinity maturation with language models and weakly supervised learning. arXiv, 2021.
Jeffrey Ruffolo, Lee-Shin Chu, Sai Pooja Mahajan, and Jeffrey J Gray. Fast, accurate antibody structure prediction from deep learning massive set of natural antibodies. bioRxiv, 2022a.
12

---

Jeffrey A Ruffolo, Jeremias Sulam, and Jeffrey J Gray. Antibody structure prediction using inter- pretable deep learning. Patterns, 3(2):100466, 2022b.
Koichiro Saka, Taro Kakuzaki, Shoichi Metsugi, Daiki Kashiwagi, Kenji Yoshida, Manabu Wada, Hiroyuki Tsunoda, and Reiji Teramoto. Antibody design using Istm based deep generative model from phage display library for affinity maturation. Scientific reports, 11(1):1-13, 2021.
Maxim V Shapovalov and Roland L Dunbrack Jr. smoothed backbone-dependdet rotamer library for proteins derived from adaptive kernel density estimates and regressions. Structure, 19(6): 844-858, 2011.
Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. In International Conference on Machine Learning, pages 9558-9568. PMLR, 2021.
Jung-Eun Shin, Adam J Riesselman, Aaron W Kollasch, Conor McMahon, Elana Simon, Chris Sander, Aashish Manglik, Andrew C Kruse, and Debora S Marks. Protein design and variant prediction using autoregressive generative models. Nature communications, 12(1))1-11, 2021.
Ken Shoemake. Uniform random rotations. In Graphics Gems III (IBM Version), pages 124 132. Elsevier, 1992.
Jascha Sohl-Dick Dicks-iin,,,, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamies. In International Conference Machine Learning, pages 2256-2265. PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Alexey Strokach, David Becerra, Carles Corbi-Verge, Albert Perez-Riba, and Philip M Kim. Fast and flexible protein design using deep graph neural networks. Cell systems, 1 11(4):402-411, 2020.
Doug Tischer, Sidney Lisanza, Jue Wang, Runze Dong, Ivan Anishchenko, Lukas F Milles, Sergey Ovchinnikov, and David Baker. Design of proteins presenting discontinuous functional sites using deep learning. Biorxiv, 2020.
Brian Trippe, Jason Yim, Doug Tischer, Tamara Broderick, David Baker, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif- scaffolding problem. arXiv preprint arXiv:2206. 04119, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Shira Warszawski, Aliza Borenstein Katz, Rosalie Lipsh, Lev Khmelnitsky, Gili Ben Nissan, Gabriel Javitt, Orly Dym, Tamar Unger, Orli Knop, Shira Albeck, et al. Optimizing antibody affinity and stability by the automated design of the variable light-heavy chain interfaces. PLoS computational biology, 15(8) 1007207, 2019.
Kevin E Wu, Kevin K Yang, Rianne van den Berg, James Y Zou, Alex X Lu, and Ava P Amini. Protein structure generation via folding diffusion. arXiv preprint arXiv:2209.15611 2022.
John L Xu and Mark M Davis. Diversity in the cdr3 region of vh is sufficient for most antibody specificities. Immunity, 13(1):37-45, 2000.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint (iX.i::2033009920 2022.
Yumeng Yan, Di Zhang, Pei Zhou, Botong Li, and Sheng- You Huang. Hdock: web server for protein-protein and pratein-dna/ma docking based on a hybrid strategy. Nucleic acids research, 45(W1):W365-W377. 2017.
Jianyi Yang, Ivan Anishchenko, Hahnbeom Park, Zhenling Peng, Sergey Ovchinnikov, and David Baker. Improved protein structure prediction using predicted interresidue orientations. Proceedings of the National Academy of Sciences, 196--1553, 2020.
13

---

Ling Yang, Zhilong Zhang, and Shenda Hong. Diffusion models: A comprehensive survey of methods and applications, arXiv preprint NNv:2209,00779,, 2022.
14


Reviews:
[{"review_id": "AMytTHJwwq8", "reviewer": "Reviewer_pxV3", "summary": "This paper addresses the problem of antigen-conditional antibody design. While there has been a line of work using ML for antibody design, existing work mostly focused on antigen-independent antibody generation, yet antigen-conditional antibody design has been far more common in practical use cases. This paper makes progress on an important problem.\n\nTwo main contributions of this paper are:\n1) A new curated dataset of \"antibody-like\" loops interacting with other protein chains, extracted from PDB.\n2) A new approach for antigen-conditional antibody design (joint sequence-structure design) based on diffusion models. ", "questions": "Major questions/suggestions:\n1. Among the three components (sequence, Ca, orientation), which one is more important to model? Ablation studies would make the paper a lot stronger.\n2. The GNN baseline in all results is antigen-independent? If so it'd be greatly appreciated to mark that (e.g. add a column in tables to say antigen dependent/independent), because otherwise it's misleading to directly compare the two.\n3. How many antigens are shared between the train and test splits? It would be important to at least show statistics of antigen overlap (e.g. min RMSD or % overlap) between train and test splits.\n4. How would the performance degrade if using HDOCK instead of known complexes for all the main results (sections 4.2-4.4)? \n5. Comparison to antibody structure prediction methods (e.g. IgFold) for RMSD?\n\nOther questions:\n1. Does \"side chains\" mean \"Cb\" specifically in the paper? It would be helpful to clarify whether it's all atoms in side chains or just Cb.\n2. Compared to using equal probabilities for all AAs in Eq. 1, would it be helpful to use e.g. BLOSUM or other priors?\n3. How is the posterior derived from Eq. 1?\n4. Is minimizing the expected KL (Eq. 4) equivalent to maximum likelihood estimation (MLE)? \n5. It'd be helpful to include simpler baselines, e.g. IMGT alignment consensus of antigen-specific antibodies. Although I expect simpler baselines to be worse, it'd still be helpful to put results into context.\n6. It would be more convincing for antibody optimization to also show results on antibody DMS datasets -- unless it's hard to evaluate likelihood in diffusion models?", "limitations": "See above (\"weaknesses\") for limitations. The authors did acknowledge the main limitations and I appreciate that as a reader. ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- (Originality & Significance) This paper considers antibody design conditioned on the antigen 3D structure. This has been an important yet challenging task, and this is one of the first ML approaches to do so.\n- (Originality) The authors curated a new dataset of antibody-like complexes (loops in contact with another chain). I believe this dataset will be useful for the research community.\n- (Clarity) This paper is mostly clearly written with high quality figures, although the writing could be improved at places (e.g. typos and more accurate language about CDRs).\n- (Quality) The empirical results seem sound, demonstrating improvements in three different antibody design tasks.\n\nWeaknesses:\n- (Significance) While the paper takes a step forward in antigen-conditioned antibody design, there is still a notable gap between the scenario in the paper and the real use case: In a real use case, we do not know the relative orientation between the antibody and the antigen, while in the design tasks in the paper the orientation is given based on known antigen-antibody complexes. The paper does address this concern in Section 4.5 by using HDOCK to dock the antibody template as a precursor step. However, this is only done for one antigen. in my view, this paper would be a lot stronger if the authors can show the design performance in Sections 4.2-4.4 both with and without known bound complexes. \n- (Quality) This paper would also be strengthened by more ablation studies on the relative merits of different model components, as well as by the inclusion of simpler baselines.\n- (Quality) The split in this paper is based on CDR sequence identity, and not based on antigen structures. It would be helpful to at least show statistics of antigen overlap (e.g. RMSD) between train and test splits.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "24poOHqLJeT", "reviewer": "Reviewer_tyV1", "summary": "This paper introduces a diffusion-based model for jointly modeling the sequence and structure of complementarity-determining regions (CDRs) of antibodies, conditioned on antigen structure and antibody framework. The method enables the design of CDRs in a wide range of scenarios (from least constrained to most): CDR sequence-structure co-design, fixed-backbone sequence design, or antibody optimization (starting from a known antibody). The approach deviates from prior work notably via the conditioning on the antigen 3D structure and factoring in the side-chain orientations. Experiments demonstrate superior performance in the aforementioned design settings. Lastly, authors curate an extended set of protein structures for model training: antibody-antigen from SAbDab and pseudo-CDRs from the PDB (assimilating loops and the corresponding chains they interact with as proxy antibody-antigen structures).", "questions": "Lines 92-93 \"It relies on an additional antigen-specific predictor to predict the neutralization of the designed antibodies, which is hard to generalize to arbitrary antigens\". Could you please clarify what you meant here regarding difficulty to generalize Vs your method? Do you mean this approach requires to train a new predictor for each new antigen (Vs your method handles any antigen by construction)? If so, isn't the antigen-specific predictor more likely to provide good results in the focus application it's been design for? Or do you have evidence the more general modeling you are proposing is both more flexible and has higher task performance?\n\nLines 274-275 -- \"We find that using the augmented training dataset could enhance the performance on the test set\". Did you train a model without the pseudo-CDR structures to give a sense of how useful this data extension is? \n\nTables 1/2: do you have a sense for the relative contributions to performance lift from the different modeling decisions you made over prior baselines (eg., conditioning on antigen structure, diffusion-based modeling, factoring in side-chain orientations, rotation & translation equivariance)\n\nMinor points:\n- Line 513 -- dead reference", "limitations": "- Limitations are very briefly discussed in the conclusion. \n- Potential negative societal impact of the work is not discussed (developed method may be re-purposed more broadly for other protein design tasks which may be malignant).", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- Significance: the ability to design novel antibodies that optimally bind to target antigens, while satisfying other constraints is critical to various antibody therapies\n- Originality: this work makes solid contributions in proposing a model that 1) handles certain design settings more naturally (eg, antibody optimization) 2) performs better than existing baselines (based on experiments reported in this work) 3) models the entire CDR structures with atomic precision through the incorporation of side-chain geometry.\n- Clarity: the paper is very well written and structured. Figures and notations are very clear. \n- Quality: sound and thorough experimental design. \n\n**Weaknesses**\n- Quality: a few minor claims / points are not fully substantiated (see Questions)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0Ck8zXfinzA", "reviewer": "Reviewer_B4br", "summary": "The paper presents a diffusion model for jointly generating the CDR sequence and structure of an antibody conditioned on its framework regions and a target antigen. Unlike existing methods, the presented method enables i) conditioning the generation on the antigen structure instead of only framework regions, and ii) predicting side-chain orientations. Lastly, the paper also presents a dataset of experimentally validated antibody structures iii) augmented by pseudo-antibody structures derived from the structure of non-antibody proteins.", "questions": "## Questions\n1) Does the diffusion model perform better than alternative models such as an autoregressive model (e.g. Jin et. al) or Transformer? Can you show this by ablation? Only change the architecture while using the same (augmented) dataset for training and tuning hyper-parameters appropriately.\n\n2) How does the proposed method compare to a simple sequence-based, non-iterative, model that generates CDR regions conditioned on the sequence (not the structure) of framework regions and the antigen, and predicting the structure with AlphaFold?\n\n3) Please compare to Jin et. al (http://arxiv.org/abs/2110.04624) also in section 4.2 and section 4.4. by training the model on the same augmented dataset of tuning hyper-parameters appropriately.\n\n4) Please show by ablation the effect of i) conditioning on framework regions and ii) the antigen structure.\n\n5) What is the training and inference time of the diffusion model compared to an autoregressive model (e.g. Jin et. al)?\n\n6) How does the model scale to long sequences, which is important for generating long CDRs or also framework regions?\n\n7) Does augmenting the training dataset by pseudo-structures improve the performance of the diffusion model and baseline model? You are referring to results in the appendix. However, I did not find any results. Since this is a major contribution of your paper, results should be in the main text.\n\n8) What is the accuracy of predicting side-chain orientations before refining them by Rosetta?\n\n9) What is the influence of applying the Rosetta packing algorithm and AMBER on the final structure? Does it change the ranking of methods?\n\n10) How do you apply the method to double-stranded antibodies? Do you generate the two different chains independently? \n\n11) Why does IMP% in Table 3 decrease with increasing t?\n\n12) What is the message of figure 3 and figure 4? Consider showing these figures in the appendix. Otherwise, interpret samples and also show samples of baseline methods.", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n* The paper is mostly clearly written\n* The use of a diffusion-based model for generating antibodies is new\n* The application of antibody design is important, e.g. for drug development\n\n## Weaknesses \n* The benefit of the diffusion-model over alternative generative models is not evaluated experimentally\n* The accuracy of predicted side-chain orientations without refinement by the Rosetta packing algorithm is not evaluated experimentally\n* The benefit of augmenting the dataset by pseudo-antibody structures is not evaluated experimentally\n* Important baselines are missing\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept