Multi-objective antibody design with constrained preference optimization
Authors: Milong Ren
ZaiKai He
Haicang Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: antibody design
diffusion generative model
preference optimization
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Antibody design is crucial for developing therapies against diseases such as cancer and viral infections. Recent deep generative models have significantly advanced computational antibody design, particularly in enhancing binding affinity to target antigens. However, beyond binding affinity, antibodies should exhibit other favorable biophysical properties such as non-antigen binding specificity and low self-association, which are important for antibody developability and clinical safety. To address this challenge, we propose AbNovo, a framework that leverages constrained preference optimization for multi-objective antibody design. First, we pre-train an antigen-conditioned generative model for antibody structure and sequence co-design. Then, we fine-tune the model using binding affinity as a reward while enforcing explicit constraints on other biophysical properties. Specifically, we model the physical binding energy with continuous rewards rather than pairwise preferences and explore a primal-and-dual approach for constrained optimization. Additionally, we incorporate a structure-aware protein language model to mitigate the issue of limited training data. Evaluated on independent test sets, AbNovo outperforms existing methods in metrics of binding affinity such as Rosetta binding energy and evolutionary plausibility, as well as in metrics for other biophysical properties like stability and specificity.

Published as a conference paper at ICLR 2025
MULTI-OBIECTIVE ANTIBODY DESIGN WITH CON- STRAINED PREFERENCE OPTIMIZATION
Milong Ren Zaikai He Haicang Zhang
Medicinal Bioinformatics Center, Shanghai Jiao Tong University School of Medicine
"Central China Research Institute of Artificial Intelligence
'Institute of Computing Technology. Chinese Academy of Sciences
*University of Chinese Academy of Sciences
ABSTRACT
Antibody design is crucial for developing therapies against diseases such as can- cer and viral infections. Recent deep generative models have significantly ad- vanced computational antibody design, particularly in enhancing binding affinity target antigens. However, beyond binding affinity, antibodies should exhibit other favorable biophysical properties such as non-antigen binding specificity and low self-association, which are important for antibody developability and clin- ical safety. To address this challenge, we propose AbNovo, framework that leverages constrained preference optimization for multi-objective antibody de- sign. First, pre-train an antigen-conditioned generative model for antibody structure and sequence co-design. Then, we tine-tune the model using binding affinity reward while enforeing explicit constraints other biophysical prop- erties. Specifically, we model the physical binding energy with continuous re- wards rather than pairwise preferences and explore primal-and-dual approach for constrained optimization. Additionally, we incorporate a structure- pro- tein language model to mitigate the issue of limited training data. Evaluated on independent test sets, AbNovo outperforms existing methods in metrics of binding affinity such as Rosetta binding energy and evolutionary plausibility, as well as metrics for other biophysical properties like stability and specificity.
INTRODUCTION
Antibodies are vital immune proteins that bind to target antigens and trigger the adaptive immune re- sponse. Antibody design is critical in developing therapeutic drugs for wide range of diseases such as cancer, autoimmune deficiency, and virus infections, with over hundred antibody drugs currently approved (Kaplon al, 2022). Structurally, antibodies consist of conserved framework regions and regions (CDRs). The framework regions maintain
highly variable complementarity-deeermmi the overall antibody structure, whereas CDRs exhibit significant variability in both structure and sequence and mainly determine antigen binding. Thus, the primary objective of computational an- tibody design is to design CDRs that bind the target antigens and possess favorable biochemical properties.
Recent deep generative models have achieved remarkable success in computational antibody de- sign, particularly in enhancing antigen-specific binding affinity (Luo al. 2022 Zhu et al. 2024
Martinkus et al. (024)) Lin et al.] 2024 /Zhou et al.] (024).. For example, DiffAb (Luo et al. 2022)
employs denoising diffusion probabilistic model for antibody structures and sequences co-design. AbX (Zhu et al. 2024) utilizes score- score-based diffusion model and incorporates geometric, physical, and evolutionary constraints guide the design process. Notably, the recent method ABDPO (Zhou et al., 2024) integrates physical energy as guidance for binding affinity within the framework of direct preference optimization.
While binding affinity is crucial, antibodies should exhibit other favorable biophysical properties such as high target specificity and low self-association,, which important for downstream devel- opability and clinical safety. For example, non-specific (off-target) binding can potentially trigger
"oorreppnndence should be addressed H. Zhang (rhanghaicang@ssjtuudd..o.

---

Published as a conference paper at ICLR 2025
unintended immune responses, causing inflammation other syndromes (Nicholson et al. 1991 Ferrigno 2016 Makowski et al. 2022). High self-association, mainly due to large amount of charged aminos on CDRs, leads to antibody aggregation and results in decreased efficacy (Sule et al. 2011 Makowski et al. 2024). wet-lab experiments, a common strategy is to generate a diverse set of candidates and then filter them based desired properties (Watson al., 2023 Ben- nett et al. [2023). However, this post-filtering approach is inefficient and has a lower success rate in designing antibodies that satisfy all specified constraints.
To address these challenges and bridge the gap between in silico design and real-world applica- tions, we propose AbNovo, framework that leverages constrained preference optimization for multi-objective antibody design. We first pre- are-train& a score-based diffusion model for antibody struc- ture and sequence co-design. We then employ constrained preference optimization to fine-tune the model using metrics of binding affinitysuuch as Rosetta binding energy and evolutionary plau- sibility-as objectives, while incorporating biophysical properties related to non-specific binding, self-association, and stability constraints. During training, we utilize primal-dual approach to iteratively optimize the policy model. contrast to the previous framework of constrained direct preference that leverages pairwise preferences, model the physical binding energy as continuous rewards and collect both reward and constraint values offline. We introduce additional techniques to improve training stability and performance, such incorporating a structure-aware protein language model to alleviate the issue of scarcity of training data.
Our contributions are summarized follows:
We propose AbNovo, a deep generative model for multi-objective antibody design that incorporates explicit constraints representing biophysical properties critical for real-world antibody development.
We extend the recent framework of constrained preference optimization from language model alignment to diffusion- generative models. This includes developing a corre- sponding training algorithm, supported by theoretical derivations and analysis.
We introduce additional simple yet effective techniques to enhance performance. For in- stance, we incorporate structure-awar protein language model trained on massive struc- tural data beyond antibodies to alleviate overfitting issues due the scarcity of antibody- antigen training data.
Experiment results show that AbNovo achieves state-of-the-art performance in metrics of binding affinity while also satisfying other biophysical properties.
2
RELATED WORKS
2.1
COMPUTATIONAL ANTIBODY DESIGN
Antibody design is to optimize the antibody structure and sequences, particularly the CDRs, to bind to target antigens while meeting other biophysical properties. Traditional methods rely on computationally intensive Monte Carlo-based simulations to optimize the antibody-antigen binding energy Bryoof- et al. 2018)).
Recent deep tearning-basee methods for antibody design can be broadly categorized into discrimina- tive models and generative models. Discriminative models typically leverage graph neural networks learn the presentations of antigen structure and to predict the most likely antibody structure and sequence (Kong et al., 2023 Lin et al. 2024). In contrast, generative models such as the denoising diffusion probabilistic models (DDPM) (Luo et al. 2022] Martinkus al. 2024 Tan et al. 2024) and score-based diffusion models (Zhu et al. [2024] Kulyté et al. (2024) build an antigen-conditioned
generative process of antibody sequences and structure.
Another trend is incorporating guidance into the generative process. AbX et al. 2024) lever- ages evolutionary, physical, and geometric constraints to narrow down the plausible structure and sequence sampling space. The method most related to our work is AbDPO (Zhou et al.) 2024), which applies preference optimization to enhance binding affinity. Our method differs from Ab- DPO in terms of framework and training objective. First, AbDPO focuses optimizing towards a lower Rosetta energy using the framework of direct preference optimization (DPO). whereas our method employs constrained preference optimization that optimizes binding affinity while imposing
2

---

Published as a conference paper at ICLR 2025
constraints related specificity and self-association. Second, regarding the preference objective, AbDPO uses pairwise preferences, whereas our method employs continuous rewards to model the physical binding energy.
2.2 PREFERENCE OPTIMIZATION OF GENERATIVE MODELS
In natural language processing, large language models (LLMs) have achieved remarkable progress in natural language generation (Achiam al. 2023 Touvron al. 2023 Dubey al. 2024). To better align these models with human values and preferences, various preference optimization frameworks (Azar et al. 2024; Chen et al. 2024b; Ethayarajh et al. 2024, Rafailov et al. 2024b; Meng et al.) 2024 Zeng et al. 2024) have been developed. For example, Reinforcement Learning from Human Feedback (RLHF) pre-trains an explicit reward model and then fine-tunes the base models through reinforcement learning (Ouyang al. (022). Direct Preference Optimization (DPO) (Rafailov et al., 2024b) derives a closed form for the optimal policy and directly fine-tunes the base model with preferences data rather than explicit reward model. More recently, DPO has been adapted from LLMs alignment to the diffusion-batt -baed generative models for image generation (Wallace et al.)
2024
However, RLHF presents several challenges in practice. First, the overoptimization issue arises (Gao et al. 2023 Coste et al. 2023 Eisenstein et al.) 2023), the reward models often serve as an imperfect proxy for true human preferences. Second, a single reward with scalar output is often inadequate capture multiple aspects of human preferences. such helpfulness and harmlessness, which are not always easily compatible (Ouyang et al.. [222] 2022| et al. 2022] Thoppilan et al., 2022). To mitigate these issues, constrained RLHF (Moskovitz et al. (202)) has been proposed, which fine-tunes LLMs by maximizing a target reward while imposing explicit constraints on aux- iliary safety objectives like harmlessness. More recently, the framework of constrained DPO (Liu
et al. 2024 Huang et al., 2024) has also been proposed for LLMs alignment.
Our method is closely related to constrained DPO, but we extend it from L.MSs alignments to the diffusion-based generative models, providing rigorous theoretical derivation. Furthermore, by in- tegrating noise contrastive estimation with constrained DPO, we model physical binding energy using continuous rewards instead of relying on pairwise preference data. Specifically, we evaluate both reward and constraint values for sampled antibodies using existing models that assess antibody biophysical properties.
METHODS
Stage 1: Training Base Model
Stage Preference Optimization
Sample Rowars: Tuncion 101 0.00,
Sample Dewarric: 1.13 0.50 0.45, 0.11
Sample
Rewards: 0.70
prediction
Soucture- aeorra A
daallesd offinne alata)
0.11 0.80 GuS
Y
Arrige
Language
AFDB (2M) POR)SOOM)
Strcttre Sequence Vislaiion Ausilary
Affaayy
Pasnibiity
Self anordation
Nar-opectfic
Sulsiiy
Objaativoss
Constriinte
Polding IPA.
Podrr
Folding ThA
Sturhhee LM
Stucture M.
B4eDub (7))
Reference model
Policy madel
Figure 1: The overview of AbNovo. In the first stage, we train the base model, diffusion-basee generative model for the co-design of antibody CDR sequences and structures. In the second stage, we update the policy model guided by biophysical properties, using a reference model initialized from the base model as regularization term during this update.
3

---

Published as a conference paper at ICLR 2025
As illustrated in Figure 1 AbNovo comprises two main phases during training. First, train an antigen-conde conditioodddiidddcceeeeeeeee generative model for the co-design of antibody structure and sequence. We refer to this co-design model the base model. Next, fine-tune the base model using constrained preference optimization. In this process, we refer to the network being optimized as the policy model. The policy model is optimized based biophysical properties such Rosetta Binding Energy and self-association, while a reference model is used as regularization term during this process, preventing over-optimization. The reference model is initialized from the base model. We briefly introduce the preliminaries and notations used in our methods in Section 3.2 Subsequently, we describe the base model and the framework of constrained preference optimization in Sections 3.2 and 3.3 respectively.
3.1
PRELIMINARIES
Each antibody consists of two identical heavy (H) chains and two identical light (L) chains. The vari- able domain of each chain is divided into framework regions and three caapllmmentarly-ddrrmminn regions (CDRs): HI, H2, and H3 in the heavy chain, and L1, L2, and L3 in the light chain.
Following previous works (Luo al.) 2022 Campbell al. 2024 Zhu et al. 2024), the structure is represented as elements of SE(3) to capture the local frame along the backbone (Jumper et al. 2021]. For antigen- antigen-antibody antibody complexes of total length N. each residues can be represented as T, (Ia, rú dg), where = ...... N. Here,
R3 is the coordinate of the C. of the i-th residue, rs E S3(3 the rotation matrix of local respect to global frame, and a; E {1. ................... 203 U {maskk]] } is one of 20 types residues or the mask state [mask]. We assume that all CDRs to be generated has m residues in total, which can be represented by PCDR = { Ti, a;)li = n + 1,..., n + m]. The antibody framework and antigen can be represented by PrC (12 Ti, a;)); N (n + I....,n + m}}. Formally, our goal is to model the distribution of PODR conditioned on Prc- We provide all the notions and their corresponding descriptions in Table 4
3.2 ANTIBODY STRUCTURE AND SEQUENCE CO-DESIGN
Diffusion Process for Sequence and Structure. Following previous works (Yim et al.) 2023 Campbell et al. 2024; Zhu et al. (024), we use a multi-modal diffusion model to jointly design antibody sequence and structure. Specifically, we use the Continuous Time Markov Chain (CTMC)- based diffusion model for discrete sequence and score-bar based SE(3) diffusion model for structure, respectively.
We use ( (T antm {(x 1.(0) a. ) I 1 to represent the antibody's s structure and -in++1
sequence at time t and (0:1) = (T T())), ..... T()) to represent the diffusion path. Here, t follows a uniform distribution 4(0 1) and we apply distinct noise schedules on t for translation (e) rotation ri and sequence az as described in Appendix A.4.2 The forward diffusion for
zy (t) (r()) ()) can be written following:
(n
x (1)
~ (z:"0 c -1-
(1) IGso(3) (1:
0t"h ~ Cat Cat( (la! (0) (1) a. (1-t) + 51a,") (aoe"aa())) 1t).
Here, ofi.j} is the Kronecker delta function which is when i equals to and is other- wise. N, NISoo)) and Cat represent the Gaussian, Isotropic Gaussian, and Categorical distri- butions, respectively. We use sy. sy. and sg to represent the score network of translation, ro- tation, and sequence, respectively. We select the prior distribution of structure and sequence as P Perfor (a.)) .r("))= = (N (d,))ddd @ IGso(3) (0, Id) @d) and pfrioo (a (a(") [[mask]] @N re- spectively. Then, the reverse diffusion process can be written as follows:
x !-a0 P# (M(
At (0) + At- (a T (0,AA--Td)) (ds)ds) Tds)).
r (1-0)) ~ ISsora) ( (cxp,.0/ (At sp((!!) (1) T( Pr,, t)), At Id).
(1-1)) ~ Cat (ola," ,
(t-at)) و + Al "ā (1-20) T(), PrC, 1))
a;
4

---

Published as a conference paper at ICLR 2025
where exp and log are the exponential and logarithmic maps, and P# is the projection matrix that removes the center of mass & ΣiEPEDa 4z-
Network Architecture.
Inspired by previous works (Huguet et al. 2024 Zhu et al. 2024 Ren al. 2024), we utilize score network similar to the architecture proven useful in protein structure prediction (Lin et al., 2023; Jumper et al. 2021). It comprises three components: a structure-and aware language model pre-trained on 2.2M structures, the Evoformer trunks, and Invariant Point Attention (IPA) @umper et al., 2021). We first extract embeddings from the structure-awans language model for both the conditional sequences of antibody framework and antigen, and noisy CDRs sequence. Structural information is encoded using a distogram of the antibody framework, antigen, and noisy CDRs, followed by linear projection. Then, these structure and sequence representations are then processed by Evoformer trunks, Finally, the IPA and an MLP output the denoised CDRs structure and sequence, respectively. Appendix 12 and Figure present more details on feature preprocessing and network architecture.
extraactemeeddinggg
Training Losses. Following previous methods (Campbell et al., 2024 Zhu al. 2024). the losses for training the base model primarily include Denoised Score Matching (DSM) losses for structure and sequence, auxiliary losses, and structural violation loss. Details of these losses are provided in Appendix A.4.5 presents the details of these losses.
3.3 CONSTRAINED PREFERENCE OPTIMIZATION
To design antibodies that bind to target antigens while satisfying other biophysical properties, the training objective is formulated as:
max (() (po) = EPrCED ET n (T 0
wm Rm (T("), PFO)] (Fc) Σ
m
- 3L
w.ne ( (P PPc)lllpee ( (T()))))))) Po
(3)
C, (T(()) Pre)] < Cn. for all n in constraints sets, s.t. Tw0 (Pe))
(Prc)
where Rm and wm denote normalized reward (c.g., Rosetta Binding Energy, Evolutionary Plau- sibility) and its weight, and C, and Cn denote a constraint (c.g.. Non-specific Binding, Self- association, Stability) and its threshold. In this equation, the first term of the objective is max- imize rewards, while the second term is to keep the policy model close to the reference model. Additionally, the optimization is constrained to ensure that expected constraints do not exceed their thresholds. Both the reward and constraint values of sampled antibodies are computed offline, as detailed in Appendix A.5
This problem can be associated with Lagrangian function and then the max-min optimization problem follows:
max min ((L) A) (po) (pe,
(4)
Po AERY
j(0) A) Σ An E
(T.0 (Poc) (oneecCCC((( (T(), pFF) - -C.)) C.)
(5)
where (R) is the objective function in Equation 3 and X PAi,... Aa] is the vector of dual variables. This equation can be interpreted as appending a penalty ((() to the original objective ((R), The penalty term, which depends on how much the antibody generated by the current model violates constraints, can be adjusted dynamically through the Lagrange multipliers X. Then this problem can be solved using the primary-dua method (Bertsekas) 2014 Ito & Kunisch 2008 Liu
al. 2024) by iteratively taking two steps
1. Update policy: Update network parameters 0 to find P = arg max J(L) (pe, A) based current value of X.
2. Update X: Update by estimating the gradient of dual function G()) = [13, based policy P
5

---

Published as a conference paper at ICLR 2025
Prior work has proved that the objective function is concave pe, and thus strong duality holds (Liu et al. 2024 Huang et al. (024). formal proof is provided in the Appendix [A.3.1
More specifically, first sample set of antibodics, denoted as gg from the reference model Pref and compute all reward and constraint values offline for these samples. Then, based on D we iteratively perform the two steps to optimize the policy model po, detailed in sections 3.3. and
3.3.2 respectively.
3.3.1
UPDATE POLICY
Given the reference model and current value of À along with corresponding preference data, previous methods (Ratailov et al.) 2024b; Liu et al. 2024: Huang et al. 2024) leverage the DPO framework to determine X Here, have significantly adapted this framework for the antibody design task.
Direct Perference Optimization with Continuous Rewards. Recent work of NCA (Chen et al.) 2024a) has extended the DPO framework to incorporate continuous reward values for large language model alignment. Since many biophysical properties (e.g., physical energy) are continuous values, have further adapted NCA for diffusion-based generative models and integrated it into the strained preference optimization framework. The detailed theoretical derivation is presented in the Appendix A.3.2 Then, the optimal policy p has the following form:
P> Pret exp(R/33)
Σ Wm Rm (T PFC) - (TC, Prc),
=
(7)
where (()) E Dg are the sampled antibodies under the reference model, and /3 is the regulartizion weight in Equation
Integrating with NCA, the original training objective in Equation 4 can be reformulated as follows (Appendix A.3.2
K
exp (Rr/3)
Cdiff (0) = E (T!") Pro.R.
log (fi) + K
log a (-fi)
exp (R₂
(8)
P9 2. (T((:1) ( Pref
= E E (rrdaaa Ppc)
log
(T PPC))
Here, we use Do to represent antibodics sampled from the reference model and or is the sigmoid function. In this equation, the first term is increase the likelihood of the samples based on their rewards and the second one serves as regularization term.
Due to the objective in Equation 8 is inefficient and intractable to train, we utilize Jensen's 's inequality and approximate the reverse process with the forward diffusion gu (Wallace et al. 2024 Zhou al. 2024). As derived in Appendix A.3.2 the simplified objective is as follows:
diiff NCA (0) = E LL Prc Thu~ m )
Σ
K
exp (R,/3)
log of
Fi) + log o
1 K
Al Fi)).
1
K
Al
exp (R./3)
F. DKL (g (m
((()) Prc) 11p ( (T (T( PFC)
DKL. (Ta ((--a))) T
(0,0)
PFo) x (T (er
Ar) p Pro)
(0.8)
Increase the Likelihood of Samples with High Rewards. Previous studies have found that the OPO-based approach makes the likelihood of the optimal preference samples decrease (Rafailov et al. 2024a). Chen al. (2024a) demonstrated that the NCA training objectives ensure that the likelihood the optimal reward does not decrease. Building on this, we kept the training objectives from the base model, Lsup, to further increase the likelihood of samples with higher rewards. Then

---

Published as a conference paper at ICLR 2025
the total loss can be written
R₁ R k Σ, R,
R
Cupdate policy
NCA + a (sup)
max (0.
Coup,i.
(10)
2, R₃
3.3.2 UPDATE X
This step involves calculating the gradient of the Lagrange multipliers by assessing the extent of constraint violation in the current policy. The gradient of (A) can be written follows:
(G(A) dx
c C(T(O) Pre)). (-T(TT),rr))
E P. ~ (T( (Prc)
(11)
where C = [C,,..., C₂] and C [c., Cr). J In this equation, the gradient of (()) can be calculated by the expected degree of constraint violation in the sampled antibodies under the current policy model px.
Considering that the optimal solution of Equation under can written as PX
7 exp(R/3), the specific estimation approach can be deduced into a closed-form:
ppat (T Pro) (xp(RR/)(CC-CC))) C(T(O) (P(c)))))))))
(A) = PECEDD Ax
(12)
E Epr exp(R/3)
By this way, we can estimate the gradient the ((A) in an offline manner. The details of the estimation process are provided in Appendix A.3.3
3.3.3 ITERATIVE OPTIMIZATION
Following previous work (Dubey et al. 2024), we update the reference model with the latest trained policy model and conduct several rounds of constrained preference optimization. Specifically, the k-th round, we use the trained policy model from the (k- - 1)-th round to update the reference model and collect sampled antibodies and their reward and constraint values offline. The entire training process for constrained preference optimization is detailed in Algorithm]
4 RESULTS
4.1 EXPERIMENTS SETUP
Training and Testing Sets. We trained AbNovo using antibody-antigen complex structures derived from the SAbDab database (Dunbar et al. 2014) and evaluated its performance on the RAbD test set, which is widely used for in silico antibody design. During testing, we simultaneously generated all six CDRs conditioned on the antigen and the antibody framework regions. Following previous work, we strictly eliminated any overlap between the training and test sets by applying 40% sequence identity threshold CDR-H3. More details on the preparation of the training and testing datasets are provided in Appendix A.4.4
Baseline Methods. We compare AbNovo with representative methods from each category: dis- criminative model dyMEAN (Kong et al., (023) and GeoAb (Lin et al. 2024);; and diffusion-based generative models DifrAb (Luo al.] 2022) and AbX (Zhu al. 2024). Since dyMEAN does not use native antibody framework structure as input, its settings difter from other methods, which may lead to an underestimation of dyMEAN's performance in the comparisons presented in our experiments. We note that AbDiffuser (Martinkus et al.) 2024) and AbDPO (Zhou et al. 2024 are unavailable for benchmarking. More details on running these methods are Appendix|A.r.
Evaluation Metries. We group the evaluation metrics into two categories: reference-based met- rics and reference-indeepeneent metrics. The reference-based metrics assess the similarity between the designed and native antibody structures and sequences. Specifically, Amino Acid Recovery (AAR, %) measures the sequence recovery accuracy by comparing the generated sequences to the native sequences. Root Mean Square Deviation (RMSD, Á) calculates the structural deviation be- tween C. coordinates of the generated and native CDRs.

---

Published as a conference paper at ICLR 2025
Algorithm Constrained Preference Optimization for Antibody Design
1: Input: Antigen-antibody complex dataset D, base model Pa
(0) preference loss L, dual function
G, reward functions (R... (T()), Pro) Mr m==1 constraint functions {C.. (T,()) PFc) initial vector of dual variables X, weights of the rewards {4n Jm= learning rate (Me '))), number of rounds of constrained preference optimization K, number of training steps during each round
B. number of data for training V.
2: for k= K do
# Initialize reference model and policy model of the current round (k-1)
(k)
3: Pref Pg
Po
4: Po Po (k-1)
(k)
Collect the antibody samples from reference model offline
5: Da = {(T P S.t. PFC, ~ D, T Pief
(8) (T Prc.)))
Compute the reward and constraint values of the antibody samples in Dg offline Rm, C.. f- Evaluate(Dg))
6:
7:
for to do
# Annotate Dg with Equation 7 Aunder current X
8:
R. (T.))) PPc)==E
= Em Wm Rmm. En An. # Update policy model 9 as Equation
9:
Pe PolicyOptimizer(p) (k)
(k)
L. 11. Dg)
# Update X as Equation 12
10:
LambdaOOttimizen
11, Dg)
11: end for
12: end for
(K) 13:
return Po
The iereeence-iddeeeedeett metrics evaluate properties without direct comparison to native antibod- ies. Rosetta Binding Energy assesses the binding affinity of the designed antibodies to target anti- gens. Evolutionary Plausibility is evaluated using the likelihood under an independent antibody language model (Shuai et al., 2021). Additionally,
consider the proportion of generated antibod- ies that satisfy constraints related to asoociisoi,,,,n, stability, and non-specific binding. For each method, we designed 128 antibodies per antigen and evaluated their average metries. Details of these metrics and the thresholds for each constraint are provided in Appendix A.5
4.1.1 EVALUATION ON MULTI-ORIECTIVV ANTIBODY DESIGN
As shown in Table I AbNovo outperforms all baseline methods across all reféreene-indepeendett metries. These results indicate that AbNovo not only excels in designing antibodies with superior binding energy and evolutionary plausibility but also achieves the lowest percentage of constraint violations compared to other methods. Furthermore, in comparison to our base model, AbNovo demonstrates significant improvements across all metrics, underscoring the effectiveness of con- strained preference optimization.
Subsequently, evaluated AbNovo reference-basee metries. As shown in Table 5 our method achieves superior performance on these metrics compared to all baseline methods.
4.2 ABLATION STUDIES
We trained several ablation models to investigate the relative importance of the core components of our method.
First, we compare constrained preference optimization with supervised fine-tuning (SFT). In the SFT setting, we selected the optimal sample for each antigen as training data, based the weighted rewards for each property (SFT in Table 2). As presented in Table 2 while SFT improves the per- formance of the base model, remains less effective compared preference optimization learning-
!GenAb only designs the CDR H3 of the antibody. For the other CDRs, we utilize the natural antibody sequence and structure for evaluation.
8

---

Published as a conference paper at ICLR 2025
Evaluation of reference-independeee metries on RAbD test set. Here, reference represents the native antibody structure and sequence in RAbD dataset.
Table 1:
Binding Energy (4)
Evolutionary Plausibility (L)
Self-
Stability (1)
Non-specific Binding (4)
Methods
association (U)
reference
19944
2.38
12.3%
0%
3.5%
DiffAb
-0,96
2.60
7.6%
15.6%
2.3%
dyMEAN
-1.74
2.82
50.8%
94.5%
1.8%
GeoAb
-1.75
2.69
38.5%
7.6%
4.3%
AbX
4.79
2.44
14.5%
4.8%
11.6 %
AbNovo (base)
-2.60
2,41
19.9%
2.9%
10.9%
AbNovo -12.05
2.36
2.3%
2.8%
1.7%
Second, we compare constrained preference optimization with preference optimization used in Ab- DPO (Zhou et al. 2024). For preference optimization, we convert all constraints considered in AbNovo into optimization objectives (Multi-ohjective in Table 2 We observed a slight increase in fulfilling all constraints but significant drop performance in Binding Energy and Evolutionary Plausibility. This indicates that for antibody design, certain biophysical properties are more suitably treated as constraints rather than optimization objectives. This phenomenon has also been observed in previous studies on language model alignment (Liu et al. 2024 Huang et al.) 2024).
Third, we included two ablation experiments to demonstrate the relative contribution the structure-aware language model that is used alleviate the scarcity of antibody-antigen complex data. i) We trained an ablation model where we excluded the embeddings input features (w.o. LM in Table We observed significant drops across nearly all metrics, indicat- ing the importance of the language model. if) We also trained an ablation model where we replaced this structure-aware language model with sequence-only language model (ESM-2 based in Table model yielded better results than the sequence-oni language
the language model as
2). It shows that the structure- model. Additionally, we assessed its ability predict long- long-distance contacts (Appendix (A.2.4) and found that the structure-aware language model significantly outperforms the pure sequence language model on independent test sets, including the CASP and CAMEO datasets.
Finally, we analyzed the effect of varying the number of iteration rounds K in constrained prefer- ence optimization on performance. As shown Table 3 we observed that increasing the number of iteration rounds resulted in improvements across overall metrics, with higher proportion of generated samples satisfying the constraints.
Table 2: Ablation studies for AbNovo on RAbD dataset. Ablation studies for AbNovo on the RAbD
dataset. The ablation experiment settings include: without using a language model (w.o. LM), replacing the structure-aware language model with ESM2 (ESM-2 based), using supervised fine- tuning instead of preference optimization (SFT), and incorporating constraints into the optimiza- tion objectives (Multi-objective).
Binding Energy (L)
Evolutionary Plausibility (1)
All Constrains (L)
Methods
AAR (t)
RMSD (L)
w.o. LM
7.54
2.67
46.5%
41.53%
3.19
ESM-2 based
1.75
2.40
30.8%
49.2%
2.55
AbNovo (base)
-2.60
2.41
26.7%
49.9%
2.19
SFT
-6.46
2.37
6.5%
48.8%
2,41
Multi-objective
-4.05
2.39
2.6%
42.7%
2.43
AbNovo
-12.05
2.35
3.9%
48.5%
2.37
4.3
CASE STUDIES
We present a case study (Figure 2 comparing the designed antibodies from different methods: dyMEAN, DiffAb, and AbNovo. This case illustrates that antibodies designed by AbNovo not

---

Published as a conference paper at ICLR 2025
Table Improvements achieved through iterative constrained preference optimization. Iterl-4 rep- resent the model's performance under constrained preference optimization with varying numbers of iterations.
Binding Energy (4)
Evolutionary Plausibility (L)
Self-
Stability (4)
Non-specific Binding (4)
Methods
association (1)
AbNovo (base)
-2.60
2.41
19.9%
2.9%
10.9%
Iterl
-6.45
2.38
6.5%
8.9%
4.7%
Iter2
-11.60
2.38
2.0%
5.2%
5.6%
Iter3
-12.05
2.36
2.3%
2.8%
1.7%
only exhibit higher binding affinity to target antigens but also fully satisfy all constraints. Previous studies have shown that larger area of negatively charged patches in the CDRs corresponds to higher risk of ceff-association in wet-lab experiments (Makowski et al. (024).. We see that dyMEAN produce large number of charged amino acids which can lead to potential risks of self-association.
In detail, we present the distribution of metrics for antibodies designed by different methods for specific antigens. As shown in Figure 3l for AbNovo-designed antibodies, there is not only higher percentage of antibodies that satisfy the constraints but also greater proportion that outperforms natural antibodics across binding energy and evolutionary plausibility.
ARDLGYCSEDT PORAEA
TREKYYDPSGDY A
TRIKRLNNDVDDY
TRICTFYOSFDY
Negativaly charged Posiively
nsriv sequanoe
TRINITLGDHHD
Hydrophobiic
Hydrophitic
dyMEAN:
DiffAb:
AbX:
Ours:
Binding energy: 3.01
Bincing energy:3.8 3.68
Binding energy: 3.02
Bincing energy 16.75
2.2.
tol::
rpl:2.2
2.2.
Btabiity selFassociatoon, speciillyy (False, False, True]
Stabilty, ail-asscciaion, specitiily. True, True, Trui]
Stubiyy sol-asoociaiinn, speckfoly. [Tus, True, False]
Stabilty sellassociation, specificity: [Tru, True, Trua]
Figure 2: Visualization of the antibody designed by dyMEAN, DiffAb, and AbNovo for given anti- gens (PDB ID: SNUZ). used orange and blue to identify all CDRs the designed antibody and all CDRs of the natural antibody. False and True used to indicate whether the designed antibody satisfies different constraints or not. We presented the CDR H3 sequences designed by different methods, indicating the biophysical properties, such hydrophilicity and hydrophobicity.
DISCUSSION
To design antibodies with strong affinity while enhancing their developability and clinical safety, propose AbNovo, a constrained preference optimization framework. Experimental results demon- strate that the AbNovo framework improves the affinity and evolutionary plausibility of designed antibodies by ensuring specificity, stability, and minimizing self-association.
The limitations of AbNovo mainly lie in the following aspects. First, while AbNovo emphasizes various in silico metrics for evaluation, limitation is the absence of wet-lab experimental valida- tion, which will be addressed our future work. Second, although AbNovo can design antibodies in scenarios with unknown antigen-antibody binding positions by integrating structure prediction and docking methods, the complexity this pipeline may lead to error accumulation. Third, though metrics like Rosetta energy widely used in evaluating antibody design, they still do not perfectly align with wet-lab experiments. Our framework adaptable and can incorporate other physico- chemical properties rewards and constraints.
10

---

Published as a conference paper at ICLR 2025
CODE AVILIBILITY
Code for AbNovo can be found at
ACKNOWLEDGMENTS
We acknowledge the financial support provided by the National Natural Science Foundation of China (Grant No. 32370657) and the Plan for Advancing Scientific Research Paradigms and Em- powering Disciplinary Upgrades Through Artificial Intelligence in Shanghai, awarded H.Z.
REFERENCES
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, al. Accurate structure prediction of biomolecular interactions with alphafold Nature, pp. 1-3, 2024.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
Jared Aryof--ryfogle, Oleks Kalyuzhniy, Michael Kubitz, Brian D Weitzner, Xiaozhen Hu, Yumiko Adachi, William R Schief, and Roland I. Dunbrack Jr. Rosettaantibodyyesiin (rabd): A general framework for computational antibody design. PLoS computational biology, 14(4):c1006112 2018.
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Danicle Calandricllo. A general theoretical paradigm to understand learn- ing from human preferences. In International Conference on Artificial Intelligence and Statisties, pp. 4447 4455. PMLR, 2024.
Nathaniel R Bennett, Brian Coventry, Inna Goreshnik, Buwei Huang, Aza Allen, Dionne Vafeados, Ying Po Peng, Justas Dauparas, Minkyung Back, Lance Stewart, et al. Improving de novo protein binder design with deep learning. Nature Communications, 14(1):2625, 2023.
Dimitri Bertsekas. Constrained optimization and Lagrange multiplier methods. Academic press, 2014.
Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. arXiv preprint wwX.v:2402.049// 2024.
Huayu Chen, Guande He, Hang Su, and Jun Zhu. Noise contrastive alignment of language models with explicit rewards. arXiv preprint arXiv:2402.05369, 2024a.
Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models. arXiv preprint arXiv:2401.01335, 2024b.
Thomas Coste, Usman Anwar, Robert Kirk, and David Krueger. Reward model ensembles help mitigate overoptimization. arXiv preprint arxhv:2310.02743, 2023.
Chai Discovery, Jacques Boitreaud, Jack Dent, Matthew McPartlon, Joshua Meier, Vinicius Reis, Alex Rogozhnikov, and Kevin Wu. Chai-1: Decoding the molecular interactions of life. bioRxiv, pp. 2024-02410 10, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.22 1783, 2024.
James Dunbar, Konrad Krawezyk, Jinwoo Leem, Terry Baker, Angelika Fuchs, Guy Georges, Jiye Shi, and Charlotte M Deane. Sabdab: the structural antibody database. Nucleic acids research, 42(D1):D1140) 146, 2014.
11

---

Published as a conference paper at ICLR 2025
Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alex D' Amour, DJ Dvijotham, Adam Fisch, Katherine Heller, Stephen Pfohl, Deepak Ramachandran, et al. Helping or herd- ing? reward model ensembles mitigate but do not climinate reward hacking. arXiv preprint arXiv:2312.09244, 2023.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306, 2024.
Paul Ko Ferrigno. 2016.
Non-antibody protein-based biosensors. Essays in biochemistry, 60(1):19-25,
Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07854 2022.
Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. In International Conference Machine Learning, pp. 10835-10866. PMLR, 2023.
Brian L Hie, Varun R Shanker, Duo Xu, Theodora UJ Bruun, Payton A Weidenbacher, Shaogeng Tang, Wesley Wu, John Pak, and Peter Kim. Efficient evolution of human antibodies from general protein language models. Nature Biotechnology, 42(2):275-283: 2024.
Xinmeng Huang, Shuo Li, Edgar Dobriban, Osbert Bastani, Hamed Hassani, and Dongsheng Ding. One-shot safety alignment for large language models via optimal dualization. arXiv e-prints, pp. arXiv-2405, 2024.
Guillaume Huguet, James Vuckovic, Kilian Fatras, Eric Thifor.au-Laufer, Pablo Lemos, Riashat Islam, Cheng- Hao Liu, Jarrid Rector- Brooks, Tara Akhound-Sadeglt Michael Bronstein, et al. Sequence-augmenteed (3)-flow matching for conditional protein backbone generation. arXiv preprint arXiv:2405.20331 2024.
Kazufumi Ito and Karl Kunisch. Lagrange multiplier approach to variational problems and appli- cations. SIAM, 2008
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-556) 2021.
Sonoko Kanai, JUN Liu, Thomas W Patapoff, and Steven J Shire. Reversible self-association of a concentrated monoclonal antibody solution mediated by fab-fab interaction that impacts solution viscosity. Journal of pharmaceutical sciences, 97(10):411944221 2008.
Hélène Kaplon, Alicia Chenoweth, Silvia Crescioli, and Janice M Reichert. Antibodies to watch in 2022. In MAbs, volume 14, pp. 2014296. Taylor & Francis, 2022.
Xiangzhe Kong, Wenbing Huang, and Yang Liu. End-to-end full-aton antibody design. In Interna- tional Conference on Machine Learning, pp. 17409-17429, PMLR, 2023.
Paulina Kulyté, Francisco Vargas, Simon Valentin Mathis, Yu Guang Wang, José Miguel Hernández- Lobato, and Pietro Liò. Improving antibody design with force- force-guided sampling in diffusion mod- els. arXiv preprint arXiv:2406. 05832, 2024.
Jiahan Li, Chaoran Cheng, Zuofan Wu, Ruihan Guo, Shitong Luo, Zhizhou Ren, Jian Peng, and Jianzhu Ma. Full-atom peptide design based on multi-modal flow matching. arXiv preprint 24X::2406000733. 2024.
Haitao Lin, Lirong Wu, Yufei Huang. Yunfan Liu, Odin Zhang, Yuanqing Zhou, Rui Sun, and Stan Li. Geoab: Towards realistic antibody design and reliable affinity maturation. In Forty- first International Conference Machine Learning, 2024.
Zeming Lin, Halil Akin, Roshan Rao. Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with language model. Science, 123- 2023.
12

---

Published as a conference paper at ICLR 2025
Zixuan Liu, Xiaolin Sun, and Zizhan Zheng. Enhancing Ilm safety via constrained direct preference optimization. arXiv preprint arXiv:2403.02475, 2024.
Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma. Antigen-specinee
antibody design and optimization with diffusion- generative models for protein structures. Advances in Neural Information Processing Systems, 35:9754 2022.
Emily K Makowski, Patrick Kinnunen, Jie Huang, Lina Wu, Matthew D Smith, Tiexin Wang, Alec A Desai, Craig N Streu, Yulei Zhang, Jennifer M Zupancie, et al. Co-optimization of ther- apeutic antibody affinity and specificity using machine learning models that generalize to novel mutational space. Nature communications, 13(1):3788, 2022.
Emily K Makowski, Tiexin Wang, Jennifer M Zupancie, Jie Huang, Lina Wu, John S Schardt, Anne S De Groot, Stephanie I. Elkins, William D Martin, and Peter M Tessier. Optimization of therapeutic antibodies for reduced self-association and non-specitic binding via interpretable machine learning. Nature Biomedical Engineering, 8(1):45-56, 2024.
Karolis Martinkus, Jan Ludwiczak, Wei-Ching Liang, Julien Lafrance- Sirrance-Vanasee, Isidro Hotzel, Arvind Rajpal, Yan Wu, Kyunghyun Cho. Richard Bonneau, Vladimir Gligorijevie. et al. Ab- diffuuse: full-atom generation of in-vitro functioning antibodies. Advances in Neural Information Processing Systems, 36, 2024.
Yu Meng, Mengzhou Xia. and Danqi Chen. Simpo: Simple preference optimization with reference-fiee reward. arXiv preprint arXiv:2405.14734. 2024.
Ted Moskovitz, Aaditya K Singh, DJ Strouse, Tuomas Sandholm, Ruslan Salakhutdinov, Anca D Dragan, and Stephen McAleer. Confronting reward model overoptimization with constrained rlhf. 2023.
arXiv preprint arXivv2310..4333
Suellen Nicholson, David E Leslie, Theodora Efandis, Christopher K Fairley, and Ian D Gust. Hep- atitis antibody testing: problems associated with non non-specific binding. Journal of virological methods, 33(3):311 1113311-11,, 1991.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Rafael Rafailov, Joey Hejna, Ryan Park, and Chelsea Finn. From r to q": Your language model secretly a q- f-function. arXiv preprint 12Xx::2404.12358 2024a.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024b.
Milong Ren, Tian Zhu, and Haicang Zhang. Carbonnovo: Joint design of protein structure and sequence using unified energy-based model. Forty-first International Conference on Machine Learning. 2024.
Richard W Shuai, Jeffrey A Ruffolo, and Jeffrey Gray. Generative language modeling for antibody design. BioRxiv, pp. 2021-12, 2021.
Shantanu Sule, Muppalla Sukumar, William Weiss, Anna Marie Harcelino-Cruz, Tyler Sam- plc, and Peter M Tessier. High-throughput analysis of concentratioon-dqeedeen antibody self- association. Biophysical
101(7):1749-1757, 201
Cheng Tan, Zhangyang Gao, Lirong Wu, Jun Xia, Jiangbin Zheng, Xihong Yang, Yue Liu, Bozhen Hu, and Stan Z Li. Cross-gate mlp with protein complex invariant embedding is one-shot antibody designer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15222-15230. 2024.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng- -ze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.082553 2022.
13

---

Published as a conference paper at ICLR 2025
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXt::2302.1397,, 2023.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1228-8238 2024.
Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eise- nach, Woody Ahern, Andrew Borst, Robert J Ragotte, Lukas Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature, 12207997):1088911100 2023.
Sandeep Yadav, Thomas M Laue, Devendra S Kalonia, Shubhadra N Singh, and Steven Shire. The influence of charge distribution on self-association and viscosity behavior of monoclonal antibody solutions. Molecular pharmacentics, 9(4):791-802, 2012.
Yumeng Yan, Huanyu Tao, Jiahua He, and Sheng- You Huang. The hdock server for integrated protiin docking, Nature protocols, 15(5):1829-1855) 2020.
Jason Yim, Brian L Trippe. Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation. arXiv preprint arXiv:2302.0222 2023.
Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang, Haifeng Zhang, and Jun Wang. Token-level direct preference optimization. arXiv preprint arXiv:2404. 11999, 2024.
1999,
Xiangxin Zhou, Dongyu Xue, Ruizhe Chen, Zaixiang Zheng. Liang Wang, and Quanquan Gu. Antigen-specific antibody design via direct anergy-based preference optimization. arXiv preprint 1657: 2024.
Tian Zhu, Milong Ren, and Haicang Zhang. Antibody design using score-based diffusion model guided by evolutionary. physical and geometric constraints. In Forty-first International Confer- on Machine Learning, 2024.
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.I NOTATIONS
Table 4: Mathematical Symbol Explanations
Symbol
Description
PеC
Antibody framework and antigen
PCDR
CDRs of antibody
x E Ra
Coordinate of C, atom Rotation matrix
r rE (3
a E (1. U {[mask]]
Amino acid type with mask Diffusion time
f
T = ((1)),())))))) (t))
Antibody's 's structure and sequence at time t
(0.)) (T) T())) .... T())
Diffusion path
80
Score network
Pprior
Prior distribution
Pref
Reference model
Pa
Policy model
PX
Optimal policy model under
u
Uniform distribution
N
Gaussian distribution
IGs0(3)
Isotropic gaussian distribution on SO(3)
Cat
Categorical distribution
R
Reward function
C
Constraint function
w e RR+ C 1
Weight of reward
Threshold of constraint
E R4
Dual variable
6 DKL
Kronecker delta function
Kullback-Leiller divergence
P
Projection matrix that removes the center of mass
g D
Dual function
Antigen-antibody complex dataset Antibodies sampled from the reference model Regularization weight
Do
B
A.2
ADDITIONAL RESULTS
A.2.1
REFERENCE-BASER METRICS EVALUATION
In this section, we evaluate AbNovo on reference-based metrics. As shown in Table AbNovo outperforms all baseline methods.
A.2.2 ANTIBODY OPTIMIZATION
In this section, further evaluate AbNovo antibody optimization tasks. Here, we specifically compare AbNovo with the generative model DiffAb and AbX. We follow the experimental process proposed by DiffAb (Luo al.) 2022). This process involves perturbing the CDR sequence and structure at time t using forward diffusion, then denoising from time t to time 0 in reverse diffusion to generate 128 antibodies for each antigen. We also follow the evaluation metrics for antibody
15

---

Published as a conference paper at ICLR 2025
Table 5: Evaluation of reference-based metrics across each CDR in RAbD test dataset.
Method
AAR(%) t RMSD(A)L
Method
AAR(%) f RMSD(A)
DiffAb
70.01
0.88
DiffAb
61.07
0.85
dyMEAN
75.71
1.09
dyMEAN
75.55
1.03
GeoAb
GeoAb
H1
L.I
AbX
80.92
0.85
AbX
80.37
0.80
AbNovo (base)
85.25
0.66
AbNovo (base)
84.34
0.66
AbNovo
84.55
0.65
AbNovo
83.50
0.65
DiffAb
38.52
0.78
DiffAb
58.58
0.55
dyMEAN
68.48
1.11
dyMEAN
83.09
0.66
GeoAb
GeoAb
H2
L2
AbX
70.73
0.76
AbX
84.53
0.45
AbNovo (base)
78.56
0.61
AbNovo (base)
88.25
0.32
AbNovo
76.60
0.62
AbNovo
88.05
0.35
DiffAb
28.05
2.86
DiffAb
47.57
1.39
dyMEAN
37.50
3.88
dyMEAN
52.11
1.44
GeoAb
41.19
2.57
GeoAb
H3
L3
AbX
44.18
2.50
AbX
65.89
1.21
AbNovo (base)
49.93
2.19
AbNovo (base)
73.88
0.86
AbNovo
48.55
2.38
AbNovo 74.45
0.86
optimization used previous works (Luo al. 2022 7hu al. 2024). We find that AbNovo out- performed other baseline methods in optimizing Rosetta Binding Energy, Evolutionary Plausibility (Table 6). the proportion of satisfying constraints ( AAR and RMSD (Table
Performance of Rosetta Binding Energy (left) and Evolutionary Plausibility (right) across different antibody optimization steps across DiffAb, AbX, and AbNovo.
Table 6:
Optimization Steps
DiffAb
AbX
AbNovo
4
-10.45/ 2.39
-8.80 /2.40
-21.02 2.39
-8.52 2.41
-2.64 2.44
-19.77 2.37
16
-7.18 2.42
2.07 / 2.42
-12.70/ 2.37
32
0.23 12.53
-3.05 2.44
-12.87 2.36
64
0.23 2.57
3.98 / 2.44
-12.87 2.36
100
-0.96 2.60
4.79 2.44
-12.05 / 2.36
Table 7: Proportion of constraint violations across varying antibody optimization steps.
Optimization Steps
DiffAb
AbX
AbNovo
13.2 %
14.0 %
12.8 %
8
13,9 %
22.7 %
7.1 %
16
13.6 %
22.5 %
6,5 %
32
15.7 %
21.9 %
4.2 %
64
21.5
23.0 %
2.6 %
100
20.8 %
23.5 %
3,9 %
16

---

Published as a conference paper at ICLR 2025
Table 8: AAR and RMSD across different antibody optimization steps.
Optimization Steps
DiffAb
AbX
AbNovo
0.88 1.09
0.80 0.97
0,85 0.80
8
0.76 1.59
0.59 / 1.51
0.66 / 1.34
16
0.48 / 1.78
0.49 / 1.54
0.51 / 1.46
32
0.39 2.05
0.45 / 1.88
0.50 / 1.66
64
0.30 2.69
0.45 / 2.33
0.48 / 2.03
100
0.28 2.86
0.44 / 2.50
0.49 / 2.38
A.2.3 DESIGN ANTIBODIES FOR UNKNOWN BINDING POSE BETWEEN THE ANTIBODY AND ANTIGEN
To enable antibody design in scenarios with unknown antigen-antibode positions and unknown antibody frameworks, we adopted previously established pipeline (Luo et al., (022) thtpps:
//githab. wew.thab.commleen wwtp:/cstt//////ccmm/ faaifffa/////ooooo seree/mainn //mmnnn)) ). Specifically, we used structure predic- tion method Chai-1 (Discovery et al. (024) and docking software HDock (Yan et al. (020) to predict the relative binding position of the antigen and antibody, followed by utilizing AbNovo for antibody design. As shown in Table g AbNovo demonstrated superiority among the comparative methods.
Table 9: Evaluation of design antibodies for unknown binding pose between the antibody and anti- gen.
Rosetta Binding
Evolutionary Plausibility
AAR
RMSD
Constraints
Energy
dyMEAN AbX
0.37
3.88
-1.7
2.82
94.5 %
0.40
2.83
11.22
2.54
39.39 %
AbNovo
0.44
2.59
-5.81
2.54
25.5%
A.2.4 EVALUATE PRE-TRAINED LANGUAGE MODEL
To demonstrate the effectiveness of structure-aware language model, we have evaluated it on CASP14, CASP15, and CAMEO (from 2022-05-01 to 2023-05-01) independent datasets. We fol- low previous work (Lin et al., 2023) and choose long-range (L >> 24) contact prediction accuracy (p@L) as the evaluation metric. The results are shown in Table I0
We observe that structure-aware language model significantly outperforms the sequence pure lan- guage model (ESM-2 3B).
Table 10: Evaluate pre-trained language model on CASP14, CASP15, CAMEO dataset.
Methods
CASP14
CASP15
CAMEO
ESM2-3B
0.37
0.44
0.51
Ours
0.58
0.59
0.75
A.2.5
CASE STUDIES
We analyze the metric distributions of antibodies designed by various methods for specific antigens. dyMEAN is not generative model and lacks the ability produce diverse outcomes, its results excluded from this analysis. As illustrated in Figure 3 AbNovo has better performance Rosetta Binding Energy, Evolutionary Plausibility, and the proportion of constraint satisfaction.
17

---

Published as a conference paper at ICLR 2025
DiffAb
AbX
AbNovo
2.5
24
24
24
Eveltoonryy Paustaly
Breuioonry Paustlll
22
22
2.2
2-1-
211
2.1
Evolutioary
1.
1.3
1.9
1.8
1.0
1.8
Rosetta Binding
Roselta Binding Energy
Roserta Binding
2.8
2.7
Eedutorrn Plaeebblly
25-
Eedutoran Plusbbltty
2.6
2.5
2.6
24
2.4
2.4
2:3
2.5
2.3
21
2.1
20
2.0
-85
-20
-15
-10
10
-25
-20
-15
-10
-30
-20
Rosetta Binding Enengy
Roselta Binding Eneegy
Rosetta Binding Energy
2.7
2.7
2.7
Exclutiran Plussbity
2.6
2.5
2.4
2.4
2.4
2.3
23
2.2
2.1
2.1
2.1
Energy
Enengy
Binding Energy
Natural antibody
Antibodies that satisfy all constraints
Antibodies that violate constraints
Figure 3: Distribution of Rosetta binding energy and evolutionary plausibility for 100 antibodies designed against the different antigens (PDB:4fqj. la2y, and 5nuz) using various methods. The red star denotes the characteristics of the natural antibodies. Antibodies that satisfy all constraints (Stability, Self-association, Non-specific Binding) are marked in blue, while those that violate the constraints are marked red. The yellow regions highlight areas where the binding energy and evolutionary plausibility metrics exceed those of the natural antibody.
A.3
ANALYTICAL RESULTS
A.3.1
STRONG DUALITY OF CONSTRAINED PREFERENCE OPTIMIZATION
This section demonstrates that strong duality holds for the constrained optimization problem (Equa- tion 3). This material is from previous works Ito & Kunisch (008); Bertsekas (2014): Huang et al. 2024 include it here only for completeness.
The Slater condition is an important concept primal-dual algorithms and mainly used to de- termine a strong duality of problem. Specifically, the Slater condition requires the existence of a strictly feasible point in the constrained optimization problem, i.e., a point that satisfies all the inequality constraints and all the inequality constraints strictly hold. If Slater's is condition is satisfied, is usually possible to ensure that the optimal solution of the primal problem is equal to the optimal solution of the dual problem, thus achieving strong duality.
18

---

Published as a conference paper at ICLR 2025
Assumption 1 (Slater condition, Striet feasible) This exists policy p and such that J[ (p. <.
In practice, this is possible because often know the optimization strategy's 's strictly feasible lution, so we can relax the constraints appropriately according to the application scenario Liu et al. 2024). Similarly, in this manner, AbNovo can satisfy the Slater condition.
Proposition 1 (Strong duality of constrained optimization) Under Slater condition (Assumption 1). there is duality gap for constrained optimization problem. Let p" be the optimal pri- mal policy such that p = arg max J(C) (pe, 2). Let X to be the optimal dual variable where
X arg min G()). Then (y","*) is saddle point of the Langrangian function: X20
max min (18) A) (p=) x²)
min max (( A).
(13)
Pe
AER
XER Po
Specific proofs can be found in previous works (Liu al. 2024
Huang al. 2024).
A.3.2 CONTINUOUS REWARD PREFERENCE OPTIMIZATION FOR SCORE-BASED DIFFUSION MODELS
In this section, derive the training objective for the score-based diffusion model using contin- uous reward from the objective form of RLHF. Following the approach of previous work (Wallace al. 2024), we first define a reparameterization of reward. Subsequently, based on this reward, can obtain the reparameterization of fo as described in (Chen et al. 2024a).
We simplify the notation in this section using q, po, and Pref represent gh respectively.
Pg 2.r.0 and Pref
We define rm rm. (T:1:)) Prc) and e, (T:1))) (00::)),Prr Prc) as the reward function and the constraint function on the whole trajectory of diffusion process, such that we define R,, (T()), Prc) and C, (T Prc) follows:
R. (T)()))), (T()),FF))) PFc) = Er po (T(Am) ((oi) I'm (T(::)) PFo)]. ( ET (T(M))) ( Pro) Icn (T:11) Pro)..
(14)
where (0:1) (T) ( T()) means the inference trajectory of diffusion process. Then, we only consider per PrC for simplification and the objective of AbNovo can be written as follows:
Σ wm Ry (m((0) PFc) Σ (T! PFc) + EnnOn
max
(Tio) (T
(15)
- (To (Pcc).
From Equation we can obtain R)) PFc) as
(T (FC)) (T (T
(TaU))T Pro) (T(((.))))))) PFc)),
(16)
where (0:1) PFc) is defined as
(01). PFc) = Σ T'
Cennmm(T(iin (T(0:)) (r() - Σ (T(:)) PrC).
(17)
m
19

---

Published as a conference paper at ICLR 2025
Then, the objective of 15 can be transformed to the following form:
min Ero pa (Poo)) R( Prc) Σ CnnCo]18
+ DRL (po (T(() (PF() |Iret (T (Pr)))
R(T Prc) + Σ An
< min Po
poo (T)) (TT)))
+ DKL (pe (T (0:1) (Pc))lPee (
(T(O:)) PPC))
((T PFc) /8
min TToa) pp po (Tom)) (Pr)))
+ (Po (T (0:1) (P:)))(()l (T(0:1) (PF)))
3 Σ On
po (0:1) Prc)
min (Tpaa po (Tea)) (Prol)
log
Pref (T(0:)) (PF))
exp (f(T((:1) PFc)/
log ((PEc)
Σ An
B n
(18)
where Z(Pro) = ET P. ( (re) (xp(f(((O.1))PFe)) and C, independent 8. The
optimal 1 (Prc)) of Equation 18 has a unique closed-form solution:
PT (0:11) (PF)) Prer(T (0:1)
(Prc) exp(f(((()1) Prc)/3)/Z(Prc).
(19)
Therefore, we have the reparameterization of ooffffTToo ((T(1:1), PFc) as follows:
pr (T(:))))
(0:1) PFc) = =Blog
+ 3 log Z(Prc).
(20)
Prof (
(Pc))
Plug this into the definition of R)( PFc) of Equation
16 hence we have:
R( PFc) (p Pre) log Pă (T(()) (PC)) + Blg Z(Pro). (21)
Pref (TO.))
Then by Equation (15) in (Chen et al. 2024a), the fa can be defined as follows: Pe (T((1))) (Pc))
fo (T()), PFc) (ppaa ( PFc) log
(22)
Prof (T(1)) (PC)
Here, considering that the computation of ((PFc) is intractable, (paa (Po) eXp(R((T) (PPc)/ to simplify the computation.
we approximate it by
Substituting this reparameterization of So into Equation (16) Chen al. (0224a), we obtain the objective of diffusion- NCA:
NCA (0)
Σ
exp (R./3)
log ( K log (-Jo(T;") :Pcc))
K
exp (R
1=1
K
exp (R,/3)
Po (T (Pp))) log
log σ
By wpo Pac)
Pret (Te
(0:1) (T.(0:) |PFc)
1=1
exp (R
po (o (T(():)) Prc)
+
loga
-B ~po [0) Peal
log
K
Prot (T:1)) (PFc)
(0:1)
(23)
20

---

Published as a conference paper at ICLR 2025
Since sampling from pe T
(At:1)
T
(0)
PFc) is intractable, we utilize q (A:1)
( Prc) for ap-
proximathon Wallace et al. 2024).
Adid
NCA
(0)
K exp (R./3)
log Byrann Te pT! pPc)
(T (T. log
Pret (T? (0:1) |Prc) 1)
Po
PPc))
exp (R₂/3) 1=1
1=1
1 logo
pa (T (0:1)
(PFC)
-Biptaru / pr. .Po)
log
K
Pref (0:1)
(PF))
exp (Ri/3)
Ps (T! (t- (T//-- PFC))
Σ
logo ETTASNN mo ((( PFc)
Σ tefAr ....)
log
K exp (R//3)
Pref (TT ( (T PFc)
==1
po (T - At) T PFC)
(T()) PFC)) 1)
1
logo
- (e (0)) .Pe))
Σ
log
K
(T
(-20)
(E(De, ...1.
Pref
K
exp (Ri/3)
po (T ((t))))
At) T PFC)
log Bygsnm (( pr prc)
E,
log
At
Pref (T (1- (
(*) PFC)
Am
exp (R₂/3)
1=1
1
pa (t At) (T( p
ft- At) T PFc) 1)
PFC)
+
log
-Bypjaci (((Taait))) (ooo PPc) At
Er log
K
Prot (T
exp (R+/3)
Σ
log
At
E.
E.E
( ( .Pre)
exp (R
1=1
Pa (t- ( (T(ht) PFc)
E B (t) (() ( Pre)
log
Prot (T! !T! PFc)
1
logo (- (A At
B.
E,
T ~a( (T)
K
Po ( (T()) PFc)
E E 0t) ( (p(() T! Pre)
log
Pret (T (T((-_1) T PFc)
(24)
21

---

Published as a conference paper at ICLR 2025
By using Jensen's 's inequality, have:
exp (Rt/3)
Aiif NCA
(0) S
E, - Prc) log
K Σ exp (R//3)
1=1
]=1
pa (T. [t- (t) ( PFc)
E p - - Ngg , Pro)
log
At
Pief (T! (t-)r) |T( PFc)
+ K
E, E p (C((" ( PPo)
log
log (T (t(tt-A))) A1) (T PFC)
Af Tip an) mTT au p ( PPc)
Prof (T (1-1)) T PFC)
exp (R./3)
E, E (T (T(! (T) Prelli
log
exp (R₂/3)
1=1
pol (T (t- Ar) ( PFc)
E (((()) T ( PPo)
log
At
Drof (T (T( PFC)
1
(- At
log pa (T (1- 21) (T! PFC)
K
log
BTp
au
~(((( pu T PPe)
Pret Pret (T((( T!! Prc)
(25)
With some algebra, the above loss simplifies to Equation 9
A.3.3 OFFLINE DUAL GRADIENT ESTIMATES
In this section, we introduce how to use offline data estimate the gradient of the dual function.
From Equation 11
the gradient of the dual function (()) can be calculated follows:
AG(A) dx
dJ
A)
dp
(1V) A)
dpp
dA
dX
(px:)
dA
((()) - ((e)
(26)
Yd
aJlo A) AA
C
(PP))
EppeED, (To) (PO)
where A (di, )2, An). An). After updating policy, the optimal solution of Equation 9 under À is
px x Prof exp(R)3).
(27)
Thus, Equation 26 could be written as
Spa Poo) (xpp(R/3)(C C((((), PEO))) (PET))
dg(A) dX PPCEDD
(28)
E mprer (T1)) Pro)) cxp(R/3)
Then, the gradient of (()) could estimated by offline data, since Equation 28 only correlated by Pref- Therefore, large offline dataset D, ([R... (T()) j.k: Pro.A), C. (T Pro, M/
22

---

Published as a conference paper at ICLR 2025
(for K antigens, J antibodies sampled for each antigen) to estimate the gradient. The specific form can be written as follows:
dg(X) dx
1 Dgl
softmax ({R,/18) j-1) C(T Pro,)) Pro,k) + Cavg + C,
(29)
where R is defined as Equation 7 C = Cp....,CCl is the constraint functions, C = [C1,..., ..., Cp. is
the thresholds for different constraints, and Cavg the global normalization term.
Here, we use {-}: i to denote the i-th element of the vector. the current Aラ is higher, while the model already satisfies the j-th constraint. Thus, higher Xj will result in greater weight for samples with high Gj, and { [ }, is larger. Eventually, the gradient of the X will increase. Conversely, if the current >j is low, while the model appears violate the j-th constraint. Such lower X will result in a smaller weight for samples with low Gy, and { d 1; will be smaller, and after being normalized, the gradient of the X3 will be negative at this step; the next step will increase the Ay.
A.4 ABNOVO BASE MODEL
This section introduces AbNovo's training strategies, training losses, and inference processes.
A.4.1 PRE-TRAINED LANGUAGE MODEL
We targeted masking 10% of the protein sequences using the BERT (Ln et al. 2023) model. Specif- ically, 85% of the masked positions were replaced with the [mask] token, 10% were substituted with random amino acids, and 5% remained unchanged. Additionally, for protein sequences longer than 200 residues, we randomly masked consecutive amino acid segments of length between 5 and 13. Concurrently, predicted the distance matrix of amino acids using the distogram loss Cdistogram for input sequences containing masked regions. We also employed a contact prediction loss function Ccontact, considering amino acids with distances greater than Á as long-distance contacts. The ESM2-3B model (Lin et al. 2023) used as our model architecture, with its pre-trained weights serving as initialization.
A.4.2 DIFFUSION PROCESS DETAILS
Following the setup from previous work (Campbell et al. 2024), our noise schedules are indepen- dently tailored for each of the three diffusion processes (Table 11
For diffusion computation and parameter settings, we follow previous works (Yim et al. 2023 Zhu
al. 2024
Table 11: Noise schedule settings.
Translation
((t) = Buin + tB3max Buin) (tt) log(t exp( 0 maxx) + t) exp(Amiaa)
Amin 0.1; Bmnx
20
Rotation
Omin 0.01; mmax 2.25
Sequence
axl) 3(1-1]
A.4.3 ABNOVO MODEL ARCHITECTURE
Networks. AbNovo adopts a network structure similar to ESMFold (Lin et al. 2023). com- prising pre-trained structure-aware language model, 4-layer Main Trunk, IPA, structure encoder and a sequence decoder. Our sequence encoder is structure-aware language model with 33 Trans former r layers, while the structure encoder is Linear layer. The sequence decoder maps the single representation obtained from the Main Trunk to amino acid species, consisting of three layers of MLPs. The specific network dimensions provided in Table [4
The model architecture of AbNovo is depicted in Figure 4 Consistent with prior studies (Jumper
al. 2021 Lin al. 2023 Zhu et al. 2024 Abramson et al. 2024), we find that employing the
23

---

Published as a conference paper at ICLR 2025
recycling technique can further enhance performance. The recycling features include the predicted
antibody sequence and the antibody distance matrix.
Antigen
Antibody Framework Structure Noiey CDR Structure
Antigen Structure Antibody Framework Structure Denaised CDR Structure
IPA
Evoformer
Sestrooee
Trunk
Wi XXXXHXX
dV. ELK
Structure-sware Language
Sepcarader MLP
VE CANGGGG VSA
DV. AEWAATYC..CC....
Antigen: ET. WWW.ANKE.LLBBB..
Antigan: EF WWKE ANDV.
Fasmaraa secustic
Ondioer de vo
Sequancs
Recycling limes
Figure 4: Encoder and score network of AbNovo.
Input Features.
The specific details of the input features shown in Table 12
Table 12: Input features of AbNovo.
Modules
Description of Input Features
Sequence Encoder
1. Sequence of antibody framework, antigen, and noisy CDR
Structure Encoder
I. Distance matrix of antibody framework, antigen, and noisy CDR
2. Backbone dihedral angles, including á, and in sine and
cosine form of antibody framework, antigen and noisy CDR
Others
1. Chain ID of antibody and antigen
2. Time embedding of Sequence and Structure
3. Relative position embedding
Table 13: Network hyper-parameter of AbNovo.
Description
Value
dimension of single representation in Evoformer Trunk
512
dimension of pair representation in Evo former Trunk dimension of single representation in IPA dimension of sequence decoder
128
384
512
A.4.4 DATASETS
Datasets for Structure-aware Language Model. In the training process, used all the proteins from the Protein Data Bank (PDB) before 2020-01-01 for training. except that we filtered some antibody data. For antibody structure data, filtered for the CDR H3 region in the training set with more than 40% sequence similarity with the test set. In the training process, we followed the data processing method of AlphaFold2 (Jumper et al. 2021), and we used MMseqs for 40% sequence similarity clustering.
For proteins AlphaFold DataBase (AFDB), we masked all amino acids with PLDDT of less than 50 and removed proteins with an overall PLDDT of less than 70. During the training process, we maintained a crop size 512 and kept the sampling probability of the PDB training data and the training data from the AFDB at 1:4. We used the parameters of the ESM2-3B (Lin al. 2023) model as initialization.
Datasets for Training base model. We follow the list of provided training from previous work (Luo et al.) (022) in infittss///ithub sabdab summary all.tsv.
Datasets for Preference Optimization. Following previous works (Zhou et al. 2024). we con- struct preference-optimizzed datasets for antigen and antibody complexes. We generate 512 antibod-
24

---

Published as a conference paper at ICLR 2025
ies for each antigen in each preference optimization process, and then the metrics are calculated for these antibodies.
When updating A, randomly selected 10000 (for 50 antigens and 200 of each antibody) antigen- antibody complexes to estimate the gradient.
A.4.5 TRAINING LOSSES
Training Losses for Structure-aware Language Model.
We use Mask Language Model Loss (Lin et al., 2023), distogram loss (umper et al. 2021), and contact prediction loss to train our base
model. The specific loss function follows:
Cpretrain = 0.5LMLM + 1.0ddistogram + 1.0Ccontact,
bm32
1
Σ
Vig log pij.
distogram
1
(30)
1=1
1 ΣΣ
Ccontact
I(di < 8A) log
L
bel
Distogram loss (Liistggrm is the prediction of distances between amino acid pairs. Specifically, we divide the distance from 2 to 22A into 32 bins and predict in which bin the distance the amino acid pair lies, where p is the probability of each bin. Contact loss (Lcontact) is the prediction of whether amino acid pairs are in contact or not (whether the distance is less than SA or not), where Ω E fcontact, not contact].
Training Losses for Base Model. We use four loss for training AbNovo (base) c(a), c(r), C(a), Lviolation and Laux- Following previous works Yim al. 2023 Campbell al,
2024). the denoising score matching (DSM) of different modal can be written as follows:
I.
V, log Pog (F)1) - V.Puo(r; (T)
ar
L
(x))
Σ
|
(0)
I;
1
(0)
(31)
L
t=1
((a) Σ
CrossEntropy ())..
We also incorporate violation loss from AlphaFold (Jumper et al., 2021 Abramson al.) 2024 to learn the geometry for inter-residues bonds and avoidance of atom clashes. Specifically, these losses can be written as follows:
Nhande
max (laassigo
Ci Tbondlength 0),
bondlength
Nhonds
(32)
Nanules
(cosonhosige - cosojil Tbondaugle: 0).
Lboudangle
max
N Nangles
1=1
Where Edesign and design are the bond length and bond angle of i-th designed antibodies. re- spectively. fi and cosa is the literature value for this bond length and bond angle. We use same tolerance value (Thondlength and Toondangle) AlphaFold.
Additionally, we add auxiliary loss for training (Yim et al. 2023). Laux is an mean squared er- ror(MSE) loss supervising the distance of designed CDR's four atoms Ω {C, C., N, O). This can be written as follows:
Laux
1 4L
min Σ
Σ acn
I i.a
(.(() 11, dalam)
(33)
1=1
25

---

Published as a conference paper at ICLR 2025
Training Losses for Preference Optimization.
From Equation
F(-) can be written as follows:
F()) = DRx.. ( (T (0.00
(T (1 (T!o))
+ DKL q (Ta) A1) (T Ip a (
= a² [DK (g= (x((t__A) x
)1pb (x: tt-
1125
(x())))
- BKI. 0² ( (x0 (x, ////0///00 ]]
a² [KKI (9" (r"(x:" (x (2-0.0) )1p((1, (1-1-)))))1 (r!0))
- -Kkk (r( (t- At) |r! (0:1) (1:00)))ppp I1pprer (I.
(11 (r!"))
(rx
a. DKL (((a(t- (0.0) 1lp (0 (a("1)
- DKl. q8 (a! (1-At) 1a (0.1) (1)00)))12 Ilefer (a. (a))) )[
(34)
where x; 1 ..., mm r, [n=+1.i fn,,, and a, ii..,, mm+.m. the first subscript represents the i-th sample in the preference optimization and the second subscript repre- sents the j-th amino acid in the CDR. With some algebra, these KL. divergences can be derived as the following form:
According to Wallace et 2024 the closed-form expressions of KL divergence in R simplifies to:
KL Divergence in R Space.
D KL.
[x (0) x (0) 12 CC
+
(35)
KL Divergence in SO(3) Space. Similar to Zhou et al. (2024). we approximately derive an empirical reconstruction loss in SO(3) as:
D V| log (10 (r))) VrPro(r"') +1)) C
(36)
KL Divergence in Discrete Space. derivation process:
For sequences in discrete space, we give the details of the
Dia DKL (a a (150aaaa
log p/ (a (a (2())
+ C1
log
q (" 101) a (p +C1
(00)
(a |a 9
log
pg (((a1)a(()) Ci
go (a(1())a((1)
< (appaat-a0// (a log (" 2( )1)
po (a (a + +CI
(a (a
E Ey ( a00) m(1))
log pz (a))aa())
((aatt
+ Ea al a0)))
log q (a aa()) + Ci
= log (a (a))) +
(37)
In practice, we choose (ar( (r)) (1.00)0 0.2], (sup) = 0.5 and K = 8. Meanwhile, we add the regularisation term a for 1 in Equation to ensure the stability of training. Here, we choose (R) / 10.0.
Ar
26

---

Published as a conference paper at ICLR 2025
A.4.6 TRAINING DETAILS
We show information about the training process, objectives, and learning rate of AbNovo in Table
14
Especially in the fine-tuning stage, for updating À for step, we update the policy for 100 steps. We show details about losses we used when updating policy and X in 3.3.1 and 3.3.2
We use Nvidia A10 (80G) for training, and the batch size is 128 for all training stages. For all training procedures, we use the Adam optimizer for training with default parameters.
Table 14: Hyper-parameter of AbNovo.
Training steps
Learning Rate
Stage
Training objective
Dataset
AFDB (2M)+PD((fltter
Pre-trained
LMIMM + Liistggram Leontact
200k
5e-5
1.0 0 + 0.2C(a)
Base model
20k
lc-4
V.i 1.0Caux
complex
Fine-tuning
Lupdate policy (Cquation[O)
20k
2e-5
Preference dataset
A.5
EVALUATION METRICS
In this section, we provide detailed descriptions of each metric.
Rosetta Binding Energy. For Rosetta Binding Energy, we followed previous work (Zhou et al., 2024 by calculating the energy between the antigen and antibody using Rosetta Software. Lower energy values represent higher antigen-antibon affinity.
We first used Rosetta software perform side-chain packing on the backbone and side chains of the designed antibodies, followed by FastRelax on the side chains. Then, we calculated the Rosetta Binding Energy. We denote the residue with the index i in the sutibody-antigen complex A, then A and Abh represent the side chain and backbone of the residue respectively. We use EP to represent the interaction energies between Paired residues, which consists of six different energy types: Enbond, Eatt, Eanl, Eelves Enk. Erep-
E Σ Σ
EP rop (A,. A + EP Errop (A.5 Abb)
JECDRs iEAntigens
+ 2EP rop (A)b. Ag))
2EP rep (A/b, AAbb)
(38)
Σ Σ
Σ
EP EP (49), EP (A7 (Ab))
JECDRs Antiigees ef eloc, rep)
The calculation incorporates various energy terms from Rosetta, using the default weights in ref2015 for each term.
Evolutionary Plausibility. Evolutionary Plausibility measures how likely designed sequence is cvolutionarily plausible in nature, reflecting adherence to general evolutionary rules of natural proteins. Recent studies show that large-scale protein language models, trained millions of nat- ural protein sequences, effectively capture these evolutionary rules (Shuai et al.] 2021] Hie et al.) 2024). Following previous works (Zhu et al. 2024 Zhou al. 2024). we evaluate Evolutionary Plausibility by calculating the perplexity of the antibody language model (Shuai et al.) (02T for the designed antibodies. Here, we use a; represent the residue type of designed CDRs. P represents the conditional probability of BERT predicting the word at masked position i, where represents the model parameters. The specific formula as follows:
Evolutionary Plausibility
Σ iECDRa
log Playla,,... 0;-1; 1..1 aN)
27

---

Published as a conference paper at ICLR 2025
We used the script provided by IgLM Shuai et al. (2021) to calculate this metric.
Stability, Stability measures the stability of the conformation of the designed antibody in isola- tion, without the antigen structure involved. This metric differs from Binding Energy, which eval- uates the interaction between the antibody and the antigen. Following the evaluation approach of prior work (Zhou et al. 2024) [i et al.] (024), we used Rosetta to calculate the total energy of the designed antibody to assess its stability.
We performed side-chain packing and FastRelax on the backbone and side chains of the designed antibody. Then output the total energy of the CDR regions using the Rosetta software.
The self-association of antibodies refers to the tendency of antibody molecules to bind to each other without interacting with the antigen. Self-association can negatively impact the stability, function, and biological properties of antibodies, especially in the context of therapeutic antibody development, where self-aggregation is generally undesirable (Yadav et al. 2012 Kanai et al. 2008).
Self-association.
Previous studies (Makowski et al., 2024) have shown that the occurrence of self-association is closely correlated with the negatively charged patches area in CDRs. Therefore, we use nega- tively charged patch areas in CDRs as proxy for evaluating the risk of self-association. Al- though other physicochemical properties are also correlated with self-association, we used the physicochemical property of CDRs with the highest correlation as the evaluation metric. We used the pipeline provided by previous work (Makowski et al.) 2024) (https: //gi thub. com/ Tessier- Lab UMi wwwcch//ccpeeee Bttp:/CCT_roo_pp__PP ) to calculate the self-association metric.
Non-specific Binding. Non-specific binding refers to the undesirable interaction of antibodies with cellular proteins other than the intended target, particularly membrane proteins of the cell (Makowski et al.) 2022). In practice, evaluating non-specific binding would require consideration of interactions with all membrane proteins and RNA-specific binding properties. Computationally modeling these interactions for all potential targets is highly complex and challenging task. Here, we followed the metric proposed in previous works (Makowski et al. 2024 2022).
Previous studies (Makowski et al. binding and the hydrophobic patches area in CDRs. Therefore, we use hydrophobic patch areas in CDRs as a proxy for evaluating the risk of non-specific binding. Although other phy physicom cal properties also have correlation with non-specific binding, we used the most strongly correlated physicochemical property in CDRs as the evaluation metric. We used the pipeline provided by pre- vious work (Makowski et al. 2024) (https: / //gtttb........ www..ooomessseeeee Lab--Labbuuln UMi CMaaa/-Miiih/SS Prop_Opt_Mir ML) to calculate the non-specific binding metric.
2024
2022 have verified correlation between non-specific
RMSD. For DiffAb, AbX, GeoAb, and AbNovo, where the antibody framework structure is provided, we directly calculate the RMSD) of the designed antibody. However, for dyMEAN, as there is no given real antibody framework, we use Kabsch alignment to align the designed antigen- antibody complex with the natural complex. Then, we calculate the Root Mean Square Deviation (RMSD) for each region of these aligned complexes.
Reward and Constraints Settings. Given the stringent requirements for compliance with clinical applications, well as the inherent biases in hydrophobicity and polarity metrics, our primary objective is to filter out antibodies that clearly violate binding rules while retaining as many potential candidates possible. Setting an excessively strict threshold would, the one hand, eliminate many viable candidate antibodies and, on the other hand, destabilize the model's ''s training.
Consequently, heuristically determined the threshold for each constraint based on training sta- bility and the empirical thresholds used in previous studies.
Specifically, we determine the thresholds based on the dataset provided in previous work (Makowski let al. 2024). This dataset comprises 80 clinical antibodies annotated with corresponding wet-lab experimental results. For non-specific binding and self-association metrics, calculate the physic- ochemical properties of all antibodies that meet the requirements, setting the threshold for constraint violation as twice the highest value of the physicochemical property observed in the antibody that satisfies the wet-lab metric.
28

---

Published as a conference paper at ICLR 2025
For stability assessment, evaluated all antibodies in RAbD and set the highest energy value as the threshold for constraint violation.
In experiments, considering the different magnitudes of various rewards and constraints, normalized all rewards and constraints during the training process. Mcanwhile, we set the initial X to [1.0, 1.0, 1.0] and the reward weights w1 and w2 1.0 and 1.0.
A.6 BASELINE METHODS
DiffAb Luo et al. 2022). We use the test script and the pre-trained model provided in the GitHub All hyper-parameter
are default.
dyMEAN (Kong et al.
2023].
We use the pre-trained model and test script provided in the GitHub All hyper-parameters used are
repository (https: /gtgttttttb..g.ggggg Swtm/TTUNLPMMT/dyae
default.
GeoAb Lin et al. 2024). We employed GeoAb from its GitHub repository (https:/)
github com/ www.///aaeeeneeeeeeeeeeee EdwaiieannnnteeeeBee..... with all default hyper-parameters and the pre-trained model provided.
We use the co-design test script and pre-trained model provided in the GitHub repository (Bttps://oithub.ccm httpa//p......aaaoootttttaaaaaa.
AbX (huu et al. 2024).
All hyper-
parameters we used are default.
AbDiffuser (Martinkus al. 2024), AbDPO (Zhou et al. 2024), and AbDesigner Tan et al. 2024). These models were not available at the time of this paper's submission. Moreover, the re- sults presented in their papers not compatible with experiment settings. For example, AbDPO only designs CDR-H3; AbDiffuser does not provide six CDRs designed results simultaneously; and AbDesigner different test set. Thus, we do include their results in our paper.
29


Reviews:
[{"review_id": "QwbFFy7OVU", "reviewer": "Reviewer_neD7", "summary": "This paper focuses on some important properties, such as non-antigen binding specificity and low self-association, and optimizes the model in a DPO-like manner. What differs it from other DPO-based methods lies in two forms, the optimization targets and continuous rewards. With a two stages training framework, the proposed AbNovo is capable of capturing generalized protein information and constraining the generated results with desired properties. Experiments also support the effectiveness that generated antibodies are well designed.", "questions": "1. In the visualization part, I don't see why results come from dyMEAN and DiffAb do not satisfy constraints like Stability, Self-association. Can you explain this in detail?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Multiple objects are considered to improve the quality of generated antibodies. Although not validated in wet lab, these kinds of properties are essential.\n2. This work does not simply integrate DPO only optimizing binding affinity, which broadens the horizons for similar works.", "weaknesses": "1. Rosetta energy is used as an alignment metric. It is well-known that forcefield energies have a weak correlation with measured binding affinity, typically around 0.3 [1,2]. This may lead to the totally wrong direction.\n2. Limited antibody optimization experiments, which should be a major highlight of antibody design. Maybe some further experiment may alleviate this, like in [3,4].\n\n[1]Luo S, Su Y, Wu Z, et al. Rotamer density estimator is an unsupervised learner of the effect of mutations on protein-protein interaction[J]. bioRxiv, 2023: 2023.02. 28.530137.\n\n[2]Ambrosetti, F., Piallini, G., & Zhou, C. Evaluating Forcefield Energies in Protein Binding Studies. National Center for Biotechnology Information, 2020.\n\n[3]Kong X, Huang W, Liu Y. End-to-end full-atom antibody design[J]. arXiv preprint arXiv:2302.00203, 2023.\n\n[4]Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma. Antigen-specific\nantibody design and optimization with diffusion-based generative models for protein structures.\nAdvances in Neural Information Processing Systems, 35:9754–9767, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FCFPAXcPlG", "reviewer": "Reviewer_vnHk", "summary": "This paper presents an antibody design method, AbNovo, achieved antibody through multi-objective optimization. By introducing a structure-aware protein language model and employing constrained preference optimization with continuous rewards, AbNovo surpasses previous methods in both reference-based metrics and reference-free metrics (i.e., biophysical properties).", "questions": "1. The announcement of \"The first deep generative model for multi-objective antibody design\" in summarized contributions, AbDPO also supports multi-objective optimization.\n\n2. In energy evaluation, if you want to assess the energy performance of the designed backbone, energy minimization is necessary for the side chains while keeping the backbone structure unchanged, and then calculate the energy. If you wish to evaluate the antibody's performance in real experiments (which implies the CDR region's structure might not maintain the designed configuration), you can use multi-chain supporting folding models like AlphaFold3 to predict the binding structure. When calculating energy, does the relaxation you used optimize only the side chain conformations, or does it also alter the main chain structure? If it's the latter, are these experiments intended to demonstrate that AbNovo can generate a better initial structure for Rosetta relaxation?\n\n3. Does the optimization of these physical properties contribute to some chemical validity? For example, does the peptide bond length get closer to the actual length?\n\n4. The standard deviation of the physical energy needs to be presented.\n\n5. The AAR performance is excessively high, and it's necessary to check whether the training data of the protein language model contains samples similar to the test set.\n\n6. I am curious about how many amino acids have mutated in those designed antibodies that outperform natural ones (at least in binding energy).\n\n7. The task setting of dyMEAN is different from others, including AbNovo. dyMEAN does not provide the real FR structure, making direct comparison somewhat unfair. Additionally, how is it achieved to use dyMEAN to generate 128 antibodies for an antigen?\n\n8. Calculating RMSD on the aligned structures seems somewhat unreasonable. Typically, for two rigid bodies that can freely undergo SE(3) transformations, alignment is performed first, followed by RMSD calculation. However, in the setting of this paper, the FR region is given, meaning the CDR region cannot undergo SE(3) transformations independently, thus requiring a direct RMSD calculation.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Achieved performance on physical metrics that significantly surpasses other methods.\n\nIntroduced a structure-aware protein language model and demonstrated its usefulness for antibody design.\n\nProvided rigorous theoretical derivation", "weaknesses": "Seems to be an updated version of AbDPO, somewhat heavier but showing better performance.\n\nThe task setting is overly simplistic. Although the structure of the antibody's FR region is relatively conserved and can be considered known, the binding pose between the antibody and antigen is typically unknown. However, given that the main goal of this work is to propose a new method for antibody optimization, this limitation is understandable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OoZw5J6FaF", "reviewer": "Reviewer_fXjU", "summary": "In this manuscript, the authors present AbNovo, a method combining constrained preference optimization with generative models for multi-objective antibody design. First, an antigen-conditioned generative model is trained to co-design antibody structure and sequence. Then this model is fine-tuned to maximize binding affinity to a target antigen while enforcing constraints on properties such as non-specific Binding, Self-association, and Stability. In their experiments, the authors compare their method to many recent works and show an improved performance.", "questions": "•\tin Section 4.2 you state that when “we incorporated all constraints into the optimization objective by taking a weighted average” a “drop in performance” is observable. However, the corresponding results show an improvement wrt. the “All Constraints” metric. Could you elaborate on that? \n  \n•\tIn Table 2, we can observe that AbNovo (base) sometimes exhibits favorable scores than AbNovo. Is there a tradeoff between fulfilling constraints and achieved AAR/RMSD? \n  \n•\tIs there a reason dyMEAN is not included in Figure 4 and AbX not in Figure 2 respectively?\n\n-------\nPost rebuttal:\nAll points were well addressed. Based on this, I am increasing my score accordingly.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "•\tThe authors provide many theoretical derivations and analysis. \n  \n•\tThe authors include many baselines for their experiments which shows the good performance of their proposed method.", "weaknesses": "•\tI would suggest introducing a background section, as there are many things in this manuscript that would benefit from a proper introduction.\n \n  \t∘\tbase model, reference model, policy model could be introduced, e.g. with an intuition. These are introduced in Figure 1, but do not come with a description on how they are related. Only in Algorithm 1, the reader is shown that those are updated iterations of the very same model.\n\n  \t∘\tdelta and G in Equation 1 is never introduced, but instead taken from Campbell et al.\n\n  \t∘\tCTMC - Continuous Time Markov Chain is never defined.\n\n  \t∘\tThe notion of time t in diffusion processes used in the manuscript, t in U([0, 1]) is based on the CTMC definition by Campbell et al. but differs to that used on many other publications, e.g. [1] J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models”, [2] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep Unsupervised Learning using Nonequilibrium Thermodynamics,”. Thus, I would recommend introducing it e.g. as being in [0, 1] in line 185. Instead, this is first done in Equation 9.\n\n  \t∘\tT^(0:1) as a diffusion path is first defined in line 284 even tough being used many times before.\n\n\n•\tThe evaluation metrics remain unclear even after reading the appendix A.3. This holds especially for “Evolutionary Plausibility”, “Stability”, “Self-association”, “Non-specific Binding”.\n  \n•\tWhile the manuscript goes in great theoretical detail, intuition is often lacking. E.g. Equation 3 is introduced but an intuition, “first term maximizes rewards, while second term keeps the model close to the reference model.”, which could facilitate understanding for the reader is missing. \n  \n•\tMany things necessary for fully understanding the paper are moved to the appendix, resulting in decreased readability. Further, this also applies to some of the most interesting results, e.g. Table 9 and especially Figure 4. \n  \n•\tSome tables are hard to read, as their caption and corresponding text do not exactly describe what is in the table. E.g. \n  \n  \t∘\tIt is unclear what “reference“ in Table 1 describes.\n  \t∘\tIn Table 3 the reader must guess that “ESM-2 based” refers to “utilizing different language models” from the text and “Multi-objective” refers to “we incorporated all constraints into the optimization objective”.\n\n•\tIn the abstract and introduction, a focus is put on “alleviate overfitting issues due to the scarcity of antibody-antigen training data”, but no analysis supporting such a claim is included. \n  \n•\tThe analysis of the “impact of utilizing different language models in training the antibody design model” is very short and not well described. \n  \n•\tFigure 4 is a very interesting figure which summarizes the capabilities of DiffAb, AbX, and AbNovo very well and highlights that AbNovo “performs best“. In there, we also observe that only a single antibody generated by DiffAb against 5NUZ does violate constraints. Therefore, it seems inadequate that the visualized antibody for DiffAb in Figure 2 is a sample which does not fulfill all constraints. Furthermore, the DiffAb sample with “Rosetta binding energy: -2.12, Evolutionary Plausibility: 2.60” violating constraints cannot be found in Figure 4. \n  \n•\tSome claims appear exaggerated:\n  \n  \t∘\t“the first deep generative model for multi-objective antibody design, which explicitly optimizes multiple biophysical properties crucial for real-world antibody development.” There have been previous works which analyze the multi-objective setting for generating antibodies, e.g. “Pareto Front Training For Multi-Objective Symbolic Optimization\" by Faris et al. which train a algorithm to optimize a pareto front of sequences regarding the objectives antibody binding quality, stability, and humanness. Perhaps the claim can be weakened or reformulated?\n\n  \t∘\tAbNovo is “bridging the gap between in silico design and practical application.” seems a bit too strong given that no practical application is contained.\n\n•\tTypo “Bolocks” in Figure 3 \n  \nIn summary, I think this manuscript offers valuable new ideas but suffers from not being self-contained, sub-optimal readabilities and depth of analysis. I hope these issues can be addressed in the rebuttal and would love to increase my score in response.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)