Gradient-guided discrete walk-jump sampling for biological sequence generation
Authors: Zarif Ikram
Dianbo Liu
M Saifur Rahman
Venue: Accepted by TMLR ()

Abstract: In this work, we propose gradient-guided discrete walk-jump sampling (gg-dWJS), a novel discrete sequence generation method for biological sequence optimization. Leveraging gradient guidance in the noisy manifold, we sample from the smoothed data manifold by applying discretized Markov chain Monte Carlo (MCMC) using a denoising model with the gradient-guidance from a discriminative model. This is followed by jumping to the discrete data manifold using a conditional one-step denoising. We showcase our method in two different
modalities: discrete image and biological sequence involving antibody and peptide sequence generation tasks in the single objective and multi-objective setting. Through evaluation on these tasks, we show that our method generates high-quality samples that are well-optimized for specific tasks.

Published in Transactions on Machine Learning Research (12/2024)
Gradient-guided discrete walk-jump sampling for biological sequence generation
Zarif Ikram
sorifllram000@@maad.ccmm com
CSE, BUET
National University of Singapore
Dianbo Liu" National University of Singapore
dianbo Cruus www.us.euu......... edu..g
M. Saifur Rahman'
mmuhman@cse buut. hd
CSE, BUET
Reviewed OpenReview: Sttps://openreen net/ forum? id= fFVu04SPfT
Abstract
In this work, we propose gradient-guided discrete walk-jump sampling (gg-dWJS), novel discrete sequence generation method for biological sequence optimization. Leveraging gradi- ent guidance in the noisy manifold, we sample from the smoothed data manifold by applying discretized Markov chain Monte Carlo (MCMC) using a denoising model with the gradient- guidance from & discriminative model. This is followed by jumping to the discrete data manifold using a conditional one-step denoising. We showcase our method in two different modalities: discrete image and biological sequence involving antibody and peptide sequence generation tasks in the single objective and multi-objective setting. Through evaluation on these lasks, we show that our method generales high-qualily samples that are well-optimized for specific tasks.
Introduction
Biological sequences like antibody sequences is critical for a broad range of problems involving diverse practical applications, from molecular imaging of cancer (Wul 2014] to immunotherapy of cancer and viral diseases (Hudson & Souriau 2003 Lu et al.] 2020). The optimization is challenging: the protein sequence has an enormous state space with high-entropy variable regions. Experimental validation, on the other hand, is both time-consuming and expensive. Gencrally, the engincering process begins with in vitro experiments to determine aspects of interest such as humanization, specificity (Makowski et al. 2022), eflicacy, and pharmacokinetic properties, followed by mutation to the samples for improved properties.
Ab initio sequence generation fascinating direction for generating novel biological sequenees given prior samples. Using generative model, these methods attempt. to generate sequences that are similar to prior data. We roughly divide these models into two groups: autoregressive models (Wang et al.) 2022 Jain ct al. 2022 and denoising models (Luo et al.) 2022 Gruver et al. 2023). Albeit compelling, both come with their drawbacks. On one hand, autoregressive models suffer from error accumulation and even scalability in the case of reward distribution fitting (e.g., Jain et al. 2022 ). On the other hand, denoising models require intricate noise scheduling, making the real discovery task difficult.
To combat the inefficiency of the autoregressive and denoising models, discrete walk-jump sampling (dWJS) (Frey et al. 2024) recently proposed sampling antibody sequences by walking noisy manifolds, followed denoising jumping to the discrete data manifold. While dWJS benefits from sampling from noisy mani- folds for discrete sequence generation, its only objective is to sample from the data distribution, leading to
"These authors co-supervised the work.

---

Published in Transactions on Machine Learning Research (12/2024)
sequences that not necessarily optimized for chosen attribute. However, in a real-world antibody opti- mization setting, we are interested in sequences that are not only antibody-like but also attribute-optimizedl Instead of focusing on a single attribute, multi-objective optimization is important for biologicial seqeunce design. For example, therapeutic antibodics must meet numerous criteria at the same time, such as binding alfinity, specificity, stability, and manufacturability Traditional single-objective oplimization can produce sequences that excel one attribute but fail in others. By including multi-objective optimization in the generative process, we can balance competing objectives and build sequences that match all of the criteria. This method assures that the antibodies produced are not only effective, but also practical for real-world use. A simple but effective way of optimizing properties in generative model is to train a discriminative model and use its gradient to guide the sampling process towards a high-fitness region. While this approach is simpler in continuous setlings, enabling direct optimization in the antibody struct ure space, it still requires consideration of the amino acid sequence for actual syuthesis, necessitating inverse- folding (Dauparas et al. 2022) of structures. However, the optimized structure does not guarantee a rcalizable sequence, and even if the sequence exists, there is no assurance of inverse- folding it, motivating the need for direct optimization in discrete sequence space. In a discrete setting such as antibody sequence generation, gradient guidance is difficult. The sampling step in JWJS, thankfully, takes place in the smoothed data manifold, which lets these discriminative models provide gradient guidance.
In this work, we present gradient-guided discrete walk-jump sampling (gg-dWJS), a novel approach for ab initio sequence generation, building on (Frey et al.) 2024), as shown in Figure [ To improve the likelihood of sequence attribute, gg-dWJS learns a denoising and discriminative model on the noisy data manifold and then walks on the noisy data, manifold using discretized Langevin MCMC with gradient guidance. Finally, using the model pair, our method performs one-step conditional denoising to jump back to the truc data manifold. We present our approach in two modalities. First, we showcase image domain in Section 5 Next, in Section 6 we detail our approach in antibody sequence generation with two examples: single objective optimization and multi objective optimization with preference conditioning. Finally, in Section 6.4 we showcase our method in a peptide generation task.
approach in discrete
To summarize, the contributions of this work are:
We propose gg-dWJS, a novel algorithm based on dWJS for target optimization using a noised discriminator model.
Using preference conditioning, we formalize our method for use in a multi-objective optimization setting.
We evaluate our method on discretized image, antibody sequence, and peptide generation tasks, showing that our method generates high-quality samples with optimized attributes.
2
Preliminary
2.1
Walk-jump sampling (WJS) with neural empirical Bayes (NEB)
By combining kernel density estimation and empirical Bayes, NEB (Saremi & Hyvärinen 2019 denoising method to recover X for smoothed random variable Y = X X (0, such that X - Y using the gradient log density of Y, where X - Y refers to the kernel density estimation smoothing. Formally, to retrieve Y - X, given Y=y, we can get the least square estimator of X = a according to Robbins (1922) Miyasawa et al. 1961
provides
f == + a Vlogy(y)
(1)
Where Vlogp(y) is known as the score function Hyvärinen & Dayan 2005) which is the gradient of log density in the smoother hed function p(y) = p(y])) ( da.
Overall, the above constitutes the WJS framework, where the walk part draws exact samples from the smoothed density by Langevin MCMC and the jump part approximates the least-squares estimator of X arbitrary time.
2

---

Published in Transactions on Machine Learning Research (12/2024)
2.2 Discrete walk-jump sampling (dWJS)
A key property of the walk-jump sampling framework is that the least square estimation (jump) part is decoupled from the Langevin MCMC (walk) part. aWJS takes advantage of this decoupling to train an energy based model or score based model on the smoothed distribution on the noised samples, sample noisy samples with Langevin MCMC, and return back to the denoised samples with a least square estimator. In our work, we restrict ourselves to score based model for both the sampler and the least square estimator.
Specfically, building on the NEB, dWJS learns the score function on the noisy data manifold parameterized 96 Rd - R Hence, equation now looks like the following:
(2)
We can learn the parameter & by performing stochastic gradient descent on the following loss function to denoise y in noisy data manifold to in discrete data manifold.
( = pe--pee) - 26 (1)ll
(3)
To sample discrete under this formulation, we perform Langevin MCMC using on the noisy data manifold, returning to the true data manifold using equation
2.3 Classifier guidance in generative models
To obtain like in diffusion models, Dhariwal & Nichol 2021 modifies the diffusion score of a model with an auxilary gradient from the log likelihood of a classifier model. The modified score is then used during sampling from the diffusion model, which affects sampling by upweighting the data the classifier labels with high probability to the chosen label, providing a conditioning to the sampling process. We adapt this effect in our method by incorporating a discriminator model the sampling process. Unlike the classfier guidance in diffusion models that uses during sampling, our method incorporates gradient guidance during the least square estimation (jump) too.
2.4 Multi-objective optimization
Multi-objective optimization (MOO) refers to discovering a set candidates x² e X such that for d different objectives R(x) (x), R2(x),..... Ra(x)], it maximizes them simultancously, i.c.,
R(x*) = max R(x) = [max Rí (z), max Ra(x),.... max Ra(x)] zEX
ZEX
ZEX
If the objectives are in tension with one another, there is no x² that simultaneously maximizes all objectives. This leads to the Pareto optimality in MOO, ensuring optimal trade-off among objectives. In this formulation, for candidate xn to dominate I2, written as I > I2,
D2 + Ri(xn) 2 Ra(xa)Vi (1.....d) d) R)(1)) (z (1....,d).
Finally,
in Parelo-oplimal si X x (1,...,d)
Poreto set constitutes the Pareto-optimal candidates, and Pareto front defines their images in the objective space. A popular method to tackle MOO is Sealarization, which decomposes the MOO problem into single- objective problem. In this method, assign convex weight w; to each objective R((x) such that w; > 0 and Di wi Using the weights, we can derive a single-objective formulation, i.e., max R(xww), where R(I(w) is the scalarization function. A widely used scalarization function is weighted-sus sum scalorization (Ehrgott 2005 Micttinen 1999), where R(x, w) = Dim w; (x.). Using this formulation, we can represent the MOO problem as a family of subproblems, each defined by a different preference vector w. Under certain assumptions, the solutions to these problems are the Pareto optimal candidates for the preference vector
3

---

Published in Transactions on Machine Learning Research (12/2024)
Repeat tirr
condition
condition
Mareat
Smoothing
Gradient-guided walk
Gradient-guided jump
paddeent-guided guided discrete walk-jump sampling process. We begin the sampling process by smoothing some discrete seed. Next, we conduct the gradient-guided walk process by combining denoising gradient with the discriminator gradient. Finally, we perform pradiettgguddd jump to return to the discrete data manifold. Here, purple tint. represents higher data density and larger circle represents higher data distribution.
Figure 1: Gradient
3 Gradient-guiden discrete walk-jump sampling
gg-dWJS augments dWJS by learning discriminator model P(C (Step A) on the noisy data Y given the true data X and its label C. Next, we train the score model Ze the noisy data Y (Step B). Later, we utilize the discriminator's gradient to guide the Langevin MCMC walk (Step C). Finally, we jump from the noisy data manifold to the true discrete data manifold using the gradient guidance (Step D). For the remainder of the paper, we use the following notations for the random variables: X refers to the true data, Y refers the noisy data, and C refers to the class labels. The lowercase of of the said variables refers the specific instances of them.
3.1 Step A: Learning noisy discriminator model
Given the true data X and its label C, we smooth X by adding Gaussian noise, obtaining Y X-N oF)))) Next, we learn the discriminator model logP(CIYY) by parameterizing it with fo(Y). Note that C can be both categorical and continuous. For example, for categorical values, parameterize fo(v) to return logits over the labels given y, while for specific attribute (e.g., Instability index), we simply learn the fo(y) over the altribute.
3.2 Step B: Learning a noisy score model
We learn the score model on the noisy data Y, parameterizing it with Jg. We learn the model by minimizing the loss L from equation 3
3.3 Step C: Gradient guided walk
We start by sampling random discrete data ~ Ug. Langevin MCMC (Sachs et al. 2017) uses the gradient of log density to guide the sampling process by providing information about the structure of the target distribution. In our method, we perform discretized Langevin MCMC on y with gradient guidance from both score function (v) and discriminator fo(v). Formally, we use
go(ue) + XV fo
to perform MCMC for steps given y/ and label c at the kn step, where the relative gradient guidance strength with which we control the effect of gradient guidence.. Intuitively, one can see this as performing gradient accent on the label c's probability with respect the input ys find the local maxima while walking towards the smoothed data. density with (y()). We summarize the process in algorithm 1 For

---

Published in Transactions on Machine Learning Research (12/2024)
detailed explanation on the discretization of Langevin MCMC, guide readers Besides, we report all our hyperparameters in the Table g
Sachs et al. (017
Algorithm gradient-guided discrete walk-jump sampling
Input:
c: targetted label
8: step size
u: inverse mass
?: friction
A: relative gradient-guidance strength
K: number of steps taken
96 R Vlogp(y): learnt score function
fo = logp(cly): learnt discriminator model
90 (O, a² [x) + Ma(00)) !)
to +
for k ..... K
// gradient-guiden walk
do
Uk+1 t- + UK
5k+1 + (b(+++1)
// score of yk+1 // discrimiintoo gradient guidance
de+1 V ff(
9k+1 s Adæ+1
uk+1 1/ + 2 Ik+1
e~ c~,,00l))
uk+1 + e ? uk+1 + ug
2 I + uu(1 - 272
Uk+1 Vk+1 + 2k+1
dK ук + 0² g + Vo
// gradient-guidet jump
Perform argmax to receive the discrete sequence from fx
3.4 Step D: Gradient guided jump
After the K steps of walking, we jump from the noisy data manifold to the clear discrete data using the gradient guidance and score model given seoothed data and label c. From NEB, given a smoothed input y, we can jump back to using the equation Without loss generality, we can extend that, for a conditional variable and y ~ p(ycc).
f = 4 V (ogp(yylc
y + - logg(clu) + V yloop(()) o Volopp(c)
+ logg(clu) + t o''V)
y () + )3go(v)
Therefore, according to the above demonstration, perform a one-step denoising to return to the true discrete data manifold using fo and 96 from steps A & B. Specifically, the jump step approximates the truc discrete point & by adding the gradient-guided adjustments to yk where yk is point at any K time in the walk process
(4) Here, (Vff(JK,,)) represents gradient from the score model, helping align with the data distribution and o² (29()) provides attribute-specific guidance, pushing the output toward regions optimized for the desired attribute. This approach is useful because it allows to sample discrete points any time of the walk process.
t = + Vy c) + (5()). +
5

---

Published in Transactions on Machine Learning Research (12/2024)
4 Multi-objective optimization with gg-dWJS
Given the added conditioning capability, can now proceed to add the formulation for preference con- ditioning to our method. In the case of preference conditioning, since Ei=1 W2 = 1, oplimizing a single objective R(x, w) = Ei= (x) refers to a single point in the pareto front. This means that we have a generative model that is controllable based on the preference vector w {wi, W2..... wa), we can generate the solutions to the sub-problems, thereby constructing the pareto front.
A naive approach for controllable generation with preference conditioning is to simply train d noisy objective function models fy w) = (y) and use d gradients to guide the generative process, i.e.,
g = go (y) V v,fi,,(y, wi) + (y W2) vyfa,oy, wa)
(5)
However, this approach requires us to train discriminator models. Instead, we can approximate the combined discriminator and use its gradient in the following way:
= (v) + V V .... .......VV...eee Vyfae(y,wa) = (v) + (v)
+ + V, vywaaay))
(y) Vy(wn R1 + u2 R2(y) +.... t.a..+ (a
(v) Vyfe(y.u)
+... waRa(y) Here, fo(y, w) se w w))((y) + (o,2(((())++..++22))))) is the preference-conditionne single-objective function. By learning this function approximation, we can have two benefits: (a) we need only one discriminator training, and (b) we can generalize to different preference w not in training by taking advantage of the shared structure in the noisy manifold. Training the preference-conddtionedd discriminator model is simple: we can sample from preference distribution p(w) during training, use to encode multiple objectives to a single-objective function, and learn a function approximator fe
5 Experiment on discrete image generation (1**) ~ 10236)
To validate our method, we compare it against dWJSS for the binarized static MNIST image generation task (Salakhutdinov & Murray] 2008; Larochelle Bengio, (2008). The high-dimensionallty of this task (28 28 2) makes it attractive one to validate approach. For this task, two questions interest
Does gradient guidance enable more realistic generation than denoising walk?
Can JS generate conditionally, i.e., can generate binarized MNIST images with specific label (say, 0)?
In figure 2 we show the results of our experiment. On the left, we compare the samples generated by dWJS. gg-dWJS without denoising walk, and gg-dWJS. Here, for gg-dWJS without denoising walk, we only use v Voo(y, c) as the gradient for the Langevin MCMC. The samples show that gg-dWJS produces the most realistic and diverse samples, which is further affirmed by the lowest Fréchet inception distance (FID) (Heusel et al. 2017) in table [ Besides, we experiment with performing the walk step using only gradient-guidance, but we lind that it cannot generate high-quality samples. On the right, we show the samples of labels 0, 3, and 8 produced by gS- They showcase the method's ability to conditionally generate samples. We report the details related to this experiment in Table g
TTriiinn and evaluation code, httpp://github... con/zar wwwaarifiiaaaggggg aw..a/
training
data,
and
model
checkpoints
are available

---

Published in Transactions on Machine Learning Research (12/2024)
Table 1: Experiment results for binarized static MNIST image generation. Here, we calculate FID on the test data. Note that by binarizing MNIST images, we lose important pixel information, contributing to the high FID.
Method
FID
dWJS
54.62
dWJs w/o denoising walk w/ gradient guidance
89.42
gg-dWJS
51.88
D
3
2
6 2 8 J 3
8
O
6 O
2 O
O
2
C
O
3 8
0 0 000000000000000 00
3
D
4
3 5 035033333300
W
0
1 8
U
9
8 3 3 3 7 9
3
9 5 333333
Q q l 3
2
4
3
6
3
B 8 8 6
8
4
65
3
0 9
6
6
Figure 2: Comparison of binarized MNIST samples generated by different dW JS methods. Left: from top to bottom: dWJS, gg-dWJS w/o denoising gradient, gg-dWJS. Right: from lop to boltom: gg-dWJS generated samples with label 0, 3, and 8.
6 Experiment on antibody sequence optimization
6.1 Paired chain antibody sequence generation (1 ~ 10393)
Figure 3: Visualization of generated heavy and light chain sequences using gg-dWJS optimized for instability index. We use IgFold (Ruffolo al. 2023) for sequence folding and Mol* viewer (Sehnal et al. 2021 for visualization.
Now, turn our attention to antibody sequence generation. Following Frey et al. (024 Gruver et al. 2202)), represent antibody sequences as = (Z1...., xa), where E 1,...,21 refers to the 20 amino acid (AA) type, augmented by the gap token ,-? We use the 1.3M sequences provided by Frey et al. (2024). These sequences are from observed antibody space (OAS) database, aligned according to the AHo number scheme (Honegger PluEckthun 2001) using the ANARCT package (Dunbar Deane 2016). The gapped heavy and light chain lengths are respectively 149 and 148, making the total dimension (149 + 148) x 21.
For the optimization task, we experiment on two simple single-objective tasks: the percentage of beta sheets and the protein instability index (Guruprasad et al. 1990). For each task, train smoothed predictor the antibody sequences and use its gradient guidance to optimize the single objective.
Results. We compare gg-dWJS with dWJS, latent diffusion method generalized for discrete sequences
(Kigmma et al. 2021 variational diffusion models (VDM)), a transformer based language model trained for antibody sequence design (Shuai et al. 20233] IgLM), a pre-trained large language model (LLM) (GPT-40), and probabilistic method for discrete sequence generation (Bengio et al., 2023 Jain al., 2022 generative flow networks (GFlowNets)).

---

Published in Transactions on Machine Learning Research (12/2024)
Table 2: Statistics of the attribute of the paired OAS dataset.
min
mean
median
max
Instability index 12.687 % Beta sheets
40.400
40.618
68.230
0.314
0.378
0.377
0,455
Experiment results for antibody sequence generation for singlo-objoctive optimization task. The results show that sg-dWJJS-generttel sequences better optimized and of higher quality.
Table 3:
Method
DCS t
Instability index 4
% Beta sheets f
dWJS
0.49 + 0.30
34.14 + 6.38
0.393 + 0.02
8g- JS w/
Beta sheet discriminator
0,51 + 0.28
35.92 + 6.25
0.4081 0.02
gg- BKJSWJS w/ Instability discriminator
0.56 + 0.27
31.32 + 5.21
0.402 + 0.023
VDM
0.34 = 0.31
39.50 + 6.79
0.37 = 0.02
IgLM
0.19 + 0.29
38.84 6.18
0.37 + 0.02
GPT-40
0.31 + 0.30
42.63 2.47
0.37 + 0.01
GelowNets
0.0 + 0.0
39.04 + 1.04
N/A
We report the results of our experiments in table Specifically, we report the distributional conformity score (DCS), percentage beta shoets, and instability index for samples generated by dWJS and gg-dWJS with the two discriminators?" 2 We choose the metrics for their fast evaluation following previous works (Stanton et al. 2022 Gruver et al., 2024), though the metrics may have limited impact for real-world designs. We describe the metrics in more details in the Appendix C.3 The results show that with beta sheet gradient and instability index guidance, gg-dWJS generates samples with improved beta sheet percentage (Paired 1-test, p-value < 0.005) and instability index (Paired t-test, p-value < 0.005) than dWJS, respectively. The results also show that our method achieves the best DCS score among all the baselines, showing its capability of generating sequences of high Ablikeness. is worth noting that we also train the VDM from the same data, but its DCS score is not high. Besides, our method with instability index and beta sheet percentage discriminator achieves the best instability index and beta sheet percentage of all methods, respectively. Surprisingly, for the beta sheet percentage counterpart, many other methods come close, but they cannot generate sequences with good ablikeness, as evident by their low DCS. The result shows that not only does gg- generate antibody sequences that are better optimized for their respective objectives, but it also produces sequences of higher quality overall. Details of this experiment are in appendix C
6.2 Antibody CDR H3 design (121 ~ 1013)
Table 4: Experiment results for antibody sequence generation for single-task optimization task. The results show that w.edWW.S-ggeeeatee sequences are better optimized and of higher quality.
Method
Pbind
Unique
Edist
gS-dWJJ w/ binders only
0.93 = 0.15
0.98
5.47 + 1.35
dWJS w/ binders only
0.87 = 0.20
0.99
5.98 + 1.37
gg-dWJS
0.78 0.33
6.30 + 1.42
dWJS
0.53 0.39
7.19 + 1.34
VDM
0.33 0.35
7.58 + 1.19
GPT-4o
0.57 0.36
7.42 + 1.57
GFlowNets
0.32 = 0.28
9.16 0.78
Because the unavailability of the open-source implementation of DCS by Frey 2024]. implement the metric following the provided algorithm, which produces the same result for dWJS (0.49) reported by the authors.
8

---

0.405 0.420 0.435 0.450 Published in Transactions on Machine Learning Research (12/2024)
Now we consider the task of generating CDR mutants hu4D5 antibody mutant dataset (Mason et al, 2021 After de-duplication and removal of multi-label samples, this dataset contains 9k binding and 25k non- bin-binding hu4D5 CDR mutants to 10 mutations. To classify the generated samples, we train a binary classifier that achioves 85% accuracy on a IID validation set. For this task, we generate 1000 samples from each baseline, classify the samples using our trained classifier, and report the binding probability.
Results. In table f we report the results of the comparison of our method with baselines such as dWJS, GFlowNets, VDM, and GPT-4o. Here, mean binding probability Phind indicates different methods' ability to generate binders. Mean uniqueness and pairwise edit distance Edist indicate the diversity of the sequences. Following Frey et al. (2024), we train our method only on the 9k binders and see a small Phind improvement (0.93) over the base method (0.87). However, this is not ideal because we are not using the full sequence data. Therefore, we train all the baselines on the entire dataset. The results indicate that our method can utilize the sequence data to achieve good diversity (6.3) while having modest Paind (0.78). Interestingly, GPT-4o achieves a comparable result dWJS, while comfortably surpassing VDM and GFlowNets. Besides, GFlowNets (top 100) achieves the best diversity with poor pbind- Compared to the methods mentioned above, our method maintains good balance of diversity and binding probability.
6.3 Antibody sequence multi-objective optimization
The main goal of this experiment capability of gg-dWJS to generate samples with pref- erence conditioning. Towards that vision, we choose
showcase the
All Points Pareto Front
instability index and beta sheet percentage as the attributes to perform preference conditioning and train discriminator fo(y, w) such
72
Instability Index
the preference-conditionee that w fw,,1 wi} where p(w,) ~ U(1). Followed by a scalar normalization of the attributes, we perform a weighted-sum scalarization to decompose the attributes into single attribute. Subsequently, using the three different preference vectors E {[0,1] we generate 1k samples. Figure shows the distribution of the attributes of the generated samples, showing that samples with different preference-conditioonigg indeed fall into distinct distribution. Besides, the generated sam- ples exhibit good diversity, shown in table 5 Finally, in figure we show pareto front for the generated samples, showcasing our method's ability to perform in multi-objective setting. The results show that our method is able to generate samples according to different preferences w, c.g., w = [o, 1] generates samples focusing on the beta sheet percentage while w = [1,0] generates samples focusing on the instability index.
60-
54
48
42
1], [1, 0J, [0.5, 0.5]}.
30
0.360
Beta Sheet Percentage
Figure 4: Pareto front of the samples generated using three preference weights with gg-dWJS.
70
[1, 0]
Instability Index
[0, 1]
60
[0.5, 0.5]
Edist 1
DCS
50
[1.0]
41.03 + 13.26
0.29 + 0.13
40
[0,
21.27 + 5.5
0.05 + 0.13
[0.5, 0.5]
51.16 + 31.79
0.10 + 0.08
30
Table 5: Experiment results for preference condition- ing with gg-dWJS for antibody sequence optimization for three different preferences. The results show that Figure 5: Distribution of the generated samples with samples generated with preference conditioning using our method for three preferences.
0.36
0.38
0.40
0.42
0.44
0.46
0.48
Beta Sheet Percentage
gS-dWJS are diverse.
9

---

Published in Transactions on Machine Learning Research (12/2024)
6.4 Effect of guidance strength
To understand the effect of the relative gradient-guidance strength X, we run the antibody sequence op- timization with the instability index discriminator guidance for À E 10",, ]00] while keeping K fixed at 40. Figure [ shows the result of our experiment, showing that optimization and diversity are in tension-grees optimization leads to sampling from a smaller region of the sampling space. Similarly, we can see that higher guidance strength only leads to improvement in DCS up to a certain extent, after which the DCS reduces.
Table 6: Ablation results on À for antibody sequence optimization task. The results show that while higher gradient-guidante strength leads to better optimization, it also leads to less diversity and data fidelity.
A
Instability index 4
DCS +
Edist
100
31.32 + 5.20
0.56 + 0.27
89.07
26.80
10
26.29 + 4.04
0.57 + 0.27
75.90
29.89
102
22.16
+ 3.63
0.58 + 0.27
34.04 + 24.11
103
16.43 + 3.42
0.51 + 0.28
28.79 = 9.79
7 Experiment on anti-microbial peptide (AMP) design
The goal of this experiment is to generate peptides with anti-microbial properties. To this end, we collect AMPs and non-AMPs from the DBAASP database (Pirtskhalava et al. 2021). Following (Jain et al.) 2022 we choose peptides of sequence length of 12 to 60 with the target group Gram-positive bacteria, which yields 6438 positive AMPs and 9222 AMPs Then, we split the dataset into two parts: D1 and D2. following Angermuuller al. (2019) where D, is visible our algorithm and D2 is used to train the oracle for validation of the results. To split the dataset, we follow the same principle as Jain et al. (2022): for any peptide x E D1; there are peptides x' E D2 such that s' belongs to x's group and vice versa. This split yields 3219 AMPs and 4611 non-AMPs in D1. Finally, train the oracle utilizing ProtTrans [Elnaggar et al.] 2007) features from D2 and utilizing MLP classifiers (89% accuracy). For the gg-dWJS implementation, we follow the same score model and discriminator model setup from the antibody CDR H3 design experiment (dctails in Sections D.1 and D.2 Finally, to adapt to the variability of sequence length, we pad all sequences with gap lokens appended to the end.
Results. We compare our method against baselines such as GFlowNet-AL (Jain et al., 2022). DynaPPO (Angermueller al.) 2019), COMs Trabucco et al.) 2021). and GFlowNets (Bengio et al. 2021 utilizing the reported data. from Jain et al. (2022) under the criteria performance, diversily, and novelly. Here, performance measures the count of samples classified as AMP by the oracle, diversity measures the difference among the generated samples, and novelty measures the difference between the generated samples and the training dataset (D1). We detail the metrics in Appendix C.3
Table 7: Results on the AMP Task.
Performance
Diversity
Novelty
gg-dWJS
0.98 + 0.015
25.78 + 1.22
15.021 + 1.02
GFlowNett-AL
0.932 + 0.002
22.34 + 1.24
28.44 + 1.32
DynaPPO
0.938 + 0.009
12.12 + 1.71
9.31 = 0.69
COMs
0.761 + 0.009
19.38 + 0.14
26.47 + 1.3
Gel
0.868 = 0.015
11.32 - 0.67
15.72 + 0.44
Table 7 reports the results. We see that gW-SWJS achieves significantly better performance and diversity
than the basclines. Besides, our mothod reports the results for 100 generated samples over 10 random seeds, while the baselines only report results for the TopK samples from K generated samples, where
3nn Jain al. (002), is mistakenly reported as 9522 non AMPs.
10

---

Published in Transactions on Machine Learning Research (12/2024)
Generated AMPs Known AMPs
0.15
Density
0.10
0.00
A
C
D
E
Q
R
S
T
W
Amino Acid
Figure 6: Distribution amino acids found in the generated AMPs by gg-dWJS matches that known AMPs while maintaining focusing on amino acid 'K", which dominant in peptides with anti-microbial activity.
= 100 and 1 == 5. However, it also evident that GFlowNet-AL achieves better novelty than our method. However, as Kirjner et al. (2023) reports, higher novelty is not necessarily equivalent to better results. For instance, a random algorithm would achieve maximum novelty.
We also see lower average instability index of 26.3 compared to 26.5 reported by GFlowNet-AA, showing that gS- generates biologically relevant sequences (score of over 40 indicates instability). Figure G shows comparison of the distribution of amino acids between the known AMPs and gg-dWJS generated AMPs. We can see that gg-dWJS generates peptides that retains the amino acid profiles of the known AMPs but boosts important characteristics generally found in AMPs, e.g,, the amino acid 'K', which dominant in peptides with anti-microbial activity (Jain et al. 2022).
8 Related Work
8.1 Discrete generative models
A large class of discrete generative models consist of autoregressive models (i.e., language models). Among many others, Austin et al. (2021) learns the posterior alphabet distribution over the prior generated data
by lcarning the bidirectional context in a language model. Recent works such as Generative Flow Networks
(GFlowNets) (Bengio et al. 2023
2021p model the flow as a DAG, and learn to sample directly propor- Zhang al. (2022) uses this idea to train and sample from an energy
tional a given reward function.
based model (EBM) (LeCun et al.
2006) using contrastive divergence (Carreira-Porpinan & Hinton
2005
Besides, Grathwohl al. (2021) improves the sampling process by leveraging local gradients using Gibbs
sampler George & McCulloch 1993: Gelfand 2000).
Another way to generate data is through demoising models (Ho et al.) 2020 Song & Ermon 2020 Song et al. 2021). These models learn the gradient log density of the data and generate continuous data from the perturbed data distribution. Thus, they are faster and more efficient than autoregressive models. Of course, one cannot simply apply the denoising gradient to the discrete data distribution because the gradients are not defined there. Many promising works attempt to remedy this by proposing different techniques. [Chen et al. (2023) simply transforms the discrete data into analogue bits, learns the denoising gradient, and applics thresholding to return the categorical data. (Snn et al. (2023) applies denoising via a contimous--ime Markov chain to the calegorical data using a stochastic jump process. For graph adjacency mat Niu et al. (2020) adds Gaussian perturbation to the upper triangular matrix, sample using the learn score model, and transform samples in the continuous space by quantizing the generated continuous adjacency matrix binary one. On similar vain, Yan al. (2024) performs post-hoc thresholding after the end of Langevin dynamics to tranform the denoised continuous adjacency matrix to binary discrete adjacency matrix.
11

---

Published in Transactions on Machine Learning Research (12/2024)
Frey al. (2024) leverages the gradient information in the smoothed data manifold and jumps to the true data manifold using NEB, which allows obtaining sample at any part of the Langevin dynamics.
8.2 Protein sequence optimization
With the advancement of language models, many recent works (Madani et al. 2020 Nijkamp et al. 2023
Ferruz et al. 2022) propose language models pre-trained on protein data to generate protein sequences, often for tasks such as structure prediction (Lin et al., 2023 These protein language models have also
been adopted for antibody generation tasks, both unguided (Shua et al. 2021) and guided (Gligorijevid ct al.) 2021] Ferruz & Höcker [022] Tagasovska et al. 2022). A key problem with language modeling of
antibodies that they struggle to caplure the data distribution of the antibody sequences given a limited amount of high-quality data and high-entropy variable regions of the antibody sequences. As such, recent works such as Luo et al. 20022) ; Gruver al. (2023); Peng et al. (2023) leverage the continuous space of
antibody structures to generate antibody structures using diffusion. In this work, we focus on the problem
of generating targeted antibody sequences.
9 Conclusion and limitations
In this work, introduce gg-dWJS, which learns discriminative model on the smoothed data and uses the gradient information to augment its denoising walk towards the local maxima given some attributes. Finally, it returns to the clean data manifold by using conditional jump with the same model. Thus, our method requires no additional score model training for different optimization tasks. Additionally, we formalize our method in a multi-objective setting using a preference-conditionn discriminator model. Using prefroace-conditioning, we show that our method can generate samples that correspond to different regions of the multi-objective space based on preference. We show our method's applicability in two modalities: discrete imagine generation and biological sequence generation in single objective and multi objective cases.
Despite our sincere efforts, our work falls short in addressing numerous issues and has limitations. For starters, while we see an improvement in DCS with gradient guidance, the improvement starts to fade away with the higher strength of the gradients. is also worth mentioning that more optimization leads to less diversity in the samples. Indeed, the general tension of data fidelity, diversity, and optimization still holds true in our case, and improving them together is still a challenge. For example, our current approach with MCMC can be limitation for diversity, as it is known to suffer from mode collapse, where it diverges to a local solution for many starting points. There are many line of work that attempts to tackle this problem (De Souza et al.] 2022 Liu et al., 2023 Wang el al. 2024b) from different viewpoints, which we believe can be an useful addition to our work. Future works can also tackle the problem of optimization and uniqueness together through the addition of an active learning pipeline (Settles) 2009 Srrnandez-Garrhh et al. 2023 to our method. Besides, one can improve fidelity of the generated sequences through the use of strcture priors, similar to the recent work by [aang et al. (2024a). Furthermore, we did not investigate different scalarization functions in our infeerence-onditionnnn setting; we left that along with the multi-objective benchmarking of our method for future work.
Broader Impact Statement
This work falls in the line of discrete generative modeling. While the negative impacts of generative modeling apply to it, the authors do not foresee any particular negative impacts.
Acknowledgments
This work funded by RISE student research grant (52024-01-013) and NUS-UToronto joint grant. Fur- thermore, 7. Ikram is thankful to his family.
12

---

Published in Transactions on Machine Learning Research (12/2024)
References
Christof Angermueller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Colwell. Model-basod reinforcement learning for biological sequence design. In International conference on learning representations, 2019. 10
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:1 34:17981-17993, 2021.
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Advances in Neural Information Processing Systems, 34:27381- 27394,, 2021.
10.
Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J Hu, Mo Tiwari, and Emmanuel Bengio. Gflownet foundations. Journal of Machine Learning Research, 24(210):1 55, 2023.
22
Miguel A Carreira-Perpinan and Geoffrey Hinton. On contrastive divergence learning. In International workshop on artificial intelligence and statistics, pp. 33 40. PMLR, 2005.
Ting Chen, Ruixiang ZHANG, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion
models with self-conditioning. In The Eleventh International Conference Learning Representations, 2023. URL https://ppenreeveev. http://www..eeeeomoooooo----////
11
Peter JA Cock, Tiago Antao, Jeffrey T Chang, Brad A Chapman, Cymon J Cox, Andrew Dalke, Iddo Friedborg, Thomas Hamelryck, Frank Kauff, Bartek Wilezynski, al. Biopython: frocly available python tools for computational molecular biology and bioinformatics. Bioinformatics, 25(1)):1422, 2009. 21
23
Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. Robust deep learning based protein sequence design using proteinmpmn. Science, 378(6615):49-3 56, 2022. 2
Daniel A De Souza, Diego Mesquita, Samuel Kaski, and Luigi Acerbi. Parallel meme without embarrassing failures. In International Conference on Arlificial Intelligence and Stalistics, pp. 1786 1804. PMLR, 2022.
12
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural
information processing systems, 34:8780 8794, 2021. 24
James Dunbar and Charlotte M Deane. Anarci: antigen receptor numbering and receptor classification.
Bioinformatics, 32(2))298 300, 2016. 7
Matthias Ehrgott. Mullicriteria optimization, volume 491. Springer Science & Business Media, 2005.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rihawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: Towards cracking the language of life's code through self-supervised deep learning and high performance computing, arxiv 2020. ar preprint 2XXt:2007.06225 2007. 10
Noclia Forruz and Birte Höcker. Controllable protein design with language models. Nature Machine Intelli-
gence, 4(6):521 532, 2022. 12
Noelia Ferruz, Steffen Schmidt, and Birte Höcker. Protgpt2 is deep unsupervised language model for protein design. Nature communications, 13(1):4348, 2022. 12
Nathan C Frey Daniel Berenberg, Karina Zadorozhny, Joseph Kleinhenz, Julien Lafrance- Vanasse, Isidro Hotzel, Yan Wu, Stephen Ra, Richard Bonneau, Kyunghyun Cho, al. Protein discovery with discrete walk-jump sampling. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreviev... http:///ww........o//////////aan
12
19
20
22
13

---

Published in Transactions on Machine Learning Research (12/2024)
Milton Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance.
Journal of the omerican statistical association, 32(200):675-700 701, 1937. 23
Milton Friedman. comparison of alternative tests significance for the problem of m rankings. The annals
of mathematical statistics, 11(1):86-98 -2, 1940. 23
Alan E Gelfand. Gibbs sampling. Journal of the Americon statistioal Association, 95(452):1300 1304, 2000.
11
Edward George and Robert E McCulloch. Variable selection via gibbs sampling. Journal of the American Statistical Association, (8(423):881-88 1899,,9, 1993. 11
Vladimir Gligorijević, Daniel Berenberg, Stephen Ra, Andrew Watkins, Simon Kelow, Kyunghyun Cho, and Richard Bonneau. conction-guided protein design by deep manifold sampling. bioRriv, pp. 2021-12, 12, 2021.
12
Will Grathwohl, Kevin Swersky, Milad Hashemi, David Duvenaud, and Chris Maddison. Oops i took a gradient: Scalable sampling for discrete distributions. In International Conference on Machine Learning, pp. 3831-333131 3841. PMLR, 2021. 11
Nate Gruver, Samuel Don Stanton, Nathan C. Frey, Tim G. J. Rudner, Isidro Hotzel, Julien Lafrance- Vanasso, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffuusion. In seirth- Conference on Neural Information Processing Systems, 2023. URL
wewte//orrrmdddaiiaaaaaa httpps:/openreevier. http:://ppaaeeeeeee..ee////ttotr
12
Nate Gruver, Samuel Stanton, Nathan Frey, Tim GJ Rudner, Isidro Hotzel, Julien Lafrance-Vannaa Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G Wilson. Protein design with guided discrete diffusion. Advances in neural information processing systems, 36, 2024. 8
Kunchur Guruprasad, BV Bhasker Reddy, and Madhusudan W Pandit. Correlation between stability of protein and its dipeptide composition: novel approach for predicting in vivo stability of protein from its primary sequence. Protein Engineering, Design and Selection, 4(2):155-161, 1990. 19
Alex Hernandez-Garcia, Nikila Saxena, Moksh Jain, Cheng-Hao Liu, and Yoshua Bengio. Multi-fidelity active learning with gflownets. arXiv preprint arXio:2306.11715, 2023. 12
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural infor- mation processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in newral information processing systems, 33:6840-6838 2020.
Annemarie Honegger and Andreas PluEckthun. Yet another numbering scheme for immunoglobulin variable domains: an automatic modeling and analysis tool. Journal of molecular biology, 309(3):657 670, 2001.
Peter J Hudson and Christelle Souriau. Engineered antibodies. Nature medicine, 9(1): 134, 2003.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005. 2
Moksh Jain, Emmanuel Bengio, Alex Hernandez- Jarrid Rector-Brooks, Bonaventure FP Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, et al. Biological sequence design with gflownets. In International Conference on Machine Learning, pp. 9786-978600 9801. PML.R, 2022. T
20
Wengong Jin, Jeremy Wohlwend, Regina Barzila,, and Tommi S Jaakkola. Iterative refinement graph neural network for antibody sequence-structuue co-design. In International Conference on Learning Representa- lions, 2021.
T9
14

---

Published in Transactions on Machine Learning Research (12/2024)
Nal Kalchbrenner, Lasse Espcholt, Karen Simonyan, Aaron van den Oord, Alex Graves, and Koray Kavukcuoglu. Neural machine translation in linear time. or preprint ar arXiv:161 10099, 2016. 20
Bahjat Kawar, Roy Ganz, and Michael Elad. Enhancing diffusion-basel image synthesis with robust classifier guidance. Transactions on Machine Learning Research, 2022.
24
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural informalion processing systems, 11-21996-27701 2021.
7
Andrew Kirjner, Jason Yim, Raman Samuseviih,, Shahar Bracha, Tommi S Jaakkola, Regina Barzilay, and Ila R Fiete. Improving protein optimization with smoothed fitness landscapes. In The Twelfth International Conference on Learning Representations, 2023.
Jack Kyte and Russell F Doolittle. A simple method for displaying the hydropathic character of a protein.
Journal of molecular biology, 157(1))105-133)) 1982. 19
Hugo Larochelle and Yoshua Bengio. Classification using discriminative restricted boltzmann machines, In Proceedings of the 25th international conference on Machine learning, pp. 3300543, 2008.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
VI Levenshtein. Binary codes capable of correcting deletions, insertions, and reversals. Proceedings of the Soviet physics doklady, 1966. 20
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Fvolutionary-seale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123 1130, 2023. 12
Haozhe Liu, Bing Li, Haoqian Wu, Hanbang Liang, Yawen Huang, Yuexiang Li, Bernard Ghanem, and Yefeng Zheng. Combating mode collapse via offline manifold entropy estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 8834-8842, 2023. 12
lly Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL
21
Ruei-Min Lu, Yu-Chyi Hwang, I-Ju Liu, Chi-Chiu Lee, Han-Zen Tsai, Hsin-Jung Li, and Han-Chung Wu. Development of therapeutic antibodics for the treatment of discases. Journal of biomedical science, 27(1):
1-30, 2020. M
Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma. Antigen- antibody design and optimization with diflusion-based generative models for protein structures. Advances in Newral Information Processing Systems, 35:9754-977 9767, 2022.
I2
Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R Eguchi, Po- Ssu Huang, and Richard Socher. Progen: Language modeling for protein generation. arXie preprint arXiv:2004.03/4 2020. 12
Emily K Makowski, Patrick C Kinnunen, Jie Huang, Lina Wu, Matthew D Smith, Tiexin Wang, Alec A Desai, Craig N Streu, Yulci Zhang, Jennifer M Zupancic, et al. Co-optimization of therapeutic antibody affinity and specificity using machine learning models that generalize to novel nutational space. Nature communications, 13(1),3788, 2022.
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory balance: Im- proved credit assignment in gflownets. Advances in Neural Information Processing Systems, 35:5955-5961 2022.
Derek M Mason, Simon Friedensohn, Cédric R Weber, Christian Jordi, Bastian Wagner, Simon M Meng, Roy Ehling, Lucia Bonati, Jan Dahinden, Pablo Gainza, et al. Optimization of therapoutic antibodics by predicting antigen specificity from antibody sequence via deep learning. Nature Biomedical Engineering. 5(6):600-6i5 612, 2021. 9
15

---

Published in Transactions on Machine Learning Research (12/2024)
Kaisa Miettinen. Nonlineor multiobjective optimization, volume 12. Springer Science & Business Media, 1999. 8
Koichi Miyasawa et al. An empirical bayes estimator the mean a normal population. Bull. Inst. Internat. Statist, 38(181-188)112, 1961.
Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. Progen2: exploring the boundaries of protein language models. Cell Systems,
2023. 12
Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Permutation invariant graph generation via score-based generative modeling. In International Conference on Artificial Intelligence and Statistics, pp. 4474-4484. PMLR, 2020. 11
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. 21
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Preltenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825-220 22300, 2011.
19
Zhangzhi Peng, Chenchen Han, Xiaohan Wang, Dapeng Li, and Fajiie Yuan. Generative diffusion models for antibody design, docking, and optimization. bioRaiv, pp. 2023 09, 2023. 12
Malak Pirtskhalava, Anthony A Amstrong, Maia Grigolava, Mindia Chubinidze, Evgenia Alimbarashvili, Boris Vishnepolsky, Andrei Gabrielian, Alex Rosenthal, Darrell E Hurt, and Michael Tartakovsky. Dbaasp v3: database of harniicrobbil/eyttttoooooo activity and structure of peptides as a resource for development new therapeutics. Nucleic acids research, 49(D)):2288 D297, 2021. I0
Herbert E Robbins. An empirical bayes approach to statistics. In Breakthroughs in Statistics: Foundations and basic theory, pp. 388 394. Springer, 1992. 2
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Compuler-Asse Assisted Intervention-MICE MICCAT 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part 1I1 18, pp. 234 241.
Springer, 2015. 19
Jeffrey A Ruffolo, Lee-Shin Chu, Sai Pooja Mahajan, and Jeffrey J Gray. Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies. Nature communications, 14(1):2389, 2023.
Matthias Sachs, Benedict Leimkuhler, and Vincent Danos. Langevin dynamics with variable coefficients and nonconservative forces: from stationary states to numerical methods. Entropy, 19(12):647, 2017. 4
Ruslan Salakhutdinov and lain Murray, On the quantitative analysis deep belief networks. In Proceedings of the 25th international conference on Machine learning, pp. 872 879. ACM, 2008 6
Saeed Saremi and Aapo Hyvärinen. Neural empirical bayes. Journal of Machine Learning Research, 20(181): 1 1-23, 2019. 2
David Sehnal, Sebastian Bittrich, Mandar Deshpande, Radka Svobodová, Karel Berka, Václav Bazgier, Sameer Velankar, Stephen K Burley, Jaroslav Koča, and Alexander S Rose. Mol* viewer: modern web app for 3d visualization and analysis of large biomolecular structures. Nucleic acids research, 49(1)):
W431-W437, 2021.
Burr Settles. Active learning literature survey. 2009. 12
Richard W Shuai, Jeffrey Ruffolo, and Jeffrey J Gray. Generative language modeling for antibody design. bioRrie, pp. 2021-12, 2021. 12
16

---

wtt.://eww....e./eooooooooooaadd Published in Transactions on Machine Learning Research (12/2024)
Richard W Shuai, Jeffrey A Ruffolo, and Jeffrey J Gray. Iglm: Infilling language modeling for antibody sequence design. Cell Systems, 14(11):979-9835 2023. 7
Yang Song and Stefano Ermon. Improved techniques for training score- based generative models. Advances in neural information processing systems, 1511248-124444 2020. 11
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL https://openrreviex. wwt.://ooorrm1dd
Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating bayesian optimization for biological sequence design with denoising autoencoders. In International Conference on Machine Learning, pp. 20459 20478. PMLR, 2022. S
Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, and Hanjun Dai. Score-based continuous-time discrete diffusion models. In The Eleventh International Conference on Learning Representations, 2023. URL
https://openreview httpe://ppareerrvr.e.ee///ottttt
Natasa Tagasovska, Nathan C. Frey, Andreas Loukas, Isidro Hotzel, Julien Lafrance- Vanusse Ryan Lewis Kelly, Yan Wu, Arvind Rajpal, Richard Bonneau, Kyunghyun Cho, Stephen Ra, and Vladimir Gligorijevie. A pareto-optimal compositional energy-based model for sampling and optimization of protein sequences. In NewrlPS 2022 Al for Science: Progress and Promises, 2022. URL http:s://ooenreiiee.ntttteeeeeee
id-U2rNKaTTXPQ
12
Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offiine model-based optimization. In International Conference on Machine Learning, pp. 10358- 10368. PMLR, 2021.
10
Chentong Wang, Sarah Alamdari, Carles Domingo-Farich, Ava Amini, and Kevin K Yang. Towards deep learning sequence-structure co-generation for protein design. arXiv preprint arXiv:2410.0170 2024a. 12
Danqing Wang, YE Fei, and Hao Zhou. On pre-training language model for antibody. Tn The Fleventh
International Conference on Learning Representations, 2022. I
Peihao Wang, Dejia Xu, Zhiwen Fan, Dilin Wang, Sreyas Mohan, Forrest landola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, et al. Taming mode collapse in score distillation for text-to-3d generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pollern Recognition, pp. 9037-90477
2024b.
12
Anna M Wu. Engineered antibodies for molecular imaging of cancer. Methods, 65(1):139 147, 2014. [
Jin Xu, Zishan Li, Bowen Du, Miaomiao Zhang, and Jing Liu. Reluplex made more practical: Leaky relu. In 2020 IEFE Symposium Computers and communications (ISCC), pp. 1-7. IEFE, 2020. 21
Qi Yan, Zhengyang Liang Yang Song, Renjie Liao, and Lele Wang. SwinGNN: Rethinking permutation invariance in diffusion models for graph generation. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL
11
Dinghuai Zhang, Nikolay Malkin, Zhen Liu, Alexandra Volokhova, Aaron Courville, and Yoshua Bengio. Generative flow networks for discrete probabilistic modeling. In International Conference on Machine Learning, pp. 22422. PMLR, 2022. 11
17

---

Published in Transactions on Machine Learning Research (12/2024)
Supplementary material for
Gradient-guided discrete walk-jump sampling for biological sequence generation Zarif Ikram, Dianbo Liu, M Saifur Rahman
Acronyms
AA amino acid.
DAG directed acyclic graph.
DCS distributional conformity score. 8 10, 19, 20, 23
dWJS discrete walk-jump sampling. 1-4, 6-8,
EBM energy based model. 11
FID Fréchet inception distance. 6, 19
GFlowNets Generative Flow Networks. 11
gg-dWJS gradient-guided discrete walk-junp sampling. 1, 2, 4 8, 12, 21, 23, 24
GRAVY grand average of hydropathicity. 19
KDE kernel density estimation. 19
MCMC Markov chain Monte Carlo. 1 4
MOO multi objective optimization. 20, 21
NEB Neural empirical Bayes. 2, 3, 5, 12
OAS observed antibody space. 7, 19 21, 23
A Notations
Table 8: Notations summary
Symbol
Description
X
discrete random variable
Y
smoothed random variable
X
Y
kernel density estimation smoothing denoising mechanism of empirical Bayes learnt score function
Y
X
(v)
fo(y)
learnt discriminator function preference condition weights
5
step size
u
inverse mass friction relative gradient-guidance strength number of steps taken
7
K
18

---

Published in Transactions on Machine Learning Research (12/2024)
B Additional details on the static MNIST experiment
We train the noisy score and discriminator model the binarized static MNIST dataset For score model architecture, we use U-Net architecture (Ronneberger et al. 2015) that takes the smoothed one-hot representation of the discrete images and returns the score of the same shape. Finally, we train CNN architecture on smoothed data and their corresponding labels for the discriminator model.
9
6 4 4 4 / 8 1 / 1
5
7
2
9
4
6
9
9
7
8
0
8
0 4
0
6
3
2
4
0
4
57 3
1
9 4
1
6
5
6
7
1
5
3
7
7
3
4
7
3 8
4
8
6
2
/
0
9
3
3
3
3
1
9
4
3
3
5
5
0
3 4 3 7
8
9
3
3
5
6
3
3
(
66 9 9 4 4
5
1
4
777006666973
0
9660 2 / 2 1 4
Y
2
6
7
6
6
9
8500
5 55 4 8 5
3
0
5
1 9
1
9
4
3
0
6
4
2
/
0
7 69
0 9 2 9 9
/
/
9 4
r
6
6
3
6
3
3
5
0
/
3
0
4
3
4
5
/
2
0
2
/
6
7
3
8
2 9
04
9 0
2
0
9
5
8
4
4
9
9
0
2
9
3
8
/
3
3
6
3
5
7
0
P
3
6
6
6 9.
3
8
7
5
3
9
8
7
9
2
9
9
7
4
4
7
3
2
6
4
6
7
R
9
0
Figure 7: Randomly chosen 400 static MNIST test-sct samples.
C
Additional details on the antibody sequence optimization experiment
C.1 Training details
All our models are trained with maximum 40 epochs with early stopping and with a learning rate of 10-1, a weight decay of 0.01, and a batch size of 64. We validate our models on a held-out validation set separate from the training data. Both the discriminator and the score based models for experiments are trained with mean square error loss, except for CDR H3 experiment, in which the discriminator is trained with a binary cross entropy loss. We save the best models based the validation loss and use the model during sampling.
C.2 distributional conformity score (DCS)
The recently proposed DCS measures sample quality using a cample-to-distribne metric, as opposed to previous sample recovery metrics (Jin et al., 2021). measures the fraction of the validation sequences that are less similar their IID samples compared to a target sample. We show the algorithm to calculate DCS from Frey et al. (2024) in algorithm 2 We implement DCS using two sequence based properties: molecular weight and grand average of hydropathicity (GRAVY) (Kyte & Doolittle, 1982). We used the kernel density estimation (KDE) implementation by Scikit-learn (Pedregosa et al. 2011) to approximate the joint distribution using a Gaussian kernel and bandwidth of 0.15. DCS is dala fala-fidelity metric similar to FID for images, Unsurprisingly, DCS and optimized attributes are in tension optimization while preserving data fidelity is harder.
C.3 Evaluation metrics
Paired OAS experiment. We show the optimization of two altributes this experiment: % beta sheets and instability index. The fraction of beta-pleated sheets provides insights into the structural characteristics of antibody soquences, influencing their stability and functionality. The protein stability index is test proposed by Guruprasad et al. (1990). A protein sequence has short half-life if its stability index is above 40. Stability is crucial metric in the context of therapeutic antibody generation, as it is closely associated with manufacturability
*Collected from Kaggle
5 Based the Py Torch example adapted for discrete data.
19

---

Published in Transactions on Machine Learning Research (12/2024)
Algorithm 2 distributional conformity score (DCS) (adapted from Frey et al. (2024))
Input:
reference distribution
X: test example
A: conformity measure
Output:
pv: the fraction of validation examples that are less similar to Dely than
Sample (zI,...,,zn) 2n) 23 ~ Dety and a held-out validation set (Ei, 2 1), 24 ~ D2ly Ek + (x.y)
for i 1, ........ do
t ,
pd x Σ las [xyl
return
CDR H3 experiment. The key attribute for this experiment is Poind, the probability of being binder, which we obtain from the binary classifier trained on the data. Besides, to measure the diversity of the generated sequences, we use mean uniqueness and edit distance Edist-
The metrics used in this experiment are similar to those listed above: % beta sheets and instability index for the optimization metric, edit distance for the diversity metric, and DCS for the data fidelity metric.
Paired OAS MOO experiment.
AMP experiment. Following Jain et al. (2022), we use the evaluation metrics described below. Here, D is the dataset of generated samples, Do is the dataset used to train the generative models, and X' is the set of all peptides.
Performance: This is simply the percentage of the generated samples identified as AMPs by the oracle trained on D2, lavoring models that can consistently generate more AMPs.
Diversity: In addition to being high performance, it is important that samples generated by the models are diverse. This diversity can be quantified by measuring the distance between all the generated samples, i.e,
E d(I,,z)) Diversity(D) 3;ED (..))
Σ
(6)
[D[[D]] -
where d is the Levenshtein distance [evennhtein 1966 calculated with the Polyleven package.
Novelty: Novelty seeks measure the difference between the proposed candidates D and the candidates that already known Do. We measure this in the proposed candidates follows:
Novelty Ex.eD ming, d(z,, 83)
[D]
where d the Levenshtein distance.
Implementation details of the baselines
D.1 Score models
We follow Frey et al.
2024) for the score model implementation available
https: com/Genentech. haahh/wwk--kkk jump We use a 35-layer Bytenet (Kalchbrenner et al. (0166) architecture with hidden layer of 128. To train the model from scratch, utilize batch size of 64 and
20

---

Published in Transactions on Machine Learning Research (12/2024)
the AdamW optimizer [Loshchilov & Hutter 2019 in PyTorch (Paszke et al. 2019) with early stopping. The training parameters include learning rate of 10-1 and a weight decay of 0.01. We conducted the training using four NVidia A100 GPUs. The training process takes approximately 10 hours. For training the score model, we use a = 1. We use the same score model training method for all experiments in this paper. Table g lists all the hyperparameters we use for sampling from the score model using dWJS and gg-dWJS.
Table 9: Hyperparameters used for dWJS and gg-dWJS sampling.
0.5
1
K
40
À (% beta sheet)
100
X (instability index)
D.2 Discriminator models
Paired OAS experiment. We train the discriminator model on two sequence-related properties: the percentage beta sheets and the protein instability index. We label the training sequences using BioPy- thon (Cock et al. 2009). For the architecture the smoothed predictor, we adopt a 3-layer 1D-CNN followed by 3-layer MLP integrated into the existing ByteNet architecture, incorporating leakyReLU (Xu et al, 2020) activations between the layers. Finally, follow the same training parameters the score model training. To use the discriminator models' prediction gradients guidance, use the torch. autograd the models prediction given label or not (in case of non-categorical variable).
CDR H3 experiment. Here, we have two similarly trained discriminator models for two different pur- poses: the discriminator model on pyind for classification and the noised discriminator model for generation using JSS. We adopt a similar approach to the one used the paired OAS experiment, except for using a sigmoid terminal activation and binary cross entropy loss function to accommodate the binary classification.
Paired OAS MOO experiment. The chosen attributes for the MOO experiment instability index and % beta sheets. For this experiment, we attempt to lower both values as the multi-objective criteria. Therefore, we train noised discriminator model that outputs a weighted sum of the normalized attributes. To achieve preference conditioning, the setup is similar to the other discriminators, except we concatenate the preference vectors with the output of the 1D-CNN.
D.3 Variational diffusion models (VDM)
In this baseline, we project the discrete sequences into a continuous latent space and perform the diffusion in the latent space. We use the implementation from to train the VDM model on the OAS https://github. gradient descent on the reconstruction and diffusion loss, We use a generalization of dataset by performing the same model to sample discrete sequences by performing a reverse diffusion process for 100 steps 1 Our implementation uses hyperparamers as listed the table 10
D.4 IgLM
1gLM is the autoregressive baseline for our comparison. We use the open-source package from https://github. com/Graylab/IgLM to sample heavy and light chain samples. We used the following prompl to generate the heavy sequences, with an adjustment to the chain token altribute to generate the light sequences.
6n layman's 's terms, perform argmax the last step while sampling.
21

---

Published in Transactions on Machine Learning Research (12/2024)
Table 10: Hyperparameters used in the VDM implementation.
Embedding dimension
128
Attention head
1
Dropout probability
0.1
Tmin
-13.3
Tiaa
5
Learning rate
2x10
Weight decay
0.01
МETHOD NAME "IgLM"
prompt_sequence sequence ""
chain_token [HEAVY] "
species_ _tckees_tkken__e token [[UMAN]
num_seqs = 1000
D.5 GPT-40
Following Frey (2024). We GPT-40 large language model (LLM) baseline. For the paired OAS generation experiment, prompt the following prompt.
You are an expert antibody engineer. I am going to give you examples of antibody heavy chain and light chain variable regions from the paired observed antibody space database. You will generate 10 antibody heavy chain and light chain that are not in the database. Output the 10 samples examples: [COMUNNSSSCEEEEEESSSSSSSSSSSSSS
python list. Here are the
DYYCQAWDY
STAVFGTGTKVIVVL))
Similarly, we perform similar prompt for the CDR H3 design experiment. The prompt is the following:.
are an expert antibody engineer. am going to give you examples CDR
variants of trastuzumab that were reported binders the HER2 antigen paper "Optimization of therapeutic antibodies by predicting antigen specificity from antibody sequence via deep learning". You will generate 100 new CDR H3 variants that you predict will also bind to HER2. Output the 100 samples as a python list. Do not post the code. Instead, show the samples directly. Here are the examples: ['MHINGFYVFH', 'FQDHGMYQHV', 'YLAFGFYVEL', 'WLNYHSYLFN', 'YNRYG- FYVED' "WRKSGFYTFD', 'WANRSFYAND', 'NPSCGMFALL', ?WSNYGMFVFS', 'ws- MGGFYVEV', WGQLGFYAYA', 'MPILGLYVFI', 'WHRNGMYAFD', 'WPLYSMYVK'', 'WGLOGLYAYQ'.)
D.6 Generative flow networks (GFlowNets)
GFlowNets popular baseline for discrete generative modeling using probabilistic method. However, for the current implementation of GFlowNets, generating samples in a state space of size 21 21 is difficult. There- fore, reduce the problem of generating the paired sequences to generating the first 50 mutations of the full sequences with trajectory balance objective (Malkin et al.) 2022p. The heavy and light chain suffixes added to the generated mutations are
WWW.E///WWWWAASKKKKKKKCCTTTTAAAT GLEWMGR FIPILNIKNYAQDI TIPLLIIKTAAADDECCCTTTEEE LEYWGQGTINSVSS and WWW.GHRRESSEEESTTTTRTTTTTTTTTTTT FAASTRATGIEE CEGGPPAASSGGEDTTTTTSSSSSEEEEAAOO
respectively. We the implementation provided in Bengio et. al. 2023 for experiment. We 3-layer ML. with
22

---

Published in Transactions on Machine Learning Research (12/2024)
256 hidden dimensions to model the forward layer, which is then followed by leaky ReLU. The forward layer takes the one-hot encoding of the states as inputs and outputs action logits. To model the forward and backward flow, we double the action space and train the MLP. with a learning rate of 10-3, including a learning rate of 0.1 for Zg. Since GFlowNets can learn to sample proportionally to a reward distribution, we transform the optimization criteria as reward function R(x) in the following way:.
Paired OAS experiment: optimizing instability index. We label the training sequences using BioPy- thon (Cock et al. 2009) and transform the instability index to reward by performing the following trans-
formation: (x)
where inder is the instability index of x.
CDR H3 experiment: optimizing binding likelihood. We use the training classifier used for eval- uation get the binding likelihood of the training sequences. The reward transformation in this task is R(x:) 2Pindd - 1.
E
Additional experiments
E.1
Effect of K
To understand the effect of the number of sampling steps K, perform an ablation experiment on K using the instability index discriminator model as the gradient guidance. For this experiment, we change K E {10, 20, 30, 40, 50} while keeping X = 1. Table I1 shows the results of the experiment. We can see that there is a minimal change in the attribute, data fidelity, and diversity. In a close inspection, we see that as K increases, sampled sequences are more optimized, yet less diverse. Notably, DCS continues to improve until K 40, after which it decrcases.
Table 11: Ablation results on number of samples steps for antibody sequence optimization task. We see generally indifferent yet a slightly optimized attribute in exchange of reduced diversity as K increases.
Instability index
DCS
Edset
10
31.69 + 5.42
0.52 + 0.28
95.35 + 24.35
20
31.96
+ 5.25
0.53 + 0.28
91.23 + 25.47
30
31.41
+ 5.42
0.54 + 0.27
90.34 + 26.55
40
31.32
+ 5.20
0.56 + 0.27
89.07
26.80
50
31.38
+ 5.33
0.55 + 0.27
87.47 26.70
E.2 Statistical analysis on preference conditional generation
To determine the statistical significance of the samples generated with different preferences using gg-SWJS In this case, we utilize three distinct preferences to represent three different treatments. Table 12 shows the result of the test. The table contains three main attributes: the proportion of beta shoets and the instability index, cach analyzed using different weight combinations. The Friedman lest yields low p-values (p 0.001), indicating statistically significant differences between weight arrangements. For example, sequences optimized with w1=1 and w2=0 have beta sheet percentage of 0.44 + 0.01, whereas those with w1=0 and w2-1 have greater instability index of 61.84 + 4.23. These results demonstrate our method's capacity to build sequences based on certain optimization preferences.
from figure 5 we run a Friedman Lest. (Friedman 1937: 1940 7
E.3 Experiments with more objectives
To understand the capability of our method under more objectives, we conduct experiments with 5 objectives; target molecular weight, inverse sequence length, negative GRAVY, % beta sheets, and inverse instability index, with our preference weight w indictating their preference in that order. We set the target molccular
We the SciPy package (bttps://acipy.org)
23

---

Published in Transactions on Machine Learning Research (12/2024)
Table 12: Experiment results for antibody sequence generation for multi-objective optimization task. Here the two objectives are percentage of beta sheets and aromaticity. The results show that og-dWWJS-generate sequences are optimized for different preferences. We use Friedman test demonstrate that the attributes are statistically different.
statistic
p-value
w1=0,w2-1 w1=1,w2-0
w1=0.5,w2-0.5
% Beta shoet Instability index
1564
<0.001
0.44 = 0.01
0.39 + 0.01
0.43 + 0.02
1522
<0.001
36.11 + 2.45
61.84 + 4.23
41.90 + 8.54
weight 23000 and use (molit (mo1vt-230000 (00000)2/23000 as our objective which maximized at 23000. Table shows that with the objectives are satisfied with their corresponding preferences. For example, with [0, 0, 0, 0, IJ the instability index is minimized while with 1, 0J. the molecular weight is closer lo the larget weight.
the results of experiment. We
Table 13: Experiment results for antibody sequence generation for multi-objective optimization task for five objectives.
Molecular weight
Sequence length
GRAVY
% Beta sheets
Instability index
1,,0,,0 0,0,0,001 0
23915.45 + 475.02
231.36 + 3.68
-0.25 + 0.08
0.39 + 0.02
36.40 = 6.14
0,
24727.72
454.44
229.23
3.53
-0.26 = 0.09
0.39 + 0.02
35.63 = 6.26
1,00,
24796.43 472.75
229.49 3.73
-0.28 + 0.09
0.39 + 0.02
35.25 6.06
1,,0,011,0
24942.65
483.04
230.78 + 3.52
-0.24 + 0.08
0.41 + 0.02
32.04 = 4.71
0,0,0,0.1
25273.25 = 373.58
231.70 = 3.24
-0.25 = 0.10
0.39 + 0.02
26.95 = 3.89
F Discussion
Why does discriminator gradient. guidance produce higher quality samples? We speculate that the guidance assists the denoising walk by reducing the discrete space. Morcover, by collaborating with the denoising score, it strengthens the actions taken in relation to particular label, thus enhancing the accuracy of the generated data. There many works, such as Dhariwal & Nichol (2021); Kawar et al. (2022). that report the same phenomenon. We leave the experimental validation of this underlying effect for future work.
24


Reviews:
[{"review_id": "OX2BJoipwM", "reviewer": "Reviewer_E2Zk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n1. Well-written and easy to follow.\n2. Extensive experiments conducted on various datasets.\n\n**Weaknesses**:\n1. What is the core difference from classifier-guided generation? Steps A-C appear to be standard design principles for diffusion models.\n  \n2. Explain the specific steps used in the \"Gradient-guided jump\" more clearly.\n  \n3. The transition back to discrete data has been explored in previous work using continuous-state models to represent discrete data as well [1-2]. It would be beneficial to cite these works and include a brief discussion.\n  \n4. Compared to dWJS, is the proposed method an adaptation for conditional generation? What is its most significant novelty?\n\n**References**:  \n[1] Yan, Q., Liang, Z., Song, Y., Liao, R., and Wang, L. SwingNN: Rethinking permutation invariance in diffusion models for graph generation. TMLR.\n\n[2] Niu, C., Song, Y., Song, J., Zhao, S., Grover, A., and Ermon, S., 2020, June. Permutation Invariant Graph Generation via Score-Based Generative Modeling. In *International Conference on Artificial Intelligence and Statistics* (pp. 4474-4484). PMLR.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper build upon existing work dWJS to improve its capacity for conditional generation. Experiments on various discrete data forms have shown its good empirical performances.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "q3pG2H0CEE", "reviewer": "Reviewer_ZDHe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n- The paper introduces an interesting idea by combining gradient guidance with dWJS for discrete sequence generation.\n- Diverse Applications: The method is evaluated on a range of tasks, including image and antibody sequence generation, demonstrating its versatility.\n- Multi-objective Optimization: The paper explores the application of gg-dWJS in multi-objective settings, an important and challenging area.\n\n**Weaknesses:**\n- Presentation and Clarity: The major weakness is the paper's presentation. Several components are not correctly defined, important details are missing, and the overall exposition needs significant improvement. This severely hinders the understanding and appreciation of the contributions.\n- Lack of Connection to Related Work: The paper lacks a clear connection to related work, particularly *classifier guidance*. The relationship between the proposed method and existing techniques should be explicitly discussed.\n- Insufficient Explanation of dWJS: As gg-dWJS builds upon dWJS, a thorough introduction to the latter is crucial. The paper only partially introduces dWJS in Section 2.2, leaving out important details about the transition from continuous to discrete representations.\n- Ambiguous Notation: See comments below.\n- Lack of Clarity. For instance:\n  - The term \"denoising walk\" in Section 5 needs clarification.\n  - The specific objective used in the antibody sequence generation task requires more detail (\"For each task, we train a smoothed predictor on the antibody sequences and use its gradient guidance to optimize the single objective\".)\n  - The role of w in the multi-objective optimization (Section 6.3) is unclear. Does  w represent the probability of choosing one objective or the other?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes gg-dWJS, a novel method for generating discrete sequences optimized for specific tasks. gg-dWJS extends the dWJS framework (Frey et al. 2024) by incorporating gradient information to guide the Markov Chain Monte Carlo (MCMC) sampling process on a smoothed data manifold. This guided sampling allows for generating high-quality discrete sequences, such as images and antibody sequences, tailored to particular objectives.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Requested Changes:**\n1. Thorough revise and rewriting: The paper requires a major rewrite to improve clarity and accessibility. Focus on clearly defining all concepts, providing essential details, and ensuring a logical flow of ideas.\n2. Detailed Explanation of dWJS: Provide a comprehensive introduction to dWJS, including all necessary components and the mechanism for transitioning between continuous to discrete spaces.\n3. Clarify Connection to *Classifier Guidance*: Explicitly discuss the relationship between gg-dWJS and classifier guidance, highlighting similarities and differences.\n4. Precise Notation: Ensure all notation is clearly defined and consistently used throughout the paper.\n5. Elaborate on Experimental Setup: Provide more details on the experimental setup, including the specific objectives, and how the guidance where trained and evaluated (e.g., how good shall this classifier needs to be?)\n6. Expand Discussion: Deepen the discussion on the trade-off between data fidelity, diversity, and optimization, and provide insights into the limitations and future directions of the work.\n\nOther Comments:\n- Section 2.1: Define the meaning of the \"half arrow.\"\n- Section 2.3: Provide a precise definition of the max operation over the vector R(x). This seems not well defined.\n- Section 3: Clearly define P(C|Y), c, C, y, and Y.  Clarify the sentence *\"Formally, we use (Math expression) to perform MCMC for K steps…\"* (It is not clear what does this mean in practice)?\n- Algorithm 1: Explain the rationale behind initializing y_0 as the sum of noise and a uniform random variable.\n- Section 5: Clarify the term \"denoising walk\" and its connection to denoising in Neural Empirical Bayes (Tweedie's formula). Provide more details on the objective used in the antibody sequence generation task (how do you train the classifier models?).\n- Section 6.3: Clearly define the role of w in the multi-objective optimization.\n- Section 6.4: Suggested change \"Effect of \\lambda\" to \"Effect of the guidance strength.\"\n- Section 8: Expand the discussion on the trade-off between diversity and optimization.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Px9uaPQYMt", "reviewer": "Reviewer_bSDE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Gradient guidance is a simple extension of dWJS that is easy to apply.\n* Evaluating gg-dWJS to multi-objective optimization is an important experiment that I'm glad was included.\n\nWeaknesses:\n* Due to gradient guidance being simple, we expect strong experiments or analysis of the technique. Gradient guidance is not a novel idea since it has been used in classifier guidance [1], Gibbs with gradients [2, 3]. Unfortunately, the antibody experiments don't present a compelling reason to use this technique.\n* The beta sheet guidance is especially puzzling since the base distribution should already capture the desired beta sheet composition. The improvement in beta sheet is 0.408 compared to 0.393 which is insignificant. There is little justification of why we care about this guidance.\n* Instability is a property I have not seen before for antibody optimization. The significance of the improvement, 31.3 compared to 34.1, is once again unclear.\n* Technical and experiment details are missing such as how the classifiers were trained. It is also unclear how a continuous valued property (beta sheet percentage, instability index) was utilized as a discriminator in gradient guidance. I have looked in the appendix and could not find these details. Hyperaparameters such as inverse mass, friction in algorithm 1 are not explained either.\n\n\nReferences:\n[1] https://arxiv.org/abs/2105.05233\n[2] http://proceedings.mlr.press/v139/grathwohl21a/grathwohl21a.pdf\n[3] https://arxiv.org/abs/2406.01572v1", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work presents gradient-guided discrete walk-jump sampling (gg-dWJS), an extension of guiding dWJS towards different properties, and applies it to antibody sequence optimization. The extension is fairly straightforward by adding the score of a classifier during dWJS sampling. To perform multi-objective optimization, multiple scores are combined with Pareto optimal scalarization. The method is first demonstrated on binary MNIST before moving to antibodies benchmarks following the original dWJS work. Two guidances are demonstrated: instability index and beta sheet. The results show there are improvements on both fronts though beta sheet improvement is minimal and the significance of the instability index improvements is also small. The baselines compared to include GPT-4o, GFlowNets, and VDM.  Ablations are performed on guidance parameters.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "* Technical details need to be clarified such as how the discriminators were trained and how their predictions were used to parameterize the gradient guidance scores. \n* $\\lambda$ is referenced in the experiments as the guidance weight but is not specified in the methods.\n* What is inverse mass and friction?\n* Please explain the significance of the improvements achieved with gradient guidance. Why should protein engineering or discrete optimization community care about the improvements?\n* My recommendation would be to find discrete optimization tasks with established benchmarks such as the original gflownets paper. There needs to be stronger benchmarks and experiments since the technical novelty of this work is limited. Another issue is the baselines such as GPT-4o or VDMs which are not meant for biological problems.\n\n\n\n\n[1] https://proceedings.mlr.press/v162/jain22a/jain22a.pdf", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: There was a significant discussion of the paper among the reviewers and I commend the authors and reviewers for thoroughly evaluating the paper. The main concerns were around the degree of technical novelty and scale of the experiments. The authors have addressed the latter with a significantly expanded set of experiments. The former seems to a matter of degree. There was some consensus that the novelty may not be high. However, since this work is highly interdisciplinary, novelty is also a matter of the perspective of the reviewer. Some may view the approach as incremental, while others may view the approach as more novel in the peptide generation area. I recommend acceptance of the revised manuscript taking into account the evaluation policy of the journal and the challenges of judging novelty of interdisciplinary work.
Decision: Accept (poster)