Steering Generative Models with Experimental Data for Protein Fitness Optimization
Authors: Jason Yang
Wenda Chu
Daniel Khalil
Raul Astudillo
Bruce James Wittmann
Frances H. Arnold
Yisong Yue
Venue: NeurIPS 2025 poster ()
Keywords: protein
generative model
diffusion model
discrete diffusion model
guidance
language model
active learning
Bayesian optimization
Primary area: machine_learning_for_sciences

Abstract: Protein fitness optimization involves finding a protein sequence that maximizes desired quantitative properties in a combinatorially large design space of possible sequences. Recent advances in steering protein generative models (e.g., diffusion models and language models) with labeled data offer a promising approach. However, most previous studies have optimized surrogate rewards and/or utilized large amounts of labeled data for steering, making it unclear how well existing methods perform and compare to each other in real-world optimization campaigns where fitness is measured through low-throughput wet-lab assays. In this study, we explore fitness optimization using small amounts (hundreds) of labeled sequence-fitness pairs and comprehensively evaluate strategies such as classifier guidance and posterior sampling for guiding generation from different discrete diffusion models of protein sequences. We also demonstrate how guidance can be integrated into adaptive sequence selection akin to Thompson sampling in Bayesian optimization, showing that plug-and-play guidance strategies offer advantages over alternatives such as reinforcement learning with protein language models. Overall, we provide practical insights into how to effectively steer modern generative models for next-generation protein fitness optimization.

Steering Generative Models with Experimental Data for Protein Fitness Optimization
Jason Yang*
Wenda Chu
Chemistry & Chemical Engineering California Institute of Technology
Computing & Mathematical Sciences California Institute Technology
Daniel Khalil
Raul Astudillo
Computing & Mathematical Sciences California Institute of Technology
Computing & Mathematical Sciences California Institute of Technology
Bruce J. Wittmann Office of the Chief Scientific Officer Microsoft Corporation
Frances H. Arnold
Chemistry & Chemical Engineering Biology & Biological Engineering California Institute of Technology
Yisong Yue
Computing & Mathematical Sciences California Institute of Technology
Abstract
Protein fitness optimization involves finding a protein sequence that maximizes desired quantitative properties in combinatorially large design space of possible sequences. Recent advances steering protein generative models (e.g.. diffu- sion models and language models) with labeled data offer a promising approach. However, most previous studies have optimized surrogate rewards and/or utilized large amounts of labeled data for steering, making it unclear how well existing methods perform and compare to each other in real-world optimization campaigns where fitness is measured through low-throughput wet-lab assays. In this study, we explore fitness optimization using small amounts (hundreds) of labeled sequence- fitness pairs and comprehensively evaluate strategies such classifier guidance and posterior sampling for guiding generation from different discrete diffusion models of protein sequences. We also demonstrate how guidance be integrated into adaptive sequence selection akin to Thompson sampling in Bayesian optimization, showing that alug-and-play guidance strategies offer advantages over alternatives such as reinforcement learning with protein language models. Overall, we pro- vide practical insights into how effectively steer modern generative models for next- sext-eenerttion protein fitness optimization.
Introduction
Proteins, sequences of amino acids, can be optimized for useful properties such as binding affinity, catalytic activity, stability, numerically quantified "fitness." However, protein optimization is
"yyue@ caltech.edu
Thhese authors contributed equally to this work.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Generative: Zero-Shot
A Trainffinetune generalive model (prior) on bulon natural
Sample with high Ikolhooods and obtain
SGPO
B
Updatelgudd de generalive model
Generative model. pretmiined on the distribution of natural sequences
Sample ww high (Kellhooda and obtlin fitness labels
Optimized Variant
Optimized Varlant
La. diflusion model, language model
Semple & verienta to have and
froh evalutioaay Hkelinood
A residusa is fexiible
Generative: Adaptive
Supervised
Sample obsain fitness labels
Sample and obtain fitness labels
Random intia!
Update generative model
Optmized Variant
Calcuulate lkclihodds for all veriante
Train a suporvised model
Optinized Varini
library
Fireet
Intialize gonerative model
Construct X% Ikelboocis)
Predict firness for entire
Sample variants likeyy high fliness
design space
eg. N residues>
e.g.
Suggesi top new varianis
Figure Comparison of steered generation for protein optimization (SGPO) to other ML- assisted workflows for protein engineering. (A) SGPO involves initializing generative prior model to sample sequences with high natural likelihoods and steering that model with assay-labeled fitness data. Optimization is difficult because the design space is vast, and the throughput of wet-lab fitness assays (Erlenmeyer flask icon) is low, so adaptive learning across multiple iterations is beneficial. Previous methods have utilized generative models such (B) fully zero-shot methods that sample highly natural sequences but do not utilize labeled fitness data or (C) those that only utilize labeled fitness. (D) Alternatively, supervised approaches involve enumerating to calculate fitness predictions for all variants in a design space, limiting them to optimizing few residues (i.e., N < 9).
challenging: the design space of proteins is enormous, as a protein of length M can be constructed in 20dd different ways, of which only a negligible fraction are functional (Romero & Arnold 2009). Moreover, most wet-lab assays only provide 102 - 10s fitness labels. Consequently, researchers often rely on directed evolution, an iterative process aiming incrementally improve protein fitness (Packer& & Liu 2015) through multiple rounds of mutation and experimental screening. In each round, a protein mutated, the variants' fitnesses are measured, and the most beneficial variant is selected for the next iteration. However, this approach can be slow, often accumulating only one mutation per round, and inefficient, as it performs a local search limited closely related protein sequences.
In recent years, there has been a growing interest in developing machine learning aMsssassisted methods to optimize protein fitness more efficiently (Yang et al. 2019 Wittmann et al. 2021a Hie &
Yang 2022; Yang et al., 2024 2025c). Many recent studies have focused on generative approaches
combining unlabeled and labeled data for protein design. Broadly, these methods achieve conditional
generation by steering generative priors of natural protein sequences (Freschlin et al.) (022) using
fitness data, thereby enabling incorporation of the steered models into adaptive optimization cycles (Hie & Yang 2022). We refer to this class of methods as Steered Generation for Protein Optimization (SGPO). These methods address the individual limitations of previous approaches (Fig. DA Table D. First, SGPO leverages labeled data, which is essential for fitness goals that deviate from natural functions (e.g., engineering enzymes for non-native activities (Arnold 2018 Yang et al. 2025b)), unlike zero-shot methods relying solely on generative priors of natural sequences (Generative: Zero- Shot, Fig. D8. Hie al. 2023) Sumida et al. 2024 Fei et al. 2025 Seki et al. (025) Lambert et al.) 2025). Second, generative priors (Wu et al. 202 Hsu et al. (024)) sample sequences with high evolutionary likelihoods and potentially higher fitness, giving these methods a significant advantage over approaches relying exclusively on labeled data (Song & Li 2023; Stanton et al.] [022] Gupta |rooeBrookes & Zou 2019 Brookes& & Listgarten 2020 Jain et al. [022] Kim et al. 2025 Angermuell et al.
2020: Hie & Yang) 2022) (Generative: Adaptive, Fig. IC). Finally, SGPO scales to larger design spaces, unlike most supervised ML-assisted directed evolution (MLDE) approaches, which require enumerating and scoring all variants in the design space (Supervised, Fig. ID) (Wu et al. 2019
Wittmann et al., 20216] Yang et al. 2025b Li et al. 2025a Vornholt et 2024, 4 Jiang et al. 2024
Hsu et al. [022 Ding et al. 2024 Hawkins-Hooker al. 2024 Zhao et al.) 2024a Thomas et al.
2025 Sun et al. 2025).
Despite these advantages, SGPO methods still face practical limitations in real-world fitness opti-
mization, particularly across two major classes of approaches: guiding discrete diffusion models
Nisonoff et al.) 2025 Stark et al. 2024 Klarner et al., 2024 Gruver et al. 2023 Lisanza et al.
2

---

Table 1: SGPO is general approach for protein fitness optimization that does not face the individual limitations of other strategies. Namely, SGPO utilizes zero-shot knowledge from the natural distribution of proteins, can be guided by assay-labeled fitness data, and can optimize many residues (N) simultaneously. Beyond those listed here, there are many other studies that combine different elements of these approaches.
Approach
Prior In- Assay formation Fitness Used?
Scales large
Protein Examples (non-exhaustive)
Used?
N?
SGPO
Lisanza et al. (2025) ) Widatalla et al.
(024))
Stocco al. (0024):
Nisonoff et 2025
Brookes et al. (2019) Blalock et al.
2025
Goel et al. (2025); Huang et al. (2025)
Generative: Zero-Shot
x
Hie et al.
2023): Sumida et al. (2024); Fei
et al. 2025
Seki et (2025):
Lambert al.
(2025)
Generative: X
Song & Li (2023); Jain et al. (2022); Anger
Adaptive
mueller al. (2020); Stanton
2022);
Brookes & Listgarten (020)
Supervised
Wittmann et al. (202 b0;;b);
Ding et al. (024))
Hawkins-Hooker et al.
(2024); ; Zhao et al.
(2024a); Sun et al. (025
2025
Goel al. 2025) and finetuning models such protein language models (PLMs) through
reinforcement learning (RL) (Ruffolo & Madanil 2024 Widatalla et al. 2024, SSocco et al. 2024
Blalock al. 2025 Wang et al. (022cc). The limitations of prior work summarized as follows:
(1) Few previous studies have explored steering with few (102 - 10$) labeled sequences (Lisanza
al. 2025; Stocco al., 2024) for protein optimization based on real fitness data, e.g. activity or
|
fluorescence, rather than computational surrogates (Lisanza et al. 2025 Blalock et al. 2025 (2)
Most studies only evaluate one type of generative prior and steering strategy, so unclear how different combinations perform in practice. (3) There is room to incorporate principles from adaptive
optimization, such as uncertainty-awar aware exploration (e.g.. Bayesian optimization), which have shown
clear benefits in protein engineering (Vornholt et al. 2024 Yang et al. 20256).
In this study, we aim understand the best practices for integrating SGPO into real-world engineering workflows. We focus here on modern generative models (i.e., discrete diffusion, language models) but acknowledge that other related methods
relevant, such as those based on variational autoencoders
(Brookes et al. 2019 Torres et al. 2024) and other adaptive search strategies (Kirjner et al.) 2024 Sinai et al.] 2020 Ren al. 20222) We explore the following questions: Which steering strategies perform best, and with which types of models? How can we utilize uncertainty to better explore the design space when performing guidance? Overall, we make the following key contributions:
1. We motivate SGPO as useful, general framework and contextualize existing methods for protein optimization under this umbrella.
2. We comprehensively evaluate design decisions for SGPO, including different generative
models for sequences and steering strategies (Fig. 2 P Section 2 for protein optimization with few fitness labels.
offering best practices
3. We introduce ideas from adaptive optimization into SGPO by proposing method that ensembles multiple plug-and-play fitness predictors and leverages their predictive uncertainty to enable more efficient exploration.
4. We are the first to adapt decoupled annealing posterior sampling (Zhang et al. 2025) for SGPO, and this type of plug-and-play guidance has the strongest performance overall.
On the TrpB, CreiLOV, and GBI protein fitness datasets, we find that SGPO methods can consistently identify high-fitness protein variants. In particular, our results highlight the advantages of plug-and- play guidance with diffusion models over finetuned language models-offee and lower computational cost. To support future research and real-world adoption, our extensive, user-friendly code is available at httptts://// htthttss://gtttii //gi: www/jsunmnn......... jumn/n//Sny//PP
greater steerability
3

---

2
Related work
Generative models for discrete sequences. The most widely adopted generative models for natural protein sequences are PLMs, such as autoregressive transformers (Nijkamp et al. (022)) and masked language models (Rives al. 202T). Increasingly. various diffusion model (Ho al. (2000)
architectures have shown efficacy for modeling discrete data (x) (Li et al. 2025b), such as protein sequences (Alamdari et al., 2024, Wang al., 2024), leveraging many similar learning techniques
such as masking or autoregressive decoding (Sahoo et al.] 2024 Lou et al. 2024 Nie et al. 2025
Shi et al. 2024) (Fig. D. These generative prior models p(x) can be categorized broadly into
Two types: those that perform diffusion in a continuous latent space (Li et al. 2022 Chen et al.
2023b Dieleman et al., 2022) and those that diffuse directly over discrete space (Fig. 2). In the protein domain, it has also been shown that latent diffusion over embeddings from PLMs can be more effective (Meshchaninov et al., 2025) Chen et al., 2023a; Torres et al. 2025). Alternatively, models performing diffusion in discrete space use transition matrix to update all discrete states in each timestep (D3PM) (Austin et al., 202T), which has later been formulated as continuous-time
Markov chains (Lou et al. [2224] Campbell al. 2022 2024 Schiff al.. 2024). Two common ways to add noise to discrete sequences are to use uniform noise matrices or absorbing state (masking) matrices (Fig. _ These have been followed by simplified frameworks showing some of the highest performance for modeling natural language, such as masked diffusion language models (MDLMs) (Sahoo al. 2024 Hoogehoom al., 2022 (Shi al. 2024 and variation that uses uniform noise called uniform diffusion language models (UDLMs) (Schiff et al. 2024). We elaborate more on these
methods in Section A.3
Continuous Diffusion
Discrete Diffusion (Uniform)
PKGYF_RREEE NAGSF K.RRIR
RFFDD__XRVEE
AFADE_XCMLLI
NKGGFF_RERIR
(e.g. ESM2)
KAGYF_RERKE PINCYF. JRQKRR
HEWYF_RGEIR
AFMDE KAMLN
rncomitnacionllas
latent representations
AERER
Decoding
Head
Discrete Diffusion (Absorbing State)
M[K]O_.IR
[*000000000000000000000
REG
HK [X] REK]R
REX]R
fonward diffusion
difluiio
Figure 2: Overview of different approaches to train diffusion models over discrete state spaces. During inference, a noised latent representation or sequence is decoded into a reasonable sequence (bottom track for each method). [X] refers to a masked token.
Plug-and-play guidance strategies. An advantage of diffusion models is the ability to perform plug-and-plas guidance based on fitness labels (y) without finetuning the generative prior model weights, resulting in reduced training costs and potentially strong signal despite having few (~ 102) labels. Guiding a continuous diffusion model often involves skewing the learned score function using gradients from supervised value function that can predict labels y from data (Chung et al. 2023 Zheng et al. 2025; Soares et al. 2025). These methods are often referred to as posterior sampling, as they aim sample from the posterior distribution, p(xly). Recent works extend this idea guiding discrete diffusion models. Classifier guidance (CG) (Nisonoff et al. (025) skews the rate matrix of the reverse time Markov chain of discrete diffusion models using a time-dependent value function,
p(y x2, t); variable splitting methods (DAPS) (Zhang et al. 2025 Chu et al. 2025) use discrete diffusion models as denoisers and only require a value function of clean data, ply xo); diffusion optimized sampling (NOS) (Gruver et al. 2023) trains value function on continuous embeddings of discrete tokens and optimizes the embedding for higher fitness; sequential Monte Carlo methods (SMC) (Li et al., 2024a] Uehara et al.) 2025 Wu et al. 2024 Lee et al. 2025a Singhal et al.) 2025)
evolve multiple particles from series of distributions to approximate the posterior distribution in limit. We explain these methods in more detail in Section A.4, along with other variations on the guidance process. In this study, we focus on CG, DAPS, and NOS as guidance techniques (Fig. 3.. Future work could also consider guidance techniques for autoregressive language models, such as future discriminators for generation (FUDGE) (Yang & Klein| 202I), plug and play language models (PPLM) (Dathathri et al. (020), and twisted SMC (Zhao et al. 2024b] Amin et al., (025b). Additionally, Xiong et al. (2025) demonstrate how guidance generalizes to masked language models and order-agnostic autoregressive models.
Reinforcement learning via model finetuning. We consider RL. broadly here as techniques that achieve conditional generation by finetuning generative models with labeled data, thus pushing

---

Prior
Train from scratch
Finetune
data homologous protein sequences
explored this study other possibiities
existing prior
ProGen2
Evo
Learning Strategy
Diffusion: Continuous
Diffusion:
Diffusion:
Autoregressive Language Model
Variational Autoencoder
Latent Space Bo
Discrete Uniform
Discrete
Absorbing State
Steering
Classifier Guidance
Posterior
NOS
SMC
FUDGE, PPLM
DDPP, RTB, DRAKES
Policy Optimization
Supervised Finetuning
Sampling
data aassy finness labels
Model Finetuning
Plug-and-Play Guidance Strategies
Figure 3: Methods design space for SGPO: non-exhaustive landscape of generative models for protein sequences and methods to steer them with labeled data. Three major types of diffusion models for sequences include those that perform diffusion over continuous space and those that perform diffusion over discrete space with a uniform or absorbing state (masking) noising process. Various types of guidance strategies are compatible with certain models, in green (NOS: diffusion optimization sampling. SMC: sequential monte carlo, FUDGE: future discriminators for generation, PPLM: plug and play language models, DDPP: discrete denoising posterior prediction, RTB: relative trajectory balance, DPLM: diffusion protein language model, BO: Bayesian optimization). Differently, language models and variational autoencoders can be aligned with labeled data via reinforcement learning such as policy optimization or supervised finetuning.
those models to produce more favorable generations. There are emerging RL techniques applied to discrete diffusion models, including discrete denoising posterior prediction (DDPP) (Rector- Brooks et al. (005),, relative trajectory balance (RTB) (Venkatraman et al. 2024 Bartoldson et al. |025]] Venkatraman et al. 2025), and direct reward backpropagation with gumbel softmax trick (DRAKES) (Wang et al. 2025a). While the above strategies specific to discrete diffusion models, supervised fine-tuning (SFT) and policy optimization are two important techniques used in RL that can be broadly applied to generative models such language models (Fig. 3). Policy optimization
has generally shown better performance than SFT (Stocco et al. 2024 Blalock et al. 2025); in particular, direct preference optimization (DPO) is often used for its algorihhmic simplicity and ease of training (Rafailov et al.) 2023) (details in Section A..4. RL has demonstrated utility for aligning generative models of proteins (language models, inverse folding models, variational autoencoders)
with properties like stability (Widatalla et al.) 2024 Blalock et al. 2025 SSocco et al. 2024 Lim al. 2025), but these methods have high computational costs of finetuning and may require large amounts of labels (> 10²) to effectively steer generations. We include DPO with an autoregressive PLM (finetuned ProGen2 Nijjaamp et al. 2023) as baseline.
Adaptive optimization. Protein engineering is commonly conducted through adaptive workflows such as directed evolution Packer & Liu (2015) or ML-based approaches such as Bayesian optimiza- tion (Frazier 2018; Stanton et al. 2022). These methods follow iterative loop: labeled data is collected via expensive wet-lab assays, a surrogate model p(y[x) is trained or updated, an acquisition function implied by the surrogate is used to propose new sequences to evaluate, and the cycle repeats (Hie & Yang] 2022 Vornholt et al. 2024 Yang et al.) (0026)). The surrogate model, often a Gaussian process or a deep ensemble, provides uncertainty estimates, which used by an acquisition function (e.g., expected improvement, Thompson sampling) to balance exploration and exploitation of the design space. In this study, we adapt these ideas guide diffusion models for protein sequence generation, as described in Section
A closely related line of work is latent space Bayesian optimization (Maus et al. 2022 Stanton et al. 2022)) Gomez-Bombarde et al. 2018; Castro et al.. 2022; Torres et al.) 2024 Lee et al. (025b)), which searches for optimal sequences within latent space- typically learned by an autoencoder, which can implicitly capture a prior on natural protein sequences. In this work, we compare against APEXGo (Torres et al. [024), method that performs trust-region Bayesian optimization in the latent space of a variational autoencoder trained over protein sequences. There are also related methods that involve conditional sampling from a prior (Brookes) (et al. 2019). However, we note that SGPO offers greater flexibility by avoiding reliance on an explicit latent space, which enables the use of modern, more powerful generative models such as diffusion models and protein language models that are not easily accommodated by traditional latent Bayesian optimization pipelines.
5

---

3
Problem setup
We focus on evaluating methods that fall under SGPO, where the primary downstream task entails starting from a known sequence with some level of fitness for target objective (i.e. activity, stability, fluorescence, binding, etc.) and identifying a modified sequence with maximized fitness, where real-world fitness can only be measured for 10 to 10 sequences. Our goal is to sample sequences with maximum fitness y from the generutive prior p(x), which is trained on the multiple sequence alignment (MSA) of homologous protein sequences that are evolutionarily related to known protein with some level of desired fitness (details in Section A.3). This model can be thought of as capturing the distribution of sequences with high likelihood from given protein family.
During inference, sequences can be sampled unconditionally from p(x), or sampling can be guided using supervised model of the form p(y|x) x cxp(f(x) where f(-) is a learned fitness predictor- also referred to as the classifier or value function. This predictor is trained on small number of labeled sequence-fitnen The goal of guided sampling is to generate protein sequences from the posterior distribution, p(x(y) y x We use computational oracle to acquire and evaluate fitness labels to simulate
pairs (typically in the hundreds) to reflect practical data limitations.
p(x) exp([(x)/8) how fitness would be measured in a real-world campaign. Details on training and guidance with the value function are provided in Section (e.4 and Table A2] As an alternative steering method to guidance. we finetune the generative prior with labeled data using an autoregressive language model (ARLM) and DPO, which serves baseline. We further compare to a baseline of latent space Bayesian optimization. The strength of steering is tuned by method-specifee hyperparameters.
4 Results
Table 2: Summary of datasets used in this work. Train and test fitness refer to the number of fitness labels used for training and testing the oracle. We focus on TrpB and CreiLOV, with some of the GB! I results moved to the Appendix. While the TrpB dataset has a lot more training labels, it may more difficult to learn due to relatively high amounts of epistatic effects between residues inon-additivity of mutation effects).
Dataset
Length Targeted Residues
Design Space
MSA
Train
Test Fit- ness
Reference
Size
Fitness
TrpB Enzyme
389
117, 118, 119, 162, 166,
N=15
5.7e4 75,618
23,313
Johnston
182, 183, 184, 185, 186.
et
al
Activity
227, 228, 230, 231, 301
(2024)
CreiLOv
119
All
N=119
3.7e5
6,842
2,401
Chen et (0023c)
Fluorescence
GB1 Binding
56 All
N=56
126
3.9e6
9.6e4
Olson al. (2014)
We study three proteins, the TrpB enzyme (Johnston al., 2024). the CreiLOV fluorescent protein (Chen et al. 2023c). and the GBI binding protein (Olson el al. 2014) due to the availability of fitness data across many residues (Table We focus protein fitness optimization to design space of 15 residues in TrpB (only these positions are allowed to vary) and all 119 and 56 residues in CreiLOV and GB1, respectively. For each protein's 's variants, we evaluate fitness by approximating it via supervised oracle trained on a large amount of real data (Section A.2).
4.1 Model pretraining captures the distribution of evolutionarily related protein sequences and enables sampling sequences with high fitness
Based on the methods explained in Section A.J and A.3 we trained generative priors on natural sequences from the MSA, focusing on continuous diffusion models (Continuous), discrete diffu- sion models with uniform (D3PM, UDLM) and absorbing state noising processes (MDLM). and autoregressive language models (ARLM) (Table Overall, the trained models capture the natural distribution of protein sequences, with the D3PM models seeming to match the distribution the most

---

Table 3: Summary of generative priors evaluated in this work. Each generative prior was trained on an MSA of homologous natural sequences. All denoising processes were modeled using transformer architecture (Section guidance experiments.
A.3 Italicized models were further explored in downstream
Model
Type
Noise
# Params
Notes
Continuous Continuous
Gaussian
27.9 M
Continuous- Diffusion
25.5 M
diffusion over ESM embeddings
ESM
D3PM-
37.9 M
Baseline
Discrete Diffusion
D3PM
Uniform
37.9 M
finetuned from Evo 38M-Uniform
(Alamdari al. 2024) uniorm diffusion langauge model
UDLM
28.6 M
MDLM
Discrete Diffusion
Absorbing
28.6 M
masked diffusion language model
ARLM
Language
n/a
151
autoregressive language model fine- tuned from ProGen2-small (Nijkamp)
Model
et al. 2023
The two different diffusion models over continuous space show comparatively lower performance, and diffusing over the latent space of ESM embeddings does not boost performance this task. The UDLM model has low performance due to mode collapse (Fig. A3 Fig. A5). Future work could finetune the pretrained diffusion protein language model (DPLM)
closely while also generating sequences with high diversity (Fig. Fig. A3
MDLM (Wang al. 2024).
Overall, we found that pretrained priors sample protein variants that have higher mean fitness, which corroborates previous studies finding that sequences with higher evolutionary likelihood are also likely to have higher fitness (Li et al., 2025a Hie et al., (203)). Based on these results, we proceeded to perform remaining experiments with model from each category of model type, namely the Continuous, D3PM, MDLM, and ARLM models.
Mean Fitness
TrpB
Random
Perplexity
Target
4.
Continuous
Continuous-ESM
D3PM-Baseline
D3PM
Perplexily
20
C
Mean Fitness
D
CreiLOv
UDLM
MDLM
ARLM
20
Diversity
Figure 4: Pretrained generative priors capture the target distribution of naturally occurring sequences that are homologous to TrpB (A-B) and CreiLOV (C-D), respectively. Lower perplexity corresponds to higher likelihood in the model. The diversity of sequences was computed as the average Shannon entropy of mutated positions with mean fitness corresponding the oracle predictions. While the various models largely achieve comparable performance, the D3PM models capture the target distribution with the highest fidelity, whereas the UDLM model is prone to mode collapse. For each model, 1000 sequences were sampled and repeats were allowed to approximate the distribution. To approximate the target distribution, 1000 sequences pretraining. Perplexity was calculated by passing generated sequences through the 764 M parameter ProGen2-base model. More details on model training can be found in Table and Section A.3 and GBI results are provided in Fig. A4
sampled from the MSA used for

---

Continuous
D3PM
MDLM
1.0
Mean Fitreess
C
DAPS
0.5
NOS
ARLM+DPO
0.0
TrpB
Unconditional
Unconditional
Mean Fitness
5-
D
E
(ARL
CreiLov
Mean Fitness
GB1
Diversity
Diversity
Diversity
Figure 5: Pareto boundaries demonstrate the trade-off between generating sequences with high fitness and high diversity for TrpB (A-C). CreiLOV (D-F), and GB1 (G-I). Sequences sampled from the generative models (Continuous, D3PM, and MDLM),, after guidance with labeled fitness data, are enriched in high-fitness protein variants, and most methods show higher performance than the ARLM+DPO baseline. Larger circle indicates stronger guidance strength hyperparameter (excluding NOS), specified in Table Each experiment was repeated using 10 different standardized sets of 200 unique sequences used for steering, each drawn from the D3PM prior, and error bars show standard deviation. Mean fitness and diversity were calculated based on 200 generated samples, with diversity calculated the average Shannon entropy of amino acids at mutated positions. Unconditional refers to sequences sampled from the prior with no guidance.
4.2 Evaluating SGPO design choices
Impressively, steering with modest amounts of labeled data (200 sequence-fitness pairs) enables most models and methods to generate sequences with even higher fitness, while sacrificing some generation diversity (Fig. 55. In this low data regime, guidance with diffusion models outperforms DPO with language models; the latter does not enable as much steerability. CG and DAPS enable the strongest steerability overall, but DAPS outperforms CG for the continuous models (Fig. 5S D). In general, guidance seems to work similarly for uniform diffusion (D3PM) and to absorbing state diffusion (MDLM). Overall, the continuous diffusion models do not perform as well as other models, as the prior does not capture the distribution of natural sequences with high fitness as well (Fig. ()) NOS does not seem to allow for as much steerability, despite an extensive hyperparameter scan (Table A3 Finally, we conducted a closer analysis of the number of unique sequences generated by the steered models and confirmed that most models produce entirely novel sequences, suggesting that they not over-steering (Fig. A6).
4.3 "Thompson sampling" using an ensemble of classifiers is effective for adaptive optimization
Next, we performed adaptive optimization experiments, which mimic real-world protein engineering scenarios and follow a setup similar to batch Bayesian optimization: in each round, a batch of sequences is sampled, evaluated for fitness, and used to retrain supervised value function that guides sampling from the pretrained prior. We focused on the MDLM models with the CG and DAPS guidance strategies, these combinations achieved the best performance in our earlier set of experiments (Fig. 5). Based on findings from these previous experiments, we selected the ideal guidance strength hyperparameter to balance fitness and diversity ensuring high predicted fitness without significantly compromising sequence diversity (Table A3).. For both guidance strategies, we employed an algorithm akin to Thompson sampling (Kandasamy et al. 2018 Russo al., 2018), drawing different value function from frequentist ensemble of neural network regressors to guide the generation of each new sample (Yang et al. 2025b). Pseudocode for our adaptive optimization algorithm is provided in Section A.5

---

Plug-and-pll and-play guidance strategies outperform baselines such as DPO with an ARLM, sampling just from the unconditional generative prior, and latent space Bayesian optimization with APEXGo (Fig. 6): Sampled sequences achieve higher values of mean and maximum fitness. Furthermore, campaigns using an ensemble of value functions and "Thompson sampling" achieve higher maximum fitness than those using only single value function for guidance (Table A4), which may be because these models enable more exploration of sequence space (Fig. A7). However, it is difficult to ascertain wither CG or DAPS works better guidance strategy, the performance is highly dependent on the guidance strength hyperparameter, and the optimal hyperparameter will not typically be known in a real-world campaign. Because the oracle may not capture the true nature of the protein fitness landscape, also suggest making relative comparisons here rather than absolute comparisons between model performance.
TrpB
CreiLov
GB1
1.5
(MDLM)
Fitness Achieved
DAPS (MDLM)
1.0
Continuous
ARLM+DPO DPO
Unconditional
0.5
(MDLM)
APEXGO
Random
0.0
200
400
600
800
200
400
600
200
400
600
# Samples
Samples
Samples
Figure 6: Maximum/mean fitness achieved ímproves over multiple iterations of steering in an adaptive setting similar to batch Bayesian optimization for TrpB (A), Creil.OV (B), and GB1 (C). 100 sequences were sampled in each round. Within each round, an ensemble of 10 value functions (classifiers) was trained on fitness data from all previously queried samples, and each new sample was generated by the MDLM model guided with value function sampled from the ensemble (akin to Thompson sampling). Only unique, novel samples were acquired. Guidance strength parameter is provided in Table A3 Error bars show standard deviation between 5 different random initializations.
5 Discussion
In this work, we conduct comprehensive study of SGPO methods and demonstrate that it is an effective approach for protein fitness optimization, by capturing the distribution of natural protein sequences with generative prior and then steering the generations with labeled data. We find that DAPS with discrete diffusion models has the highest performance overall, and plug-and-play guidance-based strategies are generally more effective than finetuning language models; the latter can be difficult when only few fitness labels are available. SGPO approaches also outperform latent space Bayesian optimization (namely APEXGo), which we attribute to the difficulty in calibrating the trust region in very low-data regimes with limited rounds of optimization and the fact that latent space Bayesian optimization relies hcavily the structure of the latent space learned during generation model training, which can limit extrapolation to high-fitness but unnatural variants.
Using plug-and-play guidance approaches has other advantages. First, only one hyperparameter (guidance strength) needs to be tuned. In real-world engineering scenarios, even the absence of ground truth fitness labels, one practical approach to selecting the guidance strength is to scan over values and choose the highest setting for which n generated sequences remain unique and novel relative to previously measured sequences, where corresponds to the screening throughput available for the next round. By contrast, for DPO, various hyperparameters need to tuned, and the training process has to be monitored closely. Even for NOS, different parameters such the step size, the number of steps, and the stability coefficient must be tuned together. A further advantage of guidance is the low computational cost required, as the prior model weights are not updated during guidance. petraininnn/inetuniig to obtain each initial prior was achieved on a single H100 GPU in less than one hour while each individual guidance experiment took minutes; pretraining language models took several hours on a single GPU.
Still, there are certain limitations of our work. We focused on proteins with fitness as mostly native function, but it would be interesting to test SGPO on other protein fitness optimization tasks where the pretrained prior may not provide as much utility. We also focused on protein optimization where

---

only z 102 fitness labels were available: different methods, such as RL, may perform better for applications where larger amounts of fitness data are available (Hie Yang 2022; Blalock et al.) 2025). We focused on guidance strategies and did not test DPO or model finetuning- methods with discrete diffusion models, but future work could adapt these methods for discrete diffusion (Borso et al. 2025). Furthermore, for TrpB and for language models, we manually mapped sequences back into the design space after generation (Section A.2), but explicitly building this into sampling techniques, such as inpainting in masked models (Blalock et al. 2025 Goel et al. 2025 may lead to improved performance. We did not consider insertions or deletions, but variable-length sequence generation could be considered in the future. Finally, we did not directly compare existing approaches for protein engineering such as directed evolution for reasons explained in Section A.2
There are several promising directions for future work to improve and extend SGPO methods. For instance, we experimented with guiding generation using value functions sampled from a Gaussian process posterior, enabling principled Thompson sampling from a fully Bayesian perspective. However, the Gaussian process struggled to model high-dimensional protein representations, leading to poor performance. This limitation could potentially be addressed with better kernel choices (Wilson et al., 2016; Michael et al. 2024, Yang et al., (025bb. Recent work has also begun to incorporate mu oiojeeejee optimization (Annadani et al. [2025] Tang et al. 2225a Li et al. 2024bb Chen et al. 2025) and uncertainty quantification (Wu et al., (0255) when guiding diffusion models. Simultaneously, alternatives to acquisition- function--assee approaches are being developed to enable Bayesian optimization in large design spaces where enumeration is infcasible (Bal et al.) (025). Other emerging approaches-cles may offer new opportunities for exploration (Davis et al. 2024 Stark et al. 2024 Tang et al. 2025b).
in spirit to flow matching are being proposed for discrete data and
scheduling could be explored to improve inference, particularly to enhance model amenability to guidance (Wang et al.] 2025b
Finally, for masked diffusion models, strategies such remasking
Peng et al., 2025 Liu al., 2025 Amin et al. 2025a). It will also be interesting to further explore guidance in other discrete domains such as natural language and small molecules (Schiff et al. 2024).
In summary, guiding generative models with labeled data offers a powerful, flexible, and principled framework for protein fitness optimization, as it effectively leverages both the evolutionary infor- mation encoded in natural protein sequences and task-specifir fitness objectives. At the same time, we recognize the potential dual-use risks: such methods could, in principle, be misused to design harmful proteins, underscoring the importance of appropriate safeguards (Baker & Church 2024 Wittmann et al. 2024). In short, our work has examined multiple effective SGPO strategies and offered insights on rest-practices for real-world protein fitness optimization, laying the groundwork for further exploration and wet-lab validation.
Acknowledgments
This work was supported by a U.S. Army Research Office cooperative agreement (W91 W991INF-19-2 0026 to F.H.A.) and Amgen Chem- Bio Engineering award. J.Y. is also supported by the NSF Graduate Research Fellowship Program and the Google PhD Fellowship. We would like to thank Hunter Nisonoff, Jacob Gershon, Lucas Arnoldt, and Chenghao Liu for helpful discussions and Trancesca-Zhoufal Li for help with the TrpB dataset. We would also like to thank Nate Gruver for help with the NOS implementation, Filippo Stocco for help with the DPO implementation, and Nathaniel Blalock for guidance on how to use the CreiLOV dataset.
References
Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Neil Tenenholtz, Bob Strome, Alan M. Moses, Alex Xijie Lu, Nicolò Fusi, Ava Pardis Amini, and Kevin K. Yang. Protein generation with evolutionary diffusion: sequence is all you need. bioRxiv, November 2024. doi: 10.1101//023.09 11.556673. URL httpp::/ htops://wwrr www. biorxiv. org/ http////neent.occmoo wtwpeenttoo..1111111111 1101/2023. 09. 11. 1166755633 Preprint; version 2 posted Nov 4, 2024.
Alan N. Amin, Nate Gruver, and Andrew Gordon Wilson. Why masking diffusion works: Condition the jump schedule for improved discrete diffusion. In Advances in Newral In- formation Processing Systems (NeurIPS), 2025a. doi: 10.48550/arXiv.25055 Xww..506..221111111111 003316 URL https: http:////mnrppp...crttaaaaaaaaa/
NeurIPS 2025 (poster), to appear.
10

---

Alan Nawzad Amin, Nate Gruver, Yilun Kuang, Yucen Lily Li. Hunter Elliott, Calvin McCarter, Aniruddh Raghu, Peyton Greenside, and Andrew Gordon Wilson. Bayesian optimization of antibodies informed by generative model of evolving sequences. In Proceedings of the Thirleenth International Conference on Learning Representations (ICLR), Singapore, April 2025b. doi: 10. Spotlight.
48550WarX1w.2411 URL htttts::///perreviieeeee wwt.
Christof Angermueller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Colwell. Model-based reinforcement learning for biological sequence design. In Proceedings of the 8th International Conference on Learning Representations (ICLR), 2020. URL https: ICLR 2020.
Yashas Annadani, Syrine Belakaria, Stefano Ermon, Stefan Bauer, and Barbara E. Engelhardt. Preference-guided diffusion for multi-objective offline optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2025. doi: 10. (0.48550/r.i...203... 17299. URL https
http////wwuuaa////////tstt///e// 116185 Poster; to appear.
Frances H. Amold. Directed evolution: Bringing new chemistry to life. Angewandte Chemie International Edition, 57(16))4143-4145 April 2018. doi: 10. 10000//ai..001008/////..... URL
https://boalinellbraary wiley wwt./oocmmmcci/coo 1002/am0eeieeee 201708408
Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne den Berg. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems, volume 34, 2021. URL
neurips cc/ wwcppapp///// paperr2021/hash.
www.eee37777-AAArAbrtta.. html NeurlPS 2021.
David Baker and George Church. Protein design meets biosecurity. Science, (8361))) Jan- uary 2024. doi: 10. 10.112//cieence.ad11000 URL btttps:/wwww.cc... org hts///oi/11..00 1126/
science ado1671
Melis Ilayda Bal, Pier Giuseppe Sessa, Mojmir Mutny, and Andreas Krause. Optimistic games for combinatorial bayesian optimization with application to protein design. In Proceedings of the Thirteenth International Conference Learning Representations (ICLR), Singapore, April 2025. doi: 10,488550/r21w.24409. 188888... Poster.
URL bttps:://penreview..eett/
Brian R. Bartoldson, Siddarth Venkatraman, James Diffenderfer, Moksh Jain, Tal Ben-Nun, Seanie Lee, Minsu Kim, Johan Obando-Ceron, Yoshua Bengio, and Bhavya Kailkhura. Trajectory balance with asynchrony: Decoupling exploration and learning for fast, scalable Ilm post-training, In Advances in Neural Information Processing Systems (NeurIPS), 2025. doi: 10.48550/arxiv.25500 18929, wRw.ttps:
Poster; to appear.
Nathaniel Blalock, Srinath Seshadri, Agrim Babbar, Sarah A. Fahlberg, Ameya Kulkarni, and Philip A. Romero. Functional alignment of protein language models via reinforeement learning. bioRxiv. May 2025. doi: 10.1 10111022222.00 05,00 .0.......2..6 02.00..65 1993. URL httph:h/tss/t www.wii...vi.o.o org/ org/content/in Preprint.
1101/2025. 05. 12. 651993v1
Umberto Borso, Davide Paglieri, Jude Wells, and Tim Rocktäschel. Preference-basee alignment of discrete diffusion models. In ICLR 2025 Workshop on Bidirectional Human-Al Aligument (Bi-Align), Singapore, April 2025, doi: 10. (0.48550/a2x..2250..0828
net het/ppo2Fid==qqSCCsc321 Workshop paper.
David H. Brookes and Jennifer Listgarten. Design by adaptive sampling, February 2020. URL
http://arxiv.ort org/ www.oorggg/bbbbbeeeeoe /acoo.///1.... 11111110010111111 03714 1eXiv:1810.0371 [cs, q-bio, stat].
David H Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International Conference on Machine Learning, 2019. URL https://arxiv
hogg/bbs/1900 10060
Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligian- nidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances in Neural Information Processing Systems (NeurIPS), vol- New Orleans, LA, USA, December 2022. Curran Asso-
ume 35, pp. 18266-28279, ciates, Inc. URL https: httpttpp://prooceeingg..toe nwwrine.....e ctpppppp..e.pppppeer....... filerffiellppppeee wttps/fflllseeee//eee/eee pttpp/pppppppppaaeeee //aah/ Sobpgd/faaaattaaiooaaaaaaaaaeeer
Sbb22330rraasff011100000eee:0::0 wtw.::0ooooo--otatt--t----- wwweaaatttoooofffeeecece html NeurlPS 2022.

---

Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications protein co-design. In Proceedings of the 4 st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pp. 5453-5512. PMLR, 21-27 Jul 2024. URL https://proceeddi
mlr. wrwssrrsssseeee ppess/v2355 v235/ campbel124a html
Egbert Castro, Abhinav Godavarthi, Julian Rubinfien, Kevin B. Givechian, Dhananjay Bhaskar, and Smita Krishnaswamy. Transformer-based protein generation with regularized latent space optimization. Nature Machine Intelligence, 4(10))840-851, 851, October 2022. doi: 10. 10310
$42256-022-00532-1 CtL.https:///ee htww:///wwwwaa...eeeeeeeeeaaaeee nawure....e..... com http:////....aaea/////2/20000000
Tianlai Chen, Pranay Vure, Rishab Pulugurta, and Pranam Chatterjee. Amp- Integrating latent diffusion with protein language models for antimicrobial peptide generation. In NeurIPS 2023 Workshop on Generative Al and Biology (GenBio), New Orleans, LA, USA, December 2023a. URL
faet/foormml id= id=145TM9VQhhx
Workshop poster; non-archival.
Ting Chen, Ruixiang Zhang, and Geoffrey E. Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning, Proceedings of the Eleventh International Conference Learning Representations (ICLR), May 2023b. URL httts:://opeereeeeeeeee wttpppperrreieeeeeeeffrrrrr (d=3itjRRQxFF ICLR 2023.
Tong Chen, Yinuo Zhang, and Pranam Chatterjee. Areuredi: Annealed rectified updates for refin- ing discrete flows with mujti-objective guidance, 2025. URL wttptttst///xii....tt abs/ 2510 00352
Yongcan Chen, Ruyun Hu, Keyi Li, Yating Zhang, Lihao Fu, Jianzhi Zhang, and Tong Si. Deep mutational scanning of an oxygen-independent fluorescent protein creilov for comprehensive profiling of mutational and epistatic effects. ACS Synthetic Biology, 12(5); (365):1461-144 doi: 10.102 (a2/l///ssbboooooo/000 2c00662
May 2023c. URL htttps://pubs..... acs org/ dt////oo11111111 1021/ 10e.:0.11110aaaeea 101////sy/2////
Wenda Chu, Zihui Wu, Yifan Chen, Yang Song, and Yisong Yue. Split gibbs discrete diffusion poste- rior sampling. In Advances in Neural Information Processing Systems (NeurlPS), 2025. doi: 00.488500/xxx..255..00111 Poster; to appear.
URL http::///aerrrssccc//uaaaa/////p
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023. URL httt:tttt////enreeeet
net/ httt/formmmiii
OnD92CAGTOO
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. In Proceedings of the 8th International Conference on Learning Representations net /ttppss//openreview
(ICLR), 2020. https:ttt::///ott
ICLR 2020.
Oscar Davis, Samuel Kessler, Mircea Petrache, Ismail Ilkan Ceylan, Michael Bronstein, and Avishek Joey Bose. Fisher flow matching for generative modeling over dis- crete data. In Advances in Neural Information Processing Systems (NeurIPS), vol- ume 37, 2024. URL htttps:/pproceeddngg/ http://owcedddng....eerr.rpp..ee nww.ineuu....e wtw.rrpp.cc/pppp..fflleeeees ott.aacce2225ll1177777777766622 htt.:9/s22s--stttaaa----ooeeeeee
html
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl, and Jonas Adler. Continuous diffusion for categorical data. arXiv preprint 1 15009, November 2022. doi: 10.488500arxi..2211.5555 Preprint.
httpss
//axxiv 2211 15089
Kerr Ding, Michael Chin, Yunlong Zhao, Wei Huang, Binh Khanh Mai, Huanan Wang, Peng Liu, Yang Yang, and Yunan Luo. Machine learning-guide guided co-optimization of fitness and diversity facilitates combinatorial library design in enzyme engineering. Nature Communications, 15 (1):6392, July 2024. doi: 10. (4008///41447-02-5506060 Articles/s41467-000 50698-9
URL. https:://// h//pp://wwww... http://wwwwttt..tt////t//
nature. com/
12

---

Hongyuan Fei, Yunjia Li, Yijing Liu, Jingjing Wei, Aojie Chen, and Caixia Gao. Advancing protein evolution with inverse folding models integrating structural and evolutionary constraints.
Cell, 1 188(7)):677446622ee19 August 2025. doi: 10. wet.///.ce...222..66...
URI. https
Peter I. Frazier. Bayesian optimization. In Recent Advances in Optimization and Modeling of
Contemporary Problems, INFORMS TutORials in Operations Research, pp. 255-278. INFORMS, Catonsville, MD, 2018. doi: 10.1 2087/ 2.0188888 URL. htt.ttts:./ppbssnnliiiiiiie www.iiieee.iifrrrmemmmmmommmmmme inf onforms
org/doi 10. 1287/educ 2018. 0188
Chase R Freschlin, Sarah A Fahlberg, and Philip A Romero. Machine learning to navigate fitness landscapes for protein engineering. Current Opinion in Biotechnology. 75:102713, June 2022. ISSN 0958-1669 1669. doi: 10. (0.1016/j.oobioo20222 102713. httthhtts:///// htttp::/www.sciinnddirrrr scieecedirectc
con/sciencee/rrticle// http:///aaaaaceee////////6666220
Shrey Goel, Vishrut Thoutam, Edgar Mariano Marroquin, Aaron Gokaslan, Arash Firouzbakht, Sophia Vincoff, Volodymyr Kuleshov, Huong T. Kratochvil, and Pranam Chatterjee. Memdlm: De novo membrane protein design with property-guides discrete diffusion. In ICLR 2025 Workshop on Leurning Meaningful Representations of Life (LMRL), Singapore, April 2025. URL https Workshop poster; non-archival.
Rafael Hoanez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobart Benjamín Sánchez- Sánchez-Lengeling Lengeling, Dennis Sheberla, Jorge mamillera-parrrguirr Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru- Aspuru-Guzik. Automatic chemical design using data-driven continu- ous representation of molecules. ACS Central Science, 4(2): 276, February 2018. doi: 10. 1021/ acscentsci.7600077. URL sattps://pulee acs. wttg/doi/11... 1021 1021/acccentsch 7b00572
Nate Gruver, Samuel Stanton, Nathan C. Frey, Tim G. Rudner, Isidro Hotzel, Julien Lafrance- Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffusion. In Advances in Neural Information Processing Systems, volume 36. 2023. URL 00289798888670004444440068111666 wow.traattttooeeeeeeeeeeeeee html 2023.
NeurIPS
Anvita Gupta and James Zou. Feedback GAN for DNA optimizes protein functions. Nature Machine Intelligence, 1(2):105-111, February 2019, ISSN 2522-5839, doi: 10. 00.1008//4225---19-007744
https:/ts::///e nature. wwtm//articleee//2266-00000 ture Publishing Group.
Number: 2 Publisher: Na-
Alex Hawkins- Jakub Kmec, Oliver Bent, and Paul Duckworth. Likelihood- fine-tuning of protein language models for few-shot fitness prediction and design. In ICML 2024 Workshop Machine Learningfo for Life and Material Science: From Theory to Industry Applications (ML4LMS), Vienna, Austria, July 2024. doi: 10.1101/2024.00 00..4.0.......55555 URL httph:tt///// https:://oennrvviess net/ Korummiid-M&YhOEUUyy Workshop poster; non-archival.
Brian Hie and Kevin K. Yang- Adaptive machine learning for protein engineering. Current Opinion in Structural Biology, 12:145-1 February 2022. ISSN 0959440X, doi: 10. s0.1006/j..bi.2022 002. URL http::ttttt::tttiiee www.iikiinuub..eeevieeeeeeeeeeee elsevier com http://r......eeeeeeee/////////0
Brian Hie, Varun R. Shanker, Duo Xu, Theodora U. Bruun, Payton A. Weidenbacher, Shaogeng Tang, Wesley Wu, John E. Pak, and Peter S. Kim. Efficient evolution of human antibodies from general protein language models. Nature Biotechnology, April 2023. ISSN 1546-1696. doi: UtL/ntts::///att
Jonathan Ho,
Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic mod- In Advances Neural Information Processing Systems (NeurIPS), vol-
els.
neurips. cc/ pppppp/p22//aaaha Accbceee55A4ff0000111111111oo0o. Abstrdctbbbbbttt...omo httml NeurIPS 2020.
ume 33, 2020. URL. https:
Emiel Hoogeboom, Alexey A. Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models. In Proceedings of the Tenth International Conference on Learning Representations (ICLR), April 2022. URL htt.pttts::// htttps://openrviieeee net. fodmm?dd Lm8T39vLDTE ICLR 2022 (poster).
13

---

Chloe Hsu, Hunter Nisonoff, Clara Fannjiang, and Jennifer Listgarten. Learning protein fitness models from evolutionary and assay- assay-labeled labeled data. Nature Biotechnology. 40(7):1144-122. January 2022. nature. ISSN 1546-1 1696. doi: 10. 10.1088/415577001--001 146-5. URL https:: //www
Bandiera Bandieraaabtest: abbest: a Cg_type: Nature Research
Journals heimary_atyped Research Publisher: Nature Publishing Group Subject_ters terem: Machine learning:Protein design Subject_term. tere__teedddd__eeeeeeee id: machinn-lcrminggpooeeineeess
Chloe Hsu, Clara Fannjiang, and Jennifer Listgarten. Generative models for protein structures and sequences, Nature Biotechnology, 42: 12:196-199, 2024.
Long-Kai Huang, Rongyi Zhu, Bing He, and Jianhua Yao. Steering protein language models. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Proceedings of the 42nd International Conference on Machine Learning. volume 267 of Proceedings of Machine Learning Research, pp. 26247-26260. PMLR, http:///rcceeeen..eeeee mlr. ppess/v267 2a0067//uaggge hhw.gguaannnaaaaaa//aa html
13-19 Jul 2025.
Moksh Jain, Emmanuel Bengio. Alex Hlemandez-Garcinn Jarrid Rector-Brooks Bonaventure F. P. Dossou, Chanakya Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, Lena Simine, Payel Das, and Yoshua Bengio. Biological sequence design with GFlowNets. In Pro- ceedings of the 39th International Conference on Machine Learning (ICML), volume 162 of Proceedings of Machine Learning Research, pp. 9786-9801 PMLR, 17-23 Jul 2022. URL https://prroceedings. mlr. prres//v62/jaabei jain22a. html
Kaiyi Jiang, Zhaoqing Yan, Matteo Di Bernardo, Samantha R. Sgrizzi, Lukas Villiger, Alisan Kayabolen, B. J. Kim, Josephine K. Carscadden, Masahiro Hiraizumi, Hiroshi Nishimasu, Jonathan S. Gootenberg, and Omar O. Abudayyeh. Rapid in silico directed evolution by protein language model with EVOLVEpro. Science, 38766732)eadr6006, November 2024. doi: 10. 1126/science.adrdo adr6006.. URL https: //t/: /www//wwwwoo...oo.. science org/ hts//oi/110 /00. 1126/ science.......//// adr6006 Publisher: American Association for the Advancement of Science.
L. Steven Johnson, Sean R. Eddy, and Elon Portugaly. Hidden Markov model speed heuristic and iterative HMM search procedure. BMC Bioinformatics, 11(1):431, August 2010. ISSN 1471-2105, doi:
11-431. URL httpp:://ti..oooo org/ 10. 1186/ 2105 431
Kadina E. Johnston, Patrick J. Almhjell, Ella J. Watkins Dulaney, Grace Liu, Nicholas J. Porter, Jason Yang, and Frances H. Amold. A combinatorially complete epistatic fitness landscape in an enzyme active site. Proceedings of the National Academy of Sciences, 121033)):2440443112,, August 2024. doi: 10. 11073/ppas.2404339911 pnas. 2400439121
URL httpt:://////tt htops::/wwww pnas wam.a..rr/da...... org/
Kirthevasan Kandasamy, Akshay Krishnamurthy, Jeff Schneider, and Barnabás Póczos. Parallelised bayesian optimisation via thompson sampling. In International conference on artificial intelligence and statistics, pp. 133-142. PMLR, 2018
Kazutaka Katoh and Daron M. Standley. MAFFT Multiple Sequence Alignment Software Version Improvements in Performance and Usability. Molecular Biology and Evolution, 3024)7772-780, April 2013. ISSN 0737-4038. doi: 10. m0.099//olle///st000.... URL aattps:///olt wwg/o0.go/oo 10 1093/ molbev/mst010
Hyeonah Kim, Minsu Kim, Taeyoung Yun, Sanghyeok Choi, Emmanuel Bengio, Alex Hernández- García, and Jinkyoo Park. Improved off-policy reinforcement learning in biological sequence design. In Proceedings of the 42nd International Conference on Machine Learning (ICML). Vancouver, Canada, July 2025. doi: 10.485500arXiv.2411 21..00041 https:// cc/ 2ttpaaaa//222//ssee///50000
ICML 2025 (poster), to appear.
Andrew Kirjner, Jason Yim, Raman Samusevich, Shahar Bracha, Tommi Jaakkola, Regina Barzilay, and lla Fiete. Improving protein optimization with smoothed fitness landscapes. In Proceedings of the 12th International Conference on Learning Representations, 2024. URL httplhhttss///att //rxxiv org/ htse/ab///0 22bb/ 00494
Leo Klarner, Tim G. J. Rudner, Garrett M. Morris, Charlotte M. Deane, and Yee Whye Teh. Context- guided diffusion for out-of-distribution molecular and protein design. In Proceedings of the 41st
14

---

International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine PMLR, 21-27 Jul 2024. URL httpss//procceedngg
Learning Research, pp. 2tmml html
mlr ppresss/2366klaareer2222
Théophile Lambert, Amin Tavakoli, Gautham Dharuman. Jason Yang, Vignesh Bhethanabotla, Sukhvinder Kaur, Matthew Hill, Arvind Ramanathan, Anima Anandkumar, and Frances H. Arood. Sequence-based generative Al-guided design of versatile tryptophan synthases. bioRxiv, Au- Btllhtttps:/wwee biorxiv. wwt.:oogggooo..o org/content/10
gust 2025. doi: 10. 0000020255.0.30.6755553 1101/2025 1.8.30.663177v7
Preprint; version posted Aug 30, 2025.
Cheuk Kit Lee, Paul Jeha, Jes Frellsen, Pietro Liò, Michael Samuel Albergo, and Francisco Vargas. Debiasing guidance for discrete diffusion with sequential monte carlo. In ICLR 2025 Workshop on Frontiers in Probabilistic Inference: Learning Meets Sampling, Singapore, April 2025a. doi: httts:s////icl cc/virtual/202555 http////wwwhhhh/2299999 Oral;
(0.48550/a2xiv.2502.00 non-archival.
Seunghun Lee, Jinyoung Park, Jaewon Chu, Minseo Yoon, and Hyunwoo J. Kim. Latent bayesian optimization via autoregressive normalizing flows. In Proceedings of the Thirteenth Inter- national Conference on Learning Representations (ICLR), Singapore, April 2025b. URL htttp:://operreee
ICLR 2025 (Oral).
Francesca-Zhoufan Li, Jason Yang, Kadina E. Johnston, Emre Gürsoy, Yisong Yue, and Frances H. Amoold. Evaluation of machine learning- directed evolution across diverse combinatorial September 2025a. doi: 10. 10.1016/j.cels 202.2001551117777 URL
landscapes. Cell Systems, 16(9):101387, Septeme
httpe:///oo..org/000111133
101387
Tianyi Li, Mingda Chen, Bowei Guo, and Zhiqiang Shen. A survey on diffusion language models.
arXiv preprint arXiv:2508 10875, August 2025b. doi: 10.48500arXx..2508.107.... URL https Preprint.
arxiii www.i.owg//a...............wo.oe abaaa//////0//./ 2508 10875
Xiang Lisa Li, John Thickstun, Ishaan Guirajani, Perey Liang, and Tatsunori B. Hashimoto. Diffusion-
LM improves controllable text generation. In Advances in Neural Information Processing Systems
(NeurIPS), volume 35, 2022. URL htttps://proceeddnngg hewpeeeedddiinnnnnneeseeeeeeeeee nwwrips. cc/ wwtp:per.pppppppcc/ paper_filess paper/ paper/2022/hash/ http////200//aecc00888886e6666a0 wwtpooddaaa-sttrtttt-eeeo--e--ee Conf erence.etar
NeurlPS 2022.
Xiner Li, Yulai Zhao, Chenyu Wang, Gabriele Scalia, Gökcen Eraslan, Surag Nair, Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, and Masatoshi Uehara. Derivative-free guidance in continuous and discrete diffusion models with soft value-based decoding. In NeurlPS 2024 Workshop on Al for New Drug Modalities, New Orleans, LA, USA, December 2024a. URL 2028/10288 Workshop poster; non-archival.
https://neur ips.
Zihao Li, Hui Yuan, Kaixuan Huang, Chengzhuo Ni, Yinyu Ye, Minshuo Chen, and Mengdi Wang. Diffusion model for data-driven black-box optimization. arXiv preprint arXiv:2400 13219, March 2024b. doi: 10.48550//rxiv.2440.. 13219. URL httph:tt//aatt //arxiv////xv....// btgg/ba//2ae 2403. 13219 Preprint.
Hocheol Lim, Geon-Ho Lee, and Kyoung Tai No. Scoring-assisted generative exploration for proteins (sage-prot): A framework for multi-objective protein optimization via iterative sequence generation and evaluation. arXiv preprint arXiv:2505.013: May 2025. doi: 10445500atxiv22555.01777 Preprint.
URL https://arxivv org/ absp///../55..00 01277
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi: Tom Sereu, Salvatore Candido, and Alexander Rives. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637) 130, March 2023. doi: 10. 226/science..dee577. URL https: http://wwwwwww..ee//////// owt. /tt/10.011100111 11t.:/...................11 /tte///ccncce.... ade2574
Sidney Lyayuga Lisanza, Jacob Merle Gershon, Samuel W. K. Tipps, Jeremiah Nelson Sims, Lucas Arnoldt, Samuel J. Hendel, Miriam K. Simma, Ge Liu, Muna Yase, Hongwei Wu, Claire D. Tharp, Xinting Li, Alex Kang, Evans Brackenbrough, Asim Bera, Stacey Gerben, Bruce Wittmann, Andrew C. McShan, and David Baker. Multistate and functional protein design using RoscTTAFold
15

---

sequence space diffusion. Nature Biotechnology, 43(8):1288-1222)) August 2025. doi: 10.1038/
$41587-024-023395 URL https://wwwz nature. http://rrcceeee11111-2222222 Epub 2024-09-25; Publisher Correction: Nat Biotechnol (4001/33444..000000003444155742 02456-0.
Sulin Liu, Juno Nam, Andrew Campbell, Hannes Stärk, Yilun Xu, Tommi Jaakkola, and Rafael Think while you generate: Discrete diffusion with planned denoising. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR), Singapore, April 2025. URL.
Poster.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pp. 32819-32848, Vienna, Austria, 21-27 Jul 2024. PMLR. URL htttps::/proceeddnnggg mlr press/v235/ 1ou22a.htmn
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. The Twelfth International Conference on Learning Representations, 2024.
Natalie Maus, Haydn Jones, Juston Moore, Matt J Kusner, John Bradshaw, and Jacob Gard- ner. Local latent space bayesian optimization over structured inputs. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems, volume 35, pp. 34505-34518. Curran Associates. Inc., 2022. URL del998228622442333311110000.
newerdddiinnnnnn............ neurips. cc/ wacc/pppr__se sirfffiees_ Fflles/paper/202/// ppppp///02//filleeeee Papes Confer--oofeeeeeee pdf
Viacheslav Meshchaninov, Pavel Strashnov, Andrey Shevtsov, Fedor Nikolaev, Nikita Ivanisenko, Olga Kardymon, and Dmitry Vetrov. Diffusion on language model encodings for protein sequence generation. In Pruceedings of the 42nd International Conference on Machine Learning (ICML), Vancouver, Canada, July 2025. doi: (05550arXii.2443.033320 Poster; to appear.
URL https://icmm.. cc/
virtual/2025posstea /43588
Richard Michael, Jacob Hansen, Peter Mørch Groth, Simon Bartels, Jesper Salomon, Pengfei Tian, Nikos S. Hatzakis, and Wouter Boomsma. A systematic analysis of regression models for protein engineering. PLOS Computational Biology, 20(5) May 2024. ISSN 1553-7358. doi: 10. 10.137//jommaa.pccbb 1ttp:/mmma..pci..11106.61.1 URL https: //tttss:/dx...r http://xxpppl......oo/../..ooost org 10.rg/10... 1371/ pcbi
1012061
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8162-8171. PMLR, 18-24 Jul 2021. URL
wtt.://a...prp...emmmpmmmmmm.mee ppress/v399 //99//niie htt//nichof www.hhollll....ooc.oooo..o html
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ri-Rong Wen, and Chongxuan Li. Large language diffusion models. Advances in Neural Information Processing Systems (NeurIPS), San Diego, CA, USA, December 2025. URL https tneeurips..c//irtual///00 http///ttteeeeeee///e/////
Erik Nijkamp, Jeffrry A. Ruffolo, Eli N. Weinstein, Nikhil Naik, and Ali Madani. ProGen2: Exploring the boundaries of protein language models. Cell Systems, http///iiiiinuubeeeeeeeeeeeeeeee elsevier. com/
November 2023.
ISSN 24054712. doi: c0.1006/j.cels..2023 reerriere/pi//SAA4441122222
URL htttp:://inkinggiiii
Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, and Jennifer Listgarten. Unlocking guid- ance for discrete state-space diffusion and flow models. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR), Singapore, April 2025. URL https://opennrevie: http://ww..eeetooommoeeexxxaaaaa
ICLR 2025 Poster.
C. Anders Olson, Nicholas C. Wu, and Ren Sun. A Comprehensive Biophysical Description of Pairwise Epistasis throughout an Entire Protein Domain. Current Biology, 24(22)2643-26511 November 2014. ISSN 09609822, doi: 10. 10. https //linkinghub
elsevier com /tter//r..oooeeeeeeeeee////////2
16

---

Michael S. Packer and David R. Liu. Methods for the directed evolution of proteins. Nature Reviews Genetics, 16(7):379- 3944, July 2015. ISSN 1471-0056, 1471-0064. doi: 10.1038/nrg3927. URL
http::/ //t:://wwwwww www. nature. com ccmm//atiiclee///r33977
Fred Zhangzhi Peng, Zachary Bezemek, Sawan Patel, Jarrid Rector- Sherwood Yao, Alexan- der Tong, and Pranam Chatterjee. Path planning for masked diffusion model sampling. In ICLR 2025 DeLTa Workshop, Singapore, April 2025, doi: 200488500arXxi.2522..000 2502.035000. URL Workshop poster, non-archival.
https://opeereviae wtwprrrewwwe..ettouunniiiiccc.. wwt.://orrr/....PPPPP.....
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Advances in Newral Information Processing Systems (NeurIPS), volume 36, 2023. URL https://papers: wwtp:pppprss.sp.....pomppp ctcapps.cc/apppe wttpcccppppppeeeeeeeeeeeee fiwefflllaaaaee wwtssspaper/aaee http:/pappp/222///0// 2aapr/222//haaas 28f606666666777668888886666
html
NeurlPS 2023.
Jarrid Rector-Brooks, Mohsin Hasan, Zhangzhi Peng, Zachary Quinn, Chenghao Liu, Sarthak Mittal, Nouha Dziri, Michael Bronstein, Yoshua Bengio, Pranam Chatterjee, Alexander Tong, and Avishek Joey Bose. Steering masked discrete diffusion models via discrete denoising posterior prediction. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR), Singapore, April 2025, doi: 10.48550krX1..2410000113 Poster.
bet/forum?id-Ombnnne
Zhizhou Ren, Jiahan Li, Fan Ding, Yuan Zhou, Jianzhu Ma, and Jian Peng. Proximal exploration for model-guided protein sequence design. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 18520-18536. PMLR, 17-23 Jul 2022. URL Atttps:/pproeedeee
mlr
html
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scaling unsupervised learning 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15), April 2021. ISSN 0027-8424, 1091-6490, doi: 10. 00073/paa.20162253 18. URL https: /wwww pnas. org/ content/ /1111////////22162111198 Publisher: National Academy of Sciences Section: Biological Sciences.
Philip A Romero and Frances Amold. Exploring protein fitness landscapes by directed evolution. Nat Rev Mol Cell Biol, 10:866-876 1766 2009, doi: 10. 10.1008/nmm2805.
Jeffrey A Ruffolo and Ali Madani. Designing proteins with language models. Nature Biotechnology, 42:200-202, February 2024.
Daniel Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen, et al. A tutorial on thompson sampling. Foundations and Trends@ in Machine Learning, 11(1):1-96, 2018.
Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. In Advances in Neural Information Processing Systems (NeurIPS), vol- ume 37 of Advances in Neural Information Processing Systems, Vancouver, Canada, De- cember 2024. URL L http::htttt///tt ptt/ttpp///prreddnng @a0011accs722aa0looogfffaaoooeee
httpa/ddwwwnn..areeeoe../p/eeeea files/ files/paper/2224// /at/p//p//////ahh html
Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Sam Boshar, Hugo Dalla-torre, Bernard P. de Almeida, Alexander Rush, Thomas Pierrot, and Volodymyr Kuleshov. Simple Guidance Mechanisms for Discrete Diffusion Models, December 2024. URL BRRL:ttt:///ee http://arxiv org/abs/2412.. 10193, arXiv:2412.10193 [cs].
Kosuke Seki, Amy B. Guo, Deniz Akpinaroglu, and Tanja Kortemme. A combinatorial mutational map of active non-native protein kinases by deep learning guided sequence design. bioRxiv, pp. 2025.082 05.550880..6683333 August 2025. doi: 10. I 1001/2025..00 08.0036663355 URL httpp::///// htpps://wwr //www biorxiv. wtt.:////oooeee/..01000... 2025.
08. 668353.
Preprint; version posted Aug
17

---

Jiaxin Shi, Kehang Han. Zhe Wang, Amaud Doucet, and Michalis K. Titsias. Simplified and generalized masked diffusion for discrete data. 2024. URL httpps://arxiv.oog////aaa orgg/bb////bb5 04329
Sam Sinai, Richard Wang, Alexander Whatley, Stewart Slocum, Elina Locane, and Eric D. Kelsic. Adalead: A simple and robust adaptive greedy search algorithm for sequence design, 2020. URL
pettps://arxiv.org///oo
.01111
Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time scaling and steering of diffusion models. In Proceedings of the 42nd International Conference on Machine Learning (ICML), Vancouver, Canada, July 2025. doi: (0.48550/arXiv.250..066
URL https://icml.cc
virtual/2025/possss 45673
ICML 2025 (poster).
Diogo Soares, Leon Hetzel, Paulina Szymczak, Fabian Theis, Stephan Günnemann, and Ewa Szezurek. Targeted AMP generation through controlled diffusion with efficient emheddings, April 2025.
URL wwt.:httts///arteee /ttp/v/rrvvvvv/m www.ariwww.........o.o... abt/ 17247 Preprint.
Yang Song, Jascha
Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proceedings of the International Conference on Learning Representations (ICLR), May 2021. URL https 7/openreview wet./oormmidddxiiiiittt.
ICLR 2021 Outstanding Paper Award.
Zhenqiao Song and Lei Li. Importance weighted inrectationmmxximizen for protein sequence de- sign. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.). Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 123499-2334, Honolulu, Hawaii, USA, Jul 2023. PMLR. URL https:://proceeddiii
mlr. .aw...pesss//.. wwwps//vr20///// http//2222/2////// song23g. html
ICML 2023.
Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders. In Proceedings of the 39th International Conference on Machine Learning. volume 162 of Proceedings of Machine Learning Research, pp. html 17-23 Jul 2022. URL httptttttt//sceeeeeeess
PMLR,
mlr malrpreee Srresss/v12//stanton
Hannes Stark, Bowen Jing, Chenyu Wang, Gabriele Corso, Bonnie Berger, Regina Barzilay, and Tommi Jaakkola. Dirichlet flow matching with applications to DNA sequence design. In Pro- ceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pp. 16549546513, Vienna, Austria, 21-27 Jul 2024. PMLR. URL htttps://proceeediggggg
mlr www.eepwss...//eeeeee stark24b. html
Martin Steinegger and Johannes Söding. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature Biotechnology,
November 2017. ISSN 3988 1546-1696. doi: 10. 1038//bt..988..... URL httph::/t///// //t://wwww /wwww nawura.ttee..a com www.//rticleeeeeeeteeeoeeoeeeeee http:///wtueeeeeo//aaaaaeeee///e Number: 11 Publisher: Nature Publishing Group.
Filippo Stocco, Maria Artigues-Lleixà Andrea Hunklinger, Talal Widatalla, Mare Güell, and Noclia Ferruz. Guiding generative protein language models with reinforcement learning. In NewrIPS 2024 Workshop on Machine Learning in Structural Biology (MLSB), Vancouver, Canada, De- cember 2024. doi: 10.488550/arxi..2412.222 12999. https /ellttpss:///lldessiiieee com/39031185/ Sriding-proteiin
wwwdangg-ooooeiiaaaaaaaaeeeeeaaa language- lation (non-archival).
htwp::/pwe....aaaaeeeeeeeaaaeeer 1earning Workshop presen-
Kiera H. Sumida, Reyes Aúñez-Fraaco, Indrek Kalvet, Samuel J. Pellock, Basile I. M. Wicky, Lukas F. Milles, Justas Dauparas, Jue Wang, Yakov Kipnis, Noel Jameson, Alex Kang. Joshmyn De La Cruz, Banumathi Sankaran, Asim K. Bera, Gonzalo Jiménez- Osés, and David Baker. Improving Protein Expression, Stability, and Function with ProteinMPNN. Journal of the American Chemical Society, 146(3):0044-061, January 2024. ISSN 0002-7863, 1520-5126. doi: 10. 1el1002/acs. 3c0041. URL org//oi/o
https://pubs. acs. org /t////oi1111//0 1021 3023/jacc..c10941 3c10941
18

---

Haoran Sun, Liang He. Pan Deng, Guoqing Liu, Zhiyu Zhao, Yuliang Jiang. Chuan Cao. Fusong Ju, Lijun Wu, Haiguang Liu, Tao Qin, and Tie- Yan Liu. Accelerating protein engineering with fitness landscape modelling and reinforcement learning. Nature Machine Intelligence, September 2025. ISSN 2522-5839. doi: 00.1038864225--025-010003 UtR..tttss://don orw.oi..rr01.1100.
142256-025-01000
Sophia Tang, Yinuo Zhang, and Pranam Chatterjee. Peptune: De novo generation of therapeutic peptides with multi-objective-guiinn discrete diffusion. In Proceedings of the 42nd International
Conference on Machine Learning (ICML 2025), Vancouver. Canada, July 2025a. doi: 10.48550/ arXiv.2412.17780. URL https: //mml..mmmmm///mm/////////// wct/iirtuall/////ppsttree//////e 45889 Poster; ICML 2025.
Sophia Tang, Yinuo Zhang, Alexander Tong, and Pranam Chatterjee. Gumbel-softmax flow matching with straight-thouugh guidance for controllable biological sequence generation. In ICLR 2025 Workshop on Interaainn Generative and Experimental Platforms for Biomolecular Design (GEM), 2025b. URL htops:///ariivv http///arx.....aab.../.// 2500. 17361 available as arXiv:2503.17361.
Workshop poster (non-archival); also
Neil Thomas, David Belanger. Chenling Xu, Hanson Lee, Kathleen Hirano, Kosuke Iwai, Vanja Polic, Kendra D. Nyberg, Kevin G. Hoff, Lucas Frenz, Charlie A. Emrich, Jun W. Kim, Mariya Chavarha, Abi Ramanan, Jeremy J. Agresti, and Lucy J. Colwell. Engineering highly active nuclease enzymes with machine learning and high-throughput screening. Cell Systems, 16(3): March 2025. URL https://::/ww wcweeceddeennccrttt....mm com/
ISSN 2405-4712, doi: wtt.:////...l..22..... science/ http://ataaeeeeeeee////1//111111
Marcelo D. T. Torres, Yimeng Zeng, Fangping Wan, Natalic Maus, Jacob Gardner, and Cesar de la Fuente-Nunez. A generative artificial intelligence approach for antibiotic optimization. bioRxiv, November 2024. doi: 10.1101/2024.1 1. 1.27.66 27.625757. http::s/://wwwt wwwp//twwwbbb....o//////oo/ org/ content/10 1101.1121/2224 11. 27. 625757v1 preprint.
Marcelo D. T. Torres, Tianlai Chen, Fangping Wan, Pranam Chatterjee, and Cesar de la Fuente. Generative latent diffusion language modeling yields anti-infective synthetie peptides. Cell Biomaterials, I: 10100183, October 2025. doi: 10. 2025, cell..coo//eel1000mmtttrrrraa/// httpr//ooooiaeriaaaaaaaaaatee//a bioRxiv preprint 10. Tw..1100/2222..000 133600630000033363003 online early Sept 2, 2025.
URL/https:// //www (25)) (25)00174- Published version of
Masatoshi Uchara, Xingyu Su, Yulai Zhao, Xiner Li, Aviv Regev, Shuiwang Ji, Sergey Levine, and Tommaso Biancalani. Reward-guided iterative refinement in diffusion models at test-time with applications to protein and dna design. In Proceedings of the 42nd International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, Vancouver, Canada, July 2025. PMLR. URL https::/t//:mmt http:///mwwc/c/caaaa//aa////a/// /ttp//20020/pptee/440/000
ICML 2025 (poster).
Siddarth Venkatraman, Moksh Jain, Luca Scimeca, Minsu Kim, Marcin Sendera, Mohsin Hasan, Luke Rowe, Sarthak Mittal, Pablo Lemos, Emmanuel Bengio, Alexandre Adam, Jarrid Broooks, Yoshua Bengio, Glen Berseth, and Nikolay Malkin. Amortizing intractable inference in diffusion models for vision, language, and control. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Vancouver, Canada, 2024. URL httts::/neuurpp.cc//ueee NeurIPS 2024.
poster/953ad 95348
Siddarth Venkatraman, Mohsin Hasan, Minsu Kim, Luca Scimeca, Marcin Sendera, Yoshua Ben- gio, Glen Berseth, and Nikolay Malkin. Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models. In Proceedings of the 42nd International Con- ference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 1-28. Vancouver, BC, Canada, July 2025. PMLR. doi: 11488550/axxi.2502.66666 .06999. URL ICML 2025 (poster). Also available as
https://opperreveer mett/oorumiid=9449hue arXiv:2502.06999,
Tobias Vornholt, Mojmír Mutný, Gregor W. Schmidt, Christian Schellhaas, Ryo Tachibana, Sven Panke, Thomas R. Ward, Andreas Krause, and Markus Jeschek. Enhanced Aequence-Activity Mapping and Evolution of Artificial Metalloenzymes by Active Learning. ACS Cenural Science, 10 (7):1357-1370, May 2024. ISSN 2374-7943, 2374-7951. URL htttsh/ts::// httpss:///bbuu.up/ acs. orrg/oig 10. tttp///cceesscciicc.c 4c00258
19

---

Chenyu Wang, Masatoshi Uchara, Yichun He, Amy Wang, Tommaso Biancalani, Avantika Lal, Tommi Jaakkola, Sergey Levine, Hanchen Wang, and Aviv Regev. Fine-tuning discrete diffusion models via reward optimization with applications to dna and protein design. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025), Singapore, April 2025a. httLhhtps:///ooeeee hwtp:rswiwwwwweeeeeeemmm weet//oruuniid-3280111
ICLR 2025 Poster.
Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Remasking discrete diffusion models with inference-time scaling. Advances in Neural Information Processing Systems 38 (NeurIPS 2025). December 2025b. URL httph:tp///// //nurrip. ce/virtual/2022 poster/118818 Poster.
Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Diffusion language models are versatile protein learners. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 52309- 52333, Vienna, Austria, Jul 2024. PMLR. URL (eil:httas:.. pettps://proceedingg mlr press/v2355. wang24ct.html
Ziwen Wang, Jiajun Fan, Ruihan Guo, Thao Nguyen, Heng Ji, and Ge Liu. Proteinzero: Self- improving protein generation via online reinforcement learning, jun 2025c. URLhttpss//arxiv
org vrg/abs//555 2506. 07459 arXiv preprint.
Talal Widatalla, Rafael Rafailov, and Brian Hie. Aligning protein generative models with experimental fitness via Direct Preference Optimization. bioRxiv, 2024. doi: 10.1101/2024.05.20.0 20.50009... URL
https://doi org/ 10. 1101/2024 05. 20. 595026
Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P. Xing. Deep kernel learning. In Proceedings of the 19th International Conference on Artificial Intelligence and Statisties, volume 51 of Proceedings of Machine Learning Research, pp. 370-378, Cádiz, Spain, 2016. PMLR. URL httpt:://procceediiiiiie
mlr A.r.press/v51/
Bruce Wittmann, Tessa Alexanian, Craig Bartling, Jacob Beal, Adam Clore, James Diggans, Kevin Flyangolts, Bryan T. Gemler, Tom Mitchell, Steven T. Murphy, Nicole E. Wheeler, and Eric Horvitz. Toward Al-Resilient Screening of Nucleic Acid Synthesis Orders: Process, Results, and Recommendations. bioRxiv, December 2024. doi: 10.1101/2024.12. httpp://wwir bis.i:////w....xivv....o orgg woc.ttent/10.1100////0...
URL
12. 12.02.22..222222222222 626439
Preprint.
Bruce J. Wittmann, Kadina E. Johnston, Zachary Wu, and Frances H. Amoll. Advances in machine 2021a. ISSN 1879033X. doi: 10.1016/j.sbi.2000 httisbbi.021.000 .0..00. URL hRL.https: httts:://do:i:: org/ 10. 1016/ sbi 2021 01.008 Publisher: Elsevier Ltd.
learning for directed evolution. Current Opinion in Structural Biology,
Bruce J. Wittmann, Yisong Yue, and Frances H. Arool. Informed training set design enables efficient machine learning-assistee directed protein evolution. Cell Systems, 1211):1006--0045 b0 ISSN 24054712. doi: 10. 20111/1..cis..202222 cels.2021.07.000. URL 2021. 0221007.008 .088
Publisher: Elsevier Inc.
Dongxia Wu, Nikki Lijing Kuang, Ruijia Niu, Yi-An Ma, and Rose Yu. Diffusion-BBO, Diffusion- Based Inverse Modeling for Online Black-Box Optimization, 2025. UtLphtts:://axxi org/ abs/2407 (tte/2477000610) arXiv preprint; also presented as a poster at NeurIPS 2024 Workshop on Bayesian Decision-makinn makiin and Uncertainty.
Luhuan Wu, Brian L. Trippe, Christian A. Naesseth, David M. Blei, and John P. Cunningham. Practical and asymptotically exact conditional sampling in diffusion models, 2024. URL https
//arxiv. org/ aba 2306 17775
Zachary Wu, S. B. Jennifer Kan, Russell D. Lewis, Bruce J. Wittmann, and Frances H. Amold.
Machine Baaring-asssisee directed protein evolution with combinatorial libraries. Proceedings of the National Academy of Sciences, (0(18)8855-88855 April 2019. ISSN 0027-8424, 1091 -6490 doi: 10. 1073//nas..... 1tt.1///aa...11199991.... 1901979116. URL http: www pnas org/Lookku//oo//1..oo.o 1073/ pnas
1901979116
20

---

Zachary Wu, Kadina E. Johnston, Frances H. Arnold, and Kevin K. Yang. Protein sequence design with deep generative models. Current Opinion Chemical Biology. 65:18-21 2021. ISSN 13675931. doi: 10. 101///00..116.... 2021 04. 004 arviv: 2104.04457 Publisher: Hlse-
URL http: http://arxtt //wpvvvivvvv///////////a///iaava org/ ab04 http//14144444
OAhttp://áx.doo vier Ltd.
Junhao Xiong, Hunter Nisonoff, Ishan Gaur, Maria E. Lukarska, Luke M. Oltrogge, David F. Savage, and Jennifer Listgarten. Guide your favorite protein sequence generative model, May 2025. URL https://arxiv. wtwpa//.rrggbb///5/000 04823 arXiv:2505.0455 00.225555.0423...
Jason Yang, Francesca-2hhoufan Li, and Frances H. Arnold. Opportunities and Challenges for Machine Learning-Assister Enzyme Engineering, ACS Central Science, 10(2):22-241, February URL https::/pubs..cc
2024. ISSN 2374-7944,,3344-79551 doi: 10.1001/acsceetsei.3oool org/doi/10 http:///...../aaccceaaaciii.. 3c0127
Jason Yang, Aadyot Bhatnagar, Jeffrey A. Ruffolo, and Ali Madani. cunction-guided conditional generation using protein language models with adapters. arXiv, June 2025a. doi: 10.48550/arXiv 2410.03634. URL. URI. http::ttt////t htapps:///arvvv.../ http:///aa.o.v.ooa/a//////ao /410 03634 arXiv:2410.03334 [q-bio.BM), v2,
last revised 2025-06-11.
Jason Yang. Ravi G. Lal, James C. Bowden, Raul Astudillo, Mikhail A. Hameedi, Sukhvinder Kaur, Matthew Hill, Yisong Yue, and Frances H. Arnold. Active leaminng-ssisstd assisted directed evolu- tion. Nature Communications, 16(1):714, January 2025b. ISSN 2041-1723. doi: 10. 1038/ 141467-025-55987 URL htttps:/wwat nature. com http:/articlsseeeeeeee wwt.rticceee/11444772----5-555-3 Publisher: Nature Publishing Group.
Jason Yang, Srancesca-Zhoufeee Li, Yueming Long, and Frances H. Amold. Illuminating the universe of enzyme catalysis in the era of artificial intelligence. Cell Systems, pp. 101372, August 2025c. ISSN 2405-4712. doi: 10. 10.1016/j.ceh, 101372. URL /tt fttts:://www.scieneeiin
com/
science/article. /0tp1///220277222002000000
Kevin Yang and Dan Klein. FUDGE: Controlled Text Generation With Future Discriminators.
In Proceedings of the 2021 Conference of the North American Chapter of the Association for
Computational Linguistics: Human Language Technologies, pp. 3511-3335, 2021. doi: 10.18653) 276. URL http::/ttt:rstt
org/ awtpa/////./// 05218 arXiv:2104.0520 [cs].
Kevin K. Yang, Zachary Wu, and Frances H. Arnold. Machine- guiiddd directed evolution for protein engineering, Nature Methods, 16(8):687-664, 2019. ISSN 15487105. doi: 10.
arXiv:
1811.10775 Publisher: Springer US.
Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2025. CVPR
doi: 10. 11111///VVV2274422255222.. 2025.0194.. URLLittpss//arxxx..or///// wwwp:////wwbbbbbboooo//o// 2025; ariiv2407.00155
Junming Zhao, Chao Zhang, and Yunan Luo. Contrastive Fitness Learning: Reprogramming Protein Language Models for Low-N Learning of Protein Fitness Landscape. preprint, Bioinformatics, February 2024a. URL http:://biooxii org/ wtgg/ookku//dii/100 1101/2024. 02. 11. 579859
Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Baker Grosse. Probabilistic Infer- ence in Language Models via Twisted Sequential Monte Carlo. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 60704-60748. PMLR, July 2024b. URL https://prooceedingg mlr wttpa//......p.erseeeeeee
Hongkai Zheng, Wenda Chu, Bingliang Zhang, Zihui Wu, Austin Wang, Berthy Feng, Caifeng Zou, Yu Sun, Nikola Borislavov Kovachki, Zachary E. Ross, Katherine L. Bouman, and Yisong Yue. Inversebench: Benchmarking plug-and-play diffusion priors for inverse problems in physical sciences. In Proceedings of the Thirteenth International Conference on Learning Representutions (ICLR 2025), Singapore, may 2025. OpenReview. URI net.ewwwwwww.weeeeeeeeeeeeeeeeee htt.riwwwweee/ooommmii formmidd ICLR 2025 Spotlight.
U3PBITXNG6
21

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The abstract summarizes the claims that we explain further in the Results and the Discussion.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
22

---

Justification: A detailed discussion of the limitations is provided in the Discussion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
All proofs have cited their sources.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The methods needed to reproduce the experimental results are provided the appendix. The code will also be released to the public.
Guidelines:
23

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The codebase will be linked in the text upon de-anonymization.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
wtteeeeiubmiisiooooooooooo. for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https c/iips.cc/public//uue
Codeeuumiiii wwabbbmmsssiinii ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
24

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
These details are provided in the appendix methods section.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Standard deviations between multiple repetitions of the same experiment are provided in relevant figures.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-simma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources used to perform the experiments provided.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
25

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps:/neuurps.. http://rwpppppp......c/aaaacsaaa
Answer: [Yes]
Justification: Yes, the research conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have cautioned the potential for negative societal impacts in the discussion.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: Our models only apply to designing proteins with minimal risk of negative impacts.
Guidelines:
26

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The assets used comply with their corresponding licenses.
Guidelines:
existing assets. The answer NA means that the paper does
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- ce-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The license will be posted with the code when released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
27

---

Justification:
No human subjects were used.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects were used.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: LLMs were not a part of the core methods of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy htte::///aarrppecece///oeerreeee for what should or should not be described.
28

---

A Appendix
A.1 Data for pretraining generative priors
The first step in our pipeline involves learning generative prior on naturally occurring protein sequences to capture the distribution of those with high evolutionary likelihood. This prior is unconditional in the sense that no labeled fitness data is used for training. However, because we are optimizing protein variants for a desired fitness, we pretrained our generative prior on sequences homologous to the parent protein be optimized (known as a multiple sequence alignment or MSA): TrpB. CreiLOV, or GBI. I. Likelihoods from MSAs have been captured by statistical models and have been shown to offer good zero-shot approximations of fitness. In other words, they capture mutational substitutions that are more favorable, based on the precedent of natural evolution.
We focused the TrpB (Johnston et al. 2024) and CreiLOV (Chen et al. 2023c) datasets due the extensive number of sequences in their MSAs and compared to GBI (Olson et al.. (014)), which has comparatively fewer sequences. MSAs were obtained by running jackhmmer (Johnson al. 2010) against Unire190 for two iterations with the parent sequence of the fitness dataset as target. For the MSA, we only used sequences where the aligned portion was least 75% the length of the parent sequence. We used the MSA that was aligned to the parent sequence, with gap tokens replaced by the corresponding amino acid found in the parent sequence, resulting full, fixed-length pseudo-nature sequences. For GBI, I, we augmented the training set with synthetic data, namely all proteins with single mutation to sequences in the MSA. For the language models on TrpB and CreiLOV, some sequences were randomly mutated by single position near the beginning of the sequence, to prevent mode collapse during autoregressive generation.
We performed sequence clustering using mmseqs2 (Steinegger & Söding 2017) at 80% identity and resampled the dataset by weighting each sample with I+In (nyj relative probability of being sampled, where n is the size of the cluster associated with that sequence. Afterward, we removed 5% of the clusters and their associated sequences as validation set.
A.2 Protein fitness optimization task
An oracle as proxy for protein fitness. We studied fitness optimization across three different protein-fitness datasets, TrpB, CreiLOV, and GB1 (Table 2 TrpB is 389 residues in length, but based on available fitness data, we limited design to 15 residues: 117, 118, 119, 162, 166, 182, 183, 184, 185, 186, 227, 228, 230, 231, and 301. Namely, we combined the fitness data from 6 combinatorially complete 3-site libraries (D-I from Johnston et al. (2024)) and the 4-site library across residues 1 183, 184, 227, and 228. We normalized the parent fitness to in each dataset and rounded all negative fitness values up to zero. The fitness here is the catalytic rate of a native reaction, the formation of tryptophan from indole and serine. To obtain a proxy fitness for all variants in the design space (2015 possibilities) we trained
oracle inspired by the dataset splitting and model architecture used in Blalock et al. (2025). Namely, we used all of the single, double, and triple mutants in the library for training, with 10% and 20% of the quadruple mutants being used for validation and testing, respectively. Our model consists of an ensemble of 20 MLPs for TrpB, and each was trained on one-hot encodings of the designed residues for 1000 epochs.
Differently, the CreiLOV dataset (length N 119) contains experimental fitnesses for all single mutations in the protein and certain higher order mutations at 15 selected positions with beneficial single mutations. Fitness here refers to associated fluorescence. To obtain a proxy fitness for all variants in the design space (20119 possibilities), we trained an oracle similar to the procedure above, using similar splits to those in Blalock et al. (2025) and were able to reproduce their high performance on the test set. Before model training, we scaled the fitnesses of the single mutants to the fitnesses of multi-mutants by adding a normalization factor to all single mutants such that the parent sequence in both datasets had the same fitness. Our model consists of an ensemble of 10 MLPs for CreiLOV, and each was trained on onehot encodings of sequences for 1000 epochs.
For GBI, the experimental finesses for nearly all double mutations across the entire protein were available, where fitness refers to binding affinity of domain of the protein. To train the oracle, we held out 10% and 20% the sequences with two mutations as a validation and test set, respectively, with remaining sequences being used for training. Our model consists of an ensemble of 10 MI.Ps for GBI, and each was trained on ooe-hot encodings of the designed residues for 50 epochs.
29

---

Our oracles show high Pearson correlation on the train and test sets (Fig. AT). As the generalization ability of our oracle was only been tested on variants that are similar the parent, we penalized the fitness of protein sequences by factor of 0.99 for every mutation accumulated beyond threshold of 60% sequence identity to the parent sequence. From here forth, we treated ground truth fitness as outputs from the oracle.
2.51
spiit trein test
Then fthees
1.3
Thue fiiress
3.5-
0
1.0
0.89
F-
0.92
Thue
r= 0.89
0.5
0.80
split
t- 0.75 0.93
train
0.0
A TrpB
2.5
CreiLOv
0.93
GB1
test
velidation
-0.5
3.0
4.0
5
10
Predicted
Predicted
Predicted fitnass
Figure A1: Oracles trained on available labeled fitness data for TrpB, CreiLOV, and GB1 extrapolate well to higher order combinations of mutations within the design space, as measured by Pearson correlation.
Processing generated sequences. Our primary method for evaluation involved examining the distribution of sampled sequences and their corresponding fitness values, diversities, and novelties. The processing pipeline for generated sequences in shown in Fig. A2 In diffusion models, sequences were generated with fixed length equal to the parent length. For the language models, nearly all generated sequences had length equal to the parent sequence length. Still, sequences were aligned with the parent sequence using mafft (Katoh & Standley 2013), and gaps were replaced with the corresponding amino acid in the parent sequence to generate complete pseudo-sequences of fixed length. Special tokens, which occurred rarely in generation, were replaced by a random amino acid. For TrpB, residues outside of the design space of 15 residues were naively mapped to the original amino acid type in the parent sequence at the end generation. We did not test inpainting, although this could be accomplished with masked (diffusion) language models.
align with parent H-YFG...SQ-FD
i.e. residues
Language Model
extract residues in the design space YRAMSHRF
MYFGD
fill in gaps with parent MKYRG... TORED
final sequences with fitness values
oracle
Diffusion Model
extract residues in the design space YFAMGHRE
MKYFG...SORDD
Figure A2: Example pipeline for generating protein sequences for evaluation, based on a hypothetical parent sequence: MKKKG...SQRFD (length=100), with residues being optimized (3, 4, 26, 27, 28, 29, 98, 99), corresponding to design space combo of KFDEACRE.
Comparison existing protein engineering methods. There are several reasons why we did not directly compare the performance of SGPO methods to existing methods used in protein engineering, such as directed evolution and MLDE. the case of directed evolution (such as random mutagenesis): (1) It is not obvious which parent sequences to use as the starting points for directed evolution for fair comparison.
(2) unclear the oracle captures the true nature the protein fitness landscape or extrapolates well to sequences with many mutations relative to the original fitness dataset from which the oracle was trained.
(3) Overall, our method enables the accumulation of many mutations in single round of experimentation, whereas directed evolution is largely limited to one mutation at a time. For example, on the CreiLOV dataset, the generated sequences with the highest fitness had on average 66 mutations from the parent reference sequence from which the original dataset was
30

---

generated, which would not be achievable with directed evolution. We also did not directly compare our method to supervised approaches in smaller design spaces, such as 4-site combinatorial libraries Yangg et al. 2025b). as we focus here on design in larger design spaces, where existing methods are lacking. Overall, traversing large swaths of sequence space will he important for faster engineering and enabling improvements to fitness that would normally be slow with directed evolution.
A.3 Generative models for sequences
Table Al: Summary of training details for generative priors in this work. Reference refers the codebase that was modified for our implementation and where the model architecture was adapted from. For all models, retained the model with the lowest validation loss. When using the ESM encoder, we used the 35M-parameter ESM2 model Lin et al. (023).
Model
Max
Learning
Batch
Warmup
Noise Schedule
Diffusion Timesteps
Model Architecture
Reference
Epochs Rate
Size
Steps
Continuous 5
1x10-4
64
10
cosine
500
BERT
Gruver et al.
2023)
Continuous- 25
1x10-4
64 10
cosine
500
BERT
Gruver al.
ESM
20)3)
D3PM-
5
1 1x10-4
64
10
Sohl-
500
ByteNet
Alamdari
Baseline
Dickstein
ct al (024)
D3PM
5
1 1x10-4
64
10
Sohl-
500
ByteNet
A Alamdari
Dickstein
et al
2024)
UDLM
5
3x 10-5
64
2500
loglinear
500
DiT
Schil
2024
MDLM
50
3x10-1
64
2500
loglinear
500
DIT
Schiff et al.
2024h
ARLM
10
1x10-4
32
10
n/a
n/a
GPT-J
Nijkamp
ct (0023)
A.3.1 Diffusion over continuous space
Diffusion models construct samples by reversing diffusion process that maps clean data points to samples from a prior distribution rxx). The forward process (Xg -> xT) is composed of conditional distributions p( (x)(xx-1). which admit closed-form expressions for the conditional distributions p(xx (x)) and ( Xo). The reverse process (xT + Xo) converts samples from the prior into samples from the learned data distribution (xp) by repeatedly predicting the denoised variable Ng from noisy values X,, using the conditional distribution p(x:-1|X,, Xy) to derive a transition distribution po (x).
Continuous noise forward process. Similarly to Gruver et al. (2023), we define a protein sequence A4, where A is the alphabet of amino acids and L is the fixed length of the sequence. To learn a distribution p(w), we first embed w into a continuous variable X9 using an embedding matrix Ug or encoder from the ESM2 language model (Lin et al. 2023), transforming discrete tokens into a continuous latent space. Gaussian noise is then applied to this embedding space. The prior distribution is defined as:
(x) = 10,
while the forward process follows Gaussian corruption schedule:
pxx( Al a.).), ā II ais 02 3x-
(2)
The variance schedule { follows the cosine schedule proposed by Nichol & Dhariwal (021) which is commonly used to stabilize training.
31

---

Reverse process. Po(W/x, t) that predicts the sequence from noised points X This done by minimizing the following objective:
The reverse process aims to recover the original sequence by learning function
((0) Ewo, [-
Polooooooooog (woo X2 p(x, Ueww)
(3)
By learning po (wr x4, t), construct the reverse transition distribution:
po (xxx)
Σ pxxe_11 Up Uo
(4)
where the posterior p(x;-1 x0) follows:
p(xi-1 xo) (ix
(5)
with mean jr and variance of given by:
1a--1
No + var(1 ā(-1)
X1.
(6)
1 á1-1 Bc. oi
att
Inference and sampling. At inference time, the learned reverse process is used to generate protein sequences from the prior (z).. This is done by iteratively sampling:
X Po (x,- xx),
(8)
and then reconstructing w by sampling:
Po ( Xn).
This denoising process iteratively refines noisy embeddings back into structured sequences.
A.3.2 Diffusion over discrete space.
Discrete diffusion models (Austin et al. 2021 Campbell et al. 2022 Lou et al. 2024) generate data in discrete spaces by reversing predefined forward Markov process. Specifically, family of distributions evolves according to the Markov chain
dp: dt
QiPt:
(10)
where Po Pdata is the data distribution and Q. E R NxN are predefined transition matrices.
This Markov process can be reversed with the help of concrete score function, s(x, t) = (x(x) as its time reversal is given by
dpr
Qr-
(11)
dt
where Q, x] = sxx, ()x,t)=Qi, (x. x) for x * x, and x xx
Qx x). To generate data
Xg ~ Pdata. we start with sampling Xy from uniform distribution and then evolve through Eq. by the Euler method.
Uniform discrete language models. Both D3PM (Austin al.
2021 and UDLM (Schiff et al. 2024) implement a uniform transition matrix Q: N 11 When T + oo, the probability distribution pr converges uniform distribution.
Masked diffusion language models. Masked diffusion language models (MDLM) (Shhoo et al.
2024) utilize an absorbing transition matrix Qe that converts tokens in a sequence to [MASK] states.
The corresponding transition matrix can be written as Q E (NN+1)x Q. = -I + e. When T + co, the limiting distribution pr converges completely masked sequence.
32

---

A.3.3 Autoregressive language models.
In this work, we finetuned the ProGen2-small decoder-only transformer (151 million parameters) based on the code and parameters used in Yang al. (2025a). Models were trained based on next token prediction and cross entropy loss. However, we did not use adapter layers, and we did not group batches based on sequence length. During inference from the autoregressive model, we used a temperature of I and Top-p value of
A.4 Steering methods
Table A2: Summary of supervised value functions used to predict fitness in this work, to guide diffusion models. All "classifiers" were trained as regressors to predict fitness. For DAPS methods, only clean data was used for training, whereas other classifiers are trained on clean and noised samples from various timesteps.
Model
Guidance Strategy
Max Epochs
Learning Rate
Batch Size
Architecture
Hidden
Dimension
Continuous Diffusion
CG
1000
1 10-3
128
4-layer MLP
256
DAPS
1000
1 10-3
128
4-layer MLP
256
Continuous
NOS
100
x 10-3
128
1-layer MLP
256
Diffusion
Discrete Diffusion
CG
1000
1 10-3
64
4-layer MLP
64
DAPS
200
x 10-3
64
4-layer MLP
64
Discrete Diffusion
NOS
200
3 x 10
64
linear layer
n/a
A.4.1 Classifier guidance
Classifier guidance (Song et al.) 2021) is technique used to steer samples generated by diffusion models toward desired attributes. The primary goal to sample from conditional distribution p(xly), where y is guiding signal of interest. In continuous space, this can be achieved by replacing the unconditional score function Vx, log (xx.) at time t by a conditional score function,
Vx, ((x.l()) = xx, (x;)) Vx, log (y(())
(12)
To obtain the conditional score function, one only needs to train a time-dependent predictor, which predicts the probability of p. (x()).) given x, and time
Continuous guidance. Classifier guidance modifies the reverse diffusion process to steer generated samples toward desired property, represented by conditioning variable The guided sampling
process modifies the update rule for x, by incorporating a classifier score Vx. log p(y into the
model's learned score function based on the relation in Eq. [2] Following [Song al. (202T). the classifier guidance term modifies the predicted &o in the denoising process:
Xo X₁ + o (so f) + Vx. logp(y(x:)).
(13)
Since our diffusion model directly predicts logits rather than the score function se f), adding classifier guidance requires modifying
predicted &o.
Instead of predicting the score function explicitly, our model predicts logits over the vocabulary, from which the denoised representation & obtained. We modify Xo by incorporating classifier gradients
as follows:
Compute the unmodified xa using the model's predicted logits:
X
p( Xx,, U)
(14)
33

---

Table A3: Hyperparameters used to tune the guidance/sterring process. The bolded parameter
was chosen as the ideal parameter for the iterative "Bayesian optimization" experiment (Fig. ( Larger guidance parameter typically implements stronger guidance strength.
Guidance Strategy
Hyperparameters
Continuous CG
1/3 - 64, 128, 256, 512, 1024 1 6.25, 15.625, 39.0625
Discrete CG
Continuous DAPS
0.250 1,2,4 10* K 50
Euler method steps 10
Langevin dynamics steps 100
Discrete DAPS
1/3 16, 32, 64, 128, 256
K = 50
Euler method steps 20
Metropolis Hastings steps 1000
Continuous NOS
À 1, 100, 1000
7 = 0,5,2.5
K 5, 10
optimizer AdaGrad
Discrete NOS
x 1, 100, 1000
7 = 0.5,2,5
K 10
optimizer AdaGrad
DPO
8 0.02, 0.1, 0.5, 2, Ir I= 1 x 10
epochs =
batch size = 8
where Ug is the embedding matrix mapping discrete tokens to continuous space. If time-dependent classifier f is available, compute the classifier guidance term:
Xx, log loggloyyyxxx. (())):))=)))))))))))))()) Vx, fxx, t)BB.
(15)
Modify No using the classifier gradient:
X V² log logp(y(x:).
(16)
This allows the diffusion model to generate samples that are more likely to satisfy the desired condition y.
Further details on training the classifier are provided in Table A2 and Table A3
Discrete guidance. Nisonoff et al. (2025) extend classifier guidance to discrete state-space dif- fusion models. In analogy classifier guidance for continuous diffusion models, they modify the unconditional rate matrix Q, (as defined Eq. 11)
to be a conditional rate matrix R. with
p(y [ Q1xx, x], VX *
(17)
ply
For classifier guidance on both continuous and discrete diffusion models, we train a time-dependent predictor (classifier) ff that predicts the fitness y given x₃ at time t. We define p(y[x) x exp( (x)) /8) where f(-) is surrogate predictor of the fitmess, and B is the guidance temperature and gov- erns the strength of guidance. Therefore, Vx, log V (). and R. [x X
exp ( (f( - f(x,t)) xx.
To obtain classifier / for discrete diffusion models, we trained an MLP regressor to predict the fitness of one-hot encoded sequence given x, and uniformly random time E [0, T). Further details provided in Table A2 and Table A3
34

---

A.4.2 Posterior sampling
Another line of guidance work (Chung et al. 2023 Mardani et al. 2024 Zhang et al. 2025) focuses on drawing samples from the posterior distribution p p(x(y) x p x), where the prior distribution is modeled by pretrained diffusion model. The conditional distribution p(y(x) can either be the likelihood function of forward model (i.e., when y is an incomplete measurement of x) or an exponential distribution with respect reward function (i.e., ply|p(pyxxp exp( (xp([(x)/8)) The major difference between posterior sampling and classifier guidance is that requires the reward function to be trained only clean data x.
While many works have studied posterior sampling in Euclidean space with continuous diffusion models, posterior sampling for discrete data has been less explored. We modified DAPS Z al. 2025) enable diffusion posterior sampling in discrete-state spaces. Suppose lies finite support X" we follow the following steps:
Initialize XT ~ px(xx)
for i = ,,..., K
1. Sample x6) ((xn/xx,,_,) by discrete diffusion model.
Run Mctropolis Hastings to sample x(( ~ p(xo(((xxxx (x,,, y) as defined in Eq. 18
3. Sample x,, ~ p(x,, (x)) following the forward Markov process.
Return XK-
Specifically, to, t1,..., tk are mono-decreasing time steps with to and tx so O. p(xolxx, y) is defined as
p(xolx,, y) oc p(y(xx)p(xx) xx)
= p(y|xx) exp(-11xx) X (x)xxx)lo/
(18)
where Xo (xx) ~ (() is point estimate of the conditional distribution, and we approximate p(xol xx) by an exponential distribution over Hamming distance. Following Proposition in Zhang et al. 22055
(i) (8)
X
Xg and X1, converge to the posterior distribution as fę goes to 0.
For posterior sampling with DAPS, obtained the value function using the same model architec- ture and training parameters as classifier guidance but only trained on clean data x (no noisy x2). We set K = 50 using the time scheduler for the original model. Further details provided in Table and Table
A.4.3 NOS
Diffusion optimized sampling (NOS) (Gruver et al. 2023) is guidance method for both continuous and discrete diffusion models, which utilizes gradient information of the continuous latent represen- tations of protein sequences. In pretrained discrete diffusion models, noisy sequences w, always have a continuous embedding in the form of hidden states of the neural network. Specifically, the denoising model that predicts Wo from Wt can be written pa (Wolg(( w.), l), where h, g(w,) is a continuous hidden states the model.
Instead of training a value function discrete sequences W., NOS proposes to train the value function on the hidden states h, In each diffusion step. NOS samples from the posterior distribution,
p(wolbe, (h. y) x po po(wolh:) exp(J(h.))
(19)
To sample from this distribution, NOS runs Langevin dynamics on h i.e.,
h < h/ V, (ADxL (W (wo/hh)) f(h.) + 21pe,, ~ (O, 7). After K iterations, denoise w. following the guided hidden state, i.e., p(Wc-1lwe.y)
(20)
Pa (ow,-1/h// [/,,
To train the value function used for guidance in NOS, following the method from Gruver et al. 2023 we trained a very shallow neural network on the final layer hidden embeddings the diffusion model.
Further details are provided in Table A2 and Table A3
35

---

A.4.4 Direct preference optimization
For DPO with language models, we used the weighted loss function from Widatalla et al. (2024)
and Stocco et al. (024)) (Eq. 21P. Ty is the policy to be updated, Tee is the original model, and
is tunable parameter describing the extent of drift from the reference model. The loss therefore
describes the cross entropy of the ratio log Ta((X)
Ta(x) and the fitness value w. Following Stocco et al.
(2024), we calculated the ratio r as the difference of the log likelihood of the sequence from the updated model minus the log likelihood of the reference model, and softmax was applied to all of the fitness values w. We used the default parameters from (Stocco et al. 2024) and tested increasing the learning rate to 10-1 but found that generation quality broke down above the levels used in Table A3 with finetuning for 5 epochs. We also tested ranked loss with other types of models, but the performance was similar.
(T( (T)
k exp
(Blog (z)
TTr((1) )
LpPOncianc (TT; Tret)
EED th
B Blog
log Σ j=k
(21)
A.5 Adaptive optimization algorithm
Algorithm 1 Adaptive Optimization with Guided Generative Models
Input: Pretrained generative prior p(x), initial empty labeled dataset Do number of rounds T. batch size B, ensemble size M
2: for t= to T do
3: Initialize batch X; 0
4: i >> 1 then Train ensemble of value functions {fe... Im=1 on Dz-1
5: while [/ < do
6: if f > then
7:
Sample value function fa Uniform(1.faa (mm) b Thompson-syy sampling Sample sequence X;, GuidedSample(p(xx). fo. GuidanceStrategy)
8:
9:
else
Sample sequence xa UnconditionaSampplin(sss)) end if
11:
x₁, & D then Add x; batch X; 13: end while
Evaluate true fitness ybb (Xp) for all x₁ X, 15: Update dataset: D, <- D.-1 U (Xx, m
16:
end for
17:
Return: Best observed sequence in DD
We used an ensemble size of M = 10 models, each trained with different random initialization of neural network weights. In practice, to speed up sampling, we sampled (B 100 samples)(MM 10 models)= 10 sequences in each GPU batch using the same Thompson-sampe than using a GPU batch size of Alternatively, for the Gaussian process model, we trained the model with the radial basis function kernel, and we sub-sampled the total amount of training pairs (when using noisy samples) to 5000 samples.
value function, rather
A.6 Latent space Bayesian optimization with
We utilized the APEXGo codebase (Torres et al. 2024). a package for training generative variational autoencoders over peptide sequences and then optimizing those sequences with latent space Bayesian optimization to maximize certain properties. We used the training code out-of-the-Be variational autoencoders over the same MSA sequences used to train priors for discrete diffusion models in SGPO. We trained until losses plateaued, to 542, 241, and 391 epochs for TrpB, CreiLOV, and GBI, respectively. Overall, the reconstruction losses were low, and generated sequences had low perplexity and high fitness, comparable to the generative models used in SGPO. We then used this latent space and the APEXGo optimization algorithm to maximize the fitness sequences as measured by the oracle used in SGPO benchmarking. Specifically, in our configuration, set
to train
36

---

the number of initialization points to 100, the number of desired diverse solutions to 1. the max number of oracle calls 800, and the batch size 100-to mimic the iterative Bayesian optimization experiments performed in Fig. 6
A.7 Additional results
Target
Continuous
Continuous- WWW.EEMMMMDMMMSHHMMMH Baseline
D3PM
UDLM
ARLM VFV
MOLM
VFVC
VFV
T
VFVG
VEVG
VEVS
VEVG
A
H
CreiLov TrpB
IS
103
124 226
194
183 184 226
183
227 228
165 184 227 228
103 184 227 228
103 184 227 228
183 184 227 228
184 227 228
RSORE
RTORE
RT
RSVRA
RSORE
RGQLR
RSÈRE RSORE
WR
ESHWR
ESTAR
RRA
AR
AAD
ETR
WA
RE
KILAD
WR
A T
KAPER
A
29 34 60
60 29
60
Figure A3: The distributions of sequences sampled from pretrained generative priors largely match those of the target distribution. The target distribution shows all sequences in the MSA. and the distributions definitions can be found in Table The residues shown for TrpB are 4 out of 15 positions studied in the dataset (parent VFVS), and 5 out of 1 119 residues for CreiLOV are shown as they correspond to those harboring favorable mutations in the original dataset (parent is AGQRD). Note that the target distribution for training the ARLM is slightly different than that shown here.
generative models are approximated by sampling 1000 sequences. Model
30
Mean Fitness
2
GB1
Random
Perplexity
Target
Continuous
20-
1
D3PM
MDLM
ARLM
2
C
Diversity
Target
Continuous
D3PM
MDLM
ARLM VDGV L
VDGV LXKA
VDGV
VDGV
VDGV
39 40
40
Figure A4: Additional results for GB corresponding to Fig.
andd Fig.
The 4 positions shown
correspond to the parent sequence of VDGV.
TrpB
0.15
B CreiLOV
Random
0.6
Target
Density
0.4
Density
0.10
Continuous ESM
Continuous
D3PM-Baseline
0.2
0.05
D3PM
0.0
0.00
5
10
40 80
120
Hamming Distance to Parent
Hamming Distance to Parent
ARLM
Figure A5: Generated sequences from pretrained priors are more similar to parent than random for
(A) TrpB and (B) CreiLOV, measured by the Hamming (or edit) distance. UDLM models exhibit mode collapse onto consensus sequence(s) in the training distribution. The parent sequence refers to the starting sequence used to generate variants in the original protein fitness dataset.
37

---

Continuous
D3PM
MDLM
1.0
Mean Fitness
C
CG
DAPS
0.5
ARLM+DPO
TrpB
Unconditional
0.0
Unconditional (ARLM)
Mean Fitness.
4-
CreiLOv
Mean Fitness
GB1
100
200
100
200
100
200
Unique & Novel
Unique & Novel
Unique & Novel
Figure A6: Pareto boundaries demonstrate the trade-off between generating sequences with high fitness and high diversity for TrpB (A-C), CreiLOV (D-F), and GB1 (G-I) showing the same experiment as Fig. [ Error bars show standard deviation. Mean fitness and diversity were calculated based on 200 generated samples, with diversity calculated as the total number of unique and novel (previously unseen) samples in the generated batch, out of 200. Larger circles indicate a stronger guidance strength, specified in Table. A3
Table A4: Adaptive optimization with an ensemble of 10 value functions and Thompson sampling, compared using single model for guidance. Max fitness refers the mean max fitness achieved at the end of the campaign using the same experimental setup as Fig. over different random initializations.
Protein
Model
Guidance
Max Fitness (Ensemble)
Max Fitness
(Single Model)
TrpB
D3PM
CG
1.551
1.542
DAPS
1.595
1.568
MDLM
CG
1.551
1.542
DAPS
1.595
1.568
CreiLOV
D3PM
CG
5,608
5.552
DAPS
5,522
5.520
MDLM
CG
5.608
5.552
DAPS
5,530
5.520
38

---

D3PM
MDLM
TrpB
2.5
CG
Diversity
DAPS
2.0
Ensemble
1.5
Single Model
C
CreiLOv
2.5
Diversity
2.0
1.5
8
Guidance Round
Guidance Round
Figure A7: Diversity of generated sequences, measured by average Shannon entropy of mutated positions, during each round of guidance. Using an ensemble of value functions sampling generally shows higher diversity than using single model. Experimental setup is the same Fig. 66 and experiments were repeated over 5 random initializations.
and Thompson
39


Reviews:
[{"review_id": "4PVW98fwXU", "reviewer": "Reviewer_kDSu", "summary": "This paper introduces SGPO (Steered Generation for Protein Optimization), a framework for finding protein sequences with desired properties. The method pretrains generative models on naturally occurring protein sequences, then steers generation using limited experimental fitness data. The authors compare generative models and steering strategies, finding that plug-and-play guidance with diffusion models outperforms reinforcement learning approaches like DPO when data is scarce. They also present an adaptive optimization strategy using ensemble fitness predictors to efficiently explore protein sequence space and identify high-fitness variants.", "questions": "- Could the authors provide additional guidance on how practitioners should select or tune the guidance strength hyperparameter in real-world, held-out experimental campaigns? Are there any unsupervised or data-driven heuristics that might mitigate this sensitivity?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\n- The paper offers an extensive comparison of generative modeling and steering strategies, including multiple types of diffusion models (continuous, uniform discrete, absorbing-state discrete), plug-and-play guidance methods, and RL finetuning baselines.\n- SGPO is contextualized as a unifying framework addressing the limitations of zero-shot, supervised, and prior-only methods. The approach is directly relevant to core challenges in protein engineering, such as efficiency in low-data regimes and scalability to large design spaces.\n\nWeaknesses:\n- The main experiments are conducted on just TrpB and CreiLOV, both of which are relatively \"tractable\" due to large multiple sequence alignments. While the justification is reasonable, this limits the assessment of broader generalizability to more challenging or less-studied protein families. Results on a more diverse set of proteins or harder fitness objectives would significantly strengthen impact.\n- While guidance-based methods require tuning only a single parameter, the results show significant performance dependence on this hyperparameter. In real optimization campaigns, it is unclear how users would choose this value in the absence of held-out ground truth.", "was_revised": true, "final_justification": "The authors' rebuttal has addressed the generalizability and hyperparameter selection issue.", "rebuttal": "We thank the reviewer for their constructive feedback and their support of our work. Below we address the specific concerns raised.\n\n### **W1**: “Results on a more diverse set of proteins or harder fitness objectives would significantly strengthen impact.”\n\n*Our response:* We thank the reviewer for this suggestion. To better demonstrate the generalizability of our methods, we have added a third case study using the GB1 dataset [1], which is commonly used in the protein design literature. Unlike TrpB and CreiLOV, GB1 has a much smaller multiple sequence alignment (only 126 sequences), making it significantly more challenging in the low-data regime. Despite this, our methods continue to perform strongly on GB1. In particular:\n\n1. DAPS consistently steers generation toward high-fitness variants, even with limited data, and does so while maintaining sequence diversity.\n2. In adaptive optimization experiments, DAPS significantly outperforms CG and other baselines, reinforcing its robustness and effectiveness across different tasks.\n\nThese new results further highlight the versatility of our framework and position DAPS as the strongest overall method in our evaluation. Figures 4–6 will be updated in the revision to include the GB1 results. For now, we provide a preview in tabular form below. \n\n**GB1 Results Table 1**\n| Model      | Method   |   Guidance Strength |   Mean Fitness (↑) |   Diversity |   # Unique & Novel |\n|:-----------|:---------|--------------------:|-------------------:|------------:|-------------------:|\n| Continuous | DAPS     |                2500 |            2.52908 |    0.320212 |              180   |\n| Continuous | DAPS     |                5000 |            2.96662 |    0.307461 |              179   |\n| Continuous | DAPS     |               10000 |            3.62737 |    0.276158 |              169.6 |\n| Continuous | DAPS     |               20000 |            4.15179 |    0.267026 |              177.9 |\n| Continuous | DAPS     |               40000 |            **5.0329**  |    0.29075  |              187.9 |\n| D3PM       | DAPS     |                  16 |            3.79125 |    0.585082 |              198.9 |\n| D3PM       | DAPS     |                  32 |            4.79    |    0.61515  |              200   |\n| D3PM       | DAPS     |                  64 |            5.91924 |    0.629356 |              200   |\n| D3PM       | DAPS     |                 128 |            6.59207 |    0.534547 |              199.9 |\n| D3PM       | DAPS     |                 256 |            **6.81778** |    0.436094 |              198.6 |\n| MDLM       | DAPS     |                  16 |            3.65391 |    0.901675 |              199.5 |\n| MDLM       | DAPS     |                  32 |            4.88479 |    0.803669 |              199.8 |\n| MDLM       | DAPS     |                  64 |            5.90884 |    0.818706 |              200   |\n| MDLM       | DAPS     |                 128 |            6.54008 |    0.65398  |              200   |\n| MDLM       | DAPS     |                 256 |            **6.74237** |    0.450862 |              199.8 |\n\n**GB1 Results Table 2**\n| Method               |   Max Fitness Achieved (↑) |\n|:---------------------|---------------------------:|\n| DAPS (MDLM)          |                  8.82489   |\n| CG (MDLM)            |                  7.78204   |\n| Unconditional (MDLM) |                  4.54604   |\n| Continuous+NOS       |                  4.3317    |\n| ARLM+DPO             |                  4.30037   |\n| Random               |                  0.0169195 |\n\n### **W2/Q1**: “Could the authors provide additional guidance on how practitioners should select or tune the guidance strength hyperparameter in real-world, held-out experimental campaigns?”\n\n*Our Response:* Thank you for raising this important point. We would first like to emphasize that the methods explored in our study, such as CG and DAPS, are intentionally designed to be simple to tune, with only a single steering hyperparameter. This is in contrast to methods like NOS and DPO, which involve more complex and sensitive hyperparameter choices. We will add the following sentence to the revision to offer more concrete guidance: \n\n“In real-world engineering scenarios, even in the absence of ground truth fitness labels, one practical approach to selecting the guidance strength is to scan over values and choose the highest setting for which the N generated sequences remain unique and novel relative to previously measured sequences, where N corresponds to the screening throughput available for the next round.”\n\nThis ensures that the chosen guidance strength balances optimization with diversity and remains actionable under practical experimental constraints.\n\nPlease let us know if you have any further feedback or questions!\n\n### Citations:\n\n[1] Olson, C. A.; Wu, N. C.; Sun, R. A Comprehensive Biophysical Description of Pairwise Epistasis throughout an Entire Protein Domain. Current Biology 2014, 24 (22), 2643–2651."}, {"review_id": "RXZjfjlynt", "reviewer": "Reviewer_LY7c", "summary": "The paper considers the problem of protein fitness optimization, where the goal is to find a protein sequence that maximizes a certain property of interest. The paper provides a framework, termed Steered Generation for Protein Optimization (SGPO), which combines a pretrained generative model of protein sequences (capturing evolutionary information) with a small, iteratively updated dataset of labeled sequence-fitness pairs. The core idea is to use this limited labeled data to train a surrogate model for the fitness, which then \"steers\" the sampling process of the generative prior model to generate new, high-fitness candidate sequences. The paper empirically evaluates various generative priors (e.g., discrete diffusion models) and steering methods (e.g., classifier guidance, posterior sampling) in a simulated adaptive optimization loop.", "questions": "Please see Strengths And Weaknesses section.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- The problem studied in the paper is important for many real world applications. \n\n- I especially commend the well-written and comprehensive python library that is released with the paper. Such artifacts can easily catalyze a lot of downstream work. \n\n---\n\n- The paper introduces \"Steered Generation for Protein Optimization (SGPO)\" as a \"useful, general framework\" (line 65) which suggests to the reader that this framework is being instantiated for the first time. I will categorize this as an instance of Bayesian optimization (BO) where the acquisition function optimization is guided by a generative model.  For example, it is mentioned in line 36 that “Firstly, SGPO utilizes labeled data, which is important for fitness goals that deviate from natural function” which is very natural and standard practice in a BO formulation. I assume the concept of adaptive optimization is probably more surprising for researchers working on developing generative models, who would typically focus on zero-shot or few-shot optimization, but this is completely standard practice for the BO community. Even more importantly, there is a large body of work combining BO with deep generative models now (please see [1-5]). Some of these papers are cited in the paper but not discussed meaningfully, mostly grouped with a big list of other methods. Please discuss this line of work and clearly differentiate your work within this context. \n\n- I think the paper should invest more effort in making the baselines strong. The main contribution of the paper is a study of different methods and their combination. For example, statements like “NOS does not seem to allow for as much steerability, although we acknowledge that different hyperparameters could be tuned for NOS and DPO to potentially improve performance“ make the contribution a bit weaker. I believe precisely this is the type of paper where one should make an effort to ensure that the hyperparameters of the baselines are tuned properly. This way the community can learn a lot from the paper than its current form. \n\n- Strong baselines like NOS[1], LOL-BO[2] were not included in the adaptive optimization part of the results. Please consider including them.\n\n- It is mentioned that “Most existing approaches fail to incorporate principles from adaptive optimization” in line 58. I think this is not entirely correct. For example, LAMBO [Gruver et al] and many other ‘BO + generative models’ approaches do incorporate adaptive optimization principles. \n\n- It is mentioned in line 150 in relation to  autoencoder style methods that “However, it can be difficult to ensure that decoded sequences are valid and biologically meaningful”. I think this is not true and in fact, the citation at the end of this sentence (Maus et al) works extremely well for the space of molecules with no such challenges.\n\n- One of the main baselines in the paper is an autoregressive LLM finetuned with DPO. The choice of DPO in this setting seems a bit unclear since it converts absolute fitness values into pairwise preferences, which necessarily discards information available to the optimization process. \n\n- I find the justification for TrPB dataset in Table 2 a bit surprising: “While the TrpB dataset has a lot more training labels, it may be more difficult to learn due to high amounts of epistatic effects between residues(non-additivity of mutation effects)”. I imagined these are precisely the kind of hard scenarios where we hope deep learning based generative models and supervised models should succeed. Otherwise, we could easily use simpler methods like a gaussian process with string kernels as surrogates and local greedy search/evolutionary search for selection of next candidates. \n\nReferences \n\n[1] Gruver, Nate, et al. \"Protein design with guided discrete diffusion.\" Advances in neural information processing systems 36 (2023): 12489-12517. \n\n[2] Maus, Natalie, et al. \"Local latent space bayesian optimization over structured inputs.\" Advances in neural information processing systems 35 (2022): 34505-34518. \n\n[3] Chu, Jaewon, et al. \"Inversion-based Latent Bayesian Optimization.\" arXiv preprint arXiv:2411.05330 (2024).\n\n[4] Kristiadi, Agustinus, et al. \"A sober look at LLMs for material discovery: Are they actually good for Bayesian optimization over molecules?.\" arXiv preprint arXiv:2402.05015 (2024).\n\n[5] Torres, Marcelo DT, et al. \"A generative artificial intelligence approach for antibiotic optimization.\" bioRxiv (2024): 2024-11.\n\n---\nOverall, the big picture idea of the paper (adaptive optimization with deep generative models) is very interesting but execution of the paper is slightly underwhelming. I am happy to increase my score if all the questions are addressed properly.", "was_revised": true, "final_justification": "The authors response and new results convinced me that this paper is a good contribution.", "rebuttal": "We thank the reviewer for their constructive feedback and for their support of our work. We respond to the specific comments below:\n\n### **W1**: “Please discuss this [latent BO] line of work and clearly differentiate your work within this context.” \n\n*Our Response:* Thank you for raising this important point. We agree that adaptive optimization with labeled fitness data is standard in the Bayesian optimization (BO) community, and we appreciate your suggestion to clarify how our work relates to this literature.\n\nOur goal in introducing SGPO is to unify and systematize a broad class of emerging techniques, including classifier guidance, posterior sampling, and reinforcement learning, for guiding generative protein models with labeled data. Many of these approaches, particularly in the protein modeling community, are not typically framed in BO terms. SGPO provides a coherent perspective that connects these different lines of work.\n\nThat said, we agree that SGPO is closely related to prior work combining BO with deep generative models that you have suggested, and we will revise the manuscript to clarify these connections. In particular, we will contrast SGPO with latent-space BO approaches and highlight key differences. SGPO offers greater flexibility by avoiding reliance on an explicit latent space, which enables the use of modern, more powerful generative models such as diffusion models and protein language models that are not easily accommodated by traditional latent BO pipelines. In response to your suggestion, we have also added APEX-GO (based on LOL-BO) from Torres et al. [1] as a representative baseline from the latent-space BO class of algorithms.\n\nThe approach of Gruver et al. [2] can be viewed as an instance within our broader SGPO framework. SGPO supports a wider range of generative models and steering strategies, and this generality is reflected in the substantial empirical improvements achieved by several of our proposed methods over NOS, the guidance technique introduced in [2].\n\n### **W2**: “I think the paper should invest more effort in making the baselines strong.”\n\n*Our Response:* Thank you for this suggestion. We have now included a more comprehensive hyperparameter search over 30 combinations of NOS hyperparameters following previous studies [2-3], across num_steps=5,10; step_size= 0.5,2,5; and stability_coefficient=0.1,1,10,100,1000. Fig. 5 will be updated in the revision, but in the meantime we report the performance of the top 2 ideal combinations of NOS hyperparameters in the table below, compared to the performance of DAPS with an optimized guidance strength. Ultimately, these new results corroborate previously reported results, confirming that DAPS methods perform better than NOS. \n\n**NOS Hyperparameters Results Table**\n|Protein|Model|Method|Mean Fitness (↑)|Diversity|nos_stability_coeff|nos_num_steps|nos_step_size|\n|:----------|:-----------|:---------|-------------------:|------------:|----------------------:|----------------:|----------------:|\n|TrpB|Continuous|DAPS|**0.904319**|1.2053||||\n|TrpB|Continuous|NOS|0.266629|1.80812|10|5|2|\n|TrpB|Continuous|NOS|0.259517|1.64129|100|10|5|\n|TrpB|MDLM|DAPS|**0.376086**|1.60617||||\n|TrpB|MDLM|NOS|0.16598|2.26534|1000|10|5|\n|TrpB|MDLM|NOS|0.161262|2.26718|0.1|5|0.5|\n|CreiLOV|Continuous|DAPS|**4.53023**|1.69757||||\n|CreiLOV|Continuous|NOS|4.19227|1.99188|100|10|2|\n|CreiLOV|Continuous|NOS|4.19123|1.98869|1000|10|2|\n|CreiLOV|MDLM|DAPS|**4.91902**|2.0076||||\n|CreiLOV|MDLM|NOS|3.81707|2.67316|1000|10|0.5|\n|CreiLOV|MDLM|NOS|3.81705|2.67689|1000|5|2|\n\nTo better demonstrate the generalizability of our methods, we have added a third case study using the GB1 dataset [4], which is commonly used in the protein design literature. Unlike TrpB and CreiLOV, GB1 has a much smaller multiple sequence alignment (only 126 sequences), making it significantly more challenging in the low-data regime. Despite this, our methods continue to perform strongly on GB1. In particular:\n\n1. DAPS consistently steers generation toward high-fitness variants, even with limited data, and does so while maintaining sequence diversity.\n2. In adaptive optimization experiments, DAPS significantly outperforms CG and other baselines, reinforcing its robustness and effectiveness across different tasks.\n\nThese new results further highlight the versatility of our framework and position DAPS as the strongest overall method in our evaluation. Figures 4–6 will be updated in the revision to include the GB1 results. For now, we provide a preview in tabular form below. \n\n**GB1 Results Table 1**\n|Model|Method|Guidance Strength|Mean Fitness (↑)|Diversity|# Unique & Novel|\n|:-----------|:---------|--------------------:|-------------------:|------------:|-------------------:|\n|Continuous|DAPS|2500|2.52908|0.320212|180|\n|Continuous|DAPS|5000|2.96662|0.307461|179|\n|Continuous|DAPS|10000|3.62737|0.276158|169.6|\n|Continuous|DAPS|20000|4.15179|0.267026|177.9|\n|Continuous|DAPS|40000|**5.0329**|0.29075|187.9|\n|D3PM|DAPS|16|3.79125|0.585082|198.9|\n|D3PM|DAPS|32|4.79|0.61515|200|\n|D3PM|DAPS|64|5.91924|0.629356|200|\n|D3PM|DAPS|128|6.59207|0.534547|199.9|\n|D3PM|DAPS|256|**6.81778**|0.436094|198.6|\n|MDLM|DAPS|16|3.65391|0.901675|199.5|\n|MDLM|DAPS|32|4.88479|0.803669|199.8|\n|MDLM|DAPS|64|5.90884|0.818706|200|\n|MDLM|DAPS|128|6.54008|0.65398|200|\n|MDLM|DAPS|256|**6.74237**|0.450862|199.8|\n\n\n**GB1 Results Table 2**\n|Method|Max Fitness Achieved(↑)|\n|:---------------------|---------------------------:|\n|DAPS (MDLM)|8.82489|\n|CG (MDLM)|7.78204|\n|Unconditional (MDLM)|4.54604|\n|Continuous+NOS|4.3317|\n|ARLM+DPO|4.30037|\n|Random|0.0169195|\n\n### **W3**: “Strong baselines like NOS, LOL-BO were not included in the adaptive optimization part of the results. Please consider including them.”\n\n*Our Response:* We have now included NOS in the adaptive optimization experiments, and it continues to underperform relative to CG and DAPS. In addition, we have incorporated APEX-GO [1] (a recent protein-specific instantiation of LOL-BO) as a representative latent-space BO method. Initial results across all three datasets suggest that APEX-GO performs poorly in this setting (see the table below for GB1 adaptive optimization results). We hypothesize that this may be due to (i) the longer sequences used in our work compared to original LOL-BO studies, and (ii) the difficulty in calibrating the trust region in very low-data regimes with limited rounds of optimization. We will provide a more thorough discussion of these limitations in the revised manuscript and will also update Figure 6 to reflect these new results.\n\n**APEXGO Results Table**\n|# Sequences|Max Fitness DAPS (↑)|Max Fitness APEXGO (↑)|\n|--------------:|----------------------:|------------------------:|\n|100|1.95523|2.04042|\n|200|4.60165|2.7743|\n|300|6.37959|2.84413|\n|400|7.1716|2.83733|\n|500|7.59393|2.90728|\n|600|7.91073|2.81892|\n|700|8.11334|2.84832|\n|800|8.30358|2.89129|\n|900|8.46983|2.86372|\n\n### **W4+5**:  Concerns about statements in Lines 58 and 150.\n\n*Our Response:* We have rewritten the specific sentence mentioned in line 58 and removed the sentence mentioned in line 150 to characterize existing work more accurately.\n\n### **W6**: “The choice of DPO in this setting seems a bit unclear since it converts absolute fitness values into pairwise preference …”\n\n*Our Response:* We explored two different DPO implementations using the “ranked” and “weighted” loss forms as outlined in [5]. Namely, the weighted form is able to capture absolute fitness values. We have now performed a more detailed hyperparameter search over these two types of models, and this comparison will be included in the revision. Overall, we found that DPO still does not enable much steerability, with “ranked” and “weighted” loss forms performing similarly.\n\n### **W7**: “I find the justification for TrPB dataset in Table 2 a bit surprising”\n\n*Our Response:* We appreciate the opportunity to clarify. Our intention was not to suggest that TrpB is uninteresting, but rather to note that the complexity of its fitness landscape (due to significant epistasis) poses a challenge despite the availability of many labeled sequences. We agree with the reviewer that such complexity is precisely what makes TrpB a valuable benchmark for testing generative modeling strategies. We have revised the phrasing accordingly.\n\nWe thank the reviewer again for their constructive suggestions. We believe the revised manuscript will be significantly strengthened as a result of your feedback. Please let us know if there are any further points you'd like us to address.\n### Citations:\n[1] Torres, M. D. T.; Zeng, Y.; Wan, F.; Maus, N.; Gardner, J.; De La Fuente-Nunez, C. A Generative Artificial Intelligence Approach for Antibiotic Optimization. bioRxiv, 2024.\n\n[2] Gruver, N.; Stanton, S.; Frey, N. C.; Rudner, T. G. J.; Hotzel, I.; Lafrance-Vanasse, J.; Rajpal, A.; Cho, K.; Wilson, A. G. Protein Design with Guided Discrete Diffusion. In Advances in Neural Information Processing Systems 36; 2023.\n\n[3] Schiff, Y.; Sahoo, S. S.; Phung, H.; Wang, G.; Boshar, S.; Dalla-torre, H.; Almeida, B. P. de; Rush, A.; Pierrot, T.; Kuleshov, V. Simple Guidance Mechanisms for Discrete Diffusion Models. In Thirteenth International Conference on Learning Representations, 2025.\n\n[4] Olson, C. A.; Wu, N. C.; Sun, R. A Comprehensive Biophysical Description of Pairwise Epistasis throughout an Entire Protein Domain. Current Biology 2014, 24 (22), 2643–2651.\n\n[5] Stocco, F.; Artigues-Lleixà, M.; Hunklinger, A.; Widatalla, T.; Güell, M.; Ferruz, N. Guiding Generative Protein Language Models with Reinforcement Learning. arXiv 2025."}, {"review_id": "cTBPBYfdqn", "reviewer": "Reviewer_ZFcc", "summary": "This paper tackles the task of protein fitness optimization, where one wants to generate realistic protein sequences that fulfill desired properties, i.e. fitness criteria. To this end, the paper investigates the performance of different guidance strategies (as well as direct preference optimization (DPO)) to sample different pre-trained generative models in a controlled manner, when only a small amount of experimental fitness data is available for guidance. The paper terms this \"Steered Generation for Protein Optimization\" (SGPO) and tests continuous-embedding diffusion models, uniform noise and masking-based discrete diffusion models as well as autoregressive models, using several different guidance methods (CG, DAPS, NOS; see paper for abbreviations and details). The paper quantitatively shows that for two protein optimization tasks, SGPO outperforms unconditional and random sequence sampling as well as autoregressive language models fine-tuned with DPO in certain situations. The paper quantifies this through diversity/fitness tradeoffs under different guidance strengths as well as a simulation of a real-world protein engineering setup, where samples are iteratively generated, then the value function is refined, and new samples are being generated.", "questions": "I do not have any further questions.", "limitations": "Limitations have been discussed appropriately.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "**Strengths:**\n- The paper is well written and easy to follow.\n- The paper provides a very thorough background section and discussion of various guidance methods, discrete diffusion models, as well as reinforcement learning-based methods.\n- The paper tackles a practically relevant task, protein engineering, which has many important applications.\n\n**Weaknesses:**\n- There is little methodological novelty in the paper. The paper merely benchmarks established discrete generative modeling frameworks and guidance/steering/fine-tuning methods for protein fitness optimization.\n- Since this is not a method development paper, but it rather focuses on benchmarking existing methods for application in real-world protein engineering, I would have hoped that the authors run very thorough experiments. However:\n  - There is no wet-lab validation. This would have been appropriate in this case, I believe, because the main target audience of this paper is not machine learning researchers but protein engineers. The experiments leverage a computational oracle for evaluation and it is not clear whether this generalizes to real-world wet-lab performance. \n  - The paper only studies two different protein optimization tasks and it is not entirely clear how generalizable the results are to other tasks. We can see that the results can vary quite a bit depending on the task. For instance, looking at Figure 6, we see that for TrpB and MDLM, the ARLM+DPO baseline performs as good as the other methods, whereas for CreiLOV, it performs worse than CG and DAPS. Hence, it is hard to make definite conclusions. Moreover, that these methods perform better than not doing any guidance or fine-tuning at all is a somewhat trivial finding.\n  - Furthermore, as the authors pointed out, most methods come with various hyperparameters that could be investigated more deeply. While it is cumbersome and challenging to tune those hyperparameters, I would expect the authors to study method tuning in depth, and include thorough discussions and analyses on this in the main paper, in particular because the focus of the paper is to carefully benchmark these existing methods.\n\n**Conclusion:** While the paper is generally well written and tackles a relevant problem, there is little methodological novelty and the benchmarking leaves some questions open; see above. Therefore, I am leaning towards rejection.", "was_revised": true, "final_justification": "While I appreciate the authors' rebuttal, it does not fundamentally change my impression of the paper. There is very little methodological novelty in this work and this paper would seem better positioned at a venue specifically for protein design and not at a core machine learning conference. Please see my review for my other points of criticism.\n\nThat being said, this is a borderline decision and if others are more enthusiastic about the work, I would also support accepting the paper.", "rebuttal": "We thank the reviewer for their feedback and for their support of our work. We respond to the specific comments below:\n\n### **W1**: “There is little methodological novelty in the paper…”\n\n*Our Response:* We respectfully disagree with the claim that the paper lacks methodological novelty. While our focus is not on proposing a new generative model or steering algorithm, our main contribution lies in the conceptual unification and rigorous comparative evaluation of a broad class of steering techniques that have emerged in recent years but have largely been studied in isolation. Our SGPO framework draws meaningful connections between approaches from classifier guidance, Bayesian optimization, and reinforcement learning, and enables their systematic evaluation across multiple generative models and tasks.\n\n### **W2**: “There is no wet-lab validation…”\n\n*Our Response:* We agree that wet-lab validation is an important next step. However, the scope of our study, which evaluates a broad range of design choices, makes comprehensive experimental validation intractable. In practice, wet-lab experiments are typically designed to validate a single modeling hypothesis due to cost and throughput constraints [1]. That said, we are actively pursuing collaborations to evaluate top-performing methods from our benchmark in targeted experimental campaigns.\n\n### **W3**: “The paper only studies two different protein optimization tasks and it is not entirely clear how generalizable the results are to other tasks. ”\n\n*Our Response:* Prior to our study, it was not clear that DAPS + D3PM/MDLM would be such a strong model, consistently outperforming baselines such as NOS and DPO, which we believe is a valuable insight to both ML researchers looking to guide discrete diffusion models and protein engineers looking to choose a strategy for wet lab campaigns. To strengthen this conclusion, we have added a third case study using the GB1 dataset [2], which is commonly used in the protein design literature. Unlike TrpB and CreiLOV, GB1 has a much smaller multiple sequence alignment (only 126 sequences), making it significantly more challenging in the low-data regime. Despite this, our methods continue to perform strongly on GB1. In particular:\n\n1. DAPS consistently steers generation toward high-fitness variants, even with limited data, and does so while maintaining sequence diversity.\n2. In adaptive optimization experiments, DAPS significantly outperforms CG and other baselines, reinforcing its robustness and effectiveness across different tasks.\n\nThese new results further highlight the versatility of our framework and position DAPS as the strongest overall method in our evaluation. Figures 4–6 will be updated in the revision to include the GB1 results. For now, we provide a preview in tabular form below. \n\n**GB1 Results Table 1**\n|Model|Method|Guidance Strength|Mean Fitness (↑)|Diversity|# Unique & Novel|\n|:-----------|:---------|--------------------:|-------------------:|------------:|-------------------:|\n|Continuous|DAPS|2500|2.52908|0.320212|180|\n|Continuous|DAPS|5000|2.96662|0.307461|179|\n|Continuous|DAPS|10000|3.62737|0.276158|169.6|\n|Continuous|DAPS|20000|4.15179|0.267026|177.9|\n|Continuous|DAPS|40000|**5.0329**|0.29075|187.9|\n|D3PM|DAPS|16|3.79125|0.585082|198.9|\n|D3PM|DAPS|32|4.79|0.61515|200|\n|D3PM|DAPS|64|5.91924|0.629356|200|\n|D3PM|DAPS|128|6.59207|0.534547|199.9|\n|D3PM|DAPS|256|**6.81778**|0.436094|198.6|\n|MDLM|DAPS|16|3.65391|0.901675|199.5|\n|MDLM|DAPS|32|4.88479|0.803669|199.8|\n|MDLM|DAPS|64|5.90884|0.818706|200|\n|MDLM|DAPS|128|6.54008|0.65398|200|\n|MDLM|DAPS|256|**6.74237**|0.450862|199.8|\n\n\n**GB1 Results Table 2**\n|Method|Max Fitness Achieved(↑)|\n|:---------------------|---------------------------:|\n|DAPS (MDLM)|8.82489|\n|CG (MDLM)|7.78204|\n|Unconditional (MDLM)|4.54604|\n|Continuous+NOS|4.3317|\n|ARLM+DPO|4.30037|\n|Random|0.0169195|\n\n### **W4**: “While it is cumbersome and challenging to tune those hyperparameters, I would expect the authors to study method tuning in depth”\n\n*Our Response:* Thank you for this suggestion. We have now included a more comprehensive hyperparameter search over 30 combinations of NOS hyperparameters, following previous studies [3–4], across num_steps = 5, 10; step_size = 0.5, 2, 5; and stability_coefficient = 0.1, 1, 10, 100, 1000. Fig. 5 will be updated in the revision, but in the meantime we report the performance of the top 2 hyperparameter configurations in the table below, compared to DAPS with an optimized guidance strength. These new results corroborate our previous findings, confirming that DAPS methods consistently outperform NOS.\n\n**NOS Hyperparameters Results Table**\n|Protein|Model|Method|Mean Fitness (↑)|Diversity|nos_stability_coeff|nos_num_steps|nos_step_size|\n|:----------|:-----------|:---------|-------------------:|------------:|----------------------:|----------------:|----------------:|\n|TrpB|Continuous|DAPS|**0.904319**|1.2053||||\n|TrpB|Continuous|NOS|0.266629|1.80812|10|5|2|\n|TrpB|Continuous|NOS|0.259517|1.64129|100|10|5|\n|TrpB|MDLM|DAPS|**0.376086**|1.60617||||\n|TrpB|MDLM|NOS|0.16598|2.26534|1000|10|5|\n|TrpB|MDLM|NOS|0.161262|2.26718|0.1|5|0.5|\n|CreiLOV|Continuous|DAPS|**4.53023**|1.69757||||\n|CreiLOV|Continuous|NOS|4.19227|1.99188|100|10|2|\n|CreiLOV|Continuous|NOS|4.19123|1.98869|1000|10|2|\n|CreiLOV|MDLM|DAPS|**4.91902**|2.0076||||\n|CreiLOV|MDLM|NOS|3.81707|2.67316|1000|10|0.5|\n|CreiLOV|MDLM|NOS|3.81705|2.67689|1000|5|2|\n\nFor DPO, we have also performed a more detailed hyperparameter search over the “ranked” and “weighted” loss variants, and this comparison will be included in the revision. Overall, we found that DPO still does not enable much steerability, with both loss forms performing similarly.\n\nPlease let us know if you have any further feedback or questions!\n### Citations:\n[1] Yang, J.; Lal, R. G.; Bowden, J. C.; Astudillo, R.; Hameedi, M. A.; Kaur, S.; Hill, M.; Yue, Y.; Arnold, F. H. Active Learning-Assisted Directed Evolution. Nat Commun 2025, 16 (1), 714.\n\n[2] Olson, C. A.; Wu, N. C.; Sun, R. A Comprehensive Biophysical Description of Pairwise Epistasis throughout an Entire Protein Domain. Current Biology 2014, 24 (22), 2643–2651.\n\n[3] Gruver, N.; Stanton, S.; Frey, N. C.; Rudner, T. G. J.; Hotzel, I.; Lafrance-Vanasse, J.; Rajpal, A.; Cho, K.; Wilson, A. G. Protein Design with Guided Discrete Diffusion. In Advances in Neural Information Processing Systems 36; 2023.\n\n[4] Schiff, Y.; Sahoo, S. S.; Phung, H.; Wang, G.; Boshar, S.; Dalla-torre, H.; Almeida, B. P. de; Rush, A.; Pierrot, T.; Kuleshov, V. Simple Guidance Mechanisms for Discrete Diffusion Models. In Thirteenth International Conference on Learning Representations, 2025."}, {"review_id": "4sFP7EfiXq", "reviewer": "Reviewer_aBuC", "summary": "The paper proposes a “steered-generation” pipeline for protein design, benchmarks three guidance strategies (CG, DAPS, NOS) across continuous and discrete diffusion models plus a DPO baseline, and shows that discrete diffusion with CG or DAPS works best on two protein datasets.", "questions": "Together with the above weaknesses, I have the following questions,\n\n1. In Fig. 3, the authors points out many conbination among prior, learning strategy, and steering methods, but this study just discussed some of them. What about the remaining approaches?\n\n2. In your experiments, why D3PM and MDLM models with the CG and DAPS guidance strategies achieved the best performance? Could you provide deeper explaination?", "limitations": "The authors have discussed the limitations. The paper would benefit by providing the source code for reviewers to evaluate the reproducibility of the proposed method.", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strength:**\n1. The study compares many guidance algorithms, diffusion architectures and an RL baseline under identical data budgets, filling a gap in prior work that typically evaluates a single combo.\n2. The ensemble-Thompson strategy integrates uncertainty into generation and outperforms single-predictor guidance.\n\n**Weakness:**\n1. The paper is not well-organised, making it difficult to read. The language should be carefully polished.\n2. In the experiment part, while the authors conducted many experiment on these two proteins, adding more datasets would strengthen the generality of the paper’s conclusions.\n3. All results rely on an in-silico oracle, and no new wet-lab validation is provided.\n4. Some citations were not be cited appropriately, such as EvoDiff in Table 3.", "was_revised": true, "final_justification": "I think the authors' detailed responses and additional experiments have addressed my concerns.", "rebuttal": "We thank the reviewer for their feedback on our work. We respond to the specific comments below:\n\n### **W1**: “The paper is not well-organised, making it difficult to read. The language should be carefully polished.”\n\n*Our Response:* We appreciate this feedback and will revise the manuscript to improve its clarity and organization. In particular, we plan to streamline the presentation of our experimental findings and rephrase several technical descriptions to ensure they are more accessible. If there are specific sections or passages that the reviewer found especially difficult to follow, we would be grateful for any pointers to help focus our revisions more effectively.\n\n### **W2**: “Adding more datasets would strengthen the generality of the paper’s conclusions.”\n\n*Our Response:* Thank you for this suggestion. To strengthen the conclusions from our paper and better demonstrate the generalization ability of our method, we have added a third case study using the GB1 dataset [1], which is commonly used in the protein design literature. Unlike TrpB and CreiLOV, GB1 has a much smaller multiple sequence alignment (only 126 sequences), making it significantly more challenging in the low-data regime. Despite this, our methods continue to perform strongly on GB1. In particular:\n\n1. DAPS consistently steers generation toward high-fitness variants, even with limited data, and does so while maintaining sequence diversity.\n2. In adaptive optimization experiments, DAPS significantly outperforms CG and other baselines, reinforcing its robustness and effectiveness across different tasks.\n\nThese new results further highlight the versatility of our framework and position DAPS as the strongest overall method in our evaluation. Figures 4–6 will be updated in the revision to include the GB1 results. For now, we provide a preview in tabular form below. \n\n**GB1 Results Table 1**\n|Model|Method|Guidance Strength|Mean Fitness (↑)|Diversity|# Unique & Novel|\n|:-----------|:---------|--------------------:|-------------------:|------------:|-------------------:|\n|Continuous|DAPS|2500|2.52908|0.320212|180|\n|Continuous|DAPS|5000|2.96662|0.307461|179|\n|Continuous|DAPS|10000|3.62737|0.276158|169.6|\n|Continuous|DAPS|20000|4.15179|0.267026|177.9|\n|Continuous|DAPS|40000|**5.0329**|0.29075|187.9|\n|D3PM|DAPS|16|3.79125|0.585082|198.9|\n|D3PM|DAPS|32|4.79|0.61515|200|\n|D3PM|DAPS|64|5.91924|0.629356|200|\n|D3PM|DAPS|128|6.59207|0.534547|199.9|\n|D3PM|DAPS|256|**6.81778**|0.436094|198.6|\n|MDLM|DAPS|16|3.65391|0.901675|199.5|\n|MDLM|DAPS|32|4.88479|0.803669|199.8|\n|MDLM|DAPS|64|5.90884|0.818706|200|\n|MDLM|DAPS|128|6.54008|0.65398|200|\n|MDLM|DAPS|256|**6.74237**|0.450862|199.8|\n\n\n**GB1 Results Table 2**\n|Method|Max Fitness Achieved(↑)|\n|:---------------------|---------------------------:|\n|DAPS (MDLM)|8.82489|\n|CG (MDLM)|7.78204|\n|Unconditional (MDLM)|4.54604|\n|Continuous+NOS|4.3317|\n|ARLM+DPO|4.30037|\n|Random|0.0169195|\n\n### **W3**: “All results rely on an in-silico oracle, and no new wet-lab validation is provided.”\n\n*Our Response:* We agree that wet-lab validation is an important next step. However, the scope of our study, which evaluates a broad range of design choices, makes comprehensive experimental validation intractable. In practice, wet-lab experiments are typically designed to validate a single modeling hypothesis due to cost and throughput constraints [2]. That said, we are actively pursuing collaborations to evaluate top-performing methods from our benchmark in targeted experimental campaigns.\n\n### **W4**: Additional citation.\n\n*Our Response:* We have now additionally included the EvoDiff citation in Table 3.\n\n### **Q1**: “In Fig. 3, the authors points out many conbination among prior, learning strategy, and steering methods, but this study just discussed some of them. What about the remaining approaches?”\n\n*Our Response:* We elected to provide readers with a broad overview of the methods that fall under the umbrella of SGPO. In our study, we focused on a manageable number of plug-and-play guidance strategies, which have low computational costs and are easy and practical for real-world engineering campaigns, compared to fine-tuning approaches. Future work could implement and compare the other approaches.\n\n### **Q2**: “In your experiments, why D3PM and MDLM models with the CG and DAPS guidance strategies achieved the best performance? Could you provide deeper explaination?”\n\n*Our Response:* We found that DAPS is the best steering strategy overall, consistently outperforming other guidance strategies such as CG and NOS. This may be because the classifier used in DAPS is only trained on clean data, whereas CG requires training the classifier on both clean and noisy data. We also found that the D3PM and MDLM models are better suited for discrete data, as they are able to capture the natural distribution of protein sequences, compared to the continuous model. Thus, for real world applications, we would suggest using DAPS + D3PM/MDLM.\n\n### **L1**: “The paper would benefit by providing the source code for reviewers…”\n\n*Our Response:* We have developed a comprehensive and well-organized codebase that is already publicly available on a GitHub repository. However, due to this year’s rebuttal policy, we are unable to share the link at this stage.\n\nPlease let us know if you have any further feedback or questions!\n\n### Citations:\n[1] Olson, C. A.; Wu, N. C.; Sun, R. A Comprehensive Biophysical Description of Pairwise Epistasis throughout an Entire Protein Domain. Current Biology 2014, 24 (22), 2643–2651.\n\n[2] Yang, J.; Lal, R. G.; Bowden, J. C.; Astudillo, R.; Hameedi, M. A.; Kaur, S.; Hill, M.; Yue, Y.; Arnold, F. H. Active Learning-Assisted Directed Evolution. Nat Commun 2025, 16 (1), 714."}]

Author rebuttal:

Meta-review: This paper proposes SGPO, a method to generate protein sequences with desired properties. The method pretrains generative models on naturally occurring protein sequences, then steers generation using limited experimental fitness data. 

Strengths：
- The overall plug-and-play nature is qutie appealing with small experimental data. 
- The paper offers an extensive comparison of generative modeling and steering strategies
 

Weaknesses:
- The main experiments are conducted on just TrpB and CreiLOV, both of which are relatively "tractable" due to large multiple sequence alignments. While the justification is reasonable, this limits the assessment of broader generalizability to more challenging or less-studied protein families. Results on a more diverse set of proteins or harder fitness objectives would significantly strengthen impact.
- the method overly relies on one hyperparameter to tune. 
- the method itself lacks novelty, but is rather a Bayesian optimization technique.
Decision: Accept (poster)