ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods
Authors: Michal Kmicikiewicz
Vincent Fortuin
Ewa Szczurek
Venue: NeurIPS 2025 poster ()
Keywords: protein design
active learning
robust exploration
fitness optimization
Primary area: machine_learning_for_sciences

Abstract: Designing protein sequences of both high fitness and novelty is a challenging task in data-efficient protein engineering. Exploration beyond wild-type neighborhoods often leads to biologically implausible sequences or relies on surrogate models that lose fidelity in novel regions. Here, we propose ProSpero, an active learning framework in which a frozen pre-trained generative model is guided by a surrogate updated from oracle feedback. By integrating fitness-relevant residue selection with biologically-constrained Sequential Monte Carlo sampling, our approach enables exploration beyond wild-type neighborhoods while preserving biological plausibility. We show that our framework remains effective even when the surrogate is misspecified.
ProSpero consistently outperforms or matches existing methods across diverse protein engineering tasks, retrieving sequences of both high fitness and novelty.

PROSPERO: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods
Michal Kmicikiewicz!12
Vincent Fortuin 2.34
Ewa Szezurek
Institute of AT for Health, Helmholtz Munich
School of Computation, Information and Technology, Technical University of Munich Helmholtz AI, "Munich Center for Machine Learning
$ Faculty Mathematics, Informatics and Mechanics, University of Warsaw (michal kmi ciliiiiiz,,,, vincent. fortuin, wwwzureeeeheenlieeeeee www.a//oott-mmmcihhhee
Abstract
Designing protein sequences of both high fitness and novelty is challenging task in data-efficient protein engineering. Exploration beyond wild-type neighborhoods often leads to biologically implausible sequences or relies on surrogate models that lose fidelity in novel regions. Here, we propose PROSPERO, an active learning framework in which a frozen pre-trained generative model guided by surrogate updated from oracle feedback. By integrating fitness-relevant residue selection with
Sequential Monte Carlo sampling, our approach enables exploration beyond wild-type neighborhoods while preserving biological plausibility. We show that our framework remains effective even when the surrogate is misspecified. PROSPERO consistently outperforms or matches existing methods across diverse protein engineering tasks, retrieving sequences both high fitness and novelty.
Introduction
Proteins are essential macromolccules that play central role in virtually all biological processes. The ability to design novel protein sequences with desired functional properties is crucial for wide range of applications, including drug design, industrial biotechnology. and beyond [1-3]. Despite this promise, optimization of protein sequences remains grand challenge in computational biology. The protein landscape [4], mapping between the space of sequences and fitnes,, their corresponding functional levels, is typically rugged, sparse, and highly non-convex [5, 6]. Moreover, upon proposing a candidate sequence from the combinatorially large search space, evaluation requires querying an expensive black-box objective function. To alleviate this, machine learning models are often used as inexpensive surrogate models that approximate the costly black-box oracle [7-9]. To further facilitate navigation of the landscape, optimization commonly begins from wild-type sequence, preserved through natural evolution and, as such, exhibiting reasonable fitness [6, 10].
Numerous strategies have emerged to traverse protein fitness landscapes. Ren et al. [6] introduced PEX, an evolutionary algorithm that exploits the wild-type neighborhood. While highly effective and favoring biologically plausible sequences thanks to its local focus, this approach limits broader exploration of the fitness landscape, potentially missing advantageous sequences that are inaccessible through local mutations. Too overcome this, reinforcement learning (RL. methods [8, 10] and Generative Flow Networks (GFNs) [9] have been employed to target novel regions of the search space. However, these global exploration strategies often encounter issues with surrogate model
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Dataset Du
B1a
Astart
A1a
Mask by charge:
A1c
(R, K, H] [MASK']
(D. Bj [MASK']
A1a
(MASK)
Ala
(A) Proxy
(B) Residue
(C) Search space constraints
training
selection
MASK
Proxy
MASR
MASX
(E) Oracle evaluation
(D) Sequential Monte Carlo guidance
Oracle
Evo
Overview of PROSPERO. Each active learning iteration begins with training surrogate model on the current dataset (A). The surrogate is then used to identify fitness-relevant residues within the top sequence (B), which are subsequently masked, yielding partially masked sequences (C), EvoDiff, guided by the surrogate, completes these sequences to generate new candidates (D), which are evaluated by the oracle and added to the dataset (E).
Figure 1:
misspecification when evaluating sequences substantially different from the surrogate's original training distribution []]]. To address this and balance exploration with robustness, GFN-AL-OCS [12] learns an unmasking policy that reconstructs partially masked sequences. Yet, since masking is applied at random, this approach may modify conserved residues, potentially degrading structural and functional integrity and yielding biologically implausible proteins. Pre- reeetrained generative models offer compelling alternative, inherently encoding rich biological priors that greatly reduce the risk of generating implausible sequences [13]. However, effectively incorporating such models in itcrative optimization workflows presents challenge, each iteration would require impractical task-specific tunent on limited pracle-annotated data or low-fidelity
annotat-d data [14, 15]. The shorteomings of existing approaches point the need for protein design framework capable of generating high-fitness sequences beyond the wild-type neighborhood, while addressing the surrogate misspecification and loss of biological plausibility that often arise from exploring such novel regions.
To meet these challenges, introduce PROSPERO. Our main contributions are:
(i) robust exploration framework, to our knowledge the first to formulate iterative design of protein sequences as inference-time guidance of pre-trained generative model by surrogate updated in an active learning loop. This enables straightforward incorporation of biological priors encoded by the generative model, helping preserve biological plausibility even when surrogate- guided exploration extends beyond wild-rype neighborhoods.
(ii) A targeted masking strategy, which focuses edits fitness-relevant residues while preserv- ing structurally and functionally important sites. In contrast, prior approaches risk disrupting essential residues through random or uninformed masking.
(iii) Biologically conpcclll---otttiinneeeee Sequential Monte Carlo (SMC) sampling, offering a novel strategy incorporate explicit biological priors into inference-time guidance in discrete se- quence space. Restricting proposals to amino acids with properties similar to their wild-type counterparts increases the likelihood of retrieving fitness sequences in novel regions of the search space, where the surrogate may be misspecified.
We conduct extensive experiments across diverse protein fitness landscapes and demonstrate that PROSPERO consistently approaches the Pareto frontier between candidate sequence fitness and novelty, while preserving biological plausibility. We perform ablation studies under varying degrees of proxy misspecification to assess contribution of individual components to the overall robustness of our method. Code is available https://github.. wtwp://///ccccccceaalaooooogoo/

---

2 Related Work
Evolutionary Algorithms Evolutionary algorithms are common approach in protein sequence design [16, 7. 6]. Notably, Sinai et al. [7] proposed a greedy algorithm, AdaLead, which adaptively mutates and recombines high-fitnee fitness sequenees selected by threshold-basee filter to balance explo- ration and exploitation. Ren et al. [6] introduced an exploration method, PEX, designed to exploit the local neighhorhood of the wild-type by prioritizing variants with fewer mutations.
Directed Evolution (MLDE) To enhance traditional directed evo- lution [17], MLDE leverages machine learning models predict sequence fitness and guide the selection of promising mutants for screening [1]. Qin et al. [15] iteratively fine-tune ESM-1b [18] with data annotated by surrogate model. Tran and Hy [13] mask k-mers of wild-type sequence and use ESM-2 [19] to propose new candidate sequences. The work of Qiu et al. [20], Qiu and Wei
[21] and Wang et al. [22] explores clustering amino acids with similar properties; however, differs from our approach by assuming a small number of fixed mutational sites.
Reinforcement Learning and Generative Flow Networks (GFNs) DyNaPPO uses an ensemble of surrogate models with varying architectures to train a generative policy that constructs candidate sequences amino acid by amino acid [8]. Rather than acting in the sequence space, LatProtRL. [10] surrogate model
lears generative policy operating in the latent space of ESM-2 [19]. GFNs to learn a stochastic policy that samples sequences proportionally to their predicted fitness values [9]: however, when the proxy misspecified, they can perform poorly [11, 12]. To address this, Kim et al. [12] proposed GFN-AL-OCS, a strategy enabling to control a trade-off between novelty and robustness based uncertainty of the proxy.
Bayesian Optimization (BO) BO is a commonly used framework for optimizing expensive black- box functions and has been widely applied to the design of biological sequences under limited evaluation budgets [23-26]. Among these approaches, Amin et al. [26] optimize antibodies by sampling from LLM trained on clonal families, using a twisted SMC procedure to incorporate knowledge about previous experimental measurements. A detailed comparison of PROSPERO and Amin et al. [26] can found Appendix E.
Generative and Energy-Based Models Frey et al. [27] use Langevin Markov-Chor Baekov-Chhinn Monte Carlo to sample from smoothed data distributions for antibody discovery. Kirjner et al. [28] construct a smoothed version of the fitness landscape prior training surrogate model, whose gradients then used to guide the design of new sequences. However, the approach depends on a smoothing-strength hyperparameter, which is highly sensitive to the underlying fitness landscape and difficult tune in practice. Frameworks introduced by Brookes et al. [14] or Song and Li [29] can propose new candidate sequences by sampling from generative models like VAEs [30]. Ghaffari al. [31] present a VAE with a sitress-truccture latent space, enabling robust optimization despite the sparsity and ruggedness of the underlying fitness landscape.
3 Problem formulation
We aim to discover protein sequences ZILL E A with high fitness E R. where A denotes the vocabulary of 20 natural amino acids and represents the length of the sequence. Unless emphasis on sequential structure is needed, we simply write for brevity. The fitness value y measures a given property of a protein, such as binding affinity or fluorescence intensity. Designed sequences are evaluated by a black-box oracle f A + R. Since oracle evaluations suchh as wet-lab experiments or costly computational simulations- are expensive and time-consuming, queries are limited to a batch of K sequences per a small number of rounds N. We assume access to an initial dataset Do ((x())))))))) H which allows to train cheap and fast to query surrogate model fo AA" + R, approximating the oracle. Additionally, let &aaatt = arg max correspond to the wild-type sequence. The main goal is to design protein sequences with high fitness values assigned by the oracle across N active learning iterations. Desirably, generated sequences should also be biologically plausible, novel, and diverse.
3

---

Proposed framework
Algorithm 1: Active Learning with PROSPERO
Input: Oracle f. proxy fe, initial dataset Du, active learning rounds N, pre-trained generative
model P. oracle budget K, SMC batch size
for n1 to N do
Fit So on dataset 1:
0 + arg ming D.-1 [(f(x) f
Select starting sequence:
Juan arg ma fxx
Get masked variants of staart
Ji=1 {
TARGETEDMASKING(/ZN fe)
// Algorithm 2
Propose new candidate sequences:
{x(0) i=1 CONSTRAINEDSMC (17(0) 1
) P.
Algorithm
Evaluate candidates with the oracle:
Dn <
Update dataset:
Dn - n UDD
PROSPERO follows the active learning loop illustrated Figure and outlined in Algorithm 1, consisting of: (i) training the surrogate model fo the current dataset Dn by minimizing the loss =(0)=BuuDD... [x) - f=()))²] (ii) identifying and masking fitness-relevant residues Tsant with targeted masking; (iii) sampling new candidate sequences using biologicalll-connstrrinee SMC; (iv) evaluating candidates with the oracle f and augmenting - As demonstrated by Antoniuk al. [32], the use of the active learning loop is expected expand the support of the surrogate model, thereby providing more reliable guiding signal to the pre- pre-trained generative model. In the remainder of this section, describe cooloiccaly--constrineedd constrained SMC (line 4 and line 5 of Algorithm 1, respectively), which represent the key methodological advances of PROSPERO over prior approaches.
core innovations of our framework: the targeted masking and the
4.1 Targeted masking
Our targeted masking strategy is inspired by alanine scanning. mutagenesis technique used both experimentally and in silico to identify functionally important residues by substituting each position with alanine-a neutral amino acid that disrupts side-chain interactions [33-36]. Traditionally, alanine scanning aims to locate critical residues at time based wet-lab experiments in silico structural modeling. In PROSPERO, we propose a batched strategy that operates purely in sequence space to identify positions within Vstart that are fitness-relevant but the same time tolerant to mutation. Specifically, we construct batches of B mutated sequences (denoted collectively as (x()) ) by randomly substituting a subset of residues at locations ( C [1...., L} with alanine. Each such mutated sequence is scored by the surrogate model fo, which returns predictive mean and uncertainty estimate: (0( (=)(z())) (z ((( We select the top B sequences according to the Upper Confidence Bound (UCB) [37] acquisition function, identifying substitutions that are not immediately harmful yet exhibit uncertainty suggestive of functional relevance of the affected residues. For each selected sequence, we construct partially masked sequence $(0) by replacing previously substituted positions with a mask token: [ = [MASK] if j E I and [ [|] [)t otherwise. The resulting batch (+(1) == is then used as input to the guided generative procedure described next.
4.2 Biologically-constraine Sequential Monte Carlo
To design sequences with high fitness, one aims to sample from the posterior p(æ: y) x p(y) x) p(x). Since querying the true fitness function given by an oracle f is assumed to be expensive, p(y x) is

---

typically approximated using surrogate fo. This yields the following target posterior distribution:
(x) fo(r)- P(xx)
(I)
Z
where P(x)) denotes a prior over sequences and Z is normalization constant. PP(x) can be modeled in various ways; here, use WroDiff-OADA [38]. This formulation poses two key challenges: (i)
surrogate models may exhibit low fidelity on out-of-distribution sequences, and (ii) direct sampling from y(x)) is infeasible due the intractability of Z. Next, we describe how PROSPERO overcomes these challenges.
Addressing surrogate misspecification with biologically constrained exploration To encourage biologically plausible exploration even under potential surrogate misxpecification, constrain candidate sampling in PROSPERO by leveraging the charge class of wild-type residues as an ex- plicit biological prior. This draws inspiration from reduced amino acid alphabets (RAAs), which simplify sequence space by grouping residues with similar physicochemical and functional prop- erties, exploiting the many-to-one relationship between sequence and structure [39, 40]. Biasing exploration toward substitutions within the same class favors alternative sequences that are more likely to preserve wild-type fitness, irrespective of the surrogate quality. We select charge as the grouping criterion, given its both fundamental and universal role in stabilizing protein structure via salt bridges, hydrogen bonding, and electrostatic interactions [41]. Specifically, for each partially masked sequence z((), we further divide the set of masked positions (+) into three disjoint subsets:
positive ( = {j E 76) [ssan E {R, K, H}}; negative Z() = {j E I(() Tatanli] E {D,E)};
and neutral I (() {j Z [sstarll] & E,) We formalize constrained sampling as sampling from the conditional distribution PRAA(- +), defined over normalized logits of the base model restricted amino acids in the same charge class as the wild-type residues at positions I())
Sampling from intractable distribution using Sequential Monte Carlo
To sample from the intractable target distribution (z), in PROSPERO we perform approximate inference using SMC. Rather than directly sampling from complex, high-ddmennsionl ((I.L) defined over full sequences, SMC decomposes the problem into sequential sampling from series of simpler, unnormalized intermediate target distributions (fr(xra))fm) relying on a tractable proposal distribution and resampling based on intermediate importance weights (for background, refer to Appendix C). This allows us to sample sequences from the approximate target posterior in a residue- borduee-b--eiddue manner. We start by capitalizing on EvoDiff's order-agnoste nature by defining the sampling permutation (()
order for each z(()) as (+) concat( (Lj & Z I (()) For simplicity, we assume that all
sequences in the batch share the same number of masked positions [Z| max ()) In practice, for sequences with fewer masked tokens, no new proposals are made once all masked positions have been filled, but these sequences remain in the population and are still included in weighting and resampling (see Algorithm 3). We proceed by performing the following operations at each unmasking step t = L -
(i) Constrained proposal: for each (((),, we sample an amino acid at position & (1) from the (i)
constrained base model: (()
(()) (
(ii) Weighting: since the surrogate model fo operates only on fully unmasked sequences, intermediate targets "e(ā=(<1)) are defined only implicitly and cannot be used directly to compute importance weights Wr- Instead, we approximate wy by first rolling out the remainder of each sequence with the base model:
T
(RA (0) II a=t+1
R(s) =
(2)
unnoll
followed by scoring it with the surrogate model g(0) |00= و (x (mro)))) + k: ( As the logits of the base model are constrained to charge-compatible amino acids, samples from PRAS do not reflect the true prior over sequences. We therefore compute the perplexity of the unconstrained model P, compensating for cases where PRAA assigns uniformly low
5

---

likelihoods across all available choices:
Perp(rumm unroll) exp
P log un ummoll
Finally, the unnormalized importance weights for each sequence 5(0 computed as " = -g(t/ Perp(sunro)) with the perplexity term correcting bias introduced by the constrained sampling.
(iii) Resampling: we sample new population of partially masked sequences based on their normalized weights, effectively discarding sequences improbable under (*(())))
B
z0()) Cat
B 1==
W Wi
j==1 Σ
After all sequences have been fully unmasked, we select the top K candidates for oracle evaluation by ranking both the final population and intermediate rollouts from the last fkeepp unmasking steps according to their predicted UCB scores ş.
Experiments
We show that PROSPERO successfully balances generation of high-fitness sequences with exploration, while maintaining both diversity (Section 5.1) and biological plausibility (Section 5.2). In Section 5.3 and Section 5.4, we demonstrate the robustness of our approach under different forms of surrogate misspecification, namely covariate shift and surrogate noise. Finally, Section 5.5 investigates the contribution of individual components of our framework to overall performance. We evaluate PROSPERO based on the following general setup, which serves as the basis for all subsequent experiments.
Datasets and oracles We evaluate our method on eight diverse protein engineering tasks, details of which can be found in Appendix A.1. For the AAV landscape, we use ground-truth fitness scores provided FLEXS [7]. For all other tasks, following Ren [6], we use TAPE [42] as the oracle to simulate wet-lab experiments. Similarly to Kim et al. [12], we replace experimental measurements in each initial dataset with scores assigned by the oracle model.
Baselines We compare our approach against suite of established methods for iterative biological sequence design, covering diverse algorithmic paradigms: (i) evolutionary algorithms- Sarriittss--aaeeaa
PEX and CMA-ES [7, 6, 16]; (ii) on- on-policy reinforcement learning DDNagpD and LatProtRL [8, 10]: (iii) GFlowNets GFN-AL and GFN-AL-8CS [9, 12]: (iv) evolutionary Bayesian Optimization (BO) [7]: (v) probabilistic framework CbAS [14]; and (vi) the machine-learning-e asstssien directed evolution (MLDE) approach of Tran and Hy [13]. Further details are provided in Appendix A.4.
Implementation details We employ PROSPERO with N = 10 active learning rounds, each ending with a batch of 128 sequences evaluated by the oracle f. To model the proxy for we follow Sinai et al. [7] and Kim et al. [12], and use an ensemble of three one-dimensional convolutional neural networks. This architecture is shared across all baselines ensure a fair comparison. Sequence selection is guided by the UCB acquisition function fa (x: = n (æ) + k (ar), where k = for the targeted masking and k = 0.1 for the biologically-constraaaa
SMC. The SMC batch size set to B 256, and during generation we retain rollouts from the last Ylkeep 10 unmasking steps. In the targeted masking, the number alanine seans S = 16, with 3-10 mutations for shorter sequences (L ~ 100) as in the AAV, E4B, and Pabl landscapes, and 5-15 for longer ones.
Evaluation metrics We evaluate performance of all methods using four primary metrics, with particular emphasis on (i) maximun fitness, defined as the highest fitness value achieved among all generated sequences. This reflects the primary objective of discovering high-performing candidates. We also report: (ii) mean filness of the top 100 generated sequences; (iii) novelly, measured as the average Hamming distance between the top 100 sequences and the starting sequence Taaan and (iv) diversity, defined as the average pairwise Hamming distance among the top 100 sequences.

---

Table 1: Maximum fitness values achieved by each method. Reported values are the mean and standard deviation over runs. Green denotes fitness improvement over wild-type stur Bold: the best overall fitness per each task. Underline: second-best, PROSPERO improves fitness on every task, ranking first on out of and second on the remaining 3.
Method
AMIE
TEM
E4B
UBE21
LGK
CMA-ES
-6.87 +: 0.257
0.037
:: 0.01
-0.429 0.252
0.553
8,03
0.000
=0.000
1.972 -0.135
0.135 :: 0.178
-1.337
0.021
DyNaPPO
3,683 1 0.575
0.067
1: 0.008
1,924 =: 0.883
0.783 1: 0.036
0.009
0.011
3.550 1: 0.012
2.796
1. 0.059
-0.007
0.015
BO
0.168
1 0.056
0.682
1 0.369
7.412 L 0.212
0.814 1 0.081
0.667
1: 0.024
3.584 1: 0.07
2.883 1 0.069
0.026
1: 0.003
PEX
0.248 0.007
1,232
1 0.000
8.099
= 0.17
1
1.:.1...30 0.343
0.665
0.022
3,603 1: 0.001
2.991 1: 0.001
0.037
0.001
Adal
0:235
0.002
0.002
R1114 0.036
1,978
0.188
0.68344 0.057
1.581 D003
2,985
0.002
ChAS
.8.002
0.002
40.569
0.092
0.351
0.043
0100
0.000
1,85 0.067
.0,056+ 0.003
-1.492 D.D3
GFN-AL
-7,853 1: 0.270
0.027 1: 0.020
0,160 1: 0.228
0.507
0.025
0.000
0.000
2,004 0.022
0.271 1. 0.443
-1,164
0.118
GPN-AL--CS
0,203
0.005
0.701
+ 0.148
7,930
= 0.55
1.297 +: 0:337
11666+00.02
3:589
+ 0.006
2,984 + D:002
0.033
0.001
I LarProtRI.
0.224
0.000
1.229
+ 0.000
7,902 0.086
1,122
0.152
0.593 + 0,018
3,590
0.003
2.983 + 0.000
0.020
0.000
MLDE
0:241
0.003
1,229
0.000
7.934
0.8%6 0.015
0.000
3,596
0.003
2.984
0.003
0.002
PROSPIXO
0.246 0.006
1,231 0.002
1,527 0.254
0.720 0.027
3.617
0.002
2,993 0.003
5.1 Protein design evaluation
Fitness optimization PROSPERO consistently achieves superior or comparable performance to the baselines in generating candidate sequences with high fitness, irrespective of the underlying fitness landscape (Table 1). Our approach obtains the highest maximum fitness values on 5 out of 8 protein engineering tasks and ranks second on the remaining Notably, among the 11 evaluated methods, only PROSPERO and PEX are able to achieve fitness improvements over wild-type Istat across all landscapes, highlighting their reliability in diverse optimization scenarios. contrast, methods fail to achieve improvements on any task. We report further results only for 6 out of 11 methods that managed to improve fitness on at least half of the tasks. Results in Table 2 demonstrate that PROSPERO is consistently able to generate a broad set of nigh-fitness candidates, achieving the highest mean fitness among the top 100 sequences on out of landscapes and ranking second on 2. Notably, mean fitness fell below that of start on only single task. Additionally, Figure 2 shows that our method discovers high-fitness sequences at earlier active learning rounds than competing approaches on half of the evaluated tasks. A detailed comparison of early- performance across all benchmarks is provided in Appendix D.2.
Exploration and diversity As showcased in Table 3, PROSPERO attains high-fitness solutions with substantially greater novelty compared to other leading approaches, outperforming them on 6 out of tasks. Remarkably, PROSPERO frequently breaks the conventional Pareto frontier between sequence fitness and novelty, achieving levels of both that remain mutually constraining for the competing methods (Figure 3). In particular, although PEX-the second- best performing in terms of maximum fitness is designed to exploit the local neighborhood of the wild-type, our method often matches or exceeds its fitness while achieving approximately 2 times greater novelty. Importantly, the exceptional performance of PROSPERO in both fitness and novelty does not come at the expense of diversity. As shown in Table 9 in Appendix D.1, among leading approaches, only GFN-AL-ÓCS exceeds our method in diversity. However, it generates lower fitness sequences across all tasks, highlighting PROSPERO'S 's ability to maintain diversity without compromising performance.
5.2 Biological plausibility
Setup We assess biological plausibility on the E4B task, where a large reference dataset of over 80,000 sequences not included in Do is available (see Appendix A.I). Following the approach
Table 2:
Mean fitness of top 100 sequences generated by leading methods. Reported values are the mean and standard deviation over 5 runs. Green: fitness improvement over wild-type Taant Bold: the best overall fitness per each task. Underline; second-best. PROSPERO improves fitness on 7 out of 8 tasks, ranking first on 5 and second on 2.
Method
AMIE
TEM
E4B
Pabl
AAV
GFP
UBE21
LGK
PEX
0.238 + 0.004
1.227 1J002
7.948 - (1446
1.307 + 0.258
0.620 + 03.17
1.597 - 00/013
2.987 + 0.001
D.033 + 01001
AdaLead
0.229 0.001
1.201
0.002
7,846
0.440
1.836
0.266
0.644
-0.031
3.563
00007
2.976
0.003
0.037
0.001
AL-
0.192
0.027
1.653
0.136
0.113
2.668
0.006
0.024
0.064
LaiProRR
0.217
0.001
1,222
0.000
7.562
0.06
0.888
0.072
0.563
1 0.009
3.582
0.003
2.975
1: 0.001
0.019
1 0.000
MLDD
0.231
1 0.004
1.131 1 0.021
7,843
- 0.122
0.877
1 0.024
0.555
1 0.000
3.591 - 0.003
2,975
1 0.005
0.036
0.002
PROSPERO
0.236 0.007
1.176 0.029
8.017
0.054
1,401
0.202
0.679
0.025
3,613
0,002
2.987
1 0,003
0,040
0.002
7

---

AMIE
TEM
E4B
Pab1
Maximum
Itteess
0:25
0:24
1.23
0.23
1.23
0.22
1.23
AAV
GFP
UBE21
LGK
0.75
3.62
3.00
0.05
Madimum
(thees
0.09
3.61
2.99
0.04
0.02
3.60
2.99
0.03
0.56
3.38
22 2.
0.03
0.50
3.57
2.98
0.02
6
10
6
10
2
4
10
2
8
10
Active loarning rounds
lsarning rounds
learning rounds
learning
(ours)
LaProffi
Figure 2: Maximum fitness recovered over 10 active learning rounds. Only methods that improved over Zstart are shown. Shaded regions indicate standard deviation across runs. PROSPERO retrieves high-fitness sequences in earlier rounds than baselines 4 out of 8 tasks.
Table 3: Average novelty of top 100 sequences generated by leading methods. Reported values the mean and standard deviation over runs. Bold: the best overall novelty. Underline: second-best PROSPERO ranks first out of tasks and second on 1.
PEX
5.19 = 1.08
1.79 +: 0.21
3,96 + 0.54
5.29 0.74
7.29 1,38
6.23 =:23=10 1.34
4.55 = 0.82
8.45
1.39
AdaLeed
3,92
14.75
8.02
GFN-AL-CCS
8,29 = 0.42
10.10 = 0.65
4.94
= 1.59
10.29
1.29
9,70 0.33
34.83 3.77
8.01
3,69
63.16
4.24
LaiProtRI.
1.09 0.04
1,10 1. 0.01
3.11 0.49
3.85
1.19
3,03
0.40
12.73
2.21
1.69
1. 0.05
1.71 0.01
MLDE
19.02 1: 2.69
4.28 1 0.55
11.8
3.49
9.67
4: 3.12
4.48
0.25
23.65
1: 5.40
9.60
2.58
50.09 1 8.08
PROSPERO
20.99 3.32
3.37 1 0.53
8.81 :: 0.98
11.83
3,52
15.03
1.59
39.85
3,95
16.45 5.11
74.33 7.75
of Surana et al. [11], we define validity as the percentage of top 100 generated sequences whose key physicochemical properties fall within the central 99% quantiles of the corresponding property distributions in the reference set. Additionally, for landscapes where Tstant folds reliably, we further assess structural quality of generated sequences using pTM and pLDDT scores from ESMFold [19], well scPerplexity [38] computed after inverse folding with ESM-IFI [43]. More information about the metrics provided in Appendix A.5.
Results Figure 4A demonstrates that PROSPERO maintains biological plausibility while proposing sequences that over twice novel those generated by baselines with comparable validity, such as PEX and LatProtRL. Moreover, it achieves this while also attaining higher fitness. Sequences generated by PROSPERO exhibit strong folding confidence, with PTM and pLDDT scores consistently above 70 (Figure 4B). Notably, this performance remains comparable to that of the wild-type even on the L.G landscape, where candidate sequences differ from the wild-type by over 70 amino acids.
Mean fttness
1.28
8.13
2.17
0.09
0.99
7.95
1.82
-0.11
0.69
7.77
1.46
7.60
-0.50
7.42
13
26
3
13
AAV
GFP
UBE2I
LGK
Mean Itnees
3.62
2.99
3.60
2.98
0.63
3.58
2.98
0.03
0.58
3.57
2.97
0.02
0.54
3.55
2.96
0.02
10
20
37
20
50
Novalty
Novelly
Novell
Novelly
ProSpero (oun))
PEX
AdaLaadd
OFNAL--OS
LarProtRL
MLDE
Figure 3: Comparison of nitness-novelty trade-off's among leading methods. Each dot represents the outcome of single run. PROSPERO achieves both higher fitness and novelty than the baselines.

---

5.3 Out-of-distribution robustness
Setup For the simulation of distribution shifts, we used UBE21 variants and predicted their pTTM scores with ESMFold [19] as the oracle. The surrogate model was trained only sequences close to the wild-type, and optimization was then initiated from increasingly distant starting sequences Eaart- We considered three cases: (i) moderate covariate shift with staat differing by 35 mutations from the wild-type; (ii) a severe covariate shift with Tstart differing by 75 mutations (approximately half of the sequence length); (iii) low-data regime, where the 35-mutation case repeated with the surrogate trained only on small subset of the available data. Full details are provided in Appendix A.2.
Results Across all settings. PROSPERO consistently outperforms competing approaches. generating sequences with the highest pTM scores (Table 4), while simultancously exploring more novel regions of the sequence space (Appendix D.3). The performance advantage over the baselines was especially pronounced under the severe shift, highlighting the effectiveness of our approach in the most challenging conditions. Even in the low-data regime, our method maintained strong performance, thereby demonstrating robustness to both distribution shifts and data scarcity. Taken together, these results show that PROSPERO remains effective in the challenging OOD settings, commonly faced when exploring sequence space beyond wild-type neighborhoods.
5.4 Noise robustness study
Setup We evaluate the robustness of exploration strategies to surrogate model misspecification by introducing increasingly noisy surrogates on the AAV landscape. where ground-truth fitness is available. Specifically, we replaced the surrogate fe with an ensemble of noisy oracles fe. each defined by adding zero-mean Gaussian noise to the ground-trutt oracle and truncating negative outputs to zero, following the perturbation scheme of Sinai et al. [7]. The magnitude of the injected noise was determined by the Signal-to-Noise Ratio (SNR), with the noise scale given by noise
/ (Vo) 10-8NR/20, where Var(Do) denotes the variance fitness scores in the initial dataset. This setup introduces both stochastic noise and systematic shift, as the truncation flattens low-fitness regions and biases predictions upward, making a strong test of robustness to surrogate error.
Results As shown in Figure 4C, PROSPERO maintains an advantage over the majority of the base- lines even at low SNR levels, demonstrating strong robustness to surrogate noise. The performance gap between our method and competing approaches widens as the noise levels decrease, highlight- ing PROSPERO's 's ability to increasingly capitalize on informative signal. Notably, our method and AdaLead exhibit a sharp performance improvement earlier than other methods, suggesting greater robustness to surrogate misspecification and ability to guide exploration toward promising regions of the search space more effectively than competing approaches.
5.5 Ablation
Setup To assess the contribution of individual components in PROSPERO, we conduct ablation studies under the same noisy surrogate setting as in the robustness analysis (Section 5.4). We compare the full method to the following ablations: (i) without SMC, corresponding to sampling from EvoDiff
Table 4: Maximum and mean pTM scores of top 100 sequences generated by leading methods under distribution shifts. Reported values are the mean and standard deviation over runs. Bold: the best overall pTM score. Underline: second-best PROSPERO demonstrates the highest robustness to covariate shifts.
Moderate shift
Severe shift
Low-data shift
Method
Max
Mean
Max
Mean
Max
Mean
PEX
0.807 = 0.023
0.760 - 0.012
0.578 0.014
0.518
0.003
0.806 + 0.013
0.752 = 0.005
AdaLead
0.796 0.013
0.755
0.011
0.593 0.028
0,526 1. 0.007
0.781 1 0.016
0.742 0.004
GFN-AL-IIC
0.791 0.010
0.729
0.005
0.630 0.024
0.542 0.006
0.782 + 0.006
0.731 0.006
LatProtRL
0.787 1 0.013
0.743 1 0.003
0.560 1 0.000
0.508
1 0.003
0.792 1 0.013
0.743
0.001
MLDE
0.810 = 0.020
0.752 = 0.004
0.652 + 0.059
0.572 0.035
0.782 + 0.022
0.735 = 0.025
PROSPERO
0.822 0.027
0.777 = 0.020
0.672 + 0.031
0.599 0.014
0.808 + 0.017
0.763 = 0.017

---

LGK
Pab1
TEM
UBE21
(A)
Soreeeoe (1) aTM #LDOT (1)
(74.33)
(11.83)
(3.37)
(16.45)
Maximum fitness
(C)
8.0
70
0.65
7.8
0.05
7.4
0.50
25
50
75
100
70-
20
10-100
(D)
Signal to noise ralio
Mecinum tiness
Novely
6.0
1.0
0.60
2.5
0.50
0.0
0.50
25
50
75
100
ProSpero
ProSparo
Pasnt
PreSparo
Satart
ProSpern
Saaart
25
20
10
Validity
Signal to noise ratio
ProSpero (ours)
LapPorR
SMC
Targeted Masking
PEX
GFN-L-OCS
MLDE
mto Targeted Masking
wO SMC Targeoed Masking RM
Figure 4: (A) Trade-offs between validity, fitness and novelty across leading methods; each dot represents the outcome single run. (B) Structural quality of top 100 sequences generated by PROSPERO across runs compared to Istant average novelty of generated sequences is shown below each task. (C) Performance leading methods on the AAV landscape under varying levels of surrogate noise. (D) Ablation of PROSPERO components under the same setting in (C). In both (C) and (D), shaded regions represent the standard deviation across 5 runs. PROSPERO generates highly biologically plausible sequences and remains robust to surrogate misspecification.
with the resampling steps omitted; (ii) without targeted masking, where masked positions are selected random; (iii) without both SMC and targeted masking; and (iv) without SMC, without targeted masking, and without restricting SMC proposals to charge-compatible amino acids (without RAA).
Results Results of the ablation are depicted in Figure 4D. Notably, PROSPERO with all components intact outperforms all ablated versions, with performance degrading only under extremely low SNR conditions. The advantage of our method over the baselines low SNR levels, as seen the noise robustness study in Section 5.4, is most likely supported by PROSPERO's constraint that limits candidate generation to sequences containing residues of the same charge class as their wild-type counterparts. This steers the generation process toward sequences with wild-type fitness regardless of the surrogate quality, providing substantial performance gains. The sharp fitness improvement at higher SNR levels likely reflects the increasing effectiveness of the SMC guidance and the targeted masking as surrogate signal improves, while at very low SNR levels, guidance appears to slightly hinder the performance. Further ablations are provided in Appendix F.
6
Conclusion
this paper, we introduced PROSPERO, an active learning framework for iterative protein sequence design based on inference-time guidance of a pre-trained generative model. Our targeted masking strategy enables edits focused on fitness-relevant residues while preserving functionally critical sites. bollogicall--consrramne SMC sampling allows incorporating biological prior knowledge while traversing fitness landscapes, increasing the likelihood of retrieving high-fitness sequences even under surrogate misspecification. By combining these innovations, PROSPERO enables robust exploration beyond wild-type neighborhoods while maintaining biological plausibility, achieving performance that matches exceeds state-of-the-ar approaches across diverse protein engineering tasks.
Acknowledgments and Disclosure of Funding
We thank Rasmus Moller-Larsent James Odgers, and Adam Izdebski for their valuable feedback and suggestions that helped us improve the manuscript. This project has received funding from the European Research Council (ERC) under the European Funding Union's 's Horizon 2020 research and innovation programme (grant agreement No 810115- DOG-AMP). VF was supported by the Branco Weiss Fellowship.
10

---

References
[1] Kevin K. Yang, Zachary Wu, and Frances H. Arnold. Machine-lcarming-guiie directed evolu- tion for protein engineering, Nature Methods, 1678),687-644 Aug 2019. ISSN 1548-7105. doi: 10.1038/s4159--019-00099
URL https: /ttp:://ddo.oo ottp:::/doooorr/1....o http://....../...0./////0/000000
[2] A Daniel Lagassé, Aikaterini Alexaki, Vijaya L Simhadri, Nobuko Katagiri, Wojciech Jankowski, Zuben E Sauna, and Chava Kimchi-Sarfaty, Recent advances in (therapeutic protein) drug development. F1000Res., 6: February 2017.
[3] Lynne Regan, Diego Caballero. Michael R Hinrichsen, Alejandro Virueta, Danielle M Williams, and Corey S JO'Hern, Protein design: Past, present, and future. Biopolymers, 104(4):334-350, July 2015.
[4] S. Wright. The roles of mutation, inbreeding, crossbreeding and selection in evolution. Pro- ceedings of the XI International Congress of Genetics, 8:209-222, 1932.
[5] J. Arjan G.M. de Visser and Joachim Krug. Empirical fitness landscapes and the predictability of evolution. Nature Reviews Genetics, 15(7):480-45) Jul 2014. ISSN 1471-0064. doi: 10. (0.1038/nr33744, URL https::///oiiii org/ 10. 1038/4r33744
[6] Zhizhou Ren, Jiahan Li, Fan Ding, Yuan Zhou, Jianzhu Ma, and Jian Peng. Proximal exploration for model- guidel protein sequence design. Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 18520-188 1836.. PMLR, 17-23 Jul 2022. URL https: pptcc//ddocccecsss///oo mlr, press// v162/ren22a. htm..
[7] Sam Sinai, Richard Wang, Alexander Whatley, Stewart Slocum, Elina Locane, and Eric Kelsic. Adalead: simple and robust adaptive greedy search algorithm for sequence design. arXiv preprint, 2020.
[8] Christof Angermueller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Colwell. Model-based reinforcement learning for biological sequence design. In International Conference on Learning Representations, 2020. URL https: htttps://ppenee //ppr//v//e.r....../e net/forum? baneerummmddd-llll BkgggBKKr..
[9] Moksh Jain, Emmanuel Bengio, Alex Hernandez-Garcia. Jarrid Rector-Brooks Bonaventure P. Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, Lena Simine, Payel Das, and Yoshua Bengio. Biological sequence design with GFlowNets. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedinggs of Machine Learning Research, pages 9786-9801. PMLR, 17-23 Jul 2022. URL 1621/122652 /2//////11 jain2aa..... 2a.222..tt.. html.
https:/ ptttps://proceedingg mlr pwwps/eeiivvvee vviess/vv6/ http:/nww..........eeeeeer
[10] Minji Lee, Luiz Felipe Vecchietti. Hyunkyu Jung, Hyun Joo Ro, Meeyoung Cha, and Ho Min Kim. Robust optimization protein fitness landscapes using reinforcement learning in latent space. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 4/st International Confer- ence Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 26976-26990, PMLR, 21 21- Jul 2024. URL https: / http///pooeeedn...ccce///// mlr. presa/v235/ 1ee24x. html.
[11] Shikha Surana, Nathan Grinsztajn, Timothy Atkinson, Paul Duckworth, and Thomas D Barrett, Overconfident oracles: Limitations of in silico sequence design benchmarking. In ICML 2024 Al for Science Workshop, 2024. URL https: http:///ppeeeeiieeeee wwwpreewwww.ee/fooeooooo pett/forummddd=fPCnnna
[12] Hyeonah Kim, Minsu Kim, Taeyoung Yun, Sanghyeok Choi, Emmanuel Bengio, Alex Hernández-GGrcfa and Jinkyoo Park. Improved off-policy reinforcement learning in bio- logical sequence design. In Forty-second International Conference on Machine Learning, 2025.
URL ftttp:://openrevicee ootp://www.eeeetooorrr//-----SS

---

[13] Thanh V. T. Tran and Truong Son Hy. Protein design by directed evolution guided by large language models. IEEE Transactions on Evolutionary Computation, 29(2):418 428, 2025. doi: 10.1 00.1109/TEWC.224444499999
[14] David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 773-782. PMLR, 09-15 Jun 2019. https::::: htt.tttps://proceete /ttp///errcee..s.....o///.////sp
mlr. S..r.pressvv9//// http///boookes1ee html.
[15] Ming Qin, Keyan Ding, Bin Wu, Haihong Yang, Zeyuan Wang, Hongbin Ye, Haoran Yu, Huajun Chen, and Qiang Zhang. Active Finetuning Protein Language Model: A Budget-Friendl Method for Directed Evolution. Frontiers in Artificial Intelligence and Applications. IOS Press, 09 2023. ISBN 9781 171466384469 doi: 10.3233/FAIA230481
[16] Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evo- lution strategies. Evolutionary Computation, (92):559-1999 06 2001. ISSN 1063-6560. doi: URL https://doli 0tt//100.122/1133350000000000000
10.1162100335561550119000
[17] Frances H. Amold. Design by directed evolution. Accounts of Chemical Research, 31)):: 125-131, 1998. doi: 10. 10.1022//r900017. URI. https////// org/10. 1//11/0..111///111 102100222//0000/ ar960017f.
[18] Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal. Zeming Lin, Jason Liu, Demi Guo, Myle Otr, C. L.Larennce Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS, 2019. doi: 10. 1 1011/0662200. URL https: /ttpps:/wwwr /www. biorxiv. org/ http//onneenttt.eee 11tpat////..111/2288333333
[19] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Allan dos Santos Costa, Maryam Fazel-Zarandi: Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv, 2022.
[20] Yuchi Qiu, Jian Hu, and Guo- Wei Wei. Cluster learning- directed evolution. Na- ture Computational Science, 1(12):80-818, Dec 2021. ISSN 2662-8457. doi: 10.1038/ s43588-00 000--002100000--- URL https: /ttttp::/ddoi.rrro 10. 1038 /0000//4300
[21] Yuchi Qiu and Guo-We Wei. CLADE 2.0: Evolution-driven cluster learning- directed evolution. J. Chem. Inf. Model., 62(19):4629-4641 October 2022.
[22] Yuhao Wang, Qiang Zhang, Ming Qin, Xiang Zhuang. Xiaotong Li, Zhichen Gong, Zeyuan Wang, Yu Zhao, Jianhua Yao, Keyan Ding, and Huajun Chen. Knowledge-aware reinforced language models for protein directed evolution. In Forty-first International Conference on Machine Learning, 2024. URL httpps://pppnrriiieeee www. fest/foormmiiee wtt.rforrmmmimdmiieeee kamimiiiit..............
[23] Dinghuai Zhang, Jie Fu, Yoshua Bengio, and Aaron Courville. Unifying tikeihhood-fre inference with black-box optimization and beyond, 2022. URL ttctps://arxivv org/ /bs/ 03372..
[24] Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating bayesian optimization for biological sequence design with denoising autoencoders. arXiv preprint arXiv:2203. 2022.
[25] Nate Gruver, Samuel Stanton, Nathan C. Frey, Tim G. Rudner, Isidro Hotzel, Julien Lafrance- Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffusion, 2023. URL https: //axxiv./// wwreeiivoorggaaae hal//bba/2220 220.. 20009.
[26] Alan Nawzad Amin, Nate Gruver, Yilun Kuang, Yucen Lily Li, Hunter Elliott, Calvin McCarter, Aniruddh Raghu, Peyton Greenside, and Andrew Gordon Wilson. Bayesian optimization of antibodies informed by a generative model of evolving sequences. In The Thirleenth International Conference on Learning Representutions, 2025. URI https::/ h/tp::s///peervveee.. met//ooummiccm ieerummidiEean wat.:middd-48-Q1P1111.
net/
12

---

[27] Nathan C. Frey, Dan Berenberg, Karina Zadorozhny, Joseph Kleinhenz, Julien Lafrance- Vanasse, Isidro Hotzel, Yan Wu, Stephen Ra, Richard Bonneau, Kyunghyun Cho, Andreas Loukas, Vladimir Gligorijevic, and Saeed Saremi. Protein discovery with discrete walk-jump sampling. The Twelfth International Conference on Learning Representations, 2024. URL. https://openrevies net. ht////ormmmmd Derrm??dd=MMHHKOOONN..
[28) Andrew Kirjner, Jason Yim, Raman Samusevich, Shahar Bracha, Tommi S. Jaakkola, Regina Barzilay, and lla R Fiete. Improving protein optimization with smoothed fitness landscapes. In The Twelfth International Conference on Learning Representations, 2024. URL https: / //openreview. net/ teet/foruuee forum/id=rx1F22288
[29] Zhenqiao Song and Lei Li. Importance weighted expecaaioon-maxiiization for protein sequence design, 2024. URL https: / /twxxv.xivvv//////// orgoooog/aa h/s//bb//3300.... http//225500006...
[30] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2014.
[31] Saba Ghaffari, Ehsan Salch, Alex Schwing, Yu-Xiong Wang, Martin D. Burke, and Saurabh Sinha. Robust model-based optimization for challenging fitness landscapes. In The Twelfth International Conference on Learning Representations, 2024. URL https: //pprreveee. pet/foorum?dd=zENOoK...
[32] Evan R. Antoniuk, Peggy Li, Nathan Keilbart, Stephen Weitzner, Bhavya Kailkhura, and Anna M. Hiszpanski. Active learning enables extrapolation in molecular generative models, 2025. URL https: //arxiv.a.ivv//// http://axxxv...oaaa////////
02059.
[33] B C Cunningham and J A Wells. High-resolution epitope mapping of hGH-rrecppee interactions by alanine-scanning mutagenesis. Science, 108444908)108111000 June 1989.
[34] Kim L Morrison and Gregory A Weiss. Combinatorial alanine-scamning. Current Opinion in Chemical Biology, 5(3):302-307, 2001. ISSN 1367-5931. doi: http://dooi.rg/1O..o 1016/ S 1136075593100000000644,, URL https: /ttp://ssceeneddrreeeeeeeeee www.:eccieeeedrrre...cooocesiaie commsciience/articc.e...
pia//$1867593100002644.
[35] Tanja Kortemme, David E Kim, and David Baker. Computational alanine scanning of protein- protein interfaces. Sei. STKE, 2004(219):12, February 2004.
[36] Dennis M Krüger and Holger Gohlke. DrugScorePPI webserver: fast and accurate in silico alanine scanning for scoring protein-protein interactions. Nucleic Acids Res., 38(Web Server issue):W480-66 July 2010.
[37] Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: no regret and experimental design. Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML' 10, page 1015-1022, Madison, WI, USA, 2010. Omnipress. ISBN 978 978160558
[38] Sarah Alamdari, Nitya Thakkar, Rianne van den Berg, Alex Lu, Nicolo Fusi, Ava P. Amini, and Kevin K. Yang. Protein generation with evolutionary diffusion: sequence is all you need. bioRxiv, 2023. doi: 10.1101/2023.00 /ttp//2223.091111 19..1..5555 1.5566673. URL https: //wp::///www.. biorxiv org/ 09. 11. 556673.
[39] J Wang and W Wang. A computational approach to simplifying the protein folding alphabet. Nat. Struct. Biol., ((1 10:13-3-1 November 1999.
[40] Riddle, J V Santiago, S T Bray-Hall, N Doshi, Grantcharova, Q Yi, and D Baker. Functional rapidly folding proteins from simplified amino acid sequences. Nat. Struct. Biol., 4 (10)8805-89, October 1997.
[41] Sandeep Kumar and Ruth Nussinov. Close-range electrostatic interactions in proteins. Chem- biochem, 3(7),604667, July 2002.
[42] R Rao, N Bhattacharya, N Thomas, Y Duan, X Chen, J Canny, P Abbeel, and Y S Song. Evaluating protein transfer learning with TAPE, Ady Neural Inf Process Syst, 32:9689-9900) 9701, 2019.
13

---

[43] Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer. and Alexander Rives. Learning inverse folding from millions of predicted structures. ICML, 2022. doi: 0ttp//22222....1..47777 carly/2022/04/11///
URL https://wwer www..iiio.ii.........ii www.ooorenc.......ooooooooo/...e 10. 487777.
[44] Emily E. Wrenbeck, Laura R. Azouz, and Timothy A. Whitehead. Single-mutation fitness landscapes for an enzyme on multiple substrates reveal specificity is globally encoded. Nature Commumicationss 8(1):15695, Jun 2017. ISSN 2041-1723. doi: 10. 10.1038/ncomms) 15695. URL https://doi. org/ /1/.//0...// 1038//0cmmmet 15695.
[45] Elad Firnberg, Jason W. Labonte, Jeffrey J. Gray, and Marc Ostermeier. A comprehensive. high-resolution map of gene's fitness landscape. Molecular Biology and Evolution, 31 (6):1581-1592, 02 2014. ISSN 0737-4038. doi: 10.1093/mllbe//ms008.. URL https: //doi.org/1il 1093 1ee110099mmmme httm////mmmmss/////////////
[46] Lea M Starita, Jonathan N Pruneda, Russell S Lo, Douglas M Fowler, Helen J Kim, Joseph B Hiatt, Jay Shendure, Peter Brzovic, Stanley Fields, and Rachel E Klevit. Activity-enhancie mutations in e3 ubiquitin ligase identified by high-throughput mutagenesis. Proceedings of the National Academy of Sciences of the United States of America, 110(14))B1263-72 April doi: 10.1073/pnas. 11tP////n..10330111111 13033091 URL https: //opppppp/rrppppmcccc org/
2013. ISSN 0027 -4244.
articles/PMC3611
[47] Daniel Melamed, David L Young, Caitlin E Gamble, Christina R Miller, and Stanley Fields. Deep mutational scanning of RRM domain of the saccharomyces cerevisiae poly(a)-binding protein. RNA, 19(11):1537-1511 November 2013.
[48] Pierce Ogden, Eric D Kelsic, Sam Sinai, and George M Church. Comprehensive AAV capsid fitness landscape reveals viral gene and enables machine-guided design. Science, 366(6469): 1439, November 2019.
[49] Karen S Sarkisyan, Dmitry A Bolotin, Margarita V Meer, Dinara R Usmanova, Alexander Mishin, George V Sharonov, Dmitry N Ivankov, Nina G Bozhanova, Mikhail S Baranov, Onuralp Soylemez, Natalya Bogatyreva, Peter Vlasov, Evgeny S Egorov, Maria Logacheva, Alexey S Kondrashov, Dmitry M Chudakov, Ekaterina Putintseva, Ilgar Z Mamedov, Dan Tawfik, Konstantin Lukyanov, and Fyodor A Kondrashov. Local fitness landscape of the green fluorescent protein. Nature, 533(7603) May 2016.
[50] Jochen Weile, Song Sun, Atina G Cote, Jennifer Knapp, Marta Verby, Joseph C Mellor, Yingzhou Wu, Carles Pons, Cassandra Wong, Natascha van Lieshout, Fan Yang. Murat Tasan, Guihong Tan, Shan Yang, Douglas M Fowler, Robert Nussbaum, Jesse D Bloom, Mare Vidal, David Hill, Patrick Aloy, and Frederick P Roth. A framework for exhaustively mapping functional missense variants. Molecular Systems Biology, 13(12)9557 2017. doi: hups://doi.org/ 10. 20.155522/sb.221777 201177990 URL https: /ttp://wwwww//.. emwopressbpeeee.eoooo org wttpg//oo/////...///o/o/ 15252//m2mm 20177908.
[51] Justin R Klesmith, John-Paul Bacik, Ryszard Michalezyk, and Timothy A Whitehead. Com- prehensive sequence-flux mapping of a levoglucosa utilization pathway in e. coli. ACS Synth. Biol., ((11):12355- 1243, November 2015.
[52] Farhan Damani, David H Brookes, Theodore Sternlieb, Cameron Webster, Stephen Malina, Rishi Jajoo, Kathy Lin, and Sam Sinai. Beyond the training set: an intuitive method for detecting distribution shift in model-based optimization, 2023. URL https: //appii:////xv.../// org/ absp///ab///// /2311. 05363.
[53] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization, 2017. URL https://arxiv, org/ abaaab///4/ /tt//111...000 6980.
[54] Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, and Koray Kavukcuoglu. Neural machine translation linear time, 2017. URL https:://////ttt //axxv.. org/abs/1610.100 10099.
14

---

[55] N.J. Gordon, D.J. Salmond, and A.F.M. Smith. Novel approach to nonlinear/hon-gause bayesian state estimation. IEE Proceedings (Radar and Signal Processing). 140:107-113, 1993. doi: 10. 1tt.9/99---2..999.0000 abs/10. 10009///p-fft f---f-22 2.1993.0015
wtt.htttp://ddiiaaattt
[56] Christian A. Naesseth, Fredrik Lindsten, and Thomas B. Schön. Elements of sequential monte
carlo. Foundations and Trends@ in Machine Learning. 12(3):307-322, 2019.
[57] Arnaud Doucet, Nando Freitas, Kevin Murphy, and Stuart Russell, Sequential monte carlo methods in practice. 01 2013. doi: 10.
[58] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL id Ad=AAWuCvz..
www.pperrrriie.eeeeoooorr... net/forum?
[59] Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, and Jennifer Listgarten. Unlocking guidance for discrete state-space diffusion and flow models. In The Thirteenth International Conference on Learning Representations, 2025. URL https:// //op://////eeeee// wet.i/ewwwee///ot id-XsgH154y07.
[60] Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Informa- fion Processing Systems, 2022. URI.https. httttps://operreviieee htappperreeeee.eee/frrrrnrrnaeei nett/forrunidd-nnTTT6GYY
[61] Cheuk Kit Lee, Paul Jeha, Jes Frellsen, Pietro Lio, Michael Samuel Albergo, and Francisco Vargas. Debiasing guidance for discrete diffusion with sequential monte carlo, 2025. URL attps://arxa org/ abs//502.......//////// 06079.
[62] Luhuan Wu, Brian L. Trippe, Christian A Nacsseth, John Patrick Cunningham, and David Blei. Practical and asymptotically exact conditional sampling in diffusion models. In shirty- Conference on Neural Information Processing Systems, 2023. URL attpps//oppereviies www.rrummm=eeegrrrrrr.. 1zcRv.
[63] Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola. Diffusion probabilistic modeling of protein backbones 3d for the motif- scaffolding problem. In The Eleventh International Conference on Learning Representations, 2023. URL https: http///pppeeeeeeeeeeeeeee/eeeeee nwt/forummidde11--a2... dd-..rmmmd..TTTTxxMM......
[64] Filip Ekström Kelvinius and Fredrik Lindsten. Discriminator guidance for autoregressive diffusion models. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li, editors, Proceedings The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pages 3403-3411 PMI.R, 02-04 May 2024. URL /ttps://ppoceeedinne www.mm............mmmmmmmm
mlr press/ v230ss//v33//eee wkstt//e kel viiius2244 html.
[65] Dongjun Kim, Ycongmin Kim, Se Jung Kwon, Wanmo Kang, and Il-Chul Moon. Refining gen- erative process with discriminator guidance in score-based diffusion models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 16567-16598. mlr ppwps//202/ veesss/v202/xim231 /im//iim23.1.... html.
PMLR, 2023. URL
[66] Emiel Hoogeboom, Alexey A. Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models. In International Conference on Learning Representations, 2022. URL
[67] Xiner Li, Yulai Zhao, Chenyu Wang, Gabriele Scalia, Gokeen Eraslan, Surag Nair, Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, and Masatoshi Uehara. Derivative-free guidance continuous and discrete diffusion models with soft value-based decoding, 2024.
URL https: //arxi/va.ii/////aaeeaa org/ altg///ss2400.... 08222 08252.
15

---

[68] Masatoshi Uchara, Xingyu Su, Yulai Zhao, Xiner Li, Aviv Regev, Shuiwang Ji, Sergey Levine, and Tommaso Biancalani. neward-guides iterative refinement diffusion models at test-time with applications to protein and DNA design. In Forty-second International Conference on Machine Learning, 2025. URL https:/:::/ http://:////peeeiieeeee/e net http://oorrmm.daapppSSSSSSS
[69] Christophe Andrieu and Gareth O. Roberts. The pseudo-marginas approach for efficient monte carlo computations. The Annals of Statistics, 37(2), April 2009. ISSN 0090-5364. doi:
10. 10.1214/07-ao5574 URL http: / /dx. doi org/10 10. 1214/07-A085777
16

---

Appendix
Experimental details
18
A.1 Protein design tasks
18
A.2 Out-of-distribution robustness
18
A.3 Surrogate training
19
A.4 Baselines implementation.
19
A.5 Evaluation metrics
20
A.6 Evodiff
20
Discussion
B.1 Limitations
21
B.2 Future work
B.3 Broader impact
C Background
21
Full results
22
D.1 Protein design
22
D.2 Early-round protein design
23
D.3 Out-of-distribution robustness
24
Extended related work
24
Further ablations
25
G Algorithms
26
17

---

A Experimental details
A.1 Protein design tasks
We evaluated PROSPERO across eight diverse protein fitness landscapes. Among these, AAV and GFP were created by Kim et al. [12] (Apache-2.0 license), while the remaining datasets were collected by Tran and Hy [13] (GPL-3.0 license). For AAV and GFP, used oracles available in FLEXS [7] (Apache-2.0 2.0 license), while for the remaining datasets oracles provided by Ren et al. [6] (Apache-2.0 license).
(i) Aliphatic Amide Hydrolase (AMIE). The objective is to optimize amidase sequences for high enzymatic activity [44]. The initial dataset Dn includes 6417 sequences of length
T. = 341, making the search space span across 20 possible variants. The fitness of the starting sequence [((aaaa) = 0.224, while the average distance between the starting sequence and Do is Hovelty(Do-. ssamn 2.
iii TEM-1 3-Lactamase (TEM). The goal is identify TEM-1 B-lactamase variants with improved thermodynamic stability [45]. Do consists of 5199 sequences with L 286, f (Tata)) 1.229, and Novelty -stan) = 2.
(iii) Ubiquitination Factor Ube4b (E4B). The goal is to enhance the activity of the E4B ubiquitination enzyme [46]. The dataset consists of 91, 91,032 sequences with 102, from which we randomly select 10,000 for the initial dataset Do- The fitness of the starting sequence (Timm) = 7.743, with Novelty (Doeltyyoo, (Do, Fsstn) 5.42.
(iv) Poly(A)-binding) Protein (Pab1). The aim is to improve the binding fitness of Pabl variants in the RNA recognition motif region [47]. The dataset contains 36,389 mutants of length L = 75. Similarly to E4B, we restrict Do to 10,000 randomly selected sequences. The fitness of the starting sequence ( 0.843, with Novelty (Do, Tstaut 3.95.
(v) Adeno-associated Viruses (AAV). The objective to discover VP1 protein sequence fragments(positions 450-540) with improved gene therapy efficiency [48]. The dataset Do. of size 15,307, was created by Kim al. [12] through random mutations of the wild-type and scoring with the oracle. Here, L 90, (x = 0.500, and Novelty (Do Fssmrt = 5.05.
(vi) Green Fluorescent Proteins (GFP)..h goal is to identify protein sequences with high log rggfluorescence intensity [49]. Do includes 10200 sequences mutated by Kim al. [12], with AAV. In this case, L 238, ( 3.572, and Novelty (Da Tatar) 42.87.
(vii) SUMO E2 conjugase (UBE2I). The aim here is to optimize variants of SUMO E2 conjugase for functional mapping applications [50]. Dy consists 3022 sequences with L = 159, f (Jraaa) 2.978, and Novelty Taaam)
(viii) Levoglucosan Kinase (L.GK). The objective is to optimize levoglucosan kinase variants for improved enzymatic activity [51]. Do contains 7633 sequences with T, = 439, f'( = 0. 0.020, and Novelty (Do, xsstat) 2.
A.2 Out-of-distribution robustness
In this experiment, we used candidate sequences generated by different exploration methods during UBE2I optimization task in Section 5.1. We employed ESMFold [19] as the oracle to predict pTM scores, as protein structure prediction models provide highly reliable feedback. better suited for assessing performance in challenging OOD settings [52]. From all UBE2I candidates, we selected those within Hamming distance < 5 from the wild-type to construct the initial dataset Do, consisting of 2624 sequence-p' -TT'M pairs, and with the average pTTM score of 0.860 0.029. The surrogate model fo trained on Do, and optimization was performed for 4 iterations starting from sequences increasingly distant from the wild-type. We considered three cases:
(i) Moderate covariate shift. Optimization was initiated from a starting sequence I'santt differing by 35 mutations from the wild-type, and with an initial pTM of approximately 0.70.
(ii) Severe covariate shift. Optimization was initiated from a starting sequence Taan differing by 75 mutations from the wild-type, and with an initial pTM of approximately 0.50.
18

---

(iii) Low-data regime shift. The moderate shift experiment was repeated with the surrogate trained on only 200 randomly selected points from Da, resulting in a reduced training set with an average p'''M score of 0.860 + 0.023.
Across all three cases, the exploration algorithms were run with their configurations corresponding to shorter sequences, as detailed Section 5 for PROSPERO and in Appendix A.4 for the baselines.
A.3 Surrogate training
Following Kim et al. [12], we trained surrogate models using the Adam optimizer [53], with both the learning rate and L2 penalty set to 0.0001, and batch size of 256. The maximum number of proxy updates was set to 3000, but we employed early stopping with 10% of the dataset reserved for validation, terminating training if the validation loss failed improve for 10 consecutive iterations.
A.4 Baselines implementation
For the following baselines, we employed the open-source implementations provided by the FLEXS benchmark [7], available at https: //gi gitt.u...... cow/samssnaaa.cccccm wttaaaai/EEEEEEEEEeee 2pache-2.0 license.
/tt.s////////////e/ee////aaaaa meatee under the
(i) AdaLead [7]: We followed the default hyperparameter settings provided by the authors. Specifically, we used recombination rate of 0.2, a mutation rate of 1/L, where the sequence length, and threshold 0.05.
(ii) DyNaPPO [8]: We altered the implementation of DyNaPPO from FLEXS following the approach of Kim et al. [12]. Specifically, replaced originally proposed proxy architectures with the same one-dimensional CNN ensembles used for other methods.
(iii) CbAS [14]: We implemented CbAS using VAE [30] the generator, retraining it each
cycle using top 20% of sequences weighted by the density ratio between the ground-trute truth- conditioned distribution and current sampling distribution.
(iv) BO [7]] We select starting sequences via Thompson sampling and use UCB to select local mutations based surrogate model's predicted mean and uncertainty.
(v) CMA-ES [16]: Following Sinai et al. [7], we convert the continuous outputs from CMA-ES one-hot representations by taking the argmax each sequence position.
We adapted the implementation of MLDE [13] from https://githut: cwmm Hom//ySnLLbb// Directed Evoouution under the GPL-3.0 license. We run 10 surrogate-based optimization steps with population size of 128 and beam size of 4. The aandon-to-importancce masking ratio was set to 0.6:0.4, and we used ESM-2 [19] with 35 million parameters for unmasking.
For comparisons with LatProtRL [10], used the code available https://githee com/ haewonc/ Beewonc/LatPrr LLttPoRRL under the MIT license. We employed a pre-trained ESM-2 [19] for both the encoder and decoder components of VED. For tasks involving shorter sequences (AAV, E4B, Pabl) we set: (i) VED latent dimension R = 16; (ii) action perturbation magnitude 5 = 0.1; (iii) episode length Tap 4; (iv) constrained decoding term Meecode = 8. For tasks involving longer sequences (GFP, AMIE, TEM, UBE21, LGK), we used: (i) R = 32; (ii) = 0.3: (iii) Tep 6; (iv) Midcode 18.
In our GFlowNet setup, we employed the implementation available at https : //github com/ eyeeonakiimmdeel htt//deltaacccc aec under the Apache-2.0 license [12]. For the conservative strategy GEN- AL-OCS proposed by Kim et al. [12], we used an adaptive with maximum masking radius set to 0.05 and rank-based proxy training with reweighting factor A 0.01. For AAV, E4B and Pabl we set the scaling factor X= = 0.1. whereas for GFP, AMIE, TEM, UBE2I and LGK X = 1. For comparisons with GFN-AL [9] modified the above configuration by removing based proxy training and using a fixed masking radius of 1.
We implemented PEX [6] using the code in https: ttt.. wwc.hh....o.aaeee MeeeliiooProttii// proximal-expl expporation wwwaatio/ttteeenaaeeeeeeee /ttte/eeeeemmmeeemeeeeeee under the Apache-2.0 license, with the default setting of 2 random mutations and a frontier neighbor size 5.
19

---

A.5 Evaluation metrics
Protein fitness optimization metrics Let Doest = =(()),,(a))
) fi=1 denote the the 100
highest-rankinn sequence-fitme fitness pairs generated across N active learning rounds. The evaluation of exploration algorithms in our experiments was based on the following metries:
(i) Maximum fitness. The primary evaluation criterion, representing the ability of an explo- ration algorithm to recover highly functional protein sequences:
MaxFiinesss (Doss) = max f(x).
(5)
iii Mean fitness. The mean fitness values of the top 100 candidate sequences:
MeanFitnnsss (Does))
f(x).
D
Dreat
(ii) Novelty, The average Hamming distance between top 100 candidates and a starting sequence, characterizing the extent of divergence from the wild-type protein:
Novelty (D Zsstart
Σ
d(x, sstart).
Dhesst
Dana
(iv) Diversity. Defined as the mean pairwise Hamming distance between the top 100 candidate sequences, reflecting the exploration algorithm's ability to explore diverse regions of the fitness landscape:
Diversity =
Σ
d(x,
(8)
DDest (|Dresl
1)
Drea inf
Biological plausibility measures To evaluate biological plausibility of candidate sequences gener- ated by various exploration algorithms we used the following measures:
(i) Validity. Defined following Surana et al. [11] as diagnostic measure to assess whether high fitness scores correspond biologically plausible sequences. Specifically, it checks whether physicochemical properties of the top 100 candidates all fall within the 0.5th to 99.5th percentile range of the reference property distribution. This provides high-confidence indication of whether elevated fitness scores reflect genuine biological plausibility or rather result from surrogate and oracle misspecification. The considered properties were:
molecular weight
aromaticity
isoelectric point
grand average of hydropathy (GRAVY)
instability index
(ii) PLDDT and PTM. Both pLDDT and pTM structure confidence scores predicted by ESMFold, scaled between 0 and 100 [19]. pLDDT measures the local per-residue confidence in structural accuracy, while pTM reflects the predicted global topology confidence. both cases, values greater than 70 are indicative of high-confidence predictions.
(iii) self-consistency Perplexity (scPerplexity). scPerplexity [38] quantifies how well a gener- ated sequence be recovered from its predicted structure. Specifically, it is defined as the negative log-likelihood of the original sequence conditioned on the structure predicted folding model. Lower scPerplexity values indicate that the sequence is more plausible under the inverse folding model given its predicted structure. For sequence folding we used ESMFold [19]; for inverse folding used ESM-IFF [43]
A.6 Evodiff
To model the prior over protein sequences in PROSPERO we used EvoDiff-OADM with million parameters, introduced by Alamdari et al. [38] and available under the MIT license.
20

---

B Discussion
B.1
Limitations
PROSPERO utilizes EvoDiff as its backbone, a model built upon a ByteNet-ttyle CNN architecture [54]. Without enforcing reproducibility the approach remains computationally efficient and exhibits reasonable runtime. However, ensuring deterministic runs with CNNs typically leads to substantially longer runtimes, representing practical limitation. Specifically, we conducted all experiments NVIDIA Tesla V100 32GB GPU, with the total runtime across all tasks being approximately 3 hours under -etermiimiiii setting and around 30 hours when enforcing reproducibility. For the noise robustness study in Section 5.4, the total runtime across all signal torial-tt-ooise ratio levels took approxi- mately 30 minutes under non-deterministic configuration and around 12 hours in the reproducible setting. We note, however, that (i) reproducibility is not strict requirement for practitioners, and (ii) even under deterministic configuration, the computational cost remains negligible compared the burden of wet-lab experiments which PROSPERO is designed to help alleviate.
PROSPERO performs well in maintaining diversity of generated sequences, as shown Section 5.1. However, approximate inference using Sequential Monte Carlo carries an inherent risk of reduced diversity, which could potentially arise depending on design choices and remains possible limitation of the approach.
B.2 Future work
An interesting direction for future work is the development of adaptive strategies for reducing the amino acid alphabet. While our charge-based grouping offers broad applicability, more tailored schemes could further enhance performance specific proteins. Another promising extension would be to apply PROSPERO in lab-in-the-loop setting, using experimental validation as the oracle and integrating structure-based alanine scanning into targeted masking to more effectively identify critical residues.
B.3 Broader impact
PROSPERO can advance protein engineering for therapeutics, enzymes, and sustainable materials by improving data efficiency and reducing experimental burden associated with wet-lab screening. However, with any enpeerl-purpose protein design tool, there is risk of misuse for designing harmful proteins or contributing to biosecurity concerns.
C Background
Sequential Monte Carlo (SMC). Sequential Monte Carlo (SMC) is class of approximate infer- ence methods for sampling from complex, high-dimensional target distributions 7(117), where i.r denotes sequence of variables or partial states [55-57]. Rather than directly sampling from ((x.r). SMC simplifies the inference by constructing a sequence of unnormalized intermediate target distribu- tions (z1.))) that progressively approximate the target. At each step a collection of weighted samples (i.c., particles) is propagated by sampling from proposal distributions {qr (x2 x1=-1)) and corrected using importance weights (wr(xi:)) to account for discrepancies between the proposal and the intermediate target. At the initial step t = 1, N particles are sampled independently from the proposal distribution qi (x1), and initial importance weights wn"") are assigned for each particle n:
wf")) (1(
(n) (xr)) 2
@1 (z) (a))
Subsequently, each step = 2.... . consists of the following three operations [56]]
ii) Optional resampling:
x [n] 1a-1 ~ Cat
(((m)) } N ( n=l
(n)
N
(10)
(m)
1=
w
21

---

ii Proposing:
st ~ gg(at =
(I1)
(iii Weighting:
7v (- (xc)
W/
(m)
(2 (2)
(m) (m) 1:1-1)0 111--1
(12)
N
D Full results
D.1
Protein design
Table 5: Mean fitness of top 100 sequences generated by each method. Reported values are the mean and standard deviation over runs.
Method
AMIE
TEM
UBE21
LGK
CMA-ES
-8.317 1 0.029
0.013 0.000
-1,009
0.029
0.232
0:012
0.000
0.0000
1.503
00000
-0.072
0.004
-1,538
0.008
DymPPO
-6.493 + 0.155
0.077
+ 0.002
0.574
- 0.148
0.481 + 0.013
0.000
+0.000
2.064 -D0K8
1.600 + D.101
-1.120
0.045
BO
-0,829 + 0,474
0.606 + 0.352
5,009
0.785
0.510 + 0:047
0.618
0.010
3,538
0.036
2.695 + D0.148
-0.017
0.020
PEX
0.238 0.004
1.227
0.002
7.948
0.045
1.307
0.258
0.620
0.017
3,597
0.003
2.987
0.001
0.033
0.001
Adalead
0.220
0.002
1,836
0.266
ChAS
-8.361
0.001
-0,20
0.068
0.162 0:082
0.000
1,666
0.003
-1.659 0.023
GEN-AL
-8.268 + 0.010
0.015
0.001
.0,415 # 0.091
0.276 0.036
0.000
00000
1.776 0.009
0.172 + 0.396
-1,345 0.037
GEN-AL-SCS
-0.244 1 0.137
0.192
:: 0.02?
7.653 = 031
1,070 1: 0.113
0.648
2 0.020
3,569 : 0.009
2.968 1: 0.006
0.024 = 0.004
LaProiRL
0.217
0.001
1.222 1 0.000
7.562 0.06
0,888 0.072
0.563
0.009
3,582 0.003
2,975
0.001
0,019 0,000
MLDE
0,231
0.021
7,843
0,877 0.024
0.000 0.555
3,591
2,955
0.005
0.036
PROSPIXO
0.236
1,176 0.029
0.054
1,401 0.202
0.679
Table 6: Median fitness of top 100 sequences generated by each method. Reported values the mean and standard deviation over runs.
Method
AMIE
TEM
E4B
Pbbl
AAV
GFP
UBE21
LGK
CMA-ES
-8,392 1 0.006
0.011
1 0.000
-1,046
0.027
0.206
0.013
0.000
1.0000
1.578
0.006
-0.079 1. 0,002
-1.563 0.008
DyNaPPO
-6.731
0.125
0.025
0.000
0
0.465
0.012
1.760
-1.198
0,032
BO
0.591
5.779
0,489
0.044
0613+ +0.009
3.541
D.169
-0.009 0.028
PEX
0,237
0.001
1,288
0.000
7.937
0.050
1.298
0.257
0.16
1.0117
3.587 0.003
2.986
0.001
0.033
0.001
AdaLoad
0,228
0.001
1.198 :: 0.001
7,32 = 0.042
1,830 1: 0.271
0.61 0:033
3.561 : 0.007
2.976 1: 0.003
0.037 - 0.001
ChAS
-8.371 + 0.023
0.009 + 0.001
-1,841
0.065
0.152 + 0.084
0.000 + +.0000
1.655 + 0.021
.0.073 + 0.004
-1.670 0.024
GEN-AL
-8.287
0.007
0.014
0.001
-0,458 0.083
0.257 0.043
0.000
00000
1,758 0.009
0.396
-1.353 0.045
GEN-AL-ACS
7.623
1,064
LadProiRL
0.218 0.001
1,222 1: 0.000
7,523 +: 0.062
0,876 1. 0.070
0.500 1: 0.010
3.582
0.003
2.975 1. 0.001
0.018 0.000
MLDE
0.251 = 0.004
1.117 + 0.038
7.834 + 0.130
0.874 0.027
0.555
0.000
3.591 + 0.003
2.975 + 0.006
0.036 0.002
PROSeruz
0.235 - 0.007
1,187 + 0.041
8.033 0.055
1,392 + 0.200
0.676 0.024
3,613 0.002
2.987
0.003
0.040 0.002
Table 7: Average diversity between top 100 sequences generated by each method. Reported values the mean and standard deviation over 5 runs.
UBE21
CMA-ES
247.97 + 1.96
203.40 + 4.60
2323344 + 0.83
49.51 + 1.67
60.34 0.89
163 73 2.44
108.01 + 1.72
316.96 5.31
DynaPPO
116.31 à: 0.82
98.68 à 1.83
24.14 1:44:000 0.31
22.00
1: 0.83
27.96 2: 0.44
79.06
1.63
49.53 = 0.81
157.11
7.56
BO
21.46
29.16 24.73
15.65
3.80
35.52
4.80
0.29
34,92
3.13
32.33 7.00
76.97
57.70
4.65
0.36
6.66
0.86
6.24 0.69
8.17
AdaLead
637 0.93
3.96 4: 0.06
5.22
:: 0.41
3.53
1 0.72
8.09
:: 2.81
26.58 = 12.68
6.04 =: 0.62
6.59
1.17
CbAS
236.10 4: 25
232.03 1: 5.38
76.31 1. 8.05
53.47
4.69
51.21
2 5.59
145.39 1: 48.19
125.98
J: 6.49
349.88 33,82
GEN-AL
323.77 0.07
270.79 1 0.25
96.13 0.66
70.61
0.16
79.70
2: 7.08
225.07
0.12
78.26 72.16
423,68
0.70
GEN-AL-SUS
1.27
5.55 0.96
3.73
LaiProdRL
1.10 0.07
1.27 1. 0.01
1,02 1 0.11
3.84 0.34
3.29 0 0.29
19.97 3.07
2.24 11 0.09
1,70 1: 0.00
MLDD
10.10 1:100:110 1.23
6.87 0.96
1.65 1 1.24
2.35 1 1.10
1.71 1:
1.62 8.80
8.86 = 1.61
7.52 1 2.06
PROSPERO
12.20
5,10 $.001.0.55
1.12 0.28
3.55 0.26
5.11 0.58
9,90 2.01
9.51 1: 2.35
17.25 5.64
22

---

Table 8: Average novelty of top 100 sequences generated by each method. Reported values are the mean and standard deviation over runs.
Mothod
AMIE
TEM
BAB
Pabl
AAV
GFP
UBE2I
LGK
CMA-ES
169.18 + 1,96
136.79 19 = 4.91
51.75 + 0.96
32.37 1.81
40.17 + 0.98
107.96 + 2.41
71.26 1.62
212.41 + 5.63
DynaPPO
0.35
15.40
0.28
27.39
BD
19.27
1.94
35.37
36.86
14.09
2.38
48.94 16.18
7.19
0.68
41.55
9.56
12.95 35,58
90.62 62.22
PEX
5.19 +: 1.08
1.79 1 0.21
3.96 0.54
5.29
0.74
7.29
1.38
6.23
0: 1.34
4.55 1:
8.45 1.39
AdaLend
4.21 + 0.89
2.93 093-4:00 0.07
3.92 + 0.62
8.47
1.58
9.86
1,55
33.79
5.84
3.92 1: 0.47
14.75 8.02
323,72
71.56
0.48
225.15
0.36
GEN-AL
$23.19 =: 0.28
272.81 0:222
97.49 0.28
71,40 0.04
84.06
1.17
226.60
2: 0.16
152.03 0.60
425.16 0.62
GFN-AL-
8.29 0.42
10.10 0.65
4,94 2: 1.59
10.29
1.29
9.70
0.33
34,83 4: 3.17
8.01 1: 3.69
63.16 4: 4,24
LatProiRL
1,09 0.04
1.10 0.01
3.11 1: 0.49
3.85 4
3.03
0.10
12.73 4: 2.21
1.69 0.05
L.71 1: 0.01
19,02 2.69
0.55
3.49
9.67
PROSPERO
20.99 3.32
3.37 0.53
8.81 0.98
3.55
13.03 = 1.59
39.83 2: 3.95
16.45 3: 5.11
74.33 + 7.75
Table 9: Average diversity between top 100 sequences generated by leading methods. Reported values are the mean and standard deviation over 5 runs. Bold: the best overall diversity. Underline: second-best. PROSPERO maintains a viable level of sequence diversity.
Method
AMIE
TEM
E4B
UBEE2
LGK
PEX
7.06 0.61
3.35 + 0.32
4.65 + 0.36
4.88 + 0.87
7,00
7,00++10 1.05
6,66 + 0.86
6,24 + 0.69
8.17 + 1.47
AdaLead
6.37
0.06
3,83
12.68
6.04
1.17
GPN-AL-6CS
14.68 0.79
17.97 + 1.27
5.55
0.96
5.68 + 1.60
8.33
+ 1.14
24.20 + 14.47
8.12 + 3.73
65.68
10.56
LaPPooRL
1.10 0.07
1.27 0.001
4.02
1 0.41
3.84
1 0.34
3.29
0.29
19.97
1: 3.07
2.24
0.09
1,70 0.00
MLDE
10.10 1.23
6,87
4: 0.96
4.65 1: 1.24
2.35
1: 1.40
1.71
0.22
8.80
4: 1.62
8,86 4: 1.61
7.52
2,06
PROSPERO
12,20 1.15
5,10 + 0.58
4.12 0.28
3.55 3.56++0.2
5.11
+
9.90 2.01
9.54 2.38
17.25 5.64
D.2 Early-round protein design
Table 10: Maximum fitness of top 100 sequences generated by leading methods limited to 4 rounds. Reported values are the mean and standard deviation over runs. Bold: the best overall fitness. Underline: second-best.
Method
AMIE
TEM
E4B
Pabl
A.V
GFP
UBE22
LGK
PEX
0.242
0.001
1.231 1 1.0001
7,971 0.078
1.064 0.007
0,801 0.018
3.597
0,002
2.967 1 0.000
0,030
0.001
Adalcad
0.2322 0,008
1:227 2-0.00
7,3022
1.997
0.506 d: 0.014
3.580
11,033
0.002
0,002
GFN-AL-ACS
0.100
0.563
0.047
0.032
LalPorR..
1,229
0.000
7.751
3.580
0.001
0.024
0.000
0.000
PROSPIRO
0.236 0.007
1.299 0.000
7.978 0.055
0.129
0.635 0.011
3.602
0.002
2.989
0.002
0.001
Table 11: Mean fitness of top 100 sequences generated by leading methods limited to 4 rounds. Reported values are the mean and standard deviation over runs. Bold: the best overall fitness. Underline: second-best.
Method
AMIE
TEM
E4B
Pabl
AAV
GFP
UBE2
LGK
PEX
0.230 1 0.001
1.176 0.008
7.886 à 0005
0.891 1: 0.006
0.551 2: 0.012
3,500 0.000
2.982 1: 0.001
0.28 = 11,00L
And
0.224 + 0.002
2.051
7.882 0.088
1.118 + 0.222
0.564 0.010
3.557 +.0.07
2.964 + 0.007
01229 0.002
GEN-AL-SCS
-0.00
1.31
0.008
0.017
1,271 0.203
0.865 1. 0.065
0.51 1 0.003
5.562
1 0.007
2,772 1. 0.11
0011 0.003
LatPredRL
0.200 0.00
1225
0.000
6.704
1 125
6.745
1: 0.841
0.525 2: 0.001
3.571
1 0.005
2.090 0.006
0018 0.000
MLDE
0.212 0.011
1.000
2: 0.024
7.538
0.100
0,786
2: 0.039
0.551 1: 0.000
3,585
2: 0.003
2,913 3: 0.019
0,255 11004
PROSPERO
0.221 + 0.010
1.004 22 0.104
7.781 à: 0.048
1,009 2: 0.0722
0.576 0.017
3.595 2 0.002
2.376 2: 0.006
0.033 0.0U2
Table 12: Average diversity between top 100 sequences generated by leading methods limited rounds. Reported values are the mean and standard deviation over 5 runs. Bold: the best overall diversity. Underline: second-bess
Method
AMIH
TEM
Pabl
AAVV
UBB121
1,3K
PEX
5.02 2: 0.50
3.07 0.22
3.55 0.36
3.93 1: 0.56
5.27 :: 0.76
5.85 0.99
1.61 0.63
7.25 4: 1.29
Ada
4.34
0.34
4.04 0.07
4.19
0:225
3.12 0.50
7.00
: 0.43
32.04 = 10.76
5.38 +
4.36
1.29
GLN-AL-
21.22
5.12
20.39 0.75
5.21
5.49 0.855
6.53
:: 0.52
21.13 2: 13.53
13.14
62.52
LaPhoIRI.
1.60 0.04
4.02
3.34 0.12
27.93 6.33
2.32+ 0.01
1.79
0.00
MLDE
10.99 1.45
6.93 4: 0.62
0.75 5.53
3.98 + 0.98
1.63 = 0.40
8000 = 1.27
7.36 1.88
8:20
1.82
PROSPERO
1.8111182
6.09 1.19
4.11
3.65 0.25
4,88 0.45
9.18 1: 0.88
9.08 0.78
21.18
23

---

Table 13: Average novelty of top 100 sequences generated by leading methods limited to 4 rounds. Reported values are the mean and standard deviation over 5 runs. Bold: the best overall novelty. Underline: second-best.
Method
AMIE
TEM
BAB
Pabl
AAV
GFH
UBE2I
LGK
PEX
2.83 a: 0.24
1.77 1: 0.12
1.97 0.0.27
2.33 4: 0.41
3.65 0.17
4.01 0.78
2.67 0.4.8
4.45 d: 1.00
2.77
0.05
4.82 0.36
8,30
GFN-AL-SCS
11.08 2.71
11:45 0.400
8.52
4.99 2: 0.81
4.52 = 0.26
31.61 2.44
1.914
45.70 :: 3.31
LatProtkI,
1.36 04054
1.61 +:000
3.10 0:118
2.87 + 0.45
1.88 0.06
16,44-13..5
1.76 0:07
1,85 + 0.01
4.71
PROSCERO
10.67 = 1.82
3.40 + 0.58
3.95 + 0.88
6.16 2.36
6.60 0.78
17.16:33.25
9:02
37.00 4.32
D.3 Out-of-distribution robustness
Table 14: Results under moderate covariate shift. Reported values are the mean and standard deviation over runs. Bold: the best overall value. Underline: second-best.
Method
Maximum PTM
Mean pTT
Diversity
Novelty
PEX
0.807 + 0.023
0.760 + 0.012
6.14 + 0.89
4.45 0.38
AdaLead
0.796 + 0.013
0.755 + 0.011
8.83 + 2.54
8.36 = 2.97
GFN-AL-8CS
0.791 + 0.010
0.729 0.005
16.92 0 0.88
9.56 0.60
LatProtRL
0.787 = 0.013
0.743 = 0.003
6.32 - 0.32
5.90 = 0.53
MLDE
0.810 = 0.020
0.752 0.004
9.89 = 1.11
20.88 + 2.98
PROSPERO
0.822 = 0.027
0.777 + 0.020
11.50 + 1.62
17.74 + 3,20
Table 15: Results under severe covariate shift. Reported values are the mean and standard deviation over 5 runs. Bold: the best overall value. Underline: second-best.
Method
Maximum PTM
Mean pTT
Diversity
Novelty
PEX
0.578 + 0.014
0.518 + 0.003
3.40 + 0.07
1.72 + 0.04
AdaLead
0.593 + 0.028
0.526 0.007
14.26 1.91
7.66
1.08
GFN-AL-SCS
0.630 = 0.024
0.542 + 0.006
14.63 + 1.16
LatProtRL
0.560 = 0.000
0.508
0.003
2.24
1.78
0.16
MLDE
0.652 = 0.059
0.572 + 0.035
13.10 1.18
21.68 + 3.85
PROSPERO
0.672 + 0.031
0.599 + 0.014
14.51 1.99
22.03 + 1.69
Table 16: Results under low-data covariate shift. Reported values are the mean and standard deviation over runs. Bold: the best overall value. Underline: second-best.
Method
Maximum pTM
Mean p'T'
Diversity
Novelty
PEX
0.806 + 0.013
0.752 0.005
6.77 + 0.45
4.39 0.51
AdaLead
0.781 = 0.016
0.742 + 0.004
7.99 + 1.39
5.95 = 2.04
GFN-AL-8CS
0.782 = 0.006
0.731 0.006
15.82
9.46 1.69
LatProtRL
0.792 = 0.013
0.743 = 0.001
6.25 + 0.23
5.57 = 0.20
MLDE
0.782 + 0.022
0.735 0.025
9.39 + 2.88
16.97 3.78
PROSPERO
0.808 = 0.017
0.763 0.017
11.25
2.51
15.87 + 1.17
E Extended related work
Inference-time guidance methods Several methods have been proposed to steer generative models during inference. Dhariwal and Nichol [58] introduced classifier guidance, where the sampling is biased toward desired properties by adjusting the generative score with the gradient of an auxiliary classifier. However, this approach is not directly applicable in the discrete data domain, where
24

---

gradients with respect to inputs are not well-defined. To address this, Nisonoff et al. [59] developed a framework that enables classifier guidance in discrete diffusion and flow matching models. Their approach leverages a continuous-time Markov chain formulation of the forward process and corre- sponding reverse-time generative process [60], where only one coordinate changes each transition, making exact guidance tractable.
Guidance can also be realized through SMC-based approaches, applicable in both discrete and continuous domains [61-63]. These methods steer generation by maintaining a population of particles that represent partial trajectories and resampling them according to their likelihood under target distribution. Ekström Kelvinius and Lindsten [64] extend discriminator guidance [65]. originally developed for score-based diffusion models. to Autoregressive Diffusion Models (ARDMs) [66] (such as leveraged in our work EvoDiff- -DDM---ODMM [38D, and further employ SMC to correct for discriminator errors at intermediate sampling steps. Li et al. [67] take a similar approach that resembles SMC in the use of importance sampling, but instead of resampling across the entire batch of particles, they generate and reweight multiple candidates from each individual sample at the previous step. Building on this idea, Uchara et al. [68] combine it with noising policy, iteratively alternating between re-noising and reward-guided denoising to progressively refine samples.
Similarities to CloneBO [26] PROSPERO and CI by Amin et al. [26] share similarities in guiding generative model at inference time using SMC. However, the approaches differ meaningfully in both scope and mechanism. First, in CloneBO, the generative model (Clonel.M) has been trained specifically for the optimization task by fitting to a distribution of clonal families. In contrast, our method uses optimization regardless of the protein family. As for the differences in the use SMC, in CIoneBO the authors compute intermediate importance weights directly, as a likelihood ratio between the base CloneLM and a twisted variant incorporating high value sequences (i.e. sequences with experimental measurements) in the conditioned clonal family. In PROSPERO, we approximate the intermediate importance weights using the surrogate model, which serves as a tractable proxy for the true likelihood. Moreover, our making certain residues impossible to sample. In contrast, CloneBO does not enforce such constraints; although twisting reduces the probability of sampling undesirable residues, it does not eliminate them entirely.
task-agnostic pre-trained generative model, enabling effortless
constrained SMC restricts proposals to conpg--b amino acids,
Connections between SMC and marggnal MCMC Pseudo-marginal MCMC [69] and SMC differ in the way they approximate the target distribution. In pseudo- maaudnaa MCMC, within a single Markov chain, a single collection of samples is generated whose marginal stationary distribution is exactly the target distribution. The approximation improves with the number of steps T and is exact in the limit of inifite T. In contrast, SMC maintains population of N samples that evolve over a sequence of T intermediate distributions. The empirical distribution formed by these samples converges to the target distribution in the limit of infinite N.
F Further ablations
Influence of the number of starting sequences on candidate generation We investigated how varying the number of best starting sequences Tsaan affects the proposed candidates by conducting experiments on the LGK landscape, which requires generating the longest sequences (L = 439). The results in Table 17 demonstrate that fewer starting points drive deeper, more directed exploration, resulting in higher novelty and fitness but lower diversity. In contrast, more starting points promote broader, more diffuse exploration, increasing diversity but limiting how far any single trajectory moves from the wild-type across subsequent optimization rounds.
Influence of reducing the amino acid alphabet To further analyze the isolated effect of constrain- ing the proposals to starge-compatible amino acids, we directly compare the performance of the full PROSPERO with its ablated counterpart lacking this restriction (without RAA). We followed the setup detailed in Section 5.5. The results in Table 18 highlight the relevance of this feature, showing consistent improvements across all signal-to-noise ratio levels.
Influence of the charge-class permutation order The sampling permutation order in biologically- constrained SMC was chosen to prioritize charge classes with fewer valid options (negatively charged
25

---

residues first, followed by positively charged, and finally neutral), as resolving the most constrained decisions early should help prevent suboptimal completions later in the sequence. To assess this, we conducted an ablation comparing PROSPERO with its standard permutation order (ascending) to both a reversed (descending) order and a random order. The results presented in Table 19 show that the reasoning behind this choice appears correct, though the performance benefits are modest.
Table 17: Results on the LGK landscape for varying numbers of starting sequences. Reported values are then mean and standard deviation over runs. The best overall values are highlighted in bold.
Ustartt
Maximum fitness
Mean fitness
Diversity
Novelty
0.043 0.002
0.040 + 0.002
17.25
5.64
74.33 + 7.75
4
0.041 + 0.001
0.039 + 0.001
19.30 + 4.58
70.20 + 6.05
16
0.038 0.002
0.037 = 0.002
19.09 5.80
64.86 - 5.44
64
0.037 + 0.002
0.034 + 0.001
29.13
56.38
128
0.033 0.002
0.030 = 0.002
33.89 = 9.54
50.45 5.52
Table 18: Results the AAV landscape across different signal-to-noise ratio levels (SNR). Reported values are then mean and standard deviation over 5 runs. The best overall values are highlighted in bold.
SNR level
-25
-20
-15
-10
PROSPERO
0.566
0.030
0.586 = 0.026
0.651 + 0.032
0.679 = 0.047
0.704 = 0.016
0.706 = 0.022
PROSPERO w/o RAA
0.507
0.025
0.040
0.029
0.605
0.032
0.653
0.042
0.029
Table 19: Results on all the landscapes with different permutation orderings. Reported values are then mean and standard deviation over 5 runs. The best overall values are highlighted in bold.
Ordering
AMIE
TEM
E4B
Publ
AAV
GFP
UBE21
LGK
Ascending
0002
0.254
0.720
0.002
Deseanding
0.003
8.139
1,363
0.141
0.706
0.003
2.991
0.003
Random
0.213 1: 0.005
1.232
0.002
8.164 1 0.015
1,338 2 0.113
0.708
0.029
3,614 = 0.003
2.993
0.002
0.012 1 0.003
G
Algorithms
Algorithm 2: Targeted Masking
Input: Starting sequence Tsan proxy fo. SMC batch size B, scans S, min substitutions 7'mins max substitutions Mmaxx exploitation-exploraiii coefficient k
Output: Masked sequences { Je
substitution locations (I(1 li=1
1 for i f-1 to B x S do
(i)
nub U (mmtt
I(a ~ UniformSubset(1, |ramll) 1. (i)
sub)
(+) + Istunt where ( [j] + for I(()
J < arg max ZE(20)) Mo(x) + k (
for (1) E J do
(1)) <- (1)) where [0] [MASK] for j E T()
8 return {B Iie {I() L
26

---

Algorithm 3: ConstrainedSMC
Input: Partially masked sequences { |= model P. proxy fo. oracle budget K, exploitation-exploriiion coefficient kept rollouts
=== mask locations (E pre-trained generative
threshold Keee
Output: Candidate sequences {z((1 1=
RolloutBuuffe t- 0
for i +- to B do
(+) concat (I I(11 I(()
100
(-) (+)
T < arg e
[
for t <- to T do
for F(0) E {
li=l do
if then
LL +
if S I then
2009
m(t+ [ PRAM (2
(0(0
((ce+120)))
11
(a ( + ROLLOUT(ZO), (((H)()))
LL - LL + log P (1-1Z -
(00)
L p.sstt)) Algorithm
12
g(D) p= + k o0 (=
invPPI
(4) exp
[Z(
15
if7 T - << Mheepp then RolloutBuffer RolloutBuffer (x
unoul: (
16
-ivPPL ( w + 5 by invPPL, i=1 17
18
for i + 1 to B do
19
Resample:
(t) CCal(w))
z(1) + (1)
(1) -- π (0)
(I) +- (Tdr(()((())
20 return [201] f- arg max K (o
Algorithm Rollout
Input: Partially masked sequences {
fim sampling permutations {T(1) | log. likelihoods
(T,T,()) pre-trained generative model P, unmasking step
Output: Unrolled sequences (1 ) log-likelihoods (LL )
<- arg max (
[Z]
2 for t + 9 to T' do
for 200 E [ do
1=
ift < [ then
(1)
7((HZ)) PRAM (i)
7(H (1)
)
L + ا (1) + log P(a 7
(000
-(<(+1(9))
7 {1r
unmoll 12 + [ 1
8 {LL( (1) 18
unroll 1=1 t {L
1==1
return (1)( (6) B
Maroll i= (LI anrot 18
i=1
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims made in the abstract and introduction reflect the paper's 's contribu- tions and scope. Section 5.1 presents our method's protein fitness optimization capabilities; Section 5.2 demonstrates the biological plausibility the generated sequences; Section 5.4
Justification:
shows robustness surrogate misspecification.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Limitations of our work are discussed in Section B.I.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
28

---

Answer:
[NA]
Justification:
We do not include any theoretical results or formal proofs in this work.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We detail the approach in Section 4, provide the algorithms in Section G and include code to reproduce our results in Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path reproducing verifying the results. 5. Open access to data and code
29

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We include code reproduce results in Section 1 and provide details the implementation of the baselines in Section A.4.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
conolicyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /ttppp.ccpppliiiiimm///tttp/nttc wwtppccppccc///ccccccccccccccclt http//uudeeee/eeee https//ooSSSuuiisseeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We detail our experimental setup Section and describe the surrogate model training procedure in Section A.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report standard deviation for all figures and tables in Section 5, as noted in each caption.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
30

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer:
[Yes]
Justification: We detail the compute resources used, including GPU type and runtimes, in Section B.I.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htap::::///nuriii httpprppp......pbpa/a//teaaaeeee
Answer: [Yes]
Justification: Our work conforms with the NeurIPS Code of Ethics
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss broader impacts of our work in Section B.3.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
31

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work does not involve the release of pre-trained models datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All asset owners properly credited, and licenses explicitly mentioned and respected.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
32

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We include documented code implementing method in Section which we will release under the GPL-3.0 license.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
33

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: This work does not involve LLMs as important, original, or non-standard component of the core methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: httpss///eeuppp.eeeee http://urrpp.../oooeeeeeeeeeeeee / for what should or should not be described.
34


Reviews:
[{"review_id": "9mBcHYwUZG", "reviewer": "Reviewer_XJMJ", "summary": "In this work a new active learning framework is proposed for the design of protein sequences. The proposed method explores functionally important residue sites instead of random sites, which are identified inspired by a technique used in the lab to perform mutagenesis. Additionally, a pre-trained generative model along with a surrogate function of the property is used to sample sequences having optimized properties, while maintaining charge-class similarity to the wild-type at functionally important residue sites. This allows for the generation of biologically plausible sequences in regions of the design space where surrogate predictions may be unreliable.", "questions": "1) How many sites are masked in each benchmark and how is this number determined?\n\n2) In each active learning round, only the best sample/sequence is used for masking and proposal of new candidates. Isn't this limiting the exploration capability of the model? I understand that simplifying the algorithm by sticking to the best sequence is a good design if it is working, but have you performed any ablation study on the number of top sequences used for candidate generation?\n\n3) I find the description for \"Sampling from ... using Sequential Monte Carlo\", lines 144-167 pretty hard to follow. It would be nice to explain what the overall procedure should look like and why before getting into its steps.\n\n4) Is the extension beyond wild-type neighborhood only measured by the novelty metric (average hamming distance between top 100 candidates and a starting sequence (wild-type))? What is the average hamming distance between the starting wild-type sequence and the rest of the sequences in the initial dataset? Or what is the average distance between the top 100 and the initial dataset? \nThe reason I am asking about the distance from the initial dataset is because it is discussed that the model is robust to the surrogate misspecification/uncertainty, so I would like to know if there is a way to quantify the extent of exploration beyond surrogate comfort zone (initial dataset) for the benchmark in Table 1.\n\n5) If I understood it correctly, all experiments are performed with fixed initial datasets. So the variability in the performance only comes from the model stochasticity. Have you examined how the initial dataset affects the model performance? This matters when the size of the initial dataset is small. \nYour algorithm starts by taking the best sample from the initial dataset, so having a smaller initial dataset, with less coverage of the landscape, could make the optimization harder. Based on the information included in the Appendix, the initial datasets are quite large (3K, 5K, 6K, 7K, 10K, 15K) which is NOT the case in many practical settings. Have you examined the variability of your model's performance on smaller initial datasets (<< 1000, e.g., 200)?\nThe size of the initial dataset affects the quality of the trained surrogate model as well as the search in the design space depending on how the initial samples are distributed within the landscape, as discussed in prior work [1]. I do not expect you to add extra experiments on this but how do you expect your model to behave for small initial datasets?\n\n6) I liked the idea and reasoning for scoring residues by Alanine replacement. Did you try other amino acids as baselines? I understand that this is not backed by what experimentalists would do but computationally, one would like to know how significant the choice of Alanine is over the rest of the amino acids. \n\n7) Nice-to-haves: Missed citation for a recent generative-based approach [1] developed to address  practical challenges of protein design.\n\nReferences:\n\n[1] Ghaffari et al. Robust Model-Based Optimization for Challenging Fitness Landscapes. ICLR 2024.\nhttps://arxiv.org/pdf/2305.13650.", "limitations": "NA", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "What I really liked about this work is basing the model design on how experimentalist scan for important sites in mutagenesis experiments.", "was_revised": true, "final_justification": "All my questions were addressed.", "rebuttal": "Thank you for taking the time to provide such a detailed and insightful review of our manuscript.\n\n## Q1: Number of mutational sites\nThe number of masked sites in each benchmark was determined based on the sequence length, with the number sampled uniformly from the range of 5–15 residues for longer, and 3–10 for shorter sequences. The range of 5–15 residues aligns with ranges commonly used by experimentalists in rational design for example on the GFP task [1], making this another aspect of our framework that reflects mutagenesis-inspired behavior. Kim et al. [2] have demonstrated that this masking range performs well *in silico*. Additionally, they experiment with masking 5% of the total residues in a sequence. Based on this, for shorter sequences, such as AAV, E4B, and Pab1, we reduced the masking range to 3–10 amino acids.\n\n\n## Q2: Ablation study on the number of top starting sequences for the candidate generation\nThank you for the helpful suggestion. We added an ablation on LGK, the task requiring generation of the longest sequences, to test the effect of varying the number of top starting sequences (1–128). Results are shown below:\n\n|N starting sequences| Maximum fitness| Mean fitness | Diversity     | Novelty      |\n|-|-|-|-|-|\n|1| **0.043 ± 0.002**   | **0.040 ± 0.002**   | 17.25 ± 5.64  | **74.33 ± 7.75** |\n|4| 0.041 ± 0.001   | 0.039 ± 0.001   | 19.30 ± 4.58  | 70.20 ± 6.05 |\n|16| 0.038 ± 0.002   | 0.037 ± 0.002   | 19.09 ± 5.80  | 64.86 ± 5.44 |\n|64| 0.037 ± 0.002   | 0.034 ± 0.001   | 29.13 ± 6.97  | 56.38 ± 7.06 |\n|128| 0.033 ± 0.002   | 0.030 ± 0.002   | **33.89 ± 9.54**  | 50.45 ± 5.52 |\n\nFewer starting points drive deeper, more directed exploration, resulting in higher novelty and fitness but lower diversity. In contrast, more starting points promote broader, more diffuse exploration, increasing diversity but limiting how far any single trajectory moves from the wild-type across subsequent optimization rounds. \n\nAs suspected by the reviewer, this new table brings important insights into the tradeoffs associated with the number of initial sequences and the exploration capabilities and we will incorporate it into the final version of the paper.\n\n\n## Q3: Clarity of lines 144-167\nThank you for the comment. We understand that this section may be difficult to follow without additional context. To support readers unfamiliar with Sequential Monte Carlo, we already refer to Appendix C in line 146, which provides a more detailed mathematical background and explains the overall procedure. To further improve the clarity of this section, the revised version of our work will feature the following changes:\n- In Equation 1 of Section 4.2, we will revise the notation of the target distribution from $\\gamma(x)$ to $\\gamma(x_{1:L})$ , and update all other occurrences of $x$ accordingly within the introductory part of Section 4.2. This change makes explicit that the target is defined over a sequence of residues, improving clarity and alignment with the sequential structure of SMC\n- We will begin the L144-167 part with the following paragraph: Rather than trying to directly sample from complex, high-dimensional target distribution $\\gamma(x_{1:L})$, in ProSpero we perform approximate inference using SMC, which decomposes the sampling process into a sequence of simpler, unnormalized, intermediate target distributions $\\{\\tilde{\\gamma}(x_{1:l})\\}_{l=1}^{L}$ (for background see Appendix C). This allows us to progressively build up sequences residue by residue, while maintaining a tractable approximation of the full design objective\"\n- We will conclude the line 162 as follows: “Finally, the unnormalized importance weights […] , with the perplexity correction term penalizing sequences that are improbable under the true prior, correcting bias introduced by the constrained sampling”\n\nWe hope this revision will make the section clearer and more accessible to the readers.\n\n\n\n## Q4: Other measures of exploration\nThank you for this insightful suggestion. We agree that the inclusion of the proposed Hamming distances will help to quantify the extent of exploration of our approach. We have computed the average Hamming distance between each starting sequence and its corresponding initial dataset. \n\n|| AMIE | TEM  | E4B  | Pab1 | AAV  | GFP   | UBE2I | LGK  |\n|-|--|--|--|--|--|--|--|--|\n| WT distance to initial dataset | 2.00 | 2.00 | 5.42 | 3.95 | 5.05 | 42.87 | 2.00 | 2.00 |\n\nAs shown, distances for the majority of the tasks are small. This is expected given that most datasets were created by wet-lab scientists applying a limited number of mutations to the wild-type sequence and measuring the activity of such mutated variants. A notable exception is the GFP task, where the initial dataset was created *in silico* by Kim et al. [2] through random mutagenesis and scoring via the oracle. \n\nWe have also computed the average distances between top 100 sequences and the initial datasets:\n| Method| AMIE| TEM | E4B|Pab1| AAV| GFP| UBE2I| LGK|\n|-|-|-|-|-|-|-|-|-|\n| AdaLead| 5.45 ± 0.62  | 3.04 ± 0.03  | 9.21 ± 0.58  | 11.84 ± 1.35 | *14.09 ± 1.43* | 70.14 ± 4.74 | 4.83 ± 0.48  | 16.31 ± 7.22 |\n| PEX| 7.18 ± 1.08  | 3.78 ± 0.21  | 9.24 ± 0.52  | 9.06 ± 0.71  | 11.75 ± 1.27 | 47.89 ± 1.09 | 6.51 ± 0.82  | 10.43 ± 1.38 |\n| GFN-AL-$\\delta$CS | 8.47 ± 0.40  | **10.19 ± 0.67** | 10.25 ± 1.55 | 12.88 ± 1.27 | 13.93 ± 0.32 | *70.98 ± 3.02* | 8.99 ± 2.97  | *63.46 ± 4.33* |\n| MLDE| *20.83 ± 2.78* | *5.99 ± 0.42*  | **16.91 ± 3.39** | *13.12 ± 3.04* | 9.14 ± 0.23  | 62.02 ± 4.40 | *11.53 ± 2.56* | 51.91 ± 7.99 |\n| LatProtRL| 2.10 ± 0.04  | 2.01 ± 0.01  | 7.59 ± 0.46  | 6.62 ± 0.84  | 7.84 ± 0.36  | 53.02 ± 1.78 | 2.51 ± 0.11  | 2.00 ± 0.00  |\n| ProSpero| **22.92 ± 3.30** | 5.31 ± 0.53  | *13.99 ± 0.95* | **15.44 ± 3.42** | **18.88 ± 1.47** | **75.09 ± 3.10** | **18.33 ± 5.08** | **75.95 ± 7.84** |\n\nSuch evaluated performance closely resembles the one in Table 3, with ProSpero achieving larger distances than compared models for the majority of the tasks. These results underscore the capabilities of our approach to explore regions of the search space beyond the initial training data, even in cases such as the GFP task, where the initial sequences are not as densely clustered around the wild-type. This new table will be featured in the revised paper.\n\n\n## Q5: Initial dataset size influence on the performance\nThank you for raising this important point. We agree that small initial datasets are common in practice. Following your suggestion, as well as suggestions of other reviewers, we've added an additional task that better reflects real-world settings. \n\nSpecifically, from all UBE2I candidate sequences generated by different methods and scored with ESMFold throughout our experiments, we randomly selected *200* of those within a Hamming distance $\\leq 5$ from the UBE2I wild-type to construct the initial dataset $D_0$, where the average pTM score of $D_0 = 0.860 ± 0.023$. We then trained a surrogate model $f_\\theta$ on $D_0$, and initiated the optimization not from the wild-type sequence, but from a sequence differing from the wild-type by a large number of 35 residues and of an initial pTM score of 0.70. We run such optimization for only 4 iterations, employing ESMFold as an oracle. The results are shown below:\n\n|Method| Maximum pTM| Mean pTM| Diversity| Novelty|\n|-|-|-|-|-|\n|AdaLead| 0.781 ± 0.016   | 0.742 ± 0.004   | 7.99 ± 1.39   | 5.95 ± 2.04  |\n|PEX | *0.806 ± 0.013*   | *0.752 ± 0.005*   | 6.77 ± 0.45   | 4.39 ± 0.51  |\n|GFN-AL-$\\delta$CS| 0.782 ± 0.006   | 0.731 ± 0.006   | **15.82 ± 1.77**  | 9.46 ± 1.69  |\n|MLDE| 0.782 ± 0.022   | 0.735 ± 0.025   | 9.39 ± 2.88   | **16.97 ± 3.78** |\n|LatProtRL | 0.792 ± 0.013   | 0.743 ± 0.001   | 6.25 ± 0.23   | 5.57 ± 0.20  |\n|ProSpero | **0.808 ± 0.017**   | **0.763 ± 0.017**   | *11.25 ± 2.51*  | *15.87 ± 1.17*  |\n\nWe believe the strong performance of our approach in comparison to the baselines is due to the incorporation of biological priors, which enable efficient exploration even when the surrogate model is weak, whether because of limited training data or distribution shifts. We will include this result in a revised version of the paper.\n\n## Q6: Amino acid choice for replacement-based residue scoring\nIn our work, we confirmed the advantage of residue scoring using Alanine scan over random amino acid substitutions, as shown in our ablation studies in Section 5.4.\nFor convenience, we summarized the key results in the table below:\n\n|SNR level|-25|-20| -15| -10| -5| 0|\n|-|-|-|-|-|-|-|\n| ProSpero   | **0.566 ± 0.03**  | **0.586 ± 0.026** | **0.651 ± 0.032** | **0.679 ± 0.047** | **0.704 ± 0.016** | **0.706 ± 0.022** |\n| ProSpero w/o TM     | 0.554 ± 0.021 | 0.581 ± 0.028 | 0.597 ± 0.027 | 0.672 ± 0.021 | 0.677 ± 0.030 | 0.649 ± 0.028 |\n\nWe did not perform systematic one-to-one comparisons with other specific amino acids. This is because alanine scanning is a well-established strategy in experimental mutagenesis, whereas there is little principled justification for selecting any other single amino acid as a universal substitute with neutral properties. As such, alternative choices lack clear biological or methodological motivation, making direct comparisons difficult to interpret meaningfully.\n\n\n## Q7: Missing citation\n\nThank you for the suggestion. We agree the cited work is relevant and will include it in the revised manuscript.\n\nThank you for the thoughtful and constructive comments, which greatly helped us improve the manuscript. We hope we have addressed your concerns, and if so, we would be grateful if you would consider **raising your rating of our submission**.\n\n#### References\n[1] T. Hayes et al. Simulating 500 million years of evolution with a language model. Science 2025. \n\n[2] H. Kim et al. Improved off-policy reinforcement learning in biological sequence design. 2024."}, {"review_id": "HtWtFicxQz", "reviewer": "Reviewer_gUKh", "summary": "The paper presents PROSPERO, an active learning framework for protein sequence design that aims to generate sequences with high fitness, novelty, and biological plausibility. Its key innovations are: (1) iterative sequence design as inference-time guidance of a pre-trained generative model within an active learning loop that incorporates biological priors; (2) a targeted masking strategy that edits fitness-relevant residues while protecting critical sites; and (3) biologically constrained Sequential Monte Carlo (SMC) sampling, which restricts mutations based on explicit priors to improve discovery of high-fitness, novel sequences even when the surrogate model is imperfect. Experiments show that PROSPERO consistently outperforms or matches existing methods across diverse tasks, and is robust to surrogate model misspecification.", "questions": "refer to above", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The PROSPERO framework is clearly presented, with comprehensive technical details and good structure. Key components—such as targeted masking based on alanine scanning and biologically-constrained SMC sampling—are introduced in detail to boost the reproducibility.\n\n2. The paper conducts a comprehensive experimental evaluation including eight diverse protein engineering tasks, comparing PROSPERO to ten strong baselines. The ablation is comprehensive, and the results consistently demonstrate the effectiveness of  the proposed approach with an emphasis on exploration.  \n\n3. By guiding a frozen generative model with an active learning-updated surrogate and introducing targeted, biologically-aware masking, PROSPERO advances protein design. The approach effectively addresses the trade-off between novelty and biological plausibility, with broad potential impact on protein engineering applications.\n\nWeaknesses\n\n1.  Some technical components—such as SMC unrolling and perplexity correction—are formally described but may lack intuitive explanations for readers less familiar with these methods. The comparison with all baselines is somewhat uneven, limiting contextual understanding.\n\n2.  While innovative in integrating various ideas, components like SMC and charge-based grouping are adapted from prior work, and experimental scope is limited to controlled scenarios. PROSPERO’s performance on more complex or out-of-distribution tasks remains untested.", "was_revised": true, "final_justification": "a sound paper for protein sequence design with relatively rough writing.", "rebuttal": "We thank the reviewer for their positive and thoughtful feedback. \n\n> Some technical components [...] are formally described but may lack intuitive explanations for readers less familiar with these methods.\n\n\nWhile the full details of the SMC procedure are included in the Appendix C, we agree that a more intuitive explanation in the main text could benefit readers less familiar with the method. We will include the following changes in the revised version of our work:\n- In Equation 1 of Section 4.2, we will revise the notation of the target distribution from $\\gamma(x)$ to $\\gamma(x_{1:L})$ , and update all other occurrences of $x$ accordingly within the introductory part of Section 4.2. This change makes explicit that the target is defined over a sequence of residues, improving clarity and alignment with the sequential structure of SMC\n- We will begin the L144-167 part with the following paragraph: Rather than trying to directly sample from complex, high-dimensional target distribution $\\gamma(x_{1:L})$, in ProSpero we perform approximate inference using SMC, which decomposes the sampling process into a sequence of simpler, unnormalized, intermediate target distributions $\\{\\tilde{\\gamma}(x_{1:l})\\}_{l=1}^{L}$ (for background see Appendix C). This allows us to progressively build up sequences residue by residue, while maintaining a tractable approximation of the full design objective\"\n- We will conclude the line 162 as follows: “Finally, the unnormalized importance weights … , with the perplexity correction term penalizing sequences that are improbable under the true prior, correcting bias introduced by the constrained sampling”\n\nWe hope this revision will make the section clearer and more accessible to the readers.\n>  The comparison with all baselines is somewhat uneven, limiting contextual understanding.\n\nWe would like to emphasize that  our evaluation covers eight diverse protein engineering tasks and in a fair manner compares against ten baselines spanning a wide range of algorithmic approaches, including evolutionary strategies, Bayesian optimization, reinforcement learning, GFlowNets, probabilistic models, and machine-learning-guided directed evolution. In addition to fitness optimization, we show tradeoffs to  exploration capabilities, biological plausibility and robustness to surrogate noise, resulting in a comprehensive and multi-faceted comparison. We believe this breadth offers a strong contextual understanding of the advantages of ProSpero relative to existing methods.\n\n> While innovative in integrating various ideas, components like SMC and charge-based grouping are adapted from prior work [...]\n\nWe agree that SMC has been applied to protein design in prior works [1, 2]. We would like to clarify, however, that introduced by us biologically-constrained SMC represents a novel and principled extension: it allows biological priors to be incorporated directly into the proposal distributions. To specifically emphasize the impact of this contribution, we will complement our existing ablation study with a direct comparison between our biologically-constrained SMC and an unconstrained formulation:\n| SNR level    | -25       | -20       | -15       | -10       | -5       | 0       |\n|-----------|--------------|--------------|--------------|--------------|--------------|--------------|\n| ProSpero | **0.566 ± 0.030** | **0.586 ± 0.026** | **0.651 ± 0.032** | **0.679 ± 0.047** | **0.704 ± 0.016** | **0.706 ± 0.022** |\n| ProSpero w/o RAA | 0.507 ± 0.025 | 0.555 ± 0.040 | 0.588 ± 0.029 | 0.605 ± 0.032 | 0.653 ± 0.042 | 0.666 ± 0.029 |\n\nAdditionally, we note that reduced amino acid alphabets (RAAs) are by no means standard or widely adopted in this context. Specifically, we refer to the work of Liang et al. [3], concluded with \"[...] RAA alphabets have not been fully and maturely used in current research.\" Given this, we believe that our biologically informed use of RAAs represents an impactful contribution in its own right, representing a notable use case.\n\n> PROSPERO’s performance on more complex or out-of-distribution tasks remains untested.\n\nThank you for raising this relevant point. To address your and other reviewers concerns, we introduced another 3 tasks evaluating effectiveness of our method under various degrees of distribution shift. In these experiments we used ESMFold as the oracle, which offers high-fidelity feedback better suited for performance assessment in more challenging and realistic settings.\n\nSpecifically, from all UBE2I candidate sequences generated by different methods and scored with ESMFold, we selected those within a Hamming distance $\\leq 5$ from the UBE2I wild-type to construct the initial dataset $D_0$, consisting of 2624 sequence---pTM score pairs. The average pTM score of $D_0=0.860 ± 0.029$. We then trained a surrogate model $f_\\theta$ on $D_0$, and initiated the optimization not from the wild-type sequence, but from a sequence differing from the wild-type by a large number of 35 residues and of an initial pTM score of 0.70. We then run such set optimization for 4 iterations. The results are shown below:\n\n| Method | Maximum pTM|Mean pTM|Diversity| Novelty|\n|-|-|-|-|-|\n| AdaLead |0.796 ± 0.013|0.755 ± 0.011|8.83 ± 2.54| 8.36 ± 2.97|\n| PEX |0.807 ± 0.023|*0.760 ± 0.012*|6.14 ± 0.89| 4.45 ± 0.38|\n|GFN-AL-$\\delta$CS |0.791 ± 0.010|0.729 ± 0.005| **16.92 ± 0.88**|9.56 ± 0.60|\n|MLDE|*0.810 ± 0.020*|0.752 ± 0.004|9.89 ± 1.11 |**20.88 ± 2.98**|\n| LatProtRL |0.787 ± 0.013|0.743 ± 0.003|6.32 ± 0.32| 5.90 ± 0.53|\n|ProSpero|**0.822 ± 0.027**|**0.777 ± 0.020**| *11.50 ± 1.62*|*17.74 ± 3.20*|\n\nwhere by novelty here we mean the distance of top 100 sequences to the starting point, not the original wild-type.\n\nMoreover, we additionally repeated the same experiment, but under an even more severe distribution shift, starting optimization from a sequence differing from the wild-type by a larger number of 75 residues and of an initial pTM score of 0.50. The results are shown below:\n\n| Method | Maximum pTM| Mean pTM| Diversity| Novelty|\n|-|-|-|-|-|\n|AdaLead | 0.593 ± 0.028| 0.526 ± 0.007| 14.26 ± 1.91| 7.66 ± 1.08|\n|PEX | 0.578 ± 0.014|0.518 ± 0.003|3.40 ± 0.07| 1.72 ± 0.04|\n|GFN-AL-$\\delta$CS|0.630 ± 0.024|0.542 ± 0.006|**24.13 ± 1.47**|14.63 ± 1.16|\n|MLDE | *0.652 ± 0.059*| *0.572 ± 0.035*|13.10 ± 1.18| *21.68 ± 3.85*|\n|LatProtRL|0.560 ± 0.000|0.508 ± 0.003|2.24 ± 0.14| 1.78 ± 0.16|\n|ProSpero|**0.672 ± 0.031**|**0.599 ± 0.014**|*14.51 ± 1.99*|**22.03 ± 1.69**|\n\nAdditionally, following the suggestion of Reviewer XJMJ, we repeated the experiment under the first distribution shift setting. To better reflect real-world constraints, we further limited the initial dataset by randomly selecting only 200 data points from $D_0$, resulting in a reduced initial training set with an average pTM score of $0.860±0.023$. \n\n|Method| Maximum pTM| Mean pTM| Diversity| Novelty|\n|-|-|-|-|-|\n|AdaLead|0.781 ± 0.016|0.742 ± 0.004|7.99 ± 1.39|5.95 ± 2.04|\n|PEX|*0.806 ± 0.013*|*0.752 ± 0.005*|6.77 ± 0.45|4.39 ± 0.51|\n|GFN-AL-$\\delta$CS|0.782 ± 0.006|0.731 ± 0.006|**15.82 ± 1.77**|9.46 ± 1.69|\n|MLDE|0.782 ± 0.022|0.735 ± 0.025|9.39 ± 2.88|**16.97 ± 3.78**|\n|LatProtRL|0.792 ± 0.013|0.743 ± 0.001|6.25 ± 0.23| 5.57 ± 0.20|\n|ProSpero|**0.808 ± 0.017**|**0.763 ± 0.017**|*11.25 ± 2.51*|*15.87 ± 1.17*|\n\n\nThese results demonstrate that our approach can robustly guide exploration toward high-fitness sequences, even under substantial distributional shifts, consistently outperforming competing methods. Moreover, in the setting of the more severe covariate shift with 75 mutations from the wild-type (which is roughly equivalent to half of the entire sequence), the performance gap between our approach and the majority of the baselines widens substantially. \n\nWe trust that this additional evaluation directly addresses the reviewer’s concern about insufficient distributional shift in our original experiments. All three tables with the results for these novel landscapes with distribution shifts will be incorporated in the paper.\n\n\nThank you for your constructive and meaningful feedback, which has helped strengthen the manuscript. We hope our responses have addressed your concerns, and would greatly appreciate it if you would consider **raising your rating of our submission** accordingly.\n\n#### References\n[1] A. N. Amin et al. Bayesian optimization of antibodies informed by a generative model of evolving sequences. ICLR 2025.\n\n[2] L. Wu et al. Practical and Asymptotically Exact Conditional Sampling in Diffusion Models. NeurIPS 2023.\n\n[3] Y. Liang et al. Research Progress of Reduced Amino Acid Alphabets in Protein Analysis and Prediction. Computational and Structural Biotechnology Journal, 2022."}, {"review_id": "zLZzijlibO", "reviewer": "Reviewer_HtS2", "summary": "PROSPERO is method for black-box optimization of proteins that combines biological priors extracted from a pretrained evolutionary diffusion model (EvoDiff) and human knowledge (e.g. Restricted amino-acid classes based on biochemical properties), with a sampling approach that operates in a restricted exploration space observing the biologically plausible priors. The method is tested on a number of in silico benchmarks for black box biological sequence design and is shown to perform well on optimization and diversity metrics within these benchmarks against standard methods in the field.", "questions": "I suggest the authors develop the paper further by doing an in vitro experiment which can be done by an outside entity or collaborative lab (costs the same as 1000-10000 GPU hours), even for one empirical round. \n\nOther than the in vitro validation, the main challenge/remedy or contribution would be to find protein optimization landscapes that show real distribution shift, and non-smooth optimization surfaces. Unfortunately at this time FLEXS and similar in silico landscapes are too low of a bar given what we know (helpful as a first check but not sufficient). The authors could think more critically about how to construct such landscapes, for instance using the Alphafold metrics and if they can make a convincing case about distribution shift in these landscapes, that's both a strong contribution and changes my view of the paper's promise.", "limitations": "yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n\nThe paper’s core thesis and application domain are relevant and correct. It should be helpful to incorporate biological priors into exploration algorithms, especially when one seeks to explore variants farther from the wild-type. \n\nIn terms of in silico benchmarking the work is well done and covers the important classes of algorithms. It also includes reasonable diversity metrics, which the method seems to perform well on. \n\nThe method also seems to be robust to noise, which is highly valuable. \n\nThe study execution within the in silico space of datasets, metrics, methods and evaluation is strong. \n\nWeaknesses\n\nMy major concerns about the work is that positioning itself as a method for “far from wild type” design, makes it more necessary to show at least one empirical result. The standard has raised in the past 3-4 years on design algorithms to move beyond in silico optimization, and empirical validation is also relatively cheap and fast. See Gruver et al Protein Design with Guided Discrete Diffusion, NeurIPS 2023. It is very hard to evaluate methods that claim to improve OOD design without any real-world validation, which while correlated with in silico, does not show state of the art performance. While the approach is entirely plausible, whether the biological priors actually help in design is a question you’d only answer if you evaluate it with real biology. I think the improvement offered here (in optimization performance in silico) is insufficient to convince me it will be an advantage in the real world. For instance Damani et al Beyond the training set.. 2023 (https://arxiv.org/pdf/2311.05363), claim almost 100% of samples far from WT that score highly on a surrogate model similar to those are broken in real life. My experience is consistent with this view and I while I think the priors incorporated in this method may help this problem, there is no clear evidence it will. \n\nMore granularly, for instance, the AAV landscape from FLEXS that is used for evaluation is a synthetic landscape, not a real one. It is built from summing the effects of single residues (which were measured). This is a convex landscape with one peak that any hill climbing algorithm can eventually (if inefficiently solve). No real biology is this way. So showing that the algorithm can climb this hill is necessary but entirely insufficient in showing that it can design sequences far from the wild type. There is also no reason why the priors incorporated into the model should help in climbing this landscape, because it is only mildly connected to those biological rules. \n\nAs mentioned above the landscapes used here do not show strong distributional shifts.  Without directly addressing this point, the main claim of the paper on its ability to do optimization far from wild type is unconvincing. \n\nSecondarily most protein design today occurs within 2-4 active learning rounds. Improved performance in that regime is far more valuable than performance improvements over more than 7 rounds of improvement", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their thoughtful and detailed feedback.\n\n## Regarding the primary concern about the lack of wet-lab validation\nWe share the reviewer’s view on the importance of real-world validation, yet want to emphasize that experimental testing is beyond the scope of our current work. Here, we focus on establishing the methodological foundation of active learning-based, robust protein design, evaluating it on a wide variety of benchmarks. At the same time, we respectfully suggest that the absence of wet-lab results should not diminish the core methodological contribution of our work.  As the reviewer acknowledged, the paper’s central thesis and application domain are relevant, and our *in silico* benchmarking is strong and comprehensive.\n\nWe thank the reviewer for highlighting the important and relevant work by Damani et al. [1]. We fully agree that great care must be taken when interpreting surrogate model predictions in regions far from the training distribution. That said, we believe the findings in Damani et al. [1] are not directly transferable to our setting. Their study and its empirical results focus on *offline* MBO. In contrast, our approach operates in an *online* active learning setting, where the surrogate is updated with oracle feedback. Crucially, it has been shown that active learning can significantly improve extrapolation and generalization in OOD regimes by enabling the model to iteratively incorporate data from previously unsupported areas of the search space [2]. We understand, however, that as long as the oracle is not a direct measure of biological function, concerns about over-optimism in unexplored regions remain justified.\n\n## Regarding the landscapes, in particular AAV\n\nWhile indeed the AAV landscape is convex in the noise-free setting, it constitutes only one of the eight tasks we evaluate on. The remaining 7 landscapes are non-convex, rugged and more representative of the complexity of real biological functions. Moreover, in the noisy setting, the AAV landscape loses its convexity and resembles the Rough Mt. Fuji model, which has been shown to capture locally valid biological behavior around wild-type sequences in empirical studies [3]. We acknowledge the reviewer’s concern that success on this task alone may not be sufficient to support claims about generalization beyond wild-type neighborhoods. However, particularly in the noisy setting, it remains a valuable component of the evaluation suite.\n\n\n## Regarding the insufficient distribution shifts on evaluated tasks\nWe appreciate and acknowledge the reviewer's concern regarding the degree of distributional shift present in our evaluation tasks. To address this more directly, following helpful suggestions of the reviewer as well as Damani et al [1], we introduced another 3 tasks evaluating effectiveness of our method under various degrees of distribution shift.  In these experiments we used ESMFold as the oracle, which offers high-fidelity feedback better suited for performance assessment in more challenging and realistic settings.\n\nSpecifically, from all UBE2I candidate sequences generated by different methods and scored with ESMFold, we selected those within a Hamming distance $\\leq 5$ from the UBE2I wild-type to construct the initial dataset $D_0$, consisting of 2624 sequence---pTM score pairs. The average pTM score of $D_0=0.860 ± 0.029$. We then trained a surrogate model $f_\\theta$ on $D_0$, and initiated the optimization not from the wild-type sequence, but from a sequence differing from the wild-type by a large number of 35 residues and of an initial pTM score of 0.70. Following the insightful suggestion of the reviewer, we run such optimization for only 4 iterations. The results are shown below:\n\n| Method | Maximum pTM|Mean pTM|Diversity| Novelty|\n|-|-|-|-|-|\n| AdaLead |0.796 ± 0.013|0.755 ± 0.011|8.83 ± 2.54| 8.36 ± 2.97|\n| PEX |0.807 ± 0.023|*0.760 ± 0.012*|6.14 ± 0.89| 4.45 ± 0.38|\n|GFN-AL-$\\delta$CS |0.791 ± 0.010|0.729 ± 0.005| **16.92 ± 0.88**|9.56 ± 0.60|\n|MLDE|*0.810 ± 0.020*|0.752 ± 0.004|9.89 ± 1.11 |**20.88 ± 2.98**|\n| LatProtRL |0.787 ± 0.013|0.743 ± 0.003|6.32 ± 0.32| 5.90 ± 0.53|\n|ProSpero|**0.822 ± 0.027**|**0.777 ± 0.020**| *11.50 ± 1.62*|*17.74 ± 3.20*|\n\nwhere by novelty here we mean the distance of top 100 sequences to the starting point, not the original wild-type.\n\nMoreover, we additionally repeated the same experiment, but under an even more severe distribution shift, starting optimization from a sequence differing from the wild-type by a larger number of 75 residues and of an initial pTM score of 0.50. The results are shown below:\n\n| Method | Maximum pTM| Mean pTM| Diversity| Novelty|\n|-|-|-|-|-|\n|AdaLead | 0.593 ± 0.028| 0.526 ± 0.007| 14.26 ± 1.91| 7.66 ± 1.08|\n|PEX | 0.578 ± 0.014|0.518 ± 0.003|3.40 ± 0.07| 1.72 ± 0.04|\n|GFN-AL-$\\delta$CS|0.630 ± 0.024|0.542 ± 0.006|**24.13 ± 1.47**|14.63 ± 1.16|\n|MLDE | *0.652 ± 0.059*| *0.572 ± 0.035*|13.10 ± 1.18| *21.68 ± 3.85*|\n|LatProtRL|0.560 ± 0.000|0.508 ± 0.003|2.24 ± 0.14| 1.78 ± 0.16|\n|ProSpero|**0.672 ± 0.031**|**0.599 ± 0.014**|*14.51 ± 1.99*|**22.03 ± 1.69**|\n\nAdditionally, following the suggestion of Reviewer XJMJ, we repeated the experiment under the first distribution shift setting. To better reflect real-world constraints, we further limited the initial dataset by randomly selecting only 200 data points from $D_0$, resulting in a reduced initial training set with an average pTM score of $0.860±0.023$. \n\n|Method| Maximum pTM| Mean pTM| Diversity| Novelty|\n|-|-|-|-|-|\n|AdaLead|0.781 ± 0.016|0.742 ± 0.004|7.99 ± 1.39|5.95 ± 2.04|\n|PEX|*0.806 ± 0.013*|*0.752 ± 0.005*|6.77 ± 0.45|4.39 ± 0.51|\n|GFN-AL-$\\delta$CS|0.782 ± 0.006|0.731 ± 0.006|**15.82 ± 1.77**|9.46 ± 1.69|\n|MLDE|0.782 ± 0.022|0.735 ± 0.025|9.39 ± 2.88|**16.97 ± 3.78**|\n|LatProtRL|0.792 ± 0.013|0.743 ± 0.001|6.25 ± 0.23| 5.57 ± 0.20|\n|ProSpero|**0.808 ± 0.017**|**0.763 ± 0.017**|*11.25 ± 2.51*|*15.87 ± 1.17*|\n\n\nThese results demonstrate that our approach can robustly guide exploration toward high-fitness sequences, even under substantial distributional shifts, consistently outperforming competing methods. Moreover, in the setting of the more severe covariate shift with 75 mutations from the wild-type (which is roughly equivalent to half of the entire sequence), the performance gap between our approach and the majority of the baselines widens substantially. \n\nWe trust that this additional evaluation directly addresses the reviewer’s concern about insufficient distributional shift in our original experiments. All three tables with the results for these novel landscapes with distribution shifts will be incorporated in the paper. We hope the reviewer will see this extended evidence and evaluation procedure as a meaningful contribution and reconsider their assessment of the paper.\n\n\n## Regarding the performance within early active learning rounds\n\nWe thank the reviewer for raising this important and practically relevant point about the typical number of active learning rounds used in real-world protein design workflows. Due to space constraints, we present only the most relevant metric below, namely the maximum fitness achieved within 4 iterations. The revised version of the manuscript will include the full set of results limited to 2 to 4 iterations across all benchmarks.\n\n| Method|AMIE|TEM|E4B| Pab1|AAV|GFP|UBE2I|LGK|\n|-|-|-|-|-|-|-|-|-|\n| AdaLead    | 0.232 ± 0.003 | 1.227 ± 0.004 | 7.962 ± 0.071 | **1.397 ± 0.329** | 0.596 ± 0.014 | 3.580 ± 0.003 | 2.982 ± 0.002 | *0.032 ± 0.002* |\n| PEX        | **0.242 ± 0.001** | **1.231** ± 0.001 | *7.971 ± 0.078* | 1.064 ± 0.071 | *0.604 ± 0.018* | *3.597 ± 0.002* | *2.987 ± 0.002* | 0.030 ± 0.001 |\n| GFN-AL-δCS | 0.160 ± 0.048 | 0.563 ± 0.119 | 7.859 ± 0.047 | 1.035 ± 0.094 | 0.596 ± 0.010 | 3.584 ± 0.005 | 2.972 ± 0.013 | 0.032 ± 0.002 |\n| MLDE       | 0.231 ± 0.006 | *1.229 ± 0.000* | 7.821 ± 0.063 | 0.866 ± 0.024 | 0.555 ± 0.000 | 3.589 ± 0.003 | 2.978 ± 0.000 | 0.028 ± 0.003 |\n| LatProtRL  | 0.224 ± 0.000 | *1.229 ± 0.000* | 7.751 ± 0.016 | 1.031 ± 0.129 | 0.565 ± 0.011 | 3.589 ± 0.003 | 2.982 ± 0.001 | 0.020 ± 0.000 |\n| ProSpero   | *0.236 ± 0.007* | *1.229 ± 0.001* | **7.978 ± 0.055** | *1.202 ± 0.129* | **0.635 ± 0.019** | **3.602 ± 0.002** | **2.989 ± 0.002** | **0.036 ± 0.001** |\n\nNotably, even when restricted to a smaller number of iterations, our approach consistently ranks first or second across all tasks, highlighting its effectiveness in low-iteration regimes commonly encountered in practice, as rightly pointed out by the reviewer.\n\n \nWe once again sincerely thank the reviewer for their thoughtful and constructive feedback, which helped us identify and address a key shortcoming in the original submission. In particular, the suggestion to better evaluate performance under distributional shift motivated the introduction of three new tasks designed to test robustness in more realistic and challenging OOD settings. We believe these additions significantly strengthen the paper and directly respond to the concerns raised. We hope the reviewer finds that these revisions adequately address their main points, and we would be grateful if they would consider **updating their evaluation of the work in light of these improvements**.\n\n#### References\n[1] F. Damani et al. Beyond the training set: an intuitive method for detecting distribution shift in model-based optimization. 2023.\n\n[2] E. R. Antoniuk et al. Active Learning Enables Extrapolation in Molecular Generative Models. 2025.\n\n[3] S. Sinai et al. AdaLead: A simple and robust adaptive greedy search algorithm for sequence design. 2020."}, {"review_id": "PzH3zzUUgx", "reviewer": "Reviewer_Dp5Q", "summary": "ProSpero introduces an iterative protein design method that goes beyond recent work PEX – that relies on local search, using a pretrained generative model to explore beyond a protein wild-type neighborhood. The pretrained generative model is used as a prior to guide the exploration, whilst proposing biologically plausible and diverse candidates in each round. \n\nProSpero is evaluated in eight multi-round design campaigns, where ground truth scores are taken from the TAPE model, and the ensemble surrogate is fine-tuned in each round. The paper introduces two methodological contributions: a targeted masking strategy to identify high-value residues to mutate, and a constrained SMC sampling process.", "questions": "RAAs: In practice, what is the impact of using the RAA based on charge particles? Was an ablation performed where you simply sample from the full AA alphabet? \n\nWhat value of n_sub was used in practice? Is this HP ablated? i.e. how many positions do you mask and the length of $\\mathcal{I}^{(i)}$? and what effect does this have on the novelty and quality of generated sequences? What is the sensitivity of this HP? My intuition is that when set too high or low, the sequences will score poorly for novelty. How was the HP set? \n\nLine 147 defines $\\pi^(i)$  as the order agnostic indices, why is it concatenated in that specific order? What is the intuition for why one would want to sample from I_neg before I_pos, followed by I_netural? Did you ablate or experiment with other potential orderings?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe paper is well written and organised. It was an enjoyable read. The reported results are good relative to benchmark methods, plus reproducibility is high, with open source code provided for the main ProSpero method and all baselines. \nFurther, efforts have been made to generate biological meaningful sequences, via constrained search (using AA charge classes), and also evaluating the generated sequences via biological validity metrics based on physicochemical properties. \n\nWeaknesses: \nProblem formulation: As far as I understand, the goal of ProSpero is to design protein sequences with high fitness values across $N$ iterations. This feels like an optimisation objective, not an active learning one - where we are trying to improve a model. \nPlease could you untie or explain the difference between your proposed active learning formulation, and a Bayesian optimization objective? It would appear to me that you have an optimisation objective, unless I am missing something. \n\nOn Clarity: \nI am confused exactly what distribution your sampling algorithm targets, and further, under what assumptions ProSpero does indeed target that distribution. Although clearly an effective proxy, it is not obvious to me what distribution the importance weights imply. \nOn the underlying assumptions: I suspect the sampling scheme is closely related to the field of pseudo-marginal MCMC (see Andrieu et al. (2009)), wherein an unbiased estimate of the unnormalised posterior is used in place of the underlying unnormalised density; precise details of this connection to such work would be appreciated. \nGenerally, I would greatly appreciate more details/precision with respect to your SMC scheme.\n\nOn novelty: \nThe significance of the proposed Targeted Masking contribution appears limited. To me this feels like random AA permutation, plus a surrogate and acquisition function applied on top.  \n\nPlease elucidate the connections between [26] and your proposed method.\nTo my understanding, CloneBO performs a similar *iterative design of protein sequences as inference-time guidance of a pre-trained generative model*, and performs the guidance *via conditioning on high value sequences*. Is there a connection between the high value sequences used to guide the tSMC samples in CloneBO and your use of a surrogate model? \n\n[26] Alan Nawzad Amin, et. al. Bayesian optimization of antibodies informed by a generative model of evolving sequences. ICLR 2025.\n\nIt would be of benefit if the authors situate their guidance method relative to other generative guidance methods, such as classifier guidance.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your meaningful and relevant feedback of our work.\n\n## W1: Problem formulation\nWe agree that the ultimate objective of our framework is optimization of protein sequences, not active learning in the traditional sense of model improvement for its own sake. Rather, the latter serves as a proxy objective to achieve the former. We appreciate the opportunity to clarify this point and will revise the manuscript to reflect this more precise framing.\n\n## W2: Clarity\nWhile full SMC details are provided in Appendix C, we agree that being more precise would improve clarity. In the revised version, we will incorporate the following changes:\n- In Equation 1 of Section 4.2, we will revise the notation of the target distribution from $\\gamma(x)$ to $\\gamma(x_{1:L})$ , and update all other occurrences of $x$ accordingly within the introductory part of Section 4.2. This change makes explicit that the target is defined over a sequence of residues, improving clarity and alignment with the sequential structure of SMC\n- We will begin the L144-167 part with the following paragraph: Rather than trying to directly sample from complex, high-dimensional target distribution $\\gamma(x_{1:L})$, in ProSpero we perform approximate inference using SMC, which decomposes the sampling process into a sequence of simpler, unnormalized, intermediate target distributions $\\{\\tilde{\\gamma}(x_{1:l})\\}_{l=1}^{L}$ (for background see Appendix C). This allows us to progressively build up sequences residue by residue, while maintaining a tractable approximation of the full design objective\"\n- We will conclude the line 162 as follows: “Finally, the unnormalized importance weights […] , with the perplexity correction term penalizing sequences that are improbable under the true prior, correcting bias introduced by the constrained sampling”\n\nWe hope this revision will make the section clearer and more accessible to the readers.\n\nImportance weights are proportional to the likelihood, as at each step they represent the ratio between the target distribution (i.e. posterior) and the proposal (prior). Resampling particles proportionally to their weights allows us to progressively concentrate samples in high-probability regions under the posterior.\n\nAs for the connections between pseudo-marginal MCMC and SMC: in pseudo-marginal MCMC, within a single Markov chain, a *single* collection of samples is generated whose marginal stationary distribution is exactly the target distribution. The approximation improves with the number of iterations $T$ and is exact in the limit of *infinite $T$*. In contrast, SMC maintains a *population* of $N$ samples that evolve over a sequence of intermediate distributions. The empirical distribution formed by these samples converges to the target distribution in the limit of *infinite $N$*.\n\n## W3: Novelty\nThe motivation behind targeted masking is a biologically motivated strategy that mirrors how experimentalists scan for functionally relevant residues. We demonstrated its effectiveness over random masking in Figure 4D in Section 5.4. For convenience, we summarized the key results in the table below:\n\n|SNR level|-25|-20|-15|-10|-5|0|\n|-|-|-|-|-|-|-|\n|ProSpero|**0.566 ± 0.03**|**0.586 ± 0.026**|**0.651 ± 0.032**|**0.679 ± 0.047**|**0.704 ± 0.016**|**0.706 ± 0.022**|\n|ProSpero w/o TM|0.554 ± 0.021|0.581 ± 0.028|0.597 ± 0.027|0.672 ± 0.021|0.677 ± 0.030|0.649 ± 0.028|\n\nResults clearly demonstrate the advantage of targeted over random masking across all noise levels. To improve the clarity of the strategy behind targeted masking, we will improve its description in Section 4.1 in the final paper.\n\n## W4: Similarities to CloneBO by Amin et al. [1]\n\nWe agree that ProSpero and CloneBO [1] share similarities in guiding a generative model at inference time using SMC. However, the approaches differ meaningfully in both scope and mechanism.\nFirst, in CloneBO, the generative model (CloneLM) has been trained *specifically* for the optimization task by fiting to a distribution of clonal families. In contrast, our method uses a *general-purpose, task-agnostic* pre-trained generative model, enabling effortless optimization regardless of the protein family.\nAs for the differences in the use of SMC, in CloneBO the authors compute intermediate importance weights directly, as a *likelihood ratio* between the base CloneLM and a twisted variant incorporating high value sequences (i.e. sequences with experimental measurements) in the conditioned clonal family. In ProSpero, we approximate the intermediate importance weights using the *surrogate model*, which serves as a tractable proxy for the true likelihood. Moreover, our biologically constrained SMC restricts proposals to charge-compatible amino acids, making certain residues impossible to sample. In contrast, CloneBO does not enforce such constraints; although twisting reduces the probability of sampling undesirable residues, it does not eliminate them entirely.\n\n>\"It would be of benefit if the authors situate their guidance method relative to other generative guidance methods [...]\"\n\nThank you for this helpful suggestion. We agree that situating our method in relation to existing guidance techniques would help readers better understand our contribution. In the revised version of the manuscript, we will include the following paragraph and expand it further with additional references and examples to strengthen its context and relevance. \n\nClassifier guidance is a technique used to steer generative models during inference, by incorporating gradients from an auxiliary classifier. The sampling is biased toward desired properties by adjusting the generative score with the gradient of the proxy classifier $\\nabla_x \\log p(y \\mid x)$ [2]. However, this approach is not directly applicable in the discrete data domain, where gradients with respect to input tokens are not well-defined. Recently, Nisonoff et al. [3] proposed a technique allowing for classifier guidance in discrete diffusion and flow matching models. Discrete-domain guidance can also be addressed using SMC-based approaches, which guide sequence generation by maintaining a population of partial samples and resampling them based on their likelihood under a target distribution. \n\n\n> RAAs: In practice, what is the impact of using the RAA based on charge particles? [..]\n\nIn our ablation study in Section 5.4, we already compare sampling from EvoDiff with and without RAAs constraints. For convenience, as well as given that both of those ablations are constant across noise levels, we summarized the results in the table below:\n\n|Method|Maximum fitness|Mean fitness|Diversity|Novelty|\n|-|-|-|-|-|\n|Evodiff|**0.570 ± 0.039**|**0.498 ± 0.024**|**7.83 ± 0.45**|**7.71 ± 1.18**|\n|Evodiff w/o RAA| 0.509 ± 0.012|0.356 ± 0.011|6.86 ± 0.50|4.11 ± 0.82|\n\nMotivated by your remark, we will extend our ablation by directly comparing the performance of ProSpero with and without RAAs constraints. The results are shown below:\n\n|SNR level|-25|-20|-15|-10|-5|0|\n|-|-|-|-|-|-|-|\n|ProSpero|**0.566 ± 0.030**|**0.586 ± 0.026**|**0.651 ± 0.032**|**0.679 ± 0.047**|**0.704 ± 0.016**|**0.706 ± 0.022**|\n|ProSpero w/o RAA|0.507 ± 0.025|0.555 ± 0.040|0.588 ± 0.029|0.605 ± 0.032|0.653 ± 0.042|0.666 ± 0.029|\n\nBoth of these tables demonstrate the relevance of RAA constraints, particularly with respect to fitness and novelty. Thank you for prompting this valuable addition.\n\n> What value of n_sub was used in practice? [...]\n\nThe number of masked sites in each benchmark was determined based on the sequence length, with the number sampled uniformly from the range of 5–15 residues for longer, and 3–10 for shorter sequences. The 5–15 residue masking range reflects common practice in rational design (e.g., for the GFP [4]) and was shown to perform well in silico by Kim et al. [5], who also explored masking 5% of residues. Based on this proportion, we reduced the range to 3–10 residues for shorter sequences like AAV, E4B, and Pab1. The reviewer’s intuition is correct, this parameter directly modulates novelty by controlling how far generated sequences deviate from the wild-type.\n\n> Line 147 defines $\\pi^{(i)}$ as the order agnostic indices, why is it concatenated in that specific order? [...]\n\nThe permutation order was chosen to prioritize charge classes with fewer valid options. The intuition behind this choice is that resolving the most constrained decisions early can help avoid suboptimal completions later in the sequence. We compared different orderings and summarized the results in the table below:\n\n|Ordering|AMIE|TEM|E4B|Pab1|AAV|GFP|UBE2I|LGK|\n|-|-|-|-|-|-|-|-|-|\n|Ascending |**0.246 ± 0.006**|1.231 ± 0.002|8.114 ± 0.037|**1.527 ± 0.254**|**0.720 ± 0.027**|**3.617 ± 0.003**|**2.993 ± 0.003**|**0.043 ± 0.002**|\n|Descending|0.244 ± 0.005|**1.232 ± 0.003**|8.139 ± 0.037|1.363 ± 0.141|0.706 ± 0.035|3.614 ± 0.003|2.991 ± 0.003|0.041 ± 0.003\n|Random|0.243 ± 0.005|**1.232 ± 0.002**|**8.164 ± 0.015**|1.338 ± 0.113|0.708 ± 0.029|3.614 ± 0.003|**2.993 ± 0.002**|0.042 ± 0.003|\n\nWhile the results suggest that the intuition is correct, the differences are modest, especially for longer sequences. Thank you for bringing attention to this aspect.\n\nThank you for the thoughtful and constructive feedback, which helped us improve the manuscript. We hope our revisions address your concerns and would greatly appreciate it if you would consider **updating your evaluation of our submission**.\n\n#### References\n[1] A. N. Amin et al. Bayesian optimization of antibodies informed by a generative model of evolving sequences. ICLR 2025\n\n[2] P. Dhariwal and A. Nichol. Diffusion Models Beat GANs on Image Synthesis. NeurIPS 2021\n\n[3] H. Nisonoff. Unlocking Guidance for Discrete State-Space Diffusion and Flow Models. ICLR 2025\n\n[4] T. Hayes et al. Simulating 500 million years of evolution with a language model. Science 2025\n\n[5] H. Kim et al. Improved off-policy reinforcement learning in biological sequence design. 2024"}]

Author rebuttal:

Meta-review: This paper introduces ProSpero, a novel active learning framework for protein design that aims to generate sequences with both high fitness and novelty by exploring beyond the local neighborhood of wild-type sequences. The method combines a pre-trained generative model with a surrogate updated via oracle feedback, using a biologically-inspired targeted masking strategy and constrained Sequential Monte Carlo sampling. The primary concern is a critical and unresolved mismatch between the paper's central claim and its validation. The paper claims to enable "robust protein design beyond wild-type neighborhoods," a significant challenge in the field. As pointed out forcefully by Reviewer HtS2, this claim cannot be substantiated by in silico benchmarks alone, as even state-of-the-art oracles like ESMFold are known to be insufficiently rugged and poor proxies for real biological function far from the training distribution.
Decision: Accept (poster)