Amortized Active Generation of Pareto Sets
Authors: Daniel M. Steinberg
Asiri Wijesinghe
Rafael Oliveira
Piotr Koniusz
Cheng Soon Ong
Edwin V. Bonilla
Venue: NeurIPS 2025 poster ()
Keywords: multi-objective optimization
multi-objective generation
black-box optimization
guided generation
multi-objective Bayesian optimization
discrete optimization
variational inference
reinforce
active generation
online black-box optimization
Primary area: probabilistic_methods
TL;DR: We learn a generative model of the Pareto set that can be conditioned on subjective preferences, without retraining, for online multi-objective optimization tasks on discrete/mixed spaces.

Abstract: We introduce active generation of Pareto sets (A-GPS), a new framework for online discrete black-box multi-objective optimization (MOO). A-GPS learns a generative model of the Pareto set that supports a-posteriori conditioning on user preferences. The method employs a class probability estimator (CPE) to predict non-dominance relations and to condition the generative model toward high-performing regions of the search space. We also show that this non-dominance CPE implicitly estimates the probability of hypervolume improvement (PHVI). To incorporate subjective trade-offs, A-GPS introduces _preference direction vectors_ that encode user-specified preferences in objective space. At each iteration, the model is updated using both Pareto membership and alignment with these preference directions, producing an amortized generative model capable of sampling across the Pareto front without retraining. The result is a simple yet powerful approach
that achieves high-quality Pareto set approximations, avoids explicit hypervolume computation, and flexibly captures user preferences. Empirical results on synthetic benchmarks and protein design tasks demonstrate strong sample efficiency and effective preference incorporation.

Amortized Active Generation of Pareto Sets
Daniel M. Steinberg'
Asiri Wijesinghe'
Rafael Oliveira
Piotr Koniusz!.23 Cheng Soon Ong 1.3
Edwin V. Bonilla'
CSIRO's Data61 UUiiversity New South Wales
Australian National University
Abstract
We introduce active generation of Pareto sets (A-GPS), a new framework for online discrete black-box multi-objective optimization (MOO). A-GPS learns a generative model of the Pareto set that supports a-posteriori conditioning on user preferences. The method employs class probability estimator (CPE) to pre- dict non-dominance relations and condition the generative model toward high- performing regions of the search space. We also show that this non-dominane CPE implicitly estimates the probability of hypervolume improvement (PHVI). To in- corporate subjective trade-offs, A-GPS introduces preference direction vectors that encode user-specified preferences in objective space. At each iteration, the model is updated using both Pareto membership and alignment with these preference directions, producing an amortized generative model capable of sampling across the Pareto front without retraining. The result is simple yet powerful approach that achieves high-quality Pareto set approximations, avoids explicit hypervolume computation, and flexibly captures user preferences. Empirical results on synthetic benchmarks and protein design tasks demonstrate strong sample efficiency and effective preference incorporation.
1 Introduction
In many scientific and engineering domains, practitioners face the challenge of optimizing complex, nigh-dimensional, discrete objects under expensive black-box evaluation processes. Examples include designing protein sequences for enhanced stability and activity, synthesizing small molecules with tailored pharmacokinetics, and engineering DNA constructs for precise gene regulation. In these settings, each candidate design must be evaluated via computationally intensive simulations laboratory assays, making efficient search strategies essential. Furthermore, these applications frequently involve multiple, often conflicting objectives. For instance, in protein engineering one may wish maximize thermal stability, catalytic turnover rate, and expression yield, yet improvements in one property can degrade another. The set of non-dominated trade-off designs (where no objective can improved without sacrificing performance in least one other objective) is known as the Pareto set. Accurately approximating this Pareto set is critical for enabling informed decision-making in downstream experimental workflows.
Traditional Bayesian optimization (BO) methods for black-box multi-objective optimization (MOO) problems, often referred to multi-objective Bayesian optimization (MOBO), rely acquisition functions such as expected hypervolume improvement (EHVI) [42, 10, 2] or their quasi-MonteC:: extensions [11, 2), which involve complex numerical integration and/or scale poorly with the number of objectives. Alternatively they may rely on random scalarizations [27, 31, 13] offering simplicity and scalability but rely on sufficient sampling density capture complex Pareto front geometries.
this work we propose a fundamentally different approach, in the spirit of multi-objective generation (MOG) [44, 43], that directly estimates generative model of the Pareto set in online sequential,
corresponning author: dan. www.sserrgdaaaaa....... csiro.an
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

fx())
Figure I: A visualization of a Pareto front, Framo and the random variables used with A-GPS. y noisy realizations
objectives, f.. When 2 1 these observations lie on the Pareto front. Preference direc- tion vectors, u. are unit vectors pointing to region of the Pareto front from reference point, Equa- tion 11. We derive aligned
Image of go (x(x
1) training pref- erence direction vectors, Uh; from observation pairs (yn: Xn), and mis-aligned preference direction vectors from permuting these pairs, (y p(n): Xn), Equation 18. The aim is to learn the distribution of the Pareto set (x(()) p(x)u, 2 1, = 1).
F Pareto
u: 0
p(u[2)
r= (0,0)
fx(
black-box optimization setting. To this end, build upon the recently proposed variational search distributions (VSD) framework for single-objective optimization [39]. VSD formulates optimization as active learning of generative model of high-perfomming designs (active generation). Thus, VSD alternates between fitting a class probability estimator (CPE) to discriminate favorable designs and updating a conditional generative model to propose new candidates directly. Hence, instead of using a fitness-threshol probability of improvement (PI)-CPE, we propose a Pareto-set CPE, which enables the generative model to focus exclusively on non-dominated designs. This approach bypasses explicit hypervolume computations and scalarizations and leads to scalable online algorithm for Pareto set approximation that is different from other recent MOG methods [44, 43]. Furthermore, we show that such non-oomnnacce CPE is implicitly estimating probability of hypervolume improvement (PHVI). Additionally, practical decision-makiin over the trade-offs among objectives. We introduce novel mechanism for embedding subjective preference conditioning into the generative model, allowing practitioners to sample candidates that align with specified trade-off directions. By using amortized variational inference (VI) with preference direction vectors, our method supports a- a-posteriori preference specification without retraining, offering flexibility in downstream design exploration.
often requires incorporating user or stakeholder preferences
We show that our method, active generation of Pareto sets (A-GPS), performs well against competing approaches on suite of challenging synthetic and real multi-objectio optimization benchmarks.
Preliminaries
2.1 Active generation
Active generation implemented by [39] reframes online black-box optimization sequential learning of a conditional generative model, guided by CPE. At each round, t E (1....,Tt T} we: (I) fit CPE (using some proper loss, LCPE),
T (x) R p(= 1|x),
parameterized by 0 and where = 1 |x E S) indicates membership in some desired set, S. For example, designs fitter than some incumbent, x", under some black box function, f. : X + R;
S = (x X fi(x) > f((x*)) Then (2) update the generative model qo (x), e.g. by minimizing the reverse Kullback-Leible (KL) divergence to the ideal conditional, p(x[z = 1). or equivalently maximizing the evidence lower bound (ELBO),
LELBO (1), 0)
Eag (x) [log कद (x)) [x
where (D)) is a prior over the design space. Formally, using data D = {(X,, 2n) 2 active generation optimizes,
8, < argmin (fr (f, D
of t- argmax (o 0,),
(3)
then samples from yo; (x) are used to propose new candidates for evaluation. New labels are acquired for these candidates, the dataset is augmented, and the process is repeated until convergence. This solution to active generation is referred to as VSD [39]. Under certain assumptions on the form of the models, this procedure has proven convergence rates to the ideal p(x|2 1).

---

2.2 Optimizing over multiple objectives
In this work we are concerned with generating discrete or mixed discrete-contin continuous designs, for example sequences x E X' = yM, where V is the sequence vocabulary and M is the sequence length, that have particular measurable properties y E R., We assume the 'black-box' relationship [f.' (x),...: f. (x),....
y = f.(x) + where fx(x)
(x)) and [p(e)[c] O. The black-box function (-) could be a noisy empirical observation, or an expensive physics/chemistry simulation (where e = 0), etc. In MOO we would like to find the global optimum.
max (x))..(x). XEX
There are number of issues that present themselves here though. Firstly, we cannot use gradient based optimization methods directly since we cannot access >x (x) as f.' are black-boxes and x is (partially) discrete. But more importantly, max not uniquely defined for the vector valued as the individual objectives can be in conflict with one (or more formally, there is no total ordering). Instead, we are interested in finding the set of designs for which we cannot increase one objective without compromising others. This is known as the Pareto set Parcto C X.
Pareto == (x x' o x, Vx EX),
(5)
where x' > refers to x' dominating x, i.e., all of the objective function values for x' are greater than or equal to those of x, and at least one is greater,
iff f'((x) (x) AI E (1,..., T.} and E1 (1,.... :.......)) L.) such that f'(x') > fx().. (6)
The Pareto set also induces the Pareto front, Faareto = (f.(x): Vx E Spacem which is the image of the Pareto set outcomes in R._. For data DN = {y k)) at round / we define the current observable Pareto set,
Spancio = {x Xy
Vi E (1, (I,...,,N,, N}. EL. NN..,
where Xj
V) 2 of W (I...., L} and 31 E (1....,L) L} such that v/ vt.
We define a Pareto set membership (or non-dominance) label 2n == 1 [x,, E Sparetol. where [[]: (True, False) - {1, 0}. which will use for training the CPE in active generation. This definition can be extended to the whole domain X' as a labeling function =xx) := 1[x E Parcto(Spare) U {x})] where PS) denotes the Pareto subset of arbitrary set S C X. Note that the definition remains unchanged for observed points already in the dataset, i.e., =(x,)) ) on. As it turns out, this non-dominance CPE is also estimating the probability of hypervolume improvement (PHVI) for new query points as the following theorem and corollary (assuming improvement (HVI) in Sec. B. as its definition is a little involved.
0). We fully define hypervolume
Theorem (Equivalence of Indicators). For every x & Sparews the HVI indicator is equivalent non-dominance indicator.
1[HVI(x) > 0] =(x).
(8)
Corollary (Non-Dominance CPE estimates PHVI). Following straighuforwardly from Theorem
P(=(x)= = 1|x) P(HVT(x) > 0|x) =: PHVT(x), Vx & Spereto
(9)
as the events are equivalent. Thus, a CPE trained on z, using a proper loss, is predicting PHVI.
See Sec. B.1 for the proof and assumptions under which this is true. We note that, for existing x E Stacto- we have a discrepancy, as =(x) = 1, whereas HVI(x) O. However, when the objectives are continuous, constructing the indicators from existing dataset for training a CPE is almost surely equivalent to constructing them in a held-out sense as there will be no ties. Furthermore. if there is observation noise it is beneficial to allow sampling at x again for noise reduction (c.g. averaging), as the true (x( may be dominated even if a single observation y at x suggests otherwise.
3 Incorporating User Preferences
In multi-ooeective optimization (MOO), practitioners invoke subjective preferences to single out subset of designs to meet application-specific requirements. Idcally, we would like not only to
3

---

incorporate these subjective preferences but also to avoid retraining our active generation framework every time new preference given.
standard approach to incorporating subjective preferences is scalarization: e.g. for convex scalar- ization we specify weight vector À Re with ||||] = and maximize
argiaax 8x,r (x), where 8x,r (x) = X (f.(x))
(10)
where E IR is reference point [27, 45, 45, 31]. While scalarization blends objectives according to explicit trade-off weights, is not optimal when learning conditional generative model of the Pareto set via CPE on labels 2n. [s (x,) ) > -] for some threshold T E IR as in [13], since each new À would require retraining. Furthermore, thresholding weighted objectives can blur the non-dominance boundary compared to directly labeling it.
3.1 Preference direction vectors and alignment indicators
As we will see in section 4, our solution incorporating user preferences for active generation is based on amortization. In other words, instead of estimating a model (x.)) as in VSD, we will learn conditional model of the form 96 (x u). Consequently, instead of scalarization, we introduce preference direction vectors u where U4 {u R || = 1}. defined from observed or desired (subjective) user specified outcomes. In our experiments, we train our method using
up = g(y)) = Yn ly, r
(11)
These unit vectors capture the relative emphasis among objectives in single geometric object. Given a trained model, a user can specify their own preferences via u. = g(y.) and our approach will generative model needs to enforce that generated
generate solutions from 9a (x u. Importantly, samples respect a user's 's desired trade-off. Therefore, we define an alignment indicator, a E {0, 1}. that labels each (x, u) pair 'aligned' if it achieves correct projection onto the preference direction. We will make clear the need for this indicator variable in the next section.
Preference directions generalize (convex) scalarization weights a (non-convex) generative setting: any À can be mapped to unit-norm vector u = A/| and conversely each u induces a unique normalized weight. In fact [8, 9] use "reference vectors". " which are similar to for scalarization of objectives and to promote diversity for many-objective evolutionary algorithms (MOEAs). By conditioning (u, a.) rather than scalarizing by X, our generative Pareto-set model becomes both more flexible (no retraining for new trade-offss offs) and more faithful to non-domen non-dominance structure. We visualize these preference direction vectors in Figure 1.
4 Amortized Active Generation of Pareto Sets
We now have all the components to describe our amortized active generation framework that learns to generate (approximate) solutions in the Pareto set, conditioned user preferences. We call our method active generation of Pareto sets (A-GPS), and it begins by generalizing the active generation objective in [39]. That is, for each round, i, we minimize the reverse KI. divergence between the generative model go (x|u) and an underlying (unobserved) true model p(x|u, 2,
of argmin
h/pp////////u/// (xxx)) a)) (u[=)),
argmin Ep(n)z) Dxulg (x) p 2,]]
(12)
Here actually conditioning avoid notational clutter. The inclusion of (uu|2) rewards learning an amortized generative model, (x u). over the distribution of the relevant preference directions. Naturally we cannot evaluate p(x u, 2, a) directly, and so we appeal to Bayes' rule,
and however leave this implicit henceforth to
1 p (x u, 2, a) z = u)p(alx, u))(x(u).
(13)
Here we have assumed conditional independence between z and a given x and u, and since p(z, a u/ is a constant w.r.t. x, will omit it from our objective. We make a further simplifying

---

assumption that a-priori p(x|u)= = p(x(Do). and then we rely on the likelihood guidance terms, p(z|x, u)p(a|x. u), capture the joint relationship between (x, u) in the variational posterior. We justify this decision by noting that the alignment relationship, x x|u, may be difficult to reason about a- a-priori, and requiring such a prior would then preclude the use of pre-trained models for p(x Do). Putting this all together results in the following equivalent amortized ELBO objective,
oi argmax (A.ELBO(())
where,
(14)
LA ELBO Ep(uls) (o)]
p(u Bgo u) log p [2|x, u) + log p(a/x, wtw.e///lallll////ll//x//a///a/a (15)
Direction dist.
Pareto CPE
Align. CPE
The B coefficient appears here to control the objective's exploration-exploiatiii tradeoff, and where 3 results in the exact minimization of Equation 12. We will now discuss how estimate each
of these components in turn, leading to the A-GPS algorithm presented in Algorithm
4.1 Estimating A-GPS's component distributions
Preference direction distribution, p(u[z). Since we observe u, we can approximate empirically p(ul2) = (En=1 2n)== Σ 1 [ = u,).. Alternativel,, we can use maximum likelihood to leamn parameterized estimator 4- (u) = p(ul2), with data D = {(xns Ups 2n kn))=
77- argmin Sporly, Div; where Lpaal,, DN)
ΣN 2n Σ n- In log T. (un). (16)
We find this occasionally aids exploration. Examples of appropriate parametric forms are von Mises- Fisher distributions, power spherical distributions [14] or normalizing flows [35]. We find Normal distributions, or mixtures, normalized to the unit sphere are more numerically stable than some of the specialized spherical distributions, see Sec. C. I. for more detail. We also find that fitting an unconditional q- (u) = p(u) for just the initial round can aid exploration of the Pareto front.
Pareto CPE, p(=|x, u). As per the original VSD, we define a CPE directly discriminate over the solution set, S. Sometimes we find setting S = Spacco leads to overly exploitative behavior. Instead, we anneal the set using the Pareto ranking method described in [15], and define the labels based on a thresholded rank, k k the next non-dominated set once St. Pareto. is removed, etc. We use T = f- : E DN }. 8,) as presented in [39], ensuring labels just Spareto The annealed method (Equation 20) applied to quantiles of k partieularly effective. So, with this label, we use the log-loss to train a CPE,
1 [x., E {Ug Sareto, : Ak S J. Here k = 1 indicates the Pareto set,
Cer N
Σ n=1
2n log Th (X,, un) + (1 2n. log(1 z (x un))
(17)
where T²(x, u) is discriminative model parameterized by 0, e.g. neural network.
Preference alignment CPE, p(a|x. u). Since do not wish to rely strong prior, p(x u)) for our sole-source of preference alignment information, we instead explicitly reward alignment in our conditional generative model by using a CPE guide. We create contrastive data for training this guide, D = {(a, = Xa, u, ) U {(a, =0, Xn, (n:(n)) J. where the second set are purposefully misaligned by permutations, P: N + N. This results in the log-loss,
N
P
N
= N PN
Σ
log T Xn, un) +
ΣΣ ==n==1
log(1 - (Xn: ue(m)) (18)
where T u) is our CPE parameterized by vh We make of two permutation methods for creating the contrastive data. The first is to just use random permutation without allowing any random alignments. The second is to use the top-k nearest neighbors, based on u, cosine distance, for replicates. The random permutation contrastive data covers the space of misalignment, while the top-k permutations improves the angular precision of the alignment scoring CPE. For all experiments use 7 random permutation replicates, and top-2 replicates, for a total of P 9.
5

---

Algorithm A-GPS optimization loop. See Figure 5 for a visual representation.
Require:
Initial dataset Dv. black-box f., prior p(x (o)). CPEs (x, u) and (x, u), variational families (u) and ( (x] threshold function f+ and 61, budget T and B.
I: function FITMODELS(DN,
Di {(2, Xn; u.)) ) where 2n lxx, {U Spmen Ak S T}) u. (yn-r)/llxx
3: Diy [[a,=1,,, Xn, un ) U ((an =0, Xn, Upe(n))
7" + argmin, C D
5: 8" argmin, (f, Dīy)
6:
v' argmin Cere (1, D
7:
3' + argmax, LO 0", 3 ,²)
return ,²,0", 0".0"""".7 .7'"
9:
for round E [1,..... ..... T do
10:
T f- (ly: E 6x)
11:
07,07
0i, 7i + FITMODELS(DN;T) sample 4,; (u) or use u.
12:
{un B
13:
{xbr 1B + sample go; (x(uar) vb E (I...., B}
14:
B + (f.(xet) + esc [yee]
DN < DN U vo))
return DN, 07, 07, 44, M
4.2 Learning A-GPS's variational distribution
To learn go (x(u), we can now re-write our amortized ELBO, Equation 15, in terms of these estimated quantities,
(o, = (g, (u) (xjo) [log T5 (x, u) + log n (x, u)] -
(19)
We find that using 'on-policy' gradient estimation methods such as REINFORCE [41, 30] are very slow when have complex variational distribution forms, (x u), e.g. causal transformers. This is because we have to set a low learning rate to avoid the variance of this estimator inducing exploding gradients for long sequences. Also, new samples have be drawn from the variational distribution every iteration of stochastic gradient descent (SGD), which can be computationally expensive. So instead we use an 'off-policy gradient estimator with importance weights emulate the on-policy estimator,
V 0, (xuu)((( wxx u) [LogTt u) + log (x u) :llog (x((u) p(x
V log go (x|Uu)
(20)
Here w(x, u) q (x/u) are the importance weights [34, 7]. Now we use samples from (()) ~ to (x(u(()), to approximate the expectation in Equation 20. we choose ø' = Φ we recover on-policy gradients, however typically only update ø' every 100 iterations of optimising A-ELBO, or if the effective sample size drops below a predetermined threshold (0.33.5). Whenever we update ø' we also resample S samples. We use these estimated gradients with appropriate SGD algorithm, such as Adam [25], to optimize for oi
4.3 Generating Pareto set candidates for evaluation
To recommend candidates for black-box evaluation round t, we sample set of designs from search distribution,
[xbe) B
II goz (x/ub));
where
+ argmax LA 03
(21)
We are free to choose Use based on preferences (u.); or if we do not have specific preferences to incorporate into the query, sample [ II trg (u) for broad Pareto front exploration.

---

Table I: Comparison of recent MOG and related techniques. ' the method has the feature, 'X' the method lacks the feature '-' the method can be easily extended to incorporate the feature. 'Modular' refers to the non-specific nature of the variational distribution used by conditioning by adaptive sampling (CbAS), VSD and A A-GPS, i.e., it can be chosen based on the task.
Online black-bon box optimization (BBO)
Amontzzdd preference contitionig
Discrete/ming /mixed X Generative pref. model. (i)) Generative obs. model, Qu (xc)
Not-ccoves conves Pareto frunt
Designed for MOO
Method
Guide
LaMBO [38]
Masked I.M
nFHVI
LaMBO-2 [2.]
Diffusion
nEHVI
Pareto Set Learning (PSL) [28]
X
Deterministic MLP
Scalarization
GHlowNets [24]
GelowNets
Scalarization
ParetoFloow [44]
x
Diffusion
Scalarization
PROUD [43]
X
Diffusion
Multiple grad. desc.
Prefereace Guided Ditfusion [3]
X
X
Diffosion
Preference CPE
ChAS [6]
X
Modular
Dominance CPE
VSD
Modular
Deminance CPE
A-GPS (ours)
Modular
Dominance CPE
5
Related Work
Our work sits the intersection of online black-box optimization, generative modeling, and user- guided multi-objective search. We organize existing methods along three dimensions: whether they operate online or offline. whether they directly optimize acquisition functions or learn conditional generative models for optimization, if they use inference-time guidance or learning for generation.
Online offline. Traditional MOBO methods, such as hypervolume-based acquisition (EHVI, noisy expected hypervolume improvement (nEHVI), and their variants), entropy search [42, 10, 11, 22] and scalarization methods [27, 45, 31, 13], operate online by sequentially querying the black-box using acquisition rules that balance exploration and exploitation. In contrast, offline MOG approaches like ParetoFlow and guided diffusion frameworks [44, 43, 3] train generative models from a fixed dataset of evaluated designs, without further oracle queries. While these offline methods can leverage rich generative priors, they have not been designed to adapt to new information.
Generative models vs. acquisition optimization. Recent advances in "active generation" recast black-box optimization fitting conditional generative models high-value regions, guided by predictors and/or acquisition functions. Methods like VSD. GFlowNets, and diffusion- solvers [39, 24, 17, 21] show that generative search can match exceed traditional direct acquisition function optimization, particularly in large search spaces. However, existing generative frameworks often need re-training to integrate subjective preferences. Similarly, all direct acquisition optimization methods require additional optimization runs to incorporate new preferences. An exception is Pareto set learning [28], which learns a neural-net that maps from scalarization weights designs.
Guidance vs. learning. Or inference-time vs. re- re-tranningffine-tumee eration methods, such those based on guided diffusion and flow matching [21, 43, 44, 3] use pre-traing pre-trained generative model, from which samples are then guided at inference time such that they are generated from a conditional generative model, leaving the original generative model unchanged. It has been noted in [26] that guided methods. though computationally efficient, may be prone to co-variate shift preventing them being guided too far from the support of the pretrained model. Con- versely, learning- based methods such as [39, 40] explicitly re- tr-irain or fine-tune the generative model to condition it, thereby circumventing these co-variate shift issues at the cost of more computation, but allowing for less constrained exploration in online scenarios.
based search. Guided gen-
Our A-GPS approach unifies these dimensions: it learns an amortized conditional generative model online, bypasses explicit acquisition optimization, and uses sequential learning to avoid co-variate shift. Table compares key features across representative MOG methods.
Experiments
We now evaluate A-GPS on number of benchmarks and compare it to some popular baselines. Firstly we apply A-GPS to number of well known continuous synthetic MOO test functions as proof-of-conceet. Then we apply it to three high-dimensional sequence design challenges
7

---

Erat Eart
D7227
(a) Negative Branin- Curain
(b) DTLZ7
(c) ZDT3
Figure 2:
Experimental results on three test functions commonly used in the MOBO literature. The top row
reports IIVI per round, the bottom row demonstrates amortized preference conditioning by generating Pareto front samples (DTLZ7 is PCA projection of the front).
A-GPS's intended application that emulate real protein engineering tasks. Our primary measure of performance is Pareto front relative HVI [46, 19] using the implementation in [4]. For all experiments set B 0.5 as the full KL regularization in Equation 15 can hamper exploitation in later rounds on some tasks. We refer the reader to Appendix for full experimental details.
6.1 Synthetic test functions
As mroof-of-concee we demonstrate A-GPS on some classical continuous paramterized MOO problems (x IR commonly used for MOBO [45, 5, 4]. Even though A-GPS has not been designed for purely continuous problems, they none-the-less allow us to demonstrate some appealing properties of A-GPS. We present three here; negative
Tranin-Currin (D 2, = 2), D'TLZ77 (D =, I 6). and ZDT3 (D = 4, T, = 2), see [16, 45, 5]. More detailed descriptions of these functions, additional experimental detail and on additional test functions are presented in Sec. D.I. We apply A-GPS to these continuous domains using conditional Gaussian generative model. (x(u) = (x ( o²² (u)). with mean and variance parameterized by a neural network (NN). The top row of Figure 2 reports mean relative hypervolume versus optimization round. The bands indicating 1 std. from 10 runs with random parameter initialisation. We compare to three Gaussian process (GP)-based baselines,
[EHHI[[1]] qEHVI and qNParEGO [10]. All methods use 64 training points, and then recommend B = candidates for T 10 rounds, and A-GPS has To set using the p = 0.25 percentile of Pareto ranks. On Branin-Currin, A-GPS (purple) rapidly outpaces the scalarization based PNParEGO, but is dominated by the methods that explicitly estimate EHVI. However, these methods do not scale to the higher dimensional DTL.Z7 problem, where A-GPS still outperforms dNParEGO. While A-GPS can model the complex Pareto front of ZDT3, the direct optimization methods outperform it, we suspect stronger generative backbone for continuous data would help here. The bottom row of Figure 2 illustrates preference conditioning: each panel plots the sampled Pareto front (dots) from qs (x(,,) colored by three representative preference directions u.. These were chosen by.
y. E (1 Pacto
(0.9), Qx (0.1)], (Av())))))) (Pllee)) , Av [Qxr,1 (0.1),2==0 (0.9)]} Pusto
(22)
Puslo
Pacto
where Q an empirical quantile function and Av denotes the set mean of each dimension, l, of the observed Pareto front, FParcto- We then use Equation 11 to convert these into u, with an automatically inferred reference point We project the higher dimensional DTLLZ7 outcomes into two dimensions using principal component analysis (PCA) for this visualisation. Overall, these results show that A- GPS supports flexible, a-posteriori preference conditioning across variety of continuous landscapes.
6.2 Ehrlich vs. naturalness
We now evaluate A-GPS on challenging two-objective synthetic 'peptide" design task that couples the Ehrlich synthetic landscape [37] with ProtBert [18] "naturalness" score. The Ehrlich function
8

---

1000000000000000000 400000
(a) M 15
(b) M 32
(c) 64
Figure 3: Ehrlich function vs. ProtBert naturalness score different sequence lengths.
has been designed to emulate key aspects of protein fitness; it maps each discrete sequence to scalar by embedding combinatorial motif interactions highly rugged, multi-modal but artificial terrain. In stark contrast, ProtBert's 's loss reflects genuine amino-acid patterns learned from 217 million real proteins. We use protein sequences X' VM where objectives are,
| and M E (15,32, 64). The two
f. Ehrlich(x),
1.2 1x2 (-Crmass()))
(23)
Here we convert ProtBert's log-loss, LroxBen (x) into likelihood (ʃ.² [0. 1J). making it output in
comparable range to the Ehrlich function (f. _ E [-13 J0, 1). We compare against a baseline that randomly mutates the set of best candidates, CbAS [6] and VSD [39]. which use the same 2-layer CNN Pareto CPE A-GPS, and against the guided diffusion based LaMBO-2 [21], which is formulated for discrete MOBO tasks using EHVI guidance. A-GPS uses the same CNN architecture for its alignment CPE. CbAS and VSD use causal transformer architecture for their gs (x). A-GPS uses the same backbone transformer, but embeds u for its prefix token and uses FiLM [32] on the transformer embeddings for conditioning the transformer (unconditional) architectures are used as priors by CbAS, VSD and A-GPS, which are trained on the initial sequences using maximum likelihood. Unlike in [39, 6], allow CbAS resample (x) between rounds (once every 100 iterations) - this drastically improves its performance. Results are reported in Figure for T 40 rounds from random starting conditions (bands indicating +1 std). All methods are given 128 training samples, and then recommend batches of size B = 32 per round, libraries and To set as p 0.25 percentile of Pareto ranks. We the poli and poli-baseling for running the benchmarks and LaMBO-2 baseline [20]. Additional experimental details and timing results in Sec. D.2. A-GPS performs similarly VSD, CbAS tends to be overly exploitative, and LaMBO-2's guided
preferences, 96 (x The same
model tends to under- reder-perform on this task.
6.3 Bi-grams
For this experiment we use the bi-grams optimization task from [38] where the aim to maximize the occurance of three bi-grams ('AV', "VC' and 'CA') in an M = 32 length sequence. We start with 512 random sequences that have no more than three of these bi-grams present. We then have T 64 rounds of B 16 to optimize the bi-gram occurances. The initial Pareto front is sparse so To is the p = 0.5 percentile of Pareto ranks. The same models as the previous experiment are used along with allows control over the number mutations to apply to an existing sequence, rather than generating complete sequence from scratch see Appendix C for details. LaMBO-2's masked diffusion backbone also has this ability, and following [38] we use a 1-mutation budget for these models. Relative hypervolume improvement results are presented in Figure 4 (a) and (b), sequence diversity is computed as the average pair-wise edit distance between all sequences. We can see the causal transformer backbone A-GPS and VSD models perform best, followed by LaMBO-2. CbAS overfits early as we can see from the diversity plot, and the mTFM models perform similarly to the random baseline and take many rounds to show any improvement on this task.
taakedd-rranffomerr model (mTFM) backbone for CbAS, VSD and A-GPS. This
6.4 Stability vs. SASA
Our final experiment uses the simulation-based protein stability vs. solvent accessible surface area (SASA) task from [38]. The aim to optimize base fluorescent proteins with M > 200 for stability (-AG) and SASA. We use the FoldX black-box implementation in poli [20], with

---

A.0
Round
(a) Bi- Bi-grams hypervolume
(b) Bi-grams diversity
Felative kape
Board
(c) Stability vs. SASA hypervolume
(d) Stability vs. SASA diversity
Figure Bi-grams and Stability vs. SASA results for relative hypervolume (HV) improvement and diversity.
512 training samples, T 64, B 16, and budget of one mutation per round. We have rich starting Pareto front To set from 0.1. We only use the m TFM backbone for CbAS, VSD and A-GPS as FoldX is best modelling only small differences to the original sequences. The results are summarised in Figure 4 (c) and (d). LaMBO-2 initially performs well, but overcome by A-GPS as the sequence diversity diminishes. See Sec. D.4 for additional plots of the estimated Pareto front.
7 Limitations and Discussion
We have considered the problem of active multi-objective generation, which frames discrete black-box multi-objective optimization as an online sequential generative learning task. Our proposed solution leverages recent advances in generative models to estimate distribution, qs (x (x/u), u). of the Pareto set directly, conditioned on user preferences u. which we call active generation of Pareto
(A-GPS).
Limitations. A limitation with A-GPS, and one that shares with many MOBO and MOG methods, is that it can be hard to specify algorithm hyper-parameteri a-priori before new data has been acquired - and the settings of these mindful of this in our implementation and design of A-GPS, and as such it comprises components that can be independently trained and validated mcaningfully on the initial training data at hand. In particular, find A-GPS, VSD and CbAS are sensitive to the prior model used, p(x PPo). To aid practitioners use these methods, we outline a general procedure we find works well in Sec. B.2 for prior choice and initialization. Another limitation with A-GPS presented in this work is the choice of the generative model for the continuous synthetic test functions. The MLP used does not appear to scale well to dighe--immensional problems, and expect using fatthigg [29] diffusion [23] backbones would remedy this issue, and incorporating these models into the VSD and A-GPS frameworks is an active research direction.
can effect real-world performance. We are
Another avenue of future work would be to explore the rich literature on MOEAs for alternative multi-objective measures beyond scalarizations and hypervolume indicators that may be suitable for guided MOG [1]. Additionally, this literature gives a thorough treatment of stochastic experiments in the multi-objective setting [36] which may be useful for MOG.
In contrast to other approaches that are dependent on diffusion models, our choice of generative model is flexible and modular. Our empirical experiments demonstrate that our method performs well on high dimensional sequence design tasks. We hope that our modular framework will result in many future extensions to different architectures for generative models, resulting in further practical algorithms for active generation in large search spaces. For code implementing A-GPS, VSD and all of the experimental results, please see
10

---

Acknowledgements
This work is funded by the CSIRO Science Digital and Advanced Engineering Biology Future Science Platforms, and was supported by resources and expertise provided by CSIRO IMT Scientific Computing. We would like to thank the anonymous reviewers for the constructive feedback and advice, and Sebastian Rojas Gonzalez for pointing us to the literature on MOEAs and for the discussion on estimating PHVI all of which greatly increased the quality of this work.
References
[1] Bekir Afsar, Jonathan E. Fieldsend, Andreia Guerreiro, Kaisa Miettinen, Sebastian Ro- jas Gonzalez, and Hiroyuki Sato. Many-Objeclive Quality Measures, pages 113-148. Springer International Publishing, Cham, 2023.
[2] Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Unexpected improvements expected improvement for Bayesian optimization. Advances in Neural Information Processing Systems, 36:20577-20612, 2023.
[3] Yashas Annadani, Syrine Belakaria, Stefano Ermon, Stefan Bauer, and Barbara E Engel- hardt. seeference-guided diffusion for multi-objective offline optimization. arXiv preprint arXiv:2503. 17299, 2025.
[4] Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. In Advances in Neural Information Processing Systems 33, 2020.
[5] Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Max-value entropy search for multi-objective bayesian optimization. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[6] David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference on machine learning, pages 773-782, PMLR, 2019.
[7] Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. In The Fourth International Conference on Learning Representations, 2016.
[8] Ran Cheng, Yaochu Jin, Markus Olhofer, and Bernhard Sendhoff. A reference vector guided evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation, 20 2016.
[9] Tinkle Chugh, Yaochu Jin, Kaisa Miettinen, Jussi Hakanen, and Karthik Sindhya. A surrogate- assisted reference vector guided evolutionary algorithm for computationally expensive many- objective optimization. IEEE Transactions on Evolutionary Computation, 22(1):129-142, 2018.
[10] Samuel Daulton, Maximilian Balandat, and Bytan Bakshy. Differentiable expected hypervolume improvement for parallel multi-objective bayesian optimization. Advances in Neural Information Processing Systems, 33:9851 1-9864, 2020.
[11] Samuel Daulton, Maximilian Balandat, and Fytan Bakshy. Parallel Bayesian optimization of multiple noisy objectives with expected hypervolume improvement. Advances in Neural Information Processing Systems, 34:2187-2200, 2021.
[12) Samuel Daulton, Sait Cakmak, Maximilian Balandat, Michael A Oshorne, Enlu Zhou, and Rytan Bakshy. Robust multi-objective Bayesian optimization under input noise. In International Conference on Machine Learning, pages 4831-4866. PMLR, 2022.
[13] George De Ath, Tinkle Chugh, and Alma AM Rahat. Mbore: multi-objective Bayesian optimisation by ratii estimation. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 776-785, 2022.

---

[14] Nicola De Cao and Wilker Aziz. The power spherical distribution.
arXiv preprint
arXiv:2006.0444 04437. 2020.
[15] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation, 6(2):182-197, 2002
[16] K. Deb, L. Thiele, M. Laumanns, and E. Zitzler. Scalable multi-objective optimization test problems. In Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH18000, volume 1, pages 825-830 vol.1, 2002.
[17] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems (NeurIPS), 34, 2021.
(18) Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang. Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, DEBSINDHU BHOWMIK, and Burkhard Rost. Prottrans: Towards cracking the language of life's code through self- supervised deep learning and high performance computing. bioRxiv, 2020.
[19] C.M. Fonseca, L. Paquete, and M. Lopez-Ibanez. An improved comension-sweey algorithm for the hypervolume indicator. In 2006 IEEE International Conference on Evolutionary Computa- sion, pages 1157-1163, 2006.
[20] Miguel González- Simon Bartels, and Richard Michael. poli: sequence objectives, January 2024.
libary of discrete
[21] Nate Gruver, Samuel Stanton, Nathan Frey, Tim GJ Rudner, Isidro Hotzel, Julien Lafrance- Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G Wilson. Protein design with guided discrete diffusion. Advances in neural information processing systems (NeurIPS). 36, 2023.
[22] Daniel Hermández-Lobato, Jose Hemmandez-Lobato, Amar Shah, and Ryan Adams. Predictive entropy search for multi-objective bayesian optimization. In International conference machine learning, pages 1492-1501. PMLR, 2016.
[23] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffuusio probabilistic models. Advances in neural information processing systems, 33:684006851,, 2020.
[24] Moksh Jain, Sharath Chandra Raparthy, Alex Hernández- Jarrid Rector-Brooks, Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-objective gflownets. In International conference on machine learning, pages 14631-14653. PMLR, 2023.
[25] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
(26) Leo Klamer, Tim G. J. Rudner, Garrett M. Morris, Charlotte M. Deane, and Yee Whye Teh. Context-guided diffusion for out-of-distribution molecular and protein design. In Proceedings of the 41st International Conference Machine Learning, ICML'24. JMLR.org. 2024.
[27] Joshua Knowles. Parego: A hybrid algorithm with on-line landscape approximation for expen- sive multiobjective optimization problems. IEEE transactions on evolutionary computation, 10(1):50-66 66, 2006.
[28] Xi Lin, Zhiyuan Yang, Xiaoyuan Zhang, and Qingfu Zhang. Pareto set learning for ex- pensive multi-objective optimization. Advances in neural information processing systems, 35:19231-19247, 2022.
[29] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arX1v:2210.02747, 2022.
[30] Shakir Mohamed, Mihaela Rosca. Michael Figurnov, and Andriy Mnih. Monte carlo gradient estimation in machine learning. Journal of Machine Learning Research, 21(132):1162. 2020.
[31] Biswajit Paria, Kirthevasan Kandasamy, and Barnabás Póczos. A flexible framework for multi-objective Bayesian optimization using random scalarizations. In Uncertuinty in Artificial Intelligence, pages 766-776. PMLR, 2020.
12

---

[32] Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018
[33] Mary Phuong and Marcus Hutter. Formal algorithms for transformers. arXiv preprint arXiv22207.0923.6 2022.
[34] Doina Precup. Richard S Sutton, and Satinder Singh. Eligibility traces for off- policy evaluation. In ICML, volume 2000, pages 759-766. Citeseer, 2000.
[35] Danilo Jimenez Rezende, George Papamakarios, Sébastien Racaniere, Michael Albergo, Gurtej Kanwar, Phiala Shanahan, and Kyle Cranmer. Normalizing flows on tori and spheres, In International Conference on Machine Learning, pages 8083-8092. PMLR, 2020.
[36] Sebastian Rojas Gonzalez, Juergen Branke, and Inneke Van Nieuwenhuyse. Bi-objective ranking and selection using stochastic kriging. European Journal of Operational Research, 322(2):599-66 614, 2025.
[37] Samuel Stanton, Robert Alberstein, Nathan Frey, Andrew Watkins, and Kyunghyun Cho. Closed-form test functions for biophysical sequence optimization algorithms. arXiv preprint arXiv:2407.002236 2024.
[38] Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating Bayesian optimization for biological se- quence design with denoising autoencoders. In International Conference on Machine Learning, pages 20459-20478. PMLR, 2022.
[39] Daniel M Steinberg, Rafael Oliveira, Cheng Soon Ong, and Edwin V Bonilla. Variational search distributions. In The Thirteenth International Conference on Learning Representations (ICLR), 2025.
[40] Chenyu Wang, Masatoshi Uehara, Yichun He, Amy Wang, Avantika Lal, Tommi Jaakkola, Sergey Levine, Aviv Regev, Hanchen, and Tommaso Biancalani. Fine-tuning discrete diffusion models via reward optimization with applications to DNA and protein design. In The Thirteenth International Conference Learming Representations, 2025.
[41] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8:229-256, 1992.
[42] Kaifeng Yang. Michael Emmerich, André Deutz, and Thomas Bäck. Efficient computation of expected hypervolume improvement using box decomposition algorithms. Journal of Global Optimization, 75:3-34, 2019.
[43] Yinghua Yao, Yuangang Pan, Jing Li, Ivor Tsang, and Xin Yao. Proud: Pareto-guided diffusion model for multi-objective generation. Machine Learning. (1attn 113(9):551--633 2024.
[44] Ye Yuan, Can Chen. Christopher Pal, and Xue Liu. Paretoflow: Guided flows in multi-objective optimization. In The Thirteenth International Conference on Learning Representations (ICLR), 2025.
[45] Qingfu Zhang and Hui Li. Moea/d: A multiobjective evoluttion algorithm based on decom- position. IEEE Transactions Evolationary Computation, 11(6):11--31, 2007.
[46] E. Zitzler and L. Thiele. Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Computation, 3(4):257-271, 1999.
13

---

A Broader Impacts
This work is motivated by applications that aim to improve societal sustainability, for example, through the engineering of enzymes to help control harmful waste. However, with many technologies, carries the risk of misuse by malicious actors. We, the authors, explicitly disavow and do not condone such uses.
B Additional Methodology Details
Model training
Discriminttive model
Geverative model
07 1Z
Dominance lnbels direction vectors
D
Unlabelled data generation
argmaa & Eo (7)
A
2.2
argiin (o. (X)) argnin Cowily 196)
IKTTTL...F LFLYGALTS #KIFTL.. LYBYL1U
AKTATL.AFLVVSILT
6.6
WKTTTL.
MKTTEL
FLLVTL..FLV
AKTTTL.&FLVSALTT
Or
LKTATI..LFLVGALTT
Labelled data
{xbe f0=1
Y6))
Experiment f.(x) +
Figure 5: A visual depiction of Algorithm A-GPS learns all the distributions involved by optimizing
different components of reverse KL loss. At time t. the optimized variational distribution ga (x/ u) with parameters of used generate designs that can incorporate user's preferences u1", We iterate until convergencc/user criterion satisfied.
B.1
Non-dominance CPE is estimating probability of hypervolume improvement
Using CPE trained on Pareto non-dominance labels, is equivalent to estimating the PHVI under some general conditions. The proof is based on the current observed Pareto set and front, Sareto F respectively, and relies the dexoppoomooiiionn definition of hypervolume. We use y as a shorthand for y(x) = f.(x) + e, and y' for yxx) = f.(x') + e', etc., where the context is clear. Firstly, proof relies the following assumptions,
Assumption (Measurement noise). No objectives exhibit measurement noise, 0.
This is so our dominance comparison, >; and hypervolume computation are exact with no ambiguity introduced from noisy measurements. We will discuss the consequences of relaxing this assumption later.
Assumption 2 (Reference Point), The reference point, r E TR is strictly dominated by every feasible objective vector,
C
y
(24)
This avoids negative- volume boxes when computing hypervolume, i.e. boxes must have positive no contribution to hypervolume.
We define box of hypervolume for any y E y under Assumption and Assumption 2 as,
B(y) : [r,]] (ri,11) [r2. (22 [r., vL].
(25)
14

---

Then let λg denote an L-dimensional Lebesgue measure, so the (dominated) hypervolume of a finite set, A R. is.
HV(A) = dy (U B(y))
(26)
YEA
With this we can define hypervolume improvement (HVT),
HVI(x): == HV Pareto U [y())))
HV
(27)
In addition, for any S X, Pareto(S) denote the Pareto subset of S, i.e.
Pareto(S)- =- {x S: x' x xx e S(x).
(28)
Note that Parcto (x}) Parcto(S) for any E X that dominated by an element of S. With these simple definitions and assumptions, and noting that:
z(x) = 1[x E Pareto(Sparee U {x})]
(29)
we have the following result.
Theorem (Equivalence of Indicators). For every x & Sparew the HVI indicator is equivalent non-dominance indicator,
1[HVI(x) > 0] = =xx).
(8)
Proof. This is straightforward if we consider the dominated and non-dominated cases.
Case 1: x is dominated. Having xxx) = implies that the condition x' X x, Vx' E Spreto is not satisfied and the Pareto set remains unchanged, i.e., Pa (Sareto U {x}) Spaeeo so that HV(FPaeto {y}) = HV (FPanco)) Hence, there is no hypervolume improvement,
2(x) =
1[HVI(x) > 0] =.
(30)
Case is non-dominated. If x of for all x' Sharww there is no y' Ftt Parçto such that
B(y) C B(y'). For HVI(x) to be positive, we need to show that HV Pareto) < i.e, whatever remains of the difference between Byy) and the previous union Uy B(y') must
Partoo U (y)))
be set of positive Lebesgue measure. Indeed, letting 7 == min max,e (1.....4) which is positive due to dominance, and setting 8 == min (2//2, minie (l....... (3t - r2) > 0 (by
V..
Assumption 2), we have that the d-box Bş := _1 [y: - 6, vil C B(y) is not covered by the union Uy'e Ryoo confirming that
Byy"). It then follows that HVI(x) HVFF Pareto U {y}) HV HVF Parcto) 2 A (BB) > 0,
(x) =1
1 > 0] = 1.
(31)
Thus, for any choice x & Spareto we have =xx) 1[HVI(x') > 0j.
Corollary (Non-Dominance CPE estimates PHVI). Following straightforwardly from Theorem 1, P(=(x)= 1|x) P(HVI(x) 0|x) =: PHVI(x), Vx
Shurcto
as the events are equivalent. Thus, a CPE trained on z, using proper loss, is predicting PHVI.
It is worth noting that we do not require Assumption for these indicators to be equivalent so long as the same noisy samples, y,, being used when computing them. However, we can no longer claim that the indicators themselves are reliably computing hypervolume improvement or non-dominance since the addition of noise makes the underlying comparisons ambiguous with respect to the true objectives, f.. It may still be possible to show convergence of A-GPS to the true Pareto set, Spaceto- while using noisy observations. The authors of VSD [39] were able to guarantee convergence using a CPE trained with noisy PI labels for single objective BBO using results from the neural tangent kernel (NTK) literature (see their Appendix F). Some of these results may be extended to Pareto set classification, though we leave this for future work, all of our experiments are noise-free.
15

---

B.2 Fitting the prior
We found that A-GPS, VSD and CbAS, which share generative backbones, were all very sensitive to the choice of prior. For complex problems, the best results were obtained in general when the prior is chosen to of the same form (or an unconditional variant) as the variational distribution, and fit to the T = 0 training data. However, for flexible backbone models like transformers, it can be easy to overfit to this training data. We found that adding dropout in conjunction with an early stopping training procedure the initial training dataset reliably led to good performance for all methods. The exact procedure used is listed as follows,
1. Set prior dropout p
2. Train prior with 10% validation set, make note of number of iterations when validation loss begins to increase
3. Train prior with all data for the number of iterations noted in the previous step
4. If appropriate, copy weights variational distribution (without dropout).
When the prior and variational models were compatible, initialized the variational model with these learned prior weights. We rum an ablation of this procedure in Sec. E.3.
C Architectural Details
C.1 Preference direction distributions
In all the experiments we use mixture of isotropic Normal distributions where the samples have been constrained to the unit norm,
2 (u)
Σ waNNiui (4|l of)
(32)
and > (H) k Typically, we find K 5 is sufficient. We learn this via maximum
1)2 so the magnitude of the mixture means is controlled (and does not decrease to 0 or increase to too). We have compared this to von Mises distributions, and find it more numerically stable, we also find no tangible benefit using more complex spherical normalizing flow representations [35]. Furthermore, we find that the performance similar to, if not slightly superior to, empirical approximation,
1 likelihood as per Equation 16, but add an extra regularisation term: - K Σ
(u)
2n
1 [
(33)
En=l
n=
where 7 = {u,, : 2, = 1] Though on occasions when only a few observations define the Pareto front, find that using this representation can lead to overly exploitative strategy.
C.2 Sequence variational distributions
In this section summarize the main variational distribution architectures considered for A-GPS VSD and CbAS.
Causal Transformer. For some the sequence experiments we implemented an auto-regressive (causal) transformer of the form,
M II m=2
qo (x) = Categ(z (softmxx(())) you (w' (1im-1), where
4ohmm (I 11mm Categ(E,, |softmax (DriiimnnnDDraaooooorrrrra (f1mm1))).
(34)
For details on the Recoder-transfomee with a causal mask see [33, Algorithm 10 Algorithm 14] for maximum likelihood training and sampling implementation details respectively.
16

---

Masked Transformer. We also implemented masked transformer model (mTFM) that learns mutate an initial sequence, x', set of positions, o [ol..... Oml where On {0, 1] and MM Om = for a mutation budget, O E f1,...,Mi M). The complete generative model is,
(x(x, qá. (x|x'|o Tmask]) (o|x')
(35)
Where the notation x'(o <- Tmusk] means we apply a masking token the original sequence the positions indicated by o. The mask generation model is,
Tdam (o|x)) Multinomial( NN e, (N, [ETTransformerr((("))
(36)
Here ETrrnsformer is an secoder-transfommen see [33, Algorithm 9] for details, and NN p. is NN decoder, with a convolutional residual layer for capturing additional local structure. The token generation model is,
M
1 [Tm s'm]
if Om 0; if Om 1,
Lhs. (x f- mmakk] II m=1
where
Categ(@z PPm)
[pI. Par) softmax =oottmax(TTrransnnooo (ETransformmr,,
c Fmmsk)))
Here softmax RIVIXMM + [0, M|| The same encoder--ransformer is shared between the mask and token heads, and only the masked positions are allowed sample new tokens. For our experiments we bias the categorical distribution so that the original token is not resampled. The choice of the set of seed sequences. {x²}}=1- can drastically impact performance of A-GPS, VSD and CbAS. A simple heuristic that we find works well in practice is to uniformly sample with replacement from the current active Pareto set, Spactoo So x' (Sprrco))) This similar to the strategy used in LaMBO-2, though since they use EHVI, they can maintain only the top-B sequences per round ranked by EHVI, where batch size.
Finally, if we use this model as prior, we drop the conditional mask model, 46c.. (olx'), ), and draw mask-positions independently with a Bernoulli distribution, Bern(Om |Pmask 0.15). Then use [33, Algorithm 12] for training.
We list the configurations of the transformer variational distributions in Table 2. positional encoding for all of these models. When using these models for priors or initialization of variational distributions, we find that over-fitting can be an issue. To eircumvent this, dropout and early stopping, see Sec. B.2 for details.
We use additive
As mentioned in the text, for the conditional generative models, (x(x)u) for A-GPS, we use the same architectures already discussed, but also learn a sequence prefix embedding from u, well as a simple 1 --hidden layer MIPs for implementing FilM [32] adaptation of the sequence token embeddings.
Conditioning.
em e'm o (1 + (u)) + fs( where
eg = Spreix (u)
(37)
Where c',, are the unmodified token embeddings (i.e. outputs of DTrrasformer or E'Transformer), and indicates element- wise product. fa and fa are the FiLM MLPs, and fpents is the prefix embedding MLP input into the transformers (often we find just linear projection is adequate). We initialize the transformer weights in these models from their non-conditional counterparts when they are used as priors.
Table Transformer network configuration the sequence experiments.
Ehrlich vs. Nat
Bi-grams
Stability vs. SASA
Property / IM
15
32
64
32
> 200 (variable)
Layers
2
Network Size
128
128
256
Attention heads
4
4
4
4
Embedding size
64
64
64
64
64
FiLM hidden size
128
128
128
128
128
Prior dropout p
0.5
0.2
0.1
Mutation budget O
17

---

C.3 Class probability estimator architectures
For all of our experiments we share the same architecture for both T u) and T3 (x, u). On the continuous synthetic test functions we the ML.P in Figure 6 (a), where simply concatenate the inputs x and u. Here Skip is skip connection which implements a residual layer, and MaxAndAvg means scalar weighted sum of max and average pooling.
For the sequence experiments we use the convolutional architecture given in Figure (b). For VSD and CbAS we simply add on another LayerNorm and LeakyReLU and then output linear layer. For A-GPS we concatenate u to output of this CNN, and then pass this concatenation into the MLP in Figure 6 (c). All architectural properties are listed in Table 3.
Table 3: CPE configurations for A-GPS, VSD and CbAS.
Ehrlich vs. Nat
Bi-grams
Stability vs. SASA
Synthetic Fns.
Property / M
15
32
64
32
200 (variable)
16
16
16
16
64
64
64
64
96
Kc
5
5
5
7
Kx
3
3
3
3
5
Sx
2
128
128
128
128
192
min(16D, 1283
hidden_layee layers
D Experimental Details
D.1 Synthetic test functions
We tested A-GPS on number of popular MOO synthetic test functions against strong GP-based baselines. All the test functions are summarised as follows, with additional results presented in Figure 7.
Tranin-Currin (D = 2): We optimize the negative Branin-Currin convex pair. We found the negative function has a more interesting Pareto front while remaining a challenging MOBO task.
DTLZ7 (D 7, L 6): A higher-dimensional constraint surface, made of L segments. The outcome dimension too high for methods that directly estimate the improvement to hypervolume.
ZDT3 (D 4, L = 2): A complex, non-convex front comprised of several disconnected segments, which stresses an optimizer's 's capacity for both exploration and front- segment coverage.
DTLZ2 (D = 3, = 2): smooth, spherical front in the negative orthant, which tests an algorithm's ability approximate non-convex curved manifolds in higher dimensions.
DTLZ2 (D 5, J. 4): A higher dimensional instantiation of the DTL.72 function for testing performance with higher dimensional inputs and objectives.
GMM (D 2, 2): Here each objective is implemented as Gaussian mixture model, and so is highly multimodal [12]. This is run without observation noise.
A detailed description of these functions and/or their Pareto-front geometries can be found in [16, 45, 5]. We use BoTorch ch [4] for the implementations of all the synthetic test functions and the baseline MOBO methods. All experiments were initialized using Latin hyper-cube sampling. The original design space is = [o. 1)2 for all problems, and we optimize directly in this space with all methods. The GP based methods
optimizer that respects these bounds directly, and clamp A-GPS's generative model to these bounds. We found transforming the space (e.g. using logit-sigmoid transforms) generally led to worse performance.
For A-GPS we use the mixture model in Equation 32 for the preference direction distribution, and for the conditional generative model
simple MLP,
(x A (x(x((u), -
(38)
Here uuu), (u) are MLPs with 2 or 4 hidden layers (if D 2 3) of size of min(16D, 256) with skip-connections and layer normalization, making them residual networks. We otherwise use the same experimental settings for the rest of the experiments, which are given in Table 4.
18

---

Sequential Linear
Sequential
Enbedding Positional nun, enbeddinga=A, enbedding.din-F
in_feeturee-D L,
out_features-!
),
LayerNorn() (),
Dropout(p=0.2)) Convid
LeakyReLU (),
Dropout (p=0.1), [3kip(
in_ in_channela=E,
channeeeC
Linear
kernal_aizertt
in featuress,,,
)
out out_featurea=h
GroupKorn (), LeakyReLU (),
()
Bropout (p=0.
LeakyReLU())
YaxAndAvgPoolldC
Dropout (p-0.1),
ternel_size
for in (aaddnn
stride=8X,
Linear
in un_ out_features-1
Skip
Convid
in_channels=t,
out out_channels-C.
kernel.eize-ke,
(a) Continuous MLP architecture
GroupKorm(),
Sequential
LeakyReLU
Linear
in featueere
Kappiieeaxxcckvgpo0ld00
out_featuressh
Linear(
featureee
LeakyReLU(), LayerNorn (),
Skip(
Linear(
(b) Sequence CNN architecture
in fe.feauueeeHH out_featurea=H
()
Linear
featuerree,H,
out out_features-i
(c)
concatenation MLP architecture
Figure 6: CPE architectures used for the experiments in PyTorch-like syntax. A = [ = L corresponding to y E R2_ and D D corresponding x E R continuous experiments. LaMBO-2 uses the same kernel size as our CNNs. See Table for specific property settings.
D.2 Ehrlich vs. Naturalness
The Ehrlich vs. naturalness score benchmark implemented using the poli benchmarking library [20], where we implemented our own ProtGPT2-basee naturalness black box, and used the inbuilt Ehrlich function [37] (not the holo version). For A-GPS
the aforementioned generative and diseriminative models, otherwise the settings are given in Table 6. We use a modified version of the LaMBO-2 algorithm [21] from poli holi-baselines [20]. We use the following Ehrlich function configurations:
M 15: motif length
motif's 2, quantization =
M 32: motif length = 4, no. motifs = 3, quantization = -4
M 64:
motif length no. motifs 4, quantization =
Additional experimental settings are given in Table 6, and runtimes in Table 5.
D.3 Bi-grams
For the bi-grams experiment we implemented our own black-box for the poli library based on the experiment [38]. All architectural details are presented in Appendix C, with additional experimental settings in Table 6. We also report runtimes in Table 7.
19

---

BTL244
DAN
(a) DTLZ2 2
(b) DTLZ2 L
(c) GMM
Figure 7: Experimental results on an additional three test functions commonly used in the MOBO literature. The top row reports IIVI round, the bottom row demonstrates amortized preference conditioning by generating Pareto front samples (DTLZ2 with 4 PCA projection of the front).
Table 4: Synthetic test functions experimental settings. Setting
Value
No
T
10
Replicates
10
B
256
interred using BoTorch's 's infer wtwrrreeeeeeeeeeeeeeeeee ()
Base Bo Borch model
SingleTaskGP
Optimizer
Adam for A-GPS (Ir = 10 10-9) and L-BEGS for the GPs
Max optimization iter. GP restarts
A-GPS
3000
10
Bounds
[0 1)
kernel
Matern v 2.5 ARD (standardized inputs)
GP hyperparameters
LogNormal priors on a and /
D.4 Stability vs. SASA
For the stability vs. SASA experiment we used bi-objective black box from the poli library, and the seed sequences and settings from the experiment [38]. All architectural details presented in Appendix C, with additional experimental settings Table 6. We also report runtimes in Table and show some samples from the A-GPS generative model, well the empirical Pareto front in Figure
D.5 Computational resources
All experiments were run Dell PowerEdge XE9640 rack server cluster with NVIDIA H100 GPUs and 4th generation Intel Xeon CPUs. All our models could easily fit on one GPU, and typically took less than 2 hours to complete the experiments.
Table 5: Ehrlich vs. Naturalness times (mins).
M = 15
M
32
M = 64
Method
mean
min
max
mean
min
max
mean
min
max
A-GPS-TFM
18.10
17.79
18.38
19.40
18.94
19.78
21.43
20.99
21.75
VSD-TFM
12.43
12.26
12.61
13.62
13.10
14.34
15.85
15.24
17.26
CbAS-TFM
9.18
8.91
9.51
9.73
9.40
9.98
11.78
11.43
12.11
LaMBO-2
14.15
13.50
14.74
16.17
15.74
16.41
17.96
17.68
18.75
Random (greedy)
0.53
0.51
0.54
0.91
0.91
0.92
2.35
2.35
2.36
20

---

Table 6: Sequence experimental settings.
Ehrlich vs. Nat.
Bi-grams
Stability vs. SASA
Setting / M +
15
32
64
32
200 (variable)
No=o
128
128
128
5T2
512
T
40
40
40
64
64
Replicates
5
5
B
32
32
32
16
16
S
256
256
256
256
256
[-1. 0]
[-1. 0]
[-1.
[0, 0.
auto
Threshold percentile
0.25
0.25
0.25
0.5
0.1
Table 7: Bigrams times (mins).
Method
min
A-GPS-TFM
30.99
30.04
31.89
A-GPS-mTFM
40.32
38.78
42.19
VSD-TFM
21.23
20.10
22.14
VSD-mTFM
28.13
27.93
28.44
CbAS-TFM
14.63
14.41
14.84
CFAS-mTFM
24.24
23.11
LaMBO-2
42.03
40.63
43.04
Random (greedy)
0.47
0.47
0.48
E
Ablation Studies
In this section we test some of the architectural decisions have made when designing A-GPS.
F.1 On-policy vs. off-policy gradients
We introduce new gradient estimator in Equation 20 based on off off-policy importance weighting approximations to the on-policy gradient estimator used by [39]. To test its efficacy, we re-run the Ehrlich naturalness score experiments with this new estimator and the original on-policy variant. We report performance and runtimes in Table 9 for M = 32. There does not seem to be a consistent difference between the gradient estimators in terms of hypervolume performance, but runtime significantly lower for the off-policy estimator, being almost an order of magnitude less for the off- off-policy variant.
E.2 Off-policy gradient estimator samples
We test the effect on performance of the number of samples, S, used for estimating the gradients of the off- off-policy estimator, Equation 20, used for A-GPS and VSD in Table 10. We also do the same for the CbAS estimator. We generally find that more samples lead to more performance for all methods, however this effect plateaus A-GPS and VSD starting = 256, whereas CbAS still sees improvement beyond S = 512. This is also something noted by the original authors in [6].
E.3 Prior regularization
We found that A-GPS, VSD and CbAS, which share generative backbones, were all very sensitive to the choice of prior. For complex problems best results were obtained in general when the prior is fit to the = 0 training data, however for flexible backbone models like transformers, it can be easy to
Table 8: Stability vs. SASA times (mins).
Method
mean
min
max
A-GPS-mTFM
120.75
118.52
122.67
VSD-mTFM
108.93
107.57
109.92
CbAS-uTFM
97.48
96.11
99.16
LaMBO-2
59.33
57.55
61.18
Random (groedy)
11.38
10.87
12.20
21

---

12250
ER STE
-0
front
30
1144
11500
11250
Detch
313
10750
2903
10500
10
10250
2000
-25
(a) Samples from the final A-GPS generative
(b) FoldX black-box evaluations of sequences generated by
model conditioned on preference direction vec- A-GPS, colored by evaluation round. tors.
Figure 8: Additional A-GPS sample visualisations from the stability vs. SASA experiment in section 6.
Table 9: Ehrlich function vs. naturalness score ablation for M = 32. Run time and performance comparison for the on-policy gradient estimators (" ('-rcinf.) the importance weighted off policy estimators for the A-GPS and VSD methods, All times are in minutes.
Time (min)
Method L-GPS-TFM
mean
min
max
T = 40 relative HV improvement
19.40
18.94
19.78
6.264 (1.159)
VSD-TFM
13.62
13.10
14.34
6.711 (0.952)
CbAS-TFM
9.73
9.40
9.98
4,398 (0.652)
LaMBO-2
16.17
15.74
16.41
3.074 (1.950)
Random (greedy)
0.91
0.91
0.92
1,260 (0. 169)
A-GPS-TFM-reinf
105.09
100.20
109.52
6.656 (0.920)
VSD-TFM-reinf. 97.95
90.74
115.81
6,257 (1.338)
overfit. Adding dropout only to the prior generative model is an effective means of controlling this overfitting when used with validation set to infer the number of learning iterations, as in the process outlined in Sec. B.2.
To demonstrate the effect of over- and under-fitting on performance, we change the transformer dropout probability while holding all else constant, e.g. fitting iterations, on the Ehrlich vs. naturalness experiment for M = 32 in Table 11. The training iterations were tuned while setting p 0.4, and so we would expect other values of dropout to be suboptimal, and we can indeed see this so. All methods severely under- slightly worse with an under-fit prior, whereas CbAS's performance actually improves. This is a trend we have noticed in all experiments; prior overfitting leads to poor performance, and under- fitting is less consequential. CbAS tends favour exploitation, which explains why a broader prior, encouraging exploration, helps the method.
with and overfit prior p E {0.2, 0.3). A-GPS and VSD perform
Table 10: Round T' 40 relative hypervolume improvement results for the varying the number of samples used estimate the gradients with the off-policy gradient estimator, Equation 20, and the CbAS gradient estimator.
M Method
S = 64
S = 128
S = 256
512
32
A-GPS-TFE TFM
5,663 (1.357)
6.212 (0.905)
6.264 (1.159)
6.245 (1.398)
VSD-TFM
5.665 (0.918)
6.218 (1.041)
6.711 (0.952)
6.964 (0.899)
CbAS-TFM
4.048 (1.249)
4.167 (0.707)
4.398 (0.652)
5.531 (1.347)
64
A-GPS-TFE TFM
5,655 (1.646)
5.522 (1.188)
6.021 (1.052)
6.269 (1.808)
VSD-TFM
5,435 (0.844)
5.198 (1.240)
5.738 (1.305)
4.633 (0.817)
CbAS-THI
4.168 (1.352)
3.803 (0.741)
3.929 (0.564)
5.092 (0.972)
22

---

Table 11: Round T 40 relative hypervolume improvement results for the varying the prior dropout probability transformer backbone. only present results
32, and the training regime was originally
optimized for = 0.4.
Method
p==02 0.2
p= 0.3
0.4
= 0.5
A-GPS-TFM
3.786 (0.925)
5.525 (1.224)
6.264 (1.159)
5.743 (0.849)
VSD-TFM
3,585 (0.894)
4.455 (0.433)
6.711 (0.952)
6,496 (1.246)
CbAS-TFM
2.543 (0.589)
4.099 (0.678)
4.398 (0.652)
5.838 (1.090)
E.4 Empirical vs. parameterized preference direction distribution
For all of our experiments we use the constrained mixture of Normal distributions (K = 5), Equa- tion 32, our parameterized preference directions distribution, (u)).. We now wish to validate this choice by comparing it to two simpler alternatives: a single constrained Normal distribution (K = 1), and the empirical distribution in Equation 33.
We make these comparisons on the synthetic test functions, as their Pareto fronts are well sampled and diverse in their shapes. As can be seen in the results in Figure 9, there is not consistent leader among the different preference distribution parameterizations across all of these functions. Perhaps the empirical preference distribution strikes the optimal blend of simplicity and performance however we do occasionally find that it under- mnder-performs compared to the parameterized distributions in cases where the Pareto front is sparsely sampled, which happens in some sequence experiments.
IratiCCarts
D7.23
2073
115
000
Sene
P0-13
1040
(a) Negative Branin Currin
(b) DTLZ7
(c) ZDT3
Gll
1.0
(d) DTL.22
(c) GMM
Figure 9: A-GPS preference distribution ablation experimental results the synthetic test functions used for Sec. 6.1.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The title describes proposed method, and the abstract and introduction identifies the problem that our method solves and places in the context of other machine learning problems.
Justification:
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The limitations are discussed in the "Related Work" and "Discussion" sections, and illustrated in our empirical results.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: have provided careful derivations of our proposed model, and also a proof of equivalence of indicators in the main text. We provide all assumptions and derivations of this proof the appendix.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: The proposed method is presented in detail in Algorithm 1, and how it is derived from first principles is described in Section 4. Details of experimental settings provided in the main text and supplement, along with supporting additional experiments.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Data reported in this paper are from existing publicly available benchmarks, described Section 6. Our open source code provided at a linked github repository.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The details of the algorithm, including the estimation of gradients and the opti- mizer are described in Section 4. Specific settings corresponding the three experimental sub-sections in Section are provided in Appendix D
Justification:
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All our experimental results in Section 6 are with appropriate uncertainty intervals.
24

---

8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Details of our computing resource, as well as required compute time provided in Appendix D.5, in the Supplement.
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: / http//eerrpp..eeee httpol/iieeccccccccceleliieiieie
Answer: [Yes]
Justification: This is an algorithmic contribution to multi-objective optimization (which has been studied for decades), and the application existing benchmark datasets, so no new ethical questions arise.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
This paper proposes an algorithmic advance, hence there are no new direct impacts. However, further discussion of societal impacts multi-objective generation is discussed in Appendix A in the Supplement.
Justification:
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
This paper does not release any large models, nor does it reveal any new datasets.
Justification:
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Data for benchmarks from previous publications (appropriately cited), Our code builds on Apache 2.0 open source licensed code.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The software implementing the algorithmic implementation well docu- mented, and the documentation is provided in the software package in the linked repository.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No crowdsourcing nor any research with human subjects.
Institutional review board (IRB) approvals or equivalent for research with human subjects
25

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No crowdsourcing nor any research with human subjects.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: The core method development in this research does not involve large language models.
26


Reviews:
[{"review_id": "XUlfcEhPwV", "reviewer": "Reviewer_uzmJ", "summary": "This paper proposes a novel approach to multi-objective optimization (MOO) by directly modeling the Pareto front, rather than relying on traditional model-based methods. This strategy avoids common pitfalls associated with hypervolume-based approaches, offering a more stable and efficient alternative that also works seamlessly for discrete problems. A key strength of the method is its ability to incorporate user preferences into the exploration of the Pareto front without requiring retraining of the Pareto classifier. Empirical results demonstrate strong performance across a representative set of baselines, highlighting the method’s practical effectiveness.", "questions": "Please can you provide a more detailed explanation for Fig 1(a) ?\nPlease can you provide detailed info about the baseline methods you ran in BOTORCH?\nPlease can you comment on why you used a sigmoid transform for the domains of the synthetic test problems? Did you then use Botorch in the unbounded space? This feels like it will impair the standard MOBO approaches.", "limitations": "No", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I really enjoyed reading this paper. As someone with expertise in Bayesian optimization but less familiarity with multi-objective generation (MOG) approaches, I found the paper informative and very accessible. The concept of active generation is clearly defined, and the overall presentation is strong. However, I found Figure 1(a) a bit hard to interpret and would have appreciated a more detailed caption—this figure plays a central role in conveying the method’s intuition. In contrast, Figure 1(b) added relatively little. While I don’t feel qualified to assess the significance of the contribution relative to prior MOG methods, I do think the work is well-executed and worthwhile. Thank you for a thoughtful and well-written paper.\n\nMy main concerns lie in the experimental section. Since the method lacks strong theoretical justification, a rigorous empirical evaluation is critical for a venue like NeurIPS. First, I would have liked to see an ablation that justifies the use of the preference alignment component described in lines 148–153—this was the one part of the method I found difficult to follow, and further evidence of its impact would be valuable. Additional ablations (e.g., on the number of samples used to approximate Equation 18) would also strengthen the paper. More seriously, a lot of implementation details for the baselines (mentioned on line 230) are missing entirely (e.g. how are acquisition functions optimized, choices of reference points e.t.c). In Bayesian optimization and MOO, these details are crucial, as implementation nuances can heavily affect performance. Without this information, it’s difficult to assess the validity of the performance comparisons. I also found it concerning that Figure 2 is based on only five repetitions—this raises questions about the statistical robustness of the results. That said, Sections 6.2 and 6.3 were compelling, and I appreciated the inclusion of more challenging, real-world examples to demonstrate the method’s applicability.", "was_revised": true, "final_justification": "The comprehensive rebuttal dealt with my concerns (noteably around repoducibility and ablation) and I raised my score. I think this work poses an intersting avenue for MOO and is well backedup with extensive emperical results.", "rebuttal": "We thank reviewer uzmJ for the thoughtful and considered review, and we are encouraged they enjoyed reading the paper! We will attempt to address the reviewer's concerns succinctly.\n\n**Q1: Please can you provide a more detailed explanation for Fig 1(a).**\n\nThank you for this feedback. We will consider moving Fig 1(b) to the appendix, or shortening its description since it is a visual representation of Alg. 1.\n\nWe will change the description of Fig 1(a) to something like the following: (a) A visualisation of a 2-dimensional Pareto front, $\\mathcal{F}\\_\\text{Pareto}$, and the random variables used with A-GPS. $y$ are the noisy realisations of the objectives, $f$, when $z=1$ these observations lie on the Pareto front. Preference direction vectors, $u$, are simply unit vectors pointing to a region of the Pareto front from a reference point, $r$ --- Equation 9. We derive aligned ($a=1$) training preference direction vectors, $u_n$ from observation pairs $(y_n, x_n)$, and mis-aligned preference direction vectors ($a=0$) from permuting these pairs, $(y_{\\rho(n)}, x_n)$ --- Equation 16. The aim is to learn the distribution of the Pareto set $q_\\phi(x|u) \\approx p(x|u, z=1, a=1)$, from which a user can input a preference encoded as $u$, for generating optimal samples. We show the image of this distribution on the Pareto front.\n\n**Q2: Please can you provide detailed info about the baseline methods you ran in BOTORCH? Please can you comment on why you used a sigmoid transform for the domains of the synthetic test problems? Did you then use Botorch in the unbounded space? This feels like it will impair the standard MOBO approaches ...**\n\nMany of these details are in the Appendix D included in the supplementary material along with the original submission, including, e.g. GP surrogate model, reference points for the problems, and details on the sigmoid transform (which is treated as part of the black box). We will expand on these for the final version.\n\nThe reference points used are supplied by the BoTorch library implementations of the black boxes where possible --- though in one case we found the supplied value erroneous. We have actually found that using the BoTorch `botorch.utils.multi_objective.hypervolume.infer_reference_point` function is a more reliable way of setting the reference point automatically, and it is used by other methods, e.g. LaMBO-2. So all of our experiments, methods and baselines have been adapted to use this method now.\n\nAll of the original synthetic test functions are only defined on $[0, 1]^D$, and so composing these black-boxes with a sigmoid function is an easy way of making them unbounded, freeing us to use more expressive generative models. The GP-based methods also optimize within this unbounded space --- though we do give them generous bounds of $[-15, 15]^D$, since the BoTorch interface requires bounds to be specified. We believe this is a fair comparison.\n\nWe will happily expand upon these details where the reviewer sees fit. We will add these details for the GP methods:\n\n| Property | Value |\n|--------|------|\n| Optimizer | Adam |\n| Iterations | 500 |\n| # Restarts | 10 |\n| Bounds | $[-15, 15]^D$ |\n| Kernel | Matern $\\nu = 2.5$ ARD (standardized inputs) |\n| Hyperparameters | LogNormal priors on $\\sigma$ and $l$ |\n\nAre there any other settings in particular we should add?\n\n**Q3: I would have liked to see an ablation that justifies the use of the preference alignment component described in lines 148–153**\n\nWe would like to clarify to the reviewer that the preference alignment indicators are *necesary* as one of the contributions of our approach is to support user preferences at test time. In order to train such a model, we need a way to tell whether a sample $x_i$ is indeed aligned with preference vector $u_i$. For this purpose, we use an alignment indicator $a_i$. Both preference direction vectors and alignment indicators are constructed internally by our framework (as described in the paper) during training so that our conditional generative model $q(x | u)$ learns to generate Pareto-optimal samples that are aligned with preferences. At test time, a user can specify a new preference vector $u_\\star$ and our model will generate samples $\\{x_\\star\\}$ that are aligned as much as possible with such preferences. We can add this clarification to the text.\n\n**Q4: Additional ablations (e.g., on the number of samples used to approximate Equation 18) would also strengthen the paper.**\n\nWe will happily include these. Here are round ten, $t=10$, results for different sample sizes, $S$, on the Ehrlich vs. Naturalness experiment,\n\n| Seq. Len | Method | $S=64$ | $S=128 $ | $S=256$ | $S=512$ |\n| -| -| -| -| -| -|\n| 32 | AGPS-TFM | 1.600 (0.307) | 1.576 (0.119) | 1.567 (0.241) | 1.584 (0.125)\n| | AGPS-LSTM | 1.563 (0.323) | 1.484 (0.225) | 1.597 (0.290)| 1.514 (0.267)\n64 | AGPS-TFM | 1.071 (0.058) | 1.094 (0.047) | 1.070 (0.058) | 1.094 (0.050)\n| | AGPS-LSTM | 1.088 (0.048) | 1.075 (0.053) | 1.075 (0.052) | 1.082 (0.048)\n\nInterestingly, for the LSTM and Transformer we do not find significant variability -- we will expand this to other experiments.\n\n**Q5: I also found it concerning that Figure 2 is based on only five repetitions --- this raises questions about the statistical robustness of the results.**\n\nWe will increase this to 10 replicates for the final version. We have implemented this already, and found little difference in the conclusions drawn, apart from improving qParEGO on DTLZ2 --- here are round 10 results presented as relative hyper-volume improvement:\n\n| Test function ($L$) | Method | Mean round 10 relative HVI (1 std.) |\n|-|-|-|\n|BraninCurrin ($L=2$)| A-GPS | 1.626 (0.085) |\n| | qParEGO | 1.374 (0.133) |\n| | qEHVI | 1.546 (0.092) |\n| | qNEHVI | 1.510 (0.107) |\n|DTLZ2 ($L=2$)| A-GPS | 1.071 (0.020) |\n| | qParEGO | 1.042 (0.027) |\n| | qEHVI | 1.060 (0.004) |\n| | qNEHVI | 1.060 (0.008) |\n|ZDT3 ($L=2$)| A-GPS | 1.267 (0.312) |\n| | qParEGO | 1.041 (0.008) |\n| | qEHVI | 1.045 (0.044) |\n| | qNEHVI | 1.082 (0.107) |\n\n\n**Q6: Since the method lacks strong theoretical justification, a rigorous empirical evaluation is critical for a venue like NeurIPS ... That said, Sections 6.2 and 6.3 were compelling, and I appreciated the inclusion of more challenging, real-world examples to demonstrate the method’s applicability.**\n\nSubsequent to the paper's submission we have realised that under certain assumptions the CPE trained on non-dominance indicators is estimating probability of hyper-volume improvement (PHVI). We have sketched the proof for reviewer yJhX. Hopefully including this will improve the method's theoretical justification.\n\nFuthermore, we have also implemented the Solvabity vs. SASA experiment from [31] which uses the Foldx simulator as a black box. This experiment is run for 64 rounds, with batches of 16, and only permits one mutation per sequence per round. This required a masked Transformer generative model (mTFM), $q$, that learns to mutate an input sequence, in a similar fashion to the generative back-bones of LaMBO and LaMBO-2. A-GPS and VSD perform similarly to LaMBO-2 on this experiment (and outperform a greedy random baseline and CbAS), and we feel it adds yet another compelling example of this method. Round 64 results summarised below,\n\n| Method | Mean round 64 relative HVI (1 std.) |\n| -| -|\n| VSD-mTFM | 1.150 (0.021)\n| LaMBO2 | 1.142 (0.025)\n| A-GPS-mTFM | 1.137 (0.025)\n| CbAS-mTFM | 1.078 (0.025)\n| Random (greedy) | 1.061 (0.013)"}, {"review_id": "TbilbYLNcT", "reviewer": "Reviewer_wcJY", "summary": "The paper proposes a new framework for online multi-objective optimization called active generation of Pareto sets. The method builds upon ideas introduced in variational search distributions, with the goal of extending to multi-objective optimization. The method learns a generative model and uses a class probability estimator, where class is Pareto optimality, to condition this model along with user preferences. This model also does not require expensive hypervolume calculation, which is common in other multiobjective optimization techniques. The authors detail the methodology used and show the results of their method compared to other multi-objective optimization algorithms on a few test functions. These results show strong performance over the test functions presented and flexibility with regard to user preferences and type of generative model.", "questions": "Each evaluation appears to have detailed setups for both the baselines and proposed method, which differ across test functions. For example, the model sometimes uses an LSTM and sometimes uses a transformer. However, the performance varies widely across test functions. What is the justification for these decisions and do they provide fair comparison against the baselines? How would you use A-GPS to approach a new problem apriori, including deciding which model to use?\n\nCan you provide further test results? The results shown are fairly limited and a wider range of evaluations would provide more convincing evidence of A-GPS performance. It is mentioned that some test functions have large variations between runs and we do see high standard deviations in the results. Showing further results from other DTLZ and ZDT test problems would be more convincing and help show the true performance of A-GPS.", "limitations": "The authors mention that specifying hyper-parameters can have a significant impact on performance, but could further expand on how these results change and other possible limitations the methodology has. It appears the selection of these hyperparameters has a significant impact on performance and may be highly tuned for the presented results.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The paper includes good insight into the ability to control user preferences and flexibility within a generative model-based optimization scheme. The results shown in Figure 2 illustrate the ability to condition on user preferences while also matching or beating the hyper-volume performance of baselines. This is particularly helpful given a trained model can be amortized, as users can input different preferences afterwards given a single trained model. This provides an advantage over other methods such as VSD, which do not support this type of amortization. Additionally, the experiments show comparable results from using a generative model as opposed to a GP-based method, which is a promising aspect of the paper.\n\nThe experiments shown are relatively limited and would be more convincing with more diversity and quantity of test functions. For example, the DTLZ and ZDT test suites contain further problems with varying pareto frontiers. Additionally, the paper would be improved with clearer discussion of its limitations. The authors mention the limitation of hyperparameter tuning, but it is not clear to what extent this is done for the included results. The results shown require selecting the best model, either transformer or LSTM, for individual test problems, suggesting the robustness of the method could be improved, as the results vary across test functions with the same setup. Therefore, if a single generative model is used with A-GPS, the results may be worse than baselines. As a whole, the paper would be improved by showing how these parameters and test setups were chosen to provide fair comparison to other methods.", "was_revised": true, "final_justification": "Updated my score from 3->4 due to the additional results and clarifications from the authors during the rebuttal.", "rebuttal": "We sincerely thank reviewer wcJY for the constructive feedback, and we will endeavour to respond to all of their concerns.\n\n**Q1: The experiments shown are relatively limited and would be more convincing with more diversity and quantity of test functions. For example, the DTLZ and ZDT test suites contain further problems with varying pareto frontiers**\n\nReviewer VCzN also requested additional synthetic function experiment, but with higher dimensional objectives. As such we have tested two more, higher dimensional, synthetic functions, along with another low dimensional GMM test function from BoTorch:\n\n| Test function ($L$) | Method | Mean round 10 relative HVI (1 std.) |\n|-|-|-|\n|DTLZ7 ($L=6$)| A-GPS | 3.073 (1.363) |\n| | qParEGO | 2.313 (1.205) |\n|DTLZ2 ($L=5$)| A-GPS | 1.053 (0.022) |\n| | qParEGO | 1.042 (0.027) |\n|GMM ($L=2$)| A-GPS | 1.057 (0.039) |\n| | qParEGO | 1.055 (0.053) |\n| | qEHVI | 1.105 (0.041) |\n| | qNEHVI | 1.157 (0.024) |\n\nUnfortunately the EHVI-based methods were not able to be used in the higher-dimensional settings (qNEVHI is claimed to scale to $L=5$, but we found difficulty running it in time for this rebuttal), which hopefully reinforces the efficacy of our method.\n\n**Q2: Additionally, the paper would be improved with clearer discussion of its limitations.**\n\nWe apologise for not elaborating on the methods limitations. We did slightly expand on the limitations in the supplementary material (Appendix A), but we will further expand the discussion to include:\n\n- A-GPS does not perform well on the GMM test function, we will include this result and attempt to explain the reason (still under investigation).\n- Miss-specification of prior, including overfitting, or poor generative model choice. This is something we have noticed in high dimensional settings, especially with small initial training datasets.\n- Extreme class imbalance when only a few Pareto-optimal solutions exist can lead to overly exploitative behaviour. We have subsequently addressed this --- see the discussion with reviewer VCzN.\n- Incoporating constraints on the design space is less straight-forward with generative models compared to direct design optimization.\n- A possible future work direction is to enhance the precision of the conditional generation as noted by reviewer yJhX.\n\n**Q3: Each evaluation appears to have detailed setups for both the baselines and proposed method, which differ across test functions. For example, the model sometimes uses an LSTM and sometimes uses a transformer. However, the performance varies widely across test functions.**\n\nWe should clarify --- in both the Ehrlich vs. Naturalness and RaSP vs. Naturaless experiments we always use both Transformer and LSTM generative models (we do not select between them depending on the experiment).\n\nWe do notice in the original experiments that the performance can differ greatly between these generative models. After some digging, we realise that our prior fitting method was too naive, and could cause the Transformers to overfit, especially with such little training data. Subsequently, we have changed the procedure to include 20% dropout (deactivated for the A-ELBO phase), and use 10% held out data to assess convergence. This has lead to much more consistent behaviour, where the transformers and LSTM models perform similarly for shorter sequences ($M \\leq 32$), and Transformers generally pull ahead of LSTMs for longer sequences ($M>64$). You can see evidence of this improved consistency in the sample ablation results requested by reviewer uzmJ. Our recommendation to practitioners would be to use Transformers for sequence design tasks so long as they are well validated.\n\n**Q4: How would you use A-GPS to approach a new problem apriori, including deciding which model to use?**\n\nThis is something we will be happy to include. We have used the same configuration for all synthetic test functions, and it seems to generalise decently. For high dimensional sequences, we do have a few recommendations, e.g.:\n\n- Generally transformers are better generative models (and more numerically stable) than LSTMs, when properly initialised.\n- We find the sequence length specific configurations given by the VSD authors work well for these problems too.\n- Our CNN CPE architecture seems to perform well with minimal reconfiguration for all sequence problems we have attempted. For longer sequences we suggest larger kernels for the CNN. Ensembles generally to not appear to make much of a performance difference.\n- Make sure the individual components are well validated on the training data where possible --- if anything, slight over-fitting the CPEs leads to preferable outcomes compared to under-fitting, as the prior provides additional regularisation.\n- Using round-varying Pareto-rank based labelling (see discussion with reviewer VCzN) makes the method more robust to cases where there are only a few Pareto-optimal solutions. We typically start with a threshold of the 75th-percentile of ranks, and by t=T, end with the Pareto set.\n\nWe will add this to the paper, or at least to the appendix and reference it in the main text."}, {"review_id": "FJJz1rK9qI", "reviewer": "Reviewer_VCzN", "summary": "This paper proposes Amortized Conditional Pareto Estimation (A-CPE), a conditional generative modeling framework for online multi-objective optimization (MOO) that enables preference specification at inference time. The authors develop an autoregressive variational distribution $q_{\\phi}(x∣u)$ (motivated by their sequence design tasks) conditioned on preference vectors $u$, which are sampled from an empirical approximation of $p(u∣z)$to amortize over $u$. Pareto-optimality and preference-alignment are modeled using discriminative classifiers $p(z∣x,u)$ and $p(a∣x,u)$, which are integrated into the ELBO objective. To accelerate optimization, the authors employ importance-weighted sampling for gradient estimation and only resample new trajectories when the likelihood ratio under significantly changes. The method is applied online per BO iteration and demonstrates competitive performance to baselines like qEHVI and qNEHVI on bi-objective benchmarks, including both continuous input and discrete sequence design tasks.", "questions": "**On the benefit of a generative model**: In low-dimensional continuous input spaces (e.g., Sec. 6.1), what is the practical advantage of using a generative model over directly optimizing $p(z=1|x,u)p(u|z=1)$ via gradient-based or heuristic methods? A controlled comparison against direct optimization would clarify the end-to-end utility of the generative model.", "limitations": "- As noted above, the reliance on an empirical $p(u|z)$ may limit exploration unless preferences are explicitly provided. This should be acknowledged as a limitation or mitigated.\n- The scalability of the method on the number of objectives remains unclear. In particular, amortizing over preference vectors  becomes increasingly difficult as the number of objectives grows, a clear elaboration of its scalability boundary, or a dicussion of future work or a limitation claim should be done on this.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- Introduces a flexible generative method for conditional Pareto set sampling that allows user preference specification at inference time without retraining.\n- The use of an autoregressive sequence model is appropriate for discrete design problems and mitigates the difficulty of searching over combinatorial spaces.\n- Avoids explicit hypervolume computation, potentially improving scalability.\n- The paper is well-written. \n\nWeakness\n- **Key concern** on $q(u|z)$ and random sampling step in Algorithm 1 (lines 11–12) relies on previously observed Pareto-optimal points. In settings where user preferences are not specified and the initial data has poor Pareto Frontier coverage, this empirical distribution will not support sufficient exploration of the Pareto front. This should be either addressed explicitly or the scope of the method should be reframed as targeting preference-aware MOO, rather than general MOO.\n- Time and computational cost profiling is missing. A breakdown of time per component (CPE training, generative sampling, ELBO optimization) would help assess scalability.\n- While the paper claims to avoid costly hypervolume (HV) computation, in bi-objective settings HV is computationally cheap. An empirical demonstration in higher-dimensional MOO (e.g., 4–5 objectives) would strengthen the claim of improved efficiency over HV-based methods.\n\nI would be happy to reconsider my score if the author could clarify my key concern and the question below.", "was_revised": true, "final_justification": "I think regarding the concern about $q(u∣z)$ potentially hindering exploration, the authors have referenced some in-house experiments suggesting that it does not cause significant differences with varying their choice of $z$, and have indicated they will incorporate further analysis in an updated version of the paper. Since new figures cannot be provided at this stage, and based on the overall consistency of the other responses, I choose to accept this explanation and have adjusted my score accordingly.", "rebuttal": "We thank reviewer VCzN for the thoughtful feedback, and will attempt to address their concerns and questions now.\n\n**Q1: Key concern on $q(u|z)$ and random sampling step in Algorithm 1 (lines 11–12) relies on previously observed Pareto-optimal points. In settings where user preferences are not specified and the initial data has poor Pareto Frontier coverage, this empirical distribution will not support sufficient exploration of the Pareto front. This should be either addressed explicitly or the scope of the method should be reframed as targeting preference-aware MOO, rather than general MOO.**\n\nThank you for this observation. Just as point of clarification, we use a constrained mixture of Gaussians over preferences, rather than an empirical distribution, for all experiments. We also study the consequence of an empirical distribution in Appendix E.2. Even so, after the submission of this version of the paper we were also concerned that a small Pareto set could lead to overly exploitative behaviour. We have subsequently made two small adjustments to A-GPS which we believe rectify this issue.\n\n1) For the first round, we fit an unconditional $q(u)$ using all available $y$.\n2) Rather than just using Pareto dominance labels, $z = 1[x \\in \\mathcal{S}]$, we\n  can use the Pareto ranking method (non-dominance sorting) described in [Deb, 2002], and then specify labels based on a thresholded rank ($s$), $z = 1[x \\in \\\\{\\bigcup \\mathcal{S}_s : \\forall s \\leq \\tau_t \\\\}]$. Here $s=1$ indicates the Pareto set, $s=2$ the next non-dominated set (shell) once $\\mathcal{S}_1$ is removed, etc. We are then able to choose a threshold, $\\tau_t$, per round $t$ using the same mechanisms as the original VSD paper --- and we find the annealed quantile method, Equation 20 of [Steinberg, 2025] (on negative ranks, $-s$), directly applicable. We typically set $\\tau_0$ to the 75th-percentile of (negative) ranks, and $\\tau_T$ so only the Pareto set is positively labelled.\n\nWe do not find much difference in the experimental results with these\nmodifications, with the exception of the ZDT3 synthetic test function, in which\nwe see an improvement for A-GPS. We report updated round-10 results to reviewer uzmJ.\n\n**Q2: Time and computational cost profiling is missing. A breakdown of time per component (CPE training, generative sampling, ELBO optimization) would help assess scalability.**\n\nWe have reported overall run-times for the methods in the supplementary material, Appendix E.1. We will expand upon these though for all seqeunce experiments. For example, here is the run-time\nsummary (in minutes) of the additional methods for the Ehrlich vs. Naturalness experiments, for M = 64, as well as VSD and A-GPS (which are in the Appendix)\n\n| Method | mean | min | max |\n|-|-|-|-|\nA-GPS-LSTM | 6.66 | 5.70 | 8.54\nA-GPS-TFM | 9.52 | 7.51 | 11.49\nVSD-LSTM | 6.11 | 4.63 | 7.62\nVSD-TFM | 7.97 | 7.11 | 8.27\nCbAS-LSTM | 3.56 | 3.35 | 3.72\nCbAS-TFM | 5.20 | 5.08 | 5.40\nLaMBO-2 | 2.32 | 2.30 | 2.33\nRandom (greedy) | 0.07 | 0.07 | 0.07\n\nThe runtime of the CPEs depends on many factors, such as their architecture,\ntraining epochs etc, but typically we find them to be less than 10% of the\ntotal runtime --- which is dominated by the A-ELBO computation. These timings\nabove, and those in Appendix E.1 should hopefully provide insight as to the\ncontribution of the generative model sampling in the runtime. We can include\nthis discussion in the paper. The main determinant of scalability is the expense of sampling the generative model. For a transformer this is quadratic in sequence length.\n\n**Q3: While the paper claims to avoid costly hypervolume (HV) computation, in bi-objective settings HV is computationally cheap. An empirical demonstration in higher-dimensional MOO (e.g., 4–5 objectives) would strengthen the claim of improved efficiency over HV-based methods.**\n\nThis is an excellent point, and we will include additional results summarised below for round 10 on two higher dimensional problems,\n\n| Test function ($L$) | Method | Mean round 10 relative HVI (1 std.) |\n|-|-|-|\n|DTLZ7 ($L=6$)| A-GPS | 3.073 (1.363) |\n| | qParEGO | 2.313 (1.205) |\n|DTLZ2 ($L=5$)| A-GPS | 1.053 (0.022) |\n| | qParEGO | 1.042 (0.027) |\n\nEvaluating these problems with EHVI-based is at or beyond their limits (it is claimed qNEHVI can scale to $L=5$, but we could not run the method in time for this rebuttal). We hope this helps to highlight the efficacy of our method.\n\n**Q4: On the benefit of a generative model: In low-dimensional continuous input spaces (e.g., Sec. 6.1), what is the practical advantage of using a generative model over directly optimizing $p(z=1 | x, u)p(u|z=1)$ via gradient-based or heuristic methods? A controlled comparison against direct optimization would clarify the end-to-end utility of the generative model.**\n\nThis is an interesting question, however, note that\n\n$\\text{argmax}_x p(z=1 | x, u)p(u|z=1)$\n\nWill optimize $x$ independently of $p(u|z=1)$, potentially ignoring any preferences. Perhaps a better alternative would be,\n\n$\\text{argmax}\\_x \\mathbb{E}_{p(u|z=1)}[p(z=1 | x, u)p(a=1 | x, u)]$,\n\nwhere the expectation could be approximated with samples (corresponding to batch size). Or for a particular preference $u_\\star$,\n\n$\\text{argmax}\\_x p(z=1 | x, u_\\star) p(a=1 | x, u_\\star)$.\n\nThis will not yield an _amortized_ model for preferences, and each new preference, $u_\\star$, _will require a separate optimization_. In low dimensional spaces, this may not be costly, in which case this is probably less direct and effective than scalarization methods like MBORE [10] or qParEGO [8]. We expect the benefit of A-GPS in the, low dimensional, continuous case comes from its stochastic nature, perhaps analogously to stochastic vs. deterministic optimization algorithms providing some robustness to local optima. If the reviewer still feels such a study is of benefit, this (modified) direct optimization is simple to implement as an ablation.\n\nOur focus is really on high-dimensional and discrete optimization problems, where direct optimization is impossible without e.g. latent space optimization or reparameterisation via a generative model and the utility of our method is clear --- we will clarify this in the text.\n\n**Q5: The scalability of the method on the number of objectives remains unclear. In particular, amortizing over preference vectors becomes increasingly difficult as the number of objectives grows, a clear elaboration of its scalability boundary, or a dicussion of future work or a limitation claim should be done on this.**\n\nAs can be seen be the DTLZ7 ($L=6$) and DTLZ2 ($L=5$) experiments the method scales more effectively than some of its EHVI-based competitors --- the main cost being the non-dominance checking, which is worst case $\\mathcal{O}(LN^2)$, or more typically $\\mathcal{O}(LN \\log N)$. We also visualised a (PCA) projection of the preference vectors and sampled points, like that in Figure 2, and found nice separated clustering of samples for each input vector. Sadly we are unable to share the figures, but these will be in the updated version of the manuscript, along with the hyper-volume improvement results.\n\n---\n\n[Deb, 2002] K. Deb, A. Pratap, S. Agarwal and T. Meyarivan, \"A fast and elitist multiobjective genetic algorithm: NSGA-II,\" in IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182-197, April 2002, doi: 10.1109/4235.996017."}, {"review_id": "V8ymUnUUOw", "reviewer": "Reviewer_yJhX", "summary": "The authors propose a method for generating Pareto fronts for black-box multi-objective optimisation tasks. They achieve this by using a generative model that creates samples conditioned on being on the Pareto set, and on alignment with preference vectors set by the user.\n\nThe modelling is based on active generation of sets guided by a class probability estimator. Where an estimator is first fit to model the distribution $p(z = 1 | x)$, and then a generative model is created to approximate $p(x | z)$. In this work, user preferences are further incorporated by considering alignment with preference vectors $u$. This results in the need to first approximate three distributions: $p(z | x, u)$ and $p(a | x, u)$ which are modelled using CPEs, and $p(u | z)$ which is modelled with  a von Mises-Fischer distribution and MLE. All the distributions together are then used to fit the ELBO of the generative model using an off-policy gradient estimator with importance weights.\n\nThe model is then shown to work well in a variety of real-world benchmarks.", "questions": "- Generally see my \"weaknesses\" section.\n\n- Is there a way a user can set a preference over the spread of the generation in the model?\n\n- Why do you think that your method which does not calculate hyper-volume improvement, beats methods that do go through the explicit calculations and try to maximise it?", "limitations": "No, the authors claim to discuss limitations in the related work section, however, these are not spelled out clearly. What the limitations are should be written more clearly.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- Quality: The paper is technically sounds, uses well known techniques and justifies the methods.\n\n- Clarity: The paper is well written, with good figures, equations, and tables. Most of the concepts get across well and can be understood easily.\n\n- Significance: Pareto front generation is of great importance in many scientific disciplines, and this paper proposes a good solution to the problem.\n\n- Originality: The general ideas of the paper rely on previous work of Steinberg et al. (2025) (ICLR), however, there is enough extensions and challenges introduced.\n\nWeaknesses:\n\n- Quality: I don't think the training of alignment vectors is well justified, indeed intuitively if $x_1$ and $x_2$ are close in space, the alignment should reflect stronger than if they were apart. However, by training on the permutations only there is no difference between the two scenarios. This is particularly concerning when we see in the experiment section, that the spread of points conditions on specific preferences can still be wide. Furthermore, the fact that pointing a preference vector ever so slightly outside the Pareto front results in poor suggestions from the algorithm is concerning, and is probably related to this fact.\n\nFurthermore, in the results, it appears that some of the methods have been implemented poorly, e.g. why is CbAS-TFM always outperformed by Random search.\n\n- Clarity: How to set the preference vectors remains a little unclear. I think a clearer example of how a practitioner would go about choosing the vectors and how this would affect the Pareto front would improve the paper. Additionally, I find it confusing about what the paper means by \"amortised\", since indeed there is re-training of the generative model at each iteration of the optimisation loop which results in a large expensive procedure between iterations -- the only amortisation that I understood related to changing preference vectors however I think this makes the paper premise and title misleading.\n\nOverall, I think the paper could be improved to address the alignment issue, which is both the main strength of the method but appears to have many pitfalls. However, it is mostly technically sound and addresses an important area, so it is in the borderline.\n\nMinor comments:\n- In Figure 3, we see the lines change for Random between the first plot and the rest, making it difficult to read\n- In Eq. (17) should the D_KL lie outside the inner expectation?", "was_revised": true, "final_justification": "Even though the rebuttal made some promising points, I keep my score, as it is difficult to verify if everything in the rebuttal will be added to the paper entirely, so I think the paper would benefit from one final iteration before publication.", "rebuttal": "Thank you to reviewer yJhX for the challenging questions. We will attempt to answer them below.\n\n**Q1: I don't think the training of alignment vectors is well justified, indeed intuitively if $x_1$ and $x_2$ are close in space, the alignment should reflect stronger than if they were apart ... the fact that pointing a preference vector ever so slightly outside the Pareto front results in poor suggestions from the algorithm is concerning, and is probably related to this fact.**\n\nThank you for this feedback. One main motivation for using the permutation method for training the alignment CPE was its simplicity --- A-GPS is already complex, and we do not wish to add more hyperparameters if they are not essential. To this end, we have found that increasing the amount of permuted pairs, $(u_{\\rho(n)}, x_n)$, to the aligned pairs $(u_n, x_n)$ by about 5:1 drastically improves the performance of the alignment component of A-GPS, and appears to improve the issue seen with the BraninCurrin test function. This procedure is similar to noise contrastive estimation (particularly InfoNCE, that uses negative example pairs), and MaxEnt estimation (where only positive labels exist, and the negative pairs are background points). This procedure rewards an estimator for discriminating relevant examples (aligned pairs) from irrelevant unaligned background/noisy samples.\n\nFurthermore, we were reluctant to make additional smoothness assumptions about the $\\mathcal{X} \\times \\mathcal{U}$ space, since in many applications these may not be valid. For example, one amino acid change in the active site of a protein can drastically impact the protein's properties (e.g. if a hydrophilic amino acid is swapped for a hydrophobic one both structural and functional properties are effected), changing the location of the protein in $\\mathcal{Y}$ and hence also $\\mathcal{U}$. We can include this point in the paper, and flag this as a direction for future investigation/improvement.\n\n**Q2: It appears that some of the methods have been implemented poorly, e.g. why is CbAS-TFM always outperformed by Random search**\n\nWe use the same implementation as [Steinberg, 2025] used in comparison to VSD --- though the authors do note that it is originally designed as an offline method, and it has been adapted to the online setting. However, we did notice the performance of the method improve when we changed the labelling strategy to Pareto rank thresholding, as discussed with reviewer VCzN. This is also more inline with how CbAS is used in both the VSD paper, and in the original implementation --- increasing thresholds. We also found an issue with the LaMBO2 baseline (as implemented in poli-baselines), where the code was not using the correct acquisition function for MOO settings. We have fixed this, and have seen a marked improvement in the performance of the method. However, We still find A-GPS and VSD generally outperform these baselines.\n\n**Q3: How to set the preference vectors remains a little unclear. I think a clearer example of how a practitioner would go about choosing the vectors and how this would affect the Pareto front would improve the paper**\n\nThank you for the feedback. We will happily clarify this in the paper. Assuming we start with a trained model, $q(x|u)$, perhaps the easiest way for a user to choose a preference vector, $u_\\star$, for generation of designs is to start with a desired outcome, $y_\\star$. We can then use Equation 9 to normalise this to a preference direction vector, $u_\\star$, to input to $x_\\star \\sim q(x|u=u_\\star)$. To control Pareto front exploration, we could parameterise a $q(u)$ directly, rather than learning similarly to that in [25]. We have opted to learn a $q(u)$ in our presentation as we are aiming for maximising the discovered hyper-volume in the experiments.\n\n**Q4: I find it confusing about what the paper means by \"amortised\", since indeed there is re-training of the generative model at each iteration of the optimisation loop which results in a large expensive procedure between iterations -- the only amortisation that I understood related to changing preference vectors however I think this makes the paper premise and title misleading.**\n\nAmortization in the context of variational inference typically refers to using a parameterized global inference function for the variational distribution, $q$, in contrast to the factored mean-field (independent) approach for each latent variable/parameter. This term came into being to describe variational models like Kingma and Welling's VAE, which can learn conditional distributions, $q(z|x)$, that can be queried with new $x$ without retraining. We can view our variational distribution, $q(x|u)$ in the same way --- we can query $q$ for various $u$ without the need for re-training, as factored mean field (and other MOBO) approaches would require. We can clarify this usage in the text, and perhaps we can re-word line 205 \"amortization->retraining\" if this was the source of confusion.\n\n**Q5: In Figure 3, we see the lines change for Random between the first plot and the rest, making it difficult to read**\n\nWe found keeping the order of the colouring consistent with algorithm performance was clearer for these sorts of plots, but we can re-style if this was confusing, and keep the style aligned with the methods.\n\n**Q6: In Eq. (17) should the $D_{KL}$ lie outside the inner expectation?**\n\nThe $D_{KL}$ term has to remain inside the expectation to be consistent with our objective in Equation 10. We can see this in the expansion in Equation 13, and then noting $q(u) \\approx p(u|z)$.\n\n**Q7: Is there a way a user can set a preference over the spread of the generation in the model?**\n\nProbably the easiest way to do this post-learning would be to query $q(x|u)$ with multiple $u$, or draw them from a specific distribution --- e.g. bounded uniform over a subset of $\\mathcal{U}$. Our framework also supports specifying a $q(u)$ rather than learning one during optimization if that is sensible for a particular application.\n\n**Q8: Why do you think that your method which does not calculate hyper-volume improvement, beats methods that do go through the explicit calculations and try to maximise it?**\n\nThis is a very good question, and we believe it is because in quite general circumstances the Pareto non-dominance label _is equivalent to a hyper-volume improvement indicator_ if a box-decomposition method is used. We sketch this now; if we define hyper-volume improvement,\n\n$\\text{HVI}(x) = \\text{HV}(\\mathcal{F} \\cup \\\\{y(x)\\\\}) - \\text{HV}(\\mathcal{F})$,\n\nwhere $\\text{HV}$ is hyper-volume, and $\\mathcal{F}$ is the Pareto front. We define hyper-volume as a union of the hyper-volume of hyper-rectangles defined by the vertices,\n\n$\\text{HV}(\\mathcal{F}) = \\lambda_L (\\bigcup_{y \\in \\mathcal{F}}[r, y])$,\n\nfor a reference point $r \\in \\mathbb{R}^L$ and a measure, $\\lambda_L$. Then the hyper-volume improvement indicator is, $1[\\text{HVI}(x) > 0]$. We make two observations,\n\n1. If $y(x)$ is dominated, then $z = 1[x \\in \\mathcal{S}] = 0$, but also the hypervolume is unchanged since $\\lambda_L([r, y(x)]) \\subseteq \\text{HV}(\\mathcal{F})$, and so $1[\\text{HVI}(x) > 0] = 0$.\n2. If $y(x)$ is not dominated, then $z = 1[x \\in \\mathcal{S}] = 1$ and also $\\lambda_L([r, y(x)]) \\not\\subseteq \\text{HV}(\\mathcal{F})$ which means it must contribute positive volume, and so $1[\\text{HVI}(x) > 0] = 1$.\n\nThis requires the reference point to be dominated by every feasible point, and also that there are no ties. The latter is not true for objectives that are discrete, and so in these circumstances the dominance indicator is not the same as HVI.\n\nTherefore, in the continuous case, with a proper loss (e.g. logistic), training a class probability estimator on the dominance indicator should also estimate the probability of hyper-volume acquisition. This nicely parallels the probability of improvement setting in VSD, and we can extend our paper to present this."}]

Author rebuttal:

Meta-review: This paper proposes Active Generation of Pareto Sets (A-GPS), a framework for online discrete black-box multi-objective optimization (MOO) that learns a generative model of the Pareto set supporting a-posteriori preference conditioning. The key scientific claims include: (1) A-GPS can effectively amortize over user preferences without requiring retraining; (2) The method avoids expensive hypervolume computation through a class probability estimator (CPE) for Pareto-optimality prediction; (3) A-GPS demonstrates competitive performance against established baselines like qEHVI and qNEHVI on both synthetic and real-world tasks including protein design benchmarks.

Strengths:
- Motivation and relevance
- Originality + novel methodological contributions
- Incorporation of preferences via conditional generative model is interesting and useful
- Technical soundness
- Empirical performance (with caveats due to some issues with evaluation)

Weaknesses:
- Empirical evaluation limited in terms of problems, baselines, and ablation studies (partially addressed during rebuttal)
- Insufficient discussion of implementation details

Reviewers generally agreed on the strong motivation and relevance of the work and the clarity of the presentation. They highlighted the novelty of the approach and the corresponding methodological contributions, arguing that while the general ideas of the paper rely on previous work by Steinberg et al. (2025), the specific setting in the paper considers additional challenges and extensions that go substantially beyond that work. They also found the performance of the method to be promising (subject to the caveats below). 

Reviewers’ concerns concentrated on the lack of rigor in the empirical evaluation, highlighting the limited coverage of problems and baselines, missing ablation studies, lack of statistical significance of some results, insufficient discussion of implementation details, and some concerns about implementation quality. In turn, the authors provided a comprehensive rebuttal addressing the different points raised. Reviewers appreciated the detailed response, and found that the authors addressed most of their concerns*, including those about reproducibility and ablations. Reviewers also appreciated the additional proof (sketch) for the equivalence between Pareto CPE and PHVI estimation.

*One thing that I believe has not been sufficiently addressed is the high variance in the experimental results. Even with the increased number of replicates, the differences between the methods are often not statistically significant. I don’t think this is a critical flaw since the paper provides a new approach to the problem (rather than just claiming to improve performance), but it is something that I would like to understand better through additional replications or problems where differences are more clearly pronounced. 

Overall, the strength of the paper significantly improved during the rebuttal and author discussion period, and reviewers generally consider the work a meaningful contribution. At this point, the primary question is whether the required changes to the manuscript are substantial enough to require a completely new review cycle. The opinions of the reviewers on this diverge somewhat and I can see both sides of the argument. However, as many if not all of the results on the requested improvements have already been provided as part of the rebuttal there is little risk in trusting the authors to properly incorporate these changes into their final manuscript. Therefore, I believe the submission should be accepted. 

That said, I do want to emphasize reviewer yJhX’s excellent point that a proper revision and additional review could “make the difference between an okay paper and a very good paper” and suggest the authors take that to heart when updating their manuscript.
Decision: Accept (poster)