Generative Bayesian Optimization: Generative Models as Acquisition Functions
Authors: Rafael Oliveira
Daniel M. Steinberg
Edwin V. Bonilla
Venue: ICLR 2026 Poster (2026)
Keywords: Bayesian optimization; generative models; black-box optimization
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We propose a general framework to turn generative models into solution samplers for black-box optimization problems

Abstract: We present a general strategy for turning generative models into candidate solution samplers for batch Bayesian optimization (BO). The use of generative models for BO enables large batch scaling as generative sampling, optimization of non-continuous design spaces, and high-dimensional and combinatorial design. Inspired by the success of direct preference optimization (DPO), we show that one can train a generative model with noisy, simple utility values directly computed from observations to then form proposal distributions whose densities are proportional to the expected utility, i.e., BO's acquisition function values. Furthermore, this approach is generalizable beyond preference-based feedback to general types of reward signals and loss functions. This perspective avoids the construction of surrogate (regression or classification) models, common in  previous methods that have used generative models for black-box optimization. Theoretically, we show that the generative models within the BO process follow a sequence of  distributions which asymptotically approximate an optimal target under certain conditions. We also evaluate the performance through experiments on challenging optimization problems involving large batches in high dimensions.

GENERATIVE BAYESIAN OPTIMIZATION: GENERATIVE MODELS AS ACQUISITION FUNCTIONS
Rafael Oliveira, Daniel M. Steinberg & Edwin V. Bonilla
CSIRO's Data61, Australia
(rofael.cossanteenee cosssaatttooooolieeerrr dan.steinberg, bevih.bomiile]abarad...c
ABSTRACT
We present general strategy for turning generative models into candidate solu- tion samplers for batch Bayesian optimization (BO). The use of generative models for BO enables large batch scaling generative sampling, optimization of non- continuous design spaces, and high-dimensional and combinatorial design. In- spired by the success of direct preference optimization (DPO), we show that one can train a generative model with noisy, simple utility values directly computed from observations then form proposal distributions whose densities propor- tional to the expected utility, i.e., BO's acquisition function values. Furthermore, this approach is generalizable beyond mreference-basee feedback to general types of reward signals and loss functions. This perspective avoids the construction of surrogate (regression classification) models, common previous methods that have used generative models for black-box optimization. Theoretically, we show that the generative models within the BO process follow a sequence of dis- tributions which asymptotically approximate an optimal target under certain con- ditions. We also evaluate the performance through experiments on challenging optimization problems involving large batches in high dimensions.
INTRODUCTION
Bayesian optimization (BO) has been a successful approach solve complex black-box optimiza- tion problems by making use of probabilistic surrogate models, such as Gaussian processes (GPs) (Rasmussen Williams, 2006), and their uncertainty estimates (Shahriari et al., 2016; Garnett, 2023). BO methods have been particularly useful in areas such as hyper-parameter tuning for ma- chine learning algorithms (Snock et al., 2012), material design (Frazier & Wang, 2016), and robot locomotion (Calandra et al., 2016). The core idea of BO is to apply Bayesian decision-theoretic framework to make optimal choices by maximizing an expected utility criterion, also known as an acquisition function. The corresponding expectations are taken under Bayesian posterior over the underlying objective function. Thus, the Bayesian model provides a principled way to account for the uncertainty inherent to the limited amount of data and the noisy observations.
In many applications such as simulated scenarios (Azimi et al., 2010), one able to run multiple evaluations of the objective function in parallel, even though the simulations themselves might be expensive to run. Common BO approaches to these batch settings incrementally build a set of can- didates by sampling "fantasy" observations from the probabilistic model and conditioning on them before selecting the next candidate in the batch (Wilson et al., 2018). Although near-optimal batches can be selected this way, this approach not scalable very large batches in high- nigh-dimensional spaces, such problems in protein design (Stanton et al., 2022; Gruver et al., 2023).
One of the most promising alternatives batch BO has been to train a generative model as a proposal distribution informed by the acquisition function and then sample a batch from the learned proposal (Brookes et al., 2019; Stanton et al., 2022: Gruver et al., 2023; Steinberg et al., 2025). This approach comes with several advantages. Firstly, given trained generative model, sampling is usually inex- pensive. Secondly, existing general-purpos generative models can be used and fine-tuned for the optimization task at hand. Lastly, sampling avoids estimating the global optimum of an acquisition function, which can be hard. However, existing generative approaches to black-box optimization usually rely on fitting surrogate (regression classification) model first then training generative

---

model on top of it (Stanton al., 2022; Gruver et al., 2023; Steinberg et al., 2025). This two-stage process compounds approximation errors from both models and can increase the computational cost significantly when compared to having a single model.
In this paper we present general framework for learning generative models for batch Bayesian optimization tasks that requires single model without the need for additional probabilistic regres- sion or classification surrogates. Our approach for generative BO (GenBO) encodes general utility functions into training objectives for generative models directly. We focus on Iwo cases, one where we train the model via a loss function for a reward model analogously to the direct preference opti- mization (DPO) formulation for large language models (Rafailov et al., 2023), and the second one where we train the generative model through divergence minimization, using utilities as part of sam- ple weights. We present theoretical analyses on the convergence of approximations and empirical results on practical applications involving high-dimensional combinatorial optimization problems.
2 BACKGROUND
We consider the problem of estimating the global optimum of an objective function f = - R as:
x² E argmax f(x),
(1)
xEX
where is an expensive-to-evaluat black-box function, i.e., Vxf is unavailable. We can only observe [(() via noisy evaluations y = [xx) + e, where is assumed sub-Gaussian (Pisier, 2016). We assume the objective I can be evaluated in parallel, and the algorithm is allowed to run up to T > 1 optimization rounds with batch of B query locations B, == {x C X' per round.
BO with regression models. Typically BO assumes a Bayesian prior over f (Garnett, 2023), often given by Gaussian process (Rasmussen & Williams, 2006). Given set of observations Di, corrupted by Gaussian noise e ~ NN, o²)), the Bayesian posterior distribution over f given the data D, is available in closed form as GP with known mean and covariance functions (see Appendix A)- BO then uses the model's posterior distribution to compute an acquisition function (x(x) mapping candidate points x E X' their expected utility value [[[(y) ) x D.]. where the utility function intuitively encodes how useful it is to collect a new observation at x. Classical examples of expected utilities include the probability of improvement (x))) == p(y 2 r|x, Dt) = E 2 r] x, Dr] and the expected improvement ar (x) == E[max(y - T, D.]]. The next candidate is then chosen as:
X1+1 E argmax a, (x).
XEx
Batch BO. This strategy can be extended to the batch setting in a variety of ways (Garnett, 2023, $11.3). For instance, one can select the first batch point Xx,1 by maximizing a, as above, and then select the next candidate as X1,2 E argmax x B[u(y) D, U {x 1., where the expectation is over both I1,1 p(y x,,, D:) and y ~ ply x, De U {xx,1, Wr.1)). and iterate over this process until B candidates have been selected for parallel evaluation. Although near optimal, evaluating this conditional expectation becomes quickly intractable as the batch size grows. Hence, one usually resorts to Monte Carlo approximations (Wilson et al., 2018). Other BO strategies allow for efficient optimization of the batch in parallel, such information-thheortie acquisition functions (Takeno et al., 2020; Teufel et al., 2024), or even asynchronously (Kandasamy et al., 2018). However, scal- ing up to large batches in nigh-dimensional domains, especially involving combinatorial or mixed contrnuu-o search spaces, remains challenging GGonzález-Duque et al., 2024).
Active generation with classification models. Instead of relying on a Bayesian surrogate model for I and then computing an acquisition function a on top of it, one can model directly, which is the main idea behind likelihood BO (Song et al., 2022). On this line, methods like variational search distributions (VSD, Steinberg et al., 2025) and batch BORE (Oliveira et al., 2022) learn probabilistic classifier ा (xx & ply 2 r) in the original space, X. based on improvement labels : M[y 2 -] and then generate batches by approximately sampling B candidates from p(xly 2 r).
The classifier can be learned by, e.g., minimizing the cross-entropy loss:
LT(T) ==
Σ 2 log π (xx) + (1 22) log(1 - T Txx)))
(3)
2

---

Given a prior Po over X' and the classifier n. that minimizes Ln, over the current na
Bt data
points in Dr, we can now learn a generative model approximating p(x y 2 T, Dr) as:
qi argmax Ex-gloo Te (xx] DKL (9lpp))
(4)
which corresponds to evidence lower bound treating (x) = plu 2 7/x, D:) as likelihood.
Direct preference optimization. The process above for learning q: can be likened to the typi- cal fine-tuning of large language models (LLMs) via reinforcement learning with human feedback (RLHF, Bai et al., 2022), which would normally involve training the LLM as an RL agent with a reward model p. In practice, we do not directly observe rewards, but have access to Given a prompt's ss context c, corresponding to the RL state, let x X ~ q(x(c) denote two answers generated by an LLM q. with x denoting the answer preferred by the user, and x the dispre- ferred one. Having a dataset Di == {cr, x x Fi one can then learn a reward function by minimizing the negative log- likelihood under a preference model, such as Bradley & Terry (1952):
preferences.
(p) == (e,,
[log (p(c, x x*) - x(),
(5)
Having learned reward model Pn. RLHF trains the LLM as to approximate an agent's optimal
policy under Ps. Regularization based on the Kullback-Leibler (KL) divergence with respect
reference model gref is further added to improve stability. The optimal generative model then solves:
On E argmax EE E. xx a) [pn(c, - /SIDL (ret)rret.
(6)
Direct preference optimization (DPO, Rafailov al., 2023) removes the need for an explicit reward model by viewing the LLM itself through the lens of a reward model. It is not hard to show that, fixing a reward model p. the optimal solution Equation 6 given by:
{p(c) Uret Uret (x|c) exp (fploxx))
1
((xlc)
where <p(c)=- = Ex Tref (x|e) exp(8-1p(c x)) is the partition function at the given context Al- though it is intractable to evaluate Sp in practice, DPO uses the fact that, in the Bradley-Terry model, the partition Lunction-depender terms cancel out. Note that the reward model p can be expressed in terms of the optimal g as:
g(x(c) (x )
plc,x) log
Blog Co(c).
(8)
Applying the substitution above to the preference-based loss (5), we get:
q(x lc) (x [c) 3 log
q(x (c) gret (x (c)
(g) - (ex log
Bog
which eliminates the partition function So terms. Therefore, we can train the generative model directly with LDPO without the need for an intermediate reward model. Such simplification to a single training loop cuts down the need for computational resources, eliminates source of approx- imation errors (from learning p). and brings in theoretical guarantees from Bradley-Terry models (Shah al., 2016; Bong & Rinaldo, 2022). The main question guiding our work is whether we can apply similar technique to simplify the training of (arbitrary, not necessarily LLM) generative models for likelihood-free frre BO by removing the need for an intermediate surrogate model for f.
A GENERAL RECIPE FOR GENERATIVE BAYESIAN OPTIMIZATION
As seen in Section 2, using generative models for BO typically involves training a regression or classification model as an intermediate step then train the candidate generator. The use of an intermediate model demands additional computational resources and brings in further sources of approximation errors which may hinder performance. Hence, we propose a framework generative model directly from (noisy) observation values. The main idea is to train the model to approximate a target distribution proportional to BO's acquisition function and then use the learned generative model as proposal for the next query locations. There are different approaches to do so, some of which have been previously explored the literature for specific acquisition functions, such as the probability of improvement (Brookes et al., 2019; Steinberg al., 2025) and upper confidence bound (Yun et al., 2025). However, we here focus on a general recipe to turn a generative model into density following any acquisition function that can be expressed an expected utility.
train the
3

---

Utility functions. Consider likeihhood-fre BO setting (Song al., 2022), where aim to directly learn an acquisition function a : X' + R at every BO round E f1...., T) based on available data. If our acquisition function takes the form of an expected utility:
ar (x) B(u((x)
(10)
we can estimate from noisy samples {x,, 14,11111 where E(u,, [D--1] = Elue(x;) For example, we have:
1. Probability of improvement (PI): W1, Ily, 2 Tol:
2. Expected improvement (EI): U2 max(y; Tr, 0):
3. Simple regret (SR): Ur,i = yz:
given a threshold 7ı for improvement-basee utilities, e.g., T2 == mäxi<t Yi or quantile of the em- pirical marginal observations distribution (Tiao et al., 2021 1). A comprehensive summary of typical utility functions for BO can be found in Wilson al. (2018). The ones listed above, however, can be directly expressed as a function of the observations. We also use a soft-plus version of EI (sEI) u(y) softplus(y -)) in our experiments, which remains positive even at low value when y T.
BO with generative models. As an illustrative example, consider the case of PI where a(x) B|ily 2 -]] = ply 2 r|x), which has been previously applied to train generative models for black-box optimization via surrogates (Steinberg et al., 2025). Given sampler for the conditional distribution pxx 2 7), by Bayes rule, we recover the original PI as:
a(x) ply r/x) p(x 2 T T)p(u 2 T)
p(x|y 2 T)
(11)
Pu (x)
Pu(x)
As the prior Po usually known, and it can even be set as uninformative (x() x we see that learning generative model to approximate the posterior above is equivalent to learning a probabilis- tic classifier for the improvement event y 2 т. Moreover, we only have a probabilistic classifier approximating p(y 2 r(x), we still need to select candidate points via optimization over the clas- sification probabilities landscape, which can be highly non-convex presenting several local optima, as in the usual BO setting choose X as the (global) maximizer of the acquisition function a. In contrast, generative model provides us with a direct way to sample candidates x ~ p(xly 2 r) which will by default concentrate at the highest density regions, and consequently highest utility according to the model. Finally, note that this same reasoning can be extended to any other non- negative expected utility function by training the generative model to approximate:
p (x) x Po(x)a;(x)
(12)
or similarly pt (x) oc (x)) exp at (x), which allows for utilities that might take negative values.
Overview. Let Q P(X) be learnable family of probability distributions over domain X. We consider general loss functions of the form:
L((g) = X.R((1) Σ l(())
(13)
where & are individual losses over points x; E X' or pairs of points X,,1X E X' and their corresponding utility values, A an optional regularization factor, and R₁ Q -> [0, oc) is complexity penalty function. The algorithm then proceeds by learning proposal distribution as:
qt argmin L 9EQ
(14)
A batch B: : (x is sampled from the learned proposal qr. We evaluate the utilities u, (vr,s) with the collected observations Vt. s)))x,)), for i (1...., B), and repeat the cycle up to a given number of iterations T N. This process is summarized in Algorithm 1. In the following, describe approaches to formulate general loss functions for learning acquisition functions and how to ensure that the sequence of batches (B.) asymptotically concentrates at the optimum x*.
4

---

Algorithm 1: Generative BO
Input: Domain X. initial data Do fort t E {l...., T do
Gt E argminge@ 14-1(()
// Fit proposal distribution // Sample batch // Collect observations // Update data
Be i.i.d. 9t Dr,i ((),,) + C,b for i ,)....,B.
D. Di-1 U (Xi,i. (1,3)
3.1 PREFERENCE BASED LEARNING
We first aim to apply similar reparameterization trick to the one in DPO to simplify generative BO methods. Note that, for a general classification loss, such as the one in Equation 3, it is not possible to eliminate the partition function resulting from DPO-like reparameterization without resorting to approximations, which might change the learned model. Hence, we need a pairwise-contrastive training objective.
Preference loss. To apply preference- basee loss, rections of the acquisition function. Assume we have a dataset D.; = {xi, uy ) with n evaluations of given utility function IR + R. We may reorganize the data into pairs of inputs and corre- sponding utility values {x,, x, 2, 04 24 nn/2 J== where Us == u(y,g), for j E (1,2). and train generative model q using the Bradley-Terr preference loss from DPO with, for i n1//]:
train model to predict preferential di-
log 9(xx,1) (o(x))))
(xx,2)
(q, Au:) llg sign(AAu)
log
(15)
po (x
where Au; U4,1 U1 as the DPO formulation, 0 (optional) temperature parameter
and the prior Po can be given by a reference model, either pre-trained or derived from expert knowl- edge about feasible solutions to the optimization problem (1). Similar to Rafailov et al. (2023), the learned generative model is seeking approximate:
()
(x(x) xp((x)exp)
(16)
Su
where is the normalization factor.
Robust preference loss. As shown in Chowdhury et al. (2024), the original DPO loss is not robust to preference noise. As in BO, one usually only observes noisy evaluations of the objective function, utility values directly derived from the observation values will also be noisy and correspondingly the sign of their differences as well. Namely, assume there is a small Ptip E (0, 1/2) probability of the preference directions being flipped w.r.t. the sign of the true expected utility:
P ssign( U4,1 (4.2) sign [E] 2 E/u,, 1l)] Pfip
(17)
Chowdhury al. (2024) showed that the original DPO preference loss is biased in this noisy case, and proposed a robust version of the DPO loss to address this issue as:
(1 - Phip (PL Au;) - Phip (f AAu)
C:PL Au;)
(18)
2ptip
which yields the robust preference loss (rPL): L;PT (g) = Σn= C:PT Aus). It follows that the loss function above is unbiased and robust to observation noise.
3.2 DIVERGENCE- BASED LEARNING
A disadvantage of DPO- based losses when applied to BO is that they only take the signs of the pair- wise utility differences into account, discarding the remaining information contained in the magni- tude of the utilities. A simpler approach is to train the generative model q to match p directly.
5

---

Forward KL. If formulate the target distribution P (x() po(x))((x), the forward Kullback-Leblle (KL) divergence of the proposal w.r.t. the target is given by:
(pill)))g) Ex-p; pirr:;,loggge (x) log gx)]]
(19)
As we do not have samples from Pus at each iteration the algorithm generates samples from the
current best approximation B, == {x,,i) Gt- An unbiased training objective can then be formulated as:
ALL (q) - ((ys log q(x,),
(20)
(x;)
which we write in a condensed form to avoid notation clutter with qi = x and n corresponding
to the total number of observations up given round. The objective above is unbiased and its global optimum can be shown to converge to Py by an application of standard results from the adaptive importance sampling literature (Delyon & Portier, 2018). A simpler version of this training objective was derived for CbAS (Brookes et al., 2019) using only the last batch for training, which would allow for convergence as the batch size goes to infinity B oc. Furthermore, as we will in our analysis, convergence to P is not sufficient to ensure convergence to the global optima of the objective function f.
Balanced forward KL. As utilities like those of Pl and EI can evaluate to 0 at the points where y < was observed, with J corresponding to an improvement threshold, every point below the threshold will not be penalized by the loss function. As a result, the model may keep high probability densities in regions of low utility. To prevent this, we may use an alternative formulation of the forward KL which comes from the definition of Bregman divergences with the convex function
u + u log u, yielding a loss:
(x.) 9i-1 (x:)
((x,)
CLI (1)
u(vs) log (xx))
(21)
1x:) (x:)
We defer the details of the derivation to the appendix. Although the additional (xx) only contributes constant term when integrated over, for finite-sample approximations, it contributes soft penalty on points where it is observed that u(y) O.
3.3 GENERALIZATIONS
In general, extend the above framework to use a proper scoring rule S P(X) X R (Gneiting & Raftery, 2007) other than the log loss. We can then learn a q approximating PT by minimizing:
(g) = - po(x,) u ((,,) (xx)
(22)
Although we leave the exploration of this formulation for future work, it is readily extensible to other types of generative models which may not have densities available in closed form, such as diffusion and flow matching (Lipman et al.. 2024), which still provide flexible probabilistic models.
4 THEORETICAL ANALYSIS
In this section, we present theoretical analysis of the algorithm's approximation of the utility- based target distribution and its performance in regards to the global optimization problem (1). We consider parametric generative models qo with a given parameter space 6 E θ C R.M For the purpose of our analysis, we will assume that models can be described as (x(x) = exp (x)), which is always possible whenever densities are strictly positive qo (x) > O. To accommodate for both the pairwise preference-based losses and the point-based divergence approximations, we introduce the following notation for the loss function:
Ln (ge) = Rn (go) Σ (el((m
(23)
where (g) corresponds to the äth model evaluation with, e.g., ms := log (x,) for KL, and m,(#) := log (x.,) log go (xx for reference-based losses, 2; encodes the dependence on utility values with 2; := u(y:) for KL and 2; = sign(u:,1 (422) for DPO losses, and w, are potential importance weights.

---

Regularity assumptions. We make few mild regularity assumptions about the problem setting and the model. Firstly, for the analysis, we assume that both the models Jo lie in a reproducing kernel Hilbert space (RKHS) H₁ shared with the true log density 9.. which is such that p (x) exp g. (x). The domain X' assumed to be a compact metric space with main results specialized for the finite discrete setting, i.e., | < The individual losses & R IR R are strictly convex and twice differentiable w.r.t. their first argument. We also assume that the regularizer Rn is strongly convex and twice differentiable. The rest of our assumptions and proofs are presented in Appendix B.
Lemma 1. Let assumptions A2 to A4 be satisfied. Then,
2 l1g 9n 122 S Lg(g) Ln S 1 (
where Hn Hy + Hx is an operator-valued lower bound on the Hessian of the loss Ln:
Ag E Hk:
('Ln(o) :n := A + arbs Σ my 2 mi
Remark 1. The result in Lemma 1 automatically ensures that the loss functional L, is strongly convex, as L (g) > Hn Ž X, > 0, for all g E H., and therefore has a unique minimizer at In. defined in Section B.3. The same, however, cannot be implied about L,, (go) over O based solely on this result, since the mapping 0 + g(-, 0) might be non-linear.
Theorem 1. Let assumptions Al to A5 hold. Then, given any 5 E (0,
Vn E N. Km,.g)> {m, 90,, >al S (,( H, + 2(m,, ( Vm E Ha: which holds with probability at least 1 - 6, where (o) is given by Lemma 5, where ((x) = denotes the unit constant function, vn := Ex-o [10(x))l 2 and on := =.
The result above shows that the approximation error for the optimal parameter 6n concentrates similarly to that of kernel method, even though we do not require the model to be kernel machine.
In addition, the term [|m]]] - is associated with the predictive variance of a Gaussian process model,
which can be shown to converge to zero, e.g., infxex' (x 2 by > 0, for all 0 E θ (see Lemma 4 in the appendix). Alternatively, one can follow a KL-focused analysis as Oliveira al. (2021) to bound DKi. (go. ll9.) via the information gain, which bounds the growth of predictive uncertainty terms [m 12, nー appearing in Theorem The asymptotic rate of 3n discussed in Remark 4.
Optimality. Theorem allows to establish that the model converges to the target g+ associated with the target distribution P" for a given utility function u if ||m] I= vanishes sufficiently fast. However, convergence to the target distribution alone does not ensure optimality of the samples gt. The latter is possible by applying results from reward-weighted regression, which shows that training proposal to maximize E By=<pv/x) [u(y) log g(x)] yields a sequence of increasing expected rewards E[((v=)] & E[(( .... (Strupl et al., 2022, Thm. 4.1). If the maximizer of the sequence of expected utilities converges to the maximizer of the objective function f. then the generative BO proposals will concentrate the true optimum Therefore, for KL-based loss functions, one may drop the proposal densities in the importance sampling weights 1/9.-1 (x;) to promote this posterior concentration phenomenon, which generally did not include importance weights. This same concentration of the learned tar- get distribution should also occur with the preference-based loss functions due to the absence of ieportance-sampliin weights. We leave a rigorous theoretical regret analysis for future work.
corroborated by our experimental findings,
RELATED WORK
Using generative models for online- optimizzttiiziiionn is becoming an increasingly popular method for optimization in discrete, mixed bascrete-contnuoon high dimensional design spaces where clas- sical BO is limited. The following discusses other works applying generative models to BO settings and contrasts them with the reward-modellfiee active generation framework we propose.

---

Latent-space BO. In latent-space BO (LSBO) methods for high high-dimensional problems (Gómez- Bombarelli et al., 2018; Stanton et al., 2022: Gruver et al., 2023), one learns a probabilistic represen- tation of (usually space, projecting query points back to the original space at evaluation time. This technique has led to numerous BO methods for high-dimensional and discrete-space optimization
manifold of the data jointly with f, and performs BO in that
et al., 2018; Gruver et al., 2023; González- Duuuee et al., 2024). Learning this latent-space can, how- ever, cause complications. LSBO can suffer poor sample efficiency if the latent-space is learned from the initial training set and then fixed (Tripp et al., 2020). Or poor performance can arise from reconstruction errors between the latent and observation space (Lee al., 2025). GenBO and other methods like VSD do not suffer from these issues as all inference is done in the observation space. Despite recent advances in the field (Chu et al., 2024; Lee et al., 2025; Moss et al., 2025), to our knowledge, LaMBO-2 remains state-of-the-art in LSBO for long sequences, like proteins.
Diffusion for BBO, There has been recent progress in adapting denoising diffusion models to black-box optimization (BBO) tasks, often by learning model that can be conditioned on observa- tion values, given a dataset of evaluations (Krishnamoorthy et al., 2023). Other approaches involve guiding the diffusion process by given utility function derived from regression model (Gruver et al.. 2023; Yun et al.. 2025). Note, however, that such methodologies are specific to diffusion, whereas we focus on a general approach that can be applied arbitrary generative models.
LLMs and BO. Recent work has begun to integrate large language models (LLMs) into BO pipelines, primarily to inject prior knowledge, improve cold-start performance, or offload certain design decisions to learned policy. Several studies use LLMs as contextual priors over the design space: for example, guiding initialization or proposal generation by loveraging natural- natural-language do- main knowledge (Liu et al., 2024), selecting acquisition functions adaptively via an LLM--riven controller (Aglietti et al., 2025). Other work treats BO as a test-time search tool that an LLM can call to refine validate its own proposals during inference (Agarwal al., 2025). Most relevant to our setting is a recent reward-model-free approach for protein engineering (Chen et al., 2025), which uses LLM preference modeling, akin to DPO, steer search without explicit surrogate. This shares the reward-model-free philosophy of GenBO, but differs fundamentally in relying on a ceneral-purposs LLM, whereas GenBO provides framework for task-specific generative black-box optimization problems with no language interface or pretrained reward structure.
6 EXPERIMENTS
We evaluate several variants of generative BO (GenBO) on a number of challenging sequence op- timization tasks against popular and strong baselines, including CbAS (Brookes et al., 2019), VSD (Steinberg et al., 2025), and LaMBO-2 (Gruver et al., 2023), besides trivial baselines, random mu- tations and a genetic algorithm (GA) implemented in POLI (González-Duque et al., 2024). As per- formance measures, we assess the simple regret, Ti f(x") ) maxign. f(x:), and the cumulative maximum, maxiga, fxx;), where n, := Bt is the number of function evaluations up to round I. In legend boxes, algorithms are sorted in descending order of final average regret. Shaded areas cor- respond to #1 standard deviation across five different random seeds. Appendix C presents further details about experiment settings and ablation studies. Table 4 and 5 summarize final results.
6.1 TEXT OPTIMIZATION
As a first experiment, we wish to optimize a short sequence (5 letters) to minimize the edit distance to the sequence ALOHA, which is implemented as a POLI black-box (González-Duque al., 2024). Here X' = VM where V is the English alphabet, and M is sequence length. Even though this sequence is relatively short, still |x] = |V| > 11.8 million elements. We increase the difficulty by only allowing |Dol = 64 where the minimum edit distance is 4, B 8, and T = 10. We compare GenBO to the classifier guided VSD (Steinberg et al., 2025) and CbAS (Brookes et al., 2019), and to simple greedy baseline that applies (3) random mutations to its best candidates per- round (González-Dugge (independent) categorical proposal distribution, q, and a uniform prior, Po. VSD and CbAS use simple embedding and 1 --hidden layer MLP classifier for estimating PI. We also varied the threshold annealing schedule. Architectural details and other experimental specifies are given in Section C.
et al., 2024). For GenBO, VSD and CbAS we use a simple mean-field
8

---

Results summarized in Figure la. We headway and CbAS under-performs due to its limited use of data (last batch only) in retraining. For this experiment, GenBO with the robust preference loss (rPL) and El-based utilities showed the quickest improvements, whereas Pl is able to reach the exact optimum at the end, with VSI) eventually also achieving good performance. In Figure 5 (appendix), we present ablation study on the threshold annealing scheme we used to balance the exploration-exploitttioo trade-off for GenBO and PI-based baselines (VSD and CbAS). The plots reveal that this problem generally favors a more exploitative approach by concentrating on higher quantiles of the observations marginal distribution. GenBO was, however, relatively less sensitive the choice of annealing scheme, as long as the final percentile was set anywhere above 90%, whereas VSD required a generally sharper rise to above the 95% quantile towards the end of the optimization process, favoring original settings suggested by Steinberg et al. (2025). We also find that in this problem the use of pre-trained informative prior po may not bring significant performance advantage, as GenBO variants with no prior (i.c., Po x 1) performed best. Lastly, we also highlight significant improvements in run time for GenBO, making it on average 3 times faster than VSD (see Table 6 in the appendix) for not needing to fit an intermediate surrogate model.
that the random baseline is not able make much
Fandory TESMOOs
Chil-Tie
Sprple NOIIE
Maximum oulcomme
Meximumn CUTCCO
Ran PLENSOOE PAALF
Cargo PL--1-1
Jandon
11500
10
00
25
30
100
125
15.0
17.
200
00
25
58
35 100 125
15.0 115 30.
Bound
Bound
(a) ALOHA
(b) Stability
(e) SASA
Figure/ I: Performance of baseline black box optimizers and GenBO variants on the (a) ALOHA, (b) stability, and (c) solvent accessible surface area optimization problems.
0.91
Sirppe ER0INE
My
Simple reggrer
2.4
Callrrhh
0.95
GRel the
Caki
2.5
Ce tireth copel ri reuoe
0.4
Oorb0
UND
Ges3o-M1-8-01
Oer 11
Ges50 Thr rouueedd--
25
Round
Round
(a) M 15
(b) M 32
(c) M = 64
Figure 2: Simple regret of the baseline black box optimizers and the GenBO variants on the Ehrlich closed-form test function protein design task for varying sequence lengths, M.
6.2 PROTEIN DESIGN
We now consider three protein sequence design tasks where |V] = 20 and we have varying M. We again use VSD, CbAS and random mutation as baselines, and add to them the guided diffusion based LaMBO-2 (Gruver al., 2023). GenBO, VSD and CbAS all share the same generative backbone, which is the causal transformer used in Steinberg et al. (2025); VSD and CbAS also use the same CNN-classifeer guide used in that work. We present additional architectural information, and additional experimental details in Section C.2. We use the black-bor box implementations in POLI for these tasks, and POLI-BAS BASELINES implementations of the random and LaMBO-2 baselines.
The first task we consider is optimization of the Ehrlich functions introduced by Stanton et al. (2024). These are challenging biologically inspired parametric closed-form functions that explicitly

---

simulate nonlinear (epistatic) effects of sequence on outcome. The outcomes are y E (-13 U [0, 1] where -1 is reserved for infeasible sequences. We use the same protocol as in Steinberg et al. (2025), where we optimize sequences of length M {15, 32,64) 64) all with motif lengths of 4, and [Pn] = 128, T 32 and = 128. The results are summarized in Figure 2. We again see that GenBO variants able to outperform or match the performance of baselines, with KL.-based losses yielding the best performance. In higher dimensions with the longest sequence setting, the benefits of the balanced forward KL loss, with its density minimization effect in areas of lower utility, are more evident. In addition, we note that exponential regularization, corresponding to assuming an exponential dot- dot-product kernel for the RKHS feature space of the model (see Remark 2), allowed for the best performance in higher dimensions. Lastly, in Figure 6 (appendix), we present an ablation study on the batch size setting B, showing monotonic improvements, especially for large B 2 32.
Calé Ihf
200
the
Far Repeenuraaat Gav3C-aE14a0a
250
Clversdi saly
1i0--E Ca Ca10out--00-rp
Dlowedlly
320
Cev DD Ri
32
12.3
15a
220
00
55
095
12.
160
260
Round
Round
(a) Stability
(b) SASA
Figure 3; Batch diversity scores per round on the FoldX protein optimization tasks.
For our final set of experiments we present two real protein optimization tasks. These experiments have been adapted from Stanton et al. (2022) where the aims are to maximize the stability and solvent accessible surface (SASA) of the proteins, respectively. The black-box the FoldX molecular simulation software (Schymkowitz et al., 2005), and is wrapped by POLI (González-Duque et al., 2024). We chose the mRouge red fluorescent protein (M 228) as the base protein for the tasks. Both tasks were given T 20 rounds, a batch size of B = 64, and an initial training set of [Pol = 88 as a subset from Stanton et al. (2022). Results are summarized in Figure lb for stability and Figure Ic for SASA. All variants of GenBO find the stability task challenging, along with the LaMBO-2 and random baselines. CbAS and especially VSD are better able to stabilize this protein. As shown by diversity scores in Figure 3a, which we measure by averaging the Levenshtein distance across the batch in the same way as Steinberg et al. (2025), algorithmic baselines with the lowest diversity yielded top performance, indicating that pure exploitation from around the starting dataset led to the highest outcomes. However, most variants of GenBO far outperform the baselines on the SASA task, and much more rapidly. We believe this task favors extrapolation away from the prior, due to the high performance of GenBO variants with uninformative prior. In contrast to the stability, the diversity scores show that increasing exploration led to better outcomes for SASA (Figure 3b).
CONCLUSION
This work introduces Generative Bayesian Optimization (GenBO), unifying framework that turns any generative model into a sampler whose density tracks BO acquisition functions. We have shown that loss functions over generative models, such as DPO and KL divergences, can be applied to directly learn samplers for batch BO. By eliminating intermediate regression or classification surro- gates, GenBO reduces approximation error, simplifies the pipeline learning just single generative model. and scales naturally to large batches and high-dimenan -ige-iimmniional or combinatorial design spaces. Theoretical results show convergence to the target distribution, and experiments on text optimization and protein design tasks demonstrate competitive performance with more complex surrogate-guidee baselines. A few challenges remain. For some variants, GenBO requires choosing and fixing the prior before optimization, and its performance depends sensible settings of utility and temperature parameters, whose theory could be further explored. Another avenue is the adaptation to acquisition strategies not expressible expected utilities, such Thompson sampling and upper confidence bound. Despite these caveats, GenBO's s minimal moving parts and principled acquisition-dii training mark simpler and more scalable alternative to multi-stage guided generation methods.
10

---

REFERENCES
Yasin Abbasi- Yadkori. Online Learning for Linearly Parametrized Control Problems. PhD, Univer- sity of Alberta, 2012.
Abbasi-Yadkori.
Dhruv Agarwal, Manoj Ghuhan Arivazhagan, Rajarshi Das, Sandesh Swamy, Sopan Khosla, and Rashmi Gangadharaiah. Searching for optimal solutions with LLMs via Bayesian optimization. In The Thirteenth International Conference on Learning Representations, 2025.
Virginia Aglietti, Ira Ktena, Jessica Schrouff, Eleni Sgouritsa, Francisco Ruiz, Alan Malek, Alexis Bellot, and Silvia Chiappa. FunBO: Discovering acquisition functions for Bayesian optimization with funsearch. In Forty-second International Conference on Machine Learning, 2025.
Javad Azimi, Alan Fern, and Xiaoli Z. Fern. Batch Bayesian optimization via simulation matching. In Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010 (NIPS 2010). 2010.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Heraandz, Tristan Hume, Scott Johnston, Shauna Kravee, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv e-prints, art. MrXiv:2204005662 April 2022. doi: 10.48550/arXiv.22 220.000000...2220055.......
Heejong Bong and Alessandro Rinaldo. Generalized results for the existence and consistency of the MLE in the Bradley-Terry -erle---Tery--ucee model. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 2160- 2177. PMLR, 2022.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-344 345, 1952.
David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 773-782, Long Beach, CA, USA, 2019. PMLR.
Roberto Calandra, André Seyfarth, Jan Peters, and Mare Peter Deisenroth. Bayesian optimization for learning gaits under uncertainty. Annals of Mathematics and Artificial Intelligence. 76(1): 5-23, 2016. doi: 000077//10472--0--446-----
Angelica Chen, Samuel Don Stanton, Frances Ding, Robert G Alberstein, Andrew Martin Watkins, Richard Bonneau, Vladimir Gligorijevic, Kyunghyun Cho, and Nathan C. Frey. Generalists vs. specialists: Evaluating LLMs on highly-constrained biophysical sequence optimization tasks. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon macoste-Julient Felix Berkenkamp, Tegan Ma- haraj, Kiri Wagstaff, and Jerry Zhu (eds.), Proceedings of the 42nd International Conference Machine Learning. volume 267 of Proceedings of Machine Learning Research, pp. 9029-9072. PMLR, 13-19 Jul 2025.
Sayak Ray Chowdhury and Aditya Gopalan. On Kernelized Multi-armed Bandits. In Proceedings of the 34th International Conference on Machine Learning (ICML). Sydney, Australia, 2017.
Sayak Ray Chowdhury, Anush Kini, and Nagarajan Natarajan. Provably robust DPO: Aligning language models with noisy feedback. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41 International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 42258-42274, PMLR, 2024.
Jaewon Chu, Jinyoung Park, Seunghun Lee, and Hyunwoo J. Kim. Inversion-basee latent Bayesian optimization. In The ehirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
11

---

Zhongxiang Dai, Yao Shu, Bryan Kian Hsiang Low, and Patrick Jaillet. Sample-thens -hepll--ten-otimizze batch neural Thompson sampling. In 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, LA. USA. 2022.
Bernard Delyon and François Portier. Asymptotic optimality of adaptive importance sampling. In S Bengio, H Wallach, H Larochelle, K Grauman, N Cesa-Bianchi, and R Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 31, Montréal, Canada, 2018. Curran Associates, Inc.
Audrey Durand, Odalric-Ambrym Maillard, and Joelle Pineau. Streaming kernel regression with provably adaptive mean, variance, and regularization. Journal of Machine Learning Research, 19 (1):650-683, 2018
Peter I. Frazier and Jialei Wang. Bayesian Optimization for Materials Design, pp. 45-75. Springer International Publishing, Cham, 2016. ISBN 978-3-319-2387-55 doi: 10.1007/ 978-3-319-238777 3.
Roman Garnett. Bayesian Optimization. Cambridge University Press, 2023. URL https://
boye.. com/.
Tilmann Gneiting and Adrian E Raftery. Strietly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-378 2007.
Rafael Gómezz-ombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengelingg Dennis Sheberla, Jorge Aguilera-Iparraauinnn Timothy Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. Automatic chemical design using data-driven continuous representation of molecules. ACS Central Science, 4(2):268-276, 2018. doi: (1022///ccceetssii700057222
Miguel González-Duque, Richard Michael, Simon Bartels, Yevgen Zainchkovskyy, Søren Hauberg, and Wouter Boomsma. A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences. The 38th Conference on Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, Vancouver, Canada, 2024.
Miguel González-Duque Duqqee Simon Bartels, and Richard Michael. Poli: A library of discrete sequence objectives, 2024. URL https: gi tituub......... com
feScience!
poli.
Nate Gruver, Samuel Stanton, Nathan Frey, Tim G.J. Rudner, Isidro Hotzel, Julien Lafrance- Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffusion. In Advances in Neural Information Processing Systems, volume 36, New Orleans, LA, USA, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1026-1034, 2015.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kemnl:: Convergence and gen- eralization neural networks. Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 8580-8589, Red Hook, NY, USA, 2018. Curran Associates Inc.
Kirthevasan Kandasamy, Akshay Krishnamurthy, Jeff Schneider, and Barnabas Poczos. Asyn- chronous parallel Bayesian optimisation via Thompson sampling. Proceedings of the 2 International Conference Artificial Intelligence and Statistics (AISTATS), Lanzarote, Spain, 2018.
Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black- box optimization. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 17842-17857. PMLR, 2023.
12

---

Seunghun Lee, Jinyoung Park, Jaewon Chu, Minseo Yoon, and Hyunwoo J. Kim. Latent Bayesian optimization via autoregressive normalizing flows. In The Thirteenth International Conference on Learning Representations, 2025.
Yucen Lily Li, Tim G. J. Rudner, and Andrew Gordon Wilson. A study of Bayesian neural network surrogates for Bayesian optimization. In 2024 International Conference on Learning Represen- tations (ICLR), Vienna, Austria, 2024. OpenReview.
Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q- Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code. arXiv e-prints, 2024.
Tennison Liu, Nicolás Astorga, Nabeel Seedat, and Mihaela van der Schaar. Large language models to enhance Bayesian optimization. In The Twelfth International Conference Learning Repre- sentations, 2024.
Henry Moss, Sebastian W. Ober, and Tom Diethe. Return of the latent space COWBOYS: Re- thinking the use of VAEs for Bayesian optimisation of structured spaces. In Forty-second Inter- national Conference on Machine Learning. 2025.
Rafael Oliveira, Lionel Ott, and Fabio Ramos. No-regret approximate inference via Bayesian opti- misation. In 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021), 2021.
Rafael Oliveira, Louis Tiao, and Fabio T. Ramos. Batch Bayesian optimisation via density-ratio estimation with guarantees. Advances Neural Information Processing Systems, 35:29816- 29829, 2022.
Mary Phuong and Marcus Hutter. arXiv:2207.09233, 2022.
Formal algorithms for transformers. arXiv preprint
Gilles Pisier. Subgaussian sequences in probability and Fourier analysis. Graduate Journal of Mathematics, 1:59-78, 2016.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. 2023.
In Advances neural information processing systems, volume 36, pp.
Carl E. Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, Cambridge, MA, 2006.
Saburou Saitoh and Yoshihiro Sawano. Theory of Reproducing Kernels and Applications. Springer, 2016.
Bernhard Schölkopf and Alexander J. Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge, Mass, 2002.
Joost Schymkowitz, Jesper Borg, Francois Stricher, Robby Nys, Frederic Rousseau, and Luis Ser- rano. The foldx web server: an online force field. Nucleic Acids Research, 33:W382-W388, July 2005. ISSN 0305-1048. doi: 10. 10.10933nar/gk1337..
Nihar Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, and Martin J. Wainwright. Estimation from pairwise comparisons: Sharp minimax bounds with topology dependence. Journal of Machine Learning Research, 17, 2016.
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando De Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1): 148- 2016.
Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 25, pp. 2951-2959 Curran Associates, Inc., 2012.
13

---

Jiaming Song, Lantao Yu, Willie Neiswanger, and Stefano Ermon. A general recipe for likelihood- free Bayesian optimization. Proceedings of the 39th International Conference on Machine Learning (ICML), Baltimore, Maryland, USA, 2022. PMLR 162.
Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias Seeger. Gaussian process op- timization in the Bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 1015-1022, 2010.
Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Green- side, and Andrew Gordon Wilson. Accelerating Bayesian optimization for biological sequence design with denoising autoencoders. In International Conference on Machine Learning, pp- 40459-20478 PMLR, 2022.
Samuel Stanton, Robert Alberstein, Nathan Frey, Andrew Watkins, and Kyunghyun Cho. Closed-form test functions for biophysical sequence optimization algorithms. arXiv preprint arXiv:2407.00236, 2024.
Daniel M. Steinberg, Rafael Oliveira, Cheng Soon Ong, and Edwin V. Bonilla. Variational search distributions. In The Thirteenth International Conference on Learning Representations, Singa-
pore, 2025.
Ingo Steinwart and Andreas Christmann. Kernels and reproducing kernel Hilbert spaces. In Support Vector Machines, chapter 4, pp. 110-163. Springer, New York, NY, 2008
Miroslav Strupl, Francesco Faccio, Dylan R Ashley, Rupesh Kumar Srivastava, and Jürgen Schmid- huber. Reward-weighted regression converges to global optimum. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8361-8369, 2022.
Shion Takeno, Hitoshi Fukuoka, Yuhki Tsukada. Toshiyuki Koyama, Motoki Shiga, Ichiro Takeuchi, and Masayuki Karasuyama. Multi- Multi-fidelity Bayesian optimization with max-value entropy search and its parallelization. In 37th International Conference on Machine Learning (ICML 2020), volume 119 of Proceedings of Machine Learning Research, pp. 9276 011669927..00 PMLR, 2020.
Felix Teufel, Carsten Stahlhut, and Jesper Ferkinghoff-Bowt Batched energy-entropy acquisition for Bayesian optimization. In 38th Conference on Neural Information Processing Systems (NeurIPS 2024), Vancouver, Canada, 2024.
Louis C. Tiao, Aaron Klein, Matthias Seeger, Edwin V. Bonilla, Cedric Archambeau, and Fabio Ramos. BORE: Bayesian optimization by density-ratio estimation. In Proceedings of the 38th International Conference on Machine Learning (ICML). PMLR, 2021.
Austin Tripp, Erik Daxberger, and José Miguel Hernández-Looatt, Sample-efficient optimization in the latent space of deep generative models via weighted retraining. Advances in Neural Infor- mation Processing Systems, 33: 1259-11272. 2020.
James T. Wilson, Frank Hutter, and Mare Peter Deisenroth. Maximizing acquisition functions for Bayesian optimization. In 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada, 2018.
Taeyoung Yun, Kiyoung Om, Jaewoo Lee, Sujin Yun, and Jinkyoo Park. Posterior Inference with Diffusson Models for High- figh-dimensional Black-box Optimization. In Forty-second International Conference on Machine Learning (ICML), Vancouver, Canada, 2025.
14

---

A GAUSSIAN PROCESSES FOR BO
Assume Gaussian process prior over f. c.g., f (P where k: X x X + IR is positive-
definite kernel (Rasmussen & Williams, 2006). Then, given a set of observations D,, := {x,, y/s / corrupted by Gaussian noise ~ (N d²). ) the posterior f\Dn, ~ gP(fns kn) is available in closed form with mean and covariance function given by:
fx(x) = -x(x)' (K, + -,.))
(24)
kn kn x') = x(x,x) - (x(x)
(K, + (x)))KKnn
(k,))
(25)
(x) = (x,,,x),
(26)
where (x) := [k(x, x;)]=_ IR." K,, == [k(,,, X3) )1ag=1 E TR yn == [we]__ R", for
x. E X. With these closed-form expressions, GP models allow BO algorithms to quantify uncer- tainty and assess expected utilities of their decisions. However, note that, due to matrix inversions, exact GP inference incurs a computational cost O(n"). Hence, one often has to resort to low-rank approximations to make GP predictions tractable in cases involving large amounts of data, such as batch evaluations with large batch size. Alternatively, one may completely discard the GP models and use other surrogates, such neural networks, and there has been increasing literature how to reliably quantify uncertainty for BO when using these models (Li et al., 2024).
B LEARNING PARAMETRIC MODELS WITH RKHS CONVEX LOSSES
In this section, we consider the general problem of learning function g. with parametric model g : x + IR, where the parameter space is an arbitrary finiteddimensional vector space. Most existing results in the Bayesian optimization and bandits literature for learning these models from inherently dependent data are only valid for linear models or kernel machines. As we will consider arbitrary generative models, need to derive convergence results applicable to wider class of models, accommodating popular modern frameworks. To do so, we will assume that there exists reproducing kernel Hilbert space (RKHS) containing the models and the target log-density 9. = log P. for a given (fixed) utility function u.
B. MAIN DEFINITIONS
We will need the following definitions to state our main assumptions and results.
Definition (RKHS). Let X be a non-empty set. A Hilbert space Ht of real-valued functions over X is called reproducing kernel Hilbert space if function evaluations bounded, that is:
E
3cx <0: vh H. h(x) S Cx Gx|h
Any Hilbert space H satisfying the condition above is associated with unique positive-semidefiiite kernel l : X' X + R with the reproducing property (Schölkopf & Smola, 2002):
Vh H ((x) = (h, kr,, x))
Vx E X,
(27)
where (4 denotes the inner product in H. which also defines the norm ||ll| V (h, h) for h E H. In fact, it can be shown that H is spanned by {k(, x) xxxx (Steinwart & CChrstmman, 2008, Thm. 4.21). Hence, given a positive-semidefin to denote the inner product and the norm, respectively, in the corresponding RKHS Hy.
kernel k, we will use the notation (-, and ||-I|x
Definition 2 (Strong convexity). A differentiable function f H + R on Hilbert space H. is a-sirongly convex over S C H. for a given > 0, if:
Yh, ES S(h) 2 S(N) + (VS(N), R1) a -1 h h 111
Definition 3 (Smoothness). A function f : H + y between Hilbert spaces H and y is rj-smothh over S C H. for a given 11 if:
Vh, h E S,
[[(()) f(1')ll> 1//h 11
(28)
15

---

Definition 4 Suub--aussianity). A random variable & taking values in a Hilbert space H is said to be E-sub-Gaussian given a positive-definite trace-class operator Σ - H. if.
vh E M4 Ejexp(h, E> I exp (1
(29)
Likewise, a H-vaalued stochastic process (5m) adapted to the filtration {8n la is conditionally if the following almost surely holds:
4h H, Elexp(h, En) 8n-1) S exp (2 Eii). Vn enNN.
(30)
B.2 AUXILIARY RESULTS
Lemma 2. Let g: X + R. represent a class of models parameterized by 0 E O. Assume that
g(x:-) E He, for all E X, where He is reproducing kernel Hilbert space associated with positive-defmite kernel lig
then follows that:
Hg = [h: : X
Bw He h(x) (w.g(x, Mouo Vx E 2
(31)
equipped with the norm:
||ll,, =- infillellee w E He, h(xx (w, g(x, MHo Vx E X constitutes the unique RKHS where ky (x,x') + (g(x,) )) g(x'x mm is the reproducing kernel.
(32)
Proof. This is a direct application of classic RKHS results (e.g., Steinwart & Christmann, 2008, Thm. 4.21) where we are treating Φ x + g(x, :) a feature map mapping into an existing Hilbert space He and taking advantage its structure to define a new one.
The RKHS Hg described above has the special property that, for any 0 E θ, the RKHS norm of the model is given by:
Remark 2.
(9 0),k,, (0,
(33)
since (< (+,)=,),, 9(, H) = g(x, 0) for all x E X, and ke(-, 0) is the unique representation of the evaluation functional at 0 in the RKHS He. The rest follows from the definition in Equation 32. Hence, each choice of ke gives us a potential RKHS norm regularizer.
Lemma 3 (Abbasi-Yadkori, Yadkori, 2012, Cor. 3.6). Let (8t be an increasing filtration, { be real-valued stochastic process, and 100 be a stochastic process taking values in a separable real Hilbert space H. with both processes adapted to the filtration. Assume that { is also predictable, i.e., de is B:-1-measurable and that e is conditionally -ub- Gaussian, for all N. Then, given any E (0, 1), with probability at least 1 - 6,
dett(I + t. V $
E
Σ E
S 20 log
8
(V+0,00))-
for any gositive-definiz operator V > 0 on H. and where we set Th = (ah...., mr).
Lemma 4 (GP variance upper bound (Steinberg et al., 2025, Lem. E.5)). Let Inni 1 be a sequence of X-valued random variables adapted the filtration (3n)a21 For a given X, assume that the following holds:
.E. ::: VT T., - 8n-1] by 0,
(34)
for a some sequence of lower bounds (b, bnnN. Then, for a bounded kernel ie : X X + R given observations at {x1 the following holds with probability 1:
xx) E (b
(35)
In addition, bn 20, then lim, xx) o2
16

---

B.3 MAIN ASSUMPTIONS
Loss. Given data D, =- (x for the analysis, we consider loss functionals defined over an RKHS Hz with the form:
n
Lg(g) (g) Σ (el(m)()) g E Hk.
(36)
where : 1 - R is fixed deterministic function, > 0 represents importance sampling weights (or Wi x when importance weights are not used), m, : H2 + IR represents a bounded linear observation functional (c.g., (g) g(x)), mz(g) ( ((x,2)), is given by utility evaluations (c.g., 24 = u(ye), or 2n.i u(y;,)) w(y.,)). for E {I....,n) n), and R,, HA + R is regularization functional.
Considering the loss structure above, the target loss functional is given by:
I., (.( B(((m(g), =))n(( dm).
(37)
For the forward KL loss, for example, the target loss is simply - [x a(x) log g(x) du(xx), where suitable base measure over X' with respect which the probability densities defined (e.g..
the counting measure if X' is discrete). We then define constrained and unconstrained targets as:
9+ argmin Lg(g)
(38)
oEHt or djm1
g- E argmin Lz) gEHr
(39)
Assumption A1 (RKHS), There exists reproducing kernel Hilbert space Hi, associated with positive-definte kernel k X' X + which bounded, supxex k(x, x) S bfj for given by > 0, such that 9x, constants, and the models can be found as elements of Ha. ie., {9(::) A Q} C Hr.
The assumption above allows us to consider functions g- that cannot be perfectly approximated by the model, though which still lie in the same underlying Hilbert space Hy. The reproducing kernel assumption is also mild, as it simply means that function evaluations are continuous (i.e., well behaved), which cannot usually be guaranteed in other types of Hilbert spaces, such as, c.g., L2- spaces. In fact, we can always find a RKHS that contains the set of models under mild assumptions, such as the minimal construction in Lemma 2. The inclusion of constant functions allows for the unnormalized target g. to also lie in Hs.
Remark 3. If an RKHS containing the model class (c.g., Hg in Lemma 2) is too small to contain 9=, we can always combine two RKHS to produce a third one containing all the elements of the two. For instance, if 9. E H. * Hy with kernel k. : X' x X IR, we can define k == k. + ky. that Hs := H. + Hl, is also RKHS (Steinwart & Christmann, 2008; Saitoh & Sawano, 2016). Such H. can be minimal, as any function g- defines a kernel k. 6x( g. (x)g.xx) (x') for the RKHS formed by the function's ' linear span H. == {ag= a E IR..
Assumption A2 (Regularization). The regularizer Rn is predictable, An-strongly corvex, twice differentiable, and O(x,)-smooth for all n 2 1, where { is non-decreasing sequence of positive real numbers, such that Xn is at most poly-logarithmic with n.
Regularization. Common choices of regularization scheme, such as the squared norm, suffice to satisfy Assumption A2. Strong convexity does not require a function to be twice differentiable, but such assumption greatly simplifies our analysis and is common in modern deep learning frameworks. Sublinearity of A,, allows for the effect of regularization to disappear as the dataset grows, so that the empirical L,, can converge to the target L, as n + co. Despite the definition of a regularization functional over the whole of He. following Remark 2, we can use any positive-definite kernel ke x θ + compatible with Lemma set Rt such that, over the model space:
An
Am ke (0.
(g) ) = | 11,,
(00)
2
2
which allows for differentiation with respect to 0, and whose Hessian (.)n()) X shows that R2 is strongly convex in H4 for X, 0. In this case, quadratic regularization penalty |00|]
17

---

corresponds to the assumption of a linear kernel, ie., kg 8¹) -0-0% restrictive, as it assumes that our models are linear functions of the parameters. However, note that, for overparameterizzd neural networks, at the infinite-width limit the model tends to show linearity in the parameters (Jacot et al., 2018). If we want to be more parsimonious, alternatively, we can choose ke as universal kernel, such as the squared exponential, yet preferably not translation invariant, so that ke(0, 0) is not simply a constant. One kernel satisfying such assumption would be the exponential dot-product kernel (0,0,, 0') == exp(0 0*), which universal for continuous functions over compact subsets of O. Nevertheless, we do not impose restrictions on the form of the regularization term R, other than Assumption A2. Lastly, Rn can also be stochastic as:
0', which might appear quite
R (g) = A - -
An
(41)
where ||-I|x denotes the norm in a general H4 satisfying Assumption A1, and In,0 may correspond to random initialization or a data-dependent term (e.g., the previous set of optimal parameters).
Assumption A3 (Loss). For any y E R. the point loss E2 := 2(: R + R is au-strongly convex, twice differentiable, and has me-smooth first-order derivatives. In addition, given any m Hk, we assume the first-order derivative & (m((])) is conditionally (it-su--Gasse when y ~ p(y|m).
Regarding sub -abb-aussianity, the pointwise derivative of the balanced forward KL loss is given by (m(()) = -u + co(x), Therefore, we have that:
(((()) x] xx]= - a(x),
(42)
which yields (x) = log axx) at the unconstrained minimizer g- under mild assumptions on the kernel and the acquisition function. Therefore, if observation noise is light tailed or the utilities are bounded, the pointwise derivatives ea(x), which is strictly positive, yielding an approximate strong convexity (when restricted to any bounded subset of the realizations of g(x)). The original Bradley-Terry model the DPO paper (Rafailov et al., 2023) is not strongly convex, but its robust version (Chowdhury et al., 2024), which accounts for preference noise, can be shown to satisfy strong convexity and smoothness.
shown to be sub-Gaussian The second derivative is simply
Assumption A4 (Weights). We have that inf nen Wn 2 b and supngN Wn S b,. almost surely, for constants
b by > 0.
Assumption AS (Density). The model class is dense in the normalized subset of Hy:
C== c=={ ge E Hu
f [dddu=)) dy=1}
These last two assumptions allow us to bound the randomness of the loss gradients and ensure that the optimal models converge to the target distribution. Density here simply means that, for any g E C, we can find a sequence {0,] such that go, - g as i + xc. Alternatively, the latter can be expressed as, for any g E C and any given A >> 0, there exists 0x E O such that 11 g0 llk S A. Hence, density is a universal approximation condition within the normalized subset of Hg. As Ha is just a theoretical construct for us, and not fixed by the algorithm, so that we can even span Ha from the model class (see Lemma 2), Assumption A5 is relatively mild.
We can now analyze the approximation error with respect to g. for the following estimators:'
O, E argmin L,, (go)
(43)
dee
In E argmin L,, (g).
(44)
gEHr,
The first one gives us the best parametric approximation go., based on the data and is what our algo- rithm will use. The second estimator corresponds to the non-parametric approximation, which we will use as a tool for our analysis, and not assume as a component of the algorithm. The assumptions above allow us to bound distances between these estimators and the true g, as a function of the loss and gradient values. We also consider the normalized version of 9n as:
gn (x)
9m. (x)
log
e du, XE X,
(45)
which we use as a reference point to bound distances to the parametric models and to the target g..
We are implicitly assuming that such glohal optima exist. This is true for the optimization in Ha. as we will show, but not always guaranteed for the optimization over θ which will depend on the parameterization.
18

---

B.4 MAIN RESULTS
Lemma 1. Let assumptions A2 to A4 be satisfied. Then,
- - 9n 122 S Ln(g) - Ln(gn) S 1
where Hn Hs + Hy is an operator-valued lower bound on the Hessian of the loss Ln:
vg E Hks
vLLn(g) Hn := An + abb Σ m, m,
Proof of Lemma 1. The Hessian of the individual losses can be lower bounded by:
Vg E He.
(el((gg), 2) = (m( & V gg)()) + V((())
(m ) & m t açm @ m,
(46)
E Vm E M,
given that V yg) = V Valg, m)n Combining this result with Assumption A2 and A4, we get:
Vg Hk.
(n) 2 A, arb Σ my m, Hn-
(47)
Now applying first order Taylor expansion Ln at any g H4. the error term is controlled by the Hessian L (gn) at an intermediate point 9n 80n (1 8)g, for some E [0, 1]. Hence, expanding Ln around gns we have that:
Vg E Ha. Ln(g) Ln(gn) LVnnggn)g gn) + 2 3n
(48)
1
1g
In l1
2
where we applied the Hessian inequality (47) and the fact that VL, (g.) 0, as gn. a minimizer. Thus, the lower bound in Lemma follows. Conversely, expanding Ln around any y and evaluating at Ins we have:
1 2119, -11
Vg His (n(gn) (g) (V g)n
(49)
Rearranging terms yields:
Vg E Has Ln(g) Ln(gn) (VLn(g) gn)n
2 l1g gnl 1 (..
< sup
(VL.(o). 9)4
|1 2 L.,. (5%))
(50)
gEHi
sup (V.. gEHx
whose right-hand side is strongly concave and has therefore a unique maximizer at:
g = H
LL,
(51)
Replacing this result into the previous equation finally leads us to the upper bound Lemma Lemma S. Consider the setting in Lemma Then,
Vn E N,
((m, G.)b - (m, gn)al [m|] Hr 18((), Vm Hi.
where B,,(5) = VR (ū-)lla + or loggdett b a M,, 1/2 and M,, "Mn,....,mn]!
19

---

Proof. By Lemma for any 9 E Hg, we have that:
(m, In)x (m, g)x]))>
(m, In g)kl
(11,212210 11, [gn g))al
S H 1/2 ||| H (gn g)ll
(52)
|mll 11 Hn
- || |V (g) n, Vm E Ha.
where the first inequality follows by Cauchy-Schward and the last is due to Lemma 1. Expanding the gradient term, we have:
V Ln (n)
(a)(o) +
Σ
wil (1mi, g)m)m.)))mm
2=1
Σ wiln, (mris g)
(53)
H
(
Σ
wilys ((mi, ()kk,)))mmm
H.
where we applied the triangle incquality to obtain the second line and the fact that Hn > And implies H < T led the last line. For g := g., sum above by setting Be as the r-algerra generated by the random variables (m; y/ M= and m++1- Then E = Vbu.a wel.. ((ğ-, m²)k) is conditionally o.-sub-Gaussin by Assumption A3 with of == R.c bung since we & bw by Assumption A4, and Φ: = Vby predictable, for all E N. An application of Lemma finally leads us to:
then apply Lemma 3 to the noisy
oz log a
det([I + " Mn)
(m)
(54)
8
which holds uniformly over all n E N with probability at least 1 - 6. Hence, it follows that:
[VL.(O)) S 1 [V.R( + Σ Ewwly (4m; 9-)x)
Vn E,
$ Bn,(d,, (55)
V
with probability at least 5. Replacing this result into Equation 52 yields the main result.
Theorem 1. Let assumptions A1 to A5 hold. Then, given any å (0,
Vn N, [mm (m, 90.. >a (( H., + 2(m, 1) (qn)), Vm Hk: which holds with probability at least 1 - 8, where (r) is given by Lemma 5, where ((x) denotes the unit constant function, vn(g) Ex [|x(x))] and qn Pit
Proof of Theorem 1. Fix any m E H4 and N, the approximation enror can then be decomposed as:
(m,..)) k (m, go. >al I (m, g. gn)xa (m,, (n.)) xl
(56)
Let L. E Hd denote the unit constant function, i.e., <(x) = 1, for all x E X. The difference hetween normalized functions is then such that:
( (m. -- - In + c log / can du - log f f du
(m, 9- -nnkkl
(57)
$ (m, Z. - -nn)al + 1m, dyal log f fean dp log fce- (e-dpl dpl
20

---

We apply Jensen's 's inequality to show that the second term on the right- hand side of the inequality bounded by the expected pointwise error between In and G.. To reduce notation clutter, let qa = Pi expg. 9. and Eg [h] == Ex~g [h(x)]. Indeed, therefore, we have that:
c
E. 15- gn Eg.
| log
ed-
c.hi
log Bg.
E9
(58)
1
/
cδ. 6
log Jed. du
du
eg.
log e du - -log
e
du
by Jensen's inequality on the convex log(-). Similarly, repeating the same steps for E.g. [gn G.).
where 9n (x) e0,(x)
con(xx
Je Jonn du get:
Egg [gn - Ø.] 2 log / e9n dp log / cB du.
(59)
Combining the two inequalities yields:
log / c0n dp - log f c dp s max (F.g [7. gnl BBg [on G-1}
(60)
Reg (0+ gnl + [Eg. I9 - Fz]
We can now bound the expectations as:
Vg P(X),
|Eg 1āx - 9n]
|xxyg [āx (x)(x - on (x)]
|x (gg -n. ((xx)()))))))))))) [x)
1/ Ins Exng [o(x)])
(61)
(o) [0(x)]lHH))
S (6) [16(())l
where the first inequality follows by Lemma 5 and the second is due to Jensen's. Therefore,
log f c t du
log
c² dj S (6)
Eg.. [1x(x) + E.g.. [10(x))1===]
(62)
Finally, applying the bound above Equation 57 followed by another application of Lemma obtain:
(m, g. - 9n)xn < (6)
(|mm|| + Km,, c)ul (Ea. [11(())]lxx. ] Eg. [110x))lxx=)))
(63)
For the remaining term in Equation 56, we have that:
(m,, go.)). (m, )n.)al Al S [|m]] 1190, - In H.
(64)
$ ||m|l 222L,, (go..) I,, (0m))
which follows from Lemma From Assumption A5, we have that:
YA O, 30 ④ 1190. 9n ll+ S A.
(65)
At the optimum, we know that Ln (go_.) S Ln(go), for all E Therefore, as + 0, by continuity, we have that:
I, (go..) T. (0n) S (90())))) - I.n (gr) - 0,
(66)
which implies (mm, )0.)) (m, gn)) - in Equation 64. Consequently, the density of the model
class allows us to replace pointwise evaluations 9. (x) = (g... Φ(x))a by go, (x), for x X, so that can swap dr in Equation 63 for q,, (x) exp go., (x). The main result then follows.
21

---

Despite the model being potentially non-linear and the loss not being required to be least-squares, Theorem shows that we recover the same kind of RKHS- based eπror bound found in the kernel- ized bandits literature (Chowdhury & Gopalan, 2017; Durand et al., 2018; Oliveira et al., 2021). Regarding the asymptotic rates, we make the following observations.
Remark For finite domain, | < oc, the RKHS becomes finite dimensional with dim( = | resembling a linear model. In this case, Srinivas et al. (2010) provides bounds for the Gram matrix log-determinant in Ba, (6) as O[Ix log n), yielding:
(6) E O(v X l)
Predictive variance. We can show connection between ||m=]] Hー and GP predictive variance. By an application Woodbury's identity, letting Oi == arbrr. we have that:
|m] H = m (And + )
=m =m (at ( I+ X- Ma)
(67)
'm (I- M, II MM Mn) M,] )m
I+
= =
m Mn (f
I -I
M,, (M,)-'Ma M,,,,,
If observations correspond to pointwise evaluations == k(,x), = xx) and m. == k(,,x)) =
((x,), for x E X' and {x;]=== C C, we end up with:
||mll?
110(x)]
= *, (k(x,)) kn (x) (n I K.,) - (x))
(68)
=1,' ,
which corresponds a scaled version of the posterior predictive variance of (x) a GP (25). More- over, the (m, c)x term in Theorem 1 is simply (o(x), c)x = <(x) = 1, for x E X. and the remaining terms depend on expectations of | which also a function the predictive variance of (x).
We can then apply the auxiliary result from VSD (Lemma 4) to show that o2, (x) is O(n asymp- totically, whenever qo, (x) has positive lower bound, allowing for asymptotic convergence of the proposal distributions to the target p., cxp g.. As result, considering Remark 4, we get:
1g. (x) 9n (x) E O (V X |
log
(69)
which vanishes + Therefore, the model converges to the target distribution P. under these assumptions. Nevertheless, the algorithm's regret still depends on how fast the time-dependent acquisition function a, concentrates around the optimum x' and how much probability mass qe places away from x" per BO round f, analyses which we leave as subject future work.
C ADDITIONAL EXPERIMENTAL DETAIL
This section presents algorithmic settings and implementation details for our experiments. We also present summary of our experimental results in Section C.4 and ablation studies in Section C.5. In addition to the descriptions in this section, code for our experiments will be released online at htt.hb..ooooeeeee cotom/csiroo co funmllummlmmmmm funml/generativvebn
C.1 TEXT OPTIMIZATION
We use the same annealing threshold scheme for setting 9: as Steinberg et al. (2025, Eqn. 20), where set 1) such that we begin Pu 0.5 end pr 0.99. For the proposal distribution, found these short sequences best generated by the simple mean-field categorical model.
M
(x
II
Categ(Em (softmax((m.))
(70)
m=l
where I' E V and ém E Rl and we directly optimize d. VSD and CbAS use the simple MLP classifier guide in Figure 4.
22

---

Sequential
Sequential(
Embedding(
Embedding (
num jon_embeddinggee mnjeddinn__dimill
num_ _un_enbedding--,,
embedding_ _im=E
Drepout (p=C.1),
Bropout (p-0.2), Convld
Flatten (), LeakyReIU(),
in_channels=B, out_channels=C, kernel sizeel_sizeKe,
Linear
in_features-16
out_features=64
) LeakyReLU MaxPoollde
LeakyReLU(),
Linear
kernel_size-Kx, stride=Sx,
fe_ fut_features=1
Convid
in_channels=C,
out_channels=C, kernel size-Ke,
LeakyReLU MaxPoolld
sizeek_, stride=Sx,
Flatten(), LazyLinear
out_features-i
LeakyRelu li hinear in settuusr,,
out fet_rratu
(a) MLP architecture
(b) CNN architecture
Figure 4: Classifier architectures used for VSD and CbAS in the experiments using Py Torch syntax.
[V] M M, and we give all other parameters in Table 2 if not directly indicated.
C.2 PROTEIN DESIGN
We use the same threshold function and setting for all of the protein design experiments as in Sec- tion C.I. However, these tasks require more sophisticated generative model that can capture local and global relationships that relate to protein's 3D structure. For this we use the auto-regressive (causal) transformer architecture also used in Steinberg al. (2025),
(xx()) ()) = Categ(z) (softmmx
g /1 1..mm1101.m)) where m=2
II
(10m//11m-11- @1.)) = Categ(zm. (attee/eeDTTaaanffooooooaa DTTransfrrner(eemm-- @1m
(71)
For the latter, see Algorithm 10 and 14 in Phuong & Hutter (2022) for maximum likelihood training and sampling implementation details, respectively. We give the architectural configuration for the transformers in each task in Table 1, and the classifier CNN used by VSD and CbAS is in Figure 4.
We use the following Ehrlich function configurations:
M = 15: motif length = 4, no. motifs = 2, quantization =
M 32: motif length 4, no. motifs 2, quantization =
M 64: motif length 4, motifs 8, quantization 4
23

---

Configuration
Stability
SASA
Ehrlich 15
Ehrlich 32
Ehrlich 64
Layers
2
2
2
Feedforward network
256
256
32
128
Attention heads
4
1
3
Embedding size
64
64
20
30
Table 1: Transformer backbone configuration.
Configuration
Stability
SASA
Ehrlich 15
Ehrlich 32
Ehrlich 64
16
16
I0
I0
10
C
96
96
16
16
16
Kc
7
7
Kx
5
5
2
2
2
Sx
4
2
H
192
192
128
128
128
Table 2: CNN guide configuration for VSD and CbAS
C.3 GENBO SETTINGS
Acronym
Meaning
EI
Expected Improvement
PI
Probability of Improvement
sEI
Soft Expected Improvement, i.e., softplus r)
SR
Simple Regret (utility function)
IKL
Forward KL loss
bfKL
Balanced forward KL loss
rPL
Robust preference loss
MF
categorical proposal model
Tim
Transformer proposal model
fr
More frequent regularization (change in A, schedule rate)
r0p10
Base regularization factor set to Ag = 0.1 Exponential regularizer, i.e., (0)
exp
X, expl| 0ul12
mp
No (informative) prior, i.e., (x) x
p
Pre-trained prior, learned from initial (randomly initialized) data Dn Importance weights
gg
1r0p10
Learning rate setting for training the generative model (e.g.. 0.1 in this case)
pemin0p50
Minimum percentile for threshold т, annealing schedule (e.g., 50% in this case)
pemax0p90
Maximum percentile for threshold 7y annealing schedule (e.g., 90% in this case)
Table 3: GenBO experiment settings acronyms
Table presents our settings for the different GenBO variants across experiments. The settings for proposal models followed VSD's configurations. Our regularization scheme penalized the Euclidean distance between the model's parameters and their random initialization (He et al., 2015) with R, (0) == Ao.100 O ll.. using an annealed regularization factor An := do log n. similar to Dai et al. (2022), which ensures enough exploration, while still A. + n - co, allowing for convergence the optimal 0,, For threshold based utilities, mainly set the quantile threshold т to follow an annealing schedule ranging from the 50th (i.c., the median) to the 99th percentile of the observations marginal distribution for both GenBO and VSD, where the percentile "t corresponding
the quantile is updated Y := X"="" where 7)
log YZ log
E (0, 1).
C.4 RESULTS SUMMARY
Besides the plots in section 6, summarize the final results in Table and 5.
24

---

ALOHA
Ehrlich-15
Ehrlich-32
Ehrlich-64
Random mut.
3.80 = 0.40
LaMBO-2
0.19 0.17
0.36 + 0.15
0.95 + 0.02
CbAS
2.20 0.40
0.57 0.12
0.61 + 0.10
0.98 + 0.01
GA
0.45 0.12
0.11
0.98 = 0.01
VSD
0.00 + 0.00
0.19 + 0.17
0.97 = 0.00
GenBO 0.20 0.40
0.00 0.00
0.28 0.16
0.94 = 0.02
Table 4: Final average regret (lower is better) for the best-performing variant of each method across the ALOHA (text optimization) and Ehrlich benchmarks
FoldX (Stability)
FoldX (SASA)
Random mut.
2.79 + 0.22
12550.26 + 56.34
LaMBO-2
3.19 + 0.58
12456.10 + 126.64
CbAS
3.65 + 0.23
12376.65 + 298.30
VSD
4.20 + 0.42
12537.97 + 186.35
GenBO
3.28 0.35
13285.42 221.60
Table 5: FoldX average maximum outcome for the best-performing variant of each method
C.5 ABLATIONS
We performed ablation studies the annealing scheme and the evaluations batch size. We vary the minimum and maximum percentile for the threshold annealing settings of both GenBO (with PI util- ity) and VSD on the text optimization problem in Figure Plots reveal a preference towards more exploitative behavior for this simpler optimization problem. In Figure 6, we vary the evaluations batch size B for GenBO on the Ehrlich benchmark problem of sequence length 32. expected, larger evaluation batches lead to lower regret, though with higher variability across the candidates.
25

---

Method
Average Runtime
CbAS
53.38 2.05
VSD
42.89 + 2.56
GenBO
14.88 0.26
Table 6: Average times with standard deviations for the ALOHA text optimization problem.
Fhal averege regret
Anol
20
1.0
08
8.90 prаxx
6d3
635 931
ods
67
305
8:00
29
8:90
6.6
050
16
(a) GenBO
(b) VSD
Figure 5: Final average simple regret (the lower the better) for GenBO and VSD as a function of the minimum and maximum percentile in the annealing schedule for the text optimization problem.
0.
0.3
Simole
0.2
18
84
128
256
Eatch site
Figure 6: Batch evaluation size B ablation on Ehrlich benchmark of length 32. The plot presents the final average simple regret for each B setting, with error bars corresponding to #1 standard deviation, All variants were run for the same number of BO rounds as the original experiment.
26


Reviews:
[{"review_id": "izAHz0gQUj", "reviewer": "Reviewer_Jdpb", "summary": "This paper introduces Generative Bayesian Optimization (GenBO), a framework where a generative model is used directly as the acquisition function for Bayesian Optimization (BO).\nTraditional generative BO methods often use a two-stage process: first, training a surrogate model (like a Gaussian Process), and second, training a generative model based on the surrogate's predictions. This can lead to compounding errors.\nGenBO, inspired by Direct Preference Optimization (DPO), bypasses the intermediate surrogate model. It trains the generative model directly on simple, observed utility values (such as Expected Improvement or Probability of Improvement). The resulting model's probability density becomes proportional to the expected utility.\nTherefore, sampling from this trained generative model is equivalent to sampling candidates from the acquisition function, simplifying the pipeline, reducing approximation errors, and enabling scalability to large batches and complex, high-dimensional spaces like protein design.", "questions": "See the weakness section above.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The author provided the theroetical analysis of their proposed method.", "weaknesses": "**Illegible Figures**\n\nThe figures are poorly rendered and extremely difficult to interpret. The hues selected for the proposed method and the various baselines are too similar, making the different plots (e.g., performance curves) indistinguishable. Compounding this issue, the legend is rendered at a font size that is far too small to be legible. As a result, it is impossible for the reviewer to accurately interpret the experimental results or validate the paper's claims.\n\n**Insufficient Experimentation and Analysis**\n\nThe empirical validation is severely lacking. The paper presents minimal experiments with what appears to be a complete absence of analysis. The claims are not adequately supported by empirical evidence.\n\n**Trivial Methodology and Limited Contribution**\n\nThe core idea seems to be training an LLM on (data, utility function value) pairs via DPO to generate better data points. This appears to be a trivial application of DPO to the Bayesian Optimization (BO) setting. If one were tasked to \"use DPO for BO,\" this is the most straightforward idea one would likely conceive. The conceptual novelty and overall contribution seem too limited.\n\n**Clarity on Training Data**\n\nThe amount of data used for training the LLM is not specified. BO inherently assumes an expensive-to-evaluate setting, which typically yields sparse data. It is highly questionable whether this data regime is sufficient to fine-tune an LLM effectively.\n\n**Baseline selection**\n\nThe field of generative model-based BO is well-established, often referred to as LBO (Latent Space Bayesian Optimization)[1~7]. The paper fails to position itself relative to this body of work. What are the specific advantages of this DPO-based approach compared to existing LBO methods? The lack of comparison makes it impossible to assess the method's practical or theoretical benefits.\n\n**Reference**\n\n[1] Tripp, Austin, Erik Daxberger, and José Miguel Hernández-Lobato. \"Sample-efficient optimization in the latent space of deep generative models via weighted retraining.\" Advances in Neural Information Processing Systems 33 (2020): 11259-11272.\n\n[2] Maus, Natalie, et al. \"Local latent space bayesian optimization over structured inputs.\" Advances in neural information processing systems 35 (2022): 34505-34518.\n\n[3] Lee, Seunghun, et al. \"Advancing bayesian optimization via learning correlated latent space.\" Advances in Neural Information Processing Systems 36 (2023): 48906-48917.\n\n[4] Chu, Jaewon, et al. \"Inversion-based latent bayesian optimization.\" Advances in Neural Information Processing Systems 37 (2024): 68258-68286.\n\n[5] Moss, Henry B., Sebastian W. Ober, and Tom Diethe. \"Return of the latent space COWBOYS: Re-thinking the use of VAEs for Bayesian optimisation of structured spaces.\" arXiv preprint arXiv:2507.03910 (2025).\n\n[6] Grosnit, Antoine, et al. \"High-dimensional Bayesian optimisation with variational autoencoders and deep metric learning.\" arXiv preprint arXiv:2106.03609 (2021).\n\n[7] Lee, Seunghun, et al. \"Latent bayesian optimization via autoregressive normalizing flows.\" arXiv preprint arXiv:2504.14889 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n6SrDkmPoW", "reviewer": "Reviewer_REEK", "summary": "This paper presents the direct use of generative models as samplers for design selection in batched Bayesian optimization. Unlike prior two-stage approaches that first fit a surrogate or regressor and then train a generator, the authors propose a single-stage method that learns a generative model whose density is proportional to the expected utility. Two approaches are introduced. The first is preference-based learning, analogous to Direct Preference Optimization (DPO), using a contrastive objective built from pairwise utilities, with a robust variant that accounts for noisy labels and sign flips. The second is a KL-divergence route that trains the generator to match the utility-weighted target distribution via a balanced forward KL, implemented with importance-weighted sampling to yield an unbiased objective, along with an alternative KL derived from Bregman divergences to avoid allocating high probability mass to low-utility regions. The paper also provides theoretical analysis showing convergence to the utility-based target distribution and concentration of the generative BO proposals at the true optimum.", "questions": "1. I am willing to raise my score if the authors can address my main concerns about the empirical results, specifically, by demonstrating stronger robustness and clearer superiority over baselines where possible. \n\n2. While the framework targets utility-based acquisitions, can you comment on extensions to other acquisition families such as Thompson Sampling and GP-UCB? Perhaps such extensions are not straightforward or not possible? An explanation would be great.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The core idea of a single-model training approach for the generative sampler is well motivated and clearly presented.\n\n2. The method attempts to generalize across expected-utility acquisition function variants.\n\n3. Scalability is linked to batched BO, making the approach practically relevant if successful.\n\n4. The paper provides theoretical analyses that support the claims, with careful attention to both convergence and optimality.\n\n5. The empirical evaluation compares the proposed methods against relevant baselines.", "weaknesses": "1. The empirical analysis is well motivated but does not make a fully convincing case for the proposed algorithm.\n\n2. Related to the above, the results do not show consistent superiority over all baselines on the more complex problems. While the authors do note cases of underperformance with provided justifications, it would help to comment on how the approach could be improved to close those gaps.\n\n3. The plots are quite difficult to read.\n\n3. Minor: while the paper claims generalization to utility-based acquisitions, it does not clearly discuss extensions to other acquisition families such as GP-UCB or Thompson Sampling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "97UByEuPbe", "reviewer": "Reviewer_szDh", "summary": "This paper proposes a framework called generative Bayesian optimization (BO), which transforms generative models into candidate samplers for batch BO. By leveraging ideas from direct preference optimization (DPO) in RLHF, the approach trains generative models directly using noisy utility values derived from observations, making the proposal distribution's density proportional to the acquisition function. This eliminates the need for intermediate surrogate models, reducing computational cost and approximation errors compared to prior methods like VSD or BORE. The framework supports two training strategies: a DPO-like loss for preference-based feedback and divergence minimization with utility-weighted samples for general rewards. It is particularly suited for large-batch scaling, high-dimensional spaces, non-continuous designs, and combinatorial problems. Theoretically, the authors show that the sequence of generative models asymptotically concentrates at the global optima under certain conditions. Empirical evaluations on challenging high-dimensional optimization tasks demonstrate improved performance for large batches.", "questions": "Please see the Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The key innovation is treating generative models as direct acquisition functions via DPO-inspired training, bypassing the two-stage process (surrogate then generative) in existing methods. This simplifies the pipeline, makes it more general to utility types, and enables efficient sampling for large batches in complex spaces. The extension to batch BO addresses a practical need in parallel evaluations, like simulations or experimental design.\n2. The paper provides convergence analysis, showing asymptotic concentration of proposal distributions around optima. Assumptions seem reasonable (e.g., bounded domain, sub-Gaussian noise, smooth objectives), building on standard BO theory (e.g., Bull, 2011 for EI; Srinivas et al., 2010 for regret). \n3. Scalable to high dimensions and combinatorial domains, where traditional GP-based BO struggles. Compatibility with pre-trained generative models allows fine-tuning for optimization, with low sampling cost. The framework generalizes beyond preferences to arbitrary utilities, broadening applicability to areas like hyperparameter tuning, materials, and robotics.\n4. Equations are well-explained, and the motivation from real-world batch scenarios is compelling.", "weaknesses": "1. While claiming reduced cost by avoiding surrogates, training generative models (especially large ones like LLMs) on utilities could still be expensive in practice, particularly for online BO with frequent retraining. No discussion of training overhead vs. baselines.\n2. Focuses on batch BO, but less emphasis on sequential cases or comparison to non-generative batch method. Robustness to noisy utilities or misspecified priors isn't addressed. For combinatorial spaces, discretization or embedding strategies aren't detailed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JO1lixrb1g", "reviewer": "Reviewer_Vkoy", "summary": "This paper tackles the challenge of how to approach Batch based Bayesian optimisation. The challenge in Batch BO is how to query a set of evaluation locations as we want them to be diverse. The traditional approach turns this into a sequential decision problem. The idea in this paper is to learn a separate model that parametrises the support of the acquisition function such that this can be used to sample candidate locations. The paper presents several different approaches on how such a model can be defined both using the acquisition function perspective and one of preference losses.", "questions": "Could you expand on what the specific model for q is and detail what p_0 is in the experimental settings that requires this?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper is generally well written and honest in what it proposes not overstating what is proposed, thank you for this. The work builds significantly on the work proposed in [1] but proposes new and novel mechanisms to train the model and the use-case here is directly BO which was only suggested in the Appendix of [1]. The preference based training is to my knowledge novel in this modelling scenario. The paper tackles a very important problem in any sequential decision making system where multiple proposals can be exploited at the same time.\n\n[1] Stein berg, D. M., Oliveira, R., Ong, C. S., & Bonilla, E. V. (2024). Variational Search Distributions. CoRR, (), .", "weaknesses": "While I appreciate the extensive background writing it somewhat also pollutes the story a bit. For example, I am not actually sure why BO using latent representations is part of the Batch-BO introduction. While it is an important class of models it is in somewhat not used to \"paint a picture\" of what you are proposing. I think the work would come into a better context if some of the background was shortened and details where only introduced if they helped the story of the paper. This might be completely personal opinion but I find the text a bit scattered up to Section 3 and some of this could be moved to after the proposed method. Again just a personal opinion that has had no impact on my score or review.\n\nOne thing that I didn't fully understand is what is the specific q that you are using and what the p_0 is for the experiments that requires this. Maybe I've missed it in the paper but it would make the results easier to interpret.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)