Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
Authors: Taeyoung Yun
Kiyoung Om
Jaewoo Lee
Sujin Yun
Jinkyoo Park
Venue: ICML 2025 poster (2025)
Keywords: Diffusion Models
Posterior Inference
High-dimensional Black-box Optimization
Primary area: probabilistic_methods
TL;DR: We adopt posterior inference with diffusion models to solve high-dimensional black-box optimization problems efficiently

Abstract: Optimizing high-dimensional and complex black-box functions is crucial in numerous scientific applications.
While Bayesian optimization (BO) is a powerful method for sample-efficient optimization, it struggles with the curse of dimensionality and scaling to thousands of evaluations. 
Recently, leveraging generative models to solve black-box optimization problems has emerged as a promising framework.
However, those methods often underperform compared to BO methods due to limited expressivity and difficulty of uncertainty estimation in high-dimensional spaces.
To overcome these issues, we introduce \textbf{DiBO}, a novel framework for solving high-dimensional black-box optimization problems.
Our method iterates two stages. First, we train a diffusion model to capture the data distribution and deep ensembles to predict function values with uncertainty quantification.
Second, we cast the candidate selection as a posterior inference problem to balance exploration and exploitation in high-dimensional spaces. Concretely, we fine-tune diffusion models to amortize posterior inference.
Extensive experiments demonstrate that our method outperforms state-of-the-art baselines across synthetic and real-world tasks. Our code is publicly available \href{https://github.com/umkiyoung/DiBO}{here}.

Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
Taeyoung Yun
Kiyoung Om"
Jaewoo Lee
Sujin Yun
Jinkyoo Park
Abstract
Bayesian optimization (BO) (Kushner, 1964; Garnett, 2023) is a powerful method for solving black-box optimization. BO constructs a surrogate model from observed data and finds an input that maximizes the acquisition function to query the black-box function. However, scales poorly to high dimensions due to the curse of dimensionality (Wang et al., 2016) and struggles scaling thousands of evalua- tions (Wang al., 2018; Eriksson et al., 2019). To address these challenges, several approaches have been suggested to scale up BO for high- high-dimensional optimization problems. Some works propose a mapping from high-dimensional space into low-dimensional subspace (Gómez-Bombaree Bombarelli et al., 2018: Maus et al., 2022; Nayebi et al.. 2019; Letham et al., 2020) or assume additive structures of the target func- tion (Duvenaud et al., 2011: Rolland et al., 2018) 88) to perform optimization in low-dimensional spaces. However, these methods often rely on unrealistic assumptions.
Optimizing high-ding -ighedimensional and complex black- box functions is crucial numerous scientific ap- plications. While Bayesian optimization (BO) is powerful method for sample-efficies optimiza- tion, it struggles with the curse of dimensionality and scaling to thousands of evaluations. Recently, leveraging generative models to solve black-box optimization problems has emerged as promis- ing framework. However, those methods often underperform compared to BO methods due to limited expressivity and difficulty of uncertainty estimation in high-dimensional spaces. To over- come these issues, we introduce DiBO, novel framework for solving high-dimensional black- box optimization problems. Our method iterates two stages. First, we train diffusion model to capture the data distribution and deep ensembles to predict function values with uncertainty quan- tification. Second, we cast the candidate selection as posterior inference problem balance ex- ploration and exploitation in high- -igh-dimensional spaces. Concretely, we fine-tune diffusion models to amortize posterior inference. Extensive experi- ments demonstrate that our method outperforms state-of-the-ar baselines across synthetic and real- world tasks. Our code is publicly available here.
Other works partition the search space into promising local regions and search candidates within such regions (Eriksson et al., 2019; Wang et al., 2020), which exhibits promising results. However, these methods may struggle escape from local optima within limited number of evaluations due to several challenges. First, it is notoriously difficult to find an input that maximizes the acquisition function in high-dimensional spaces since the function is often highly non-convex and contains numerous local optima (Ament et al., 2023). Furthermore, as the data points lie on a tiny manifold compared to the entire search space, the uncer- tainty becomes extremely high in regions too far from the dataset (Oh et al., 2018).
1. Introduction
Optimizing high-dimensional and complex black-box func- tions crucial in various scientific and engineering applica- tions, including hyperparameter optimization (Šchié al., 2022), material discovery (Hernáneez-Lobato al., 2017), et al., 2018; Griffiths
Recently, generative model-based approaches have shown promising results in black-box optimization (Brookes et al., 2019; Kumar Levine, 2020; Wu et al., 2024). These meth- ods utilize an inverse mapping from function values the input domain and propose candidates via sampling from the trained model conditioned on high score. While they alleviate aforementioned issues in BO by converting opti- mization as sampling (Janner et al., 2022), their performance degrades in higher dimensions due to the limited expressiv- ity of underlying models (Brookes et al., 2019; Kumar Levine, 2020) or the difficulty of uncertainty estimation in high-dimensional spaces (Wu et al., 2024).
chemical design et al., 2023), and control systems (Candelieri et al., 2018).
Eqqual contribution 'Korea Advanced Institute of Science and Technology (KAIST). Correspondence to: Tacyoung Yun <99yty ac.ist.ac.kr.. kr>>, Kiyoung Om <sc99an krist.ac.kr>..
Proceedings of the 4rnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Posterior Inference with Diffusion Models for High-ddmensiooal Black-box Optimization
Target Function /00
UCB score He 00) + (X)
Prior Distribution
Posterior Distribution
Po (X)
Pe (X) - exp (s (Fo y + (0))
Figure 1. Motivating example of our method. In high-dimensional spaces, directly searching for input that maximizes UCB score may lead to sub-optimal results (As depicted in x the second figure). Sampling from the posterior distribution prevents overemphasized exploration in the boundary of search space and leads to efficient exploration (As depicted in tor of the last figure).
To overcome these issues, we introduce Diffusion models for Black-box Optimization (DiBO), a novel generative model-based approach for high-ding figh-dimensional black-bon op- timization. Our key idea is to cast the candidate selection problem as a posterior inference problem. Specifically, we first train a proxy and diffusion model, which serves as reward function and the prior. Then, construct a poste- rior distribution by multiplying two components and sample candidates from the posterior to balance exploration and ex- ploitation in high-dimensional spaces. Instead of searching for input that maximizes the acquisition function, sampling from the posterior prevents us from choosing the samples that lie too far from the dataset, as illustrated in Figure 1.
performance on a variety of tasks compared to state-of- the-art baselines, including BO methods, generative model- based methods, and evolutionary algorithms.
2. Preliminaries
2.1. Black-box Optimization
In black-box optimization, our objective is to find design x E X that maximizes the target black-box function f(x).
find x" = arg max fxx x
with R rounds of B batch of queries
(1)
Our method iterates two stages. First, train diffusion model to effectively capture high-dimensional data distri- bution and an ensemble of proxies to predict function val- ues with uncertainty quantification. During training, we adopt a reweighted training scheme proposed prior gener- ative model-based approaches focus on high-scoring data points (Kumar& & Levine, 2020; Tripp al., 2020). Second, sample candidates from the posterior distribution. As the sampling from the posterior distribution is intractable, we train an amortized sampler to generate unbiased sam- ples from the posterior by fine-tuning the diffusion model. While the posterior distribution remains highly non-convex, we can capture such complex and multi-modal distribution by exploiting the expressivity of the diffusion models. By repeating these two stages, we progressively get close to the high-scoring regions of the target function.
Querying designs in batches is practical in many real-world scenarios, such as biological sequence designs (Jain et al., 2022; Kim et al., 2024a). As the evaluation process is expensive in most cases, developing an algorithm with high sample efficiency is critical in black-box optimization.
2.2. Diffusion Probabilistic Models
Diffusion probabilistic models (Sohl-Dickstein et al., 2015; Ho et al., 2020) a class of generative models that aim approximate the true distribution (xxo) with a parametrized model of the form: (x = - po (Xer) dx1:r, where x and latent variables X1. Xx share the same dimension- ality. The joint distribution po (xn:)), often referred to as the reverse process, is defined via Markov chain that starts from standard Gaussian prior pr = IO,
As we have a limited budget for evaluations, it is beneficial to choose modes of the posterior distribution as propos- ing candidates. To accomplish this, we propose two post- processing strategies after training the amortized sampler: local search and filtering. Concretely, we generate many samples from the amortized sampler and improve them via local search. Then, we filter candidates with respect to the unnormalized posterior density. By incorporating these strategies, we can further boost the sample efficiency of our method across various optimization tasks.
T po (xx))))x))
(x pr (x() II
(2)
1=1
Po (x,) N (Ho( t), E.)
pe (x,- ) is a Gaussian transition from step & to to 1.
We choose forward process as fixed to be a Markov chain that progressively adds Gaussian noise the data according to a variance schedule $1...., BT:
We conduct extensive experiments on four synthetic and three real-world high-dimensional black-hox optimization tasks. We demonstrate that our method achieves superior
q(xi.r Xn) = II
q (xx, (x-1))
X
(4)
1=1
(xx/xx-1)=N((
x 3.I).
(5)
2

---

Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
Phase Training Models
Phase Sampling Candidates
Evaluate Samples
Diffusion model Ps
Proxy (x), op (x)
Posterior Inference
Py(()) x (x) exp (B (Mx tv
+ Gp))
Local Search
Filtering
Roweighting
Moving Dataset
Dataset
X1. "1, Xg
Figure 2. Overview of our method. Phase 1: Train diffusion models and ensemble of proxies. Phase 2: Sampling candidates from the posterior distribution and post-procette post-processing via local search and filtering. Then, we evaluate samples, update the dataset, and repeat the process until find optimal design.
Training Diffusion Models. We can train diffusion mod- els by optimizing the variational lower bound of nega- tive log-likelihood, 12 [ log po (xo)]. Following Ho et al.
can train the model with the samples from the distribution different from the current policy Py to ensure mode coverage (Sendera et al., 2024; Akhound-Sadegh et al., 2024).
(2020), we use a simplified loss with noise parameterization:
3. Method
(0) Ex t-U(), I- [ll co +)] (6)
In this section, we introduce DiBO, a novel approach for high-dimensional black-box optimization by leveraging dif- fusion models. Our method iterates two stages to find an optimal design in high-dimensions spaces. First, we train a diffusion model to capture the data distribution and an en- semble of proxies to predict function values with uncertainty quantification. During training, we apply a reweighted train- ing scheme to focus on high-scoring regions. Next, we sample candidates from the posterior distribution. To fur- ther improve sample efficiency, we adopt local search and filtering to select diverse modes of posterior distribution as candidates. We then evaluate the selected candidates, update the dataset, and repeat the process until we find an optimal design. Figure 2 shows the overview of our method.
Here, eø (xe: f) is the learned noise estimator, and
BBc
no(x,, 1) Van X1
c0 ex( ()),
(7)
VI
where at 1 Be and at IT atg-
Fine-tuning Diffusion Models for Posterior Inference. Given a diffusion model (x( and a reward function r(x), we can define a posterior distribution y x po (x)rr(x), where the diffusion model serves as a prior. Sampling from the posterior distribution enables us to solve various down- stream tasks (Chung et al., 2023; Lu al., 2023; Venka- traman et al., 2024). For example, in offline reinforcement learning, if we train conditional diffusion model p(a (c)s) as behavior policy, it requires sampling from the product distribution of behavior policy and Q-function (Nair et al., 2020), i.c., T (aass) s) xx u(als) exp(3 Q(s,a)))
3.1. Phase 1: Training Models
In each round r, we have a pre-collected dataset of input- output pairs D, = {(x, v:)) where is the number of data points.
When the prior modeled as a diffusion model, the sam- pling from the posterior distribution is intractable due to the hierarchical nature of the sampling process. Fortunately, we can utilize relative trajectory balance (RTB) loss suggested by Venkatraman et al. (2024) to learn amortized sampler Pv that approximates the posterior distribution by fine-tuning the prior diffusion model as follows:
Training Diffusion Model. We first train a diffusion model po po(x) using D, to capture the data distribution. We choose diffusion model it has a powerful capability learn the distribution of high-dimensional data across various domains (Ramesh et al., 2022; Ho et al., 2022).
Zy Pa (Xo:T)
2
Training Ensemble of Proxies. We also train a proxy model to predict function values using the dataset D,.. As it is notoriously difficult to accurately predict all possible regions in high-dimensional spaces with a limited amount of samples, we need to properly quantify the uncertainty of our proxy model. To this end, we train an ensemble of proxies fer. Sox to estimate the epistemic uncertainty of the model (Lakshminarayanan al., 2017).
L(X0:T; 15) (log r(xo) Po (Xn:T)
(8)
where and Z, is the partition function estimator. If the loss converges to zero for all possible trajectories Xo:7, the amortized sampler matches the posterior distribution.
One of the main advantages of RTB loss is that we can train the model in an off-policy manner. other words, we
3

---

Posterior Inference with Diffusion Models for Higg-dimennional Black-box Optimization
Reweighted Training. In the training stage, we intro- duce a reweighted training scheme focus high-scoring data points since our objective is to find optimal design that maximizes the target black-box function. Reweighted training has been widely used in generative modeling for black-box optimization, especially in offline settings (Ku- mar & Levine, 2020; Krishnamoorthy et al., 2023; Kim
Algorithm DiBO
1: Input: Initial dataset Do; Max rounds R; Batch size B: Buffer size L,; Diffusion model po: pei Proxy
for .. fox
2: for r r O...., - do
3: Phase 1. Training Models
4: Compute weights w(y. D,) with Equation (9)
al., 2024b). Formally, we can compute the weight for each data point as follows:
Train for. Sox with Equation (10)
6: Train pe with Equation (11)
exp(y)
7:
wly D.) Exx DED,, exp(v')
8: Phase 2. Sampling Candidates
9:
Initialize pa f Po
Then, our training objective for proxies and diffusion models can be described as follows:
10:
Train pa with Equation (14) using X0:T from pe or from the dataset D,
11:
Sample {x;) [xs];=1 ~ Pe
K
12:
Update JM into {x]]
J.M with Equation (15)
[(01.k)=
Σ
Σ
w(y D,) (v for (x))²
(10)
13:
Filter top-1 samples {x))
1B among {x] Ji=
L.M
Aml (X.y)ED.
14:
[(0) =
Σ
wly, D,) log (x
(11)
15:
Evaluation and Moving Dataset
(x.y)ED,
16:
Evaluate y/b = f(xx); Ab = yb=1.... 1...... B
17:
Update Dr+1 + D, U {(xx] 1)))
3.2. Phase 2: Sampling Candidates
18:
if [Dr+1l L then
19:
Remove bottom- (/D,+1 L) samples from D,+1
After training models, we sample candidates from the pos- terior distribution to query the black-box function. As sampling from the posterior intractable, we introduce an amortized sampler that generates unbiased samples from the posterior by fine-tuning the pre-trained diffusion model. Then, to further boost the sample efficiency, we apply post- processing strategies, local search and filtering, to select diverse modes of the posterior distribution as candidates.
20: end if
21:
end for
from Pur is intractable. Therefore, we introduce amortized sampler p& R Purs which can be obtained by fine-tuning the trained diffusion model with relative trajectory balance loss suggested by Venkatraman et al. (2024). Formally, we train the parameters of Pv with the following objective:
Amortizing Posterior Inference. Our key idea is to sample candidates from the probability distribution that satisfies two desiderata: (1) promote exploration towards both high- rewarding and highly uncertain regions and (2) prevent the sampled candidates from deviating excessively from the data distribution. To accomplish these objectives, we can define our target distribution as follows:
Zap Pol (Xa.)
2
((xoTTi() = log exp (3 r (xo)) po (X
(14)
where xo = x, and Zy is parameterized partition function.
As mentioned in the previous section, we can employ off- policy training to effectively match the target distribution. To this end, we train pe with the on-policy trajectories from the model mixed with the trajectories generated by samples from the pre-collected dataset D,.. Please refer to Appendix B.2.1 for more details on off-policy training.
1
Ptar (x) = arg max Expo [x)( - DKL (p |1 pe) (12) PEP
where Ta (x) = u (x)x)) + (x(x) UCB score from the proxy, and Ha (x), (x) indicate mean and the standard deviation from proxy predictions, respectively. P denotes the set of feasible probability distributions and ,3 is an in- verse temperature. Target distribution that maximizes the right part of the Equation (12) can be analytically derived as follows (Nair et al., 2020):
After training Pv, we can generate unbiased samples from our target distribution. However, as we have a limited eval- uation budget, it is advantageous to refine candidates to exhibit a higher probability density of target distribution, i.e., modes of distribution. To achieve this, we introduce two post-processing strategies: local search and filtering.
Pear (x) = z po (x) exp (B (x))
(13)
Local Search. First, we generate a set of candidates [x,)M
LM by sampling from Ps. For each candidate x,, we perform a local search using gradient ascent to move it to- ward high-density regions. Formally, we update the original
where = SxEx po (x) exp (8 . (x)) is a partition func- tion. As we do not know the partition function, sampling
4

---

Posterior Inference with Diffusion Models for High-dimenniionl Black-box Optimization
Rastrigin-200D
Ackley-200D
Levy-200D
Rasenbrock-2000
114
0.20
Btee: Voltve Found
Best Value Found
Btet Valure Found
Best Valuee Found
1000
-200
-150
203
Number
Number Evalustions
00
Number Evalustions
Number Evalustions
Rastrigin-400D
Ackley-400D
Levy-400D
Rosenbrock-400D
(tesd Veluee round
-1.00
Best Value Found
Best
Best
Number Evaluations
Number Evaluations
Number of Evaluations
Number of Evaluations
DIBO
TURBO
MCTS
MCMC
DI1-80
Figure 3. Comparison between our method against baselines in synthetic tasks. Experiments are conducted with four random seeds, and the mean and one standard deviation reported.
candidate X, to X as follows:
4. Experiments
x +1 - x + V x= (po(x) exp(3 101---ro (x)())) (15)
In this section, we present experimental results on high- dimensional black-box optimization tasks. First, we conduct experiments on four synthetic functions commonly used in the BO literature (Eriksson al., 2019; Wang et al., 2020). Then, we perform experiments three high-dimensional real-world tasks, including HalfCheetah- 102D from Mu- JoCo Locomotion, RoverPlanning- 100D, and DNA-180D from LassoBench. The description of each task is available in Appendix A.
for j = ..... J 1, where x = x,, x = (16)
where 17 is the step size, and is the number of updates. To estimate the marginal probability px),,, we employ the probability flow ordinary differential equation (PF ODE) Song et al. (2021) with differentiable ODE solver. Please refer to Appendix B.2.2 for more details on local search.
As the amortized sampler Py is parametrized as a diffusion model, we can expect that it generates samples across di- verse possible promising regions. Then, the local search procedure guides samples towards modes of each promising region (Kim al., 2024c).
4.1. Baselines
We evaluate our method against state-of-the-art (SOTA) baselines for high-dimensional black-box optimization, in- cluding BO methods: TuRBO (Eriksson al., 2019), LA- MCTS (Wang al., 2020), MCMC-BO (Yi al., 2024), CMA-BO (Ngo al., 2024), an evolutionary search ap- proach: CMA-ES (Hansen, 2006), and existing genera- tive model-based algorithms: CbAS (Brookes et al., 2019), MINs (Kumar & Levine, 2020), DDOM (Krishnamoorthy et al., 2023), and Diff-BBO (Wu et al., 2024). Details about baseline implementation can be found in Appendix C.
Filtering. After the local search, we introduce filtering to select B candidates for evaluation among generated samples {xi,- .xxx J. To be specific, we select the top-1 samples with respect to the unnormalized target density, pe (x) exp(3-rg(x)) Through filtering, we can effectively capture the high-quality modes of the posterior distribution, thereby significantly improving the sample efficiency.
3.3. Evaluation and Moving Dataset
4.2. Synthetic function tasks
After selecting candidates, we evaluate their function values by querying the black-box function. Then, we update the observations. When updating the dataset, we remove the samples with the lowest function values if the size of the dataset is larger than the buffer size L. We empirically find that it reduces the computational complex- ity during training and ensures that the model concentrates more on the high-scoring regions.
We benchmark four synthetic functions that are widely used for evaluating high-dimensional black-box optimization al- gorithms: Rastrigin, Ackley, Levy, and Rosenbrock. We evaluate each function in both D 200 and 400 dimen- sions and set the search space X' [lb, ub] for each func- tion following previous works (Wang et al., 2020; Yi et al., 2024). All experiments are conducted with initial dataset
dataset with
5

---

Posterior Inference with Diffusion Models for Hiig--dimensional Black-box Optimization
HalfCheetah-
RoverPlanning-1000
DNA-180D
-120
-3.7
Best Volue Found
Best Valua Found
Best Value Found
--330
-31
-208
Number of Evalustions
Number of Evaluations
Number of Evaluacions
[Ours]
TuRaO
BC
CM
DB1.80
CMA
Figure+ 4. Comparison between method against baselines in real-world tasks. Experiments are conducted with random seeds, and the mean and one standard deviation are reported.
size [Dol = 200, batch size B = 100, and 10, 000 as the maximum evaluation limit.'
and DNA-180D. Each experiment starts with [Dol = 100 initial samples, a batch size of B = 50, and a maximum evaluation limit of 2, 000.
As shown in Figure 3, our method significantly outperforms all baselines across four synthetic functions. Furthermore, observe that our method not only discovers high-scoring designs but also achieves high sample efficiency. Specif- ically, our method exhibits significant gap compared to baselines on Rastrigin and Ackley tasks, which have numer- ous local optima near the global optimum. It highlights that our key idea, sampling candidates from the posterior distribution, enables effective exploration of promising re- gions in iighddimensional spaces and mitigates the risk of converging sub-optimal regions.
The results are illustrated in Figure Our method ex- hibits superiority in terms of both the performance and the sample efficiency compared to baseline approaches. While other methods show inconsistent performance excelling on some tasks but underperforming on others-our method consistently surpassed the baselines, highlighting its robust- ness across a broader range of tasks.
5. Additional Analysis
In this section, we carefully analyze the effectiveness of each component of our method. We conduct additional analysis in Rastrigin-200D and HalfCheetah- 102D tasks.
We find that generative model- based approaches such as CbAS and MINs perform well the early stage but struggle to improve the performance through subsequent iterations. While diffusion-basee methods (DDOM, Diff-BBO) show consistent improvements across various tasks, the perfor- mance lags behind that of recent BO methods. These results demonstrate that the superiority of our method stems not just from using diffusion models but also from our novel framework calibrated for high-dimensional black-box op- timization. We also conduct an ablation study on each component of our method in the following section.
A blation on Reweighted Training. We examine the im- pact of reweighted training during the model training stage. We conduct experiments by omitting the reweighting compo- nent. As shown in Figure 5a, the performance of our method sig significantly drops if we remove the reweighted training. It underscores that focusing on high-scoring regions acceler- ates the optimization process. We also conduct analysis on the number of training epochs in Appendix D. We find that naively increasing the number of training epochs does not lead to an improvement in performance.
We also compare our method with high-dimensional Bayesian optimization methods. While these approaches hibit comparable performance and often outperform genera- tive model-based baselines, they remain relatively sample- inefficient compared to our method. It reveals that sampling diverse candidates from the posterior distribution can be a sample-efficien solution for high-dimensional black-box optimization compared to choosing inputs that maximize the acquisition function.
Ablation on Sampling Procedure. We analyze the effect of strategies we have proposed during the sampling stage. We conduct experiments without filtering, local search, and finally completely remove the amortized inference stage and propose samples from the diffusion model po-
As depicted Figure 5b, each component of our method significantly affects the performance. Notably, when we remove both local search and filtering strategies, we observe that the sample efficiency of our method significantly drops,
4.3. Real-World Tasks
To evaluate the performance and adaptability of our method in real-world scenarios, conduct experiments on three additional tasks: HalfCheetah- 102D, RoverPlanning-1000
For MCMC-BO, report the score of budget 000 tasks with D = 400 due to memory constraints.
6

---

Posterior Inference with Diffusion Models for High-dimenniooal Black-box Optimization
Rastrigin-200D
HollChestah-1020
Rastrigin-2000
HallCheetah-102D
-32
-200
-120
Valu
250
Best Value
1.09
1:200
1200
12200
-100
0.80
fittedig
-100
littring
grting
2000
400
LEG
LSS 200
sen
So
1oe con si 1oe oec
(a) Reweighted Training
(b) Sampling Procedure
Rastrigin-2000
HaliChestah-1020
Rastrigin-2000
MollCheetah-1022
-.009
Bood
Best
3-10"6
5-10"5
1-10"5
(Delauti (Delautt
1=100% (Defart)
-200
$-100"3
4-10"
$=10"2
3-10"3
1-200
1:500
1000 (Gehrut) Meving Datanat
L-100 L11200
1-0300 (Detect))
Moxing Detaset
2000
Kumber Eoslaations
400
D800
Number
2007
Number Evaluntion
2006
Number
50
LoW 20 uoe awc
(c) Analysis on |3
(d) Analysis on L
Figure 5. Ablation on various components in DiBO. Experiments are conducted on the Rastrigin-2000 and HalfChectah- 102D tasks.
demonstrating the effectiveness of the proposed components. We also conduct further analysis on the number of local search steps J in Appendix D.2 and the effect of off- off-policy training for amortized inference in Appendix D.3.
Analysis on initial dataset size [Pol and batch size B. Initial dataset size and batch size for each round can be crucial in the performance of the method. To verify the effeet of these components, we conduct additional analysis in Appendices D.4 and D.5 and find that our method is robust to different initial experiment settings.
Analysis on Inverse Temperature B. The parameter :3 in Equation (12) governs the trade-off between exploitation and exploration. If ( is too small, the method tends to ex- ploit already discovered high-scoring regions. On the other hand, if (3 is too large, we generate samples that deviate too far from the current dataset and overemphasize exploration of the boundary of search space.
Analysis on uncertainty estimation. We further analyze how the uncertainty estimation affects the performance of our method in Appendix D.6. We find that using ensembles to measure uncertainty is powerful approach, and using the UCB score based on the uncertainty can effectively guide the model to explore promising regions.
As shown in Figure 5c, when / is too small, it often leads to convergence on a local optimum due to limited exploration. Conversely, if B too large, the model becomes overly dependent on the proxy function, resulting in excessive exploration and ultimately slowing the convergence.
Time complexity of the method. We conduct analysis on the time complexity of our method in Appendix D.7. Our findings show that our method exhibits relatively low or comparable running time compared to other baselines.
Analysis on Buffer size I.. As we descrihed in Section 3.3, we introduce buffer size L to maintain the dataset with high- scoring samples collected during the evaluation cycles. The choice of L impacts both the time complexity and the sample efficiency of our method.
6. Related Works
6.1. -iig--dimeniional Black-Box Optimization
Various approaches have been proposed to address high- dimensional black-box optimization. In Bayesian optimiza- tion (BO), some methods assume that objective function can be decomposed into sum of several low-dimensional functions and train a large number of Gaussian processes (GPs) (Duvenaud et al., 2011; Kandasamy et al., 2015; Gard- ner et al., 2017). However, relying on training multiple
As illustrated in Figure 5d, using a small buffer size results in reaching suboptimal results, also causing early perfor- mance saturation. In contrast, a larger buffer size can reach the optimal value as in our default setting but significantly decelerate the rate of performance improvement.
7

---

Posterior Inference with Diffusion Models for High-dimenniioal Black-box Optimization
GPs reduces its scalability to a large number of evaluations. Other approaches assume that high-dimensional objective functions reside in a low-dimensional active subspace and introduce mapping to low wwwdimmesiooal spaces (Chen et al., 2012; Garnett et al., 2014; Nayebi et al., 2019; Letham et al., 2020). However, these methods make strong assumptions that often fail to align with real-world problems.
the conditioning target value during candidate sampling.
Diff-BBO is closely related to our work, particularly in its use of diffusion models and its focus on online scenarios. However, utilizing multiple diffusion models results in a significant computational burden throughout the optimiza- tion process. Additionally, estimating uncertainty using diffusion models in high-dimensional spaces remains chal- lenging, which can impact the effectiveness of uncertainty- based acquisition and may lead to suboptimal results. In contrast, our approach alleviates the computational burden by introducing moving dataset and effectively scaling up to high- high-dimensional tasks with our posterior sampling strategy.
Another line of BO methods utilizes local modeling or par- titioning of the search space address high dimensionality and scalability. TuRBO (Eriksson et al., 2019) fits multiple local models and restricts the search space to small trust regions to improve scalability. LA-MCTS (Wang et al., 2020) trains an unsupervised K-means classifier to parti- tion the search space, identifies promising regions for sam- pling, and then employs BO-based optimizers such as BO or TuRBO. Including LA-MCTS, aega-aaggosssmss that can be adapted to existing high-dimensional BO methods have also emerged. MCMC-BO (Yi et al., 2024) adopts Markov Chain Monte Carlo (MCMC) to adjust a set of candidate points towards more promising positions, and CMA-BO (Ngo et al., 2024) utilizes covariance matrix adaptation strat- egy to define local region that has the highest probability of containing global optimum.
6.3. Amortized Inference in Diffusson Models
As diffusion models generate samples through a chain of stochastic transformations, sampling from the posterior dis- tribution p x px(x)r(x) is intractable. One of the widely used methods is estimating the guidance term by training a classifier on noised data (Dhariwal & Nichol, 2021; Lu et al., 2023). However, such data is unavailable in most cases, and it is often hard to train such a classifier in high- -igh-dimensional settings. Although Reinforcement learn- ing (RL) methods have recently been proposed and shown interesting results (Black et al., 2024; Fan et al., 2024), naive RL fine-tuinng does not provide an unbiased sampler of the target distribution (Uchara et al., 2024; Domingo-Enrich et al., 2024). To this end, we choose relative trajectory balance proposed by Venkatraman et al. (2024) to obtain an unbiased sampler of the posterior distribution without training an additional classifier. Furthermore, we propose two post-processing strategies, local search and filtering, to improve the sample efficiency of our method.
Evolutionary Algorithms (EAs) are also widely applied in high- dighesinnnio black-box optimization. The most repre- sentative method, the covariance matrix adaptation evolution strategy (CMA-ES) (Hansen, 2006), operates by building and refining a probabilistic model of the search space. It iteratively updates the covariance matrix of a multivariate Gaussian distribution to capture the structure of promising solutions and uses feedback from the objective function to guide the search towards the optimal solution.
6.2. Generative Model-based Optimization
7. Conclusion
Several methods have been developed that utilize generative models to optimize black-box functions. Most approaches learn an inverse mapping from function values to the input domain and propose promising solutions by sampling from the trained model, conditioned on a high score (Brookes et al., 2019; Kumar& & Levine, 2020; Krishnamoorthy et al., 2023; Wu et al., 2024; Kim al., 2024b).
In this work, we introduce DiBO, a novel generative model- based framework for high- nigh-dimensional and scalable black- box optimization. We repeat the process of training models and sampling candidates with diffusion models to find a global optimum a sample-efficient manner. Specifically, by sampling candidates from the posterior distribution, we can effectively balance exploration and exploitation in high- dimensional spaces. We observe that our method surpasses various black-box box optimization methods across synthetic and real-world tasks.
Building on the success of diffusson models, widely known for their expressivity in high-dimensional spaces (Ramesh et al., 2022; Ho et al., 2022), leveraging diffusion models for black-box optimization has also emerged (Krishnamoor- thy et al., 2023; Wu et al., 2024; Kong et al., 2024; Yun et al., 2024). DDOM (Krishnamoorthy et al., 2023) trains conditional diffusion model with classifier-free guidance (Ho& & Salimans, 2021) and incorporates reweighted training to enhance the performance. Diff-BBO (Wu et al., 2024) trains an ensemble of conditional diffusion models, then employs an uncertainty-be based acquisition function to select
Limitation and Future work. While our method shows superior performance on a variety of tasks, we need to train the diffusion model with the updated dataset in every round, Although we analyze that our method exhibits low computa- tional complexity compared to the state-of-the-art baselines, one may consider constructing framework that can effi- ciently reuse the trained models from the previous rounds.
8

---

Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
Acknowledgement
Chung, H., Kim, J., Mecann, M. T., Klasky, M. L.,, and Ye, J. C. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Confer- ence on Learning Representations, 2023.
This work is supported by the National Research Founda- tion of Korea (NRF) grant funded by the Korea govern- ment(MSITT) (No. RS-2025-00563763).
Dhariwal, P. and Nichol, A. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Impact Statement
Real-worid design optimization can lead to impressive breakthroughs, but it also carries certain risks. For example, while advanced pharmaceutical research could treat diseases once seen incurable, those same methods could be mis- used create harmful biochemical substances. Researchers have responsibility to prioritize the well-being of socicty.
Domingo-Enrich, C.,, Drozdzal, M., Karrer, B., and Chen, T. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. arXiv preprint arXiv:2409.0088 2024.
Duvenaud, K., Nickisch, H., and Rasmussen, C. Addi- tive gaussian processes. Advances in neural information processing systems, 24, 2011.
References
Sahound-Sadegh T., Rector-Brooks, J., Bose, J., Mittal, S., Lemos, P., Liu, C.-H., Sendera, M., Ravanbakhsh, S., Gidel, G., Bengio, Y., et al. Iterated denoising en- ergy matching for sampling from boltzmann densities. In Forty-first International Conference on Machine Learn- ing, 2024.
Eriksson, D., Pearce, M., Gardner, J., Turner, R. D., and Poloczek, M. Scalable global optimization via local bayesian optimization. Advances neural information processing systems, 32, 2019.
Fan, Y., Watkins, O., Du, Y., Liu, H., Ryu, M., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Lee, K., and Lee, K. Rein- forcement learning fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Sys- tems, 36, 2024.
Ament, S., Daulton, S., Eriksson, D., Balandat, M., and Bakshy, Unexpected improvements to expected im- provement for bayesian optimization. Advances in Neural Information Processing Systems, 36:20577-20612, 2023.
Gal, and Ghahramani, Z. Dropout as a bayesian approx- imation: Representing model uncertainty in deep learn- ing. In international conference on machine learning. pp. 1050-1059, PMLR, 2016.
Ba, L. Layer normalization. arXiv: EXXi:1607.064550 2016.
arXiv preprint
Black, K., Janner, M., Du, Y., Kostrikov, I., and Levine, S. Training diffusion models with reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024.
Gardner, J., Guo, C., Weinberger, K., Garnett, R., and Grosse, R. Discovering and exploiting additive struc- ture for bayesian optimization. Artificial Intelligence and Statistics, pp. 1311-1319, PMLR, 2017.
Brookes, D., Park, H., and Listgarten, J. Conditioning by adaptive sampling for robust design. In International conference machine learning, pp- 773-782. PMLR, 2019.
Garnett, R. Bayesian optimization. Cambridge University Press, 2023.
Garnett, R., Osbore,, M. A., and Hennig, P. Active learning of linear embeddings for gaussian processes. In Pro- ceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, pp. 230-2390 2014.
Candelieri, A., Perego, R., Archetti, F., et al. Bayesian optimization of pump operations in water distribution systems. Journal of Global Optimization, 71:213-235, 2018.
R., Wei, J. N., Duvenaud, D., Hernández-Lobatee J. M., Sánchez-L B., She- berla, D.,, equilera-Iparraguing P., and Aspuru Guzik, A. Automatic chemical de- sign using a data-driven continuous representation of molecules. ACS central science, 4(2):268-2 276, 2018.
Chen, B., Castro, R., and Krause, A. Joint optimization and variable selection of high-dimensional gaussian processes. In 26th International Conference on Machine Learning (ICML 2009), pp. 1423-1430 2012.
J., Hirzel, T. D., Adams,
Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Grathwohl, W., Chen, R. T., Bettencourt, J., Sutskever, I., and Duvenaud, D. Ffjord: Free-fom continuous dynam- ics for scalable reversible generative models. In Interna- tional Conference on Learning Representations, 2019.
Chen, R. T. Q. torchdiffeq. 2018. URL https:// gi github. www.ctooo/roeeeeee httm://tggccee/ttccciifeee

---

Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
Griffiths, R.-R., Klarner, L.,, Moss, H., Ravuri, A., Truong, S., Du, Y., Stanton, S., Tom, G., Rankovic, B., Jamasb, A., et al. Gauche: a library for gaussian processes in chemistry. Advances in Neural Information Processing Systems, 36:76923-76946, 2023.
Kim, H., Kim, M., Yun, T., Choi, S., Bengio, E., Hernández- Garefa, A., and Park, J. Improved off-policy reinforce- ment learning in biological sequence design. In NeurIPS 2024 Workshop on Al for New Drug Modalities, 2024a.
Kim, M., Berto, E,, Ahn, S., and Park, J. Bootstrapped training of coore-conditioned generator for offline design of biological sequences. Advances in Neural Information Processing Systems, 36, 2024b.
Hansen, N. The ema evolution strategy: A comparing re- view. Towards a New Evolutionary Computation: Ad- vances on Estimation of Distribution Algorithms, 192:75, 2006.
Kim, M., Yun, T., Bengio, E., Zhang, D., Bengio, Y., Ahn, S., and Park, J. Local search gflownets. In The Twelfth International Conference on Learning Representations, 2024c.
Hansen, N., Akimoto, Y., and Baudis, P. CMA-ES/pycma on Github. Zenodo, D01:10.5281/zenodo 25596344, Febru- ary 2019. URL https: //oii. org/1 028111 zenodo. 2559634.
Kingma, D., Salimans, T., Poole, B., and Ho, Varia- tional diffusion models. Advances in neural information processing systems, 11-21696-21701, 2021.
Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelus). arXiv preprint 16XX::1600600415,, 2016.
Hernández-Lobato, M., Requeima, J., Kyzer-Knapp. E. O., and Aspuru-Guzik, A. Parallel and distributed thomp- son sampling for large-scale accelerated exploration of chemical space. In International conference on machine learning, pp. 1470-1479. PMLR, 2017.
Kingma, D. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR), 2015.
Kong, L., Du, Y., Mu, W., Neklyudov, K., De Bortoli, V., Wu, D., Wang. H., Ferber, A., Ma, Y.A.,, Gomes, C. P., et al. Diffusion models as constrained samplers for optimization with unknown constraints. arXiv preprint arXiv:2402. 18012, 2024.
Ho, J. and Salimans, T. Classifier-free diffusion guidance. In NeurlPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Krishnamoorthy, S., Maskkaria, S. M., and Grover, A. Diffu- sion models for black- box optimization. In International Conference on Machine Learning, pp. 17842-17857. PMLR, 2023.
Ho, J., Chan, W.. Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. Imagen video: High definition video generation with diffusion models. arXiv preprint 10.0:322,0002330, 2022.
Kumar, A. and Levine, S. Model inversion networks for model-based optimization. Advances in neural informu- tion processing systems, 33:5126-5117, 2020.
Hutchinson, M. F. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Com- munications in siriistiss-Simmaatiion and Computation, 18(3):005-11076 1989.
Kushner, H. J. A new method of locating the maximum point of an arbitrary multipeak curve the presence of noise. 1964.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Jain, M., Bengio, E., Harrena A., Rector- Brooor J., Dossou, B. F., Ekbote, C. A., Fu, J., Zhang, T., Kil- gour, M., Zhang, D., et al. Biological sequence design with gflownets. In International Conference Machine PMLR, 2022.
Learning, pp.
Letham, B., Calandra, R. Rai, A., and Bakshy, Re-examining linear embeddings for high-dimensional bayesian optimization. Advances in neural information processing systems, 33:1546-155 1558, 2020.
Janner, M., Du, Y., Tenenbaum, J., and Levine, S. Planning with diffusion for flexible behavior synthesis. In Interna- tional Conference on Machine Learning, pp. 9902-9915. PMLR, 2022.
Lu, C., Chen, H., Chen, J., Su, H., Li, C.,, and Zhu, J. Con- trastive energy prediction for exact energy-guider diffu- sion sampling in offline reinforcement learning. In Inter- national Conference on Machine Learning, pp. 22825 22855. PMLR, 2023.
Kandasamy, K., Schneider, J., and Póczos, B. High di- mensional bayesian optimisation and bandits via additive models. In International conference on machine learning, pp. 295-304 PMLR, 2015.
10

---

Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
Maus, N., Jones, H., Moore, J., Kusner, M. J., Bradshaw, J., and Gardner, J. Local latent space bayesian optimization over structured inputs. Advances in neural information processing systems, 35:34505 -4444555--4188,, 2022.
Todorov, E., Erez, T., and Tassa, Y. Mujoco: A physies engine for model-based control. In 2012 IEEF/RSJ inter- national conference intelligent robots and systems, pp. 5026-5033. IEEE, 2012.
Nair, A., Gupta, A., Dalal, M., and Levine, S. Awac: Accel- erating online reinforcement learning with offline datasets. arXiv preprint wwt.iv:200600933. 2020.
Trabucco, B., Geng, X., Kumar, A., and Levine, S. Design- bench: Benchmarks for data-driven offline model-based optimization. In International Conference on Machine Learning, pp. 21668-21676. PMLR, 2022.
Nayebi, A., Munteanu, A., and Poloczek, M. A framework for bayesian optimization in embedded subspaces. In International Conference on Machine Learning, pp. 4752- 4761. PMLR, 2019.
Tripp, A., Daxberger, E., and HemááneezLobato, J. M. Sample-efficient optimization in the latent space of deep generative models via weighted retraining, Advances in Neural Information Processing Systems, 33:1 1259- 11272, 2020.
Ngo, L., Ha, H., Chan, J., Nguyen, V., and Zhang, H. High- dimensional bayesian optimization via covariance matrix adaptation strategy. Transactions on Machine Learning Research, 2024.
Uehara, M., Zhao, Y., Black, K., Hajiramezanali, E., Scalia, G., Diamant, N. L., Tseng, A. M., Biancalani, T., and Levine, S. Fine-tuning of continuous timeuuuuu--meeeeeee diffusion models as entropy-regularizer control. arXiv preprint arXiv:2402.15194, 2024.
Oh, C., Gavves, E., and Welling, M. Bock: Bayesian opti- mization with cylindrical kernels. In International Con- ference on Machine Learning, pp. 3868-3877 PMLR, 2018.
Venkatraman, Jain, M., Scimeca, L.,, Kim, M., Sendera, M., Hasan, M., Rowe, L., Mittal, S., Lemos, P., Bengio, E., Adam, A., Rector-Brooks, J., Bengio, Y.. Berseth, G., and Malkin, N. Amortizing intractable inference in diffusion models for vision, language, and control. In The Thirty-eighth Annual Conference Neural In- formation Processing Systems, 2024. URL https: / / opppeevvee.... net/ forumm Seerrummdd-VVTMMaa. d=ggdd/Ma//...........
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3 2022.
Rolland, P., Scarlett, J., Bogunovic, I., and Cevher, V. High- dimensional bayesian optimization via additive models with overlapping groups. In International conference on artificial intelligence and statistics, pp. 298-307. PMLR, 2018.
Wang, L., Fonseca, R., and Tian, Y. Learning search space partition for black-box optimization using monte carlo tree search. Advances in Neural Information Processing Systems, 13:19511-19522 2020.
Šehic, K., Gramfort, A., Salmon, J., and Nardi, L. Las- sobench: A high-dimensional hyperparameter optimiza- tion benchmark suite for lasso. In International Confer- ence on Automated Machine Learning, pp. 2-1. PMLR, 2022.
Wang. 7., Hutter, F., Zoghi, M., Matheson, D., and
De Feitas, N. Bayesian optimization in a billion dimen- sions via random embeddings. Journal of Artificial Intel- ligence Research, 55:361-387 2016.
Sendera, M., Kim, M., Mittal, S., Lemos, P., Scimeca, L., Rector-Bi Brooos, J., Adam, A., Bengio, Y., and Malkin, N. Improved off- off-policy training of diffusion samplers. In The Shirrt-Eiggtth Annual Conference on Neural Information Processing Systems, pp. 1-30. ACM, 2024.
Wang. Z., Gehring, C., Kohli, P., and Jegelka, S. Batched large-scale hayesian optimization in high-dimensional spaces. In International Conference on Artificial Intelli- gence and Statistics, pp. 745-754. PMLR, 2018
Skilling, J.
The eigenvalues of mega-dimennsooal matrices. Maximum Entropy and Bayesian Methods: Cambridge, England, 1988, pp. 455 466, 1989.
Wu, D., Kuang, N. L., Niu, R., Ma, Y--.,, and Yu, R. Diff- bbo: Diffusion- inverse modeling for black-box optimization. arXiv preprint Niv:2407.006/11 2024.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. In International conference on machine learning, pp. 2256-2265. PMLR, 2015.
Yi, Z., Wei, Y., Cheng, C. X., He, K., and Sui, Y. Improving sample efficiency of high dimensional bayesian optimiza- tion with meme. arXiv preprint arXiv:2401.0266 2024.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score- generative modeling through stochastic differrnntial equations. In International Conference on Learning Representations, 2021.
Yun, T., Yun, S., Lee, J., and Park, J. Guided trajectory generation with diffusion models for offline model-based optimization. In The Neural Information Processing Systems, 2024.
Annual Conference on

---

Posterior Inference with Diffusion Models for Higg-ddimensionll Black-box Optimization
A. Task Details
In this section, we present detailed description the benchmark tasks used our experiments.
A.1. Synthetic Functions
We conduct experiments on four complex synthetic functions that are widely used in BQ literature: Rastrigin, Ackley, Levy, and Rosenbrock. Levy and Rosenbrock have global optima within long, flat valleys, whereas Rastrigin and Ackley have numerous local optima. This characteristic makes all four functions particularly challenging as the dimensionality increases. Following previous studies (Wang et al., 2020; Yi al., 2024), we set the search space for each function as, Rastrigin: [-5,5]2 Ackley: [-5, 10) Levy: [-10, 10)", and Rosenbrock: [-5, 10)
A.2. MuJoCo locomotion
MuJoCo locomotion task (Todorov al., 2012) is a popular benchmark in Reinforcement Learning (RL). In this context, we optimize linear policy W described by the equation and our goal is to identify the weight matrix that maximizes this return. We specifically focus the HalfCheetah task, which has dimensionality of 102. Each entry of the weight matrix W is constrained to the range [-1, 1], and we utilize 3 rollouts for each evaluation. We followed the implementation of these tasks from the prior work Ngo et al. (2024).
Ws. The average return of this policy serves as objective,
A.3. Rover Trajectory Optimization
Rover Trajectory Optimization is task determining the trajectory of a rover in a 2D environment suggested by Wang et al. (2018). Following previous work Ngo et al. (2024), we utilized a much harder version with a 100-dimensional variant, optimizing 50 distinct points. This task requires specifying a starting position s, a target position g. and a cost function applicable the state space. We can calculate the cost c(x) for a specific trajectory solution by integrating the cost function
along the trajectory x E [o, 1] The reward function is defined as: f(x) c(x) + x([xx, - -s|12 [g99,100 9ll1) + b. We followed implementation from Wang et al. (2018).
A.4. LassoBench
LassoBench (Šchić al., 2022)* is challenge focused on optimizing the hyperparameters of Weighted LASSO (Least Absolute Shrinkage and Selection Operator) regression. The goal is to fine-tune of hyperparameters to achieve balance between least-squares estimation and the sparsity-inducing penalty term. LassoBench serves both synthetic (simple, medium, high, hard) and real-world tasks, including (Breast cancer, Diabetes, Leukemia, DNA, and RCVI). Specifically, we focused on the DNA task, which is 180-dimensional hyperparameter optimization task that utilizes a DNA dataset from a microbiological study. In Figure 4, we present the original results multiplied by -1 for improved visibility.
http:htt..oo....o/////aaaaaaaaaa -n zat on
12

---

Posterior Inference with Diffusion Models for Higghddimensional Black-box Optimization
B. Methodology Details
In this section, we provide a detailed overview the methodology, covering model implementations and architectures, training procedures, hyperparameter settings, and computational resources.
B.1. Training Models
B.1.1. TRAINING PROXY MODEL
We train five ensembles proxies. To implement the proxy function, we use MLP with three hidden layers, each consisting of 256 (512 for 400 dim tasks) hidden units and GELU (Hendrycks Gimpel, 2016) activations. We train proxy model using Adam (Kingma, 2015) optimizer for 50 (100 for 400 dim tasks) epochs per round, with learning rate 1 x 10-3, We set the batch size 256. The hyperparameters related to the proxy are listed in Table 1.
Table I. Hyperparameters for Training Proxy
Parameters
Values
Num Ensembles
Architecture
Number of Layers
3
Num Units
256 (Default) 512 (400D)
Batch size
256
Optimizer Learning Rate
Adam
Training
10-3
Training Epochs
50 (Default) / 100 (400D)
1.2. TRAINING DIFFUSION MODEL
We utilize the temporal Residual MLP architecture from Venkatraman et al. (2024) as the backbone of our diffusion model. The architecture consists of three hidden layers, each containing 512 hidden units. We implement GELU activations alongside layer normalization (Ba, 2016). During training, we use the Adam optimizer for 50 epochs (100 for 400 dim tasks) per round with a learning rate of x 10-3, We set batch size to 256. We employ linear variance scheduling and noise prediction networks with 30 diffusion steps for all tasks. The hyperparameters related the diffusion model summarized in Table 2.
Table 2. Hyperparameters for Training Diffusion Models
Parameters
Values
Number Layers
3
Architecture
Num Units
512
Batch size
256
Optimizer
Adam
Training
Learning Rate
1 x 10-3
Training Epochs
50 (Default) 100 (400D)
Diffusion Settings
Num Timesteps
30
13

---

Posterior Inference with Diffusion Models for High-dimensiooal Black-box Optimization
B.2. Sampling Candidates
B.2.1. FINE-TUNING DIFFUSION MODEL
We use relative trajectory balance (RTB) loss to fine-tune the diffusion model for obtaining an amortized sampler of the posterior distribution.
Zv (x((X)T) L(xoT: 16) = (log exp (3 r (xx)) "Po
(17)
As stated in the original work by Venkatraman et al. (2024), the gradient of this objective concerning s does not necessitate backpropagation into the sampling process that generates a trajectory Xp:r. Consequently, the loss can be optimized in an off-policy manner. Specifically, we can optimize Equation (17) with (1): on-policy trajectories Xo:T ~ Ps (x.:)) or (2): off-policy trajectories Xo:r generated by noising process given X sampled from the buffer.
To effectively fine-tune our diffusion model, we train Pe using both methods. For each iteration, we select a batch of on-policy trajectories with probability of 0.5 and off-policy trajectories otherwise. When sampling from the buffer, use reward-prie prioriiiii sampling to focus data points with high UCB scores. We conducted additional analysis off-policy training in Appendix D.3.
We initialize & +- 0 with each iteration, so the architecture and diffusion timestep is the same with Table 2. During training, we use Adam optimizer for 50 epochs (100 epochs for 400D) with learning rate x 10-4 We set the batch size to 256. The hyperparameters for fine-tuning the diffusion model are summarized Table 3.
Table 3. Hyperparameters for Finetuning Diffusion Models
Parameters
Values
Number of Layers
3
Architecture
Num Units
512
Batch size
256
Optimizer
Adam
Training
Learning Rate
1 X 10-4
Training Epochs
50 (Default) 100 (400D)
Diffusion Settings
Num Timesteps
30
All the training is done with a Single NVIDIA RTX 3090 GPU.
B.2.2. ESTIMATING MARGINAL LIKELIHIOD
During the local search and filtering in Section 3.2, we use the probability flow ordinary differential equation (PF ODE)
to estimate the marginal log- ligeliiel of the diffusion prior log (x)). We consider the diffusion forward process as the following stochastic differential equation (SDE):
dx = f(x, f)dt g())dw
(18)
and the corresponding reverse process is
dx
g(()dw
(19)
where w and w are forward and reverse Brownian motions, and and are drift coefficient and diffusion coefficient respectively. The quantity (x) denotes the marginal distribution of x at time t. As we do not have direct access to score Vx log P. (x), it should be modeled with network approximation sø(x, t) se Vx log pe (xx., while in our case implicitly modeled by noise prediction network co (x, t) (Kingma et al., 2021).
14

---

Posterior Inference with Diffusion Models for Higg-dimenniionll Black-box Optimization
There also exists deterministic PF ODE,
dx [(x, - (o)p(
V
(20)
which evolves the sample x through the same marginal distributions {x) as Equations (18) and (19), under suitable regularity conditions. (Song et al., 2021)
With the trained so (x. t), we can estimate log log px(x) by applying the instantaneous change-of-variables
formula (Chen al., 2018) the PF ODE:
log (xp) logpr(XT) + f" V Fo((()))
(21)
where
(xx()),))=(((x1 (()) (x),
(22)
However, directly computing the trace of fa is computationally expensive. Following Grathwohl et al. (2019); Song et al. (2021), we use the
trace estimator (Skilling, 1989; Hutchinson, 1989) to estimate the trace efficiently:
- (
(23)
where the is sampled from the Rademacher distribution.
We solve Equation (21) using a differentiable ODE solver torchdi ffeq (Chen, 2018) with 4th-order Runge-Kutta Kutta (RK4) integrator, accumulating the divergence term in the integral to approximate log (Xp)). Since the PF ODE deterministic, this entire simulation is fully differentiable, enabling gradient-based optimization with respect to the Xo, thereby supporting the local search stage.
B.2.3. HYPERPARAMETERS
For the upper confidence bound (UCB), fixed 1.0 that controls the exploration-exploottaie For the target posterior distribution, the inverse temperature parameter / controls the trade-off between the influence of exp(r,(x)) and (x().. When selecting querying candidates, we sample M = B x 102 candidates from Ps (x), perform local search for J steps, and retain B candidates for batched querying. After querying and adding candidates, we maintain our training dataset to contain high-scoring samples. We present the detailed hyperparameter settings in Table We also conduct several ablation studies explore the effect of each hyperparameter on the performance.
Table 4. Hyperparameters during sampling candidates
Inverse Temperature 3
Local Search Steps J
Buffer Size I
Synthetic 200D
105
10
1000 (Rastrigin) 500 (Others)
Synthetic 400D
105
15
1000 (Rastrigin) 500 (Others)
HalfCheetah 102D
10
10
300
RoverPlanning 100D
105
30
300
DNA 180D
105
50
300
15

---

Posterior Inference with Diffusion Models for Higghddimensional Black-box Optimization
C. Baseline Details
In this section, we provide details of the baseline implementation and hyperparameters used in our experiments.
TuRBO (Eriksson et al., 2019): We use the original code and keep all settings identical to those in the original paper. For all the algorithms utilizing TuRBO base algorithm (TuRBO, LA-MCTS, MCMC-BO), use TuRBO-1 (No parallel local models).
LA-MCTS (Wang et al., 2020): We use the original code 6 and keep all settings identical to those in the original paper.
MCMC-BO (Yi et al., 2024): We utilize the original code and adjust the standard deviation of the proposal distribution during the Metropolis- Hatstngs steps to replicate the results from the original paper. Due to memory constraints during the MCMC steps, we report a total 6,000 evaluations for the D 400 tasks.
CMA-BO (Ngo al., 2024): We the original code and keep all settings identical to those in the original paper.
CbAS (Brookes et al., 2019): We reimplement the original code 9 with PyTorch and keep all settings identical to those in the original paper.
MINs (Kumar & Levine, 2020): We reimplement the code from Trabucco et al. (2022) 10 with Py PyTorh and keep all settings identical those the original paper.
DDOM (Krishnamoorthy al., 2023): To ensure fair comparison, we reimplement the original code to work with diffusion models and add classifier free guidance for conditional generation. We use the same method-specifie hyperparameters following the original paper and tune the training epochs (200 as default, 400 for D 400 tasks) for each task to optimize performance.
Diff-BBO (Wu et al., 2024): As there no open-source code, we implement it according to the details provided in the original paper. As with DDOM, we use the original method-specific hyperparameters and tune the training epochs (200 as a default, 400 for D 400 tasks) for each task improve performance.
CMA-ES (Hansen, 2006): We existing library pycma (Hansen et al., 2019) and adjust the initial standard deviation as 0.1, which gives better performance on all tasks.
16

---

Posterior Inference with Diffusion Models for Higghddimensional Black-box Optimization
D. Extended Additional Analysis
In this section, we present additional analysis on DiBO that is not included in the main manuscript due to the page limit.
D.1. Effect of Training Epochs
The number of epochs for training models can be crucial in the performance black-hox optimization algorithms. If we use too small a number of epochs, the proxy may underfit, and the diffusion model may find it hard to capture the complex data distribution accurately. On the other hand, we use too large number of epochs, the proxy and the diffusion may overfit the dataset, and the overall procedure takes longer time for each round.
To this end, we conduct experiments Rastrigin-200D and HalfCheetah- by varying training epochs. As shown in the Figure 6, when we use large training epochs, the performance improves significantly the early stage but eventually converges to the sub-optimal results due to the overfitting of the proxy and diffusion model, which may hinder exploration towards promising regions.
Rastrigin-200D
HalfCheetah-1022
-1000
Best Found Veluc
Best Found Value
1500
2000
-2500
Epochts33
Epochs-50 (DXfaulit)
Epocha-50 (Default)
Epochs=100
-1000
Epochs=100
Epochs=150
Epochs=150
2009
4020
6930
6000
10000
-2000
250
500
750
1000
1250
1500
1750
2000
Number of Evaluations
Number of Evaluations
Figure Performance of DiBO in Rastrigin- 200D HalfCheetah- 102D by varying training epochs. Experi- ments are conducted with four random seeds. Mcan and one standard deviation are reported.
17

---

Posterior Inference with Diffusion Models for Hiighddimensional Black-box Optimization
D.2. Analysis Local Search Steps J
We conduct additional analysis on local search steps J. Through local search, can capture the modes of the target distribution, which leads high sample efficiency. However, using too large local search steps may focus exploiting single mode with the highest density of the target distribution, resulting in sub-optimal results.
To this end, we conduct experiments on Rastrigin- 200D and HalfCheetah-102D by varying J. As shown in the Figure 7, our method shows a relatively slow learning curve when we remove the local search. On the other hand, if we use too large J, struggles to escape from local optima and eventually results in sub-optimal results.
Rastrigin-200D
HalfCheetah-1022
-900
Best Found Value
Best Found Value
-1500
-200
-2500
w/o Local Search
J-30
-1000
W/ Local Search (Default)
30
-2500
(Refault)
J-50
2003
4023
5020
5000
12000
-2000
250
300
750
1000
1250
2500
1750
2005
Number of Evaluations
Number of Evaluations
Figure Performance of DiBO in Rastrigin-2001 and HalfCheetah- 102D by varying J. Experiments are conducted with four random seeds. Mean and one standard deviation are reported.
D.3. Effect of Off-poolicy Training in Amortized Inference
During the fine-tuning stage, we employ off-policy training with the RTB loss function, as detailed in Appendix B.2.1. To assess the impact of this approach, we conduct a comparative experiment using only on-policy training.
As illustrated in Figure 8, off off-policy training demonstrates a significant performance advantage over on-policy training. In on-policy training, the model is restricted to learning only from the generated samples. Consequently, crucial data points, particularly those associated with significant events, are rarely encountered during training. In contrast, off- off-policy training effectively captures these critical regions by directly leveraging information from replay buffer, enabling the model to learn from informative data points efficiently.
Rastrigin-
HalfCheetah-1o0
3000
Best Found Value
Best Found Value
-1500
2000
-2000
-200
Dn-Policy
On-Polley
Palicy (Default)
Policy (Default)
2003
4020
5020
5000
10000
-2000
250
500
750
1000
1250
1500
1758
2000
Number of Evaluations
Number Evaluations
Figure Performance DiBO in Rastrigin- 200D and HalfCheetah- 102D and without off off-policy training. Experiments are conducted with four random seeds. Mean and one standard deviation are reported.
18

---

Posterior Inference with Diffusion Models for High-dimenniionl Black-box Optimization
D.4. Analysis on Initial Dataset Size (Pol
The size of the initial dataset [Do] can be crucial in the performance of black-box optimization algorithms. If the initial dataset is too small and concentrates a small region compared to the whole search space, it is hard to explore diverse promising regions without proper exploration strategies.
To this end, we conduct experiments by varying [Pol on the synthetic tasks. As shown in the Figure 9, our method demonstrates robustness regarding the size of the initial dataset. It indicates that our exploration strategy, proposing candidates by sampling from the posterior distribution, is powerful for solving practical high-dimensional black-box optimization problems.
Rastrigin-2000
Ackley-200D
Levy-200D
Rosenbrock-200D
Best Found Volue
Dast Fourdd Vollar
Best Found Value
-000
Best ( (Defaul)) Dyj-500
10,-100
[Da=1000
200 (Defautt
200 (Defaut)
[0 200 (Detsut)
|Py|-500
[Dy=500
10, -500
[00|-0000
Number Exaluaaionns
Nambar of Evalcations
Number Evolattions
Number Evalcations
Figure 9. Performance DiBO in synthetic functions by varying (Pol. Experiments are conducted with four random seeds. Mean and one standard deviation are reported.
D.5. Analysis on Batch Size B
The batch size B can be crucial in the performance of black box optimization algorithms. As the number of evaluations is mostly limited, if we use too large B, it is hard to focus on high-scoring regions. On the other hand, if we use too small B, it hinders exploration, and it is hard to escape from local optima.
To this end, we conduct experiments by varying B on the synthetic tasks. Note that we fix the batch size for all main experiments as B 100. We visualize the experiment results in Figure 10. We can observe that our method shows robust performance across different B while using large batch size leads to slightly slow convergence compared to using a small batch size. Using smaller batch size shows better sample efficiency but also leads to an increase in computational time.
Rastrigin-2000
Ackley-200D
Levy
Rasenbrack-200
+1000
Dest Found Value
Best Found Valuee
Beed Found
-000
M-20
Bm20
B0
Bi:50
B=50
B-50
8-50
1=100 (Dwfault)
10100 (Delaut)
1=100 (Delauk)
di 10 (Dwlaut)
B-200
B-200
8-200
8-200
120
Number Evalustions
Number
200
Number Evaluntions
400
200
Number Evalustionn
Figure 10. Performance of DiBO in synthetic functions by varying B. Experiments are conducted with four random seeds. Mean and one standard deviation are reported.
19

---

Posterior Inference with Diffusion Models for Higg-dimmnnsionll Black-box Optimization
D.6. Analysis Uncertainty Estimation
To promote exploration, we estimate uncertainty with an ensemble of proxies and adopt an upper confidence bound (UCB) to define the target distribution. Specifically, we use: rx(x) M4 (x) + d (x), where y controls the degree of uncertainty bonus. We evaluated two aspects of this approach in the HallChectah- task.
To analyze the effectiveness of the ensemble strategy for uncertainty estimation, besides our ensemble method, we test Monte Carlo (MC) dropout (Gal & Ghahramani, 2016) and a setup without uncertainty estimation (one proxy). As shown in Figure Ila, the ensemble strategy effectively estimates the uncertainty and improves sample efficiency compared to others.
To analyze if UCB with uncertainty bonus promotes exploration, we conduct experiments by varying the parameter y and analyzing its impact performance. Figure 11b demonstrate that increasing 7 leads more extensive search space exploration. However, excessively large values 10.0) dilute the focus exploitation, slowing convergence.
HalfCheetah- 102D
HalfCheetah-1022
3000
3000
Best Fourd Value
Best Found Value
2003
2000
1000
1000
Ensemble (Default)
MC Dropout
y=0.0
5=5.0
None
1-1.0 (Defaurt)
y-10.0
Number Evaluations
Number Evaluations
(a) Analysis on uncertainty estimation methods
(b) Analysis on
Figure 11. Performance of DiBO in HalfCheetah- -02DD with varying uncertainty estimation methods and gamma Experiments conducted with four random seeds. Mean and one standard deviation reported.
These findings demonstrate that an ensemble of proxies effectively captures uncertainty even in high-dimensional spaces. Moreover, designing a target distribution that incorporates UCB helps balance exploration and exploitation, improving sample efficiency during optimization.
20

---

Posterior Inference with Diffusion Models for Hiighddimensional Black-box Optimization
D.7. Time Complexity of method
We report the average running time per each round in Table 5. All training is done with single NVIDIA RTX 3090 GPU and Intel Xeon Platinum CPU @ 2.90GHZ. As shown in the table, the running time of method is similar to generative model-based approaches and mostly faster than BO methods. It demonstrates the efficacy of our proposed method.
Table 5. Average time (in seconds) for each round in each method.
Rastrigin-200D
Rastrigin-400D
Ackley-200D
Ackley-400D
Levy-200D
Levy-400D
Roscnbrock-200D
Rosenbrock-400D
TuRBO
244.39 0.21
1089.09 a 0.04
33.91 a: 0.24
41.50 a 0.06
167.58 0.21
59,70 a 0.02
452.21 4: 0.15
45.05 =: 0.01
LA-MCTS
222.95 = 6.59
256.82 = 8.89
150.27 = 3.97
184.14 + 1.67
90.47 = 1.69
229.80 11.99
154.56 + 4.08
223.59 = 7.52
MCMC-BO
341.98 3.54
429.02 + 4.61
370.03 + 4.17
345.60 3.42
337.87 + 3.65
429.02 4.61
419.07 + 5.12
448.56 5.13
CMA-BO
4.00
833.97 + 2.12
661.15
854,23 2.77
694.67 =
871.33 3.66
645.65 +
857.39 2.53
CbAS
2.226
213.64
207.35 4.01
213.67
6.62
212.61 = 4.05
212.69 9.83
203.87 #
221.68 5.01
28.36
32.20
28.83
0.17
0.37
+ 23.19 0.12
DDOM
0:28
26.57 0.11
23.53 0.04
26.62 0,16
0.15
26,46 0.15
26.55 0.15
Dif' BBO
128.75 0.89
143.80 1.16
131.16 0.86
143.96 1.68
130.07 0.74
143.97 1.79
128.33 a: 1.74
144.03 1.58
CMA-ES
0.00
0.05 0.00
0.03 0.00
0.04 0.00
0.04 0.00
0.05 0.00
0.03 0.00
0.04 0.00
DIBO
0.26
79.63 1.07
71.99 0.18
39.55
72.21 0.55
39.57 0.28
72.88
D.8. Analysis more computation time
In this section, present supplementary results demonstrating that more computing time significantly improve performance beyond the results reported in the main text. For example, on the Ackley-200D benchmark, with the local search steps J = 50 and buffer size, L 2000 improves performance from the default configuration value of -0.6443 to score of 0.2660. While increasing the number of local search steps has possibility of converging to the local optimum, a large buffer size complements this issue. However, using larger // and leads to an increase in computational complexity. Exploring methods that reduce the complexity of training while maintaining large buffer sizes and local search steps may be promising research direction. We leave future work.
Ackley-200D
1 & 3DU (Uefault)
5- & 2000
2000
6866 Number of Evaluations
4093
800
1000
Figure 12. Performance of DiBO in Ackley-200D with local search epochs J 50, and buffer size L = 2000. Experiments are conducted with random seeds. Mean and one standard deviation are reported.
21


Reviews:
[{"review_id": "MsqD7gUuTV", "reviewer": "Reviewer_rUT1", "summary": "This paper utilizes the diffusion model for high-dimensional black-box optimization. At each iteration, they sample the candidates from the posterior distribution. The empirical results show that the proposed method outperforms other baselines.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your concrete review. Below we answer the questions and concerns you raised.\n\n> **Claims and Evidence)** There is no measurement regarding the uncertainty of the generative model or discussion about the relationships between two terms of uncertainty.\n\nWhile utilizing the uncertainty of the diffusion model can be an interesting future work, measuring the uncertainty of the diffusion models is mostly complex [1]. We conducted a brief literature search on this topic [2, 3], but most of them focus on detecting poor-quality images, which lies outside of our research scope. \n\n[1] Wu, Dongxia, et al. \"Diff-BBO: Diffusion-Based Inverse Modeling for Black-Box Optimization.\"\n\n[2] Kou, Siqi, et al. \"Bayesdiff: Estimating pixel-wise uncertainty in diffusion via bayesian inference.\"\n\n[3] Jazbec, Metod, et al. \"Generative Uncertainty in Diffusion Models.\"\n\n> **Methods and Evaluation Criteria)** It is helpful to verify that by including the black-box optimization benchmarks with structured data in the design space, such as molecular optimization tasks. \n\nOur method can be directly applied to benchmarks with structured data, such as molecular optimization tasks. To this end, we conducted additional experiments on benchmarks with structured inputs. We follow the standard  evaluation pipeline with [4]. \n\nAs shown in the table, our method achieves both higher performance and better sample efficiency compared to recent BO baselines for structured inputs. We promise to add these results in our final manuscript.\n\n\n**Experiment results on structured inputs. Experiments are conducted with 4 random seeds.**\n| Tasks|# Evaluation Budget |LOL-BO [4]|CoBO [5]|DiBO|\n|:-|-|:-|:-|:-:|\n|Zaleplon MPO|20000|0.711 ± 0.014|0.724 ± 0.004|0.739 ± 0.034|\n||30000|0.723 ± 0.006|0.728 ± 0.002|0.771 ± 0.002|\n||40000|0.739 ± 0.000|0.738 ± 0.002|0.771 ± 0.002|\n|Perindopril MPO | 20000|0.734 ± 0.000|0.715 ± 0.025|0.815 ± 0.004|\n||30000|0.771 ± 0.014|0.788 ± 0.024|0.818 ± 0.006|\n||40000|0.798 ± 0.021|0.796 ± 0.018|0.825 ± 0.009|\n\n[4] Maus, Natalie, et al. \"Local latent space bayesian optimization over structured inputs.\"\n\n[5] Lee, Seunghun, et al. \"Advancing bayesian optimization via learning correlated latent space.\"\n\n> **Theoretical Claims)** There is no theoretical guarantee provided for the sampling approach.\n\nWhile we acknowledge that the theoretical guarantee of our algorithm could further enhance its reliability, the guarantee of finding optimal solutions using deep learning models is almost impossible. Our paper makes a methodological and empirical contribution to solving high-dimensional black-box optimization problems effectively. We believe that our method represents a new departure for solving high-dimensional black-box optimization by importing ideas from diffusion models and amortized posterior inference.\n\n>**Experimental Designs or Analyses)** My concern is that many baselines for high-dimensional black-box optimization are missing. For the ablation study in the appendix, it will be helpful if the authors can include other baselines in the analysis of batch size and initial dataset size.\n\nWe apologize for missing some crucial baselines in high-dimensional BO. However, we would like to emphasize that we present 4 strong BO-based baselines (We also already include LA-MCTS, which you mentioned in [5]). In particular, MCMC-BO and CMA-BO, which published last year, outperform most of the baselines listed above in various benchmarks.\n\nNevertheless, we conducted experiments with additional baselines, logEI and MCTS-VS. \nAs shown in the table, we outperform those baselines in terms of performance. We promise to conduct experiments on all benchmarks and update the results in our final manuscript.\n\n**Experiment results of DiBO and additional baselines. Experiments are conducted with four random seeds.**\n|| TuRBO (LogEI) | MCTS-VS-TuRBO  | DiBO (Ours) |\n|-|-|:-|:-|\n|Rastrigin|-584.09|-1089.62|-560.364|\n|HalfCheetah|-511.99|-223.175|3378.353|\n\n\nRegarding ablation studies, we also include other baselines in the analysis of batch size and initial dataset size. As shown in the table, even with different experiment settings, our method consistently outperforms other baselines by a large margin. Furthermore, as depicted in Figure 9 in the Appendix, our method does not degrade in terms of performance even with a small initial dataset size. \n\n**Ablation studies on other baselines in terms of initial experiment settings. Experiments are conducted with four random seeds.**\n||Batch size|TuRBO|Diff-BBO|DiBO (Ours)|\n|:-|:-|:-|-|:-|\n|Rastrigin|20|-797.520|-1728.317|-573.528|\n||50| -812.958|-1702.763|-545.124|\n||100(Default)|-950.376|-1730.651|-560.364|\n\n||Initial Dataset size|TuRBO|Diff-BBO|DiBO (Ours)|\n|:-|:-|-|-|:-|\n|Rastrigin|10|-1012.730|-1745.659|-586.761|\n||50|-952.407|-1700.911|-629.776|\n||200 (Default)|-950.376|-1730.651 |-560.364|", "claims_and_evidence": "The authors claim that by sampling candidates from the posterior distribution, the proposed method can effectively balance exploration and exploitation. However, they only measure the uncertainty of a portion of their model, which is from the ensemble of proxies. There is no measurement regarding the uncertainty of the generative model or discussion about the relationships between two terms of uncertainty. It remains unclear why this sampling approach can effectively balance exploration and exploitation. And there is no theoretical guarantee provided.", "methods_and_evaluation_criteria": "Both the synthetic and real-world benchmark datasets have been evaluated. The authors follow the standard problem setting of high-dimensional bayesian optimization. Since the authors claim their method can effectively capture complex and multi-modal data distribution. It is helpful to verify that by including the black-box optimization benchmarks with structured data in the design space, such as molecular optimization tasks [1].\n\n[1] Maus, Natalie, et al. \"Local latent space bayesian optimization over structured inputs.\"", "theoretical_claims": "There is no theoretical guarantee provided for the sampling approach. It would be helpful if the authors can include some theoretical analysis of their proposed algorithm. For example, can it be proven that the proposed approach guarantees an optimal or near-optimal solution for black-box optimization under certain assumptions?", "experimental_designs_or_analyses": "My concern is that many baselines for high-dimensional black-box optimization are missing. I conducted a brief literature search and listed some of them [1-9]. DDOM is not an appropriate baseline as it is designed for offline optimization. For the ablation study in the appendix, it will be helpful if the authors can include other baselines in the analysis of batch size and initial dataset size. I assume the performance of DiBO will degrade when there is insufficient data for the diffusion model to learn or update the data distribution.\n\n[1] Ament, Sebastian, et al. \"Unexpected improvements to expected improvement for bayesian optimization.\"\n[2] Eriksson, David, and Martin Jankowiak. \"High-dimensional Bayesian optimization with sparse axis-aligned subspaces.\"\n[3] Nayebi, Amin, Alexander Munteanu, and Matthias Poloczek. \"A framework for Bayesian optimization in embedded subspaces.\" \n[4] Wang, Zi, et al. \"Batched large-scale Bayesian optimization in high-dimensional spaces.\"\n[5] Wang, Linnan, Rodrigo Fonseca, and Yuandong Tian. \"Learning search space partition for black-box optimization using monte carlo tree search.\"\n[6] Letham, Ben, et al. \"Re-examining linear embeddings for high-dimensional Bayesian optimization.\" \n[7] Song, Lei, et al. \"Monte carlo tree search based variable selection for high dimensional bayesian optimization.\"\n[8] Ziomek, Juliusz Krzysztof, and Haitham Bou Ammar. \"Are random decompositions all we need in high dimensional Bayesian optimisation?.\"\n[9] Nguyen, Quan, et al. \"Local Bayesian optimization via maximizing probability of descent.\"", "relation_to_broader_scientific_literature": "This paper extends the diffusion model from offline to online black-box optimization, which has been studied in Diff-BBO [1]. The posterior sampling approaches of the proposed algorithm appears similar to the ones used in offline optimization with diffusion models [2,3].\n\n[1] Wu, Dongxia, et al. \"Diff-BBO: Diffusion-Based Inverse Modeling for Black-Box Optimization.\"\n[2] Yu, Peiyu, et al. \"Latent energy-based odyssey: Black-box optimization via expanded exploration in the energy-based latent space.\"\n[3] Kong, Lingkai, et al. \"Diffusion models as constrained samplers for optimization with unknown constraints.\"", "essential_references_not_discussed": "Please refer to the previous sections to add the references.", "other_strengths_and_weaknesses": "Please refer to the previous sections.", "other_comments_or_suggestions": "Please refer to the previous sections.", "questions_for_authors": "Please refer to the previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fXPimvfyRF", "reviewer": "Reviewer_jXRP", "summary": "This paper proposes a novel high-dimensional black-box optimization method, where the authors train a diffusion model based on the weighted data as the prior and performs posterior sampling when combined with a uncertainty-aware function proxy. The authors also use local search and filtering strategies to further refine the posterior samples. Over extensive benchmarks the proposed DiBO demonstrates improved optimization performances compared to representative high-dimensional optimization methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comment and for considering our key idea, incorporating the diffusion model as prior and casting sampling as posterior inference for solving high-dimensional black-box optimization, as novel. We answer your questions below.\n\n>**Other Comments Or Suggestions)** Given the expressive power of diffusion model, I think some tasks including structured input space may further enhance the paper (e.g. chemical/protein design).\n \nAs you mentioned, tasks including structured input space further enhance the usefulness of our method. To this end, we conduct experiments on molecular optimization following [1]. As shown in the table, our method achieves not only higher performance but also high sample efficiency compared to recent BO baselines for structured inputs. We promise to add these results in our final manuscript.\n\n\n**Experiment results on structured inputs. Experiments are conducted with four random seeds.**\n| Tasks|# Evaluation Budget |LOL-BO [1]|CoBO [2]|DiBO|\n|:-|-|:-|:-|:-:|\n|Zaleplon MPO|20000|0.711 ± 0.014|0.724 ± 0.004|0.739 ± 0.034|\n||30000|0.723 ± 0.006|0.728 ± 0.002|0.771 ± 0.002|\n||40000|0.739 ± 0.000|0.738 ± 0.002|0.771 ± 0.002|\n|Perindopril MPO | 20000|0.734 ± 0.000|0.715 ± 0.025|0.815 ± 0.004|\n||30000|0.771 ± 0.014|0.788 ± 0.024|0.818 ± 0.006|\n||40000|0.798 ± 0.021|0.796 ± 0.018|0.825 ± 0.009|\n\n[1] Maus, Natalie, et al. \"Local latent space bayesian optimization over structured inputs.\"\n\n[2] Lee, Seunghun, et al. \"Advancing bayesian optimization via learning correlated latent space.\"\n\n> **Questions For Authors)** How to set the number of ensembles to guarantee the uncertainty quantification is reasonable?\n\nThank you for your interest in our work. The number of ensembles is crucial to reasonably quantify the uncertainty of the surrogate model. To this end, we conduct experiments by varying number of ensembles, $K$. As shown in the table, there is no big difference in performance when we increase $K$ more than $K=5$. However, without uncertainty quantification or too small number of ensembles leads to poor performance, which indicates that uncertainty quantification is crucial for high-dimensional black-box optimization problems.\n\n**Ablation studies on the number of ensembles ($K$). Experiments are conducted with four random seeds.**\n\n|             | $K$         | DiBO (Ours) |\n|:----------- |:----------- |:----------- |\n| HalfCheetah | 1 (None)    | 2750.765    |\n|             | 3           | 2604.994    |\n|             | 5 (Default) | 3191.215    |\n|             | 7           | 3131.849    |\n|             | 9           | 2926.619    |", "claims_and_evidence": "The main contributions that the authors claim:\n\n1. The proposed diffusion-based algorithm to address the scalability and efficiency in high-dimensional optimization.\n\n2. Superior performances over a variety of tasks compared to state-of-the art baselines.\n\nI think the proposed method and the experimental results support the claimed contributions.", "methods_and_evaluation_criteria": "I think the method makes sense and the used benchmarks are representative in high-dimensional black-box optimization.", "theoretical_claims": "Not applicable.", "experimental_designs_or_analyses": "I think the experiment setting and ablation studies are comprehensive.", "relation_to_broader_scientific_literature": "I think the idea of incorporating diffusion model for input prior learning and casting the sampling as posterior inference is novel and a suitable usage of diffusion model to address high-dimensional issues.", "essential_references_not_discussed": "I think essential references are discussed.", "other_strengths_and_weaknesses": "The paper is clear and well-written. And I don't have major concerns in terms of the weakness.", "other_comments_or_suggestions": "I think it is a good work which well applies diffusion models for high-dimensional black-box optimization.\n\nGiven the expressive power of diffusion model, I think some tasks including structured input space may further enhance the paper (e.g. chemical/protein design).", "questions_for_authors": "How to set the number of ensembles to guarantee the uncertainty quantification is reasonable?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0om6XLqow9", "reviewer": "Reviewer_DZqC", "summary": "## Summary\n* The authors propose a two-stage approach for black box optimization using diffusion model. The first stage is training stage. The authors propose to train a weighted unconditional model for density estimation, and an ensemble of proxy models to capture the value and uncertainty of target. This diffusion model + discriminative approach is commonly adopted, e.g. in classifier based guidance.\n* Next, the authors propose to findtune the model (amortized inference) using a relative trajectory balance. The target is designed to balance exploration and exploitation. To further enhance the performance, the authors adopts two post processing technique: local search and filtering. The local search is a gradient ascent on finetuning target, and the filtering is a selection of candidates.\n* Empirical results on multiple dataset show the effectiveness of their approach.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive assessment of our paper's extensive experiment results. We've attempted to answer your questions below.\n\n>**Claims And Evidence)** Therefore, whether simple reweighted training achieve weighted likelihood such as Eq. 11 remains questionable.\n\nThank you for pointing out the question regarding Eq.11. As you mentioned, we try to maximize the ELBO instead of the marginal likelihood. We will fix the Eq.11 in the final manuscript.\n\n> **Other Strengths And Weaknesses)** One part that I do not like about this paper is that it contains to many sub-parts and tricks. The paper is driven by a clear performance target, but there is no clear technical clue that lead this paper.\n\nThank you for your constructive feedback. While there are several sub-parts in our method, please note that we systemically analyze the effect of each component through extensive ablation studies to verify that each component is crucial for improving performance. \n\nFurthermore, please note that several ideas we imported in this paper are already considered as a reasonable choice for training diffusion models as an amortized sampler effectively. For example, off-policy training is suggested in various GFlowNets literature [1, 2, 3].\n\n[1] Venkatraman, Siddarth, et al. \"Amortizing intractable inference in diffusion models for vision, language, and control.\"\n\n[2] Akhound-Sadegh, Tara, et al. \"Iterated denoising energy matching for sampling from boltzmann densities.\"\n\n[3] Rector-Brooks, Jarrid, et al. \"Steering masked discrete diffusion models via discrete denoising posterior prediction.\"\n\nThank you again for your comments. We hope we have addressed them satisfactorily above, but do not hesitate to let us know if you have further questions. We are always ready to engage in further discussion!", "claims_and_evidence": "## Claims And Evidence\n* The reweighting scheme is interesting. However, the score matching loss of diffusion model is not a direct max likelihood target (See [Maximum Likelihood Training of Score-Based Diffusion Models]) Therefore, whether simple reweighted training achieve weighted likelihood such as Eq. 11 remains questionable.", "methods_and_evaluation_criteria": "## Methods And Evaluation Criteria\n* The benchmark datasets look standard for this field and the ablation studys are sufficient to support the claims.", "theoretical_claims": "## Theoretical Claims\n* There are no theoretical claims.", "experimental_designs_or_analyses": "## Experimental Designs Or Analyses\n* The experimental result and analysis are sufficient. Abundant results show the effectiveness of the proposed method, in terms of performance and complexity. The effectiveness of different component proposed by the authors is also verified.", "relation_to_broader_scientific_literature": "## Relation To Broader Scientific Literature\n* The proposed approach will likely to be a strong baseline in black box optimization.", "essential_references_not_discussed": "## Essential References Not Discussed\n* The reference to prior works is sufficient.", "other_strengths_and_weaknesses": "* One part that I like about this paper is its empirical evaluation. Through benchmark and real world problems, the authors successfully show the advantage of their approach over prior works.\n* One part that I do not like about this paper is that it contains to many sub-parts and tricks. The paper is driven by a clear performance target, but there is no clear technical clue that lead this paper.", "other_comments_or_suggestions": "-", "questions_for_authors": "-", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work considers a two-stage approach for black box optimization using diffusion model. The proposed method is novel and the experiments support the claims that it outperforms baselines such as BO. While no theoretical results are part of this work and better understanding the type of exploration exploitation trade-off the method performs would be of general interest (and strengthen the paper), the contributions and results are valuable to the community. The authors also provided additional convincing evidence supporting the results and claims, resolving most of the concerns raised by the reviewers.
Decision: Accept (poster)