Robust Guided Diffusion for Offline Black-Box Optimization
Authors: Can Chen
Christopher Beckham
Zixuan Liu
Xue Liu
Christopher Pal
Venue: Accepted by TMLR ()

Abstract: Offline black-box optimization aims to maximize a black-box function using an offline dataset of designs and their measured properties. Two main approaches have emerged: the forward approach, which learns a mapping from input to its value, thereby acting as a proxy to guide optimization, and the inverse approach, which learns a mapping from value to input for conditional generation.  (a) Although proxy-free~(classifier-free) diffusion shows promise in robustly modeling the inverse mapping, it lacks explicit guidance from proxies, essential for generating high-performance samples beyond the training distribution.  Therefore, we propose \textit{proxy-enhanced sampling} which utilizes the explicit guidance from a trained proxy to bolster proxy-free diffusion with enhanced sampling control. (b) Yet, the trained proxy is susceptible to out-of-distribution issues. To address this, we devise the module \textit{diffusion-based proxy refinement}, which seamlessly integrates insights from proxy-free diffusion back into the proxy for refinement. To sum up, we propose \textit{\textbf{R}obust \textbf{G}uided \textbf{D}iffusion for Offline Black-box Optimization}~(\textbf{RGD}), combining the advantages of proxy~(explicit guidance) and proxy-free diffusion~(robustness) for effective conditional generation. RGD achieves state-of-the-art results on various design-bench tasks, underscoring its efficacy. Our code is \href{https://github.com/GGchen1997/RGD}{here}.

Published in Transactions on Machine Learning Research (12/2024)
Robust Guided Diffusion for Offline Black-Box Optimization
Can (Sam) Chent Christopher Beckham Zixuan Liu", Xue Liuh?2, Christopher Pal2,3,4
'MeGill University, 2MILA Quebec AI Institute, $Polytechnique Montreal, *Canada CIFAR AI Chair, "University of Washington
Reviewed on OpenRevicw:
Abstract
Offline black-box oplimization aims to maximize a black-box function using an oflline dataset of designs and their measured properties. Two main approaches have emerged: the forward approach, which learns a mapping from input to its value, thereby acting as a proxy to guide optimization, and the inverse approach, which learns a mapping from value to input for conditional generation. (a) Although proxy-free (classiffer-free) diffusion shows promise in robustly modeling the inverse mapping, it lacks explicit guidance from proxies, essential for generating high-performance samples beyond the training distribution. Therefore, we propose prozy-enhenced sompling which utilizes the explicit guidance from trained proxy to bolster proxy-free diffusion with enhanced sampling control. (b) Yet, the trained proxy is susceptible to out-of-distribution issues. To address this, we devise the module diffusion--unn prozy refinement, which seamlessly integrates insights from proxy- free diffusion back into the proxy for refinement. To sum up, we propose Robust Guided Diffusion for Offline Black-bor box Optimization (RGD), combining the advantages of proxy (explicit guidance) and proxy-free dilfusion (robustness) for effective conditional generation. RGD achieves state-of-the-art results on various design-benchl tasks, underscoring its efficacy. Our code is here
1 Introduction
Creating new objects to optimize specific properties is a ubiquitous challenge that spans a multitude of fields, including material science, robotic design, and genetic engineering. Traditional methods generally require interaction with a black-box function to generate new designs, process that could be financially burdensome and potentially perilous (Hamidich) 2018 Sarkisyan et al. 2016). Addressing this, recent research endeavors have pivoted loward a more relevant and practical context, termed offline black-box optimization (BBO) (Trabucco al. 2022 Krishnamoorthy et al., 2023). In this context, the goal is to maximize a Jack-box function exclusively utilizing an offline dataset of designs and their measured properties.
There are two main approaches for this task: the forward approach and the reverse approach. The forward approach entails training deep neural network (DNN), parameterized as J&(+), using the offline datasct. Once trained, the DNN acts as a proxy and provides explicit gradient guidance to enhance existing designs. However, this technique is susceptible to the out-of-die of-iiiriiitttioonn (OOD) issue, leading to potential overestimation of unscen designs and resulting adversarial solutions (Trbbucco et al. 2021
The reverse approach aims to learn mapping from property value to input. Inputting high value into this mapping directly yields high-performance design. For example, MINs (Kumar & Levine 2020 adopts
GAN [Goodfellow et al.] 2014 to model this inverse mapping, and demonstrate some success. Recent
works Krishnamoorthy et al. 2023) have applied proxy-free diffuusoo!] (Ho & Salimans (022). parameterized
by 0, to model this mapping, which proves its elficacy over oth generative models. Proxy-free diffusion
chencan421 www.maailloomccccceeeeeeee Classifer--fve diffusion classification and adapted to prozy--rres free diffusioo to generalize regression.

---

Published in Transactions on Machine Learning Research (12/2024)
employs score predictor so(.,,ww.. This represents linear combination of conditional and unconditional scores, modulated by strength parameter w to balance condition and diversity in the sampling process. This guidance significantly diverges from proxy (classifier) diffusion that interprets scores as classifier gradients and thus generates adversarial solutions. Such a distinction grants proxy-free diffusion its inherent robustness in generating samples.
Nevertheless, the inverse approach, proxy-free diffu- sion, initially designed for in-distribution generation, such synthesizing specific image categories, faces limitations in offline BBO. Particularly, it struggles to generate high--performacce samples that exceed the training distribution due to the lack explicit guidanod Consider, for example, the optimization of two-dimensional variable (Id1, Td22) to maxi- mize the negative Rosenbrock function (Rosenbrock 19600: (( x222 (11 za1)² 100 = as depicted in Figure 1 The objective is to steer the initial points (indicated in pink) towards the high- performance region (highlighted in yellow). While proxy-free diffusion can nudge the initial points closer to this high- performance region, the generated points (depicted in blue) fail to reach the high-perforee perforrrfrmmeec region due to its lack of explicit proxy guidance.
Inicial points
Proxy-free diffusion
Proxy-erhanced sampling
Negative Rasenbrack Function
20
15-
1200
1800
0.0
-05-
2400
-10
3000
-15
3600
-20
220
-1.5
-1.0
-0.5
0.0
05
1.0
1.5
29
Xd,
Figure 1: Motivation of explicit proxy guidance.
To address this challenge, we introduce a prozy-enhanced sampling module as illustrated in Figure 2 aa). It incorporates the explicit guidance from the proxy Jo (x) into proxy-free diffusion to enable enhanced control over the sampling process. This modulc hinges on the strategic optimization of the strength parameter w to achieve a better balance between condition and diversity, per reverse diff'usion step. This incorporation not only preserves the inherent robustness of proxy-free diffusion but also leverages the explicit proxy guidance, thereby enhancing the overall conditional generation efficacy. As illustrated in Figure 1 samples (depicted in red) generated via prory-enhanced sampling more effectively guided towards, and often reach, the high-performance area (in yellow).
Yet, the trained proxy is susceptible to out-of-distribution (OOD) issues. To address this, we devise a module diffusion-bused procy refinement as detailed in Figure 2b). This module seamlessly integrates insights from proxy-free diffusion into the proxy Jx(x) for refinement. Specif- ically, we generate diffusion distribution pe (yllt) on adversarial samples &,, using the associated probability flow ODE This distribution is derived independently of a proxy, thereby exhibiting greater robustness than the proxy distribution on adversarial samples. Subsequently, we calculate the Kullback-Leible divergence between the two distributions on adversarial samples, and use this divergence minimization fortify the proxy's robustness and reliability.
(a)
Reverse diffusion
Final design
selen
Rd
FT
PT-1
T1
3o
Proxy-enhanced sampling (optimizing w)
(b)
Proxy Ja(in)
Diffusion-based
Diffusion distribution
proxy refinement
poly
Probabilty fow CDE
4T
frr
f
regularization strategy to
Forward diffusion
To sum up, we propose Robust Guided Diffusion for Of- fline Black-boz Optimization (RGD), a novel framework that combines the advantages of proxy (explicit guidance) and proxy-free diffusion (robustness) for effective condi- tional generation. Our contributions throe-fold:
Figure 2: Overview of RGD: Module (a) incor- porates proxy guidance into proxy-free diffusion to enable enhanced sampling control; Module (b) integrates insights from proxy-free diffusion back into the proxy for refinement.
2 free diffusion cannot interpreted as proxy and thus does not provide explicit guidance Ho & Salimans 2022 Ordinary Differential Equation
2

---

Published in Transactions on Machine Learning Research (12/2024)
We propose prozy-enhanced sampling module which incorporates proxy guidance into proxy-free diffusion to enable enhanced sampling control.
We further develop diffusion-based prozy refinement which integrates insights from proxy-frec diffusion back into the proxy for refinement.
RGD delivers state-of-the-ar performance on various design-bench tasks, emphasizing its efficacy.
2
Preliminaries
We provide the key notations used in this paper in Appendix
2.1 Offline Black-box Optimization
Offline black-box optimization (BBO) aims to maximize a black-box function with an offline dataset. Imagine a design space as X' = Rd, where d is the design dimension. The offline BBO (Trabucco et al. 2022 is:
x arg max ((x). DEX
(1)
In this equation, J(.) is the unknown objective function, and æ E X is possible design. In this context, there is an offline dataset, D, that consists of pairs of designs and their measured properties. Specifically, each a denotes a particular design, like the size of robot, while indicates its related metric, such as its speed.
A forward approach gradient ascent fits a proxy distribution po(y)æ) = ((J)())) to the offline dataset where & denote the proxy parameters:
arg min log
log
(x)) + (y J 20 (x)
(2)
arg min E EDwDD log( (
For the sake of consistency with terminology used in the forthcoming subsection on guided diffussin, we will refer to (+)-)) the proxy distribution and J+(-) the proxy. Subsequently, this approach performs gradient ascent with Jæ) leading to high-performance designs x":
Iy+1 xz + 1) V
for e 10 1].
converging to æM after M steps. However, this method suffers from the dittof-distribution issue where the proxy predicts values that are notably higher than the actual values.
2.2 Diffusion Models
Diffusion models, type of latent variable models, progressively introduce Gaussian noisc to data in the forward process, while the reverse process aims to iteralively remove this noise through a learned score estimator Ho et al. (2020). In this work, we utilize continuous time diffusion models governed by stochastic differential equation (SDE), as presented in Song et al. (2021). The forward SDE formulated as:
dx = f(z, t)dt g(t)dw.
(4)
where f(.,t): + Rd represents the drift coefficient, g(.) : R + denotes the diffusion coefficient and w is the standard Wiener process. This SDE transforms data distribution into noise distribution. The reverse is:
dx = (f( - v log lox) dt + g(t)du
with V log) representing the score of the marginal distribution at time t, and symbolizing the reverse Wiener process. The score function V. logp(x) estimated using a time-dependent neural network so(2,, (x,,t), enabling us to transform noise into samples. For simplicity, will s0(x;), implicitly including t.
3

---

Published in Transactions on Machine Learning Research (12/2024)
2.3 Guided Diffusion
Unconditional diffusion modcls capture the natural data distribution, while guided diffusion soeks to produce samples with specific desirable attributes, falling into two calegories: procy diffusson (Dhariwal & Nichol 2021 and proxy-free diffusion (Ho & Salimans 2022). While these were initially termed classifier diffusion and classifier-free diffusion in classification tasks, we have renamed them to prozy diffusion and prozy-free diffuusion, respectively, to generalize to our regression context. Proxy diffusion combines the model's score estimate with the gradient from the proxy distribution, providing explicit guidance in line with the forward approach. However, it can be interpreted as a gradient- based adversarial attack.
In proxy-free diffusion, guidance is not dependent proxy gradients, which enables an inherent robustness of the sampling process. Particularly, it models the score linear combination of an unconditional and a conditional score. A unified neural network sø( v) parameterizes both score types. The score so(2,, approximates the gradient of the log probability Va, log p(axly)) i.e., the conditional score, while 8o (at) estimates the gradient of the log probability Vz, p(x)), ie., the unconditional score. The score function follows:
(m y, w) (1 s))0 v) w8)())))
Within this context, the strength parameter w specifies the generation's adherence to the condition y, which is set to the maximum value Ymaz in the offline datasct following Krishnamoorthy al. (2023). Optimization of w balances the condition and diversily. Lower w values increase sample diversily at the expense of conformity to y, and higher values do the opposite.
3 Related Work
Offline black-box optimization. A recent surge in research has presented two predominant approaches for offline BBO. The forward approach deploys a DNN to fit the offline dataset, subsequently utilizing gradient ascent to enhance existing designs. Typically, these techniques, including COMs (Trabucco et al.,
2021), ROMA (Yu et al., 2021). NEMO (Fu & Levine, 2021), BDI (Chen et al. 2022b
IOM QQi (Chen et al., 2023a), are designed to embed prior knowledge within
2023b),
ct al. 2022a and Parallel-mentorte
the surrogate model to alleviate the OOD issue. The reverse approach (Kumar & Levine 2020 Chan et al., 2021) is dedicated to learning a mapping from property values back to inputs. Feeding a high value into this inverse mapping directly produces design of elevated performance. Additionally, methods in Brookes et all (2019): Fannjiang & Listgarten (2020) progressively tailor a generative model towards the optimized design via & proxy function and BONET (Mashkaria et al. (023) introduces an autoregressive model trained on fixed-length trajectories to sample high-scoring designs. Recent investigations (Krishnamoorthy et al.) 2023) have underscored the superiority of diffusion models in delincating the inverse mapping. However, rescarch specialized guided diffusion for offline BBO remains limited. This paper addresses this research gap.
Guided diffuuion. Guided diffusion seeks to produce samples with specific desirable attributes. Contempo- rary research in guided diffussin primarily concentrates on enhancing the elficiency of its sampling process. Meng et al. 2023) propose a method for distilling a classifier free guided diffusion model into a more efficient single model that necessitates fewer steps sampling. [Sadat et al. (2023) introduce the Dynamic CFG, which initially compels the model depend more on the unconditional score and progressively shifts towards the standard CFG. However, unlike our method, it does not optimize the strength parameter. [Kynkäännemmj et al. (2024) restrict the guidance to specific range of noise levels, which improves both sampling speed and quality. Wizadwongsa & Suwajanakorn (2023) introduce an operator splitting method to expedite classifier guidance by separating the update process into two key functions: the diffusion function and the conditioning function. Additionally, [ansal et al.] (2023) presents an efficient and universal guidance mechanism that
utilizes a readily available proxy to enable diffusion guidance across time steps. In this work, we explore the application of guided diffusion in offiine BBO, with the goal of creating tailored algorithms efficiently generate high-performance designs.
4

---

Published in Transactions on Machine Learning Research (12/2024)
4 Method
In this section, we present our method RGD, melding the strengths of proxy and proxy-free diffusion for effective conditional generation. Firstly, we describe a newly developed module termed prozy-enhanced sampling. It integrales explicit proxy guidance into proxy-free diffusion to enable enhanced sampling control, as detailed in Section 4.1 Subsoquently, we explore diffusion-based prozy refinement which incorporates
insights gleaned from proxy-free diffusion back into the proxy, further elaborated in Section 4.2
The overall
algorithm is shown in Algorithm
4.1 Proxy-enhanced Sampling
As discussed in Section 2.3 proxy-free diffu- sion trains an unconditional model and condi- tional models. Although proxy-free diffusion can generate samples aligned with most con- ditions, it traditionally lacks control due to the absence of an explicit proxy. This is par- ticularly significant in offline BBO where we aim to obtain samples beyond the training dis- tribution, as the offline dataset is inherently limited. Therefore, we require explicit proxy guidance to achieve enhanced sampling con-
Algorithm 1 Robust Guided Diffusion for Offline BBO
Input: offine dataset D, of diffusion steps T.
1: Train proxy distribution pø(u(z) on DD by Eq. (2)
2: Train proxy-free diffusion model s0 sa( on D.
3:
prozy refénement #/
4: Identify adversarial samples & via Eq. (3)
5: Compute diffusion distribution Po (b) by Eq.
12
6: Compute KT. divergence loss per Eq. 13
7: Refine proxy distribution p (v x) through Eq.
15
trol. This module is outlined in Algorithm Line 8- Line 16.
1
8:
sampl linglngg */
9: Begin with KT ~ (O,
Optimization of Directly updating the design æ, with proxy gradient suffers from the OOD issue and determining proper condi- tion y necessitates the manual adjustment of multiple hyperparameter (Kunar & Levine 2020). Thus, we propose to introduce proxy guidance by only optimizing the strength pa- rameter w within šs (Zt, y,w) in Eq. 66. As discussed in Section 2.3] the parameter wa bal- ances the condition and diversity, and an optimized w could achieve better balance in the sampling process, leading to more effective generation.
10: for t= 7 1 to 0 do
11: Derive the score 80 50 y, w) from Eq.
12: Update I++1 to xr(w)) using w as per Eq.
13: Optimize to @ following Eq. (8)
Finalize the update of æ; with via Eq. 99
15: end for
16: Return x² = x0
Enhanced Sampling. With the score function, the update of a noisy sample 2e41 is computed as:
[r(w)= solver(x141 50 w)),
where the solver the second-order Heun solver (Süli & Mayers 2003), chosen for its enhanced accuracy through predictor-corroctor method. A proxy is then trained to predict the property of noise æ, at time step 1, denoted as Ja( By maximizing Jø (x(x,(w) (w), () with respect to w, we can incorporate the explicit proxy guidance into proxy-free diffusion to facilitate improved sampling control in the balance between condition and diversity. This maximization process is:
s = w + OJ (a)().()).))
(8)
Ow
where 77 denotes the learning rate. We leverage the automatic differentiation capabilities of PyToorch Paszke ct al. 2019) to efficiently compute the above derivatives within the context of the solver's operation. The optimized @ then updates the noisy sample x++1 through:
x solver(xt41 (1141,1,4))
(9)
This process iteratively denoises Fa utilizing it in successive steps to progressively approach Da which represents the final high-scoring design x..
5

---

Published in Transactions on Machine Learning Research (12/2024)
Proxy Training. Notably, (z t) can be directly derived from the proxy Jø(æ), the mean of the proxy distribution P=(-(ar) in Eq. [ This distribution is trained exclusively at the initial time step & = 0, eliminating the need for training across time steps and reducing computational cost. To achieve this derivation, we reverse the diffusion from 2e back to Zu using the Tweedie formula Robbins 11922)
Do + s0())) a(t)² p(1)
(10)
where s0())) is the estimated unconditional score at time step I, and (1)² and μu()) are the variance and the mean coefficient of the perturbation kernel at time t, as detailed in equations (32-33) in [Song et al. (2021)
For more detailed derivation, refer to the Appendix B Consequently, we express
( (x,, + (1)2
Jø 1) = Js
(11)
((t)
This formulation allows for the optimization of the strength parameter w via Eq. refer to (.) in subsequent discussions.
8
For simplicity, we will
4.2 Diffusion-based Proxy Refinement
In the prozy-enhanced sampling module, the proxy (.).) is employed to update the parameter w to enable enhanced control. However, (+) may still be prone to the OOD issue, especially on ad advessarial samples (Tra- bucco al.) 2021). To address this, we refine the proxy by using insights from proxy-free diffusion. The procedure of this module is specified in Algorithm Lines 3-7.
Diffusion Distribution. Adversarial samples are identified by gradient ascent on the proxy per Eq. 1 to form the distribution g(xr). We utilize vanilla proxy to perform 300 gradient ascent steps, identifying samples with unusually high prediction scores as adversarial. This method is based on the limited extrapolation capability of the vanilla proxy, as demonstrated in Figure 3 in COMs (Trabucco et al. 2021). Consequently, these samples are vulnerable to the proxy distribution. Conversely, the proxy-free diffusion, which functions without depending on a proxy, inherently offers greater resilience against these samples, thus producing more robust. distribution. For an adversarial sample & q(x),, we compute po and po (o using the probability flow ODE associated with the SDE, as detailed in Appendix D of [Song et al. (2021). The implementation can be accessed here. We estimate p(y) using Gaussian kernel-density estimation. The diffusion distribution regarding y is:
po(y (o(l)) ply) ()
(12)
which demonstrates inherent robustness over the proxy distribution (y((). Yet, directly applying diffusion
distribution to design optimization by gradient ascent computationally intensive and potentially unstable due to the demands of reversing ODEs and scoring steps.
Proxy Refinement. We opt for a more feasible approach: refine the proxy distribution (o(
((J by minimizing its distance to the diffusion distribution po (y(l.). The distance is quantified by the Brobackk-Libbeer (KL) divergence:
[p[KL(psllpo]] = Eg(=)) / Ps (y a)) log
Po (u ax) (+) dy.
P
(13)
Po
We avoid the reparameterrizttion trick for minimizing this divergence as it necessitates backpropagation through pe(y(é), which prohibitively expensive. Instead, for the sample &, the gradient of the KL divergence KI(pplleg (Io)) with respect to the proxy paramcters & computed as:
d d (vl do
(1 log Po(v)() Po (
Epo(yla)
(14)
Complete derivations are in Appendix A
The KL divergence then acts as regularization in our loss C:
C(D, a))
Ep( E5[[-loe log (1)(2=)) (FFy(a)) (KT(pppl1o))
(15)

---

Published in Transactions on Machine Learning Research (12/2024)
where D is the training dataset and is hyperparameteer. We propose to optimize a based on the validation The computational effort of this module can exceed that of the sampling itself, a topic we explore further in Appendix E Notably, even without this module, our method maintains strong performance, as detailed in Table B
loss via bi-level optimization as detailed in Appendix C
5 Experiments
In this section, we conduct comprehensive experiments to evaluate our method's performance.
5.1 Benchmarks
Tasks, Our experiments encompass variety of tasks, split into continuous and discrete categories.
The continuous category includes four tasks: (1) Superconductor (SuperC): The objective here is to engineer superconductor composed of 86 continuous elements. The goal is to enhance the critical temperature using 17, 010 design samples. This task is based on the dataset from Hamidieh (2018). (2) Ant. Morphology (Ant): In this task, the focus is on developing a quadrupedal ant robot, comprising 60 continuous parts, to augment- its crawling velocity. It uses 10, 004 design instances from the dataset in Trabucco et al.) (2022); Brockman et al. (2016) (3) D'Kitty Morphology (D'Kitty): Similar to Ant Morphology, this task involves the design of quadrupodal D'Kitty robot with 56 components, aiming to improve its crawling speed with 10, 004 designs,
as described in Trabucco et al. (0022); Ahn et al. (2020). (4) Rosenbrock (Rosen): The aim of this task is to optimize a 60-dimension continuous vector to maximize the Rosenbrock black-box function. It uses 50000
designs from the low-scoring part (Rosenbrock 1960).
For the discrele category, we explore three lasks: (1) TF Bind 8 (TF8): The goal is to identify an 8-unit. DNA sequence that maximizes binding activity. This task uses 32, 898 designs and detailed in [Barrera] ct al. (016)) (2) TF Bind 10 (TF10): Similar to TF8, but with 10-unit DNA sequence and a larger pool of 50, 000 samples, described in (Barrera et al. 2016). (3) Neural Architecture Search (NAS): This task focuses discovering the optimal neural network architecture to improve test accuracy on the CIFAR-10 dataset, using 1, 771 designs (Zoph & Le 2017).
Evaluation. In this study, we utilize the oracle evaluation from design-bench (Trabucco et al., 2022 Adhering this established protocol, we analyze the top 128 promising designs from each method. The evaluation metric employed is the 100th percentile normalized ground-truth score, calculated using the formula y- Smin where Ymin and Vmax signify the lowest and highest scores respectively in the comprehensive,
yn Braa - Smmtn
yet unobserved, dataset. In addition to these scores, we provide an overview of each method's effectiveness through the mcan and median rankings across all evaluated tasks. Notably, the best design discovered in the offline dataset, designated as D(best), is also included for reference. For further details on the 50th percentile (median) scores, please refer to Appendix C.
5.2 Method Comparison
Our approach is evaluated against two primary groups of baseline methods: forward and inverse approaches. Forward approaches enhance existing designs through gradient ascent. This includes: (i) Grad: utilizes simple
gradient ascent on current designs for new creations; (ii) ROMA (Yu et al.) 2021 implements smoothness
regularization on proxies; (iii) COMs (Trabucco et al. 2021 ):
applies regularization to assign lower scores to
adversarial designs; (iv) NEMO (Fu & Levine [021]: bridges the gap between proxy and actual functions using normalized maximum likelihood; (v) BDI (Chen et al. 2022b): utilizes both forward and inverse mappings to transfer knowledge from offline datasstss to the designs; (vi) 10M (Qi et al.) 2022b): ensures consistency between representations of training datasets and optimized designs.
Inverse approaches locus on learning a mapping from a design's property value back to its input. High property values are input into this inverse mapping to yield enhanced designs. This includes: (1) CbAS (Brookes ct al. 2019): : CbAS employs VAE model to implicitly implement the inverse mapping. It gradually tunes its distribution toward higher scores by raising the scoring threshold. This process can be interpreted as incrementally increasing the conditional score within the inverse mapping framework. (ii) Autofocused CbAS
7

---

Published in Transactions on Machine Learning Research (12/2024)
(Auto. CbAS) Fannjjang & Listgarten
2020 adopts importance sampling for retraining a regression model
based on CbAS. (iii) MIN (Kumar &
Levine 2020): maps scores to designs via GAN model and explore
this mapping for optimal designs. (iv) BONET (Mashkaria et al. 2023); introduces an autoregressive model
for sampling high-scoring designs. (v) DDOM
Krishnamoorthy et al. 2023): utilizes proxy-free diffusion to
model the inverse mapping.
Traditional methods as detailed in Trabucco et al. (0022) are also considered: (i) CMA-ES (Hansen 2006): modifies the covariance matrix to progressively shift the distribution towards optimal designs; (fi) BO-qEI (Wilson et al. 2017]: implements Bayesian optimization to maximize the proxy and utilizes the peess--ecceeee-mppooeemee acquisition function for design suggestion, labeling designs using the proxy;
(iii) REINFORCE (Willims 1992): enhances the input space distribution using the learned proxy model.
5.3 Experimental Configuration
In alignment with the experimental protocols established in Trabucco el al. (2022); Chen el al. (2022b), we have tailored our training methodologies for all approaches, utilizing three- MLP architecture for all involved proxics. For methods such as BO-qEI, CMA-ES, REINFORCE, CbAS, and Auto.CbAS that do not utilize gradient ascent, we base our approach on the findings reported in Trabucco et al. (2022). J We adopted T = 1000 diffusion sampling steps, set the condition y to Ymarx and initial strength w as 2 line with Krishnamoorthy al. (2023). To ensure reliability and consistency in our comparative analysis, each experimental sctting was replicated across 8 independent runs, unless stated otherwise, with the presentation of both mean values and standard deviations. These experiments were conducted using NVIDIA GePorce V100 GPU, We have detailed the computational overhead of our approach in Appendix D to provide a comprehensive view of its practicality.
Table 1: Results (maximum normalized score, mean + std) on continuous tasks.
Method
Superconductor
Ant Morphology
D'Kitty Morphology
Rosenbrock
D(best)
0.399
0.565
0.884
0.518
BO-qEI
0.402 + 0.034
0.819 + 0.000
0.896 = 0.000
0.772 + 0.012
CMA-ES
0.465 + 0.024
1.214 + 0.732
0.724 + 0.001
0.470 + 0.026
REINFORCE
0.481 + 0.013
0.266 + 0.032
0.562 + 0.196
0.558 + 0.013
Grad
0.490 = 0.009
0.932 = 0.015
0.930 = 0.002
0.701 = 0.092
COMs
0.504 + 0.022
0.818 0.017
0.905 + 0.017
0.672 + 0.075
ROMA
0.507 + 0.013
0.898 + 0.029
0.928 + 0.007
0.663 + 0.072
NEMO
0.499 = 0.003
0.956 + 0.013
0.953 = 0.010
0.614 0.000
10M
0.524 0.022
0.929 + 0.037
0.936 = 0.008
0.712 = 0.068
BDI
0.513 + 0.000
0.906 + 0.000
0.919 = 0.000
0.630 + 0.000
CbAS
0.503 + 0.069
0.876 + 0.031
0.892 = 0.008
0.702 + 0.008
AuAo.CbAS
0.421 + 0.045
0.882 + 0.045
0.906 + 0.006
0.721 + 0.007
MIN
0.499 + 0.017
0.445
0.892 + 0.011
BONET
0.422 0.019
0.925 0.010
0.941 0.001
0.780 = 0.009
DDOM
0.495 0.012
0.940
0.935 0.001
0.789 + 0.003
RGD
+.015
0.968
0.943 0.004
0.797 + 1.011 0.011
5.4 Results and Analysis
In Tables I and showcase experimental results for both continuous and discrete tasks. To clearly differentiate among the various approaches, distinct lines separate traditional, forward, and inverse approaches within the tables For every task, algorithms performing within a standard deviation of the highest score are emphasized by bolding following Trabucco et al. (2021)
We make the following observations. (1) As highlighted in Table 2 RGD not only achieves the top rank but also demonstrates the best performance in six out of seven tasks, emphasizing the robustness and superiority of our method. (2) RGD outperforms the VAF-based CbAS, the GAN-based MIN and the Transformer-based
8

---

Published in Transactions on Machine Learning Research (12/2024)
Table 2: Results (maximum normalized score, mean std) on discrete tasks & ranking on all tasks.
Method
TF Bind 8
TF Bind 10
NAS
Rank Mean
Rank Median
D(best)
0.439
0.467
0.436
BO-qEI
0.798 = 0.083
0.652 = 0.038
1.079 = 0.059
9.1/15
11/15
CMA-ES
0.953 0.022
0.670 + 0.023
0.985 0.079
7.3/15
4/15
REINFORCE
0.948 0.028
0.663 + 0.034
1.895 0.000
11.3/15
14/15
Grad
0.872 0.062
0.646 = 0.052
0.624 0.102
9.0/15
10/15
COMs
0.517 = 0.115
0.613 + 0.003
0.783 + 0.029
10.3/15
10/15
ROMA
0.927 0.033
0.676 + 0.029
0.927 + 0.071
6.1/15
6/15
NEMO
0.942 0.003
0.708 + 0.022
0.737
5.3/15
5/15
IOM
0,823 = 0.130
0.650 0.042
0.089
7,4/15
6/15
BDI
0.870 0.000
0.605 + 0.000
0.722
9.6/15
9/15
CbAS
0.927 0.051
0.651 + 0.060
0.683
8.7/15
8/15
Auto.Clo CbAS
0.910 + 0.044
0.630 + 0.045
10.3/15
10/15
MIN
0.905 0.052
0.616 0.021
0.017
10.4/15
10/15
BONET
0.913 0.008
0.621 + 0.030
0.724
7.7/15
8/15
DDOM
0.957 + 0.006
0.657 + 0.006
4.9/15
5/15
RGD
0.974 0.003
0.694 + 0.018
0.825 0.063
2.0/15
2/15
BONET. This result highlights the superiority of diffusion models in modeling inverse mappings compared to other generative approaches. (3) Upon examining TF Bind 8, we observe that the average rankings for forward and inverse methods stand at 10.3 and 6.0, respectively. In contrast, for TF Bind 10, both methods have the same average ranking of 8.7, indicating no advantage. This notable advantage of inverse methods in TF Bind implies that the relatively smaller design space TF Bind 8 (4%) facilitates easier inverse mapping, as opposed to the more complex space in TF Bind 10 (419). (4) RGD's performance is less impressive on NAS, where designs are encoded as 64-length sequences of 5-calegory one-hot vectors. This may stem from the design-bench's encoding not fully capturing the sequential and hierarchical aspects of network architectures, affecting the efficacy of inverse mapping modeling.
5.5 Ablation Studies
In this section, we present a series of ablation studies to scrutinize the individual contributions of distinct components in our methodology. We employ our proposed approach as benchmark and methodically exclude key modulcs, such as the prony-enhanced sampling and diffusion-based prozy refinement, to assess their influence on performance. These variants are denoted as w/o prozy-e and w/o diffusion-b r. Additionally, we explore the strategy of directly performing gradient ascent on the diffusion intermediate state with a learning rate of 0.1, referred
direct grod update. The results from these ablations are detailed in Table 3
Our analysis reveals that omitting either module results in a decrease in performance, thereby affirming the importance of each component. The w/o diffusion-b r variant generally surpasses w/o prozy-e, highlighting the utility of the proxy-enhanced sampling even with a basic proxy setup. Conversely, direct grad update tends to produce subpar results across tasks, likely attributable to the proxy's limitations in handling out-of-distribution samples, leading to suboptimal design optimizations.
To further dive into the proxy-enhanced sampling module, we visualize the strength ratio w/wp where wo represents the initial strength- across diffusion steps t. This analysis is depicted in Figure 3 for two tasks: Ant and TF10. We observe pattern of initial decrease followed by increase in w' across both tasks. This pattern can be interpreted as follows: The decrcase in facilitates the generation of a more diverse set of samples, enhancing exploratory capabilities. Subsequently, the increase in w signifies a shift towards integrating high- performaeeee features into the sample generation. Within this context, conditioning on the maximum y is not aimed at achieving the datasct's maximum but at enriching samples with high-scoring attributes. Overall, this adjustment of effectively balances between generating novel solutions and honing in high-quality ones.
9

---

Published in Transactions on Machine Learning Research (12/2024)
Table Ablation studies on RGD (maximum normalized score, mean = std).
Task
D
RGD
w/o proxy-e
w/o diffusion-b r direct grad update
SuperC
86
0.515 + 0.011
0.495 0.012
0.502 + 0.005
0.456 + 0.002
Ant
60
0.968 + 0.006
0.940 0.004
0.961 + 0.011
-0,006 0.003
D'Kitty
56
0.943 = 0.004
0.935
0.001
0.939 + 0.003
0.714 + 0.001
Rosen
60
0.797 = 0.011
0.789 0.003
0.813 + 0.005
0.241 + 0.283
TF8
8
0.974
0.003
0.957 0.007
0.960 + 0.006
0.905 + 0.000
TF10
10
0.694
0.018
0.657 0.006
0.667 + 0.009
0.672 + 0.018
NAS
64
0.825 = 0.063
0.745 = 0.070
0.717 + 0.032
0.718 + 0.032
1.4
Ant
Proxy Distribution po(y
TF10
Peak of Proxy Distribution
1.2
Diffusion Distribution Poly|x)
Strength ratio w/ul
1.0
Prob Density
w
Peak of Diffusion Distribution
Ground-truth
0.8
0.6
0.4
0.2
1000
800
700
600
500
400
300
200
100
0.6
0.8
1.0
1.2
Diffusion step
Figure 3: This adjustment of w effectively balances Figure 4: The proxy distribution overestimates the between generating novel solutions and honing in on ground truth, while the diffusion distribution closely aligns with it, demonstrating its robustness.
high-quality ones during sampling.
In addition, we visualize the proxy distribution alongside the diffusion distribution for a sample & from the Ant task in Figure 4 to substantiate the efficacy of diffusion-based proxy refinement. The proxy distribution significantly overestimates the ground truth, whereas the diffusion distribution closely aligns with it, demonstrating the robustness of diffusion distribution. For more quantitative analysis, compute the expectation of both distributions and compare them with the ground truth. The mean of the diffusson Po(yl)) ie)
distribution is calculated as (pl()) [y] (v)(v))) Pe(xa)) 9 The MSE loss for the proxy distribution is 2.88, while for the diffusion distribution, it is 0.13 on the Ant task. Additionally, evaluate this the TFB10 task, where the MSE loss for the proxy distribution is 323.63 compared to 0.82 for the diffusion distribution. These results further corroborate the effectiveness of our proposed module.
Furthermore, we (1) investigate the impact of replacing our trained proxy model with alternative approaches, specifically ROMA and COMs, (2) analyze the performance with an optimized condition y and (3) explore a simple annealing approach of w. For comprehensive discussion on these, readers are referred Appendix F where the results further highlight the effectiveness of our trained proxy and the adaptation strategy.
5.6 Hyperparameter Sensitivity Analysis
This section investigates the sensitivity of RGD to various hyperparameters. Specifically, we analyze the effects of (1) the number diffusion sampling steps T, (2) the condition y, and (3) the learning rate 7 of the proxy-enhanced sampling. These parameters are evaluated on two tasks: the continnous Ant task and the discrete TFB10 task. Our method is generally robust to these hyperparameters. For detailed discussion, see Appendix G
10

---

Published in Transactions on Machine Learning Research (12/2024)
6 Conclusion
In conclusion, we propose Robust Guided Diffusion for Offline Black-bor Optimization (RGD). The prozy- enhanced sampling module adeptly integrates proxy guidance to facilitate improved sampling control, while the diffusion-based prozy refinement module leverages proxy-free diffusion insights for proxy improvement. Empirical evaluations on design-bench have showcased RGD's outstanding performance, further validated by ablation studies on the contributions of these novel components. We discuss the broader impact and limitation in Appendix !
References
Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, and Vikash Kumar. Robel: robotics benchmarks for learning with low-cost robots. In Conf. on Robot Lea. (CoRL), 2020.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldbhum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proc. Comp. Vision. Pattern. Rec. (CVPR), 2023.
Luis A Barrera et al. Survey of variation in human transcription factors reveals provalent DNA binding changes. Science, 2016.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint 1xXx:1666601540,, 2016.
David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In Proc. Int. Conf. Machine Learning (ICML), 2019.
Alvin Chan, Ali Madani, Ben Krause, and Nikhil Naik. Deep extrapolation for attribute-enhanced generation. Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS), 2021.
Can Chen, Yingxue Zhang, Jie Fu, Xue Liu, and Mark Coates. Bidirectional learning for offline infinite-width model-based optimization. Proc. Adu. Newr. Inf. Proc. Syst (NewrIPS), 2022a.
Can Chen, Yingxue Zhang, Jie Fu, Xue Liu, and Mark Coates. Bidirectional learning for offline infinite-width model-based optimization. In Proc. Adv. Neur, Inf. Proc. Syst (NewrIPS), 2022b.
Can Chen, Christopher Beckham, Zixuan Liu, Xue Liu, and Christopher Pal. Parallel-mentoring for offline model-based optimization. In Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS), 2023a.
Can Chen, Yingxue Zhang, Xue Liu, and Mark Coates. Bidirectional learning for offiine model- model-based biological sequence design. In Proc. Ind. Conf. Machine Lea. (ICML.,, 2023b.
Prafulla. Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Proc. Adu. Newr. Inf. Proc. Syst (NeurIPS), 2021.
Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model-based design. Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS), 2020.
Justin Fu and Sergey Levine. Offline model-based optimization via normalized maximum likelihood estimation. Proc. Inl. Conf. Learning Rep. (ICLR), 2021.
Tan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS), 2014.
Kam Hamidich. A data-driven statistical model for predicting the critical temperature of superconductor. Computational materials science, 2018.
11

---

Published in Transactions on Machine Learning Research (12/2024)
Nikolaus Hansen. The CMA evolution strategy: comparing review. Towards A New Evolutionary Compulation, 2006.
Jonathan Ho and Tim Salimans. Classifier-fre diffusion guidance. arXiv preprint orXim:2007.1559. 2022.
Jonathan Ho, Ajay Jain, and Picter Abbecl. Denoising diffusion probabilistic models. Advonces in newral information processing systems, 2020.
Chin-Wei Huang, Jac Hyun Lim, and Aaron C Courville. A variational perspective on diffusion- generative models and score matching, Advances in Neural Information Processing Systems, pp. 22863 22876, 2021.
Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black-box optimization. Proc. Int. Conf. Machine Learning (ICML), 2023.
Aviral Kumar and Sergey Levine, Model inversion networks for model-based optimization. Proc. Adv. Newr. Inf. Proc. Syst (NeurIPS), 2020.
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. arXiv preprint arXiv:2404.07722 2024.
Satvik Mehul Mashkaria, Siddarth Krishnamoorthy, and Aditya Grover. Generative pretraining for black-box optimization. In Proc. Ind. Conf. Machine Learning (ICMT.), 2023.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proc. Comp. Vision. Pattern. Rec. (CVPR)) 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: an imperative style, high-performance deep learning library. Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS), 2019.
Han Qi, Yi Su, Aviral Kumar, and Sergey Levine. Data-driven model-based optimization via invariant representation learning. In Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS), 2022a.
Han Qi, Yi Su, Aviral Kumar, and Sergey Levine. Data-driven model- based optimization via invariant representation learning. In Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS), 2022b.
Herbert E Robbins. An empirical bayes approach to statistics. In Breakthroughs in Statistics: Foundations and basic theory. 1992.
HoHo Rosenbrock. An automatic method for finding the greatest or least value of a function. The computer journal, 1960.
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M Weber. Cads: Unleash- ing the diversity of diffusion models through condition-annealed sampling. arXiv preprint ar Xr::233100.77//7
2023.
Karen S Sarkisyan et al. Local fitness landscape of the green fluorescent protein. Nature, 2016.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishck Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. Proc. Int. Conf. Learning Rep. (ICLR), 2021.
Endre Süli and David F Mayers. An introduction to numerical analysis. Cambridge university press, 2003.
Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for elfective offline model-based optimization. In Proc. Inl. Conf. Machine Learning (ICML), 2021.
Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Besign-Bench: benchmarks for data-driven offline model-based optimization. Xiv preprint orXiv:2202.08. 08450, 2022.
12

---

Published in Transactions on Machine Learning Research (12/2024)
Ronald J Williams.
Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 1992.
James T Wilson, Riccardo Moriconi, Frank Hutter, and Mare Peter Deisenroth. The reparameterization trick for acquisition functions. arXiv preprint Xrxi 171.00044 2017.
Suttisak Wizadwongsa and Supasorn Suwajanakorn. Accelerating guided diffusion sampling with splitting numerical methods. In Proc. Int. Conf. Learning Rep, (ICLR), 2023. URL http:://oppeneeiiee..c net/
Sihyun Yu, Sungsoo Ahm, Le Song, and Jinwoo Shin. Roma: robust model adaptation for offline model-based optimization. Proc. Adv. Neur. Inf. Proc. Sysl (NewrIPS), 2021.
Barret Zoph and Quoc V. Le. Neural architecture search with reinforcement learning. arXiv preprint arvi 161.00178,, 2017.
13

---

Published in Transactions on Machine Learning Research (12/2024)
A
Derivation
This section provides a derivation of the gradient of the KL divergence. Let's consider the KL divergence term, defined
( ②t) Pe ( (
KL(pellpe) = (o((() ) log
dy.
(16)
The gradient with respect to the parameters is computed as follows:
dKL(pøllpo)
/ dpo(ula)
( log (s(y(() (
dy
dep
deb
Pa (t)
/
d dlog po (ul (t) (1 + log
Pe (yl (t) dy
poll)a
(17)
dΦ Idlog (oyyat)
po (3) (i)) )
1 log Po (ft) (3/ (t)
Ev s))
dΦ
B
Proxy Training
We follow (33) from Song al. (2021) where p(x+|xo) ((t)xxo()xxo))))) (t) Given this, we can sample
X, from Xo using: x, px()xx + (t). To recover Xy from X4, need to know c, which approximates
~ -o() (x,). Using this approximation, we derive Xo Ximea(t) RJ (t)
(())
This approach originates from Song et al. [2021) and utilize the implementation framework detailed in another seminal work (Huang et al. 2021 Our code, available here implements this process as follows:
Line 24 implements μ(t)
Line 27 implements (2())
Line 37 describes the sampling process: =(()xx + cat()
Line 112 optimizes: c - (x,), where e the target, (t) is the std, and a is so(x.).
Additionally, it's worth noting that Eq. 10 in work aligns closely with (15) from the seminal work DDPM (Ho et al. 2020). In DDPM, they present the equation an s 2z- TT-vlaor) which is derived in a discrete setting.
C Hyperparameter Optimization
We propose adjusting a based on the validation loss, establishing bi-level optimization framework:
a" arg min ED, [log Por (a)
(18)
s.t. o' ('(a) = arg min L(d.a).
(19)
Within this context, D, represents the validation datasel sampled from the ofline dataset. The inner optimization task, which seeks the optimal o² (a), is efficiently approximated via first-order gradient descent methods. We use batch optimization, with each batch containing 256 training samples and 256 validation samples. The bi-level optimization process updates the hyperparameter with a single iteration for both the inner and outer levels.
14

---

Published in Transactions on Machine Learning Research (12/2024)
D Evaluation of Median Scores
While the main text of our paper focuses on the 100th percentile scores, this section provides an in-depth analysis of the 50th percentile scores. These median scores, previously explored in TTrbbucoo al. (2022). , serve additional metric to assess the performance of our RGD method. The outcomes for continuous tasks are detailed in Table G and those pertaining to discrete tasks, along with their respective ranking statisties, are outlined in Table 7 An examination of Table 7 highlights the notable success of the RGD approach, as it achieves the top rank in this evaluation. This finding underscores the method's robustness and effectiveness.
E
Computational Overhead
Table 4: Computational Overhead (in seconds).
Process
SuperC
Ant
D'Kitty
NAS
Proxy training
40.8
74.5
24.7
7.8
Diffusion training
405.9
767.9
251.1
56.0
Proxy-e sampling
30.0
29.7
29.6
31.5
Diffusion-b b proxy r
3104.6
4036.7
2082.8
3096.2
Overall cost
3581.3
4908.8
2388.2
3191.5
In this section, we analyze the computational overhead of our method. RGD consists of two core components: proxy-enhanced sampling (prozy-e sampling) and diffusion-based proxy refinement. (diffusion-b prozy r). Additionally, RGD employs trained proxy and proxy- diffusion model, whose computational demands are denoted as prozy training and diffusion training, respectively.
Table $ indicates that experiments can be completed within approximately one hour, demonstrating elficiency. The diffusion-based prozy refinement module is the primary contributor to the computational overhead, primarily due to the usage one-time process for refining the proxy, its high computational cost is offset by its non-recurring nature. We have also compared the time costs of various competitive methods for the 86-dimension continuous SuperC task. Our method (RGD) requires approximately 3581.3 seconds, Auto. requires 921.4 seconds, BONET requires 673.2 seconds, DDOM requires 460.7 seconds, and BDI requires 618.4 seconds. For the discrete 64-dimension NAS task, our method (RGD) takes approximately 3191.5 seconds, while Auto. CbAS takes 389.3 seconds, MIN takes 879.1 scconds, BONET takes 485.3 seconds, DDOM takes 103.4 seconds, and BDI lakes 498.0 seconds. The evaluation any mentioned method NAS entails training the CIFAR-10 dataset over 20 epochs for 128 architectural designs, accumulating a total of 153.6 hours. Tn comparison, the one-hour computation time of our method appears negligible. This comparative analysis illustrates the computational overhead of RGD relative to other methods.
probability flow ODE for sample likelihood computation. However, as this is a
requires 425.2 scconds, MIN
The computational bottleneck of our method is the diffusion-lussed prozy refinement module. When we remove this module, this adjustment significantly reduces computational overhead: from 3581.3 seconds to 476.7 seconds on SuperC, and from 3191.3 seconds to 95.3 seconds on NAS, rendering our method more efficient than the comparison methods. Following this adjustment, we recalculate the rankings based on the results presented in Tables 2 and 3 The new ranking, as shown in the Table 5 reaffirms that our method continues hold its position as the top-performing method in terms of ranking.
In contexts such as robotics or bio-chemical research, the most time-intensive part of the production cycle is usually the evaluation of the unknown objective function. Therefore, the time differences between methods for deriving higl-peeformance designs are less critical in actual production environments, highlighting RGD's practicality where optimization performance are prioritized over computational speed. This aligns with recent literature (A.3 Computational Complexity in Chen et al. (202aa) and A.7.5. Computational Cost in Chen ct al. (2023b)) indicating that in black- box optimization scenarios, computational time is relatively minor compared to the time and resources dedicated to experimental validation phases.
15

---

Published in Transactions on Machine Learning Research (12/2024)
Table 5: Ranking Statistics of Methods.
Method
Rank Mean
Rank Median
BO-qEI
9.14
11.00
CMA-ES
7.14
3.00
REINFORCE
11.29
14.00
Grad
9.00
10.00
COMs
10.00
10.00
ROMA
5.86
6.00
NEMO
5.14
5.00
IOM
7.43
6.00
BDI
9.29
8.00
CbAS
8.57
8.00
Auto. CbAS
10.29
10.00
MIN
10.29
10.00
BONET
7.43
7.00
DDOM
4.71
5.00
RGD
3.71
3.00
Table 6: Results (median normalized score, mean + std) on continuous tasks.
Method
Superconductor
Ant Morphology
D'Kitty Morphology
Roscnbrock
BO-qEI
0.300 = 0.015
0.567 = 0.000
0.883 = 0.000
0.761 = 0.004
CMA-ES
0.379 + 0.003
-0.045 + 0.004
0.684 + 0.016
0.200 + 0.000
REINFORCE
0.463 + 0.016
0.138 + 0.032
0.356 + 0.131
0.553 + 0.008
Grad
0.339 + 0.013
0.532 + 0.014
0.867 + 0.006
0.540 + 0.025
COMs
0.312 + 0.018
0.568 + 0.002
0.883 + 0.000
0.419 0.286
ROMA
0.364 + 0.020
0.467 0.031
0.850 + 0.006
-0.121 + 0.242
NEMO
0.319
0.592 0.001
0.882 = 0.002
0.510 = 0.000
TOM
0.343 0.018
0.513 + 0.024
0.873 + 0.009
0.126 + 0.443
BDI
0.474 0.000
0.855 + 0.000
0.561 + 0.000
CbAS
0.111 + 0.017
0.384 + 0.016
0.753 + 0.008
0.676 + 0.008
Auto.CbAS
0.131 + 0.010
0.364 + 0.014
0.736 + 0.025
0.695 + 0.008
MIN
0.336 0.016
0.618 0.040
0.887 1.004
0.634 + 0.082
BONET
0.319 0.014
0.615 0.004
0.895 - 0.021
0.630 0.009
DDOM
0.295 + 0.001
0.590 0.003
0.870 + 0.001
0.640 + 0.001
RGD
0.308 1 0.003
0.684 0.006
0.874 + 0.001
0.644 + 0.002
F Further Ablation Studies
In this section, we extend our exploration to include alternative proxy refinement schemes, namely ROMA and COMs, to compare against our diffusion-basse proxy refinement module. The objective is to assess the relative offectiveness these schemes the context of the Ant and TFB10 tasks. The comparative results are presented Table g Our investigation reveals that proxies refined through ROMA and COMs exhibit performance akin to the vanilla proxy and they fall short of achieving the cnhancements seen with our diffusion-based proxy refinement. We hypothesize that the ceffusioo-bbsedd proxy refinement, by aligning closely with the characteristies of the diffusion model, provides more relevant and impactful signal. This alignment improves the proxy's ability to enhance the sampling process more effectively.
Additionally, we contrast our approach, which adjusts the strength parameter w, with the MIN method that focuses on identifying an optimal condition y. The MIN strategy entails optimizing a Lagrangian objective with respect to y, a process that requires manual tuning of four hyperparameters. We adopt their methodology determine optimal conditions y and incorporate these into the proxy-free diffusion for tasks Ant and TF10. The normalized scores for Ant and TF10 are 0.950 + 0.017 and 0.660 + 0.027, respectively. The outcomes
16

---

Published in Transactions on Machine Learning Research (12/2024)
Table 7: Results (median normalized score, mean + std) on discrete tasks & ranking on all tasks.
Method
TF Bind 8
TF Bind 10
NAS
Rank Mean
Rank Median
BO-qEI
0.439 + 0.000
0.467 0.000
0.544 + 0.099
6.4/15
7/15
CMA-ES
0.537 = 0.014
0.484 0.014
0.591 + 0.102
8.0/15
5/15
REINFORCE
0.462 = 0.021
0.475 0.008
-1.895 + 0.000
9.7/15
9/15
Grad
0.546 0.022
0.526 + 0.029
0.443 = 0.126
6.6/15
8/15
COMs
0.439 = 0.000
0.467 + 0.000
0.529 + 0.003
7.7/15
8/15
ROMA
0.543 = 0.017
0.518 0.024
0.529 = 0.008
7.6/15
5/15
NEMO
0.436 + 0.016
0.453 + 0.013
0.563 + 0.020
8.3/15
8/15
10M
+
0.474 0.014
-0.083 + 0.012
9.3/15
8/15
BDI
0.000
0.476 - 0.000
0.517 + 0.000
7.3/15
8/15
CbAS
1128
0.292 = 0.027
11.3/15
12/15
Auto.CbAS
0.419 0.007
0.461 0.007
0.217 + 0.005
11.9/15
13/15
MIN
0.421 = 0.015
0.468 0.006
0.433 = 0.000
7.0/15
7/15
BONET
0.507 + 0.007
+.600+0000
0.571 + 0.095
5.9/15
6/15
DDOM
0.553 = 0.002
0.488 + 0.001
0.367 + 0.021
6.9/15
5/15
RGD
0.557 = 0.002
0.545 = 0.006
0.371 0.019
4.9/15
4/15
Table 8: Comparative Results of Proxy Integration with COMs, ROMA, and ours.
Method
Ant Morphology
TF Bind 10
No proxy
0.940 0.004
0.657 0.006
Vanilla proxy
0.961 0.011
0.667 + 0.009
COMs
0.963 = 0.004
0.668 + 0.003
ROMA
0.953 + 0.003
0.667 + 0.003
Ours
0.968 + 0.006
0.694 + 0.018
fall short of those achieved by method detailed in Table g
This discrepancy likely stems from the complexity involved in optimizing whereas dynamically adjusting w proves to be more efficient strategy for enhancing sampling control.
Last but not least, we explore simple annealing approaches for w. Specifically, we test two annealing scenarios considering the default w as 2.0: (1) decrease from 4.0 to 0.0, (2) an increase from 0.0 to 4.0, both modulated by a cosine function over the time step (t) and (3) a high constant w We apply these strategies to the Ant. Morphology and TF Bind 10 tasks, and the results are as follows:
Table 9: Results of Anncaling Approaches.
Method
Ant Morphology
TF Bind 10
RGD
0.968
0.694
w = 2.0
0.940
0.657
W = 4.0
0.927
0.655
Increase
0.948
0.654
Decrease
0.924
0.647
The empirical results across both strategies illustrate their inferior performance compared to our approach, thereby demonstrating the efficacy of our proposed method.
G Hyperparameter Sensitivity Analysis
RGD's performance is assessed under different settings of T, y, and n,. We experiment with T values 500, 750, 1000, 1250, and 1500, with the default being T 1000. For the condition ratio y/ ymaz, we test values of
17

---

Published in Transactions on Machine Learning Research (12/2024)
0.5, 1.0, 1.5, 2.0, and 2.5, considering 1.0 the default. Similarly, for the learning rate 1/ explore values of 2.5e-, 5.0e-3, 0.01, 0.02, and 0.04, with the default set to n = 0.01. Results are normalized by comparing them with the performance obtained at default values.
As depicted in Figures and 7 RGD demonstrates considerable resilience to hyperparameter variations. The Ant task, particular, exhibits more marked sensitivity, with gradual enhancement performance as these hyperparameters are varied. The underlying reasons for this trend include: (1) An increase in the number of diffusion steps (T) enhances the overall quality of the generated samples. This improvement, in conjunction with more effective guidance from the trained prexy, leads to better results. (2) Elevating the condition (v) enables the diffusion model to extend its reach beyond the existing datasct, paving the way for superior design solutions. However, selecting an optimal y can be challenging and may, as observed in the TFB10 task, sometimes lead to suboptimal results. (3) A higher learning rate (1) integrates an enhanced guidance signal from the trained proxy, contributing to improved performances.
In contrast, the discrete nature of the TFB10 lask seems to endow it with certain robustnnsss to variations in these hyperparameters, highlighting distinet behavioral pattern in response to hyperparameter adjustments.
1.011
1011
1.020
Ant
TF10
1,008
Ant
TP10
1P10
1.004
1.000
1.005
Score rato
Score ratlo
Score ratlo
1.005
1,000
0.995
0.995
0.990
500
750
1000
1250
1500
0.960
0.3
1.3 Condition y/ymax
23
0.985
0.0025
0.005
0.01
0.02
0.04
Diffusion step
Learning rate
Figure 5: The ratio of
Figure 6: The ratio of
Figure 7: The ratio of
the performance of our RGD method with T' to the perfor- mance with 1000.
the performance of our RGD method with ymynaz to the performance with 1.0.
the performance our RGD method with 7 to the perfor- mance with ry = 0.01.
H Notations
We provide the key notations used in this paper in Table 10
Broader Impact and Limitation
Broader impact. Our research has the potential to significantly accelerate advancements in fields such new material development, biomedical innovation, and robotics technology. These advancements could lead to breakthroughs with substantial positive societal impacts. However, we recognize that, like any powerful tool, there are inherent risks associated with the misuse of this technology. One concerning possibility the exploitation of our optimization techniques to design objects or entities for malicious purposes, including the creation of more efficient weaponry or harmful biological agents. Given these potential risks, it is imperative to enforce strict safeguards and regulatory measures, especially in areas where the misuse of technology could lead to significant ethical and societal harm. The responsible application and governance of such technologies are crucial to ensuring that they serve to benefit society whole.
Limitation. We recognize that the benchmarks utilized in our study may not fully capture the complexities of more advanced applications, such as protein drug design, primarily due to our current limitations in accessing wet-lab experimental setups. Moving forward, we aim to mitigate this limitation by fostering partuerships with domain experts, which will enable us to apply our method more challenging and diverse problems. This direction not only promises to validate the efficacy of our approach in more complex scenarios but also aligns with our commitment to pushing the boundaries of what our technology can achieve.
18

---

Published in Transactions on Machine Learning Research (12/2024)
Table 10: Key notations used this paper.
Notations
Descriptions
Proxy parameters
Diffusion model parameters
Jo
Learned mean
(+).)
Learned standard deviation
N
Gaussian distribution
Proxy distribution
Particular design
Property of design
a
Adversarial designs
((x)
Adversarial distribution
Time step of diffusion sampling
T
Total number of diffusion steps
Gradient optimization step design
M
Total number of gradient optimization steps
Noisy sample at time step t
f(,tt)
Drift coefficient Diffusion cocfficient Standard Wiener process
9(.)
w
Reverse Wiener process Score of the marginal distribution
Va log p(x)
s-
Learned score function of x;
van /
Two-dim variable of Rosenbrock
X
Design space
d
Design dimension
Optimized strength parameter
Strength parameter
D
Oflline dataset.
(t)²
Variance of perturbation kernel
((t)
Mean of perturbation kernel
Hyperparameter of KL loss
19


Reviews:
[{"review_id": "oSm8FeULlz", "reviewer": "Reviewer_pK3b", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The authors outline all steps of the proposed framework, present unified notation, and provide equations and derivations underlying all steps.\n- The authors perform comprehensive benchmarking of their method on a range of continuous and discrete tasks and against a suite of existing methods that follow traditional, forward, and inverse approaches.\n- The proposed framework includes novel modules with sound theoretical formulation that are empirically demonstrated to perform well on a range of benchmarks and in ablation studies.\n\nWeaknesses\n- The order of presentation in the manuscript does not facilitate clarity. Figures and equations are referenced before they have appeared in the main text; *Related Work* appears at the end and disconnected from the *Preliminaries* and *Comparison Methods* sections; concepts are introduced without clearly relating to each other (e.g. forward/inverse approach, proxy diffusion and proxy-free diffusion, diffusion model, guided diffusion, and diffusion distribution).\n- The issues that the two proposed modules aim to address are described in similar terms: the proxy is susceptible to the out-of-distribution issue and overestimation, while proxy-free diffusion struggles with generating samples exceeding the training distribution. Describing the differences between these two settings more clearly would help with motivating the method.\n- The authors included additional analyses in the Appendix and briefly mention their existence in Sections 4.5 and 4.6. However, the results of these analyses are not reported in the main text. While details and figures of supplementary analyses may be included in the appendix, all results must be reported in the main text.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes a new optimization framework based on a guided diffusion model. The task at hand is optimization of the desired value metric $y$ given measured designs $x$ and the authors utilize a classifier-free diffusion model $s_\\theta(x_t, y)$, updating the solution over time steps $t=T-1,\\dots,0$ and returning a design $x_0$ with the maximized value. The authors use the term \"proxy-free diffusion\" throughout to highlight their application to regression as opposed to classification.\n\nThe authors propose two novel modules incorporated into this process: (i) *proxy-enhanced sampling*, which improves the quality of generated samples by leveraging the proxy $J_\\phi(x)$ that predicts $y$ and (ii) *diffusion-based proxy refinement*, which modifies the proxy distribution to be more robust against adversarial examples. \n\nSpecifically, the *proxy-enhanced sampling* module optimizes the parameter $w$ in the classifier-free guidance score, which balances sample diversity with conformity to the designs with the highest $y$. At each timestep $t$, the previous sample $x_{t+1}$ is updated using the classifier-free score with the current weight $w$, then $w$ is updated to $\\hat{w}$ using the gradient of the proxy $J_\\phi(x_t(w))$, and finally $x_{t+1}$ is updated again using $\\hat{w}$. The *diffusion-based proxy refinement* module regularizes the loss fitting the proxy distribution $p_\\phi(y|x)$ (parameterized by $J_\\phi$) with a KL-divergence term between the proxy distribution and the diffusion distribution evaluated at adversarial samples, i.e. samples for which the proxy predicts unusually high $y$ values. The proxy is refined prior to the diffusion process.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical requested changes:\n- Frame *Introduction*, *Preliminaries* and *Method* within the forward/inverse approach dichotomy (does the proposed framework follow the inverse approach?). Define how guided diffusion relates to diffusion models and how proxy diffusion and proxy-free diffusion (as well as proxy distribution and diffusion distribution) relate to the forward/inverse approach.\n- Modify Figure 2 to have a clear flow of the inference corresponding to Algorithm 1, denote the proxy distribution and proxy-free diffusion model in Figure 2.\n- Unify the *Method* section, Figure 2, and Algorithm 1 to present the steps of RGD in the same order to improve clarity.\n- Define probability flow ODE and explain whether the proposed framework involves both forward and reverse diffusion.\n- Restructure *Related Work* to appear prior to or combine with Section 4.2.\n- Modify the nearly identical sentences describing the novel modules to convey more information about the method (e.g. a sentence \"proxy-enhanced sampling integrates proxy guidance to enable enhanced sampling control\" appears in a nearly identical way 5 times in the manuscript).\n- Give more details on adversarial samples and how they manifest as a weakness of the proxy distribution. Provide more context for the inability of the proxy-free diffusion to generate samples beyond the training distribution and why this setting is particularly applicable to offline BBO.\n- Explain why $J_\\phi(x_t, t)$, which is used for optimizing $w$, is defined w.r.t. $x_0$.\n- Provide more details on the \"direct grad update\" setting in the ablation studies.\n- Report brief results of the supplementary analyses given in Appendices E, F in the main text and specify why those two tasks were selected.\n\nMinor requested changes:\n- Specify what abbreviations stand for (MINs, GAN, COMs).\n- Figure 2 caption: Overall -> Overview\n- Provide citations when introducing diffusion models.\n- Section 3.2, *Proxy Refinement*: $\\mathcal{D}$ notation for the KL divergence is overloaded as it has previously denoted the dataset.\n- Section 3.2, *Proxy Refinement*: parameterization trick -> reparameterization trick\n- Provide more details on the bi-level optimization of $\\alpha$ hyperparameter in terms of the order of operations and the use of training vs. validation set.\n- Section 4.1: footnote 4 seems to be in the wrong place.\n- Section 4.2: *Comparison Methods* -> *Method Comparison*\n- Expand the statement \"we have tailored our training methodologies for all approaches\" and include more details on the training procedure in the Appendix.\n- Add more detailed captions for the figures, especially Figures 3, 4.\n- Specify in the caption what is reported (e.g. mean +- std) and bolded for all Tables.\n\nQuestions for authors:\n- How much value does explicit optimization of $w$ provide over simply setting $w$ to a constant high value? Does one of the benchmarked methods implement this approach and thus could provide an empirical answer to this?\n- How is the unconditional score $s_\\theta(x_t)$ computed from the neural network  $s_\\theta(x_t, y)$?\n- Section 3.1, *Proxy Training*: given the for-loop in the Algorithm 1, should it be initial time step $t=T$ instead of $t=0$?", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vrqr3HSo4U", "reviewer": "Reviewer_rX3i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is clearly written and well-structured, making it easy to follow the proposed framework and experimental results. I especially appreciate Figure 1 for providing a visual motivation of the RGD framework\n- Addition of proxy guidance to diffusion-based models to address OOD issues\n- The method is compared against state-of-the-art baselines and ablations, providing a comprehensive evaluation of the proposed framework\n- The implementation via shared documented code repository enhances reproducibility and transparency\n\nWeaknesses:\n- The refinement requires additional computational overhead, which may limit its practicality in resource-constrained environments, notably on the SuperC task it requires 3581.3 seconds in comparison to Auto.CbAS which requires 425.2 seconds, MIN requires 921.4 seconds, BONET requires 673.2 seconds, DDOM requires 460.7 seconds, and BDI requires 618.4 seconds, which is a significant increase in computational cost. This cost should also be more prominently mentioned in the main body of the paper because it is a significant limitation of the proposed method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a framework called RGD that combines proxy-based and proxy-free diffusion techniques to enhance black-box optimization tasks. The paper integrates proxy guidance into the diffusion process to improve control and robustness during sample generation, focusing on out-of-distribution (OOD) issues. The authors demonstrate the effectiveness of RGD on a variety of continuous and discrete optimization tasks. The downside of the proposed method is the additional computational overhead required for the refinement process, which appears significant in comparison to other state-of-the-art methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please comment more on the computational overhead of the proposed method and compare it on more datasets with respect to related work, with different number and type of features, to provide a better understanding of the trade-off between performance and computational cost also possibly with other hyperparameters like T the number of diffusion steps. I understand your argument that the experiment design's cost is not as significant as the experiment itself, however, the computational overhead currently seems significant and it would be useful to have more data on this to better understand the available trade-offs between algorithmic and hardware performance. The computational cost, since it appears as a significant limitation of the proposed method, should be more prominently mentioned in the main body of the paper.\n- It would be useful to strengthen the motivation by pointing out specific scenarios/datasets on which you benchmarked where it is challenging to generate optimal samples, due to the OOD issues, to provide a more concrete benefit of the proposed method\n- How did you determine \"300 gradient ascent steps\" as the optimal number?\n\nMinor: \n\n- It would be useful to have a notation table in the Appendix to be able to quickly reference the symbols used in the paper\n- To improve clarity it would be useful to move text from footnotes to the main text where appropriate to avoid breaking the flow of the paper - I would move the details on the datasets/challenges from Experiments to Appendix to make space\n- We’ve detailed -> We have detailed", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qrNY9lsiZY", "reviewer": "Reviewer_i1gZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### **Strengths**\n\nThe offline BBO problem is an interesting and challenging one, and the authors do a good job of motivating its importance and applications. The core idea at work, which is to get the \"best of both worlds\" by using a hybrid forward/inverse approach and tuning the diffusion guidance procedure, makes sense. The experimental results are well reported and support the authors' claim to have developed a method that is robust across a variety of BBO tasks. The ablation studies carried out in the paper are sensible and reasonably comprehensive. The authors have also made the code for the method publicly available.\n\n### **Weaknesses**\n\nWhile I consider the related work most similar to the current submission quite easy to understand, I had difficulty following some parts of the technical exposition. Some of this was due to my having to tax my working memory while keeping track of the authors' notational choices and terminology for things much more familiar to me under other names. This minor issue went away by the time I gave the paper a second reading, but I was left with the sense that the paper would greatly benefit from an additional editorial pass, both for clarity (preferably by a naïve reader) and general proofreading. When carrying this out, the authors should consider whether the paper is as self-contained as it could be and whether a reasonably skilled reader could implement the described method based on the paper alone.\n\nHere are a couple of examples of where I got hung up:\n* In Algorithm 1, line 5 refers to the diffusion distribution in Eq. 12, but it is not clear to me how $p_{\\\\theta}(\\\\hat{\\\\mathbf{x}}|y)$ and $p\\_{\\\\theta}(\\\\hat{\\\\mathbf{x}})$ are computed \"via the probability flow ODE,\" since this ODE (which is not actually defined in the paper, while the SDE is) describes the dynamics on $\\\\mathbf{x}$ and not the probability distributions. That would be a job for the Fokker-Planck equation, but I can't imagine that this is actually what's being computed under the hood.\n* On page 5, another proxy is introduced, somewhat confusingly, as $J\\_{\\\\phi}(\\\\mathbf{x}\\_t, t)$, which the authors say is trained to \"predict the property of noise $\\\\mathbf{x}\\_t$ at time step $t$.\" This new proxy shares notation and, apparently, parameterization with the regression model $J_{\\\\phi}(\\\\mathbf{x})$, but it is not clear from the surrounding text how this training happens. A direct relationship is drawn between this new proxy and the original (regression) proxy when the authors \"reverse the diffusion from $\\\\mathbf{x}\\_t$ back to\n$\\\\mathbf{x}\\_0$\" in a single step in Eq. 10. But despite the authors' reference, I cannot reconcile their Equation 10 (which bears some resemblance to Tweedie's formula) with anything in Song et al. (2021), nor can I connect it to any well-known relationship between $\\\\mathbf{x}\\_t$ and $\\\\mathbf{x}\\_0$ in diffusion.\n\nThese are two examples that are representative of various areas where the technical exposition is insufficiently clear. Of course, if I have misread or missed something in the above cases, I would appreciate it if the authors could set me straight. \n\n### **Bottom Line**\n\nThe core ideas in this paper seem sound, and I believe this work would be of interest to the TMLR readership. The experimental results also show that the authors have developed a method that seems to work well on benchmark cases. That the authors have released the code is also a major selling point, which I believe further helps toward fulfilling the claims-and-evidence criterion. However, the paper's clarity issues have me on the fence as to whether this criterion has fully been satisfied. For now, I will give the authors the benefit of the doubt and put down a *qualified* yes on this criterion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Offline black-box optimization (BBO) is concerned with the problem of learning the optimum/optimizer of an objective function $J: \\\\mathbf{x} \\\\to y$ without explicit access to the function $J$ but rather only implicit knowledge of the objective through a fixed sample of input-output pairs. This limitation of one's knowledge of the objective to a finite sample of data poses a significant challenge to the optimization task. \n\nMost approaches to this problem adopt either a *forward* approach (essentially learning some representation of $p(y| \\\\mathbf{x})$, such as through regression) or an *inverse* approach (essentially learning $p(\\\\mathbf{x} | y)$, which is typically a one-to-many mapping). Each approach has its advantages and limitations. This paper proposes Robust Guided Diffusion (RGD), a method for offline BBO that combines elements of both the forward and inverse approaches to leverage their advantages while mitigating their drawbacks. A large focus of the proposed method is on the use of *guidance* in its inverse component by optimizing the guidance weight during the diffusion sampling process. The paper's experimental results show performance matching the state of the art on a number of BBO benchmark tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* The paper would greatly benefit from an additional editorial pass. In addition to the expository clarity issues mentioned above, there are several passages in which words or punctuation are omitted, and Figure 2 seems to have an incompletely written caption.\n* In Algorithm 1, line 4 would read better as \"Identify adversarial examples $\\\\hat{\\\\mathbf{x}}$ via Eq. 3.\"\n* Two papers [1, 2] come to mind that explore different strategies for adjusting guidance during diffusion sampling, which I believe could be relevant related work, despite their focus on image models. The authors should consider the relevance of this work but should not feel obligated to include it.\n* I appreciate the authors' inclusion of the computational overhead involved. I am also sympathetic to the authors' statement that \"the time differences between methods for deriving high-performance designs are less critical in actual production environments.\" Nevertheless, it would be interesting to see how RGD's roughly one-hour requirement for these experiments compares with the time requirements of the competing methods.\n\n[1] Sadat, Seyedmorteza, et al. \"CADS: Unleashing the diversity of diffusion models through condition-annealed sampling.\" arXiv preprint arXiv:2310.17347 (2023).\n\n[2] Kynkäänniemi, Tuomas, et al. \"Applying guidance in a limited interval improves sample and distribution quality in diffusion models.\" arXiv preprint arXiv:2404.07724 (2024).", "broader_impact_concerns": "I do not have any ethical concerns about this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper describes a new sampling scheme for conditional sampling for offline black-box optimization. The claims of the paper are well supported by experiments. Reviewers pointed out some clarifications issues in the first version, which however were resolved during the rebuttal.
Decision: Accept (poster)