Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization
Authors: Taeyoung Yun
Sujin Yun
Jaewoo Lee
Jinkyoo Park
Venue: NeurIPS 2024 poster (2024)
Keywords: Offline Model-based Optimization
Diffusion Models
Decision Making
Primary area: generative_models
TL;DR: Generate trajectories towards high-scoring regions for solving offline model-based optimization problems

Abstract: Optimizing complex and high-dimensional black-box functions is ubiquitous in science and engineering fields. Unfortunately, the online evaluation of these functions is restricted due to time and safety constraints in most cases. In offline model-based optimization (MBO), we aim to find a design that maximizes the target function using only a pre-existing offline dataset. While prior methods consider forward or inverse approaches to address the problem, these approaches are limited by conservatism and the difficulty of learning highly multi-modal mappings. Recently, there has been an emerging paradigm of learning to improve solutions with synthetic trajectories constructed from the offline dataset. In this paper, we introduce a novel conditional generative modeling approach to produce trajectories toward high-scoring regions. First, we construct synthetic trajectories toward high-scoring regions using the dataset while injecting locality bias for consistent improvement directions. Then, we train a conditional diffusion model to generate trajectories conditioned on their scores. Lastly, we sample multiple trajectories from the trained model with guidance to explore high-scoring regions beyond the dataset and select high-fidelity designs among generated trajectories with the proxy function. Extensive experiment results demonstrate that our method outperforms competitive baselines on Design-Bench and its practical variants. The code is publicly available in \url{https://github.com/dbsxodud-11/GTG}.

Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimization
Taeyoung Yun' Sujin Yun' Jaewoo Lee' Jinkyoo Park
Korea Advanced Institute of Science and Technology (KAIST) *Omelet.ai (99yty, yunsj0625, jaewoo, jinkyoo. (aakyoo.park) @kaist. ac. ac.kr
Abstract
Optimizing complex and high-dimensional black-box functions is ubiquitous in science and engineering fields. Unfortunately, the online evaluation these functions is restricted due to time and safety constraints in most cases. In offline model-based optimization (MBO), we aim to find a design that maximizes the target function using only pre-existing offline dataset. While prior methods consider forward or inverse approaches to address the problem, these approaches are limited by conservatism and the difficulty of learning highly multi-modal mappings. Recently, there has been an emerging paradigm learning to improve solutions with synthetic trajectories constructed from the offline dataset. In this paper, we introduce a novel conditional generative modeling approach to produce trajectories toward high-scoring regions. First, we construct synthetic trajectories toward high-scoring regions using the dataset while injecting locality bias for consistent improvement directions. Then, we train conditional diffusion model to generate trajectories conditioned on their scores. Lastly, we sample multiple trajectories from the trained model with guidance explore high-scoring regions beyond the dataset and select high-fidelit designs among generated trajectories with the proxy function. Extensive experiment results demonstrate that our method outperforms competitive baselines on Design- Bench and its practical variants. The code is publicly available https:://// //tg//gtttt//tt//////// com /bpp/dddxdddddddo 11/GTG
Introduction
Optimizing complex and high-dimensional black-box functions ubiquitous science and engineer- ing fields, including biological sequence design []] materials discovery 22, and mechanical design Traditional methods like Bayesian optimization have been developed solve the problem by iteratively querying blxck-box function. However, the online evaluation of the black-bon function is restricted in most real-world situations due time and safety constraints.
Fortunately, we often have access to previously collected offline dataset. This problem setting referred offline model-based optimization (MBO), and objective is find design that maximizes a target function using solely an offline dataset (5). As no online evaluation is available, key challenge of MBO the out-t--distiitttion (OOD) issue arising from limited data coverage. Suppose we train proxy that predicts function values given input designs and naively apply a gradient-basee optimizer based the proxy to identify the optimal design. It would fall into sub-optimal results due to inaccurate predictions of the proxy in unseen regions.
To mitigate this issue, forward approaches mostly consider training robust surrogate model against adversarial optimization inputs and applying gradient- madde maximization. Trabucco et al. [6] train proxy with the regularization term to prevent overestimation OOD designs. Fu and Levine leverage normalized maximum likelihood estimator to handle uncertainty on unseen regions.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

There are also several works that focus on fine-tuning the proxy for robustness on unexplored regions 8 9 [0]. However, the generalization of the proxy outside of the dataset still remains challenging.
On the other hand, inverse approaches learn a mapping from function values to the input domain. Then, they generate high-scoring designs by querying the learned mapping with high score. Prior approaches utilize expressive generative models to learn a mapping, such as variational autoencoders [2]. generative adversarial nets [3], autoregressive models [4] diffusion models [T5] While these methods show promising results, they still suffer from the difficulty of learning highly unsmooth distributions and utilizing valuable information about the landscape of the black-box function.
Recently, new perspective has emerged on tackling the MBO by learning to improve solutions with synthetic trajectories constructed from the dataset [6 17]. These methods aim to generate sequence designs toward high-scoring regions. It seems more promising than learning an inverse mapping that generates only single design, that can help better understand the landscape the target function. However, there still room for improvement in this perspective, First, prior approaches construct trajectories with simple heuristics, which may lead to generating trajectories with inconsistent directions of improvement. Furthermore, the sequential nature of autoregressive models may lead to error accumulation during sampling [
utilize information from sequences of designs
To this end, we propose a novel conditional generative modeling approach to solve the MBO problem. Unlike prior inverse approaches, which generate single design, we generate sequence of designs toward high-scoring regions with guided sampling. Our method consists of four stages. First, we construct trajectories from the dataset while incorporating locality bias to distill the knowledge of the landscape of the target function into the generator. Then, we train a conditional diffusion model that generates the whole trajectory at once bypass training, we sample multiple trajectories conditioned on context data points and high score values. Finally, we select high-fidelity designs among generated trajectories by filtering with the proxy.
accumulation and auxiliary proxy. After
We empirically demonstrate that our method achieves superior performance on Design-Bench, well-known benchmark for MBO with variety of real-world tasks. Furthermore, explore more practical settings, such as sparse or noisy datasets, verifying the generalizability of our method.
Preliminaries
2.1 Problem setup
In offline model-based optimization (MBO), we aim to find a design x that maximizes the target black-box function f. Unlike the typical black-box optimization setting, dataset D, and online evaluations are unavailable. The problem setup can be described as follows:
only access an offline
find x arg max f(x) s.t only an offline dataset DD {(x,, ) is given xER
(1)
where x is decision variable and y fx) is a target property we want to maximize.
2.2 Diffusion probabilistic models
Diffusion probabilistic models 19 20
class of generative models that approximate the true distribution Go with a parametrized model of the form: po = ] ) where Zo ~ go and ... , .I latents with the same dimensionality. The joint distribution pe (xF.T) is called the reverse process, defined as a Markov chain starting from standard Gaussian (r( = (0,
T
Po(Zp:r) = (x( II
Po po(2x-1112) Po(x)-1/z.) = ((H
(2)
where (o parametrized Gaussian transition from timestep & 1.
We define a forward process, which also fixed as a Markov chain that adds Gaussian noise to the data with the variance schedule .,, ........... .rr
T
(( = II (x-1). g(zr(o,-1) = NNV1 1 B, 3,1)
(3)

---

Trajectory
Step 3: Sampling from the Diffusion Model
Step 4: Select Candidatas
Conttrutioon
Plrs
Aggrepate
Posst
(x, p)
Poyd
Content Conditioinn
Conditional Diftuiion
Step Training
Prony f
Tokking
Ry"
Po(ti)())
Clysilee fiee Cuidence
Top Q Fiterling
Canditional Offuaias
fe
(e(+((()))
N Times
Figure I: Overview of our method. Step 1: Construct trajectories from the dataset. Step 2: Train diffusion model and proxy. Step 3: Sample trajectories from the diffusion model with classifier-free guidance and context conditioning. Step 4: Select candidates for evaluation by filtering with proxy.
Training diffusion models can be performed by maximizing the variational lower bound on the log-likelihood Eg [log po (2o)]. which equivalent minimizing the following loss:
(0)) (tto:-ggg.tt(((.1.)........ [lle - c t)]]3]
where ( 1) is the parameterization suggested by 20 p0( 1)
va. (1 Vi-a. (T 0).
For modeling conditional distribution (<>ly), we can use classifier free guidance (2]] In classifier- free guidance, we train both conditional eg (x y, t) and unconditional model cn t) with the following loss:
[()) E
ca [lle-ca(ze,
BBy + {)
(5)
For sampling, we start from Gaussian noise zr and refine zt into rt-1 with the perturbed noise from the learned model at each diffusion timestep t:
Ett) = <I 0, t) + weee(ar v, - c0 0,t))
(6)
where w is scalar value that controls the guidance scale.
Methodology
In this section, we introduce GTG, Guided Trajectory Generation, conditional generative modeling approach for solving MBO problem by learning to improve solutions using the offline dataset. We first construct trajectories towards high-scoring regions while incorporating locality bias for consistent improvement directions. Then, we train the conditional diffusion model to generate trajectories and a proxy model. Finally, sample multiple trajectories using the diffusion model with guided sampling and filter high-fidelity designs with the proxy. Figure shows the overview of the proposed method.
3.1 Constructing trajectories
We construct set of trajectories Draj from the offline dataset D to gather information on learning to improve designs. In this paper, each trajectory T E Diraj set of input-output pairs and can be represented two-dimensional array:
y/ ]. (xh,, vn.) E vh - 1.... H
!/
y/2
While prior works construct trajectories via sorting heuristics or sampling from high-scoring regions, we focus constructing trajectories that give more valuable information for learning to improve designs towards higher scores. To achieve this, we develop a novel method to construct trajectories based two desiderata.
First, the trajectory should be towards high-scoring regions while containing information on landscape of the target black-box function. Second, the trajectories should be diverse and not converge single data point with the highest score of the dataset, as our objective is to discover high-scoring designs beyond the offline dataset by generalizing the knowledge of learning to improve solutions.

---

Algorithm/ Trajectory construction procedure of GTG
Input: Offline dataset D, Trajectory length H, Number of trajectories N, initial percentile p, number of nearest neighbors K, and perturbation coefficient
Output: Duaj
1: Initialize trajectory dataset Prrj
2: for n== 1, .N do
3: Sample (x1, V1) from yth percentile of D and initialize trajectory < {(x1, y1}}
4: for h = 1,... H do
5:
Find K nearest neighbors of x whose score is higher than max [81. ... , .yhk 6: Sample (x from the K neighbors and update T f- T U {xx++1, 31++1)
7: end for
8: Update Dinj+ < Dunj (T}
9:
end for
To this end, we introduce a novel strategy to construct trajectories from the dataset. We illustrate the procedure in Algorithm I For each trajectory, we first sample an initial data point (x1,)) y1) , from relatively low score distribution, pth percentile of D. After initialization, we employ local search strategy to select the next data point generate smooth trajectory toward high-scoring regions that contain the information on the landscape of the target function. Specifically, for each round h, we find K nearest neighbors of X,, whose score is higher than max( small, non-negative real number. By allowing small perturbations using c, we can prevent generated trajectories from converging single maximum of the offline dataset. Then, we sample (xn, yn. from the K neighbors randomly to generate diverse trajectories. We repeat the procedure until constructing a trajectory of length H. By moving towards high-scoring regions while staying in local region, we can effectively guide the generator to learn diverse and consistent paths for improving solutions.
2h - c, where is
Note that identifying K nearest neighbors of a data point whose values are above certain threshold does not require substantial computational time compared to training and evaluation. We explain in more detail our trajectory construction procedure in Appendix B.I
3.2 Training models
Given our trajectory dataset D Duaj. our objective is to learn the conditional distribution of trajectories towards high-scoring regions. We choose diffusion models, which have a powerful capability to learn the distribution of complex and high-dimensional data [22 23). to generate trajectories. Our objective then transformed from searching high-scoring designs to maximizing the conditional likelihood of trajectories, which can be achieved by minimizing the loss in Equation (5):
0" = arg max E,wDus [logPo po (TT))
(8)
where = Yh. is the sum of scores in the trajectory т. By training diffusion model to efficiently distill the knowledge of
generate sequence of designs instead of single design, the complex landscape of the target function into the diffusion model.
In addition, also train forward proxy fe using the dataset D. We can use the proxy to filter high-scoring designs from the trajectories generated by the trained diffusion model.
3.3 Sampling trajectories from the diffusion model
After training, sample trajectories with guided sampling. We classifier-free guidance to generate trajectories. To be specific, we sample t from the diffusion model using Equation (6), where y" (T) is the target conditioning value. Following prior works [13 [6], we assume that we know the maximum score y" and set (T( = a (Hy"), ), where a controls the exploration level of the generated trajectories. We discuss the role of in
detail in the subsequent section.
To fully utilize the expressive power diffusion models, we introduce an additional strategy, context conditioning, during the sampling. We generate trajectory with diffusion model while inpainting the C' context data points of the trajectory with Tax. which a subtrajectory sampled from Dinj- By conditioning trajectories in different contexts, can effectively explore diverse high-scoring regions.

---

Algorithm 2 Sampling procedure of GTG
Input: Offline dataset D, Trajectory dataset Dmj, Conditional diffusion model po. Proxy model fø. Context length C. Trajectory length H, Evaluation budget Q-
Output: Deand Initialize cand 0
2: for n = 1, 1. N do
3: Initialize (TT) (0, I) and Tax Dinj
4: for t= T, ....................... do
5:
Compute (tt) using Po(Tly" (T)) by Equation 6
b Classifier-free Guidance Context- Coniiitonnniiinnnn
6: Compute ((t-1) using Tax. é(I) and ( by Equation 9
7: end for
8: Update Deand < Deand (xC+1," xx) from T== (0))
9:
end for
10: Set Doand as top-Q scoring samples filtering by fs
Filtering
110
-10
-6
-16
(b)
Figure 2: (a) Trajectories constructed by BONET (blue) and PGS (green). (b) Diverse trajectories constructed by GTG (red). (c) Trajectories generated by trained diffusion model with guided sampling. Red dots indicate context data points, and blue dots represent generated data points.
Formally, for each denoising timestep i, we refine (() into T (t-1) with the following procedure:
3t
(t-1)
O + m)
T
at E(t)
Var
where is the mask for the first context data points and é(t) is computed from the Equation (6)
3.4 Selecting candidates
After generating trajectories, we introduce filtering to select candidates for evaluation. In other words, we select top-Q samples in terms of the predicted score from the proxy. By filtering with the proxy, we can exploit the knowledge from the dataset to search high-scoring designs
Experimental evaluation
In this section, we present the results of our experiments various tasks. First, we analyze our method in a toy 2D experiment. Then, we present the results on the Design-Bewt Bench and its practical variants to verify the effectiveness of the method. We also conduct extensive analyses on various aspects to deepen our understanding of the proposed method.
4.1 Toy 2D experiment
We first evaluate our method using toy setting to analyze each component of our method thoroughly. We choose Branin, synthetic 2D function with three distinct global maxima. Figure 2 shows the contour plot of the Branin function. The analytical form of the Branin function is as follows:
f(11,12) -a (x22 - ba? cr r)² cos(21))
(10)
5

---

Table Experiments on Design-Bench Tasks. We report max score (100 percentile) among Q=128 candidates. Blue denotes the best entry in the column, and Violet denotes the second best.
Method
TFBind8
TFBind10
Superconductor
Ant
Kitty
Mean Rank
D (best)
0.439
0.467
0.399
0.565
0.884
BO-QEI
0.794 0.103
0.631 0.041
0.486 = 0.025
0.812 a: 0.000
0.89 0.000
11.4/55
CMA-ES
0.919 0.055
0.649
0.020
0.478 = 0.010
2.222 1.550
0.724 0.001
8.6/ 15
REINFORCE
0.947 0.029
0.628 # 0.025
0.485 = 0.011
0.244 0.031
0.558 0.193
11.6/15
Grud Ascent
0.983 0.015
0.648 = 0.044
0.509 0.018
0.295 = 0.021
0.877 0.023
7,8/ 15
COMs
0,968
0.619 = 0.038
0,444 = 0.035
0.927 0.043
0.957 0.016
8,2/
NEMO
0
0.705 0.000
0.502 0.002
0.952 0.002
0.950 0.001
4.8
RoMA
0,924 =
0,666 0.035
0.510 0.015
0.030
0.927 0.013
7/
BDI
0.000
0.630 0.025
0.508 0.011
0.000
0.939 0.000
6.6 15
ICT
0.944 0.015
0.598 0.020
0.507 0.014
0,946 0.021
0.960 0.014
7/
ChAS
0,895 0.043
0.638 = 0.040
0.468 = 0.058
0.825 0.030
0.898 0.011
11.2/15
MINs
0.884 0.039
0.660 = 0.048
0.500 = 0.036
0.908 0.031
0.942 0.005
8 15
DDOM
1.966=0000
0.666 = 0.024
0,476 = 0.029
0.926
0.948 0.011
7/15
BONET
0.831 = 0.199
0.606 # 0.044
0.405 = 0.017
0.957 = 0.004
0.950 = 0.014
10/
PGS
0.968 0.019
0.693 = 0.031
0.475 = 0.048
0.748 = 0.049
0.948 0.014
7.6/15 15
GTG (Ours)
0.976 0.020
0.698 0.127
0.519 0.045
0.963 0.009
0.971 0.009
1.6 15
where a = 1, 4 c == 10, / -= 87
and the range of (11, 222 is [-5, 5, 10) x [o, 15]-
For the MBO setting, uniformly sample 5000 data points and remove the top 10% percentile to make the task more challenging. We construct trajectories with a length of 64 using our trajectory construction strategy and other strategies suggested by prior methods, BONET [16] and PGS [17].
Figure 2a shows the trajectories generated from prior methods. As shown the figure, we find that constructed trajectories show uncorrelated movements, which makes the model hard to capture knowledge on the landscape of the target black-box function. Unlike prior methods, our method constructs trajectories that improve the solution with the local movements, as illustrated in Figure 2b Such trajectories help the diffusion model to learn how improve solutions efficiently. We also find that trajectories do not converge into a single data point and toward diverse high-scoring regions via random sampling from K neighbors and perturbations from e.
Figure 2c shows the trajectories generated by the trained diffusion model with context conditioning and classifie--free guidance. As shown in the figure, GTG can generalize the knowledge on improving solutions to find diverse high-scoring solutions. GTG achieves a maximum score of -0.490 + 0.070, which is near-optimal compared the global optimum (-03398) and far beyond the maximum value of the dataset (-6.031). Please refer to Appendix A.1 for more details of the toy experiment.
4.2 Design-Bench tasks
In this section, we present the experiment results of our method on Design-Bende tasks [L. We conduct experiments on two discrete tasks and three continuous tasks. For each task, we have an offline dataset from an unknown oracle function. We present the detailed task description below.
TFBind8 and TFBind10 (T]. We aim to find a DNA sequence of lengths and 10 with maximum binding affinity with particular transcription factor.
Superconductor (2]. We aim to design chemical formula, represented by 86-dimensional vector, for a superconducting material with a high critical temperature.
Ant and D'Kitty Morphology 44, 255. We aim to optimize the morphological structure of two simulated robots. The morphology parameters include size, orientation, and the location of the limbs. Ant has 60 continuous parameters, and D' Kitty has 56 continuous parameters.
4.3 Baselines
For baselines, we prepare four main categories to solve MBO problems. First, we compare our method with traditional methods widely used in online black-box optimization settings, such as BO-qEI (26). CMA-ES (27). REINFORCE [28] and Gradient Ascent.

---

Table 2: Experiments Sparse Datasets.
TFBind8
Dkitty
Method
1%
20%
50%
1%
20%
50%
BDI
0.898 + 0.000
0.952 + 0,000
0.988 # 0.000
0.865 =: 0.000
0,927 + 0.000
0.938 = 0.000
ICT
0.899 = 0.045
0.925 + 0.035
0.962 + 0.019
0,946 = 0.010
0,949 + 0.010
0.954
DDOM
0.851 #: 0.082
0.906 0.050
0.896 4: 0.048
0.938 #: 0.007
0,945 + 0.011
0.944 = 0.008
BONET
0.791 4: 0.079
0.824 a: 0.061
0.884 à 0.072
0.875 a: 0.004
0,939 +: 0.007
0.940 +: 0.009
GTG (Ours)
0.948 0.009
0.964 = 0.025
0.973 = 0.016
0.949 + 0.013
0.957 = 0.009
0.968 + 0.002
Table 3: Experiments on Noisy Datasets.
TFBind8
Dkitty
Method
1%
20%
50%
1%
20%
50%
BDI
0,980 = 0.005
0.886 + 0.051
0.873 + 0.048
0.929 = 0.008
0.908 = 0.010
0.918 0.016
ICT
0.941 # 0.013
0.950 + 0.023
0.921 + 0.054
0.940 = 0.029
0.914 = 0.024
0.896 = 0.000
DDOM
0.896 = 0.048
0.887 # 0.065
0.887 = 0.065
0.944 0.009
0.945 = 0.011
0,926 = 0.020
BONET
0.904 = 0.044
0.822 # 0.113
0.773 0.143
0.942 = 0.008
0.927 = 0.024
=.924
GTG (Ours)
0.976 0.015
0.967 0.026
0.948 0.029
0.955 0.008
0.947 0.015
0.937 0.013
The second category comprises recently proposed forward approaches, including COMs 6. NEMO 77.. RoMA [8]. BDI (24), and ICT [9]. The third category encompasses inverse approaches, and we select CbAS MT]. MINs [3], and DDOM [15] as our baselines. Finally, we also compare with baselines which construct synthetic trajectories and generalize the knowledge of learning to improve solutions, BONET [16 and PGS
4.4 Evaluation metries
For evaluation, we follow the protocol of prior works. We identify Q = 128 designs selected by the algorithm and report normalized score of 100 percentile design. For all algorithms, we run experiments over 8 different seeds and report mean and standard errors.
To evaluate our method, construct trajectories of length H 64 and train conditional diffusion model for each task. After training, we sample N = 128 trajectories conditioning on = 32 context data points and setting a 0.8 across all tasks. Finally, we filter top-128 candidates among generated designs with the predicted score from the proxy for evaluation.
4.5
Main results
As shown in the Table GTG achieves average rank of 1.6, the best among all competitive baselines. It performs best on two tasks and is runner-up on three tasks, demonstrating superior performance across different tasks. We observe that GTG generally surpasses forward approaches, which struggle to fall into OOD designs, especially in high-dimensional settings. We also observe that method outperforms inverse approaches, including DDOM, which also utilizes diffusion model. It demonstrates that generating trajectories towards high-scoring regions can be more effective than generating single design, as we can distill the knowledge of the landscape of the target function into the generator. Our method achieves higher performance compared to BONET, which also generates trajectories. It indicates that our novel trajectory construction strategy effectively guides the diffusion model to explore diverse paths toward high-scoring regions.
4.6 Practical variants of Design-Bench tasks
In this section, we present experiment results in more practical setting of Design-Benee tasks. While Design-Bench Bench assumes large, unbiased offline dataset containing thousands of data points for the training model, such a setting is impractical in most cases. Therefore, we prepare two additional practical settings, sparse and noisy datasets, to verify the robustness of our method in such extreme cases. In a sparse setting, we only provide 2% of the original dataset for training. For the noisy setting, we add 1% of standard Gaussian noise to the normalized score values. We choose recent papers published after 2022, BDI, ICT, DDOM, and BONET for primary baselines. Please refer to Appendix A.2 for detailed experiment settings and Appendix D.5 for results with more baselines.
7

---

Table 4: Ablation study trajectory construction strategy.
Method
TFBind8
TFBind10
Superconductor
Ant
D' D'Kiuy
SORT-SAMPLE
0.954 0.026
0.697 = 0.126
0.487 = 0.016
0.946 0.011
0.966 0.005
Top-p Percentile
0.948 = 0.030
0.669 + 0.033
0.439 = 0.039
0.946 0.018
0,964 0.003
Ours
0.976 = 0.020
0.698 + 0.127
0.519 = 0.045
0.963 + 0.009
0.971 = 0.009
Table 5: Ablation study on sampling procedure of GTG.
Method
TFBind8
TFBind10
Superconductor
Ant
D' D'Kitty
0
0,923 1 0.054
0.636 + 0.047
0.499 + 0.040
0.867 0.051
0.926 + 0.048
{CF)
0.914 = 0.053
0.687 + 0.065
0.502 = 0.040
0.918 + 0.064
0.943 = 0.011
CC,, CC}
0.920 = 0.036
0.687 + 0.065
0.502 0.024
0.927 0.022
0.945 = 0.014
FF.F)
0.963 + 0.019
0.628 + 0.036
0.483 + 0.034
0.952 0.026
0.965 + 0.007
CCF, CC, F}
0.976 = 0.020
0.698 = 0.127 0.127
0.519 # 0.045
0.963 0.009
0.971 0.009
Table 2 shows the results of our method and recent baselines sparse datasets. The table shows that our method mostly outperforms other baselines even in sparse datasets, demonstrating the superiority of exploiting knowledge of the target function by constructing diverse trajectories from the dataset. Table 3 reports the experiment results on the noisy settings. We find that even with 50% of noise, our method can find relatively high-scoring designs, demonstrating its robustness in practical settings.
5 Additional analysis
In this section, we carefully analyze the effectiveness of each component in our method.
Ablation trajectory construction. We propose a novel trajectory construction strategy by incorporating locality bias. To verify the effectiveness of the strategy, we compare our strategy with prior approaches. SORT-SAMPLE and Top-p Percentile, suggested by BONET and PGS, respectively. Table shows that our strategy outperforms prior strategies across various tasks. We conduct additional analysis on trajectory construction strategies in Appendix [.]
Ablation on sampling procedure. We analyze the effectiveness of strategies we introduced during the sampling procedure, namely context conditioning (CC), classified-free guidance (CF), and filtering (F). Across various tasks, it evident that all components are crucial for improving performance as demonstrated in Table We conduct further analysis on sampling strategies in Appendix D.2]
Hyperparameter sensitivity. We also conduct experiments on the effect of various hyperparameters we introduced in this paper. We first train conditional diffusion model with various lengths (H). As shown in Figure Ba] increasing H leads to achieving higher performance. We also conduct experiments by varying the number of contexts (C) and the exploration level (a). Figure 3bb shows that = 32 achieves superior performance while conditioning with too many contexts degrades performance. Finally, Figure[3]shhows strong correlation between and the score, demonstrating the effectiveness of guided sampling. We conduct further analysis on hyperparameters in Appendix [D.2]
Varying evaluation budget. We provide experiment results with a small number of evaluation budgets (Q). As shown in Figure we generally outperform most baselines even with a relatively low evaluation budget.
Assumption on optimal score. We assume that the optimal value y* is known, following prior works [13 [6]. We conduct experiments by relaxing the aforementioned assumption in Appendix D.2 and find that GTG can achieve comparable performance even without knowing y"
Effect of unsupervised pretraining. It might be beneficial to pretrain the diffusion model when we have large-scale unlabeled dataset and few designs of labeled points [29]. To this end, discuss the effectiveness of pretraining diffusion models with unlabeled datasets Appendix [.3]
Time complexity of sampling procedure. We also conduct analysis on the time complexity of the sampling procedure of our method in Appendix [D.4] Experiment results demonstrate that we can decrease the number of denoising timesteps even one-tenth with minimal loss in performance.
8

---

1.3
0115
1:56
Ngmaloon Soore
Nornailos
Normioon
Mix
Max
03
Manlion
124
13
044 0.0
0.7
Trgeetuyy Length 127)
Contest Length (C)
Explecation Level (6)
(a) Ablation on H
(b) Ablation on
(c) Ablation on or
Figure 3: Ablation on hyperparameters of GTG. Experiments conducted on D'Kitty task.
Supereondnaeoor
Kitty
0.50
0.50
0.54
Normsllont soaaldedd Score
llnddlldddd Store
0.40
0.00
0.40
Narmaltzod
80:00
2.00
0.88
0.58
0.54
0.2
0.70
0.00
Exatustion
Badggei (Q2)
Evechad (uig
Exaluntion Bodged((2)
GTG
DOOM
RONET
C5A8
MTN
COMs
PG5
RDI
TCT
Figure 4: Ablation on varying evaluation budget Q.
6 Related works
6.1
Offline model-based optimization
In offline MBO, generalization outside the offline dataset is crucial for success. While there have been attempts to train robust surrogate model to achieve accurate predictions on unseen regions g 10 effectively exploring high-scoring regions remains challenging.
Recently, a new perspective on solving the MBO problem has emerged by learning to improve solutions from synthetic trajectories and generalizing the knowledge to find designs beyond the dataset 17] BONET [16] trains an autoregressive model to generate optimal trajectories conditioned on low regret budget. PGS [17] trains RL policy with trajectories consisting of high-scoring designs to roll out optimal trajectories. MATCH-OP (30) also constructs monotonic trajectories and matches the gradient field with the proxy. GTG falls under this category but adopts unique approach to constructing trajectories with local search and utilizing diffusion models to enhance performance.
Generative models for decision making
Generative models have emerged as powerful tool for decision-making problems, including bandit
problems 31, reinforcement learning [18 32
33
14 35), and optimization [15] (6). offline
MBO, there are inverse approaches to learning mapping from function values to input domains
with generative models and sample designs from high-scoring regions I2
14 [5]. DDOM
15] utilizes conditional diffusion model and generates high-scoring samples with reweighted training and classifier-fres guidance. DiffOPT (36] considers a constrained optimization setting and introduces two-stage framework that begins with guided diffusion process for warm-up. followed by a Langevin dynamics stage for further correction.
As concurrent works, DEMO 37] trains a diffusion model to match a pseudo-target target distribution constructed by gradient ascent and uses the model to edit designs in the offline dataset. Diff-BBO 38 measures the uncertainty of generated designs to select the optimal target value for conditioning the diffusion model. Our method distinguishes itself from prior works by utilizing diffusion models to generate trajectories toward high-scoring regions by learning improve solutions from the dataset.

---

7 Discussion and conclusion
In this paper, we introduce GTG, novel conditional generative modeling approach for learning to improve solutions from synthetic trajectories constructed with the dataset. First, we construct diverse trajectories toward nigh-scoring regions while incorporating locality bias. Then, we train the conditional diffusion model and proxy function. After training, we generate trajectories with classifier-free guidance and nonexx-conndtioonng to generalize the knowledge on how to improve solutions. Lastly, our filtering strategy for selecting candidates further improves the performance. Our extensive experiments demonstrate the generalizability of GTG.
Limitation and future work. While our method shows powerful generalizability on Design-Bench tasks, some evaluation methods may fully capture real-world complexities. For example, in the superconductor task, we find that the offline dataset has multiple copies of the same inputs but with different outputs. As result, the random forest oracle which fit this offline data not reliable. Moreover, we resort to filtering designs with the proxy function, which may result in inaccurate predictions on OOD regions. Although our filtering strategy works well in sparse and noisy settings, one may consider constructing robust proxy model to handle the uncertainty of its predictions.
Acknowledgements
We thank the anonymous reviewers for their insightful comments and suggestions which signifi- cantly improve our manuscript. This work was supported by the National Research Foundation of Korea(NRFF) grant funded by the Korea (SV)) (No. RS-2024-0041008)..
10

---

References
[1] Luis A Barrera, Anastasia Vedenko, Jesse V Kurland, Julia M Rogers, Stephen $ Gisselbrecht, Elizabeth J Rossin, Jaie Woodard, Luca Mariani, Kian Hong Kock, Sachi Inukai, et al. Survey of variation in human transcription factors reveals prevalent dna binding changes. Science, 2016.
[2] Kam Hamidich. data-driven statistical model for predicting the critical temperature of superconductor. Computational Materials Science, 2018.
[3] Felix Berkenkamp, Angela P Schoellig, and Andreas Krause. Safe controller optimization for quadrotors with gaussian processes. In International conference robotics and automation (ICRA), 2016.
[4] Thomas Liao, Grant Wang, Brian Yang. Rene Lee, Kristofer Pister, Sergey Levine, and Roberto Calandra. Data-efficient learning morphology and controller for microrobot. In International Conference on Robotics and Automation (ICRA), 2019.
[5] Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-bench: Bench- marks for data-driven offline model-based optimization. In International Conference on Machine Learning (ICML), 2022.
[6] Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In International Conference on Machine Learning (ICML), 2021.
[7] Justin Fu and Sergey Levine. Offline model-based optimization via normalized maximum likelihood estimation. In International Conference on Learning Representations (ICLR), 2021.
[8] Sihyun Yu, Sungsoo Ahn, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for offline model-based optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[9] Ye Yuan, Can Sam Chen, Zixuan Liu, Willie Neiswanger, and Xue Steve Liu. Importance- aware co-teaching for offline model-based optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[10] Can Sam Chen, Christopher Beckham, Zixuan Liu, Xue Steve Liu, and Chris Pal. Parallel- mentoring for offline model-based optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[11] David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International Conference on Machine Learning (ICML), 2019.
[12] Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model-based design. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[13) Aviral Kumar and Sergey Levine. Model inversion networks for model-bases optimization. In Advances Neural Information Processing Systems (NeurIPS), 2020.
[14] Minsu Kim, Federico Berto, Sungsoo Ahn, and Jinkyoo Park. Bootstrapped training of score- conditioned generator for offline design of biological sequences. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[15] Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black-box optimization. In International Conference on Machine Learning (ICML), 2023.
[16] Satvik Mehul Mashkaria, Siddarth Krishnamoorthy, and Aditya Grover. Generative pretraining for black-box optimization. In International Conference on Machine Learning (ICML), 2023.
[17] Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, and Janardhan Rao Doppa. Offline model-based optimization via policy-guided gradient search. Proceedings of the AAAI Conference Artificial Intelligence (AAAI), 2024.

---

[18] Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning (ICML), 2022.
[19] Jascha Sohl-Dick Dicksiicsttin, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning (ICML), 2015.
[20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[21] Jonathan Ho and Tim Salimans. Classifier-Iree diffiusion guidance. In NeurIPS Workshop on Deep Generative Models and Downstream Applications, 2021.
[22] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 000 1(2):3 2022.
[23] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao. Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint wriiv2210.02300 2022.
[24] Can Chen, Yingxueff Zhang, Jie Fu, Xue Steve Liu, and Mark Coates. Bidirectional learning for offline mafin model-based optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[25] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv: 160600/60000 2016.
[26] James T Wilson, Riccardo Moriconi, Frank Hutter, and Marc Peter Deisenroth. The reparame- terization trick for acquisition functions. arXiv preprint arXiv:1712.004224, 2017.
[27] Nikolaus Hansen. The cma evolution strategy: a comparing review. Towards a new evolutionary computation: Advances in the estimation of distribution algorithms, pages 75-102, 2006.
[28] Ronald J Williams. Simple statistical saadient-folloingg algorithms for connectionist reinforce- ment learning. Machine learning, 8:229-256 1992.
[29] Tung Nguyen, Sudhanshu Agrawal, and Aditya Grover. Expt: Synthetic pretraining for few-shot experimental design. In Advances in Neurul Information Processing Systems (NeurIPS), 2023.
[30] Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, and Trong Nghia Hoang. Learning surrogates for offline black-box optimization via gradient matching. In International Conference on Machine Learning (ICML), 2024.
[31] Yu-Guan Hsieh, Shiva Prasad Kasiviswanathan, Branislav Kveton, and Patrick Blöbaum. Thompson sampling with diffusion generative prior. In International Conference on Machine Learning (ICML), 2023.
[32] Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B Tenenbaum, Tommi S Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision making? International Conference Learning Representations (ICLR), 2023.
[33] Zhixuan Liang, Yao Mu, Mingyu Ding, Fei Ni, Masayoshi Tomizuka, and Ping Luo. Adaptd- iffuser: Diffusion models as adaptive self-evolving planners. In International Conference on Machine Learning (ICML), 2023.
[34] Haoran He, Chenjia Bai, Kang Xu, Zhuoran Yang, Weinan Zhang, Dong Wang. Bin Zhao, and Xuelong Li. Diffusion model is an effective planner and data synthesizer for multi-task reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS). 2023.
[35] Jaewoo Lee, Sujin Yun, Taeyoung Yun, and Jinkyoo Park. Gta: Generative trajectory augmen- tation with guidance for offline reinforcement learning. In Advances Neural Information Processing Systems (NeurIPS), 2024.
12

---

[36] Lingkai Kong, Yuanqi Du, Wenhao Mu, Kirill Neklyudov, Valentin De Bortol, Haorui Wang, Dongxia Wu, Aaron Ferber, Yi-An Ma, Carla P Gomes, et al. Diffusson models as constrained samplers for optimization with unknown constraints. arXiv preprini arXiv: 18012, 2024.
[37] Ye Yuan, Youyuan Zhang, Can Chen, Haolun Wu, Zixuan Li, Jianmo Li, James J Clark, and Xue Liu. Design editing for offline model-basee optimization. arXiv preprint arXiv22405.1396 2024.
[38] Dongxia Wu, Nikki Lijing Kuang, Ruijia Niu, Yi-An Ma, and Rose Yu. Diff-bbo: Diffusion- based inverse modeling for black-box optimization. arXiv preprint arXiv:2407.006// 2024.
[39] Anna Gaulton, Louisa J Bellis, A Patricia Bento. Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, al. Chembl: large-scale bioactivity database for drug discovery. Nucleic acids research, 40(D1):D1100- D1107, 2012.
[40] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Interna- tional Conference on Learning Representations (ICLR), 2015.
[41] Jacob Austin, Daniel D Johnson, Jonathan Ho. Daniel Tarlow, and Rianne Van Den Berg. Struc- tured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[42] Nate Gruver, Samuel Stanton, Nathan Frey, Tim GJ Rudner, Isidro Hotzel, Julien Lafrance- Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G Wilson. Protein design with guided discrete diffusion. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[43] Austin Tripp, Erik Daxberger, and José Miguel Hernández- Lobatto Sample-efficient optimiza- tion in the latent space of deep generative models via weighted retraining. In Advunces in Neural Information Processing Systems (NeurIPS), 2020.
[44] Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[45] Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[46] Christopher Williams and Carl Rasmussen. Gaussian processes for regression. In Advances in Neural Information Processing Systems (NeurIPS), 1995.
[47] Zecheng Wang, Che Wang, Zixuan Dong, and Keith W Ross. Pre-training with synthetic data helps offline reinforcement learning. In International Conference on Learning Representations (ICLR), 2024.
[48] Zi Wang. George E Dahl, Kevin Swersky, Chansoo Lee, Zelda Marict, Zachary Nado, Justin Gilmer, Jasper Snock, and Zoubin Ghahramani. Pre-training helps bayesian optimization too. In ICML Workshop on Adaptive Experimental Design and Active Learning in the Real World, 2022.
13

---

Appendix
A
Task Details
We present additional information on Branin and Design-Bench tasks.
A.1 Toy Branin Task
Branin is well-known synthetic function for benchmarking black-bou optimization methods. It has three distinct global maxima, (-T,, 12.275). (æ,, 2.275), and (9.42478, 2.475) with maximum value of -0.398. We create synthetic offline dataset by uniform sample N 5000 data points and remove the top 10% percentile. Figure shows the visualization the dataset used for evaluation.
Contour plot of the Branin function
Offline Dataset
-35
12
70
-005
-140
-175
-10
245
280
-2 Xo
10
Figure 5: Visualization of the offline dataset used for Branin task.
We compare GTG with competitive baselines, BONET, and PGS for the Branin task. For all methods, we generate 400 trajectories with horizon 64 using construction strategies suggested by each method. with
For GTG, we train the diffusion model with a length H = 64 and apply context-conditioni C' = 32 and classificr-free guidance with a 0.8 for guided sampling. We generate four trajectories for evaluation. Table shoows the best function values achieved by each method on the Branin task. As shown in the table, GTG successfully generalizes the knowledge to improve solutions and achieve better performance compared to baselines.
Table 6: Experiment results on Branin task. We report 100th percentile among Q = 128 samples from each method. Experiments are conducted with three different random seeds.
Optima
(best)
BONET
PGS
GTG
-0.398
-6.031
-0.769 0.227
-1.295 + 0.459
-0.490 0.070
14

---

A.2 Design-Bench Tasks
Design-Bent Bench [5] is the most widely used benchmark for evaluating MBO algorithms. Table shows the details of each task. For discrete tasks, we convert discrete input into a continuous vector by approximating logits with soft interpolation between one-hot encoding and uniform distribution using mixing factor of 0.6. We present detailed statistics of each task in Table
Table 7: Detail Setting of Design-Bence Tasks.
Task
Dataset Size
Dimensions
Type
Oracle
Max
TFBind8
32898
Discrete
Exact
1.0
TFBind10
50000
10
Discrete
Exact
2.128
Superconductor
17014
86
Continuous
Random Forest
185.0
Ant
10004
60
Continuous
Exact
590.0
Dkitty
10004
56
Continuous
Exact
340.0
A.2.1 Excluded Design-Bench Tasks
Following from prior works [5 16 29 we exclude Hopper 25 and ChEMBL [39 tasks for
evaluation. As noted in previous works, the oracle for the Hopper task is heavily skewed towards low-function values and gives inconsistent results. For the ChEMBL task, all methods already produce nearly the same results, which makes it not meaningful task for evaluation.
A.2.2 Practical Variants of Design-Bench Tasks
We prepare two practical variants of Design-Bende tasks to verify the robustness of GTG in terms of data sparsity and label noise. We present the distribution of function values in the original offline dataset and its practical variants on the TFBind8 task and D' Kitty tasks. As shown the figure, the score distributions of sparse and noisy datasets significantly differ from the original ones, making the task more challenging.
11:00
Ohtgiral
Digiool
Spane (205))
(205)
IATTS
0.55
Noiey (2055)
11100
0:23
Denotty
0.115
0:31
2:11
IVC3
0008
2.0
Scorre
Soure
(a) TFBind8
(b) D'Kitty
Figure 6: Distribution of function values in the original offline dataset and its pratical variants.
15

---

B Methodology Details
In this section, we present the method details, including model implementations and architectures, training schemes, hyperparameter configurations, and computing resources.
B.1 Trajectory Construction
In terms of constructing trajectories, introduce variables, K and which control the level of locality and optimality of the trajectories. For too large value of K, we construct trajectories with inconsistent directions of improvement, while the extremely small value of K leads trajec- tories wandering the initial data point. If we lower the close to zero, we only allow monotonic improvement, while large values lead to suboptimal trajectories. We present the hyperparameters for our experiments in the Table g. We also conduct additional analysis on trajectory construction in Appendix D.I
Table 8: Hyperparameters for Trajectory Construction.
Task
p
H
N
K
e
TFBind8
20
64
1000
50
0,05
TFBind10
20
64
1000
50
0,05
Superconductor
20
64
4000
20
0.05
Ant
20
64
4000
20
0.05
Dkitty
20
64
4000
20
0.01
To identify K nearest neighbors of certain data point, we pre-compute the distance matrix between pairwise designs. For discrete tasks, we use hamming-be distance as a distance metric, and for continuous tasks, the computational time for crepoompu distance matrix and constructing trajectory dataset from the offline dataset. As shown in the table, constructing trajectories does not require significantly large amount time, even in high-dimensional settings.
Euclidean distance to measure the similarity between designs. shbls
Table 9: Time complexity of trajectory construction on Design-Bench Tasks. We use Intel Xeon@ Gold 5317 CPU @ 3.00GHz and report mean and standard deviation across five different runs.
Method
TFBind8
TFBind10
Superconductor
Ant
D' O'Kitty
Distance Matrix (sec)
5.38 0.08
14.14 2.17
7.34 0.10
1.67 1.67
1.49 0.07
Trajectory Construction (sec)
22.36 0.44
28.63 0.26
73.19 # 0.56
53.74 3.05
56.24 4.33
16

---

B.2 Training Models
B.2.1 Training Diffusion Model
We use temporal U-Net architecture from Diffuser [8] as a backbone of the diffusion model. For discrete tasks, we train the model using Adam optimizer 40 for x 10* training steps with the learning rate of 10-3 While one could use discrete diffusion models 42] for discrete tasks, we use continuous diffusion models with continuous relaxation of discrete inputs for simplicity. For continuous tasks, we train the model for 10a steps with a learning rate of 10-4, The hyperparameters we used for modeling and training are listed in Table 10
Table 10: Hyperparameters for Training Diffusion Models
Parameters
Values
Number of Layers
6
Architecture
Num Channels
32 (Discrete), 128 (Continuous)
Channel Multipliers
(1,4,8,
Batch size
128
Optimizer
Adam
Training
Learning Rate
10 (Discrete), x 10 (Continuous)
Training Steps
1 10a (Discrete), x 10 (Continuous)
Conditioning
Conditional dropout (p)
0.25
B.2.2
Training Proxy Model
We use MLP with 2 hidden layers with 1024 hidden units and ReLU activations to implement the proxy function. As our objective filtering high-fidelit designs with the proxy, we introduce a rank- based reweighting suggested by 43 during training to make the proxy model focus high-scoring regions. For discrete tasks, we train proxy model using Adam optimizer for 1 x 10 training steps
with learning rate of 1 x 10 For continuous tasks, we train the model for 5 x 103 training steps
with learning rate x 10 The hyperparameters we used for modeling and training are listed
in Table 11
Table 11: Hyperparameters for Training Proxy
Parameters
Values
Number of Layers
Architecture
Num Units
1024
Batch size
128
Optimizer
Adam
Training
Learning Rate
1 x 10-3
Training Steps
1 10 (Discrete), 10 (Continuous)
All training is done with single NVIDIA RTX 3090 GPU and takes approximately 30 minutes for discrete tasks and 2 hours for continuous tasks.
B.3 Sampling Procedure
We sample trajectories with T' 200 denoising steps across all tasks. For classifier-free guidance, we set the guidance scale w as 1.2. In practice, we sample batch of trajectories to generate multiple trajectories in parallel. We analyze the time complexity of sampling trajectories from the diffusion model in Appendix D.4
17

---

C Baseline Details
In this section, we provide more details the baselines used for our experiments.
Baselines from Design-Bench [5]. We take the implementations of most baselines from open-source code It contains baselines of BO-qEI (26), CMA-ES [27]. REINFORCE [28], Gradient Ascent,
CbAS [I]. MINs (13), and COMs 44 We reproduce the results with 8 independent random seeds.
NEMO 7 NEMO leverages normalized maximum likelihood estimator to handle uncertainty in unseen regions and prevent adversarial optimization while performing gradient ascent. As there no open-source code, refer to the results of NEMO from 9
BDI [24]. BDI learns forward mapping from low-scoring regions to high-scoring regions, and its backward mapping distills the knowledge of the offline dataset to search for optimal designs. We follow the hyperparameter setting of the paper and reproduce the results with the open-source code
ICT 19. ICT maintains three symmetric proxies and enhances the performance of the ensemble by co-teaching and importance-aware sample reweighting. We follow the hyperparameter setting of the paper and reproduce the results with the open-source code
DDOM [1]. DDOM leverages diffusion models to model distribution over high-scoring regions and sample designs with classifier-fi free guidance. We follow the hyperparameter setting the paper except for the evaluation budget Q for fair comparison. We find that there is performance drop in several tasks when we use Q 128 instead of 256. We reproduce the results with the open-source code
BONET [6]. BONET trains an autoregressive model with trajectories constructed from the offline dataset and generalizes the knowledge to explore high-scoring regions. We follow the hyperparameter setting the paper except for the evaluation budget Q for fair comparison. find that there is performance drop in several tasks when we use 128 instead of 256. We reproduce the results with the open-source code
PGS 17 PGS trains a policy to guide gradient- based optimization by reformulating the MBO problem as an offline RL problem. We follow the hyperparameter setting of the paper and reproduce the results with the open-source code
httpp://gituu..co////iii hww.h//tto...oooooooooooooooooee basclinnes
"htps://github.co https///gghuuuoooo////////e/////
hhap:://gthuu..coo//dddaattt///
18

---

D Extended Additional Analysis
In this section, we present additional analysis on GTG which is not included in the main section due to the page limit.
D.1
Additional Analysis on Trajectory Construction
D.1.1 Analysis Score Distribution of Trajectories
We conduct additional analysis on trajectory construction method. We try to generate diverse trajectories toward high-scoring regions by randomly selecting subsequent designs from K neighbors and allowing local perturbations. To this end, we visualize the shift in the distribution of function values via various trajectory construction strategies in the Superconductor task. As shown in Figure 7 the SORT-SAMPLE strategy suggested by BONET constructs trajectories solely on high-scoring designs, which can be easily trapped into local optima. Unlike SORT-SAMPLE our method shifts distribution towards nigh-scoring regions while using the information of low-scoring regions to distill the knowledge of the landscape of the target function to the generator.
Orgged
1200
2000
130
0.5
8.0
4.15
0.58
0.5
C.0
0.5
300070
S00th
Store
(a) Original Dataset
(b) SORT-SAMPLE
(c) Ours
Figure 7: Distribution of function values in the offline dataset and trajectory datasets constructed by different strategies.
D.1.2 Analysis on Hyperparameters in Trajectory Construction
We also conduct additional analysis on hyperparameters trajectory construction, K and e. Figure shows the performance of GTG in TFBind8 task by varying K and e. While using too large K or too small e may lead to relatively low performance, we do not see much variation with different values.
1130
1,200
0.35
0.95
0180
Normalized Scare
Norpalipd Seore
0.56
0:06
0,550
0.800
0.555
0.925
0.50
0.30
10
20
30
100
0.01
0.05
0.1
0.2
Number of Nearest Neighbors (K)
Perturbation Cocllicienl (c)
Varying K
(b) Varying
Figure 8: Performance of GTG in TFBind8 task by varying K and c. Experiments are conducted with random seeds and mean and standard deviation are reported.
19

---

D.2
Additional Analysis on Sampling Procedure
D.2.1 Various Strategies for Guided Sampling
In this section, we explore various strategies for guiding diffusion models to generate high-scoring designs. As we also generate score values, it could be possible to guide diffusion models to generate high-scoring designs by inpainting score values with the desired values. To this end, we conduct additional experiments on Dessinn-Bench tasks by generating trajectories with inpainting instead of classifier-free guidance. Specifically, we inpaint the y values of the generated trajectories as y". the normalized score of the optimal design.
Table T2 shows the performance of different guiding strategies. It confirms that conditioning by classifier-free free guidance performs better than the inpainting strategy, justifying our decision choice.
Table 12: Exploring various guiding strategies.
Method
TFBind8
TFBind10
Superconductor
Ant
D' D'Kitty
GTG (Inpainting) GTG (CF)
0,963 0.026
0.652 0.062
0.503 0.035
0.938 0,014
0,966 0,007
0.976 0.020
0.698 0.127
0.519 0.045
0.963 0.009
0.971 0.009
D.2.2 Diversity Analysis
In this section, we explore the trade-off between performance and diversity via filtering strategy. While the filtering strategy boosts the performance of our method by eliminating potentially sub- optimal designs, may reduce the diversity of candidates, which may be crucial in tasks such drug discovery due to proxy misspecification [45].
To this end, measure the diversity of the candidates, following the procedure of [4] For measurement, we use the average of the pairwise distance between candidates as below.
Diversity(D)
d(x,x') D -) XED xx)
Σ
(11)
where d(x, x') is pairwise distance between samples. For discrete tasks, we use the hamming-ball distance metric. For continuous tasks, we compute L2 distance.
Table 13 illustrates the effect of filtering on performance and diversity. As expected, we achieve higher performance through filtering while sacrificing the diversity of the candidate set. It might be beneficial to automatically balance performance and diversity trade-off by measuring the uncertainty of the proxy function. We leave it as future work.
Table 13: Impact of filtering on performance and diversity of designs
TFBind8
Ant
D'K D'Kitty
Method
Performance
Diversity
Performance
Diversity
Performance
Diversity
GTG
0.976 0.020 0.920 1 0.036
0.03
0.963 0.009
9.41 + 1.96
0.971 0.009
0.41 0.07
GTG w/o Filtering
1.17 1 0.01
0.952 0.026
17.02 3.56
0.965 1 0.007
0.73 1. 0.06
20

---

D.2.3 Impact of Exploration Level (a)
In this section, we explore the impact of the exploration level (a) the generated samples. As depicted in Figure[3c| increasing a leads to higher performance, indicating the importance of classifier- free guidance. However, we observe that conditioning on extremely high leads to sub-optimal performance, as illustrated Figure 9 Conditioning on extremely high a guides the diffusion model to over-exploration, resulting in sub-optimal out-of-distribution designs. Note that we do not fine-tune a for each task and fix with the value of across all tasks, which generally exhibits good performance.
10
09
Nomnalozdd Sooge
Normalted aaaaled Soote
0.8
07
06
0.-
04
Max
Max
Medlaa
0.3
Median
0.6
1.5
Exploration Level (a)
Exploration Level (a)
(a) Ant
(b) D' D'Kitty
Figure Performance of GTG in Ant and D' D'Kitty tasks with extremely high a values
D.2.4 Assumption on y"
We assume that the optimal value y" of each task known, following prior works [3, 16]. However, it is not always possible to know the exact optima. To this end, we estimate y" with Vmmx where ymax the maximum value of the dataset and evaluate GTG by conditioning on the estimated value. As depicted in Table [4] conditioning
7 ymax achieves comparable performance and even outperforms the performance of conditioned on exact optima the TFBind8 task. However, introduces additional perparameeter 7, whose optimal value varies across tasks. Therefore, rely on assuming the exact optima, which is not an issue in many problems.
Table 14: Analysis on relaxing assumption of known y"
Method
TFBind8
TFBind10
Superconductor
Ant
D' D'Kitty
?' = 1.0
0.973 0.020
0.687 0.122
0.490 + 0.055
0.898 + 0.027
0.965 + 0.011
= 1.5
0.984 0.010
0.684 0.123
0.494 # 0.052
0.960 = 0.010
0.947 # 0.012
7 2.0
0.976 0.020
0.684 0.123
0.490 # 0.046
0.957 = 0.011
0.925 = 0.022
y² is known
0.976 0.020
0.698 0.127
0.519 0.045
0.963 # 0.009
0.971 # 0.009
21

---

D.3 Effect of Unsupervised Pretraining
might be beneficial to pretrain the diffusion model with unlabeled data when we have limited data points. Specifically, there is a recent work EXPT [29), which trains an autoregressive model using synthetic trajectories constructed from the large-scale unlabeled dataset and adapts new tasks by conditioning on a few labeled points. To this end, we discuss the effect of pre-training GTG with unlabeled datasets. We follow a similar procedure of EXPT to generate synthetic dataset. Formally, we sample synthetic functions from Gaussian Processes [6 with an RBF kernel and assign pseudo values the unlabeled data points from synthetic functions. Please refer to [9] for more detailed setting, Given synthetic dataset, we pretrain diffusion models with trajectories constructed from the dataset using the proposed method. Then, we generate samples by conditioning on context data points from the labeled dataset. For labeled dataset, we randomly select 1% of the original dataset.
Table 15 shows the experiment results on various Design-Benche tasks. As shown in the table, pretraining generally improves the performance of GTG in the sparse data setting. We also find that GTG with pretraining outperforms ExPT in 3 of 5 tasks. While we do not assume the existence of the large-scale unlabeled dataset in the main experiment and pretraining is not main focus of our research, it might be beneficial to analyze the effect of pretraining with synthetic datasets in offline MBO thoroughly as in other problems [7 48
Table 15:
Impact of pretraining with synthetic dataset performance. Experiments are conducted with three random seeds.
Method
TFBind8
TFBind10
Superconductor
Ant
D' D'Kitty
ExPT
0.837 + 0.036
0.635 + 0.036
0.471 + 0.030
0.955 + 0.021
0.961 = 0.006
GTG
0,948 + 0.009
0.666 + 0.051
0,526 + 0.032
0.655 + 0.051
0.949 : 0.013 0.930 # 0.005
GTG w Pretraining
0.953 = 0.030
0.703 # 0.018
0.564 + 0.038
0.897 = 0.015
D.4 Time Complexity of Sampling Procedure
In this section, we analyze the time complexity of the sampling procedure of GTG. To generate trajec- tories, we run T 200 denoising timesteps with classifier-free guidance and context-conditioning to sample N = 128 trajectories, which takes approximately 9.41s and 9.47s in wall clock time for the Ant and D' Kitty tasks, respectively. We visualize the trade-off between the performance and runtime of sampling by varying the number of denoising timesteps. As shown in Figure 10 we can decrease the number of denoising timesteps one-tenth with minimal loss in performance. Please note that sampling time is negligible compared to evaluating black-box functions, which is mostly expensive in real-world settings.
Normallzd Score
Normallad Seare
0x3-
4.78
079
15.0P
3x1F
2.
3'30P
Sampling Time (seconda)
Sampling Time (sacoed))
(a) Ant
(b) D' Kitty
Figure 10: Trade-off between Performance and Sampling time in Ant and D' D'Kitty tasks.
22

---

D.5 Extended Experiment Results
In this section, we present extended experiment results in sparse and noisy datasets. As shown in Tables 16 [n] our method outperforms most baselines in various practical settings. Note that we cannot conduct experiments with NEMO and RoMA, as there is no code publicly available.
Table 16: Experiments on Sparse Datasets.
TFBind8
Dkitty
Method
1%
20%
50%
1%
20%
50%
RO-qEI
0.878 0.048
0.878 0.082
0.863 # 0.085
0.884 + 0.001
0.891 + 0.005
0.891 0.003
CMA-ES
0.879 = 0.066
0.039
0,927 = 0.037
0.723 0.723 0.002
REINFORCE
0.945 0.036
0
0,910 0.032
0.615 0.178
0.614
0.130
Grad Ascent
0.897 0.060
0.037
0.951 0.027
0.610 0.172
0.822 + 0.043
0.868 0.016
COMs
0.941 = 0.032
0.954 0.029
0,969 = 0.016
0.918 = 0.005
0.915 # 0.057
0.958 = 0.015
BDI
0.898 + 0.000
0.952 0.000
0,988 + 0.000
0.865 + 0.000
0.927 +: 0.000
0.938 + 0.000
ICT
0.899 = 0.045
0.925 0.035
0.962 = 0.019
0,946 + 0.010
0.949 + 0.010
0.954 = 0.008
CbAS
0,908 0.043
0.915 = 0.036
0.909 = 0.040
0.887 = 0.016
0.895 + 0.010
0,900 = 0.008
MINs
0.871 # 0.083
0.882 0.021
0.935 # 0.027
0.926 + 0.008
0,941 + 0.008
0.938 + 0.007
DDOM
0.851 + 0.082
0.906 0.050
0.896 # 0.048
0.938 + 0.007
0.945 + 0.011
0.944 = 0.008
BONET
0.791 0.079
0.824
0.061
0.884 # 0.072
0.875 + 0.004
0.939 # 0.007
0.940 + 0.009
PGS
0.914 0.043
0.866 0.064
0.896 0.100
0.939 + 0.023
0.952 + 0.022
0.963 0.023
GTG (Ours)
0.948 0.009
0.964 0.025
0.973 0.016
+ 0.013
0.957 + 0.009
0.968 0.002
Table 17: Experiments on Noisy Datasets.
TFBind8
Dkitty
Method
1%
20%
50%
1%
20%
50%
BO-qEI
0.744 # 0.089
0.091
0.579 # 0.114
0.891 = 0.003
0.891 # 0.012
0.884 = 0.000
CMA-ES
0.968 + 0.011
0.961 0.013
0.876 + 0.061
0.863 +: 0.022
0.852 + 0.014
0.839 + 0.012
REINFORCE
0.825 0.054
0.819 = 0.051
0.409 + 0.171
0.560 + 0.194
0.619 = 0.193
Grad Ascent
0.955 0.022
0.938 0.025
0.917 + 0.034
0.911 + 0.009
0.856 + 0.024
0.830 + 0.047
COMs
0.928 + 0.028
0.938 # 0.040
0.915 # 0.057
0.936 + 0.009
0,926 + 0.012
0.925 + 0.013
BDI
0.980 = 0.005
0.886 0.051
0,873 # 0.048
0.929 + 0.008
0,908 + 0.010
0.918 + 0.016
ICT
0.941 0.013
0.950 0.023
0,921 = 0.054
0.940 + 0.029
0.914 + 0.024
0.896 = 0.000
CbAS
0.916 0.041
0.916 0.034
0.906 0.033
0.901 0.009
0.898 0.017
0.888 0.013
MINs
0.885 + 0.057
0.947 0.032
0.883 0.068
0.941 = 0.006
0.938 + 0.008
0.932 0.008
DDOM
0.896 = 0.048
0.887 0.065
0.887 = 0.065
0.944 = 0.009
0.945 + 0.011
0.926 0.020
BONET
0.904 0.044
0.822
0.773 0.143
0.942 +: 0.008
0.927 & 0.024
0.924 0.010
PGS
0.906 0.030
0.911 0.033
0.869 0.039
0.942 0.005
0.923 0.009
0.891 0.016
GTG (Ours)
0.976 + 0.015
0.967 0.026
0.948 0.029
0.955 + 0,008
0.947 + 0.015
0.937 + 0.013
23

---

D.6 Additional Visualization on Toy 2D Experiment
We present additional visualization results from the Toy 2D experiment. shown in Figure GTG is able to generate diverse trajectories toward high-scoring designs by conditioning on different context points and classifier-fi frree guidance.
Certea
Figure 11: Extra visualization of generated trajectories with GTG in Branin Task.
E
Broader Impact
Optimization for real-world designs presents both opportunities and risks. For instance, while the design of new pharmaceuticals holds the promise of curing previously untreatable diseases, there is the potential for misuse, such as creating harmful biochemical agents. Researchers should be diligent to ensure that their innovations are employed in ways that contribute positively to societal welfare.
24

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We clearly state the main claims in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected generalize to other settings. It is fine include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
We do not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide detailed experiment settings in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
If the contribution is primarily new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model limited some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
We provide open access to the data and code to reproduce the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
onPolicy for more details.
While we encourage the release code and data, we understand that this might not be possible, "No" acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurlPS code and data submission guidelines (https
httpcc//pbb. wttpccpptc.gggp.ii......oooocttt http:/uuddssscddceeeeee wetoommssssonPii onPllicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All of these can be found in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We conduct all experiments with multiple random seeds and report error bars.
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap. etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should clear whether the error bar the standard deviation the standard error of the mean.
27

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e-g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide information on the computer resources in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttlsssttts////t htttps:/neeurips.ctt http:/euurricccccc/ http://ppw/.rrrrp...//oooo/////a
Answer: [Yes]
Justification:
We use Design-Benchs which does not contain harmful offensive contents.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there is a special consider- ation due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the broader impact of the paper in Appendix
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
28

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: We do
controversial dataset.
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should deseribe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We mention the license the README.md of our code.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset,
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswi paperswithcode. thcode. com datasets has
curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's creators.
13. New Assets
Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification: We provide our code publicly available.
The answer NA means that the paper does not release assets.
29

---

Researchers should communicate the details of the dataset/code/mmodll as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We do not conduct crowdsourcing experiments.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: We do not conduct experiments with human subjects.
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "P6SeAGt9uV", "reviewer": "Reviewer_gJu1", "summary": "This paper introduces Guided Trajectory Generation (GTG), a novel conditional generative modeling approach to solve the MBO problem. GTG consists of three parts, including trajectories construction, model training and trajectories sampling and filtering. Experimental results on various tasks, including a toy 2D experiment and Design-Bench variants, demonstrate the method's effectiveness.", "questions": "1. Please clarify the difference between the method proposed in this paper and Decision Diffuser [1].\n2. In Section 3.4 and Appendix D.3, the distribution of data used to train models must be close to the real distribution. Otherwise, out of distribution could occur, leading to an error evaluation of trajectories.\n3. Please clarify the concept of filtering in Section 3.4 and Section 5. Whether they are the same operation or not?\n\n[1] Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657, 2022.", "limitations": "Yes, the authors have discussed the limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written with a great motivation and description. Especially, authors introduce the conditional guidance to explore high-scoring regions.\n2. The effectiveness of the proposed method is verified through experiments on multiple tasks and a detailed comparison with existing methods is presented.\n3. The paper analyzes the hyperparameter settings and explores the effects of different hyperparameter settings on the model performance.", "weaknesses": "1. The method proposed is similar to Decision Diffuser [1]. The method proposed focuses on the offline model-based optimization problem while Decision Diffuser is used to sovle the offline RL problem.\n2. It is not clear that in loss function, only x is used to calculate loss or both x and y are used.\n3. In this paper, the concept of filtering in Section 3.4 and Section 5 are not clarified clearly.\n\n[1] Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the insightful review and positive feedback!\n> (Weakness 1 & Question 1) Clarification on the difference between the proposed method and Decision Diffuser.\n\nWe would like to highlight that our method aims to find a design that maximizes the target black-box function while Decision Diffuser is a planner to solve offline RL problems by generating high-rewarding trajectories that follow environment dynamics. \n\nInstead of utilizing diffusion model to generate a single design, we train diffusion model with synthetic trajectories constructed from the dataset. While several prior works construct trajectories from datasets, we developed a novel method for constructing trajectories with locality bias to help generator better understand the landscape of the target function. As reviewer MZpe mentioned, how to construct trajectories is a interesting problem in the context of MBO. The ablation study summarized in Table 4 underscores that our method outperforms prior strategies across various tasks.\n> (Weakness 2) It is not clear that in loss function, only x is used to calculate loss or both x and y are used.\n\nWe apologize for the confusion in the notation. The trajectory $\\tau$, which is a set of $(x, y)$ input pairs, should be used to calculate the loss in line 5 of Algorithm 2. We train diffusion model to generate trajectories, not a single design $x$. I will modify the manuscript to clarify this.\n\n> (Question 2) In Section 3.4 and Appendix D.3, the distribution of data used to train models must be close to the real distribution. Otherwise, out of distribution could occur, leading to an error evaluation of trajectories.\n\nFor both Section 3.4 and Appendix D.3, we train proxy model with only offline dataset. As our objective is filtering high-fildelity designs with the proxy, we introduce a rank-based reweighting suggested by [1] during training to make the proxy model focus on high-scoring regions. To prevent high error in evaluation, we train ensembles of MLP for robustness of the proxy model. For more details, please refer to Appendix B.2 and our code.\n\n[1] Austin Tripp, Erik Daxberger, and José Miguel Hernández-Lobato. Sample-efficient optimization in the latent space of deep generative models via weighted retraining. Advances in Neural Information Processing Systems, 33:11259–11272, 2020.\n\n> (Weakness 3 & Question 3) Clarification of the concept of filtering in Section 3.4 and Section 5\n\nWe apologize for the confusion regarding the concept of filtering in the manuscript. Both Section 3.4 and Section 5 refer to the same concept. Among samples from multiple generated trajectories, we select candidates for evaluation by filtering with the proxy function.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SI5PQCjbJA", "reviewer": "Reviewer_MZpe", "summary": "The paper consider the problem of offline optimization where the goal is to find the optima of a black-box function in a zero-shot manner without online evaluations. The key idea is to generate trajectories with a locality biased heuristic and employ a conditional diffusion model to learn the distribution of these trajectories. Experiments are performed on multiple tasks from design bench benchmark.", "questions": "Please see weaknesses section.", "limitations": "Please see weaknesses section.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- I think looking at construction of trajectories is a really interesting problem in the context of offline optimization. Other than the two references mentioned in the paper, another recent ICML paper (listed below) also uses monotonic trajectories in the context of this problem that shows this is a recurring theme.\n\n    Hoang, M., Fadhel, A., Deshwal, A., Doppa, J., & Hoang, T. N. (2024). Learning Surrogates for Offline Black-Box Optimization via Gradient Matching. In Forty-first International Conference on Machine Learning.\n\n- I like the ablation experiments in the experimental section that shows the efficacy of different components of the proposed approach.", "weaknesses": "- This is not specific to the paper as such but the tasks in design-bench benchmark have multiple issues and as a result, some of the results are not reliable. For example, the offline dataset in superconductor task has multiple copies of the same inputs but with different outputs. As a result, the random forest oracle which is fit on this offline data is not reliable.\n\n- The paper calls existing trajectory generation approaches (like sorted monotonic trajectories) as heuristics but the proposed approach is also a heuristic. Is there more principled/theoretical explanation for the proposed approach?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable review and positive feedback!! As only two questions have been raised by the reviewer, we will address it in this response. If you have any additional questions, please do not hesitate to let us know!\n\n> (Weakness 1) Unreliable results of superconductor task.\n\nWe acknowledge the issues with Design-Bench, particularly in the Superconductor task. However, we have found that by fixing the seed for evaluation, we can obtain consistent outputs for multiple copies of the same inputs. Please note that we have reproduced baselines whose code is publicly available using the same evaluation procedure for a fair comparison.\n\n> (Weakness 2) The paper calls existing trajectory generation approaches (like sorted monotonic trajectories) as heuristics but the proposed approach is also a heuristic. Is there more principled/theoretical explanation for the proposed approach?\n\nWe apologize for overstating our proposed method. Our trajectory construction method can also be considered a heuristic since it does not include learning components. Nevertheless, we focus on constructing trajectories that give us more valuable information for learning to improve designs toward diverse high-scoring regions. We have empirically found that constructing trajectories with locality bias can guide a diffusion model to explore high-scoring regions in the Design-Bench and its variants. Furthermore, we also find that the computational time for constructing trajectories does not require a significantly large amount of time, as shown in Table 9 in Appendix B.1.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qh2SYPBhGG", "reviewer": "Reviewer_nEhC", "summary": "The paper introduces Guided Trajectory Generation (GTG) for offline model-based optimization (MBO). GTG creates synthetic trajectories from offline data, using locality bias to ensure consistent improvement. A conditional diffusion model generates these trajectories based on their scores. The method then uses guided sampling and a proxy function to identify the best designs. GTG shows superior performance on the Design-Bench benchmark compared to other methods.", "questions": "- Please check the weaknesses\n- For the discrete tasks, Is a latent space used or is everything kept discrete during diffusion and proxy training?\n- How many gradient steps are used for gradient ascent methods like COMs and RoMA? Is it 64? And is the trajectory length the same for BONET and PGS?\n- Were the hyperparameters picked using the proxy or the oracle? \n- Are designs picked at the end of the generation process, or can they be selected from the middle of the trajectory? Is the filtering applied to results generated by BONET or PGS? How does this affect their outcomes? \n- How many designs are initially considered, and how many are selected at the end (128 out of how many)?\n- Why is the perturbation epsilon different for DKitty?", "limitations": "The authors address the limitations and potential negative impacts in their paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The authors effectively motivate their approach with a toy 2D Branin example, highlighting the differences from previous synthetic trajectory-based methods.\n- Extensive ablation studies demonstrate the robustness and effectiveness of the proposed method.\n- The paper is well-written and easy to follow, with a straightforward and effectively communicated idea.", "weaknesses": "- TFBind8 and TFBind10 are very similar. It would be better to see experiments on more varied discrete benchmarks like NAS. Also, why do the results for TFBind10 have such high variance?\n- It’s not clear how this approach deals with out-of-distribution (OOD) problems. Diffusion models usually generate in-distribution data, so where does the robustness against OOD desings come from?\n- The proxy function uses the standard regression approach, which can be quite fragile when estimating OOD designs. Have you investigated using COMS or ROMA models as a proxy?\n- There isn't a principled mathematical explanation for why this approach works.\n- The paper doesn’t include 50th percentile results, which are a standard part of evaluation in this field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the critical reviews, insightful feedback, and further suggestions to enhance our manuscript.\n\n> (Weakness 1) More varied discrete benchmarks and high variance in TFBind10 results\n\nWe excluded NAS as it takes too long to evaluate. Instead, to verify the effectiveness of our method on various discrete tasks, we conduct additional experiments on UTR. We use the reported scores in PGS for other baselines. As shown in Table 19 attached in PDF, we achieve the second-best results in the UTR task.\n\nFor high variance in TFBind10, we observe that it is due to the variability of the seeds. When we conduct experiments on more seeds (16 seeds), we find that our method still showed high performance with lower variance.\n\n**Performance of GTG on TFBind10 with more random seeds.**\n|| TFBind10|\n| -|:-|\n| GTG (8 seeds)  | 0.698 ± 0.127 |\n| GTG (16 seeds) | 0.699 ± 0.091 |\n\n> (Weakness 2) It’s not clear how this approach deals with out-of-distribution (OOD) problems.\n\nThank you for your interest in how our method addresses out-of-distribution (OOD) problems. We introduce classifier-free guidance to guide the diffusion model towards exploring high-scoring regions. To mitigate the OOD issue, we introduce $\\alpha$, which controls the exploration level of the generated trajectories. We find that setting $\\alpha=0.8$ generally exhibits good performance while high $\\alpha$ results in sub-optimal OOD designs. Please refer to Section 5 and Appendix D.2 for a more discussion on the impact of $\\alpha$.\n\n> (Weakness 3) Using COMS or ROMA models as a proxy\n\nAs you mentioned, the proxy function can be fragile when estimating OOD designs. To mitigate this issue, we use ensembles of MLP as a proxy with rank-based reweighting during training to focus on high-scoring regions as written in Appendix B.2. Nevertheless, using COMs and ROMA is a promising approach to enhance the robustness against OOD designs. \n\nTo this end, we replace proxy with COMs and ROMA and evaluate the performance of our method on TFBind8 and Dkitty tasks. As shown in Table 19 attached in PDF, we find no significant difference in performance based on the choice of different proxy functions.\n\n> (Weakness 4) Absence of principled mathematical explanation\n\nWhile there is no principled mathematical explanation, our method shows promising results across various real-world tasks. We also conduct a thorough analysis of our method using Toy 2D experiment visualizations in Figure 2. Additionally, we perform experiments on practical settings such as sparse and noisy datasets, demonstrating the versatility of our method.\n\n> (Weakness 5) Absence of median performance\n\nWe apologize for missing the median performance results. We will update the manuscript with median performance results. As shown in the table 20 attached in the PDF, we also achieve the highest mean rank (4.0) in terms of the median performance.\n\n> (Question 1) For the discrete tasks, Is a latent space used, or is everything kept discrete during diffusion and proxy training?\n\nAs mentioned in Appendix A.2, we convert discrete input into a continuous vector and adopt a continuous diffusion and proxy model.\n\n> (Question 2) How many gradient steps are used for gradient ascent methods like COMs and RoMA? Is it 64? And is the trajectory length the same for BONET and PGS?\n\nWe strictly follow the original procedures of the baselines written in their papers and publicly available code. For BONET, while the original paper generates 4 trajectories with a prediction length of 64, we generate 4 trajectories with a prediction length of 32 to match the evaluation budget of 128.\n\n> (Question 3) Are designs picked at the end of the generation process, or can they be selected from the middle of the trajectory?\n\nWe aggregate all designs generated by trajectories and select candidates for evaluation with a filtering procedure.\n\n> (Question 4) Is the filtering applied to results generated by BONET or PGS? How does this affect their outcomes?\n\nBoth methods do not apply a filtering strategy after generation. While PGS generates synthetic trajectories, it ultimately selects final candidates using a policy trained with the trajectories. Therefore, we examine the effect of filtering only in BONET. As shown in the table, filtering can improve the performance of BONET. However, we find that our method still outperforms BONET in terms of performance.\n\n**Performance of BONET with filtering strategy.**\n| | TFBind8| Dkitty|\n|:- |:- |:-|\n| BONET| 0.831 ± 0.109 | 0.950 ± 0.014 |\n| BONET + Filtering | 0.913 ± 0.122 | 0.954 ± 0.011 |\n| GTG (Ours) | 0.976 ± 0.020 | 0.971 ± 0.009 |\n\n> (Question 5) How many designs are initially considered, and how many are selected at the end (128 out of how many)?\n\nAs mentioned in Section 4.4, we sample $N=128$ trajectories conditioning on $C=32$ context data points, so overall $4,096$ designs are initially considered. Then $Q=128$ designs are finally selected. As sampling procedure can be conducted in parallel, time complexity is not too problematic. For time complexity of sampling procedure, please refer to Appendix D.4.\n\n> (Question 6) Why is the perturbation epsilon different for DKitty?\n\nWe introduce perturbation epsilon ($\\epsilon$) to prevent synthetic trajectories from converging into a single maxima in the offline dataset. We use small $\\epsilon$ for Dkitty as there are relatively many samples with high scores in the offline dataset compared to other datasets.\n\nWe also observe that even with the same $\\epsilon=0.05$ for the Dkitty task, there is no big difference in performance, and it still outperforms other baselines, as shown in the table. For analysis on $K$ and $\\epsilon$ for constructing trajectories, please refer to Appendix D.1.\n\n**Performance of GTG on DKitty task with different perturbation epsilon ($\\epsilon$).**\n|| D'Kitty|\n| -- |:--- |\n| ICT (Best among baselines) | 0.960 ± 0.014 |\n| GTG ($\\epsilon=0.01$)| 0.971 ± 0.009 |\n| GTG ($\\epsilon=0.05$)| 0.965 ± 0.008 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g6BJlh3Nta", "reviewer": "Reviewer_FGy7", "summary": "The paper proposes a novel conditional generative modeling approach using diffusion models for offline model-based optimization (MBO). The method constructs synthetic trajectories toward high-scoring regions, trains a conditional diffusion model, and samples multiple trajectories to explore and select high-fidelity designs.", "questions": "Please see weaknesses", "limitations": "n/a", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tIntroduces a new approach to generating trajectories that consistently move towards high-scoring regions.\n2.\tLeverages the powerful capabilities of diffusion models to handle complex and high-dimensional data.\n3.\tThe method utilizes classifier-free guidance and context conditioning during the sampling process, enhancing the exploration of high-scoring regions.", "weaknesses": "1.\tThe authors have adopted the Diffusion model for offline model-based optimization, which is a current trending approach. It is recommended that the authors emphasize their unique contributions and clarify the distinctions between their method and previous approaches.\n2.\tThe model filters candidates through a trained proxy model. The authors should elaborate on how data handling is managed during experiments to ensure that there is no label leakage and discuss whether the comparisons with other models are conducted fairly.\n3.\tThe authors should provide a more detailed description and discussion of the experimental results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback and further suggestions that could enhance our manuscript.\n\n> (Weakness 1) The authors have adopted the Diffusion model for offline model-based optimization, which is a current trending approach. It is recommended that the authors emphasize their unique contributions and clarify the distinctions between their method and previous approaches.\n\nWe acknowledge that several concurrent works adopt diffusion models for offline model-based optimization. DDOM [1] is the pioneering work utilizing diffusion models for offline MBO. Following DDOM, DEMO [2] trains a diffusion model to match a pseudo-target distribution constructed by gradient ascent and uses the model to edit designs in the offline dataset. Diff-Opt [3] considers a constrained optimization setting and introduces a two-stage framework that begins with a guided diffusion process for warm-up, followed by a Langevin dynamics stage for further correction. Diff-BBO [4] measures the uncertainty of generated designs to select the optimal target value for conditioning the diffusion model.\n\nOur method uniquely distinguishes itself from prior and concurrent works by using diffusion models to generate trajectories toward high-scoring regions, learning to improve solutions from the dataset. While we mention this part in the related work section, we will modify the manuscript to emphasize our unique contributions in the introduction.\n\n> (Weakness 2) The model filters candidates through a trained proxy model. The authors should elaborate on how data handling is managed during experiments to ensure that there is no label leakage and discuss whether the comparisons with other models are conducted fairly.\n\nIn the filtering stage, we train our proxy model with only offline datasets for fair comparison.\n\n> (Weakness 3) The authors should provide a more detailed description and discussion of the experimental results.\n\nSorry for missing detailed description and discussion on the experiment results. We will provide more thorough analysis on our main experiment in the manuscript as written below.\n\nOur experiment results generally surpass forward approaches, which struggle to fall into OOD designs, especially in high-dimensional settings. We also observe that our method outperforms inverse approaches, including DDOM, which utilizes the diffusion model. It demonstrates that generating trajectories towards high-scoring regions can be more effective than generating a single design, as we can distill the knowledge of the landscape of the target function into the generator. Our method achieves higher performance compared to BONET, which also generates trajectories. It indicates that our novel trajectory construction strategy effectively guides the diffusion model to explore diverse paths toward high-scoring regions.\n\n[1] Krishnamoorthy, Siddarth, Satvik Mehul Mashkaria, and Aditya Grover. \"Diffusion models for black-box optimization.\" International Conference on Machine Learning. PMLR, 2023.\n\n[2] Yuan, Ye, et al. \"Design Editing for Offline Model-based Optimization.\" arXiv preprint arXiv:2405.13964 (2024).\n\n[3] Kong, Lingkai, et al. \"Diffusion models as constrained samplers for optimization with unknown constraints.\" arXiv preprint arXiv:2402.18012 (2024).\n\n[4] Wu, Dongxia, et al. \"Diff-BBO: Diffusion-Based Inverse Modeling for Black-Box Optimization.\" arXiv preprint arXiv:2407.00610 (2024).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank the review committee for their detailed feedback. We appreciate the recognition of our paper's strengths, highlighted by the reviewers: **well-written** (gJu1, nEhc), **novel** (nEhc, FGy7), and **extensive experiments and ablations** (gJu1, MZpe, nEhc). In response to the reviewers' feedback, we provide a summary of conducted additional experiments:

- **More discrete tasks:** We conduct additional experiments on UTR tasks to demonstrate the effectiveness of our method on various discrete tasks. We achieved the second-best results on the UTR task.

- **Different proxy functions**: We conduct additional experiments with different proxy functions such as COMs and ROMa for filtering strategy. We find that there is no significant difference in performance depending on the choice of proxy function.

- **Median Performance**: We report the median score of our method and all baselines. We achieve the highest mean rank (4.0) regarding median performance.

Detailed responses have been provided for each point raised by the reviewers.

Meta-review: This paper is concerned with model-based optimization (MBO). The authors propose to leverage the diffusion models with guided sampling methods to synthesize trajectories to improve the solutions. Overall, I think the authors make a good contribution to MBO as they successfully demonstrate that the diffusion models can be used as a trajectory synthesizer for MBO tasks, which will motivate more studies on along this direction. The authors also provide good responses to ease the concerns from the reviewers. As a result, I recommend acceptance of this paper.
Decision: Accept (poster)