Design Editing for Offline Model-based Optimization
Authors: Ye Yuan
Youyuan Zhang
Can Chen
Haolun Wu
Melody Zixuan Li
Jianmo Li
James J. Clark
Xue Liu
Venue: Accepted by TMLR ()

Abstract: Offline model-based optimization (MBO) aims to maximize a black-box objective function using only an offline dataset of designs and scores. These tasks span various domains, such as robotics, material design, and protein and molecular engineering. A common approach involves training a surrogate model using existing designs and their corresponding scores, and then generating new designs through gradient-based updates with respect to the surrogate model. This method suffers from the out-of-distribution issue, where the surrogate model may erroneously predict high scores for unseen designs. To address this challenge, we introduce a novel method, Design Editing for Offline Model-based Optimization} (DEMO), which leverages a diffusion prior to calibrate overly optimized designs. DEMO first generates pseudo design candidates by performing gradient ascent with respect to a surrogate model. While these pseudo design candidates contain information beyond the offline dataset, they might be invalid or have erroneously high predicted scores. Therefore, to address this challenge while utilizing the information provided by pseudo design candidates, we propose an editing process to refine these pseudo design candidates. We introduce noise to the pseudo design candidates and subsequently denoise them with a diffusion prior trained on the offline dataset, ensuring they align with the distribution of valid designs. Empirical evaluations on seven offline MBO tasks show that, with properly tuned hyperparamters, DEMO's score is competitive with the best previously reported scores in the literature.

Published in Transactions on Machine Learning Research (04/2025)
Design Editing for Offline Model-based Optimization
Ye Yuan"t
ye. mmnannmmmall.mmglll.om
McGill University, Mila Quebec Al Institute
Youyuan Zhang*
youguuan. wwnguan..eamg@@mmiiimmmmm.cccm
McCill University
Can (Sam) Chen
can. won.chen@mill..po quebec
McGill University, Mila Quebec Al Institute
Haolun Wu
haolun, wwt mwm@mail.mmggil...c
McCill Universily, Mila Quebee AI Institute
Zixuan (Melody) Li
zizuan Satmam.ii@@@mil.meeee...
McGill Universily, Mile Quebec Al Institute
Jianmo Li McCill Universiby
jionmo Janmo.li@mmiil.mmggll.
James J. Clark
dames.clarks @megill.c
McGill University, Mila Quebec Al Institute
Xue Liu
meliu@es.mecgll.
McCill University, Mila Quebee AI Institute
Reviewed on OpenReview: https: // opeereview..eee.o net/ forum? id- 17FF
Abstract
Offline model-based optimization (MBO) aims to maximize black-box objective function using only an ofllne dataset of designs and scores. These tasks span various domains, such as robotics, material design, and protein and molecular engineering. A common approach involves training a surrogate model using existing designs and their corresponding scores, and then generating new designs through pradient-based updates with respect to the surrogate model. This method suffers from the aut-of-distrribution issue, where the surrogate model may erroneously predict high scores for unseen designs. To address this challenge, introduce novel method, Design Editing for Offline Model-based Optimization (DEMO), which leverages a diffusion prior to calibrate overly optimized designs. DEMO first. generales pseudo design candidates by performing gradient ascent with respect to a surrogate model. While these pseudo design candidates contain information beyond the offline dataset, they might be invalid or have erroneously high predicted scores. Therefore, address this challenge while utilizing the information provided by pseudo design candidates, we propose editing process to refine these pseudo design candidates. We introduce noise to the pseudo design candidates and subsequently denoise them with diffusion prior trained on the offline dataset, ensuring they align with the distribution of valid designs. Empirical evaluations seven offline MBO tasks show that, with properly tuned hyperparameters, DEMO's score competitive with the best previously reported scores in the literature. The source code provided here.
"Equal contribution with random order. i "Correspondigg author.

---

Published in Transactions on Machine Learning Research (04/2025)
(b)
Increesing Noise Injection Level
1.2
Offiin Dataset
1.2
Offline Data
Pseudo Design Candidates
Pe Po
Pseudo Design Candidates
Py
P
1.0
1.0
Criical Temperatur
Chttccll Termpersture
Micce
Remove
Pi
0.6
Ph
0.6
Pi
In distribution Prior
distisriibutin
0.4
0.4
Surrogate Model
Surragate Model
0.2
Blackbox Objective Function
0.2
Objective Function
-0.5
0.0 Superconductor Material
0.5
-0.5
0.0
0.5
Superconductor Material
Figure 1: Illustration of DEMO: A diffusion model, acting as the prior distribution, is trained on the offline dataset. Pseudo design candidates are acquired by performing gradient ascent with respect to a learned surrogate model. New designs are generated by modifying pseudo design candidates toward the valid distribution captured by the diffusion prior.
1 Introduction
Designing objects with specific desired traits is a primary goal in many fields, spanning areas such as robotics,
material design, and protein and molecular engineering (Tabuucco et al. 2022 Liao et al. 2019 Sarkisyan Conventionally, this goal pursued by itcratively
ct al. 2016 Angermüllerr et al., 2020 Hamidich 2018).
lesting a black-box objective function that maps a design to its property score. However, this process can be
costly, time-consuming, or hazardous (Sarkisyan et al. 2016 Angermüüler et al. 2020 Hamidieh 2018
Barrera et al. 2016 Sample al.) 2019). Thus, it is more feasible to utilize an existing offline dataset of designs and their scores to find optimal solutions without further real-world testing (Trabucco et al. 2022
This approach is known as offline model-based optimization (MBO), where the objective is to identify a design that optimizes the black-box function using only the offline dataset.
Gradient ascent is commonly used address the offiine MBO challenge. For instance, as shown in Figure ! (a), offline dataset might consist of five pairs of superconductor materials and their corresponding critical temperature, denoted P1,2,3,455- A deep neural network (DNN) model, referred as the surrogate model and represented by fa(-), is trained to approximate the unknown objective function based on this dataset. Gradient ascent is then applied to existing designs with respect to the surrogate model fe(-) for generating a new design that achioves a higher score. This approach, however, encounters an out-of-distribution (OOD) problem, where the surrogate struggles to accurately predict data outside the training distribution. This mismatch between the surrogate and the true objective function, as depicted in Figure 1 (a), can result
overly optimistic scores for the new designs generated by gradient ascent (Yu et al. 2021
To tackle this OOD issue, recent research has proposed the use of regularization techniques, either applied directly to the surrogate model (Yu et al. 2021 Trabucco et al. 2021 Fu & Levine 2021 Qi et al. 2022a
Yuan et al., 2023 Chen et al. 2023a) or to the design under consideration (Chen et al. 2022 2023b). ). These
strategies improve the surrogate's robustness and generalization. However, calibrating design candidates generated by gradient ascent remains unexplored. Instead of regularizing the surrogate models, we could tailor these design candidates toward a prior distribution and avoid over optimization.
In this work, introduce innovative and effective approach, Design Editing for Offline Model-based Optimization (DEMO) to fill this gap. Initially, surrogate model, represented fo(-), is trained the offlinc dataset D, and gradient ascent is applicd to existing designs with respect to the surrogate model. This process generates several designs which may have wrongly high predicted scores but low ground-truth scores due to the inaccuracies of the surrogate model, and we denote them as pseudo design condidates. As illustrated in Figure [ (a), the surrogate model fits the offline data P1 to Ps: generating pscudo design candidates Pa and ps through gradient ascent. While these pseudo design candidates might be overly optimized, they contain information beyond the offline dataset. We then propose the second phase to calibrate these
2

---

Published in Transactions on Machine Learning Research (04/2025)
pseudo design candidates and align them with the in-distribution area. Specifically, conditional diffusion model, denoted sp(+), is trained on all existing designs along their corresponding scores within the offline dataset to characterize a manifold of valid designs, as all existing designs are valid. After that, we edit the pscudo design candidates by introducing random noise to them and employing the diffusion prior to remove the noise. Comparing to directly leveraging a generative model lo craft new design candidates from pure noise, our method benefits from the pscudo design candidates, which incorporate more information than the generative model merely trained on the offline dataset. As illustrated in Figure (b), after injecting noise, the distribution of pseudo design candidates (represented by the orange contour) has more overlap with the valid design distribution (represented by the purple contour). By progressively removing the noise, we gradually project these pseudo design candidates to the manifold of valid designs, as demonstrated in Figure (b). A central assumption underlying our approach is that candidate designs located near the offline data manifold are less prone to being "spurious optima" of the surrogate model. In regions far from the obscrved data, the model must extrapolate, which often leads to unreliable predictions and artificially inflatod performance istims Essentially, the model may identify a high-scoring design that, in reality, is merely an artifact of overfitting or model bias. By contrast, when candidate points remain close to the data manifold, they are supported by the training data and the model's predictions in these regions are more trustworthy. Therefore, by guiding our search toward these well-supported regions, we reduce the risk of selecting designs that appear optimal only due to the model's extrapolative errors, and instead focus on candidates that are both perforrfff and realistically feasible. In essence, DEMO produces new designs which are first wildly optimized and then calibrated under the constraints captured by the diffusion prior. We empirically validate DEMO across different offiin MBO tasks.
In summary, this paper makes three principal contributions:
We introduce a novel method, Design Editing for Offline Model-based Optimization (DEMO). DEMO first performs gradient ascent with respect to learned surrogate model and share the information to the second phase of DEMO through pseudo design candidates.
The second phase trains a conditional diffusion model on the offline dataset as the in-distribution prior and calibrate the pseudo design candidates with this diffusion prior to generate final designs.
Exporiments on the descgn-bench dataset show that, with properly tuned hyperparamters, DEMO's score is competitive with the best previously reported scores in the literature.
2 Preliminary
2.1
Offline Model-based Optimization
Offline model-based optimization (MBO) addresses a range of optimization challenges with the aim of maximizing a black-box objective function based on an offline datasct. Mathematically, we define the valid design space as X = IRd, with d representing the dimension the design. Offline MBO is formulated as:
x= = arg max f(x), xE
(1)
where f(.) is the black-box objective function, and x E X is a potential design. For the optimization process, we utilize an offline dataset D = {(2,, 1))) with a; representing an existing design, such as a superconductor material, and y; representing the associated property score, such as the critical temperature. Usually, this optimization process outputs K candidates for optimal designs, where K is a small budget to test the black-box objective function. The offline MBO problem also finds applications in other areas, like robot design, as well as protein and molecule engineering.
A prevalent approach to solving offline MBO involves approximating the unknown objective function f(-) with a surrogate function, typically deep neural network (DNN) fo(+), trained on the ollline dataset:
N
0 = arg min 9
Σ (fo(z,) - v1)
(2)
N
1=1
3

---

Published in Transactions on Machine Learning Research (04/2025)
Once the surrogate is trained, design optimization is performed using gradient ascent:
2t = z + v a
for E [1,
(3)
Here, T represents the number of steps, and 77 denotes the learning rate. The optimal design x" is identified as Tr. This gradient ascent method faces an -idt----iittiiittonn (OOD) issue, where the surrogate may fail to accurately predict. the scores of unseen designs, resulting in suboptimal solutions.
2.2 Diffusion Models
Diffusion models stand out in the family of generative models due to their unique approach involving forward diffusion and backward denoising processes. The essence of diffusion models is to gradually add noise to a sample, followed by training a neural network to reverse this noise addition, thus recovering the original data distribution. In this work, we follow the formulation of diffusion models with continuous time (Song let al. 2021b Huang al. 2021I. Here, xt)) is random variable denoting the state of data point at time E [0, T]. The diffusion process is defined by a stochastic differential equation (SDE):
dæ f(x t)dt + g())du,
(4)
where f(-,t) is the drift coefficient, g(-) is the diffusion coefficient, and w is a standard Wiener process. The backward denoising process is given by the reverse time SDE:
dæ = fxx,t) g(t)² V log pr()] dt + gt) dw.
(5)
where dt represents a negative infinitesimal step time, and w a reverse time Wiener process. The gradient of the log probability, V (a)), is approximated by neural network so(x(tt, with score-matching objectives (Vincent, 2011 Song & Ermon 2019
Beyond basic diffusion models, our focus is to train conditional diffusson model that learns the conditional probability distribution of designs based on their associated property scores. To incorporate conditions to diffusion models, Ho & Salimans (2022) achieve it. by dividing the score function into a combination of conditional and unconditional components, known as classifier-fres diffusion models. Specifically, a single neural network, s (at, t. y), is trained to handle both components by utilizing y as the condition or leaving it emply for unconditional functions. Formally, we can write this combination as follows:
s ( t, y) (1 + w)so(z,, t. v) ws,(
(6)
where is parameter that adjusts the influence of the conditions. A higher value of w ensures that the generation process adheres more closely to the specified conditions, while lower w value allows greater ficxibility in the outputs.
3
Related Works
3.1 Offline Model-based Optimization
Recent offline model-based optimization (MBO) techniques broadly fall into two categories: (i) those that employ gradient-basod optimizations and (ii) those that create new designs via generative models. Gradient- based methods directly optimize the surrogate model's predictions using gradient-based search. However, straightforward gradient ascent can lead to candidate designs that are far from the observed data, where the surrogate is prone to unreliable extrapolations. Therefore, to address this problem several works employ regularization techniques that enhance either the surrogate model (Yu et al. 2021 Trabucco et al. 2021 Fu
& Levine 2021 Qi el. al.) 2022a) or the design itself (Chen el al. 2023b 2022), thus improving the model 'ss robustness and generalization capacity. BOSS (Dao et al.) 2024bb introduces a mensitivity-infornn regularizer to mitigate the overfitting and narrow prediction margins offline surrogates. IGNITE (Dao et al. 2024a
presents model-agnostic sharpness regularization method that theoretically reduces the generalization error of offline surrogate models. aSCR (Yao et al. 2024) constrains the optimization trajectory to regions
4

---

Published in Transactions on Machine Learning Research (04/2025)
where the surrogate is reliable by dynamically adjusting regularization strength. Some approaches involve synthesizing new data with pseudo labels (Yuan et al. 2023 Chen et al. 2023a), where they aim to identify useful information from these synthetic data to correct the surrogate model's inaccuracies. Other approaches like PGS (Chemingui et al. 2024) employs policy-guided gradient search approach that explicitly learns the best policy for a given surrogate model. Match-OPT (Hoang et al. 2024] provides a theoretical framework for offline black-box optimization by quantifying the performance gap due to surrogate inaccuracies and introduces a black-box gradient matching algorithm to improve surrogate quality. The second category encompasses methods that learn to replicate the conditional distribution of existing designs and their scores, including approaches such as MIN (Kumar & Levine, 2020). CbAS (Brookes et al.) 2019). Auto CbAS Pannjiang & Listgarten 2020), DDOM (Krishnamoorthy et al. 2023 and BONET (Mashkaria et al.) 2023). ExPT (Nguyen el al. 2023) trains a foundation model for few-shot experimental design that leverages unsupervised pretraining and in-context learning on unlabeled data to efficiently generate candidate optima with minimal labeled examples. A concurrent work (Dao et al.] 2025) unsurprisingly sharcs the core idea of leveraging diffusion processes to bridge the gap between low-and-high performing designs. This work explicitly trains generalized diffusion process to map directly between the distributions of low- and high-value designs. In contrast, we leverage diffusion prior in two-phase process by first generating pseudo design candidates via surrogate-based gradient ascent, and then employing a diffusion model to edit these candidates so that they remain on the valid design manifold. Our decoupled two optimization phases and modularity provide enhanced stability and flexibility. These methods are known for their ability to generate designs by sampling from learned distributions conditioned on higher target scores.
In DEMO, candidate design is first generated via gradient-based optimization, ensuring that the candidate is high-scoring according to the surrogate model. Recognizing that direct gradient-basck updates can push the candidate into regions where the model's predictions are unreliable, we then employ diffusion model to "edit" the design, explicitly moving it closer to the data manifold. This two-stage process decouples the search for high performance from the regularization required to maintain realism. While traditional gradient- based methods embed regularization within the optimization loop and generative methods rely solely on learned distributions, DEMO leverages the flexibility of gradient-based search alongside the robust.ness of diffusion-basse based editing. This approach effectively reduces the risk of spurious optima-candidater that appear optimal due to extrapolative errors, by cnsuring that final designs remain in regions well-supported by data. In summary, while existing oflline MBO methods typically focus on either direct gradient-based optimization with embedded regularization or on generative modeling of the design space, DEMO's ' hybrid approach offers a novel balance between performance and realism, thereby addressing key challenges inherent in both paradigms.
3.2 Diffusion- Based Editing
Diffusion models have shown remarkable success in various generation tasks across multiple modalities, especially for their ability to control the generation process based on given conditions. For instance, recent, advancements have utilized diffusion models for zero-shot, test-time editing in the domains of text-based image and video generation. SDEdit (Meng et al. 2022) employs an editing strategy to balance realism and faithfulness in image generation. To improve the reconstruction quality, methodologies such as DDIM Inversion (Song et al. 2021a), Null-text Inversion (Mokady al. 2023 and Negative-prompi Inversion (Miyake et al. 2023) concentrate on deterministic mappings from source latents to initial noise, conditioned on source text. Building on these, CycleDiffusion (Wu & la Torre, 2023 and Direct Inversion (Ju et al. 2023) leverage source latents from each inversion step and further improve the faithfulness of the target image to the source image. Following the image editing technique, several video editing methods (Qi ct al. 2023 Ceylan et al.,
2023 Yang el al. 22023 Geyer et. al.) 22024: Cong el al. 2024 Zhang el al.) 2024) adopt image dilfusion models and enforce temporal consistency across frames, offering practical and efficient solutions for video editing. Inspired by the success of these editing techniques in the field of computer vision, DEMO distinguishes itsclf in the context of offline MBO by first leveraging the surrogate model to craft pseudo design candidates and then refining them toward the in-distribution area.
5

---

Published in Transactions on Machine Learning Research (04/2025)
4 Methodology
In this section, we elaborate the details of our proposed Design Editing for Offline Model-based Optimization (DEMO). Typically, the out-of-distribution (OOD) issue in offline MBO arises due to incomplete observation of the entire distribution of designs and scores. During gradient ascent optimization, the pseudo design candidates are optimized with respect to the surrogate model without constraints, which leads them into the OOD region and causes overestimation of the scores. To address this, we introduce a design editing process to calibrate the pseudo design candidates toward the in-distribution area, which mitigates the OOD problem through the use of learned diffusion prior. Algorithm | illustrates the complete process of DEMO.
4.1 Acquirement of Pseudo Design Candidates
Initially, a deep neural network (DNN), denoted as fa()) with parameters 0, is trained on the oflline dataset D= (=.,,,,))) 3;) where x; and ys denote a design and its associated score, respectively. The parameters 0 are optimized as shown Eq. (2) The solution fo- (.) obtained from Eq. (2) serves as a surrogate for the unknown black-box objective function f(-) in Eq. (1. New data are then generated by performing gradient ascent on the existing designs with respect to the learned surrogale model fo- The initial point 2p an existing design selected from D. We update it as shown in Eq. (3). The design ær acquired at step T is an overly optimized design, as constraints are applied during the optimization process. By itcratively using the top K designs in the oflline dataset D as the initial points, a batch of pseudo design candidates, denoted as D', is acquired. This process is outlined from line to line 8 in Algorithm
4.2 Training of Diffusion Prior
We employ a classificr-free conditional diffusion model (Ho & Salimans
2022) to learn the conditional probability distribution of existing designs and their scores in oflline datasel D. Following the approach in DDOM (Krishnamoorthy et al.) 2023), we use the Variance Preserving (VP) stochastic differential equation (SDE) for the forward diffusion process, as specified in Song et al. (2021b)
3(t) dæ 2
ædt + v ((t)dw,
(7)
where ((L) is a continuous time function for / E 1]. The forward process in DDPM (Ho el al.) 2020 is
proved to be discretization of Eq. (7) (Song et al. 2021b). To integrate conditions in the backward denoising process, we need to train a DNN S& $0(2,, f, with paramcters Φ, conditioned on the time t and the score y associated with the unperturbed design 2n corresponding to xz- The parameters & are optimized as:
Φ² arg min E, [A(t) [mm [[| (, - V
where X(t) is a positive weighting function depending time. Since we train the offline datasct DD the model optimized according to Eq. captures the manifold of valid designs. This part is described in Line 10 of Algorithm !
Howevor, since the offline dataset may contain very low-score existing designs, naively applying the trained model to the subsequent design editing process might be harmful for calibrating the pseudo design candidates. Therefore, we propose to use the distribution conditioned on the maximum property score among the offline dataset as the prior for later usage, which is simple yet effective method for mitigating the negative impacts of very low-score designs.
4.3 Design Editing Process
Due to potential inaccuracies of the surrogate model fe- (.) in representing the black-bor objective function, the set of pscudo design candidates D' might include samples that have high predicted scores but low ground-truth scores. Inspired by the success of editing techniques in image synthesis tasks (Meng et al. 2022 Su et al. 2023). we explore the potential of calibrating these pseudo design candidates to obtain new designs
6

---

Published in Transactions on Machine Learning Research (04/2025)
with valid high scores. We perturb pseudo design candidate x(r) E D' by introducing noise at specific time out of (1,.., ;M M) and auxiliary noise levels $1,-.. .9M:
a ())
perturb = =v (()) + v1 - Cm
(9)
where am 11- Smi Om I1 Ogs and (0, This results in closed-form expression that samples i (p)
I.) The perturbed design is then used as the starting point. A final optimized design is synthesized by using any numerical solver for the backward denoising process with the model se (.), conditioned on the maximum property score among the offline dataset, to remove the noise. In our implementation, we follow existing studies (Krishnamoorlhy el al., 2023 and use Heun's 's method as the solver. To yield K final optimized designs, we utilize all pseudo design candidates from D', obtain various perturbed designs, and denoise them, pushing them toward the prior distribution. Lines 12 17 of Algorithm present the process of this procedure.
"ertuurb ~ ~N(V
Algorithm Design Editing for Offline Model-based Optimization
Input: Offline dataset D {( u=) and a time m.
Output: candidate optimal designs.
1: /* Acquirement of Pseudo Design Candidates */
2: Initialize surrogate model ʃa(.) and optimize 0 with Eq. (2 to obtain fa- (.)-
3: D' = -0-
for == 1,2. K do Zo - -x with the i-th best score within D.
for 1,2, ,T do
Update 2e with Eq. (3)
8: Append dr to D'.
/+ Training of Diffusion Prior */
10: Initialize s-(-) and optimize o with Eq.
8 on D to obtain $6- (.).
11: /* Destgn Editing Process */
12: Candidates 0
13: for i =1==1, 1,1, ... K do
14: (P) - x, D
15: Perturb x(r) with Eq. 9 and the given time m.
(p) 16: Denoise x: pertturb and generate Knew using the Heun's method with s conditioned on max max((y:)
17: Append Knew to Candidates.
18:
return Candidates
5
Experiments
This section first describes the experiment setup, followed by the implementation details and results. We aim to answer the following questions in this section: (Q1) Is proposed DEMO more effective than baseline methods in addressing the offline MBO problem? (Q2) One can alternatively generate new designs from the diffusion prior or directly use the pseudo design candidates. Is DEMO better than these partial alternatives by introducing the editing process?
5.1 Dataset and Tasks
We carry out experiments on 7 tasks selected from Design-Bence TTabucco et al. 2022 and Bayes Benchmarks Kim 2023), including 4 continuous lasks and 3 discrete tasks. The continuous lasks are as follows: (i) Superconductor (SuperC) (Hamidieh) 2018), where the goal to create a superconductor with (ii) Ant Morphology (Ant) (Trabucco et al. 2022 Brockman al. 20166). where the objective is to design a four-legged ant with 60 continuous components to increase crawling speed, based on 10, 004 designs; (iii D'Kitty Morphology
continuous components maximize critical temperature, using 17, 014 designs;
7

---

Published in Transactions on Machine Learning Research (04/2025)
(D'Kitty) [Trabucco et al. 2022 Ahn al. 2020), where the focus is on designing four-legged D'Kitty with 56 continuous components to enhance crawling speed, using 10, 004 designs; (iv) Inverse Levy Function (Levy) (Kim) 2023), where the aim is to maximize function values of the inverse black-box Levy function with 60 imput dimensions, using 15, 000 designs. The discrete tasks include: (v) TF Bind 8 (TF8) (Barrera et al., 2016), where the goal is to identify an 8-unit DNA sequence that. maximizes binding activity score, with 32, 898 designs; (ui) TF Bind 10 (TF10) (Barrera et al., 2016), where the objective is to find a 10-umit DNA sequence that optimizes binding activity score, using 30, 000 designs; (vii) NAS (Zoph & Le] 2017), where the aim is to discover the optimal neural network architecture to improve test accuracy on the CIFAR-10
dataset (Hinton et al. 2012). using 1, 771 designs.
5.2 Evaluation and Metrics
Following the evaluation protocol used in previous studies (Trabucco et al. 2022), we assume the budget K = 128 and generate 128 new designs for each method. The 100-th (max) percentile normalized ground-truth score is reported in this section, and the 50-th (median) percentile score is provided in Appendix AA.1 This normalized score is calculated as Yn = ymaas Yeun where Ymin and Umax are the minimum and maximun scores in the entire offiine dataset, respectively. For better comparison, we include the normalized score of the best design in the offline dataset, denoted as D(best). Additionally, we provide mean and median rankings across all 7 tasks for a comprehensive performance evaluation.
U-Biinn....
5.3 Comparison Methods
We benchmark DEMO against three groups of baseline approaches: (i) traditional methods, (ii those utilizing gradient optimizations from current designs, and (iii) those employing generative models for sampling. Traditional methods include: (1) BO-qEI (Wilson et al. 2017): conducts Bayesian Optimization to maximize the surrogate, proposes designs using the quasi-
acquisition function, and labels the designs using the surrogate model. (2) CMA-ES (Hanson 2006): progressively adjusts the distribution toward the optimal design by altering the covariance matrix. (3) REINFORCE (Williams) 1992):
optimizes the distribution over the input space using the learned surrogate. The second category includes: (4) Mean: optimizes the average prediction of the ensemble of surrogate models. (5) Min: optimizes
the lowest prediction from a group of learned objective functions. (6) COMs (Trabucco et al. 2021):
applies regularization to assign lower scores to designs derived through gradient ascent. (7) ROMA (Yu ltt al. 2021h: introduces smoothness regularization to the DNN. (8) NEMO (Fu & Levine 2021 limits the discrepancy between the surrogate and the black-bos box objective function using normalized maximum likelihood before performing gradient ascent. (9) BDI (Chen et al. 2022) employs forward and backward
mappings to transfer knowledge from the offline dataset to the design. (10) IOM (Qi et al. 2022b): : ensures representation consistency between the training dataset and the optimized designs. (II) ICT (Yuan et al. 2023): : identifies useful information from a pseudo-labeled dataset to improve the surrogate model. (12) Tri-mentoring (Chen et al. 2023a):: leverages information of pairwise comparison data to enhance ensemble performance. (13) PGS (Chomingui et al.] 2024]]: learns the best policy for a given surrogate model. Generative model-based methods include: (14) CbAS (Brookes et al. 2019), which adapts a VAE model to steer the design distribution toward areas with higher scores. (15) Auto CbAS (Fannjiang & Listgarten 2020), which uses importance sampling to update a regression model based on CbAS. (16) MIN (Kumar & Levine 2020), which establishes relationship between scores and designs and seeks optimal designs within this Tramework. (17) DDOM (Krishnanoorthy el al. 2023), which learns a generative diffusson model conditioned on the score values. (18) BONET (Mashkaria et al. 2023), which employs an autoregressive model trained on the offline datasct.
5.4 Implementation Details
We follow the training protocols from Trabucco et al. (2021) for all comparative methods unless stated otherwisc. 3-layer MLP with ReLU activation is used for both fo(.) and s.),,, with a hidden layer size of 2048. In Algorithm I Adam optimizer (Kingma & Ba 2015) is utilized to train the surrogate models over 200 epochs with batch
the iteration count, T, established at 100 for both continuous and discrete tasks. The
8

---

Published in Transactions on Machine Learning Research (04/2025)
Table 1: Experimental results on continuous tasks for comparison.
Method
Superconductor
Ant Morphology
D D'Kitty Morphology
Levy
D(best)
0.399
0.565
0.884
0.613
BO-qEI
0,402 + 0.034
0.819 - 0.000
0.896 +: 0.000
0.810 + 0.016
CMA-ES
0.465 + 0.024
1.214 0.732
0.724 0.001
0.887 + 0.025
REINFORCE
0.481 + 0.013
0.266 + 0.032
0.562 + 0.196
0.561 + 0.090
Mean
0.505 1: 0.013
0.940 + 0.014
0.956 0.014
0.984 1: 0.023
Min
0.501 = 0.019
0.918 0.034
0.942 0.009
0.964 + 0.023
COMs
0,481 1 0.028
0.842 + 0.037
0.926 1: 0.019
0.936 1 0.025
ROMA
0.509 + 0.015
0.916 = 0.030
0.929 0.013
0.976 + 0.019
NEMO
0.502 = 0.002
0.955 0.006
0.952 0.004
0.969 + 0.019
BDI
0.513 + 0.000
0.906 + 0.000
0.919 0.000
0.938 + 0.000
IOM
0.518 + 0.020
0.922 0.030
0.944 0.012
0.988 + 0.021
ICT
0.503 = 0.017
0.961 = 0.007
0.968 = 0.020
0.879 + 0.018
Tri-mentoring
0.514 0.018
0.948 1 0.014
0.966 0.010
0.924 1 0.035
PGS
0.563 + 0.058
0.949 = 0.017
0.966 0.013
0.963 + 0.027
CbAS
0.503 + 0.069
0.876 + 0.031
0.892 0.008
0.938 + 0.037
Auto CbAS
0.421 = 0.045
0.882 = 0.045
0.906 0.006
0.797 = 0.033
MIN
0.499 + 0.017
0.455 + 0.080
0.892 + 0.011
0.761 + 0.037
DDOM
0.486 + 0.013
0.952 + 0.007
0.941 0.006
0.927 + 0.031
BONET
0.437 = 0.022
0.976 0.012
0.954 = 0.012
0.918 + 0.025
DEMO,F (ours)
0.525 1 0.009
0.968 1 0.009
0.970 1 0.007
1.007 1: 0.015
size of 128, and a learning rate set 10-1, The step size, 17, in Eq. is configured at 10-3 for continuous tasks and 10-1 for discrete tasks. The diffusion model, (.()), undergoes training for 200 epochs with batch size of 128. Previous works such as Krishnamoorthy et al. (023) employ 1000 epochs for training diffusion models, but we find 200 epochs is enough for the diffusion models to converge. For the design editing process, following precedents set by previous studies (Krishnamoorthy et al. 2023), we set M at 1000. The selected value of m is 600, with furthor claboration provided in Appendix A.2 Results from traditional mothodologics are referenced from Trabucco et al. (2022). and we conduct independent trials for other methods, reporting the mean and standard error. All experiments are conducted on a workstation with single Intel Xeon Platinum 8160T CPU and a single NVIDIA Tesla V100 GPU, with execution times per trial ranging from 10 minutes to 20 hours (including evaluation time), depending on the specific tasks.
5.5 Results
Following existing studies [Trabucco et al. 2021 Yuan et al. 2023 in Table and Table in bold if its mcan is at least as high the highest mean minus standard deviation of the corresponding method. Alternatively, method is also bolded if its mean plus standard deviation reaches exceeds the method with the highest mean.
we mark a method
Performance in Continuous Tasks. Table | presents the results of the four continuous tasks. DEMO achieves competitive performance to existing approaches across all of contimous tasks. DEMO outperforms gradient-based methods, such as NEMO and ICT, by leveraging the design editing process to calibrate the pseudo design candidates rather than performing unconstrained optimization against regularized surrogate models. It is worth noting that some methods, such COMs, employ constrained optimization process by penalizing the value at lookahead gradient ascent optimization point. The superior performance of DEMO compared to COMs indicates that the design editing process is a more effective method. Morcover, when compared to other generative model-based approaches, such as MIN and DDOM, DEMO generally outperforms them because these methods train models solely on the offline dataset and may not benefit from the information provided by surrogate models. DEMO achieves better performance by effectively utilizing the surrogate model acquire a batch of pseudo design candidates. These results strongly support the effectiveness of DEMO for contun tasks.
9

---

Published in Transactions on Machine Learning Research (04/2025)
Table 2: Experimental results on discrete tasks, and ranking on all tasks for comparison.
Method
TF Bind 8
TF Bind 10
NAS
Rank Mean
Rank Median
D(best)
0.439
0.467
0.436
BO-qEI
0.798 + 0.083
0.652 + 0.038
1.079 + 0.059
13.9/19
16/19
CMA-ES
0.953 + 0.022
0.670 0.023
0.985 + 0.079
9.1/19
7/19
REINFORCE
0.948 0.028
0.663 0.034
11.95 + 0.000
15.1/19
19/19
Mean
0,895 = 0.020
0.654 = 0.028
0.663 + 0.058
9.3/19
9/19
Min
0.931 0.036
0.634 0.033
0.708 0.027
10.7/19
11/19
COMs
0.474 + 0.053
0.625 = 0.010
0.796 0.029
13.1/19
14/19
ROMA
0.921 +1: 0.040
0.669 1 0.035
0.025
7.9/19
7/19
NEMO
0.942 0.003
0.708 + 0.010
0.735 +
6.7/19
7/19
BDI
0,870 + 0.000
0.605 1.000
0.722 0.000
12.1/19
13/19
IOM
0.870 0.074
0.618 0.025
0.411 0.04
10.0/19
10/19
ICT
0.958 = 0.008
0.691 1 0.023
0.667 0.091
7.7/19
6/19
mri-mentoring
0.970 0.001
0.722 0.017
0.709
5.4/19
4/19
PGS
0.981 0.015
0.658 1 0.021
0.727 T. 0.033
5.4/19
7/19
CbAS
0.927 + 0.051
0.651 + 0.060
0.683 1.007
12.0/19
12/19
Auto CbAS
0.910 0.044
0.630 0.045
0.506 +
15.6/19
16/19
MIN
0.905 = 0.052
0.616 0.021
0.717 0.046
15.4/19
16/19
DDOM
0.961 0.024
0.640 + 0.029
0.737 + 0.014
9.4/19
10/19
BONET
0.975 0.004
0.681 0.035
0.724 0.008
8.0/19
6/19
DEMO (our))
0.982 0.016
0.762 0.058
0.753 + 0.017
2.1/19
1/19
Performance in Discrete Tasks. Table displays the results of the three discrete tasks. DEMO achieves top performance in TF Bind 8 and TF Bind 10, with the results on TF10 surpassing those of other methods by significant margin, suggesting DEMO's capability in solving discrete offline MBO tasks. However, DEMO underperforms on NAS, which may be due to two reasons. First, each neural network architecture is encoded as a sequence of one-hot vectors, which has a length of 64. This encoding process might be insufficient for precisely representing all features of a given architecture, leading suboptimal performance NAS. Additionally, after examining the NAS offline dataset, we found that many existing designs share commonalities. This redundancy means that the NAS dataset contains less useful information compared to other tasks, which further explains why DEMO's performance on NAS is not as strong.
Summary. These results on both continuous and discrete lasks provide a clear answer to Q1. DEMO a attains the highest rankings with a mean of 2.1/19 and a median of 1/19, as detailed Table 2 and Figure 2b and secures top performance all tasks. We perform Welch's 's t-test across the seven tasks to compare DEMO against all bascline methods. To account for multiple hypothesis tests, we apply the Bonferroni correction all reported p-values to control the family-wise error rate. The complete set of corrected p-values is provided in Appendix A.3 In summary, at significance level of 0.05, the results confirm that DEMO achieves statistically significant improvements over 10 bascline methods in the Superconductor task, 14 in the Ant task, 13 in the D'Killy (ask, 16 in the Levy lask, 11 in the TF8 task, 14 in the TF10 lask, and in the NAS task. We further examine the reliability of DEMO in Appendix A.4 Additional quantitative analysis of the predicted and ground-truth property scores of the pseudo design candidates and final candidates is provided in Appendix A.5) We also study the influence of the number of gradient ascent steps in the convergence study detailed in Appendix A.5
5.6 Ablation Study
New designs can be alternatively generated from the prior distribution using the diffusion model or by directly using the pscudo design candidates after the gradient ascent process. We refer to these two methods as Diflusion-only and Grad-only, respectively. To rigorously assess whether the introduction of the editing phase within our DEMO method is beneficial, ablation experiments are conducted by systematically comparing the complete DEMO method with Diffusion-only and Grad-only. The results, summarized in Table 2a provide clear insights into the impact of our design editing process. For the four continuous tasks, DEMO consistently achieves higher performance compared to its ablated versions. For instance, in the SuperC
10

---

Published in Transactions on Machine Learning Research (04/2025)
Task
DEMO
Grad-only
Diffusion-only
E5
REIM REIMFORCE
SuperC
0.525 = 0.009
0,482 0.013
0.346 = 0.011
Meai
Min
Aut
0.968 = 0.009
0.963 = 0.008
0.377 = 0.004
COMs
ROMA
D'Kitty
0.970
0.007
0.933 0.002
0.762 + 0.023
NEKO
Levy
1.007 = 0.015
0.990 = 0.020
0.483 + 0.015
TF8
0.980
0.004
0.965 + 0.008
0.420 + 0.004
TF10
0.762
0.058
0.638 + 0.019
0.465 + 0.006
CbAs
Auto ChAs
NAS
0.753 0.017
0.668 = 0.084
0.274 0.013
MIN
DOOM
BOME
DEMO
10 12133411
Rank
(a) Ablation studies of DEMO.
(b) Comparison of ranks methods.
Figure
10
Effectiveness (mean ranking!
Task
Dimension
Memory
Training
Optimization
SuperC
(17014, 86)
1179
235.92
Ant
(10004, 60)
1177
220.82
0.82
D'Kitty
(10004, 56)
1177
222.68
0.68
Levy
(15000, 60)
1177
227.80
0.80
X
Mean Ti-mentoring
TF8
(32898, 8)
1177
253.81
0.81
DDOM
TF10
(30000, 10)
1177
311.73
0.73
DEMD (aurs)
NAS
(1771, 64)
1201
233.81
0.81
50
100
150
200
250
300
Efficiency (seconds)
(a) Dimension records the number of samples in the offline dataset of each task and the corresponding input dimension. Memory summarizes the GPU usage in MB for training the models, and the last two columns demonsttate the time used for training models and optimizing candidate designs in seconds, respectively.
(b) Comparison of effoctiveness-effciinnn. trade- off with other representative methods. The left- bottom corner represents that methods are both effective (with higher
ranking) and elfi- cient (less training and optimization latency).
Figure 3: Computational efficiency analysis.
lask, DEMO achieves a score of 0.525 + 0.009, significantly higher than both Grad-only (0.482 +: 0.013) and Diffusion-only (0.346 = 0.011). Unlike baseline methods of conditional generative models based on a target score higher than all existing designs, such as Auto CbAS and DDOM, the Diffusion-only approach generates new designs conditionod on the maximum score within the offline dataset for the ablation study purpose. Similar improvements
observed in the Ant. D'Kitty, and Levy tasks, underscoring the effectiveness of integrating the design editing process in continuous tasks. In the discrete tasks TF8, TF10, and NAS, DEMO's superior performance over both alternative solutions is evident, highlighting its comprchensive effectiveness in managing discrete challenges. Overall, the ablation studies validate the importance of the design editing process within the DEMO method, answering Q2 conclusively. The complete DEMO method collectively contributes to enhancements across a range both continuous and discrete tasks and various input dimensions.
5.7 Computational Efficiency Analysis
To evaluate the computational efficiency and scalability of DEMO, we additional experiments workstation with a single Intel Xcon Platinum 8160T CPU and a single NVIDIA Tesla V100 GPU. The results are summarized in Table 3a. The table reports the sizes of the oflline dalaset, the corresponding input dimension, the GPU memory usage for training models in megabytes (MB), and the time for training models and optimizing design candidates in seconds. Furthermore, we closcly examine the time efficiency for optimizing design candidates of another diffusion- method, DDOM, on every task. DDOM takes 2.35 seconds in the Superconductor task, 2.21 seconds in the Ant. task, 1.84 seconds in the D'Kitty task, 2.07
11

---

Published in Transactions on Machine Learning Research (04/2025)
seconds in the Levy task, 1.89 seconds in the TF Bind 8 task, 1.83 seconds in the TF Bind 10 task, and 1.85 seconds in the NAS task. This indicates that our DEMO is more time-efficient than DDOM in every task for optimizing design candidates. In addition, Figure 3b illustrates the effectiveness-eficcene compared to DDOM and two representative surrogate-based methods, Mcan Ensemble and Tri-mentoring Mean Ensemble is a widely used traditional method for offline MBO that aggregates predictions from an ensemble of surrogate models to provide robust estimates, while Tri-mentoring is more recent bascline that has consistently achieved the best mean and median ranking among all surrogate-based methods in our experiments. These two methods are chosen because they represent both the established and state-of-the-art performance of surrogate-baset approaches in offline optimization. As shown, although DEMO requires slightly more runtime than other selected methods, it achieves significantly higher effectiveness within the same time scale, as indicated by its superior mean ranking. It is noteworthy that Tri-mentoring exhibits similar time efficiency to DEMO, which may be attributed to its use of pairwise ranking information and bi-level optimization. These results demonstrate that DEMO not only mitigates the out-of-distribution challenges in offline model-based optimization but does with practical computational demands, making it viable solution for real-world applications.
trade-off for DEMO
6 Conclusion and Discussion
In this study, we introduce Design Editing for Offline Model-based Oplimization (DEMO). DEMO begins by training a surrogate model on the offline dataset as an estimate of the ground-truth black- box objective function. A batch of pseudo design candidates is then generated by performing gradient ascent with respect to the surrogate model. Subsequently, diffusion model is trained on the offline dataset to capture the distribution of all existing valid designs. The design editing process introduces random noise to the pseudo design candidates and employs the learned diffusion model to denoise them, ensuring that the final optimized designs are not far from the prior distribution and have valid high scores. In essence, DEMO generates now designs by first extensively oplimizing pseudo design candidates and then refining them with the diffusiio prior. Experiments on the design-bench dataset show that, with properly tuned hyperparamters, DEMO's score competitive with the best previously reported scores in the literature. We further discuss the connection between our method and Bayesian inference in Appendix A.6 One limitation of our proposed DEMO method is that its performance depends on the tuning of the hyperparameter amount of noise added during the design editing process, ensuring that the final design candidates are not extremely influenced by the extrapolation error of the surrogate while also avoiding an overly conservative edit that would revert them back to the low-scoring regime. is important to note that such hyperparameter tuning for regularizations to balance overestimation and averconservatism not unique to our approach; other methods in offline MBO may face similar challenges. Nonetheless, our experiments show that when m is appropriately tuned, DEMO achieves favorable trade-off and delivers competitive performance. We discuss additional limitations and potential negative impacts in Appendix A.7 and Appendix A.8 respectively.
which controls the
12

---

Published in Transactions on Machine Learning Research (04/2025)
References
Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, and Vikash Kumar. Robel: Robotics benchmarks for learning with low-cost robots. In Conf. on Robot Lea. (CoRL), 2020.
Christof Angermüller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Colwell. Model-based reinforcement learning for biological sequence design. In Sth International Conference on Learning Representations, TCLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. Open net, 2020.
Luis A Barrera et al. Survey of variation in human transcription factors reveals prevalent dna binding changes. Science, 2016.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. ArXiv preprint, 00t//6606001440,, 2016.
David H. Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 773 782. Pmlr, 2019.
Duygu Ceylan, Chun-Hao Paul Huang, and Niloy J. Mitra. Pix2video: Video editing using image diffusion. In JEEE/CV /CVV International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp.
Ieee, 2023. doi:
Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, and Janardhan Rao Doppa. Offline model-based optimization via policy-guides gradient search. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (eds.), Thirty-Eightt AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourleenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pp. 11230-11230. AAAI Press, 2024. doi: 10.1609/ /aaii. v38110.20001.
Can Chen, Yingxue Zhang, Jie Fu, Xue (Steve) Liu, and Mark Coates. Bidirectional learning for offline infinite-width model-based optimization. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022.
Can Chen, Christopher Beckham, Zixuan Liu, Xue (Steve) Liu, and Chris Pal. Parallel-menttringg for offline model- optimization. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (ods.), Advances in Neural Information Processing Systems 36: Annual Conference Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023a.
Can Chen, Yingxue Zhang, Xue Liu, and Mark Coates. Bidirectional lcarning for offline model-based biological sequence design. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proccedings of Machine Learning Rescarch, pp. 5351-5366, Pmlr, 2023b.
Yuren Cong, Mengmeng Xu, Christian Simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Mamuel Pérez- Rúa, Bodo Rosenhahn, Tao Xiang, and Sen He. FLATTEN: optical flow-guided attention for consistent text-lo-video editing. In The Twelfth international Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.ner net, 2024.
Cuong Dao, Phi Le Nguyen, Truong Thao Nguyen, and Nghia Hoang. Incorporating surrogate gradient norm to improve offline optimization techniques. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Urrich Paquet, Jakub M. Tomczak, and Cheng Zhang (ods.), Advances Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024a.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Manh Cuong Dao, Phi Le Nguyen, Truong Thao Nguyen, and Trong Nghia Hoang. Boosting offline optimizers with surrogate sensitivity. In Forty-first International Conference on Machine Learning, ICML, 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024b.
Manh Cuong Dao, The Hung Tran, Phi Le Nguyen, Thao Nguyen Truong, and Trong Nghia Hoang. From
Low To High-value Designs: Offline Optimization Via Generalized Diffusion, 2025.
Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model-based design. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Alsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Justin Fu and Sergey Levine. Offline model-based optimization via normalized maximum likelihood estimation. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.neg net, 2024.
Kam Hamidieh. A data-driven stalistical model for predicting the critical temperature of a superconductor. Computational Materials Science, 2018.
Nikolaus Hansen. The CMA evolution strategy: A comparing review. In Towards a New Evolutionary Computation: Advances the Estimation of Distribution Algorithms, 2006.
Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580. 2012.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusson probabilistis models. In Hugo Larochelle, Mare'Auree Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Amnual Conference Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, and Trong Nghia Hoang. Learning surrogates for offline black-box optimization via gradient matching. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. Open nnc, 2024.
Chin-Wei Huang, Jae Hyun Lim, and Aaron C. Courville. A variational perspective on diffusion-based generative models and score matching. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 22863 22876, 2021.
Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. ArXiv preprint, abs/23100.113 2023.
Jungtaek Kim. BayesO Benchmarks: Benchmark functions for Bayesian optimization, 2023.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Adilya Grover. Difflusion models for black-box optimization. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (cds.), International Conference on Machine Learning, ICML, 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 17842-178577 Pmlr, 2023.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Aviral Kumar and Sergey Levine. Model inversion networks for model-based optimization. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systemns 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Seul Lee, Jaehyeong Jo, and Sung Ju Hwang. Exploring chemical space with score-based out-of-dittribution generation. In Andreas Krause, Em Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proccedings of Machine Learning Research, pp. 18872-18892 Pmlr, 2023.
Thomas Liao, Grant Wang, Brian Yang, Rene Lee, Kristofer Pister, Sergey Levine, and Roberto Calandra. Data-efficient learning of morphology and controller for a microrobot. ArXiv preprint, 0bss/1905.011 2019.
Satvik Mehul Mashkaria, Siddarth Krishnamoorthy, and Aditya Grover. Generative pretraining for black-box optimization. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volune 202 of Proceedings of Machine Learning Research, pp. 24173 24197. Pmlr, 2023.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Daiki Miyake, Akihiro lohara, Yu Saito, and Toshiyuki Tanaka. Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models. ArXiv preprint,
2023.
Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 1038-6047, Icce, 2023. doi: 111100/ccp5527792223..000
Tung Nguyen, Sudhanshu Agrawal, and Aditya Grover. Expt: Synthetic pretraining for few-shot experimental design. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. In IEFE CREE/CVE /CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 15886-15896 Icce, 2023. doi:
10.1109//ccw51007.22222 000.000000000000044400000.
Han Qi, Yi Su, Aviral Kumar, and Sergey Levine. Data-driven offline decision-making via invariant repre- sentation learning. Tn Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (cds.), Advances in Neural Information Processing Systems 35: Annual Conference Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022a.
Han Qi, Yi Su, Aviral Kumar, and Sergey Levine. Data-driven model-based optimization via invariant representation learning. In Proc. Adv. Neur. Inf. Proc. Syst (NeurIPS). 2022b.
Paul J Sample, Ban Wang, David W Reid, Vlad Presnyak, lain J McFadyen, David R. Morris, and Georg Seelig. Human UTR design and variant effect prediction from a massively parallel translation assay. Nature Biotechnology, 2019.
Karen S Sarkisyan et al. Local fitness landscape of the green fluorescent protein. Nature, 2016.
15

---

Published in Transactions on Machine Learning Research (04/2025)
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net, 2021a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc, Emily B. Fax, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 11895 11907, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.nct 2021b.
Xuan Su, Jiaming Song, Chenlin Meng, and Stefano Ermon. Dual diffusion implicit bridges for image-to-image translation. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 10858-10366 Pmlr, 2021.
Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-benche Benchmarks for data-driven offline modcl-based optimization. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 21658 21676. Pmir, 2022.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23 (7):1661-1674, 2011. doi: 10.1162/NECO __0422.
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 1992.
James T Wilson, Riccardo Moriconi, Frank Hutter, and Marc Peter Deisenroth. The reparameterization trick for acquisition functions. ArXiv preprint, abs/1772.00422, 2017.
Chen Henry Wu and Fernando De la Torre. A latent space of stochastic diffusion models for zero-shot image editing and guidance. In IEFE/CVP International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 7344 7353. leee, 2023. doi:
Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender video: Zero-shot text-guided video-to-video translation. ArXiv preprint,
2023.
Michael S. Yao, Yimeng Zeng, Hamsa Bastani, Jacob R. Gardner, James C. Gee, and Osbert Bastani. Generative adversarial model-based optimization via source critic regularization. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024.
Sihyun Yu, Sungsoo Alm, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for oflline model-based optimization. In Mare Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jen- nifer Wortman Vaughan (eds.). Advances Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 4619- 4631, 2021.
16

---

Published in Transactions on Machine Learning Research (04/2025)
Ye Yuan, Can Chen, Zixuan Liu, Willic Neiswanger, and Xue (Steve) Liu. Importance-aware co-teaching for of- fline model-based optimization. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Anmual Conference on Noural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
Ye Yuan, Can Chen, Christopher Pal, and Xue Liu. Paretollow: Guided flows in multi-objective oplimization, 2024.
Youyuan Zhang, Xuan Ju, and James J Clark. Fastvideoedit: Leveraging consistency models for efficient. text-to-vidoo editing. ArXiv preprint, abs/2403.062699 2024.
Barret Zoph and Quoe V. Le. Neural architecture search with reinforcement learning. 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
17

---

Published in Transactions on Machine Learning Research (04/2025)
Table 3: Experimental results on continuous tasks for comparison.
Method
Superconductor
Ant Morphology
D D'Kitty Morphology
Levy
D(best)
0.399
0.565
0.884
0.613
BO-qEI
0.300 = 0.015
0.567 = 0.000
0.883 0.000
0.643 + 0.009
CMA-ES
0.379 + 0.003
-0.045 + 0.004
0.684 0.016
0.410 + 0.009
REINFORCE
0,463 = 0.016
0.138 = 0.032
0.356 = 0.131
0,377 = 0.065
Mean
0.334 1: 0.004
0.569 0.011
0.876 0.005
0.561 1: 0.007
Min
0.364 = 0.030
0.569 = 0.021
0.873 0.009
0.537 + 0.006
COMs
0.316 1 0.024
0.564 + 0.002
0.881 0.002
0.511 +: 0.012
ROMA
0.370 + 0.019
0.477 0.038
0.854 0.007
0.558 + 0.003
NEMO
0.320 = 0.008
0.592 = 0.000
0.883 = 0.000
0.538 = 0.006
BDI
0.412 + 0.000
0.471 0.000
0.855 0.000
0.531 + 0.003
IOM
= 1350
0.513 = 0.035
0.876 0.006
0.562 + 0.007
ICT
0.399 1 0.012
0.592 1 0.025
0.874 1 0.005
0.691 1: 0.009
Tri-mentornng
0.355 + 0.003
0.606 0.007
0.866 0.001
0.687 + 0.012
PGS
0.379 + 0.016
0.532 = 0.016
0.941 + 0.008
0.476 + 0.014
CbAS
0.111 0.011
0.384 0.016
0.753 0.008
0.479 + 0.020
Auto CbAS
0.131 0.010
0.364 0.014
0.736 0.025
0.499 + 0.022
MIN
0.336 + 0.016
0.618 + 0.040
0.887 0.004
0.681 + 0.030
DDOM
0.346 = 0.009
0.615 0.007
0.861 0.003
0.595 + 0.012
BONET
0.369 1 0.015
0.819 0.032
0.907 1 0.020
0.604 1 0.008
DEMO (ours)
0.400 + 0.007
0.604 + 0.005
0.891 0.002
0.762 1 0.008
Table 4: Experimental results discrete lasks, and ranking all tasks for comparison.
Method
TF Bind 8
TF Bind 10
NAS
Rank Mean
Rank Median
D(best))
0.439
0.467
0.436
BO-qEI
0.439 + 0.000
0.467 = 0.000
0.544 = 0.099
9.4/19
10/19
CMA-ES
0.537 + 0.014
0,484 = 0.014
0.591 + 0.102
10.6/19
7/19
REINFORCE
0.462 0.021
0.475 1 0.008
1.895 0.000
13.7/19
18/19
Mean
0.539 1 0.030
0.539 1 0.010
0.494 1: 0.077
8.3/19
8/19
Min
0.569 + 0.050
0.485 + 0.021
0.567 + 0.006
7.3/19
8/19
COMs
0.439 + 0.000
0.467 0.002
0.525 1 0.003
11.1/19
11/19
ROMA
0.555 + 0.020
0.512 0.020
0.525 + 0.003
8.6/19
7/19
NEMO
0.438 + 0.001
0.454 0.001
0.564 0.016
1044/19
11/19
BDI
0.439 + 0.000
0.476 0.000
0.517 + 0.000
10.4/19
10/19
IOM
0.439 1 0.000
0.477 1 0.010
0.050 1 0.011
11.0/19
10/19
ICT
0.551 + 0.013
0.541 + 0.004
0.494 + 0.013
5.6/19
4/19
Tri-mentoring
0.609 1 0.021
0.527 = 0.008
0.516 + 0.028
6.1/19
4/19
PGS
0.375 + 0.014
0.443 0.005
0.508 + 0.017
12.0/19
12/19
CbAS
0.428 + 0.010
0.463 + 0.007
0.292 + 0.027
16.3/19
16/19
Auto CbAS
0.119 1 0.007
0.461 4 0.007
0.217 + 0.005
16.9/19
17/19
MIN
0.421 + 0.015
0.468 + 0.006
0.433 0.000
9.3/19
12/19
DDOM
0.401 1 0.008
0.464 1 0.006
0.306 0.017
11.9/19
13/19
BONET
0.505 + 0.055
0.496 + 0.037
0.571 + 0.095
4.6/19
5/19
DEMO, (ours)
0.533 0.010
0.480 = 0.003
0.564
4.4/19
4/19
A Appendix
A.1 Median Normalized Scores
Performance in Continuous Tasks. Table 3 showcases the median normalized scores for various baseline methods across contimuus tasks. DEMO, while not always topping the charts, demonstrates robust performance across these tasks, consistently outperforming several bascline methods. For example, in the Levy function task, DEMO's score of 0.762 + 0.008 is the highest one among all approaches. This highlights DEMO's capability to mitigate the OOD issue of surrogates based methods effectively. Notably, DEMO
18

---

Published in Transactions on Machine Learning Research (04/2025)
Superconductor
TF Bind 8
0.525
0.500
Normalized Score
Normalized Score
0.9
0.475
0.8
0.450
DEMO (ours)
DEMO (ours)
Grad-only
0.7
Grad-only
0.425
Difrusi on-only
Diffusion-only
0.400
0.375
0.5
0.4
200
400
600
800
1000
0
200
400
600
800
1000
Value of m
Value of m
Figure Selecting m near results generated designs that retains most properties of pseudo design candidates. Conversely, setting m near 1000 generates designs that align closely with the distribution of existing designs. Optimal designs are achieved by choosing m in the mid-range, effectively utilizing information from both the pseudo design candidates and the diffusion prior.
outperforms traditional generative models like CbAS and Auto CbAS by significant margins across all tasks. It also maintains competitive edge against more recent generative methods like MIN and DDOM.
Performance in Discrete Tasks. Moving to discrete tasks, detailed in Table 4 DEMO exhibits performance on par with other baseline methods. This performance can be attributed to DEMO's method- ology which, although highly effective in calibrating the pscudo design candidates, might struggle in task enviromments with redundancy in design features.
Summary. The results presented in Tables 3 and collectively validate DEMO's efficacy across both continuous and discrete optimization lasks, providing further support for answering Q1 affirmatively. With a mean rank of 4.4/19 and a median rank of 4/19 in terms of the median normalized scores, DEMO stands out among 19 competing methods. This comprehensive performance underscores DEMO's capacity to integrate and leverage information from the distribution of existing designs and pseudo design candidates.
A.2 Sensitivity to the Choice of m
In Eq. selecting a time m close to M results in &perturb resembling random Gaussian noise, which introduces greater flexibility into the new design generation process. On the other hand, if m is closer to 0, the resulting design retains more characteristics of the pscudo design candidates. Thus, m serves as a critical hyperparameter in our methodology. This section explores the robustness of DEMO to various choices of We perform experiments to 1000 in increments of 100. As illustrated in Figure 4 DEMO generally outperforms the Diffusion-only and Grad-only methods, Nevertheless, overly extreme values of m, whether too high or too low, can diminish performance. Selecting an excessively low m causes the model to adhere too closely to the pseudo design candidates, while choosing an overly high m biases the model towards the distribution of existing designs, neglecting the guidance of pseudo design candidates. Choosing m from mid-range effectively balances the influences from both the prior distribution and the pseudo design candidates, leading set 600 for all tasks.
continuous task, SuperC, and one discrete task, TF8, with ranging from 0
A.3 Details of Corrected P-Values
As mentioned in the main lext, all reported p-values are adjusted using the Bonferroni correction to account for multiple hypothesis testing. Specifically, each uncorrected p-value is multiplied by the total number of null hypotheses, which corresponds to the number of bascline methods (18). Table 5 presents the corrected p-values from comparisons between DEMO and other bascline methods. We bold the values where DEMO demonstrates statistically significant. improvement the corresponding baseline method at significance
19

---

Published in Transactions on Machine Learning Research (04/2025)
Table 5: Corrected p-values on all tasks.
Method
Superconductor
Ant
D D'Kitty
Levy
TF8
TF10
NAS
BO-qEI
0.000
0.000
0.000
0.000
0.003
0.007
0.000
CMA-ES
0.001
3.362
0.000
0.000
0.091
0.021
0.000
REINFORCE
0.000
0.000
0.005
0.000
0.111
0.013
0.000
Mean
0.032
0.004
0.264
0.319
0.000
0.007
0.025
Min
0.081
0.035
0.000
0.007
0.042
0.002
0.017
COMs
0.023
0.000
0.002
0.000
0.000
0.002
0.035
ROMA
0.221
0.013
0.000
0.027
0.027
0.021
0.000
NEMO
0.001
0.046
0.000
0.006
0.001
0.306
0.269
BDI
0.063
0.000
0.000
0.000
0.000
0.001
0.012
TOM
3.496
0.027
0.002
0.524
0.030
0.005
0.000
ICT
0.075
0.952
7.162
0.000
0.030
0.093
0.289
Tri-mentoring
1.370
0.048
3.344
0.001
0.645
0.875
7.866
PGS
0.970
0.162
4.140
0.018
8.093
0.010
0.670
CbAs
3,600
0.000
0.000
0.007
0.168
0.019
0.371
Auto CbAs
0.002
0.008
0.000
0.000
0.017
0.002
0.000
MIN
0.027
0.000
0.000
0.000
0.033
0.001
0.613
DDOM
0.000
0.014
0.000
0.001
0.553
0.003
0.538
BONET
0.000
1.398
0.067
0.000
2.383
0.052
0.013
DEMO (ours)
10
14
13
16
11
14
7
1.0
Seaticaly
JEMO louns
Proportion
Propartion
0.6
0.6
0.4
0.4
Supert
Ant
D'Kitty
Levy
TF8
TF10
MAS
SuperC
Ant
D'kitty
Levy
TF8
TF10
NAS
Figure 5: The proportion calculated as the num- ber of new designs which surpass D(best) divided by the budget 128, indicating the reliability to consistently generale new higher-scoring designs. This figure demonstrates that DEMO is more reli- able than Diffusion-only
Figure 6: The proportion calculated the mum- ber of new designs which surpass D(best) divided by the budget 128, indicating the reliability to consistently generate new higher-scoring designs. This figure demonstrates that DEMO is more reli- able than Grad-only in 5/7 tasks.
tasks.
level of 0.05. The last row summarizes the number bascline mothods for which DEMO achieves statistically significant improvement in each lask.
A.4 Reliability Study
In this subsection, we assess the ability of DEMO to reliably produce superior designs compared to selected surrogate based methods and generative model based methods. To measure reliability, we compute the proportion of new designs that exceed the best scores the offline dataset D(best). The results are depicted in Figure] DEMO consistently outperforms Diffusion-only across all tasks, achieving notable improvements, particularly in the SuperC and NAS tasks. This confirms DEMO's enhanced reliability over the state-of-the-art generative model-based baseline in both continuous and discrete settings. We then extends the reliability study to compare DEMO with a gradient-based based approach. When compared to Grad-only, DEMO demonstrates greater consistency in out of 7 tasks. However, Grad-only outperforms DEMO in Levy and TF10 tasks, which can be attributed to the gradient- based method's tendency to generate new designs within narrower distribution. While Grad-only achieves higher proportion of higher-scoring new designs in these two tasks, DEMO generates new designs within a wider distribution and thus produces candidates with higher maximum scores, evidenced in Table 2
20

---

Published in Transactions on Machine Learning Research (04/2025)
A.5 Quantitative Analysis and Convergence Study
Table 6: Quantitative Analysis and Convergence Study for Ant
Gradient Ascent Step
100
200
300
Predicted Score of Pseudo Design Candidates
0.950
1.210
1.340
Ground-truth Score of Pseudo Design Candidates
0.942
0.887
0.845
Ground-trute truth Score of Edited Final Design Candidates
0.968
0.935
0.928
Table Quantitative Analysis and Convergence Study for TF8
Gradient Ascent Step
100
200
300
Predicted Score of Pseudo Design Candidates Ground-truth Score of Pseudo Design Candidates
2.916
5.570
8.311
0.912
0.895
0.895
Ground-truth Score of Edited Final Design Candidates
0.982
0.958
0.914
We run additional experiments on one continuous task Ant and one discrete task TF8. As shown in Table G and Table 7 the pseudo design candidates usually have over optimistic predicted score, but their ground-truth scores evaluated by the oracle are not such high. After editing the pseudo design candidates by our approach, the ground-truth scores are effectively increased.
In our original setting, we run 100 gradient ascent steps to acquire the pseudo design candidates. 200 and 300 gradient ascent steps are considered in the additional experiments, where we can mimic the distribution very far away from the training distribution. As we can see from Table G and Table 7 cven though the performance degrades as the number of gradient ascent steps increases, our proposed approach still helpful for editing the pseudo design candidates and thus improving the quality of the final candidates.
A.6 Connection to Bayesian Inference
From a Bayesian perspective, our objective is to find the design æ that maximizes the posterior probability:
p(ælu) x p(v)=))((r),
(10)
where p(y]az) is the likelihood of obtaining property score y given design æ and p(x) is the prior distribution over designs learned from the offline dataset. Taking the logarithm, we have:
logp(æly) log p(ylx) + log ((æ)( + +C,
(11)
with C. being a constant that does not alTect the optimization. To find the design that maximizes the posterior,
we aim to solve:
at = arg max [log =EX [log p(y logr(
(12)
In the offline model-based optimization (MBO), we typically do not have a direct expression for p(y (z).
Instead, we train a surrogate model fe(x) to predict the property score y. By following Lee et al. (2023 |;
Yuan et al. 2024). we can model the probability density using the Boltzmann distribution as follows:
p(yl=) po(v)=) e(f((2)
(13)
Z
where y is the scaling factor, and Z the normalization constant. Taking the logarithm of this expression, obtain:
logp(y(=)) mz) (z) log Z.
(14)
21

---

Published in Transactions on Machine Learning Research (04/2025)
To maximize the posterior, we wish to solve Eq. 12 In practice, can perform gradient ascent. By differentiating the log-posterior, the update rule becomes:
Lt+l = St + 1 V [log((])) + p(r)( ===i
(15)
where v is the learning rate. Since ply x) is approximated by the surrogate model as discussed above, we have:
Vn logply/z) R VaJo(a)),
(16)
and the diffusion model is trained to capture the prior, yielding an approximation:
V log)(æ) x sa 0),
(17)
where (a 0) is the score function learned by the diffusion model on the original data at time 0. At t = 0, no noise has been added, so the score function idcally estimates the gradient of the log probability of the original data distribution. It is worth noting that, in practice, diffusion models are trained learn the score function over contimuous range of time steps. Although t = 0 represents the ideal scenario, small nonzero l should be used for numerical stability. This update can be interpreted as performing approximate maximum a posteriori (MAP) estimation, where the first term drives the design toward higher predicted scores, namely maximizing the likelihood. The second term enforces that the design remains within the valid design manifold, that is incorporating the prior.
In Bayesian perspective, DEMO can be seen as performing an approximate posterior inference in two steps: (1) Posterior Approximation, that is the acquirement of pseudo design candidates, using the surrogate to imagine where high-score probability mass might lie, and (2) Posterior Refinement, namely the design editing process, using the diffusion prior to correct and refine those candidates. As detailed above, while a single-phase Bayesian inference formulation offers an elegant theoretical framework by directly optimizing the combined log-posterior, our two-phase approach in DEMO provides practical advantages. In the single-phase formulation, the update rule simultaneously balances the gradient of the likelihood, derived from the surrogate model, and the gradient of the prior, obtained via the diffusion model, within a single unified update. This balance can be challenging to achieve in practice, which may require additional hyperparameters to trade-off. More importantly, directly applying the diffusion prior in the update rule may not be enough to edit the out-of-distribution candidates back to the valid range. In contrast, our two-phase method decouples these tasks: the first phase focuses exclusively on generating high-scoring candidates via gradient ascent on the surrogate model, while the second phase leverages the diffusion model lo edit and refine these candidates, ensuring they remain within the valid design manifold. This modular design simplifies the optimization process by isolating each objective, achieving a more stable and effective balance between high performance and design realism compared to direct ingle-phase Bayesian optimization.
A.7 Limitations
We have demonstrated the effectiveness of DEMO across a wide range of lasks. However, some evaluation methods may not fully capture real-world complexities. For example, in the superconductor task (Hamidieh) 2018), we follow traditional practice by using random forest regression model as the oracle, done in prior studies (Trabucco et al., 2022). Unfortunately, this model might not entirely reflect the intricacies of real-world situations, which could lead to discrepancies between our oracle and actual ground-truth outcomes. Engaging with domain experts in the future could help enhance these evaluation approaches. Nevertheless, given DEMO's straightforward approach and the empirical evidence supporting its robustness and efficacy across various tasks detailed in the Design-Bende Bench (Trabucco et al., 2022) and BayesO Benchmarks (Kim we remain confident in its ability generalize effectively to different contexts.
2023
A.8 Negative Impacts
This study secks to advance the field of Machine Learning. However, it's important to recognize that advanced optimization techniques can be used for either beneficial or detrimental purposes, depending on their application. For example, while these methods can contribute positively to society through the development.
22

---

Published in Transactions on Machine Learning Research (04/2025)
of drugs and materials, they also have the potential to be misused to create harmful substances or products. As researchers, we must stay aware and ensure that our contributions promote societal betterment, while also carefully assessing potential risks and ethical concerns.
23


Reviews:
[{"review_id": "9Qi5T1J5Iz", "reviewer": "Reviewer_YKnZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper identifies out-of-distribution errors as a key challenge in offline MBO and explains how surrogate models often extrapolate unreliably when optimizing outside the training data distribution. The idea of using a diffusion prior to project pseudo design candidates back into the valid distribution is a conceptually sound and novel approach to mitigating these issues. The paper also includes a thorough empirical evaluation across seven diverse MBO tasks, covering both continuous and discrete domains. The paper is well-written and easy to understand. The figures are also clear and informative.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents Design Editing for Offline Model-based Optimization (DEMO), a novel method to improve offline model-based optimization (MBO) using gradient ascent on a surrogate model to generate pseudo design candidates. The model is able to address out-of-distribution (OOD) issues by refining pseudo design candidates through diffusion-based editing. Empirical validation across multiple MBO tasks show performance improvements over existing optimization methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "While the results show that DEMO outperforms baselines, there is limited discussion on computational efficiency and scalability. Since diffusion models are typically more expensive than traditional optimization approaches, an analysis of runtime, memory usage, and efficiency trade-offs would help assess its practicality for large-scale problems.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HHQaeJG0om", "reviewer": "Reviewer_6yVc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nFrom a technical perspective, the paper presents an approach that effectively combines gradient-based optimization with diffusion models, providing a way to address the out-of-distribution problem. The theoretical justification for staying near the data manifold is well-reasoned and clearly explained. The empirical validation is particularly strong, with comprehensive experiments across seven different tasks, thorough ablation studies, and statistical significance tests demonstrating meaningful improvements. The method achieves impressive performance metrics. The paper is also well-written with clear motivation, problem setup, and effective visualizations that help explain the method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces DEMO (Design Editing for Offline Model-based Optimization), which combines gradient-based optimization with diffusion models in a two-phase approach for offline model-based optimization. The first phase generates pseudo design candidates through gradient ascent on a surrogate model, while the second phase uses a diffusion prior to \"edit\" these candidates through a noise-and-denoise process to ensure they remain within the valid design distribution. The key insight driving this approach is that designs closer to the observed data manifold are more likely to be genuinely good rather than artifacts of model extrapolation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Theoretical Framework: Is it possible to formulate DEMO as a single-phase Bayesian inference problem with diffusion as prior and gradient descent on posterior given y. \n\n2. Quantitative Analysis: What's the difference of expected scores and oracle scores between pseudo-candidates and diffusion-edited designs? This would better justify the two-phase approach.\n\n3. Convergence Study: Examine how gradient descent convergence in first phase affects diffusion editing benefits. Does editing remain helpful with very long gradient descent runs (i.e. very far away from training distributions)?", "broader_impact_concerns": "I don't see any potential concerns here.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TYPG2NR2Cd", "reviewer": "Reviewer_4Lsc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\n1. The paper is well-written.\n2. The idea might be relatively new but the concept of design editing is not quite well-positioned with related work (see 2nd weakness)\n3. The overall workflow is well-illustrated. I like figure 1 which is nice & informative.\n4. The reported experiments appear strong.\n\nWeaknesses:\n\nFirst, it is unclear why the proposed idea work. From 4.3, the authors use the standard forward diffusion process to add Gaussian noise to the original input and then, learn a backward process to denoise it back. \n\nAs such, this is a vanilla diffusion prior mapping back and forth between the Gaussian noise distribution and the implicit offline data distribution. Why does it help map from the base designs to high-scoring design if it was not trained to do so?\n\nSecond, I believe this is also not the first work on design editing. For example, prior to this work, this has been explored here:\nhttps://openreview.net/pdf?id=K9Elg2JrvY\n\nThe above work explicitly trained a generalized diffusion process to map between the distributions of low- and high-value designs. The authors should at least position with this work, i.e., why does learning to map back and forth between Gaussian noises and offline data would have similar effect of mapping between low- and high-value designs during inference?\n\nThird, the authors do miss quite a number of more recent works (aside from the above) in their related work section. To point out a few:\n\n1. ExPT: https://arxiv.org/abs/2310.19961 (NeurIPS-23) \n2. Match-OPT: https://openreview.net/pdf?id=mv9beA1wDF (ICML-24)\n3. BOSS: https://openreview.net/pdf?id=aLSA3JH08h (ICML-24)\n4. aSCR: https://arxiv.org/abs/2402.06532 (NeurIPS-24)\n5. IGNITE: https://openreview.net/pdf?id=ag7piyoyut (NeurIPS-24)\n\nWhile I do understand that it might be hard to compare empirically with all existing works, it is still important to provide a thorough positioning and focus the baseline on most relevant work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a method to address the offline optimization problem in which an offline dataset of a black-box function is given and we want to find its maxima. A naive approach is to learn a surrogate of this function using the offline data and then perform gradient ascent on the input to this surrogate to find its maxima. \n\nHowever, the main issue with this approach is that the surrogate might predict inaccurately as the gradient ascent process moves away from its offline data regime, which results in overestimation and suboptimal overall performance. The existing literature has provided numerous approaches to constraint the gradient ascent or search process in general such that it focuses more on the safe regime where the surrogate is reliable.\n\nUnder this context, this paper presents a design-editing method where top K maximizers of the surrogate are further edited via a diffusion process. The corresponding diffusion model seems to be learned to map between the offline data and Gaussian noises which however does not explain why it has the effect of mapping from base designs (maxima of surrogate) to designs with higher oracle value.\n\nOverall, this is a relatively interesting idea but not well justified and also not well-positioned against diffusion-related works. The empirical comparison has also ignored a number of more recent works in offline optimization (see my comments below).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please revise the manuscript to address my concerns above in the weakness sections. \n\n1. Most importantly, please provide a clear mathematical explanation to explain why does learning to map back and forth between Gaussian noises and offline data would have similar effect of mapping between low- and high-value designs during inference? \n\n2. A positioning & comparison with https://openreview.net/pdf?id=K9Elg2JrvY is in my opinion important here because this prior work addresses the same problem using the same concept.\n\n3. Please revise the related work section to include the missing references to provide a comprehensive review of literature.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "shX5pVdOTW", "reviewer": "Reviewer_5gy3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- Idea in the paper is reasonable\n- Experimental results are promising (although I have some issues with the precise claims)\n- Paper is reasonably well-written\n\nWeaknesses:\n\n- I don't think the empirical results fully support the claims about DEMO being state-of-the-art, first because the results don't all seem statistically significant, and second because the method is only tested at a single setting of hyperparameters. More details in \"requested changes\"\n- Offline MBO is a field full of very similar methods, which all try to generate high-scoring points, but regularized to be near the data manifold. A deeper analysis and explanation of DEMO's approach vs the approach of baseline methods would be nice. At the moment, §3 basically just mentions the existence of many papers.\n- This is somewhat of a personal opinion, but\n    1. I don't think offline MBO is a realistic problem (specifically, in any of the applications mention, is there really only one opportunity to generate designs, because if it is ≥2 then it is not offline anymore)\n    2. I don't think design bench tasks are particularly realistic (eg, in all my years of machine learning I have never heard of anybody doing offline neural architecture search).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Design Editing for Offline Model-based Optimization (DEMO). It is essentially a two stage procedure for offline black-box optimization:\n\n1. A surrogate model $\\hat f$ is trained on offline data, and then a candidate point $\\tilde x^*\\in\\arg\\max_x \\hat f(x)$ is chosen\n2. A diffusion model is learned to diffuse $\\tilde x^*$ closer to the data manifold, producing a final candidate $x^*$\n\nThe key assumption is that points closer to the data manifold are less likely to be \"spurious optima\" of the surrogate model (note: I think the paper would be improved if the authors explained this intuition more clearly). The authors evaluate their method on design-bench and find that it performs competitively.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think the empirical evidence does not support the claim from the introduction that:\n\n> Extensive experiments demonstrate DEMO effectively and reliably generates new designs, yielding state-of-the-art results across 7 offline MBO tasks, with the mean rank of 2.1 and the median rank of 1 among 19 methods.\n\nIn my opinion, a more honest summary of the results would be:\n\n> Experiments on the design-bench dataset show that, with properly tuned hyperparamters, DEMO's score is competitive with the best previously reported scores in the literature\n\nTo explain the differences:\n\n- I think it is an exaggeration to call design bench \"extensive experiments\"\n- The \"state-of-the-art results\" claims seem exaggerated, hence \"competitive\" instead of \"state-of-the-art\":\n   - Many performance differences seem small and likely not statistically significant (the most egregious example I spotted was TF Bind-8, where #1 rank is claimed for 0.982±0.016, compared with 0.981±0.015 for PGS).\n  - The t-test results seem wrong. For example, a p-value of 0.03 is claimed on TF8, despite the difference in means only being $10^{-3}$ and the standard errors of each estimate being $\\approx 10^{-2}$. It also looks like no correction for multiple tested hypotheses was done.\n- The results are obtained for only one set of hyperparameters (e.g. training length, weight decay coefficients, number of gradient steps to obtain candidate point, diffusion schedule). From my experience in ML I guess that the results are quite sensitive to the settings of these parameters.\n\nIf the authors would like to maintain their original claims I think more experimental evidence would be necessary.\n\nFinally, please write $10^{-3}$ instead of $1e-3$:\n\n$1e-3=e-3\\approx -0.281718$.", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers initially raised concerns about statistical methodology, performance claims, theoretical justification, and hyperparameter sensitivity, but ultimately recommended acceptance after the authors comprehensively addressed each issue by implementing proper statistical testing with Bonferroni corrections, revising performance claims from "state-of-the-art" to "competitive," adding Bayesian theoretical framing, and acknowledging limitations regarding hyperparameter tuning. The experimental validation across seven diverse MBO tasks demonstrates the method's effectiveness and generalizability, making it relevant to researchers in optimization, generative models, and their applications.

TMLR's audience would find value in this paper on Design Editing for Offline Model-based Optimization, which addresses the out-of-distribution challenge in offline MBO through a novel two-phase approach combining gradient-based optimization with diffusion models.
Decision: Accept (poster)