Preference-Guided Diffusion for Multi-Objective Offline Optimization
Authors: Yashas Annadani
Syrine Belakaria
Stefano Ermon
Stefan Bauer
Barbara E Engelhardt
Venue: NeurIPS 2025 poster ()
Keywords: Offline Optimization
Multi-Objective Optimization
Diffusion Models
Classifier Guidance
Primary area: general_machine_learning
TL;DR: We propose a classifier guided diffusion model for Offline Multi-Objective Optimization. The guidance classifier is a preference model that is defined based on the notion of pareto-dominance.

Abstract: Offline multi-objective optimization aims to identify Pareto-optimal solutions given a dataset of designs and their objective values. In this work, we propose a preference-guided diffusion model that generates Pareto-optimal designs by leveraging a classifier-based guidance mechanism. Our guidance classifier is a preference model trained to predict the probability that one design dominates another, directing the diffusion model toward optimal regions of the design space. Crucially, this preference model generalizes beyond the training distribution, enabling the discovery of Pareto-optimal solutions outside the observed dataset. We introduce a novel diversity-aware preference guidance, augmenting Pareto dominance preference with diversity criteria. This ensures that generated solutions are optimal and well-distributed across the objective space, a capability absent in prior generative methods for offline multi-objective optimization. We evaluate our approach on various continuous offline multi-objective optimization tasks and find that it consistently outperforms other inverse/generative approaches while remaining competitive with forward/ surrogate-based optimization methods. Our results highlight the effectiveness of classifier-guided diffusion models in generating diverse and high-quality solutions that approximate the Pareto front well.

Preference-Guided Diffusion for Multi-Objective Offline Optimization
Yashas Annadani 1.3 Syrine Belakaria²
Stefano Ermon
Stefan Bauer
Barbara Engelhardt
1 TU Munich
2 Stanford University
Heimholtz AI, Munich Gladstone Institutes
Abstract
Offline multi-objective optimization aims to identify Pareto-optimal solutions given a dataset of designs and their objective values. In this work, we propose preference-guiee diffusion model that generates Pareto-optimal designs by leveraging classifier-based guidance mechanism. Our guidance classifier is a preference model trained to predict the probability that one design dominates another, directing the diffusion model toward optimal regions of the design space. Crucially, this preference model generalizes beyond the training distribution, enabling the discovery of Pareto-optimal solutions outside the observed dataset. We introduce novel diversity-aware preference guidance, augmenting Pareto dominance preference with diversity criteria. This ensures that generated solutions are optimal and well-distributed across the objective space, a capability absent prior generative methods for offline multi-objective optimization. We evaluate our approach on various continuous offline multi-objective optimization tasks and find that it consistently outperforms other inverse/generative approaches while remain- ing competitive with forward/ surrogate-base based optimization methods, Our results highlight the effectiveness of bussifierrguided diffusion models in generating diverse and high-quality solutions that approximate the Pareto front well.
Introduction
Several design problems science and engineering require optimizing a black-box, expensive-to- evaluate function. For example, in antibiotic drug discovery, the goal is to identify novel molecules with high antibacterial activity (39]. This can be formulated as a single-ohjective optimization problem. However, in practice, most real-world design challenges involve balancing multiple conflicting objectives. For example, in drug discovery, in addition to maximizing antibacterial activity, we also want to minimize toxicity and production costs (38). This constitutes a multi- objective experimental design problem.
Prior work in both single and multi-objective optimization (MOO) has largely focused on adaptive experimental design using online methods such as Bayesian optimization [33]. These approaches rely on training surtogate models for each objective function and designing acquisition functions that are typically optimized via gradient-based techniques 3 or evolutionary algorithms to determine the next candidate for evaluation. This process is iteratively repeated to optimize the objectives. However, in many real-world applications, sequential evaluations where inputs are tested one time or in small batches-as are impractical. In some cases, we have only single opportunity evaluate the function, and we must allocate the entire evaluation budget efficiently.
"oorespondencce to yashas. annadani @ gmail.com Code available at https//gittbbbooyymmaaadnngggggu
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

For example, in drug design, scientists cannot test molecules one by one in wet lab experiments due to the high cost, slow turnaround, and the inherently parallelizable nature of the process [38.. Instead, it is common to evaluate all candidate molecules in a single batch. This setting is referred to as offline black-box optimization [4.. While recent work has explored offline optimization in the single-objective setring [23] 46],. where extensive prior data is leveraged to model the objective function and identify potential optima, the MO case remains relatively underexplored.
Offline black-box optimization presents distinct challenges compared to traditional online optimiza- tion. Since the algorithm cannot iteratively refine the learned model using newly acquired data, it must effectively leverage the available dataset to generalize beyond observed data points. This is particularly challenging because the true optima are often expected to lie outside the existing dataset, requiring robust extrapolation. Additionally, the goal is not merely to identify data points with high function val- ues but find solutions that satisfy a well-defined notion of optimality, such as the Pareto optimality.
Prior work in single-objective offline optimization can generally be categorized into two main approaches: forward and inverse methods. Forward approaches attempt mimic strategies used in online optimization while leveraging offline data [4]. These methods train surrogate model of the objective function and optimize using gradient-based techniques to propose a set of promising inputs for evaluation. They are effective when the search space is well-defined. In contrast, inverse inputs, approaches use generative models to learn an inverse mapping from function values
enabling the generation of new candidates with potentially high objective values [6] [6] 24 (25). This
distinction is critical when optimal inputs are not known in advance. For example, in chemistry, if the goal is to evaluate a known molecule, surrogate models are effective in predicting its properties. However, if the goal is to discover entirely new molecules with desired properties, inverse methods are essential, as they directly generate novel candidates rather than selecting from predefined space. Some generative modeling approaches also draw inspiration from online methods by constructing synthetic optimization trajectories from offline data, aiming to generate new optimal points by extrapolating from the learned trajectories 23
Multi-objecttee offline optimization introduces additional challenges beyond those encountered in the single-objective setting. In single-obeective optimization, the goal is simply to maximize (or minimize) a function value. However, in the multi-objectivg case, we seek to achieve best trade- offs among competing objectives, which is formalized by Pareto optimality. Beyond identifying Pareto-optimal solutions, another critical challenge is ensuring diversity on the Pareto front. A well-structured Pareto front should provide solutions spread across different regions of the objective space, representing broad range of Pareto-optimal designs. Although the problem of ensuring diversity of solutions is addressed in online MOO M. ensuring both optimality and diversity is challenging in the offline setting the algorithm must infer these solutions solely from existing data.
Xue et al. [45 has recently explored benchmarking offline multi multi-objective optimization (MOO) by building offline datasets for variety of MOO benchmarks and proposing several potential algorithms. Their work extends some offline single-objective optimization (SOO) approaches to the MOO setting, such as fitting surrogate models to the offline data and optimizing over the surrogates using evolutionary algorithms. However, this work has not explored the possibility of using generative models for offline MOO.
Our contributions are:
We formulate offline MOO as an inverse problem, and propose novel algorithm based on diffusion models with preference- classifier guidance. This classifier is trained to compare two candidate designs and determine which is more likely dominate the other in the objective space.
We address the diversity challenge in the offline MOO setting. Rather than just identifying high-value designs, we train the preference model favor solutions that are not only optimal but also well-distributed across the objective space.
We demonstrate through extensive experiments on both synthetic and real-world settings that our approach performs better than other inverse methods in terms of convergence to the Pareto front and the diversity of obtained solutions.
Among inverse techniques, our approach consistently achieves the strongest results; making it competitive choice in cases where forward methods not feasible. Even when forward

---

Unnoering divene deigon
Traidegg
Trikrecn GEGAE
Pakmnt Trefernne Clasdier
Furk
Traling
polloae buod deerened
Diffusion Model
A
Zulviia palionas
Y1
Figure I: A schematic representation of the proposed preference guided diffusion approach for offline multi- objective optimization.
methods applicable, algorithm remains competitive while maintaining the benefits of generative exploration.
Background
2.1 Offline Multi-Objective Optimization
Multi-objective optimization (MOO) seeks to find the design E X'd that minimizes (or maximizes) m different objectives:
min f(x) = #EX4
where fi 2' -> an unknown and expensive-to-evalue objective. For most practical problems, the objectives are not simultaneously optimizable by single design. Hence, the goal instead is to find the set of designs that are Pareto optimal.
Definition 2.1 (Pareto Dominance). A design Pareto dominates another design E dd (denoted by x < 2) if fa(x) < fz( Vi and 3j (z) < fz(2).
Definition 2.2 (Pareto Optimality and Pareto Front). A design x" is Pareto optimal if a E x such that æ < x=" The of all Pareto optimal designs is called a Pareto set. Correspondingly, the objective values of the Pareto set {f(x") x² is called the Pareto front (PF).
The Pareto front provides optimal set of trade-offs that can be achieved from the objectives when they are not simultaneously optimizable.
Sequential methods that collect data by selecting designs and evaluating their function values the most common approach for MOO, making use of surrogate models with uncertainty quantification to learn the unknown objectives. However, for many practical problems, these sequential methods not feasible due to prohibitive cost or time constraints (or both). Instead of iteratively allocating an evaluation budget to refine the design choices, offline optimization uses the entire budget in single round of function evaluations. In offline optimization, we have access to dataset of N non-optimal design-objective values pairs D == f { The goal of offline MOO to find the Pareto-optimal set by relying only the existing dataset D.
2.2 Diffusion Models
Diffusion models
35 are class of generative models defined by a Markov chain that sequentially adds noise to data samples and then leams to denoise from white noise (i.e., generate sample) by reversing the Markov chain. this work, we follow the Denoising Diffusion Probabilistic Models (DDPM) 117 approach, which we summarize here. Given a sample æn g(æ), time- dependent forward noising process defined as:
g( xt-1) = Ma, VI - 3, 8,1le)
(2)
where B, is the variance of the noising schedule at timestep L such that 31 B2 < .... Br, and T
the total number of timesteps. Let a₂ 1 3r and ā, Ii aus then the noised sample

---

be obtained in closed form:
2t Vā + Via G 4~,N(,
The reverse process conditional ((x;-1 2t.) not tractable. Therefore, a denoising model defined as po(#;=11 xta) == (Z Eg (xxe) is learned by optimizing parameters 0. Instead of parameterizing the reverse process to estimate (o(())) it is common to reparameterize this reverse process to predict the noise that was added to produce 2t (equation B). If cæ( t) is the denoising model to predict the added noise, reparameterization yields the mean (1x as:
1 Vlz
at (0
Hg (xt)
(4)
VI- 0c,
The reverse process variance is set to be the same as the forward process variance at time t, i.c., Ng (xxe = Bylla. The denoising model can be trained with mean squared error (MSE) loss:
((0)
E
[|<e - (2),,
(5)
A new sample can be generated by first sampling äा ~ NO 1a) and then autoregressively sampling from (1 (o
(o.)),, SeTa) get Ty.
2.3 Classifier Guidance
If label o corresponding to each sample ag available, then classifier guidance allows one to generate new samples from a trained diffusion model specific to desired label. To do this, classifier guidance [13] trains an additional time-dependent classifier of the input Po (o 2t, (). Along with
the trained denoising model eø (2t, t), new conditional sample can be generated by first sampling ØT ~ NO, L4) and then, from = T' = L, autoregressively sampling:
セt-1 o, 2t ~ ((x--1; ne (ă;) + w&t V log ã., B,Ia)
(6)
where is the guidance strength. In this work, we use classifier guidance to generate samples that approximate the optimal Pareto sets where the classifier is a preference model that predicts the dominance of one input over the other.
3 Related Work
Online Multi-Objecfin Black-Box Optimization: Adaptive experimental design has primarily explored multi-objective optimization (MOO) in an online or sequential fashion, where solutions are iteratively refined as new data arrives D 4. Although less studied than single-objective optimization, several sequential approaches have proven effective. The most established is Bayesian optimization (BO) 18 33), which typically uses Gaussian processes
to model objectives, especially in data- scarce regimes. Multi-objective extensions of BO often reduce the problem via scalarization [2] or employ acquisition functions such as expected hypervolume improvement (EHVI) [5] and information gain 4 to balance trade-offss across objectives. Recent work also introduced batch selection strategies [1, though most methods emphasize Pareto dominance over diversity. A few exceptions, such as Konakovic Lukovic et al. [22) and Ahmadianshalchi et al. [1] explicitly promote Pareto front diversity. Beyond traditional BO, neural approaches extend online MOO using generative models like variational autoencoders (VAEs) combined with Gaussian processes over the latent space [36]. However, these methods inherit VAE iimitatioss-psterte collapse and non-identifiability- that restrict their robustness in practical optimization tasks.
Offline Single-Objectivn Black-Box Optimization: Offline single-objective optimization focuses on leveraging existing datasets without additional data collection. Forward methods [1]] 46) train surrogate models to approximate the objective function and then optimize these surrogates for high-performing inputs. Inverse methods instead employ conditional GANs [6, [6] 25 to learn mappings from function outputs back to inputs. To bridge forward and inverse strategies, Chemingui et al. 77). Krishnamoorthy et al. (33 proposed hybrid methods that synthesize opttido-ttiiizziionn trajectories from offline data, mimicking online behavior. Diffusion- diffusion losses with weighted importance terms, while classifier guided models (8] bias generation toward globally optimal designs. Kim et al. [19] provides a comprehensive review of model-based offline optimization methods, summarizing progress across forward, inverse, and hybrid approaches. Additionally, Trabucco et al. [22 established a standardized benchmarking suite for consistent evaluation of offline optimization algorithms.
frameworks (4 adjust

---

Pande
baviniged or Iil
2.0
Pukning Dato
1.0
Davinated
0.0
Prefenncs
0.2 0.4 0.6
0.8
0.2
0.4
0.6
9.8
y2
V2
Figure 2: Generalization the preference model on regions unseen the training data on ZDT2 task The preference model gives good prediction of Pareto dominance between the reference design (in black) with other designs (in pink and green). Pink indicates that the preference model predicts these designs to be dominated by the reference design and green indicates that these designs predicted dominating the reference design. The figure the right is zoomed-in version the left, excluding the training data (in bluc).
Offline Multi-Objective Black-Box Optimization: Research on offline multi-objective optimiza- tion (MOO) remains limited. Xue et al. [45] recently proposed a benchmarking framework with offline datasets and baseline algorithms across MOO benchmarks. Their work mainly extends forward-styls single-objective methods by fitting surrogate models to offline data and optimizing them via evolutionary algorithms but does not explore generative inverse techniques. Concurrently, Yuan et al. [88) introduced ParetoFlow, a flow-based generative model for offline MOO that embeds objective weighting directly into the loss function, effectively scalarizing multiple objectives. While this inverse approach is promising, it does not explicitly address Pareto front diversity- an essential factor for capturing representative trade-offs in MOO.
4 Preference-Guided Diffusion for Offline Multi-Objective Optimization
We present a new effective approach for offline MOO by using classifier guidance to generate samples from Pareto optimal sets with a diffusion model trained offline data. Our approach does require training individual surrogate models for each objective. It relies on an inverse strategy while ensuring the ability to generate diverse samples from the Pareto optimal set. We refer our method as preference preferrene-guided diffusion for multi-objectiiv offline optimization (PGD-MOO).
Let E d C Rdd be any d-dimensional design with corresponding objective values y; fa( f() defined by unknown and expensive-to-evaluat functions fi X + R. Let y I be the vector of objective values for an m-objective problem. In offline MOO, we have access to dataset == (1) y (6) ] of N previously evaluated design-objective pairs. Given D, the goal is to generate designs x:" from the unknown optimal Pareto set.
While diffuusion models capture the distribution over data p(x), ). in offline MOO, we often interested in samples that lie outside the training data, closer to the Pareto front. This motivates the use of classifier guidance. Directly using classifier guidance in diffusion models involves training surrogate models for each objective, which often requires scalarization and hence can be suboptimal. We propose use preference-basee guidance capture Pareto dominance relations between data points.
4.1 Preference Guided Diffusion
In this work, we explore an alternate guidance strategy that does not involve training surrogate models for every objective. Instead, we model the space of multiple-objectives through preference pairs, as defined by Bradley-TTerry model [ of designs.
In this regard, train a preference model that predicts the (log)- probability of whether design Pareto dominates (Definition (2J] another design. During sampling of the designs, Given two designs and we train (immeconditioned) binary classifier that predicts log Ps (x < x, t). We parameterize this distribution with a multi-layer perceptron (MLP) that takes in two inputs (designs) of size 2 x d and outputs the logit of the Bernoulii distribution predicting whether the first input Pareto dominates the second input.
5

---

Algorithm 1 Sampling from Preference Guided Diffusion
Require: Trained eo (2 (), preference model pø(æ x Φ ar, dt, l), guidance weight w and the most
dominant design in the dataset x ((best)
1: < x= P(best)
2: ØT ~ ~N, Ma)
3: to do
4: (a(x;;) ya,
I
E 1-00-oo co(,,,))
vi-á.
5: Compute preference score sp= Va. Po - Ze, t)
6: Sample - (x H4 (xx) ) + w; BcTdd) Be
7: r < 2.
8: end for
9: return 20o
Training the Preference Classifier. To train the preference classifier, we first sort the points in the training data by their Pareto dominance. This divides the data into multiple fronts in increasing order of dominance, with points in the same front considered equally dominant. Next, we select (x: t) pairs randomly from the dataset. If one design strictly dominates the other, is labeled as preferred. If both æ and & are equally dominant, we assign the label æ < & if x has more diversity contribution than è wrt other points that belong to the front. The diversity contribution of each point is calculated using the crowding distance [2] of a selected design w.r.t all other points that belong to the same front in the dataset. Crowding distance for any point computed as follows:
dCD(
Σ 1=1
Di
y
mas
V
min
where v," and y; are the 7thh objective values of neighboring designs of x in the corresponding front sorted according to ith objective value. y/ and y!th are maximum and minimum values of the objective in the current front. Crowding distance has been used secondary selection criterion in evolutionary algorithms such as NSGA-2 [12] to maintain the diversity of solutions. Using crowding distance to create a binary label encourages the preference model to not only guide the diffusion model towards more Pareto-dominant regions, but also ensure that the designs that make up the Pareto front are diverse. For the denoising model, we train unconditional diffusion model eØ fo(2,,)) 2.2 on the designs x in the dataset, similar to DDPM
Sampling Designs. With trained denoising model ca(x,, t) and preference model Po < v, & t), we sample variable at the current timestep Σ; as well as from the previous realization of denoising, i.e., ēt+1 for the preference model to estimate Va, Pa,Pa (z, < Æt41 ã 2241, t). At the beginning of the denoising procedure when It+1 is not defined, we input the most dominant design in the dataset for preference comparison. The most dominant design in the dataset is chosen based non-dominated sorting. The sampling procedure is summarized in Algorithm|] T Intuitively, a preference model that generalizes well beyond its training data should guide the denoising process such that, each step of denoising, the resulting sample ã; is in a more Pareto-dominant region.
design by using classifier guidance ($..3) We input both the denoised
With enough timesteps, the resulting denoised sample do will be close to the Pareto front. Morcover, the connection to Bradley-Terry model of preference is apparent: the gradient of the logit which indicates the probability of the first input dominating the second, carries information about the reward 32]. Here, we use this to guide the diffusion model such that can sample from regions closer to the Pareto front through the denoising process.
This approach does not require training surrogate models, thus providing simple alternative approach to offline MOO. We find that the preference model generalizes well outside of the training data (see Fig. 2 therefore providing guidance to the diffusion model to generate designs outside of the training data close to the Pareto front, while maintaining diversity in the samples.

---

Experiments
We perform several experiments standard benchmarks for offiine MOO. Through these experiments, we would like to understand how close the generated samples are to the Pareto front, as well as the diversity of the solutions.
Our evaluation closely follows the benchmarking effort provided in prior work [45]. We evaluate our approach on two sets of tasks: synthetic and real-world applications- based RE engineering suite [40]. Each task consists of dataset of 60k offline datapoints. As in [45]
Benchmark Tasks:
we use 54k randomly chosen data points for training and the remaining for validation.
1. Synthetic set of tasks consist of 12 distinct tasks each with their own dataset. The objectives are analytic functions with tractable Pareto-fronts. This set of tasks has been widely used MOO problems. Every task consists of 2-3 objectives with d ranging from 10 to 30.
RE engineering suite of problems are set of 12 distinct tasks, each with their own dataset. These tasks are based on real-world applications in engineering, for instance, rocket injector design and disc brake design. ranges from 3-7 variables and the number of objectives m varies from to 6.
3. MO-NAS are set of tasks are based on neural architecture search benchmarks from 14 27 30
which consists of multiple objectives like prediction error and hardware constraints like GPU latency. In particular, we use the datasets corresponding to (10/MOP(199) and INIK/MOP(199 tasks provided by Lu al. [30]. As the design space is discrete, we operate on the continuous space of corresponding logits, as in prior work 45 48).
Details of individual tasks and their corresponding datasets given in Appendix B
Baselines: We compare ap- proach with two categories base- lines:
Table Average ranking of hypervolume obtained by differ- ent methods across synthetic, RE and MO-NAS tasks.
We compare with ParetoFlow 48 classifier guided generative model based on flow-matching (28] Classi-
Method
Synthetic
RE
MO-NAS
MuliHHad
5.73
10.11
MultiHead PeGrad
6.08
6.06
8.00
fiers for guidance trained surrogate models for each objective, followed by scalarization. ParetoFlow is our primary baseline since our work is mainly concerned with studying in- verse approaches for offline MOO.
MultiHead - GradNorm
9.75
11.4
8.50
MultipleModels
6.5
3.67
9.39
MultipleModdeh COM
7.83
7.27
2.72
MultipleModdels IOM
4.6
5.44
MultipleMMddess ICT
6.67
4.67
MultipleModels RoMA
6.58
7.67
7.44
MultipleModelss TriMentoring
8.33
4.3
8.11
ParctoFlow 2. Forward approaches using evolu- tionary algorithms: As suggested in prior work [45. a standard approach to offline MOO is to train a surrogate model for each objective and then use evolutionary algorithms such NSGA-2 [12 search over the design space. Although there are various ways to learn surrogate models, we compare with deep neural network (DNN)-based approaches, which are shown perform best according to benchmarks (45). The DNN approaches we compare with are: i) A Multi-Head Model: Uses multi-task learning (50) to train joint surrogate for all objectives. Training techniques for this approach such as GradNorm 10 and PeGrad [47] are also compared. ii) Multiple Models: Maintain m independent surrogate models, each making
7.38
6.17
4.67
PGD-MOO + Data Pruning (Ours)
5.08
9.06
2.14
PGD-MOO (Ours)
4.42
7.67
1.86
of single optimization technique, including COMs [1 ROMA 46), IOM 311, ICT 44), and Tri-mentorin (
Other forward approaches like multi nulti-objective Bayesian optimization algorithms are shown to perform worse than evolutionary algorithms on both synthetic and RE set of tasks [45), hence we exclude them from comparisons in this work.
Evaluation Metrics: For each algorithm, we evaluate the convergence of solutions using the hypervolume metric (52] standard metric in MOO for measuring the closeness of the proposed designs to the Pareto front. Hypervolume measures the volume of the objective space between
7

---

t-1000
To
0.0
0.4
0.6
0.8
0.0
0.6
0.8
0.0
0.4
0.6
0.0
0.0
0.4
0.6
1.0
Figure 3: Plot of samples from our preference-gun guidedd diffusion model (in green) on the ZDT2 task [53 different timesteps of denoising. Convergence of samples close to the Pareto front (in red) outside the training data (blue) highlights the importance of preference guidance.
Table 2: Hypervolume results of DTLZ subtasks (part of the synthetic task). Each method is run for five random seeds and evaluated on 256 designs.
Method
DITIZ1
DTLZ2
DILZ3
DITLZ4
DTLZ5
DTLZ6
DTL.27
D-ett
10.00
9.91
10.00
10.76
9.35
8,55
8,56
MalnHHaa
10:51 + 0.25
9.03
0.56
10.48
0.23
6.73
8.41 0.15
8.72 = 1.07
10,66 0.09
MaltiHead- PeGind
10.64
10.55
9.95
9.02
0.34
9.90
MalniHead- - GraNNorm
10.64 + 001
8.86 +:66+100 1.27
10.26
0.28
7.45 + 0.75
7.87 + 1.06
8.16 1: 2.21
10.31 + 0.22
MultipleMeokls
10.64 :: 0.01
9.03 0.80
10.58
0,03
7.66 2:
7.65 4: 1.39
9.58 = 0.31
10.6 : 0.16
MaltipleMedels- COM
10.64
10.27
0..3
MallipleModdss 1OM
10.61 1 0.01
10.10 1: 0.27
0.13
10.03 1: 0.53
9.77 0.188
9.30 0.31
10.60 1 0.05
MullipleMockls- ICT
10.64 1: 0.01
8.68
0.88
10.25
2. 0.42
10:33 0.24
9.25 0.228
9.10 1.16
10.29 : 0.05
MaltipleMookls RoMA
10.64 0.01
10.04 1: 0.05
10.61 2 0.03
9.25
0.11
8.71
0.44
9.84 0.25
10.53 0.04
MulliplcMooks- TriMenttring
10.64
1.03
ParcteHow
10:60 0.02
16.13:
10,41 0.09
10:29
0.17
9.63
+ 0.23
9.25 = 0,43
894 0.188
FGD-MOO + Data Pruning (Ours)
10.61 0.01
10.55 1: 0,01
10,63
0,01
10.63 1 0.01
10.07
1 0.02
10.15
0,03
9:57 0.07
PGD-MOO (Ours)
10.65
reference point and the objective vectors of the solution set, and does not require access the true Pareto front. The reference point used for evaluation of hypervolume is taken from [Appeendix B).
In addition to the hypervolume, we also measure the diversity of the obtained solutions using the A-spread metric [22 [1) The A-spread measures the extent of the spread achieved in a computed Pareto front approximation (2). It is important to consider the diversity of the obtained solutions, especially in the case of MOO wherein there is no single "best" design, but rather an entire set of solutions based on the Pareto front. In addition, in the case of offline optimization, the acquisition is single-shot. Therefore, solutions that are diverse and hence provide more coverage over the objective space are preferable. In this work, provide the first effort to evaluate and benchmark the diversity of solutions obtained by different approaches in offline MOO. Further details of the evaluation metrics and their computation is provided in Appendix A.I
We evaluate all methods on 5 random seeds, and compute the metrics using budget of 256 designs.
5.1 Training Details
We parameterize the unconditional denoising model to be multi-layer perceptron (MLP) with two 11-dimmesioonal hidden layers, followed by a ReLU nonlinearity and layer normalization [6). We also incorporate sinusoidal time embedding 43 for conditioning. We parameterize the preference model to be an MLP with three hidden layers, with first two hidden layers having the same number of units the input, while the last hidden layer is having 512 units. Similar to denoising model, we also use ReLU nonlinearity followed by layer normalization and sinusoidal time embedding.
The denoising model is trained with AdamW optimizer 22 with learning rate of 5e 4 for up to 200 epochs. Following Ho et al. [17,, we employ a linear noise schedule such that the noise 3, grows linearly from 1e 4 to 0.02. The preference model is trained with Adam optimizer 2 with learning rate of le for up to 500 epochs. During sampling, we set the guidance weight w to 10. For the preference model, we also experiment with pruning the training data only contain the top 30% of points, sorted according to their dominance, We refer to this method as PGD-MOO Data Pruning in the results.
8

---

Table 3: Hypervolume results of ZDT subtasks (part of the synthetic task). Each method is run for five random seeds and evaluated on 256 designs.
Method
ZDTI
ZDT2
ZDT3
ZDT4
ZDT6
D (best)
4.17
5.15
5.45
4.61
MultiHead
4.8 0.03
5.57
0.07
5.58
0.2
4.59
1 0.26
0.01
:: 0.01
MultiHead - PcGrad
4.84 0.01
5.51
3.68
0.70
4.67 0.1
MultiHead - GradNorm
4.63 = 0.15
5.37
5.2
3.28
1.2
MultipleModels
=
5.57
5.03
4.78 - 0.01
MullipleModels- COM
4.52 = 0.02
5.49 4.12
5.10
4.41
MultiplcModels- IOM
4.68 = 0.12
5.61 5.11
4.21
4.75 0.01
MultipleModels- ICT
4.82 = 0.01
5,58 =
5.59
4.63
4.75 + 0.01
MultipleModels RoMA
4,84 = 0.01
5,43 + 0.35
5,89 + 0.04
0.04
4.13 0.11
1.71 0.10
MulripleModels TriMentoring
4.64 - 0.10
5.22 + 0.11
5.16 + 0.04
0.04
5.12 + 0.12
2.61 +
PoretoFlow
4,23 0.04
5,65
0.11
5.29
- 0.14
5.00 1 0.22
4.48 + 0.11
PGD-MOO + Data Pruning (Ours)
4.54 0.08
5.21
0.06
5.61
= 0.06
5.06 + 0.07
4.56 0.14
PGD-MOO (Ours)
4.41 0.08
5.33
0.05
5.54
- 0.10
5.02 0.03
4.82 + 0.01
Table 4: Selected results of hypervolume on RE task. Results are evaluated on 256 designs with five
different random seeds.
Medlod
RE21
RE22
RE
RE.3
RC3
RE37
RE41
RE61
Dibestt
4.78
4.79
10.36
10.08
5.57
18.27
97.49
Malillend Krr
4.7
4.32
17.06
MultiHead
0.36
4.78 0.14
10.51
0.0
4.74
10.6 0.05
10:49
0.07
6.67
20.62
0.11
10R
MulkippleMsshh- COM
4.38 1.0.9
4.84 1.00
4.83 0.01
10.64
1. 0.01
10.55
0.02
6.35 1.1
20.37
0.06
107.99
0.48
MustipisMMsst-ICTT
4.6 4.0
4.84 +.0.0
4.84 0.0
1064
+ 0.0
10.5 + 0.01
6.75=000
20.58
0.04
108.68
0.27
Molliip-MMakk-IOMM
4.58 : 0.92
4.84 +:00
4.83 0001
10,5 :: 0.0
10.57 0.01
6.1 0:000
20,66
0.05
107
0.5
MalkiplsModdl- RoMA
4.57 0.0
4,61 0:111
4,83 0.01
10.64 0.0 0.0
10.53 1 0.03
6.67 1: 0.02
20.39
0.09
108.4
0.28
4.6 16-+-0.
4.84 +:00
4.84 - 0
10.22
= 0.01
10.59 + 0.0
6.73 + 0.81
20.68
0.04
108
0.29
MokiipleMooell
4..10.
4.84 1.00
4.63.1..33 0.25
10,62 L 0.02
10.55 0.00
6.73 0:00
20.77
0.08
108.96 0.06
PGD-M00
Prnning (Ours)
4.84
10:4
539
19.37
0.13
PGD-MOO
4.46 1.000
4.84 0.0
4.84
10,65 0.0
10.32 0.0.
6.13 0.012
19.31 0.46
105.02 1.14
5.2 Results
Evaluation of Convergence. We provide detailed results of hypervolume for various baselines and our approach on the synthetic task (Tables and 3). We find that our approach performs competitively with respect to baselines. Treference-guide guided diffusion performs another generative model-based approach using guidance. This shows the benefits of having a preference model classifier for guidance. Vizualization of the final sampled designs from approach is provided in Appendix [..2. Overall, our method performs better than other baselines, which learn surrogate models and evolutionary algorithms in the synthetic task setting (Fig. 3). In addition, we also find that our method performs competitively in the RE engineering suite (Tables and 22) and the MO-NAS tasks (Table []. In problems with higher number of objectives, find that our approach is slightly worse compared to the baselines in terms of hypervolume. However, we note that our approach is much simpler train in these settings, while still achieving diverse solutions (discussed further below).
average better than ParetoFlow,
Evaluation of Diversity. Average ranking in terms of performance of the A-spread metric for all algorithms (Table 55) shows that our approach gives more diverse solutions than all the other baselines including in the RE setting. These results highlight the importance of having diversity constraint in the training procedure for the classifier through the data selection procedure (Appendix F.3)
Across experiments, we find that our approach has competitive performance in terms of hypervolume (convergence) while being better terms of the A-spread metric (diversity) than the baselines.
5.3 Ablation Studies
We also study the impact of choice of two important aspects: the guidance-weight hyperaprameter w, and the role of diversity criteria in training the preference classifier.
Results for various choices of w for ZDT subtask are provided in Fig. 4a With w 0 (no preference guidance), the results are lower, as expected. Increasing the guidance weight generally results in better hypervolume at the slight cost of diversity. This indicates that if diversity is more important in the resulting designs, extreme values of are less preferable.

---

2011
2013
201
20
2313
207
2019
200
Exooike
Figure 4: Ablation study of (a) different guidance weights w on both hypervolume and A A-spread metrics (b) various diversity criteria (Crowding, SubCrowding, No Diversity and Hypervolume) on the A A-spread metric. All evaluation done with ZDT subtask 256 sampled designs across random seeds.
We also perform ablation studies on the role of diversity criteria that is used to pick preference pairs for training the preference classifier. For effective comparison, also compare with hypervolume improve- ment, in addition Crowding, SubCrowd- ing and having no diversity criteria. In the hypervolume improvement diversity met- ric (indicated as just Hypervolume), we measure diversity of a point based on the MultipleMoodels-ICT
Table 5: Average ranking of the A-spread metric ob- tained by different algorithms on both synthetic and tasks. Detailed results are provided in Appendix E
Method
Synthetic
RE
HaltiHead-GradNeee
7.12
8.27
MultiHead-
5.92
7.0
MultiHead
8.83
7.2
MultipleModeessCCMM
6.5
5.47
6.5
5.87
change in hypervolume it brings when it is MultipleModees-IOM
6.42
6.8
added the sampled set of designs. Re- MuliplleModeel-RRMAAA
5.92
6.4
sults for A A-spread metric for the ZDT sub- MulinleeModdl--Tiieennnnng MultipleModels
6.0
4.6
task is given Fig. [4b] We find that in- corporating diversity criteria for preference generally results in better diversity the resulting solutions. In addition, Crowd- ing and SubCrowding gives better diversity than using hypervolume improvement. Detailed results for tasks are provided Appendix E.I
9.25
7.53
ParetoFlow
10.0
9.0
PGD-MOO Pruning (Ours)
2.67
4.0
PGD-MOO (Ours)
2.83
4.28
6 Conclusion
In this work, we presented a novel classifier-guided diffusion approach for offline multi-objective optimization (MOO). Our method leverages preference model that predicts Pareto dominance between pairs of inputs, incorporating diversity considerations to ensure that designs on the same Pareto front are well-distributed. Empirical results show that technique performs competitively in terms of convergence to the true Pareto front, while also generating diverse set of solutions.
Limitations. A key limitation of our current approach is that it relies soicly dominance information rather than the individual function values the objectives. Consequently, it does not allow fine- grained control over trade-off's among different objectives, which can be important if a practitioner needs to emphasize or de-emphasize specific objective values.
Future Directions. One promising extension would be to integrate additional guidance signals, such as the actual function values, enabling more preference- form MOO. This would allow users to explicitly prioritize certain objectives over others specify desired performance ranges. Another avenue for future work combining forward (surrogate-bast based) and inverse (generative) approaches, where candidates proposed the generative model are iteratively refined using surrogate models.
10

---

Acknowledgements
This work was supported by a CIFAR Catalyst grant, the Helmholtz Foundation Model Initiative and the Helmholtz Association. The authors also acknowledge the Gauss Centre for Supercomputing e.V. (www.gauss-cente censure.eeeeeeeeu eu) for funding this project by providing computing time on the GCS Supercomputer JUQUEEN [37] at Jülich Supercomputing Centre (JSC). BEE was funded in part by grants from the Parker Institute for Cancer Immunology (PICT), the Chan-Zuckerbery Institute (CZI), NIH NHGRI R0I NG0119677 and NIH NHGRI R01 HG013733 BEE is a CIFAR Fellow in the Multiscale Human Program. Syrine Belakaria is supported by the Stanford Data Science fellowship.
References
[1] Alaleh Ahmadianshalchi, Syrine Belakaria, and Janardhan Rao Doppa. Pareto front-diverse batch multi-objective bayesian optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10784-10794, 2024.
[2] Charles Audet, Jean Bigeon, Dominique Cartier, Sébastien Le Digabel, and Ludovic Salomon. Performance indicators in multiobjective optimization. European journal of operational re- search, 292(():397 422, 2021.
[3] Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wil- son, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian optimization. Advances neural information processing systems, 33:21524 21538, 2020.
[4] Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Output space entropy search framework for multi-objective bayesian optimization. Journal of artificial intelligence research, 72:667-715, 2021.
[5] Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: the method of paired comparisons. Biometrika, 39(3/4):34-345, 1952.
[6] David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference on machine learning, pages 773-782. PMLR, 2019.
[7] Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, and Janardhan Rao Doppa. Offline model-based optimization via policy-guided gradient search. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 1 11230-112 11239, 2024.
[8] Can Sam Chen, Christopher Beckham, Zixuan Liu, Xue Liu, and Christopher Pal. Robust guided diffusion for offline black-box optimization. arXiv preprint arXiv:2410.00988. 2024.
[9] Can Sam Chen, Christopher Beckham, Zixuan Liu, Xue Steve Liu, and Chris Pal. Parallel- mentoring for offline model-based optimization. Advances in Neural Information Processing Systems, 36, 2024.
[10] Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. Gradnorm: Gra- dient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, pages 794-803. PMLR, 2018.
[11] Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. Differentiable expected hypervolume improvement for parallel multi-objective bayesian optimization. Advances in Neural Information Processing Systems, 33:9851 1-864, 2020.
[12] Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, (2):182-197, 197, 2002.
[13] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances neural information processing systems, 34:8780-8794 2021.
[14] Xuanyi Dong and Yi Yang. Nas-bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326, 2020.

---

[15] Michael Emmerich and Jan-willem Klinkenberg. The computation of the expected improve- ment in dominated hypervolume pareto front approximations. Rapport technique, Leiden University, 34:7-3, 2008
[16] Clara Fannjiang and Jennifer Listgarten. Autofocused oracles for model- based design. Advances in Neural Information Processing Systems, 33:12945--1299 2020.
[17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 60010 2020.
[18] Donald R Jones, Matthias Schonlau, and William Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13:455-4922 1998.
[19] Minsu Kim, Jiayao Gu, Ye Yuan, Taeyoung Yun, Zixuan Liu, Yoshua Bengio, and Can Chen. Offline model- based optimization: Comprehensive review. arXiv preprint wrXV::2503.12886 2025.
[20] Diederik Kingma. Adam:
method for stochastic optimization. arXiv preprint
arXiv: 2014.
[21] Joshua Knowles. Parego: A hybrid algorithm with on-line landscape approximation for expen- sive multiobjective optimization problems. IEEE transactions on evolutionary computation, 10 (1):50-66, 2006.
[22] Mina Konakovic Lukovic, Yunsheng Tian, and Wojeiech Matusik. Diversity-guider multi- objective bayesian optimization with batch evaluations. Advances in Neural Information Processing Systems, 33:17708-17720 2020.
[23] Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Generative pretraining for black-box optimization. arXiv preprini urXiv:2206. 10786, 2022.
[24] Siddarth
Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black-box optimization. In International Conference on Machine Learning, pages 17852-17557.
PMLR, 2023.
[25] Aviral Kumar and Sergey Levine. Model inversion networks for model-based optimization. Advances in neural information processing systems, 33:5126-5137, 2020.
[26] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. ArXiv e-prints, pages A607 2016.
[27] Chaojian Li, Zhongzhi Yu, Yonggan Fu, Yongan Zhang, Yang Zhao, Haoran You, Qixuan Yu, Yue Wang, and Yingyan Lin. Hw-nas-bench: Hardware-aware neural architecture search benchmark. arXiv preprint arXiv:2 10584, 2021.
[28] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:22 2022.
[29]
Loshchilov. Decoupled weight decay regularization. arXiv preprint 17Xiv:1111.051000 2017.
[30] Zhichao Lu, Ran Cheng, Yaochu Jin, Kay Chen Tan, and Kalyanmoy Deb. Neural architecture search as multiobjective optimization benchmarks: Problem formulation and performance assessment. IEEE transactions evolutionary computation, (422)322--337,202
[31] Han Qi, Yi Su, Aviral Kumar, and Sergey Levine. Data-driven offline decision-making via invariant representation learning. Advances in Neural Information Processing Systems, 35: 13226-13237, 2022.
[32] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, (0653728-537444 2023.
[33] Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEFE, 104 (1):148-175, 2015.
12

---

[34] Jascha Sohl- Dill-Disstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-22655. PMLR, 2015.
[35] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-hased generative modeling through stochastic differential equations. arXiv preprint arXiv:201 134466, 2020.
[36] Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating bayesian optimization for biological se- quence design with denoising autoencoders. In International Conference on Machine Learning. pages 20459-20478, PMLR, 2022.
[37] Michael Stephan and Jutta Docter. Juqueen: Ibm blue gene/q@ supercomputer system at the jüllich supercomputing centre. Journal of large-scale research facilities JLSRF, I:Al-A1, 2015.
[38] Jonathan M Stokes, Kevin Yang, Kyle Swanson, Wengong Jin, Andres Cubillos-Ruiz. Nina M Donghia, Craig R MacNair, Shawn French, Lindsey Carfrae, Zohar Bloom- Ackermann, et al. A deep learning approach to antibiotic discovery. Cell, 180(4):688-702. 2020.
[39] Kyle Swanson, Gary Liu, Denise B Catacutan, Autumn Arnold, James Zou, and Jonathan M Stokes. Generative ai for designing and validating easily synthesizable and structurally novel antibiotics. Nature Machine Intelligence, 6(3):338-353, 2024.
[40] Ryoji Tanabe and Hisao Ishibuchi. easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing, 89:106078, 2020.
[41] Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. International Conference on Machine Learning, pages 10358-10368. PMLR, 2021.
[42] Brandon Trabucco, Xinyang Geng, Aviral Kumar, and Sergey Levine. Design-bench: Bench- marks for data-driven offline model-based optimization. In International Conference on Machine Learning, pages 21658-21677 PMLR, 2022.
[43] A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
[44] Christopher KI Williams and Carl Edward Rasmussen. Gaussian processss for machine learning, volume 2. MIT press Cambridge, MA, 2006.
[45] Ke Xue, Rong-Xi Tan, Xiaobin Huang, and Chao Qian. Offline multi-objective optimization. arXiv preprint arXiv:2406.03:: 037222 2024.
[46] Sihyun Yu, Sungsoo Ahn, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for offline model-based optimization. Advances Neural Information Processing Systems, 34: 4619-4631, 2021.
[47] Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33:5824-5836, 2020.
[48] Ye Yuan, Can Chen, Christopher Pal, and Xue Liu. Paretoflow: Guided flows in multi-objective optimization. arXiv preprint arXiv:2412.03718. 2024.
[49] Ye Yuan, Can Sam Chen, Zixuan Liu, Willie Neiswanger, and Xue Steve Liu. Importance-aware co-teaching for offline model- optimization. Advances in Neural Information Processing Systems, 36, 2024.
[50] Yu Zhang and Qiang Yang. survey on multi-task learning. IEFE transactions on knowledge and data engineering, 34(124 34(12):5586-5699 2021.
13

---

[51] Aimin Zhou, Yaochu Jin, Qingfu Zhang, Bernhard Sendhoff, and Edward Tsang. Combining model- boseel-bsedd and genetics- based offspring generation for multi-objective optimization using convergence criterion. In 2006 IEEE international conference evolutionary computation, pages 892-899. IEFE, 2006.
[52] Eckart Zitzler and Lothar Thiele. Multiobjective optmization using evolutionary algorithms comparative case study. In International conference on parallel prohlem solving from nature, pages 292-301. Springer, 1998.
[53] Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele. Comparison multiobjective evolutionary algorithms: Empirical results. Evolutionary computation, ((2):173-195 2000.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The paper's 's contributions are using a preference based guidance for diffusion models an offline MOO setting. This laid out the abstract.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are laid
$G
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [No]
Justification: The paper does not provide any formal theoretical results.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The required details for reproducibility provided in $5
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification; The reproducible code is included as part of the submission in the supplemen- tary material.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training and testing details are laid out in $
7. Experiment statistical significance
14

---

Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All experiments conducted with multiple seeds and confidence intervals are provided.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details regarding computational resources needed for experiments is provided in Appendix C
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: pattpa://neuripee hcc/public/Bthiccciieeeeee
Answer: [Yes]
Justification: The authors have reviewed the NeurlPS code of ethics and the research conducted in the paper conforms this code.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impact statement is provided in Appendix
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: The paper does not deal with an algorithm or model that have potential for high risk of misuse.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: The licenses for the code and which this paper relies on is from 45
whose
code has been released on Github with an unknown license.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
15

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does involve LLMs.
16

---

Appendix: Preference-Guided Diffusion for Multi-Objective Offline Optimization
A Experimental Setting
A.1 Metries
We provide computational details regarding the metrics used in the work hypervolume and A A-spread.
Hypervolume is common metric which is used in Multi-Objectinn Optimiza- tion to measure the quality of the solutions. It does not require access to the Pareto Pareto-Front. It instead relies on reference point, which is any point in the imput space that is worse than all points in the solution set across every objective. Given this reference point aher hypervolume is computed the area enclosed by hyperrectangle from every point in the solution set S to Feo
Hypervolume:
/ U Eret, [x22, Tretz X X [a Erela dA #ES
HV(S) =
(8)
where is the Lebesgue measure and [., corresponds to hyperrectangle between the two points. While relatively easier for objective problems, computing hypervolume more involved for solutions with higher number of objectives.
A-Spread: A-Spread 12, 51 measures the uniformity of solutions which belong to the same front. In this work, we take into account the extreme points of the predicted front to calculate the A-spread of a solution set S as follows:
E miny Ily vll + - (y 3/(1²)) de
((S)
(9)
E miny |ly yl1 + (15) - 1)de
where miny Ily yll is the distance between the extreme points of the set and the corresponding extreme points in the Pareto front. If the Pareto front is not known, this quantity is evaluated be zero. d' ( (/(y²)) corresponds the distance between consecutive points of set S, sorted according to objective values of one of the objectives, and d' is the mean of df across all elements of set S.
B Dataset Details
We provide further details the datasets used our experiments for both Synthetic and RE set of tasks in Tables 6 and 7 All the datasets directly taken from 45 well as the evaluation of hypervolume computation and the needed reference points.
Table 6: Dataset information and reference point for bypervolumeecomputatioo Synthetic set of tasks.
Name
m
Pareto Front Shape
Reference Point
DTLZ1
Linear
(558.21, 552.30, 568.36)
DTLZ2
10
Concave
(2.77, 2.78. 2.93)
DTLZ3
10
Concave
(1703.72, 1605.54, 1670.48)
DTLZZ
10
Concave
(3.03, 2.83, 2.78)
DTL.75
10
Concave (2d)
(2.65, 2.61, 2.70)
DTL.76
10
Concave (2d)
(9.80, 9.78, 9.78)
DTLZ7
10
3
Disconnected
(1.10, 1.10, 33.43)
ZDTI
30
Convex
(1.10, 8,58)
ZDT2
30
2
Concave
(1.10, 9.59)
ZDT3
30
Disconnected
(1.10, 8.74)
ZDT4
10
2
Convex
(1.10, 300.42)
ZDT6
Concave
(1.07, 10.27)
17

---

Table 7: Dataset information and reference point for hypervolume computation for RE set of tasks.
Name
m
Pareto Front Shape
Reference Point
RE21
(Four bar truss design)
Convex
(3144.44, 0.05)
RE22 (Reinforced concrete beam design)
Mixed
(829.08, 2407217.2 25))
RE23 (Pressure vessel design)
Mixed, Disconnected
(713710.88, 1288669. 78)
RE24 (Hatch cover design)
Convex
(5997.83. 43,67)
RE25 (Coil compression spring design)
Mixed, Disconnected
(124.79, 10038735, 500)
RE31 (Two har truss design)
Unknown
(808.85, 6893375.88 82, 6793450.00)
RE32 (Welded beam design)
Unknown
(290.66, 16552.46, 388265024,00 00)
RE33 (Disc brake design)
Unknown
(8.01, 8.84, 2343.30)
RE34 (Vchicle wrashworthiness design)
Unknown
(1702.52. 11.6, 0.26)
RE35 (Speed reducer design)
Unknown
(7050.79, 1696.67, 397,83)
RE36 (Gear train design)
Concave, Disconnected
(10.21, 60.00, 0.97)
RE37 (Rocket injector design)
Unknown
(0.99, 0.96, 0,99)
RE41 (Car side impact design)
Unknown
(42.65, 4,43, 13.08, 13.45)
RE42 (Conceptual marine design)
Unknown
(-26.39, 19904,90, 28546.79 79, 14.98)
RE61
(Water resource planning)
Unknown
(83060. 03, 1350.00, 2853469.06
16027067 .0, 357719, 99660.36)
Table 8: Dataset information and reference point for MO-NAS set of tasks.
Name
Search space
d
Reference Point
C-10/MOP1
NAS-Bench-
26
2
(3.49 x 10 1,314 x 107)
C-10/MOP2
NAS-Bench-IDe
26
3
(9.05 x 10 1001 ,3.05 x 10",8.97 10%)
C-10/MP3
NATS
5
3
(2.31 x 10 101, 10-
102)
C-10/MOP4
NATS
4
(2.31 x 10 7.14
x..4 102,
10-2)
C-10/MPP5
JAS-Bench-201
(9.03
10 1.53 x 109 2.20 10
10
C-10/MOP6
NAS- Bench-201
(9.03
10!, 1.53 10 2.20 10 1.05 101,
10" 10
10
C-10/MP77
NAS-Bench-
6
(9.03 x 10', x 10
109, 10P,
4.8 10', 1.05
10
2.23
10",
10
C-10/MOP8
DARTS
32
2
(2.61 x 10 1,,
10%)
C-10/MOP9
DARTS
32
3
(4.85 10 1,-9,3.9
109)
IN-IK/MOP1
ResNet50
25
(2.81 10
107)
IN-IK/MOP2
ResNet50
25
2
(2.80 10 1.15 10
IN-IK/MOP3
ResNet50
25
3
(2.81 10
3,887 x 107 1.26 10403
IN-K//OPP4
Transformer
34
2
(1.83 x 10', 7.25 107)
IN- N-IKSMOPS
Transformer
34
3
(1.83 x 10 1.49 10
IN- IN-IK/MOP6
Transformer
34
3
(1.83 10
7.10 x 10", 1.48 10
IN-IK/MOP7
MNV3
21
2
(2.64 10
109)
IN-IK/MOP8
MNV3
21
3
(2.65 x 10
1.00 x 10" 1.34 109)
IN-IK/MOP9
MNV3
21
4
(2.65 x 10
1.03
10 1.31 x 10" 6.30
10
C Computational Resources
All the experiments are run on an NVIDIA A100 GPU. Our proposed approach takes on average of 1300 seconds for 2 objective task of 30 dimensions. Approaches like MultipeeMdeels walltime is directly proportional to the number of objectives. Consequently, Paretoflow (48). which relies on MultipleModels also scales poorly wrt number of objectives. Overall, our approach takes roughly 56 GPU hours to train and test on all the datasets for seeds. A representative runtime of our approach as well as the baselines is given in Table
D Broader Impact Statement
This paper addresses the problem of offline optimization with multiple objectives. Advances in the problem being addressed are impactful for discovering new drugs for curing diseases, discovering new materials with certain physical properties, to name few. The authors do not forsee any negative societal impacts of this work beyond what might be enabled due to general advancements in machine learning.
18

---

Table Runtime (in seconds, for training) of different approaches on two representative tasks. Both tasks have 60k samples in total. The runtime of PGD mainly depends on the dimensionality of the problem (for RE it is 3 and ZDT1 is 30) while for approaches like MultiHead and ParetoFlow it is proportional to the number of objectíves (for RE 61 it is 6 and for ZDTI it is 2), since they require training a surrogate model for each objective. We expect these runtimes to be the same if the dimensionality (or correspondingly the number of objectives) for other tasks are the same. Sampling times for all these approaches are fairly fast (at most 30 sec for 256 samples).
Method
ZDTI
RE61
MultiHead
450
470
MultipleModels
920
2820
Pareto
1200
3100
PGD-MOO (Ours)
1300
930
E
Additional Results
E.1
Ablation Study Full Results
In addition to the main results, we perform evaluation of the impact of guidance weight w on the
resulting designs. Tables 10 12 and [4 presents results of hypervolume evaluated on 256 sampled designs. Tables I1 13
and presents the corresponding diversity evaluation with A-spread metric.
In addition to the above evaluation, we also evaluate the impact of the incporated diversity criteria for selecting the preference pairs for training the classifer. In addition to Crowding and SubCrowding, we also consider not incorporating any diversity (mentioned as No diversity), as well as computing the improvement in hypervolume due specific design in the sampled designs (mentioend just Hypervolume). Hypervolume improvement due to a specific sample can be computed as the difference between hypervolume due the entire set and the hypervolume due the entire set except for this specific sample.
Results for different evaluation criteria are provided in Tables 16 18 and 20 for hypervolume, as well as in Tables 17 19 and 21 for diversity.
Table 10: Hypervolume results with 256 sampled designs of different guidance weights w for DTLZ subtasks (part of synthetic task).
Guidance Weight (w)
DTLZI
DTLZ2
DTLZ3
DTLZZ
DTLZ5
DTLZ6
DTLZ7
w
10.64+0.00
10.53++0.22
10.61+00.2
10.66+0.01
10.07+00.02
10.15++0.05
9.48-+0.14
10.65=+0.00
10.54+0.02
10.63-10.00
10.64:0.01
10.08-10.01
10.17440.2
9.64-10.14
10
10.65=#0.00
10.56:0.00
10.63++0.00
10.65++.01
10.05=0001
10.17:+0.04
9.76=10.14
w 20
10.64:0.00
10.5510.01
10.62:1.001
10.64:0.01
10.09:10022
10.16:10.04
9.81-10.3
10.62:0.01
10.55:0.01
10.63:10.00
10.63-1.01
10.09:10.02
10.16:10.09
9.80-1.36
Table 11: Diversity evaluation results with 256 sampled designs of different guidance weights for DTLZ subtasks (part synthetic task).
Guidance Weight (w)
DTLZI
DTLZ2
DTLZ3
DTLZ4
DTLZ5
DTTL26
DTLZ7
w
0
0.54=0.02
0.54+0.02
0.48:10.01
1.66+0.03
0.50=00.22
0.64+0.01
0.52+00.22
w
0.5911000
0.54.10.3
0.48:10.03
1.62:10.44
0.52.10.04
0.62-10.02
0.65
w
0.61::0.4
0.5640.02
0.47:0002
1.62-10.03
0.54:0.02
0.6240002
0.70:10.05
0.6111005
0.54
0.4710001
1.53.10.02
0.50.10.03
0.60.10.03
0.73.10.07
50
0.6010.01
0.56110.2
0.49100.3
1.49.10.05
0.5310.03
0.6010.02
0.66.10.06
19

---

Table 12: Hypervolume results with 256 sampled designs of different guidance weights for ZDT subtasks (part of synthetic task).
Guidance Weight (w)
ZDTI
ZDT2
ZDT3
ZDT4
ZDT6
w 0
4.53
5.20-10.03
5.57=0.09
5.08:10.04
4.60=0.12
w 5
0.53+0004
5.31
5.55=0.03
5.06:10.07
4.81=0002
w 10
0.55
5.39:+0.05
5.55=0.06
4.97-10.07
4.82-+0.00
w 20
0.53
5.52:0..2
5.60=0.10
5.0010.07
4.82=1.01
w 50
4.51
5.62:10.04
5,55=0.04
5,04:1.04
4.63=10.3
Table 13: Diversity evaluation results with 256 sampled designs of different guidance weights w in ZDT subtasks (part of synthetic task).
Guidance Weight (w)
ZDTI
ZDT2
ZDT3
ZDT4
ZDT6
w 0
0.64=0.04
0.62+00.01
0.62=0.04
0.61+0.02
0.77=0.05
W 5
0.63=0.03
0.66
0.62=0.02
0.61
0.81
10
0.63+00.09
0.58++0.01
0.60=0.02
0.60+-0.02
0.83+00.4
w 20
0.63+0.02
0.58++0.02
0.60=0.02
0.58++0.02
0.90++0.06
W 50
0.60+0000
0.60
0.62=0.03
0.62-+0.01
0.91+0.04
Table 14: Hypervolume results with 256 sampled designs of different guidance weights w for various
RE tasks.
Guidance Weight (w)
RE21
RE24
RE25
RE35
RE41
4.38
4.84:10.00
4.84:10.00
10.42:10.06
18.98:10.25
w
4.43:10.05
4.84.10.00
4.84
10.37-10.06
19.00:10.26
10
4.46::0.04
4.83=1.00
4.84
10.27=0.11
19.17=0.33
w 20
4.45::0.05
4.84=0.00
4.84+0.00
10.16:10.15
18.40:10,84
50
4.44
4.83+1.00
4.84
9.55
17.67=00.9
Table 15: Diversity evaluation results with 256 sampled designs of different guidance weights w for various RE tasks.
Guidance Weight (w)
RE21
RE24
RE25
RE35
RE41
0
0.60=0.03
1.15+0.07
1.05=0.05
0.70+10.03
0.42=0002
w = 5
0.61=0.04
1.20
1.08=0.05
0.81+0.12
0.45=0.02
10
0.61+0.02
1.20+-0.05
1.19=0.08
1.00-10.12
0.47=+0.04
w 20
0.61+0003
1.18++0.03
1.21-0.06
0.83+H0.11
0.49+00.04
50
0.63+0.02
1.17+0008
1.21-0.06
0.82-H0.09
0.62++0.19
Table 16: Hypervolume results with 256 sampled designs when specific diversity criterion is used for
training the preference classifier. Results evaluated with w 10 DTL.Z subtasks (part synthetic tasks).
Diversity Criteria
DTLZI
DTLZ2
DTLZ3
DTLZ4
DTLZS
DTLZ6
DTLZ7
Crowding
10.65=0.00
10.56:20.00
10.63=0..00
10.65=0.01
10.05:0.00
10.17=0.04
9.76+t..14 14
Hypervolume
10.64=0.00
10.55:10.01
10.63+0..00
10.64=0.00
10.06+0.022
10.13+0.01
9.72++0.09
SubCrowding
10.64+0.00
10.56++0.01
10.62+0.01
10.65+0.02
10.08+00001
10.10+0.07
9.49++0.05
No diversity
10.64.00.0
10.56:10.00
10.63:10.00
10.64:10.01
10.06-10.00
10.13:0.00
9.66:10.00
E.2 Visualization the Sampled Designs
We provide visualization of the sampled designs from model on 2-objective problems in Fig.
20

---

Table 17: Diversity evaluation results with 256 sampled designs when specific diversity criterion is used for training the preference classifier. Results evaluated with to 10 on DTLZ subtasks (part of synthetic tasks).
Diversity Criteria
DTLZ1
DTLZ2
DTLZ3
DTLZ4
DTLZ5
DTLZ6
DTLZ7
Crowding
0..14:004
0.56:0.02
0.47=0.02
1.62=0.03
0.5410.02
0.62:0.02
0.70
Hypervolume
0.58:10.00
0.54:10.3
0.4910.01
1.63=0004
0.53:0002
0.63+0.02
0.76:10.07
SubCrowding
0.55++0.03
0.54=0.02
0.48=0003
1.68=0.04
0.51+0.02
0.65+0.02
0.58++0.2
No diversity
0.58:10.00
0.55
0.48:0.00
1.56-0.03
0.5340.00
0.63+0000
0.68:10.00
Table 18: Hypervolume results with 256 sampled designs when specific diversity criterion is used for
training the preference classifier. Results evaluated with w = 10 on ZDT subtasks (part of synthetic tasks).
Diversity Criteria
ZDTI
ZDT2
ZDT3
ZDT4
ZDT6
Crowding
4.55
5.39:00.55
5.55-10.06
4.97:00.07
4.82:10.00
Hypervolume
4.49:10.12
5.40+10.05
5.59-10.09
5.01
4.82++0.01
SubCrowding
4.54
5.24
5.60-10.03
5.08+10.08
4.59
No diversity
4.59:10.08
5.46:0.03
5.58+0.00
5.21=0.00
4.74
Table 19: Diversity evaluation results with 256 sampled designs when specific diversity criterion is used for training the preference classifier. Results evaluated with on ZDT subtasks (part of synthetic tasks).
Diversity Criteria
ZDTI
ZDT2
ZDT3
ZDT4
ZDT6
Crowding
0.63
0.58+0.01
0.60+:0.02
0.60+1.02
0.83+0.04
Hypervolume
0.64
0.6140.03
0.60
0.60+10.02
0.86
SubCrowding No diversity
0.70+L0.02
0.90+F0.04
0.66-10.03
0.60+10.04
0.77++0.06
0.72++0.04
0.60++0.03
0.66++0.03
0.65+-0.00
0.84+0001
Table 20: Hypervolume results with 256 sampled designs when specific diversity criterion is used for training the preference classifier. Results evaluated with w = 10 on RE tasks.
Diversity Criteria
RE21
RE24
RE25
RE35
RE41
Crowding 4.46+10.04
4.83+0.00
4.84+10.00
10.27+0.11
19.1740.30
Hypervolume
4.41=:0.3
4.83+0.00
4.84+0.00
10.36::000
19.09=0.00
SubCrowding
4.39:0.05
4.83+0.00
4.84+0.00
10.41=0.05
19.02=0.18
No diversity
4.42+10.00
4.83++000
4.84++0.00
10.35+0000
19.09+0000
Table 21: Diversity evaluation results with 256 sampled designs when specific diversity criterion is used for training the preference classifier. Results evaluated with w 10 on RE tasks.
Diversity Criteria
RE21
RE24
RE25
RE35
RE41
Crowding
0.6020.02
1.20:10.05
1.19-10.08
1,00-10.12
0.47:10.04
Hypervolume
0.62+40.2
1.16+0.09
1.14=10.09
0.99+0.08
0.47+00.3
SubCrowding
0.60+0001
1.1740.05
1.1240.06
0.70+0.03
0.43+0.02
No diversity
0.62+0.01
1.16+0.06
1.39-10.06
0.90+0.02
0.50+0.04
21

---

ZOT
20T
Z0T
Z0T
125
25
0.0
0.4
0.6
0.0
0.6
0.0
0.4
9.6
0.5
65
0.6
0.9
24
3.0
4000
2.5
2.0
2000
2000
0.8
$1.5
1.5
0.4
1.0
1.0
1000
0.2
0.5
0.0
0.0
0.0
100 250
1000 2000
2000
4800 5000
6000
Figure 5: Plot of the samples from diffusion model (in green) on different 2-objective tasks. Top row shows results for synthetic set of benchmarks, while the bottom shows results for RE engineering suite. Blue dots correspond training data and red dot corresponds the Pareto front. The blue dots omitted for the bottom row for clarity. Results ZDT2 available Fig.
Table 22: Evaluation of hypervolume with 256 sampled designs on subsets of the RE task. Results averaged over 5 different random seeds.
4.78
4/6
4.79
10.5
10.56
10.56
4.8
0.39
4.7
4 0:44
1.12
1.63 0.8.
4.5% 1.52.00.
10.6 0.1
10:4 0.15
10.05
MuliiHead--oCind
4.59 0.01
4.73 + 0.36
4.84 + 0.0
4.15 0.66
4.78
0.14
10.64
0.01
10.6 + 0.01
1:.9-=0000
MuliiHld
4.6.1..00
4,84 1:00
4.541.0.0 0.01
4.73 0.22
4.74 1. 0.2
10.65 1.00
10.6 0.05
10.62 0:00
MullipieModelsCCOM
0.09
2.84
4.84
0:2
10.64
0.01
MullipieMoiels-ee
MulippieMxiei-IIMM
MullipleModellRool RoMA
4.57 46- 0.0
4.61
4 0.01
3.9 1:22
4,83
10.64
10.64
10.58
InliploMsisl-TTiMemeet
4.84 000
4.84 0.0
4.84 +00
4.84
10.65 +.00
10,2 + 0.01
10.6 + 0.01
MullipleModels
4.6
1,84 1.00
4.84 1 0.0
4,83 1 0.01
4.63 0.25
10.65 1.00
10,62 1 0,02
10.82 1122.100
PaneoFlow
4.1017
4.85 1.001
10.66 0.12
10.61 0.0
10.75 02
FGD-MO0 Tata Pruning
E.3
Detailed Results
Table 27: Evaluation of the A-spread metric with 256 sampled designs on ZDT subtask, part of the
synthetic set of tasks. Results are averaged over 5 different random seeds. Lower values are better. Method
ZDTI
ZDT2
ZDI3
ZDT4
ZDT6
DaltHHaad-GradNoon
0.96 0.19
0.94
0.93 + 0.17
0.79 0.15
0.77 + 0.16
0.83 0.1
0.98 0.13
0.79 1 0.03
0.67
0.04
0,82 : 0.11
MultHHead
1.13
1.04
0.83 1 0.06
0.68
1.00
1.22 - 0.07
MullppleModles-COM
0.89
0.11
0.83 0.05
0.64
0.98 - 1.09
MultipleMociets-ICT
1.1 0.02
1.1
0.86
0.09
MultiplcMosdes-IIOM
0.1
0.0
0.81 +.07
0.74
0.46 - 0.1
MuliipleMedeel-RoMAA
0.64 + 0.06
0.9
0.79 0.07
0.69
0.78 + 0.06
MalippeeModels-TnMMennne
0.86 +: 0.03
0.86
0.9
0.73
0.78 0,06
MuldipleModels
1.07 + 0.06
1.01 0.03
0.84 0.03
0.0
1.19 0.04
ParetoFlow
1.46 0.03
1.19 0.1
1.46 0.14
1.1+
0.71 + 0.05
PGD-MOO DataPruning (Ours)
0.66 0.08
0.61
0.03
0.6 0.03
0.6 0.04
0.8 0.05
PGD-MOO (Ours)
0.68 0.07
0.78 0.09
0.65 + 0.03
0.6 0.03
0.76 + 0.03
The detailed results given in Tables 22
23 and
26
29
22

---

Table 23: Evaluation of hypervolume with 256 sampled designs on subsets of the RE task. Results are averaged over 5 different random seeds.
Method
RE34
R1:35
R1336
RI:37
KF41
RE42
RK61
18best)
9.3
10:08
7.61
5.57
18.27
11.52
97,19
Malnitaad-GraaNeer
8.47 + L.77
9.76 = 1.3
9.67 = 0.43
5.67 1 1.4
17.06
2: 3.82
18.77 = 2.99
[08.01 = 1.0
MulnHHead-PoCind
+
+ 0.05
10.17
6.68
0.06
20.66
22.57
0.226
108.
00.28
MulniHead
10.1
10.49 0.07
10.23
6.67
0.05
20.62
22.38
0.35
108.92
MultipleMooks-COM
9.96 0.09
10.55
+: 0.02
9.22 = 0.35
6.35 0.1
20.37
+ 0.06
17.44-0.71
197.999
+: 0.48
MallipleMedeS-ICT
10.1+000
10.5 + 0.01
10.29 0,03
6.73 +.0
20.58 + 0.04
22.27
0011
108.8 + 0.22
MoltpplMMokksIOOM
10.11 0:01
1: 0.01
10.29
MollipteMooel-RoMMA
9.91 0.00
10.53 -: 0.03
9.72 = 0.28
6.67 1: 0.02
20.39
0.00
21:41
0.37
108.7 +: 0.28
MaltipleMockk-TriMeeee
10.08 2: 0.02
10.59 1: 0.0
9.61 = 1.42
6.73 1: 0.01
20.68 4 0.01
21.6
0.19
108.61 1 0.29
MallipleMokkss
10.11 0.0
10.35 4: 0.01
10.24 0.03
6.73 4: 0.03
20.77
0.08
22.59
0.11
108.96 1 0.06
ParetoHlo
11.2 1.32-+0.35
11.12 0.02
842 + 0.35
6.55 0.59
19.41
0.92
20.35
5,31
107.1 6.96
PGD-MOO + Data Pruning (Ours)
9.15
10:43
00:3:00000
9.88 = 0.33
5.99 0.11
19.37
- 0.15
17.4
3 0.63
103.04 1 1.71
PGD-MOO (Ours)
9.39 0.16
10.32 0.1
9.37 = 0.17
6.13::0.11
19.31
0.46
19.01 = 0.68
105,02 + 1.14
Table 24: Evaluation of hypervolume with 256 sampled designs on subsets of the C10MOP tasks. Results are averaged over 5 different random seeds.
CUDHUPS
$1.03
11000 1.279
7116
24.
BCCH
213
41%0
L2768 3.1.3
L4013 118
11/00:1 2.29
1131
43.000 1-:409
18:100mm 8:844
48:38--00 6600
1:2. 1181
5m0 8.01
11818:
77/023 11.1.00
2 2--121--30 -9 5/h)
3 2211
5.305
13377 31831
101111
200-3190001 177000
8-32000 1011500
11:310 11177
Table 25: Evaluation of hypervolume with 256 sampled designs on the IN1K tasks. Results
averaged over 5 different random seeds.
10030004 11000 101313
12515 21-07
11 12SEESS B2T
1283 1963
CU8 0780
14.79:1
100%
4386144 523
11836 11007
L.90. 4.1803
+11330 3.506
128033
5480 1.8 1112
5,8911000 01615
16.811 11.177
6989 1.0
1J8011880
13105 1.3
Mel Heal all
0-300- 0180
11:001110
2118)-00811
6187
411-2-6794
41000 +-3000
11488-41:00
1.855
LOU370
8-3355
TIREE 117331
53017 T000
2711
Table 26: Evaluation of the A-spread metric with 256 sampled designs on DTLZ subtask, part of the synthetic set of tasks. Results are averaged over 5 different random seeds. Lower values are better.
Method
DTTZI
DTLZ2
DTTLZ3
DTLZZ
DTTZ5
DTLZ6
DTLZ7
MultiHedd-GradNoon
0.61 1 0.03
0.8910.23
0.8
0.29
U.66
0.14
0.75
0.1
0.28
1.200.18
MultiHead-PoGna
0.65 : 0.04
0.73 + 0.05
0.76
2 0.11
1.08 = 0.2
0.77
0.06
0.87 +: 0.06
1.15 + 0.2
MuliHHea
0,86 + 0.08
0.93
+ 0.15
0.93
0.16
0.88
0.
0.92 0.14
0.88 0.2
0.93 + 0.11
MulippeeMsdelsCOM
0.69 1 0.02
0.85 0.15
0.73
0.66
1.091...1 0.12
1.09
0.89
0.08
1.15 0.15
0.78
1 0.03
MultppleModel-1CT
0.7 0.7-0.02 0.02
0.79
+ 0.02
0.63 +.11
0.92
+ 0.03
0.8
088+0.0 0.05
0.91 +0066
0.77 + 0.05
Mulipplakaaae MuliiplaMadellRookee RoMA
MulsipleMadees-100
0.96
0.67 0.04
1.23
0.91
0.03
0.62 -: 0.03
1.06 1 0.08
0.89
0.11
1.28 0.08
0.93 0.13
0.76=0.1
0.69 1 0.03
MulápleModeec, TriMentrring
0.72 1: 0.03
0.91 0.1
0.66 0.09
0.95 0.07
0.7 1: 0.06
1 0.09
0.81 1 0.05
MultipleModels
0.88 1 0.07
1.11 1 0.26
1.0
:: 0.22
0.93 1:133
0.92 0.17
1.0 0.24
0.85 1 0.12
ParetoFlow
0.82 = 0.02
1:07
1 0.07
0.68
0.09
1.63 0.13
1.04
= 0.06
0.34 0.16
0.0
PGD-MOO + Data Pruning (Ours)
0.58 2: 0.04
0.52 1 0.03
0.46
1 0.02
1.66 1 0.04
0.51 0.02
0.63 1: 0.02
0.53 1 0.04
PGD-MOO (Ours)
0.62 = 0.02
0.5 1.5:200 0.03
0.46 = 0.02
1.6
0,51 0.02
0.61 = 0.03
0.63 # 0.02
Table 28: Evaluation of the A-sprea metric with 256 sampled designs on subsets of the RE task. Results are averaged over 5 different random seeds. Lower values are better.
Method
RE31
RK22
RE33
RE24
RE25
RE31
RE32
REB
MultHead GradNorm
0.77 : 0.15
161 0.37
1.7
0.98 0.4
1.54 0.5
0.91 0.17
126 022
0.92
0.14
Multileead PeGraa
0.47
1.8
1.14 0.221
0.4
161028 0.28
0.13
MultHHed
042
1.43 0.2
0.223
1.3 1.19
1.84
0.4 0.144
0.84 0.14
1.16 0.05
MultppeeModels-COM COM
0.56 - 0.12
1.22 0.42
0.49
108 0:2
1.63 0.1
1.24
0.17
1.322.199
0.87
4 0.07
MultipleMocddsH ICT
0.38 1 0.02
1.78 0.12
0.84 0.07
0.67 10112
1.4
1.88
0.19
0.91 0.09
1 0.21
MultspleMookhs-l IOM
0.98 0.4
181 008
1.26 0.23
1.24
1.5
1.07 0.22
10.9
0.99 0.06
MolkipleMMokks-Bee
0.09
1.57
1.33 0:440
1:24
1.15
MultspleMoodli TriMentrrng
0.37 0.01
1.73
0.91
0.04
0.14
0.99 0.09
MulkipleMMdels
0.37 0.00
1.85 - 0.12
0.82 0.46
0.99 1.0.2
1.36
1.65 1 0.22
09 09910.2 0.26
1,16 1,16
Parerol-ow
0.12
1.37 2111
0.03
134 - 0.0
L.7
PGD-MOO- Datu Pruning (Ours)
0.61 0.03
1.29 0.08
1.42
0021..11
113 0.01
L12 0.08
1.34 0.26
162 0.1
0.83 1 0.17
PGD-MOO (Ours)
0.003
0.07
1.06 0.06
1.14 0.02
0.07
1.59
23

---

Table 29: Evaluation of the A --spread metric with 256 sampled designs on subsets of the RE task. Results are averaged over 5 different random seeds. Lower values are better.
Method
RE3
RF3
RF3
RE37
RE41
RF42
RF6T
Muliilead GramNorm
0.96
0.23
1.17
0.14
1.19
0.19
1.19
0.31
1:13
1.12
0.72 0.03
MuliHHead-PoGrai
1.11 1: 0.88
0.99 1: 0.13
0.11
0.17
0.06::0..0
0.62 0.01
0.9 1 0.08
0.72 1: 0.03
MultHHead
1.18 0.02
1.03
0.06
0.16
0.73
0.44
::
0.74 0.06
MullpplaMesees-COM
1.16 0.00
0.02
0.89 1 0.03
0.94
0.13
0.73
0.56 0.00
0.68 0.07
0.66 1: 0.03
MultipleMedals-ICT
MultiplaModcls110
1.09 0.05
1.03 + 0.06
1.15
0.07
0.57
0.73 0.06
0.62 + 0.08
MuliipleModel- RoMA
1,02 - 0.03
1.22 : 0.07
0.93 0.13
0.00
0.59
0.66 0:05
0.59 1 0.05
MulippleModal-TriMMee TriMcctoin
1,05
0.82 -
122
0.4
0.08
0.78 0.07
0.05
MultipleModels
1.22 + 0.03
1.07 + 0.12
1.03 + 0.07
0.2+
0.22-0.44 0.04
0,83 0.08
0.7 + 0.06
PGD + Dnta Pruning (Ours)
0.58 + 0,05
0.67 + 0.04
0.7 = 0.05
0
0.42 6.0
0.49 0.03
0.52 + 0.83
PGD-MOO (Ours)
24


Reviews:
[{"review_id": "yhsELb7OJE", "reviewer": "Reviewer_79Tz", "summary": "This paper proposes a diffusion-based generative method for offline MOO, which aims to identify the Pareto-optimal solutions only given the offline dataset $\\mathcal{D}= \\\\{ ( \\mathbf{x}_i, \\mathbf{y}_i ) \\\\} _{i=1}^{N}$. The proposed method first trains a pairwise preference classifier based on NSGA-II selection [1] (integrating both dominance and solution diversity) to determine which design is better, then trains an unconditioned diffusion model. After that, it uses the classifier to guide the denoising procedure. Experimental results demonstrate the effectiveness of the proposed method.", "questions": "How do you set up the noise schedules $\\beta_t$ in your experiments?", "limitations": "Limited experimental studies:\n\n- Effectiveness of the classification-based preference guider is not examined. For example, I suggest compare it with weighted sum regression-based predictor [4] or ranking-based preference guider [5] (use ranking loss to train the model).\n- I cannot understand the benefits of the inverse approaches in offline MOO based on the experiments. Upon the same preference classifier, I suggest comparing your diffusion-based method to searching inside the classifier by replacing the selection operator of NSGA-II with the model’s output.\n- Ablation of the guidance weight $w$ is needed, as in [6].\n\nI understand that conducting additional experimental evaluations in all tasks of Off-MOO-Bench [2] is time-consuming during the rebuttal period. I suggest that you compare those in some representative tasks. I will be glad to increase the score once you address my concerns.\n\n## References\n\n[1] A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE TEvC, 2002.\n\n[2] Offline multi-objective optimization. ICML 2024.\n\n[3] Robust Guided Diffusion for Offline Black-Box Optimization. TMLR 2024.\n\n[4] ParetoFlow: Guided Flows in Multi-Objective Optimization. ICLR 2025.\n\n[5] Offline model-based optimization by learning to rank. ICLR 2025.\n\n[6] Diffusion models for black-box optimization. ICML 2023.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- This proposed method is clear and well motivated.\n- The code implementation is clean and well constructed.\n\n## Weaknesses\n\n- In line 253, you mentioned that “This approach does not require training surrogate models, thus providing a simple alternative approach to offline MOO”. As in [2], the end-to-end surrogate model is prediction-based MLP regressor. Don’t you need to train an MLP-based preference classification-based surrogate model too? Based on my comprehension, this discussion is not appropriate.\n- Based on my comprehension, your approach is somehow similar to RGD [3] in offline single-objective optimization. Relevant discussion on both this work and the difference between your framework and RGD is needed.", "was_revised": true, "final_justification": "This paper proposes a method based on diffusion model and classifier-based preference guider for offline multi-objective optimization. The method incorporates training unconditional diffusion model $p(x)$ and sampling with classifier guidance, where the classifier is based on NSGA-II selection. It is different from recent offline MOO work like ParetoFlow. This method is direct and intuitive, and the experimental results are competitive. \n\nOther reviewers mainly have concerns on: (1) missing related discussion of diffusion models; (2) limited experiments on Off-MOO-Bench [1]. For (1), the authors have added necessary discussion. For (2), there are so many tasks in [1] where the evaluations of some tasks are expensive, and adding all tasks during rebuttal is difficult. The authors have promised to add more tasks in the revised version. Besides, for optimization tasks that have constraints raised by other reviewers, studies in offline black-box optimization rarely consider this situation, and the authors have discussed potential solutions. \n\nOverall, I believe this paper would be of high quality after revision. Thus I've raised my score.\n\n[1] Offline multi-objective optimization. ICML 2024.", "rebuttal": "Dear Reviewer, \n\nThanks a lot for your thoughtful and detailed feedback. We are glad that you found our work clear and well motivated with a clean constructed code. \nWe answer the concerns and questions you have raised below:\n\n- “_Don’t you need to train an MLP-based preference classification-based surrogate model too?_”: In line 253, we refer to surrogate models as they are commonly used in the literature, namely, models that predict the output $y$ for any input $\\mathbf{x}$. In multi-objective optimization (MOO) settings, forward approaches typically train a separate surrogate model for each objective, and these are then used to search over a pre-defined design space. This is in contrast to our method, where the classifier is used to guide the diffusion-based generation process, rather than search within a fixed design space. This distinction is important: in inverse approaches like ours and Pareto Flow, no explicit search is required. Instead, designs are generated directly by the diffusion model, guided by learned preferences or classifiers, without needing to enumerate or evaluate points in a pre-set space.\n- “_your approach is somehow similar to RGD [3] in offline single-objective optimization_”: Thanks for bringing this paper to our attention. As you mention, RGD focuses mainly on offline single-objective setting, whereas the goal of this work is to explicitly address the harder problem of multi-objective setting, where the notions of Pareto Front and Pareto dominance arise. In addition, while RGD uses guided diffusion like the ParetoFlow approach, the guidance mechanism is different from ours, wherein we incorporate Pareto dominance relations for the guidance classifier. We will include RGD in related work and discuss these differences. \n- “_How do you set up the noise schedules $\\beta_t$ in your experiments?_”: Following the DDPM paper (Ho et al. 2020), we set the noise schedule $\\beta_t$ such that it grows linearly from 1e-4 to 0.02 with time $t$. We will add this detail in the experiment section.\n- “_Effectiveness of the classification-based preference guider is not examined. For example, I suggest compare it with weighted sum regression-based predictor [4] or ranking-based preference guider [5] (use ranking loss to train the model)_”: Thanks for the suggestion. [4] is the Pareto Flow approach, which is an inverse model similar to ours, and we compare to it extensively in our work. We find that our approach performs competitively across many datasets as compared to ParetoFlow. [5] mainly deals with single objective settings, and thus a direct comparison is not feasible since the actual outputs (a single best design vs the pareto set) and the corresponding metrics (hypervolume vs $y$ spread) would be very different. \n\n- “_cannot understand the benefits of the inverse approaches in offline MOO based on the experiments. Upon the same preference classifier, I suggest comparing your diffusion-based method to searching inside the classifier by replacing the selection operator of NSGA-II with the model’s output_”: Thank you for your suggestion. As explained in our introduction, there are several important settings—such as drug discovery—where generating entirely new designs is essential, and relying on a pre-defined search space is not always feasible. In such cases, inverse approaches become crucial and are often the only viable strategy. Forward approaches that depend on search procedures and the output of a surrogate model are not practical in these contexts. We do, however, compare against several search-based methods in our experiments for benchmarks where this is a feasible approach, which aligns with your recommendation. \n- “_Ablation of the guidance weight $w$ is needed_\": Thanks for the suggestion. We perform ablation experiments on the guidance weight $w$. Due to the space limitation, we are including only a subset of representative results below, but we will include the full set of results in the appendix in the camera-ready. We find that the guidance weight has an effect on both the hypervolume and diversity. With $w=0$ (no preference guidance), the results are lower, as expected. Increasing the guidance weight generally results in better hypervolume at the slight cost of diversity. We provide detailed results below (all results are evaluated on 256 solutions with 5 random seeds):\n\n\n### ZDT TASKS - Hypervolume Metric\n\n| Guidance | ZDT1 | ZDT2 | ZDT3 | ZDT4 | ZDT6 |\n| :--- | :---: | :---: | :---: | :---: | :---: |\n| $w=0$ | 4.53±0.04 | 5.20±0.03 | 5.57±0.09 | 5.08±0.04 | 4.60±0.12 |\n| $w=5$ | 4.53±0.04 | 5.31±0.03 | 5.55±0.03 | 5.06±0.07 | 4.81±0.02 |\n| $w=10$ | 4.55±0.04 | 5.39±0.05 | 5.55±0.06 | 4.97±0.07 | 4.82±0.00 |\n| $w=20$ | 4.53±0.13 | 5.52±0.02 | 5.60±0.10 | 5.00±0.07 | 4.82±0.01 |\n| $w=50$ | 4.51±0.18 | 5.62±0.04 | 5.55±0.04 | 5.04±0.04 | 4.63±0.33 | \n\n### ZDT TASKS - Spread Metric (Lower is better)\n\n| Guidance | ZDT1 | ZDT2 | ZDT3 | ZDT4 | ZDT6 |\n| :--- | :---: | :---: | :---: | :---: | :---: |\n| $w=0$ | 0.64±0.04 | 0.62±0.01 | 0.62±0.04 | 0.61±0.02 | 0.77±0.05 |\n| $w=5$ | 0.63±0.03 | 0.66±0.03 | 0.62±0.02 | 0.61±0.01 | 0.81±0.03 |\n| $w=10$ | 0.63±0.09 | 0.58±0.01 | 0.60±0.02 | 0.60±0.02 | 0.83±0.04 |\n| $w=20$ | 0.63±0.02 | 0.58±0.02 | 0.60±0.02 | 0.58±0.02 | 0.90±0.06 |\n| $w=50$ | 0.60±0.03 | 0.60±0.02 | 0.62±0.03 | 0.62±0.01 | 0.91±0.04 |\n\n### RE TASKS - Hypervolume\n\n| Guidance Weight | RE21 | RE24 | RE25 | RE35 | RE41 |\n| :--- | :---: | :---: | :---: | :---: | :---: |\n| $w=0$ | 4.38±0.07 | 4.84±0.00 | 4.84±0.00 | 10.42±0.06 | 18.98±0.25 |\n| $w=5$ | 4.43±0.05 | 4.84±0.00 | 4.84±0.00 | 10.37±0.06 | 19.00±0.26 |\n| $w=10$ | 4.46±0.04 | 4.83±0.00 | 4.84±0.00 | 10.27±0.11 | 19.17±0.30 |\n| $w=20$ | 4.45±0.05 | 4.84±0.00 | 4.84±0.00 | 10.16±0.15 | 18.40±0.84 |\n| $w=50$ | 4.44±0.05 | 4.83±0.00 | 4.84±0.00 | 9.55±0.54 | 17.67±0.96 | \n\n### RE TASKS - Spread (Lower is better)\n\n| Guidance Weight | RE21 | RE24 | RE25 | RE35 | RE41 |\n| :--- | :---: | :---: | :---: | :---: | :---: |\n| $w=0$ | 0.60±0.03 | 1.15±0.07 | 1.05±0.05 | 0.70±0.03 | 0.42±0.02 |\n| $w=5$ | 0.61±0.04 | 1.20±0.09 | 1.08±0.05 | 0.81±0.12 | 0.45±0.02 |\n| $w=10$ | 0.61±0.02 | 1.20±0.05 | 1.19±0.08 | 1.00±0.12 | 0.47±0.04 |\n| $w=20$ | 0.61±0.03 | 1.18±0.03 | 1.21±0.06 | 0.83±0.11 | 0.49±0.04 |\n| $w=50$ | 0.63±0.02 | 1.17±0.08 | 1.21±0.06 | 0.82±0.09 | 0.62±0.19 |\n\n \n- “_Additional experiments on representative tasks_”: Thanks for the suggestion. We extensively evaluate 17 different datasets across three different sets of tasks DTLZ, ZDT and RE. However, we are happy to include additional experiments on representative tasks like Multi-Objective  NAS and Multi-Objective RL. Given the limited space for the rebuttal, we will make these results available in the revision. Note that the provided code, which will be made available on acceptance, has the option to run it on all datasets and tasks that are part of the benchmark Xue et al 2024 paper.\n\nWe hope our response satisfactorily addresses all your concerns. If there are still any open questions or concerns, we are happy to answer them.\n\nReferences:\n\nHo, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems (NeurIPS)."}, {"review_id": "gBbeALOL0X", "reviewer": "Reviewer_1hyQ", "summary": "This paper presents Preference-Guided Diffusion for Multi-Objective Offline Optimization (PGD-MOO), which uses a preference-trained classifier to guide diffusion sampling in offline multi-objective optimization. A preference classifier is trained on pairwise comparisons in which the preferred design Pareto dominates its counterpart; when two designs are mutually equally dominating, the tie is resolved by favoring the one with a larger crowding distance, so the model learns to value diversity along the Pareto front. The authors demonstrate that PGD-MOO outperforms other baselines in DTLZ and ZDT tasks.", "questions": "1. Could you elaborate on why conditional guidance was chosen as the core mechanism for multi-objective offline optimization? In line 205, the paper mentions the goal of generating samples near the Pareto front, including those outside the training data distribution. However, Yuan et al [8] has suggested that conditional guidance can be difficult to tune effectively, and its generalization to out-of-distribution (OOD) regions tends to be limited. In contrast, approaches that fine-tune the diffusion model using proxy models such as DRaFT [9] and DDPO [10] have been shown to generate samples from desirable regions with strong performance. Finetuning methods are more robust in OOD area because finetuning moves the model distribution towards high-rewarding space. Moreover, recent approaches like GFlowNets [11] have been used to mitigate mode collapse in reinforcement learning, offering alternative sampling strategies with improved diversity. While [8][9][10][11] is not for MOO, in light of these alternatives, I am curious about the rationale behind choosing conditional guidance over diffusion finetuning methods.\n\n2. A follow-up question related to the clarity of the methodology: could you confirm whether the preference model was trained using the Bradley-Terry model? This is suggested implicitly, but an explicit clarification would be very helpful.\n\n[8] Yuan, Hui, et al. \"Reward-directed conditional diffusion: Provable distribution estimation and reward improvement.\" Advances in Neural Information Processing Systems 36 (2023): 60599-60635.\n\n[9] Clark, Kevin, et al. \"Directly fine-tuning diffusion models on differentiable rewards.\" arXiv preprint arXiv:2309.17400 (2023).\n\n[10] Black, Kevin, et al. \"Training diffusion models with reinforcement learning.\" arXiv preprint arXiv:2305.13301 (2023).\n\n[11] Liu, Zhen, et al. \"Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets.\" arXiv preprint arXiv:2412.07775 (2024).", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\nThe paper is clearly structured and easy to follow. One of its most compelling aspects is the idea of overweighting more diverse samples over concentrated ones, thereby encouraging the classifier to favor diversity. To the best of my knowledge, this is a novel approach, and it appears to improve the ∆-spread metric effectively. In the synthetic function benchmarks, PGD-MOO shows that a preference-based classifier outperforms the surrogate model approach, as well as multi-head and multi-model baselines.\n\n### Weakness\n- **Narrow experimental coverage.** PGD-MOO is tested on two synthetic tasks (DTLZ, ZDT) and the RE engineering tasks, whereas Xue et al. [1] include a broader set that includes MO-Swimmer, MO-Hopper, MO-Portfolio, and Molecule tasks, characterized by continuously represented data. It would be better to add more continuous representation tasks to the experiment section for more solid demonstrations. \n\n- **PGD-MOO underperforms ParetoFlow** On the RE suite, PGD-MOO underperforms ParetoFlow [2] despite using the same guidance mechanism—the main difference being the classifier. This result suggests that a preference-based classifier may not outperform a surrogate regressor in practice.\n\n- **Limited methodological novelty.** Classifier-guided diffusion sampling was first introduced by Dhariwal et al. [3] and later extended to classifier-free guidance by Ho et al. [4]; these two variants are fundamentally the same in that both condition the reverse process at inference time. For offline optimization, Krishnamoorthy et al. [5] already explored classifier-free guidance for offline model-based optimization. Even if one considers conditioned generation to be a universal technique, the use of preference classifiers is already a well-established idea in RLHF for language models [6]. As such, the novelty of this work remains incremental.\n\n- **Restricted to continuous design spaces.** Because PGD-MOO inherits the continuous DDPM backbone [7], it cannot natively handle discrete variables, whereas ParetoFlow [2] works for both discrete and continuous problems. Adopting a discrete-friendly diffusion variant (or a data-type–agnostic flow) would make the approach more generally useful.\n\n- **Clarity.** The explanation of the training method for the preference model in Section 4.1 could benefit from greater clarity. It seems to be based on the Bradley-Terry model, but this is not made explicit in the text. Providing a more detailed and transparent description would help readers better understand the underlying methodology.\n\n- **Limited ablation about diversity contribution.** The paper provides limited ablation on the *diversity contribution* to the selection of winning samples. To better understand the effectiveness of isolating diversity as a design choice, it would be valuable for the authors to include an ablation study specifically examining its impact on performance.\n\n[1] Xue, Ke, et al. \"Offline multi-objective optimization.\" arXiv preprint arXiv:2406.03722 (2024).\n\n[2] Yuan, Ye, et al. \"ParetoFlow: Guided Flows in Multi-Objective Optimization.\" arXiv preprint arXiv:2412.03718 (2024).\n\n[3] Dhariwal, Prafulla, and Alexander Nichol. \"Diffusion models beat gans on image synthesis.\" Advances in neural information processing systems 34 (2021): 8780-8794. \n\n[4] Ho, Jonathan, and Tim Salimans. \"Classifier-free diffusion guidance.\" arXiv preprint arXiv:2207.12598 (2022).\n\n[5] Krishnamoorthy, Siddarth, Satvik Mehul Mashkaria, and Aditya Grover. \"Diffusion models for black-box optimization.\" International Conference on Machine Learning. PMLR, 2023.\n\n[6] Ouyang, Long, et al. \"Training language models to follow instructions with human feedback.\" Advances in neural information processing systems 35 (2022): 27730-27744.\n\n[7] Ho, Jonathan, Ajay Jain, and Pieter Abbeel. \"Denoising diffusion probabilistic models.\" Advances in neural information processing systems 33 (2020): 6840-6851.", "was_revised": true, "final_justification": "All of my concerns have been resolved by the authors through their rebuttal. I thank them for their efforts in providing thorough experimental results.", "rebuttal": "Dear Reviewer,\n\nThanks a lot for your detailed feedback. We are glad that you find our work structured and easy to follow as well as our approach novel in terms of incorporating diversity for offline MOO.\nBelow we address the questions and concerns you have raised:\n\n- “_Narrow experimental coverage_”: We would like to emphasize that DTLZ, ZDT and RE contain multiple datasets within each task, with various objectives and dimensions. The total number of datasets is 17. In this regard, we believe testing on these tasks already provides a good representation of various approaches, including our method. However, we are happy to include additional experiments on representative tasks like Multi-Objective  NAS and Multi-Objective RL. Given the limited space for the rebuttal, we will make these results available in the revision. Note that the provided code, which will be made available on acceptance, has the option to run it on all datasets and tasks that are part of the benchmark Xue et al 2024 paper.\n- “_PGD-MOO underperforms ParetoFlow On the RE suite_”: We find that ParetoFlow is slightly better in terms of hypervolume than our approach when the number of objectives is 4 or greater, but our method is competitive or better otherwise. More importantly, our approach results in much more diverse solutions than ParetoFlow, an important aspect in MOO that has not been given enough attention so far. In this regard, we argue that our method is still better in practice when considering both diversity and hypervolume simultaneously. We have reported all findings transparently to highlight both the strengths and weaknesses of all methods, and we hope this balanced presentation will not be held against our paper.\n- “_Limited methodological novelty_“: Our method is novel across two different aspects - 1. A classifier based on preference defined by Pareto dominance and 2. As you have also acknowledged, ensuring diversity of resulting samples by explicitly training the classifier using the preference samples. While conditional diffusion and the use of preference in RLHF has been done before for other problems, we believe that the aspect of preference from a Pareto viewpoint while addressing the aspect of diversity is entirely novel for MOO.\n- “_Restricted to continuous design spaces_“: We note that the ParetoFlow method cannot be natively adapted to combinatorial problems. However, they do address the problem of discrete (categorical) cases by treating the discrete classes as continuous (through the logits of one hot representation). Since this essentially renders the approach as continuous, we can adopt the same approach for our method for discrete cases. Multi-Objective Neural Architecture Search benchmark provides such a case. As mentioned before, we will include this result in the revision.\n- “_Clarity,......,The explanation of the training method for the preference…. Is it following the Bradley-Terry Model_”: Thanks for bringing this point. Indeed, our preference model is based on Bradley-Terry. We train the preference model as just a binary classifier over preference pairs (which are designs with well defined dominance relations). During sampling, we use this trained preference model to guide the diffusion model using the logits of the class which indicates the first input dominates the other. This logit could be interpreted as the “reward” to the diffusion model for preferring the first input over the second, which corresponds to the Bradley-Terry setup. We will make this aspect more explicit in the revision.\n- “_Limited ablation about diversity contribution_\" : Thanks for the suggestion. We perform ablation experiments on diversity contribution. Due to the space and limitation, we are including only a subset of representative results below, but we will include the full set of results in the appendix in the camera-ready. Apart from the Crowding and SubCrowding approaches which are explained in the paper, we also include results for two different diversity criteria - _No Div. Criteria_ and _HV_. _No Div. Criteria_ is the approach without any explicit diversity contribution and HV is the approach with diversity measured as the hypervolume improvement due to any sample in the dataset.  We find that incorporating diversity criteria for preference generally results in better diversity of the resulting solutions. In addition, Crowding and SubCrowding gives better diversity than using hypervolume improvement. We provide detailed results below (all results are evaluated on 256 solutions with 5 random seeds):\n\n## ZDT Tasks -Spread (Lower is better)\n\n| Diversity Criteria   | ZDT1          | ZDT2          | ZDT3          | ZDT4          | ZDT6          |\n|:---------------------|:--------------|:--------------|:--------------|:--------------|:--------------|\n| No Div. Criteria        | 0.72$\\pm$0.04 | 0.60$\\pm$0.03 | 0.66$\\pm$0.03 | 0.65$\\pm$0.00 | 0.84$\\pm$0.01 |\n| Crowding             | 0.63$\\pm$0.09 | 0.58$\\pm$0.01 | 0.60$\\pm$0.02 | 0.60$\\pm$0.02 | 0.83$\\pm$0.04 |\n| HV                   | 0.64$\\pm$0.06 | 0.61$\\pm$0.03 | 0.60$\\pm$0.04 | 0.60$\\pm$0.02 | 0.86$\\pm$0.05 |\n| SubCrowding          | 0.70$\\pm$0.02 | 0.90$\\pm$0.04 | 0.66$\\pm$0.03 | 0.60$\\pm$0.04 | 0.77$\\pm$0.06 |     \n\n## RE Tasks - Spread (Lower is better)\n\n| Diversity Criteria   | RE21          | RE24          | RE25          | RE35          | RE41          |\n|:---------------------|:--------------|:--------------|:--------------|:--------------|:--------------|\n| No Div. Criteria        | 0.62$\\pm$0.01 | 1.16$\\pm$0.06 | 1.39$\\pm$0.06 | 0.90$\\pm$0.02 | 0.50$\\pm$0.04 |\n| Crowding             | 0.61$\\pm$0.02 | 1.20$\\pm$0.05 | 1.19$\\pm$0.08 | 1.00$\\pm$0.12 | 0.47$\\pm$0.04 |\n| HV                   | 0.62$\\pm$0.02 | 1.16$\\pm$0.09 | 1.14$\\pm$0.09 | 0.99$\\pm$0.08 | 0.47$\\pm$0.03 |\n| SubCrowding          | 0.60$\\pm$0.01 | 1.17$\\pm$0.05 | 1.12$\\pm$0.06 | 0.70$\\pm$0.03 | 0.43$\\pm$0.02 |\n\n- “_I am curious about the rationale behind choosing conditional guidance over diffusion finetuning methods_”: We choose conditional guidance as it is a relatively simple and straightforward method to generate samples beyond the model training distribution. We notice that for the datasets and benchmarks that we have tested these models on, they do in fact generate samples from beyond the training data, close to the Pareto front. However, diffusion finetuning methods are definitely an interesting alternative. We note that even when diffusion finetuning methods are used, conditional guidance usually has complementary strengths that can be used to further enhance the quality of designs.\n- “_It appears that the references cited in the main text (e.g., [1]) are not listed in the References section_”: Xue et al. 2024 has been listed in line 459 in references. Kindly let us know if we misunderstood something regarding this comment.\n\nWe hope our response satisfactorily addressed all your concerns. If there are still any open questions or concerns, we are happy to answer them."}, {"review_id": "XQgT4NLiRP", "reviewer": "Reviewer_U1bb", "summary": "The paper tackles the challenge of offline black-box multi-objective optimization (MOO), where new evaluations are infeasible, and the goal is to infer Pareto-optimal solutions beyond the observed dataset. To this end, the authors propose PGD-MOO, a novel inverse approach that combines diffusion models with preference-guided classifiers. Unlike forward methods that rely on training surrogate models for each objective, PGD-MOO leverages the generative capabilities of diffusion models to learn the mapping from objective values to input designs. Furthermore, a preference classifier model is trained to guide the denoising process toward the solution space. In addition to solution dominance, the classifier innovatively incorporates a diversity criterion to obtain more varied and sparsely distributed solutions. Experimental results demonstrate that PGD-MOO outperforms other inverse methods while maintaining strong generalization capabilities. It achieves solution quality comparable to forward methods, which may not be feasible in many real-world scenarios. Moreover, it exhibits significantly better diversity, enabling a more comprehensive approximation of the Pareto front.", "questions": "1. In practical optimization problems, there are often various constraints. How can the proposed method be extended to handle constrained multi-objective optimization problems and ensure that the generated solutions satisfy constraints?\n2. The training process of diffusion models is often computationally intensive. Are there ways to optimize the training process to reduce computational costs and improve training efficiency, such as through model architecture design or training algorithm improvements?\n3. As the number of objectives grows, Pareto dominance information tends to have a diminished impact. Why not consider incorporating hypervolume (HV) contribution or other metrics instead of relying solely on crowding distance when comparing two designs?\n4. The experiments conducted thus far have focused on small-dimension datasets. How can the proposed method be effectively extended to tackle problems characterized by high-dimensionality and complex constraints?", "limitations": "Yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n1. The paper pioneers a diffusion-based offline MOO method that integrates diversity optimization. It deftly replaces scalarized surrogates with a unified dominance/diversity classifier, thereby streamlining the overall architecture.\n2. The method incorporates crowding distance (a concept borrowed from NSGA-II) into preference labeling, ensuring uniform and comprehensive coverage of the Pareto front.\n3. Through evaluations on a diverse array of continuous offline multi-objective optimization tasks—spanning synthetic scenarios to real-world engineering problems—the proposed method consistently demonstrates superior performance. It excels in terms of both convergence to the Pareto front and the diversity of solutions.\n\n### Weaknesses\n1. The paper provides only a brief description of the diffusion model component. As the foundational element of the proposed method, the diffusion model's training process is not sufficiently detailed, particularly regarding how it is adapted to multi-objective optimization settings. The paper should elaborate on how multi-objective datasets are incorporated into the standard diffusion modeling pipeline and how the multi-objective nature of the problem is addressed during training.\n2. In the experimental section, only one inverse approach is included for comparison, which may not be sufficient to fully demonstrate the advantages of the proposed method. It is recommended to include a diffusion-based baseline without classifier guidance as part of an ablation study to better highlight the contribution of the preference-guided mechanism.\n3. The introduction section transitions rather abruptly into listing the contributions. Additionally, Section 3 offers relatively limited discussion of the proposed method, which appears disproportionate given the repeated exposition of the offline multi-objective optimization background earlier in the text. It is recommended to streamline the problem formulation to reduce redundancy and instead devote more space to elaborating on PGD-MOO, particularly the application and adaptation of the diffusion model.", "was_revised": true, "final_justification": "I have carefully reviewed the author’s rebuttal and taken into account all the points raised. After thorough consideration, I believe that my original score remains appropriate and I will maintain it as is.", "rebuttal": "Dear reviewer,\n\nThanks a lot for your detailed feedback. We are glad that you find that the strength of our work in excelling at both the diversity of solutions while also addressing the convergence to Pareto front. We address your concerns and questions below:\n\n- “_The paper provides only a brief description of the diffusion model component_” : We have explained the diffusion model in Sec 2.2 as well as the training details in Sec 5.1. The diffusion model is an unconditional one, i.e. it just models the distribution of the data. We incorporate multi-objective optimization entirely through our preference model, which is used for guidance. This is detailed in Sec 4. We wanted to keep the diffusion model explanation succinct in order to explain the main contribution of the paper, i.e. the preference model in more detail. But we are happy to add more details about diffusion models in the appendix for the camera-ready.\n- “_In the experimental section, only one inverse approach is included for comparison_”: ParetoFlow, which is another inverse approach, is the only approach which exists which directly addresses the problem of Offline MOO. If there are other inverse approaches which are relevant that we might have missed, please let us know and we are happy to compare them.\n- “_recommended to include a diffusion-based baseline without classifier guidance as part of an ablation study to better highlight the contribution of the preference-guided mechanism_”: Thanks a lot for the suggestion. In response to the reviewer 79Tz, we have provided results for ablation studies over the guidance weight $w$, including for the case of $w=0$, which corresponds to the simple diffusion based baseline without preference guidance. We find that having preference is essential to obtain designs close to the Pareto front. We will include these results in the camera-ready.\n- “_..introduction section transitions rather abruptly….It is recommended to streamline the problem formulation to reduce redundancy and instead devote more space to elaborating on PGD-MOO, particularly the application and adaptation of the diffusion model_”: Thanks for the suggestion. We have tried to structure the writing in such a way that the main contribution of the paper and the problem statement are laid out well. However, we will try to streamline a bit of the background section explain the diffusion model component in more detail in the revision.\n- “_In practical optimization problems, there are often various constraints. How can the proposed method be extended to handle constrained multi-objective optimization problems_”: That’s a great question. Note that input constraints—i.e., restricting inputs to a certain range—are already incorporated into our training procedure. If one instead wants to enforce constraints on the outputs (i.e., the objective values), this becomes a significantly harder problem, and to our knowledge, has not been addressed in the existing offline MOO literature. One intuitive way to approach this would be to train a classifier to distinguish between constraint-satisfying and constraint-violating solutions based on the training data, and then incorporate multiple classifier guidance during generation. However, this approach does not guarantee that the generated designs will always satisfy the constraints. We acknowledge that properly handling such output constraints requires careful treatment and is outside the scope of the current work.\n- “_The training process of diffusion models is often computationally intensive. Are there ways to optimize the training process to reduce computational costs_”: While training a diffusion model can be computationally expensive, our method benefits from general advancements in  faster training and sampling strategies proposed for diffusion models in the literature. For instance, one could use DDIM sampling [1] to speed up sampling. Other techniques used for faster training of diffusion models can also be used.\n- “_Why not consider incorporating hypervolume (HV) contribution or other metrics_”: Indeed, that is a very good point. While incorporating HV is desirable, it might be sensitive to scale due to the reference point. The actual value of HV can be different for different reference points. So for a practical problem where the reference point is harder to define, HV is harder to incorporate. However, if a reference point is well established, HV is definitely a reasonable alternative to define preference. In the response to reviewer 1hyQ, we have provided results on incorporating HV contribution as a way to measure diversity. We find that it is helpful, although the proposed _Crowding_ and _SubCrowding_ approaches are slightly better. We will add this discussion in the revision.\n- “_The experiments conducted thus far have focused on small-dimension datasets_\": We have conducted experiments on settings where the dimensions are up to 30. The advantage of using a diffusion model is that it is shown to scale very well with the number of dimensions, as opposed to forward approaches based on surrogate models. We believe that a similar observation will hold true to this setting as well. Realistic high dimensional multiobjective datasets are also needed to thoroughly evaluate the model’s effectiveness for high-dimensional settings.\n\nWe hope our response satisfactorily addressed all your concerns. If there are still any open questions or concerns, we are happy to answer them.\n\n## References\n\n[1] Song, Jiaming, Chenlin Meng, and Stefano Ermon. \"Denoising diffusion implicit models.\" arXiv preprint arXiv:2010.02502 (2020)."}, {"review_id": "e1t91e2En6", "reviewer": "Reviewer_rK4P", "summary": "This paper proposes a novel generative approach for offline multi-objective optimization using a classifier-guided diffusion model. The core idea is to guide the generation process using a preference model trained to predict whether one design dominates another. This preference model is not the same as the surrogate model generall used in offline MOO.  This guidance steers the diffusion model toward Pareto-optimal regions, including those beyond the training data.", "questions": "See Strengths And Weaknesses*", "limitations": "See Strengths And Weaknesses*", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: the paper handles an important problem and uses advanced techs to achieve impressive results.  I like the idea of using preference model to guide diffusion generation. \n\nWeakness:\n1. The paper misses an important citation: Kim M, Gu J, Yuan Y, et al. Offline Model-Based Optimization: Comprehensive Review[J]. arXiv preprint arXiv:2503.17286, 2025.\n\n2. Figure 1 seems never mentioned in the text. \n\n3. It looks like diversity contribution is dataset-dependent while the preference model is dataset-independent. How can u train a preference model using the diversity contribution or \"equally dominant\" is very rare?\n\n4. I think the author may misunderstand some point: the preference model is trained on pairs from the offline dataset (time step 0) while used to guide designs across different time steps.", "was_revised": true, "final_justification": "I’m leaning toward accepting.", "rebuttal": "Dear reviewer,\n\nThanks a lot for your detailed and positive feedback. We are glad that you find our work addresses an important problem with impressive results. We address your concerns and questions below:\n\n- \"_Missing Citation_\": Thanks for bringing the Kim et al 2025 paper to our attention. While this paper directly does not address offline MOO, it does provide an overview of various approaches in the general offline setting. We will include the citation in the revision and discuss this aspect.\n- \"_Reference to Figure 1_\": Thanks for bringing this to our attention. We will include a reference to figure 1 in the main text.\n- “_How to train preference model when equally dominant is rare_”: That is a good question. We hypothesize that a sufficiently large dataset typically contains samples within each front, such that diversity can be computed. When the number of samples are low such that equally dominant points are rare, the preference model in this case would be trained more on the aspect of dominance rather than diversity. Given that the dataset is limited in this case, we argue that such a behaviour is expected and even preferable.\n- “_The preference model is trained on pairs from the offline dataset (time step 0) while used to guide designs across different time steps_”: The preference model is not just trained on the offline dataset at time step 0, but also on the noisy versions of the offline dataset, similar to what is done in classifier guidance for data modalities like images or text.\n\nWe hope our response satisfactorily addressed all your concerns. If there are still any open questions or concerns, we are happy to answer them."}]

Author rebuttal:

Meta-review: This work proposes PGD-MOO, a preference-guided diffusion method for tackling multi-objective offline optimization problems. The key idea of PGD-MOO is to build a diffusion model with preference-based classifier guidance to generate Pareto optimal solutions based on offline dataset. Inspired by the idea of the seminal multi-objective optimization algorithm NSGA-II, the guidance classifier is trained to predict the domination relation between two different solutions, which can then be used to direct the diffusion model to generate Pareto optimal solutions (the ones that dominate other non Pareto optimal solutions). In addition, a diversity-aware preference guidance approach using crowding distance in NSGA-II is proposed to further encourage the model to generate well-distributed solutions on the Pareto front. Experimental results show PGD-MOO can achieve promising performance on the ZDT/DTLZ, RE, and MO-NAS problems (provided in the rebuttal) under the offline optimization setting.

The reviewers find this paper clearly structured and easy to follow, the proposed PGD-MOO method clear and well-motivated, the diversity-aware approach compelling and can ensure uniform coverage of the Pareto front, and the code implementation clean and well-constructed. Some concerns have been raised on clarifications, relations to existing works, missing citation, methodological novelty and extension, experimental coverage and performance. Most of these concerns have been properly addressed during the rebuttal.

Finally, this work has mixed ratings, where one reviewer votes to clearly accept this work, 2 reviewers lean toward weak acceptance, and 1 reviewer recommends weak rejection. In the final justification, the most positive reviewer 79Tz believes this paper would be of high quality after revision. The original most negative reviewer 1hyQ believes all of his/her concerns have been resolved and hence raised the rating to weak acceptance. The remaining negative reviewer U1bb did not respond to the author's further response.

I read the paper myself and believe that offline multi-objective optimization is an important problem, and this work is a timely and valuable contribution to this research direction. I agree with the reviewers that the proposed PGD-MOO method with a diversity-aware approach is well-motivated and compelling. In addition, I believe the authors' rebuttal has properly addressed the concerns raised by reviewer U1bb, and the new results on 18 practical MO-NAS problems provided in the rebuttal are also highly appreciated. Therefore, I recommend acceptance of this work.   

Please make sure all the discussion and revisions are carefully incorporated into the final paper, and release the code as promised.
Decision: Accept (poster)