SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusion
Authors: Sedjro Salomon Hotegni
Sebastian Peitz
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-objective optimization
Denoising Diffusion Probabilistic Models
Multiple gradient descent
Offline multi‑objective optimization
Multi-objective Bayesian optimization
Diffusion Transformer
Primary area: generative models
TL;DR: We propose SPREAD, a diffusion-based generative framework for multi-objective optimization that couples adaptive gradient updates with a repulsion mechanism, achieving competitive efficiency, scalability, and Pareto front coverage across benchmarks.

Abstract: Developing efficient multi-objective optimization methods to compute the Pareto set of optimal compromises between conflicting objectives remains a key challenge, especially for large-scale and expensive problems. To bridge this gap, we introduce SPREAD, a generative framework based on Denoising Diffusion Probabilistic Models (DDPMs). SPREAD first learns a conditional diffusion process over points sampled from the decision space and then, at each reverse diffusion step, refines candidates via a sampling scheme that uses an adaptive multiple gradient descent-inspired update for fast convergence alongside a Gaussian RBF–based repulsion term for diversity. Empirical results on multi-objective optimization benchmarks, including offline and Bayesian surrogate-based settings, show that SPREAD matches or exceeds leading baselines in efficiency, scalability, and Pareto front coverage.  Code is available at https://github.com/safe-autonomous-systems/moo-spread .

Published as a conference paper at ICLR 2026
SPREAD: SAMPLING-BASED PARETO FRONT
REFINEMENT VIA EFFICIENT ADAPTIVE DIFFUSION
Sedjro Salomon Hotegni, Sebastian Peitz
Department of Computer Science, TU Dortmund University Lamarr Institute for Machine Learning and Artificial Intelligence (sal (salomon saalomon.. hotegni, sebastian petpaastiiaapeetteeeeeeee pwwp:ee:..ttooddrrmoddd..
ABSTRACT
Developing efficient multi-objective optimization methods to compute the Pareto set of optimal compromises between conflicting objectives remains a key chal- lenge, especially for large-scale and expensive problems. To bridge this gap, we introduce SPREAD, a generative framework based on Denoising Diffu- sion Probabilistic Models (DDPMs). SPREAD first learns conditional dif- fusion process over points sampled from the decision space and then, at each reverse diffusion step, refines candidates via sampling scheme that an adaptive multiple gradient descent-inspired update for fast convergence along- side Gaussian RBF-based repulsion term for diversity. Empirical results on multi-objective optimization benchmarks, including offline and Bayesian surrogate-bar based settings, show that SPREAD matches or exceeds leading base- lines in efficiency, scalability, and Pareto front coverage. Code is available at com safe aut systems/moo sprooosppaad...
INTRODUCTION
Multi-objective optimization (MOO) is fundamental in numerous scientific and engineering disci- plines, where decision-makers often face the challenge of optimizing conflicting objectives simulta- neously (Rangaiah) 2016 Malakooti] 2014 Zhang et al. 2024b). The primary aim is to identify the Pareto front: set of non-dmminteed solutions where improving one objective would deteriorate at least one other. Traditional methods for approximating the Pareto front include evolutionary algo- rithms (Deb) 2 Zhou et al. 2011] scalarization techniques (Braun al., 2015 Hotegni Peitz 2025), and -rraiieet descent (MGD) (Désidéri 2012, Sener & Koltun 2018) combined with multi-start techniques (i.e., using random initial guesses to obtain multiple points). However, these approaches may struggle with scalability, especially in high nigh-dimensional or resource-coossrraieed settings (Cheng et al. 202T] Li et al.) 2024a).. As workaround, domain-specific MOO algorithms have been developed to leverage domain knowledge to improve efficiency and solution quality in targeted settings (c.g., offline MOO Yuaan et al. 2025b), Bayesian MOÔ (Daulton et al. (002)). federated learning (Hartmann al., (0255), but at the expense of broader applicability. These challenges underscore the need for MOO methods capable of efficiently adapting to large-scale, high- dimensional, and computationally expensive problem settings. Developing such universal approaches would not only streamline the optimization process, but would also broaden the applicability of MOO techniques across different domains.
Recent advancements in generative modeling have shown promise in addressing complex optimization problems (Garciarena al. 2018, Yuan et al. 2025a). In particular, diffusion models such as Denoising Diffusion Probabilistic Models (DDPMs), have demonstrated remarkable capabilities in generating high-quality samples across various domains (Ho al., 2020 Yang et al., 2023). Their iterative refinement process aligns well with the principles of MOO, offering potential pathway to efficiently approximate the Pareto front. this work, we introduce SPREAD (Sampling-base Pareto front Refinement via Efficient Adaptive Diffusion), novel diffusion-driven generative framework designed to tackle multi-objective optimization across diverse problem settings. Our approach leverages the strengths of diffusion models to iteratively generate and refine candidate solutions, guiding them towards Pareto optimality. SPREAD applics a conditional diffusion modeling approach, where conditional DDPM is trained points sampled from the input space, allowing

---

Published as a conference paper at ICLR 2026
the model to effectively learn the underlying structure of the objective functions and steer the generation process toward promising regions. To further enhance convergence towards Pareto optimality, SPREAD incorporates adaptive guidance mechanism inspired by the multiple gradient descent algorithm (Désidéri 2012), dynamically guiding the sampling process to regions likely to contain optimal solutions. Furthermore, to promote diversity among the generated solutions and ensure an approximation of the entire Pareto front, SPREAD utilizes a Gaussian RBF repulsion mechanism (Buhmann 2000) that discourages clustering, mitigates mode collapse and encourages exploration in the objective space.
We evaluate SPREAD on diverse MOO problems including two challenging, resource-constrained scenarios: offline multi-objective optimization (Xue et al.] 2024) and Bayesian multi-objective optimization (Knowles, 2006). each case, we benchmark our method against state-of-the- approaches specifically tailored for these settings. Our empirical results demonstrate that SPREAD not only achieves competitive performance but also offers superior scalability and adaptability across different problem domains. Our contributions can be summarized as follows: (a) We propose a novel diffusion-based generative framework for MOO that effectively approximates the Pareto front. (6) We introduce novel conditioning approach along with adaptive guidance mechanism inspired by MGD to improve convergence towards Pareto optimal solutions. (c) We implement a diversity- promoting strategy to ensure comprehensive and well-distributed set of solutions. (d) We validate our approach on challenging MOO tasks, demonstrating its effectiveness and generalizability.
2 RELATED WORK
We now situate our approach within prior work on generative modeling for multi-objective optimiza- tion, and gradient-based methods relevant to our settings. An extended discussion of related work is provided in Appendix
Generative Modeling for Multi-Objective Optimization Recent work explores alternatives to traditional search methods, such as evolutionary algorithms or ccquisition-based optimization, by directly generating Pareto optimal candidates. ParetoFlow (Yuan et al., 2025a) uses flow-matching with multi-objective redictor-guidance module to steer samples toward the front in the offline setting, showing that guided generative samplers can cover non-convex fronts efficiently. In parallel, preference classifier and uses it for diffusion guidance to obtain diverse Pareto optimal designs from data. For Bayesian settings, CDM- PSI. (Li al. 2025a) couples unconditional/connittonn diffusion with Pareto set learning propose candidate points under tight evaluation budgets. Our approach differs by conditioning a diffusion transformer on objectives and applying step-wise, MGD-inspired guidance together with an explicit diversity force, yielding both convergence and spread without a separate preference classifier.
PGD-MOO (Annadani al. 2025 trains a dominance-
Gradient-based Methods for Pareto Set Discovery A complementary line of research explicitly profiles the Pareto set by moving population with repulsive interactions or by smoothing scalar- izations. PMGDA (Zhang al., 2025) extends the classical MGDA (Désidéri (012) by sampling multiple descent directions in a probabilistic manner, thus improving stability and coverage in high dimensions. Smooth Tchebycheff scalarization (STCH) provides lightweight differentiable scalarization with favorable guarantees (Lin et al. 2024). Leveraging hypervolume gradients, HV- Grad (Deist al. [ updates solutions toward the Pareto front while preserving diversity, while MOO-SVGD (Liu et 202T) employs Stein variational gradients to transport particles and obtain well-spaced fronts. Extending this line of work, SPREAD integrates MGD directions into DDPM denoising process, using them as adaptive guidance signals within diffusion sampling.
3 PRELIMINARIES
To set the stage for our method, we review the fundamental concepts on which our approach is based.
3.1 DENOISING DIFFUSION PROBABILISTIC MODELS (DDPMS)
As powerful generative models, Denoising Diffusion Probabilistic Models excel at producing high- quality samples in wide range of applications. such as image synthesis (Dhariwal & Nichol 2021) speech generation (Kong et al. 2020), and molecular design (Hoogeboom et al.) 2022). These models operate by simulating forward diffusion process, where Gaussian noise is incrementally added to
2

---

Published as a conference paper at ICLR 2026
data, followed by a learned reverse process that denoises the data step by step. In the conditional setting, DDPMs generate data samples conditioned on auxiliary information c, enabling controlled generation aligned with specific attributes constraints. The forward diffusion process gradually corrupts a data point Xo over 7 ttiesstes:
(xxx(xx-1) = ( V 1 1xx
where is variance scheduling parameter, often chosen according a linear (Ho al. 2020) or a cosine (Nichol & Dhariwal] 2021) schedule. After 7 steps, Xr approaches a standard Gaussian distribution. The aim is reconstruct X from xT by learning a parameterized model (.(-) that predicts the added noise at cach timestep t, conditioned on The model is trained to minimize the following loss:
(0) Exo.t.t.c ||e- fle--élfe (x)
(2)
where c <~ (O, I) is the true noise added to Xu randomly chosen timestep t {1...., T to
obtain X2, at each epoch. Specifically, from equation we have after t timesteps X = vā, Xo
vI - āg with ā₂ I 3.).
At inference (post-training), sampling starts from pure noise XT -N(00, I) and iteratively denoises using the learned reverse process:
B. Xx VI- a
(x) VBz,
Dt
where z (O, I). To enhance sample quality and control, guidance techniques can be applied: classifier guidance introduces gradients from a separately trained classifier to steer the generation process (Dhariwal & Nichol [02II) while classifier free guidance interpolates between conditional and unconditional predictions within the same model, allowing for flexible control without additional classifiers (Ho & Salimans 2022).
3.2 MULTI-OBEECTTO OPTIMIZATION (MOO)
Multi-objective optimization involves optimizing multiple conflicting objectives simultaneously (Eich-
felder 2008 Peitz& & Hotegni 2025):
min F(x) (..x) (x)),
(MOP)
xEX
where X' is the decision space, and each fj X IR., j E (1,...,m, m) represents an objective function. Throughout this paper, we assume that each objective function is continuously differentiable. Definition 1 (Pareto Stationarity). solution x" X' is said to be Pareto stationary if there exist nonnegative scalars AI,..., Amm with ΣØ=1 >3 = 1, such that ",
Such points are necessary candidates for Pareto optimality but may include non-optimal solutions.
Definition (Dominance). A solution x' E X' is said to dominate another solution x E X (denoted
x' x) if: fx(x)= S (x) for all = =,...., m, and ži E {l,..., .... | f((x') < fi(x).
Definition 3 (Pareto Optimality). A solution x* E X is called Pareto optimal if there is no x' E X such that x' < x". is called weakly Pareto optimal if there is no x' E X such that fz(x') < (x for all 1,...,m. m.
The set P of all Pareto optimal solutions is called Pareto set, and its image F F(P) (F E P). is known Pareto front. Among the various strategies for solving multi-objectio optimization problems, gradient-basse based techniques are of particular relevance, and in our case, multiple gradient descent serves as the key inspiration for the update mechanism within SPREAD.
3.3 MULTIPLE GRADIENT DESCENT (MGD)
Multiple gradient descent is technique designed to find descent directions that simultaneously improve all objectives in MOO (Désidéri 2012). Given the gradients V (x) for each objective f,, MGD seeks a convex combination of these gradients that yields common descent direction each iteration. This is achieved by solving the following optimization problem:
X = arg min AEA...
X (x)
(4)
3

---

Published as a conference paper at ICLR 2026
(xoc)
Input Projection (.
Quiper Projection fe di
Layer Norm 1j
Laser Norn 1
Layer (A,
Nota
N (n,
MB Sei-Atication 7
NH Cros-Allation J.
PW Vradierant (b, 2,
1imattpp
(s. 2.
Conditias C (i n))
Enthed (s, J,
Layers
Farward Pas
Concotenation
Element-wise Addition
Figure I: DIT-MOO architecture. Diffusson Transformer adapted for multi-objective optimization, where noise prediction conditioned on objective values condition via multi-head cross- cross-attention.
where Am {> I プソ = 1, 2 0} is the standard simplex. The optimal weights X define the aggregated gradient g(x) = Ey= fz (x), whose negative serves as the common
descent direction. The decision variable is then updated using this direction: X = X, 77: (x)), with 77, being the step size at iteration t. While MGD ensures convergence a Pareto stationary point, employing classical multi-start approach does not inherently promote diversity among solutions. To overcome this drawback, our method incorporates mechanism that promotes diversity, as detailed in the next section.
4 METHOD
In this section, we first present the core components of our method for solving an MOP in an online setting (full access to the objective functions), and then discuss how we adapt them to different resource-cone conuuree-onstraieedddeeeeee settings. We adopt Transformer-bar (Fig. [ adapted from the Diffusion Transformer (DiT) architecture (Peebles Xie 2023), for stable and scalable sampling. The model takes input a batch of n noisy decision variables X, E R together with a timestep and a condition C. and outputs the predicted noise é (X,, t, C). A cosine schedule (Nichol & Dhariwal 2021 is considered for the variance scheduling parameter B, Further architectural details are provided in Appendix A.4
noise-prediction network, DiT-MOO
Training For a given MOP, we sample N points (x)) J = X from the decision space X' C Rd
via Latin hypercube sampling (McKay et al. 2000) to create the training dataset. Our DiT-MOO is then trained using the loss L. (equation on pairs (x', c²) with
c F +
E o0)
(5)
During sampling, however, we condition on the original objective vector Fxx'). The shift E can be any vector with strictly positive entries, fixed for the entire dataset or varying per point or batch. The following theorem establishes the key advantage of this conditioning approach.
Theorem 1 (Objective Improvement). Let X C. X be training dataset with distribution Px. Let (0, orm independent of X. and define the training label
C = +
(6)
For conditioning value in the support of C, denote by Pxic=e the true conditional data distribu- tion and by Qo(- c) the distribution produced by a conditional DDPM when sampling conditioned on c. Assume the sampler approximates the true conditional training distribution total-variation TV distance by at most E (0,1):
(Qo c), Px|c=e) = supp (2 c) - -x)c=c(A) S T.
(7)
Fix any initialization X1 X and set Cr F (xr). Ifcr lies in the support of C, and we draw Xg ~ Qo( |T), then:
Pi(x xXT) 1 T-
(8)
4

---

Published as a conference paper at ICLR 2026
In other words, conditioning the reverse diffusion F(xx) yields, with probability at least 1 sample that dominates Xy-
The proof of this theorem is provided in Appendix A.1
Sampling Let X [ X' denote random initial points. We refine them by iteratively applying the reverse diffusion step (equation 3) augmented with guided update that (i) aligns each sample with its MGD direction and (ii) encourages dispersion in the objective space to promote diversity. Specifically, this guidance is implemented via an additive term, balancing objective improvement (in the spirit Section[3.3] with spreading along the Pareto front, together with a small noise term. At each sampling step I, the update is therefore:
B,
X
VI B X
(, (x,,,,, + BB.z
Eo
VI de X - X nu (X,)
(9)
where the condition C' is the batch of the objective values related to X,, and (x()) (h:)²= (b:) + +
(10)
the guidance directions. Here, 8 Fd is random perturbation added to the main directions (h)) and (.t....,77)) 7²"²" E I are scaling parameters that control the strength of this perturbation. We choose hi, i ....., balance objectives:
ii) Alignment with the MGD directions: Let gi = g(x;",, I,..., n be obtained as defined in Section 3.3 The main directions are chosen to maximize the average inner product
n
(l bz).
(11)
(ii) Diversity in objective space: Define (yi)=1 = Y,, F (X; Tt ((h)) + 7f 6.)) The main directions are chosen so as to minimize the Gaussian RBF repulsion function Buuh
mann 2000)
T, n(n 1)
2
Σ ISK4jSn
expl
ly; yd 20²
(12)
where > 0 is the length-scale.
Balancing the alignment objective (equation 11 with the diversity requirement (equation 12 we obtain the main directions by solving the following sub-problem:
(ht) arg min (u))
1 n gi u²) E
[X -e ((n') + 7 6.))
F
(13)
where 14 0. In practice, solve this sub-problem by performing fixed number of gradient descent steps, which provides an approximation the main directions while keeping the computational cost manageable. In the case where v, O, the main directions h 1,..., n, are well aligned with the MGD directions and thus inherit their descent properties. This assumption leads to the following theorem:
Theorem 2. Assume each objective function fÚ is continuously differentiable, and that v 0.ror all t te (I,...,T) T). Let, at reverse timestep 1,
aij (Vi)(''),,
bis = (Vi,(x('),
with az5 > for all 1,..., , n and = -,..., "...,,m, m. Define
ni ( min
(- dij
0 0< bi.j"
if any b 0.
jibe,
(14)
(c, <>
otherwise,
where p controls the magnitude of the scaling parameters 7i and denotes arbitrary positive scalar. Then, -h; --
(h₁ + 7(6c) serves common descent direction for all objectives x²
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 SPREAD (Online Setting)
Algorithm 2 Training (Online Setting)
Input: DIT-MOO architecture (untrained model), a multi- objective optimization problem (MOP).
Input: DiT-MOO as the noise prediction net-
work éa(-). multti-objective optimization problem [OP].
Parameter: epochs E, timesteps 7', sample size n.
Output: approximate pareto optimal points Po.
Parameter: epochs E, timesteps T.
I: DiT-MOO training via Algorithm 2
Output: a trained noise prediction network (o(+)
2: Initialize random points XT (xr C X
3: Pr Xxr
Sample N points {x] Ji= X X using
for T to
Latin hypercube sampling (Appendix A.5
5: (g:)
)=
<- Get the MGD directions via Sec-
2: {Be f- Get the variances via a cosine schedule (opeendi([A.5
tion 3.3
6:
(h,)
Get the main directions via equa-
3: for epoch = to E do
tion
4:
Uniform( (1,.-..,7) X, f Vā
7:
(h:)
t Get the guidance directions via equa-
5:
vI 8z with e ~ (0, I) and ā. + IT. i= (1 0.).
tion 10
8:
X,-1
Get the denoised points via equation g
6:
C - F(X,) + E. with E
(0, )m) an
arbitrary vector with strictly positive entries.
9: Pt-1 + Use crowding distance (Appendix A.5)
to get the top-n non-dominated points from XI-UUP-
7: Take gradient descent
step on
Va |le és (
10: end for
end for Return: Ex(-)
Return: Po
We provide the proof of this theorem in Appendix A.2 While vs 0 guarantees common descent direction for all objectives, it is very strong and restrictive assumption, since a moderate value Pe necessary to achieve good coverage of the Pareto front. A discussion
the theory for the general case 2/2 > O can be found in in Appendix A..] including the sketch of a proof. An ablation study illustrating this trade-off is presented in Appendix D (Figure 99. To determine the batch 7) of step sizes at timestep / (equation 9). we employ an Armijo backtracking line search (Armijo 1966). This ensures sufficient decrease in the objective functions each timestep I, prevents overly aggressive steps, and adapts to local curvature (Fliege& & Svaiter 2000).
The proposed SPREAD framework for solving multi-objective optimization problems is summarized in Algorithm|] | The final set Po of approximate solutions is obtained as the top-n non-dominated points from the union X, U... XT. More specifically, for two successive reverse timesteps and t t- 1. we define Pt-1 as the top-n non-dominated points from the union X,-1 U P. (with Pr XT initially), using crowding distance (Deb al. 2002a) to preserve diversity (preferring non-dominated solutions that are less crowded in objective space).
4.1 EXTENSION TOWARDS SURROGATE-BASE OPTIMIZATION
Beyond the classical (online) setting, SPREAD extends naturally to resource-constee constraine multi- objective optimization, where true objective evaluations expensive limited and surrogate models are used. Such challenges arise domains like offline MOO and Bayesian MOO, which require dedicated multi-objective optimization methods to handle restricted costly evaluations.
In offline multi-objective optimization, the true objective functions are unavailable. Instead, relies a pre-collected dataset D {(x, F(x)), X' to train surrogate function which serves as a proxy model for the objectives (Xue et al. 2024). To adapt SPREAD to this and
Offline MOO:
setting, we set X DD in Algorithm and use = F in Algorithms
A key constraint in multi-objective Bayesian optimization (MOBO) is the limited evaluation budget of an expensive F, which typically addressed by employing iteratively updated Gaussian process surrogate models. Using simulated binary crossover (SBX) (Deb) 1995) as an auxiliary escape mechanism to avoid local optima, together with the data extraction strategy proposed in CDM-PSL (Li et al. 2025a), adapt SPREAD to the MOBO setting. The procedure is described in Appendix B (Algorihm B), along with further details. Moreover, Algorithm [ from the online setting is adapted to Algorithm 4 using Gaussian processes.
Bayesian MOO:

---

Published as a conference paper at ICLR 2026
5
EXPERIMENTS
5.1 ONLINE MOO SETTING
We evaluate our method on a diverse suite of problems, ranging from synthetie benchmarks (ZDT ZZit-
Zler al. ),00), DTTLZ (Deb et al.) 2002b) real-world engineering design tasks RE (Tanabe &
Ishibuchi 2020p. All synthetie problems use an input dimension of d = 30. The selected woald
tasks use N with continuous decision spaces. The baselines considered are gradient- based
MOO methods for Pareto set discovery: PMGDA (Zhang et al. 2025). STCH (Lin et al 2024).
MOO-SVGD (Liu et al. 202T), and HVGrad (Deist al.[ For SPREAD, we train MOO for 1000 epochs with early stopping after 100 epochs. We set the number of timesteps to T = 5000,
and each baseline is also run for 5000 iterations. Each method produces set of 200 points, and the quality of the solutions is assessed using the hypervolume (HV) indicator (Guerreiro et al. 2020). More detailed descriptions of the experimental protocols appear in Appendix C
Table I: Hypervolume results averaged over 5 independent runs. The best values are bold.
HV 171
m-
3
Method
ZIPT1
701
2013
RE:31
P71.72
D1T-74
DT1.75
KF.33
RB34
RKF
RE41
PMCIDA
MOO SVGD
5,0030.00
6.21:000
6.880002
20.34-0.2
22.1-002
19.09
1357:005
16.26-0.17
156.2010:57
1.5=0009
579.5366442
STCH
571:0.000
5.9:0000
5:4400133
19.0760.00
14.55:0000
1746:00.0
12:14:000
1967240.00
031=0002
506.3342.86
HVGrad
5.72:0.00
6,22:0000
6.10::0.00
43.65.0.00
19.98.0004
17,48:0055
30.13:0000
156.7240.000
1.4.0000
930. 178.1728.91
SPREAD
572219.00
6,23:0000
6.10::0.00
70.10:1.011
229
20.22:0001
18,07:00011
133.763172
24
142-0000
1008.75::00 26.377
Table 2: Results of the A-sprea diversity measure. The best value, along with those whose mean falls within one standard deviation of it, are shown in bold.
specad (1)
m
Method
ZDTI
ZDT2
2013
RE:11
DTL
DTL.44
D11.77
RE333
RE34
RK37
KK41
FMGDDA
04220.17
0.23
13720.02
1.550000
0.66
1.71:007
LXC
1:11:0000
1,4030000
0.2930.01
1.4010000
MOD-VVG
0.78+0020
1.1-0.11
8.000.08
101+0000
1.110001
1.02-0099
077+0.8
100+000
1.20-0,17
0.58+00.07
1.13+00.04
STCH
10110004
1:000000
105=043
10050000
100:0004
100=0000
106
10000000
1000:0000
0.30240.4
1.3861000
HVGra
0.36+006
0.070005
108+0.10
100+0000
188+005
1.56-0.6
0.66-1833
1.0000000
1.00-0.00
0.50+
1.00-0.02
SPREAD
0.32+8.01
0.2900002
0.53-0.01
0.44:0002
09350005
0.80-9.06
0.0::1005
0.97:8002
0.88
0.0000001
0.92=00003
Table I reports hypervolume results for problems with two to four objectives. On the bi-objective synthetic problems ZDT1-3, SPREAD matches the best values, while clearly outperforming the baselines on the real-world task RE21. For three objectives, SPREAD achieves the best results on 4 out of the 6 evaluated problems. On the four-objective problem RE41, it attains the highest hypervolume overall. To assess the diversity of the generated solutions for each method, we evaluate the A A-spread measure as introduced in Deb et al. (002a). By convention, A-spread is set to too when the solutions collapse a single point. As reported in Table 2 our method yields more diverse solutions on most problems. These results indicate that SPREAD maintains superior performance as the number of objectives increases, providing superior coverage and diversity of the Pareto front in both synthetic and engineering benchmarks. In Appendix D (Figure we show the approximate Pareto optimal points produced by the different methods for four synthetic and four real-world problems.
Scalability Analysis We further investigate the scalability of all methods by comparing their computational time as the number m of objectives increases (ZDTI with m = 2. DTLZ2 with m = 3, and RE41 with m = 4) and as the number n of required samples grows (DTLZ4 with n = 200, 400, 600, 800). Unlike the baselines, SPREAD requires a training phase, so we account for both training and sampling times to ensure fair comparison. As shown in Figure a and Figure bb) PMGDA exhibits the largest growth rate in computational time with increasing m and n. In contrast, SPREAD achieves substantially lower computational time than PMGDA, while being moderately more costly than MOO-SVGD, HVGrad, and STH. However, as shown in Figure ( and Figure 2 d), SPREAD consistently offers superior performance in hypervolume and A-spread compared to the other methods. Therefore, our method provides a favorable trade-off between efficiency and performance.
Ablation Study We present in Table an ablation study on the diversity-promoting mechanisms in SPREAD. Specifically, we evaluate three variants: SPREAD(W/o diversity), with (ht) = (gi)
SPREAD(W/o repulsion), with (h))², (gt) y + Tf; and SPREAD(W/o perturbation). with
(h:) (h:),,- The results indicate that SPREAD(W/o diversity) and SPREAD(w// repulsion) tend to collapse the solutions to a single point (A-spread = +00). Ignoring the perturbation (SPREAD(W/6 perturbation)) has milder impact on solution quality for some problems. However,
7

---

Published as a conference paper at ICLR 2026
MOD-SVGD
Time (minutes Tel tio
HVGrad
19:03 50.00
6-spread
PMIDDA
SPREAD (Durs)
109 200
19V
20.0
1.2
SAN
2012)mm21
660
600
200
450
650
600
(a)
(b)
(c)
(d)
Figure 2: Scalability. Comparison of (a) computational time as the number of objectives increases (ZDTI with 2. DTLZ2 with m 3, and RE41 with 4), and (b-d) computational time, hypervolume, and A-spread, respectively, as the number of required samples increases (DTLZ4).
Table 3: Ablation study on the highlighted in bold. For A-spread, any mean value within one standard deviation the best is also shown in bold. Worst values are shown in red, while best values are shown in blue (HV) and green (A- (A-spread).
mechanisms in SPREAD. Best values are
SPREAD
SPREAD A (w/o diversity)
SPREAD PREA (w/o perturbation)
SPREAD D (wlo repulsion)
Problem
HV
A-spread
HV
Aspread
HV
A-spread
HV
A-spread
ZDTI
5.72=0.00
0.32:10.01
5.06=0.00
+00
5.72::0.00
0.32=0.02
4.25=0.08
0.8820.05
ZDT2
6.22=0000
0.29::0.22
5,89=0.00
6.22-0000
0.28=0.02
4.40=0.14
ZDT3
6.10=0.00
0.53=0001
5.06=0.00
0.66+0.00
6.10=0.00
0,51=0.01
4.34=0.07
0.84+0005
RF21
70.10-0.00
0.44+0.02
70.03++0.01
0.41+00.22
+0.11
0.84-0005
70.03-+0.03
0.51+0.03
DTL22
22.91=0.00
0.93+00.5
22.94+0.00
0.734-0.33
22.8=0.01
0.9140.07
22.79:40.04
100000088
DTL774
20.22:10.01
1.80000.06
20.36.10.00
0.8911.11
20.01 10002
0.8910.2
20.34
0.9710.115
DTT1
18.07-00001
0.69-+0.05
16.7+0..0
4:00
18.05++0.01
0.80++0.03
1288440.33
0.87+00004
RE33
133.76:t:1.72
0.97:40.02
8.72=0065
0.99-40.00
125 46
0.97=0.04
99 99.89-49.7
1.04+0.11
RE34
0.97:10.03
237.34.0.77
RE37
1.42+-0.00
0,80++0001
1.32+0..0
0.98++0.03
1.42-40.00
0.75+40.3
1.40=0.000
0.79+00.05
RE41
1008.75..6.3
0,92.10.03
950.45.17.32
0.81
969.43.16.44
0.93.00.33
1011.03
10.06
maintain a good balance between convergence (HV) and Pareto front coverage (A-spread), all diversity-promottnng mechanisms of SPREAD are important. The diversity gain observed with SPREAD(w/o diversity) on some problems shows that the stochasticity inherent DDPM sampling (injected in X (equation 9 contributes to the overall diversity of SPREAD. Ablation studies on additional hyperparameters of SPREAD, including Vt, the perturbation scaling factor p, and the number of blocks L, are provided in Appendix
5.2 OFFLINE MOO SETTING
In the offline setting. we conduct our evaluation using Off- MOO-Bench (Xue et al. 2024). a unified collection of offline multi-objective optimization benchmarks. Each task is asso- ciated with dataset D and an evaluation oracle F. During optimization, F remains inaccessible and is only used to compute the hypervolume of the final solutions. The base- lines comprise DNN-based approaches that employ either Multiple Models (MM) or Multi-Head Models (MH), in con- junction with gradient-bases algorithms (GradNorm (Chen)
Table 4: Offfine MOO. Average rank results (4) per task group. Within each group, the overall best method is shown in bold, and the best gen- erative approach is highlighted in light gray
Method
Synthetic
RE
D(best)
9.08
1188
et al. 2018). and PcGrad Yuu et al. (020)) or model-based op-
MM
5.92
3.92
timization methods (COM (Trabucco al. 202T), IOM (Qi et al. 2022), ICT (Yuan et al. (2023), RoMA Yu et al.) 2021). and TriMentoring (Chen
MM-COM
8.00
7.42
MM-IOM
5.67
4.33
2023) to refine candidate solutions. Additionally, we evaluate the ability of the evolu-
MM-ICT
6.50
3.83
MM-RoMA
6.08
7.75
tionary algorithms NSGA-III and MOEA/D to solve offline
MM- TriMentoring
8.17
4.58
MOO tasks in Appendix D (Tables 16 and 17] The most relevant baselines for our approach are the generative meth- ods ParetoFlow (Yuan et al. 2025b) and PGD-MOO (An- nadani et al. 2025). ParetoFlow utilizes flow-matching mod- preerence-guided diffusion technique. Each algorithm is run with five different random
MH
7.58
5.67
MH-PeGrad
5.75
6.92
MH- GradNorm
9.58
11.50
ParetoFlow
7.50
6.83
PGD-MOO
4.58
8.75
els. while PGD-MOO employs
SPREAD
3,50
1.83
seeds, producing 256 solutions per seed. We evaluate task groups, Synthetic and RE, with 12
8

---

Published as a conference paper at ICLR 2026
m=
m
Branin and Camin
2011
Pericillo Production
DILZ2
Ear Side Impact (R641)
3.10
02
30
2012
2013
DT125
D1L27
0.1
RD80
10-
(POTS
PS.. M020
08
CONI15L
02
0.7
04
(Ours)
Enilation
Era
Ovalatera
Endlation
Figure 3: Bayesian MOO. Log-hypervolume difference (LHD) over 20 post-initialization steps (totaling 100 function evaluations) on nine MOBO benchmarks: Branin and Currin, ZDTI, ZDT2, ZDT3, Penicillin Production, DTLZ2, DTLZ5, DTLZ7, and Car Side Impact (RE41).
problems in each. Following Xue al. (024), rank algorithms within each task group with respect to their hypervolumes, and use the resulting average rank (J) as our primary comparison metric. The average rank results are reported in Table while the individual hypervolume results are provided in Appendix (Tables and Here, "D(best)" denotes the dataset's 's non-dominated points, serving as simple baseline. Our method achieves the best average rank across both the synthetic (3.50) and real-world (1.83) task groups, and it outperforms the other generative approaches on most problems in terms of hypervolume (see Tables [4 and These results show that SPREAD effectively leverages static datasets to generate high-quality approximate Pareto fronts without any online queries, matching or even surpassing the performance of oftte-ff-the-rt offline multi obbti-objective optimization techniques.
5.3 BAYESIAN MOO SETTING
We compare our method against three groups of baselines in multi Bayesian optimization:
(1) Pareto learning- based methods (PSL-MOBO (Lin al., (022). SVH-PSL (Nguyen al.
2025)) (2) acquisition-based methods (PDBO (A (Ahmadianshalchi et al. 2024), qPOTS (Renganathan
Carlson 2023),, ) and (3) CDM-PSL L et al. 2025a), diffusion- based generative approach. We consider nine MOBO problems with 2 or 3 objectives, including the real-world RE41 problem (Car Side Impact), which has 4 objectives All methods were initialized with 100 solutions and then run for 20 iterations, selecting 5 new solutions per iteration, for total of 100 function evaluations. We repeat each experiment with 5 independent random seeds, and Figure 3 shows the mean and standard deviation of the nog-hhyervolume difference (LHD) across the 20 instaaiitiaiizoiionn iterations. LHD is computed at each iteration as the logarithm of the difference between the maximum reachable hy- pervolume and the obtained hypervolume (see equation 40 in Appendix AA6)) SPREAD delivers solid performance across the benchmark suite, achieving the lowest final values in most cases. It converges particularly rapidly on the 3-objective DTLZ2 and DTLZ5 problems and the 4-objective Car Side Impact problem. Notably, SPREAD consistently outperforms CDM-PSL, another diffusion- based generative method. This advantage stems from our novel conditioning strategy and our adaptive guidance mechanism, which steers samples more accurately toward the Pareto front, yielding stronger approximations overall. To assess the ability both generative methods to fully solve MOBO problems, we compare their performance without employing SBX to escape local optima (step 8, Algorithm in Appendix D (Figure 13 which demonstrates the superiority our method.
40
CONCLUSION
We introduced SPREAD, a diffusion-basse generative framework for multi-objective optimization that refines candidate solutions through adaptive, MGD-insping -eD-inspired guidance and repulsion mechanism. By integrating these components into a conditional diffusion process, SPREAD achieves both convergence toward Pareto optimality and broad coverage of the front. Experiments on

---

Published as a conference paper at ICLR 2026
synthetic and real-world tasks show that SPREAD consistently outperforms oftte-of-theartt baselines in terms of hypervolume, diversity, and scalability, particularly in offline and Bayesian settings. A promising direction for future work is the design of a proper constraint-hande handling mechanism to extend SPREAD to multi-objective optimization problems with constraints on the decision variables.
7 ETHICS STATEMENT
This work does not involve human subjects, sensitive personal data, or applications with direct societal risks. The experiments are conducted entirely on publicly available benchmark problems and synthetic test functions commonly used in the multi-objective optimization community. No new datasets are collected, and all code and experimental protocols are designed for scientific reproducibility. We believe our contributions align with the ICLR Code of Ethics and do not raise ethical concerns beyond standard research integrity.
REPRODUCIBILITY STATEMENT
We have made significant efforts to ensure reproducibility our results. All algorithmic details, hyper- parameter choices, and evaluation metrics described in the main paper (Sections 4 5) and the ap- pendix (Appendices C Complete proof's of theoretical results are provided in Appendix A For all baseline methods, rely publicly available implementations official repositories, referenced in Appendix [ To facilitate full reproducibility, we release our implementation and scripts for repro- ducing all experimental results https://github com safe autonomous systems/ mnc spresp
ACKNOWLEDGMENT
This project received funding from the German Federal Ministry of Education and Research (BMBF) through the AI junior research group "Multicriteria Machine Learning". All experiments were performed on the compute cluster of the Lamarr Institute for Machine Learning and Artificial Intelligence.
REFERENCES
Alaleh Ahmadianshalchi, Syrine Belakaria, and Janardhan Rao Doppa. Pareto front-diverse batch multi-objective bayesian optimization. In Proceedings of the AAAI Conference Artificial Intelligence, number 10, pp. 10784-10794, 2024.
Yashas Annadani, Syrine Belakaria, Stefano Ermon, Stefan Bauer, and Barbara E Engel- hardt. Preference- preference-guided diffusion for objeivobeective offline optimization. arXiv preprint arXiv:2503. 17299, 2025.
Larry Armijo. Minimization of functions having lipschitz continuous first partial derivatives. Pacific Journal of mathematics, (6(1):1-3, 1966.
Manuel Berkemeier and Sebastian Peitz. Derivative-free multiobjective trust region descent method using radial basis function surrogate models. Mathematical and Computational Applications, 26 (2):31, 2021.
Francesco Biscani and Dario Izzo. A parallel global multiobjective framework for optimization: pagmo. Journal of Open Source Software, 5(53):2338, 2020. doi: 10.21 10.21105/joss.02333. URL https: http:///oo.org11..1
joss. 02338
Blank and Deb. pymoo: Multi-objective optimization in python. IEEE Access, 8:89497-89509, 2020.
Marlon Alexander Braun, Pradyumn Kumar Shukla, and Hartmut Schmeck. Obtaining optimal pareto front approximations using scalarized preference information. In Proceedings of the 2015 Annual Conference Genetic and Evolutionary Computation, pp. 631-638, 2015.
10

---

Published as a conference paper at ICLR 2026
Martin Dietrich Buhmann. Radial basis functions. Acta numerica, 9:1-38, 2000.
Can Sam Chen, Christopher Beckham, Zixuan Liu, Xue Steve Liu, and Chris Pal. mertolel- for offline model-based optimization. Advances in Neural Information Processing Systems, 36: 76619-76636, 2023.
Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, pp. 794-803. PMLR, 2018.
George H Cheng, G Gary Wang, and Yeong-Maw Hwang. Multi-objective optimization for high- dimensional expensively constrained black-box problems. Journal of Mechanical Design, 143(11): 111704, 2021.
Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Multi-objective bayesian optimization over high-dimensional search spaces. In Uncertainty in Artificial Intelligence, pp. 507-517. PMLR, 2022.
Kalyanmoy Deb. Real-coded genetic algorithms with simulated binary crossover: Studies multimodal and multiohjective problems. Complex systems, 9:431-454 1995.
Kalyanmoy Deb. Multi-objective optimisation using evolutionary algorithms: an introduction. In Multi-objeccive evolutionary optimisation for product design and manufacturing, pp. 3-34. Springer, 2011.
Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. A fast and elitist multiob- jective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, 1(2):182-197, 2002a.
Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. Scalable multi-objective optimization test problems. In Proceedings of the 2002 congress on evolutionary computation. CEC'02 (Cat. No. 02TH8600). volume 1, pp. 825- 830. IEEE, 2002b.
Kalyanmoy Deb, Proteek Chandan Roy, and Rayan Hussein. Surrogate modeling approaches for multiobjective optimization: Methods, taxonomy, and results. Mathematical and Computational Applications, 26(1):5, 2020.
Timo M Deist, Monika Grewal, Frank JWM Dankers, Tanja Alderliesten, and Peter AN Bosman. Multi-objective learning predict pareto fronts using hypervolume maximization. arXiv preprint arXiv:2102.04523, 2021.
Jean-Antoine Désidéri. Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350(5-6):31333) 2012.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794 8794, 2021.
Gabriele Fichfelder. Adaptive scalarization methods in multiobjective optimization. Springer, 2008
Jörg Fliege and Benar Fux Svaiter. Steepest descent methods for multicriteria optimization. Mathe- matical methods of operations research, 51(3):479 494, 2000.
Unai Garciarena, Roberto Santana, and Alexander Mendiburu. Evolved gans for generating pareto set approximations. In Proceedings of the genetic and evolutionary computation conference, pp. 434 441, 2018.
Andreia P Guerreiro, Carlos M Fonseca, and Luis Paquete. The hypervolume indicator: Problems and algorithms. arXiv preprint arXiv:2005.0033 2020.
Maria Hartmann, Grégoire Danoy, and Pascal Bouvry. Multi-objective methods in federated learning: survey and taxonomy. arXiv preprint arXiv:2502.03008, 2025.
Jonathan Ho and Tim Salimans. Classifier- free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
11

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pp. 8867-8887. PMLR, 2022.
Sedjro Salomon Hotegni and Scbastian Peitz. Enhancing adversarial robustness through multi- objective representation learning. In Walter Senn, Marcello Sanguineti, Ausra Saudargiene, Igor V. Tetko, Alessandro E. Villa, Viktor Jirsa, and Yoshua Bengio (eds.), Artificial Neural Networks and Machine Learning - ICANN 2025, pp. 442-454, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-032-04558--2
Joshua Knowles. Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE transactions evolutionary computation, 10(1): 50-66, 2006.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprini WoXiv:20090096/1 2020.
Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black- box optimization. In International Conference Machine Learning, pp. 17842-17857. PMLR, 2023.
Bingdong Li, Zixiang Di, Yongfan Lu, Hong Qian, Feng Wang, Peng Yang, Ke Tang, and Aimin Zhou. Expensive multi-objective bayesian optimization based on diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 27063-27071, 2025a.
Jaaning Li, Sijia Xu, Jiaming Zheng, Guoqing Jiang, and Weichao Ding. Research on multi-objective evolutionary algorithms based on large-scale decision variable analysis. Applied Sciences, 14(22): 10309, 2024a.
Miqing Li, Shengxiang Yang, and Xiaohui Liu. Shift-based density estimation for pareto- based algorithms in many-objective optimization. IEEE Transactions on Evolutionary Computation, 18 (3):448-365, 2013.
Zhiyong Li, Mingfeng Huang. and Ziyi Wang. Surroatte-ssisted multi-objective optimization of interior permanent magnet synchronous motors with a limited sample size. Applied Sciences, 15 (8):4259, 2025b.
Zihao Li, Hui Yuan, Kaixuan Huang, Chengzhuo Ni, Yinyu Ye, Minshuo Chen, and Mengdi Wang. Diffusion model for data-driven black-box optimization. CoRR, 2024b.
Xi Lin, Zhiyuan Yang, Xiaoyuan Zhang, and Qingfu Zhang. Pareto set learning for expensive multi- objective optimization. Advances in neural information processing systems, 35:19231-19247, 2022.
Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang. Fei Liu, Zhenkun Wang, and Qingfu Zhang, Smooth tchebycheff scalarization for multi-objective optimization. In International Conference on Machine PMLR, 2024.
Learning, pp.
Xingchao Liu, Xin Tong, and Qiang Liu, Profiling pareto front with multi-objective stein variational gradient descent. Advances in neural information processing systems, 34: 34:14721-1473,, 2021.
Behnam Malakooti. Operations and production systems with multiple objectives. John Wiley & Sons, 2014.
Michael D McKay, Richard J Beckman, and William Conover. A comparison of three methods for selecting values of input variables in the analysis of output from computer code. Technometrics, 42(1):55-61, 2000.
Minh-Duc Nguyen, Phuong Mai Dinh, Quang-Huy Nguyen, Long P Hoang, and Dung D Le. Improving pareto set learning for expensive multi-objective optimization via stein variational hypereeworkss. In Proceedings of the AAAI Conference on Artificial Intelligence, number 18, pp. 19677-19685, 2025.
12

---

Published as a conference paper at ICLR 2026
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-88888888 8171. PMLR, 2021.
Biswajit Paria, Kirthevasan Kandasamy, and Barnabás Póczos. A flexible framework for multi- objective bayesian optimization using random scalarizations. In Uncertainty Artificial Intelli- gence, pp. 766-776. PMLR, 2020.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVF international conference on computer vision, pp- 4195-4205, 2023.
Sebastian Peitz and Michael Dellnitz. Gradient-based multiobjective optimization with uncertainties. In Y. Maldonado, L.. Trujillo, O. Schütze, A. Riccardi, and M. Vasile (eds.), NEO 2016, volume 731, pp. 159-182. Springer, 2018a.
Sebastian Peitz and Michael Dellnitz. A survey of recent trends in multiobjective optimal con- trol trol-surrogte models, feedback control and objective reduction. Mathematical and computational applications, 23(2):30, 2018b.
Sebastian Peitz and Sedjro Salomon Hotegni. Multi-objective deep learning: Taxonomy and survey of the state of the art. Machine Learning with Applications, pp. 100700, 2025.
Han Qi Yi Su, Aviral Kumar, and Sergey Levine. Data-driven offline decision-making via invariant representation learning. Advances Neural Information Processing Systems, 35:13226-13237, 2022.
Gade Pandu Rangaiah. Multi-objective optimization: techniques and applications in chemical engineering, volume 5, world scientific, 2016.
Ashwin Renganathan and Kade Carlson. qpots: Efficient batch multiobjective bayesian optimiza- tion via pareto optimal thompson sampling. arXiv preprint arXiv:2310.15788 2023.
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-oojectivve optimization. Advances in neural information processing systems, 31, 2018.
Ryoji Tanabe and Hisao Ishibuchi. An easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing, 89:106078, 2020.
Brandon Trabucco, Aviral Kumar, Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In International Conference on Machine Learning, pp. 10358-10368. PMLR, 2021.
Lian Wang, Rui Deng, Liang Zhang, Jianhua Qu, Hehua Wang, Lichui Zhang, Xing Zhao, Bing Xu, Xindong Lv, and Caspar Daniel Adenutsi. A novel surrogate-asisted multi-objective well control parameter optimization method based on selective ensembles. Processes, 12(10):2140, 2024.
Lyndon While and Lucas Bradstreet. Applying the wfg algorithm to calculate incremental hypervol- umes. In 2012 IEEE Congress on Evolutionary Computation, pp. 1-8. IEEE, 2012.
Dongxia Wu, Nikki Lijing Kuang, Ruijia Niu, Yian Ma, and Rose Yu. Diff-bbo: Diffusion-bass inverse modeling for black-box optimization. NeurIPS 2024 Workshop on Bayesian Decision- making and Uncertainty, 2024.
Ke Xue, Rong-Xi Tan, Xiaobin Huang, and Chao Qian. Offline multi-objective optimization. arXiv preprint arXiv:2406.0377 2024.
Donghaa Yang, Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. Diffsound: Discrete diffusion model for text-to-sound generation. IEEE/ACM Transactions Audio, Speech, and Language Processing, 31:1720-1733 2023.
Sihyun Yu, Sungsoo Ahn, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for offline model-based optimization. Advances in Neural Information Processing Systems, 34:4619-4p5 1631, 2021.
13

---

Published as a conference paper at ICLR 2026
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in neural information processing systems, 33: 5824-5836, 2020.
Ye Yuan, Can Sam Chen, Zixuan Liu, Willie Neiswanger, and Xue Steve Liu. Importance-aware co-teaching for offline model-based optimization. Advances in Neural Information Processing Systems, 36:55718-55733, 2023.
Ye Yuan, Can Chen, Christopher Pal, and Xue Liu. Paretoflow: Guided flows in multi-objective optimization. In The Thirleenth International Conference on Learning Representations, 2025a. URL
net doormmiiddmyyyBleee5
Ye Yuan, Can Chen, Christopher Pal, and Xue Liu. Paretoflow: Guided flows in multi-objective optimization. In The Thirteenth International Conference on Learning Representations, 2025b.
Xiaoyuan Zhang, Liang Zhao, Yingying Yu, Xi Lin, Yifan Chen, Han Zhao, and Qingfu Zhang. Libmoon: A gradient-based multiobjective optimization library in pytorch. Advances in Neural Information Processing Systems, 2024a.
Xiaoyuan Zhang, Xi Lin, and Qingfu Zhang. Pmgda: A preference- based multiple gradient descent algorithm. IEEE Transactions on Emerging Topics in Computational Intelligence, 2025.
Xingyi Zhang, Ran Cheng, Ye Tian, and Yaochu Jin. Evolutionary Large-Scale Multi-Objectivve Optimization and Applications. John Wiley & Sons, 2024b.
Aimin Zhou, Bo- Yang Qu, Hui Li, Shi-Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, and Qingfu Zhang. Multiobjective evolutionary algorithms: A survey of the state of the art. Swarm and evolutionary computation, 1(1):32-44 49, 2011.
Eckart Zitzler and Lothar Thiele. Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE transactions on Evolutionary Computation, 224)257-771, 2002.
Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary computation, 132):173-195, 2000.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
Additional Proofs and Details
15
Proof of Theorem!!
15
2
Proof of Theorem
16
A.3
A discussion v 0 in Theorem
17
A.4
Extended Architectural Details
18
A.5
Additional Methodological Details
19
A.6
Evaluation Metrics
20
SPREAD in the MOBO Setting
21
Implementation Details
22
Additional Results
23
Extended Related Work
31
A ADDITIONAL PROOFS AND DETAILS
A.1 PROOF OF THEOREM 1
We recall Theorem [
Theorem 1. E (0, oo)m, independent of X, and define the training label
(Objective Improvement) Let X C X' be training dataset with distribution Px. Let
C = F(X) +
(15)
For a conditioning value in the support of C, denote by Px|C=e the true conditional data distribution and by Qe c) the distribution produced by a conditional DDPM when sampling conditioned on c. Assume the sampler approximates the true conditional training distribution total-variation TV
distance by at most r E [0, 1):
TV c), Pxic=e) sup 10 c) -X)c=e(AA)
(16)
Fix any initialization Xx E X' and set c: = Cr = F(xr). ). If cr lies in the support of C, and we draw Xg Q+ CT), then:
F(x; < XT) > 1-7.
(17)
In other words, conditioning the reverse diffusion F(xT) yields, with probability least r, sample that dominates Xy.
Proof. We proceed in two steps.
1. Characterization of the true conditional training distribution Px P=i-=
Conditioning the training data on forces F(X) = E. Because each component of E is strictly positive, we have F F(X) C. Formally, letting Ae := fx X F(x) c), have
Px C=e 1.
(18)
This means that, if we sample point x from the training data distribution conditioned on a label that lies in the support of C, then the objective vector of that sample is almost surely strictly below e component-we wise (because among the data points that carry the training label c, essentially all of them have objective values strictly below c).
15

---

Published as a conference paper at ICLR 2026
2. Transfer guarantee from the true conditional training distribution to the learned sampler via TV.
The total ooal-variation assumption (equation 16] says that for every measurable set A,
Qo(A Px]==(A) s
(19)
Simply put, when we condition on the label c, the model's sampling probabilities are uniformly close to the true training data probabilities, within for every event we could ask about.
Applying equation 19 with A = -e and using equation 8 yields
Qo c) 2 Px(c=((Ae) - Qa (Ac c) 2 1-T.
(20)
(21)
So, if we draw Xo Qa(- c), then
((F(x)) < c) 2 1
(22)
Finally, assume we conditioned the sampler on с F (xr) Under the assumption "c Fx lies in the support of C", all the conditional probabilities above are well-defined, and equation 22 immediately implies FxF(xo) < F(xT)) 1 T. This completes the proof.
To empirically support the dominance claim in Theorem we provide a direct visualization of the initial points and their corresponding sampled points. Since dominance be visually verified in two objectives, we conduct this experiment on the bi-objective problems considered in the online setting. Figure presents the results for ZDTI, ZDT2, ZDT3, and RE21. No guidance mechanism is used so that the effect of the diffusion model alone can be clearly assessed. As shown in the plots, the sampled points consistently dominate their initializations, providing empirical validation of the theoretical improvement stated in Theorem
2DT1
ZOT2
2DT3
500
500 20t via 200 200 300 200 1800
Initial points
Sampled points
Figure 4: Empirical validation of the objective improvement guaranteed by Theorem 1. The figure shows the movement of random initial points (blue) toward the corresponding sampled points (red) on ZDTI, ZDT2, ZDT3 and RE21 in the online setting. Guidance is disabled so that only the effect of the diffusion model is visualized.
A.2 PROOF OF THEOREM 2
We recall Theorem 2
Theorem 2. Assume each objective function fy is continuously differentiable, and that 2/2 = 0 for all t {1....,T} T). Let, at reverse timestep
Gi = (V.g(x/),, hi)
bis = (VI,(x()),o))
16

---

Published as a conference paper at ICLR 2026
with 0r 0 for all 1...., n and j 1....,m. Define
ai big
p min J.b
0 0<p<l, if any b, < O,
Vi
<0
(23)
S. 0,
otherwise,
where p controls the magnitude of the scaling parameters 7/s and Ç denotes an arbitrary positive scalar. Then, -hj = - (hj + 7/6c)) serves as a common descent direction for all objectives at x
Proof. For each point x reverse timestep 6, define
oi (V]s(x)), hi). and big (VII(x)))
It suffices to prove that, for all j 1, (x(" hi) >
(24)
(V](())) (b) +(/8)) > 0
(25)
ag + 7ibs >
(26)
Since it 0, the main direction hē is well aligned with the MGD direction at x/", and thus inherits its descent property, i.e. ai, > 0 for all j = 1...., m
If bi.j > for all j = -,...... m, then any choice of 7/ = C (with ( > 0) works.
Otherwise, if bi.j < 0 for some fl....,m) m).
For indices j with b,.5 O, the inequality in (equation 26 is equivalent to
aijj
(27)
bi.j
For indices j with búj > O, the inequality in (equation 26 is satisfied with any o3 > 0. Therefore, valid choice is
0 < 95
min j.b,, <0
ai with 0 < 1
(28)
bi.j
which ensures that the inequality in (equation 26 is satisfied for all j = 1....,
A.3 A DISCUSSION ON 144 > IN THEOREM
As outlined in the main text, the assumption that we turn of the diversity criterion for the Pareto set by choosing 14 = 0 in Theorem is quite restrictive and limiting. It is possible to prove an alternative version of Theorem when dropping this assumption. Since this would lead to the requirement of a sample-wise online adaptation of the penalty parameter computational cost, we have decided to pursue this approach in practice. However, proof would follow along the arguments laid out next.
well as significant additional
First, we note that the optimization problem equation 13 the first term -/nn Σ (g) u²) can simply be decomposed into sample-wise formulation, where we try to align the descent direction u' as much with the multi-objective steepest descent direction gi. The coupling occurs only in the second term, where try to diversify the directions using the repulsion term I' from equation 12 Setting % = 0 thus simply leads to u = gi for all samples i. Setting P4 > 0 thus leads to a divergence between the two, and need to bound the maximum angle between u' and g) in order to ensure that u' remains a common descent direction for sample For simplicity, we are now going to drop the indices i and in the following.
17

---

Published as a conference paper at ICLR 2026
Next, following the arguments above, need to ensure that the angle between any individual gradient (x) and u remains sufficiently large. In Peitz& & Dellnitz (2018a), an additional a-posterion criterion was derived for the optimization problem (4), which the weights X; are derived that ultimately lead to the construction g(x) = ーーー A (x)(x... To bound the angle between descent direction g(x) and (x) from above (i.e., to make it strictly smaller than the otherwise allowed maximum 2 for non-increasing direction), the corresponding weight X7 has to be larger than
lower bound m,min that can be computed when knowing g(x) as well as all the (x)... As consequence, one obtains a bound of the form (lg(x)) Vfz (x)]22 <.
Our strategy is now to bound the second term in equation 13 from above such that the difference between (x) and u is smaller than the smallest difference between g(x) and V (x),
(x)( ul S max JE JE(.....m)
[l(())
(x)(x)] < <g--
We observe that the maximum value for re in equation 12 is one in the case where all samples coincide. We thus have F E [o, 1}, and consequently Te) [0) Since the optimization problem (1) trades between the deviation of from g (x) and the spreading, we find that the inner product (x)), u) (and thus the angle) is also bounded.
In summary, appropriate choice of leads to maximum deviation between and g(x). By making sure that this deviation smaller than the minimal angle between g(x) and V ʃ; (x), descent can be guaranteed. However, in practice this would require to perform costly calculation and sample-wise adaptation of 1/, which proves to be impractical. Moreover, we find that allowing a temporary increase in favor of a better diversity is in the end beneficial for the Pareto set coverage.
A.4 EXTENDED ARCHITECTURAL DETAILS
Figure 1 shows our noise-prediction network DiT-MOO, conditioned on the objective values via multi-head cross-attention module (MH Cross-Attentions MHCA). The input projection, time embedding, and condition embedding are implemented as linear layers with hidden dimension while the output projection is linear layer with input dimension e. At timestep let Z, E =x*xI Jlayernoormalied features from X,) and B, E Rax2xe (the concatenation of the condition and time embeddings) be the inputs to MHCA. With attention heads, where each head has dimension d₂ = d, e/h, the MHCA module computes, for each head E (1....,4]:
Q Z, w (
Q E Raxixd,
K B, w Rnx2xds
v B Rux2xd,
where w Q
E gexd, w K E gexd, and w e Redd are learnable projection
The attention weights are
Q [K(K())
A ( softmax
(29)
Vdz
and the head output is Finally, the head outputs are concatenated and projected:
000 = A (1) ( E xxxxxd
(30)
MHCA(Z,, B,) (concat 00) Wo E gaxixe
(31)
where Wo E Rexe is learnable projection matrix.
For a fixed number of blocks L. the number of parameters in DiT-MOO depends on the input dimension d and the objective space dimension m of the considered multi-objective optimization problem. In all our experiments with SPREAD, we set I 3, which yields total of approximately 800k parameters (ranging from 797, 571 to 804, 894).
'eere, we use the convention that a tensor in X represents matrices of size b, and matrix multiplications carried in parallel across the batch dimension.
18

---

Published as a conference paper at ICLR 2026
A.5 ADDITIONAL METHODOLOGICAL DETAILS
Latin Hypercube Sampling (LHS) Latin hypercube sampling is a stratified sampling technique for generating well-distributed initial points in Rd (McKay al. 2000). Given sample size N, the range of each decision variable 2, (j L..... d) is partitioned into N disjoint intervals of equal probability under the uniform distribution. One value is drawn uniformly random from each interval, yielding N candidate values per dimension. The values across dimensions are then randomly permuted and paired, so that each sample x , x²4)
X C IR contains exactly one value from each interval of every variable. This makes LHS particularly useful for covering the decision space uniformly with relatively few samples.
Cosine Variance Schedule The cosine variance schedule (Nichol & Dhariwal 2021 is technique for defining the forward diffusion noise schedule in smooth, non-linear fashion. Instead of linearly increasing the variance 3, over timesteps .,... T, the cumulative product of the noise-retention coefficients, ā₂ ITT'_1 3e), is parameterized using shifted cosine function:
cos?
1/T+s 1+s
t= t
(32)
cos2
1+s
where s 2 0 a small offset to avoid singularities near O. The corresponding variances 3r are then recovered from ã₁ Compared to linear schedules, the cosine schedule allocates more steps to low-noise regions, resulting in improved sample quality and training stability in practice.
Armijo Backtracking Line Search At each timestep i of the sampling process in SPREAD, the step size Ty is determined using the Armijo backtracking line search (Armijo) 1966). Given a search direction h, (X(), we start from an initial step size = no and iteratively reduce it by a factor b E (0, 1) until the Armijo condition is satisfied:
F(X: - ,,,((;))) < F(X)) - an VF(X)) (x,)).
(33)
where a E (0, 1) is a fixed parameter. This condition ensures a sufficient decrease the objective function while avoiding overly aggressive steps. We set 10-4 and b 0.9 in our experiments.
Crowding Distance The crowding distance (Deb et al. 2002a) is density estimator widely used in evolutionary multi-objective optimization to preserve diversity along the Pareto front. Given non-dominated set S = [xl ..... x") }. the crowding distance of solution x* E Rd is computed by summing the normalized objective-wist wise distances to its immediate neighbors. For each objective j E {l,..., m). sort the solutions by fj. and assign infinite distance to the boundary solutions. For interior points, the contribution objective j
(x +1 ff fx
max f, (x*) min& (xt) df
(34)
The overall crowding distance of x is then
CD(x') = Σ
(35)
with CD(x)) = too for boundary points. Larger crowding distances indicate that a solution lies in a less crowded region of the objective space, making it more likely to be selected.
Computational and Memory Complexity Analysis Let K be the number of gradient steps used to solve the sub-problem in equation [3 In each reverse step of Algorithm SPREAD performs four main operations: performing DIT-MOO demoising pass, computing the multi-gradient descent directions, solving the diversity denoising update uses single forward pass through DiT-MOO, which processes each sample independently and costs O(n(d + m)). Computing the MGD directions requires evaluating all objective gradients Vfz yielding cost of (nmdd) Solving the sub-problem in equation 13 for
sub-problem. and applying the guidance update. The
19

---

Published as a conference paper at ICLR 2026
K' gradient steps requires evaluating the objectives for all points, ((mmd), together with computing the pairwise RBF-basee based repulsion term in objective space, O(n2m). Thus the full sub-problem costs O(K(nmd + n n2m)... Finally, the guidance update involves only vector projections and costs O(nd).
Summing these contributions and keeping only the dominant terms, one reverse-diffusion step has complexity
(wmd K (nmd + n n²m),
(36)
and running all T' steps yields the total sampling complexity
O (Tumd + (eeTKK(ng (nmd n²m)).
(37)
which is dominated by the O(T term when objective evaluations are inexpensive and is large.
The memory usage during sampling determined by storing the n points (O(nd)), their objective values (mmm) the pairwise distances required for the repulsion term (O(n²m)), and the DiT-MOO activations (O(neL)), in addition to the model parameters. Therefore, the total memory scales as
(n(( + m) + n'm + neL + 101).
(38)
A.6 EVALUATION METRICS
Hypervolume (HV) The hypervolume indicator measures the portion of objective space that is weakly dominated by the approximated Pareto front with respect to a fixed reference point (Zitzler & Thiele (202).
Formally, for a set of non-dominated solutions P x² Idd, the hypervolume is defined
{x' Jime with
HV(P) = (f Rd | A II [I(())) 1) (
(39)
where E IRmh is reference point dominated by all Pareto optimal solutions, and A--) is the Lebesgue measure (see Figure 5) We report Appendix [ the reference points used in our experiments. The HV is maximized when the solution set covers the Pareto front broadly and accurately, making it widely used indicator for comparing multi- objective optimization methods.
Figure 5: Hypervolume
bi-objective example, cor- responding to the shaded region defined by the ob- tained solutions (red) and the reference point (blue).
Log Hypervolume Difference (LHD) The log hypervolume difference is a commonly used indicator assess the convergence of multi-objective Bayesian optimization methods. Let HV* denote the maximum reachable hypervolume (i.e., the hypervolume of the true Pareto front). At iteration f, let Pe be the approximated Pareto front obtained by the algorithm. The LHD then defined as:
LHD, = log(HV" - HV(P:)).
(40)
A lower LHD value indicates that the current solution set is closer to the optimal hypervolume.
A-spread The A-spread (Deb et al., 2002a) evaluates the diversity an approximated Pareto front by comparing the spacing between consecutive solutions to the average spacing, while also accounting for the coverage the true extreme points. Let y = [F )..... F(x")) denote the F(x')]l as the Euclidean distance between consecutive solutions, a-1 Σn=1 d; as their mean, and dy, di as the distances from the extreme solutions in Y the true Pareto front endpoints (if available, otherwise set to 0). The A-spread is given by:
non-dominated solutions, sorted along chosen objective. Define d, [
A-spread dy + di
d, d
(41)
dj + di 1)d
lower value indicates a more uniform spread of solutions along the Pareto front. By convention, A A-spread +oo if the solution collapses to single point.
20

---

Published as a conference paper at ICLR 2026
B SPREAD IN THE MOBO SETTING
In a MOBO framework, the true objectives are expensive to evaluate, so surrogate models (e.g.. Gaussian processes) are trained on an initial dataset of evaluated solutions. At each iteration, the surrogate models are updated with newly evaluated solutions, and search strategy proposes new candidate solutions to evaluate (Paria et al. 2020; Lin et al., 2022). This iterative cycle of modeling, proposing, and evaluating continues until the evaluation budget is exhausted, yielding an approximation of the Pareto front. To adapt our method to this setting, we use SPREAD as the search strategy for proposing new candidate solutions. The full procedure is given in Algorithm [ Following (Li et al. 2025a), we employ simulated binary crossover (SBX) as an auxiliary operator to escape local minima, i.e. when no improvement is observed for fixed number of iterations. Batch selection is then performed using the hypervolume metric, as in (Lin et al. 2022). To increase the number of training samples per iteration k, we adopt the data augmentation strategy of (Li 2025a). Specifically, data points are first extracted from ( using shift-based density estimation (Li et alll, 2013). Three transformations are then applied: small random perturbations. interpolation of randomly chosen pairs, and Gaussian noise injection. The augmented samples are shuffled, truncated to match the target augmentation factor, and merged with the extracted points to form the enhanced dataset used for training DiT-MOO.
Simulated Binary Crossover (SBX) Simulated Binary Crossover (SBX) (Deb 1995
real- parameter recombination operator used to generate two offspring from two parents. Given parent vectors P1.P2 E (*) C Rd and distribution index parameter x > 0, SBX first samples a random vector u E [0, 1]S, then computes a spread factor Tj for each j 1, ..... d as
(2ug) mtl, (1))
S 0.5,
T
(42)
U > 0.5. xtI
The two offspring are then set as
offspringl, = (15((11 + T2) offspring2, 0.5 + (1 + Ty) P2.j).
(43)
higher x makes offspring closer the parents (less exploratory), while lower x allows more distant (diverse) offspring. This operation is repeated 1000 times, and the resulting points are passed to the batch selection step (Step 10 in Algorithm our experiments, we use x = 15.
Algorithm 3 MOBO with SPREAD
DIT-MOO the noise prediction network (a(:), black-box oujti-objective function F(-) defined on X. Parameter: initial sample size Yinis, number of iterations K, batch size b, boolean flag es escape (initialized to False).
Input:
Output: approximate Pareto optimal points.
1: Get the initial solutions (x) Ti=l"
(0) via LHS, and evaluate ( F(X
2: for to K do
3: Train Gaussian-Proceet surrogates [ using (x(e) (R), y(O)).
4: X (A)
Ataiin + Get training data for DIT-MOO using (x (k) (k) as in CDM-PSL Li et al. 2025a
(Appendix [B]
5: if ieeccaee Fal then
S + Generate offspring with SPREAD via Algorithm 4p.
7: else
8:
S Apply SBX to escape local-optima (Appendix B
9:
end if
10:
(() +- Batch selection: select the top-b solutions from based on their hypervolume contributions.
11:
((+1) (()) U F(X()) new)
12: ((+1)) ( U ((k
13: Decide whether to invert escape based on the latest hypervolume values.
14:
end for
Return: X
21

---

Published as a conference paper at ICLR 2026
Algorithm 4 Offspring generation with SPREAD (MOBO setting)
Input: DiT-MOO the noise prediction network Eg()). black-hox multi-ohjective function F(-) defined y== (gp())=
X. training dataset X Gaussian- Proccsss surrogates (sp
GP.
Parameter: epochs E, timesteps T, number of generation Ngen. required offspring size n.
Output: offspring $ generated by SPREAD.
1: Train Eo(-) for E epochs on ( gt
trin via Algorithm Jusing GP instead of F.
2: S
for to Ngen
Initialize random points Xy = [xx]== C
5: Pr f- Xx
6:
for = T to do
7:
(gi)-=1 Get the MGD directions via Section
3.3
using GP instead of F.
8:
(hi) +
Get the main directions via equation
13
using GP instead of F.
(hi) Get guidance directions via equation
10
10:
X,-1 Get the denoised points via equation
11:
Pt-1
Use crowding distance (Appendix
A.S
to the top-n non-dominated points
from X PP.
end for
13:
$ 5- S (Pa}
14: end for
Return: S
C IMPLEMENTATION DETAILS
In this section, provide further details about the experimental settings. For each training batch X = (xt) set = Im where 1mm denotes the m-dimennsioall vector of ones. The value of w is determined as
min (F(*t) xf E X, F F(x*) 0}, if this set is nonempty,
10
otherwise.
In all experiments, fix the number of DIT blocks L == 3. DIT-MOO is trained for maximum of 1000 epochs with early stopping after 100 epochs, except in the Bayesian MOO setting, where maximum of 250 epochs used. The number of solutions produced by all methods is 200 in the main experiments and 256 in the offline setting. For the Bayesian MOO setting, solutions are selected at each of the 20 steps. We consider T 5000 timesteps for the main experiments, and 1000 and 25 timesteps for the offline and Bayesian settings, respectively. At each timestep, we solve for the main directions (hr) using gradient descent with 10 iterations and a fixed V; = In the repulsion function, the length scale is set adaptively from the pairwise squared distances as
median [[ F, 12,12 1
i.j 4))
20 10
(44)
log n
where F, E RD are the objective vectors in the batch of samples and is the number of samples In all experiments, we use fixed values of determined solely by the number of objectives and, a few cases, by the specific problem instance:
Online setting:
- m = 2: 0.9
m >>2 2: 0.001
Offline setting:
m p:p= = 0.9, except for ZDT4(p = 0.0001), and ZDT6(p = 0.1).
m 2: p= 0.001, except for DTLZ4, and DTLZ6: = 0.0001.
Bayesian setting:
2: p 0.9
II implementation, follow PyTorch workaround from Liu et al. 2021 simulate NumPy's medi
22

---

Published as a conference paper at ICLR 2026
m > 2: p 0.01
We determined the default P values from few preliminary checks and used them unchanged throughout our experiments. We did not conduct any extensive hyperparameter search. While Figure I0 shows that SPREAD's performance varies across wide range of p values, the relative ordering with respect to the baselines remains stable: even with non-optimal choices of p. SPREAD does not exhibit any drastic performance degradation relative the competing methods. For new problems, we recommend using the default values, which depend only on the number of objectives. In the main experiments, we use the implementations provided by the PyTorch library LibMOON (Zhang et al. 2024a for HVGrad, PMGDA, and STH. For MOO-SVGI we rely the authors' official code In the offline setting, implementation and evaluation protocols follow Off-MOO-Bende (Xue et al. 2.24), and adopt baseline results from Annadani et al. (2025). Five independent seeds (1000, 2000, ..... 5000) are used, and we report mean and standard deviation across runs. The same seeds are used across all experiments in all settings, and for ablation studies where mean and standard deviation are not reported, we fix the seed to 1000. In the Bayesian MOO setting, we use the publicly available codes provided by the respective authors. We report in Tables 5 6 7 and detaile information about the synthetic and real-world problems considered across the different settings. The experiments were run on single NVIDIA A A100-SXM4440GB GPU. To facilitate reproducibility, our code is available at the following anonymous repository:
https: / annnymouuuu... 4open. science/ r/ SPREAD-RRED
Table 5: Benchmark problems. Problem settings and reference points in the main experiments.
Name
d
Type
Pareto Front Shape
Reference Point for Hypervolume Computation
ZDTI
30
2
Continaous
Convex
(0.9994, 6.0576)
2DT2
2
Contimaous
Concave
(0.9994, 6.8960)
ZOT3
30
2
Continsous
Disconnated
(0.9994, 6.0571)
Continuous
(2,8390, 2, 2.8575)
D1TLZ4
30
3
Continsous
Concave
(3.2675, 2.6443, 2.4263)
DTLZ7
30
Continaous
Disconneted
(0.9984, 22.8144)
RE21 (Four design)
2
Continaous
Convex
(3144,44, 0.05)
RE33 (Dise brake design)
3
Continaous
Unknown
(5.01, 9.84,
RF34 (Vehicle crashwortiness design)
Continaous
1.8642022ee43, L181192944-01, 2.9019933e-001)
RE37 (Recket injoctor desiga)
3
Continaous
Unknown
(1,1022, 1,20726899, 1.20318656)
RF41 (Car side impact design)
4
Continaous
Unknown
(47 (47.04480682, 4,86993366, 14.40091177, 10.3941957)
Table 6: Offline MOO benchmarks: task properties.
Task Name
Dataset size
Dimensions
# Objectives
Search space
Synthetic Function
60000
7-30
Continuous Continuous, Integer & Mixed
Real-world Application
60000
3-7
2-6
D ADDITIONAL RESULTS
MGD+RBF Baseline and Guidance Ablation Experiments We present experiments using the straightforward MGD+RBF baseline, well ablation of SPREAD without any guidance mechanism. For MGD+RBF, we apply the second update of equation [oo T iterations (using the
same T as in the main experiments) refine random initializations, without using any diffusion model. For the SPREAD ablation without guidance, we discard the second update of equation 9 and rely solely on the reverse-diffusion update (the first update of equation Dp. As shown in Table 9
SPREAD consistently dominates both baselines in the online setting. In the offline setting (Table 10
SPREAD(w/o guidance) achieves the best performance on many DTLZ problems, while SPREAD surpasses both baselines on most ZDT and RE problems; MGD+RBF, however, shows strong performance on the RE61 task. In the MOBO setting, Figure (6 demonstrates that SPREAD and SPREAD(W/o guidance) both outperform MGD+RBF on most tasks, with SPREAD performing best overall. These results indicate that both diffusion and guidance are essential components contributing to the strong performance of SPREAD.
'https: wttptth.ccc///chag55555221 'https: / /i thub. wwt tat/uuuttaaaaaallaaa t objjecti veSamp1
ibmon
23

---

Published as a conference paper at ICLR 2026
Table 7: Offline MOO benchmarks: problem settings and reference points.
Nume
Pareto Shape
Hypersolume Computation
Continouss
8.55)
23T2
Contisuous
Coscave
(1.10, 9.59)
20713
Continuous
Disconncted
(1.10, 8.74)
Contisuons
300
2DT6
Continuous
Concave
(1,07, 10.27)
DTT21
7
Contisuon:
Linear
(558.21, 552.30, 568.36)
DTL22
Continoous
Concave
(2.77. 2.78,
Continoous
Concave
(1783
167045)
DILZ4
10
Continous
Concave
(3.03, 2,83, 2.75)
DTL25
Contisuous
Concave (24)
(2.65, 2.61, 2,70)
DTL.26
Continoous
D7
Continonu
Disoemetted
(1.10, 1,10, 71,4)
RE:11 (Four bes truss design)
4
Continoous
Convex
(3144.44, 0.05)
RE22 (Reinforced concrete boom
Mixed
Mixed
(829006, 2407217.25)
(Coll compresiion spring design)
Mised
Mited, Disconsectad
RE:11 (Two bas inus design)
3
3
Continouus
Unkaowa
(MU8355 6993575 6793450.000
RF32 (Wolded hesm design)
4
Continuos
Unknows
(290.66, 16552.46, 368265024000
(Disc design)
Continoous
Unknowa
RE35 (Speed roducer design)
7
Mixed
Unknows
(7050, 1696.77 397.83)
RE26 (Gear train design)
4
3
Integer
Concave, Disconnected
(10.21, 60.00, 0.97)
RE:37 (Rocket usjector
4
Contiauous
Unkoowa
RE:1 side impact desigs)
Conminous
Unksowa
(42.65, 9.43, 13,08, 13:45)
RF42 (Concegtual marine desiign)
6
Contisuon
Unkwows
(-26. 19, 19904 90, 28546 79, 14,98)
RE61 (Water resource platsing)
3
Contisuous
Unkoowa
(83050:33, 1350,00, 2653469.06 161277666 6E, 357719.74, 99660.36
Table 8: Bayesian MOO benchmarks: problem settings and reference points.
Name
d
m
Type
Pureto Front Shape
Reference Point for Hypervolume Computation
ZDTI
20
Continuous
Convex
(0.9994, 6.0576)
ZDT2
20
Continuous
Concave
(0,9994, 6.8960)
ZDT3
20
Continuous
Disconnected
(0.9994, 6,0571)
DTLZ2
20
Continuous
Concave
(2.8390, 2,901 2.8575)
DTLZS
Continuous
Concave
(2,6672, 2,8009, 2.8575)
DTL27
Continuous
Disconnected
(0,9984, 0,9961, 0.9961,22.81144) 22.8114)
Branin and Currin
2
Continunus
Convex
(18.0, 6.0)
Penicillin Production
7
Continuous
Unknown
(1.8500, 86.9300, 514.7000)
Car Side Impact (RE41)
7
Continuous
Unknown
(45.4872, 445 13.3394. 10.3942)
Table 9: (Online) MGD+RBF baseline and ablation study without guidance. The random seed is fixed to 1000, and the best results are highlighted in bold.
SPREAD
MGD+RBF
SPREAD (wio guidance)
Problem
HV
A-ppredd
HV
A-spread
HV
A-spread
ZDTI
5.72
0.32
5.06
1.00
4.08
0.84
7DT2
6,22
0.30
5.89
1.00
4.00
0.85
ZDT3
6.10
0.53
5.06
1.00
4.18
0.86
RE21
70.11
0.41
32.84
1.00
70.05
0.45
DTLZ2
22.92
0.92
0.00
1.63
22.25
0.94
DTLZ4
20.22
18.84
1.14
17.88
1.37
DTLZ77
18.08
0.61
16.72
1.00
11.88
0.74
RE33
135.26
0.97
119.22
1.11
0.00
1.00
RE34
242.69
0.93
210.07
1.14
237.07
0.78
RE37
1.42
0.81
0.23
1.70
1.37
0.87
RE41
1005.08
0.88
382.80
1.28
860.07
0.94
Table 10: (Offline) MGD+RBF baseline and ablation study without guidance. The random seed is fixed to 1000, and the best results are highlighted in bold.
Mothod
ZDTI
ZDT2
ZDT3
ZD14
ZDT6
DTL21
DTLZ2
DTL.23
DTLZA
DTLZS
DTLZ6
DTL27
SPREAD
4.1
6.52
5.75
4.78
11.72
13.21
10.21
10.85
8.56
11.35
MGD+RBF
0.00
4.64
4.97
5.27
4.39
10.17
12.83
10.26
35,38
10.79
10.92
11.23
SPREAD (w/ guidance)
468
6.16
5.48
3.30
3.86
11.95
13.57
10.61
35.38
11.98
9.37
11.04
Method
RF21
RF22
RF25
RF31
RF32
RF33
RE35
RF36
RE37
RF41
RF42
RF61
SPREAD
4,83
5.18
5.33
11.87
11.34
13,50
10.82
10.03
8.39
23.16
22.87
201.43
MGD+RBF
3.01
4.84
4.89
11.44
11.38
8.87
10.53
6.54
6.78
13.56
19.21
2004.69
SPREAD (w/o guidance)
4.79
5.14
5.24
11.87
11.17
12.72
10.85
10.13
8.51
21.34
22.40
314.52
24

---

Published as a conference paper at ICLR 2026
m=
m83
Branin and Curtin
23311
Pericillio Production
D1L22
Side Impact (NE41)
5.00
9.4
24
6.48
23-
2013
DTL25
D1L22
0.-
0.6
100 R187
(Wh dridar))
02
(0UG)
Exdlutiora
Deluttioa
Ovelattre
Figure 6: (Bayesian) MGD+RBF baseline and ablation study without guidance. The random seed is fixed to 1000.
Approximate Pareto Fronts To provide complete view of the results in Tables and Figure illustrates the approximate Pareto optimal points obtained by the different methods on four synthetic and four real-world problems. SPREAD provides broader coverage of the Pareto fronts, particularly on the real- world problems.
Runtime Scaling with Decision-Space Dimensionality. To analyze the runtime scaling with decision space dimensionality, we perform an ablation study on DTT.74 the online setting for (d 10, 20, 30, 40). As shown in Figure 8 the computational cost of SPREAD remains lower than that of PMGDA and higher than that of the other baselines. However, SPREAD delivers better hypervolume and A-sprea performance than all baselines the number of decision variables increases.
Ablation Study it To assess the impact of the parameter is (equation 13) of SPREAD, we conduct an ablation study with values ranging from 0 to 100. As shown in Figure 9 setting Vt provides poor trade-off between convergence and diversity. In the bi-objective ZDT2 problem, lower positive value (P; = 0.5) yields both better convergence and good diversity, while the 4-objective RE41 problem, higher value (D, 50) achieves a more favorable balance. For the 3-objective DTLZ4 problem, smaller values of 14 improve diversity at the expense of convergence, whereas larger values reduce diversity but improve hypervolume. Overall, v; a key parameter for balancing convergence and diversity in SPREAD, with moderate positive values typically yielding the best performance. our experiments, we set 1/2 = 10
the performance
The adaptive perturbation added to the main direction in equation 14 scaled by < < 1, which controls its magnitude. We evaluate its effect on SPREAD's performance in Figure 10 The results suggest that when the number of objectives is small (e.g.. = 2). lower values of yield good performance, whereas problems with more objectives benefit from moderate or larger values of p.
Ablation Study
Ablation Study on the Number of Blocks L Figure II shows the performance of SPREAD as the number of blocks increases on ZDT2 (m 2), DTLZA (m = 3), and RE41 (m 4). The results indicate that larger number of blocks does not necessarily improve performance, even on problems with more objectives, compared moderate values such L 2.
Effect of the Number of Main-Direction Optimization Steps. We performed an ablation on the number of main direction optimization steps (5, 10, 50) on ZDT2, DTLZA, and RE41. The results are shown in Figure I2] The runtime plot shows that increasing the number of inner steps increases the cost linearly. but the cost remains modest compared to a full DDPM denoising pass. Regarding performance, the hypervolume and A --speedd curves do not show a consistent monotonic trend across problems: on RE41, performance improves with more steps, while on DTLZA, the best values appear smaller step counts. Importantly, all curves remain within comparable range across the tested step
25

---

Published as a conference paper at ICLR 2026
1V5
.70
PMC
SPREAD
CATT
04
04
04
120
610
0.20
10-
610
5AREAD
10/
1/0
7(51
(12)
1.30
SPREAD
0.00
1.00
Vi0 5y00
186m
PHIG2
SFREAD
3.955
3.01
0.01
0030
6007
000
A000
1000
0005
20.5
2500
05.5
288
2500
051
285
2500
12:
240
280
2593
Ser
Figure 7: Approximate Pareto optimal points for multiple benchmark problems. Solutions from 5 independent runs are merged, and the non-dominated points are shown.
counts. These observations suggest that using a small fixed number of steps (c.g., 5 25) is practical and that the method does not appear particularly sensitive full convergence of the inner update.
26

---

Published as a conference paper at ICLR 2026
125
20
Time. (minutes)
100
MOO-SVGD
d.spread
13
HV
10
HVGrad
50
1.2
STCH
16
PMGDA
22
SPREAD (Ours)
10
10
20
30
40
10
20
30
40
(a)
(b)
(c)
Figure 8: Computational time, hypervolume, and A A-spread, respectively, the decision-space dimensionality increases (DTLZA).
ZDT2
DTLZ4
RE41
6.20
20.2
1850
6.15
20.25
950
6.10
19.
20.20
6.03
19.7
940
1'
Vt
V2
Vt
1.2
4-Spread
3.0
0.A
spread
L-spread
1.4
0.35
9.85
0.90
0.3
0.80
2.6
0.75
0:85
2.4
0.70
3.0
0.65
0.0
53
1.0
10.0
00.00.0
0.
0.50
05
50
15.0
50.0100
0.0
0.55 01
1.0
60
10.0
56.100.
Vt
Ve
Figure 9: Ablation study vs. evaluated ZDT2 (m 2), DTLZ7 (m
3), and RE41 (m 4).
M
RE41
222
1060
M
20.225
3,000000
M
20.00
1010
3.00575
20.75
1920
20350
20.25
1020
3.000255
20.000
910
500
69
0,000
0.03
02
63
04
6.5
0.
0.
24
0.0
0.56
A-spread
4-spread
D.spread
5.50
3.10
0.9
077
6.75
0.001
0.003
01
22
0.5
57
58
0,2002
0.00
0.01
53
27
0
Figure 10: Ablation study on the perturbation scaling factor p, evaluated on ZDT2 (m = 2), DTLZ4
(m 3), and RE41 (m 4).
Ablation with explicit shift conditioning. In Table 11 report the ablation results for explicit shift conditioning in the online setting. For each training sample x the condition is defined as (F) instead of the implicit form F(x*) =. This requires embedding Fxx')
27

---

Published as a conference paper at ICLR 2026
16.222
RE41
1010
0.00060
20.23
HV
0.00055
20.21
0.00050
3
0.310
0.95
0.94
4-spread
0.305
4-spread
0.90
4-spread
0.92
0.300
0.85
0.295
0.80
0.290
0.75
Figure 11: Ablation study the number of blocks L {1,2,.... .,,,,................ 5) in DIT-MOO, evaluated
ZDT2 (m = 2), DTLZ4 (m = 3), and RE41 (m = 4).
ZDT2
DTLZ4
RE41
20.27
3.00000
20.26
3.00588
HV
20.23
2.00052
1000
20222
5.00550
950
25
30
10
50
10
25
50
Steps
Steps
Steps
1.2
0.915
-ppread
5.31
d-spread
1.1
ispread
0.905
3.0
0.045
0.3
5.05
Steps
Steps
Steps
Avg (seconds)
2.0
DTLZ4
RE41
15
1.
0.5
Steps
Figure 12: Ablation study on the number of optimization steps in equation 13, evaluated on ZDT2
(m 2), DTLZ4 (m 3), and RE41 (m 4).
and E separately and concatenating their embeddings with the time embedding, Consequently, the conditioning input to the multi-head cross-attention module has shape (n, 3, e), while the output retains the shape (n, 1,e) in Figure [] During sampling, set the condition (F)
28

---

Published as a conference paper at ICLR 2026
Overall, the results indicate that the implicit shift conditioning used in SPREAD slightly outperforms explicit shift conditioning in terms of hypervolume on most problems, with the difference being especially notable on the 4-objective RE41 problem. Explicit conditioning, however, improves solution diversity on the ZDT problems and achieves higher hypervolume on RE33 and RE34. Nevertheless, the theoretical improvement guarantee established for implicit shift conditioning in Theorem does not directly carry over to the explicit variant.
Table 11: Ablation study with explicit shift conditioning. The random seed fixed to 1000, and the best results are highlighted in bold.
SPREAD
SPREAD (explicit)
Problem
HV
A-spread
HV
A-spread
ZDTI
5.72
0.32
5.72
0.29
ZDT2
6.22
0.30
6.22
0.28
ZDT3
6.10
0.53
6.10
0,50
RE21
70.11
0.41
70.10
0,43
DTLZ2
22.92
0.92
22.91
0.94
DTL.74
20.22
0.76
20.22
0,76
DTL.77
18.08
0.61
18.07
0.69
RE33
135.26
0.97
135.29
1.03
RE34
242.69
0.93
242.91
0.84
RE37
1.42
0.81
1.42
0.73
RE41
1005.08
0.88
968.46
0.99
Training Cost and Feasibility of Transfer Learning Across Related Problems. We acknowledge that training a separate diffusion model for each problem introduces additional computational over- head. However, when multiple problems share the same decision space and number of objectives, transfer learning becomes feasible and can substantially reduce the training effort in both online and offline settings. In this experiment, perform transfer learning by training single diffusion model on groups related problems. Two groups are considered both the online and offline settings:
Online setting: (ZDT1-3) and (DTLZ2, DTLZA, DTLZ7)
Offline setting: (ZDT1-3) and (DTLZ2-7)
Tables and and 13 present the results. In the online setting, transfer learning achieves performance comparable to training individual models, with only one exception (DT1.72) where the hypervolume is slightly lower. In the offline setting, transfer learning even improves performance most problems. Overall, these experiments suggest that transfer learning is practical strategy for reducing training cost with minimal risk of performance degradation.
Table 12:
(Online) Transfer learning results. The random seed is fixed to 1000, and the best results highlighted in bold.
SPREAD
SPREAD (transfer)
Prohlem
A-spread
A-spread
ZDTI
5.72
0.32
5.72
0.32
ZDT2
6.22
0.30
6.22
0.29
ZDT3
6.10
0.53
6.10
0.53
DTLZ2
22.92
0.92
22.91
0.90
DTLZZ
20.22
0.76
20.22
0.79
DTL27
18.08
0.61
18.08
0.70
Table 13: (Offline) Transfer learning results. are highlighted in bold.
The random seed is fixed 1000, and the best results
Method
ZDT1
ZDT2
ZDT3
DTLZ2
DTLZ3
DTL.Z4
DTL.75
DTLZ6
DTLZ7
SPREAD
4.91
6.52
5.75
13.21
10.21
30.26
10.85
8.56
11.35
SPREAD (transfer)
6.52
5.85
13.17
10.22
31.09
10.85
29

---

Published as a conference paper at ICLR 2026
Table 14: (Offline) Hypervolume results of synthetic functions averaged over independent runs.
x:25
1711
10073
10:15
9:31
MM
010
100
01:31
5.05 1:0.19
0.7% D/l
10.54
4-0.0
5113
13:48 0163
3.66
1130
1-0-3
10:51
MM CTM
553. 0:47
119 0:13
5:68
m1.0
5:0 0.18
4:11
8.21
10.44
0.00
3:40 01.91
13.77
03:11
4.72 0..34
5:61 [111
0:11 0:35
10:08 - 0.36
M34-00
4.6. 8:112
545 0.31
5A1
0.06
4:5-2:0.21
475 = 001
10.54
0.00
10.10:100 0:22
10:34
94.33
1003 2-035
5.77 0..15
0.30-20.31
10.80 0.006
MML-UT
1:1
11.0
1:44:
BU.44 1:000
8:64 11:33
13:25
16:2
18:31 1:031
11:28
10.24 1.11
4:13 +::11
0:0
0.11
10,53 -00.4
0.00
0:30=035
10:43
1.00
0.00
MII
4.0
1071
11.00
4:391:00.3
4.78
H655 0.23
0101 0.35
13:08
1123
6.75
1.115
1:17
1109 0:00
MH al
2:1
1.55
0.)
5.5
0.00
3.68
4.67
0.10
10.54 0.01
9/63 0:33
13.55
0.13
9.95 1.93
5/02 0.34
9:00--9.25
10.51 : 0.0.0
Mil-CesNNom
0:11
337
3,5
0.20
3.23
9:000
331
1.20
1056
0:01
1:05
1.21
10:26
6624
7.48 0:75
1,87
1.26
2:164-2-21
11,31 0.22
5m
170
01
01
3101077
3.18
011
1030 0100
10:11
0110
10:11
111110117
0171
B118
PGD-MDO
0:08
005
0.10
1502 3:030
10.55
10.55 501
1800
088
10.4
10.00
00:
Table 15: (Offline) Hypervolume results of real-world tasks averaged over independent runs.
11:55
1075g
1:51
15:35
11:41
47.09
4.50 01.00
4,54 (13)
4,5 2:335
RLES 0.00
1002 1032
13 3310
1033 01:)
10:35 1003
6:15 11.0
20.1 = 00.05
12.30 6.11
IR0.95 2 1015
NBH-COM
4.30 2.,09
4,50 0.00
4.5 0.00
00.4-20..1
1064 600
6] 2:10
10.55 0.00
0.81 0.35
6.35
0.10
20.37 0.005
17:44
6.71
107.29 00.4
4.19 11.00
4.36 6001
433
H185 1.000
1005 (03)
10:2:1010 -310
01.57 1.:0
19:29
8/6
6.11
11.0
21.58 11:99
12.15
6.10
KBM
4.5000.0
4,6
4-..
00:440000
1064
0.55-=000
005
0.22
0.000000
21 -037
MM MMMTriMenniinggg
0.000
1,50 0.00
4...00000
0.000
1062 001
10:00-301
0.59
964 1.43
6.73-4
0..0
20.58 0:54
31:0 0.19
105.51 0.29
XBH
4.00 1..00
4,6 D00
4.4-2:00 2:00
1025 0:00
1000 101
10 2:10
13:25
545
0.67 0.00
2152:2113
22.38 0.35
1M6-22 0.22
NiH-Pslirad
4.59
4.75
0-16
1003
10:59
10.31 0.25
13:17 E/65
0.58 0.106
21.5 11.30
32.37 0.26
106.44
1.70
IGD-NDO
2.55
4,5
4.5
0.00
0..
1063
000
10:51
3.14
16.32
01.1
0.3120..3
6.13
10.31 0.44
1530
0.66
105.22 1.34
SPREAD
0.0 10.00
5:8
533
9.6
1131 1880
11.27
0.17
11:00
810
16.76 643
9.5 0031
033 0.66
22.38 18.26
3538
8066
351.00
$9.0
Table 16: (Offline: comparison with evolutionary algorithms) Hypervolume results of synthetic functions. Best values are highlighted in bold.
11
DYLZI
NSCA
135
10.0
MOEND
0.00
Table 17: (Offline: comparison with evolutionary algorithms) Hypervolume results of real-world tasks. Best values are highlighted in bold.
Matlodd
M8220
823
RL.35
REJ5
61.36
REW7
R541
RE4L
439 0.00
4:54 5 300
0000
1003 000
1000
0.01
1981 500
RC49
= 010
3.88 1.00
6,9 101
3077 2
2230 = 0.75
10:04:000000
MORADD
1.91
1 om
4.3 8000
19:9
1.04
10.5
810
1949 om
1616
0.10
1.00
6,66
1009
21.09 .0m)
22.09 1.00
NA
SPIRAD
483 105--Mm 0.0
5.85
9/0
335
11.9
9:00
1550
1943
163
088
9.5
0.21
K33 Mi
$3.39 6.36
251.81 1 4157
Hypervolume Results in the Offline Setting As mentioned in Section 5.2 Table reports the average rank results. For reference, we provide here the corresponding individual hypervolume results: Table 14 summarizes the results for synthetic problems, while Table 1.5 reports the results for real-world tasks. For each problem, the overall best method is shown in bold, and the best generative approach is highlighted in light gray- In addition, we evaluate the ability of the well- known evolutionary algorithms NSGA-III and MOEA/D, originally designed for the online setting, on offline MOO tasks. We use the pymoо Bllank& & Deb 2020) implementation, with the evaluation adapted to rely on pretrained proxy models instead of the true objective functions. The label "NA" denotes runs that failed due to memory exhaustion when handling many objectives. As shown in Tables [6 and these algorithms struggle adapt the offline setting, indicating that state-of-the- art online MOO methods are not necessarily suitable for resource-constraing domains. By contrast, our SPREAD framework provides this flexibility.
Ablation Study in the MOBO Setting Figure + complements the results in Section comparing the SPREAD and CDM-PSL generative methods without SBX (step 8, Algorithm The figure highlights that SPREAD achieves superior performance compared to CDM-PSL under this setting. Interestingly, on the 4-objective Car Side Impact problem, SPREAD achieves a better convergence rate without SBX than with it.
Many-Objective Experiments (m 10). We consider the DTLZ problem family (online set- ting), which is flexible with respect to the number of objectives. Since HVGrad is a hypervolume- maximization method, it cannot be applied in this setting because hypervolume computation becomes
30

---

Published as a conference paper at ICLR 2026
m=
m=3
4
Branin and Curnin
20111
Perscillio Production
D1L22
Side Impact (RE41)
1.0
0.5
82
027
2311
2013
D1L25
D1L27
0.1
03-
044
COM-
0.5
02
(Duns)
Exalutiora
Ovelatore
Figure 13: (Bayesian) Ablation study on the local-optima escaping technique step 8 of Algorithm 3 SPREAD compared against CDM-PSL, where variants without step are marked with an asterisk (*).
prohibitively expensive in many-objective optimization, and were therefore unable to HVGrad. therefore exclude from this experiment. To evaluate performance, we use an approximate hyper- volume calculation based the Wowff(WWlkinng Fish Group algorithm (While & Bradstreet] 2012) implementation from the pyşmo library (Biscani & Izzo 2020). As shown in Figure [4] SPREAD outperforms the remaining three baselines D1IL.74 and D1LL77, while PMGDA achieves the best performance on DTL.22. Overall, these results demonstrate that SPREAD remains competitive the many-objective setting and scales favorably higher numbers of objectives.
DTLZE 110 abjectiveal
DTLZE 110 dejectives)
BTLZ7 110 objective))
SHEAR
MED bU1p
200P
Figure 14: Performance comparison for many-objective DTLZ problems with 10. We report hypervolume (higher is better) and A-ppread (lower is better). The random seed is fixed to 1000.
E EXTENDED RELATED WORK
To complement the discussion in Section 2. we provide a broader overview of related work, ranging from diffusion-based approaches for single-objective black-box optimization (BBO) surrogate- assisted methods for multi-objective optimization.
Diffusion Models as Data-Driven Samplers for Black-Box Optimization In the single-objective, offline setting, Krishnamoorthy et al. (2023) introduced Denoising Diffusion Optimization Models
31

---

Published as a conference paper at ICLR 2026
(DDOM), which learn conditional generative model over designs given target values and, test time, employ guidance to sample high-reward candidates. This approach highlights the potential of inverse modeling without relying explicit surrogates. Beyond the offline setting, Diffusion-BBO (Wu et al., 2024) extends this idea to the online regime by scoring in the objective space through an uncertainty-aware acquisition function and then conditionally sampling designs, with theoretical and empirical results showing sample-efficient improvements over BO baselines. Parallel efforts broaden the scope of data-driven BBO with diffusion by training reward-directed conditional models that combine large unlabeled datasets with small labeled sets and provide sub-optimality guarantees, or by constraining sampling to learned data manifolds to enforce feasibility, both yielding improvements across black-box optimization tasks (Li et al., 2024b). Together, these works highlight the potential of diffusion models reformulate single-objective black-box optimization as a generative sampling task, paving the way for extensions to multi-objective settings.
Surrogate- Multi-Objective Optimization Surrogate techniques have long been com- bined with multi-objective optimization reduce the cost of expensive evaluations by approximating the true objectives, either globally or locally. Deb et al.] (2020) offer broad taxonomy of surrogate modeling strategies and propose adaptive switching scheme (ASM) that cycles among different surrogate types, demonstrating that ASM often outperforms any individual surrogate model. With- out requiring explicit gradients, Berkemeier Peitz (202T) introduce a derivative-free trust-region descent method for multi-objective problems, which builds radial basis function surrogates within each local region and proves convergence to Pareto-critical points. In practical engineering settings, surrogate-assisted MOO has been applied to optimize permanent magnet synchronous motors us- ing neural networks, Kriging, or support vector regression under small sample regimes (Li et al. 2025b). In the context of multi-objective control problems and PDE-constraiie Dellnitz (2018b) survey how surrogate modeling or reduced-ordes models help accelerate decision making feedback control under real-time constraints. In reservoir modeling and well control, the MOO-SESA framework of Wang et al. (2024) combines a selective ensemble of SVR surrogates with NSGA-II to strike balance between surrogate robustness and multi-objective accuracy, yielding faster convergence and more reliable Pareto fronts in benchmark reservoir. Overall, surrogate-assisted methods provide the foundation for offline and Bayesian multi-objective optimization, where surro- gates not only reduce evaluation costs but also serve as probabilistic models to guide exploration and exploitation under uncertainty.
systems, Peitz &
32


Reviews:
[{"review_id": "SdrvzctalH", "reviewer": "Reviewer_uYzd", "summary": "This paper proposes SPREAD, a diffusion-based generative framework for multi-objective optimization (MOO).\nThe method integrates a conditional DDPM with a Multiple Gradient Descent (MGD)-inspired adaptive guidance mechanism and a Gaussian RBF repulsion term to jointly ensure convergence to Pareto optimality and coverage of the Pareto front.\nSPREAD can operate in three major settings:\nOnline optimization, directly refining decision variables;\nOffline optimization, using static datasets via surrogate functions;\nBayesian optimization, using Gaussian process surrogates under tight evaluation budgets.\n\nThe paper provides:\n- Theoretical results (Theorem 1–2) establishing that the conditional diffusion process yields Pareto-dominant samples with high probability;\n- A Transformer-based diffusion architecture (DiT-MOO);\n- Extensive experiments across online, offline, and Bayesian benchmarks (ZDT, DTLZ, RE, OffMOO-Bench, and MOBO suites).\n\nResults show that SPREAD outperforms state-of-the-art baselines (PMGDA, HVGrad, STCH, ParetoFlow, PGD-MOO, CDM-PSL) in terms of hypervolume, diversity, and scalability.", "questions": "1. Theorem 1 assumes that the learned conditional distribution Qθ(⋅∣c) approximates the true conditional training distribution PX∣C=c within TV distance ≤ τ.\n  - How is τ estimated or controlled in practice?\n  - Can you provide empirical evidence (e.g., KL or Wasserstein estimates) showing that the learned conditional diffusion distribution indeed remains close to the true data distribution during training?\n  - If τ > 0.2, how sensitive is Theorem 1’s conclusion to this deviation?\n2. Theorem 2 assumes νₜ = 0 for all t, which guarantees a common descent direction.\n  - However, experiments rely on νₜ > 0 to ensure diversity.\n  - Could you provide an approximate convergence argument or empirical validation that Pareto dominance still holds under small νₜ > 0?\n  - How large can νₜ be before the descent guarantee breaks down?\n3. The “main direction” optimization mixes alignment and diversity terms and is solved via “a fixed number of gradient steps.”\n  - What is the typical number of inner-loop iterations and their computational cost relative to one DDPM denoising step?\n  - Does convergence of this inner optimization affect the overall sampling stability?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. First systematic framework unifying diffusion models and multi-objective optimization.The adaptive gradient guidance that couples DDPM sampling with MGD directions is conceptually novel and practically effective.Theoretical justification (Theorem 1 & 2) provides insight into convergence and dominance guarantees, uncommon in diffusion-based optimization works.\n2. Mathematical exposition is rigorous, with precise definitions (Pareto stationarity, dominance, MGD formulation). Algorithmic presentation (Algorithmic 1–4) is detailed enough for reproducibility. Ablation studies (Table 3) isolate the role of each design component (perturbation, repulsion, diversity), showing necessity.Extensive cross-domain experiments strengthen the claim of generality.\n3. Addresses three major MOO paradigms (online/offline/Bayesian) under a single framework. Potentially impactful for the community exploring generative optimization or diffusion-driven design. Outperforms strong baselines across different domains and scales; results are consistent and statistically robust.", "weaknesses": "1. Theorem 1 assumes total variation (TV) distance ≤ τ between the learned conditional and the true data distribution, but τ is not quantified or empirically verified.\n2. Theorem 2 depends on νₜ = 0 for descent guarantees, which does not hold in practice (since diversity requires νₜ > 0). A stability bound for nonzero νₜ would improve rigor.\n3. Although Figures 2(a–b) show runtime comparisons, there is no formal analysis of asymptotic computational cost (e.g., O(n·T·d)) or memory scaling.\n4. All tested problems have ≤ 4 objectives; no experiments on “many-objective” problems (m ≥ 10).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ess6MyQ31p", "reviewer": "Reviewer_CrTm", "summary": "This paper proposes a diffusion-based approach to multi-objective optimization. The core novel mechanism is straightforward: during training, the model learns to map from artificially worsened objectives (real objectives plus a positive shift) to designs, then at inference conditions on actual objectives to encourage improvement. At each reverse diffusion step, they apply post-processing that combines multiple gradient descent and RBF-based repulsion borrowed from existing MOO literature. The method trains a separate model per problem and is evaluated in three settings. The online setting (cheap evaluations) shows marginal gains over simpler gradient methods. The offline setting enables the model to learn from static datasets and propose new designs beyond the dataset. The Bayesian setting trains the conditional diffusion model on small datasets with data augmentation, and demonstrates promising results against some MOBO baselines like PSL-MOBO.", "questions": "- Have the authors compared their approach against MGD+RBF without the diffusion model in all three settings?\n- As mentioned in weakness, can the authors provide ablations with explicit shift conditioning $p(x|c,\\Xi)$ versus the current implicit approach?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- I think the idea of shifting condition is interesting and new in MOO literature.\n- The algorithm is straightforward, and the idea of data augmentation that enables the method to work in Bayesian setting is interesting.", "weaknesses": "- I find it difficult to grasp the theoretical insight that makes the method work. Here is my understanding: (1) the diffusion model learns the input-output mapping from the training data, (2) it also implicitly learns the global shift vector $\\Xi$. Then, if the diffusion model has learned the input-output mapping well and can generalize, it should be able to shift its output by conditioning on the new $C$ to infer $F$ via point 2, and thereby generate $X$ through its generalization capability. This hypothesis could be partially verified by explicitly feeding the shift vector to the conditional diffusion model (i.e., conditioning on both $C$ and $\\Xi$) to see if it gives better performance as an ablation study. However, if this is the mechanism, I cannot understand why it would work well in data-sparse settings like the online scenario wher obj is expensive .\n- The shift vector introduces a tricky hyperparameter with unclear guidance on how to properly choose it.\n- Although benchmarked across different problem settings, the claims seem somewhat overstated. I am not fully convinced this provides significant benefit in the cheap online setting. For the Bayesian setting, it may be beneficial, which is interesting but warrants deeper investigation with more ablation studies since PSL-MOBO is not a very standard baseline. Overall, I think this is more of an algorithm suited for offline settings.\n- The method trains a separate model per problem with no mechanism for transfer learning or generalization across problems. This limits scalability and requires expensive retraining (1000 epochs) for each new problem.\n- The paper lacks comparison against simpler baselines, particularly: (1) MGD+RBF with multiple random initializations (no diffusion model) in the online setting, (2) guidance-free diffusion sampling to quantify the guidance contribution, and (3) explicit vs implicit shift conditioning. Without these ablations, it is unclear whether the diffusion model provides meaningful benefit beyond the gradient-based guidance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NScWrFGutw", "reviewer": "Reviewer_rKVB", "summary": "This paper introduces SPREAD, a generative framework for multi-objective optimization (MOO) built upon Denoising Diffusion Probabilistic Models (DDPMs). The central idea is to learn a conditional diffusion process over the decision space and refine candidate solutions during reverse diffusion via an adaptive guidance mechanism. This mechanism combines a Multiple Gradient Descent (MGD)-inspired update for efficient convergence and a Gaussian RBF-based repulsion term for maintaining diversity. The paper clearly motivates the use of a generative, diffusion-based formulation by highlighting the limitations of conventional MOO approaches. The main contribution lies in unifying these components into a single conditional diffusion process, applicable to online, offline, and Bayesian MOO settings. Extensive experiments on synthetic and real-world benchmarks demonstrate that SPREAD achieves competitive or superior performance in hypervolume and diversity, particularly as the number of objectives increases.", "questions": "Kindly address weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The paper introduces a fresh perspective on MOO by leveraging diffusion models for iterative refinement toward the Pareto front\n- The combination of MGD-based guidance and RBF repulsion is both intuitive and well-motivated, yielding a good balance between convergence and diversity.\n- The theoretical results (Theorems 1 and 2) are rigorous, supporting the conditioning and convergence properties.\n- SPREAD consistently outperforms or matches state-of-the-art methods in hypervolume and diversity metrics.\n- Ablation studies (Table 3, Figs. 6–8) are particularly informative, showing that each component—MGD alignment, repulsion, and perturbation—contributes meaningfully to performance.", "weaknesses": "- The notation is overly dense, with multiple primes and sub-/superscripts that reduce readability. \n- The framework involves training a diffusion transformer and solving an optimization (Eq. 13) at every sampling step. Although Figure 2 shows SPREAD is faster than PMGDA, it remains slower than lighter baselines such as MOO-SVGD or HVGrad. Particularly, solving Eq. 13 not only depends on the data size $n$ but also the decision-space dimensionality. Some analysis of runtime scaling with decision-space dimensionality would strengthen the discussion.\n- SPREAD introduces two hyperparameters, $\\nu_t$ and $\\rho$. While $\\nu_t = 10$ is reported as the default, the default for $\\rho$ is unspecified. Given the cost of expensive optimization runs, hyperparameter tuning can be prohibitive. Moreover, Figures 6 and 7 indicate that performance is somewhat sensitive to these values, suggesting that robustness should be discussed in greater detail.\n- The largest tested problem (RE61) has $m = 6$ objectives. In combination with the weakness of *hyperparameter sensitivity*, it would be valuable to evaluate SPREAD's performance in higher-dimensional settings (e.g., $m \\geq 10$) to assess effectiveness of SPREAD.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)