Pareto Set Learning for Expensive Multi-Objective Optimization
Authors: Xi Lin
Zhiyuan Yang
Xiaoyuan Zhang
Qingfu Zhang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Multi-Objective Optimization
Expensive Multi-Objective Optimization
Bayesian Optimization
Multi-Objective Bayesian Optimization
Pareto Set Learning

Abstract: Expensive multi-objective optimization problems can be found in many real-world applications, where their objective function evaluations involve expensive computations or physical experiments. It is desirable to obtain an approximate Pareto front with a limited evaluation budget. Multi-objective Bayesian optimization (MOBO) has been widely used for finding a finite set of Pareto optimal solutions. However, it is well-known that the whole Pareto set is on a continuous manifold and can contain infinite solutions. The structural properties of the Pareto set are not well exploited in existing MOBO methods, and the finite-set approximation may not contain the most preferred solution(s) for decision-makers. This paper develops a novel learning-based method to approximate the whole Pareto set for MOBO, which generalizes the decomposition-based multi-objective optimization algorithm (MOEA/D) from finite populations to models. We design a simple and powerful acquisition search method based on the learned Pareto set, which naturally supports batch evaluation. In addition, with our proposed model, decision-makers can readily explore any trade-off area in the approximate Pareto set for flexible decision-making. This work represents the first attempt to model the Pareto set for expensive multi-objective optimization. Experimental results on different synthetic and real-world problems demonstrate the effectiveness of our proposed method. 

Pareto Set Learning for Expensive Multi-Objective Optimization
Xi Lin, Zhiyuan Yang, Xiaoyuan Zhang, Qingfu Zhang
Department of Computer Science, City University of Hong Kong
(xi. fii,inn zhiyuyang4-c, xzhang2523-c) cityu. edu.kkk sungfu.zhangocitge tyu edu.hk
Abstract
Expensive multi-objective optimization problems be found in many real-world applications, where their objective function evaluations involve expensive compu- tations or physical experiments.
desirable obtain an approximate Pareto front with a limited evaluation budget. Multi-objective Bayesian optimization (MOBO) has been widely used for finding finite set of Pareto optimal solutions. However, it is well-known that the whole Pareto set is on a continuous manifold and can contain infinite solutions. The structural properties of the Pareto set are not well exploited in existing MOBO methods, and the sinite-set approximation may not contain the most preferred solution(s) for deciisionmmkeens This paper develops novel learning-basee method to approximate the whole Pareto set for MOBO, which generalizes the decomposition- algorithm (MOEA/D) from finite populations to models. We design simple and powerful acquisition search method based on the learned Pareto set, which naturally supports batch evaluation. In addition, with our proposed model, decision-makes can readily explore any trade-off area in the approximate Pareto set for flexible decision decision-making This work represents the first attempt model the Pareto set for expensive multi-objective optimization. Experimental results on different synthetic and real-world problems demonstrate the effectiveness of our proposed method.
multi-ohjective optimization
Introduction
Many real-world applications involve optimizing mul- tiple costly-to-evaluate and potentially competing objectives, such as finding strong yet ductile mate- rial (38) building a neural network with high accu- racy and low latency (28) and improving the quality while minimizing total charge in particle accelera- tor tuning [73]. Very often, these objectives conflict each other and cannot be optimized simultaneously by single solution. Instead, there set of so- lutions with different optimal trade-off's among the objectives, called the Pareto set. For Pareto op- timal solution, none of its objective values can be further improved without deteriorating others. In ad- dition, the evaluation of each solution could require time-consuming computation or costly physical ex- periments, and thus large number of evaluations are unbearable. Different multi-objective Bayesian optimization (MOBO) algorithms 44 48 43), typically directly generalized from the single-objective Bayesian optimization (BO)
40
160
(a) ParEGO
(b) Pareto Set Learning
Figure 1: Pareto Set Learning can approxi- mate the whole Pareto set, and let decision- makers easily explore any trade-off among objectives to choose their preferred solutions.
77 have been proposed to find small set of approximate Pareto solutions with a small amount of objective function evaluation budget.
660
39
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

The finite set approximation has some undesirable drawbacks. For a nontrivial multi- multi-objective optimization problem, the Pareto set is on a continuous manifold and has infinite solutions with different optimal trade-offs among the objectives [58]. This Pareto set structure could be helpful to better select candidate solutions for expensive evaluation, and hence accelerate the optimization process of MOBO. In addition, small set of solutions may not contain the one(s) that exactly satisfy the decision-maker preferences. Finding the most preferred trade-off solution(s) could require several rounds of interaction with the decision-makers. These approaches would be extremely time-consuming, especially when the optimization modeler and the decision-maie are not in the same team, which is common in real-world MOBO applications [56]-
This paper proposes a novel Pareto set learning (PSL) method to approximate the whole Pareto set for expensive multi-objective optimization problems with limited evaluation budget. Our proposed method can accelerate the multi-objective Bayesian optimization process, and also provide decision-maken with more useful information to support flexible decision-making To the best of our knowledge, this is the first attempt to learn the whole Pareto set for expensive multi-objective optimization. Our main contributions include:
We propose a novel set model to map any trade-off preference to its corresponding Pareto solution, along with surrogate model-based method to approximate the whole Pareto set with a limited evaluation budget.
We develop lightweight yet powerful batch acquisition search method for efficient MOBO, which can outperform other MOBO approaches in terms of both performance and computa- tional cost. We demonstrate that the learned Pareto set can support flexible user-involved decision-making.
We test our proposed method on both synthetic benchmarks and real-world application problems. The results validate the efficiency and usefulness of PSL.
2 Related work
Bayesian Optimization. Surrogate model-based methods have been widely used and studied for expensive optimization 47 65 79]. These methods iteratively build a surrogate model to approximate the black-box objective function, and uses an acquisition function to search for the optimal solution. Much effort has been made on various design issues Bayesian optimization, such as acquisition functions [8I) nigh-dimensional optimization [1 92], batch evaluation (22), scalable
optimization [S 277, and theoretical analysis [42]. Most work for BO are on single-objective optimization. We refer readers to Garnett (30] for a comprehensive introduction.
Multi-Objective Bayesian Optimization. MOBO extends single-objective Bayesian optimization for solving expensive multi-objective optimization problems. Although the Pareto set could contain infinite solutions, the MOBO methods typically focus on finding single or finite set of solutions. The scalarization-base algorithms, such as ParEGO 45) and TS-TCH [62] iteratively scalarize the multi-objectivy problem into single-objective ones with random preferences, and then apply single-objective BO to solve them. MOEA/D-EGO [99] adopts the MOEA/D framework [97] to solve a set of surrogate scalarized subproblems simultaneously. SMS-EGO 667 and PAL [04 105
generalize the upper confidence bound (UCB) to multi-objective optimization. Emmerich et al. 26 and Emmerich and Klinkenberg (25 propose the probability of improvement (PI) and expected
improvement (EI) for multi-objective hypervolume. Entropy search methods 33
35 37 90] have
also been studied in multi-objective optimization [34 6 84). Bradford et al. [10 and Belakaria et al.
77) consider Thompson sampling and uncertainty maximization for multi-objective optimization, respectively. Different algorithms can be hybridized to achieve better performances [82].
Different new developments have been recently proposed to handle the issues of diverse batch evaluation (53] efficient hypervolume improvement calculation [14], noisy optimization [15 [6].
high-dimensional optimization [17], and decision criteria beyond Pareto optimality [56]. Some
attempts have been made to incorporate the decision-maker's preference into MOBO 3 62 They
typically need the decision-maket preference before or during optimization, which may not always
be available in real- weal-world applications. All these MOBO methods aim to provide finite set of approximate Pareto optimal solutions to decision-makers.

---

fxo
1,00
/,00
7000
(a) (Weakly) Pareto Sols.
(b) Weakly Pareto Front
(c) Pareto Front
(d) PSL for MOBO
Figure 2: (Weakly) Pareto solutions and (Weakly) Pareto front. (a) Examples of Pareto solutions, weakly Pareto solutions, and dominated solutions. The Pareto solutions are also weakly Pareto optimal. The weakly Pareto optimal but not Pareto optimal solutions (e.g., purple points) are dominated but not strietly dominated by at least one Pareto solution.
(b) The weakly Pareto front f(Mweak) is the image of all weakly Pareto optimal solutions Mweak in the objective space. (c) The Pareto front f(Mps) is the image of all Pareto optimal solution MMps (Pareto set) in the objective space. (d) Our proposed method approximates the whole Pareto set and uses it to select solutions for expensive evaluation, which can improve the search efficiency of multi-objeccive Bayesian optimization.
Structure Learning. In addition the surrogate objective model, some methods have been proposed to explore the problem structure for Bayesian optimization. Sener and Koltun [76] learn the geometric
structure of the problem in an online manner accelerate optimization. Wang et [8 and Zhao et al. [101] apply Monte Carlo tree search (MCTS) to divide the search space for efficient modeling and searching. Novel latent space modelings with complex solution representations.
32 86
have been proposed for optimization problems
From Population to Pareto Set Learning. For the last several decades, most multi objet-ob optimization methods have focused on finding single or a finite set of Pareto optimal solutions (e.g.. a population) approximate the Pareto [58] 24). A few attempts have been made approximate the whole Pareto set with simple mathematical models [68
[36] 98, 31]. Pirotta et al. [66] and Parisi al. 633 have proposed conduct Pareto manifold approximation for multi-objective reinforcement learning. Recently, different approaches have also been investigated to incorporate the preference information into deep neural networks for image style transfer (78 23), multi-task learning with finite solutions [75] 55 54) or approximate Pareto front [50] 61 74), reinforcement learning [96] [], and neural combinatorial optimization [5.. In this work, we generalize the decomposition- based multi-objective optimization algorithm (MOEA/D) 197). and propose to learn a set model which maps all valid trade-off preferences to the Pareto set for efficient MOBO.
3 Expensive multi-objective optimization
We consider the following expensive continuous multi-ohjective optimization problem:
f(x) = = (2())) zex
min
fm(()),
(I)
where is solution in the decision space X R", f X 1 is an m- nd-dimensional valuor objective function, and the evaluation is expensive for all individual objectives fa(x), i 1,.... m. For non-trivial problem, no single solution can optimize all objectives at the same time, and we have to make a trade-off among them. We have the following definitions for multi-objective optimization:
Definition 1 (Pareto Dominance and Striet Dominance) Let x ,", x E X. x is said to dominate 2², denoted as x x", , if and only if f((x") S fi(a"), Vi E [l...,,m) m] and 3j (l, m) such that f2²) < < (x))))) In addition, x" is said to strictly dominate x² (x Karia x²) ), if and only if f((x") < f((x"), Vi m...
Definition 2 (Pareto Optimality) A solution xx E X is Pareto optimal if there is no a E X such that & x x². A solution x' E X weakly Pareto optimal if there is no àt E X such that à Lari x'.
Definition 3 (Pareto Set/Front) The set of all Pareto optimal solutions Mps S X is called the Pureto set, and its image in the objective space f(Mps) = (f(x)) ) x E Mpd is called the Pareto front. Similarly, we can define the weakly Pareto set Mweat and weakly Pareto frons f (MMeak).
3

---

The strict dominance relation is stronger than the Pareto dominance since it requires strictly better values for all objectives. Therefore, the set of weakly Pareto optimal solutions Mweak (c.g.. the solutions that are not strictly dominated) would be larger than Mps. and it is straightforward to see Mps S Mwakk The illustration of (weakly) Pareto solution and Pareto front shown in Figure 2
Each Pareto solution æ E .Ms represents a different optimal trade-off' among the objectíves for problem (I). Under mild conditions, the Pareto set Mps and Pareto front f(Mps) are both on (m dimensional manifold in the decision space X' E _ and objective space R", respectively 36 98 The number of Pareto solutions could be infinite (i.e. |Mpps oc).
Bayesian Optimization (BO) is a model based method for solving expensive black-box optimization problems. Given a set of
solutions {X, y). BO builds surrogate models (e.g.. Gaussian process) for each objective, and defines acquisition function(s) to leverage the surrogate objective values for navigating the search space. Only promising solutions will be selected for expensive evaluation. We refer interesting reader to 30 for a detailed introduction.
Pareto Set Learning. The current MOBO methods aim to find small set of finite solutions = {x(1), _x)22_._...)))))))))) .... ........llsl) x((S)) to approximate the Pareto set Mps- In addition to the evaluated solutions S, our proposed Pareto set learning (PSL) method also learns an estimated Pareto set Mpat
with the predicted Pareto front pMMp)) to approximate the Pareto set Mps and Pareto front (Mp)) The whole approximate Pareto set can be easily explored by adjusting the trf preference as illustrated in Figure 3 With the learned Pareto set, we also develop an efficient batched solution selection approach for efficient MOBO, which will be introduced in the next section.
4 Pareto Set Learning for MOBO
4.1 Pareto set model
As pointed out in Section B] the Pareto set can contain infinite solutions with different trade- offs. In addition, there is no complete order among the Pareto solutions. A Pareto set model for MOBO should be powerful enough to ap- proximate the whole Pareto set, and convenient enough to easily explore any trade-off solutions, In this work, we propose to build a set model that maps any trade-off off preferences to their cor- responding Pareto solutions with scalarization.
Koonmene ner
bp
" M
14
(a) Preference to PS
(b) Preference to PF
Scalarization. The scalarization method pro- vides a natural connection from a set of pref- erences A = (X E RTI E> = }} among the m objectives to the Pareto set Mps- The most simple and straightforward approach is the weight-sum scalarization:
Figure 3: Mapping from Preferences to Ap- proximate Pareto Set/Front: Our proposed PSL. method learns the connection from set of valid (infinite) trade-off preference A = {A e RT > = =} to (a) the approximate Pareto set Mpst and hence (b) the corresponding predicted Pareto front f-Mpu)). The whole Pareto set (front) can be easily explored by adjusting the preference. The preference simplex have been resized and ro- tated for better visualization.
min Iws (x/A) #Ex
= min Em (xx)
(2)
=ex
However, this method can only find the con-
vex hull of Pareto front (Mps) 24]. In this
work, we use the following weighted Tcheby- cheff approach:
min Sich (x|A) = min max (A(f{(z)) - (=, -e)] x
#EX 15s5m
where 2" (27. , ) is the ideal vector for the objective vector f(z) (i.e. lower-bound for
minimization problem), E > 0 is a small positive scalar, and u, = (2) - E) is called an (unachievable) utopia value for the i-th objective fi(æz). This scalarization method has a promising property:
Theorem (Choo and Atkins [3). A feasible solution & E X is weakly Pareto optimal if and only if there is a weight vector X > 0 such that x is an optimal solution of the problem 3

---

Preference R"
Parameter R!
Solution R
Esitimated Obj.R"
Tch. Scalarization E
Model 0
GP Model(s)
f(x)
Itch (2|12)
f(x) Objectives R
Pareto Set Model
Gaussian Process Model
Figure 4: Pareto Set Learning for Multi-Objective Bayesian Optimization: (a) The Pareto set model learns a parameterized mapping from any valid preference À E A {} R Σ>; A = 1} to its corresponding solution x²)) E R (b) We build independent Gaussian process models for each objective function. With these surrogate models, the set model can be efficiently trained to approximate the Pareto set. (c) In this work, we use the augmented Tchebycheff scalarization to connect each preference its corresponding Pareto solution.
In other words, all Pareto solutions a E Mps can found by solving the Tchebycheff scalarized subproblem (3) with a specific (but unknown) trade-off preference X. We let Miah be the solution set for problem 3 with all valid preferences A and have Mps S .weakk = -Mah- The weakly Pareto optimal but not Pareto optimal solutions (Mweak MMps are dominated (but not strictly dominated) by some Pareto solutions, and are usually not desirable for decision-making. They can be further avoided by the augmented Tchebycheff approach [83 41
In this work, we the following scalarization:
Stch_aug (x|A)
max (M((f(()) (2) - =))} + Σ Aff(z)) AAA 1m5m
where P sufficiently small positive scalar depends on the problem and current solution location. This form of augmentation has also been used in ParEGO [45]. With the augmentation term, the weakly dominated solutions will have larger scalarized values than the corresponding Pareto solutions in (4) and will ultimately be eliminated with the optimization process (e.g., Much aug Mps). In this work, we simply set p = 0.001, dynamically update 2 as the current best value for each objective and let 0. This setting is robust for all problems we considered. The traditional methods focus on solving the scalarization problem with a finite set different preferences a sequential [45) or collaborative manner [97 199].
Set Model. With augmented Tchebycheff scalarization, we propose build set model for mapping preferences to their solutions:
(AA)= = ho()),
where À is any valid preference in A == ( RT RTI = 1}, x (A) X' is its corresponding Pareto
solution, and (A) is the Pareto set model with parameter 0. The input preference X has (m - 1) degree of freedom, and the output solution set Mpss = fo ho (>){A E A} is on an (m - 1)- dimensional manifold in X E R" In other words, the set model maps the (m 11) 1i-dimensional regular preference simplex A to the (m 1)-dimensional solution set Mpa with complicated structure.
We want to find the optimal parameters 0" such that the generated set Mpal matches the solution set for augmented Tchebycheff scalarization Mich sug (x- (A)) AJ. where
(X) ho- (A) arg min amg (x: A), A. =EX
(6)
The learned mapping is illustrated in Figure 3 Once the connection is learned, we can explore the whole approximate Pareto set/front by simply adjusting the preferences among objectives. We use an MLP neural network as the set model for all MOBO problems, which is good at capturing complicated problem structures [76]. The model details can be found Appendix [
4.2 Pareto Set Learning with Gaussian Process
Since the evaluation of (a)()) = (h)g is expensive, we use the surrogate model-based approach to learn the Pareto set model (A) shown in Figure Our method is orthogonal to the choice of surrogate models, and we build independent Gaussian process models for each objective as in other MOBO methods [4 553.
5

---

Gaussian Process Model. A single-objective Gaussian process 69 has a prior distribution defined on the function space:
f(x) ~ (P())):))) k(x, æ)):
(7)
where μ X + I is the mean function and k 2' x X R is the covariance kernel function. With n evaluated solutions {X, y = {[x(), [f( = I,.... n) the posterior distribution can be
updated by maximizing the marginal likelihood based the data. For a new solution x*+1 the posterior mean and variance are:
((xx(n+)) (((:((+))))) -
(a = k(x(((1)) (((ltt)),, (x(+))))
k:" K-1 k
(8)
where k = kxxxx X) is the kernel vector, K = 1 x) is the kernel matrix, Matérn 5/2 kernel are used for all models in this work. For m independent GP models, we let (xx) = [ (az)... A.m (x)] and 6² (xx)
[o] (z). o (x)] be the predicted mean and variance for the objective vector. Suppose we have a learned Pareto set Mpat. the GP models give us both predicted value (M(Mpa) = {f( 3 and uncertainty o (MMpa) (o (2)) E MMpa for the whole approximate
Pareto set.
Pareto Set Learning. Now we propose an effi-
Algorithm 1 PSL with GP Models
cient algorithm find the optimal parameter 0* for the Pareto set model ho (A). The optimal so-
1: Input: Model x (A) = (A)
lution set Mich aug for augmented Tchebycheff scalarization (4) is unknown, hence we need to optimize all solutions generated by our model a (A) he( with respect to their correspond- ing augmented Tchebycheff scalarization sub-
2: Initialize the parameters dg
3: for I. == 1 to 7 do
4: Sample preferences { A
5: Update 0 with gradient descent in 10
6: end for
problems for all valid preferences:
7: Output: Model (<A) = hor (A)
0- arg min Ex Ex~A ( he
(9)
If the model is perfectly learned, the obtained approximate Pareto set M - f
should be the same as Mich aug- However, is difficult to directly optimize (9) due to the expectation over infinite preferences (IA = oo). We use Monte Carlo sampling and gradient descent to iteratively learn the model with the surrogate model:
02+1 O.
V Votich ang ang ho (Ax) As))
(10)
k=1
where we randomly sample K = 10 different valid preferences (A1,.... Ax) A at each iteration in this work. Here Jugh (.) is the augmented Tehehycheff scalarization with predicted objective values:
m
Juch (x: A)
max (A.(f((z) (2, I5i5m
- -))) + Σ xxf((z).
(11)
One design issue left is how to set the surrogate objective f(x). If we only want to obtain the current predictive Pareto front, it is straightforward to use the posterior mean as the surrogate value. The approximate Pareto front under the posterior mean could provide valuable information to decision- makers. However, for Bayesian optimization, we have to take the uncertainty into account to balance exploitation and exploration. Many widely-used criteria, such as expected improvement (EI) [59] and upper confidence bound (UCB) [81) could be more reasonable choice. In this work, we use the lower confidence bound (LCB) for minimization problems.
(xx) fa(x:) =3
(12)
We simply set B = and discuss the performance with other surrogate values in Appendix F.9
The expensive objective function f(x) ) is usually black-box and non-diffcrentiablle but we can easily obtain the gradients for the Gaussian process and the set model. Indeed, gradient-based methods have been widely used for optimizing the acquisition function in both BO 195 93 and MOBO [14 53) The max operator in Tchebycheff scalarization is technically only subdifferentiable, but it is known to have good subgradients [94] for surrogate optimization and can preserve convexity if the objectives (.((() ) are all convex 9

---

The Pareto set learning algorithm with Gaussian process models is summarized in Algorithm 1 We find that the simple random initialization and gradient descent are enough to learn a good Pareto set approximation. The overparameterized neural network could be beneficial to overcome potential non-convexity 52
4.3 Batched selection on approximate Pareto set
Algorithm 2 MOBO with PSL
Algorithm 3 Batch Selection with PSL
Input: Black-box vector-valued function fa:)
1: Input: Model mx)) ho(A), Batch
2: Initial Samples (X Vo
Size B
3: for t = 1 to T do
2: Sample P Ppreferences
4: Train GPs based on (X 9111
{A { }-1 ~
5: Learn set model he, (A) with GPs (Alg. 1)
3: Generate solutions
6: Select (x) Jo= with the set model (Alg. 3)
X = ( ( {} on Mp
7:
X (z(0))
4: Find subset (2(())
1b=1 C X that has the highest (VI(f((a(0))
Y/ f yt-1 U (f
8: end for
5: Output: Batch solutions (a h-1
9: Output: {X,, 3 and final set model hor (A)
In this subsection, we propose a lightweight yet efficient batched acquisition search for MOBO with the learned Pareto set model. The algorithm framework is shown in Algorithm 2 The crucial difference with other MOBO approaches is that we build set model at each iteration for batched solution selection as shown in Algorithm | and Algorithm 3
The batched selection procedure
contains two closely related steps:
Batch Sampling on Approximate Pareto Set. Our model naturally supports generating an arbitrary number of solutions batch. If the decision-maken preferences are available, we can use preference- based sampling in this step. In this work, without any prior knowledge, we uniformly sample P the valid preferences {X (P) pp=1. and generate the corresponding solutions X { approximate Pareto set Mpal-
Batch Selection. At each iteration of MOBO, we typically select small number B (e.g., 5) of solutions Xn (x) ] from the sampled solutions X for expensive evaluations. To take all
already evaluated solutions {X+-1, 1 , into consideration, we the hypervolume [03] as the
selection criteria. The hypervolume HV(y) = Vol(S) measures the volume of S dominated by set y in the objective space:
S fr RM By y such that y x rr").
(13)
where is a reference point that dominated by y E y. The hypervolume improvement (HVI) of set X 3 with respect to the already evaluated solutions {X,-1. 1 can be defined as:
(x)()(x))) LIVVy,1-uO - HV
(14)
where XB
(b) B
jb are selected solutions, and f(X B) are the surrogate values. In this work, as the surrogate value for Bayesian optimization, and provide an ablation (2)
we mainly use the LCB study of different surrogate values in Appendix F.9
A better trade-off set will have larger hypervolume, and the true Pareto set always has the largest one. We want to select set of X B such that their corresponding objective values f(X B)) maximize H XX ))) It would be computationally expensive to jointly optimize a set of solutions to exactly maximize the hypervolume improvement (14). and therefore sequential greedy selection is typically used [1]. this work, we select the set X B in a sequential greedy manner from X where
X P 1, 000 for all problems. More details can be found in Appendix D.2
7

---

5 Experiments
In this section, we compare the proposed PSL method with other MOBO approaches on the perfor- mance of evaluated solutions. We also analyze the quality of the learned Pareto set model, which other methods cannot produce.
Baseline Algorithms. We consider several widely-used MOBO methods and two model-free ap-
proaches as baselines. The implementations of NSGA-II 200, MOEA/D-EGO [99). TSEMO [TO]
USeMO-EI 77], DGEMO [53] are from DGEMO's 's open-source codebase 1I based on pymoo g
The implementations of scrambled Sobol sequence, qParEGO 45), TS-TCH 622 qEHVi 14 and
QNEHVI [5] and from BoTorch [5]. We implement the proposed PSI in Pytorch (64]
Benchmarks and Real-World Problems. The algorithms are first compared six newly pro-
posed synthetic test instances (see Appendix E.T. as well as the widely-used VLMOP1-3 [8] and
DTLZ2 21
benchmark problems. Then we also conduct experiments on 5 different real-world multi-objective engineering design problems (RE) [85] including 1) four bar truss design [2] 2)
pressure vessel design [46]: 3) disk brake design [70]; 4) gear train design [9 and 5) rocket injector
design 87 Details of these problems can be found in Appendix
Experiment Setting. For each experiment, we randomly generate 10 initial solutions for expensive evaluations, and then conduct MOBO with 20 batched evaluations with batch size 5. Therefore, there total 110 expensive evaluations. For an experiment, all algorithms are independently run 10 times. We use the hypervolume indicator [103] as the metric to compare the quality evaluated solutions chosen by different MOBO algorithms, which is monotonic to the Pareto dominance relation. The ground truth Pareto front will always have the best (highest) hypervolume.
5.1
Experimental results and analysis
Table I: Algorithm runtime per iteration (in seconds).
Problem
#objs
MOEA/D-EGO TSEMO USeMO-EI
DGEMO
qEHVI PSL(Ours): Model + Selection
FI
2
40.95
4.82
6.12
61.48
36.71
5.26 + 1.33 6.59
DTL.22
3
71.83
7.28
8.76
83.57
75.92
7.02 + 1.59 8.61
MOBO Performance. We compare PSL with other MOBO methods on the performance of evaluated solutions. Figure Shoows the log hypervolume difference to the true/approximate Pareto front for the synthetic/real- -world problems during the optimization process. The approximate Pareto fronts for the real-world design problems are from Tanabe and Ishibuchi [85] with a large number of evaluations, which are also used in other MOBO works. In most experiments, our proposed PSL method has better comparable performance with other MOBO algorithms. Especially, generalized scalarization- based method, PSL significantly outperforms the model-free counterparts such qParEGO 45 15 MOEA/D- EGO 199. and TS-TCH [62]. These promising results validate the efficiency and usefulness of Pareto set learning for MOBO. More discussion of the proposed algorithm can be found in Appendix A. and Appendix A.2
As shown in Table [ PSL has shorter or comparable total runtime (e.g., for modeling and batch selection) per iteration with other MOBO methods, which can be ignored in the expensive optimization problems (might take days). The algorithm runtimes for all problems can be found in Appendix F. These results confirm that the Pareto set learning approach has a low computational overhead which is affordable for MOBO.
The Learned Pareto Set. We present the approximate Pareto set learned by PSL under the posterior mean after optimization in Figure 6 which not supported by other MOBO methods. According to the results, PSL can successfully learn the Pareto sets for different benchmarks and real-world application problems with different shapes of Pareto fronts. For benchmark problems, PSL can match the ground truth Pareto front with a small evaluation budget. For real-world applications, the approximate Pareto fronts can capture the trade-off among objectives and provide valuable information to support decision-making, We further discuss the the practicality of the approximate Pareto set in Appendix A.3
httpee///gthhi..ccm/mmmmaaggaggs
httpss//ggshhu...coyyyorrrr////p nttpp::/ggithun wtw.hhubb.coo/1--LPSSL...

---

19360
-16
-14
20
100
100
(I)
(b) F2
F3
-44
1.0
-14
Nancton Evenutioo
Fundion Evaluatora
Function Eve catiro
(d) F44
(e) F5
(i) F6
32
-16
10g
Tunction Everutoro
Furction Evatotora
Fuection Eversloro
(g) VLMOP1
(b) VLMOP2
VLMOP3
3.2
10g 301emnc
0.2
-2.0
15
-12
Tuncioo Evetubboro
Fancdon Eva
Pancion Calarisr
(i) DTLZ2
Design
Design
04
Tuvition Srllaroon
Finetton Tastadonn
Function catioe
(m) Disk Brake Design
(n) Gear Train Design
(0) Rocket Injector Design
Figure 5: The log hypervolume difference w.r.t. the number of expensive evaluation of all algorithms for 15 different problems. The solid line the mean value averaged over 10 independent runs for each algorithm, and the shaded region is the standard deviation around the mean value. The labels of all algorithms can found in Subfigure (a).

---

(a) Fl (3.4 10
(b) x 10-
(c) "LMOP2((.3. 10
DLL 10
(e) Four Bar Thuss(2 x 10
(1 Disk Brake (7.7 10
(g) Gear Thain (2.4 x 10
(b) Rocket Injector (5 x 10 x001)
Figure 6: The Learned Pareto Fronts (Relative Hypervolume Difference) by PSL: Our learned Pareto fronts can match the ground truth Pareto fronts for the synthetic benchmarks, and have small relative hypervolume differences the approximate Pareto fronts for real-world design problems. The learned Pareto front can well represent the optimal trade-offs among different objectives and provide valuable information to support flexible -ecision-making.
Flexible Trade-off Adjustment. With our model, the cecisionnmmkerr can easily explore the whole ap- proximate Pareto set by themselves to select the most preferred trade-off solution(s) shown in Figure 7 No consuming communication between the opti- mization modeler and the decision-make is required. By directly exploring the approximate Pareto front in interactive manner, the decision-makers can observe and understand the connection between the trade-off preferences and corresponding solutions in real-time. It is also beneficial for decision-maker to further adjust and assign their most accurate prefer- ences. The ability to incorporate user's knowledge into decision making [30] could be crucial for many real-world applications. More experimental results and analyses can be found in Appendix
(a) Different Preferences
(b) DTLZ2
Figure 7: Different trade-off preferences and their corresponding solutions on the approxi- mate Pareto set.
Conclusion, limitation and future work
Conclusion. We have proposed a novel Pareto learning method, which is a first attempt to approximate the whole Pareto set for expensive multi-objective optimization. The advantages of this approach are two-fold. First, by learning and utilizing the approximate Pareto set, can serve as an efficient MOBO method that outperforms different existing approaches. Secondly, it allows decision makorrs to readily explore the whole approximate Pareto set, which supports flexible and interactive decision- making. We believe the proposed Pareto set learning method could provide novel way for solve expensive multi-objective optimization.
decision-making.
Limitation and Future Work. The quality of the approximate Pareto set mainly depends on the accuracy of the surrogate models and the performance of the Pareto set learning algorithm, which could be poor for problems with insufficient evaluation budget and/or large-scale search space. A more detailed discussion of limitations and potential future work can be found in Appendix B and potential societal impact can be found in Appendix C
Acknowledgements
This work was supported by the Hong Kong General Research Fund (11208121, CiyY3-9043148)
10

---

References
[1] A. Abdolmaleki, S. Huang, L.. Hasenclever, M. Neunert, F. Song, M. Zambelli, M. Martins, N. Heess, R. Hadsell, and M. Riedmiller. A distributional view multi-objective policy optimization. In International Conference on Machine Learning (ICML), pages 11-22. PMLR, 2020.
[2] A. Abdolmalcki, S. H. Huang, G. Vezzani, B. Shahriari, J. T. Springenberg, S. Mishra, D. TB, A. Byravan, K. Bousmalis, A. Gyorgy, al. On multi-objective policy optimization as tool for reinforcement learning. arXiv preprint arXiv:2 106.08199 2021.
[3] M. Abdolshah, A. Shilton, S. Rana, S. Gupta, and S. Venkatesh. Multi-objective bayesian optimisation with preferences over objectives. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[4] Astudillo and P. Frazier. Multi-attribute bayesian optimization with interactive preference learning. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
[5] M. Balandat, B. Karrer, D. Jiang, S. Daulton, B. Letham, A. G. Wilson, and E. Bakshy, Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in Neural Information Processing Systems (NeurIPS), 2020.
[6] S. Belakaria and A. Deshwal. Max-value entropy search for multi-objective bayesian optimiza- tion. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[7] S. Belakaria, A. Deshwal, N. K. Jayakodi, and R. Doppa. Uncertainty-awar search framework for multi-objective bayesian optimization. In AAAI Conference on Artificial Intelligence (AAAI), 2020.
[8] J. Blank and K. Deb. pymoo: Multi-oojective optimization in python. IEEE Access, 8: 89497-89509, 2020.
[9] S. Boyd and L. Vandenberghe. Convex optimization. Cambridge University Press, 2004.
[10] E. Bradford, A. M. Schweidtmann, and A. Lapkin. Efficient multiobjective optimization employing gaussian processes, spectral sampling and genetic algorithm. Journal of global optimization, 71 2018.
[11] E. Brochu, M. Cora, and N. De Freitas. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv: 2012,259, 2010.
[12] F. Cheng and X. Li. Generalized center method for multiobjective engineering optimization. Engineering Optimization, 31 316666641-61,, 1999.
[13] E. U. Choo and D. Atkins. Proper efficiency in nonconvex multicriteria programming. Mathe- matics of Operations Research, 8(3):467-470) 1983.
[14] S. Daulton, M. Balandat, and E. Bakshy. Differentiable expected hypervolume improvement for parallel multi-objective bayesian optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[15] S. Daulton, M. Balandat, and E. Bakshy. Parallel bayesian optimization of multiple noisy objectives with expected hypervolume improvement. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[16] S. Daulton, S. Cakmak, M. Balandat, M. A. Osborne, E. Zhou, and Bakshy. Robust multi- objective bayesian optimization under input noise. In International Conference Machine Learning (ICML), 2022.
[17] S. Daulton, D. Eriksson, M. Balandat, and E, Bakshy. Multi-objective bayesian optimization over high-dimensional search spaces. Conference on Uncertainty in Artificial Intelligence (UAI), pages 507-517. PMLR, 2022.

---

[18] G. De Ath, R. M. Everson, A. A. Rahat, and J. E. Fieldsend. Greed is good: Exploration and exploitation trade-offs in bayesian optimisation. ACM Transactions on Evolutionary Learning and Optimization, 1(1):1-22, 2021.
[19] K. Deb and A. Srinivasan. Innovization: Innovating design principles through optimization. In Genetic and Evolutionary Computation Conference (GECCO), 2006.
[20] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation, 6(2): 2002.
[21] K. Deb, L. Thiele, M. Laumanns, and E. Zitzler. Scalable multi-objective optimization test problems. In IEEE Congress on Evolutionary Computation (CEC), 2002.
[22] T. Desautels, A. Krause, and J. Burdick. Parallelizing gaussian process bandit optimization. The Journal of Machine Learning Research, 15(1): 3873-3923, 2014.
tradeoffs in
[23] A. Dosovitskiy and J. Djolonga. You only train once: Loss-conditional training of deep networks. International Conference on Learning Representations (ICLR), 2019.
[24] M. Ehrgott. Multicriteriaa optimization, volume 491. Springer Science & Business Media, 2005.
[25] M. Emmerich and Klinkenberg. The computation of the expected improvement in dominated hypervolume of pareto front approximations. Rapport technique, Leiden University, 34, 2008.
[26] M. Emmerich, K. Giannakoglou, and B. Naujoks. Single-and multiobjective evolutionary opti- mization assisted by gaussian random field metamodels. IEEE Transactions on Evolutionary Computation, 10(4):421-439 2006.
[27] Eriksson, M. Pearce, J. Gardner, D. Turner, and M. Poloczek. Scalable global optimiza- tion via local bayesian optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[28] D. Eriksson, P. I.JJ Chuang, S. Daulton, A. Aly, A. Babu, A. Shrivastava, P. Xia, S. Zhao, Venkatesh, and M. Balandat. Latency-aware neural architecture search with -bjei-obeecctive bayesian optimization. arXiv preprint arXivv2106.1 1890, 2021.
[29] P. I. Frazier. A tutorial on bayesian optimization. arXiv preprint
2018.
[30] R. Garnett, Bayesian Optimization. Cambridge University Press, 2022. in preparation.
[31] I. Giagkiozis and P. J. Fleming. Pareto front estimation for decision making. Evolutionary computation, 22(4) 1678, 2014.
[32] R. Homezz-ommarellit J. N. Wei, D. Duvenaud, J. M. Hemmnndez-Lobato B. Sánchez- Lengeling, Sheberla, J. Iparluuuup,, T. D. Hirzel, R. P. Adams, and A. Aspuru- Guzik. Automatic chemical design using data-driven continuous representation of molecules. ACS central science, 4(2))268-227) 2018.
[33] P. Hennig and C. J. Schuler. Entropy search for cetimaatioo--fficienn global optimization. Journal of Machine Learning Research, 13(6), 2012.
[34] D. Hernández-LLobtto, J. Hernandez-Lobato, A. Shah, and R. Adams. Predictive entropy search for multi-objective bayesian optimization. In International Conference Machine Learning (ICML), 2016.
[35] J. M. JdemándezzLobatte M. W. Hoffman, and Z. Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. In Advances in Neural Information Processing Systems (NeurIPS), 2014.
[36] Hillermeier. Generalized homotopy approach to multiobjective optimization. Journal of Optimization Theory and Applications, 110(3):557-883 2001.
12

---

[37] M. W. Hoffman and Z. Ghahramani. Output-space predictive entropy search for flexible global optimization. NeurIPS Workshop on Bayesian Optimization, 2015.
[38] K. M. Jablonka, G. M. Jothiappan, S. Wang, B. Smit, and B. Yoo. Bias free multiobjective active learning for materials design and discovery. Nature Communications, 12(1):1-10, 2021.
[39] D. R. Jones. A taxonomy of global optimization methods based on response surfaces. Journal of global optimization, 21(4):345-38 (3055,3332222 2001.
[40] D. R. Jones, M. Schonlau, and W. J. Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455-43) 1998.
[41] I. Kaliszewski. A modified weighted tchebycheff metric for multiple objective programming. Computers & operations research, 14(4):315 --33,,,,11,111 1987.
[42] K. Kawaguchi, L. P. Kaelbling, and T. Lozano-Pérez. Bayesian optimization with exponential convergence. In Advances in Neural Information Processing Systems (NeurIPS), 2015.
[43] A. J. Keane. Statistical improvement criteria for use in multiobjective design optimization. AIAA journal, 44(4):877-89) 2006.
[44] N. Khan, D. E. Goldberg, and M. Pelikan. Multi-objective bayesian optimization algorithm. In Genetic and Evolutionary Computation Conference (GECCO), pages 684-684. Citeseer, 2002.
[45] J. Knowles. ParEGO: hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 10 (1):50-66, 2006.
[46] S. Kramer. An augmented lagrange multiplier based method for mixed integer discrete continuous optimization and its applications to mechanical design. Journal of Mechanical Design, 116:405, 1994.
[47] J. Kushner. new method of locating the maximum point of arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86(1):97-106, 1964.
[48] M. Laumanns and J. Ocenasek. Bayesian optimization algorithms for multi-objective op- timization. In International Conference on Purallel Problem Solving from Nature (PPSN), 2002.
[49] X. Lin, H.-L. Zhen, Z. Li, Q. Zhang, and S. Kwong. Pareto multi-task learning. In Advances in Neural Information Processing Systems, pages 12060-12070, 2019.
[50] X. Lin, Z. Yang, Q. Zhang, and S. Kwong. Controllable pareto multi-task learning. arXiv 2020.
preprint
[51] X. Lin, Z. Yang, and Q. Zhang. Pareto set learning for neural multi-objective combinatorial optimization. In International Conference on Learning Representations (ICLR), 2022.
[52] D. Lopez-Paz and L. Sagun. Easing non-convex optimization with neural networks. In International Conference on Learning Representations (ICLR) Workshops, 2018.
[53] M. K. Lukovic, Y. Tian, and W. Matusik. Diversity-guided multi-objective bayesian optimiza- tion with batch evaluations. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[54] P. Ma, T. Du, and W. Matusik. Efficient continuous pareto exploration in multi-task learning. International Conference on Machine Learning (ICML), 2020.
[55] D. Mahapatra and V. Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. Shirty-seventh International Conference on Machine Learning, 2020.
[56] G. Malkomes, B. Cheng, E. H. Lee, and M. Mccourt. Beyond the pareto efficient frontier: Constraint active search for multiobjective experimental design. In International Conference Machine Learning (ICML), 2021.
13

---

[57] M. D. McKay, J. Beckman, and W. J. Conover. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 42(1): 55-61, 2000.
[58] K. Miettinen. Nonlinear multiobjectivv optimization. Springer Science & Business Media, 1998.
[59] J. Močkus. On bayesian methods for seeking the extremum. In Optimization techniques IFIP technical conference, pages 400-404. Springer, 1975.
[60] J. Mockus. Bayesian approach global optimization: theory and applications. Kluwer Academic Publishers., 1989.
[61] A. Navon, A. Shamsian, G. Chechik, and E. Fetaya. Learning the pareto front with hypernet- works. In International Conference on Learning Representations (ICLR), 2021.
[62] B. Paria, K. Kandasamy, and B. Póczos. A flexible framework for multi-objective bayesian op- timization using random scalarizations. In Conference on Uncertainty in Artificial Intelligence (UAT), 2020.
[63] S. Parisi, M. Pirotta, and M. Restelli. Multi-objective reinforcement learning through contin- uous pareto manifold approximation. Journal of Artificial Intelligence Research (JAIR), 57: 187-227, 2016.
[64] A. Paszke, S. Gross, F. Massa, A. Lerer. J. Bradbury, G. Chanan, T. Killcen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances Neural Information Processing Systems (NeurlPS), 2019.
[65] M. Pelikan, D. E. Goldberg, E. Cantú-Paz, al. Boa: The bayesian optimization algorithm. In Genetic and Evolutionary Computation Conference (GECCO), 1999.
[66] M. Pirotta, S. Parisi, and M. Restelli. Multi-objective reinforcement learning with continuous pareto frontier approximation. AAAI Conference on Artificial Intelligence (AAAI), 2015.
[67] W. Ponweiser, Wagner, D. Biermann, and M. Vineze. Multiobjective optimization on a limited budget of evaluations using model-assisted s-metric selection. In International Conference on Parallel Problem Solving from Nature (PPSN), 2008.
[68] J. Rakowska, R. T. Haftka, and L. T. Watson. Tracing the efficient curve for multi-objective control-structure optimization. Computing Systems in Engineering, 261-6) 1991.
[69] C. E. Rasmussen and C. K. Williams, Gaussian processes for machine learning. MIT Press, 2006.
[70] T. Ray and K. Liew. A swarm metaphor for multiobjective design optimization. Engineering optimization, 34(2)1141-133, 2002.
[71] F. Rehbach, M. Zaefferer, B. Naujoks, and T. Bartz-Beielstein. Expected improvement versus predicted value in surrogate-based optimization. In Genetic and Evolutionary Computation Conference (GECCO), 2020.
[72] C. Romero. Extended lexicographic goal programming: a unifying approach. Omega, 29(1): 63-71, 2001.
[73] R. Roussel, A. Hanuka, and A. Edelen. Multiobjective bayesian optimization for online accelerator tuning. Physical Review Accelerators and Beams, 24(6):062801, 2021.
[74] M. Ruchte and Grabocka. Scalable pareto front approximation for deep multi-objecttive learning. In IEEE International Conference on Data Mining (ICDM), 2021.
[75] O. Sener and V. Koltun. Multi-task learning as multi-objective optimization. In Advances in Neural Information Processing Systems, pages 525-536. 2018.
[76] Sener and V. Koltun. Learning to guide random search. In International Conference on Learning Representations (ICLR), 2020.
14

---

[77] B. Shahriari, K. Swersky, Z. Wang, R. Adams, and N. De Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148-175, 2016.
[78] A. Shoshan, R. Mechrez, and Deinik-Manon Dynamic-net: Tuning the objective without re-training for synthesis tasks. In /EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
[79] J. Snoek, H. Larochelle, and R. Adams. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems (NeurIPS), 2012.
[80] J. Snock, O. Rippel. K. Swersky, R. Kiros, N. Satish, N. Sundaram, M. Patwary. M. Prabhat, and R. Adams. Scalable bayesian optimization using deep neural networks. In International Conference on Machine Learning (ICML), 2015.
[81] N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In International Conference on Machine Learning (ICMI), 2010.
[82] I. Steponavičé, R. Hyndman, K. Smith-Miles, and L. Villanova. Dynamic algorithm selection for pareto optimal set approximation. Journal of Global Optimization, 67(1):263-282 2017.
[83] Steuer and E.-U. Choo. interactive weighted tchebycheff procedure for multiple objective programming. Mathematical Programming, 26(3):326 344, 1983.
[84] S. Suzuki, S. Takeno, T. Tamura, K. Shitara, and M. Karasuyama. Multi-objective bayesian optimization using pareto-frontier entropy. In International Conference on Machine Learning (ICML), 2020.
[85] R. Tanabe and H. Ishibuchi. An easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing, 89:106078, 2020.
[86] Tripp, E. Daxberger, and M. Hernández-Lobate Sample-efficient optimization in the latent space of deep generative models via weighted retraining. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[87] Vaidyanathan, K. Tucker, N. Papila, and W. Shyy. Cfd-based design optimization for single element rocket injector. In Aerospace Sciences Meeting and Exhibir, 2003.
[88] D. A. Van Veldhuizen and G. B. Lamont. Multiobjective evolutionary algorithm test suites. In CM Symposium on Applied Computing (SAC), 1999.
[89] L. Wang, Fonseca, and Y. Tian. Learning search space partition for black-box optimiza- tion using monte carlo tree search. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[90] Z. Wang and S. Jegelka. Max-value entropy search for efficient bayesian optimization. In International Conference on Machine Learning (ICML), 2017.
[91] 7. Wang, M. Zoghi, F. Hutter, D. Matheson, and N. De Freitas. Bayesian optimization in high dimensions via random embeddings. In International Joint Conferences on Artificial Intelligence (IJCAI), 2013.
[92] Z. Wang, C. Gehring, P. Kohli, and S. Jegelka. Batched large-scale bayesian optimization in high-dimensional spaces. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2018
[93] J. Wilson, F. Hutter, and M. Deisenroth. Maximizing acquisition functions for bayesian optimization. In Advances in Neural Information Processing Systems (NeurIPS), volume 31, 2018.
[94] J. T. Wilson, R. Moriconi, F. Hutter, and M. P. Deisenroth. The reparameterizzatoon trick for acquisition functions. arXiv preprint 1rXiv:1712.00424, 2017.
[95] J. Wu and Frazier. The parallel knowledge gradient method for batch bayesian optimization.
In Advances in Neural Information Processing Systems (NeurlPS), 2016.
15

---

[96] R. Yang. X. Sun, and K. Narasimhan. A generalized algorithm for multi-objectio reinforce- ment learning and policy adaptation. Advances in Neural Information Processing Systems (NeurIPS), 2019.
[97] Q. Zhang and H. Li. MOEA/D: A multiobjective evolutionary algorithm based on decomposi- tion. IEEE Transactions on Evolutionary Computation, 11(6):712-731, 2007.
[98] Q. Zhang, A. Zhou, and Y. Jin. Rm-meda: regularity model-based multiobjective estimation of distribution algorithm. IEEE Transactions on Evolutionary Computation, 12(1):41-63,
2008.
[99] Q. Zhang, W. Liu, E. Tsang, and B. Virginas. Expensive multiobjectiiv optimization by moea/d with gaussian process model. IEEE Transactions on Evolutionary Computation, 14(3): 456 474, 2010.
[100] R. Zhang and D. Golovin. Random hypervolume scalarizations for provable multi-objective black box optimization. In International Conference on Machine Learning (ICML). 2020.
[101] Y. Zhao, L. Wang, K. Yang. T. Zhang, Guo, and Y. Tian. Multi-objective optimization by learning space partitions. In International Conference on Learning Representations (ICLR), 2022.
[102] E. Zitzler, K. Deb, and L. Thiele. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary Computation, 8(2):173-195, 2000.
[103] E. Zitzler, D. Brockhoff, and Thiele. The hypervolume indicator revisited: On the design of pareto-compliant indicators via weighted integration. In International Conference on Evolutionary Multi-Criterion Optimization (EMO), 2007.
[104] M. Zuluaga, G. Sergent, A. Krause, and M. Püschel. Active learning for multi-objectivv optimization. In International Conference Machine Learning (ICML), 2013.
[105] M. Zuluaga, A. Krause, and M. Püschel. E-pal: an active learning approach the multi- objective optimization problem. The Journal of Machine Learning Research, 17(1):66193663 2016.
16

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Appendix
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix [
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material URL)? [Yes] See https://gittuul httpg//aaaa.ooooo/////SL..e Li////XLLLPLLMOOO
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section S and Appendix
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix F1
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [Yes] See Appendix G
(c) Did you include any new assets either in the supplemental material or as a U
(d) Did you discuss whether and how consent was obtained from people whose data you's 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
17


Reviews:
[{"review_id": "xhO79u5oQ8T", "reviewer": "Reviewer_MK65", "summary": "This work proposes a novel method for multi-objective Bayesian optimization (MOBO). The work argues that the proposed method achieves greater sample efficiency and better runtime than existing alternatives. The proposed method leverages the structure of the Pareto set, acknowledging that the Pareto set is typically infinite and the often the result of optimization is a finite approximation. ", "questions": "Why is using NN policy for candidate generation a good idea? What is the benefit?\n  * The NN serves as a way to generate candidates. Why not just optimize the acquisition function under one scalarization (or more for generating a batch)?\n  * Can you provide an ablation study that removes the NN and simply optimizes the acquisition function? The importance of each the two main contributions---1) using augmented Tchebyshev scalarization with an ideal point and epsilon and 2) using a NN to predict the optimal designs---is currently not clear.\n* Regarding batch selection: qParEGO also selects different scalarizations, randomly, for each point in the batch. The proposed batch selection criteria of using HVI (based on the predicted LCB) to select points is not well-motivated. Why not just maximize expected improvement or LCB of HVI? The choice of using HVI for batch selection is a large departure from proposed approach for sequential optimization and is not well-justified\n* This work is missing an introduction to Bayesian optimization, which would make it hard for a non-BO expert to read this paper.\n* Hypervolume improvement is introduced before hypervolume. I recommend switching this ordering (and potentially introducing these concepts earlier)\n* “It would be computationally intensive to find a set to exactly maximize the hypervolume improvement (12). “ This is commonly done in expected hypervolume improvement, albeit with a sequential greedy approximation. I recommend clarifying this statement to state that jointly maximizing B candidates is difficult and therefore sequential greedy selection is typically used (Daulton et al., 2020).\n* The pareto set of designs for the ZDT problems exist in a tiny slice of the search space. This biases performance towards the PSL based approach where a NN could very well predict the optimal designs in they are close to together and at least one has been observed. Furthermore, this makes many improvement-based acquisition function hard to optimize. Therefore, using the proposed epsilon adjustment has outsized impact, but this could be done for any acquisition function. In addition, the initial designs used for acquisition optimization are particularly important as often these will be zero. Hence, acquisition optimization is sensitive to the initialization and techniques such as perturbing the best designs observed so far with a small amount of zero-mean Gaussian noise significant influence optimization performance. This feature if ZDT is particularly important because one contribution of this work is the method for generating candidates using PLS with circumvents the classical acquisition optimization. Heuristics such as perturbing the best points with a small amount of zero-mean Gaussian noise should be compared against to understand this paper's contribution.\n* Appendix, L198: “qEHVI needs an extremely long runtime  to calculate the expected hypervolume improvement for more than 3 objectives at each step.” Daulton et al., 2020 run qEHVI on vehicle safety with 3 objectives and Daulton et al., 2021 run qEHVI and qNEHVI on car side impact with 4 objectives.\n* Why is LaMoo (Zhao et al., 2022) not compared against? \n* Why is qNEHVI (Daulton et al., 2021) not compared against, since is scales better with the batch size than qEHVI?\n* Line 76: “expectation improvement” -> expected improvement\n* Line 126: Figure 1b does not show “our proposed PSL method also learns an estimated Pareto set Mpsl with the predicted Pareto front fˆ (Mpsl ) to approximate the Pareto set Mps and Pareto front f (Mps ) “\n* Line 250: “\\hat{f} are the surrogate values” – it is unclear whether this is the posterior mean or the LCB?\n\n", "limitations": "The paper briefly touches on limitations in the last paragraph, but those limitations are for MOBO in general rather than focusing on this methodology. The paper would benefit from a discussion of worst-case behavior or situations were the method might fail or struggle would be good to include. Negative societal impacts are not discussed.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The problem of multi-objective optimization of expensive function is important. However, many existing works have already tackled this problem, as noted in the paper. Hence, subsequent efforts can at best achieve incremental gains. The method amounts to 1) using the known connection between an augmented TCH scalarization and pareto optimal points, 2) training a neural network select the next candidate designs (these would be believed to be Pareto optimal if the \\hat{f} were set to the posterior mean, but instead this work sets it to a lower confidence bound). \n\nThe paper is generally well-written, but the augmented Chebyshev scalarization is already used by ParEGO with EI. It is unclear why a NN should be used here to select designs (the main novel component). The empirical results show that the proposed methods works quite well, but raises questions. Given that there are no theoretical results, the burden is on the empirical validation and the motivation for the methodology, but both currently fall short of making a compelling case for the utility of this method. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "kc_Rbldz-Pe", "reviewer": "Reviewer_KcfL", "summary": "The paper proposes an algorithm for batch multi-objective Bayesian optimization. The acquisition function uses Chebyshev scalarization. The goal is to solve problems with an infinite size pareto optimal set and find a pareto manifold of solutions. ", "questions": "Several applications have an infinite pareto set of solutions. However, in practice, having access to an approximation of the manifold over the input space in the context of expensive evaluations, might not be necessarily useful. Given the cost of the objectives, the budget might be exhausted by the time a good approximation of the manifold is built. At that point of the optimization process, access to the pareto set manifold would not be useful without access to output values. Also relying on only the approximate input space manifold can be very risky in several applications given the uncertainty of the prediction.", "limitations": "Yes the authors discussed limitations briefly. I suggest including a discussion about the challenges in defining the preference over objectives in the context of black-box functions", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n+The problem setting is interesting. Continuous pareto set There are several real-world problems with a manifold of solutions. The Idea of creating a projection of the manifold based on the preference defined by the practitioner is appealing.\n+ the approach is well structured and novel \n+ The paper is well written, and the proposed solution is clear.\n+ The paper covered lots of baselines on several synthetic and real-world problems \n\nWeaknesses: \n\n+ The practicality of accessing the pareto set manifold in the context of expensive experiments might be questionable.\n+ Defining user preference in terms of scalars is challenging in practice and there is no clear way on how to do it\n+ The experiment comparison is missing an important relevant algorithm from the preference over objectives literature [1]\n+ The contribution of the batch selection is minor and incremental\n\n[1] Abdolshah, Majid, et al. \"Multi-objective Bayesian optimization with preferences over objectives.\" Neurips (2019).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "MV1fZT-qbRF", "reviewer": "Reviewer_HJkG", "summary": "This paper presents a new Pareto set learning method to approximate the whole Pareto set for expensive multi-objective Bayesian optimization. it first generated a model that maps any trade-off preferences to their corresponding Pareto solutions with scalarization, then uses a batched acquisition search with the learned Pareto set model. ", "questions": "The setting of the test problems is strange. The experiments on other test problems from the ZDT and DTLZ suites need to be added.", "limitations": "The impact of the aggregation function on the algorithm needs to be analyzed.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The expensive multi-objective Bayesian optimization problems studied in this paper are a long-standing concern in the field of Bayesian optimization, and this paper presents a new algorithm for this. The main problem is that all the problems tested in the experiments have been proven to be quite simple. Moreover, the dimensionality of the test problem does not exceed 6. These raise concerns about the effectiveness of the algorithm proposed in this paper because the tested problems are too simple.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "E7WSLk2kVA-", "reviewer": "Reviewer_YcbD", "summary": "In the context of expensive multi-objective black-box optimization, this article proposes to construct a surrogate model of the Pareto set. It is a neural network built upon surrogate models of the objectives, here Gaussian processes. The interest is first to help decision makers to navigate the estimated Pareto set and front. Then it is useful for selecting batches on candidates for sequential optimization, using the existing expected hypervolume improvement. The method is benchmarked over different classical test functions, against appropriate baselines.", "questions": "Could you show the learned Pareto front and set on a case where they are disconnected?\nAppendix line 57: Is it still Expected improvement in this form?\nAppendix C5: could you define the what ''uncertainty'' means here?", "limitations": "Limitations are discussed.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- the method integrates the structure of the Pareto set to help optimize the qEHVI criterion;\n- the Pareto set model is useful for working with practitioners. The method to build it with backpropagation is original.\n- the results are convincing in terms of speed and sample efficiency.\n\nWeaknesses:\n- models for the Pareto sets have been proposed in the multi-objective literature.\n- it uses the qEHVI criterion.\n- the behavior on disconnected Pareto front is not shown.\n- a few aspects could be clarified, see Questions below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept