Original: Multi-Objective Bayesian Optimization with Pareto Set Learning 
Revised: Pareto Set Learning for Expensive Multi-Objective Optimization

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Expensive multi-objective optimization problems can be found in many real-worldapplications, where their objective function evaluations involve expensive computation and/or physical experiments.
Revised: Expensive multi-objective optimization problems can be found in many real-world applications, where their objective function evaluations involve expensive computations or physical experiments.

Original: It is desirable to obtain an approximate Pareto front with a small evaluation budget.
Revised: It is desirable to obtain an approximate Pareto front with a limited evaluation budget.

Original: Multi-objective Bayesian optimization(MOBO) has been widely used for ﬁnding a ﬁnite set of Pareto optimal solutionsto this problem.
Revised: Multi-objective Bayesian optimization (MOBO) has been widely used for ﬁnding a ﬁnite set of Pareto optimal solutions.

Original: However, it is well-known that the whole Pareto set could haveinﬁnite optimal solutions.
Revised: However, it is well-known that the whole Pareto set is on a continuous manifold and can contain inﬁnite solutions.

Original: The structural properties of the Pareto set are not wellutilized in existing MOBO methods, and the ﬁnite-set approximation may notcontain the most preferred solution(s) for decision-makers.
Revised: The structural properties of the Pareto set are not well exploited in existing MOBO methods, and the ﬁnite-set approximation may not contain the most preferred solution(s) for decision-makers.

Original: This paper develops asimple yet efﬁcient Pareto set learning method to approximate the whole Paretoset for MOBO.
Revised: This paper develops a novel learning-based method to approximate the whole Pareto set for MOBO, which generalizes the decomposition-based multi-objective optimization algorithm (MOEA/D) from ﬁnite populations to models.

Original: We design a simple yet powerful acquisition search method withthe learned Pareto set, which naturally supports batch evaluation.
Revised: We design a simple and powerful acquisition search method based on the learned Pareto set, which naturally supports batch evaluation.

Original: In addition,with our proposed model, decision-makers can readily explore any trade-off areain the approximate Pareto set for ﬂexible decision-making.
Revised: In addition, with our proposed model, decision-makers can readily explore any trade-off area in the approximate Pareto set for ﬂexible decision-making.

Revised: This work represents the ﬁrst attempt to model the Pareto set for expensive multi-objective optimization.

Original: Experimental resultson different synthetic and real-world problems demonstrate the effectivenessandefﬁciency of our proposed method.
Revised: Experimental results on different synthetic and real-world problems demonstrate the effectiveness of our proposed method.

Original:  Introduction
Revised:  Introduction

Original: [Figure]
Revised: [Equation]

Original: mate the whole Pareto set.
Revised: ParEGO [Equation] Dominated

Original: With our proposed model, decision-makers can easily explore any trade-off among the objectives to choose their most preferred solutions.
Revised: [Equation] (a) ParEGO f 1 ( x ) 0.2 0.4 0.6 0.8 1. f 2 ( x ) 0. [Figure]

Original: Many real-world applications involve optimizing multiple costly-to-evaluate and potentially competingobjectives, such as ﬁnding strong yet ductile material [34], building a neural network with high accuracy and low latency [24], and improving the qualitywhile minimizing total charge in particle accelerator tuning [66].
Revised: Many real-world applications involve optimizing multiple costly-to-evaluate and potentially competing objectives, such as ﬁnding strong yet ductile material [38], building a neural network with high accuracy and low latency [28], and improving the quality while minimizing total charge in particle accelerator tuning [73].

Original: Very often, these objectives couldconﬂict with each other and cannot be optimized simultaneously by a single solution.
Revised: Very often, these objectives conﬂict each other and cannot be optimized simultaneously by a single solution.

Original: Instead, there isa set of solutions with different optimal trade-offsamong the objectives, called the Pareto set.
Revised: Instead, there is a set of solutions with different optimal trade-offs among the objectives, called the Pareto set.

Original: For a Pareto optimal solution, none of its objective valuescan be further improved without deteriorating others.
Revised: For a Pareto optimal solution, none of its objective values can be further improved without deteriorating others.

Original: In addition, the evaluation of each solution requirestime-consuming computation or costly physical experiment, and thus a large number of evaluations arenot acceptable.
Revised: In addition, the evaluation of each solution could require time-consuming computation or costly physical experiments, and thus a large number of evaluations are unbearable.

Original: Different multi-objective Bayesianoptimization (MOBO) algorithms [40, 44, 39], typically directly generalized from the single-objective
Revised: Different multi-objective Bayesian optimization (MOBO) algorithms [44, 48, 43], typically directly generalized from the single-objective

Original: Bayesian optimization (BO) [53, 35, 9, 68, 25], have been proposed to ﬁnd a small set of approximate
Revised: Bayesian optimization (BO) [60, 39, 11, 77, 29], have been proposed to ﬁnd a small set of approximate

Original: Pareto solutions with a limited evaluation budget.
Revised: Pareto solutions with a small amount of objective function evaluation budget.

Original: However, the ﬁnite set approximation has  undesirable drawbacks.
Revised: The ﬁnite set approximation has some undesirable drawbacks.

Original: For a nontrivial multi-objectiveoptimization problem, the Pareto set is on a continuous manifold and has inﬁnite solutions withdifferent optimal trade-offs among the objectives [51].
Revised: For a nontrivial multi-objective optimization problem, the Pareto set is on a continuous manifold and has inﬁnite solutions with different optimal trade-offs among the objectives [58].

Original: This Pareto set structure could be helpfulto better select candidate solutions for expensive evaluation, and hence accelerate the optimizationprocess of MOBO.
Revised: This Pareto set structure could be helpful to better select candidate solutions for expensive evaluation, and hence accelerate the optimization process of MOBO.

Original: In addition, a small set of solutions may not contain the one(s) that precisely satisfythe decision-maker ’s preferences.
Revised: In addition, a small set of solutions may not contain the one(s) that exactly satisfy the decision-maker’s preferences.

Original: Finding the most preferred trade-off solution(s) could requireseveral rounds of communication with the decision-makers.
Revised: Finding the most preferred trade-off solution(s) could require several rounds of interaction with the decision-makers.

Original: These approaches would be extremelytime-consuming and undesirable, especially when the optimization modeler and the decision-makerare not in the same team, which is common in real-world MOBO applications [49].
Revised: These approaches would be extremely time-consuming, especially when the optimization modeler and the decision-maker are not in the same team, which is common in real-world MOBO applications [56].

Original: In this work, we propose a novel Pareto set learning (PSL) method to approximate the whole Paretoset for expensive multi-objective optimization problems.
Revised: This paper proposes a novel Pareto set learning (PSL) method to approximate the whole Pareto set for expensive multi-objective optimization problems with a limited evaluation budget.

Original: Our proposed method can accelerate themulti-objective Bayesian optimization process, and also provide decision-makers with more usefulinformation to support ﬂexible decision-making.
Revised: Our proposed method can accelerate the multi-objective Bayesian optimization process, and also provide decision-makers with more useful information to support ﬂexible decision-making.

Original: We believe it is a principled way to deal with MOBO.
Revised: To the best of our knowledge, this is the ﬁrst attempt to learn the whole Pareto set for expensive multi-objective optimization.

Original: This work has the following main contributions:
Revised: Our main contributions include:

Original: • We propose a novel set model to map any trade-off preference to its corresponding Pareto solution, along with a surrogate model-based method to approximate the whole Pareto set with a limited evaluation budget.
Revised: • We propose a novel set model to map any trade-off preference to its corresponding Pareto solution, along with a surrogate model-based method to approximate the whole Pareto set with a limited evaluation budget.

Original: • We develop a lightweight yet powerful batch acquisition search methodon the approximate Pareto set for efﬁcient MOBO, which can outperform other MOBO approaches in bothperformance and runtime.
Revised: • We develop a lightweight yet powerful batch acquisition search method for efﬁcient MOBO, which can outperform other MOBO approaches in terms of both performance and computational cost.

Original: Wealso demonstrate that the learned Pareto set can support ﬂexible user-involved decision-making.
Revised: We demonstrate that the learned Pareto set can support ﬂexible user-involved decision-making.

Original: • We test our proposed method on both synthetic benchmarks and real-world application problems.
Revised: • We test our proposed method on both synthetic benchmarks and real-world application problems.

Original: The resultsfully validate the efﬁciency and usefulness of PSL.
Revised: The results validate the efﬁciency and usefulness of PSL.

Original:  Related work
Revised:  Related work

Original: Bayesian Optimization.
Revised: Bayesian Optimization.

Original: Surrogate model-based methods have been widely  studied for expensiveoptimization [43, 36, 58, 70].
Revised: Surrogate model-based methods have been widely used and studied for expensive optimization [47, 40, 65, 79].

Original: This method iteratively builds a surrogate model to approximatethe black-box objective function, and uses an acquisition function to search  optimal solution.
Revised: These methods iteratively build a surrogate model to approximate the black-box objective function, and uses an acquisition function to search for the optimal solution.

Original: Dueto its importance in machine learning, much effort has been made on various design issues in Bayesian optimization, such as acquisition functions [72], high-dimensional optimization [82], batchevaluation [20], scalable optimization [71], and theoretical analysis [38].
Revised: Much effort has been made on various design issues in Bayesian optimization, such as acquisition functions [81], high-dimensional optimization [91, 92], batch evaluation [22], scalable optimization [80, 27], and theoretical analysis [42].

Original: Most of them are onsingle-objective optimization.
Revised: Most work for BO are on single-objective optimization.

Original: We refer readers to Garnett [26] for a comprehensive introduction.
Revised: We refer readers to Garnett [30] for a comprehensive introduction.

Original: Multi-Objective Bayesian Optimization.
Revised: Multi-Objective Bayesian Optimization.

Original: MOBO extends single-objective Bayesian optimization for solving expensive multi-objective optimization problems.
Revised: MOBO extends single-objective Bayesian optimization for solving expensive multi-objective optimization problems.

Revised: Although the Pareto set could contain inﬁnite solutions, the MOBO methods typically focus on ﬁnding a single or a ﬁnite set of solutions.

Original: The scalarization-based algorithms, such as ParEGO [41] and TS-TCH [55] iteratively scalarize the multi-objective probleminto single-objective ones with random preferences, and then apply single-objective BO to solvethem.
Revised: The scalarization-based algorithms, such as ParEGO [45] and TS-TCH [62] iteratively scalarize the multi-objective problem into single-objective ones with random preferences, and then apply single-objective BO to solve them.

Original: MOEA/D-EGO [87] extend the MOEA/D framework [85] to solve a set of surrogate scalarizedsubproblems simultaneously.
Revised: MOEA/D-EGO [99] adopts the MOEA/D framework [97] to solve a set of surrogate scalarized subproblems simultaneously.

Original: SMS-EGO [60] and PAL [92, 93] generalize the upper conﬁdencebound (UCB) to multi-objective optimization.
Revised: SMS-EGO [67] and PAL [104, 105] generalize the upper conﬁdence bound (UCB) to multi-objective optimization.

Original: Emmerich et al.
Revised: Emmerich et al.

Original: [23] and Emmerich and Klinkenberg[22] propose the probability of improvement (PI) and expected improvement (EI) for multi-objectivehypervolume.
Revised: [26] and Emmerich and Klinkenberg [25] propose the probability of improvement (PI) and expected improvement (EI) for multi-objective hypervolume.

Original: Entropy search methods [29, 31, 33, 81] have also been explored in multi-objectiveoptimization [30, 4, 75].
Revised: Entropy search methods [33, 35, 37, 90] have also been studied in multi-objective optimization [34, 6, 84].

Original: Bradford et al.
Revised: Bradford et al.

Original: [8] and Belakaria et al.
Revised: [10] and Belakaria et al.

Original: [5] consider Thompson sampling anduncertainty maximization for multi-objective optimization, respectively.
Revised: [7] consider Thompson sampling and uncertainty maximization for multi-objective optimization, respectively.

Original: Different algorithms canalso be efﬁciently combined to achieve better overall performance [73].
Revised: Different algorithms can be hybridized to achieve better performances [82].

Original: The current works have investigated the  diverse batch evaluation [47], efﬁcient hypervolume improvement calculation[13], multiple noisy objectives [14],  and decision criteria beyond Paretooptimality [49].
Revised: Different new developments have been recently proposed to handle the issues of diverse batch evaluation [53], efﬁcient hypervolume improvement calculation [14], noisy optimization [15, 16], high-dimensional optimization [17], and decision criteria beyond Pareto optimality [56].

Original: Some attempts have been made to incorporate the decision-maker’s preference into MOBO [1, 55, 2].
Revised: Some attempts have been made to incorporate the decision-maker’s preference into MOBO [3, 62, 4].

Original: They typically need  decision-maker’s preference before or during optimization,which could be unavailable inmany real-world applications.
Revised: They typically need the decision-maker’s preference before or during optimization, which may not always be available in real-world applications.

Revised: All these MOBO methods aim to provide a ﬁnite set of approximate Pareto optimal solutions to decision-makers.

Revised: [Figure]

Revised: The weakly Pareto optimal but not Pareto optimal solutions (e.g., purple points are dominated but not strictly dominated by at least one Pareto solution weakly front weak the image of all weakly

Original: Structure Learning.
Revised: Structure Learning.

Original: In addition to the surrogate objective model, some attempts have been made toexplore the problem structure for Bayesian optimization.
Revised: In addition to the surrogate objective model, some methods have been proposed to explore the problem structure for Bayesian optimization.

Original: Sener and Koltun [67] learn the geometricstructure of the problem in an online manner to accelerate optimization.
Revised: Sener and Koltun [76] learn the geometric structure of the problem in an online manner to accelerate optimization.

Original: Wang et al.
Revised: Wang et al.

Original: [80] and Zhaoet al.
Revised: [89] and Zhao et al.

Original: [89] apply Monte Carlo tree search (MCTS) to divide the search space for efﬁcient modeling [Figure] [Figure]
Revised: [101] apply Monte Carlo tree search (MCTS) to divide the search space for efﬁcient modeling and searching.

Original: f
Revised: Novel latent space modelings [32, 86] have been proposed for optimization problems with complex solution representations. From Population to Pareto Set Learning.

Original: [Figure] [Figure] x f
Revised: For the last several decades, most multi-objective optimization methods have focused on ﬁnding a single or a ﬁnite set of Pareto optimal solutions (e.g., a population) to approximate the Pareto set [58, 24].

Original: [Figure] The idea to approximate the whole Pareto set manifold has been studied in the multi-objectiveoptimization community [61, 32, 27].
Revised: A few attempts have been made to approximate the whole Pareto set with simple mathematical models [68, 36, 98, 31].

Original: Pirotta et al.
Revised: Pirotta et al.

Original: [59] and Parisi et al.
Revised: [66] and Parisi et al.

Original: [56] have explored the Paretomanifold approximationmethod for multi-objective reinforcement learning.
Revised: [63] have proposed to conduct Pareto manifold approximation for multi-objective reinforcement learning.

Original: Apart from optimization,different approaches have  been investigated to incorporate the preference information into deep neuralnetworks for image style transfer [69, 21] and multi-task learning [48, 54, 45].
Revised: Recently, different approaches have also been investigated to incorporate the preference information into deep neural networks for image style transfer [78, 23], multi-task learning with ﬁnite solutions [75, 49, 55, 54] or approximate Pareto front [50, 61, 74], reinforcement learning [96, 1, 2], and neural combinatorial optimization [51].

Original: In this work, wepropose to learn a set model which maps all valid trade-off preferences to theapproximate Pareto set,and use it for efﬁcient MOBO.
Revised: In this work, we generalize the decomposition-based multi-objective optimization algorithm (MOEA/D) [97], and propose to learn a set model which maps all valid trade-off preferences to the Pareto set for efﬁcient MOBO.

Original:  Expensive multi-objective optimization
Revised:  Expensive multi-objective optimization

Original: We consider the following expensive continuous multi-objective optimization problem:
Revised: We consider the following expensive continuous multi-objective optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where x is a solution in the decision space
Revised: where x is a solution in the decision space

Original: X ⊂ R n , f : X → R m is an m -dimensional vector-valuedobjective function, and the evaluation is expensive for all individual objectives{ f i ( x ) } mi =1 .
Revised: X ⊂ R n , f : X → R m is an m -dimensional vector-valued objective function, and the evaluation is expensive for all individual objectives f i ( x ) , i = 1 , . . . , m .

Original: For anon-trivial problem, no singlebest solution can optimize all objectives at the same time, and we haveto make a trade-off among them.
Revised: For a non-trivial problem, no single solution can optimize all objectives at the same time, and we have to make a trade-off among them.

Original: We have the following deﬁnitions for multi-objective optimization:
Revised: We have the following deﬁnitions for multi-objective optimization:

Original: Deﬁnition 1 (Pareto Dominance and Strict Dominance) Let x a , x b ∈ X , x a is said to dominatex b , denoted as x a ≺ x b , if and only if f i ( x a ) ≤ f i ( x b ) , ∀ i ∈ { 1 , ..., m } and ∃ j ∈ { 1 , ..., m } suchthat f j ( x a ) <
Revised: Deﬁnition 1 (Pareto Dominance and Strict Dominance) Let x a , x b ∈ X , x a is said to dominate x b , denoted as x a ≺ x b , if and only if f i ( x a ) ≤ f i ( x b ) , ∀ i ∈ { 1 , ..., m } and ∃ j ∈ { 1 , ..., m } such that f j ( x a ) <

Original: f j ( x ( b ) ) .
Revised: f j ( x ( b ) ) .

Original: In addition, x a is said to strictly dominate x b ( x a ≺ strict x b ), if and only iff i ( x a ) <
Revised: In addition, x a is said to strictly dominate x b ( x a ≺ strict x b ), if and only if f i ( x a ) <

Original: f i ( x b ) , ∀ i ∈ { 1 , ..., m } .
Revised: f i ( x b ) , ∀ i ∈ { 1 , ..., m } .

Original: Deﬁnition 2 (Pareto Optimality) A solution x ∗ ∈ X is Pareto optimal if there is no ˆ x ∈ X suchthat ˆ x ≺ x ∗ .
Revised: Deﬁnition 2 (Pareto Optimality) A solution x ∗ ∈ X is Pareto optimal if there is no ˆ x ∈ X such that ˆ x ≺ x ∗ .

Original: A solution x (cid:48) ∈
Revised: A solution x (cid:48) ∈

Original: X is weakly Pareto optimal if there is no ˆ x ∈ X such that ˆ x ≺ strict x (cid:48) .
Revised: X is weakly Pareto optimal if there is no ˆ x ∈ X such that ˆ x ≺ strict x (cid:48) .

Original: Deﬁnition 3 (Pareto Set/Front) The set of all Pareto optimal solutions M ps ⊆
Revised: Deﬁnition 3 (Pareto Set/Front) The set of all Pareto optimal solutions M ps ⊆

Original: X is called the
Revised: X is called the

Original: Pareto set, and its image in the objective space f ( M ps )
Revised: Pareto set, and its image in the objective space f ( M ps )

Original: =
Revised: =

Original: { f ( x ) | x ∈ M ps } is called the Paretofront.
Revised: { f ( x ) | x ∈ M ps } is called the Pareto front.

Original: Similarly, we can deﬁne the weakly Pareto set M weak and weakly Pareto front f ( M weak ) .
Revised: Similarly, we can deﬁne the weakly Pareto set M weak and weakly Pareto front f ( M weak ) .

Original: The strict dominance relation is stronger than the Pareto dominance since it requires strictly bettervalues for all objectives.
Revised: The strict dominance relation is stronger than the Pareto dominance since it requires strictly better values for all objectives.

Original: Therefore, the set of weakly Pareto optimal solutions M weak (e.g., thesolutions that are not strictly dominated) would be larger than M ps , and it is straightforward to see
Revised: Therefore, the set of weakly Pareto optimal solutions M weak (e.g., the solutions that are not strictly dominated) would be larger than M ps , and it is straightforward to see

Original: M ps ⊆ M weak .
Revised: M ps ⊆ M weak .

Original: The illustration of (weakly) Pareto solution and Pareto front is shown in Figure 2.
Revised: The illustration of (weakly) Pareto solution and Pareto front is shown in Figure 2.

Original: Each Pareto solution x ∈ M ps represents a different optimal trade-off among the objectives forproblem (1).
Revised: Each Pareto solution x ∈ M ps represents a different optimal trade-off among the objectives for problem (1).

Original: Under mild conditions, the Pareto set M ps and Pareto front f ( M ps ) are both on ( m − 1) - dimensional manifold in the decision space X ∈ R n and objective space R m , respectively [51, 32, 86].
Revised: Under mild conditions, the Pareto set M ps and Pareto front f ( M ps ) are both on ( m − 1) - dimensional manifold in the decision space X ∈ R n and objective space R m , respectively [36, 98].

Original: The number of Pareto solutions could be inﬁnite ( |M ps | = ∞ ).
Revised: The number of Pareto solutions could be inﬁnite (i.e. |M ps | = ∞ ).

Original: Bayesian Optimization (BO)[26] is a model-based method for solving expensive black-box optimization problems.
Revised: Bayesian Optimization (BO) is a model-based method for solving expensive black-box optimization problems.

Original: Given a set of already-evaluated solutions { X , y } , BO builds surrogate models(e.g., Gaussian process) for each objective, and deﬁnes acquisition function(s) to leverage the surrogate objective values for navigating the search space.
Revised: Given a set of already-evaluated solutions { X , y } , BO builds surrogate models (e.g., Gaussian process) for each objective, and deﬁnes acquisition function(s) to leverage the surrogate objective values for navigating the search space.

Original: Only promising solutions will be selected forexpensive evaluation.
Revised: Only promising solutions will be selected for expensive evaluation.

Original: We refer interesting reader to [26] for a detailed introduction.
Revised: We refer interesting reader to [30] for a detailed introduction.

Original: Pareto Set Learning.
Revised: Pareto Set Learning.

Original: The current MOBO methods aim to ﬁnd a small set of ﬁnite solutions
Revised: The current MOBO methods aim to ﬁnd a small set of ﬁnite solutions

Original: S =
Revised: S =

Original: { ¯ x (1) , ¯ x (2) , · · · , ¯ x ( |S| ) } to approximate the Pareto set M ps .
Revised: { ¯ x (1) , ¯ x (2) , · · · , ¯ x ( |S| ) } to approximate the Pareto set M ps .

Original: In addition to the evaluatedsolutions S , our proposed PSL method also learns an estimated Pareto set M psl with the predicted Pareto front ˆ f ( M psl ) to approximate the Pareto set M ps and Pareto front f ( M ps ) .
Revised: In addition to the evaluated solutions S , our proposed Pareto set learning (PSL) method also learns an estimated Pareto set M psl with the predicted Pareto front ˆ f ( M psl ) to approximate the Pareto set M ps and Pareto front f ( M ps ) .

Original: The wholeapproximate Pareto set can be easily explored by adjusting the trade-off preference as illustrated in Figure 3.
Revised: The whole approximate Pareto set can be easily explored by adjusting the trade-off preference as illustrated in Figure 3.

Original: With the learned Pareto set, we also develop an efﬁcient batched solution selection approachfor efﬁcient MOBO.
Revised: With the learned Pareto set, we also develop an efﬁcient batched solution selection approach for efﬁcient MOBO, which will be introduced in the next section.

Original:  Pareto Set Learning for MOBO 
Revised:  Pareto Set Learning for MOBO 4.1 Pareto set model

Original: [Figure]
Revised: [Equation]

Original: [Table]{+0.8 1.0 x0.0 0.2 0.4 0.6 0. x0.0 0. PreferenceApproximate Pareto Set (a) Preference to PS f 1 ( x ) 0.2 0.4 0.
 0.8 1.0 f 2 ( x ) 0.20.40.60.81.+}
Revised: {+0.8 1.0 x0.0 0.2 0.4 0.6 0. x0.0 0. PreferenceApproximate Pareto Set (a) Preference to PS f 1 ( x ) 0.2 0.4 0.
 0.8 1.0 f 2 ( x ) 0.20.40.60.81.+}

Original: [Figure]
Revised: f 3 ( x ) 0.4 0.

Original: [Table]
Revised: PreferenceApproximate Pareto Front

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: Preference

Original: [Figure]
Revised: [Figure]

Original:  As discussed in Section 3, the Pareto set cancontain inﬁnite solutions with different tradeoffs which are non-dominated with each other.
Revised: PF As pointed out in Section 3, the Pareto set can contain inﬁnite solutions with different tradeoffs.

Original: In other words, there is no clear order and rankamong the Pareto solutions.
Revised: In addition, there is no complete order among the Pareto solutions.

Original: A Pareto set modelfor MOBO should be powerful enough to approximate the whole Pareto set, and convenientenough to easily explore any trade-off solutions.
Revised: A Pareto set model for MOBO should be powerful enough to approximate the whole Pareto set, and convenient enough to easily explore any trade-off solutions.

Original: In this work, we propose to build a  model thatmaps any trade-off preferences to their corresponding Pareto solutions with scalarization.
Revised: In this work, we propose to build a set model that maps any trade-off preferences to their corresponding Pareto solutions with scalarization.

Original: Scalarization.
Revised: Scalarization.

Original: The scalarization method provides a natural connection from a set ofwellorganized preferences Λ = { λ ∈ R m + | (cid:80) λ i =1 } among the m optimization objective to the Pareto set M ps .
Revised: The scalarization method provides a natural connection from a set of preferences Λ = { λ ∈ R m + | (cid:80) λ i = 1 } among the m objectives to the Pareto set M ps .

Original: The most simple and straightforward approach is thelinear weight-sum scalarization:
Revised: The most simple and straightforward approach is the weight-sum scalarization:

Original: [Equation]
Revised: [Equation]

Original: However, this method can only ﬁnd the convexhull of Pareto front f ( M ps ) [12].
Revised: However, this method can only ﬁnd the convex hull of Pareto front f ( M ps ) [9, 24].

Original: In this work, we focus on the  weighted Tchebycheff approach:
Revised: In this work, we use the following weighted Tchebycheff approach:

Original: [Equation]
Revised: [Equation]

Original: (3) where z ∗ = ( z ∗ 1 , · · · , z ∗ m ) is the ideal vector for the objective vector f ( x ) (lower-bound for minimization problem), ε > 0 is a small positive scalar, and u i =
Revised: where z ∗ = ( z ∗ 1 , · · · , z ∗ m ) is the ideal vector for the objective vector f ( x ) (i.e. lower-bound for minimization problem), ε > 0 is a small positive scalar, and u i =

Original: ( z ∗ i − ε ) is called an (unachievable)utopia value for the i -th objective f
Revised: ( z ∗ i − ε ) is called an (unachievable) utopia value for the i -th objective f

Original: i ( x ) .
Revised: i ( x ) .

Original: This scalarization method has a promising property:
Revised: This scalarization method has a promising property:

Original: Theorem 1 (Choo and Atkins [11]).
Revised: Theorem 1 (Choo and Atkins [13]).

Original: A feasible solution x ∈ X is weakly Pareto optimal if and onlyif there is a weight vector λ > [-
-] such that x is an optimal solution of the problem (3).
Revised: A feasible solution x ∈ X is weakly Pareto optimal if and only if there is a weight vector λ > [-
-]0 such that x is an optimal solution of the problem (3).

Revised: [Figure]

Original: In other words, all Pareto solutions x ∈ M ps can be found by solving the Tchebycheff scalarizedsubproblem (3) with a speciﬁc (but unknown) trade-off preference λ .
Revised: In other words, all Pareto solutions x ∈ M ps can be found by solving the Tchebycheff scalarized subproblem (3) with a speciﬁc (but unknown) trade-off preference λ .

Original: We let M tch be the solutionset for problem (3) with all valid preferences Λ and have M ps ⊆ M weak =
Revised: We let M tch be the solution set for problem (3) with all valid preferences Λ and have M ps ⊆ M weak =

Original: M tch .
Revised: M tch .

Original: The weakly Pareto optimal but not Pareto optimal solutions (e.g., M weak \ M ps ) are dominated (but not strictlydominated) by some Pareto solutions, and are usually not desirable for decision-making.
Revised: The weakly Pareto optimal but not Pareto optimal solutions ( M weak \M ps ) are dominated (but not strictly dominated) by some Pareto solutions, and are usually not desirable for decision-making.

Original: They can be[Figure] further avoided by the augmented Tchebycheff approach [74, 37].
Revised: They can be further avoided by the augmented Tchebycheff approach [83, 41].

Original: In this work, we use the followingscalarization:
Revised: In this work, we use the following scalarization:

Original: [Equation]
Revised: [Equation]

Original: where ρ is a sufﬁciently small positive scalar depends on the problem and current solution location.
Revised: where ρ is a sufﬁciently small positive scalar depends on the problem and current solution location.

Original: This form of augmentation has also been used in ParEGO [41].
Revised: This form of augmentation has also been used in ParEGO [45].

Original: With the augmentation term, theweakly dominated solutions will have larger scalarized values than the corresponding Pareto solutionsin (4), and will ultimately be eliminated with the optimization process (e.g., M tch_aug =
Revised: With the augmentation term, the weakly dominated solutions will have larger scalarized values than the corresponding Pareto solutions in (4), and will ultimately be eliminated with the optimization process (e.g., M tch_aug =

Original: M ps ).
Revised: M ps ).

Original: In thiswork, we simply set ρ = 0 .
Revised: In this work, we simply set ρ = 0 .

Original: 001 , dynamically update z ∗ i as the current best value for each objectiveand let ε = 0 .
Revised: 001 , dynamically update z ∗ i as the current best value for each objective and let ε = 0 .

Original: | z ∗ | .
Revised: | z ∗ | .

Original: This setting is robust for all problems we considered.
Revised: This setting is robust for all problems we considered.

Revised: The traditional methods focus on solving the scalarization problem (4) with a ﬁnite set of different preferences λ in a sequential [45] or collaborative manner [97, 99].

Original: Set Model.
Revised: Set Model.

Original: With augmented Tchebycheff scalarization, we  build a set model for mapping preferencesto their solutions:[Equation]
Revised: With augmented Tchebycheff scalarization, we propose to build a set model for mapping preferences to their solutions:

Original: [Equation]
Revised: [Equation]

Original: where λ is any valid preference in Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , x
Revised: where λ is any valid preference in Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , x

Original: ( λ ) ∈
Revised: ( λ ) ∈

Original: X is its corresponding Pareto solution, and h θ ( λ ) is the Pareto set model with parameter θ .
Revised: X is its corresponding Pareto solution, and h θ ( λ ) is the Pareto set model with parameter θ .

Original: Since all valid preferencesλ has ( m − 1 ) degree of freedom,  the  setof all generated solutions M psl = { x = h θ ( λ ) | λ ∈ Λ }is on an ( m − 1 )-dimensional manifold in X ∈ R n .
Revised: The input preference λ has ( m − 1 ) degree of freedom, and the output solution set M psl = { x = h θ ( λ ) | λ ∈ Λ } is on an ( m − 1 )- dimensional manifold in X ∈ R n .

Original: In other words, the set model maps the( m − 1 )-dimensional regular preference simplex Λ to the ( m − 1 )-dimensional solution set M pslwith complicated structure.
Revised: In other words, the set model maps the ( m − 1 )-dimensional regular preference simplex Λ to the ( m − 1 )-dimensional solution set M psl with complicated structure.

Original: We want to ﬁnd the optimal parameters θ ∗ such that the generated set M psl matches the  augmented Tchebycheff scalarization’s set M tch_aug
Revised: We want to ﬁnd the optimal parameters θ ∗ such that the generated set M psl matches the solution set for augmented Tchebycheff scalarization M tch_aug

Original: = { x ∗ ( λ ) | λ ∈ Λ } , where
Revised: = { x ∗ ( λ ) | λ ∈ Λ } , where

Original: [Equation]
Revised: [Equation]

Original: An illustration of the learned mapping is shown in Figure 3.
Revised: The learned mapping is illustrated in Figure 3.

Original: Once the connection is learned, we canefﬁciently navigate the whole approximate Pareto set/front by simply adjusting the preferences amongobjectives.
Revised: Once the connection is learned, we can explore the whole approximate Pareto set/front by simply adjusting the preferences among objectives.

Original: In this work, we use an MLP neural network as the setlearning model for all MOBOproblems, which was shown to be powerful enough to capture complicated problem structure [67].
Revised: We use an MLP neural network as the set model for all MOBO problems, which is good at capturing complicated problem structures [76].

Original: The model details can be found in Appendix.
Revised: The model details can be found in Appendix D.

Original: 4.2 Pareto Set Learning with Gaussian Process
Revised: 4.2 Pareto Set Learning with Gaussian Process

Original: Since the evaluation of f ( x ( λ ))
Revised: Since the evaluation of f ( x ( λ ))

Original: =
Revised: =

Original: f ( h θ ( λ )) is expensive, we use  surrogate models to learn the Pareto set model h θ ( λ ) as shown in Figure 4.
Revised: f ( h θ ( λ )) is expensive, we use the surrogate model-based approach to learn the Pareto set model h θ ( λ ) as shown in Figure 4.

Original: Our method is orthogonal to the choice of surrogatemodels, and we build independent Gaussian process model for each objective as in other MOBOapproaches [47, 49].
Revised: Our method is orthogonal to the choice of surrogate models, and we build independent Gaussian process models for each objective as in other MOBO methods [14, 53].

Original: Gaussian Process Model.
Revised: Gaussian Process Model.

Original: A single-objective Gaussian process [62] has a prior distribution deﬁnedon the function space:
Revised: A single-objective Gaussian process [69] has a prior distribution deﬁned on the function space:

Original: [Equation]
Revised: [Equation]

Original: where µ : X → R is the mean function and k :
Revised: where µ : X → R is the mean function and k :

Original: X × X
Revised: X × X

Original: → R is the covariance kernel function.
Revised: → R is the covariance kernel function.

Original: Withn evaluated solutions { X , y } = { [ x ( i ) ] , [ f ( x ( i ) ] | i = 1 , . . . , n ) } , the posterior distribution can beupdated by maximizing the marginal likelihood based on the data.
Revised: With n evaluated solutions { X , y } = { [ x ( i ) ] , [ f ( x ( i ) ] | i = 1 , . . . , n ) } , the posterior distribution can be updated by maximizing the marginal likelihood based on the data.

Original: For a new solution x n +1 , theposterior mean and variance are:
Revised: For a new solution x n +1 , the posterior mean and variance are:

Original: [Equation]
Revised: [Equation]

Original: where k = k ( x , X ) is the kernel vector and K
Revised: where k = k ( x , X ) is the kernel vector,

Original:  = k
Revised: K = k

Original: ( X , X ) is the kernel matrix (Matérn 5/2kernel is used for all models).
Revised: ( X , X ) is the kernel matrix, Matérn 5/2 kernel are used for all models in this work.

Original: For m independent GP models, we let ˆ µ ( x ) =
Revised: For m independent GP models, we let ˆ µ ( x ) =

Original: [ˆ µ 1 ( x ) , · · · , ˆ µ m ( x )]and ˆ σ 2 ( x ) =
Revised: [ˆ µ 1 ( x ) , · · · , ˆ µ m ( x )] and ˆ σ 2 ( x ) =

Original: [ˆ σ 21 ( x ) , · · · , ˆ σ 2 m ( x )] be the predicted mean and variance for the objective vector.
Revised: [ˆ σ 21 ( x ) , · · · , ˆ σ 2 m ( x )] be the predicted mean and variance for the objective vector.

Original: Suppose we have a learned Pareto set M psl , the GP models give us both predicted value ˆ µ ( M psl )
Revised: Suppose we have a learned Pareto set M psl , the GP models give us both predicted value ˆ µ ( M psl )

Original: ={ ˆ µ ( x ) | x ∈ M psl } and uncertainty ˆ σ 2 ( M psl ) =
Revised: = { ˆ µ ( x ) | x ∈ M psl } and uncertainty ˆ σ 2 ( M psl ) =

Original: { ˆ σ 2 ( x ) | x ∈ M psl } for the whole approximate
Revised: { ˆ σ 2 ( x ) | x ∈ M psl } for the whole approximate

Original: Pareto set.
Revised: Pareto set.

Original: 1: Input: Model x ( λ ) =
Revised: 1: Input: Model x ( λ ) =

Original: h θ ( λ ) 2: Initialize the parameters θ3: for t = 1 to T do 4:
Revised: h θ ( λ ) 2: Initialize the parameters θ3: for t = 1 to T do 4:

Original: Sample preferences { λ k } Kk =1 ∼ Λ 5:
Revised: Sample preferences { λ k } Kk =1 ∼ Λ 5:

Original: Update θ t with gradient descent in (9) 6: end for 7: Output: Model x ( λ ) =
Revised: Update θ t with gradient descent in (10) 6: end for 7: Output: Model x ( λ ) =

Original: h θ T ( λ )
Revised: [Equation]

Original: Pareto Set Learning.
Revised: Pareto Set Learning.

Original: Now we propose an efﬁcient algorithm to ﬁnd the optimal parameter θ ∗for the Pareto set model h θ ( λ ) .
Revised: Now we propose an efﬁcient algorithm to ﬁnd the optimal parameter θ ∗ for the Pareto set model h θ ( λ ) .

Original: The optimal solution set M tch_aug for augmented Tchebycheffscalarization (4) is unknown, hence we need tooptimize all solutions generated by our modelx ( λ ) =
Revised: The optimal solution set M tch_aug for augmented Tchebycheff scalarization (4) is unknown, hence we need to optimize all solutions generated by our model x ( λ ) =

Original: h θ ( λ ) with respect to their corresponding  Tchebycheff scalarization subproblems forall valid preferences:
Revised: h θ ( λ ) with respect to their corresponding augmented Tchebycheff scalarization subproblems for all valid preferences:

Original: [Equation]
Revised: [Equation]

Original: If the model is perfectly learned, the obtained approximate Pareto set M psl = { x = h θ ( λ ) | λ ∈ Λ }should be equal to M tch_aug .
Revised: If the model is perfectly learned, the obtained approximate Pareto set M psl = { x = h θ ( λ ) | λ ∈ Λ } should be the same as M tch_aug .

Original: However, it is difﬁcult to directly optimize (8) due to the expectationover inﬁnite preferences ( | Λ | = ∞ ).
Revised: However, it is difﬁcult to directly optimize (9) due to the expectation over inﬁnite preferences ( | Λ | = ∞ ).

Original: We use Monte Carlo sampling and gradient descent to iterativelylearn the model with the surrogate model:
Revised: We use Monte Carlo sampling and gradient descent to iteratively learn the model with the surrogate model:

Original: [Equation]
Revised: [Equation]

Original: where we randomly sample K = 10 different valid preferences { λ 1 , · · · , λ K } ∼ Λ at each iterationin this work.
Revised: where we randomly sample K = 10 different valid preferences { λ 1 , · · · , λ K } ∼ Λ at each iteration in this work.

Original: Here ˆ g tch_aug ( · ) is the augmented Tchebycheff scalarization with predicted objectivevalues:
Revised: Here ˆ g tch_aug ( · ) is the augmented Tchebycheff scalarization with predicted objective values:

Original: [Equation]
Revised: [Equation]

Original: One design choice left is  the surrogate objective ˆ f ( x ) .
Revised: One design issue left is how to set the surrogate objective ˆ f ( x ) .

Original: If we only want to get the current predictive Pareto front, it is straightforward to use the posterior mean as the surrogate value.
Revised: If we only want to obtain the current predictive Pareto front, it is straightforward to use the posterior mean as the surrogate value.

Original: The approximate Pareto front under the posterior mean could provide valuable information to decision-makers.
Revised: The approximate Pareto front under the posterior mean could provide valuable information to decisionmakers.

Original: However, for Bayesian optimization, we should take the uncertainty into account to balance exploitationand exploration.
Revised: However, for Bayesian optimization, we have to take the uncertainty into account to balance exploitation and exploration.

Original: Many widely-used criteria, such as expected improvement (EI) [52] and upperconﬁdence bound (UCB) [72], could be a more reasonable choice.
Revised: Many widely-used criteria, such as expected improvement (EI) [59] and upper conﬁdence bound (UCB) [81], could be a more reasonable choice.

Original: In this work, we use the lowerconﬁdence bound  for minimization problems:
Revised: In this work, we use the lower conﬁdence bound (LCB) for minimization problems.

Original: [Equation]
Revised: [Equation]

Original: We simply set β = 12 and discuss the performance with other surrogate objectives in Appendix.
Revised: We simply set β = 12 and discuss the performance with other surrogate values in Appendix F.9.

Original: The expensive objective function f ( x ) is usually black-box and non-differentiable, but we can easilyobtain the gradients for the Gaussian process and the set model.
Revised: The expensive objective function f ( x ) is usually black-box and non-differentiable, but we can easily obtain the gradients for the Gaussian process and the set model.

Original: Indeed, gradient-based methods havebeen widely used for optimizing the acquisition function in both BO [84, 83] and MOBO [47, 13].
Revised: Indeed, gradient-based methods have been widely used for optimizing the acquisition function in both BO [95, 93] and MOBO [14, 53].

Original: The max operator in Tchebycheff scalarization is technically only subdifferentiable, but it is knownto have good subgradients [83] for surrogate optimization and  preserve convexity if the objectives{ ˆ f i ( x ) } mi =1 are all convex [7].
Revised: The max operator in Tchebycheff scalarization is technically only subdifferentiable, but it is known to have good subgradients [94] for surrogate optimization and can preserve convexity if the objectives { ˆ f i ( x ) } mi =1 are all convex [9].

Original: The Pareto set learning algorithm with Gaussian process models is summarized in Algorithm 1 .
Revised: The Pareto set learning algorithm with Gaussian process models is summarized in Algorithm 1 .

Original: Weﬁnd the simple random initialization and gradient descent are enough to learn a good Pareto setapproximation.
Revised: We ﬁnd that the simple random initialization and gradient descent are enough to learn a good Pareto set approximation.

Original: The overparameterized neural network could be beneﬁcial to overcome potentialnon-convexity [46].
Revised: The overparameterized neural network could be beneﬁcial to overcome potential non-convexity [52].

Original: 3 Batched selection on approximate Pareto set
Revised: 4.3 Batched selection on approximate Pareto set

Original: In this subsection, we propose a lightweight yet efﬁcient batched acquisition search for MOBOwith the learned Pareto set model.
Revised: In this subsection, we propose a lightweight yet efﬁcient batched acquisition search for MOBO with the learned Pareto set model.

Original: The algorithm framework is shown in Algorithm 2 .
Revised: The algorithm framework is shown in Algorithm 2 .

Original: The crucialdifference with other MOBO approaches is that we build a set model at each iteration for batchedsolution selection as shown in Algorithm 1 and Algorithm 3 .
Revised: The crucial difference with other MOBO approaches is that we build a set model at each iteration for batched solution selection as shown in Algorithm 1 and Algorithm 3 .

Original: The batched selection procedurecontains two closely related steps:
Revised: The batched selection procedure contains two closely related steps:

Original: Batch Sampling on Approximate Pareto Set.
Revised: Batch Sampling on Approximate Pareto Set.

Original: Our model naturally supports generating arbitrarynumber of solutions in batch.
Revised: Our model naturally supports generating an arbitrary number of solutions in batch.

Original: If the decision-maker’s preferences are available, we can use preferencebased sampling in this step.
Revised: If the decision-maker’s preferences are available, we can use preferencebased sampling in this step.

Original: Without any prior knowledge, we uniformly sample P valid preferences{ λ ( p ) }
Revised: In this work, without any prior knowledge, we uniformly sample P valid preferences { λ ( p ) }

Original: Pp =1 , and generate the corresponding solutions
Revised: Pp =1 , and generate the corresponding solutions

Original: X
Revised: X

Original: = { x ( λ ( p ) ) }
Revised: = { x ( λ ( p ) ) }

Original: Pp =1 on the approximate Paretoset M psl .
Revised: Pp =1 on the approximate Pareto set M psl .

Original: Batch Selection.
Revised: Batch Selection.

Original: At each iteration of MOBO, we typically select a small number B (e.g., 5 ) ofsolutions X B = { x ( b ) } Bb =1 from the sampled solutions X for expensive evaluations.
Revised: At each iteration of MOBO, we typically select a small number B (e.g., 5 ) of solutions X B = { x ( b ) } Bb =1 from the sampled solutions X for expensive evaluations.

Original: To take allalready evaluated solutions { X t − 1 , y t − 1 } into consideration, we use the hypervolume [91] as theselection criteria.
Revised: To take all already evaluated solutions { X t − 1 , y t − 1 } into consideration, we use the hypervolume [103] as the selection criteria.

Original: The hypervolume HV ( y ) =
Revised: The hypervolume HV ( y ) =

Original: Vol ( S ) measures the volume of S dominated by aset y in the objective space:
Revised: Vol ( S ) measures the volume of S dominated by a set y in the objective space:

Original: [Equation]
Revised: [Equation]

Original: where r ∗ is a reference point that dominated by all y ∈ y .
Revised: where r ∗ is a reference point that dominated by all y ∈ y .

Original: The hypervolume improvement (HVI)of a set X B with respect to the already evaluated solutions { X t − 1 , y t − 1 } can be deﬁned as:
Revised: The hypervolume improvement (HVI) of a set X B with respect to the already evaluated solutions { X t − 1 , y t − 1 } can be deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where X B = { x ( b ) } Bb =1 are selected solutions, and ˆ f ( X B ) are the surrogate values.
Revised: where X B = { x ( b ) } Bb =1 are selected solutions, and ˆ f ( X B ) are the surrogate values.

Original: In this work,we mainly use the LCB (11) as the surrogate value for Bayesian optimization, and provide an ablationstudy of different surrogate values in Appendix. F.3.
Revised: In this work, we mainly use the LCB (12) as the surrogate value for Bayesian optimization, and provide an ablation study of different surrogate values in Appendix F.9.

Original: A better trade-off set will have a larger hypervolume, and the ground truth Pareto set always hasthe largest one.
Revised: A better trade-off set will have a larger hypervolume, and the true Pareto set always has the largest one.

Original: We want to select a set of X B such that their corresponding objective values ˆ f ( X B )maximize HVI (ˆ f ( X B )) .
Revised: We want to select a set of X B such that their corresponding objective values ˆ f ( X B ) maximize HVI (ˆ f ( X B )) .

Original: It would be computationally intensive to jointly optimize a set of solutionsto exactly maximize the hypervolume improvement (13), and therefore sequential greedy selectionis typically used [13].
Revised: It would be computationally expensive to jointly optimize a set of solutions to exactly maximize the hypervolume improvement (14), and therefore sequential greedy selection is typically used [14].

Original: In this work, we select the set X B with a sequential greedy manner from X where | X | =P = 1 , 000 for all problems.
Revised: In this work, we select the set X B in a sequential greedy manner from X where | X | =

Original: More details can be found in Appendix.

Original:  Experiments
Revised:  Experiments

Original: In this section, wemainly compare the proposed PSL method with other MOBO approaches on theperformance of evaluated solutions.
Revised: In this section, we compare the proposed PSL method with other MOBO approaches on the performance of evaluated solutions.

Original: We also analyze the quality of the learned Pareto set, whichisnot supported by other methods.
Revised: We also analyze the quality of the learned Pareto set model, which other methods cannot produce.

Original: Baseline Algorithms.
Revised: Baseline Algorithms.

Original: We consider different widely-used MOBO methods and two model-freeapproaches as baselines.
Revised: We consider several widely-used MOBO methods and two model-free approaches as baselines.

Original: The implementations of NSGA-II [18], MOEA/D-EGO [87], TSEMO [8], [Figure] [Table] [Figure]
Revised: The implementations of NSGA-II [20], MOEA/D-EGO [99], TSEMO [10],

Original: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: USeMO-EI

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure] n [Figure] USeMO-EI [5], DGEMO [47] are from DGEMO’s open-source codebase 1 based on pymoo 2 [6].
Revised: [7], DGEMO [53] are from DGEMO’s open-source codebase 1 based on pymoo 2 [8].

Original: The implementations of scrambled Sobol sequence,TS-TCH [55], MESMO [4], qParEGO [41], andqEHVI [13] and from BoTorch 3 [3].
Revised: The implementations of scrambled Sobol sequence, qParEGO [45], TS-TCH [62], qEHVI [14] and qNEHVI [15] and from BoTorch 3 [5].

Original: We implement  PSL  in Pytorch [57], and will open-source ourcode upon publication 4 .
Revised: We implement the proposed PSL 4 in Pytorch [64].

Original: Benchmarks and Real-World Problems.
Revised: Benchmarks and Real-World Problems.

Original: The algorithms are ﬁrst compared on  the widely-used 
Revised: The algorithms are ﬁrst compared on six newly proposed synthetic test instances (see Appendix E.1), as well as the widely-used VLMOP1-3 [88] and

Original: ZDT1-3 [90], VLMOP1-3 [79] and DTLZ2 [19] benchmark problems.
Revised: DTLZ2 [21] benchmark problems.

Original: Then we also conduct experiments on 5 different real-world multi-objective engineering design problems (RE) [76], including1) four bar truss design [10]; 2) pressure vessel design [42]; 3) disk brake design [63]; 4) gear traindesign [17] and 5) rocket injector design [78].
Revised: Then we also conduct experiments on 5 different real-world multi-objective engineering design problems (RE) [85], including 1) four bar truss design [12]; 2) pressure vessel design [46]; 3) disk brake design [70]; 4) gear train design [19] and 5) rocket injector design [87].

Original: Details of these problems can be found in Appendix.
Revised: Details of these problems can be found in Appendix E.

Original: Experiment Setting.
Revised: Experiment Setting.

Original: For each experiment, we randomly generate 10 initial solutions for expensiveevaluations, and then conduct MOBO with 20 batched evaluations with batch size 5 .
Revised: For each experiment, we randomly generate 10 initial solutions for expensive evaluations, and then conduct MOBO with 20 batched evaluations with batch size 5 .

Original: Therefore,there are total 110 expensive evaluations.
Revised: Therefore, there are total 110 expensive evaluations.

Original: For an experiment, all algorithms are independently run 10times with different random seeds.
Revised: For an experiment, all algorithms are independently run 10 times.

Original: We use the hypervolume indicator [91] as the metric to comparethe quality of evaluated solutions chosen by different MOBO algorithms, which is monotonic tothe Pareto dominance relation.
Revised: We use the hypervolume indicator [103] as the metric to compare the quality of evaluated solutions chosen by different MOBO algorithms, which is monotonic to the Pareto dominance relation.

Original: The ground truth Pareto front will always have the best (highest)hypervolume.
Revised: The ground truth Pareto front will always have the best (highest) hypervolume.

Original: Detailed experimental setting can be found in Appendix.

Original: 5.1 Experimental results and analysis
Revised: 5.1 Experimental results and analysis

Original: [Table]
Revised: [Table]

Original: MOBO Performance.
Revised: MOBO Performance.

Original: We compare PSL with other MOBO methods on the performance of evaluatedsolutions.
Revised: We compare PSL with other MOBO methods on the performance of evaluated solutions.

Original: Figure 5 shows the log hypervolume difference to the true/approximate Pareto front for the To [Table] [Figure]
Revised: Figure 5 shows the log hypervolume difference to the true/approximate Pareto front for the synthetic/real-world problems during the optimization process.

Original: [Table] [Figure] [Table] [Figure]
Revised: The approximate Pareto fronts for the real-world design problems are from Tanabe and Ishibuchi [85] with a large number of evaluations, which are also used in other MOBO works.

Original: [Table] [Figure] [Table] [Figure]
Revised: In most experiments, our proposed PSL method has better or comparable performance with other MOBO algorithms.

Original: [Table] [Figure] [Table] [Figure]
Revised: Especially, as a generalized scalarization-based method, PSL signiﬁcantly outperforms the model-free counterparts such as qParEGO [45, 15], MOEA/D-EGO [99], and TS-TCH [62].

Original: synthetic/real-world problems during the optimization process. The approximate Pareto fronts for thereal-world design problems are from Tanabe and Ishibuchi [76] with a large number of evaluations,which are also used in other MOBO works. In most experiments, our proposed PSL method convergesfaster and achieves better ﬁnal results than other MOBO methods.
Revised: These promising results validate the efﬁciency and usefulness of Pareto set learning for MOBO. More discussion of the proposed algorithm can be found in Appendix A.1 and Appendix A.2.

Original: As shown in Table 1, PSL has ashorter or comparable total runtime  per iteration with other MOBO methods, which can be ignored inthe expensive optimization problems (might take days).The promising results validate that the PSLapproach can signiﬁcantly improve the MOBO performance for expensive optimization problems.f 1 ( x ) 0.0 0.2 0.4 0.6 0.8 1. f 2 ( x ) 0.0 0.2 0.4 0. f 3 ( x ) 0.0 0.2 0.
Revised: As shown in Table 1, PSL has a shorter or comparable total runtime (e.g., for modeling and batch selection) per iteration with other MOBO methods, which can be ignored in the expensive optimization problems (might take days).

Original: [-(a) Different Preferences f 1 ( x ) 0.0 0.2 0.4 0.
 f 2 ( x ) 0.0 0.2 0.4 0. f 3 ( x ) 0.0 0.2 0. [Figure]-]
Revised: [-(a) Different Preferences f 1 ( x ) 0.0 0.2 0.4 0.
 f 2 ( x ) 0.0 0.2 0.4 0. f 3 ( x ) 0.0 0.2 0. [Figure]-]The algorithm runtimes for all problems can be found in Appendix F.1.

Original: ) DTLZ
Revised: These results conﬁrm that the Pareto set learning approach has a low computational overhead which is affordable for MOBO.

Original: The Learned Pareto Set.
Revised: The Learned Pareto Set.

Original: We present the approximate Pareto set learned by PSL under the posteriormean after optimization in Figure 6, which is notsupported by other MOBO methods.
Revised: We present the approximate Pareto set learned by PSL under the posterior mean after optimization in Figure 6, which is not supported by other MOBO methods.

Original: According tothe results, PSL can successfully learn the Pareto setsfor different benchmarks and real-world applicationproblems with different shapes of Pareto fronts.
Revised: According to the results, PSL can successfully learn the Pareto sets for different benchmarks and real-world application problems with different shapes of Pareto fronts.

Original: Forbenchmark problems, PSL can match the ground truth
Revised: For benchmark problems, PSL can match the ground truth

Original: Pareto front with a small evaluation budget.
Revised: Pareto front with a small evaluation budget.

Original: For realworld applications, the approximate Pareto fronts cancapture the trade-off among objectives and providevaluable information to support decision-making.
Revised: For real-world applications, the approximate Pareto fronts can capture the trade-off among objectives and provide valuable information to support decision-making.

Revised: We further discuss the the practicality of the approximate

Original: Flexible Trade-off Adjustment.
Revised: Pareto set in Appendix A.3. [Figure] i [Figure]

Revised: i

Revised: [Figure]

Revised: (g) VLMOP

Revised: [Figure]

Revised: l o g H V D i ff e r e n c e

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Figure]

Revised: their corresponding solutions on the approximate Pareto set.

Revised: Flexible Trade-off Adjustment.

Original: With our model,the decision-makers can easily explore the whole approximate Pareto set by themselves to selectthe most preferred trade-off solution(s) as shown in Figure 7.
Revised: With our model, the decision-makers can easily explore the whole approximate Pareto set by themselves to select the most preferred trade-off solution(s) as shown in Figure 7.

Original: No time-consuming communicationbetween the optimization modeler and the decision-maker is required.
Revised: No time-consuming communication between the optimization modeler and the decision-maker is required.

Original: By directly exploring theapproximate Pareto front in an interactive manner, the decision-makers can observe and understandthe connection between  preferences and corresponding solutions in real-time.
Revised: By directly exploring the approximate Pareto front in an interactive manner, the decision-makers can observe and understand the connection between the trade-off preferences and corresponding solutions in real-time.

Original: It is also beneﬁcial fordecision-makers to further adjust and assign their most accurate preferences.
Revised: It is also beneﬁcial for decision-makers to further adjust and assign their most accurate preferences.

Original: The ability to incorporateuser’s knowledge into decision making [26] could be crucial for many real-world applications.
Revised: The ability to incorporate user’s knowledge into decision making [30] could be crucial for many real-world applications.

Revised: More experimental results and analyses can be found in Appendix F.

Original:  Conclusion, limitation and future work
Revised:  Conclusion, limitation and future work

Original: Conclusion.
Revised: Conclusion.

Original: In this work, we have proposed a novel Pareto set learning method for expensive multiobjective optimization with a limited evaluation budget.
Revised: We have proposed a novel Pareto set learning method, which is a ﬁrst attempt to approximate the whole Pareto set for expensive multi-objective optimization.

Original: The beneﬁts of this approach are two-fold.
Revised: The advantages of this approach are two-fold.

Original: First of all, by learning and utilizing the approximate Pareto set, it can serve as an efﬁcient MOBOmethod that outperforms different existing approaches.
Revised: First, by learning and utilizing the approximate Pareto set, it can serve as an efﬁcient MOBO method that outperforms different existing approaches.

Original: Secondly, italso allows the decision-makersto easily exploreany trade-off region on the  approximate Pareto set, which supports ﬂexible andinteractive decision-making.
Revised: Secondly, it allows decision-makers to readily explore the whole approximate Pareto set, which supports ﬂexible and interactive decision-making.

Original: We believe the proposed Pareto set learning method could provide anovel way to solve expensive multi-objective optimization problems. Checklist For all authors...
Revised: We believe the proposed Pareto set learning method could provide a novel way for solve expensive multi-objective optimization.

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope? [Yes] (b) Did you describe the limitations of your work? [Yes] See Section 6.
Revised: Limitation and Future Work.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] We do notﬁnd any potential negative societal impacts of our work.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [No] Our code will

Original: be released upon publication.

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,

Original: how they

Original: were chosen)?

Original: [Yes] See Section 5 and Appendix.

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi- ments multiple times)? [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: The quality of the approximate Pareto set mainly depends on the accuracy of the surrogate models and the performance of the Pareto set learning algorithm, which could be poor for problems with insufﬁcient evaluation budget and/or large-scale search space.

Original: [Yes] Yes. See Appendix.

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets? [Yes] See Appendix. (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: A more detailed discussion of limitations and potential future work can be found in Appendix B, and potential societal impact can be found in Appendix C.

Original: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [N/A]
Revised: Acknowledgements

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?

Original: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
Revised: This work was supported by the Hong Kong General Research Fund (11208121, CityU-9043148).

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]

Original: We provide more discussions, algorithm and problem details, and extra experimental results in thisappendix:

Original: • More discussions of the proposed algorithm are temporally provided in Section A.

Original: • The limitations and potential future improvement for PSL are discussed in Section B.

Original: • Potential societal impact of this work is discussed in Section C.

Original: • Details of the set model and batch selection algorithm are provided in Section D.• Details of the benchmark and real-world application problems are given in Section E.(deﬁnition of 6 new problems are added.)

Original: • More experimental results and analyses are presented in Section F. (many new results)

Original: A More Discussion

Original: Since no extra page is allowed for the main paper during the response period, we temporally put allnew discussions for our proposed algorithm in this section.

Original: Will move part of these discussions intothe main paper and reorganize the rest materials once an extra page is allowed.

Original:  A.1 Motivation: Pareto set model over simple scalarization
Revised: A More discussion A.1 Motivation: Pareto set model over simple scalarization

Original: Our proposed Pareto set learning method has a close relation to the scalarization-based methods.
Revised: Our proposed Pareto set learning method is closely related to the scalarization-based methods.

Original: Butit can overcome a major disadvantage of the current scalarization methods for expensive optimization.
Revised: But it can overcome a major disadvantage of the current scalarization methods for expensive optimization.

Original: The scalarization methods randomly select one (e.g., ParEGO) or a batch of scalarized subproblems(e.g., MOEA/D-EGO, TS-TCH, and qParEGO) at each iteration.
Revised: The scalarization methods randomly select one (e.g., ParEGO [45]) or a batch of scalarized subproblems (e.g., MOEA/D-EGO [99], TS-TCH [62], and qParEGO [15]) at each iteration.

Original: By optimizing the acquisitionfunction (e.g., EI, LCB, Thompson Sampling etc) on each scalarization, they generate a batch ofsolutions for expensive evaluation.
Revised: By optimizing the acquisition function (e.g., EI, LCB, Thompson Sampling) on each scalarization, they generate a batch of solutions for expensive evaluation.

Original: A major limitation of all these scalarization methods is that they do not explicitly consider thosealready-evaluated solutions, neither for choosing the weighted scalarization, nor for selecting thenext (batch of) solution(s) for expensive evaluation.
Revised: A major limitation of all these scalarization methods is that they do not explicitly consider those already-evaluated solutions, neither for choosing the weighted scalarization, nor for selecting the next (batch of) solution(s) for expensive evaluation.

Original: Therefore, the selected weighted scalarization(s)might be close to those already-evaluated solutions.
Revised: Therefore, the selected weighted scalarization(s) might be close to those already-evaluated solutions.

Original: In other words, even the obtained solution(s), canmaximize the EI/LCB for the selected scalarization,  could still be similar to those already-evaluatedones, and indeed not an optimal choice for multi-objective optimization.
Revised: In other words, even the obtained solution(s) can maximize the EI/LCB for the selected scalarization, they could still be similar to those alreadyevaluated ones, and indeed not an optimal choice for multi-objective optimization.

Original: This limitation leads toinferior performance of these scalarization methods, as reported in [13, 47] and in our experimentalresults.
Revised: This limitation leads to inferior performance of these scalarization methods, as reported in [14, 53] and in our experimental results.

Original: To overcome this limitation, our proposed method has a two-stage approach:
Revised: To overcome this limitation, our proposed method has a two-stage approach:

Original: • Stage 1: Using the Pareto set model, it ﬁrst efﬁciently samples a dense set of candidatesolutions to cover the whole approximate Pareto front (of posterior mean, EI, or LCB etc).• Stage 2: Then it selects a small batch of appropriate solutions from this dense set forexpensive evaluation.
Revised: • Stage 1: Using the Pareto set model, it ﬁrst efﬁciently samples a dense set of candidate solutions to cover the whole approximate Pareto front (of posterior mean, EI, or LCB etc). • Stage 2: Then it selects a small batch of appropriate solutions from this dense set for expensive evaluation.

Original: For selection, we use the HVI criteria to take both the selected andalready-evaluated solutions into consideration.
Revised: For selection, we use the HVI criteria to take both the selected and already-evaluated solutions into consideration.

Original: In this way, our proposed method can efﬁciently explore the whole approximate Pareto front, andchoose the most appropriate solutions (on the approximate Pareto front, while far from the selectedand already-evaluated ones) for expensive evaluation.
Revised: In this way, our proposed method can efﬁciently explore the whole approximate Pareto front, and choose the most appropriate solutions (on the approximate Pareto front, while far from the selected and already-evaluated ones) for expensive evaluation.

Original: The experimental results have validated theefﬁciency of the proposed method.
Revised: The experimental results have validated the efﬁciency of the proposed method.

Original: A.2 Hypervolume improvement for batch selection
Revised: A.2 Hypervolume improvement for batch selection

Original: As discussed above, the current scalarization approach (e.g., (q)ParEGO) randomly selects differentscalarization(s) at each iteration, which is to randomly generate a small set of sparse solutions on theapproximate Pareto set (with EI, LCB) for evaluation.

Original: In other words, the selected solutions are asmall random subset of the whole approximate Pareto set (our Stage 1).

Original: In our proposed algorithm, weuse the HVI to select a small batch of solutions from the dense approximate Pareto set, which can beaware of both selected and already-evaluated solutions and leads to better optimization performance(our Stage 2).

Original: Efﬁciently ﬁnding a batch of solutions to optimize the EI/LCB of HVI could be very challenging.qEHVI [13] and qNEHVI [14] are two promising approaches along this direction.
Revised: Efﬁciently ﬁnding a batch of solutions to optimize the EI/LCB of HVI could be very challenging. qEHVI [14] and qNEHVI [15] are two promising approaches along this direction.

Original: From the viewpointof optimizing HVI, our proposed method restricts the search procedure only on the approximate Pareto set, which is a low-dimensional (m-1) manifold in the decision space.
Revised: From the viewpoint of optimizing HVI, our proposed method restricts the search procedure only on the approximate Pareto set, which is a low-dimensional (e.g., ( m − 1) -dimensional) manifold in the decision space [36, 98].

Original: Therefore, we can use asimple two-stage sample-then-select approach to ﬁnd the batch of solutions for evaluations.
Revised: Therefore, we can use a simple two-stage sample-then-select approach to ﬁnd the batch of solutions for evaluations.

Original: For theoptimal situation, the set of solutions that optimize HVI should all be on the Pareto front.
Revised: For the optimal situation, the set of solutions that optimize HVI should all be on the Pareto front.

Original: However,we agree with the reviewer that if an efﬁcient method exists, directly optimizing the EI/LCB of HVIon the whole search space should be a principled approach for batch selection.
Revised: However, if an efﬁcient method exists, directly optimizing the EI/LCB of HVI on the whole search space should be a principled approach for batch selection.

Original: On the other hand, the scalarization-based approach could have a close relationship with the hypervolume [88].
Revised: On the other hand, the scalarization-based approach could have a close relationship with the hypervolume [100].

Original: It will be very interesting to study how to better leverage this relation for designing a moreefﬁcient algorithm, such as learning the whole Pareto set while inference the location of solutions (onthe learned Pareto set) to optimize HVI at a single stage.
Revised: It will be very interesting to study how to better leverage this relation for designing a more efﬁcient algorithm, such as learning the whole Pareto set while inference the location of solutions (on the learned Pareto set) to optimize HVI at a single stage.

Original: These will be our future work.
Revised: These will be our future work.

Revised: In summary, the advantages of PSL can be summarized from the following two viewpoints:

Revised: • From the viewpoint of scalarization based methods, PSL proposes a novel two-stage approach to select a small batch of appropriate solutions from the approximate Pareto front, which takes those already-evaluated solutions into consideration.

Revised: As a result, it signiﬁcantly outperforms other scalarization-based methods.

Revised: • From the viewpoint of hypervolume improvement based methods, in an ideal case, the set of solutions that optimize HVI should all be on the Pareto front.

Revised: Restricting the search on the approximate Pareto set, PSL leads to an efﬁcient HVI algorithm.

Original: A.3 Practicality of the approximate Pareto set
Revised: A.3 Practicality of the approximate Pareto set

Original: In real-world applications, the learned approximate Pareto set could not always be accurate.
Revised: The learned approximate Pareto set could not always be accurate in real-world applications.

Original: Therefore,it is risky to only rely on the approximate Pareto set to make a decision (see more discussion in thelimitation section).
Revised: Therefore, it is risky to only rely on the approximate Pareto set to make a decision (see more discussion in Section.

Original:  With our proposed method, decision-makers can  haveboth the evaluated solutionsand the approximate Pareto set at the same time.
Revised: B). With our proposed method, decision-makers can simultaneously have the evaluated solutions and the approximate Pareto set.

Original: The approximate Pareto set can provide extra usefulinformation to better support their decision-making.
Revised: The approximate Pareto set can provide extra useful information to better support their decision-making such as:

Original: More speciﬁcally,• It can helpthe decision-makers to better understand the (approximate) trade-offs they willmake when they choose an already evaluatedsolution as their ﬁnal solution.
Revised: • It can help decision-makers to better understand the (approximate) trade-offs they will make for choosing any already evaluated solution.

Original: The approximate Pareto set and the corresponding surrogate objective values around the chosen solution canprovide valuable information to understand what we can gain or lose by adjusting thecurrent chosen solution.
Revised: The approximate Pareto set and the corresponding surrogate objective values around the chosen solution can provide valuable information to understand what we can gain or lose by adjusting the chosen solution.

Original: • It allows the decision-maker to explore the whole approximate Pareto set/front easily.
Revised: • It allows decision-makers to explore the whole approximate Pareto set easily.

Original: If none of the already evaluated solutions canprecisely satisfy the decision-maker’s preferredtrade-off (which could be possible for applications with inﬁnite Pareto sets), the decisionmaker can rely on the approximate Pareto set/front to choose the preferred solution for further evaluation.
Revised: If none of the already evaluated solutions can satisfy the decision-maker’s preferred trade-off, the decision-maker can rely on the approximate Pareto set/front to choose the preferred solution for further evaluation.

Original: • When the optimization modeler and the ﬁnal decision-maker are not the same person, theapproximate Pareto set/front provides a much more efﬁcient way for them to communicateand discuss the whole trade-offs among different objectives during or post the optimization process.
Revised: • When the optimization modeler and the ﬁnal decision-maker are not the same person, the approximate Pareto set/front provides a much more efﬁcient way for them to communicate and discuss the whole trade-offs among different objectives during or after the optimization process.

Original: This demand is common and important for many applications [49].
Revised: This demand is common and essential for many applications [56].

Original: • Finally, an important application for MOBO is to help domain experts efﬁciently conduct experiments.
Revised: • Finally, an important application for MOBO is to help domain experts efﬁciently conduct experiments.

Original: The approximate Pareto set/front might contain useful patterns and structures which can help the domain expert obtain moreuseful information from the experiment.
Revised: The approximate Pareto set might contain useful patterns and structures which can help the domain expert obtain more information from the experiment.

Original: It 
Revised: It also provides

Original: also provides a way for domain experts to incorporate their knowledge into the optimization process (e g., choose the most concerned region, and eliminate some uninteresting locations).The
Revised: a way for domain experts to incorporate their knowledge into the optimization process (e.g., choose the most concerned region, and eliminate some uninteresting locations).

Original:  proposed Pareto set learning method could be a novel approach to support "bring decision maker back into the loop"  for Bayesian optimization [26]. Therefore,
Revised: The proposed Pareto set learning method could be a novel approach to support the "bringing decision-makers back into the loop" approach for Bayesian optimization [30].

Original: the

Original: learned

Original: approximate Pareto set

Original: could

Original: be a

Original:  useful tool to support ﬂexible decision making for expensive multi-objective optimization
Revised: Therefore, the learned approximate Pareto set could be a useful tool to support ﬂexible decisionmaking for expensive multi-objective optimization.

Original: B Limitation and potential future work 
Revised: B Limitation and potential future work B.

Original: B.

Original: 1 The approximate Pareto set could be inaccurate
Revised: 1 The approximate Pareto set could be inaccurate

Original: It could be very risky, especially in those safety-critical applications, to only rely on an approximate
Revised: It could be very risky, especially in those safety-critical applications, to only rely on an approximate

Original: Pareto set to make the ﬁnal decision.
Revised: Pareto set to make the ﬁnal decision.

Original: The quality of the approximate Pareto set heavily depends onthe quality of the surrogate models.
Revised: The quality of the approximate Pareto set heavily depends on the quality of the surrogate models.

Original: If the evaluation budget is insufﬁcient to build a good set ofsurrogate models (for each objective), we cannot obtain a good approximate Pareto set.
Revised: We cannot obtain a good approximate Pareto set if the evaluation budget is insufﬁcient to build a good set of surrogate models.

Original: This is also achallenge for the general  (multi-objective)
Revised: This is also a general challenge for (multi-objective)

Original: Bayesian optimization methods.
Revised: Bayesian optimization [30].

Original: One possible method to address this challenge is to leverage the user preference information into theexpensive optimization process [1, 55, 2].
Revised: One possible method to address this challenge is to leverage the user preference information into the expensive optimization process [3, 4, 62].

Original: In this way, the general MOBO method can spend the limited evaluation budget mainly on the user-preferred region rather than the whole decision space.
Revised: In this way, the general MOBO method can spend the limited evaluation budget mainly on the user-preferred region rather than the whole decision space.

Original: Similarly, our proposed method can only build a partial approximate Pareto set with the user-preferredtrade-offs.
Revised: Similarly, our proposed method can only build a partial approximate Pareto set with the user-preferred trade-offs.

Original: However, the preference-based approach could be affected by the following limitation.
Revised: However, the preference-based approach could be affected by the following limitation.

Original:  preference is challenging for black-box optimization
Revised: B.2 Deﬁning the user preference is challenging for black-box optimization

Original: Although some scalarization methods (e.g., Chebyshev scalarization) have good theoretical propertiesto connect the scalars to their corresponding Pareto solutions, it could  be not so easy to deﬁne userpreferences in terms of scalars, especially in the black-box expensive optimization setting.
Revised: Although some scalarization methods (e.g., Chebyshev scalarization) have good theoretical properties to connect the scalars to their corresponding Pareto solutions [13], it could still be hard to deﬁne user preferences in terms of scalars, especially in the black-box expensive optimization setting.

Original: In many applications, the decision-makers mighteven do not  know their actual preferences beforemaking their decision.
Revised: In many applications, the decision-makers might not even know their actual preferences before making their decision.

Original: If the approximate Pareto set can be properly learned, instead of asking usersto provide their preferences in terms of ﬁxed scalars, our proposed approach can let them interactivelyexplore the whole approximate Pareto set/Pareto front to select their most preferred solutions.
Revised: Suppose the approximate Pareto set can be learned appropriately, instead of asking users to provide their preferences, our proposed approach can let them interactively explore the whole approximate Pareto set/front to select their most preferred solutions.

Original: In thisinteractive way, it could be much easier for the decision-makers to accurately express and assign theirpreferences.
Revised: In this interactive way, it could be much easier for the decision-makers to accurately express and assign their preferences.

Original: However, it could be difﬁcult to precisely obtain user preference if a good approximate Pareto set is not available (e.g., in the early stage of optimization, or without enough budget).
Revised: However, it could be difﬁcult to precisely obtain user preference if a good approximate Pareto set is unavailable (e.g., in the early stage of optimization, or without enough budget).

Original: The algorithm proposed in Abdolshah et al.
Revised: The algorithm proposed in Abdolshah et al.

Original: [1] is an elegant and promising way to incorporatepreference into the MOBO process via the preference-order constraints, which does not requireany prior knowledge on the (approximate) Pareto front.
Revised: [3] is an elegant and promising way to incorporate preference into the MOBO process via the preference-order constraints, which does not require any prior knowledge on the (approximate) Pareto front.

Original: The preference-order constraint also has aclose relationship with the lexicographic approach for multi-objective optimization [65], which is anon-scalarizing method with good theoretical property (e.g., connection to weakly Pareto optimalsolution).
Revised: The preference-order constraint also has a close relationship with the lexicographic approach for multi-objective optimization [72], which is a non-scalarizing method with good theoretical property (e.g., connection to weakly Pareto optimal solution).

Original: How to incorporate this information into our proposed Pareto set learning model for moreﬂexible preference incorporation could be interesting for future work.
Revised: Incorporating this information into our proposed Pareto set learning model for more ﬂexible preference incorporation could be interesting for future work.

Original: B.3 Scalability
Revised: B.3 Scalability

Original: The scalabilitywith the dimensions of  search space  is a main challenge forthe general MOBOalgorithms.
Revised: The scalability of the search space dimension is a major challenge for general MOBO algorithms.

Original: It is also very difﬁcult for our proposed PSL method to learn a good enough Pareto set forsuch large-scale problems.
Revised: It is also very difﬁcult for our proposed PSL method to learn a good enough Pareto set for such large-scale problems.

Original: The PSL’s performance depends on a set of good surrogate models, whichrequires a large number of evaluated solutions for the problem with a high-dimensional search space.
Revised: The PSL’s performance depends on a set of good surrogate models, which requires a large number of evaluated solutions for the problem with a high-dimensional search space.

Original: Recently, some efﬁcient search region decomposition/management methods have been proposedto better allocate the limited computational budget for problems with high-dimensional searchspace.
Revised: Recently, some efﬁcient search region decomposition/management methods have been proposed to better allocate the limited computational budget for problems with high-dimensional search space [27, 89].

Original: These ideas can be naturally generalized to solve expensive multi-objective optimizationproblems, such as MORBO  and LaMOO.
Revised: These ideas can be naturally generalized to solve expensive multi-objective optimization problems, such as MORBO [17] and LaMOO [101].

Original: Since the search region management methods are algorithmagnostic (e.g., a meta-algorithm), they can be combined with different (multi-objective) optimizationalgorithms.
Revised: Since the search region management methods are algorithm-agnostic (e.g., a meta-algorithm), they can be combined with different (multi-objective) optimization algorithms.

Original: Studying how to efﬁciently combine the MORBO/LaMOO approach with our proposed PSL method will be an important future work to tackle this limitation.
Revised: Studying how to efﬁciently combine the MORBO/LaMOO approach with our proposed PSL method will be an important future work to tackle this limitation.

Original: C Potential societal impact
Revised: C Potential societal impact

Original: Our proposed Pareto set learning method mainly has two strengths: (1) it is good for better multiobjective Bayesian optimization in terms of speed and sample efﬁciency, and (2) it can help decisionmakers to navigate the estimated Pareto setand front for better decision-making.
Revised: Our proposed Pareto set learning method mainly has two strengths: (1) it is good for better multiobjective Bayesian optimization in terms of speed and sample efﬁciency, and (2) it can help decisionmakers to navigate the estimated Pareto set for better decision-making.

Original: These strengthscould lead to many positive potential societal impacts.
Revised: These strengths could lead to many positive potential societal impacts.

Original: For example, as an efﬁcient MOBO algorithm,it can reduce the cost of obtaining a set of diverse Pareto solutions in many applications such asmaterials science, engineering design, and recommender systems.
Revised: For example, as an efﬁcient MOBO algorithm, it can reduce the cost of obtaining a set of diverse Pareto solutions in many applications, such as materials science, engineering design, and recommender systems.

Original: The learned approximate Paretoset provides a novel way for decision-makers to easily explore different trade-offs, which could be animportant component for a more user-friendly multi-criteria decision system.
Revised: The learned approximate Pareto set provides a novel way for decision-makers to easily explore different trade-offs, which could be an important component for a more user-friendly multi-criteria decision system.

Original: On the negative side, as discussed in the limitation section, there iscurrently no guarantee that thelearned approximate Pareto set could be accurate.
Revised: On the negative side, as discussed in the limitation section, there is no guarantee that the learned approximate Pareto set could be accurate.

Original: Solely relying on the approximate Pareto set tomake the ﬁnal decision could be risky.
Revised: Solely relying on the approximate Pareto set to make a decision could be risky.

Original: The decision-makers should leverage all the informationthey have to make the ﬁnal decision.
Revised: The decision-makers should leverage all the information they have to make the ﬁnal decision.

Original: In addition, the leakage of the learned Pareto set model mightunintentionally reveal the problem information and user preference, which should be avoided inreal-world applications.
Revised: In addition, the leakage of the learned Pareto set model might unintentionally reveal the problem information and user preference, which should be avoided in real-world applications.

Original: D Model and algorithm details 
Revised: D Model and algorithm details D.1 Pareto set model

Original: D.1 Pareto set model

Original: Model Structure.
Revised: Model Structure.

Original: In this work, we use a multi-layered perceptron (MLP) as the Pareto set modelx ( λ ) =
Revised: In this work, we use a multi-layered perceptron (MLP) as the Pareto set model x ( λ ) =

Original: h θ ( λ ) .
Revised: h θ ( λ ) .

Original: For all experiments, the Pareto set model has 2 hidden layers each with 256 hiddenunits, and the activation is ReLU.
Revised: For all experiments, the Pareto set model has 2 hidden layers each with 256 hidden units, and the activation is ReLU.

Original: The input and output dimension of the set model is the number ofobjectives m and the dimension of decision variables n , respectively.
Revised: The input and output dimension of the set model is the number of objectives m and the dimension of decision variables n , respectively.

Original: Therefore, the set model h θ ( λ )has the following structure:[Equation]
Revised: Therefore, the set model h θ ( λ ) has the following structure:

Original: [Equation]
Revised: [Equation]

Original: where the input is an m -dimensional preference λ ∈ Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , the output isa n -dimensional decision variables x ∈ R n , and θ ∈ R d represents all learnable parameters forthe Pareto set model.
Revised: where the input is an m -dimensional preference λ ∈ Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , the output is a n -dimensional decision variables x ∈ R n , and θ ∈ R d represents all learnable parameters for the Pareto set model.

Original: We build independent Gaussian process models with Matérn 5/2 kernel foreach objective as the surrogate models.
Revised: We build independent Gaussian process models with Matérn 5/2 kernel for each objective as the surrogate models.

Original: The objective to optimize is the augmented Tchebycheffscalarization on the surrogate value with respect to all valid preferences.
Revised: The objective to optimize is the augmented Tchebycheff scalarization on the surrogate value with respect to all valid preferences.

Original: Model Training.
Revised: Model Training.

Original: In the proposed Pareto set learning method, we have the following model structure:
Revised: In the proposed Pareto set learning method, we have the following model structure:

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] where the preference λ goes through the Pareto set model h θ ( λ ) and Gaussian process model, thengets the augmented Tchebycheff scalarized value ˆ g tch_aug ( x ( λ )) at the end (a detailed version can befound in Figure 4 in the main paper).
Revised: where the preference λ goes through the Pareto set model h θ ( λ ) and Gaussian process model, then gets the augmented Tchebycheff scalarized value ˆ g tch_aug ( x ( λ )) at the end (a detailed version can be found in Figure 4 in the main paper).

Original: To train the Pareto set model, we use  gradient-based method tooptimize the model parameter θ with respect to the scalarized value ˆ g tch_aug ( x  λ )) .
Revised: To train the Pareto set model, we use a gradient-based method to optimize the model parameter θ with respect to the scalarized value ˆ g tch_aug ( x ( λ )) .

Original: With the chainrule, we have:
Revised: With the chain rule, we have:

Original: [Equation]
Revised: [Equation]

Original: where we can calculate each term respectively.
Revised: where we can calculate each term respectively.

Original: • For the ﬁrst term ∂ ˆ g tch_aug ∂ ˆ f , the augmented Tchebycheff scalarization ˆ g tch_aug
Revised: • For the ﬁrst term ∂ ˆ g tch_aug ∂ ˆ f , the augmented Tchebycheff scalarization ˆ g tch_aug

Original: x | λ ) = max 1 ≤ i ≤ m { λ i ( ˆ f i [Equation] x ) − ( z 
Revised: x | λ ) = max 1 ≤ i ≤ m { λ i ( ˆ f i ( x ) − ( z ∗ i − ε )) }

Original: [Equation] − ε )) } + ρ [Equation]
Revised: + ρ (cid:80) mi

Original: mi =

Revised: =

Original: λ i ˆ
Revised: λ i ˆ

Original: f i ( x ) has a max operator which is techni- cally only subdifferentiable, but it is known to have good subgradients [83] for surrogate  optimization.
Revised: f i ( x ) has a max operator which is technically only subdifferentiable, but it is known to have good subgradients [94] for surrogate model based optimization.

Original: We simply take the subgradients for this term.
Revised: We simply take the subgradients for this term.

Original: • The  term ∂ ˆ f ∂ x is the gradient of surrogate values to the input x .
Revised: • The second term ∂ ˆ f ∂ x is the gradient of surrogate values to the input x .

Original: In this work, we build independent Gaussian process models for each objective with Matérn 5/2 kernel.The
Revised: In this work, we build independent Gaussian process models for each objective with Matérn 5/2 kernel.

Original:  gradients of the kernel ∂ k i ∂ x , predictive mean ∂ ˆ µ i ∂ x and standard deviation ∂ ˆ σ i ∂ x for each objective can be easily calculated.
Revised: The gradients of the kernel ∂ k i ∂ x , predictive mean ∂ ˆ µ i ∂ x and standard deviation ∂ ˆ σ i ∂ x for each objective can be easily calculated.

Original: With these terms, the calculation for the surrogate value (e
Revised: With these terms, the calculation for the surrogate value (e

Original: g., predictive mean, UCB and EI) for each objective ∂
Revised: g., predictive mean, UCB and EI) for each objective ∂

Original: ˆ f i ∂ x is also straightforward.
Revised: ˆ f i ∂ x is also straightforward.

Original: • The  term ∂ x ∂ θ
Revised: • The third term ∂ x ∂ θ

Original: =
Revised: =

Original: ∂h θ ( x ) ∂ θ is the gradient of the Pareto set model to the parameter θ .
Revised: ∂h θ ( x ) ∂ θ is the gradient of the Pareto set model to the parameter θ .

Original: In this work, we use a simple MLP as the Pareto set model, and its gradient be easily obtained by auto-differentiation such as in PyTorch [57].
Revised: In this work, we use a simple MLP as the Pareto set model, and its gradient be easily obtained by auto-differentiation such as in PyTorch [64].

Original: For Algorithm 1 in the main paper, we randomly sample K = 10 different preferences { λ k } K =10 k = ∼Λ in batch at each step.
Revised: For Algorithm 1 in the main paper, we randomly sample K = 10 different preferences { λ k } K =10 k = ∼ Λ in batch at each step.

Original: Without any prior information (e.g., user’s preference), we uniformly sample preference λ from [0 , 1] m and then normalize it such that (cid:80) mi [-=
-] λ i = 1 .
Revised: Without any prior information (e.g., user’s preference), we uniformly sample preference λ from [0 , 1] m and then normalize it such that (cid:80) mi [-=
-]=1 λ i = 1 .

Original: The total update step is
Revised: The total update step is

Original: T = 500 for training the Pareto set model at each iteration.
Revised: T = 1000 for training the Pareto set model at each iteration.

Original: The model optimizer is Adam withlearning rate η = 1 e − 3 and no weight decay.
Revised: The model optimizer is Adam with learning rate η = 1 e − 3 and no weight decay.

Original: The learning process typically only requires lessthan 10 seconds to ﬁnish, which is enough to obtain a good Pareto set approximation.
Revised: The learning process typically only requires less than 10 seconds to ﬁnish, which is enough to obtain a good Pareto set approximation.

Original: Detailedexperimental results on the runtime can be found in Table 3.
Revised: Detailed experimental results on the runtime can be found in Table 3.

Original: We use the same model and training procedure for all experiments, and the only problem-dependentsetting is the input/output dimension m and n .
Revised: We use the same model and training procedure for all experiments, and the only problem-dependent setting is the input/output dimension m and n .

Original: The proposed Pareto set model and  learning approach have robust and promising performances for different expensive multi-objective optimizationbenchmarks and real-world problems.
Revised: The proposed Pareto set model and its learning approach have robust and promising performances for different expensive multi-objective optimization benchmarks and real-world problems.

Original: The MOBO with Pareto Set Learning framework (e.g., Algorithm 2 in the main paper) is similar toother MOBO methods, and the main difference is on the batch selection with the learned Pareto set( Algorithm 3 ).
Revised: The MOBO with Pareto Set Learning framework (e.g., Algorithm 2 ) is similar to other MOBO methods, and the main difference is on the batch selection with the learned Pareto set ( Algorithm 3 ).

Original: At each iteration, we ﬁrst randomly sample P = 1000 preferences { λ ( p ) } P =1000 p = ∼ Λand directly obtain their corresponding solutions 
Revised: At each iteration, we ﬁrst randomly sample P = 1000 preferences { λ ( p ) } P =1000 p = ∼ Λ and directly obtain their corresponding solutions X

Original: X
Revised: =

Original: = { x ( λ ( p ) ) } P =1000 p = on the current learned Paretoset M psl .
Revised: x ( λ ( p ) ) } P =1000 p = on the current learned Pareto set M psl .

Original: We then use the predicted hypervolume improvement (HVI) with respect to the alreadyevaluated solutions { X t − 1 , y t − 1 } to select a subset X B ⊂ X for expensive evaluation.
Revised: We then use the predicted hypervolume improvement (HVI) with respect to the already evaluated solutions { X t − 1 , y t − 1 } to select a subset X B ⊂ X for expensive evaluation.

Original: Thesurrogate values for the candidate solutions are directly used for the predicted HVI calculation if theyare the predicted mean ˆ µ or the lower conﬁdence bound ˆ µ − β σ .
Revised: The surrogate values for the candidate solutions are directly used for the predicted HVI calculation if they are the predicted mean ˆ µ or the lower conﬁdence bound ˆ µ − β σ .

Original: When the surrogate values areexpected improvement, although the approximate Pareto set is learned with EI for each scalarization,we simply let ˆ f ( x s ) =
Revised: When the surrogate values are expected improvement, although the approximate Pareto set is learned with EI for each scalarization, we let ˆ f ( x s ) =

Original: ˆ µ − σ for the predicted HVI calculation.
Revised: ˆ µ − σ for the predicted HVI calculation.

Original: The main reasons for this choiceare: 1) we want to avoid the (repeatedly) time-consuming Monte Carlo integration for calculating theexpected hypervolume improvement; 2) the LCB (or the posterior mean only) is on the same scale asthe value of those already-evaluated solutions, which make the calculation of HVI meaningful.
Revised: The main reasons for this choice are: 1) we want to avoid the (repeatedly) time-consuming Monte Carlo integration for calculating the expected hypervolume improvement; 2) the LCB (or the posterior mean only) is on the same scale as the value of those already-evaluated solutions, which make the calculation of HVI meaningful.

Original: Since it is computationally intensive to ﬁnd B solutions to exactly optimize HVI, we select thesolution batch X B in a sequentially greedy manner from X .
Revised: Since it is computationally intensive to ﬁnd B solutions to exactly optimize HVI, we select the solution batch X B in a sequentially greedy manner from X .

Original: Similar approaches have also been usedin the current MOBO methods [47, 15].
Revised: Similar approaches have also been used in the current MOBO methods [53, 17].

Original: The greedy selection approach is presented in Algorithm 4 .
Revised: The greedy selection approach is presented in Algorithm 4 .

Original: We sequentially sample solutions from the candidate set X C , and add the single solution x s with thebest HVI value into the selected set X B .
Revised: We sequentially sample solutions from the candidate set X C , and add the single solution x s with the best HVI value into the selected set X B .

Original: For all problems, the acquisition search procedure (solutionsampling + individual search + batch selection) with batch size B = 10 typically costs less than 3seconds.
Revised: For all problems, the acquisition search procedure (solution sampling + individual search + batch selection) with batch size B = 5 typically costs less thanseconds.

Original: The detailed results can be found in Table 3.
Revised: The detailed results can be found in Table 3.

Original: E Problem details
Revised: E Problem details

Original: [Table]
Revised: [Table]

Original: E.1 Synthetic benchmark problems
Revised: E.1 Synthetic benchmark problems

Original:  We  test our proposed PSL algorithm on different widely-used synthetic multi-objective optimizationbenchmark problems, namely ZDT1-3 [90], VLMOP1-3 [79], and DTLZ2 [19].
Revised: To better evaluate our proposed PSL method [Table] we propose 6 new synthetic test problems with different shapes of Pareto sets which can be found in next page. We also test our proposed PSL algorithm on different widely-used synthetic multi-objective optimization benchmark problems, namely VLMOP13 [88] and DTLZ2 [21].

Original: The input and outputdimensions of these algorithms are shown in Table 2.
Revised: The input and output dimensions of these problems are shown in Table [Table]

Original: These synthetic problems have known Paretosets M ps and Pareto fronts f ( M ps ) with the nadir point:[Equation]
Revised: These synthetic problems have known Pareto sets M ps and Pareto fronts f ( M ps ) with the nadir point:

Original: [Equation]
Revised: [Equation]

Original: where x 1 , . .
Revised: where x 1 , . .

Original: x m are solutions in thePareto set. We set the reference point r = 1 . 1 × z nadir foreach problem. E.2 New problems with various shape of Pareto front
Revised: x m are solutions in the

Original: Among the synthetic benchmarks, the ZDT problem has a simple Pareto set, which is a line on theboundary of the decision space, and most decision variables of all Pareto optimal solutions share thesame values (e.g., 0).

Original: These properties could make ZDT problems not appropriate for comparingdifferent MOBO algorithms.

Original: We chose the ZDT problem for comparison solely due to it is widelyused in multi-objective optimization, and our proposed PSL method is not specially designed forsuch problems.

Original: In this subsection, to better evaluate our proposed PSL method, we propose 6 new test problemsof which the Pareto sets are curves in the search space.

Original: The test problems (F1 - F6) are deﬁnedin the next page, and the experimental results on these newly proposed problems can be found in

Original: Appendix. F.1.

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original:  where s 1 = (cid:88)
Revised:  where s 1 = (cid:88)

Original: j
Revised: j

Original: [Equation]
Revised: [Equation]

Original: J
Revised: J

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: and s 2 = (cid:88)
Revised: and s 2 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: where s 1 = (cid:88)
Revised: where s 1 = (cid:88)

Original: j ∈ J
Revised: j ∈ J

Original: [Equation]
Revised: [Equation]

Original: − x
Revised: − x

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) 2 and s 2 = (cid:88)
Revised: ) 2 and s 2 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original:  where s 1 = (cid:88)
Revised:  where s 1 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: ∈ J( x j − sin
Revised: ∈ J( x j − sin

Original: [Equation]
Revised: [Equation]

Original: 4 π x
Revised: 4 π x

Original: [Equation]
Revised: [Equation]

Original: jπ n
Revised: jπ n

Original: [Equation]
Revised: [Equation]

Original: ) 2 and s 2 = (cid:88)
Revised: ) 2 and s 2 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original:  where s 1 = (cid:88)
Revised:  where s 1 = (cid:88)

Original: j ∈ J
Revised: j ∈ J

Original: ( x j − 0 .
Revised: ( x j − 0 .

Original: 8 x 1 cos(4 π x 1 + jπ n )) 2 and s 2 = (cid:88)
Revised: 8 x 1 cos(4 π x 1 + jπ n )) 2 and s 2 = (cid:88)

Original: j ∈ J( x
Revised: j ∈ J( x

Original: [Equation]
Revised: [Equation]

Original: − 0 .
Revised: − 0 .

Original: 8 x 1 sin(4 π x 1 + jπ n )) 2 ,
Revised: 8 x 1 sin(4 π x 1 + jπ n )) 2 ,

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: [Equation]
Revised: [Equation]

Original: where s 1 = (cid:88) j ∈ J
Revised: where s 1 = (cid:88) j ∈ J

Original: ( x j − 0 .
Revised: ( x j − 0 .

Original: 8 x 1 cos(4 π x 1 + jπn
Revised: 8 x 1 cos(4 π x 1 + jπn

Original: [Equation]
Revised: [Equation]

Original: )) 2 and s 2 = (cid:88)
Revised: )) 2 and s 2 = (cid:88)

Original: j ∈ J
Revised: j ∈ J

Original: ( x j − 0 .
Revised: ( x j − 0 .

Original: 8 x 1 sin
Revised: 8 x 1 sin

Original: [Equation]
Revised: [Equation]

Original: 4 π
Revised: 4 π

Original: [Equation]
Revised: [Equation]

Original: 1 + jπ n ))
Revised: 1 + jπ n ))

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: f
Revised: f

Original: [Equation]
Revised: [Equation]

Original: where s 1 = (cid:88) j ∈ J{ x j − [0 .
Revised: where s 1 = (cid:88) j ∈ J{ x j − [0 .

Original: 3 x 21 cos(12 π x 1 + 4 jπ n ))
Revised: 3 x 21 cos(12 π x 1 + 4 jπ n ))

Original: + 0 .
Revised: + 0 .

Original: 6 x 1 ] cos(6 π x 1 + jπ n ) } and s 2 = (cid:88)
Revised: 6 x 1 ] cos(6 π x 1 + jπ n ) } and s 2 = (cid:88)

Original: j ∈ J{ x j − [0 .
Revised: j ∈ J{ x j − [0 .

Original: 3 x 21 cos(12 π x 1 + 4 jπ n ))
Revised: 3 x 21 cos(12 π x 1 + 4 jπ n ))

Original: + 0 .
Revised: + 0 .

Original: 6 x 1 ] sin(6 π x 1 + jπ n ) } ,
Revised: 6 x 1 ] sin(6 π x 1 + jπ n ) } ,

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: E.3 Real-world application problems
Revised: E.2 Real-world application problems

Original: We also conduct experiments on 5 real-world multi-objective engineering design problems [76] thatare initially proposed in different communities for different applications:
Revised: We also conduct experiments on 5 real-world multi-objective engineering design problems [85] that are initially proposed in different communities for different applications:

Original: Four Bar Truss Design.
Revised: Four Bar Truss Design.

Original: This problem is to design a four-bar truss.
Revised: This problem is to design a four-bar truss.

Original: The two objectives to optimizeare its structural volume and joint displacement.
Revised: The two objectives to optimize are its structural volume and joint displacement.

Original: The decision variables are the length of the fourbars.
Revised: The decision variables are the length of the four bars.

Original: The details of this problem can be found in Cheng and Li [10].
Revised: The details of this problem can be found in Cheng and Li [12].

Original: Pressure Vessel Design.
Revised: Pressure Vessel Design.

Original: This problem is to design a cylindrical pressure vessel.
Revised: This problem is to design a cylindrical pressure vessel.

Original: The two objectivesto minimize are the total cost (material, forming, and welding) and the violations of three differentdesign constraints.
Revised: The two objectives to minimize are the total cost (material, forming, and welding) and the violations of three different design constraints.

Original: This problem has four decision variables, which are the shell thicknesses, thepressure vessel head, the inner radius, and the length of the cylindrical section.
Revised: This problem has four decision variables, which are the shell thicknesses, the pressure vessel head, the inner radius, and the length of the cylindrical section.

Original: The details of thisproblem can be found in Kramer [42].
Revised: The details of this problem can be found in Kramer [46].

Original: Disk Brake Design.
Revised: Disk Brake Design.

Original: This problem is to design a disc brake.
Revised: This problem is to design a disc brake.

Original: The three objectives to minimize arethe mass, the minimum stopping time, and the violations of four design constraints.
Revised: The three objectives to minimize are the mass, the minimum stopping time, and the violations of four design constraints.

Original: This problemhas four decision variables: the inner radius, the outer radius, the engaging force, and the number offriction surfaces.
Revised: This problem has four decision variables: the inner radius, the outer radius, the engaging force, and the number of friction surfaces.

Original: The details of this problem can be found in Ray and Liew [63].
Revised: The details of this problem can be found in Ray and Liew [70].

Original: Gear Train Design.
Revised: Gear Train Design.

Original: This problem is to design a gear train with four gears.
Revised: This problem is to design a gear train with four gears.

Original: The three objectivesto minimize are the difference between the realized gear ration and the required speciﬁcation, themaximum size of four gears, and the design constraint violations.
Revised: The three objectives to minimize are the difference between the realized gear ration and the required speciﬁcation, the maximum size of four gears, and the design constraint violations.

Original: The decision variables are thenumbers of teeth in each of the four gears.
Revised: The decision variables are the numbers of teeth in each of the four gears.

Original: The details of this problem can be found in Deb and
Revised: The details of this problem can be found in Deb and

Original: Srinivasan [17].
Revised: Srinivasan [19].

Original: Rocket Injector Design.
Revised: Rocket Injector Design.

Original: This problem is to design a rocket injector that needs to minimize themaximum temperature of the injector face, the distance from the inlet, and the temperature on the posttip.
Revised: This problem is to design a rocket injector that needs to minimize the maximum temperature of the injector face, the distance from the inlet, and the temperature on the post tip.

Original: It has four decision variables, namely, the hydrogen ﬂow angle, the hydrogen area, the oxygenarea, and the oxidizer post tip thickness.
Revised: It has four decision variables, namely, the hydrogen ﬂow angle, the hydrogen area, the oxygen area, and the oxidizer post tip thickness.

Original: The details of this problem can be found in Vaidyanathanet al.
Revised: The details of this problem can be found in Vaidyanathan et al.

Original: These real-world multi-objective design problems do not have known exact Pareto fronts.
Revised: These real-world multi-objective design problems do not have known exact Pareto fronts.

Original: We use theapproximate Pareto fronts provided by Tanabe and Ishibuchi [76] with a large number of evaluationsas our refereed Pareto fronts, which have also been used in other MOBO works.
Revised: We use the approximate Pareto fronts provided by Tanabe and Ishibuchi [85] with a large number of evaluations as our refereed Pareto fronts, which have also been used in other MOBO works.

Original: We set the referencepoint r = 1 .
Revised: We set the reference point r = 1 .

Original: 1 × ˆ z nadir where ˆ z nadir is the nadir point of the approximate Pareto front.
Revised: 1 × ˆ z nadir where ˆ z nadir is the nadir point of the approximate Pareto front.

Original: The probleminformation and reference points can be found in Table 2.
Revised: The problem information and reference points can be found in Table 2.

Original: E.4 Experiment settings
Revised: E.3 Experiment settings

Original: For all experiments, we ﬁrst randomly sample and evaluate 10 valid solutions with Latin hypercubesampling [50], and set them as the initial solutions { X 0 , y 0 } .
Revised: For all experiments, we ﬁrst randomly sample and evaluate 10 valid solutions with Latin hypercube sampling [57], and set them as the initial solutions { X 0 , y 0 } .

Original: Then we further run the MOBOalgorithm to optimize the given problem with a limited evaluation budget ( 100 for all problems).
Revised: Then we further run the MOBO algorithm to optimize the given problem with a limited evaluation budget ( 100 for all problems).

Original: Inthe main paper, we report the results with a batch size 5 , and there are only 100 / 5 = 20 batchediterations for each algorithm.
Revised: In the main paper, we report the results with a batch size 5 , and there are only 100 / 5 = 20 batched iterations for each algorithm.

Original: In section F in this appendix, we also report experimental results withdifferent batch sizes.F Additional experiments
Revised: In section F in this appendix, we also report experimental results with different batch sizes.

Original: [Figure]

Original:  F.1 Results on new challenging problems F1-F
Revised: F Additional experiments F.1 Run time

Original: [Figure] l o g H V D i

Original: [Figure]

Original: e r e n c e

Original: [Figure]

Original: We run experiments on more challenging problems proposed in Appendix E.2 to further validate our

Original: [Figure]

Original: proposed PSL method.

Original: The Pareto sets of these problems are all different complicated space curves in

Original: [Figure]

Original: the search space.

Original: We have run the experiments with two different settings: 1) 6-dimensional problems

Original: [Figure]

Original: with 100 evaluations, and 2) 20-dimensional problems with 200 evaluations.

Original: The experimental results

Original: [Figure]

Original: in Figure 8 show that our proposed PSL method still has a promising performance on these problems.

Original: [Table]
Revised: [Table]

Original: We report the run time for each algorithm with batch size 10 per iteration in Table 3.
Revised: We report the run time for each algorithm with batch size 10 per iteration in Table 3.

Original: For our proposed
Revised: For our proposed

Original: PSL algorithm, we further report the detailed run time for learning the set model and for searchingthe batch solutions.
Revised: PSL algorithm, we further report the detailed run time for learning the set model and for searching the batch solutions.

Original: According to the results, PSL can efﬁciently learn the set model and select abatch of solutions for evaluation at each iteration with a total time budget of less than 10 seconds.
Revised: According to the results, PSL can efﬁciently learn the set model and select a batch of solutions for evaluation at each iteration with a total time budget of less than 10 seconds.

Original: The short PSL batch selection run time is expected, since all solutions are directly sampled fromthe learned Pareto set with a few further local search steps, rather than optimizing the acquisitionfunction(s) from scratch as in other MOBO methods.
Revised: The short PSL batch selection run time is expected, since all solutions are directly sampled from the learned Pareto set with a few further local search steps, rather than optimizing the acquisition function(s) from scratch as in other MOBO methods.

Original: We want to emphasize that the run time for different MOBO algorithms strongly depends on theimplementation, which is also highlighted in the current works [13, 47].
Revised: We want to emphasize that the run time for different MOBO algorithms strongly depends on the implementation, which is also highlighted in the current works [14, 53].

Original: The objective evaluationsin real-world applications are usually very expensive and involve costly real-world experiments orsimulations that take days to run.
Revised: The objective evaluations in real-world applications are usually very expensive and involve costly real-world experiments or simulations that take days to run.

Original: All the MOBO algorithms are efﬁcient and have negligible run-timeoverheads in these real-world application scenarios.F.3 Performance with different acquisition functions [Figure] [Equation]
Revised: All the MOBO algorithms are efﬁcient and have negligible run-time overheads in these real-world application scenarios.

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: ( =0.1) LCB ( =0.5) LCB ( =1.0) LCB ( =

Original: [Figure]

Original: [Figure]

Original: 100 Function Evaluations 7.

Original: l o g H V D i

Original: ff e r e n c e

Original: EIPredicted

Original: [Figure]

Original: LCB ( =0.1) LCB ( =0.5) LCB ( =1.0) LCB ( =2.0)

Original: [Figure]

Original: For the proposed PSL method, We can use different acquisition functions as the surrogate value forthe Pareto set learning approach.

Original: Here, we compare the performance with 6 different acquisitionfunctions/settings, namely, the Expected Improvement (EI), Predicted Mean, and the Lower Conﬁdence Bound (LCB) with β = 0 .

Original: [-5 , 1 . 0 and 2 . 
-]
Revised: [-5 , 1 . 0 and 2 . 
-]

Original: on four benchmark and real-world applicationproblems.

Original: According to the results shown in Figure 9, PSL with all 6 acquisition functions has reasonablygood performance.

Original: Although PSL with EI performs slightly worse than the other acquisitions, itcan still outperform many other MOBO algorithms with the results shown in Figure 5 of the mainpaper.

Original: These results conﬁrm that the learning the approximate Pareto set during optimization couldbe beneﬁcial for MOBO.

Original: Among the other acquisition functions and settings, no single choice can achieve the best performancefor all problems.

Original: The predicted mean acquisition can be seen as LCB with β = 0 , which is a pureexploitation strategy without considering the uncertainty (e.g., predicted variance) for exploration.

Original: It might be surprising that this greedy acquisition can still have a pretty good performance.

Original: Indeed,this greedy and purely exploitative approach has been recently studied and shown to have promisingperformance for single-objective BO [64, 16].

Original: DGEMO [47] also uses the predicted mean as anacquisition function for MOBO.

Original: The BO textbook [26] brieﬂy discusses this pure exploitative strategyat the end of Section 7.10 (posterior mean acquisition function and lookahead).

Original: One limitation ofthis greedy approach is that it might be stuck at a non-optimal location with no well-ﬁtted surrogatemodel due to overexploitation [26].

Original: There are many acquisition policies for Bayesian optimization, but ﬁnding the most suitable one for a

Original: [Table]

Original: given problem is still an open question [26], which is also the case for our PSL method.

Original: [Table]

Original: F.4 Quality of the learned Pareto front
Revised: F.2 Quality of the learned Pareto set

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: l o g H V D i ff e r e n c e

Original: TS-TCHTS-TCH (Approx. PF) [Figure] In the main paper, we have demonstrated that the proposed PSL method can successfully approximatethe ground truth Pareto front with a limited evaluation budget as in Figure 6.
Revised: In the main paper, we have demonstrated that the proposed PSL method can successfully approximate the ground truth Pareto front with a limited evaluation budget as in Figure 6.

Original: In this subsection, wefurther investigate the performance of the learned Pareto fronts during the optimization process.
Revised: In this subsection, we further investigate the performance of the learned Pareto fronts during the optimization process.

Original: Asshown in Figure 10, the learned Pareto fronts have good quality (with 1,000 random samples) duringoptimization and lead to promising evaluated solutions performance.
Revised: As shown in Figure 8, the learned Pareto fronts have good quality (with 1,000 random samples) during optimization and lead to promising evaluated solutions performance.

Original: In addition, we also compare our Pareto set model with another modeling approach based on evaluatedsolutions.
Revised: In addition, we also compare our Pareto set model with another modeling approach based on evaluated solutions.

Original: Speciﬁcally, we run the TS-TCH algorithm and record the evaluated solutions with theircorresponding preferences during the optimization process.
Revised: Speciﬁcally, we run the TS-TCH algorithm and record the evaluated solutions with their corresponding preferences during the optimization process.

Original: Then we build a model to approximatethe Pareto front based on the evaluated and non-dominated solutions at each iteration step.
Revised: Then we build a model to approximate the Pareto front based on the evaluated and non-dominated solutions at each iteration step.

Original: The modelwe built has an identical structure (2-layer MLP) to the PSL model, but is now trained in a supervisedmanner.
Revised: The model we built has an identical structure (2-layer MLP) to the PSL model, but is now trained in a supervised manner.

Original: According to the results in Figure 10, our proposed PSL model has better quality than themodel trained in the supervised manner with evaluated solutions. F.5 Impact of the aggregation function and Pareto set model
Revised: According to the results in Figure 8, our proposed PSL model has better quality than the model trained in a supervised manner with evaluated solutions

Original: [Figure]
Revised: [Figure]

Original: ff e r e n c e TCH Aggregation (without NN) PSL ( [Figure] NN)
Revised: F.3 Impact of the Pareto set model for scalarization-based method

Original: [Figure]
Revised: [Figure]

Original: 100 Function Evaluations 1. l o g H V D i ff e r e n c e TCH Aggregation (without NN) PSL (with NN)
Revised: In this work

Original: [Figure]
Revised: [Figure]

Original: In this work, we choose the weighted Chebyshev scalarization with an ideal point and epsilon mainlydue to its good theoretical property.
Revised: we choose the weighted Chebyshev scalarization with an ideal point and epsilon mainly due to its good theoretical property.

Original: In this subsection, we conduct an ablation study on our proposed
Revised: In this subsection, we conduct an ablation study on our proposed

Original: PSL method with and without the Pareto set model.
Revised: PSL method with and without the Pareto set model.

Original: The results in Figure 11 show that simplyoptimizing the acquisition function for scalarization could lead to a signiﬁcant performance drop.
Revised: The results in Figure 9 show that simply optimizing the acquisition function for scalarization could lead to a signiﬁcant performance drop.

Original: Itconﬁrms that the proposed Pareto set model is important for the overall promising performance.
Revised: It conﬁrms that the proposed Pareto set model is important for the overall promising performance.

Original: [Figure]
Revised: [Figure]

Original: We show the approximate Pareto set and Pareto front learned by PSL for the ZDT3 problem in Figure.

Original: The ZDT3 problem has a continuous search space, but its Pareto set is disconnected.

Original: According to theresult, PSL can ﬁnd an approximate continuous set that contains the disconnected parts of the Paretoset.

Original: In this work, we use a simple fully-connected neural network as the Pareto set model, and do nottake the disconnected Pareto set into consideration. How to design a more powerful Pareto set modelto capture the disconnected patterns of the  Pareto set (e.g., maybe with search region decomposition)is worthwhile studying in the future. F.7 PSL with different preferences 1.0 f 1 ( x ) 0.
Revised: In this subsection, we conduct an ablation on small perturbing the best points v.s. Pareto set model for generating the candidate set.

Original: [Figure]

Original: f 2 ( x )

Original: Pref 1 Pref 2 Pref

Original: [Figure]

Original: Pref 4 Pref

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Table]

Original: [Table] [Table] [Figure] Suppose the decision-makers have different sets of preferred trade-offs, our model can successfully generate the corresponding parts of the Pareto fronts.
Revised: Based on the results shown in Figure 10, the Gaussian perturbing method can improve the performance of simple scalarization on problems with simple Pareto set (e.g.,

Original: The preference assignment is not ﬁxed, and the decision-makers can adjust it and obtain the corresponding approximate solutions in real-time.

Original: Our proposed PSL method enables the decision-makers to easily adjust their preference to explore thewhole approximate Pareto set for making decisions (e.g., see Figure 3 and Figure 6 in the main paper).

Original: This ability is not supported by other MOBO methods, which is important to ﬂexibly incorporate thedecision-maker’s preference and prior knowledge into Bayesian optimization [26].

Original: The decision-making process for real-world applications could be subjective.

Original: We follow a similar waywith other preference-based MOBO to show our method can successfully generate different trade-offsolutions.

Original: In Figure 13, suppose the decision-makers have different sets of trade-off preferences, ourmodel can generate the corresponding parts of the Pareto fronts for different problems.

Original: We uniformlysample 1 , 000 and 10 , 011 preferences for the two-objective and three-objective problems respectively,and use the learned Pareto set models to generate the corresponding approximate Pareto solutions.

Original: Wefurther divide the preferences into 5 exclusive parts, and label them with the corresponding solutionsin different colors.

Original: As shown in Figure 13, the corresponding connection from the preferences to thesolutions is clear for all problems.

Original: By directly exploring the approximate Pareto front in an interactive manner, the decision-makerscan observe and understand the connection between preferences and corresponding solutions inreal-time.

Original: This ability could be beneﬁcial for the decision-makers to further adjust and assign theirmost accurate preferences.

Original: [Figure]

Original: In addition to the predicted mean, our PSL method also provides the uncertainty information (e.g.,predicted standard deviation) to support decision-making.

Original: Figure 14 shows the (log) predictedstandard deviations for different objectives on the approximate Pareto front of DTLZ2.

Original: According tothe results in Figure 14(a)(b)(c), we can observe that the PSL model has a low predicted standarddeviations for almost all locations on the approximate Pareto front except the top corner and the topleft boundary.

Original: From the log standard deviations in Figure 14(d)(e)(f), we ﬁnd that PSL has a higheroverall uncertainty level for objective 1/2 than objective 3, where the uncertainty distributions arequite similar for the ﬁrst two objectives.

Original: [Figure] l o g H V D i ff e r e n c e
Revised: ZDT1 [102]) but not the problems with complicated Pareto set (e.g., F5 and F6).

Original: NSGA-IIMOEA/D-EGOTSEMOUSeMO-EI

Original: [Figure] -TCHqEVHIMESMOPSL (Ours) [Figure] n
Revised: On all problems, our proposed PSL method still achieves the best performance.

Original: [Figure] e [Figure] l o g H V D i ff e r e n c e (c) ZDT     200 Function Evaluations
Revised: DGEMO [53] also has a local search approach to expand the candidate set around the best points (in terms of surrogate value) with the ﬁrst and second derivatives of the GP surrogate model.

Original: H y p

Original: e r v o l u m e 1e SobolNSGA-IIMOEA/D-EGOqParEGOTS-TCH TSEMOUSeMO-EIMESMOPSL (Ours) (d) DTLZ We also test PSL  on problems with a larger number of decision variables and objectives.
Revised: In our experiments, PSL can outperform DGEMO on most problems, which conﬁrms the importance of the

Original: Figure 15shows the results on 2-objective ZDT1-3 problems with 20 decision variables and the 10 -objective DTLZ2 problem with 50 decision variables. For these problems, all algorithms have 50 randomlygenerated initial solutions, and we conduct 20 batched evaluations with batch size 10. Therefore,there are total 250 evaluations for each algorithm.
Revised: Pareto set model for generating the candidate set.

Original: According to the results, PSL still performs well on ZDT problems with 20 decision variables. Forthe 10 -objective
Revised: On the other hand, the local search methods might provide complementary candidate solutions to

Original: DTLZ2 problem, all algorithms have 0 hypervolume since the obtained solutions arefar from the Pareto front. We do not include the results for DEGMO and qEHVI. The DEGMO codedoes not support problems with larger than 3 objectives. The calculation of expected hypervolume improvement could be time-consuming, and the efﬁcient computation methods proposed in qEHVI [13]and qNEHVI [14] make it possible to properly handle problems with 3-4 objective.
Revised: PSL, especially at the early stage of optimization when the approximate Pareto set is inaccurate.

Original: However, thescalability is still a key limitation for the current MOBO methods.

Original: How to efﬁciently solve problemswith a large number of decision variables and objectives could be an important research direction for MOBO. F.10 Results on other ZDT and DTLZ problems [Figure]
Revised: We will investigate how to efﬁciently combine PSL with the local search approaches in future work.

Original: We run experiments on ZDT4, ZDT6, and DTLZ1 in this subsection.

Original: The results show that theproposed PSL method can achieve promising results on the ZDT6 problem, while none of the MOBOalgorithms can have satisfactory performance for ZDT4 and DTLZ1. Since the obtained solutionsare all far from the ground truth Pareto front and the default reference point, all MOBO methodshave a zero hypervolume value and hence lead to the same value for hypervolume difference. Onepossible reason for the poor performance is that the ZDT4 and DTLZ1 problems have a complicatedoptimization landscape (e.g., with a large number of local optimums). With a limited evaluationbudget, the MOBO methods all fail to build a good surrogate model for such problems, and hencehave poor performance.
Revised: F.5 Comparisons with HVI-LCB

Original: [Figure]
Revised: [Figure]

Original: In this subsection, we conduct an ablation study on maximizing the hypervolume improvement with
Revised: In this subsection, we conduct an ablation study on maximizing the hypervolume improvement with

Original: LCB on the approximate Pareto set vs. searching across the whole design space.
Revised: LCB on the approximate Pareto set vs. searching across the whole design space.

Original: The experimentalresults on three newly proposed problems (i.e., F4, F5, and F6) with complicated Pareto sets areshown in Figure 17.
Revised: The experimental results on three newly proposed problems (i.e., F4, F5, and F6) with complicated Pareto sets are shown in Figure 11.

Original: We also provide the qEHVI results as reference.
Revised: We also provide the qEHVI results as reference.

Original: Based on these results, it is clear that our proposed PSL method can outperform directly searchingthe entire search space to maximize HVI with LCB.
Revised: Based on these results, it is clear that our proposed PSL method can outperform directly searching the entire search space to maximize HVI with LCB.

Original: The qHVI-LCB method performs similarly toqEHVI, which indicates that different acquisitions (for HVI) do not have a signiﬁcant impact on theoptimization performance.
Revised: The qHVI-LCB method performs similarly to qEHVI, which indicates that different acquisitions (for HVI) do not have a signiﬁcant impact on the optimization performance.

Original: In other words, with the learned Pareto set, our proposed PSL methodcan efﬁciently conduct the acquisition optimization mainly on the promising low-dimensional (e.g.,(m-1)) manifold, and then lead to signiﬁcantly better Bayesian optimization performance.
Revised: In other words, with the learned Pareto set, our proposed PSL method can efﬁciently conduct the acquisition optimization mainly on the promising low-dimensional (e.g., (m-1)-dimensional) manifold, and then lead to signiﬁcantly better Bayesian optimization performance.

Original: One possible reason for this performance gap could be the difﬁculty of directly searching the wholedesign search for maximizing HVI.
Revised: One possible reason for this performance gap could be the difﬁculty of directly searching the whole design search for maximizing HVI.

Original: As shown in recent works [89, 15], the performance of qEHVIcan be improved using meta search region management algorithms such as LaMOO [89] and trustregion methods [15].
Revised: As shown in recent works [101, 17], the performance of qEHVI can be improved using meta search region management algorithms such as LaMOO [101] and trust region methods [17].

Original: It implies that directly searching the whole design space is not efﬁcient.
Revised: It implies that directly searching the whole design space is not efﬁcient.

Original: These meta-algorithms adaptively decompose the whole search region into different subregionsalong with the optimization process, while our PSL method directly learns the whole approximate Pareto set at each step.
Revised: These meta-algorithms adaptively decompose the whole search region into different subregions along with the optimization process, while our PSL method directly learns the whole approximate Pareto set at each step.

Original: We will add the comparison with those meta-algorithms in the ﬁnal version(LaMOO [89] is not open-sourced yet, and the code for the trust-region method [15] was just releasedseveral days ago).

Original: Studying how to efﬁciently combine PSL with these algorithms could be animportant future work as discussed in Appendix B.
Revised: Studying how to efﬁciently combine PSL with these algorithms could be an important future work as discussed in Appendix B.

Original: Together with the discussion in our initial response, the advantages of PSL can be summarized fromthe following two viewpoints:

Original: • From the viewpoint of scalarization based methods, PSL proposes a novel two-stage approach to select a small batch of appropriate solutions from the approximate Pareto front, which takes those already-evaluated solutions into consideration. As a result, it signiﬁcantlyoutperforms other scalarization-based methods.• From the viewpoint of hypervolume improvement based methods, in an ideal case, the setof solutions that optimize HVI should all be on the Pareto front. Restricting the search onthe approximate Pareto set, PSL leads to an efﬁcient HVI algorithm.
Revised: F.6 The HVI acquisition values during optimization

Original: [Figure]
Revised: [Figure]

Original: Following your suggestion
Revised: In this subsection, we compare the HVI-LCB acquisition values obtained by different search methods. We run our proposed PSL as the optimization algorithm

Original: [Figure]
Revised: [Figure]

Original: we have added the experiment to compare the HVI-LCB acquisitionvalues obtained by different search methods in Appendix F.11. We run our proposed PSL as theoptimization algorithm, and conduct different search methods at each iteration.
Revised: and conduct different search methods at each iteration.

Original: The compared searchmethods are: 1) searching across the entire design space, 2) ourPLS search on the approximate
Revised: The compared search methods are: 1) searching across the entire design space, 2) our

Original:  Pareto front, and 3) the HVI-LCB acquisition values for the whole approximate Pareto front with1,000 sampled solutions (which can be treated as the upper bound for our method).
Revised: PLS search on the approximate Pareto front, and 3) the HVI-LCB acquisition values for the whole approximate Pareto front with 1,000 sampled solutions (which can be treated as the upper bound for our method).

Original: Based on theresults, it is clear that our proposed PSL search can produce better HVI-LCB acquisition values thandirectly searching the entire design space.
Revised: Based on the results, it is clear that our proposed PSL search can produce better HVI-LCB acquisition values than directly searching the entire design space.

Original: It should be noticed that the reported results are obtained by running PSL, and the search methods[Figure] have the same surrogate functions and baseline HV (from the same evaluated solutions chosen by
Revised: It should be noticed that the reported results are obtained by running PSL, and the search methods have the same surrogate functions and baseline HV (from the same evaluated solutions chosen by

Original: PSL) at each iteration.
Revised: PSL) at each iteration.

Original: By running qHVI-LCB alone, the inefﬁcient acquisition search at eachiteration will lead to the accumulated worse overall performance as shown in Figure.17.
Revised: By running qHVI-LCB alone, the less efﬁcient acquisition search at each iteration will lead to the accumulated worse overall performance as shown in Figure 11.

Original: [Figure]
Revised: [Table]

Original: 4.0 Learned Pareto Front (f) PS: T =0.
Revised: [Table] ( x ) 0 1 2 3 4f 2 ( x ) 1.251. [Figure]

Original: 4.0 Learned Pareto Front (g) PS: T =0. Learned Pareto Front (h) PS: T =0.
Revised: [Figure] f 3 ( x )(g) Disk Brake f 1 ( x ) 0.2 0.4 0.6 0.8 1.

Original: Learned Pareto Front (i) PS: T =0.{+f 2 ( x ) 0.2 0.4 0.
 f 3 ( x ) 0.40.20.00.20.+}
Revised: {+f 2 ( x ) 0.2 0.4 0.
 f 3 ( x ) 0.40.20.00.20.+}

Original: Learned Pareto Front (j) PS: T =
Revised: (h) Rocket Injector Suppose the decision-makers have different sets of preferred trade-offs model can successfully generate the corresponding parts of the Pareto fronts. The preference assignment is not ﬁxed, and the decision-makers can adjust it and obtain the corresponding approximate solutions in real time.

Original: Due to  the simple Pareto set, the optimization performance on ZDT1 could be signiﬁcantly affectedby the initialization.
Revised: Our proposed PSL method enables the decision-makers to easily adjust their preference to explore the whole approximate Pareto set for making decisions.

Original: In the experiment, we did not leverage this property for an unfair comparison onthe ZDT problems.
Revised: This ability is not supported by other

Original: PSL randomly initializes the parameters of the Pareto set model at each step, andthen uses the gradient-based method to learn theapproximate Pareto set.
Revised: MOBO methods, which is important to ﬂexibly incorporate the decision-maker’s preference and prior knowledge into Bayesian optimization [30]. The decision-making process for real-world applications could be subjective. We follow a similar way with other preference-based MOBO to show our method can successfully generate different trade-off solutions. In Figure 13, suppose the decision-makers have different sets of trade-off preferences, our model can generate the corresponding parts of the Pareto fronts for different problems.

Original: An illustration on how PSLlearns the  approximate Pareto set/front is shown in Figure 19.
Revised: We uniformly sample 1 , 000 and 10 , 011 preferences for the two-objective and three-objective problems respectively, and use the learned Pareto set models to generate the corresponding approximate Pareto solutions. We further divide the preferences into 5 exclusive parts, and label them with the corresponding solutions in different colors. As shown in Figure 13, the corresponding connection from the preferences to the solutions is clear for all problems. By directly exploring the approximate Pareto front in an interactive manner, the decision-makers can observe and understand the connection between preferences and corresponding solutions in real-time.

Revised: This ability could be beneﬁcial for the decision-makers to further adjust and assign their most accurate preferences.

Revised: [Figure]

Revised: In addition to the predicted mean, our PSL method also provides the uncertainty information (e.g., predicted standard deviation) to support decision-making.

Revised: Figure 14 shows the (log) predicted standard deviations for different objectives on the approximate Pareto front of DTLZ2.

Revised: According to the results in Figure 14(a)(b)(c), we can observe that the PSL model has low predicted standard deviations for almost all locations on the approximate Pareto front except the top corner and the top left boundary.

Original: In our approach, the initially learned{+log standard deviations in Figure 14(d)(e)(f), we ﬁnd that PSL has a higher overall uncertainty level for objective 1/2 than objective 3, where the uncertainty distributions are quite similar for the ﬁrst two objectives. [Figure] For the proposed PSL method, we can use different acquisition functions as the surrogate value for the Pareto set learning approach. Here, we compare the performance with 6 different acquisition functions/settings, namely, the Expected Improvement (EI), Predicted Mean, and the Lower Conﬁdence Bound (LCB) with β = 0 . 1 , 0 . 5 , 1 . 0 and 2 . 
+}
Revised: From the {+log standard deviations in Figure 14(d)(e)(f), we ﬁnd that PSL has a higher overall uncertainty level for objective 1/2 than objective 3, where the uncertainty distributions are quite similar for the ﬁrst two objectives. [Figure] For the proposed PSL method, we can use different acquisition functions as the surrogate value for the Pareto set learning approach. Here, we compare the performance with 6 different acquisition functions/settings, namely, the Expected Improvement (EI), Predicted Mean, and the Lower Conﬁdence Bound (LCB) with β = 0 . 1 , 0 . 5 , 1 . 0 and 2 . 
+}

Revised: on four benchmark and real-world application problems.

Revised: According to the results shown in Figure 15, PSL with all 6 acquisition functions has reasonably good performance.

Revised: Although PSL with EI performs slightly worse than the other acquisitions, it can still outperform many other MOBO algorithms with the results shown in Figure 5 of the main paper.

Original:  Pareto set  is a random curve in the search space far away from the boundary.
Revised: These results conﬁrm that learning the approximate Pareto set during optimization could be beneﬁcial for MOBO. Among the other acquisition functions and settings, no single choice can achieve the best performance for all problems. The predicted mean acquisition can be seen as LCB with β = 0 , which is a pure exploitation strategy without considering the uncertainty (e.g., predicted variance) for exploration. It might be surprising that this greedy acquisition can still have a pretty good performance.

Original: Then it graduallyconverges to  the Pareto set with the efﬁcient gradient-based update.
Revised: Indeed, this greedy and purely exploitative approach has been recently studied and shown to have promising performance for single-objective BO [71, 18]. DGEMO [53] also uses the predicted mean as an acquisition function for MOBO. Garnett [30] brieﬂy discusses this pure exploitative strategy at the end of Section 7.10 (posterior mean acquisition function and lookahead). One limitation of this greedy approach is that it might be stuck at a non-optimal location with no well-ﬁtted surrogate model due to overexploitation [30].

Original: [Figure]
Revised: There are many acquisition policies for Bayesian optimization, but ﬁnding the most suitable one for a given problem is still an open question [30], which is also the case for our PSL method.

Original: [Figure]
Revised: [Figure]

Revised: each algorithm can be found in subﬁg(a).

Original: In this subsection, we conduct an ablation on small perturbing the best points v.s. Pareto set modelfor generating the candidate set. Based on the results shown in Figure 20, the Gaussian perturbing method can improve the performanceof simple scalarization on  problems  with simple Pareto set (e.g., ZDT1) but not the problems withcomplicated Pareto set (e.g., F4-F6).
Revised: We conduct experiments on more challenging problems proposed in Appendix E.1 with a 20dimensional decision space to further validate our proposed PSL method. The Pareto sets of these problems are all different complicated space curves in the search space. We set the batch size to 10, and run all algorithms with 20 batched evaluations.

Original: On all problems, our proposed PSL method still achieves thebest performance.
Revised: In other words, the evaluation budget is 200 for all problems. The experimental results in Figure 16 show that our proposed PSL method still has a promising performance on these problems.

Original: DGEMO [47] also has a local search approach toexpand the candidate set around the best points(in term of surrogate value) with  the ﬁrst and second  derivatives of the GP surrogate model. In ourexperiments, PSL can outperform DGEMO on most problems, which conﬁrms the importance of the NN for generating the candidate set.
Revised: According to the experimental results, the DGEMO [53] performs the best on these problems with larger decision space. The DGEMO algorithm has a well-designed local search strategy, which fully leverages the ﬁrst and second order derivatives of the Gaussian process surrogate model to enrich its large candidate set for selection.

Original: On the other hand, the local search methods might provide complementary candidate solutions to
Revised: In addition, it also explicitly encourages exploring diverse solutions during the optimization process. In contrast, for problems with larger decision space, our proposed

Original: PSL, especially at the early stage of optimization when the approximate Pareto set is inaccurate.
Revised: PSL method could learn an inaccurate Pareto set, which could mislead and slow down the search process.

Original: Wewill investigate how to efﬁciently combine PSL with the local search approaches in future work.
Revised: In this work, we focus on the main idea of learning the Pareto set for MOBO problems, and do not aggressively include other powerful methods into our proposed algorithm. Its performance might be further improved by considering the second-order derivatives of GPs and diversity-guided batch selection as in DGEMO [53].

Original: F.15 Performance with Different Batch Sizes
Revised: F.11 Performance with Different Batch Sizes

Original: We conduct additional experiments with different batch size ( B = 10 ) as well as sequential evaluation( B = 1 ) for all problems.
Revised: We conduct additional experiments with different batch size ( B = 10 ) as well as sequential evaluation ( B = 1 ) for all problems.

Original: The total number of evaluations is still 10 + 

Revised: The total number of evaluations is still 10 +

Original: =
Revised: =

Original: 110 for all experiments.
Revised: 110 for all experiments.

Original: In other words, these settings have 10 and 100 batched evaluations, respectively.
Revised: In other words, these settings have 10 and 100 batched evaluations, respectively.

Original: The experimentalresults are shown in Figure 21 and Figure 22.
Revised: The experimental results are shown in Figure 17 and Figure 18.

Original: According to the results, our proposed PSL methodconsistently has the best overall performance for different batch sizes. It outperforms many other 
Revised: According to the results, our proposed PSL method has a better or comparable performance with other MOBO methods in most experiments.

Original: MOBO approaches on a largemajority of experiments and has comparable performance with the bestalgorithm when it is not the best one. In this work, we focus on the main idea of learning the Pareto set for MOBO problems, and donot aggressively include other powerful methods into our proposed algorithm. Its performancemight be further improved by considering the second-order derivatives of GPs and diversity-guidedbatch selection in DGEMO [47], as well as Thompson sampling and uncertainty maximization in TSEMO [8] and USeMO-EI [5].
Revised: It also consistently outperforms the model-free scalarization methods with a large performance gap.

Original: These promisingalgorithm performances, along with the efﬁcient run time, conﬁrm that it is worthbuilding the Pareto set model for expensive multi-objective optimization problems.
Revised: These promising performances, along with the efﬁcient run time, conﬁrm that it is worth building the Pareto set model for expensive multi-objective optimization problems.

Original: We believe the
Revised: [Figure] i

Original: Pareto set learning approach could be an efﬁcient off-the-shelf tool for multi-objective Bayesianoptimization.
Revised: [Figure] [Figure] l o g H V D i ff e r e n c e

Original: [Figure]
Revised: [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: r
Revised: r

Original: [Figure]
Revised: [Figure]

Original: n 
Revised: n c e

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Original: (d) VLMOP
Revised: [Figure] [Figure] [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: (f) VLMOP
Revised: l o g H V D i ff e r e n c e

Original: [Figure]
Revised: [Figure]

Original: V D

Original: i
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: n c e (g) DTLZ
Revised: l o g

Original: [Figure]
Revised: [Figure]

Original: V D
Revised: V D

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure] ) VLMOP
Revised: e r e n c e

Original: [Figure]
Revised: [Figure]

Original: (g) DTLZ
Revised: l o g H V D

Original: [Figure]
Revised: i

Original: [Figure]
Revised: [Figure]

Original: l o g H V D i ff e r e n c e 
Revised: e r e n c e (f

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: G Licenses
Revised: G Licenses

Original: Table 4 lists the licenses for the codes and problem suite we used in this work.
Revised: Table 4 lists the licenses for the codes and problem suite we used in this work.

Original: [Table]
Revised: [Table]