Original: Multi-Objective Bayesian Optimization with Pareto Set Learning
Revised: Multi-Objective Bayesian Optimization with Pareto Set Learning

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Expensive multi-objective optimization problems can be found in many real-worldapplications, where their objective function evaluations involve expensive computation and/or physical experiments.
Revised: Expensive multi-objective optimization problems can be found in many real-worldapplications, where their objective function evaluations involve expensive computation and/or physical experiments.

Original: It is desirable to obtain an approximate
Revised: It is desirable to obtain an approximate

Original: Pareto front with a small evaluation budget.
Revised: Pareto front with a small evaluation budget.

Original: Multi-objective Bayesian optimization(MOBO) has been widely used for ﬁnding a ﬁnite set of Pareto optimal solutionsto this problem.
Revised: Multi-objective Bayesian optimization(MOBO) has been widely used for ﬁnding a ﬁnite set of Pareto optimal solutionsto this problem.

Original: However, it is well-known that the whole Pareto set could haveinﬁnite optimal solutions.
Revised: However, it is well-known that the whole Pareto set could haveinﬁnite optimal solutions.

Original: The structural properties of the Pareto set are not wellutilized in existing MOBO methods, and the ﬁnite-set approximation may notcontain the most preferred solution(s) for decision-makers.
Revised: The structural properties of the Pareto set are not wellutilized in existing MOBO methods, and the ﬁnite-set approximation may notcontain the most preferred solution(s) for decision-makers.

Original: This paper develops asimple yet efﬁcient Pareto set learning method to approximate the whole Paretoset for MOBO.
Revised: This paper develops asimple yet efﬁcient Pareto set learning method to approximate the whole Paretoset for MOBO.

Original: We design a simple yet powerful acquisition search method withthe learned Pareto set, which naturally supports batch evaluation.
Revised: We design a simple yet powerful acquisition search method withthe learned Pareto set, which naturally supports batch evaluation.

Original: In addition,with our proposed model, decision-makers can readily explore any trade-off areain the approximate Pareto set for ﬂexible decision-making.
Revised: In addition,with our proposed model, decision-makers can readily explore any trade-off areain the approximate Pareto set for ﬂexible decision-making.

Original: Experimental resultson different synthetic and real-world problems demonstrate the effectiveness andefﬁciency of our proposed method.
Revised: Experimental resultson different synthetic and real-world problems demonstrate the effectiveness andefﬁciency of our proposed method.

Original:  Introduction
Revised:  Introduction

Original: [Figure]
Revised: [Figure]

Original: mate the whole Pareto set.
Revised: mate the whole Pareto set.

Original: With our proposed model, decision-makers can easily explore any trade-off among the objectives to choose their most preferred solutions.
Revised: With our proposed model, decision-makers can easily explore any trade-off among the objectives to choose their most preferred solutions.

Original: Many real-world applications involve optimizing multiple costly-to-evaluate and potentially competingobjectives, such as ﬁnding strong yet ductile material [34], building a neural network with high accuracy and low latency [24], and improving the qualitywhile minimizing total charge in particle accelerator tuning [66].
Revised: Many real-world applications involve optimizing multiple costly-to-evaluate and potentially competingobjectives, such as ﬁnding strong yet ductile material [34], building a neural network with high accuracy and low latency [24], and improving the qualitywhile minimizing total charge in particle accelerator tuning [66].

Original: Very often, these objectives couldconﬂict with each other and cannot be optimized simultaneously by a single solution.
Revised: Very often, these objectives couldconﬂict with each other and cannot be optimized simultaneously by a single solution.

Original: Instead, there isa set of solutions with different optimal trade-offsamong the objectives, called the Pareto set.
Revised: Instead, there isa set of solutions with different optimal trade-offsamong the objectives, called the Pareto set.

Original: For a
Revised: For a

Original: Pareto optimal solution, none of its objective valuescan be further improved without deteriorating others.
Revised: Pareto optimal solution, none of its objective valuescan be further improved without deteriorating others.

Original: In addition, the evaluation of each solution requirestime-consuming computation or costly physical experiment, and thus a large number of evaluations arenot acceptable.
Revised: In addition, the evaluation of each solution requirestime-consuming computation or costly physical experiment, and thus a large number of evaluations arenot acceptable.

Original: Different multi-objective Bayesianoptimization (MOBO) algorithms [40, 44, 39], typically directly generalized from the single-objective
Revised: Different multi-objective Bayesianoptimization (MOBO) algorithms [40, 44, 39], typically directly generalized from the single-objective

Original: Bayesian optimization (BO) [53, 35, 9, 68, 25], have been proposed to ﬁnd a small set of approximate
Revised: Bayesian optimization (BO) [53, 35, 9, 68, 25], have been proposed to ﬁnd a small set of approximate

Original: Pareto solutions with a limited evaluation budget.
Revised: Pareto solutions with a limited evaluation budget.

Original: However, the ﬁnite set approximation has undesirable drawbacks.
Revised: However, the ﬁnite set approximation has undesirable drawbacks.

Original: For a nontrivial multi-objectiveoptimization problem, the Pareto set is on a continuous manifold and has inﬁnite solutions withdifferent optimal trade-offs among the objectives [51].
Revised: For a nontrivial multi-objectiveoptimization problem, the Pareto set is on a continuous manifold and has inﬁnite solutions withdifferent optimal trade-offs among the objectives [51].

Original: This Pareto set structure could be helpfulto better select candidate solutions for expensive evaluation, and hence accelerate the optimizationprocess of MOBO.
Revised: This Pareto set structure could be helpfulto better select candidate solutions for expensive evaluation, and hence accelerate the optimizationprocess of MOBO.

Original: In addition, a small set of solutions may not contain the one(s)
Revised: In addition, a small set of solutions may not contain the one(s)

Original: that precisely satisfythe decision-maker
Revised: that precisely satisfythe decision-maker

Original: ’s preferences.
Revised: ’s preferences.

Original: Finding the most preferred trade-off solution(s) could requireseveral rounds of communication with the decision-makers.
Revised: Finding the most preferred trade-off solution(s) could requireseveral rounds of communication with the decision-makers.

Original: These approaches would be extremelytime-consuming and undesirable, especially when the optimization modeler and the decision-makerare not in the same team, which is common in real-world MOBO applications [49].
Revised: These approaches would be extremelytime-consuming and undesirable, especially when the optimization modeler and the decision-makerare not in the same team, which is common in real-world MOBO applications [49].

Original: In this work, we propose a novel Pareto set learning (PSL) method to approximate the whole Paretoset for expensive multi-objective optimization problems.
Revised: In this work, we propose a novel Pareto set learning (PSL) method to approximate the whole Paretoset for expensive multi-objective optimization problems.

Original: Our proposed method can accelerate themulti-objective Bayesian optimization process, and also provide decision-makers with more usefulinformation to support ﬂexible decision-making.
Revised: Our proposed method can accelerate themulti-objective Bayesian optimization process, and also provide decision-makers with more usefulinformation to support ﬂexible decision-making.

Original: We believe it is a principled way to deal with
Revised: We believe it is a principled way to deal with

Original: MOBO.
Revised: MOBO.

Original: This work has the following main contributions:
Revised: This work has the following main contributions:

Original: • We propose a novel set model to map any trade-off preference to its corresponding Pareto
Revised: • We propose a novel set model to map any trade-off preference to its corresponding Pareto

Original: solution, along with a surrogate model-based method to approximate the whole Pareto set
Revised: solution, along with a surrogate model-based method to approximate the whole Pareto set

Original: with a limited evaluation budget.
Revised: with a limited evaluation budget.

Original: • We develop a lightweight yet powerful batch acquisition search method on the approximate
Revised: • We develop a lightweight yet powerful batch acquisition search method on the approximate

Original: Pareto set for efﬁcient MOBO, which can outperform other MOBO approaches in bothperformance and runtime.
Revised: Pareto set for efﬁcient MOBO, which can outperform other MOBO approaches in bothperformance and runtime.

Original: We also demonstrate that the learned Pareto set can support
Revised: We also demonstrate that the learned Pareto set can support

Original: ﬂexible user-involved decision-making.
Revised: ﬂexible user-involved decision-making.

Original: • We test our proposed method on both synthetic benchmarks and real-world application
Revised: • We test our proposed method on both synthetic benchmarks and real-world application

Original: problems.
Revised: problems.

Original: The results fully validate the efﬁciency and usefulness of PSL.
Revised: The results fully validate the efﬁciency and usefulness of PSL.

Original:  Related work
Revised:  Related work

Original: Bayesian Optimization.
Revised: Bayesian Optimization.

Original: Surrogate model-based methods have been widely studied for expensiveoptimization [43, 36, 58, 70].
Revised: Surrogate model-based methods have been widely studied for expensiveoptimization [43, 36, 58, 70].

Original: This method iteratively builds a surrogate model to approximatethe black-box objective function, and uses an acquisition function to search optimal solution.
Revised: This method iteratively builds a surrogate model to approximatethe black-box objective function, and uses an acquisition function to search optimal solution.

Original: Dueto its importance in machine learning, much effort has been made on various design issues in
Revised: Dueto its importance in machine learning, much effort has been made on various design issues in

Original: Bayesian optimization, such as acquisition functions [72], high-dimensional optimization [82], batchevaluation [20], scalable optimization [71], and theoretical analysis [38].
Revised: Bayesian optimization, such as acquisition functions [72], high-dimensional optimization [82], batchevaluation [20], scalable optimization [71], and theoretical analysis [38].

Original: Most of them are onsingle-objective optimization.
Revised: Most of them are onsingle-objective optimization.

Original: We refer readers to Garnett [26] for a comprehensive introduction.
Revised: We refer readers to Garnett [26] for a comprehensive introduction.

Original: Multi-Objective Bayesian Optimization.
Revised: Multi-Objective Bayesian Optimization.

Original: MOBO extends single-objective Bayesian optimization for solving expensive multi-objective optimization problems.
Revised: MOBO extends single-objective Bayesian optimization for solving expensive multi-objective optimization problems.

Original: The scalarization-based algorithms, such as ParEGO [41] and TS-TCH [55] iteratively scalarize the multi-objective probleminto single-objective ones with random preferences, and then apply single-objective BO to solvethem.
Revised: The scalarization-based algorithms, such as ParEGO [41] and TS-TCH [55] iteratively scalarize the multi-objective probleminto single-objective ones with random preferences, and then apply single-objective BO to solvethem.

Original: MOEA/D-EGO [87] extend the MOEA/D framework [85] to solve a set of surrogate scalarizedsubproblems simultaneously.
Revised: MOEA/D-EGO [87] extend the MOEA/D framework [85] to solve a set of surrogate scalarizedsubproblems simultaneously.

Original: SMS-EGO [60] and PAL [92, 93] generalize the upper conﬁdencebound (UCB) to multi-objective optimization.
Revised: SMS-EGO [60] and PAL [92, 93] generalize the upper conﬁdencebound (UCB) to multi-objective optimization.

Original: Emmerich et al.
Revised: Emmerich et al.

Original: [23] and Emmerich and Klinkenberg[22] propose the probability of improvement (PI) and expected improvement (EI) for multi-objectivehypervolume.
Revised: [23] and Emmerich and Klinkenberg[22] propose the probability of improvement (PI) and expected improvement (EI) for multi-objectivehypervolume.

Original: Entropy search methods [29, 31, 33, 81] have also been explored in multi-objectiveoptimization [30, 4, 75].
Revised: Entropy search methods [29, 31, 33, 81] have also been explored in multi-objectiveoptimization [30, 4, 75].

Original: Bradford et al.
Revised: Bradford et al.

Original: [8] and Belakaria et al.
Revised: [8] and Belakaria et al.

Original: [5] consider Thompson sampling anduncertainty maximization for multi-objective optimization, respectively.
Revised: [5] consider Thompson sampling anduncertainty maximization for multi-objective optimization, respectively.

Original: Different algorithms canalso be efﬁciently combined to achieve better overall performance [73].
Revised: Different algorithms canalso be efﬁciently combined to achieve better overall performance [73].

Original: The current works have investigated the diverse batch evaluation [47], efﬁcient hypervolume improvement calculation [13], multiple noisy objectives [14], and decision criteria beyond Paretooptimality [49].
Revised: The current works have investigated the diverse batch evaluation [47], efﬁcient hypervolume improvement calculation [13], multiple noisy objectives [14], and decision criteria beyond Paretooptimality [49].

Original: Some attempts have been made to incorporate the decision-maker’s preference into
Revised: Some attempts have been made to incorporate the decision-maker’s preference into

Original: MOBO [1, 55, 2].
Revised: MOBO [1, 55, 2].

Original: They typically need decision-maker’s preference before or during optimization,which could be unavailable in many real-world applications.
Revised: They typically need decision-maker’s preference before or during optimization,which could be unavailable in many real-world applications.

Original: Structure Learning.
Revised: Structure Learning.

Original: In addition to the surrogate objective model, some attempts have been made toexplore the problem structure for Bayesian optimization.
Revised: In addition to the surrogate objective model, some attempts have been made toexplore the problem structure for Bayesian optimization.

Original: Sener and Koltun [67] learn the geometricstructure of the problem in an online manner to accelerate optimization.
Revised: Sener and Koltun [67] learn the geometricstructure of the problem in an online manner to accelerate optimization.

Original: Wang et al.
Revised: Wang et al.

Original: [80] and Zhaoet al.
Revised: [80] and Zhaoet al.

Original: [89] apply Monte Carlo tree search (MCTS) to divide the search space for efﬁcient modeling
Revised: [89] apply Monte Carlo tree search (MCTS) to divide the search space for efﬁcient modeling

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: f
Revised: f

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: x f
Revised: x f

Original: [Figure]
Revised: [Figure]

Original: The idea to approximate the whole Pareto set manifold has been studied in the multi-objectiveoptimization community [61, 32, 27].
Revised: The idea to approximate the whole Pareto set manifold has been studied in the multi-objectiveoptimization community [61, 32, 27].

Original: Pirotta et al.
Revised: Pirotta et al.

Original: [59] and Parisi et al.
Revised: [59] and Parisi et al.

Original: [56] have explored the Paretomanifold approximation method for multi-objective reinforcement learning.
Revised: [56] have explored the Paretomanifold approximation method for multi-objective reinforcement learning.

Original: Apart from optimization,different approaches have been investigated to incorporate the preference information into deep neuralnetworks for image style transfer [69, 21] and multi-task learning [48, 54, 45].
Revised: Apart from optimization,different approaches have been investigated to incorporate the preference information into deep neuralnetworks for image style transfer [69, 21] and multi-task learning [48, 54, 45].

Original: In this work, wepropose to learn a set model which maps all valid trade-off preferences to the approximate Pareto set,and use it for efﬁcient MOBO.
Revised: In this work, wepropose to learn a set model which maps all valid trade-off preferences to the approximate Pareto set,and use it for efﬁcient MOBO.

Original:  Expensive multi-objective optimization
Revised:  Expensive multi-objective optimization

Original: We consider the following expensive continuous multi-objective optimization problem:
Revised: We consider the following expensive continuous multi-objective optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where x is a solution in the decision space
Revised: where x is a solution in the decision space

Original: X ⊂ R n , f : X → R m is an m -dimensional vector-valuedobjective function, and the evaluation is expensive for all individual objectives { f i ( x ) } mi =1 .
Revised: X ⊂ R n , f : X → R m is an m -dimensional vector-valuedobjective function, and the evaluation is expensive for all individual objectives { f i ( x ) } mi =1 .

Original: For anon-trivial problem, no single best solution can optimize all objectives at the same time, and we haveto make a trade-off among them.
Revised: For anon-trivial problem, no single best solution can optimize all objectives at the same time, and we haveto make a trade-off among them.

Original: We have the following deﬁnitions for multi-objective optimization:
Revised: We have the following deﬁnitions for multi-objective optimization:

Original: Deﬁnition 1 (Pareto Dominance and Strict Dominance) Let x a , x b ∈ X , x a is said to dominatex b , denoted as x a ≺ x b , if and only if f i ( x a ) ≤ f i ( x b ) , ∀ i ∈ { 1 , ..., m } and ∃ j ∈ { 1 , ..., m } suchthat f j ( x a ) <
Revised: Deﬁnition 1 (Pareto Dominance and Strict Dominance) Let x a , x b ∈ X , x a is said to dominatex b , denoted as x a ≺ x b , if and only if f i ( x a ) ≤ f i ( x b ) , ∀ i ∈ { 1 , ..., m } and ∃ j ∈ { 1 , ..., m } suchthat f j ( x a ) <

Original: f j ( x ( b ) ) .
Revised: f j ( x ( b ) ) .

Original: In addition, x a is said to strictly dominate x b ( x a ≺ strict x b ), if and only iff i ( x a ) <
Revised: In addition, x a is said to strictly dominate x b ( x a ≺ strict x b ), if and only iff i ( x a ) <

Original: f i ( x b ) , ∀ i ∈ { 1 , ..., m } .
Revised: f i ( x b ) , ∀ i ∈ { 1 , ..., m } .

Original: Deﬁnition 2 (Pareto Optimality) A solution x ∗ ∈ X is Pareto optimal if there is no ˆ x ∈ X suchthat ˆ x ≺ x ∗ .
Revised: Deﬁnition 2 (Pareto Optimality) A solution x ∗ ∈ X is Pareto optimal if there is no ˆ x ∈ X suchthat ˆ x ≺ x ∗ .

Original: A solution x (cid:48) ∈
Revised: A solution x (cid:48) ∈

Original: X is weakly Pareto optimal if there is no ˆ x ∈ X such that ˆ x ≺ strict x (cid:48) .
Revised: X is weakly Pareto optimal if there is no ˆ x ∈ X such that ˆ x ≺ strict x (cid:48) .

Original: Deﬁnition 3 (Pareto Set/Front) The set of all Pareto optimal solutions M ps ⊆
Revised: Deﬁnition 3 (Pareto Set/Front) The set of all Pareto optimal solutions M ps ⊆

Original: X is called the
Revised: X is called the

Original: Pareto set, and its image in the objective space f ( M ps )
Revised: Pareto set, and its image in the objective space f ( M ps )

Original: =
Revised: =

Original: { f ( x ) | x ∈ M ps } is called the Paretofront.
Revised: { f ( x ) | x ∈ M ps } is called the Paretofront.

Original: Similarly, we can deﬁne the weakly Pareto set M weak and weakly Pareto front f ( M weak ) .
Revised: Similarly, we can deﬁne the weakly Pareto set M weak and weakly Pareto front f ( M weak ) .

Original: The strict dominance relation is stronger than the Pareto dominance since it requires strictly bettervalues for all objectives.
Revised: The strict dominance relation is stronger than the Pareto dominance since it requires strictly bettervalues for all objectives.

Original: Therefore, the set of weakly Pareto optimal solutions M weak (e.g., thesolutions that are not strictly dominated) would be larger than M ps , and it is straightforward to see
Revised: Therefore, the set of weakly Pareto optimal solutions M weak (e.g., thesolutions that are not strictly dominated) would be larger than M ps , and it is straightforward to see

Original: M ps ⊆ M weak .
Revised: M ps ⊆ M weak .

Original: The illustration of (weakly) Pareto solution and Pareto front is shown in Figure 2.
Revised: The illustration of (weakly) Pareto solution and Pareto front is shown in Figure 2.

Original: Each Pareto solution x ∈ M ps represents a different optimal trade-off among the objectives forproblem (1).
Revised: Each Pareto solution x ∈ M ps represents a different optimal trade-off among the objectives forproblem (1).

Original: Under mild conditions, the Pareto set M ps and Pareto front f ( M ps ) are both on ( m − 1) - dimensional manifold in the decision space X ∈ R n and objective space R m , respectively [51, 32, 86].
Revised: Under mild conditions, the Pareto set M ps and Pareto front f ( M ps ) are both on ( m − 1) - dimensional manifold in the decision space X ∈ R n and objective space R m , respectively [51, 32, 86].

Original: The number of Pareto solutions could be inﬁnite ( |M ps | = ∞ ).
Revised: The number of Pareto solutions could be inﬁnite ( |M ps | = ∞ ).

Original: Bayesian Optimization (BO) [26] is a model-based method for solving expensive black-box optimization problems.
Revised: Bayesian Optimization (BO) [26] is a model-based method for solving expensive black-box optimization problems.

Original: Given a set of already-evaluated solutions { X , y } , BO builds surrogate models(e.g., Gaussian process) for each objective, and deﬁnes acquisition function(s) to leverage the surrogate objective values for navigating the search space.
Revised: Given a set of already-evaluated solutions { X , y } , BO builds surrogate models(e.g., Gaussian process) for each objective, and deﬁnes acquisition function(s) to leverage the surrogate objective values for navigating the search space.

Original: Only promising solutions will be selected forexpensive evaluation.
Revised: Only promising solutions will be selected forexpensive evaluation.

Original: We refer interesting reader to [26] for a detailed introduction.
Revised: We refer interesting reader to [26] for a detailed introduction.

Original: Pareto Set Learning.
Revised: Pareto Set Learning.

Original: The current MOBO methods aim to ﬁnd a small set of ﬁnite solutions
Revised: The current MOBO methods aim to ﬁnd a small set of ﬁnite solutions

Original: S =
Revised: S =

Original: { ¯ x (1) , ¯ x (2) , · · · , ¯ x ( |S| ) } to approximate the Pareto set M ps .
Revised: { ¯ x (1) , ¯ x (2) , · · · , ¯ x ( |S| ) } to approximate the Pareto set M ps .

Original: In addition to the evaluatedsolutions S , our proposed PSL method also learns an estimated Pareto set M psl with the predicted
Revised: In addition to the evaluatedsolutions S , our proposed PSL method also learns an estimated Pareto set M psl with the predicted

Original: Pareto front ˆ f ( M psl ) to approximate the Pareto set M ps and Pareto front f ( M ps ) .
Revised: Pareto front ˆ f ( M psl ) to approximate the Pareto set M ps and Pareto front f ( M ps ) .

Original: The wholeapproximate Pareto set can be easily explored by adjusting the trade-off preference as illustrated in
Revised: The wholeapproximate Pareto set can be easily explored by adjusting the trade-off preference as illustrated in

Original: Figure 3.
Revised: Figure 3.

Original: With the learned Pareto set, we also develop an efﬁcient batched solution selection approachfor efﬁcient MOBO.
Revised: With the learned Pareto set, we also develop an efﬁcient batched solution selection approachfor efﬁcient MOBO.

Original:  Pareto Set Learning for MOBO
Revised:  Pareto Set Learning for MOBO

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: As discussed in Section 3, the Pareto set cancontain inﬁnite solutions with different tradeoffs which are non-dominated with each other.
Revised: As discussed in Section 3, the Pareto set cancontain inﬁnite solutions with different tradeoffs which are non-dominated with each other.

Original: In other words, there is no clear order and rankamong the Pareto solutions.
Revised: In other words, there is no clear order and rankamong the Pareto solutions.

Original: A Pareto set modelfor MOBO should be powerful enough to approximate the whole Pareto set, and convenientenough to easily explore any trade-off solutions.
Revised: A Pareto set modelfor MOBO should be powerful enough to approximate the whole Pareto set, and convenientenough to easily explore any trade-off solutions.

Original: In this work, we propose to build a model thatmaps any trade-off preferences to their corresponding Pareto solutions with scalarization.
Revised: In this work, we propose to build a model thatmaps any trade-off preferences to their corresponding Pareto solutions with scalarization.

Original: Scalarization.
Revised: Scalarization.

Original: The scalarization method provides a natural connection from a set of wellorganized preferences Λ = { λ ∈ R m + | (cid:80) λ i =1 } among the m optimization objective to the
Revised: The scalarization method provides a natural connection from a set of wellorganized preferences Λ = { λ ∈ R m + | (cid:80) λ i =1 } among the m optimization objective to the

Original: Pareto set M ps .
Revised: Pareto set M ps .

Original: The most simple and straightforward approach is the linear weight-sum scalarization:
Revised: The most simple and straightforward approach is the linear weight-sum scalarization:

Original: [Equation]
Revised: [Equation]

Original: However, this method can only ﬁnd the convexhull of Pareto front f ( M ps ) [12].
Revised: However, this method can only ﬁnd the convexhull of Pareto front f ( M ps ) [12].

Original: In this work, we focus on the weighted Tchebycheff approach:
Revised: In this work, we focus on the weighted Tchebycheff approach:

Original: [Equation]
Revised: [Equation]

Original: (3) where z ∗ =
Revised: (3) where z ∗ =

Original: ( z ∗ 1 , · · · , z ∗ m ) is the ideal vector for the objective vector f ( x ) (lower-bound for minimization problem), ε > 0 is a small positive scalar, and u i =
Revised: ( z ∗ 1 , · · · , z ∗ m ) is the ideal vector for the objective vector f ( x ) (lower-bound for minimization problem), ε > 0 is a small positive scalar, and u i =

Original: ( z ∗ i − ε ) is called an (unachievable)utopia value for the i -th objective f
Revised: ( z ∗ i − ε ) is called an (unachievable)utopia value for the i -th objective f

Original: i ( x ) .
Revised: i ( x ) .

Original: This scalarization method has a promising property:
Revised: This scalarization method has a promising property:

Original: Theorem 1 (Choo and Atkins [11]).
Revised: Theorem 1 (Choo and Atkins [11]).

Original: A feasible solution x ∈ X is weakly Pareto optimal if and onlyif there is a weight vector λ > 

Revised: A feasible solution x ∈ X is weakly Pareto optimal if and onlyif there is a weight vector λ >

Original: such that x is an optimal solution of the problem (3).
Revised: such that x is an optimal solution of the problem (3).

Original: In other words, all Pareto solutions x ∈ M ps can be found by solving the Tchebycheff scalarizedsubproblem (3) with a speciﬁc (but unknown) trade-off preference λ .
Revised: In other words, all Pareto solutions x ∈ M ps can be found by solving the Tchebycheff scalarizedsubproblem (3) with a speciﬁc (but unknown) trade-off preference λ .

Original: We let M tch be the solutionset for problem (3) with all valid preferences Λ and have M ps ⊆ M weak =
Revised: We let M tch be the solutionset for problem (3) with all valid preferences Λ and have M ps ⊆ M weak =

Original: M tch .
Revised: M tch .

Original: The weakly
Revised: The weakly

Original: Pareto optimal but not Pareto optimal solutions (e.g., M weak \ M ps ) are dominated (but not strictlydominated) by some Pareto solutions, and are usually not desirable for decision-making.
Revised: Pareto optimal but not Pareto optimal solutions (e.g., M weak \ M ps ) are dominated (but not strictlydominated) by some Pareto solutions, and are usually not desirable for decision-making.

Original: They can be
Revised: They can be

Original: [Figure]
Revised: [Figure]

Original: further avoided by the augmented Tchebycheff approach [74, 37].
Revised: further avoided by the augmented Tchebycheff approach [74, 37].

Original: In this work, we use the followingscalarization:
Revised: In this work, we use the followingscalarization:

Original: [Equation]
Revised: [Equation]

Original: where ρ is a sufﬁciently small positive scalar depends on the problem and current solution location.
Revised: where ρ is a sufﬁciently small positive scalar depends on the problem and current solution location.

Original: This form of augmentation has also been used in ParEGO [41].
Revised: This form of augmentation has also been used in ParEGO [41].

Original: With the augmentation term, theweakly dominated solutions will have larger scalarized values than the corresponding Pareto solutionsin (4), and will ultimately be eliminated with the optimization process (e.g., M tch_aug =
Revised: With the augmentation term, theweakly dominated solutions will have larger scalarized values than the corresponding Pareto solutionsin (4), and will ultimately be eliminated with the optimization process (e.g., M tch_aug =

Original: M ps ).
Revised: M ps ).

Original: In thiswork, we simply set ρ = 0 .
Revised: In thiswork, we simply set ρ = 0 .

Original: 001 , dynamically update z ∗ i as the current best value for each objectiveand let ε = 0 .
Revised: 001 , dynamically update z ∗ i as the current best value for each objectiveand let ε = 0 .

Original: | z ∗ | .
Revised: | z ∗ | .

Original: This setting is robust for all problems we considered.
Revised: This setting is robust for all problems we considered.

Original: Set Model.
Revised: Set Model.

Original: With augmented Tchebycheff scalarization, we build a set model for mapping preferencesto their solutions:
Revised: With augmented Tchebycheff scalarization, we build a set model for mapping preferencesto their solutions:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where λ is any valid preference in Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , x
Revised: where λ is any valid preference in Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , x

Original: ( λ ) ∈
Revised: ( λ ) ∈

Original: X is its corresponding
Revised: X is its corresponding

Original: Pareto solution, and h θ ( λ ) is the Pareto set model with parameter θ .
Revised: Pareto solution, and h θ ( λ ) is the Pareto set model with parameter θ .

Original: Since all valid preferencesλ has ( m − 1 ) degree of freedom, the set of all generated solutions M psl = { x = h θ ( λ ) | λ ∈ Λ }is on an ( m − 1 )-dimensional manifold in X ∈ R n .
Revised: Since all valid preferencesλ has ( m − 1 ) degree of freedom, the set of all generated solutions M psl = { x = h θ ( λ ) | λ ∈ Λ }is on an ( m − 1 )-dimensional manifold in X ∈ R n .

Original: In other words, the set model maps the( m − 1 )-dimensional regular preference simplex Λ to the ( m − 1 )-dimensional solution set M pslwith complicated structure.
Revised: In other words, the set model maps the( m − 1 )-dimensional regular preference simplex Λ to the ( m − 1 )-dimensional solution set M pslwith complicated structure.

Original: We want to ﬁnd the optimal parameters θ ∗ such that the generated set M psl matches the augmented
Revised: We want to ﬁnd the optimal parameters θ ∗ such that the generated set M psl matches the augmented

Original: Tchebycheff scalarization’s set M tch_aug
Revised: Tchebycheff scalarization’s set M tch_aug

Original: = { x ∗ ( λ ) | λ ∈ Λ } , where
Revised: = { x ∗ ( λ ) | λ ∈ Λ } , where

Original: [Equation]
Revised: [Equation]

Original: An illustration of the learned mapping is shown in Figure 3.
Revised: An illustration of the learned mapping is shown in Figure 3.

Original: Once the connection is learned, we canefﬁciently navigate the whole approximate Pareto set/front by simply adjusting the preferences amongobjectives.
Revised: Once the connection is learned, we canefﬁciently navigate the whole approximate Pareto set/front by simply adjusting the preferences amongobjectives.

Original: In this work, we use an MLP neural network as the set learning model for all MOBOproblems, which was shown to be powerful enough to capture complicated problem structure [67].
Revised: In this work, we use an MLP neural network as the set learning model for all MOBOproblems, which was shown to be powerful enough to capture complicated problem structure [67].

Original: The model details can be found in Appendix.
Revised: The model details can be found in Appendix.

Original: 4.2 Pareto Set Learning with Gaussian Process
Revised: 4.2 Pareto Set Learning with Gaussian Process

Original: Since the evaluation of f ( x ( λ ))
Revised: Since the evaluation of f ( x ( λ ))

Original: =
Revised: =

Original: f ( h θ ( λ )) is expensive, we use surrogate models to learn the
Revised: f ( h θ ( λ )) is expensive, we use surrogate models to learn the

Original: Pareto set model h θ ( λ ) as shown in Figure 4.
Revised: Pareto set model h θ ( λ ) as shown in Figure 4.

Original: Our method is orthogonal to the choice of surrogatemodels, and we build independent Gaussian process model for each objective as in other MOBOapproaches [47, 49].
Revised: Our method is orthogonal to the choice of surrogatemodels, and we build independent Gaussian process model for each objective as in other MOBOapproaches [47, 49].

Original: Gaussian Process Model.
Revised: Gaussian Process Model.

Original: A single-objective Gaussian process [62] has a prior distribution deﬁnedon the function space:
Revised: A single-objective Gaussian process [62] has a prior distribution deﬁnedon the function space:

Original: [Equation]
Revised: [Equation]

Original: where µ : X → R is the mean function and k :
Revised: where µ : X → R is the mean function and k :

Original: X × X
Revised: X × X

Original: → R is the covariance kernel function.
Revised: → R is the covariance kernel function.

Original: Withn evaluated solutions { X , y } =
Revised: Withn evaluated solutions { X , y } =

Original: { [ x ( i ) ] , [ f ( x ( i ) ] | i = 1 , . . . , n ) } , the posterior distribution can beupdated by maximizing the marginal likelihood based on the data.
Revised: { [ x ( i ) ] , [ f ( x ( i ) ] | i = 1 , . . . , n ) } , the posterior distribution can beupdated by maximizing the marginal likelihood based on the data.

Original: For a new solution x n +1 , theposterior mean and variance are:
Revised: For a new solution x n +1 , theposterior mean and variance are:

Original: [Equation]
Revised: [Equation]

Original: where k = k ( x , X ) is the kernel vector and K
Revised: where k = k ( x , X ) is the kernel vector and K

Original: = k
Revised: = k

Original: ( X , X ) is the kernel matrix (Matérn 5/2kernel is used for all models).
Revised: ( X , X ) is the kernel matrix (Matérn 5/2kernel is used for all models).

Original: For m independent GP models, we let ˆ µ ( x ) =
Revised: For m independent GP models, we let ˆ µ ( x ) =

Original: [ˆ µ 1 ( x ) , · · · , ˆ µ m ( x )]and ˆ σ 2 ( x ) =
Revised: [ˆ µ 1 ( x ) , · · · , ˆ µ m ( x )]and ˆ σ 2 ( x ) =

Original: [ˆ σ 21 ( x ) , · · · , ˆ σ 2 m ( x )] be the predicted mean and variance for the objective vector.
Revised: [ˆ σ 21 ( x ) , · · · , ˆ σ 2 m ( x )] be the predicted mean and variance for the objective vector.

Original: Suppose we have a learned Pareto set M psl , the GP models give us both predicted value ˆ µ ( M psl )
Revised: Suppose we have a learned Pareto set M psl , the GP models give us both predicted value ˆ µ ( M psl )

Original: ={ ˆ µ ( x ) | x ∈ M psl } and uncertainty ˆ σ 2 ( M psl ) =
Revised: ={ ˆ µ ( x ) | x ∈ M psl } and uncertainty ˆ σ 2 ( M psl ) =

Original: { ˆ σ 2 ( x ) | x ∈ M psl } for the whole approximate
Revised: { ˆ σ 2 ( x ) | x ∈ M psl } for the whole approximate

Original: Pareto set.
Revised: Pareto set.

Original: 1: Input: Model x ( λ ) =
Revised: 1: Input: Model x ( λ ) =

Original: h θ ( λ ) 2: Initialize the parameters θ3: for t = 1 to T do 4:
Revised: h θ ( λ ) 2: Initialize the parameters θ3: for t = 1 to T do 4:

Original: Sample preferences { λ k } Kk =1 ∼ Λ 5:
Revised: Sample preferences { λ k } Kk =1 ∼ Λ 5:

Original: Update θ t with gradient descent in (9) 6: end for 7: Output: Model x ( λ ) =
Revised: Update θ t with gradient descent in (9) 6: end for 7: Output: Model x ( λ ) =

Original: h θ T ( λ )
Revised: h θ T ( λ )

Original: Pareto Set Learning.
Revised: Pareto Set Learning.

Original: Now we propose an efﬁcient algorithm to ﬁnd the optimal parameter θ ∗for the Pareto set model h θ ( λ ) .
Revised: Now we propose an efﬁcient algorithm to ﬁnd the optimal parameter θ ∗for the Pareto set model h θ ( λ ) .

Original: The optimal solution set M tch_aug for augmented Tchebycheffscalarization (4) is unknown, hence we need tooptimize all solutions generated by our modelx ( λ ) =
Revised: The optimal solution set M tch_aug for augmented Tchebycheffscalarization (4) is unknown, hence we need tooptimize all solutions generated by our modelx ( λ ) =

Original: h θ ( λ ) with respect to their corresponding Tchebycheff scalarization subproblems forall valid preferences:
Revised: h θ ( λ ) with respect to their corresponding Tchebycheff scalarization subproblems forall valid preferences:

Original: [Equation]
Revised: [Equation]

Original: If the model is perfectly learned, the obtained approximate Pareto set M psl = { x = h θ ( λ ) | λ ∈ Λ }should be equal to M tch_aug .
Revised: If the model is perfectly learned, the obtained approximate Pareto set M psl = { x = h θ ( λ ) | λ ∈ Λ }should be equal to M tch_aug .

Original: However, it is difﬁcult to directly optimize (8) due to the expectationover inﬁnite preferences ( | Λ | = ∞ ).
Revised: However, it is difﬁcult to directly optimize (8) due to the expectationover inﬁnite preferences ( | Λ | = ∞ ).

Original: We use Monte Carlo sampling and gradient descent to iterativelylearn the model with the surrogate model:
Revised: We use Monte Carlo sampling and gradient descent to iterativelylearn the model with the surrogate model:

Original: [Equation]
Revised: [Equation]

Original: where we randomly sample K = 10 different valid preferences { λ 1 , · · · , λ K } ∼ Λ at each iterationin this work.
Revised: where we randomly sample K = 10 different valid preferences { λ 1 , · · · , λ K } ∼ Λ at each iterationin this work.

Original: Here ˆ g tch_aug ( · ) is the augmented Tchebycheff scalarization with predicted objectivevalues:
Revised: Here ˆ g tch_aug ( · ) is the augmented Tchebycheff scalarization with predicted objectivevalues:

Original: [Equation]
Revised: [Equation]

Original: One design choice left is the surrogate objective
Revised: One design choice left is the surrogate objective

Original: ˆ f ( x ) .
Revised: ˆ f ( x ) .

Original: If we only want to get the current predictive
Revised: If we only want to get the current predictive

Original: Pareto front, it is straightforward to use the predictive mean as the surrogate value. 
Revised: Pareto front, it is straightforward to use the posterior mean as the surrogate value. The approximate

Revised: Pareto front under the posterior mean could provide valuable information to decision-makers.

Original: However,for Bayesian optimization, we should take the uncertainty into account to balance exploitation andexploration.
Revised: However, for Bayesian optimization, we should take the uncertainty into account to balance exploitationand exploration.

Original: Many widely-used criteria, such as expected improvement (EI) [52] and upper conﬁdencebound (UCB) [72], could be a more reasonable choice.
Revised: Many widely-used criteria, such as expected improvement (EI) [52] and upperconﬁdence bound (UCB) [72], could be a more reasonable choice.

Original: In this work, we use the lower conﬁdencebound for minimization problems:
Revised: In this work, we use the lowerconﬁdence bound for minimization problems:

Original: [Equation]
Revised: [Equation]

Original: We simply set β = 12 and discuss the performance with other surrogate objectives in Appendix.
Revised: We simply set β = 12 and discuss the performance with other surrogate objectives in Appendix.

Original: The expensive objective function f ( x ) is usually black-box and non-differentiable, but we can easilyobtain the gradients for the Gaussian process and the set model.
Revised: The expensive objective function f ( x ) is usually black-box and non-differentiable, but we can easilyobtain the gradients for the Gaussian process and the set model.

Original: Indeed, gradient-based methods havebeen widely used for optimizing the acquisition function in both BO [84, 83] and MOBO [47, 13].
Revised: Indeed, gradient-based methods havebeen widely used for optimizing the acquisition function in both BO [84, 83] and MOBO [47, 13].

Original: The max operator in Tchebycheff scalarization is technically only subdifferentiable, but it is knownto have good subgradients [83] for surrogate optimization and preserve convexity if the objectives{ ˆ f i ( x ) } mi =1 are all convex [7].
Revised: The max operator in Tchebycheff scalarization is technically only subdifferentiable, but it is knownto have good subgradients [83] for surrogate optimization and preserve convexity if the objectives{ ˆ f i ( x ) } mi =1 are all convex [7].

Original: The Pareto set learning algorithm with Gaussian process models is summarized in Algorithm 1 .
Revised: The Pareto set learning algorithm with Gaussian process models is summarized in Algorithm 1 .

Original: Weﬁnd the simple random initialization and gradient descent are enough to learn a good Pareto setapproximation.
Revised: Weﬁnd the simple random initialization and gradient descent are enough to learn a good Pareto setapproximation.

Original: The overparameterized neural network could be beneﬁcial to overcome potentialnon-convexity [46].
Revised: The overparameterized neural network could be beneﬁcial to overcome potentialnon-convexity [46].

Original: 4.3 Batched selection on approximate Pareto set
Revised: 3 Batched selection on approximate Pareto set

Original: In this subsection, we propose a lightweight yet efﬁcient batched acquisition search for MOBOwith the learned Pareto set model.
Revised: In this subsection, we propose a lightweight yet efﬁcient batched acquisition search for MOBOwith the learned Pareto set model.

Original: The algorithm framework is shown in Algorithm 2 .
Revised: The algorithm framework is shown in Algorithm 2 .

Original: The crucialdifference with other MOBO approaches is that we build a set model at each iteration for batchedsolution selection as shown in Algorithm 1 and Algorithm 3 .
Revised: The crucialdifference with other MOBO approaches is that we build a set model at each iteration for batchedsolution selection as shown in Algorithm 1 and Algorithm 3 .

Original: The batched selection procedurecontains two closely related steps:end
Revised: The batched selection procedurecontains two closely related steps:

Original: Batch Sampling on Approximate Pareto Set.
Revised: Batch Sampling on Approximate Pareto Set.

Original: Our model naturally supports generating arbitrarynumber of solutions in batch.
Revised: Our model naturally supports generating arbitrarynumber of solutions in batch.

Original: If the decision-maker’s preferences are available, we can use preferencebased sampling in this step.
Revised: If the decision-maker’s preferences are available, we can use preferencebased sampling in this step.

Original: Without any prior knowledge, we uniformly sample P valid preferences{ λ ( p ) }
Revised: Without any prior knowledge, we uniformly sample P valid preferences{ λ ( p ) }

Original: Pp =1 , and generate the corresponding solutions
Revised: Pp =1 , and generate the corresponding solutions

Original: X
Revised: X

Original: = { x ( λ ( p ) ) }
Revised: = { x ( λ ( p ) ) }

Original: Pp =1 on the approximate Paretoset M psl .
Revised: Pp =1 on the approximate Paretoset M psl .

Original: Batch Selection.
Revised: Batch Selection.

Original: At each iteration of MOBO, we typically select a small number B (e.g., 5 ) ofsolutions X B = { x ( b ) } Bb =1 from the sampled solutions X for expensive evaluations.
Revised: At each iteration of MOBO, we typically select a small number B (e.g., 5 ) ofsolutions X B = { x ( b ) } Bb =1 from the sampled solutions X for expensive evaluations.

Original: To take allalready evaluated solutions { X t − 1 , y t − 1 } into consideration, we use the hypervolume [91] as theselection criteria.
Revised: To take allalready evaluated solutions { X t − 1 , y t − 1 } into consideration, we use the hypervolume [91] as theselection criteria.

Original: The hypervolume HV ( y ) =
Revised: The hypervolume HV ( y ) =

Original: Vol ( S ) measures the volume of S dominated by aset y in the objective space:
Revised: Vol ( S ) measures the volume of S dominated by aset y in the objective space:

Original: [Equation]
Revised: [Equation]

Original: where r ∗ is a reference point that dominated by all y ∈ y .
Revised: where r ∗ is a reference point that dominated by all y ∈ y .

Original: The hypervolume improvement (HVI)of a set X B with respect to the already evaluated solutions { X t − 1 , y t − 1 } can be deﬁned as:
Revised: The hypervolume improvement (HVI)of a set X B with respect to the already evaluated solutions { X t − 1 , y t − 1 } can be deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where X B = { x ( b ) } Bb =1 are selected solutions, and ˆ f ( X B ) are the surrogate values.
Revised: where X B = { x ( b ) } Bb =1 are selected solutions, and ˆ f ( X B ) are the surrogate values.

Original: In this work, wemainly use the LCB (11) as the surrogate value, and provide an ablation study of different surrogatevalues in Appendix.
Revised: In this work,we mainly use the LCB (11) as the surrogate value for Bayesian optimization, and provide an ablationstudy of different surrogate values in Appendix.

Original: F.3.
Revised: F.3.

Original: A better trade-off set will have a larger hypervolume, and the ground truth Pareto set always hasthe largest one.
Revised: A better trade-off set will have a larger hypervolume, and the ground truth Pareto set always hasthe largest one.

Original: We want to select a set of X B such that their corresponding objective values ˆ f ( X B )maximize HVI (ˆ f ( X B )) .
Revised: We want to select a set of X B such that their corresponding objective values ˆ f ( X B )maximize HVI (ˆ f ( X B )) .

Original: It would be computationally intensive to jointly optimize a set of solutionsto exactly maximize the hypervolume improvement (13), and therefore sequential greedy selectionis typically used [13].
Revised: It would be computationally intensive to jointly optimize a set of solutionsto exactly maximize the hypervolume improvement (13), and therefore sequential greedy selectionis typically used [13].

Original: In this work, we select the set X B with a sequential greedy manner from
Revised: In this work, we select the set X B with a sequential greedy manner from

Original: X where | X | = P = 1 , 000 for all problems.
Revised: X where | X | = P = 1 , 000 for all problems.

Original: More details can be found in Appendix.
Revised: More details can be found in Appendix.

Original:  Experiments
Revised:  Experiments

Original: In this section, we mainly compare the proposed PSL method with other MOBO approaches on theperformance of evaluated solutions.
Revised: In this section, we mainly compare the proposed PSL method with other MOBO approaches on theperformance of evaluated solutions.

Original: We also analyze the quality of the learned Pareto set, which isnot supported by other methods.
Revised: We also analyze the quality of the learned Pareto set, which isnot supported by other methods.

Original: Baseline Algorithms.
Revised: Baseline Algorithms.

Original: We consider different widely-used MOBO methods and two model-freeapproaches as baselines.
Revised: We consider different widely-used MOBO methods and two model-freeapproaches as baselines.

Original: The implementations of NSGA-II [18], MOEA/D-EGO [87], TSEMO [8],
Revised: The implementations of NSGA-II [18], MOEA/D-EGO [87], TSEMO [8],

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: n
Revised: n

Original: [Figure]
Revised: [Figure]

Original: USeMO-EI [5], DGEMO [47] are from DGEMO’s open-source codebase 1 based on pymoo 2 [6].
Revised: USeMO-EI [5], DGEMO [47] are from DGEMO’s open-source codebase 1 based on pymoo 2 [6].

Original: The implementations of scrambled Sobol sequence, TS-TCH [55], MESMO [4], qParEGO [41], andqEHVI [13] and from BoTorch 3 [3].
Revised: The implementations of scrambled Sobol sequence, TS-TCH [55], MESMO [4], qParEGO [41], andqEHVI [13] and from BoTorch 3 [3].

Original: We implement PSL in Pytorch [57], and will open-source ourcode upon publication 4 .
Revised: We implement PSL in Pytorch [57], and will open-source ourcode upon publication 4 .

Original: Benchmarks and Real-World Problems.
Revised: Benchmarks and Real-World Problems.

Original: The algorithms are ﬁrst compared on the widely-used
Revised: The algorithms are ﬁrst compared on the widely-used

Original: ZDT1-3 [90], VLMOP1-3 [79] and DTLZ2 [19] benchmark problems.
Revised: ZDT1-3 [90], VLMOP1-3 [79] and DTLZ2 [19] benchmark problems.

Original: Then we also conduct experiments on 5 different real-world multi-objective engineering design problems (RE) [76], including1) four bar truss design [10]; 2) pressure vessel design [42]; 3) disk brake design [63]; 4) gear traindesign [17] and 5) rocket injector design [78].
Revised: Then we also conduct experiments on 5 different real-world multi-objective engineering design problems (RE) [76], including1) four bar truss design [10]; 2) pressure vessel design [42]; 3) disk brake design [63]; 4) gear traindesign [17] and 5) rocket injector design [78].

Original: Details of these problems can be found in Appendix.
Revised: Details of these problems can be found in Appendix.

Original: Experiment Setting.
Revised: Experiment Setting.

Original: For each experiment, we randomly generate 10 initial solutions for expensiveevaluations, and then conduct MOBO with 20 batched evaluations with batch size 5 .
Revised: For each experiment, we randomly generate 10 initial solutions for expensiveevaluations, and then conduct MOBO with 20 batched evaluations with batch size 5 .

Original: Therefore,there are total 110 expensive evaluations.
Revised: Therefore,there are total 110 expensive evaluations.

Original: For an experiment, all algorithms are independently run 10times with different random seeds.
Revised: For an experiment, all algorithms are independently run 10times with different random seeds.

Original: We use the hypervolume indicator [91] as the metric to comparethe quality of evaluated solutions chosen by different MOBO algorithms, which is monotonic tothe Pareto dominance relation.
Revised: We use the hypervolume indicator [91] as the metric to comparethe quality of evaluated solutions chosen by different MOBO algorithms, which is monotonic tothe Pareto dominance relation.

Original: The ground truth Pareto front will always have the best (highest)hypervolume.
Revised: The ground truth Pareto front will always have the best (highest)hypervolume.

Original: Detailed experimental setting can be found in Appendix.
Revised: Detailed experimental setting can be found in Appendix.

Original: 5.1 Experimental results and analysis
Revised: 5.1 Experimental results and analysis

Original: [Table]
Revised: [Table]

Original: MOBO Performance.
Revised: MOBO Performance.

Original: We compare PSL with other MOBO methods on the performance of evaluatedsolutions.
Revised: We compare PSL with other MOBO methods on the performance of evaluatedsolutions.

Original: Figure 5 shows the log hypervolume difference to the true/approximate Pareto front for the
Revised: Figure 5 shows the log hypervolume difference to the true/approximate Pareto front for the

Original: To
Revised: To

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: synthetic/real-world problems during the optimization process.
Revised: synthetic/real-world problems during the optimization process.

Original: The approximate Pareto fronts for thereal-world design problems are from Tanabe and Ishibuchi [76] with a large number of evaluations,which are also used in other MOBO works.
Revised: The approximate Pareto fronts for thereal-world design problems are from Tanabe and Ishibuchi [76] with a large number of evaluations,which are also used in other MOBO works.

Original: In most experiments, our proposed PSL method convergesfaster and achieves better ﬁnal results than other MOBO methods.
Revised: In most experiments, our proposed PSL method convergesfaster and achieves better ﬁnal results than other MOBO methods.

Original: As shown in Table 1, PSL has ashorter or comparable total runtime per iteration with other MOBO methods, which can be ignored inthe expensive optimization problems (might take days).
Revised: As shown in Table 1, PSL has ashorter or comparable total runtime per iteration with other MOBO methods, which can be ignored inthe expensive optimization problems (might take days).

Original: The promising results validate that the PSLapproach can signiﬁcantly improve the MOBO performance for expensive optimization problems.f 1 ( x ) 0.0 0.2 0.4 0.6 0.8 1.
Revised: The promising results validate that the PSLapproach can signiﬁcantly improve the MOBO performance for expensive optimization problems.f 1 ( x ) 0.0 0.2 0.4 0.6 0.8 1.

Original: f 2 ( x ) 0.0 0.2 0.4 0.
Revised: f 2 ( x ) 0.0 0.2 0.4 0.

Original: f 3 ( x ) 0.0 0.2 0.
Revised: f 3 ( x ) 0.0 0.2 0.

Original: (a) Different Preferences f 1 ( x ) 0.0 0.2 0.4 0.

Revised: (a) Different Preferences f 1 ( x ) 0.0 0.2 0.4 0.

Original: f 2 ( x ) 0.0 0.2 0.4 0.
Revised: f 2 ( x ) 0.0 0.2 0.4 0.

Original: f 3 ( x ) 0.0 0.2 0.
Revised: f 3 ( x ) 0.0 0.2 0.

Revised: [Figure]

Original: (b) DTLZ
Revised: ) DTLZ

Original: The Learned Pareto Set.
Revised: The Learned Pareto Set.

Original: We present the approximate Pareto set learned byour proposed PSL methodat the end of optimization in Figure 6, which is notsupported by other MOBO methods.
Revised: We present the approximate Pareto set learned by PSL under the posteriormean after optimization in Figure 6, which is notsupported by other MOBO methods.

Original: According tothe results, PSL can successfully learn the Pareto setsfor different benchmarks and real-world applicationproblems with different shapes of Pareto fronts.
Revised: According tothe results, PSL can successfully learn the Pareto setsfor different benchmarks and real-world applicationproblems with different shapes of Pareto fronts.

Original: Forbenchmark problems, PSL can match the ground truth
Revised: Forbenchmark problems, PSL can match the ground truth

Original: Pareto front with a small evaluation budget.
Revised: Pareto front with a small evaluation budget.

Original: For realworld applications, the approximate Pareto fronts cancapture the trade-off among objectives and providevaluable information to support decision-making.
Revised: For realworld applications, the approximate Pareto fronts cancapture the trade-off among objectives and providevaluable information to support decision-making.

Original: Flexible Trade-off Adjustment.
Revised: Flexible Trade-off Adjustment.

Original: With our model,the decision-makers can easily explore the whole approximate Pareto set by themselves to selectthe most preferred trade-off solution(s) as shown in Figure 7.
Revised: With our model,the decision-makers can easily explore the whole approximate Pareto set by themselves to selectthe most preferred trade-off solution(s) as shown in Figure 7.

Original: No time-consuming communicationbetween the optimization modeler and the decision-maker is required.
Revised: No time-consuming communicationbetween the optimization modeler and the decision-maker is required.

Original: By directly exploring theapproximate Pareto front in an interactive manner, the decision-makers can observe and understandthe connection between preferences and corresponding solutions in real-time.
Revised: By directly exploring theapproximate Pareto front in an interactive manner, the decision-makers can observe and understandthe connection between preferences and corresponding solutions in real-time.

Original: It is also beneﬁcial fordecision-makers to further adjust and assign their most accurate preferences.
Revised: It is also beneﬁcial fordecision-makers to further adjust and assign their most accurate preferences.

Original: The ability to incorporateuser’s knowledge into decision making [26] could be crucial for many real-world applications.
Revised: The ability to incorporateuser’s knowledge into decision making [26] could be crucial for many real-world applications.

Original:  Conclusion, limitation and future work
Revised:  Conclusion, limitation and future work

Original: Conclusion.
Revised: Conclusion.

Original: In this work, we have proposed a novel Pareto set learning method for expensive multiobjective optimization with a limited evaluation budget.
Revised: In this work, we have proposed a novel Pareto set learning method for expensive multiobjective optimization with a limited evaluation budget.

Original: The beneﬁts of this approach are two-fold.
Revised: The beneﬁts of this approach are two-fold.

Original: First of all, by learning and utilizing the approximate Pareto set, it can serve as an efﬁcient MOBOmethod that outperforms different existing approaches.
Revised: First of all, by learning and utilizing the approximate Pareto set, it can serve as an efﬁcient MOBOmethod that outperforms different existing approaches.

Original: Secondly, it also allows the decision-makersto easily explore any trade-off region on the approximate Pareto set, which supports ﬂexible andinteractive decision-making.
Revised: Secondly, it also allows the decision-makersto easily explore any trade-off region on the approximate Pareto set, which supports ﬂexible andinteractive decision-making.

Original: We believe the proposed Pareto set learning method could provide anovel way to solve expensive multi-objective optimization problems. 
Revised: We believe the proposed Pareto set learning method could provide anovel way to solve expensive multi-objective optimization problems. Checklist For all authors...

Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Revised: contributions and scope?

Revised: [Yes]

Original: A More Discussion
Revised: (b) Did you describe the limitations of your work? [Yes] See Section 6. (c) Did you discuss any potential negative societal impacts of your work? [No] We do notﬁnd any potential negative societal impacts of our work.

Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Revised: them?

Revised: [Yes]

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [N/A]

Revised: (b) Did you include complete proofs of all theoretical results?

Revised: [N/A]

Revised: If you ran experiments...

Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Revised: mental results (either in the supplemental material or as a URL)?

Revised: [No] Our code will

Revised: be released upon publication.

Revised: (b) Did you specify all the training details (e

Revised: g., data splits, hyperparameters,

Revised: how they

Revised: were chosen)?

Revised: [Yes] See Section 5 and Appendix.

Revised: (c) Did you report error bars (e

Revised: g., with respect to the random seed after running experi-

Revised: ments multiple times)?

Revised: [Yes]

Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Revised: [Yes] Yes. See Appendix.

Revised: If you are using existing assets (e

Revised: g., code, data, models) or curating/releasing new assets...

Revised: (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes]

Revised: (b) Did you mention the license of the assets?

Revised: [Yes] See Appendix.

Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [N/A]

Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: using/curating?

Revised: [N/A]

Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Revised: information or offensive content?

Revised: [N/A]

Revised: If you used crowdsourcing or conducted research with human subjects...

Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Revised: applicable?

Revised: [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: Board (IRB) approvals, if applicable?

Revised: [N/A]

Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: spent on participant compensation?

Revised: [N/A]

Revised: We provide more discussions, algorithm and problem details, and extra experimental results in thisappendix:

Revised: • More discussions of the proposed algorithm are temporally provided in Section A.

Revised: • The limitations and potential future improvement for PSL are discussed in Section B.

Revised: • Potential societal impact of this work is discussed in Section C.

Revised: • Details of the set model and batch selection algorithm are provided in Section D.• Details of the benchmark and real-world application problems are given in Section E.(deﬁnition of 6 new problems are added.)

Revised: • More experimental results and analyses are presented in Section F. (many new results)

Original: Since no extra page is allowed for the main paper during the response period, we temporally put allnew discussions for our proposed algorithm in this section.
Revised: Since no extra page is allowed for the main paper during the response period, we temporally put allnew discussions for our proposed algorithm in this section.

Original: Will move part of these discussions intothe main paper and reorganize the rest materials once an extra page is allowed.
Revised: Will move part of these discussions intothe main paper and reorganize the rest materials once an extra page is allowed.

Original: A.1 Motivation: Pareto set model over simple scalarization
Revised: A.1 Motivation: Pareto set model over simple scalarization

Original: Our proposed Pareto set learning method has a close relation to the scalarization-based methods.
Revised: Our proposed Pareto set learning method has a close relation to the scalarization-based methods.

Original: Butit can overcome a major disadvantage of the current scalarization methods for expensive optimization.
Revised: Butit can overcome a major disadvantage of the current scalarization methods for expensive optimization.

Original: The scalarization methods randomly select one (e.g., ParEGO) or a batch of scalarized subproblems(e.g., MOEA/D-EGO, TS-TCH, and qParEGO) at each iteration.
Revised: The scalarization methods randomly select one (e.g., ParEGO) or a batch of scalarized subproblems(e.g., MOEA/D-EGO, TS-TCH, and qParEGO) at each iteration.

Original: By optimizing the acquisitionfunction (e.g., EI, LCB, Thompson Sampling etc) on each scalarization, they generate a batch ofsolutions for expensive evaluation.
Revised: By optimizing the acquisitionfunction (e.g., EI, LCB, Thompson Sampling etc) on each scalarization, they generate a batch ofsolutions for expensive evaluation.

Original: A major limitation of all these scalarization methods is that they do not explicitly consider thosealready-evaluated solutions, neither for choosing the weighted scalarization, nor for selecting thenext (batch of) solution(s) for expensive evaluation.
Revised: A major limitation of all these scalarization methods is that they do not explicitly consider thosealready-evaluated solutions, neither for choosing the weighted scalarization, nor for selecting thenext (batch of) solution(s) for expensive evaluation.

Original: Therefore, the selected weighted scalarization(s)might be close to those already-evaluated solutions.
Revised: Therefore, the selected weighted scalarization(s)might be close to those already-evaluated solutions.

Original: In other words, even the obtained solution(s), canmaximize the EI/LCB for the selected scalarization, could still be similar to those already-evaluatedones, and indeed not an optimal choice for multi-objective optimization.
Revised: In other words, even the obtained solution(s), canmaximize the EI/LCB for the selected scalarization, could still be similar to those already-evaluatedones, and indeed not an optimal choice for multi-objective optimization.

Original: This limitation leads toinferior performance of these scalarization methods, as reported in [13, 47] and in our experimentalresults.
Revised: This limitation leads toinferior performance of these scalarization methods, as reported in [13, 47] and in our experimentalresults.

Original: To overcome this limitation, our proposed method has a two-stage approach:
Revised: To overcome this limitation, our proposed method has a two-stage approach:

Original: • Stage 1: Using the Pareto set model, it ﬁrst efﬁciently samples a dense set of candidatesolutions to cover the whole approximate Pareto front (of posterior mean, EI, or LCB etc). • Stage 2: Then it selects a small batch of appropriate solutions from this dense set forexpensive evaluation.
Revised: • Stage 1: Using the Pareto set model, it ﬁrst efﬁciently samples a dense set of candidatesolutions to cover the whole approximate Pareto front (of posterior mean, EI, or LCB etc).• Stage 2: Then it selects a small batch of appropriate solutions from this dense set forexpensive evaluation.

Original: For selection, we use the HVI criteria to take both the selected andalready-evaluated solutions into consideration.
Revised: For selection, we use the HVI criteria to take both the selected andalready-evaluated solutions into consideration.

Original: In this way, our proposed method can efﬁciently explore the whole approximate Pareto front, andchoose the most appropriate solutions (on the approximate Pareto front, while far from the selectedand already-evaluated ones) for expensive evaluation.
Revised: In this way, our proposed method can efﬁciently explore the whole approximate Pareto front, andchoose the most appropriate solutions (on the approximate Pareto front, while far from the selectedand already-evaluated ones) for expensive evaluation.

Original: The experimental results have validated theefﬁciency of the proposed method.
Revised: The experimental results have validated theefﬁciency of the proposed method.

Original: A.2 Hypervolume improvement for batch selection
Revised: A.2 Hypervolume improvement for batch selection

Original: As discussed above, the current scalarization approach (e.g., (q)ParEGO) randomly selects differentscalarization(s) at each iteration, which is to randomly generate a small set of sparse solutions on theapproximate Pareto set (with EI, LCB) for evaluation.
Revised: As discussed above, the current scalarization approach (e.g., (q)ParEGO) randomly selects differentscalarization(s) at each iteration, which is to randomly generate a small set of sparse solutions on theapproximate Pareto set (with EI, LCB) for evaluation.

Original: In other words, the selected solutions are asmall random subset of the whole approximate Pareto set (our Stage 1).
Revised: In other words, the selected solutions are asmall random subset of the whole approximate Pareto set (our Stage 1).

Original: In our proposed algorithm, weuse the HVI to select a small batch of solutions from the dense approximate Pareto set, which can beaware of both selected and already-evaluated solutions and leads to better optimization performance(our Stage 2).
Revised: In our proposed algorithm, weuse the HVI to select a small batch of solutions from the dense approximate Pareto set, which can beaware of both selected and already-evaluated solutions and leads to better optimization performance(our Stage 2).

Original: Efﬁciently ﬁnding a batch of solutions to optimize the EI/LCB of HVI could be very challenging.qEHVI [13] and qNEHVI [14] are two promising approaches along this direction.
Revised: Efﬁciently ﬁnding a batch of solutions to optimize the EI/LCB of HVI could be very challenging.qEHVI [13] and qNEHVI [14] are two promising approaches along this direction.

Original: From the viewpointof optimizing HVI, our proposed method restricts the search procedure only on the approximate
Revised: From the viewpointof optimizing HVI, our proposed method restricts the search procedure only on the approximate

Original: Pareto set, which is a low-dimensional (m-1) manifold in the decision space.
Revised: Pareto set, which is a low-dimensional (m-1) manifold in the decision space.

Original: Therefore, we can use asimple two-stage sample-then-select approach to ﬁnd the batch of solutions for evaluations.
Revised: Therefore, we can use asimple two-stage sample-then-select approach to ﬁnd the batch of solutions for evaluations.

Original: For theoptimal situation, the set of solutions that optimize HVI should all be on the Pareto front.
Revised: For theoptimal situation, the set of solutions that optimize HVI should all be on the Pareto front.

Original: However,we agree with the reviewer that if an efﬁcient method exists, directly optimizing the EI/LCB of HVIon the whole search space should be a principled approach for batch selection.
Revised: However,we agree with the reviewer that if an efﬁcient method exists, directly optimizing the EI/LCB of HVIon the whole search space should be a principled approach for batch selection.

Original: On the other hand, the scalarization-based approach could have a close relationship with the hypervolume [88].
Revised: On the other hand, the scalarization-based approach could have a close relationship with the hypervolume [88].

Original: It will be very interesting to study how to better leverage this relation for designing a moreefﬁcient algorithm, such as learning the whole Pareto set while inference the location of solutions (onthe learned Pareto set) to optimize HVI at a single stage.
Revised: It will be very interesting to study how to better leverage this relation for designing a moreefﬁcient algorithm, such as learning the whole Pareto set while inference the location of solutions (onthe learned Pareto set) to optimize HVI at a single stage.

Original: These will be our future work.
Revised: These will be our future work.

Original: A.3 Practicality of the approximate Pareto set
Revised: A.3 Practicality of the approximate Pareto set

Original: In real-world applications, the learned approximate Pareto set could not always be accurate.
Revised: In real-world applications, the learned approximate Pareto set could not always be accurate.

Original: Therefore,it is risky to only rely on the approximate Pareto set to make a decision (see more discussion in thelimitation section).
Revised: Therefore,it is risky to only rely on the approximate Pareto set to make a decision (see more discussion in thelimitation section).

Original: With our proposed method, decision-makers can have both the evaluated solutionsand the approximate Pareto set at the same time.
Revised: With our proposed method, decision-makers can have both the evaluated solutionsand the approximate Pareto set at the same time.

Original: The approximate Pareto set can provide extra usefulinformation to better support their decision-making.
Revised: The approximate Pareto set can provide extra usefulinformation to better support their decision-making.

Original: More speciﬁcally,• It can help the decision-makers to better understand the (approximate) trade-offs they willmake when they choose an already evaluated solution as their ﬁnal solution.
Revised: More speciﬁcally,• It can help the decision-makers to better understand the (approximate) trade-offs they willmake when they choose an already evaluated solution as their ﬁnal solution.

Original: The approximate
Revised: The approximate

Original: Pareto set and the corresponding surrogate objective values around the chosen solution canprovide valuable information to understand what we can
Revised: Pareto set and the corresponding surrogate objective values around the chosen solution canprovide valuable information to understand what we can

Original: gain or lose by adjusting the current
Revised: gain or lose by adjusting the current

Original: chosen solution.
Revised: chosen solution.

Original: • It allows the decision-maker to explore the whole approximate Pareto set/front easily.
Revised: • It allows the decision-maker to explore the whole approximate Pareto set/front easily.

Original: If
Revised: If

Original: none of the already evaluated solutions can precisely satisfy the decision-maker’s preferredtrade-off (which could be possible for applications with inﬁnite Pareto sets), the decisionmaker can rely on the approximate Pareto set/front to choose the preferred solution forfurther evaluation.
Revised: none of the already evaluated solutions can precisely satisfy the decision-maker’s preferredtrade-off (which could be possible for applications with inﬁnite Pareto sets), the decisionmaker can rely on the approximate Pareto set/front to choose the preferred solution for further evaluation.

Original: • When the optimization modeler and the ﬁnal decision-maker are not the same person, theapproximate Pareto set/front provides a much more efﬁcient way for them to communicateand discuss the whole trade-offs among different objectives during or post the optimization
Revised: • When the optimization modeler and the ﬁnal decision-maker are not the same person, theapproximate Pareto set/front provides a much more efﬁcient way for them to communicateand discuss the whole trade-offs among different objectives during or post the optimization

Original: process.
Revised: process.

Original: This demand is common and important for many applications [49].
Revised: This demand is common and important for many applications [49].

Original: • Finally, an important application for MOBO is to help domain experts efﬁciently conduct
Revised: • Finally, an important application for MOBO is to help domain experts efﬁciently conduct

Original: experiments.
Revised: experiments.

Original: The approximate Pareto set/front might contain useful patterns and structures
Revised: The approximate Pareto set/front might contain useful patterns and structures

Original: which can help the domain expert obtain more useful information from the experiment.
Revised: which can help the domain expert obtain more useful information from the experiment.

Original: It
Revised: It

Original: also provides a way for domain experts to incorporate their knowledge into the optimization
Revised: also provides a way for domain experts to incorporate their knowledge into the optimization

Original: process (e
Revised: process (e

Original: g., choose the most concerned region, and eliminate some uninteresting locations). The
Revised: g., choose the most concerned region, and eliminate some uninteresting locations).The

Original: proposed
Revised: proposed

Original: Pareto set learning method could be a novel approach to support "bringdecision maker back into the loop" for Bayesian optimization [26].
Revised: Pareto set learning method could be a novel approach to support "bring decision maker back into the loop" for Bayesian optimization [26].

Original: Therefore, 
Revised: Therefore, the

Original: the learned approximate Pareto set could be a
Revised: learned approximate Pareto set could be a

Original: useful tool to
Revised: useful tool to

Original: support ﬂexible decision-
Revised: support ﬂexible decision

Original: making for expensive multi-objective optimization
Revised: making for expensive multi-objective optimization

Original: B Limitation and potential future work
Revised: B Limitation and potential future work

Original: B.
Revised: B.

Original: 1 The approximate Pareto set could be inaccurate
Revised: 1 The approximate Pareto set could be inaccurate

Original: It could be very risky, especially in those safety-critical applications, to only rely on an approximate
Revised: It could be very risky, especially in those safety-critical applications, to only rely on an approximate

Original: Pareto set to make the ﬁnal decision.
Revised: Pareto set to make the ﬁnal decision.

Original: The quality of the approximate Pareto set heavily depends onthe quality of the surrogate models.
Revised: The quality of the approximate Pareto set heavily depends onthe quality of the surrogate models.

Original: If the evaluation budget is insufﬁcient to build a good set ofsurrogate models (for each objective), we cannot obtain a good approximate Pareto set.
Revised: If the evaluation budget is insufﬁcient to build a good set ofsurrogate models (for each objective), we cannot obtain a good approximate Pareto set.

Original: This is also achallenge for the general (multi-objective)
Revised: This is also achallenge for the general (multi-objective)

Original: Bayesian optimization methods.
Revised: Bayesian optimization methods.

Original: One possible method to address this challenge is to leverage the user preference information into theexpensive optimization process [1, 55, 2].
Revised: One possible method to address this challenge is to leverage the user preference information into theexpensive optimization process [1, 55, 2].

Original: In this way, the general MOBO method can spend the limited evaluation budget mainly on the user-preferred region rather than the whole decision space.
Revised: In this way, the general MOBO method can spend the limited evaluation budget mainly on the user-preferred region rather than the whole decision space.

Original: Similarly, our proposed method can only build a partial approximate Pareto set with the user-preferredtrade-offs.
Revised: Similarly, our proposed method can only build a partial approximate Pareto set with the user-preferredtrade-offs.

Original: However, the preference-based approach could be affected by the following limitation.
Revised: However, the preference-based approach could be affected by the following limitation.

Original: B.2 Deﬁning the user
Revised: B.2 Deﬁning the user

Original: preference 
Revised: preference is

Original: is challenging for black-box optimization
Revised: challenging for black-box optimization

Original: Although some scalarization methods (e.g., Chebyshev scalarization) have good theoretical propertiesto connect the scalars to their corresponding Pareto solutions, it could be not so easy to deﬁne userpreferences in terms of scalars, especially in the black-box expensive optimization setting.
Revised: Although some scalarization methods (e.g., Chebyshev scalarization) have good theoretical propertiesto connect the scalars to their corresponding Pareto solutions, it could be not so easy to deﬁne userpreferences in terms of scalars, especially in the black-box expensive optimization setting.

Original: In many applications, the decision-makers might even do not know their actual preferences beforemaking their decision.
Revised: In many applications, the decision-makers might even do not know their actual preferences beforemaking their decision.

Original: If the approximate Pareto set can be properly learned, instead of asking usersto provide their preferences in terms of ﬁxed scalars, our proposed approach can let them interactivelyexplore the whole approximate Pareto set/Pareto front to select their most preferred solutions.
Revised: If the approximate Pareto set can be properly learned, instead of asking usersto provide their preferences in terms of ﬁxed scalars, our proposed approach can let them interactivelyexplore the whole approximate Pareto set/Pareto front to select their most preferred solutions.

Original: In thisinteractive way, it could be much easier for the decision-makers to accurately express and assign theirpreferences.
Revised: In thisinteractive way, it could be much easier for the decision-makers to accurately express and assign theirpreferences.

Original: However, it could be difﬁcult to precisely obtain user preference if a good approximate
Revised: However, it could be difﬁcult to precisely obtain user preference if a good approximate

Original: Pareto set is not available (e.g., in the early stage of optimization, or without enough budget).
Revised: Pareto set is not available (e.g., in the early stage of optimization, or without enough budget).

Original: The algorithm proposed in Abdolshah et al.
Revised: The algorithm proposed in Abdolshah et al.

Original: [1] is an elegant and promising way to incorporatepreference into the MOBO process via the preference-order constraints, which does not requireany prior knowledge on the (approximate) Pareto front.
Revised: [1] is an elegant and promising way to incorporatepreference into the MOBO process via the preference-order constraints, which does not requireany prior knowledge on the (approximate) Pareto front.

Original: The preference-order constraint also has aclose relationship with the lexicographic approach for multi-objective optimization [65], which is anon-scalarizing method with good theoretical property (e.g., connection to weakly Pareto optimalsolution).
Revised: The preference-order constraint also has aclose relationship with the lexicographic approach for multi-objective optimization [65], which is anon-scalarizing method with good theoretical property (e.g., connection to weakly Pareto optimalsolution).

Original: How to incorporate this information into our proposed Pareto set learning model for moreﬂexible preference incorporation could be interesting for future work.
Revised: How to incorporate this information into our proposed Pareto set learning model for moreﬂexible preference incorporation could be interesting for future work.

Original: B.3 Scalability
Revised: B.3 Scalability

Original: The scalability with the dimensions of search space is a main challenge for the general MOBOalgorithms.
Revised: The scalability with the dimensions of search space is a main challenge for the general MOBOalgorithms.

Original: It is also very difﬁcult for our proposed PSL method to learn a good enough Pareto set forsuch large-scale problems.
Revised: It is also very difﬁcult for our proposed PSL method to learn a good enough Pareto set forsuch large-scale problems.

Original: The PSL’s performance depends on a set of good surrogate models, whichrequires a large number of evaluated solutions for the problem with a high-dimensional search space.
Revised: The PSL’s performance depends on a set of good surrogate models, whichrequires a large number of evaluated solutions for the problem with a high-dimensional search space.

Original: Recently, some efﬁcient search region decomposition/management methods have been proposedto better allocate the limited computational budget for problems with high-dimensional searchspace.
Revised: Recently, some efﬁcient search region decomposition/management methods have been proposedto better allocate the limited computational budget for problems with high-dimensional searchspace.

Original: These ideas can be naturally generalized to solve expensive multi-objective optimizationproblems, such as MORBO and LaMOO.
Revised: These ideas can be naturally generalized to solve expensive multi-objective optimizationproblems, such as MORBO and LaMOO.

Original: Since the search region management methods are algorithmagnostic (e.g., a meta-algorithm), they can be combined with different (multi-objective) optimizationalgorithms.
Revised: Since the search region management methods are algorithmagnostic (e.g., a meta-algorithm), they can be combined with different (multi-objective) optimizationalgorithms.

Original: Studying how to efﬁciently combine the MORBO/LaMOO approach with our proposed
Revised: Studying how to efﬁciently combine the MORBO/LaMOO approach with our proposed

Original: PSL method will be an important future work to tackle this limitation.
Revised: PSL method will be an important future work to tackle this limitation.

Original: C Potential societal impact
Revised: C Potential societal impact

Original: Our proposed Pareto set learning method mainly has two strengths: (1) it is good for better multiobjective Bayesian optimization in terms of speed and sample efﬁciency, and (2) it can help decisionmakers to navigate the estimated Pareto set and front for better decision-making.
Revised: Our proposed Pareto set learning method mainly has two strengths: (1) it is good for better multiobjective Bayesian optimization in terms of speed and sample efﬁciency, and (2) it can help decisionmakers to navigate the estimated Pareto set and front for better decision-making.

Original: These strengthscould lead to many positive potential societal impacts.
Revised: These strengthscould lead to many positive potential societal impacts.

Original: For example, as an efﬁcient MOBO algorithm,it can reduce the cost of obtaining a set of diverse Pareto solutions in many applications such asmaterials science, engineering design, and recommender systems.
Revised: For example, as an efﬁcient MOBO algorithm,it can reduce the cost of obtaining a set of diverse Pareto solutions in many applications such asmaterials science, engineering design, and recommender systems.

Original: The learned approximate Paretoset provides a novel way for decision-makers to easily explore different trade-offs, which could be animportant component for a more user-friendly multi-criteria decision system.
Revised: The learned approximate Paretoset provides a novel way for decision-makers to easily explore different trade-offs, which could be animportant component for a more user-friendly multi-criteria decision system.

Original: On the negative side, as discussed in the limitation section, there is currently no guarantee that thelearned approximate Pareto set could be accurate.
Revised: On the negative side, as discussed in the limitation section, there is currently no guarantee that thelearned approximate Pareto set could be accurate.

Original: Solely relying on the approximate Pareto set tomake the ﬁnal decision could be risky.
Revised: Solely relying on the approximate Pareto set tomake the ﬁnal decision could be risky.

Original: The decision-makers should leverage all the informationthey have to make the ﬁnal decision.
Revised: The decision-makers should leverage all the informationthey have to make the ﬁnal decision.

Original: In addition, the leakage of the learned Pareto set model mightunintentionally reveal the problem information and user preference, which should be avoided inreal-
Revised: In addition, the leakage of the learned Pareto set model mightunintentionally reveal the problem information and user preference, which should be avoided inreal-world applications.

Original: D Model and algorithm details
Revised: D Model and algorithm details

Original: D.1 Pareto set model
Revised: D.1 Pareto set model

Original: Model Structure.
Revised: Model Structure.

Original: In this work, we use a multi-layered perceptron (MLP) as the Pareto set modelx ( λ ) =
Revised: In this work, we use a multi-layered perceptron (MLP) as the Pareto set modelx ( λ ) =

Original: h θ ( λ ) .
Revised: h θ ( λ ) .

Original: For all experiments, the Pareto set model has 2 hidden layers each with 256 hiddenunits, and the activation is ReLU.
Revised: For all experiments, the Pareto set model has 2 hidden layers each with 256 hiddenunits, and the activation is ReLU.

Original: The input and output dimension of the set model is the number ofobjectives m and the dimension of decision variables n , respectively.
Revised: The input and output dimension of the set model is the number ofobjectives m and the dimension of decision variables n , respectively.

Original: Therefore, the set model h θ ( λ )has the following structure:
Revised: Therefore, the set model h θ ( λ )has the following structure:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the input is an m -dimensional preference λ ∈ Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , the output isa n -dimensional decision variables x ∈ R n , and θ ∈ R d represents all learnable parameters forthe Pareto set model.
Revised: where the input is an m -dimensional preference λ ∈ Λ = { λ ∈ R m + | (cid:80) λ i = 1 } , the output isa n -dimensional decision variables x ∈ R n , and θ ∈ R d represents all learnable parameters forthe Pareto set model.

Original: We build independent Gaussian process models with Matérn 5/2 kernel foreach objective as the surrogate models.
Revised: We build independent Gaussian process models with Matérn 5/2 kernel foreach objective as the surrogate models.

Original: The objective to optimize is the augmented Tchebycheffscalarization on the surrogate value with respect to all valid preferences.
Revised: The objective to optimize is the augmented Tchebycheffscalarization on the surrogate value with respect to all valid preferences.

Original: Model Training.
Revised: Model Training.

Original: In the proposed Pareto set learning method, we have the following model structure:
Revised: In the proposed Pareto set learning method, we have the following model structure:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the preference λ goes through the Pareto set model h θ ( λ ) and Gaussian process model, thengets the augmented Tchebycheff scalarized value ˆ g tch_aug ( x ( λ )) at the end (a detailed version can befound in Figure 4 in the main paper).
Revised: where the preference λ goes through the Pareto set model h θ ( λ ) and Gaussian process model, thengets the augmented Tchebycheff scalarized value ˆ g tch_aug ( x ( λ )) at the end (a detailed version can befound in Figure 4 in the main paper).

Original: To train the Pareto set model, we use gradient-based method tooptimize the model parameter θ with respect to the scalarized value ˆ g tch_aug ( x( λ )) .
Revised: To train the Pareto set model, we use gradient-based method tooptimize the model parameter θ with respect to the scalarized value ˆ g tch_aug ( x λ )) .

Original: With the chainrule, we have:
Revised: With the chainrule, we have:

Original: [Equation]
Revised: [Equation]

Original: where we can calculate each term respectively.
Revised: where we can calculate each term respectively.

Original: • For the ﬁrst term ∂ ˆ g tch_aug ∂ ˆ f , the augmented Tchebycheff scalarization ˆ g tch
Revised: • For the ﬁrst term ∂ ˆ g tch_aug ∂ ˆ f , the augmented Tchebycheff scalarization ˆ g tch_aug

Original: [Equation]

Original: aug ( x 
Revised: x | λ ) =

Revised: max 1 ≤ i ≤ m { λ i ( ˆ f i

Original: [Equation]
Revised: [Equation]

Original: λ ) =max
Revised: x ) − ( z

Original: [Equation]
Revised: [Equation]

Original: { λ [Equation] [Equation]
Revised: − ε )) }

Original: )) } + 
Revised: + ρ

Original: [Equation]
Revised: [Equation]

Original: ={+mi =
+}
Revised: {+mi =
+}

Original: λ i 
Revised: λ i ˆ

Original: ˆ f i ( x ) has a max operator which is techni-
Revised: f i ( x ) has a max operator which is techni-

Original: cally only subdifferentiable, but it is known to have good subgradients [83] for surrogate
Revised: cally only subdifferentiable, but it is known to have good subgradients [83] for surrogate

Original: optimization.
Revised: optimization.

Original: We simply take the subgradients for this term.
Revised: We simply take the subgradients for this term.

Original: • The term ∂ ˆ f ∂ x is the gradient of surrogate values to the input x .
Revised: • The term ∂ ˆ f ∂ x is the gradient of surrogate values to the input x .

Original: In this work, we build
Revised: In this work, we build

Original: independent Gaussian process models for each objective with Matérn 5/2 kernel.
Revised: independent Gaussian process models for each objective with Matérn 5/2 kernel.

Original: The
Revised: The

Original: gradients of the kernel ∂ k i ∂ x , predictive mean ∂ ˆ µ i ∂ x and standard deviation ∂ ˆ σ i ∂ x for each
Revised: gradients of the kernel ∂ k i ∂ x , predictive mean ∂ ˆ µ i ∂ x and standard deviation ∂ ˆ σ i ∂ x for each

Original: objective can be easily calculated.
Revised: objective can be easily calculated.

Original: With these terms, the calculation for the surrogate value
Revised: With these terms, the calculation for the surrogate value

Original: (e
Revised: (e

Original: g., predictive mean, UCB and EI) for each objective ∂
Revised: g., predictive mean, UCB and EI) for each objective ∂

Original: ˆ f i ∂ x is also straightforward.
Revised: ˆ f i ∂ x is also straightforward.

Original: • The term ∂ x ∂ θ
Revised: • The term ∂ x ∂ θ

Original: =
Revised: =

Original: ∂h θ ( x ) ∂ θ is the gradient of the Pareto set model to the parameter θ .
Revised: ∂h θ ( x ) ∂ θ is the gradient of the Pareto set model to the parameter θ .

Original: In this
Revised: In this

Original: work, we use a simple MLP as the Pareto set model and its gradient be easily obtained by
Revised: work, we use a simple MLP as the Pareto set model, and its gradient be easily obtained by

Original: auto-differentiation such as in PyTorch [57].
Revised: auto-differentiation such as in PyTorch [57].

Original: For Algorithm 1 in the main paper, we randomly sample K = 10 different preferences { λ k } K =10 k = ∼Λ in batch at each step.
Revised: For Algorithm 1 in the main paper, we randomly sample K = 10 different preferences { λ k } K =10 k = ∼Λ in batch at each step.

Original: Without any prior information (e.g., user’s preference), we uniformly samplepreference λ
Revised: Without any prior information (e.g., user’s preference), we uniformly sample

Original:  from
Revised: preference λ from

Original: [0 , 1] m and then normalize it such that (cid:80) mi =

Revised: [0 , 1] m and then normalize it such that (cid:80) mi =

Original: λ i = 1 .
Revised: λ i = 1 .

Original: The total update step is
Revised: The total update step is

Original: T = 500 for training the Pareto set model at each iteration.
Revised: T = 500 for training the Pareto set model at each iteration.

Original: The model optimizer is Adam withlearning rate η = 1 e − 3 and no weight decay.
Revised: The model optimizer is Adam withlearning rate η = 1 e − 3 and no weight decay.

Original: The learning process typically only requires lessthan 10 seconds to ﬁnish, which is enough to obtain a good Pareto set approximation.
Revised: The learning process typically only requires lessthan 10 seconds to ﬁnish, which is enough to obtain a good Pareto set approximation.

Original: Detailedexperimental results on the runtime can be found in Table 3.
Revised: Detailedexperimental results on the runtime can be found in Table 3.

Original: We use the same model and training procedure for all experiments, and the only problem-dependentsetting is the input/output dimension m and n .
Revised: We use the same model and training procedure for all experiments, and the only problem-dependentsetting is the input/output dimension m and n .

Original: The proposed Pareto set model and learning approach have robust and promising performances for different expensive multi-objective optimizationbenchmarks and real-world problems.
Revised: The proposed Pareto set model and learning approach have robust and promising performances for different expensive multi-objective optimizationbenchmarks and real-world problems.

Original: The MOBO with Pareto Set Learning framework (e.g., Algorithm 2 in the main paper) is similar toother MOBO methods, and the main difference is on the batch selection with the learned Pareto set( Algorithm 3 ).
Revised: The MOBO with Pareto Set Learning framework (e.g., Algorithm 2 in the main paper) is similar toother MOBO methods, and the main difference is on the batch selection with the learned Pareto set( Algorithm 3 ).

Original: At each iteration, we ﬁrst randomly sample P = 1000 preferences { λ ( p ) } P =1000 p = ∼ Λand directly obtain their corresponding solutions
Revised: At each iteration, we ﬁrst randomly sample P = 1000 preferences { λ ( p ) } P =1000 p = ∼ Λand directly obtain their corresponding solutions

Original: X
Revised: X

Original: = { x ( λ ( p ) ) } P =1000 p
Revised: = { x ( λ ( p ) ) } P =1000 p

Original: = on the current learned Paretoset M psl .
Revised: = on the current learned Paretoset M psl .

Original: We then use the predicted hypervolume improvement (HVI) with respect to the alreadyevaluated solutions { X t − 1 , y t − 1 } to select a subset X B ⊂ X for expensive evaluation.
Revised: We then use the predicted hypervolume improvement (HVI) with respect to the alreadyevaluated solutions { X t − 1 , y t − 1 } to select a subset X B ⊂ X for expensive evaluation.

Original: Thesurrogate values for the candidate solutions are directly used for the predicted HVI calculation if theyare the predicted mean ˆ µ or the lower conﬁdence bound ˆ µ − β σ .
Revised: Thesurrogate values for the candidate solutions are directly used for the predicted HVI calculation if theyare the predicted mean ˆ µ or the lower conﬁdence bound ˆ µ − β σ .

Original: When the surrogate values areexpected improvement, although the approximate Pareto set is learned with EI for each scalarization,we simply let ˆ f ( x s )
Revised: When the surrogate values areexpected improvement, although the approximate Pareto set is learned with EI for each scalarization,we simply let ˆ f ( x s )

Original: =
Revised: =

Original: ˆ µ − σ for the predicted HVI calculation.
Revised: ˆ µ − σ for the predicted HVI calculation.

Original: The main reasons for this choiceare: 1) we want to avoid the (repeatedly) time-consuming Monte Carlo integration for calculating theexpected hypervolume improvement; 2) the LCB (or the posterior mean only) is on the same scale asthe value of those already-evaluated solutions, which make the calculation of HVI meaningful.
Revised: The main reasons for this choiceare: 1) we want to avoid the (repeatedly) time-consuming Monte Carlo integration for calculating theexpected hypervolume improvement; 2) the LCB (or the posterior mean only) is on the same scale asthe value of those already-evaluated solutions, which make the calculation of HVI meaningful.

Original: Since it is computationally intensive to ﬁnd B solutions to exactly optimize HVI, we select thesolution batch X B in a sequentially greedy manner from X .
Revised: Since it is computationally intensive to ﬁnd B solutions to exactly optimize HVI, we select thesolution batch X B in a sequentially greedy manner from X .

Original: Similar approaches have also been usedin the current MOBO methods [47, 15].
Revised: Similar approaches have also been usedin the current MOBO methods [47, 15].

Original: The greedy selection approach is presented in Algorithm 4 .
Revised: The greedy selection approach is presented in Algorithm 4 .

Original: We sequentially sample solutions from the candidate set X C , and add the single solution x s with thebest HVI value into the selected set X B .
Revised: We sequentially sample solutions from the candidate set X C , and add the single solution x s with thebest HVI value into the selected set X B .

Original: For all problems, the acquisition search procedure (solutionsampling + individual search + batch selection) with batch size B = 10 typically costs less than 3seconds.
Revised: For all problems, the acquisition search procedure (solutionsampling + individual search + batch selection) with batch size B = 10 typically costs less than 3seconds.

Original: The detailed results can be found in Table 3.
Revised: The detailed results can be found in Table 3.

Original: E 
Revised: E Problem details

Original: [Table]
Revised: [Table]

Original: E.1 Synthetic benchmark problems
Revised: E.1 Synthetic benchmark problems

Original: We test our proposed PSL algorithm on different widely-used synthetic multi-objective optimizationbenchmark problems, namely ZDT1-3 [90], VLMOP1-3 [79], and DTLZ2 [19].
Revised: We test our proposed PSL algorithm on different widely-used synthetic multi-objective optimizationbenchmark problems, namely ZDT1-3 [90], VLMOP1-3 [79], and DTLZ2 [19].

Original: The input and outputdimensions of these algorithms are shown in Table 2.
Revised: The input and outputdimensions of these algorithms are shown in Table 2.

Original: These synthetic problems have known Paretosets M ps and Pareto fronts f ( M ps ) with the nadir point:
Revised: These synthetic problems have known Paretosets M ps and Pareto fronts f ( M ps ) with the nadir point:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where x 1 , . .
Revised: where x 1 , . .

Original: x m are solutions in the Pareto set.
Revised: x m are solutions in the Pareto set.

Original: We set the reference point r = 1 .
Revised: We set the reference point r = 1 .

Original: 1 × z nadir foreach problem.
Revised: 1 × z nadir foreach problem.

Original: E.2 New problems with various shape of Pareto front
Revised: E.2 New problems with various shape of Pareto front

Original: Among the synthetic benchmarks, the ZDT problem has a simple Pareto set, which is a line on theboundary of the decision space, and most decision variables of all Pareto optimal solutions share thesame values (e.g., 0).
Revised: Among the synthetic benchmarks, the ZDT problem has a simple Pareto set, which is a line on theboundary of the decision space, and most decision variables of all Pareto optimal solutions share thesame values (e.g., 0).

Original: These properties could make ZDT problems not appropriate for comparingdifferent MOBO algorithms.
Revised: These properties could make ZDT problems not appropriate for comparingdifferent MOBO algorithms.

Original: We chose the ZDT problem for comparison solely due to it is widelyused in multi-objective optimization, and our proposed PSL method is not specially designed forsuch problems.
Revised: We chose the ZDT problem for comparison solely due to it is widelyused in multi-objective optimization, and our proposed PSL method is not specially designed forsuch problems.

Original: In this subsection, to better evaluate our proposed PSL method, we propose 6 new test problemsof which the Pareto sets are curves in the search space.
Revised: In this subsection, to better evaluate our proposed PSL method, we propose 6 new test problemsof which the Pareto sets are curves in the search space.

Original: The test problems (F1 - F6) are deﬁnedin the next page, and the experimental results on these newly proposed problems can be found in
Revised: The test problems (F1 - F6) are deﬁnedin the next page, and the experimental results on these newly proposed problems can be found in

Original: Appendix. F.1.
Revised: Appendix. F.1.

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original:  where s 1 = (cid:88)
Revised:  where s 1 = (cid:88)

Original: j
Revised: j

Original: [Equation]
Revised: [Equation]

Original: J
Revised: J

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: and s 2 = (cid:88)
Revised: and s 2 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: where s 1 = (cid:88)
Revised: where s 1 = (cid:88)

Original: j ∈ J
Revised: j ∈ J

Original: [Equation]
Revised: [Equation]

Original: − x
Revised: − x

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) 2 and s 2 = (cid:88)
Revised: ) 2 and s 2 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original:  where s 1 = (cid:88)
Revised:  where s 1 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: ∈ J( x j − sin
Revised: ∈ J( x j − sin

Original: [Equation]
Revised: [Equation]

Original: 4 π x
Revised: 4 π x

Original: [Equation]
Revised: [Equation]

Original: jπ n
Revised: jπ n

Original: [Equation]
Revised: [Equation]

Original: ) 2 and s 2 = (cid:88)
Revised: ) 2 and s 2 = (cid:88)

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original:  where s 1 = (cid:88)
Revised:  where s 1 = (cid:88)

Original: j ∈ J
Revised: j ∈ J

Original: ( x j − 0 .
Revised: ( x j − 0 .

Original: 8 x 1 cos(4 π x 1 + jπ n )) 2 and s 2 = (cid:88)
Revised: 8 x 1 cos(4 π x 1 + jπ n )) 2 and s 2 = (cid:88)

Original: j ∈ J( x
Revised: j ∈ J( x

Original: [Equation]
Revised: [Equation]

Original: − 0 .
Revised: − 0 .

Original: 8 x 1 sin(4 π x 1 + jπ n )) 2 ,
Revised: 8 x 1 sin(4 π x 1 + jπ n )) 2 ,

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: [Equation]
Revised: [Equation]

Original: where s 1 = (cid:88) j ∈ J
Revised: where s 1 = (cid:88) j ∈ J

Original: ( x j − 0 .
Revised: ( x j − 0 .

Original: 8 x 1 cos(4 π x 1 + jπn
Revised: 8 x 1 cos(4 π x 1 + jπn

Original: [Equation]
Revised: [Equation]

Original: )) 2 and s 2 = (cid:88)
Revised: )) 2 and s 2 = (cid:88)

Original: j ∈ J
Revised: j ∈ J

Original: ( x j − 0 .
Revised: ( x j − 0 .

Original: 8 x 1 sin
Revised: 8 x 1 sin

Original: [Equation]
Revised: [Equation]

Original: 4 π
Revised: 4 π

Original: [Equation]
Revised: [Equation]

Original: 1 + jπ n ))
Revised: 1 + jπ n ))

Original: [Equation]
Revised: [Equation]

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: f
Revised: f

Original: [Equation]
Revised: [Equation]

Original: where s 1 = (cid:88) j ∈ J{ x j − [0 .
Revised: where s 1 = (cid:88) j ∈ J{ x j − [0 .

Original: 3 x 21 cos(12 π x 1 + 4 jπ n ))
Revised: 3 x 21 cos(12 π x 1 + 4 jπ n ))

Original: + 0 .
Revised: + 0 .

Original: 6 x 1 ] cos(6 π x 1 + jπ n ) } and s 2 = (cid:88)
Revised: 6 x 1 ] cos(6 π x 1 + jπ n ) } and s 2 = (cid:88)

Original: j ∈ J{ x j − [0 .
Revised: j ∈ J{ x j − [0 .

Original: 3 x 21 cos(12 π x 1 + 4 jπ n ))
Revised: 3 x 21 cos(12 π x 1 + 4 jπ n ))

Original: + 0 .
Revised: + 0 .

Original: 6 x 1 ] sin(6 π x 1 + jπ n ) } ,
Revised: 6 x 1 ] sin(6 π x 1 + jπ n ) } ,

Original: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =
Revised: J 1 = { j | j is odd and 2 ≤ j ≤ n } and J 2 =

Original: { j | j is even and 2 ≤ j ≤ n } .
Revised: { j | j is even and 2 ≤ j ≤ n } .

Original: E.3 Real-world application problems
Revised: E.3 Real-world application problems

Original: We also conduct experiments on 5 real-world multi-objective engineering design problems [76] thatare initially proposed in different communities for different applications:
Revised: We also conduct experiments on 5 real-world multi-objective engineering design problems [76] thatare initially proposed in different communities for different applications:

Original: Four Bar Truss Design.
Revised: Four Bar Truss Design.

Original: This problem is to design a four-bar truss.
Revised: This problem is to design a four-bar truss.

Original: The two objectives to optimizeare its structural volume and joint displacement.
Revised: The two objectives to optimizeare its structural volume and joint displacement.

Original: The decision variables are the length of the fourbars.
Revised: The decision variables are the length of the fourbars.

Original: The details of this problem can be found in Cheng and Li [10].
Revised: The details of this problem can be found in Cheng and Li [10].

Original: Pressure Vessel Design.
Revised: Pressure Vessel Design.

Original: This problem is to design a cylindrical pressure vessel.
Revised: This problem is to design a cylindrical pressure vessel.

Original: The two objectivesto minimize are the total cost (material, forming, and welding) and the violations of three differentdesign constraints.
Revised: The two objectivesto minimize are the total cost (material, forming, and welding) and the violations of three differentdesign constraints.

Original: This problem has four decision variables, which are the shell thicknesses, thepressure vessel head, the inner radius, and the length of the cylindrical section.
Revised: This problem has four decision variables, which are the shell thicknesses, thepressure vessel head, the inner radius, and the length of the cylindrical section.

Original: The details of thisproblem can be found in Kramer [42].
Revised: The details of thisproblem can be found in Kramer [42].

Original: Disk Brake Design.
Revised: Disk Brake Design.

Original: This problem is to design a disc brake.
Revised: This problem is to design a disc brake.

Original: The three objectives to minimize arethe mass, the minimum stopping time, and the violations of four design constraints.
Revised: The three objectives to minimize arethe mass, the minimum stopping time, and the violations of four design constraints.

Original: This problemhas four decision variables: the inner radius, the outer radius, the engaging force, and the number offriction surfaces.
Revised: This problemhas four decision variables: the inner radius, the outer radius, the engaging force, and the number offriction surfaces.

Original: The details of this problem can be found in Ray and Liew [63].
Revised: The details of this problem can be found in Ray and Liew [63].

Original: Gear Train Design.
Revised: Gear Train Design.

Original: This problem is to design a gear train with four gears.
Revised: This problem is to design a gear train with four gears.

Original: The three objectivesto minimize are the difference between the realized gear ration and the required speciﬁcation, themaximum size of four gears, and the design constraint violations.
Revised: The three objectivesto minimize are the difference between the realized gear ration and the required speciﬁcation, themaximum size of four gears, and the design constraint violations.

Original: The decision variables are thenumbers of teeth in each of the four gears.
Revised: The decision variables are thenumbers of teeth in each of the four gears.

Original: The details of this problem can be found in Deb and
Revised: The details of this problem can be found in Deb and

Original: Srinivasan [17].
Revised: Srinivasan [17].

Original: Rocket Injector Design.
Revised: Rocket Injector Design.

Original: This problem is to design a rocket injector that needs to minimize themaximum temperature of the injector face, the distance from the inlet, and the temperature on the posttip.
Revised: This problem is to design a rocket injector that needs to minimize themaximum temperature of the injector face, the distance from the inlet, and the temperature on the posttip.

Original: It has four decision variables, namely, the hydrogen ﬂow angle, the hydrogen area, the oxygenarea, and the oxidizer post tip thickness.
Revised: It has four decision variables, namely, the hydrogen ﬂow angle, the hydrogen area, the oxygenarea, and the oxidizer post tip thickness.

Original: The details of this problem can be found in Vaidyanathanet al.
Revised: The details of this problem can be found in Vaidyanathanet al.

Original: These real-world multi-objective design problems do not have known exact Pareto fronts.
Revised: These real-world multi-objective design problems do not have known exact Pareto fronts.

Original: We use theapproximate Pareto fronts provided by Tanabe and Ishibuchi [76] with a large number of evaluationsas our refereed Pareto fronts, which have also been used in other MOBO works.
Revised: We use theapproximate Pareto fronts provided by Tanabe and Ishibuchi [76] with a large number of evaluationsas our refereed Pareto fronts, which have also been used in other MOBO works.

Original: We set the referencepoint r = 1 .
Revised: We set the referencepoint r = 1 .

Original: 1 × ˆ z nadir where ˆ z nadir is the nadir point of the approximate Pareto front.
Revised: 1 × ˆ z nadir where ˆ z nadir is the nadir point of the approximate Pareto front.

Original: The probleminformation and reference points can be found in Table 2.
Revised: The probleminformation and reference points can be found in Table 2.

Original: E.4 Experiment settings
Revised: E.4 Experiment settings

Original: For all experiments, we ﬁrst randomly sample and evaluate 10 valid solutions with Latin hypercubesampling [50], and set them as the initial solutions { X 0 , y 0 } .
Revised: For all experiments, we ﬁrst randomly sample and evaluate 10 valid solutions with Latin hypercubesampling [50], and set them as the initial solutions { X 0 , y 0 } .

Original: Then we further run the MOBOalgorithm to optimize the given problem with a limited evaluation budget ( 100 for all problems).
Revised: Then we further run the MOBOalgorithm to optimize the given problem with a limited evaluation budget ( 100 for all problems).

Original: Inthe main paper, we report the results with a batch size 5 , and there are only 100 / 5 = 20 batchediterations for each algorithm.
Revised: Inthe main paper, we report the results with a batch size 5 , and there are only 100 / 5 = 20 batchediterations for each algorithm.

Original: In section F in this appendix, we also report experimental results withdifferent batch sizes.
Revised: In section F in this appendix, we also report experimental results withdifferent batch sizes.

Original: F
Revised: F

Original: Additional experiments
Revised: Additional experiments

Original: [Figure]
Revised: [Figure]

Original: F.1 Results on new challenging problems F1-F 
Revised: F.1 Results on new challenging problems F1-F

Original: [Figure]
Revised: [Figure]

Original: l o g H V D
Revised: l o g H V D

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: e r e n c e
Revised: e r e n c e

Original: [Figure]
Revised: [Figure]

Original: We run experiments on more challenging problems proposed in Appendix E.2 to further validate our
Revised: We run experiments on more challenging problems proposed in Appendix E.2 to further validate our

Original: [Figure]
Revised: [Figure]

Original: proposed PSL method.
Revised: proposed PSL method.

Original: The Pareto sets of these problems are all different complicated space curves in
Revised: The Pareto sets of these problems are all different complicated space curves in

Original: [Figure]
Revised: [Figure]

Original: the search space.
Revised: the search space.

Original: We have run the experiments with two different settings: 1) 6-dimensional problems
Revised: We have run the experiments with two different settings: 1) 6-dimensional problems

Original: [Figure]
Revised: [Figure]

Original: with 100 evaluations, and 2) 20-dimensional problems with 200 evaluations.
Revised: with 100 evaluations, and 2) 20-dimensional problems with 200 evaluations.

Original: The experimental results
Revised: The experimental results

Original: [Figure]
Revised: [Figure]

Original: in Figure 16 show that our proposed PSL method still has a promising performance on these problems.
Revised: in Figure 8 show that our proposed PSL method still has a promising performance on these problems.

Original: [Table]
Revised: [Table]

Original: We report the run time for each algorithm with batch size 10 per iteration in Table 3.
Revised: We report the run time for each algorithm with batch size 10 per iteration in Table 3.

Original: For our proposed
Revised: For our proposed

Original: PSL algorithm, we further report the detailed run time for learning the set model and for searchingthe batch solutions.
Revised: PSL algorithm, we further report the detailed run time for learning the set model and for searchingthe batch solutions.

Original: According to the results, PSL can efﬁciently learn the set model and select abatch of solutions for evaluation at each iteration with a total time budget of less than 10 seconds.
Revised: According to the results, PSL can efﬁciently learn the set model and select abatch of solutions for evaluation at each iteration with a total time budget of less than 10 seconds.

Original: The short PSL batch selection run time is expected, since all solutions are directly sampled fromthe learned Pareto set with a few further local search steps, rather than optimizing the acquisitionfunction(s) from scratch as in other MOBO methods.
Revised: The short PSL batch selection run time is expected, since all solutions are directly sampled fromthe learned Pareto set with a few further local search steps, rather than optimizing the acquisitionfunction(s) from scratch as in other MOBO methods.

Original: We want to emphasize that the run time for different MOBO algorithms strongly depends on theimplementation, which is also highlighted in the current works [13, 47].
Revised: We want to emphasize that the run time for different MOBO algorithms strongly depends on theimplementation, which is also highlighted in the current works [13, 47].

Original: The objective evaluationsin real-world applications are usually very expensive and involve costly real-world experiments orsimulations that take days to run.
Revised: The objective evaluationsin real-world applications are usually very expensive and involve costly real-world experiments orsimulations that take days to run.

Original: All the MOBO algorithms are efﬁcient and have negligible run-timeoverheads in these real-world application scenarios.
Revised: All the MOBO algorithms are efﬁcient and have negligible run-timeoverheads in these real-world application scenarios.

Original: F.3 Performance with different acquisition functions
Revised: F.3 Performance with different acquisition functions

Original: [Figure]
Revised: [Figure]

Original: [Equation][Figure] [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: ( =0.1) LCB ( =0.5) LCB ( =1.0) LCB ( =2.0)
Revised: ( =0.1) LCB ( =0.5) LCB ( =1.0) LCB ( = [Figure]

Original: [Figure]
Revised: [Figure]

Original: 100 Function Evaluations 7.
Revised: 100 Function Evaluations 7.

Original: l o g H V D i
Revised: l o g H V D i

Original: ff e r e n c e
Revised: ff e r e n c e

Original: EIPredicted
Revised: EIPredicted

Original: [Figure]
Revised: [Figure]

Original: LCB ( =0.1) LCB ( =0.5) LCB ( =1.0) LCB ( =2.0)
Revised: LCB ( =0.1) LCB ( =0.5) LCB ( =1.0) LCB ( =2.0)

Original: [Figure]
Revised: [Figure]

Original: For the proposed PSL method, We can use different acquisition functions as the surrogate value forthe Pareto set learning approach.
Revised: For the proposed PSL method, We can use different acquisition functions as the surrogate value forthe Pareto set learning approach.

Original: Here, we compare the performance with 6 different acquisitionfunctions/settings, namely, the Expected Improvement (EI), Predicted Mean, and the Lower Conﬁdence Bound (LCB) with β = 0 .
Revised: Here, we compare the performance with 6 different acquisitionfunctions/settings, namely, the Expected Improvement (EI), Predicted Mean, and the Lower Conﬁdence Bound (LCB) with β = 0 .

Original: 5 , 1 . 0 and 2 . 

Revised: 5 , 1 . 0 and 2 .

Original: on four benchmark and real-world applicationproblems.
Revised: on four benchmark and real-world applicationproblems.

Original: According to the results shown in Figure 9, PSL with all 6 acquisition functions has reasonablygood performance.
Revised: According to the results shown in Figure 9, PSL with all 6 acquisition functions has reasonablygood performance.

Original: Although PSL with EI performs slightly worse than the other acquisitions, itcan still outperform many other MOBO algorithms with the results shown in Figure 5 of the mainpaper.
Revised: Although PSL with EI performs slightly worse than the other acquisitions, itcan still outperform many other MOBO algorithms with the results shown in Figure 5 of the mainpaper.

Original: These results conﬁrm that the learning the approximate Pareto set during optimization couldbe beneﬁcial for MOBO.
Revised: These results conﬁrm that the learning the approximate Pareto set during optimization couldbe beneﬁcial for MOBO.

Original: Among the other acquisition functions and settings, no single choice can achieve the best performancefor all problems.
Revised: Among the other acquisition functions and settings, no single choice can achieve the best performancefor all problems.

Original: The predicted mean acquisition can be seen as LCB with β = 0 , which is a pureexploitation strategy without considering the uncertainty (e.g., predicted variance) for exploration.
Revised: The predicted mean acquisition can be seen as LCB with β = 0 , which is a pureexploitation strategy without considering the uncertainty (e.g., predicted variance) for exploration.

Original: It might be surprising that this greedy acquisition can still have a pretty good performance.
Revised: It might be surprising that this greedy acquisition can still have a pretty good performance.

Original: Indeed,this greedy and purely exploitative approach has been recently studied and shown to have promisingperformance for single-objective BO [64, 16].
Revised: Indeed,this greedy and purely exploitative approach has been recently studied and shown to have promisingperformance for single-objective BO [64, 16].

Original: DGEMO [47] also uses the predicted mean as anacquisition function for MOBO.
Revised: DGEMO [47] also uses the predicted mean as anacquisition function for MOBO.

Original: The BO textbook [26] brieﬂy discusses this pure exploitative strategyat the end of Section 7.10 (posterior mean acquisition function and lookahead).
Revised: The BO textbook [26] brieﬂy discusses this pure exploitative strategyat the end of Section 7.10 (posterior mean acquisition function and lookahead).

Original: One limitation ofthis greedy approach is that it might be stuck at a non-optimal location with no well-ﬁtted surrogatemodel due to overexploitation [26].
Revised: One limitation ofthis greedy approach is that it might be stuck at a non-optimal location with no well-ﬁtted surrogatemodel due to overexploitation [26].

Original: There are many acquisition policies for Bayesian optimization, but ﬁnding the most suitable one for agiven problem is still an open question [26], which is also the case for our PSL method.
Revised: There are many acquisition policies for Bayesian optimization, but ﬁnding the most suitable one for a [Table] given problem is still an open question [26], which is also the case for our PSL method.

Original: [Table]
Revised: [Table]

Original: F.4 Quality of the learned Pareto front
Revised: F.4 Quality of the learned Pareto front

Original: [Table]
Revised: [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Figure]

Original: [Figure]
Revised: [Figure]

Original:  g H V D i ff e r e n 
Revised: l o g H V D i ff e r e n c e

Original: [Figure] e TS
Revised: TS-TCHTS-TCH (Approx. PF)

Original: [Figure]
Revised: [Figure]

Original: In the main paper, we have demonstrated that the proposed PSL method can successfully approximatethe ground truth Pareto front with a limited evaluation budget as in Figure 6.
Revised: In the main paper, we have demonstrated that the proposed PSL method can successfully approximatethe ground truth Pareto front with a limited evaluation budget as in Figure 6.

Original: In this subsection, wefurther investigate the performance of the learned Pareto fronts during the optimization process.
Revised: In this subsection, wefurther investigate the performance of the learned Pareto fronts during the optimization process.

Original: Asshown in Figure 12, the learned Pareto fronts have good quality (with 1,000 random samples) duringoptimization and lead to promising evaluated solutions performance.
Revised: Asshown in Figure 10, the learned Pareto fronts have good quality (with 1,000 random samples) duringoptimization and lead to promising evaluated solutions performance.

Original: In addition, we also compare our Pareto set model with another modeling approach based on evaluatedsolutions.
Revised: In addition, we also compare our Pareto set model with another modeling approach based on evaluatedsolutions.

Original: Speciﬁcally, we run the TS-TCH algorithm and record the evaluated solutions with theircorresponding preferences during the optimization process.
Revised: Speciﬁcally, we run the TS-TCH algorithm and record the evaluated solutions with theircorresponding preferences during the optimization process.

Original: Then we build a model to approximatethe Pareto front based on the evaluated and non-dominated solutions at each iteration step.
Revised: Then we build a model to approximatethe Pareto front based on the evaluated and non-dominated solutions at each iteration step.

Original: The modelwe built has an identical structure (2-layer MLP) to the PSL model, but is now trained in a supervisedmanner.
Revised: The modelwe built has an identical structure (2-layer MLP) to the PSL model, but is now trained in a supervisedmanner.

Original: According to the results in Figure 12, our proposed PSL model has better quality than themodel trained in the supervised manner with evaluated solutions.
Revised: According to the results in Figure 10, our proposed PSL model has better quality than themodel trained in the supervised manner with evaluated solutions.

Original: F.5 Impact of the aggregation function and Pareto set model
Revised: F.5 Impact of the aggregation function and Pareto set model

Original: [Figure]
Revised: [Figure]

Original: ff e r e n c e
Revised: ff e r e n c e

Original: TCH Aggregation (without NN) PSL (
Revised: TCH Aggregation (without NN) PSL (

Original: [Figure]
Revised: [Figure]

Original: NN)
Revised: NN)

Original: [Figure]
Revised: [Figure]

Original: 100 Function Evaluations 1.
Revised: 100 Function Evaluations 1.

Original: l o g H V D i
Revised: l o g H V D i

Original: ff e r e n c e
Revised: ff e r e n c e

Original: TCH Aggregation (without NN) PSL (with NN)
Revised: TCH Aggregation (without NN) PSL (with NN)

Original: [Figure]
Revised: [Figure]

Original: In this work, we choose the weighted Chebyshev scalarization with an ideal point and epsilon mainlydue to its good theoretical property.
Revised: In this work, we choose the weighted Chebyshev scalarization with an ideal point and epsilon mainlydue to its good theoretical property.

Original: In this subsection, we conduct an ablation study on our proposed
Revised: In this subsection, we conduct an ablation study on our proposed

Original: PSL method with and without the Pareto set model.
Revised: PSL method with and without the Pareto set model.

Original: The results in Figure 11 show that simplyoptimizing the acquisition function for scalarization could lead to a signiﬁcant performance drop.
Revised: The results in Figure 11 show that simplyoptimizing the acquisition function for scalarization could lead to a signiﬁcant performance drop.

Original: Itconﬁrms that the proposed Pareto set model is important for the overall promising performance.
Revised: Itconﬁrms that the proposed Pareto set model is important for the overall promising performance.

Original: [Figure]
Revised: [Figure]

Original: We show the approximate Pareto set and Pareto front learned by PSL for the ZDT3 problem in Figure.
Revised: We show the approximate Pareto set and Pareto front learned by PSL for the ZDT3 problem in Figure.

Original: The ZDT3 problem has a continuous search space, but its Pareto set is disconnected.
Revised: The ZDT3 problem has a continuous search space, but its Pareto set is disconnected.

Original: According to theresult, PSL can ﬁnd an approximate continuous set that contains the disconnected parts of the Paretoset.
Revised: According to theresult, PSL can ﬁnd an approximate continuous set that contains the disconnected parts of the Paretoset.

Original: In this work, we use a simple fully-connected neural network as the Pareto set model, and do nottake the disconnected Pareto set into consideration.
Revised: In this work, we use a simple fully-connected neural network as the Pareto set model, and do nottake the disconnected Pareto set into consideration.

Original: How to design a more powerful Pareto set modelto capture the disconnected patterns of the Pareto set (e.g., maybe with search region decomposition)is worthwhile studying in the future.
Revised: How to design a more powerful Pareto set modelto capture the disconnected patterns of the Pareto set (e.g., maybe with search region decomposition)is worthwhile studying in the future.

Original: F.7 PSL with different preferences
Revised: F.7 PSL with different preferences

Original: 1.0 f 1 ( x ) 0.
Revised: 1.0 f 1 ( x ) 0.

Revised: [Figure]

Original: f 2 ( x )
Revised: f 2 ( x )

Original: Pref 1 Pref 2 Pref3 Pref 4 Pref
Revised: Pref 1 Pref 2 Pref

Original: [Figure]
Revised: [Figure]

Revised: Pref 4 Pref

Original: [Figure]
Revised: [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Suppose the decision-makers have different sets of preferred trade-offs, our model can successfully generate the corresponding parts of the Pareto fronts.
Revised: Suppose the decision-makers have different sets of preferred trade-offs, our model can successfully generate the corresponding parts of the Pareto fronts.

Original: The preference assignment is not ﬁxed, and the decision-makers can adjust it and obtain the corresponding approximate solutions in real-time.
Revised: The preference assignment is not ﬁxed, and the decision-makers can adjust it and obtain the corresponding approximate solutions in real-time.

Original: Our proposed PSL method enables the decision-makers to easily adjust their preference to explore thewhole approximate Pareto set for making decisions (e.g., see Figure 3 and Figure 6 in the main paper).
Revised: Our proposed PSL method enables the decision-makers to easily adjust their preference to explore thewhole approximate Pareto set for making decisions (e.g., see Figure 3 and Figure 6 in the main paper).

Original: This ability is not supported by other MOBO methods, which is important to ﬂexibly incorporate thedecision-maker’s preference and prior knowledge into Bayesian optimization [26].
Revised: This ability is not supported by other MOBO methods, which is important to ﬂexibly incorporate thedecision-maker’s preference and prior knowledge into Bayesian optimization [26].

Original: The decision-making process for real-world applications could be subjective.
Revised: The decision-making process for real-world applications could be subjective.

Original: We follow a similar waywith other preference-based MOBO to show our method can successfully generate different trade-offsolutions.
Revised: We follow a similar waywith other preference-based MOBO to show our method can successfully generate different trade-offsolutions.

Original: In Figure 13, suppose the decision-makers have different sets of trade-off preferences, ourmodel can generate the corresponding parts of the Pareto fronts for different problems.
Revised: In Figure 13, suppose the decision-makers have different sets of trade-off preferences, ourmodel can generate the corresponding parts of the Pareto fronts for different problems.

Original: We uniformlysample 1 , 000 and 10 , 011 preferences for the two-objective and three-objective problems respectively,and use the learned Pareto set models to generate the corresponding approximate Pareto solutions.
Revised: We uniformlysample 1 , 000 and 10 , 011 preferences for the two-objective and three-objective problems respectively,and use the learned Pareto set models to generate the corresponding approximate Pareto solutions.

Original: Wefurther divide the preferences into 5 exclusive parts, and label them with the corresponding solutionsin different colors.
Revised: Wefurther divide the preferences into 5 exclusive parts, and label them with the corresponding solutionsin different colors.

Original: As shown in Figure 13, the corresponding connection from the preferences to thesolutions is clear for all problems.
Revised: As shown in Figure 13, the corresponding connection from the preferences to thesolutions is clear for all problems.

Original: By directly exploring the approximate Pareto front in an interactive manner, the decision-makerscan observe and understand the connection between preferences and corresponding solutions inreal-time.
Revised: By directly exploring the approximate Pareto front in an interactive manner, the decision-makerscan observe and understand the connection between preferences and corresponding solutions inreal-time.

Original: This ability could be beneﬁcial for the decision-makers to further adjust and assign theirmost accurate preferences.
Revised: This ability could be beneﬁcial for the decision-makers to further adjust and assign theirmost accurate preferences.

Original: [Figure]
Revised: [Figure]

Original: In addition to the predicted mean, our PSL method also provides the uncertainty information (e.g.,predicted standard deviation) to support decision-making.
Revised: In addition to the predicted mean, our PSL method also provides the uncertainty information (e.g.,predicted standard deviation) to support decision-making.

Original: Figure 14 shows the (log) predictedstandard deviations for different objectives on the approximate Pareto front of DTLZ2.
Revised: Figure 14 shows the (log) predictedstandard deviations for different objectives on the approximate Pareto front of DTLZ2.

Original: According tothe results in Figure 14(a)(b)(c), we can observe that the PSL model has a low predicted standarddeviations for almost all locations on the approximate Pareto front except the top corner and the topleft boundary.
Revised: According tothe results in Figure 14(a)(b)(c), we can observe that the PSL model has a low predicted standarddeviations for almost all locations on the approximate Pareto front except the top corner and the topleft boundary.

Original: From the log standard deviations in Figure 14(d)(e)(f), we ﬁnd that PSL has a higheroverall uncertainty level for objective 1/2 than objective 3, where the uncertainty distributions arequite similar for the ﬁrst two objectives.
Revised: From the log standard deviations in Figure 14(d)(e)(f), we ﬁnd that PSL has a higheroverall uncertainty level for objective 1/2 than objective 3, where the uncertainty distributions arequite similar for the ﬁrst two objectives.

Original: F.9 Problems with larger number of decision variables and objectives
Revised: F.9 Problems with larger number of decision variables and objectives

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: l o g H V D i ff e r e n c e
Revised: l o g H V D i ff e r e n c e

Original: NSGA-IIMOEA/D-EGOTSEMOUSeMO-EI
Revised: NSGA-IIMOEA/D-EGOTSEMOUSeMO-EI

Original: DGEMOSobolqParEGOTS-TCHqEVHIMESMOPSL (Ours)(a
Revised: [Figure] -TCHqEVHIMESMOPSL (Ours)

Original: [Figure]
Revised: [Figure]

Original: ZDT
Revised: n

Original: [Figure]
Revised: [Figure]

Original: Evaluations 1.

Original: l o g H V D i ff e r e n c e (b) ZDT     100 Function Evaluations 1. l o g H V D i ff e r e n c e
Revised: e

Original: [Figure]
Revised: [Figure]

Original:      200 Function Evaluations
Revised: l o g H V D i ff e r e n c e (c) ZDT     200 Function Evaluations

Original: H y p
Revised: H y p

Original: e r v o l u m e 1e
Revised: e r v o l u m e 1e

Original: SobolNSGA-IIMOEA/D-EGOqParEGOTS-TCH
Revised: SobolNSGA-IIMOEA/D-EGOqParEGOTS-TCH

Original: TSEMOUSeMO-EIMESMOPSL (Ours) [Figure] ) DTLZ
Revised: TSEMOUSeMO-EIMESMOPSL (Ours) (d) DTLZ

Original: We also test PSL on problems with a larger number of decision variables and objectives.
Revised: We also test PSL on problems with a larger number of decision variables and objectives.

Original: Figure 15shows the results on 2-objective ZDT1-3 problems with 20 decision variables and the 10 -objective
Revised: Figure 15shows the results on 2-objective ZDT1-3 problems with 20 decision variables and the 10 -objective

Original: DTLZ2 problem with 50 decision variables.
Revised: DTLZ2 problem with 50 decision variables.

Original: For these problems, all algorithms have 50 randomlygenerated initial solutions, and we conduct 20 batched evaluations with batch size 10.
Revised: For these problems, all algorithms have 50 randomlygenerated initial solutions, and we conduct 20 batched evaluations with batch size 10.

Original: Therefore,there are total 250 evaluations for each algorithm.
Revised: Therefore,there are total 250 evaluations for each algorithm.

Original: According to the results, PSL still performs well on ZDT problems with 20 decision variables.
Revised: According to the results, PSL still performs well on ZDT problems with 20 decision variables.

Original: Forthe 10 -objective
Revised: Forthe 10 -objective

Original: DTLZ2 problem, all algorithms have 0 hypervolume since the obtained solutions arefar from the Pareto front.
Revised: DTLZ2 problem, all algorithms have 0 hypervolume since the obtained solutions arefar from the Pareto front.

Original: We do not include the results for DEGMO and qEHVI.
Revised: We do not include the results for DEGMO and qEHVI.

Original: The DEGMO codedoes not support problems with larger than 3 objectives.
Revised: The DEGMO codedoes not support problems with larger than 3 objectives.

Original: The calculation of expected hypervolume improvement could be time-consuming, and the efﬁcient computation methods proposed in qEHVI [13]and qNEHVI [14] make it possible to properly handle problems with 3-4 objective.
Revised: The calculation of expected hypervolume improvement could be time-consuming, and the efﬁcient computation methods proposed in qEHVI [13]and qNEHVI [14] make it possible to properly handle problems with 3-4 objective.

Original: However, thescalability is still a key limitation for the current MOBO methods.
Revised: However, thescalability is still a key limitation for the current MOBO methods.

Original: How to efﬁciently solve problemswith a large number of decision variables and objectives could be an important research direction for
Revised: How to efﬁciently solve problemswith a large number of decision variables and objectives could be an important research direction for

Original: MOBO.
Revised: MOBO.

Original: F.10 Results on other ZDT and DTLZ problems
Revised: F.10 Results on other ZDT and DTLZ problems

Original: [Figure]
Revised: [Figure]

Original: We run experiments on ZDT4, ZDT6, and DTLZ1 in this subsection.
Revised: We run experiments on ZDT4, ZDT6, and DTLZ1 in this subsection.

Original: The results show that theproposed PSL method can achieve promising results on the ZDT6 problem, while none of the MOBOalgorithms can have satisfactory performance for ZDT4 and DTLZ1.
Revised: The results show that theproposed PSL method can achieve promising results on the ZDT6 problem, while none of the MOBOalgorithms can have satisfactory performance for ZDT4 and DTLZ1.

Original: Since the obtained solutionsare all far from the ground truth Pareto front and the default reference point, all MOBO methodshave a zero hypervolume value and hence lead to the same value for hypervolume difference.
Revised: Since the obtained solutionsare all far from the ground truth Pareto front and the default reference point, all MOBO methodshave a zero hypervolume value and hence lead to the same value for hypervolume difference.

Original: Onepossible reason for the poor performance is that the ZDT4 and DTLZ1 problems have a complicatedoptimization landscape (e.g., with a large number of local optimums).
Revised: Onepossible reason for the poor performance is that the ZDT4 and DTLZ1 problems have a complicatedoptimization landscape (e.g., with a large number of local optimums).

Original: With a limited evaluationbudget, the MOBO methods all fail to build a good surrogate model for such problems, and hencehave poor performance. 
Revised: With a limited evaluationbudget, the MOBO methods all fail to build a good surrogate model for such problems, and hencehave poor performance. [Figure]

Original: F.11 Performance with Different Batch Sizes
Revised: In this subsection, we conduct an ablation study on maximizing the hypervolume improvement with LCB on the approximate Pareto set vs. searching across the whole design space. The experimentalresults on three newly proposed problems (i.e., F4, F5, and F6) with complicated Pareto sets areshown in Figure 17. We also provide the qEHVI results as reference.

Revised: Based on these results, it is clear that our proposed PSL method can outperform directly searchingthe entire search space to maximize HVI with LCB.

Revised: The qHVI-LCB method performs similarly toqEHVI, which indicates that different acquisitions (for HVI) do not have a signiﬁcant impact on theoptimization performance.

Revised: In other words, with the learned Pareto set, our proposed PSL methodcan efﬁciently conduct the acquisition optimization mainly on the promising low-dimensional (e.g.,(m-1)) manifold, and then lead to signiﬁcantly better Bayesian optimization performance.

Revised: One possible reason for this performance gap could be the difﬁculty of directly searching the wholedesign search for maximizing HVI.

Revised: As shown in recent works [89, 15], the performance of qEHVIcan be improved using meta search region management algorithms such as LaMOO [89] and trustregion methods [15].

Revised: It implies that directly searching the whole design space is not efﬁcient.

Revised: These meta-algorithms adaptively decompose the whole search region into different subregionsalong with the optimization process, while our PSL method directly learns the whole approximate

Revised: Pareto set at each step.

Revised: We will add the comparison with those meta-algorithms in the ﬁnal version(LaMOO [89] is not open-sourced yet, and the code for the trust-region method [15] was just releasedseveral days ago).

Revised: Studying how to efﬁciently combine PSL with these algorithms could be animportant future work as discussed in Appendix B.

Revised: Together with the discussion in our initial response, the advantages of PSL can be summarized fromthe following two viewpoints:

Revised: • From the viewpoint of scalarization based methods, PSL proposes a novel two-stage approach to select a small batch of appropriate solutions from the approximate Pareto front,

Revised: which takes those already-evaluated solutions into consideration.

Revised: As a result, it signiﬁcantlyoutperforms other scalarization-based methods.• From the viewpoint of hypervolume improvement based methods, in an ideal case, the setof solutions that optimize HVI should all be on the Pareto front.

Revised: Restricting the search onthe approximate Pareto set, PSL leads to an efﬁcient HVI algorithm.

Revised: [Figure]

Revised: Following your suggestion

Revised: [Figure]

Revised: we have added the experiment to compare the HVI-LCB acquisitionvalues obtained by different search methods in Appendix F.11.

Revised: We run our proposed PSL as theoptimization algorithm, and conduct different search methods at each iteration.

Revised: The compared searchmethods are: 1) searching across the entire design space, 2) our PLS search on the approximate

Revised: Pareto front, and 3) the HVI-LCB acquisition values for the whole approximate Pareto front with1,000 sampled solutions (which can be treated as the upper bound for our method).

Revised: Based on theresults, it is clear that our proposed PSL search can produce better HVI-LCB acquisition values thandirectly searching the entire design space.

Revised: It should be noticed that the reported results are obtained by running PSL, and the search methods

Revised: [Figure]

Revised: have the same surrogate functions and baseline HV (from the same evaluated solutions chosen by

Revised: PSL) at each iteration.

Revised: By running qHVI-LCB alone, the inefﬁcient acquisition search at eachiteration will lead to the accumulated worse overall performance as shown in Figure.17.

Revised: [Figure]

Revised: 4.0 Learned Pareto Front (f) PS: T =0.

Revised: 4.0 Learned Pareto Front (g) PS: T =0.

Revised: Learned Pareto Front (h) PS: T =0.

Revised: Learned Pareto Front (i) PS: T =0.

Revised: Learned Pareto Front (j) PS: T =

Revised: Due to the simple Pareto set, the optimization performance on ZDT1 could be signiﬁcantly affectedby the initialization.

Revised: In the experiment, we did not leverage this property for an unfair comparison onthe ZDT problems.

Revised: PSL randomly initializes the parameters of the Pareto set model at each step, andthen uses the gradient-based method to learn the approximate Pareto set.

Revised: An illustration on how PSLlearns the approximate Pareto set/front is shown in Figure 19.

Revised: In our approach, the initially learned

Revised: Pareto set is a random curve in the search space far away from the boundary.

Revised: Then it graduallyconverges to the Pareto set with the efﬁcient gradient-based update.

Revised: [Figure]

Revised: [Figure]

Revised: In this subsection, we conduct an ablation on small perturbing the best points v.s. Pareto set modelfor generating the candidate set.

Revised: Based on the results shown in Figure 20, the Gaussian perturbing method can improve the performanceof simple scalarization on problems with simple Pareto set (e.g., ZDT1) but not the problems withcomplicated Pareto set (e.g., F4-F6).

Revised: On all problems, our proposed PSL method still achieves thebest performance.

Revised: DGEMO [47] also has a local search approach to expand the candidate set around the best points(in term of surrogate value) with the ﬁrst and second derivatives of the GP surrogate model.

Revised: In ourexperiments, PSL can outperform DGEMO on most problems, which conﬁrms the importance of the

Revised: NN for generating the candidate set.

Revised: On the other hand, the local search methods might provide complementary candidate solutions to

Revised: PSL, especially at the early stage of optimization when the approximate Pareto set is inaccurate.

Revised: Wewill investigate how to efﬁciently combine PSL with the local search approaches in future work.

Original: We conduct additional experiments with different batch size ( B = 10 ) as well as sequential evaluation( B = 1 ) for all problems.
Revised: We conduct additional experiments with different batch size ( B = 10 ) as well as sequential evaluation( B = 1 ) for all problems.

Original: The total number of evaluations is still 10 + 

Revised: The total number of evaluations is still 10 +

Original: =
Revised: =

Original: 110 for all experiments.
Revised: 110 for all experiments.

Original: In other words, these settings have 10 and 100 batched evaluations, respectively.
Revised: In other words, these settings have 10 and 100 batched evaluations, respectively.

Original: The experimentalresults are shown in Figure 17 and Figure 18.
Revised: The experimentalresults are shown in Figure 21 and Figure 22.

Original: According to the results, our proposed PSL methodconsistently has the best overall performance for different batch sizes.
Revised: According to the results, our proposed PSL methodconsistently has the best overall performance for different batch sizes.

Original: It outperforms many other
Revised: It outperforms many other

Original: MOBO approaches on a large majority of experiments and has comparable performance with the bestalgorithm when it is not the best one.
Revised: MOBO approaches on a large majority of experiments and has comparable performance with the bestalgorithm when it is not the best one.

Original: In this work, we focus on the main idea of learning the Pareto set for MOBO problems, and donot aggressively include other powerful methods into our proposed algorithm.
Revised: In this work, we focus on the main idea of learning the Pareto set for MOBO problems, and donot aggressively include other powerful methods into our proposed algorithm.

Original: Its performancemight be further improved by considering the second-order derivatives of GPs and diversity-guidedbatch selection in DGEMO [47], as well as Thompson sampling and uncertainty maximization in
Revised: Its performancemight be further improved by considering the second-order derivatives of GPs and diversity-guidedbatch selection in DGEMO [47], as well as Thompson sampling and uncertainty maximization in

Original: TSEMO [8] and USeMO-EI [5].
Revised: TSEMO [8] and USeMO-EI [5].

Original: These promising algorithm performances, along with the efﬁcient run time, conﬁrm that it is worthbuilding the Pareto set model for expensive multi-objective optimization problems.
Revised: These promising algorithm performances, along with the efﬁcient run time, conﬁrm that it is worthbuilding the Pareto set model for expensive multi-objective optimization problems.

Original: We believe the
Revised: We believe the

Original: Pareto set learning approach could be an efﬁcient off-the-shelf tool for multi-objective Bayesianoptimization.
Revised: Pareto set learning approach could be an efﬁcient off-the-shelf tool for multi-objective Bayesianoptimization.

Original: [Figure]
Revised: [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: r
Revised: r

Original: [Figure]
Revised: [Figure]

Original: n
Revised: n

Original: [Figure]
Revised: [Figure]

Original: (d) VLMOP
Revised: (d) VLMOP

Original: [Figure]
Revised: [Figure]

Original: (f) VLMOP
Revised: (f) VLMOP

Original: [Figure]
Revised: [Figure]

Original: V D
Revised: V D

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: n c e (g) DTLZ
Revised: n c e (g) DTLZ

Original: [Figure]
Revised: [Figure]

Original: V D
Revised: V D

Original: i
Revised: i

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: ) VLMOP
Revised: ) VLMOP

Original: [Figure]
Revised: [Figure]

Original: (g) DTLZ
Revised: (g) DTLZ

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: l o g H V D i ff e r e n c e
Revised: l o g H V D i ff e r e n c e

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: G 
Revised: G Licenses

Original: Licenses

Original: Table 4 lists the licenses for the codes and problem suite we used in this work.
Revised: Table 4 lists the licenses for the codes and problem suite we used in this work.

Original: [Table]
Revised: [Table]

Original: [Table]