M ULTI - OBJECTIVE O PTIMIZATION BY L EARNING S PACE P ARTITIONS
Anonymous authors
Paper under double-blind review
A BSTRACT
In contrast to single-objective optimization (SOO), multi-objective optimization (MOO) requires an optimizer to ﬁnd the Pareto frontier, a subset of feasible solutions that are not dominated by other feasible solutions.
In this paper, we propose
LaMOO, a novel multi-objective optimizer that learns a model from observed samples to partition the search space and then focus on promising regions that are likely to contain a subset of the Pareto frontier.
The partitioning is based on the dominance number , which measures “how close” a data point is to the Pareto frontier among existing samples.
To account for possible partition errors due to limited samples and model mismatch, we leverage Monte Carlo Tree Search (MCTS) to exploit promising regions while exploring suboptimal regions that may turn out to contain good solutions later.
Theoretically, we prove the efﬁcacy of learning space partitioning via LaMOO under certain assumptions.
Empirically, on the HyperVolume (HV) benchmark, a popular MOO metric, LaMOO substantially outperforms strong baselines on multiple real-world MOO tasks, by up to 225% in sample efﬁciency for neural architecture search on Nasbench201, and up to 10% for molecular design.
I NTRODUCTION
Multi-objective optimization (MOO) has been extensively used in many practical scenarios involving trade-offs between multiple objectives.
For example, in automobile design (Chang, 2015), we must maximize the performance of the engine while simultaneously minimizing emissions and fuel consumption.
In ﬁnance (Gunantara, 2018), one prefers a portfolio that maximizes the expected return while minimizing risk.
Mathematically, in MOO we optimize M objectives f ( x ) =
[ f 1 ( x ) , f 2 ( x ) , . .
, f M ( x )] ∈ R M : min f 1 ( x ) , f 2 ( x ) , ..., f M ( x ) (1) s .
t .
x ∈ Ω
While we could set arbitrary weights for each objective to turn it into a single-objective optimization (SOO) problem, modern MOO methods aim to ﬁnd the problem’s entire Pareto frontier : the set of solutions that are not dominated by any other feasible solutions 1 (see Fig.
1 for illustration).
The
Pareto frontier yields a global picture of optimal solution structures rather than focusing on one speciﬁc weighted combination of objectives.
As a result, MOO is fundamentally different from SOO.
Instead of focusing on a single optimal solution, a strong MOO optimizer should cover the search space broadly to explore the Pareto frontier.
Popular quality indicators in MOO, such as hypervolume (HV), capture this aspect by computing the volume of the currently estimated frontier.
Speciﬁcally, given a reference point R ∈ R M , as shown in Fig. 1(a), the hypervolume of a ﬁnite approximate Pareto set P is the M-dimensional
Lebesgue measure λ M of the space dominated by P and bounded from below by R .
That is,
HV ( P , R ) =
λ M ( ∪ |P| i =
[ R, y i ]) , where [ R, y i ] denotes the hyper-rectangle bounded by reference point R and y i .
Consequently, the optimizer must consider the diversity of solutions in addition to their optimality.
𝑯𝑽 𝓟 (o (𝒙)=0) 𝓡 o (𝒙)= 𝛀 𝒈𝒐𝒐𝒅
SearchSpace
High o (𝒙)
Low o (𝒙) 𝒇𝒇
ObjectiveSpace 𝛀 𝒃𝒂𝒅 (a) (b) o
(𝒙)=2 o (𝒙)= 𝑦𝑦𝑦𝑦 : 𝑦 ;
MOO methods
Sampling Method
Objectives>3 MOEA/D (Zhang & Li, 2007)
Evolution × CMA
[Figure]
ES (Igel et al., 2007a) × NSGA
[Figure]
II (
[Figure]
et al., 2002a)
[Figure]
[Figure] [Figure]
) √
PAREGO (Knowles,
[Figure]
Bayesian optimization √
[Figure] [Figure]
et al
[Figure] [Figure]
) √
LaMOO
[Figure]
Space partition √
While several previous works have proposed approaches to capture this diversity-optimality trade-off (Deb et al., 2002a; Knowles, 2006; Igel et al., 2007; Deb & Jain, 2014; Daulton et al., 2020), in this paper, we take a fundamentally different route by learning promising candidate regions from past explored samples.
Ideally, to ﬁnd the Pareto frontier in as few function evaluations as possible, we want to sample heavily in the Pareto optimal set Ω P , deﬁned as the
region of
input vectors that corresponds
to the Pareto
frontier.
One way to focus samples on Ω P is to gradually narrow the full search space down to the subregion containing Ω P via partitioning.
For example, in the case of quadratic objective functions, Ω P can be separated from the non-optimal set Ω \ Ω P via simple linear classiﬁers (see Observation 1,2).
Motivated by these observations, we thus design LaMOO, a novel MOO meta-optimizer that progressively partitions regions into sub-regions and then focuses on sub-regions that are likely to contain
Pareto-optimal regions, where existing solvers can help.
Therefore, LaMOO is a meta-algorithm.
Unlike cutting-plane methods (Loganathan & Sherali, 1987; Hinder, 2018; Vieira & Lisboa, 2019) that leverage the (sub)-gradient of convex objectives as the cutting plane, with global optimality guarantees, LaMOO is data-driven: it leverages previous samples to build classiﬁers to learn the partition and focuses future samples in these promising regions.
No analytical formula of objectives or their sub-gradients is needed.
LaMOO is a multi-objective extension of recent works (Wang et al., 2020; Yang et al., 2021) that also learn space partitions but for a single black-box objective.
Empirically, LaMOO outperforms existing approaches on many benchmarks, including standard benchmarks in multi-objective black-box optimization, and real-world multi-objective problems like neural architecture search (NAS) (Cai et al., 2019; 2020) and molecule design.
For example, as a meta-algorithm, LaMOO combined with CMA-ES as an inner routine requires only 62.5%, 8%, and 29% as many samples to reach the same hypervolume as the original CMA-ES (Igel et al., 2007a) in
BraninCurrin (Belakaria et al., 2019), VehicleSafety (Liao et al., 2008) and Nasbench201 (Dong &
Yang, 2020), respectively.
On average, compared to qEHVI, LaMOO uses 50% samples to achieve the same performance in these problems.
In addition, LaMOO with qEHVI (Daulton et al., 2020) and
CMA-ES require 71% and 31% fewer samples on average, compared to naive qEHVI and CMA-ES, to achieve the same performance in molecule discovery.
R ELATED W ORK
Bayesian Optimization (BO) (Zitzler et al., 2003; Knowles, 2006; Ponweiser et al., 2008; Couckuyt et al., 2014; Paria et al., 2018; Yang et al., 2019; Daulton et al., 2020) is a popular family of methods to optimize black-box single and multi-objectives.
Using observed samples, BO learns a surrogate model ˆ f ( x ) , search for new promising candidates based on acquisition function built on ˆ f ( x ) , and query the quality of these candidates with the ground truth black-box objective(s).
In multi-objective
Bayesian optimization (MOBO), most approaches leverage Expected Hypervolume Improvement (EHVI) as their acquisition function (Zitzler et al., 2003; Couckuyt et al., 2014;
Yang et al., 2019), since ﬁnding the Pareto frontier is equivalent to maximizing the hypervolume given a ﬁnite search space (Fleischer, 2003).
There are methods (Knowles, 2006; Ponweiser et al., 2008; Paria et al., 2018) that use different acquisition functions like expected improvement (Jones et al., 1998) and
Thompson sampling (Thompson, 1933).
EVHI is computationally expensive: its cost increases exponentially with the number of objectives.
To address this problem, qEHVI (Daulton et al., 2020) accelerates optimization by computing EHVI in parallel, and has become the state-of-the-art MOBO algorithm.
In this paper, we leverage qEHVI as a candidate inner solver in our proposed LaMOO algorithm.
Evolutionary algorithms (EAs) (Deb et al., 2002a; Igel et al., 2007a; Zhang & Li, 2007; Beume et al., 2007; Fang et al., 2018) are also popular methods for MOO tasks.
One category of MOOEAs (Srinivas & Deb, 1994; Deb et al., 2002a; Deb & Jain, 2014) leverages Pareto dominance to simultaneously optimize all objectives.
A second category (e.g., (Zhang & Li, 2007)) decomposes a multi-objective optimization problem into a number of single-objective sub-problems, converting a difﬁcult MOO into several SOOs.
Another category is quality indicator-based methods, such as (Beume et al., 2007) and (Igel et al., 2007a).
They scalarize the current Pareto frontier using quality indicators (e.g., HV) and transfer a MOO to a SOO.
New samples are generated by crossover and mutation operations from existing ones.
However the drawbacks of non-quality indicator-based methods (i.e., the ﬁrst two categories) can be not overlooked.
Speciﬁcally, for MOO with many objectives, NSGA-II (Deb et al., 2002a) easily gets stuck in a dominance resistant solution (Pang et al., 2020) which is far from the true Pareto frontier.
while MOEA/D perform better in many-objective problems but specifying the weight vector for problems with unknown Pareto front is the main obstacle (Deb & Jain, 2014).
In addition, A* search based algorithms are also considered to be extended to MOO (Stewart & White, 1991; Tung Tung & Lin Chew, 1992; De la Cruz et al., 2005).
Quality
Indicators .
Besides hypervolume, there are several other quality indicators (Van Veldhuizen & Lamont, 1998; Zitzler et al., 2000; Bosman & Thierens, 2003) for evaluating sample quality, which can be used to scalarize the MOO to SOO.
The performance of a quality indicator can be evaluated by three metrics (Deng et al., 2007; Li et al., 2014), including convergence (closeness to the Pareto frontier), uniformity (the extent of the samples satisfy the uniform distribution), and spread (the extent of the obtained approximate Pareto frontier).
Sec.
B speciﬁcally illustrates the merits of each quality indicator.
HyperVolume is the only metric we explored that can simultaneously satisfy the evaluation of convergence, uniformity, and spread without the knowledge of the true Pareto frontier while it may suffer from expensive calculation in many-objective problems.
Therefore, throughout this work, we use HV to evaluate the optimization performance of different algorithms.
L EARNING S PACE P ARTITIONS : A T HEORETICAL U NDERSTANDING
Searching in high-dimensional space to ﬁnd the optimal solution to a function is in general a challenging problem, especially when the function’s properties are unknown to the search algorithm.
The difﬁculty is mainly due to the curse of dimensionality: to adequately cover a d -dimensional space, in general, an exponential number of samples are needed.
For this, many works use a “coarse-to-ﬁne” approach: partition the search space and then focusing on promising regions.
Traditionally, manually deﬁned criteria are used, e.g., axis-aligned partitions (Munos, 2011b), Voronoi diagrams (Kim et al., 2020), etc.
Recently, (Wang et al., 2019; 2020; Yang et al., 2021) learn space partitions based on the data collected thus far, and show strong performance in NeurIPS black box optimization challenges (Sazanovich et al.
; Kim et al.).
On the other hand, there is little quantitative understanding of space partition.
In this paper, we ﬁrst give a formal theoretical analysis on why learning plays an important role in space-partition approaches for SOO.
Leveraging our understanding of how space partitioning works, we propose
LaMOO which empirically outperforms existing SoTA methods on multiple MOO benchmarks.
3. P ROBLEM S ETTING
Intuitively, learning space partitions will yield strong performance if the classiﬁer can determine which regions are promising given few data points.
We formalize this intuition below and show why it is better than ﬁxed and manually deﬁned criteria for space partitioning.
Consider the following sequential decision task.
We have N samples in a discrete subset S
and there exists one sample x ∗ that achieves a minimal value of a scalar function f .
Note that f can be any property we want, e.g., in the Pareto optimal set.
The goal is to construct a subset S T ⊆ S
after T steps, so that (1) x ∗ ∈ S T and (2) | S T | is as small as possible.
More formally, we deﬁne the reward function r as the probability that we get x ∗ by randomly sampling from the resulting subset S T :
[Equation]
It is clear that 0 ≤ r ≤ 1 .
r = 1 means that we already found the optimal sample x ∗ .
Here we use discrete case for simplicity and leave continuous case (i.e., partitioning a region Ωinstead of a discrete set S 0 ) to future work.
Note N could be large, so here we consider it infeasible to enumerate S
to ﬁnd x ∗ .
However, sampling from S 0 , as well as comparing the quality of sampled solutions are allowed.
An obvious baseline is to simply set S T :
= S 0 , then r b = N − 1 .
Now the question is: can we do better?
Here we seek help from the following oracle :
Deﬁnition 1 ( ( α, η ) -Oracle) .
Given a subset S that contains x ∗ , after taking k samples from S , the oracle can ﬁnd a good subset S good with | S good | ≤ | S | / 2 and
[Equation]
Lemma 1.
The algorithm to uniformly draw k samples in S , pick the best and return is a (1 , 1) -oracle.
See Appendix for proof.
Note that a (1 , 1) -oracle is very weak, and is of little use in obtaining higher reward r .
We typically hope for an oracle with smaller α and η (i.e., both smaller than 1).
Intuitively, such oracles are more sample-efﬁcient: with few samples, they can narrow down the region containing the optimal solution x ∗ with high probability.
Note that α < 1 corresponds to semi-parametric models .
In these cases, the oracle has generalization property : with substantially fewer samples than N (i.e., on the order of N α ), the oracle is able to put the optimal solution x ∗ on the right side.
In its extreme case when α = 0 (or parametric models ), whether we classify the optimal solution x ∗ on the correct side only depends on the absolute number of samples collected in S , and is independent of its size.
For example, if the function to be optimized is linear, then with d + 1 samples, we can completely characterize the property of all | S | samples.
Relation with cutting plane.
Our setting can be regarded as a data-driven extension of cutting plane methods (Loganathan & Sherali, 1987; Vieira & Lisboa, 2019; Hinder, 2018) in optimization, in which a cutting plane is found at the current solution to reduce the search space.
For example, if f is convex and its gradient ∇ f ( x ) is available, then we can set S good :=
{ x : ∇ f ( x 0 ) (cid:62) ( x − x 0 ) ≤ 0 , x ∈ S 0 } , since for any x ∈ S 0 \ S good , convexity gives f ( x ) ≥ f ( x 0 ) + ∇ f ( x 0 ) (cid:62) ( x − x 0 ) >
f ( x 0 ) and thus x is not better than current x 0 .
However, the cutting plane method relies on certain function properties like convexity.
In contrast, learning space partition can leverage knowledge about the function forms, combined with observed samples so far, to better partition the space.
3. R EWARDS UNDER OPTIMAL ACTION SEQUENCE
We now consider applying the ( α, η ) -oracle iteratively for T steps, by drawing k t samples from
S t − 1 and setting S t := S good ,t − 1 .
We assume a total sample budget K , so (cid:80) Tt =1 k t = K .
Note that
T ≤ log 2 N since we halve the set size with each iteration.
Now the question is twofold.
(1) How can we determine the action sequences { k t } in order to maximize the total reward r ?
Following the optimal action sequences { k ∗ t } , can r ∗ be better than the baseline r b = N − 1 ?
The answer is yes.
Theorem 1.
The algorithm yields a reward r ∗ lower bounded by the following:
[Equation]
where r b := N − 1 and φ ( α, T ) :
= (1 − 2 − αT ) / (1 − 2 − α ) .
Remarks.
Following Theorem 1, a key condition to make r ∗ > r b is to ensure log 2 > ηN α φ ( α,T ) K .
This holds if when ηN α φ ( α,T ) K → 0 .
Note that since T ≤ log 2 N , the ﬁnal reward r ∗ is upper bounded by 1 (rather than goes to + ∞ ).
We consider some common practical scenarios below.
Non-parametric models ( α = 1 ).
In this case, φ ( α, T ) ≤ 2 and the condition becomes 12 log 2 > ηN/K .
This happens when the total sample budget K = Θ( N ) , i.e., on the same order of N , which means that the partitioning algorithm obtains little advantage over exhaustive search.
Semi-parametric models ( α < 1 ).
In this case, φ ( α, T ) ≤ 1 / (1 − 2 − α ) and the condition becomes (1 − 2 − α ) log 2 > ηN α /K .
This happens when the total sample budget K
= Θ( N α ) .
In this case, we could use many fewer samples than exhaustive search to achieve better reward, thanks to the generalization property of the oracle.
Parametric models ( α = 0 ).
Now φ ( α, T ) =
T and the condition becomes log 2 > ηTK .
Since
T ≤ log 2 N , the total sample budget can be set to be K = Θ(log N ) .
Intuitively, the algorithm performs iterative halving (or binary search) to narrow down the search toward promising regions.
3. E XTENSION TO M ULTI -O BJECTIVE O PTIMIZATION
Given our understanding of space partitioning, we now extend this idea to MOO.
Intuitively, we want “good” regions to be always picked by the space partition.
For SOO, it is possible since the optimal solution is a single point.
How about MOO?
Unlike SOO, in MOO we aim for a continuous region, the Pareto optimal set Ω P :=
{ x : (cid:64) x (cid:48) (cid:54) = x : f ( x (cid:48) )
≺ f ( x ) } .
A key variable is the regularity of Ω P : if it is highly non-regular and not captured by a simple partition boundary (ideally a parametric boundary), then learning a space partition would be difﬁcult.
Interestingly, the shape of Ω P can be characterized for quadratic objectives:
Observation 1.
If all f j are isotropic, f j ( x ) = (cid:107) x − c j (cid:107) 22 , then Ω P = ConvexHull( c 1 , . .
c q ) .
Observation 2.
If M = 2 and f j ( x ) =
( x − c j ) (cid:62) H j ( x − c j ) where H j are positive deﬁnite symmetric matrices, then there exists w 1 :
=
H 2 ( c 2 − c 1 ) and w 2 :
= H 1 ( c 1 − c 2 ) , so that for any x ∈ Ω P , w (cid:62) 1 ( x − c 1 ) ≥ 0 and w (cid:62) 2 ( x − c 2 ) ≥ 0 .
In both cases, Ω P can be separated from non-Pareto regions Ω \ Ω P via a linear hyperplane.
Empirically, Ω P only occupies a small region of the entire search space (Sec. 4), and quickly focusing samples on the promising regions is critical for high sample efﬁciency.
In the general case, characterizing Ω P is analytically hard and requires domain knowledge about the objectives (Li et al., 2014).
However, for MOO algorithms in practice, knowing that Ω P can be separated from Ω \ Ω P via simple decision planes is already useful: we could learn such decision planes given previous data that are already collected, and sample further in promising regions.
L A MOO: L ATENT A CTION M ULTI - OBJECTIVE O PTIMIZATION
In Sec. 3, for convenience, we only analyze a greedy approach, which makes decisions on space partitions and never revises them afterwards.
While this greedy approach indeed works (as shown in
Sec. 5.3), an early incorrect partition could easily rule out regions that turn out to be good but weren’t identiﬁed with few samples.
In practice, we want to keep the decision softer: while exploiting the promising region, we also explore regions that are currently believed to be sub-optimal given limited samples.
It is possible that these regions turn out to contain good solutions when more samples are available, and the oracle can then make a different partition.
To balance the trade-off between exploration and exploitation to cope with the generalization error of the learned classiﬁer, we leverage Monte Carlo Tree Search (MCTS) (Kocsis & Szepesvári, 2006) and propose our algorithm LaMOO.
As shown in Alg. 1, LaMOO has four steps: (1) learn to partition the search space given previous observed data points D t , which are collected { x i , f( x i )} from iterations 0 to t .
(2) With this information, we partition the region into promising and non-promising regions, and learn a classiﬁer h ( · ) to separate them.
(3) We select the region to sample from, based on the
UCB value of each node.
(4) We sample selected regions to obtain future data points D t +1 .
Learning Space Partitions .
We construct the partition oracle using the dominance number .
Let D t be the collected samples up to iteration t and D t,j :=
D t ∩ Ω j be the samples within the region
Ω j we want to partition.
For each sample x ∈ D t,j , its dominance number o t,j ( x ) at iteration t is deﬁned as the number of samples in Ω j that dominate x (here I [ · ] is the indicator function):
[Equation]
While naive computation requires O ( | D t,j | 2 ) operations, we use Maxima Set (Kung et al., 1975) which runs in O ( | D t,j | log | D t,j | ) .
For x ∈ Ω P , o ( x ) = 0 .
For each D t,j , we then get good (small o ( x ) ) and bad (large o ( x ) )
samples by ranking them according to o ( x ) .
The smallest 50% are labeled to be positive while others are negative.
Based on the labeled samples, a classiﬁer (e.g., Support Vector Machine (SVM)) is trained to learn a decision boundary as the latent action.
We choose SVM since the classiﬁer needs to be decent in regions with few samples, and has the ﬂexibility of being parametric or non-parametric.
A
B
C
Learn to partitionh(·)
Expand the tree 𝛀 𝐃 𝛀 𝐂
SearchSpace 𝛀 𝐄
D
E
A
B
C
D
E 𝑈𝐶𝐵 ( = 𝑈𝐶𝐵 ) =
𝑈𝐶𝐵 * =7 𝑈𝐶𝐵 + =
A
B
C
D
E 𝛀 𝐃 𝛀 𝐂 𝛀 𝐄
Only samples from 𝛀 𝐄 basedon samplemethods (a)Split (b)Select (c)Sample
Select w.r.tucb
𝓟 (o (𝒙)=0)
[Figure] [Figure]
)=
[Figure] [Figure]
[Figure] [Figure]
[Figure]
Exploration Using Upper Conﬁdence Bounds(UCB) .
As shown in Fig.
2, LaMOO selects the ﬁnal leaf node by always choosing the child node with larger UCB value.
The UCB value for a node j is deﬁned as UCB j :=
v j + 2 C p (cid:112) 2 log n parent(j) /n
j , where n j is the number of samples in node j ,
C p is a tunable hyperparameter which controls the degree of exploration, and v j is the hypervolume of the samples in node j .
The selected leaf corresponds the partitioned region Ω k as shown in Alg. 1.
Sampling in Search Region .
We use existing algorithms as a sampling strategy in a leaf node, e.g., qEHVI (Daulton et al., 2020)) and CMA-ES (Igel et al., 2007a).
Therefore, LaMOO can be regarded as a meta-algorithm , applicable to any existing SOO/MOO solver to boost its performance.
LaMOO with qEHVI.
As a multi-objective solver, qEHVI ﬁnds data points to maximize a parallel version of Expected Hypervolume Improvement (EHVI) via Bayesian Optimization (BO).
To incorporate qEHVI into LaMOO’s sampling step, we conﬁne qEHVI’s search space using the tree-structured partition to better search MOO solutions.
LaMOO with CMA-ES.
CMA-ES is an evolutionary algorithm (EA) originally designed for singleobjective optimization.
As a leaf sampler, CMA-ES is used to pick a sample that maximizes the dominance number o ( x ) within the leaf.
Since o ( x ) changes over iterations, at iteration t , we ﬁrst update o t (cid:48)
( x ) of all previous samples at t (cid:48) < t to o t ( x ) , then use CMA-ES.
Similar to the qEHVI case, we constrain our search to be within the leaf region.
Once a set of new samples D new is obtained (as well as its multiple function values f ( D new ) ), we update all partitions along its path and the entire procedure is repeated.
E XPERIMENTS
We evaluate the performance of LaMOO in a diverse set of scenarios.
This includes synthetic functions, and several real-world MOO problems like neural architecture search, automobile safety design, and molecule discovery.
In such real problems, often a bunch of criteria needs to be optimized at the same time.
For example, for molecule (drug) discovery, one wants the designed drug to be effective towards the target disease, able to be easily synthesized, and be non-toxic to human body.
5. S MALL - SCALE P ROBLEMS
Synthetic Functions.
Branin-Currin (Belakaria et al., 2019) is a function with 2-dimensional input and 2 objectives.
DTLZ2 (Deb et al., 2002b) is a classical scalable multi-objective problem and is popularly used as a benchmark in the MOO community.
We evaluate LaMOO as well as baselines in
DTLZ2 with 18 dimensions and 2 objectives, and 12 dimensions and 10 objectives, respectively.
Structural Optimization in Automobile Safety Design (vehicle safety) is a real-world problem with 5-dimensional input and 3 objectives, including (1) the mass of the vehicle, (2) the collision acceleration in a full-frontal crash, and (3) the toe-board intrusion (Liao et al., 2008).
Nasbench201 is a public benchmark to evaluate NAS algorithms (Dong & Yang, 2020).
There are 15625 architectures in Nasbench201, with groundtruth #FLOPs and accuracy in CIFAR10 (Krizhevsky, 2009).
Our goal is to minimize #FLOPs and maximize accuracy in this search space.
We normalized #FLOPs to range [ − 1 , 0] and accuracy to [0 , 1] .
We compare LaMOO with 4 classical evolutionary algorithms (CMA-ES (Igel et al., 2007a),
MOEA/D (Zhang & Li, 2007), NSGA-II (Deb et al., 2002a), and NSGA-III (Deb & Jain, 2014)) and 2 state-of-the-art BO methods (qEHVI (Daulton et al., 2020) and qParego (Knowles, 2006)).
Evaluation Criterion .
we ﬁrst obtain the maximal hypervolume (either by ground truth or from the estimation of massive sampling), then run each algorithm and compute the log hypervolume difference (Daulton et al., 2020):
[Equation]
(6) where HV cur is the hypervolume of current samples obtained by the algorithm with given budget.
Result.
As shown in Fig.
3, LaMOO with qEHVI outperforms all our BO baselines and LaMOO with CMA-ES outperforms all our EA baselines, in terms of HV log _ diﬀ .
Evolutionary algorithms rely on mutation and crossover of previous samples to generate new ones, and may be trapped into local optima.
Thanks to MCTS, LaMOO also considers exploration and greatly improves upon vanilla CMA-ES over three different tasks with 1000 samples.
In addition, by plugging in BO, LaMOO+qEHVI achieves 225% sample efﬁciency compared to other BO algorithms on Nasbench201.
This result indicates that for high-dimensional problems (6 in Nasbench201), space
[Figure]
partitioning leads to faster optimization.
We further analyze very high-dimensional problems on
Sec.
For visualization of Pareto frontier by LaMOO+qEHVI,
Fig.
12 in Appendix.
Optimization of Many Objectives .
While NSGA-II and NSGA-III perform well in the two-objective problems, all evolutionary-based baselines get stuck in the ten-objective problems.
In contrast,
LaMOO performs reasonably well.
From Fig.
4, qEHVI+LaMOO shows strong performance in ten objectives.
When combined with a CMA-ES, LaMOO helps it escape the initial region to focus on a smaller promising region by space partitioning.
5. M ULTI -O BJECTIVE M OLECULE D ISCOVERY
[Figure]
Next
[Figure]
we tackle the practical problem of multi-objective molecular generation, which is a highdimensional problem (search space is 32-dimensional).
Molecular generation models are a critical component of pharmaceutical drug discovery
[Figure]
wherein a cheap-to-run in silico model proposes promising molecules which can then be synthesized and tested in a lab (Vamathevan et al., 2019).
However
[Figure]
one commonly requires the generated molecule to satisfy multiple constraints: for example, new drugs should generally be non-toxic and ideally easy-to-synthesize, in addition to their primary purpose.
Therefore, in this work, we consider several multi-objective molecular generation setups from prior work on molecular generation (Yu et al., 2019;
Jin et al., 2020b; Yang et al., 2021): (1) activity against biological targets GSK3 β and JNK3, (2) the same targets together with QED (a standard measure of “drug-likeness”) and SA (a standard measure of synthetic accessibility), and (3) activity against SARS together with QED and SA.
In each task, we propose samples from a pretrained 32-dimensional latent space from (Jin et al., 2020a), which are then decoded into molecular strings and fed into the property evaluators from prior work.
Fig.
5 shows that LaMOO+qEHVI outperforms all baselines by up to 10% on various combinations of objectives.
While EA struggles to optimize these high-dimensional problems due to the limitations mentioned in Sec. 2, LaMOO helps them (e.g., CMA-ES) to perform much better.
5. A BLATION STUDIES
Visualization of LaMOO.
To understand how LaMOO works, we visualize its optimization procedure for Branin-Currin.
First, the Pareto optimal set Ω P is estimated from 10 6 random samples (marked as black stars), as shown in both search and objective space (Fig. 6(b) and bottom left of
Fig. 6(c)).
Over several iterations, LaMOO progressively prunes away unpromising regions so that the remaining regions approach Ω P (Fig 6(c)).
Fig 6(a) shows the ﬁnal tree structure.
The color of each leaf node corresponds to a region in the search space (Fig 6(b)).
The selected region is recursively
A
B
C
D
H
I
E
F
G
J
K
[Figure]
bounded by SVM classiﬁers corresponding to nodes on the selected path (red arrow in Fig 6(a)).
The new samples are only generated from the most promising region Ω J , improving sample efﬁciency
[Figure]
Ablation of Design Choices.
We show how different hyperparameters and sampling methods play a role in the performance.
We perform the study in VehicleSafety below.
Sampling methods .
LaMOO can be integrated with different sample methods, including Bayesian
Optimization (e.g., qEHVI) and evolutionary algorithms (e.g., CMA-ES).
Fig.
7(left) shows that compared to random sampling, qEHVI improves a lot while CMA-ES only improves slightly.
This is consistent with our previous ﬁnding that for MOO, BO is much more efﬁcient than EA.
The exploration factor C p controls the balance of exploration and exploitation.
A larger C p guides
LaMOO to visit the sub-optimal regions more often.
Based on the results in Fig.
7(middle), greedy search ( C p = 0 ) leads to worse performance compared to a proper C p value (i.e. 10% of maximum hypervolume), which justiﬁes our usage of MCTS.
On the other hand, over-exploration can also yield even worse results than greedy search.
Therefore, a "rule of thumb" is to set the C p to be roughly 10% of the maximum hypervolume HV max .
When HV max is unknown, C p can be set empirically.
SVM kernels .
As shown in Fig.
7(right), we ﬁnd that the RBF kernel performs the best, in agreement with (Wang et al., 2020).
Thanks to the non-linearity of the polynomial and RBF kernels, their region partitions perform better compared to a linear one.
C ONCLUSION
We propose a search space partition optimizer called LaMOO as a meta-algorithm that extends prior single-objective works (Wang et al., 2020; Yang et al., 2021) to multi-objective optimization.
We demonstrated both theoretically and via experiments on multiple MOO tasks that LaMOO signiﬁcantly improves the search performance compared to strong baselines like qEHVI and CMAES.
In particular, LaMOO + qEHVI achieves state-of-the-art results on a variety of tasks including synthetic functions, neural architecture search, and molecule discovery.
R EPRODUCIBILITY S TATEMENT
Anonymous code is available at https://drive.google.com/drive/folders/ 1CMdg5iBdbKe3nkboIjiS998rnBEV09EB?usp=sharing .
A full theoretical proof of
LaMOO can be found in Appendix A.
A P ROOFS
Lemma 1.
The algorithm to uniformly draw k samples in S , pick the best and return is a (1 , 1) -oracle.
Proof.
Consider the following simple (1 , 1) -oracle for single-objective optimization: after sampling k samples, we rank them according to their function values, and split them into two k/ 2 smaller subsets ˜ S
good and ˜ S bad .
Other points are randomly assigned to either of the two subsets.
Then if x ∗ happens to be among the k collected samples (which happens with probability k/ | S | ), deﬁnitely we have x ∗ ∈ S good .
Therefore, we have:
[Equation]
which is an oracle with α = η = 1 .
The last inequality is due to e x ≥ 1 + x (and thus e − x ≥ 1 − x ).
Lemma 2.
Deﬁne g ( λ ) : R + (cid:55)→ R + as:
[Equation]
The following maximization problem
[Equation]
has optimal solutions
[Equation]
where λ is determined by g ( λ )
=
K .
With optimal { z ∗ t } , the objective reaches − (cid:80) t log(1 + λw t ) .
Proof.
Its Lagrange is:
[Equation]
Taking derivative w.r.t.
z t and we have:
[Equation]
Lemma 3.
Both g ( λ ) and g − 1 ( y ) are monotonously decreasing.
Furthermore, let ¯ w :=
T (cid:16)(cid:80)
Tt =1 w − 1 t (cid:17) − 1 be the Harmonic mean of { w t } and w max :=
max Tt =1 w t , we have:
[Equation]
[Figure]
Proof.
It is easy to see when λ increases, each term in g ( λ ) decreases and thus g ( λ ) is a decreasing function of λ .
Therefore, its inverse mapping g − 1 ( y ) is also decreasing.
Let µ ( y ) :
= 1 /g − 1 ( y ) > 0 .
Then we have:
[Equation]
It is clear that when y = 0 , µ ( y ) = 0 .
Taking derivative with respect to y in both side, we have:
[Equation]
15) where µ (cid:48)
( y ) =
d µ ( y ) d y is the derivative of µ ( y ) .
Using the property of Harmonic mean, we have:
[Equation]
This gives:
[Equation] [Equation]
Integrate on both side starting from y = 0 , we have:
[Equation] [Equation]
Using µ (0)
=
we thus have:
[Equation] [Equation]
This leads to µ ( y ) ≤ ¯ w (cid:2) exp
( y ¯ w − 1 /T )
− 1 (cid:3) .
With g − 1 ( y ) =
1 /µ ( y ) , we arrive the ﬁnal lower bound for g − 1 ( y ) .
For an alternative upper bound of g − 1 ( y ) , we just notice that (here w max :=
max t w t ):
[Equation] [Equation]
Using the same technique as above, we have µ ( y ) ≥ w max (cid:2) exp( yw − 1max /T ) − 1 (cid:3) and the upper bound of g − 1 ( y ) follows.
Finally, note that e x ≥ 1 + x , we have
[Equation]
[Equation]
Theorem 1.
Following optimal sequence, the algorithm yields a reward r ∗ lower bounded by the following:
[Equation]
where r b := N − 1 and φ ( α, T ) :
= (1 − 2 − αT ) / (1 − 2 − α ) .
Proof.
First note that | S T | ≤ | S 0 | / 2 T and thus 1 | S T | ≥ 2 T /N .
So we just need to bound P ( x ∗ ∈ S T ) , which can be written as:
[Equation]
Therefore we have
[Equation]
We want to ﬁnd the action sequence { k t } so that log P ( x ∗ ∈ S T ) is maximized.
Let w t := η | S t − 1 | α and z t :=
k t /w t , applying Lemma 2, and we know that
[Equation]
where the Lagrangian multiplier λ satisﬁes the equation g ( λ ) = K .
Now we have:
[Equation]
Here 1 (cid:13) is due to Lemma 3 which tells that λ
= g − 1 ( K ) ≤ T/K , 2 (cid:13) is due to w t :
= η | S t − 1 | α and | S t − 1 | ≤ N/ 2 t − 1 , and 3 (cid:13) due to log(1 + x ) ≤ x .
Putting all of them together, we know that
[Equation]
Optimal action sequence { k ∗ t } .
From the proof, we could also write down the optimal action sequence that achieves the best reward: k ∗ t = w t log (cid:16) 1 + 1 λw t (cid:17) , where w t :=
η | S t − 1 | α .
Using
Lemma 3, we could compute the upper and lower bound estimation of λ = g − 1 ( K ) .
Here ¯ w :=
T (cid:16)(cid:80) T t =1 w − 1 t (cid:17) − 1 be the Harmonic mean of { w t } and w max :=
max T t =1 w t :
[Equation]
With λ , we could compute approximate { k ∗ t } .
Here we make a rough estimation of { k ∗ t } if we terminate the algorithm when | S T | is still fairly large.
This case corresponds to the setting T = β log 2 N where β < 1 and all w t ∼ N α .
With K = Θ( N α ) as in semi-parametric case, ¯ w − 1 K = Θ(1) , exp( ¯ w − 1 K/T ) − 1 ≈ ¯ w − 1 K/T and λw t ∼ log 2 N (cid:29) 1 .
Since log(1 + x ) ≈ x for small x , we have k ∗ t ≈ w t 1 λw t = 1 /λ , which is independent of t .
Therefore, a constant amount of sampling at each stage is approximately optimal.
Observation 1.
If all f j are isotropic, f j ( x ) = (cid:107) x − c j (cid:107) 22 , then Ω P = ConvexHull( c 1 , . .
. , c M ) .
Proof.
Consider J ( x ; µ ) :
= (cid:80) Mj =
1 µ j f j ( x ) where the weights µ j ≥ 0 satisﬁes (cid:80) j µ j = 1 .
For brevity, we write the constraint as ∆ :=
{ µ : µ j ≥ 0 , (cid:80) j µ j = 1 } .
Now consider the Pareto Set Ω P :=
{ x : ∃ µ ∈ ∆ : ∇ x J ( x ; µ ) = 0 } .
We have the following:
[Equation] [Equation]
[Equation] [Equation]
[Equation]
The last step is due to the fact that (cid:80) j µ j = 1 .
Therefore, for any x ∈ Ω P , x is a convex combination of { c 1 , . .
. , c M } and thus x ∈ ConvexHull( c 1 , . .
. , c M ) .
Conversely, for any x ∈
ConvexHull( c 1 , . . . , c M ) , we know ∇ x J ( x ; µ ) = 0 and thus x ∈ Ω P .
Observation 2.
If M = 2 and f j ( x ) =
( x − c j ) (cid:62) H j ( x − c j ) where H j are positive deﬁnite symmetric matrices, then there exists w 1 :
=
H 2 ( c 2 − c 1 ) and w 2 :
= H 1 ( c 1 − c 2 ) , so that for any x ∈ Ω P , w (cid:62) 1 ( x − c 1 ) ≥ 0 and w (cid:62) 2 ( x − c 2 ) ≥ 0 .
Proof.
Following Observation 1, similarly we have for all x ∈ Ω P , (cid:80)
j µ j H j ( x − c j ) = 0 , which gives:
[Equation]
Note that this is an expression of the Pareto Set Ω P .
Let A j :=
( (cid:80) j µ j H j ) − 1 µ j H j .
Then (cid:80)
j
A j = I .
Note that while (cid:80) j µ j H j and ( (cid:80) j µ j H j ) − 1 are positive deﬁnite matrix.
A j may not be.
Let M :=
(cid:80) i µ i H i .
Since µ ∈ ∆ , M is a PD matrix.
Note that we have
[Equation]
Using Eqn.
36, we know that x
= M − 1 (cid:80) j µ j H j
[Equation]
−
[Equation]
j − c k ) .
For M = 2 , we have x = c 2 + M − 1 µ 1 H 1 ( c 1 − c 2 ) .
So we have
[Equation]
This is because ( c 1 − c 2 )
H 1 M − 1 H 1 ( c 1 − c 2 ) ≥ 0 since H 1 M − 1 H 1 is a PSD matrix.
Therefore, let w 2 :
=
H 1 ( c 1 − c 2 ) and we have w (cid:62) 2 ( x − c 2 ) ≥ 0 , which is independent of µ ∈ ∆ .
This means it holds for any x ∈ Ω P .
Let w
=
H 2 ( c 2 − c 1 ) , then similarly we have w (cid:62) 1 ( x − c 1 ) ≥ 0 for all x ∈ Ω P .
B Q UALITY I NDICATORS C OMPARISON
[Table]
Generational Distance(GD)
(Van Veldhuizen & Lamont, 1998) measures the distance the pareto frontier of approximation samples and true pareto frontier, which requires prior knowledge of true pareto frontier and only convergence is considered.
IGD (Bosman & Thierens, 2003) is improved version of GD.
IGD calculates the distance the points on true pareto frontier to the closest point on pareto frontier of current samples.
Inverted Generational Distance(IGD) satisﬁes all three evaluation metrics of QI but requires true pareto frontier which is hardly to get in real-world problem.
Maximum
Spread(MS)
(Zitzler et al., 2000) computes the distance between the farthest two points of samples to evaluate the spread.
Spacing(S)
(Bandyopadhyay et al., 2004) measures how close the distribution of pareto frontier of samples is to uniform distribution.
Overall Non-dominated Vector Generation and
Ratio(ONVGR) is the ratio of number of samples in true pareto-frontier.
The table 1 demonstrates the good characteristics of each quality indicators.
C E ND - TO - END L A MOO P SEUDOCODE
Below we list the pseudocode for the end-to-end workﬂow of LaMOO in Algorithm 2.
Specﬁcally, it includes search space partition in Function Split .
Node(promising region) selection in Function
Select , and new samples generation in Function Sample .
Algorithm 1: Inputs: Initial
uniform sampling, sample [Table] : for =
do [Table] Set root
of to 4: V , v, n ← Split ( L , D t ) 5: k ← Select ( C p , D t )
[Table]
D
) 7: end for 8:9: Function Split ( t ) 10: while L (cid:54) do 11: Ω j ← pop ﬁrst element( ) , D t,j ← D t ∩ Ω j , n t,j ← | D t,j | .
Compute dominance number o t,j of D t,j using Eqn.
5 and train SVM model h ( · ) . If ( D t,j , o t,j ) is splittable by SVM, then L ← L ∪ Partition(Ω j , h ( · )) .
14: end while 15:16: Function Select ( C p , D t ) 17: for k k is do 18:
D t,k ← D
∩ Ω k , v t,k ← HyperVolume( D t k ) , n t,k ← | D t,
| .
19: k arg max c
UCB t,c := v t,c
C n t,c 20: end
21: return k 22:23: Function Sample ( k ) 24: is drawn qEHVI or CMA
25: return
D E XPLORATION FACTOR ( C p ) SETUP WITH UNKNOWN MAXIMUM
H YPERVOLUME
[Figure]
As we mentioned in the paper, a "rule of thumb" is to set the C p to be roughly 10% of the maximum hypervolume HVmax.
If HVmax is unknown, C p can be dynamically set to 10% of the hypervolume of current samples in each search iteration.
The ﬁgures below demonstrate the difference between 10% HVmax and 10% current hypervolume in three problems(Branin-Currin, VehicleSafety, and
Nasbench201 from left to right).
The ﬁnal performances by using 10% HVmax and 10% current hypervolume are similar.
E W ALL CLOCK TIME IN DIFFERENT PROBLEMS
[Figure]
Fig.
10 shows the wall clock time of different search algorithms in BraninCurrin(Belakaria et al., 2019), VehicleSafefy (Liao et al., 2008) and Nasbench201 (Dong & Yang, 2020).
F D ETAILS OF B ENCHMARK P ROBLEMS F. P ROBLEM DESCRIPTION
BraninCurrin (Belakaria et al., 2019) : f (1) ( x 1 , x 2 ) =
(15 x 2 − 5 .
1 ∗ (15 x 1 − 5) 2 4 π+ 75 x 1 −π − 5)
+ (10 − 10 8 π )) ∗ cos(15 x 1 − 5))
f (2) ( x 1 , x 2 ) =
(cid:20) 1 − exp (cid:18) − (2 x 2 ) (cid:19)(cid:21) 2300 x 31 + 1900 x 21 + 2092 x 1 +100 x 31 + 500 x 21 + 4 x 1 + where x 1 , x 2 ∈
VehicleSafefy (Liao et al., 2008) : f 1 ( x ) = 1640 .
2823 + 2 .
3573285 x 1 + 2 .
3220035 x 2 + 4 .
5688768 x 3 + 7 .
7213633 x 4 + 4 .
4559504 xf 2 ( x ) = 6 .
5856 + 1 .
15 x 1 − 1 .
0427 x 2 + 0 .
9738 x 3 + 0 .
8364 x 4 − 0 .
3695 x 1 x 4 + 0 .
0861 x 1 x+ 0 .
3628 x 2 x 4 + 0 .
1106 x 21 − 0 .
3437 x 23 + 0 .
1764 xf 3 ( x ) = − 0 .
0551 + 0 .
0181 x 1 + 0 .
1024 x 2 + 0 .
0421 x 3 − 0 .
0073 x 1 x 2 + 0 .
024 x 2 x 3 − 0 .
0118 x 2 x− 0 .
0204 x 3 x 4 − 0 .
008 x 3 x 5 − 0 .
0241 x 22 + 0 .
xwhere x ∈ [1 , 3] 5 .
Nasbench201 (Dong & Yang, 2020) :
[Figure]
In Nasbench201, the architectures are made by stacking the cells together.
The difference among architectures in Nasbench201 is the design of the cells, see ﬁg 11.
Speciﬁcally, each cell containsnodes, and there is a particular operation connecting to two nodes including zeroize, skip-connect, 1x1 convolution, 3x3 convolution, and 3x3 average pooling.
Therefore, there are C 24 = 6 edges in a cell and
5 6 =15625 unique architectures in Nasbench201.
According to this background, Each architecture can be encoded into a 6-dimensional vector with 5 discrete numbers (i.e., 0, 1, 2, 3, 4 that corresponds to zeroize, skip-connect, 1x1 convolution, 3x3 convolution, and 3x3 average pooling).
[Equation]
where x ∈ { 0 , 1 , 2 , 3 , 4 } 6 .
DTLZ2 (Deb et al., 2002b) :
[Equation]
where g ( x ) =
(cid:80) x i ∈ x M ( x i − 0 . 5) 2 , x ∈ [0 , 1] d , and x M represents the last d − M + 1 elements of x .
F. V ISUALIZATION OF P ARETO -F RONTIER FOR B ENCHMARK P ROBLEMS
[Figure]
F. R EFERENCE P OINTS
† : M represents the number of objectives.
Problem
Reference Point
BraninCurrin (18.0, 6.0)
VehicleSafety (
[Table] [Table]
[Table]
Nasbench (-3.0, -6.0)
DTLZ (1.1, .., 1.1) ∈
[Table]
Molucule Discovery
[Table]
Table 2 elaborates the reference points in the problems throughout the paper.
F. M AXIMUM H YPERVOLUME OF EACH PROBLEM
Problem
Maximum
[Table]
BraninCurrin 59.
VehicleSafety 246.
Nasbench 8.
DTLZ2(2 objectives) 1.
DTLZ2(10 objectives) 2.
[Table]
Table 3 elaborates the observed maximal hypervolume in the problems throughout the paper.
We used these value to calculate the log hypervolume difference in ﬁg 3 and ﬁg 4.
G E XPERIMENTS S ETUP
Experiment details:
For small-scale problems(i.e.
Branin-Currin, VehicleSafety, and Nasbench201) and DTLZ2 with 2 and 10 objectives.
We randomly generate 10 samples as the initialization.
For multi-objective molecule discovery, the number of initial samples is 150.
In each iteration, we update 5 batched samples(q value) for all search algorithms.
Hyperparameters of LAMOO:
For all problems, we leverage polynomials as the kernel type of SVM and the degree of the polynomial kernel function is set to 4.
The minimum samples in the leaf of MCTS is 10.
The cp is roughly set to 10% of maximum of hypervolume(i.e.
Branin-Currin -> 5, VehicleSafety -> 20, Nasbench201 -> 6, DTLZ2(2 objectives) -> 0.1, DTLZ2(10 objectives) -> 0.25, molecule discovery(2 objectives) -> 0.03, molecule discovery(3 objectives) -> 0.2, molecule discovery(4 objectives) -> 0.06).
Hyperparameters of qEHVI and qParEGO:
The number of q is set to 5.
The acquisition function is optimized with L-BFGS-B (with a maximum of 200 iterations).
In each iteration, 256 raw samples used for the initialization heuristic are generated to be selected by the acquisition function.
As the same claim in Daulton et al.
(2020), each generated sample is modeled with an independent Gaussian process with a Matern 5/2 ARD kernel.
H V ERIFICATION OF LAMOO ON M ANY - OBJECTIVE P ROBLEMS
[Figure]
While it is theoretically hard to label the samples into good and bad based on their dominance number in many-objective problems due the the lack of dominance pressure(All samples are non-dominated with each other).
If number of objective is not too large(i.e.
≤ 10 ), the samples can be still split by dominance number.
Given the problem(DTLZ2 with 10 objectives) shown in ﬁg 4, we randomly generate 50 samples in the search space and draw the dominance distribution of them(see ﬁg 13).
We did this experiment 5 times.
We then partition the search space by a SVM classiﬁer based on the labeled samples into “good” and “bad”, and randomly generate 50 samples in “good region”, “bad region”, and whole space, respectively.
We did this process 5 times with different initial samples.
Fig.
14 shows the range of hypervolume of the samples generated from good regions, the whole space, and bad regions.
From the ﬁgure, we can see that the hypervolume of samples generated from good regions are signiﬁcantly higher than others.
I C OMPUTATIONAL COMPLEXITY ANALYSIS OF L A MOO
Here is a detailed breakdown of the computational complexity of our algorithm (Alg. 1)
Line.6: Compute dominance: O ( MN node 2 ) where N node is the number of samples in the node and
M is the number of dimensions.
Line 7: Time complexity of SVM : O ( N 2 ) where N node is number of searched samples in total. Line 10:
Hypervolume: O ( N M 2 + N log N ) ( M > 3 ) (Beume & Rudolph, 2006) or O ( N log N ) ( M ≤ 3 ) (Beume et al., 2009), where N is number of searched samples in total and D is the number of dimensions.
HV computation is to be the dominating factor in case of M > 3 , otherwise, it should be the computation of SVM.
J V ARIATION OF L A MOO WITH A CHEAPER OVERHEAD
k ← D t ∩ Ω k , n t,k ← | D t,k | .
11: end for 12: for l is leaf node do 13: v t,l ← HyperVolume( D t,l ) 14: end for 15: k ← arg max l ∈ leaf nodes UCB t,l , where UCB t,l := v t,l + 2 C p (cid:113) 2log( n t,l ) n t,p , where p is the parent of l .
D t +1 ← D t ∪ D new , where D new is drawn from Ω k based on qEHVI or CMA-ES.
17: end for
[Figure] [Figure]
Instead of traversing down the search tree to trace the current most promising search path, this variation of LaMOO directly select the leaf node with the highest UCB value .
Algorithm.
3 illustrates the detail of this variation.
Therefore, this variation avoids calculating the hypervolume in the non-leaf nodes of the tree, where hypervolume calculation is the main computational cost of LaMOO especially in many-objective problems.
Figure.
15 and ﬁgure.
16 validate the variation that is able to reach similar performance of searched samples but saves lots of time.
We leave the validation of others problems in the future works.
K ADDITIONAL RELATED WORKS
(Hashimoto et al., 2018a; Kumar et al., 2018; Hashimoto et al., 2018b) indeed leverage classiﬁers to partition search space and draw the new samples from the good region.
However, (Hashimoto et al., 2018a; Kumar et al., 2018; Hashimoto et al., 2018b) randomly sampled in selected regions without integrating existing optimizers (e.g. Bayesian optimization, evolutionary algorithms).
In addition, they progressively select the good regions without a trade-off of exploration and exploitation as we did by leveraging Monte Carlo Tree Search (MCTS).
(Munos, 2011a; Wang et al., 2014; Kawaguchi et al., 2015) can be seen as the ﬁrst work to use MCTS to build hierarchical space partitions. But their partitions are predeﬁned (e.g., Voronoi graph, axis-aligned partition, etc) without learning (or adapting to) observed samples so far, except for (Wang et al., 2019; 2020; Yang et al., 2021), which are learning extensions coupled with MCTS. However, they all deal with single-objective optimization. For multi-objective optimization, (Loshchilov et al., 2010; Seah et al., 2012; Pan et al., 2019) learns to predict the dominance rank of samples, without computing them algorithmic-ally, a slow process with many previous samples, in order to speed up the MOEAs algorithms.
Unlike our paper, they do not partition the search space into good/bad regions.
In contrast, LaMOO computes the rank algorithmic-ally.
Therefore, our contributions are complementary to theirs.
We leave a combination of both as one of the future works.