Non-Myopic Multi-Objective Bayesian Optimization
Authors: Syrine Belakaria
Alaleh Ahmadian
Barbara E Engelhardt
Stefano Ermon
Jana Doppa
Venue: Accepted by TMLR ()

Abstract: We consider the problem of finite-horizon sequential experimental design to solve multi-objective optimization (MOO) of expensive black-box objective functions. This problem arises in many real-world applications, including materials design, where we have a small resource budget to make and evaluate candidate materials in the lab. We solve this problem using the framework of Bayesian optimization (BO) and propose the first set of non-myopic methods for MOO problems. Prior work on non-myopic BO for single-objective problems relies on the Bellman optimality principle to handle the lookahead reasoning process. However, this principle does not hold for most MOO problems because the reward function needs to satisfy some conditions: scalar variable, monotonicity, and additivity. We address this challenge by using hypervolume improvement (HVI) as our scalarization approach, which allows us to use a lower-bound on the Bellman equation to approximate the finite-horizon using a batch expected hypervolume improvement (EHVI) acquisition function (AF) for MOO. Our formulation naturally allows us to use other improvement-based scalarizations and compare their efficacy to HVI. We derive three non-myopic AFs for MOBO: 1) the Nested AF, which is based on the exact computation of the lower bound, 2) the Joint AF, which is a lower bound on the nested AF, and 3) the BINOM AF, which is a fast and approximate variant based on batch multi-objective acquisition functions. Our experiments on multiple diverse real-world MO problems demonstrate that our non-myopic AFs substantially improve performance over the existing myopic AFs for MOBO.

Published in Transactions on Machine Learning Research (04/2025)
Non-Myopic Multi-Objective Bayesian Optimization
Syrine Bclakaria* Stanford University
syrinebastan edu
Alaleh Ahmadianshalchi* Washington State University
w.a.madionshhlcci@sssu...m edvu
Barbara E Engelhardt Stanford University
barbarueeSstann edu
Stefano Ermon Stanford University
ermon@ccsstanforde edu
Janardhan Rao Doppa Washington State University
jona. dooaa.dppaaooo @wsu. edu
Reviewed on OpenReview: https: h/t:://opppee openreview. net/ forum? id= 2e1a22d880
Abstract
We consider the problem of finitehhorionn sequential experimental design to solve multi- objective optimization (MOO) of expensive black-box objective functions. This problem arises in many real-world applications, including materials design, where we have a small resource budget to make and evaluate candidate materials in the lab. We solve this problem using the framework of Bayesian optimization (BO) and propose the first set of non-myopic methods for MOO problems. Prior work on non-myopic Bо for single-objective problems relies on the Bellman optimality principle to handle the lookahcad reasoning process. However, this principle does not hold for most. MOO problems because the reward function needs satisfy some conditions: scalar variable, monotonicity, and additivity. We address this challenge by using hypervolume improvement (HVI) as our scalarization approach, which allows us to use a lower-bound on the Bellman equation to approximate the finite-horizon using batch expected hypervolume improvement (EHVT) acquisition function (AF) for MOO. Our formulation naturally allows us to use other improvement- based scalarizations and compare their efficacy to HVI. We derive three non-myopic AFs for MOBO: 1) the Nested AF, which is based on the exact computation of the lower bound, 2) the Joint AF, which is a lower bound on the nested AF, and 3) the BINOM AF, which fast and approximate variant based on batch multi-objective acquisition functions. Our experiments multiple diverse real-world MO problems demonstrate that our non-myopic AFs substantially improve performance over the existing myopic AFs for MOBO.
1 Introduction
Many engineering design and scientific applications involve performing a sequence of experiments under a small-resource budget to optimize multiple cxpensive-to-eralue objective functions. Consider the example of searching nanoporous materials optimized for hydrogen-powent need to perform physical lab experiments to make a candidate material and evaluate its volumetric and gravimetric hydrogen uptake properties. Our goal is to approximate the optimal Pareto front of solutions under a given small resource budget for experiments.
vehicles. In this real-world problem,
"These authors contributed equally to this work

---

Published in Transactions on Machine Learning Research (04/2025)
We consider the problem of finite-horizon sequential experimental design (SED) to solve multi-objective optimization (MOO) problems over expensive- to--evllllae objectives to find high-quality Pareto solutions. We solve this problem within the framework of Bayesian optimization (BO) (Shahriari et al. 2015), which has been shown to be sample-effiiintt in practice. The koy idea behind BO is to train a surrogate model (o.g., a Gaussian process) from past experimental data, and intelligently select the future experiments guided by an acquisition function. Most of the prior acquisition functions for BO are myopic in nature (i.c., focused on the immediate advancement of the goal). There are & few non-myopic acquisition functions for BO in the single-objective setting (Jiang et al.] 2020a Lam & Willcox] 2016). These non-myopic strategies rely on formulating the optimal policy for SED as a dynamic program using the recursive Bellman optimality principle, and then solving for the optimal policy using methods with varying approximations (e.g,, rollout stralegies).
This work studies the design of non-myopic aequisition functions for MOO problems for the first time. The Bellman optimality principle does not hold in most MOO problems and requires the reward function of the underlying Markov decision process (MDP) to satisly some conditions (Roijers al. 2013: Van Moffaerl & Nowé 2014) 1) only scalar rewards; 2) monotonically increasing rewards with respect to better actions; and 3) additivity condition-foe for the optimality to hold. These conditions are challenging to satisfy in MOO problems and require a careful design of the reward function. We address this challenge by using hypervohume improvement (HVT) as our scalarization approach to define the reward function. Unlike the hypervolume function, hypervolune improvement satisfies the additivity condition. A key factor for these conditions to hold is to define optimality as Pareto optimality with respect to HVI rather than individual optimality for each objective function. This reward function allows us to use a lower-bound on the Bellman equation to approximately solve the finite-horizon SED problem using a batch utility function for MOO (c.g., qEHVI (Daulton et al. 20200). Our approach generalizes to other scalarization approaches that satisly the above conditions.
We derive three non-myopic multi-objective (NMMO) acquisition functions (AFs) for MOO with varying speed and accuracy trade-offs: 1) the NMMO-Nested AF, based on exact. computation of the lower bound and computationally intractable even for small horizons, 2) the NMMO-Joint AF, a scalable lower bound the nested AF, and 3) the BINOM AF, fast and approximate version of EHVI. Our experiments on real-world MOO problems demonstrate that the proposed non-myopic acquisition functions substantially improve performance over the baseline BO methods for MOO.
Contributions. The key contribution of our work is the development and evaluation of principled non-myopic acquisition functions solve MOO problems. In particular, we
validate the use of lower-bound on the Bellman optimality principle for MOO problems by using hypervolume improvement (HVT) to define appropriate reward function;
develop three non-myopic acquisition functions based on the derived lower Bound-NMMONNesteer NMMO- Joint, and BLNOM with varying speed and accuracy trade-offs;
instantiatc our framework with several information gain- based scalarizations and compare with HVI.
experimentally evaluate the proposed non-myopic acquisition functions and baseline BO methods on multiple real-world MOO problems. Our code is provided at hattps://g thub. com/ Alaleh/XMMO
2 Problem Setup and Background
This section formally defines the problem sctting and provides the background necessary for our solution approach.
Multi-objective optimization (MO0). Let X C Rd be the input space of d design variables, where each candidate input x E & is a d-dimensional vector. Let [f..... .fk] with K > 2 represent the black-box objective functions defined over the input space X, where (x), ... ".,......))))..........."..""."" x + R. We denote the function evaluation at x as y "y,....,,, ,yk], where y: = fxx for all E fl. K}. For example, to optimize nanoporous materials for hydrogen-powered vehicles, represents candidate material, fi [x) and fx(x)
2

---

Published in Transactions on Machine Learning Research (04/2025)
corresponds to a physical lab experiment to evaluate the volumetric and gravimetric hydrogen uptake of the material. WLOG, we assume maximization for all K objective functions.
An input x Pareto-dominates another input x' if and only if Vi fi(x) 2 fi(x') and 3j fx(x > (x'). The optimal solution of the MOO problem is a set of inputs X' C & such that no input x E 3 X Pareto-dominates another input x' E X The set input solutions X is called the optimal Pareto sel, and the corresponding set of function values ) is called the optimal Pareto front. In SED, we select one input for evaluation in each iteration, and our goal is to uncover high-quality Pareto front while minimizing the total number of expensive function evaluations. A typical measure to evaluate the quality of a Pareto front is the Pareto hypervolume (PHV) indicator [Zizzler & Thicle 1999).
Pareto Hypervolume (HV) Indicator. The hypervolume indicator HV of a finite approximate Parelo set y C. RK is the K K-dimensional Lebesgue measure Ax of the space dominated by y and bounded from below by a given reference point r E R
HV(2r)) = Ax (UI..)):
(1)
yEY
where yr denotes the hyper-rectangle formed by the reference point and the outcome vector y. The hypervolume quantifies the quality of the Pareto front approximation, with larger values indicating better solutions. In SED, aim to select inputs that maximize the increase in hypervolume to efficiently improve the Pareto front.
Non-myopic MOO problem. Our goal is approximate the optimal Pareto front for MOO problem within maximum experimental budget T, which is typically small (i.e., finite-horizon SED). This problem can be formulated as:
max max (fxx, ... [x(x),, s.t. [X|=
=
XEP(X) XEX
where P(x) denotes the power set of and X (x1...xxx) is the set of inputs selocted for function evaluation. We solve this problem using non-myopic decision policy, where, al each iteration, the algorithm selects one input for evaluation while reasoning about candidate experimental design plans within the budget T.
Overview of Bayesian optimization (BO). BO is an effective framework for solving expensive black-box optimization problems. BO has three main steps: (i) conducting an expensive experiment with an input selected by BO, (ii) updating the surrogate model by including the new evaluations, and (iii) selecting a candidate input for the next function evaluation. The two key components of BO are the surrggate model and acquisition function. Surrogate models are probabilistic models (typically Gaussian processes (Williams & Rasmussen 2006)) that are trained on data from past function evaluations. A surrogate model is able to predict. the output of unknown inputs without requiring an expensive experiment and to quantily the uncertainty of its predictions. The acquisition function (e.g., expected improvement (El) (Mockus 1989) is used to decide which input to select for the next function evaluation. It uses the surrogate model score the utility of selecting inputs for the next experiment by trading off exploitation and exploration. The goal is to quickly direct the search toward high-quality solutions with the least number of iterations. BO in multi-objective setting requires particular consideration for modeling and acquisition function design. For MOO problems, we model the objective functions f using K independent GP models "PI,...,GP.
and discuss acquisition functions in Section 3
Non-myopic decision policies for BO. We review important facts for optimal sequential decision-making
Osborne 2010). Consider collecting & observations D, (x),, E [0, []}, and let denote a utility function defining a quality metric over the data D.. The marginal gain in the utility of query x w.r.t. D, is expressed as
u(y|x, D.) u(D, U (x, - w(D.).
(3)
3

---

Published in Transactions on Machine Learning Research (04/2025)
Consider the where T - steps remaining. The expected marginal gain for T t-steps is expressed through the Bellman recursion (Jiang et al. 2020a) as
Ur-((x|D)) = By (y(yxx Dj) + By max (x.
(4)
exploitation
exploration
The optimal expected policy to select the T Equation at each step
lookahead steps can be obtained by sequentially maximizing
x" = argmax Ur -x
(5)
XEx
Being a sequence T - nested integrals, solving the optimization problem in equation 5 to optimality is intractable for even a small value of (Lam al. 2016 Jiang et al., 2020b
Recent work made connection between non-myopie input selection and batch input selection (Jiang et al., 2017 2020a Assuming parallel evaluation, the expected marginal utility of a new batch of evaluations Yr. . given their associaled batch of inputs X =
is defined as:
D( = Ey [u(Y|X, D.)]
(6)
Respectively, the maximum expected marginal utility of a batch of evaluations at input locations X' of size XX' = T - t conditioned on the addition of an observation (x,y) to the data is defined as
X- D.
Comparing equations A and 7 the second term in equation has adaptive utility with nested maximization while equation has joint batch utility with outer maximization. This loads to lower-bound connection between the two:
max X
Ey (x, (x, & By ( U (x,
(8)
Consequently, we obtain a lower bound on the expected marginal gain defined by the Bellman recursion in equation
a(x(D.) S (xT-)(
(9)
with a(x(D,) = Eylu(yyx,DDD)
D.)l
max -T
E,U((X)) (x,y)))
(10)
explostation
exploration
The lower bound on the Bellman equation (Equation 10 provides an efficient alternative for optimizing the sequential decision-making process in non-myopic single-objective BO (Jiang et al., (020a)), where the utility function u(D,) is defined as the incumbent, and the marginal utility u(y D.) as the incumbent improvement. However, applying the same strategy in the multi-objective BO (MOBO) setting is non-trivial. The Bellman equation is inherently singlo-dimensionall with respect to the output of the utility function and, therefore, presents challenges when faced with the multi-oulput nature of MOBO, where conflicting objectives require simultaneous consideration. This necessitates an adapted formulation capable of capturing the trade-offs across multiple objectives and guiding the exploration and exploitation balance. The methodological challenges address the extension of the Bellman equation for MOBO and the proposed solutions are described in Section 4
4

---

Published in Transactions on Machine Learning Research (04/2025)
3 Related Work
Most of the prior work on acquisition functions in BO falls under the myopic category. There is less work on non-myopic acquisition functions with focus on single-objective BO.
Myopic single-objective BO. Recent advances in single-objective BO have predominantly focused on
refining and extending traditional acquisition functions. The most notable techniques (Garnett
2023 Wang
& Jegelka models.
2017
Hernández-Lobato al. 2014 Ament et al. 2024) use Gaussian processes as surrogate
Non-myopic single-objective BO. Non-myopic approaches in single-objective B0 (Jiang et al. 2020a
Lam & Willcox 2016 González et al.; 2016 Ginsbourger & Le Riche. 2009. Osborne 2010: Osborne et al.
2009:
Belakaria et al. 2022 Kharkovskii el al. 2020; Marchant et al. 2014 Ling et al.) 2016) represent
an emerging direction aimed overcoming the limitations of myopic acquisition functions, which typically
optimize for immediate gains. These lookahead methods consider the future impact of current decisions (Wul
& Frazier 2019 Lee et al.] 2021), thereby enhancing long-term optimization outcomes when the experimental resource budget is small. Jiang el al. (2017) also addressed non-myopic active search by introducing batch selection to approximate non-myopic sequential decision-maken making.
Multi-objective BO. When compared single-objective BO, multi-objectivve BO (MOBO) has received relatively less attention. The predominant methods for dosigning MOBO strategics include information-
theoretic methods (Tu et al. 2022; Hernández-Looato et al. 2016 Belakaria et al. 2019 Suzuki et al.
2020: Garrido & Hernández- Lobato 2021), uncertainty based methods (Belakaria et al. 2020b).
hypervolume- based methods (Konakovic Lukovic et al. 2020
Daulton et al. 2020 Ahmadianshalchi et al.
2024ah, scalarization- based methods (Daulton et al.] 2022
Knowles 2006 and batch MOBO methods
AAhaddanshallhhi et al. (0244b). The joint entropy search for MOBO (JESMO) algorithm (Tu el al. 2022
aims maximize information gain about the optimal Pareto front by evaluating inputs that maximally reduce its uncertainty. The expected hypervolume improvement (EHVT) method (Daulton et al. 2020 extends the concept of EI from single-objective to MOO problem by assessing the potential improvement in hypervolume from new input evaluations. Several efforts to extend these approaches to handle constraints and preferences have resulted in more general frameworks that accommodate complex real-world decision-making scenarios GaariidoMMercánn & Hernández-Lobato, 2019 Belakaria et al.) 2020a Ahmadianshalchi et al.,
2024ch. MOBO methods have been applied to solve challenging engineering design applications (Chen et al. 2024 Belakaria et al. 2020cc Deshwal et al.) 2021a Yang ct al. 2021 Zhou al. 2020) and can also be
employed with surrogate models for combinatorial spaces (Deshwal & Doppa 2021 Deshwal et al. 2023
2021b 2022
Non-myopic MOBO. Despite advances in myopic strategies for MOBO, a notable knowledge gap remains.
There is no prior work on non-myopic acquisition functions for MOBO problems with a finite horizon. A recent approach proposed for settings with decoupled function evaluation, Daulton et al. (2023) considers a one-step lookahead only and thus lacks strategic foresight. Our goal is to precisely fill this crilical gap in the current state of knowledge.
4 Non-Myopic Multi-Objective BO
In this section, we provide details of our proposed approach for non-myopic MOBO. We first discuss the challenge of maintaining the validity of the Bellman optimality principle and, by consequence, the ability to use the Bellman equation and its bower-bound to solve the SED problem in the multi-objectivee setting. We then discuss design choices for the marginal utility function that enables the effective use of the Bellman equation in our setting. We finally provide the details of the non-myopic acquisition functions we propose, their computational challenges, and practical algorithms.
5

---

Published in Transactions on Machine Learning Research (04/2025)
4.1 Challenges of Non-Myopin MOBO
For the Bellman optimality principle to hold and for the optimal policy to be computable by solving the Bellman equation, the reward function has to satisfy several criteria. In our problem setting, we refer to the reward and the marginal gain in utility defined in equation reward function requirements that are not straightforward to satisfy when dealing with a multi-objective problem:
g interchangeably. We state below some of the
Scalarization: the reward has to be defined as single scalar value and not vector of values (Boutilier ett al.) 1999 Roijers et al.) 2013). In single-objective optimization, the reward is defined as the improvement in the best-found function value (Lam et al. 2016; Jiang et al. 2020a). However, in MOBO problems, every input evaluation leads to a vector of values representing a trade-off where some of the objectives may improve while others degrade.
Monotonicity: the reward has to be monotonically increasing function with respect to the quality of the actions (i.e., better actions lead to higher rewards). This monotonic property ensures that the optimization trajectory aligned with improving performance (Roijers et al. 2013
Additivity: the reward must satisfy the additivity condition, i.e, the total reward is a sum of the rewards
obtained at each step of the decision-making process (Boutilier et al. 1999 Roijers et al. 2013 Van Moffaert
& Nowé 2014 Additivity is essential for the recursive decomposition of the sequential decision process.
It is important to note that the choice of the scalarization approach plays a key role in preserving the monotonicity and additivity conditions. In MOBO problems, the challenge lies in combining both the improvement and degradation incurred in several objectives into a single, qfiali metrie that accurately reflects the quality of an imput in termss of Pareto dominance regardless of trade-offs in individual objectives. In broader applications of the Bellman equation (c.g., multi-objective reinforccment learning), linear scalarization is the most common technique known to preserve the additivity the reward, where each objective is multiplied by a predetermined nonnegative weight reflecting its relative importance (Barrett & Narayanan, 2008). Since linear scalarization computes convex combination, it has the known fundamental limitation that cannot recover solutions laying outside the convex regions of the Pareto front (Van Moffaert & Nowe 2014
4.2 Our Proposed Non-Myopic MOBO Method
In this section, we provide the details of the design choices of our utility function that enable the application of the Bellman equation in the MO setting. We then provide the details of the different acquisition functions propose for non-myopic MOBO.
4.2.1 Utility for Multi- ObjeitObeeetive Non-Myopic Setting
The hypervolume (HV) quality indicator provides scalar measure of a particular Pareto front y. HV is the volune between reference point and the Pareto front. This measure has been of particular interest in MOBO problems because it is known to be strictly monotonically increasing with regard to Pareto dominance. However, the HV measure does not satisfy the additivity condition necessary for Bellman's 's optimality principle (Van Moffaert et al. 2013). In this work, we propose to use the hypervolume improvement (HVI) to scalarize our multiple single-objective rewards defined as the improvement of each of the functions. This approach quantifies the quality of new point y by the volume of the objective space that is newly encompassed by extending the Pareto front to include the new point.
- HV())
(11)
HV1 provides a suitable scalarization because it naturally satisfies all of the reward requirements. First, it integrates the improvement of multiple objectives into a single scalar value. Second, preserves the monotonicity by reflecting the Pareto quality of new input (action) given the current Pareto front (state space representation).
6

---

Published in Transactions on Machine Learning Research (04/2025)
Intuitively, HVI satisfies the additivity condition because the improvement in hypervohume by successive imput evaluations (actions) can be summed to give total improvement over the sequence of evaluations, thus adhering to the required recursive structure.
We denole HVI. the hapervolume improvement hypervolume improvement collected at iteration T. The additivity condition is satisfied because the total hypervolume collected at iteration T can be decomposed as a summation of the intermediate hypervolume improvements
Lemma 1.
with t E [1, and Itotal be total
T HVIental = Σ HVI.
(12)
We provide a proof of lemma 1 in Appendix B Tt is important to note that the Bellman optimality principle holds with respect to HVI and Pareto optimality defined by the HV measure, rather than with respect to each objective function separately. This definition optimality aligns well with the goals of MOBO, where the focus on maximizing the objective space coverage efficiently, and where defining optimality with respect to HV and using HV as the primary evaluation metric is the most common approach (Belakaria et al.) 2019 Daulton al. 2021). We make this distinction to avoid confusion with the multi-objective RL setting, where theoretical optimality guarantees with respect to each function independently might not hold when using nonlinear scalarizations RRojers et al. 2013).
4.2.2 Proposed Acquisition Functions
Given our choice of reward as the HVI, can define SED problem using the Bellman equation. We formally define the marginal gain in the utility of our multi-objective setting as follows:
u(y|x, D,) HVI(y)))
(13)
Similar to the single-objective case, computing the optimal policy by fully solving the Bellman equation to optimality is intractable even for a moderately large horizon T t (Jiang et al. 2020b). An effective approach to navigating this complexity is to approximate the optimal policy by optimizing the lower-bound on the Bellman equation (Equation 9h. In what follows, we denote by X the full remaining horizon at iteration with |X| = T 1, the next input. to evaluate and the first input in the horizon X, and X' the lookahead horizon with X T T and X = fx, X'). The selection of the next input to evaluate while accounting for the lookahead horizon (the remaining sequence of inputs in the horizon) can be achieved by maximizing the acquisition function a;) (Equation 10 By defining the marginal gain in utility as the HVI, we rewrite the acquisition function as:
(Nested(x(D.) = EHVI(*ID)
Ey (x,,)) XIX
max
(14)
The first term on the r.h.s is the expected hypervolume improvement (EHVI) for the input x given the observed data. Dr. The second term the maximum of the expectation over the batch expected hypervolume improvement (BEHVI) computed at the batch of inputs X' conditioned on the addition of the new observation yx.y) to the data. The expectation is computed with respect to the possible values of y sampled from the posterior of the surrogate models at the input We refer to this new acquisition function the nested non-myopic multi-object acquisition function. Since the first part corresponds to the EHVI acquisition function |Emmerich & Klinkenberg) 2008), which corresponds to the one-step myopic policy, input selection by optimizing this lower bound always at least as tight as optimizing the myopic policy. This approach, while approximate, leverages the strengths of the EHVI and BEHVT acquisition functions (Emmerich & Klinkenberg 2008 Daulton et al.) (2020) efficiently compute the immediate impact of the selection of input and to approximate the impact of the selection of the lookahead horizon X'.
7

---

Published in Transactions on Machine Learning Research (04/2025)
Algorithm NMMO Algorithm
Input: input space X; K blackbox objective functions fi (1()),,))) and maximum no. of iterations T, selected non-myopic method E JMMMO-Joint, WMMO-Nested, BINOM}.
1: Initialize GP models .P...PP..... ... PPk by evaluating at No initial points
2: for each iteration t = 1 to T do
3: if mettood==WMOONeestt
then
Select X, < arg maxxex (.Neste Dx), where ONNested defined in Equation 14
if method==WMM0-oos then
6:
Select x,, X" t arg mo XEX.X' Ex77--- Oooinn (x, X (Dt), where J.oint is defined in Equation 15
if method==BINOM then
8: Select X arg XEXT- ABINOM(X(D.)) where aBINOM is defined in Equation 16
9:
Select X, f arg maxxEX EHVI(x((;)
10: Evaluate X₁: Yt - (fi(x))... (x(xx))
11: Aggregate data: D141 t D₃ yi)
12: Update models GPI 1 2P: ,9PK
13:
end for
14: return Pareto front of (x)), fK [x(x based on Dr
Computational challenges. The expression of the acquisition function (Equation 14 involves two computational challenges for which we propose solutions.
1. Nested Mazimization: The second term in the nested non-myopic AF for MOO (Equation 14 as a maximization problem. Thus, evaluating the acquisition function for each candidate imput x requires solving an internal optimization problem to estimate the batch representing the lookahead horizon. This leads to computationally prohibitive acquisition function optimization. To address this challenge, propose two alternative acquisition functions:
is defined
A Aternative lower-bound computation: One approach to circumvent the nested maximization problem is to reformulate the acquisition function a new lower-bound that jointly optimizes over both the first input in the horizon and the lookahead horizon X'. We refer to this approach the joint non-myopic MO acquisition function.
Joint (x, D.) (x,
EE/EE/II))))) (x,,))
(15)
Approximation via batch selection: Approximating the selection of all the inputs in the horizon by a batch acquisition function has been previously used for non-myopic single-objective BO (González et al. 2016
Jiang et al.) 2020a). Let X", s.t | - & be a set of inpuls selected by maximizing a batch acquisition function U (,) Jiang et al. 2020a) showed that choosing any input x* E X can be approximated by selecting inputs that maximize a Dx) several times, with a as the acquisition function in equation 10 With
this approximation, we propose to follow Jiang et al. (2020a), and approximate the non-myopic acquisition
function by a fully joint batch acquisition lunction. We select the full horizon by maximizing the BEHVI. We call this selection approach Batch- Informed NOnmyopic Multi-objective optimization (BINOM):
OBINOM (EINOM(X(D.)) = BEHVT(X(D.).
-
(16)
However, an important issue arises. The strategy does not differentiate the next input for evaluation x" from the lookahead horizon X' in the way the acquisition functions in equations 10 and 15 do. By analogy Jiang et al. (2020a). we select the next input evaluate the input with the highest immediate EHVI among the inputs in the selected batch.
8

---

Published in Transactions on Machine Learning Research (04/2025)
2. Expectation computation via posterior sampling: The second term of the acquisition function (Equation 10 requires expectation computed over the BEHVT. This expectation can be estimated by sampling several realizations of y from the posterior of the surrogate models. The accuracy of the expectation estimation depends on the mumber of samples used. In practice, more samples lead to a slower and more computationally expensive acquisition function, while fewer samples leads to inaccurate estimation and a higher variance. This results in unstable optimization behavior with unreliable outcomes. To mitigate the former issues, we propose substitute the expectation by the use of one-step lookahead GP model (Lyu et al.] 2021). In this approach, the mean function of the GP remains unchanged, while the variance is updated based on the posterior conditioning on the newly added input.
In summary, these computational strategies are crucial for enhancing the elficiency and stability of the acquisition functions multi-objective optimization scenarios. With these strategies, our decision-making process for finite- horizoo sequential experimental design using MOBO becomes computationally tractable and more robust.
5 Experiments and Results
This section describes our experimental evaluation comparing the proposed non-myopic methods with basclines on several real-world and synthetic MOO problems.
Benchmark MOO problems: We provide a brief description of our real-world MOO benchmarks, including the number of input dimensions (d) and the number of objectives (K).
Metal-organéc Framework Design (d = 7, K 2) (Kitagawa et al. 2014 Boyd al. 2019 Nanoporous materials (NPMs) (Lu & Zhao 2004) represent class of structures characterized by their microscopic pores at the nanoscale level. Metal-organic frameworks (MOFs) (Ahmed et al. 2017 Ahmed & Siegel 2021: Boyd et al. 2019: Deshwal et al. (021cc) are class of NPMs, instrumental in several sustainable applications due to their selective gas absorption capabilities. The goal of this problem is to identify the Pareto front of MOFs from & 100K candidates that optimize gravimetric and volumetric hydrogen uptake for hydrogen-poweree vehicles.
Reinforced Conerete Beam Design (d 3, K 2) (Amir & Hasegawa 1989 :: The goal is to optimize the
cost of concrete and reinforeing steel used in the construction of a beam. Its input features are the area of reinforcement, the beam's width, and its depth. This setup enabled us to explore a mixed input space to find cost-effective yet robust designs.
(Cheng & Li 1999h: The goal is to minimize the structural volume and joint displacement of a four-bar truss to reduce the weight of the structure and the displacement of a specific node. The features are the areas of the four-member cross-sections.
Four-Bar Truss Design (d = K. =
The goal is to design compound four-gear train to achieve a specific target gear ratio between the driver and driven shafts. The objectives are to minimize the error between the obtained gear ratio and the desired gear ratio and to minimize the maximum size of the four gears to optimize the compactness of the design.
Gear Train Design (d 4, K 3) (Deb & Srinivasan 2006
Tanabe & Ishibuchi 2020
Welded Beam Design (d = 4, K = 3) (Coello & Montes 2002 Rao 2019): The goal is to minimize the cost of fabrication and the end deflection of a welded beam by precisely adjusting four of its side lengths. To focus more directly on the relationship between the beam's dimensions and the optimization objectives, we adopted non-constrained version of this problem as in prior work [anabee & Ishibuchi 2020 Konakovic Lukovie
et al. 2020
= 3) 3) RRa & Liew 2002 Tanabe & Ishibuchi 2020): The goals are to minimize the mass of the brake and the stopping time. The variables are the inner and outer radius of the discs, the engaging force, and the number of friction surfaces.
Disc Break Design (d 4. K-
We also include the ZDT-3 (d=9, K= 2) Zitzler et al. 2000) problem as a synthetic MOO benchmark. We provide additional results in the Appendix (Section A.4) and an overview of all our MOO benchmarks(Appennis Table
9

---

Published in Transactions on Machine Learning Research (04/2025)
Disc brake design
Four-bar truss design
Gear train design
100
320
Hypervolume
40
Hypervolume
75
Hyperyg
300
50
38
280
25
0
20
40
60
36
20
40
60
0
20
40
60
Iteration Number
Iteration Number
Iteration Number
organio framework design
1e10 Welded beam design
2100
Reinforced concrete beam design
1.80
Hypervolume
2000
Hyagrvolume
620000
Hypervolume
800000
L.78
1900
580000
1.76
190000
560000
1700
40
20
40
60
1.74
Iteration Number
Iteration Number
Iteration Number
BINOM (H=4)
EHVI
JESMO
NMMO-Joint (H=4)
NMMO-joint (H=8)
Figure 1: Hypervolume results on real-world problems: Non-myopie methods vs. EHVI and JESMO.
Baselines: We proposed three non-myopic acquisition functions (NMMO-Joint, NMMO-Nested, and BINOM) with varying trade-offss between computational complexity and approximation accuracy. In an ideal scenario with a perfect surrogate model, evaluating the entire lookahead horizon H = T - at each iteration would yield the closest results to the optimal policy. However, previous research on non-myopic Bayesian optimization (Jiang et al. 2020a| b Lee et al., 2021 2020) has shown that models often struggle with long-term predictions due compounding uncertainty at each lookahead step. This is because the model's predictions at each step are based the uncertain predictions from the previous steps. As result, querying an extended horizon can hinder the optimization process by including suboptimal or mislcading inputs in the non-myopic horizon, potentially guiding the search in less promising directions and incurring higher compulational costs. To address this issue, we adopt the same strategy from previous work and introduce a maximum horizon length to limit the size of the horizon.
The NMMO-Nested method stands out with its potential for superior optimization performance due to its comprehensive approach to handling nested optimizations. However, this method has practical challenges. Notably, this method requires a pro-defined grid for optimization, as internal maximization prevents using gradient- cradient-based optimizers. Furthermore, Its computational complexity constrains the size of the grid that can be feasibly used, which in turn limits the method's 's operational speed and accuracy. Hence, the nested approach does not perform as well as expected compared to the other proposed non-myopic approaches. We provide results including NMMO-Joint with lookahead horizon H E {4, 8} and BINOM with H = 4. We report the results for NMMO- with H
the Appendix Section A.5.
We consider two state-of-the-at MOBO baselines to benchmark the performance of our non-myopic methods. First, we use the multi-objective variant of the joint entropy search for multi-objective Bayesian optimization (JESMO) (u et al. 2022 Balandat et al.) 2020). Second, use the expected hypervolume improvement (EHVT) method (Emmerich et al.) 2006; Balandat et al. 2020) as a hypervolume-base bascline. Another baseline is the hypervolume knowledge gradient (HVKG), a one-step lookahead method for cost-aware MOBO problems (Daulton et al. 2023). While the HVKG algorithm designed for cost-aware settings with decoupled evaluations, we use a modified version of this method where the objective function costs are set to zero, and we calculate all objectives for a true evaluation of the hypervolume. We report the results in the Appendix Section A.5
10

---

Published in Transactions on Machine Learning Research (04/2025)
The baseline methods were chosen due to their distinct but complementary approaches to handling multi- objective trade-offs, providing comprehensive comparative perspective against our proposed NMMO method. The effectiveness of each method is measured using the hypervolume indicator.
Evaluation metric. The Hypervolume is widely-used metric in MOO that measures the volume of the space dominated by the Pareto front relative to a predefined reference point: a larger hypervolume indicates beller Pareto solutions. We will compare the hypervolume metric after each BO ileration.
Experimental details: All experiments were averaged among 15 runs, initialized with five points randomly generated from Sobol sequences, and run for 65 iterations. We emphasize that non-myopic solutions are well- suited for the smaller experimental budgets. We model each of the multiple objectives using independent GP with Matérn 5/2 ARD kernel. We use implementations from the BoTorch Python package (Balandat et al. 2020 for all baselines, including JESMO, EHVT, and HVKG. All methods were evaluated under identical initial conditions. This includes consistent initialization, identical computational budgets, and the same ovaluation criteria across all experiments.
5.1 Results and Discussion
Figure I shows the hypervolume versus BO iterations for NMMO-Joint with lookahead horizon H E {4, and BINOM with lookahead horizon H = 4 against existing state-of-the-al baselines.
Non-myopic methods perform better than myopic ones. The non-myopic methods outperform the state-of-the-ae aart-rr- myopic MOBO algorithms all six real-world MOO problems (Figure D These results highlight the capability of non-myopic methods in maximizing the hypervolume when the experimental budget is small (i.e., 65 BO iterations).
Non-myopic methods with different horizons. From our results, we often observe a slight decrease in performance with increasing horizon values in non-myopic MOO. This trend can be primarily attributed to the increased complexity and model uncertainty as the decision horizon extends. A longer horizon necessitates the consideration of a larger set of future outcomes and their interactions, which not only complicates the oplimization lask but also introduces greater uncertainty and potential for errors in prediction. Consequently, these accumulated errors can degrade the optimization strategy, making it challenging for the algorithm effectively balance short-term gains against long-term goals, and negatively impacting performance. This finding highlights the challenges in implementing long-term strategic planning in complex optimization environments. These challenges be seen in the NMMO-Joint and BINOM methods run each benchmark with increasing lookahead horizon H {2, 6, 8} (Appendix Figure and Figure
Computational runtime comparison. In our comparative analysis of myopic and non-myopic MOO methods, there are computational complexity and optimization performance tradc-offs. Myopic MOO methods, due to their focus on immediate gains, have lower computational complexity, which translates into faster runtimes (Table 2h. However, this computational efficiency comes at the cost of performance. Our results indicate that, while myopic methods are faster, they underperform in comparison to their non-myopic counterparts in limited-budget settings. Hence, while non-myopic approaches require more computational resources, their better optimization results justify the additional complexity.
Ablation study of information gain within BINOM. In addition to the EHVI acquisition function, we incorporated information-gall acquisition functions, specifically MESMO and JESMO, into proposed BINOM approach. Even though information gain based scalarization does not always satisfy the monotonicity condition as discussed in section A.3 These instantiations of our framework provide a complementary approach to the hypervolume- based strategies. The results, provided in the Appendix demonstrate that the non-myopic mothods maintain strong performance across different instantiations of BINOM, highlighting the versatility and effectiveness of our framework. The results also show that the HVI instantiation performs better in most cases.
11

---

Published in Transactions on Machine Learning Research (04/2025)
6 Summary
This paper introduced novel set of non-myopic methods for multi-objective Bayesian optimization (MOBO); to specifically address the problem of finite-horizon sequential experimental design (SED) for MOO problems. By addressing the difficulties related to scalarization, monotonicity, and additivity of the reward function typically encountered in MOO scenarios, we successfully adapted the Bellman optimality principle for MOBO by adopting a new approach to scalarization using hypervolmme improvement (HVI). Our proposed methods, including the Nested, Joint, and BINOM acquisition functions, represent the first non-myopic acquisition functions tailored for MOBO problems. Our experiments on multiple real-world problems, show that our methods show significant improvements by consistently outperforming the state-of-the-ar myopic approaches in limited-budger settings.
Acknowledgments. Belakaria is supported by Stanford Data Science Postdoctoral Fellowship. Ahmadian- shalchi and Doppa are supported by the NSF CAREER award IIS-1845922.
References
Alaleh Ahmadianshalchii, Syrine Belakaria, and Janardhan Rao Doppa. Pareto front-diverse batch multi- objective Bayesian optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 38: 10784-10794, Mar. 2024a, doi: 10.1609/aami http://aaai.881..22550.. URL https://ojs. aaai. org/ org/index. iid/xxndxxxppp hrg/innex.phhp/
AAAT/articcle/veee//888
Alalch Ahmadianshalchi, Syrine Bclakaria, and Janardhan Rao Doppa. Pareto front-diverse batch multi- objective bayesian optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 38(10):
2024b.
Alaleh Ahmadianshalchi, Syrine Belakaria, and Janardhan Rao Doppa. Preference-aware constrained multi- objective bayesian optimization. In Proceedings of the 7th Joint International Conference on Data Science & Management of Data (11th ACM IKDD CODS and 29th COMAD), pp. 182 191, 2024c.
Alauddin Ahmed and Donald J. Siegel. Predicting hydrogen storage in mofs via machine learning. Patterns, 2(7):100291, 2021. ISSN 2666-3899.
Alauddin Ahmed, Yiyang Liu, Justin Purewal, Ly D. Tran, Antek G. Wong-Foy, Mike Veenstra, Adam J. Matzger, and Donald J. Siegel. Balancing gravimetric and volumetric hydrogen density in mofs. Energy Environ. Sci., 10:2459 2471, 2017.
Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Unexpected improvements to expected improvement for Bayesian optimization. Advances in Neural Information Processing Systems, 36, 2024.
Hossain M Amir and Takashi Hasegawa. Nonlinear mixed--iisretee structural optimization. Journal of Structural Engineering, 115(3):626 646, 1989.
Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy. BoTorrh: A Framework for Efficient Monte-Carrl Bayesian Optimization. In Advances in Neural Information Processing Systems 33, 2020. URL http:://arxiiiii wttps/..rrg//bs/111..... 06403
Leon Barrett and Srini Narayanan. Learning all optimal policies with multiple criteria. In Proceedings of the 25th internalional conference on Machine learning, pp. 41 47, 2008.
Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Max-value entropy search for multi-objective Bayesian optimization. In Conference on Neural Information Processing Systems, 2019.
Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Max-value entropy search for multi-objective Bayesian optimization with constraints. Xrv preprint arXiv:2009. 01721, 2020a.
12

---

Published in Transactions on Machine Learning Research (04/2025)
Syrine Belakaria, Aryan Deshwal, Nitthilan Kannappan Jayakodi, and Janardhan Rao Doppa. Uncertainty- aware search framework for multi-objective Bayesian optimization. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 34, 2020b.
Syrine Belakaria, Derek Jackson, Yue Cao, Janardhan Rao Doppa, and Xiaonan Lu. Machine learning enabled fast multi-objective optimization for electrified aviation power system design. In IEEE Energy Conversion Congress and Erposition (ECCE), 2020c.
Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Output space entropy search framework for multi-objective bayesian optimization. Journal of artificial intelligence research, 72:667-716 -15, 2021.
Syrine Belakaria, Rishit Sheth, Janardhan Rao Doppa, and Nicolò Fusi. Bayesian optimization over iterative learners with structured responses: A budget-aware planning approach. CoRR, abs/ hes/2206.12708 2022.
Craig Boutilier, Thomas Dean, and Steve Hanks. theistcn- planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research, 11:1 94, 1999.
Peter G. Boyd, Arunraj Chidambaram, Enrique García-Diez, Christopher P. Ireland, Thomas D. Daff, Richard Bounds, Andrzej Gladysiak, Pascal Schouwink, Scyed Mohamad Moosavi, M. Mercedes Maroto- Valer, Jeffrey A. Reimer, Jorge A. R. Navarro, Tom K. Woo, Susana Garcia, Kyriakos C. Stylianou, and Berend Smit. Data-driven design of metal organic frameworks for wet flue gas co2 capture. Nature, 576(7786):253 256, December 2019. ISSN 1476-4687. doi: 10.1038//411586-019-117888 URL http:
//dx.doi.ov org. 10.
Eric Chen, Alaleh Ahmadianshalchi, Sonja Sparks, Chuchu Chen, Aryan Deshwal, Janardhan R Doppa, and Kaiyan Qiu. Machine learning enabled design and optimization for 3d-printing of high-fidelity presurgical organ models. Advanced Materiols Technologies, pp. 2400037, 2024.
FY Cheng and XS Li. Generalized center method for multiobjective engineering optimization. Engineering Oplimization, (155::641 661, 1999.
Carlos A Coello Coello and Efrén Mezura Montes. Constraint--aandling in genetic algorithms through the use of Nominance-based tournament selection. Advanced Engineering Informatics, 16(3):193 203, 2002.
Sam Daulton, Maximilian Balandat, and Eytan Bakshy. Hypervolume knowledge gradient: a lookahead approach for multi-objective Bayesian optimization with partial information. In International Conference on Machine Learning, pp. 7167-7204. PMLR, 2023.
Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. Differentiable expected hypervolume improvement for parallel multi-objective Bayesian optimization. Advances in Neural Information Processing Systems, 33: 9851-9864, 2020.
Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. Parallel Bayesian optimization of multiple noisy objectives with expected hypervolume improvement. NewrIPS, 34, 2021.
Samuel Daulton, Sait Cakmak, Maximilian Balandat, Michael A. Osborne, Enlu Zhou, and Eytan Bakshy, Robust multi-objective Bayesian optimization under input noisc. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Mochine Learning Research, pp. 4831 4866. PMLR, 17-23 Jul 2022. URL htt.attpss//proceedng....i/ .mw...s......g...........
htmm1 html
Kalyanmoy Deb and Aravind Srinivasan. Innovization: Innovating design principles through optimization. Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp. 1629 1636, 2006.
Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. Scalable test problems for evolutionary multiobjective optimization. In Evolutionary multiobjective optimization. Springer, 2005.
Aryan Deshwal and Janardhan Rao Doppa. Combining latent space and structured kernels for Bayesian optimization over combinatorial spaces. In Advances in Neural Information Processing Systems (NeurIPS), pp. 8185 82508 2021.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Aryan Deshwal, Syrine Belakaria, Ganapati Bhat, Janardhan Rao Doppa, and Partha Pratim Pande. Learning pareto-frontic resource management, policies for heterogeneous soes: An information-theoreiic approach. In (DAC), 2021a.
Aryan Deshwal, Syrine Belakaria, and Janardhan Rao Doppa. Bayesian optimization over hybrid spaces. Tn Proceedings of the 38th International Conference Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Research, pp. 2632 2643. PMLR, 2021b.
Aryan Deshwal, Cory M. Simon, and Janardhan Rao Doppa. Bayesian optimization of nanoporous materials. Mol. Syst. Des. Eng., 6:1066 1086661066, 2021c
Aryan Deshwal, Syrine Belakaria, Janardhan Rao Doppa, and Dae Hyun Kim. Bayesian optimization over permutation spaces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 2022.
Aryan Deshwal, Sebastian Ament, Maximilian Balandat, Eytan Bakshy, Janardhan Rao Doppa, and David Eriksson. Bayesian optimization over high-dimensional combinatorial spaces via dictionary-based embed- dings. ColR, abs/ abs/2333.01177 2023.
Michael Emmerich and Jan-willem Klinkenberg. The computation of the expected improvement in dominated hypervolume of pareto front approximations. Technical Report, Leiden University, 34, 2008.
M.T.M. Emmerich, K.C. Giannakoglou, and B. Naujoks. Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels. IEEE Transactions on Evolutionary Computation, 10(4): 421-439, 2006. doi: 211100/TEVC.20055.555
Roman Garnett. Bayesian optimization. Cambridge University Press, 2023.
Eduardo C Merchhn and Daniel Seemández-Lobato Predictive entropy search for multi-objective Bayesian optimization with constraints. Neurocomputing, 361:50-68, 2019.
Eduardo C. Merchén and Daniel Hernández-LLoatoo Parallel predictive entropy search for multi-
objective Bayesian optimization with constraints, 2021.
David Ginshourger and Rodolphe Le Riche. Towards gp-based optimization with finite time horizon. Advances in Model- Oriented Design and Analysis: Proceedings of the 9th International Workshop in Mfodel-Oriented Design and Analysis, 10 2009.
Javier González, Michael Osborne, and Neil Lawrence. Glasses: Relieving the myopia of Bayesian optimisation. In Artificial Intelligence and Statistics, pp. 790-799. PMLR, 2016.
Daniel Hernández-LLobtto, Jose Hernandez-Lobato, Amar Shah, and Ryan Adams. Predictive entropy search for multi-objective Bayesian optimization. In Proceedings of International Conference Machine Learning (ICML), pp. 1492-1501, 2016.
José Miguel Hernández-Lobate. Matthew W Hoffman, and Zoubin Ghahramani. Predictive entropy scarch for elficient global optimization of black-box functions. In NewrIPS, 2014.
Shali Jiang, Gustavo Malkomes, Gcoff Converse, Alyssa Shofner, Bonjamin Moscley, and Roman Garnett. Efficient nonmyopic active search. In International Conference on Machine Learning. JMLR. org, 2017.
Shali Jiang, Henry Chai, Javier Gonzalez, and Roman Garnett. BINOCULARS for elficient, nonmyopic sequential experimental design. In International Conference on Machine Learning, pp. 4803. PMLR, 2020a.
Shali Jiang, Daniel Jiang, Maximilian Balandat, Brian Karrer, Jacob Gardner, and Roman Garnett. Efficient. nonmyopic Bayesian optimization via one-shot multi-step trees. In NeurIPS, 2020b.
Dmitrii Kharkovskii, Chun Kai Ling, and Bryan Kian Hsiang Low. Nonmyopic gaussian process optimization with macro-actions. In International Conference on Artificial Intelligence and Statistics, pp. 1599-4604. PMIRR 2020.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Susumu Kitagawa et al. orgal-organic frameworks (mofs). Chemical Society Reviews, 43(16)):441-5444 2014.
Joshua Knowles. Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjec- tive optimization problems. IEEE Transactions on Evolutionary Compulation, 10(1):50 66, 2006.
Mina Konakovic Lukovic, Yunsheng Tian, and Wojeiech Matusik. Diversity-guided multi-objective Bayesian optimization with batch evaluations. Advances Neural Information Processing Systems, 33:17708- 2020.
Andreas Krause and Carlos Guestrin. Nonmyopic active learning of Gaussian processes: an exploration- exploitation approach. In Proceedings of the 24th international conference Machine learwing, pp- 449 456, 2007.
R Lam and Karen E Willcox. Lookahead Bayesian optimization with inequality constraints. Proceedings of the 34th International Conference on Machine Learning, 56:1230-1239, 2016.
Remi Lam, Karen Willcox, and David H Wolpert. Bayesian optimization with finite budget: An approximate
dynamic programming approach. Adoances in Neural Information Processing Systems, 29, 2016.
Eric Lee, David Eriksson, David Bindel, Bolong Cheng, and Mike Mccourt. Efficient rollout strategies for Bayesian optimization. In UAL. PMLR, 2020.
Eric Hans Lee, David Eriksson, Valerio Perrone, and Matthias Seeger. A nonmyopic approach to cost- constrained Bayesian optimization, 2021.
Chun Kai Ling, Kian Hsiang Low, and Patrick Jaillet. Gaussian process planning with lipschitz continuous reward functions: Towards unifying bayesian optimization, active learning, and beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.
GQ Max Lu and Xiu Song Zhao. Nanoporous malerials: science and engineering, volume 4. World Scientific, 2004.
Xiong Lyu, Mickaöl Binois, and Michael Ludkovski. Evaluating Gaussian process metamodels and sequential designs for noisy level set estimation. Statistics and Computing, (1)4):33, 2021.
Roman Marchant, Fabio Ramos, Scott Sanner, et Sequential bayesian optimisation for spatial-temporal monitoring. In UAT, pp.
Citeseer, 2014.
Jonas Mockus. Boyesian Approach to Global Optimization: Theory and Applications. Kluwer Academic Publishers, 1989.
Michael Osborne. Bayesian Gaussian processes for sequential prediction, optimisation and quadrature. PhD thesis, Oxford University, UK, 2010.
Michael A. Osborne, Roman Garnett, and Stephen J. Roberts. Gaussian processes for global optimization. In 3rd international conference on learning and intelligent optimization (LION3), pp. 15, 2009.
Singiresu Rao. Engineering optimization: theory and practice. John Wiley & Sons, 2019.
Tapabrata Ray and KM Liew. A swarm melaphor for multiobjective design optinization. Engineering optimization, 34(2):141-155 153, 2002.
Diederik Roijers, Peter Vamplew, Shimon Whiteson, and Richard Dazeley. A survey of multi-objectivve soquential docision-making. Journal of Artificial Intelligence Research, 48:67-113, 2013.
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEFF, 2015.
15

---

Published in Transactions on Machine Learning Research (04/2025)
Shinya Suzuki, Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Multi-objective Bayesian optimization using pareto-frontier entropy. In International Conference on Machine Learning, pp- 9279 9288. PMLR, 2020.
Ryoji Tanabe and Hisao Ishibuchi. An easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing, 89:106078, 2020.
Ben Tu, Axel Gandy, Nikolas Kantas, and Behrang Shafei. Joint entropy search for multi-objective Bayesian optimization. Advances in Neural Information Processing Systems, 35:9922-9938, 2022.
Kristof Van Moffaert and Ann Nowé, Multi-objective reinforcement learning using sets of pareto dominating policies. The Journal of Machine Learning Research, 15(1):3483 3512, 2014.
Kristof Van Moffacrt, Madalina M Drugan, and Ann Nowé. Hypervolume-basee multi-objective reinforcement Mulli-Crilerion Oplimizalion: 7th International Conference, EMO 2013, Sheffield, learning. In Evolulionary Mulli- Crilerion
UK, March 19-22, 2013. Proceedings 7, pp.
Springer, 2013.
Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient Bayesian optimization. In Proceedings of International Conference Machine Learning (ICML), 2017.
Christopher KI Williams and Carl Edward Rasmussen, Gaussian processes for machine learning. MIT press Cambridge, MA, 2006.
Jian Wu and Peter Frazier. Practical two-step lookahcad Bayesian optimization. Aduances in newral informalion processing systems, 32, 2019.
Xiaoxuan Yang, Syrine Belakaria, Biresh Kumar Joardar, Huanrui Yang, Janardhan Rao Doppa, Partha Pra- tim Pande, Krishnendu Chakrabarty, and Hai Helen Li. Multi-objective optimization of reram crossbars for robust DNN inferencing under stochastic noise. Tn IEEE/ACM ACM International Conference On Computer Aided Design, ICCAD 2021, Munich, Germany, November 1-4, 2021, pp. 1 9. IEEE, 2021.
Zhiyuan Zhou, Syrine Belakaria, Aryan Deshwal, Wookpyo Hong, Janardhan Rao Doppa, Partha Pratim Pande, and Deukhyoun Heo. Design of multi-output switcheed-cpaccior voltage regulator via machine learning. In 2020 Design, Aulomation & Test in Europe Conference & Eahibition (DATE), pp. 502-507. IEEE, 2020.
Eckart Zitzler and Lothar Thiele. Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEFE transactions Evolutionary Computation, 3(4):257-271, 1999.
Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele. Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary computation, 8(2):173-195, 2000.
16

---

Published in Transactions on Machine Learning Research (04/2025)
A Appendix
A.1
Benchmarks details
In Table 1 we include concise overview of all of our MOO benchmarks consisting of the problem names, number of imput dimensions (d), number of output dimensions (K), and the reference point (r) used for caleulating the hypervolume indicator for each benchmark.
A.2 Analysis of lookahead horizons
We include figures illustrating the performance of our non-myopic methods across different horizon values. The results reveal a general trend where larger horizons correlate with slight drops in performance. This observation can primarily be attributed to the increased uncertainty associated with extending the optimization horizon. As the horizon increases, the predictive model must account for broader range of potential future outcomes. Their uncertainties accurately estimating the long-ten ter-term consequences of current decisions. Consequently, while longer horizons thelretically allow for more strategic and informed decision-making by considering future impacts, the added uncertainty can undermine performance by introducing greater variability in the optimization outcomes. Figure and Figure 3 show the hypervolume results for NMMO-Joint and BINOM with horizon lengths 2, 4, 6, and
complicate the decision-making process, as the model faces increased difficulty in
Disc brake design
Four-bar truss design
Gear train design
320
41.5
Hypervolume
100
hypervolume
Sypervolume,
315
95
40.5
90
310
40.0
85
20
40
60
39.5
0
20
40
60
80
20
40
60
Iteration Number
Iteration Number
Iteration Number
Mrtal-organic framework design
1e10 Welded beam design
2100
61500 Reinforce concrete beam design
Hypervolune
Hypervolume
2000
Asgervolursnn
@610000
1900
8605000
1.79
1800
5600000
595000
1.78
1700
20
40
20
40
60
20
60
Iteration Number
Iteration Number
Iteration Number
NMMO-Joint (H=2)
NMMO-Joint (H=4)
NMMO-Joint (H=6)
NMMO-Joint (H=8)
Figure 2: Hypervolume results for NMMO-Joint with lookahead horizon E {2, 6,8
A.3 Alternative scalarization approaches: an ablation study
The proof of additivity in Lemma 1 relies the fact that HVI is baaeeee scalarization. Therefore, this lemma thcoretically holds for any improvement-based scalarization. Notably, information gain (IG) based scalarizations, defined as the decrease in the entropy of the Pareto front/set, can indeed satisfy the additivity condition. IG can be computed with respect to the input space (leading to an instantiation of our method with PESMO (Horaándzz-Lobato et al. 20166). the output space (leading to an instantiation of our method with MESMO (Belakaria et al. 2019 2021 )) both (leading an instantiation of our method with JFSMO (Tu et al. 2022) ). However, it is important. to point out that the IG monotonieity with
17

---

Published in Transactions on Machine Learning Research (04/2025)
Disc brake design
Four-bar truss design
Gear train design
41.5
hypervolume..
320
servolume
41.00
Hypervolume
104
315
40.5
100
40.00
98
310
40
60
20
60
96
40
60
Iteration Number
Iteration Number
Iteration Number
o0 Metal-organic framework design
2100
Gisofeiinorredd concrete beam design
1.795 le10 Welded beam design
1e10
Hypervolume
2000
Hypervalune
6600000
Hypervolume
1.790
1900
605000
1.785
1800
7600000
#1.780
1700
20
40
595000
20
40 Iteration Number
60
Iteration Number
Iteration Number
BINOM (H=2)
BINOM (H=4)
BINOM (H=6)
BINOM (H=8)
Figure 3: Hypervolume results for BINOM with lookahead horizon H E {2, 4, 6,88).
respect to better actions might not always hold. Typically, an input is considered better choice/action if it dominates existing points in the Pareto front and thus enhances the quality of the Pareto front. However, an input can have a high information gain even is not Pareto dominating other inputs, but it provides high information about a new region the search space. This couflicting issue does not occur with HVI, since HVI is always monotonically increasing with respect to Pareto dominance.
Nevertheless, provide ablation study comparing variants of acquisition functions with EHVI, MESMO, and JESMO based scalarization
We present the results of our ablation study comparing the performance of the BINOM acquisition function using Expected Hypervolume Improvement (EHVI), Max-value Entropy Scarch (MES), and Joint Entropy Search (JES) with horizon lengths of 2, 4, and 6, across six real-world benchmark problems. For clarity, we refer the BINOM variant using EHVI as simply BINOM. The hypervolume plots for each benchmark illustrate that BINOM-EHVI generally outperforms the other acquisition functions, demonstrating its superiority in guiding the optimization process towards high-quality Pareto front. Even in instances where BINOM-EHVI does not achieve the best performance, it consistently surpasses our myopic baseline methods, reaffirming the robustness and effectiveness of the BINOM approach in diverse optimization scenarios.
A.4 Synthetic Benchmarks
In addition to the benchmarks presented in the main paper, we evaluated our method on the ZDT3 benchmark in Figure 5 The ZDT3 problem is widely used test in multi-objective optimization literature
Konakovic Lukovic et al. 2020).
demonstrate the effectiveness of our non-myopic methods on problems with a larger number of objectives, we evaluated our most computationally efficient method, BINOM, on various DTLZ benchmark problems Deb et al. 2005). Figure presents results for DTLZ-1 (d=11, K-4), DTLZ-1 (d=7, K=5), DTLZ-2 (d=8, K=4), and DTLZ-3 (d-9, K-5).
A.5 Additional Baselines
HVKG (Daulton et al. 2023 HVKG is notable for its cost-aware decoupled strategy, which assigns specific costs to each objective function and avoids evaluating all objectives in every iteration. To ensure fair
18

---

Published in Transactions on Machine Learning Research (04/2025)
Disc brake design
Four-bar truss design
Gear train design
Hypervolume
320
Hypervolume
e000
300
30
30
90
25
280
80
20
40
60
60
20
40
60
Iteration Number
Iteration Number
Iteration Number
Metal-organic framework design
1c10 Welded beam design
2100
Reinforced concrete beam design
1.80
Hypervolume
2000
Hypervolune
620000
Hypervolume
600000
1.78
1900
1y911.66
580000
1.76
1800
560000
1700
40 Iteration Number
20
40
60
Iteration Number
Iteration Number
BINOM-JES (H-2)
BINOM-JES 6)
BINOM-MES (H=4)
BINOM (H=2)
BINOM (H-6)
BINOM-JES --S (H=4)
BINOM MES (H=2)
BINOM-M MES (H=6)
BINOM (H=4)
Figure 4:
Hypervolume results for BINOM with MESMO (BINOM-MES) JESMO (BINOM-JES), and EHVI (BINOM) acquisition functions with lookahead horizon // E 62,
ZDT3
ZDT3
ZDT3
120
120
122.5
Nypervolume
120.0
115
BNON (l+4))
116
Dil
Hyto
117.5
BBUN [t|]]
114
jonnt
SNO [tt-]
110
Neve Masted (nw2)
wusc
115.00
SIOO [Hed]
SIOO (twd]
112
NHVO--oit (Hud)
NMMMO (H+B)
20
40
60
20
40
60
20
40
60
Iteration Number
Iteration Number
Iteration Number
(a)
(b)
(c)
Figure 5: With respect to the ZDT3 problem, Figure 5a shows the comparison of all baselines, Figure 5b shows the hypervolume comparison for BINOM with varying horizon lengths, and Figure 5c depicts the hypervolume comparison for NMMO-joint with different horizon lengths.
19

---

Published in Transactions on Machine Learning Research (04/2025)
1e10
(1121
1e13
(TLZ1(7,5)
1.00
Hypervolume
2.0
Hypervolum ervolume
JFSM
90.99.995
20.95
5
BNOM (1
BNOM 0t=4)
BIOOM
00.90
NHNG-jont
0
20
40
60
0
20
40
60
Iteration Number
Iteration Number
DTLZ2(8,4)
1e20
(9,53
1.000
Hypervolume
0.4
spprrolurge
0.995
20.990
20.88
0
20
40
60
0.980
0
20
40
60
Iteration Number
Iteration Number
Figure 6: Hypervolume results for variants of the DTLZ problem.
comparison between HVKG and our NMMO approaches, we adjusted the objective function costs to zero for all problems in our experiments, which allows all methods to operate under similar cost conditions. Morcover, to accurately measure the hypervolume indicalor, we implemented procedure where all objective functions of the selected inputs under the HVKG method are evaluated at each iteration. These expensive evaluations are then used calculate the hypervolume, ensuring that all methods' performances are assessed similarly.
NMMO-Nested. We include the NMMO-Nested with H 2 results in Figure Due to compulational cost, were not able to run experiments of NMMO-Nested with additional horizon values. As explained in section 5 while the NMMO-Nested method should theoretically outperform the other non-myopie baselines, its optimization is very time-consuming and inefficient in practice, leading to slightly subpar results.
MESMO. We included MESMO (Max-value Entropy Search for Multi-objective Optimization (Belakaria et al. 2019) in our baseline comparisons to assess its performance on various benchmarks. MESMO employs an advanced output space entropy search strategy, aiming to maximize the information gained about the Pareto front with each iteration. Despite its innovative approach, MESMO has been somewhat overshadowed in recent cvaluations by newer methods like JESMO, which combines the strengths of both output and input space entropy searches. We include comparison of how MESMO's performance stacks up against other state-of-the-art multi-objective Bayesian optimization techniques.
We use implementations from the BoTorch python package for all basclines (JESMO, EHVI, MESMO, HVKG).
A.6 Run times
The following are the computational complexities of proposed methods.
BINOM: O(KN"H) Where K is the number of objectives, N is the number of observations, and H is the lookahead horizon.
NMMO-Joint: O(KN2H2) This method has an additional factor of due to the joint optimization over the lookahead horizon.
NMMO-Nested: M( Where M the size of the discretized input space used for the nested optimization.
20

---

Published in Transactions on Machine Learning Research (04/2025)
Table 1: Benchmark details.
Problem name
d
Disc Brake Design
4
3
(6.1356, 6.3421, 12.9737)
Four Bar Truss Design
4
2
(2967 (2967.0243, 0.0383)
Gear Train Design
4
3
(6.6764, 59.0, 0.4633)
Reinforced Concrete Beam Design
3
(703.6860, 899.2291)
Welded Beam Design
(202.8569, 42.0653, 2111643.6209)
Mctal-Organic Framowork Design
(1.0, 1.0)
ZDT-3
9
(11.0, 11.0)
For context, the baseline EHVI method typically has a complexity of O(KN3).
While our non-myopic methods do introduce additional computational overhcad compared to myopic ap- proaches, the increase is not prohibitively high, especially considering the performance gains achieved and its intended use in budget- lndget-limited settings for scientific applications.
Table 2 shows the average run-time per itcration of cach bascline on multiple benchmarks. We utilized an NVIDIA Quadro RTX 6000 GPU with 24,576 MiB memory capacity. We provide the average and standard deviation of 65 iterations in one run the algorithm (in seconds). It is noteworthy that while the non-my methods take longer, they also provide better performance with a limited budget.
Table 2: Runtime comparison (seconds per iteration) rounded to the closest integer.
Reinforeed conerete beam design
Four-bar truss design
Gear train design
Metal-organic framework design
Method
EHVI
1 +
0 0
+ 0
5 + 0
JESMO
2 +
4
# 3
6 +
HVKG
47 + 20
32 + 5
52 + 14
102 + 27
4 + 2
9 +
13 d:3
17 17:22 2 17
BINOM
7 4
22 22+09 9
24 +
28 + 13
22 5
59 + 10
47 7
NMMO-Joint
52 + 19
35 +
75 10
55 1546
104 33
125 + 41
181 + 52
73 + 13
NMMO-Nested 2
560 + 58
802 + 62
616 + 151
754 + 141
A.7 Additional Experimental Details
Joint Entropy Search for Multi-Obe Objti-Objecttve Optimization (JESMO): We use the JESMO (Tu et al. 2022) algorithm implemented in the Bo'oorc python package Balandat et al 2020) baseline for comparison in our study. However, we encountered significant challenge with this approach: the JESMO implementation in BoTorch failed almost immediately when applied to many of our real-world multi-objective problems. This failure was primarily due to the algorithm's inability to identify a sufficient number of optimal Pareto sample points. Specifically, it struggled to find the required sample points. To accommodate this limitation and ensure the feasibility of using JESMO in our experiments, we adjusted the requirement a smaller set of optimal Pareto sample points. We determined the optimal number of Pareto sample points by monitoring
21

---

Published in Transactions on Machine Learning Research (04/2025)
Disc brake design
Four-bar truss design
Gear train design
100
Hypervolume
320
Hypervolume
40
Hypervolume
75
300
38
25
280
20
40
60
36
20
40
60
0
20
40
60
Iteration Number
Iteration Number
Iteration Number
Metal-organic framework design
1e10 Welded beam design
2100
Reinforced concrete beam design
1.80
Hypervolumm pepervolumee
620000
2000
Hypervolume
sypervolum
600000
1.78
1900
5880000
1.76
1800
560000
1700
20
40
60
1.74
20
40
60
Iteration Number
lteration Number
Iteration Number
BINOM (H=4)
HVKG
MESMO
NMMO-Joint (H=8)
NMMO-Nested (H=2)
EHVI
JESMO
NMMO-Joint (H=4)
Figure 7: Hypervolume results on real-world benchmarks including additional baselines
the instances where JESMO failed to identify a sufficient number of optimal points across various runs. This empirical approach allowed us to adjust the number of required sample points to a level where the algorithm could consistently perform without failure.
A.8 Limitations and Future Work.
Despite the proposed advancoments in solving the finite-horizon multi-objcctive SED problem using hy- pervolume improvement stralegies, several limitations remain in our approach, opening the way for future improvemenss. While the introduction of alternative lower bounds and the use of BEHVT are designed provide tractable alternative to solving the Bellman equation, these strategies raise questions about the tightness of the lower bounds and its ability to accurately represent the intractable optimal policy. This question is closely related to prior work on the adaptivity gap (Krause & Guestrin 2007). Given that we established the possibility of using the Bellman equation with HVI, promising future direction would be to build a multi-step expected HVI approach analogous to the multi-step expected improvement (Jiang et al. 2020b
B Hypervolume Improvement Additivity Proof
Let Vo be the initial Pareto front, and let (xt, yc) be the new data point added at time step 1, where & E [). We denote the updated Pareto front at time step Vr. The hypervolume improvement time step is given by:
HVL
=V -VVOL-L)))H) -VV)) HV
(17)
Now, let's consider the final hypervolume improvement after T time steps, we denote the final hypervolume improvement as HV Total
TVTTot -V())) HV(O6)
22

---

Published in Transactions on Machine Learning Research (04/2025)
We want to prove that:
t=T
IV
HVL
By expanding the summation, we get a telescoping series:
T HVL
Σ t=1
Σ - -V(O)-))((())))) t=1
(18)
+ (HV()) - HV()))) + -
(19)
The intermodiate terms of the series canceling out lead to:
IVI = -V) HV(O%)
(20)
Therefore,
oV Σ HVI
(21)
This proves that the final hypervolume improvement is indeed the summation of hypervolume improvements occurring at individual time steps.
The intuition behind this proof is that the hypervolume improvement at each time step represents the additional volume dominated by the updated Pareto front compared the previous Pareto front. By summing up these individual improvements, account for the total volume gained from the initial Pareto front to the final Pareto front.
23


Reviews:
[{"review_id": "EaIZPuVCTM", "reviewer": "Reviewer_eCCK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is nicely motivated and clearly discusses the challenges faced by non-myopic multi-objective BO.\n- The paper is well written, firstly listing the requirements on the scalarization approach and then proposing a scalarization method which satisfy these requirements.\n- It is commendable that the paper proposed several variants of their main algorithm, which offer different trade-offs between performance and computation. This offers practitioners significant flexibility when adopting their proposed method.\n- The paper provides discussions at many places, which are very useful for helping the reader understand the essential insights about the method and experimental findings.\n\nWeaknesses:\n- In the experiments, the maximum number of objectives $K$ is 3. I think it would help reinforce the empirical conclusions from the paper if the proposed methods are applied to problems with a larger number of objectives.\n- Just to clarify, does BINOM incur less computation or more computation than NMMO-Joint? Why is NMMO-Joint tested using a larger horizon than BINOM?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel method for non-myopic multi-objective Bayesian optimization (BO). The paper firstly identifies the requirements on the reward/utility function in order to satisfy the Bellman optimality principle in non-myopic BO, and then proposes to use HVI as the scalarization approach which satisfies these requirements. Several methods are proposed which differ in terms of the degree of approximation and the trade-off between computation and performance. Experimental results show that the proposed methods perform better than classical myopic BO methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "A few places may require some clarifications:\n- Equation (1), it is unclear what is the objective being maximized here.\n- Section 4.2.1 and Equation (10), what is $\\mathcal{Y}$ here? How is it chosen?\n- In Section 4.2.2., unless I missed it, I think the three variants of the proposed method are not clearly discussed in this section, more specifically, which algorithm name corresponds to which version. It is only discussed in Algorithm 1.", "broader_impact_concerns": "I do not have any concern about the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "l9LhYz4spR", "reviewer": "Reviewer_uVHU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Strengths*\n\n- This research project is highly relevant to the community. Multi-objective black-box optimization is an important topic for both researchers and practitioners, and the introduction of a non-myopic acquisition function in this setting is a valuable contribution.\n\n- The derivation of the proposed acquisition function is well contextualized within previous work and is presented in a relatively clear and easy-to-follow.\n\n- The selection of benchmark problems is particularly interesting. They look like compelling design problems that could significantly benefit from multi-objective optimization.\n\n*Weaknesses*\n\nThe primary weakness of this paper is the clarity of its presentation. The mathematical expressions require greater precision, and all elements critical to the proposed method should be explicitly defined. I strongly recommend a thorough and detailed revision of the presentation to enhance its clarity.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel acquisition function for multi-objective black-box optimization that is non-myopic—arguably the first in this setting. Specifically, the authors explore the use of Hypervolume Improvement as a utility function for optimal sequential decision-making. Building on the framework of [Shali Jiang et al. (ICML 2020)](https://proceedings.mlr.press/v119/jiang20b/jiang20b.pdf), they derive a *lower bound* on the true expected utility. The paper discusses three strategies for solving the multi-step Bellman equation:\n- Nested Non-Myopic Multi-Objective Acquisition Function – provides an exact computation of the lower bound.\n- Joint Non-Myopic Multi-Objective Optimization – a scalable alternative to the previous approach.\n- Batch-Informed Non-Myopic Multi-Objective Optimization – a fast, approximate version of Expected Hypervolume Improvement (EHVI).\n\nThe proposed methods are evaluated on at least six real-world experimental design problems, with promising results. Additionally, empirical results for different horizons and a computational runtime comparison are presented. An ablation study of information-gain within the proposed framework is also investigated.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Avoid making claims about information gain in this setting. There is no compelling evidence demonstrating that information gain is monotonic or additive in this context, and including such suggestions weakens the main point of this paper. I would adhere to the principle of \"one idea, one paper.\" If you intend to propose a new information-theoretic-inspired acquisition function, it should be thoroughly evaluated and discussed in depth, and not in two paragraphs.\n\n- Consider improving the mathematical definitions. For example, the function $\\alpha(x)$ is overloaded so many times that it gets confusing. Consider adding a new terminology for each definition such as $\\alpha_{BINOM} (x)$. You can use a similar strategy that the authors (Shali Jiang et al.) used in equation 5 of this paper [Efficient Nonmyopic Active Search (ICML 2017)](https://proceedings.mlr.press/v70/jiang17d/jiang17d.pdf). In fact, this paper I think was the first to point out that you can do a batch selection to approximate a non-myopic sequential decision making (it assumes independency when you do so). \n\n- Correct the citations by using \\citep and \\citet appropriately and consistently.\n\n- In Figure 1, either use a log scale for the bottom row or normalize all objectives for consistency.\n\n- Address typographical errors, particularly in Appendix B, where they are prevalent.\n\n- In Section 5, the evaluation metric should be referred to as Hypervolume, not HVI.\n\n- I don't see a formal definition of hypervolume. \n\n- Equation 1 is not precise. $f_1(x) ... f_2(x) $ could be anything. Specifically, a casual reader my infer that this is an addition. \n\n- $D_t$ is never defined as a set of $t$ observations $(x, y)$. In some contexts, *observation* is synonymous of the response variable $y$.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KJlTXOoGGI", "reviewer": "Reviewer_GEYU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n1. The experimental results suggest that the proposed non-myopic algorithms significantly outperform baselines in the tested settings.\n2. The experiments are comprehensive, with many objective functions, runtime analyses, and ablation studies.\n2. The paper is written clearly. The ideas developed are straightforward to follow.\n\n**Weaknesses**:\n1. The paper lacks significant novelty. The algorithms are largely straightforward combinations of ideas developed in previous non-myopic and multi-objective BO papers.\n\nOverall, the paper is likely to be of interest to BO practitioners dealing with multi-objective BO problems, while it is of little interest from a theoretical perspective.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes non-myopic algorithms for multi-objective Bayesian optimization. It uses expected hypervolume improvement as the scalarization metric with previous non-myopic algorithms for single-objective Bayesian optimization. It proposes 3 algorithms, NMO-Nested, NMO-Joint, and BINOM, listed in increasing levels of approximation error (and decreasing levels of computational complexity). The algorithms are compared to several baselines with several objective functions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Would strengthen work**:\n1. It is not clear what Equation (1) means. How is this the Pareto hypervolume indicator with respect to a reference $r$ as described in the previous paragraph?\n2. In the first paragraph of page 4, should Equation 5 refer to Equation 6?\n3. The citation Osbourne & Osbourne (2010) is wrong, both refer to the same person.\n4. In lines 6 and 8 of Algorithm 1, the argmax's should include all arguments to the function (other than $D_t$), otherwise the statement is not well-defined.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: A summary of the strength and weaknesses based on the reviews and rebuttal is provided below:

**STRENGTHS**

(1) The problem of nonmyopic MOBO and its challenges are well motivated.

(2) The paper is well-written and incorporates important insights and experimental findings.

(3) Several variants of the main algorithm are proposed, each with a different trade-off between performance vs. computation, allowing users to have the flexibility of options.

(4) The experimental results show that the proposed algorithms outperform the baselines.


**WEAKNESSES**

(1) Experimental results on a larger number of objectives are initially lacking. The authors have provided them in the rebuttal to address this concern.

(2) There were initial concerns on clarity for which the authors have clarified in the rebuttal.

Since the main concerns of the reviewers have been addressed, the pros outweigh the cons.


For completeness, the authors can consider citing and commenting on the following earlier references on nonmyopic BO:

Nonmyopic Gaussian Process Optimization with Macro-Actions. AISTATS 2020.

Sequential Bayesian optimisation for spatial-temporal monitoring. UAI 2014.

Gaussian process planning with Lipschitz continuous reward functions: Towards unifying Bayesian optimization, active learning, and beyond. AAAI 2016.
Decision: Accept (poster)