Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement

1 Introduction . Black-box optimization problems that involve multiple competing noisy objectives are ubiquitous in science and engineering . For example , a real-time communications service may be interested in tuning the parameters of a control policy to adapt video quality in real time in order to maximize video quality and minimize latency [ 10 , 17 ] . In robotics , scientists may seek to design hardware components that maximize locomotive speed and minimize energy expended [ 8 , 38 ] . In agriculture , development agencies may seek to balance crop yield and environmental impact [ 28 ] . For such multi-objective optimization ( MOO ) problems , there typically is no single solution that is best with respect to all objectives . Rather , the goal is to identify the Pareto frontier : a set of optimal trade-offs such that improving one objective means deteriorating another . In many cases , the objectives are expensive to evaluate . For instance , randomized trials used in agriculture and the internet industry may take weeks or months to conduct and incur opportunity costs , and manufacturing and testing hardware is both costly and time-consuming . Therefore , it is imperative to be able to identify good trade-offs with as few objective evaluations as possible . Bayesian optimization ( BO ) , a method for efficient global black-box optimization , is often used to tackle such problems . BO employs a probabilistic surrogate model in conjunction with an acquisition function to navigate the trade-off between exploration ( evaluating designs with high uncertainty ) and exploitation ( evaluating designs that are believed to be optimal ) . Although a significant number of works have explored multi-objective Bayesian optimization ( MOBO ) , most available methods 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) [ 3 , 39 , 51 , 60 ] do not take into account the fact that , in practice , observations are often subject to noise . For example , results of an A/B test are highly variable due to heterogeneity in the underlying user population and other factors . Agricultural trials are affected by the stochastic nature of plant growth and environmental factors such as soil composition or wind currents . In robotics , devices are subject to manufacturing tolerances , and observations of quantities such as locomotive speed and efficiency may be corrupted by measurement error from noisy sensors and environmental factors such as temperature or surface friction . While previous work has shown that a principled treatment of noisy observations can significantly improve optimization performance in the single-objective case [ 24 , 37 ] , this issue is understudied in the multi-objective setting . Furthermore , many applications in which evaluations take a long time require evaluating large batches of candidates in parallel in order to achieve reasonable throughput . For example , when firms optimize systems via A/B tests , it may take several weeks to test any particular configuration . Because of this , large batches of candidate policies are tested simultaneously [ 36 ] . In biochemistry and materials design , dozens of tests can be conducted parallel on a single microplate [ 63 ] . Even in sophisticated high throughput chemistry settings , these batches may take several hours or days to set up and evaluate [ 42 ] . Most existing MOBO methods , however , are either designed for purely sequential optimization [ 3 , 51 ] or do not scale well to large batch sizes [ 11 ] . Contributions : In this work , we propose a novel MOBO algorithm , based on expected hypervolume improvement ( EHVI ) , that scales to highly parallel evaluations of noisy objectives . Our approach is made possible by a general-purpose , differentiable , cached box decomposition ( CBD ) implementation that dramatically speeds up critical computations needed to account for uncertainty introduced by noisy observations and generate new candidate points for highly parallel batch or asynchronous evaluation . In particular , our CBD-based approach solves the fundamental problem of scaling parallel EHVI-based methods to large batch sizes , reducing time and space complexity from exponential to polynomial . Our proposed algorithm , noisy expected hypervolume improvement ( NEHVI ) , is the onestep Bayes-optimal policy for hypervolume improvement and provides state-of-the-art performance across a variety of benchmarks . To our knowledge , our work provides the most extensive evaluation of noisy parallel MOBO to date . A high-quality implementation of qNEHVI , as well as many of the baselines considered here , will be made available as open-source software upon publication . 2 Preliminaries . Our goal is to find the set of optimal designs x over a bounded set X ⊂ Rd that maximize one or more objectives f ( x ) ∈ RM , with no known analytical expression nor gradient information of f . Multi-Objective Optimization ( MOO ) aims to identify the set of Pareto optimal objective tradeoffs . We say a solution f ( x ) = [ f ( 1 ) ( x ) , ... , f ( M ) ( x ) ] dominates another solution f ( x ) f ( x′ ) if f ( m ) ( x ) ≥ f ( m ) ( x′ ) for m = 1 , ... , M and ∃m ∈ { 1 , ... , M } s.t . f ( m ) ( x ) > f ( m ) ( x′ ) . We define the Pareto frontier as P∗ = { f ( x ) : x ∈ X , @ x′ ∈ X s.t . f ( x′ ) f ( x ) } , and denote the set of Pareto optimal designs as X ∗ = { x : f ( x ) ∈ P∗ } . Since the Pareto frontier ( PF ) is often an infinite set of points , MOO algorithms usually aim to identify a finite approximate PF P . A natural measure of the quality of a PF is the hypervolume of the region of objective space that is dominated by the PF and bounded from below by a reference point . Provided with the approximate PF , the decision-maker can select a particular Pareto optimal trade-off according to their preferences . Bayesian Optimization ( BO ) is a sample-efficient optimization method that leverages a probabilistic surrogate model to make principled decisions to balance exploration and exploitation [ 19 , 50 ] . Typically , the surrogate is a Gaussian Process ( GP ) , a flexible , non-parametric model known for its well-calibrated predictive uncertainty [ 47 ] . To decide which points to evaluate next , BO employs an acquisition function α ( · ) that specifies the value of evaluating a set of new points x based on the surrogate ’ s predictive distribution at . While evaluating the true black-box function f is timeconsuming or costly , evaluating the surrogate is cheap and relatively fast ; therefore , numerical optimization can be used to find the maximizer of the acquisition function x∗ = arg maxx∈X α ( x ) to evaluate next on the black-box function . BO sequentially selects new points to evaluate and updates the model to incorporate the new observations . Evolutionary algorithms ( EAs ) such as NSGA-II [ 12 ] are a popular choice for solving MOO problems ( see Zitzler et al . [ 67 ] for a review of various other approaches ) . However , EAs generally suffer from high sample complexity , rendering them infeasible for optimizing expensive-to-evaluate black-box functions . Multi-objective Bayesian optimization ( MOBO ) , which combines a Bayesian surrogate with an acquisition function designed for MOO , provides a much more sample-efficient alternative . 3 Related Work . Methods based on hypervolume improvement ( HVI ) seek to expand the volume of the objective space dominated by the Pareto frontier . Expected hypervolume improvement ( EHVI ) [ 16 ] is a natural extension of the popular expected improvement ( EI ) [ 29 ] acquisition function to the MOO setting . Recent work has led to efficient computational paradigms using box decomposition algorithms [ 59 ] and practical enhancements such as support for parallel candidate generation and gradient-based acquisition optimization [ 11 , 58 ] . However , EHVI still suffers from some limitations , including ( i ) the assumption that observations are noise-free , and ( ii ) the exponential scaling of its batch variant , qEHVI , in the batch size q , which precludes large-batch optimization . DGEMO [ 39 ] is a recent method for parallel MOBO that greedily maximizes HVI while balancing the diversity of the design points being sampled . Although DGEMO scales well to large batch sizes , it does not account for noisy observations . TSEMO [ 5 ] is a Thompson sampling ( TS ) heuristic that can acquire batches of points by optimizing a random fourier feature ( RFF ) [ 46 ] approximation of a GP surrogate using NSGA-II and selecting a subset of points from the EA ’ s population to sequentially greedily maximize HVI . This heuristic approach for maximizing HVI currently has no theoretical guarantees and relies on zeroth-order optimization methods , which tend to be slower and exhibit worse optimization performance than gradient-based approaches . Entropy-based methods such as PESMO [ 25 ] , MESMO [ 3 ] , and PFES [ 51 ] are an alternative to EHVI . Of these three methods , PESMO is the only one that accounts for observation noise . However , PESMO involves intractable entropy computations and therefore relies on complex approximations , as well as challenging and time-consuming numerical optimization procedures [ 25 ] . Garrido-Merchán & Hernández-Lobato [ 21 ] recently proposed an extension to PESMO that supports parallel candidate generation . However , the authors of this work provide limited evaluation and have not provided code to reproduce their results.1 MOO can also be cast into a single-objective problem by applying a random scalarization of the objectives . ParEGO maximizes the expected improvement using random augmented Chebyshev scalarizations [ 32 ] . MOEA/D-EGO [ 64 ] extends ParEGO to the batch setting using multiple random scalarizations and the genetic algorithm MOEA/D [ 65 ] to optimize these scalarizations in parallel . Recently , qParEGO , another batch variant of ParEGO was proposed that uses compositional Monte Carlo objectives and sequential greedy candidate selection [ 11 ] . Additionally , the authors proposed a noisy variant , qNParEGO , but the empirical evaluation of that variant was limited . TS-TCH [ 45 ] combines random Chebyshev scalarizations with Thompson sampling [ 54 ] , which is naturally robust to noise when the objective is scalarized . Golovin & Zhang [ 23 ] propose to use a hypervolume scalarization with the property that the expected value of the scalarization over a specific distribution of weights is equivalent to the hypervolume indicator . The authors propose a upper confidence bound algorithm using randomly sampled weights , but provide a very limited empirical evaluation . Many prior attempts by the simulation community to handle MOO with noisy observations found that accounting for the noise did not improve optimization performance : Horn et al . [ 26 ] suggest that the best approach is to ignore noise , and Koch et al . [ 33 ] concluded that further research was needed to determine if modeling techniques such as re-interpolation could improve BO performance with noisy observations . In contrast , we find that accounting for noise does substantially improve performance in noisy settings . Lastly , previous works have considered methods for quantifying and monitoring uncertainty in the Pareto frontiers during the optimization [ 4 , 7 ] . In contrast , we provide a solution to performing MOBO in noisy settings , rather than purely reasoning about the uncertainty in the Pareto frontier .

Summary: The authors describe an BO algorihm for multi-objective optimization using hypervolume improvement. The main contribution is the incorporation of noise into the estimation of the improvement measure. The algorithm is evaluated on four functions and theoretical analysis is performed. In the experiments, the algorithm performed best on the set of functions.