Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement

1 Introduction . Black-box optimization problems that involve multiple competing noisy objectives are ubiquitous in science and engineering . For example , a real-time communications service may be interested in tuning the parameters of a control policy to adapt video quality in real time in order to maximize video quality and minimize latency [ 10 , 17 ] . In robotics , scientists may seek to design hardware components that maximize locomotive speed and minimize energy expended [ 8 , 38 ] . In agriculture , development agencies may seek to balance crop yield and environmental impact [ 28 ] . For such multi-objective optimization ( MOO ) problems , there typically is no single solution that is best with respect to all objectives . Rather , the goal is to identify the Pareto frontier : a set of optimal trade-offs such that improving one objective means deteriorating another . In many cases , the objectives are expensive to evaluate . For instance , randomized trials used in agriculture and the internet industry may take weeks or months to conduct and incur opportunity costs , and manufacturing and testing hardware is both costly and time-consuming . Therefore , it is imperative to be able to identify good trade-offs with as few objective evaluations as possible . Bayesian optimization ( BO ) , a method for efficient global black-box optimization , is often used to tackle such problems . BO employs a probabilistic surrogate model in conjunction with an acquisition function to navigate the trade-off between exploration ( evaluating designs with high uncertainty ) and exploitation ( evaluating designs that are believed to be optimal ) . Although a significant number of works have explored multi-objective Bayesian optimization ( MOBO ) , most available methods 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) [ 3 , 39 , 51 , 60 ] do not take into account the fact that , in practice , observations are often subject to noise . For example , results of an A/B test are highly variable due to heterogeneity in the underlying user population and other factors . Agricultural trials are affected by the stochastic nature of plant growth and environmental factors such as soil composition or wind currents . In robotics , devices are subject to manufacturing tolerances , and observations of quantities such as locomotive speed and efficiency may be corrupted by measurement error from noisy sensors and environmental factors such as temperature or surface friction . While previous work has shown that a principled treatment of noisy observations can significantly improve optimization performance in the single-objective case [ 24 , 37 ] , this issue is understudied in the multi-objective setting . Furthermore , many applications in which evaluations take a long time require evaluating large batches of candidates in parallel in order to achieve reasonable throughput . For example , when firms optimize systems via A/B tests , it may take several weeks to test any particular configuration . Because of this , large batches of candidate policies are tested simultaneously [ 36 ] . In biochemistry and materials design , dozens of tests can be conducted parallel on a single microplate [ 63 ] . Even in sophisticated high throughput chemistry settings , these batches may take several hours or days to set up and evaluate [ 42 ] . Most existing MOBO methods , however , are either designed for purely sequential optimization [ 3 , 51 ] or do not scale well to large batch sizes [ 11 ] . Contributions : In this work , we propose a novel MOBO algorithm , based on expected hypervolume improvement ( EHVI ) , that scales to highly parallel evaluations of noisy objectives . Our approach is made possible by a general-purpose , differentiable , cached box decomposition ( CBD ) implementation that dramatically speeds up critical computations needed to account for uncertainty introduced by noisy observations and generate new candidate points for highly parallel batch or asynchronous evaluation . In particular , our CBD-based approach solves the fundamental problem of scaling parallel EHVI-based methods to large batch sizes , reducing time and space complexity from exponential to polynomial . Our proposed algorithm , noisy expected hypervolume improvement ( NEHVI ) , is the onestep Bayes-optimal policy for hypervolume improvement and provides state-of-the-art performance across a variety of benchmarks . To our knowledge , our work provides the most extensive evaluation of noisy parallel MOBO to date . A high-quality implementation of qNEHVI , as well as many of the baselines considered here , will be made available as open-source software upon publication . 2 Preliminaries . Our goal is to find the set of optimal designs x over a bounded set X ⊂ Rd that maximize one or more objectives f ( x ) ∈ RM , with no known analytical expression nor gradient information of f . Multi-Objective Optimization ( MOO ) aims to identify the set of Pareto optimal objective tradeoffs . We say a solution f ( x ) = [ f ( 1 ) ( x ) , ... , f ( M ) ( x ) ] dominates another solution f ( x ) f ( x′ ) if f ( m ) ( x ) ≥ f ( m ) ( x′ ) for m = 1 , ... , M and ∃m ∈ { 1 , ... , M } s.t . f ( m ) ( x ) > f ( m ) ( x′ ) . We define the Pareto frontier as P∗ = { f ( x ) : x ∈ X , @ x′ ∈ X s.t . f ( x′ ) f ( x ) } , and denote the set of Pareto optimal designs as X ∗ = { x : f ( x ) ∈ P∗ } . Since the Pareto frontier ( PF ) is often an infinite set of points , MOO algorithms usually aim to identify a finite approximate PF P . A natural measure of the quality of a PF is the hypervolume of the region of objective space that is dominated by the PF and bounded from below by a reference point . Provided with the approximate PF , the decision-maker can select a particular Pareto optimal trade-off according to their preferences . Bayesian Optimization ( BO ) is a sample-efficient optimization method that leverages a probabilistic surrogate model to make principled decisions to balance exploration and exploitation [ 19 , 50 ] . Typically , the surrogate is a Gaussian Process ( GP ) , a flexible , non-parametric model known for its well-calibrated predictive uncertainty [ 47 ] . To decide which points to evaluate next , BO employs an acquisition function α ( · ) that specifies the value of evaluating a set of new points x based on the surrogate ’ s predictive distribution at . While evaluating the true black-box function f is timeconsuming or costly , evaluating the surrogate is cheap and relatively fast ; therefore , numerical optimization can be used to find the maximizer of the acquisition function x∗ = arg maxx∈X α ( x ) to evaluate next on the black-box function . BO sequentially selects new points to evaluate and updates the model to incorporate the new observations . Evolutionary algorithms ( EAs ) such as NSGA-II [ 12 ] are a popular choice for solving MOO problems ( see Zitzler et al . [ 67 ] for a review of various other approaches ) . However , EAs generally suffer from high sample complexity , rendering them infeasible for optimizing expensive-to-evaluate black-box functions . Multi-objective Bayesian optimization ( MOBO ) , which combines a Bayesian surrogate with an acquisition function designed for MOO , provides a much more sample-efficient alternative . 3 Related Work . Methods based on hypervolume improvement ( HVI ) seek to expand the volume of the objective space dominated by the Pareto frontier . Expected hypervolume improvement ( EHVI ) [ 16 ] is a natural extension of the popular expected improvement ( EI ) [ 29 ] acquisition function to the MOO setting . Recent work has led to efficient computational paradigms using box decomposition algorithms [ 59 ] and practical enhancements such as support for parallel candidate generation and gradient-based acquisition optimization [ 11 , 58 ] . However , EHVI still suffers from some limitations , including ( i ) the assumption that observations are noise-free , and ( ii ) the exponential scaling of its batch variant , qEHVI , in the batch size q , which precludes large-batch optimization . DGEMO [ 39 ] is a recent method for parallel MOBO that greedily maximizes HVI while balancing the diversity of the design points being sampled . Although DGEMO scales well to large batch sizes , it does not account for noisy observations . TSEMO [ 5 ] is a Thompson sampling ( TS ) heuristic that can acquire batches of points by optimizing a random fourier feature ( RFF ) [ 46 ] approximation of a GP surrogate using NSGA-II and selecting a subset of points from the EA ’ s population to sequentially greedily maximize HVI . This heuristic approach for maximizing HVI currently has no theoretical guarantees and relies on zeroth-order optimization methods , which tend to be slower and exhibit worse optimization performance than gradient-based approaches . Entropy-based methods such as PESMO [ 25 ] , MESMO [ 3 ] , and PFES [ 51 ] are an alternative to EHVI . Of these three methods , PESMO is the only one that accounts for observation noise . However , PESMO involves intractable entropy computations and therefore relies on complex approximations , as well as challenging and time-consuming numerical optimization procedures [ 25 ] . Garrido-Merchán & Hernández-Lobato [ 21 ] recently proposed an extension to PESMO that supports parallel candidate generation . However , the authors of this work provide limited evaluation and have not provided code to reproduce their results.1 MOO can also be cast into a single-objective problem by applying a random scalarization of the objectives . ParEGO maximizes the expected improvement using random augmented Chebyshev scalarizations [ 32 ] . MOEA/D-EGO [ 64 ] extends ParEGO to the batch setting using multiple random scalarizations and the genetic algorithm MOEA/D [ 65 ] to optimize these scalarizations in parallel . Recently , qParEGO , another batch variant of ParEGO was proposed that uses compositional Monte Carlo objectives and sequential greedy candidate selection [ 11 ] . Additionally , the authors proposed a noisy variant , qNParEGO , but the empirical evaluation of that variant was limited . TS-TCH [ 45 ] combines random Chebyshev scalarizations with Thompson sampling [ 54 ] , which is naturally robust to noise when the objective is scalarized . Golovin & Zhang [ 23 ] propose to use a hypervolume scalarization with the property that the expected value of the scalarization over a specific distribution of weights is equivalent to the hypervolume indicator . The authors propose a upper confidence bound algorithm using randomly sampled weights , but provide a very limited empirical evaluation . Many prior attempts by the simulation community to handle MOO with noisy observations found that accounting for the noise did not improve optimization performance : Horn et al . [ 26 ] suggest that the best approach is to ignore noise , and Koch et al . [ 33 ] concluded that further research was needed to determine if modeling techniques such as re-interpolation could improve BO performance with noisy observations . In contrast , we find that accounting for noise does substantially improve performance in noisy settings . Lastly , previous works have considered methods for quantifying and monitoring uncertainty in the Pareto frontiers during the optimization [ 4 , 7 ] . In contrast , we provide a solution to performing MOBO in noisy settings , rather than purely reasoning about the uncertainty in the Pareto frontier .

Summary: The paper proposes a novel method to perform multi-objective BO based on Expected HyperVolume Improvement. Specifically, the authors argue that taking into account noisy observations and dealing with that noise improves the optimisation as they introduce a new acquisition function NEHVI. They moreover introduce a parallelisation framework enabling scalable batch acquisition in their novel acquisition function. This paper is well written. The ability to use EHVI in a noisy environment is useful in practice and in my opinion, the improvement in scalability is important for the community. However, I would have liked to see experiments led with varying numbers of objectives as it is known that it’s a main bottleneck of EHVI in practice.