Risk-averse Heteroscedastic Bayesian Optimization
Authors: Anastasia Makarova
Ilnura Usmanova
Ilija Bogunovic
Andreas Krause
Venue: NeurIPS 2021 Poster (2021)
Keywords: bayesian optimization
gaussian process
robustness
black-box

Abstract: Many black-box optimization tasks arising in high-stakes applications require risk-averse decisions. The standard Bayesian optimization (BO) paradigm, however, optimizes the expected value only.  We generalize BO to trade mean and input-dependent variance of the objective, both of which we assume to be unknown a priori.  In particular, we propose a novel risk-averse heteroscedastic Bayesian optimization algorithm (RAHBO) that aims to identify a solution with high return and low noise variance, while learning the noise distribution on the fly.  To this end, we model both expectation and variance as (unknown) RKHS functions, and propose a novel risk-aware acquisition function.  We bound the regret for our approach and provide a robust rule to report the final decision point for applications where only a single solution must be identified. We demonstrate the effectiveness of RAHBO on synthetic benchmark functions and hyperparameter tuning tasks.

Risk-averse Heteroscedastic Bayesian Optimization
Anastasiia Makarova ETH Zürich anmakaro@ethz.c
Ilnura Usmanova
Ilija Bogunovic ETH Zürich ilijabEethz.ch ch
Andreas Krause ETH Zürich krauseaßethz.co ch
ETH Zürich ilmurau@ethz.ch
Abstract
Many black-box optimization tasks arising in high-stakes applications require risk- averse decisions. The standard Bayesian optimization (BO) paradigm, however, optimizes the expected value only. We generalize BO to trade mean and input- dependent variance the objective, both of which we assume to be unknown priori. In particular, optimization algorithm (RAHBO) that aims to identify a solution with high return and low noise variance, while learning the noise distribution the fly. To this end, we model both expectation and variance as (unknown) RKHS functions, and propose novel risk-awart acquisition function. We bound the regret for approach and provide a robust rule to report the final decision point for applications where only single solution must be identified. We demonstrate the effectiveness of RAHBO on synthetic benchmark functions and hyperparameter tuning tasks.
propose novel risk-averse heteroscedastic Bayesian
1 Introduction
Black-box optimization tasks arise frequently in high-stakes applications such drug and material discovery [17, 22, 28), genetics [16, 27], robotics [5, 12, 25), hyperparameter tuning of complex learning systems [13, 21, 34], to name a few. In many of these applications, there is often trade-off between achieving high utility and minimizing risk. Moreover, uncertain and costly evaluations are an inherent part of black-box optimization tasks, and modern learning methods need to handle these aspects when balancing between the previous two objectives.
Bayesian optimization (BO) is a powerful framework for optimizing such costly black-box functions from noisy zeroth-order evaluations. Classical BO approaches are typically risk-neutral as they seek to optimize the expected function value only. In practice. however, two different solutions might attain similar expected function values, but one might produce significantly noisier realizations. This of major importance when comes to actual deployment of the found solutions. For example, when selecting hyperparameters of a machine learning algorithm, we might prefer configurations that lead to slightly higher test errors but at the same time lead to smaller variance.
In this paper, we generalize BO to trade off mean and input-dependent noise variance when sequen- tially querying points and outputting final solutions. We introduce a practical setting where both the black-box objective and input-depent needs to estimate them on the fly. We propose a novel optimistic risk-averse algorithm RAHBO that makes sequential decisions by simultancously balancing between exploration (learning about uncertain actions), exploitation (choosing actions that lead high gains) and risk (avoiding unreliable actions). We bound the cumulative regret of RAHBO as well as the number of samples required to output a single near-optimal risk-averse solution. In our experiments, we demonstrate the rver-everse performance of our algorithm and show that standard BO methods can severely fail in applications where reliability of the reported solutions of utmost importance.
noise variance are unknown a priori, and the learner
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

RAHBO a=
Empirica frequency
RAJIBO US a=1
Roo. x))
300
0
10
31
30
15
Variance (*o,
(a) Unknown objective f
(b) Unknown variance p²
(c) Histogram of variance
When there a choice between identical optima with different noise level, standard BO tends query points corresponding higher noise. (a) Unknown objective with 3 global maxima marked as (A, B, C); (b) Heteroscedastic noise variance over the same domain: the noise level (A, B, C) varies according to the sigmoid function; (c) Empirical variance distribution all points acquired during BO procedure (over 9 experiments with different seeds). The three bumps correspond the three global optima with different noise variance. RAHBO dominates in choosing the risk-averse optimum, consequently yielding lower risk-averse regret in Figure 5a.
Figure 1:
Related work. Bayesian optimization (BO) [26] refers to approaches for optimizing noisy black- box objective that is typically expensive to evaluate. A great number of BO methods have been developed over the years, including significant number of variants of popular algorithms such as GP-UCB [35], Expected Improvement [26], and Thompson Sampling [14]. While the focus of standard BO approaches mainly on trading-off exploration vs. exploitation and optimizing for the expected performance, in this work, we additionally focus on the risk that is involved when working with noisy objectives, illustrated Figure 1.
The vast majority of previous BO works assume (sub-) Gaussian and homoscedastic noise (i.e., imput independent and of some known fixed level). Both assumptions be restrictive in practice. For example, as demonstrated in [15], the majority hyperparameter tuning tasks exhibit heteroscedas- ticity. few works relax the first assumption and consider, e.g., heavy-tailed noise models [31] and adversarially corrupted observations [8]. The second assumption is typically generalized via heteroscedastic Gaussian process (GP), allowing explicit dependence of the noise distribution the evaluation points [6, 7, 10, 20J. Similarly, in this work, we consider heteroscedastic GP models, but unlike the previous works, we specifically focus the risk that is associated with querying and reporting noisy points.
Several works have recently considered robust and risk-averse aspects in BO. Their central focus is on designing robust strategies and protecting against the change/shift in uncontrollable covariates. They study various notions including worst-case robustness [9]. distributional robustness [19, 30], robust mixed strategies [33] and other notions risk-aversion [11, 1, 18, 29), and while some of them report robust regret guarantees, their focus is primarily the robustness in the homoscedastic GP setting. Instead, in our setting, we account for the risk that comes from the realization of random noise with unknown distribution. Rather than optimizing the expected performance, in risk-averse setting, we prefer inputs with lower variance. To this end, we incorporate the learning of the noise distribution into the optimization procedure via mean-variance objective. The closest our setting risk-aversion with respect to noise in multi-armed bandits [32]. Their approach, however, fails to exploit correlation in rewards among similar arms.
Contributions. We propose a novel Risk-averse Heteroscedastic Bayesian optimization (RAHBO) approach based on the optimistic principle that trades off the expectation and uncertainty of the arean-vriance objective function. We model both the objective and variance as (unknown) functions belonging to RKHS space of functions, and propose a practical risk-aware algorithm in heteroscedastic setting. In our theoretical analysis, we establish rigorous sublinear regret guarantees for our algorithm, and provide robust reporting rule for applications where only single solution required. We demonstrate the effectiveness RAHBO on synthetic benchmarks, as well hyperparameter tuning tasks for the Swiss free-clectron laser and machine learning model.

---

2
Problem setting
Let X' be given compact set of inputs (X' C R. for some d N). We consider a problem of sequentially interacting with a fixed and unknown objective f 22 IR At every round of this procedure, the learner selects an action Ia X, and obtains a noisy observation
= [xx) + E(tee;
(I)
where {(z;) is zero-mean noise independent across different time steps t. In this work, we consider Goossaussian heteroscedastic noise that depends on the query location.
Definition 1. zero-mean real-valued random variable & --sub-Gaussian if there exists variance- proxy p² such that AA R, E[cAE] Se -42
For a sub-Gaussian &. its variance Var(c] lower bounds any valid variance-pront p. ie., Var(e] & p². In case War[g] = p² holds, { is said to be strictly --sub-Gaussien Besides zero-mean Gaussian random variables, most standard symmetric bounded random variables (e.g.. Berouuli,, beta, uniform, binomial) are strictly sub-Gaussian (see [2, Proposition I.1]). Throughout the paper, consider sub-Gaussian noise, and in Section 3.3, we specialize to the case of strictly sub-Gaussian noise.
Optimization objective. Unlike the previous works that mostly focus on sequential optimization of f in the homoscedastic noise case, in this work, we consider the trade-off between risk and return in the heteroscedastic case. While there exist number of risk-averse objectives, consider the simple and frequently used mean-variance objective (MV) [32]. Here, the objective value at a E X is trade-off between the (mean) return fxx) and the risk expressed by its variance-proxy p2()):
MV(x) = f(x) ap²(x))
(2)
where a 2 is so-called coefficient of absolute risk tolerance. In this work, we assume a is fixed and known to the learner. In the case of a = 0, maximizing MV(x) coincides with the standard BO objective.
Performance metrics. We aim to construct a sequence of input evaluations 2e that eventually maximizes the risk-averse objective MV(x). To assess this convergence, we consider two metrics. The first metric corresponds to the notion of cumulative regret similar to the one used in standard BO and bandits. Here, the learner's goal is maximize its risk- cumulative reward over a time horizon T, or equivalently minimize its risk-averse cumulative regret:
T
RT Σ [Mv((a") - -
(3)
where x² arg MV.x). A sublinear growth of Ry with T implies vanishing average regret Ry/TT + as T 20. Intuitively, this implies the existence of some I such that MV(x.) is arbitrarily close to the optimal value MV")").
The second metric is used when the learner seeks to simultaneously minimize the number of expensive function evaluations T. Namely, for a given accuracy O, report single "good" risk-averse point ēT E X' after a total of T rounds, that satisfies:
MV 2 MV(x") c.
(4)
Both metrics are important for choosing risk-averse solutions and which one is preferred depends on the application at hand. For example, risk-averse cumulative regret Ry might be of a greater interest in online recommendation systems, while reporting single point with high MV value might be more suitable when tuning machine learning hyperparameters. We consider both performance metrics in our experiments.
Regularity assumptions. We consider standard smoothness assumptions [9, 35] when it comes to assume that f(-) belongs to reproducing
the unknown function f X + I.. In particular, kernel Hilbert space (RKHS) H. (a space of smooth and real-valued functions defined on X),, i.e., H.. induced by a kernel function s(..)). We also assume that <(2, x²) for every x. X. Moreover, the RKHS norm of f(-) is assumed to be bounded [| I By for some fixed constant By > O. We assume that the noise {(x) is p(x) sub-Gaussian with variance-pros proxy (xx) uniformly bounded p(z) e [g, Ø for some constant values ō 2 g 0.
3

---

3
Algorithms
We first recall the Gaussian process (GP) based framework for sequential learning of RKHS functions from observations with heteroscedastic noise. Then, in Section 3.2, we consider a simple risk-averse Bayesian optimization problem with known variance-proxy, and later on in Section 3.3, we focus on our main problem setting in which the variance-proxy is unknown.
3.1 Bayesian optimization with heteroscedastic noise
Before addressing the risk-averse objective, we briefly recall the standard GP-UCB algorithm [35] in the setting of heteroscedastic sub-Gaussian noise. The regularity assumptions permit the construction of confidence bounds via GP model. Particularly, to decide which point to query at every round, GP-
UCB makes use of the posterior GP mean and variance denoted by pe(-) and o.()). respectively, They
are computed based on the previous measurements V1.t = [M] [31......34 Vel and the given kernel k(;,))
(x) = <a(=)² (Ke " "yy.
(5)
(x) 1 (M(x, x) - (x) [K, AE,) (x())
(6)
where E, diag(p²(x1) (... (are), (Kt)ay x(5;, (x)²²²
À 0 and prior modelling assumptions are E(:) (O, p²)()²)) (-)) and GP(O,
At time f, GP-UCB maximizes the upper confidence bound of f.)).
Zy E arg max #EX
(x) + 1
=nucb/
If the noise (a(e)) heteroscedastic and (aae)--ss---ussss the following confidence bounds hold:
Lemma 1 (Lemmaa 7 in [20]). Let f E Has and pe(-) and of (-) be defined as in Eqs. (5) and (6)
with X O. Assume that the observations (Ie, Vt)22 satisfy Eq. (1). Then the following holds for all t 2 and x E X with probability at least 6:
(Vam (det(EE, K2)
f(x))
VAllfII 02-1(x).
(8)
5 det(AA))1/2
1=8 8
Here, 3r stands for the parameter that balances between exploration exploitation and ensures the validity of confidence bounds. The analogous concentration inequalities in case of homoscedastic noise were considered in [1, 14, 35].
Failure GP-UCB in the risk-averse setting. GP-UCB is guaranteed to achieve sublinear cumu- lative regret with high probability in the risk-neutral (homosedaasiccheeersseeddd [14, 35], However, for the risk-averse setting in Eq. (2), the maximizers x E arg max, MV and u E arg max (x) might not coincide, and consequently, MV(x") can be significantly larger than MV(z)). This is illustrated in Figure I, where GP-UCB most frequently chooses optimum A of the highest risk.
BO setting
3.2 Warm up: Known variance-proxy
We remedy the previous issue with GP-UCB by proposing a natural Risk-averse Heteroscedastic BO (RAHBO) in case of the known variance-proxy (²(.). At each round I, RAHBO chooses the action:
It arg (z) 3,0 (z))
#EX
where 3g from Lemma and from Eq. (2). In the next section, we further relax the assumption of the variance- -roxy and consider a more practical setting when p²(-) is unknown to the learner. For the current setting, the performance RAHBO formally captured the following proposition. Proposition 1. Consider any f H. with || < By and sampling model from Eq. (1) with known variance- proxy (zr). Let (T be set as in Lemma with X = 1. Then, with probability at least 1 - 8, RAHBO attains cumulative risk-averse regret Rr O Trr +1)).

---

Here, denotes the maximum information gain [35] at time T defined via mutual information Iy firr) between evaluations F1.T and fı:T [f(z)....... f2r)] at points A C D:
max Tyn:T: JuT), "T ACX,
(10)
where (UIIT,IIT)) Σ t=1
In 1
of , (2e) ((x))
(11)
in case of heteroscedastic noise (see Appendix A.1.I). The upper bounds on YT are provided in [35] widely used kernels. These upper bounds typically scale sublinearly in T"; for linear kernel yr Od log T), and in case of squared exponential kernel "r = O(d(log 1) While these bounds are derived assuming the homoscedastic GP setting with some fixed constant noise variance, we show (in Appendix A.1.3) that the same rates (up multiplicative constant factor) apply in the heteroscedastic case.
3.3 RAHBO for unknown variance-proxy
In the case of unknown variance- proxxc the confidence bounds for the unknown f()) in Lemma 1 can not be readily used, and we construct new ones on the combined mean-variance objective. To learn about the unknown p² (x), we make some further assumptions.
Assumption 1. The variance-proxy (x(x belongs to an RKHS induced by some kernel keer, i.e, p² Hygver, and RKHS norm is bounded [|p2 Ieve I Bear some fiite Bear > O. Moreover, the noise E(x) in Eq. (1) is strictly --su--aaussinn i.., Varlş(x)] = -²((x) for every E X.
As consequence of our previous assumption, we can now focus on estimating the variance since Var[g(-)] and p- coincide. In particular, to estimate Var[g(-)] we consider a repeated experiment setting, where for each IG we collect k > 1 evaluations (xx (e(x)) f(xx) + (xx.). Then, the sample mean and variance of E(1ra) are given as:
1
(x()) and (x
(ui(2s) mix (xx))
(12)
The key idea is that for strictly sub-Gaussian noise E(x), $1 = 8k _ yields unbiased, but noisy evaluations of the unknown variance-proxy pL. = (08(31)...., i.e.
$L = p² (z+) + 7/(x))
(13)
with zero-mean noise 71())). In order to efficiently estimate p²()), we need an additional assumption. Assumption 2. The noise nπ(x) in Eq. (13) is (y( Gaussian with known (x)x and the realizations (22 are independent between t.
We note that similar assumption is made in [32] in the multi-armed bandit setting. The fact that (.) known is rather mild as Assumption I allows controlling its value. For example, in of strietly Lub-Gaussian 17(x) we show (in Appendix A.2) that Var[n(-)] = p²(()) s 2p)(()) - 1). Then, given that p²(() (-) < Φ², we can utilize the following (rather conservative) bound as a variance-proxy. i... p (x) = 2gt (k
RAHBO algorithm. We present our Risk-averse Heteroscedastic BO approach for unknown variance-proxy in Algorithm 1. Our method relies on building the following GP models.
Firstly, we use sample variance evaluations 87.4 to construct a GP model for p²()). The corresponding
" (-) and (-) are computed as in Eqs. (5) and (6) by using kernel x ,eer, variance-proxy / (-)
and noisy observations si..- Consequently, build the upper and lower confidence bounds ucb (.)
and lcb: (.) of the variance-proxy (²(() and we set 3 according to Lemma 1:
ucb, var (x) = 11 (xx) + 3ear ( (x), lcb, var (x) = 1i (I) 3 0 (x).
(14)
(15)
Secondly, we use sample mean evaluations vh1. = (rig (x1). T to construct a GP model for f(-). The mean (+(+)) and variance of(-) in Eqs. (5) and (6), however, rely the unknown
5

---

Algorithm Risk-averse Heteroscedastic Bayesian Optimization (RAHBO)
Require: Parameters a, (5 3t 1221, A, k, Prior 11 = PD 0, Kernel functions K, Ke
1: for t = .... do
2: Construct confidence bounds ucb, and lcb, (-) as in Eqs. (14) and (15)
3: Construct ucb/ (.) as in Eq. (7)
4: Select 22 E arg z ucb, (x)
lcb, (x:)
5:
Observe samples: y/ (z)) ) = f(x)) + (x()) for every E [k]
6: Use samples (u)(=))] to compute sample mean (x4) and variance (x)))) as in Eq. (12)
7: Use It; & (Iz) to update posterior (4/ar() (-) and of ofar()) as Eqs. (32) and (33)
8: Use ucb,"" ver (.) to compute Σ, as in Eq. (16)
9: Use Te, (Iz) and D to update posterior (e(-) and or(() as in Eqs. (5) and (6)
end for
variance-proxy p²(() in E,, an we thus use its upper confidence bound ucb;"" truncated with g²:
$ = diag( min (ucb, (x1), (xi) (x)),
min fucb ao (x+), )²
(16)
where Σ; is corrected by k since every evaluation in mis is an average over k samples. This substitution of the unknown variance-proxy by its conservative estimate guarantees that the confidence bounds ucb: (x:) := pe- 1 (x) + (x) on I also hold with high probability (conditioning on the confidence bounds for pl-) holding true; see Appendix A.3 for more details).
Finally, we define the acquisition function ucb, Te arg max (ebM((x) at each round L.
MV (x) ucb! (z) aleb, (x)))"())) i.e. selecting
The proposed algorithm leads new maximum information gains îr max (m,:T, fi.)) and Ir max I [6 1: Pi:T) for sample mean 11. and sample variance $ evaluations. The corresponding mutual information in frr and I'r computed according to Eq. (11) for heteroscedastic noise with variance- proxy d² /k and př, respectively (see Appendix A.4). The performance of RAHBO captured in the following theorem.
Theorem 1. Consider any I Hr with || S B, and sampling model in Eq. (1) with unknown variance-proxy (x) that satisfies Assumptions and 2. Let (xx) denote the set of actions chosen by RAHBO (Algorithm 1) over T rounds. Set T= and [t-rr]==1 according to Lemma with = 1, R² max o (xx) and pl-) E [e, Ø). Then, the risk-averse cumulative regret Ry of RAHBO is bounded as follows:
2T
2TT 1n(1+R-2)* 1
Pr RT S Brky In(1 + k/o2)
+
guar k a
21 8.
(17)
The risk-averse cumulative regret of RAHBO depends sublinearly on T for most the popularly used kernels. This follows from the implicit sublinear dependence on T in Br, 3yor and fr. rr (the bounds in case of heteroscedastic noise replicate the ones used in Proposition 1 as shown in Appendices A.1.2 and 1.1.).. Finally, the result of Theorem 1 provides a non-trivial trade-off for number of repetitions k where larger increases sample complexity but also leads to better estimation of the noise model. Fur- thermore, we obtain the bound for the number of rounds T required for identifying e-optimal point:
Corollary 1.1. Consider the setup of Theorem 1. Let = {21TT=1 denote actions selected by RAHBO over T rounds. Then, with probability at least 5, the reported point êт arg mag,, lebyy (It)), where leb (xx) leb/ (x) a ueb, (x), achieves e-accuracy, i.e. MV(x*) -MV("))-((TT) I after T> 112/5/11((1+//8) +4n / lm(1 rounds.
The previous result demonstrates the sample complexity rates when a single risk-averse reported solution is required. We note that both Theorem I and Corollary 1.1 provide guarantees for choosing risk-averse solutions, and depending on application at hand, we might consider either one of the proposed performance metrics. We demonstrate use-cases for both in the following section.

---

2.5
True
Nuisy ovaluations
Sample variance
True variance
0.0
0.5
1.5
2.0
0.0
1.0
2.0
(a) Illustration of the sine function (left) and noise variance (right)
0.4
02
RAHBO am2
RAHIO d=
RAHBO a-
QP-ICC
LCB
--- RAHBO--US 2
MVC MV(R))
RAHIOO US a=1
RARIBO-US c-1
0.1
60
20
0.0
0.0
iterations
BO iterations
(b) Cumulative regret
(c) Suboptimality w.r.t. MV
(d) Suboptimality w.r.t.
Figure 2: (a) Unknown true objective along with noisy evaluations with varying noise level (left) and unknown true noise variance and its evaluations (right). (b) Cumulative regret. (c) Simple MV regret for reporting rule बτ = arg max lebr(xx)) (c) Simple regret f(x*) - ((TT) for the unknown function at the reported point år from (d). RAHBO not only leads to strong results in terms of MV but also in terms of the mean objective fz))
4 Experiments
In this section, we experimentally validate RAHBO on two synthetic examples and two real hyperparameter tuning tasks, and compare it with the baselines. We provide open-source implementation of our method."
Baselines. We compare against two baselines: As the first baseline, we use GP-UCB with het- eroscedastic noise as a standard risk-neutral algorithm that optimizes the unknown fx). As the second one, we consider a risk-averse baseline that uniformly learns variance-pree proxy p² (z) before the optimization procedure,
contrast to RAHBO which learns the variance-proxy on the fly. We call it R RAHBO-US, standing for R RAHBO with uncertainty sampling. It consists of two stages: (i) uni- formly learning p² (a:) via uncertainty sampling, (ii) GP-UCB applied to the mean-variance objective, in which instead of the unknown p² (a:) we use the mean of the learned model. Note that RAHBO-LU US is the closest to the contextual BO setting [18], where the context distribution is assumed to be known.
Experimental setup. At each iteration f, an algorithm queries a point H; and observes sample mean and sample variance of & observations {y (xx)) We use a heteroscedastic GP for modelling (() and homoscedastic GP for (z)). We set X = 1 and B = 2. which is commonly used in practice to improve performance over the theoretical results. Before the BO procedure, we determine the GP hyperparameters maximizing the marginal likelihood. To this end, we use initial points that are same for all the baselines and are chosen via Sobol sequence that generates low discrepancy quasi-random samples. We repeat each experiment several times, generating new initial points for every repetition. We use two metrics: (a) risk-averse cumulative regret R, computed for the acquired inputs; (b) simple regret MVxx)) MV(+T) computed for inputs reported via Corollary 1.1. For each metric, we report its mean + two standard errors over the repetitions.
Example function We first illustrate the methods performance on a sine function depicted in Figure 2a. This function has two global optimizers. We induce a heteroscedastic zero-mean Gaussian noise on the measurements. We use sigmoid function for the noise variance, as depicted in Figure 2a, that induces small noise on [0, 1] and higher noise on (1, 2). We initialize the algorithms by selecting 10 inputs x: at random and keep these points the same for all the algorithms. We use k = 10 samples each chosen F4, The number of acquisition rounds is T 60. We repeat the experiment 30 times for each method and show their average performances in Figure 2.
wwwse-heeeero-be
7

---

Evaluations
Objective GP-UCB
Reported point argmax, LCBr(x)
Mean
Confideace bounds
RAHBO
2
-2
-2
2.0
0.0
Figure 3: GP models fitted for GP-UCB (left) and RAHBO (right) for sine function. After initialization with the same sampled points, GP-UCB concentrates on the high-noise region whereas RAHBO prefers small variance. Additional plots are presented in Appendix A.6.
Branin benchmark Next, we evaluate the methods on the (negated) Branin benchmark function in Figure la, achieving its optimum value f = -0.4 at (-x, 12.3), (T, 2.3), (9.4, 2.5). The heteroscedastic variance function illustrated Figure lb defines different noise variances for the three optima. We initialize all algorithms by selecting 10 inputs. We use k 10 samples to estimate the noise variance. The number of acquisition rounds is = 150. We repeat BO 25 times and show the results in Figures Ic and 5a. Figure Ic provides more intuition behind the observed regret: UCB exploits the noisiest maxima the most, while RAHBO prefers smaller variance.
Tuning Swiss free-electron laser In this experiment, we tune the parameters of Swiss X-ray free- electron laser (SwissFEL.), an important scientific instrument that generates very short pulses of X-ray light and enables researchers to observe extremely fast processes. The main objective is to maximize the pulse energy measured by a gas detector, that is a time-consuming and repetitive task during the SwissFEL operation. Such (re-)tuning takes place while user experiments on SwissFEL are running, and thus the cumulative regret is the metric of high importance in this application.
We use real SwissFEL measurements collected in [21] to train neural network surrogate model, and use it to simulate the SwissFEL objective fax for new parameter settings æ. We similarly fit a model of the heteroscedastic variance by regressing the squared residuals via a GP model. Here, we focus on the calibration of the four most sensitive parameters.
We report our comparison in Figure 4 where we also assess the effect of varying the coefficient of absolute risk tolerance a. We use 30 points initialize the baselines and then perform 200 acquisition rounds. We repeat each experiment 15 times. In Figure 4a we plot the empirical frequency of the true (unknown to the methods) values fxx) and o (x;) at the inputs x; acquired by the methods. The empirical frequency for p²(x) illustrates the tendency of risk-neutral GP- UCB to query points with higher noise, while risk-averse achieves substantially reduced variance and minimal reduction in mean performance. Sometimes, risk-neutra GP-UCB also fails to succeed in querying points with the highest f-value. That tendency results in lower cumulative regret for RAHBO in Figures 4c and 4d. We also compare the performance of the reporting rule from Corollary 1.1 in Figure 4b, where we plot error bars with standard deviation both for ((TT) and p² ( at the reported point ãr. As before, RAHBO drastically reduces the variance compared to GP-UCB, while having only slightly lower mean performance. Additional results are presented in Figure 10 in Appendix.
Random Forest tuning BO is widely used by cloud services for tuning machine learning hyperpa- rameters and the resulting models might be then used in high high-stakes applications such as credit scoring or fraud detection. In k-fold crass-validation, the average metric over the validation sets is optimized a canonical example of the repeated experiment setting that we consider in the paper. High across- folds variance is a practical problem [24] where the mean-variance approach might be beneficial.
In our experiment, we tune hyperparameters of random forest classifier (RF) on dataset of fraudulent credit card transactions [23].2 It consist of 285k transactions with 29 features (processed due to confidentiality issues) that are distributed over time, and only 0.2% are fraud examples (see Appendix for more details). The search space for the RF hyperparameters is also provided in the Appendix. We use the balanced accuracy score and validation folds, i.e., 5, and each validation fold is shifted in time with respect to the training data. We seek not only for high performance on average but also for low variance across the validation folds that have different time shifts with respect to the training data.
Phttps: hwwp::/wwwwwwwgl.loo www.kaaggw....ggao..ooee.... wow.///g-uub/creeeeeeooooooo wttp///reetttarrrraanddddd
8

---

RAHBO a-0.
RAHBO-US a=0.5
Frequency
RAIIBO
RAIDDOUS a=
Frequency
RAHBO
RAHBO-ES a=2
RAIIBO
RAIDO-USSas
GP-DCB
6.1
0.4
[(0 (a) Empirical distribution of true (x) (left) and pxx) (right) for SwissFEL
pf (x)
RANSO
100
RAHBO
GIMIIC
GI-LCC
0.97
RAHBO US
0.5
RAHIBO-US am
0.95
RAIIBO am0.5
RAIIBO-US gm05
30
RANIBO
RAIBOOUS
0.93
RAHIBO
RAIBBOUS 2=2
GP-UCB
20
0.0
0.1
0.2
0.3
0.4
0.5
0.
00
iterations
itceations
(b) Mcan- -can-variance tradcoff
(c) Cum. regret (a 0.5) (d) Cum. regret (a 1)
Figure 4: Experimental results for SwissFEL: (a) Distributions of f(x) and p²x)) for all points queried during the optimization. GP-UCB queries points with higher noise (but not necessarily high return f) in contrast to the risk-averse methods. (b) Mean fārr) and variance p² (2r) at the reported fy = arg max,, lcby(z:) (z;): for each method, we repeat BO experiment 15 times (separate points) and plot corresponding standard deviation error bars. RAHBO reports solutions with reasonable mean-variance tradeoff, while GP-UCB produces solutions with high mean value but also high noise variance. (c-d) Cum. regret for 0.5 and = 1 (see more in Appendix A.7.1).
We initialize the algorithms by selecting 10 hyperparameter settings and keep these points the same for all algorithms. We use Matérn 5/2 kernels with Automatic Relevance Discovery (ARD) and normalize the input features to the unit cube. The number of acquisition rounds in one experiment is 50 and we repeat each experiment 15 times. We demonstrate we plot mean 2 standard errors. While both RAHBO and GP-UCB perform comparable in terms of the mean error, its standard deviation for RAHBO is smaller.
results in Figures 5b and 5e where
800
RAHRO
RAHIBO a- 20
0.07
RAHBO 100
GP-UUB
0,025
GR-UCR
GP-TCC
- MVIC2)
Kandom Search
MVis"" Mrvie") MVVG))
700
0.05
0,015
0.03
200
0.05
.0
100
150
20
40
20
40 BO iterations
60
BO iterations
BO iterations
(a) Branin benchmark
(b) RF Tuning
(c) RF Tuning
Figure 5: Branin: (a) Cumulative regret. Random Forest: (b-c) Simple regret for the reported
ft arg max Mv(x)) for (b) 20 and (c) 100. While both methods have comparable mean, RAHBO has consistently lower variance.
5 Conclusion
In this work, we generalize Bayesian optimization to the risk-averse setting and propose RAHBO algorithm aiming to find an input with both large expected return and small input-dependent noise variance. Both the mean objective and the variance are assumed to be unknown priori and hence estimated online. RAHBO is equipped with theoretical guarantees showing (under reasonable

---

assumptions) sublinear dependence on the number of evaluation rounds T both for cumulative risk-averse regret and e-aceurate mean- verannariance metric. The empirical evaluation of the algorithm on synthetic benchmarks and hyperparameter tuning tasks demonstrate promising examples of heteroscedastic use-cases benefiting from RAHBO.
Acknowledgements
This research has been gratefully supported by NCCR Automation grant 5INF40 180545, by ERC under the European Union's Horizon grant 815943, SNSF grant 200021 1127711 and ETH Zürich Postdoctoral Fellowship 19-2 FEL-47. The authors thank Sebastian Curi, Mojmír Mutný and Johannes Kirschner as well the anonymous reviewers of this paper for their helpful feedback.
References
[1] Y. Abbasi-Yadkorfi Online learning for linearly parametrized control problems. PhD thesis, Edmonton, Alberta, 2012.
[2] Arbel, O. Marchal, and H. D. Nguyen. On strict Gaussianity, optimal proxy variance and symmetry for bounded random variables. arXiv:1901.09180 2019.
[3] M. Balandat, B. Karrer, D. R. Jiang, S. Daulton, B. Letham, A. G. Wilson, and E. Bakshy. BoTorch: a framework for efficient Monte-Carlo Bayesian optimization. Neural Information Processing Systems (NeurIPS), 2020.
[4] E. Benhamou. A few properties of sample variance. arXiv: 1809.0377 2018.
[5] Berkenkamp, A. Krause, and A. Schoellig. Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics. arXiv:1602.04445 2016.
[6] M. Binois, R. B. Gramacy, and M. Ludkovski. Practical heteroscedastic Gaussian process modeling for large simulation experiments. Journal of Computational and Graphical Statistics, 2018.
[7] M. Binois, J. Huang, R. B. Gramacy, and M. Ludkovski. Replication or exploration: Sequential design for stochastic simulation experiments. In Technometrics, 2019.
[8] I. Bogunovic. A. Krause, and J. Scarlett. caruption-tolerant Gaussian process bandit optimiza- tion. In Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
[9] Bogunovic, Scarlett, Jegelka, and V. Cevher. Adversarially robust optimization with Gaussian processes. In Neural Information Processing Systems (NeurIPS), 2018
[10] I. Bogunovic, J. Scarlett, A. Krause, and V. Cevher. Truncated variance reduction: A unified approach to Bayesian optimization and level-set estimation. In Neural Information Processing Systems (NeurIPS), 2016.
[11] S. Cakmak, R. Astudillo, P. L. Frazier, and E. Zhou. Bayesian optimization of risk measures. In Neural Information Processing Systems (NeurIPS), 2020.
[12] R. Calandra, A. Seyfarth, J. Peters, and M. P. Deisenroth. Bayesian optimization for learning gaits under uncertainty. Annals of Mathematics and Artificial Intelligence, 2016.
[13] Y. Chen, A. Huang, Z. Wang, I. Antonoglou, J. Schritwieserr D. Silver, and N. de Freitas. Bayesian optimization in Alphago. arXiv:1812.06833 2018.
[14] S. R. Chowdhury and A. Gopalan. On kernelized multi- bandits. International
Conference Machine Learning (ICML), 2017.
[15] A. I. Cowen-Rivers, W. Lyu, R. Tutunov, Z. Wang, A. Grosnit, R. R. Griffiths, H. Jianye, Wang, and B. Ammar. An empirical study of assumptions in Bayesian optimisation. arXiv:20/2.0 03826, 2021.
[16] J. González, J. Longworth, D. C. James, and N. D. Lawrence. Bayesian optimization for synthetic gene design. S.Ki:1505..06221 2015.
[17] R.-R. Griffiths and J. M. Hernández- Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chemical Science, 2020.
[18] Iwazaki, Y. Inatsu, and I. Takeuchi. Mean-variance analysis Bayesian optimization under uncertainty. In Conference Artificial Intelligence and Statistics (AISTATS), 2021.
10

---

[19] J. Kirschner, I. Bogunovic, S. Jegelka, and A. Krause. Distributionally robust Bayesian optimization. In Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
[20] Kirschner and A. Krause. Information directed sampling and bandits with heteroscedastic noise. In Conference On Learning Theory (COLT), 2018.
[21] Kirschner, M. Mutny, N. Hiller, R. Ischebeck, and A. Krause. Adaptive and safe Bayesian optimization in high dimensions via onedimensionall subspaces. In International Conference on Machine Learning (ICML), 2019.
[22] K. Korovina, Xu, K. Kandasamy, W. Neiswanger, B. Poezos, Schneider, and E. Xing. Chembo: Bayesian optimization of small organic molecules with synthesizable recommenda- tions. urXiv: 01425, 2019.
[23] Y.-A. Le Borgne and G. Bontempi. Machine Learning for Credit Card Fraud Detection Practical Handbook. Université Libre de Bruxelles, 2021.
[24] A. Makarova, H. Shen, V. Perrone, A. Klein, J. B. Faddoul, A. Krause, M. Seeger, and
C. Archambeau. Overfitting in Bayesian optimization: empirical study and carly-stopping solution. In ICLR Workshop on Neural Architecture Search, 2021.
[25] A. Marco, P. Hennig, J. Bohg, S. Schaal, and S. Trimpe. Automatic LQR tuning based on Gaussian process global optimization. In International Conference on Robotics and Automation (ICRA), 2016.
[26] Močkus. On Bayesian methods for seeking the extremum. Optimization Techniques, 1975.
[27] H. Moss, D. Leslie, D. Beck, J. González, and P. Rayson. BOSS: Bayesian optimization over string spaces. In Neural Information Processing Systems (NeurlPS), 2020.
[28] D. M. Negoescu, P. Frazier, and W. B. Powell. The knowledge-gradient algorithm for sequencing experiments in drug discovery. INFORMS J. Computing, 2011.
[29] Q. P. Nguyen, Z. Dai. B. K. H. Low, and P. Jaillet. Value-at-risk optimization with Gaussian processes. In International Conference on Machine Learning (ICML). 2021.
[30] T. Nguyen, S. Gupta, H. Ha, S. Rana, and S. Venkatesh. Distributionally robust Bayesian quadrature optimization. In Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
[31] S. Ray Chowdhury and A. Gopalan. Bayesian optimization under heavy-tailed payoffs. In Neural Information Processing Systems (NeurIPS), 2019.
[32] A. Sani, A. Lazaric, and Munos. Risk-aversion multi-armed bandits. In Neural Information Processing Systems (NeurIPS), 2012.
[33] P. G. Sessa, I. Bogunovic, M. Kamgarpour, and A. Krause. Mixed strategies for robust optimization of unknown objectives. In Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
[34] Snock, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learning algorithms. In Neural Information Processing Systems (NeurIPS), 2012.
[35] N. Srinivas, A. Krause, S. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. International Conference on International Conference on Machine Learning (ICML), 2010.


Reviews:
[{"review_id": "xo8GC4dkHl", "reviewer": "Reviewer_ZDq3", "summary": "This paper proposes a risk-averse Bayesian optimization algorithm, which aims to find an input with both large expected return and small noise-dependent noise variance. Both the mean function and input-dependent noise variance are assumed to be unknown functions from a reproducing kernel Hilbert space, and hence both need to be estimated online. Both theoretical guarantees and empirical evaluations are given.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper aims to tackle an important problem, i.e., designing BO algorithms that are risk-averse in terms of the risk introduced by input-dependent noise. The approaches taken by the paper, both theoretical and empirical, are all natural and reasonable. However, I have some concerns/questions regarding the theoretical and empirical results of this paper, please see the details below.\n\nWeaknesses/questions:\n- It looks like the Gaussian process posterior mean and variance for heteroscedastic GP adopted in this paper (equations 5 and 6) have a few differences between their homogenous counterpart (e.g., refer to equations 2 and 3 in reference [12]), even when a homogeneous noise variance is used, equations 5 and 6 in the current paper don't reduce to equations 3 and 3 in [12]. For example, the noise variance doesn't appear in the homogeneous case given in [12], but appears in the heteroscedastic case in equations 5 and 6 in the current paper. Is there an intuitive explanation for this difference?\n- Proposition 1: I have a few questions regarding these theoretical results. Firstly, $\\beta_T$ defined in equation 8 seems to be data-dependent, i.e., it depends on the history of selected input locations. So, I think a concrete expression regarding the growth rate of $\\beta_T$ should be given, in order to support the claim that the proposed algorithms are no-regret (as claimed in line 67). Next, is there an intuitive interpretation regarding the dependence on $\\lambda$? What is the recommended value of $\\lambda$? In the homogeneous case like [12], $\\lambda$ is set to a value dependent on $T$, but here it seems like a free parameter. In fact, the same questions apply to Theorem 2 as well.\n- Lines 164-165: it says here that \"the same upper bounds are still applicable in the considered heteroscedastic case (i.e., by setting $\\sigma^2$ to $\\varrho^2$)\". If I understood correctly, if we set $\\sigma^2$ to $\\varrho^2$, the corresponding information gain (see the formula on line 160) will be smaller than the information gain with heteroscedastic noise variance, since the information gain is inversely related to the noise variance. That means the max information gain derived using $\\varrho^2$ is smaller than by the $\\gamma_T$ from Proposition 1, so that doesn't really give us a concrete regret upper bound in terms of homoscedastic noise variance. Does this contradict what is said here in these few lines, or is my understanding inaccurate?\n- Equation 15: what's the intuition behind the use of $\\text{ucb}^{var}_t(x_1)$ here? BTW I think the second $x_1$ should be $x_t$.\n- Theorem 2: in addition to the questions I have regarding the dependence on $\\beta_T$ and $\\lambda$ as mentioned above, the dependence on $k$ here seems a little counter-intuitive: why is the regret bound increasing in $k$? Does that suggest we should set $k$ to the smaller value of 1?\n- Corollary 2.1: does this result hold \"with high probability\"?\n- Experiments: does the horizontal axis in Figure 2 (and other figures) consider the value of $k$? That is, if $k=10$ experiments are repeated for every evaluated input, does that correspond to 1 iteration or 10 iterations in the figure? If it corresponds to 1 iteration, the comparison may not be fair for GP-UCB. Moreover, I think an ablation w.r.t. the value of $k$ is needed, because I think it will have a huge impact on the estimated variance. For example, if $k$ is larger, I suppose RAHBO-US will perform better. \n- lines 221-224: are the initial steps for variance estimation included in the comparisons shown in the figures?\n- lines 250-251: why does standard GP-UCB prefer noisier inputs? Shouldn't it be indifferent between different inputs with the same expected function value?\n- Figure 5: it seems that the advantage of the proposed method isn't so clear in these figures, especially in the hyperparameter tuning tasks of Figures b and c.\n- Lines 268-269: are you trying to refer to Figure 4 instead of Figure 10 here? Because if it's Figure 10, the word \"drastically\" (line 271) will be an overstatement.\n\nSome more minor points:\n- Figure 2 caption, third and fourth lines: I think (c) and (d) are reversed.\n- I think Figure 3 is never referred to in the text.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6 hours"}, "extra_text": {}}, {"review_id": "gve-yK2jyb7", "reviewer": "Reviewer_hxC9", "summary": "This paper presents an approach for risk-averse Bayesian Optimization in the case that the function to be optimized is subject to heteroskedastic observation noise. For a mean-variance model, the authors propose an acquisition function and provide bound on the cumulative regret for the case when the variance proxy is known. They then extend this to model the unknown variance proxy (assuming strict sub-gaussianity) and provide regret bounds also for this case. The algorithm is empirically evaluated on a set of synthetic and real problems, where it is shown to improve upon non-robust BO approaches in terms of the regret defined.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### High-level summary:\n- The problem setting is relevant of interest for the community.\n- The discussion of related work is overall good.\n- The contributions are novel and appear technically sound.\n- The paper is generally well written and easy to follow.\n- The empirical evaluation is solid, but could benefit from additional baselines.\n- Overall a good paper; with room for improvement, but a pretty clear accept in my mind.\n\n\n### Notes:\n- Why are all of the named applications \"high-stakes\" applications? This is understandable for drug discovery and robotics, but not really for the other examples, please substantiate.\n  - More generally, the paper could do a better job at explaining the practical implications of this work by giving concrete examples.\n- It is quite a bit into the paper before the authors discuss that the setting of heteroskedastic noise and the setting of risk-averse BO are not new but have been studied before. They should do  this in the introduction to avoid any potential misconceptions and clearly identify the novelty of their work over existing literature.\n- The discussion of related work is relatively comprehensive. However, the setting of this paper is quite close to that of [1] (which focuses purely on modeling). The authors should add that reference and discuss how their work relates to that of [1].\n- I am not fully convinced that the mean-variance-model is a practical choice; it seems much harder for me to properly choose the coefficient of absolute risk tolerance than, say, to optimizat VaR or CVaR. From my experience working with practitioners, the assumption that alpha is fixed and known will generally not hold in practice. It would be good to address this rather than stating it as an assumption without any discussion.\n- The formulation of the acquisition function (eq 9) is pretty intuitive and the straightforward choice. An alternative would be to not consider the posterior variance sigma(x)  of the latent function, but consider the the overall posterior predictive sqrt(sigma(x)^2 + rho(x)^2). It would be worth discussing such option and compare the choice made here to other approaches.\n- The paper could benefit from additional discussion of the algorithm. For instance, in eq. 9 it may appear that there is a tension between beta and alpha, and it took me a bit to think through the difference implication of the posterior uncertainty of the latent function and the variance of the observation. It would be helpful to provide this kind of intuition for what is going on to the reader.\n- The repeated experiment setting is a natural thing to do but appears limiting, and it seems like it could really hurt sample efficiency. With a smoothness assumption on the behavior of the variance proxy, it should be possible to devise a more sample efficient algorithm than simply re-evaluating each candidate k times. This is discussed also in [1] and the authors should discuss how their approach compares to that of [1].\n- The theoretical results appear sound, though I did not check them in any detail. It's nice to show some regret bounds, and this appears to require a good amount of technical finesse. However, I feel like value of these theoretical contributions for practical purposes is somewhat limited. For instance, the results rely on finding the maximizer of the acquisition function exactly, but in actual implementations this requires solving very hard non-convex optimization problems repeatedly, where and one generally cannot guarantee finding the globally optimal solution for all but the smallest problems.\n  - The theoretical results do get quite technical, it would be useful for the non-expert to have some high-level summary for how the theoretical analysis and proof technique relates to that in Srinivas et al.\n- The empirical results are quite convincing and are presented well. What I feel is missing are some other existing robust BO methods (e.g. optimizing VaR or CVaR). While those may not specifically target the mean-variance model formulation, they could still perform quite well in terms of the regret defined here. The method from [3] has been implemented in botorch and should thus be quite easy to hook up into the submitted code: https://github.com/pytorch/botorch/blob/master/tutorials/risk_averse_bo_with_environmental_variables.ipynb\n- The FEL setup sounds quite interesting, it would be nice to get some more details into the non-anonymized version.\n- The authors state that \"In settings where the noise is homoscedastic, our approach does not bring additional value\". I think there is more to that, likely sample efficiency will suffer compared to approaches that were not designed with robustness in mind. It could be valuable to provide some analysis of the performance in case the noise is indeed homoskedastic; this would allow to get some idea of the \"price of being robust to heteroskedasticity\". To understand this, it would be good to have an ablation study that uses a homoskedasatic setting.\n- Please honor botorch's request for citation https://github.com/pytorch/botorch#citing-botorch [2]\n- Misc:\n    - l. 229: be clear what you mean by heteroskedastic GP (looks like it's a botorch FixedNoiseGP)\n    - Fig 5: \"Simple regret fat the reprted\"\n    - The term \"zeroth-order observation\" may not be clear to everyone, consider rephrasing (no gradient info)\n\n\n### Questions:\n- Can you explain the difference between your modeling setup and the HeteroskedasticSingleTaskGP from https://github.com/pytorch/botorch/blob/master/botorch/models/gp_regression.py#L339 ?\n- The fact that `k` (the number of function samples to estimate the local variance) is fixed and non-adaptive is somewhat disappointing. It should be possible to improve sample efficiency by in later stages making this adaptive to the posterior uncertainty of the variance proxy at the point to be evaluated. In fact, you could be Bayesian about this, too. This would mean interpreting the posterior distribution of the variance proxy as your prior, and then the `k` samples as observations, and then use the observations to update this prior. Did you consider this?\n\n### References:\n\n[1] K. Kersting, C. Plagemann, P. Pfaff, and W. Burgard. Most likely heteroscedastic gaussian process regression. In Proceedings of the 24th International Conference on Machine Learning, ICML ’07, pages 393–400, New York, NY, USA, 2007. ACM.\n\n[2] M. Balandat, B. Karrer, D. R. Jiang, S. Daulton, B. Letham, A. G. Wilson, and E. Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. Advances in Neural Information Processing Systems 33, 2020.\n\n[3] S. Cakmak, R. Astudillo, P. Frazier, and E. Zhou. Bayesian optimization of risk measures, 2020\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- In my mind, the main practical limitation of the paper is that the parameter alpha will often be unknown in practice (w/o already having a reasonably good understanding of both the behavior of the function and the noise). The authors have not really addressed this and should expand on it in their discussion.\n- The authors state that \"In settings where the noise is homoscedastic, our approach does not bring additional value\". I think there is more to that, likely sample efficiency will suffer compared to approaches that were not designed with robustness in mind. Ideally this could be quantified in an ablation study, and at the very least discussed.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "b1W0HAkZ-A", "reviewer": "Reviewer_2VoZ", "summary": "This paper proposes a new UCB strategy for Bayesian optimization that focuses on mean-variance optimization instead of expected value. The authors argue that this setting is relevant for heteroscedastic settings where one seeks high mean values with lower risk. Regret bounds are given to provide theoretical results about the convergence of the proposed algorithm for this setting. Empirical results on two synthetic problems and two \"real-world\" problems show that the proposed approach returns solutions with a higher mean and lower risk than standard risk-neutral GP-UCB.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\nA risk-averse strategy for Bayesian optimization is surely relevant in many applications. Noisy observations are ubiquitous in science and engineer problems. In fact, it is somehow surprising that previous work hasn't addressed this setting before. The \"Tuning a Free Electron Laser\" example was the most interesting experiment. I found this work fairly complete and relatively easy to understand, despite the heavy notation for the theorems.\n\n**Weakness**\n\nNo major issues. I would only suggest that the empirical section could be more substantial if all the results were motivated by real applications or if the synthetic experiments were conducted over multiple test functions (and not only two). It would be interesting to see aggregate results across a more extensive set of problems.\n\n**Minors**\n\n- lines 41 (suggestion). Exploration vs. Exploitation for optimizing the expected performance; instead of an 'and optimizing.'\n- Maybe I've missed that, but I was expecting RAHBO-US to execute x% observations of US and (1-x)% of RAHBO. What values of x were considered?\n\n**After author's response**\n\nThanks for all the detail answers! ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "InVRvy23hWj", "reviewer": "Reviewer_UhGC", "summary": "The paper proposes a Bayesian optimization method for robust optimization of black-box functions under heteroskedastic observation noise. The method uses an objective that trades off mean and noise variance to identify solutions with high mean and low noise variance. The authors propose modeling the black-box function and the black-box noise variance function using independent GPs and using a UCB-style acquisition function. The authors prove a sublinear cumulative regret bound for the method.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n-\tTheoretical grounding: the sublinear cumulative regret bound is nice\n-\tUnder-explored and interesting problem class of robustness with respect to stochastic evaluations\n  -\tMany works on robust optimization consider robustness with respect to input noise rather than output noise\n-\tThe FEL experiment is a nice demonstration \n\nWeaknesses:\n-\tTypically, expected performance under observation noise is used for evaluation because the decision-maker is interested in the true objective function and the noise is assumed to be noise (misleading, not representative). In the formulation in this paper, the decision maker does care about the noise; rather the objective function of interest is the stochastic noisy function. It would be good to make this distinction clearer upfront.\n-\tThe RF experiment is not super compelling. It is not nearly as interesting as the FEL problem, and the risk aversion does not make a significant difference in average performance. Overall the empirical evaluation is fairly limited.\n-\tIt is unclear why the mean-variance model is the best metric to use for evaluating performance\n  -\tWhy not also evaluate performance in terms of the VaR or CVaR?\n  -\tThe MV objective is nice for the proposed UCB-style algorithm and theoretical work, but for evaluation VaR and CVaR also are important considerations\n\nWriting:\n-\tVery high quality and easy to follow writing\n-\tGrammar:\n  -\tL164: “that that”\n  -\tFigure 5 caption: “Simple regret fat the reprted”\n\nQuestions:\n-\tFigure 2: “RAHBO not only leads to strong results in terms of MV, but also in terms of mean objective”? Why is it better than GP-UCB on this metric? Is this an artifact of the specific toy problem?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are discussed and potential future directions are interesting. “We are not aware of any societal impacts of our work” – this (as with an optimization algorithm) could be used for nefarious endeavors and could be discussed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper makes an important contribution to the theory of Bayesian optimization. Namely, it provides a sublinear cumulative regret bound in the heteroscedastic setting, the first of its kind, and show that this regret bound additionally applies in the risk averse setting. This result is particularly significant given that the overwhelming majority of Bayesian optimization papers assume homoscedastic noise, which may arguably less common than the heteroscedastic setting. Be sure to incorporate suggested minor corrections by the reviewers in the final camera ready version.
Decision: Accept (Poster)