Title: Efficient Robust Bayesian Optimization for Arbitrary Uncertain inputs
Venue: NeurIPS 2023
Accepted: yes
Keywords: bayesian optimization

robust optimization
Abstract: Bayesian Optimization (BO) is a sample-efficient optimization algorithm widely employed across various applications. In some challenging BO tasks, input uncertainty arises due to the inevitable randomness in the optimization process, such as machining errors, execution noise, or contextual variability. This uncertainty deviates the input from the intended value before evaluation, resulting in significant performance fluctuations in the final result. In this paper, we introduce a novel robust Bayesian Optimization algorithm, AIRBO, which can effectively identify a robust optimum that performs consistently well under arbitrary input uncertainty. Our method directly models the uncertain inputs of arbitrary distributions by empowering the Gaussian Process with the Maximum Mean Discrepancy (MMD) and further accelerates the posterior inference via Nystrom approximation. Rigorous theoretical regret bound is established under MMD estimation error and extensive experiments on synthetic functions and real problems demonstrate that our approach can handle various input uncertainties and achieve a state-of-the-art performance.

Efficient Robust Bayesian Optimization for Arbitrary Uncertain inputs

Abstract

Bayesian Optimization (BO) is a sample-efficient optimization algorithm widely employed across various applications. In some challenging BO tasks, input uncertainty arises due to the inevitable randomness in the optimization process, such as machining errors, execution noise, or contextual variability. This uncertainty deviates the input from the intended value before evaluation, resulting in significant performance fluctuations in final result. In this paper, we introduce a novel robust Bayesian Optimization algorithm, AIRBO, which can effectively identify a robust optimum that performs consistently well under arbitrary input uncertainty. Our method directly models the uncertain inputs of arbitrary distributions by empowering the Gaussian Process with the Maximum Mean Discrepancy (MMD) and further accelerates the posterior inference via Nyström approximation. Rigorous theoretical regret bound is established under MMD estimation error and extensive experiments on synthetic functions and real problems demonstrate that our approach can handle various input uncertainties and achieve a state-of-the-art performance.

1 Introduction

Bayesian Optimization (BO) is a powerful sequential decision-making algorithm for high-cost blackbox optimization. Owing to its remarkable sample efficiency and capacity to balance exploration and exploitation, BO has been successfully applied in diverse domains, including neural architecture search [32], hyper-parameter tuning [4, 29, 12], and robotic control [18, 5], among others. Nevertheless, in some real-world problems, the stochastic nature of the optimization process, such as machining error during manufacturing, execution noise of control, or variability in contextual factor, inevitably introduces input randomness, rendering the design parameter x to deviate to x ′ before evaluation. This deviation produces a fluctuation of function value y and eventually leads to a performance instability of the outcome. In general, the input randomness is determined by the application scenario and can be of arbitrary distribution, even quite complex ones. Moreover, in some cases, we cannot observe the exact deviated input x ′ but a rough estimation for the input uncertainty. This is quite common for robotics and process controls. For example, consider a robot control task shown in Figure 1a, a drone is sent to a target location x to perform a measurement task. However, due to the execution noise caused by the fuzzy control or a sudden wind, the drone ends up at location x ′ ∼ P ( x ) and gets a noisy measurement y = f ( x ′ ) + ζ . Instead of observing the exact value of

Executionnoise

Machining error: P()

Measurement noise: N(0, o2)

2

可

xn → xn = xn +

→f(xn)

yn =f(xn)

+

S

x'~P（x)

/x1,y1

x2,y2

BO

GP(μ*,E*)

xnyn

(a) An example case: drone measurement with execution noise.

(b) Problem formulation.

x ′ , we only have a coarse estimation of the input uncertainty P ( x ) . The goal is to identify a robust location that gives the maximal expected measurement under the process randomness.

To find a robust optimum, it is crucial to account for input uncertainty during the optimization process. Existing works [24, 3, 7, 10] along this direction assume that the exact input value, i.e. , x ′ in Figure 1b, is observable and construct a surrogate model using these exact inputs. Different techniques are then employed to identify the robust optimum: Nogueira et al. utilize the unscented transform to propagate input uncertainty to the acquisition function [24], while Beland and Nair integrate over the exact GP model to obtain the posterior with input uncertainty [3]. Meanwhile, [7] designs a robust confidence-bounded acquisition and applies min-max optimization to identify the robust optimum. Similarly, [10] constructs an adversarial surrogate with samples from the exact surrogate. These methods work quite well but are constrained by their dependence on observable input values, which may not always be practical.

An alternative approach involves directly modeling the uncertain inputs. A pioneering work by Moreno et al. [20] assumes Gaussian input distribution and employs a symmetric Kullback-Leibler divergence (SKL) to measure the distance of input variables. Dallaire et al. [13] implement a Gaussian process model with an expected kernel and derive a closed-form solution by restricting the kernel to linear, quadratic, or squared exponential kernels and assuming Gaussian inputs. Nonetheless, the applicability of these methods is limited due to their restrictive Gaussian input distribution assumption and kernel choice. To surmount these limitations, Oliveira et al. propose a robust Gaussian process model that incorporates input distribution by computing an integral kernel. Although this kernel can be applied to various distributions and offers a rigorous regret bound, its posterior inference requires a large sampling and can be time-consuming.

In this work, we propose an Arbitrary Input uncertainty Robust Bayesian Optimization algorithm (AIRBO). This algorithm can directly model the uncertain input of arbitrary distribution and propagate the input uncertainty into the surrogate posterior, which can then be used to guide the search for robust optimum. To achieve this, we employ Gaussian Process (GP) as the surrogate and empower its kernel design with the Maximum Mean Discrepancy (MMD), which allows us to comprehensively compare the uncertain inputs in Reproducing Kernel Hilbert Space (RKHS) and accurately quantify the target function under various input uncertainties(Sec. 3.1). Moreover, to stabilize the MMD estimation and accelerate the posterior inference, we utilize Nyström approximation to reduce the space complexity of MMD estimation from O ( m 2 ) to O ( mh ) , where h ≪ m (Sec. 3.2). This can substantially improve the parallelization of posterior inference and a rigorous theoretical regret bound is also established under the approximation error (Sec. 4). Comprehensive evaluations on synthetic functions and real problems in Sec.5 demonstrate that our algorithm can efficiently identify robust optimum under complex input uncertainty and achieve state-of-the-art performance.

2 Problem Formulation

In this section, we first formulize the robust optimization problem under input uncertainty then briefly review the intuition behind Bayesian Optimization and Gaussian Processes.

2.1 Optimization with Input Uncertainty

As illustrated in Figure 1b, we consider an optimization of expensive black-box function: f ( x ) , where x is the design parameter to be tuned. At each iteration n , we select a new query point x n

according to the optimization heuristics. However, due to the stochastic nature of the process, such as machining error or execution noise, the query point is perturbed to x ′ n before the function evaluation. Moreover, we cannot observe the exact value of x ′ n and only have a vague probability estimation of its value: P x n . After the function evaluation, we get a noisy measurement y = f ( x ′ n ) + ζ n , where ζ n is homogeneous measurement noise sampled from N (0 , σ 2 ) . The goal is to find an optimal design parameter x ∗ that maximizes the expected function value under input uncertainty:

Depending on the specific problem and randomness source, the input distribution P x can be arbitrary in general and even become quite complex sometimes. Here we do not place any additional assumption on them, except assuming we can sample from these input distributions, which can be easily done by approximating it with Bayesian methods and learning a parametric probabilistic model [16]. Additionally, we assume the exact values of x ′ are inaccessible, which is quite common in some real-world applications, particularly in robotics and process control [25].

2.2 Bayesian Optimization

In this paper, we focus on finding the robust optimum with BO. Each iteration of BO involves two key steps: I) fitting a surrogate model and II) maximizing an acquisition function.

Gaussian Process Surrogate: To build a sample-efficient surrogate, we choose Gaussian Process (GP) as the surrogate model in this paper. Following [34], GP can be interpreted from a weightspace view: given a set of n observations, D n = { ( x i , y i ) | i = 1 , ..., n } . Denote all the inputs as X ∈ R D × n and all the output vector as y ∈ R n × 1 . We first consider a linear surrogate:

where w is the model parameters and ζ is the observation noise. This model's capacity is limited due to its linear form. To obtain a more powerful surrogate, we can extend it by projecting the input x into a feature space ϕ ( x ) . By taking a Bayesian treatment and placing a zero mean Gaussian prior on the weight vector: w ∼ N (0 , Σ p ) , its predictive distribution can be derived as follows (see Section2.1 of [34] for detailed derivation):

where A = ϕ T ( X )Σ p ϕ ( X ) and I is a identity matrix. Note the predictive distribution is also a Gaussian and the feature mappings are always in the form of inner product with respect to Σ p . This implies we are comparing inputs in a feature space and enables us to apply kernel trick. Therefore, instead of exactly defining a feature mapping ϕ ( · ) , we can define a kernel: k ( x, x ′ ) = ϕ ( x ) T Σ p ϕ ( x ′ ) = ψ ( x ) · ψ ( x ′ ) . Substituting it into Eq. 3 gives the vanilla GP posterior:

From this interpretation of GP, we note that its core idea is to project the input x to a (possibly infinite) feature space ψ ( x ) and compare them in the Reproducing Kernel Hilbert Space (RKHS) defined by kernel.

Acquisition Function Optimization: Given the posterior of GP surrogate model, the next step is to decide a query point x n . The exploitation and exploration balance is achieved by designing an acquisition function α ( x |D n ) . Through numerous acquisition functions exist [28], we follow [25, 7] and adopt the Upper Confidence Bound (UCB) acquisition:

where β is a hyper-parameter to control the level of exploration.

3 Proposed Method

To cope with randomness during the optimization process, we aim to build a robust surrogate that can directly accept the uncertain inputs of arbitrary distributions and propagate the input uncertainty into

the posterior. Inspired by the weight-space interpretation of GP, we empower GP kernel with MMD to compare the uncertain inputs in RKHS. In this way, the input randomness is considered during the covariance computation and naturally reflected in the resulting posterior , which then can be used to guide the search for a robust optimum(Sec. 3.1). To further accelerate the posterior inference, we employ Nyström approximation to stabilize the MMD estimation and reduce its space complexity (Sec. 3.2).

3.1 Modeling the Uncertain Inputs

Assume P x ∈ P X ⊂ P are a set of distribution densities over R d , representing the distributions of the uncertain inputs. We are interested in building a GP surrogate over the probability space P , which requires to measure the difference between the uncertain inputs.

To do so, we turn to the Integral Probabilistic Metric (IPM) [23]. The basic idea behind IPM is to define a distance measure between two distributions P and Q as the supremum over a class of functions G of the absolute expectation difference:

where G is a class of functions that satisfies certain conditions. Different choices of G lead to various IPMs. For example, if we restrict the function class to be uniformly bounded in RKHS we can get the MMD[15], while a Lipschitz-continuous G realizes the Wasserstein distance [14].

In this work, we choose MMD as the distance measurement for the uncertain inputs because of its intrinsic connection with distance measurement in RKHS. Given a characteristic kernel k : R d × R d → R and associate RKHS H k , define the mean map ψ : P → H k such that ⟨ ψ ( P ) , g ⟩ = E P [ g ] , ∀ g ∈ H k . The MMD between P, Q ∈ P is defined as:

Without any additional assumption on the input distributions, except we can get m samples { u i } i m =1 , { v i } i m =1 from P, Q respectively, MMD can be empirically estimated as follows [21]:

̸

To integrate MMD into the GP surrogate, we design an MMD-based kernel over P as follows:

with a learnable scaling parameter α . This is a valid kernel, and universal w.r.t. C ( P ) under mild conditions (see Theorem 2.2, [11]). Also, it is worth to mention that, to compute the GP posterior, we only need to sample m points from the input distributions, but do not require their corresponding function values.

With the MMD kernel, our surrogate model places a prior GP (0 , ˆ k ( P x , P x ′ )) and obtain a dataset D n = { (ˆ x i , y i ) | ˆ x i ∼ P x i , i = 1 , 2 , ..., n ) } . The posterior is Gaussian with mean and variance:

where y n := [ y 1 , · · · , y n ] T , ˆ k n ( P ∗ ) := [ ˆ k ( P ∗ , P x 1 ) , · · · , ˆ k ( P ∗ , P x n )] T and [ ˆ K n ] ij = ˆ k ( P x i , P x j ) .

3.2 Boosting posterior inference with Nyström Approximation

To derive the posterior distribution of our robust GP surrogate, it requires estimating the MMD between each pair of inputs. Gretton et al. prove the empirical estimator in Eq. 8 approximates MMD in a bounded and asymptotic way [15]. However, the sampling size m used for estimation greatly affects the approximation error and insufficient sampling leads to a high estimation variance(ref. Figure 3a).

Such an MMD estimation variance causes numerical instability of the covariance matrix and propagates into the posterior distribution and acquisition function, rendering the search for optimal query point a challenging task. Figure 3b gives an example of MMD-GP posterior with insufficient samples, which produces a noisy acquisition function and impedes the search of optima. Increasing the sampling size can help alleviate this issue. However, the computation and space complexities of the empirical MMD estimator scale quadratically with the sampling size m . This leaves us with a dilemma that insufficient sampling results in a highly-varied posterior while a larger sample size can occupy significant GPU memory and reduce the ability for parallel computation.

To reduce the space and computation complexity while retaining a stable MMD estimation, we resort to the Nyström approximation [33]. This method alleviates the computational cost of kernel matrix by randomly selecting h subsamples from the m samples( h ≪ m ) and computes an approximated matrix via ˜ K = K mh K + h K T mh . Combining this with the MMD definition gives its Nyström estimator:

where U = K ( u , u ′ ) , V = K ( v , v ′ ) , W = K ( u , v ) are the kernel matrices, 1 m represents a m-by-1 vector of ones, m defines the sampling size and h controls the sub-sampling size. Note that this Nyström estimator reduces the space complexity of posterior inference from O ( MNm 2 ) to O ( MNmh ) , where M and N are the numbers of training and testing samples, m is the sampling size for MMD estimation while h ≪ m is the sub-sampling size. This can significantly boost the posterior inference of robust GP by allowing more inference to run in parallel on GPU.

4 Theoretical Analysis

Assume x ∈ X ⊂ R d , and P x ∈ P X ⊂ P are a set of distribution densities over R d , representing the distribution of the noisy input. Given a characteristic kernel k : R d × R d → R and associate RKHS H k , we define the mean map ψ : P → H k such that ⟨ ψ ( P ) , g ⟩ = E P [ g ] , ∀ g ∈ H k .

We consider a more general case. Choosing any suitable functional L such that ˆ k ( P, P ′ ) := L ( ψ P , ψ P ′ ) is a positive-definite kernel over P , for example the linear kernel ⟨ ψ P , ψ P ′ ⟩ k and radial kernels exp( -α ∥ ψ P -ψ P ′ ∥ 2 k ) using the MMD distance as a metric. Such a kernel ˆ k is associated with a RKHS H ˆ k containing functions over the space of probability measures P .

One important theoretical guarantee to conduct GP model is that our object function can be approximated by functions in H ˆ k , which relies on the universality of ˆ k . Let C ( P ) be the class of continuous functions over P endowed with the topology of weak convergence and the associated Borel σ -algebra, and we define ˆ f ∈ C ( P ) such that

which is just our object function, For ˆ k be radial kernels, it has been shown that ˆ k is universal w.r.t C ( P ) given that X is compact and the mean map ψ is injective [11, 22]. For ˆ k be linear kernel which is not universal, it has been shown in Lemma 1, [26] that ˆ f ∈ H ˆ k if and only if f ∈ H and further ∥ ˆ f ∥ ˆ k = ∥ f ∥ k . Thus, in the remain of this chapter, we may simply assume ˆ f ∈ H ˆ k .

Suppose we have an approximation kernel function ˜ k ( P, Q ) near to the exact kernel function ˆ k ( P, Q ) . The mean ˆ µ n ( p ∗ ) and variance ˆ σ 2 n ( p ∗ ) are approximated by

where y n := [ y 1 , · · · , y n ] T , ˜ k n ( P ∗ ) := [ ˜ k ( P ∗ , P 1 ) , · · · , ˜ k ( P ∗ , P n )] T and [ ˜ K n ] ij = ˜ k ( P i , P j ) .

The maximum information gain corresponding to the kernel ˆ k is denoted as

Denote e ( P, Q ) = ˆ k ( P, Q ) -˜ k ( P, Q ) as the error function when estimating the kernel ˆ k . We suppose e ( P, Q ) has an upper bound with high probability:

Assumption 1. For any ε > 0 , P, Q ∈ P X , we may choose an estimated ˜ k ( P, Q ) such that the error function e ( P, Q ) can be upper-bounded by e ε with probability at least 1 -ε , that is, P ( | e ( P, Q ) | ≤ e ε ) > 1 -ε.

Remark. Note that this assumption is standard in our case: we may assume max x ∈X ∥ ϕ ∥ k ≤ Φ , where ϕ is the feature map corresponding to the k . Then when using empirical estimator, the error between MMDempirical and MMD is controlled by 4Φ √ 2 log(6 /ε ) m -1 with probability at least 1 -ε according to Lemma E.1, [8]. When using the Nyström estimator, the error has a similar form as the empirical one, and under mild conditions, when h = O ( √ m log( m )) , we get the error of the order O ( m -1 / 2 log(1 /ε )) with probability at least 1 -ε . One can check more details in Lemma 1.

Now we restrict our Gaussian process in the subspace P X = { P x , x ∈ X} ⊂ P . We assume the observation y i = f ( x i ) + ζ i with the noise ζ i . The input-induced noise is defined as ∆ f p x i :=

f ( x i ) -E P x i [ f ] = f ( x i ) -ˆ f ( P x i ) . Then the total noise is y i -E P x i [ f ] = ζ i +∆ f p x i . We can state our main result, which gives a cumulative regret bound under inexact kernel calculations,

Theorem 1. Let δ > 0 , f ∈ H k , and the corresponding ∥ ˆ f ∥ ˆ k ≤ b, max x ∈X | f ( x ) | = M . Suppose the observation noise ζ i = y i -f ( x i ) is σ ζ -sub-Gaussian, and thus with high probability | ζ i | < A for some A > 0 . Assume that both k and P x satisfy the conditions for ∆ f P x to be σ E -sub-Gaussian, for a given σ E > 0 . Then, under Assumption 1 with ε > 0 and corresponding e ε , setting σ 2 = 1+ 2 n , running Gaussian Process with acquisition function

we have that the uncertain-inputs cumulative regret satisfies:

with probability at least 1 -δ -nε . Here ˜ R n = ∑ n t =1 ˜ r t , and ˜ r t = max x ∈X E P x [ f ] -E P x t [ f ]

The proof of our main theorem 1 can be found in appendix B.3.

The assumption that ζ i is σ ζ -sub-Gaussian is standard in GP fields. The assumption that ∆ f P x is σ E -sub-Gaussian can be met when P x is uniformly bounded or Gaussian, as stated in Proposition 3, [26]. Readers may check the definition of sub-Gaussian in appendix, Definition 1.

To achieve an regret of order ˜ R n ∈ O ( √ n ˆ γ n ) , the same order as the exact Improved GP regret (23), and ensure this with high probability, we need to take ε = O ( δ/n ) , e ε = O ( n -5 2 ˆ γ n (ˆ γ -2 n ∧ n -1 2 )) , and this requires a sample size m of order O ( n 5 ˆ γ -2 n (ˆ γ 4 n ∨ n ) log( n )) for MCMC approximation, or with a same sample size m and a subsample size h of order O ( n 5 2 + ν ˆ γ -1 -ν n (ˆ γ 2 n ∨ n 1 2 )) for Nyström approximation with some ν > 0 . Note that (16) only offers an upper bound for cumulative regret, in real applications the calculated regret may be much smaller than this bound, as the approximation error e ϵ can be fairly small even with a few samples when the input noise is relatively weak.

To analysis the exact order of ˆ γ n could be difficult, as it is influenced by the specific choice of embedding kernel k and input uncertainty distributions P x i , x i ∈ X . Nevertheless, we can deduce the following result for a wide range of cases, showing that cumulative regret is sub-linear under mild conditions. One can check the proof in appendix B.4.

Theorem 2 (Bounding the Maximum information gain) . Suppose k is r -th differentiable with bounded derivatives and translation invariant, i.e., k ( x, y ) = k ( x -y, 0) . Suppose the input uncertainty is i.i.d., that is, the noised input density satisfies P x i ( x ) = P 0 ( x -x i ) , ∀ x i ∈ X . Then if the space X is compact in R d , the maximum information gain ˆ γ n satisfies

Thus, when r > d ( d +1) , the accumulate regret is sub-linear respect to n , with sufficiently small e ε .

GP

xc_Is=0.121,xc_0s=2.011,noise=0.000

10

8

2std

mean

X

gdt

samples

6

Z

-2

0

-6

-4

0.0

0.2

0.4

0.6

0.8

1.0

MMDGP-nystrom-200/20

xc_Is=0.222,xc_0s=1.245,noise=0.122

2

0

2std

mean

X

gdt

samples

0.0

0.2

0.4

0.6

0.8

1.0

MMDGP-nystrom-500/50

xc_Is=0.231, xc_0s=1.330, noise=0.046

2std

mean

X

p6

samples

0

0.0

0.2

0.4

0.6

0.8

1.0

MMDGP-nystrom-200/20

xc_Is=0.167,xc_0s=1.129,noise=0.095

4

X

2

0

-2

2std

mean

X

gdt

samples

0.0

0.2

0.4

0.6

0.8

1.0

5 Evaluation

In this section, we first experimentally demonstrate AIRBO's ability to model uncertain inputs of arbitrary distributions, then validate the Nyström-based inference acceleration for GP posterior, followed by experiments on robust optimization of synthetic functions and real-world benchmark.

5.1 Robust Surrogate

Modeling arbitrary uncertain inputs : We demonstrate MMDGP's capabilities by employing an RKHS function as the black-box function and randomly selecting 10 samples from its input domain. Various types of input randomness are introduced into the observation and produce training datasets of D = { ( x i , f ( x i + δ i )) | δ i ∼ P x i } 10 i =1 with different P x configurations. Figure 2a and 2b compare the modeling results of a conventional GP and MMDGP under a Gaussian input uncertainty P x = N (0 , 0 . 01 2 ) . We observe that the GP model appears to overfit the observed samples without recognizing the input uncertainty, whereas MMDGP properly incorporates the input randomness into its posterior.

To further examine our model's ability under complex input uncertainty, we design the input distribution to follow a beta distribution with input-dependent variance: P x = beta ( α = 0 . 5 , β = 0 . 5 , σ = 0 . 9(sin 4 πx +1)) . The MMDGP posterior is shown in Figure 2c. As the input variance σ changes along x , inputs from the left and right around a given location x i yield different MMD distances, resulting in an asymmetric posterior ( e.g. , around x = 0 . 05 and x = 0 . 42 ). This suggests that MMDGP can precisely model the multimodality and asymmetry of the input uncertainty.

Moreover, we evaluated MMDGP using a step-changing Chi-squared distribution P x = χ 2 ( g ( x ) , σ = 0 . 01) , where g ( x ) = 0 . 5 if x ∈ [0 , 0 . 6] , and g ( x ) = 7 . 0 otherwise. This abrupt change in g ( x ) significantly alters the input distribution from a sharply peaked distribution to a flat one with a long tail. Figure 2d illustrates that our model can accurately capture this distribution shape variation, as evidenced by the sudden posterior change around x = 0 . 6 . This demonstrates our model can thoroughly quantify the characteristics of complex input uncertainties.

Comparing with the other surrogate models: We also compare our model with the other surrogate models under the step-changing Chi-squared input distribution. The results are reported in Figure 7 and they demonstrate our model outperforms obviously under such a complex input uncertainty (see Appendix D.1 for more details)

5.2 Accelerating the Posterior Inference

Estimation variance of MMD: We first examine the variance of MMD estimation by employing two beta distributions P = beta ( α = 0 . 4 , β = 0 . 2 , σ = 0 . 1) and Q = beta ( α = 0 . 4 , β = 0 . 2 , σ = 0 . 1) + c , where c is an offset value. Figure 3a shows the empirical MMDs computed via Eq. 8 with varying sampling sizes as Q moves away from P . We find that a sampling size of 20 is inadequate, leading to high estimation variance, and increasing the sampling size to 100 stabilizes the estimation.

We further utilize this beta distribution P as the input distribution and derive the MMDGP posterior via empirical estimator in Figure 3b. Note that the MMD instability caused by inadequate sampling subsequently engenders a fluctuating posterior and culminates in a noisy acquisition function, which prevents the acquisition optimizer ( e.g. , L-BFGS-B in this experiment) from identifying the optima. Although Figure 3c shows that this issue can be mitigated by using more samples during empirical

Beta(0.4,0.2)

xc_Is=0.123, xc_0s=0.550, noise=0.150

MMDGP-Empirical-20

xc_Is=0.124, xc_0s=0.701, noise=0.149

MMDGP-Empirical-100

xc_Is=0.029, xc_0s=0.729, noise=0.161

MMDGP-nystrom-100/10

sampling_size=20

2std

2std

MMD2 estimation

m

sampling_size=100

mean

gdt

samples

mean

gdt

samples

mean

gdt

samples

2

1

0

1.0

sampling_size=100

sampling_size=20

UpperConfidenceBound

new_P

new_p

UpperConfidenceBound

new_p

UpperConfidenceBound

80

Covariance

0.6

3

-E

0.4 -

2

2

2

0.2

0.0

0.2

0.4

0.6

0.8

1.0

Sample mean distance

00

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

(a) The empirical MMD and covariance values between two beta distributions P and Q as Q moves away from P .

MMDestimation, it is crucial to note that a larger sampling size significantly increases GPU memory usage because of its quadratic space complexity of O ( MNm 2 ) ( M and N are the sample number of training and testing, m is the sampling size for MMD estimation). This limitation severely hinders parallel inference for multiple samples and slows the overall speed of posterior computation.

Table 1 summarizes the inference time of MMDGP posteriors at 512 samples with different sampling sizes. We find that, for beta distribution defined in this experiment, the Nyström MMD estimator with a sampling size of 100 and sub-sampling size of 10 already delivers a comparable result to the empirical estimator with 100 samples (as seen in the acquisition plot of Figure 3d). Also, the inference time is reduced from 8.117 to 0.78 seconds by enabling parallel computation for more samples. For the cases that require much more samples for MMD estimation ( e.g. , the input distribution is quite complex or high-dimensional), this Nyström-based acceleration can have a more pronounced impact.

Effect of Nyström estimator on optimization: To investigate the effect of Nyström estimator on optimization, we also perform an ablation study in Appendix D.2, the results in Figure 8 suggest that Nyström estimator slightly degrades the optimization performance but greatly improves the inference efficiency.

5.3 Robust Optimization

Experiment setup: To experimentally validate AIRBO's performance, we implement our algorithm 1 based on BoTorch [2] and employ a linear combination of multiple rational quadratic kernels [6] to compute the MMD as Eq. 9. We compare our algorithm with several baselines: 1) uGP-UCB [26] is a closely related work that employs an integral kernel to model the various input distributions. It has a quadratic inference complexity of O ( MNm 2 ) , where M and N are the sample numbers of the training and testing set, and m indicates the sampling size of the integral kernel. 3) GP-UCB is the standard GP with UCB acquisition, which represents a broad range of existing methods that

(a) The RKHS function

(b)

Robust

regrets

of

RKHS function.

(c) The double-peak func- tion

(d) Robust regrets of double-peak function

4

Regret

30

20

40

UGP

MMDGP-nystrom

GP

skl

ERBF

8

6

gdt

exact

true

est_norm

12

10

14

8

UGP

MMDGP-nystrom

skl

ERBF

2

10

6

4

4

0

0

2

-10

2

0

0.0

0.2

0.4

0.6

0.8

1.0

0

10

20

step

30

40

50

0

0.0

0.2

0.4

0.6

0.8

1.0

0

10

20

step

30

40

50

focus on non-robust optimization. 3) SKL-UCB employs symmetric Kullback-Leibler divergence to measure the distance between the uncertain inputs [20]. Its closed-form solution only exists if the input distributions are the Gaussians. 4) ERBF-UCB is the robust GP with the expected Radial Basis Function kernel proposed in [13]. It computes the expected kernel under input distribution using the Gaussian integrals. Assuming the input distributions are sub-Gaussians, this method can efficiently find the robust optimum. Since all the methods use UCB acquisition, we simply distinguish them by their surrogate names in the following tests.

At the end of the optimization, each algorithm needs to decide a final outcome x r n , perceived to be the robust optimum under input uncertainty at step n . For a fair comparison, we employ the same outcome policy across all the algorithms: x r n = arg max x ∈D n ˆ µ ∗ ( x ) , where ˆ µ ∗ ( x ) is the posterior mean of robust surrogate at x and D n = { ( x i , f ( x i + δ i )) | δ i ∼ P x i } are the observations so far. The optimization performance is measured in terms of robust regret as follows:

where x ∗ is the global robust optimum and x r n represents the outcome point at step n . For each algorithm, we repeat the optimization process 12 times and compare the average robust regret.

1DRKHSfunction: Webegin the optimization evaluation with an RKHS function that is widely used in previous BO works [1, 24, 10]. Figure 4a shows its exact global optimum resides at x = 0 . 892 while the robust optimum is around x = 0 . 08 when the inputs follow a Gaussian distribution N (0 , 0 . 01 2 ) . According to Figure 4b, all the robust BO methods work well with Gaussian uncertain inputs and efficiently identify the robust optimum, but the GP-UCB stagnates at a local optimum due to its neglect of input uncertainty. Also, we notice the regret of our method decrease slightly slower than uGP works in this low-dimensional and Gaussian-input case, but later cases with higher dimension and more complex distribution show our method is more stable and efficient.

1D double-peak function: To test with more complex input uncertainty, we design a blackbox function with double peaks and set the input distribution to be a multi-modal distribution P x = beta ( α = 0 . 4 , β = 0 . 2 , σ = 0 . 1) . Figure 4c shows the blackbox function (black solid line) and the corresponding function expectations estimated numerically via sampling from the input distribution ( i.e. , the colored lines). Note the true robust optimum is around x = 0 . 251 under the beta distribution, but an erroneous location at x = 0 . 352 may be determined if the input uncertainty is incorrectly presumed to be Gaussian. This explains the results in Figure 4d: the performance of SKL-UCB and ERBF-UCB are sub-optimal due to their misidentification of inputs as Gaussian variables, while our method accurately quantifies the input uncertainty and outperforms the others.

10D bumped-bowl function: we also extend our evaluation to a 10D bumped-bowl function [27] under a concatenated circular distribution. Figure 9 demonstrates AIRBO scales efficiently to high dimension and outperforms the others under complex input uncertainty(see Appendix D.3).

Robust robot pushing: To evaluate AIRBO in a real-world problem, we employ a robust robot pushing benchmark from [31], in which a ball is placed at the origin point of a 2D space and a robot learns to push it to a predefined target location ( g x , g y ) . This benchmark takes a 3-dimensional input ( r x , r y , r t ) , where r x , r y ∈ [ -5 , +5] are the 2D coordinates of the initial robot location and r t ∈ [0 , 30] controls the push duration. We set four targets in separate quadrants, i.e. , g 1 = ( -3 , -3) , g 2 = ( -3 , 3) , g 3 = (4 . 3 , 4 . 3) , and a 'twin' target at g ′ 3 = (5 . 1 , 3 . 0) , and describe the input uncertainty via a two-component Gaussian Mixture Model (defined in Appendix D.4). Following [7, 10], this blackbox benchmark outputs the minimum distance to these 4 targets under squared and linear distances: loss = min( d 2 ( g 1 , l ) , d ( g 2 , l ) , d ( g 3 , l ) , d ( g ′ 3 , l )) , where d ( g i , l ) is the Euclidean distance between the ball's ending location l and the i -th target. This produces a loss landscape as

10.5

93= (4.3,4.3)★

9.0

2.5

GP

MMDGP-nystrom

uGP

ERBF

1.0

GP

MMDGP-nystrom

uGP

ERBF

93 = (5.1,3.0) ★

sk1

skl

Z

g1 = (-3, -3)

7.5

Regret(mean)

2.0

Regret(std)

0.8

6.0

1.5

0.6

4.5

0.4

1.0

★

3.0

0.2

g2 = (-3,3)

1.5

0.5

0

20

40

60

80

100

0

20

40

60

80

100

-6

-4

2

0

2

4

0'0

step

step

(a) Contour of the robot push world

(b) Robust regrets of different algorithms

M-GP, total=48

M-ERBF, total=48

M-skl, total=48

M-MMDGP, total=48

M-uGP, total=48

[Q2] count:36(75.00%), rt=23.00+-3.76

[Q2] count:32(66.67%), rt=23.72+-4.14

[Q2] count:13(27.08%), rt=27.66+-2.56

[Q3] c0unt:8(16.67%),rt=30.00+-0.00

[Q3] count:8(16.67%), rt=25.86+-0.60

[Q3]count:0(o.00%),rt=nan+-nan

[Q3] count:21(43.75%).rt=28.78+-1.09

[Q3]count:24(50.00%),rt=29.12+-1.08

[Q4] count:20(41.67%), rt=23.54+-0.98

[Q4] count:4(8.33%), rt=20.87+-0.00

[Q4] count:12(25.00%), rt=22.92+-1.98

[Q4] count:11(22.92%), rt=25.07+-2.33

30

30

30

30

30

Z

4

26

28

2

28

26

2

26

28

2

28

26

Z

28

26

24

24

24

24

24

0

Q1

22

0

Q4

Q1

22

0

Q4

Q1

22

0

Q4.

Q1

22

0-

Q1.

22

20

2

20

2

20

Z-

20

Z-

20

Q2

18

16

Q3

Q2

16

18

Q3

Q2

18

16

Q3

Q2

00

18

16

Q2

18

16

-4

-2

0

2

4

=4

2

0

2

4

4

2

0

2

4

4

2

0

2

4

4

=2

0

2

4

shown in Figure 5a. Note that g 2 is a more robust target than g 1 because of its linear-form distance while pushing the ball to quadrant I is the best choice as the targets, g 3 and g ′ 3 , match the dual-mode pattern of the input uncertainty. According to Figure 5b, our method obviously outperforms the others because it efficiently quantifies the multimodal input uncertainty. This can be further evidenced by the push configurations found by different algorithms in Figure 6, in which each dot represents the robot's initial location and its color represents the push duration. We find that AIRBO successfully recognizes the targets in quadrant I as an optimal choice and frequently pushes from quadrant III to quadrant I. Moreover, the pushes started close to the origin can easily go far away under input variation, so our method learns to push the ball from a corner with a long push duration, which is more robust in this case.

6 Discussion and Conclusion

In this work, we generalize robust Bayesian Optimization to an uncertain input setting. The weightspace interpretation of GP inspires us to empower the GP kernel with MMD and build a robust surrogate for uncertain inputs of arbitrary distributions. We also employ the Nyström approximation to boost the posterior inference and provide theoretical regret bound under approximation error. The experiments on synthetic blackbox function and benchmarks demonstrate our method can handle various input uncertainty and achieve state-of-the-art optimization performance.

There are several interesting directions that worth to explore: though we come to current MMD-based kernel from the weight-space interpretation of GP and the RKHS realization of MMD, our kernel design exhibits a deep connection with existing works on kernel over probability measures [22, 11]. Along this direction, as our theoretic regret analysis in Section 4 does not assume any particular form of kernel and the Nyström acceleration can also be extended to the other kernel computation, it is possible that AIRBO can be further generalized to a more rich family of kernels. Moreover, the MMDused in our kernel is by no means limited to its RKHS realization. In fact, any function class F that comes with uniform convergence guarantees and is sufficiently rich can be used, which renders different realizations of MMD. With proper choice of function class F , MMD can be expressed as the Kolmogorov metric or other Earth-mover distances [15]. It is also interesting to extend AIRBO with the other IPMs.

References

John-Alexander M Assael et al. 'Heteroscedastic treed bayesian optimisation'. In: arXiv preprint arXiv:1410.7172 (2014).

Maximilian Balandat et al. 'BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization'. In: Advances in Neural Information Processing Systems 33 . 2020. URL: http: //arxiv.org/abs/1910.06403 .

Justin J Beland and Prasanth B Nair. 'Bayesian Optimization Under Uncertainty'. In: (2017), p. 5.

James Bergstra et al. 'Algorithms for hyper-parameter optimization'. In: Advances in neural information processing systems 24 (2011).

Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. 'Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics'. In: Machine Learning (2021), pp. 1-35.

Mikołaj Bi´ nkowski et al. Demystifying MMD GANs . 2021. DOI: 10.48550/arXiv.1801. 01401 . arXiv: 1801.01401 .

Ilija Bogunovic et al. 'Adversarially Robust Optimization with Gaussian Processes'. In: Advances in Neural Information Processing Systems . Vol. 31. Curran Associates, Inc., 2018.

Antoine Chatalic et al. 'Nyström Kernel Mean Embeddings'. In: International Conference on Machine Learning . 2022.

Sayak Ray Chowdhury and Aditya Gopalan. 'On Kernelized Multi-armed Bandits'. In: International Conference on Machine Learning . 2017.

Ryan B. Christianson and Robert B. Gramacy. Robust Expected Improvement for Bayesian Optimization . 2023. arXiv: 2302.08612 .

Andreas Christmann and Ingo Steinwart. 'Universal Kernels on Non-Standard Input Spaces'. In: NIPS . 2010.

Alexander I Cowen-Rivers et al. 'HEBO: pushing the limits of sample-efficient hyperparameter optimisation'. In: Journal of Artificial Intelligence Research 74 (2022), pp. 12691349.

Patrick Dallaire, Camille Besse, and Brahim Chaib-draa. 'An Approximate Inference with Gaussian Process to Latent Functions from Uncertain Data'. In: Neurocomputing . Adaptive Incremental Learning in Neural Networks 74.11 (2011), pp. 1945-1955. DOI: 10.1016/j. neucom.2010.09.024 .

Arthur Gretton, Dougal Sutherland, and Wittawat Jitkrittum. 'Interpretable comparison of distributions and models'. In: Advances in Neural Information Processing Systems [Tutorial] (2019).

Arthur Gretton et al. 'A kernel two-sample test'. In: The Journal of Machine Learning Research 13.1 (2012), pp. 723-773.

Trevor Hastie et al. The elements of statistical learning: data mining, inference, and prediction . Vol. 2. Springer, 2009.

Thomas Kühn. 'Eigenvalues of integral operators with smooth positive definite kernels'. In: Archiv der Mathematik 49 (1987), pp. 525-534. URL: https://api.semanticscholar. org/CorpusID:121372638 .

Ruben Martinez-Cantin et al. 'Active policy learning for robot planning and exploration under uncertainty.' In: Robotics: Science and systems . Vol. 3. 2007, pp. 321-328.

Seyedali Mirjalili and Andrew Lewis. 'Obstacles and difficulties for robust benchmark problems: A novel penalty-based robust optimisation method'. In: Information Sciences 328 (2016), pp. 485-509.

Pedro Moreno, Purdy Ho, and Nuno Vasconcelos. 'A Kullback-Leibler Divergence Based Kernel for SVM Classification in Multimedia Applications'. In: Advances in Neural Information Processing Systems . Vol. 16. MIT Press, 2003.

Krikamol Muandet et al. 'Kernel Mean Embedding of Distributions: A Review and Beyond'. In: Foundations and Trends® in Machine Learning 10.1-2 (2017), pp. 1-141. DOI: 10.1561/ 2200000060 . arXiv: 1605.09522 .

Krikamol Muandet et al. 'Learning from Distributions via Support Measure Machines'. In: NIPS . 2012.

Alfred Müller. 'Integral probability metrics and their generating classes of functions'. In: Advances in applied probability 29.2 (1997), pp. 429-443.

Jose Nogueira et al. 'Unscented Bayesian Optimization for Safe Robot Grasping'. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Daejeon, South Korea: IEEE, 2016, pp. 1967-1972. ISBN: 978-1-5090-3762-9. DOI: 10.1109/IROS.2016. 7759310 .

Rafael Oliveira, Lionel Ott, and Fabio Ramos. 'Bayesian Optimisation under Uncertain Inputs'. In: Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics . PMLR, 2019, pp. 1177-1184.

Rafael Oliveira, Lionel Ott, and Fabio Ramos. Bayesian optimisation under uncertain inputs . 2019. arXiv: 1902.07908 .

Nicholas D. Sanders et al. Bayesian Search for Robust Optima . 2021. arXiv: 1904.11416 .

Bobak Shahriari et al. 'Taking the Human Out of the Loop: A Review of Bayesian Optimization'. In: Proceedings of the IEEE 104.1 (Jan. 2016), pp. 148-175. DOI: 10.1109/JPROC. 2015.2494218 .

Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 'Practical bayesian optimization of machine learning algorithms'. In: Advances in neural information processing systems 25 (2012).

Niranjan Srinivas et al. 'Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design'. In: International Conference on Machine Learning . 2009. URL: https://api.semanticscholar.org/CorpusID:59031327 .

Zi Wang and Stefanie Jegelka. 'Max-value entropy search for efficient Bayesian optimization'. In: International Conference on Machine Learning . PMLR. 2017, pp. 3627-3635.

Colin White, Willie Neiswanger, and Yash Savani. 'Bananas: Bayesian optimization with neural architectures for neural architecture search'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 35. 12. 2021, pp. 10293-10301.

Christopher Williams and Matthias Seeger. 'Using the Nyström Method to Speed Up Kernel Machines'. In: Advances in Neural Information Processing Systems . Vol. 13. MIT Press, 2000.

Christopher KI Williams and Carl Edward Rasmussen. Gaussian processes for machine learning . Vol. 2. 3. MIT press Cambridge, MA, 2006.

A Nyström Estimator Error Bound

Nyström estimator can easily approximate the kernel mean embedding ψ p 1 , ψ p 2 as well as the MMD distance between two distribution density p 1 and p 2 . We need first assume the boundedness of the feature map to the kernel k :

Assumption 2. There exists a positive constant K ≤ ∞ such that sup x ∈X ∥ ϕ ( x ) ∥ ≤ K

The true MMD distance between p 1 and p 2 is denoted as MMD ( p 1 , p 2 ) . The estimated MMD distance when using a Nyström sample size m i , sub-sample size h i for p i respectively, is denoted as MMD ( p i ,m i ,h i ) . Then the error

and now we have the lemma from Theorem 5.1 in [8]

Lemma 1. Let Assumption 2 hold. Furthermore, assume that for i ∈ 1 , 2 , the data points X i 1 , · · · , X i m i are drawn i.i.d. from the distribution ρ i and that h i ≤ m i sub-samples ˜ X i 1 , · · · , ˜ X i h i are drawn uniformly with replacement from the dataset { X i 1 , · · · , X i m i } . Then, for any δ ∈ (0 , 1) , it holds with probability at least 1 -2 δ

provided that, for i ∈ { 1 , 2 } ,

where c 1 = 2 K √ 2 log(6 /δ ) , c 2 = 4 √ 3 K log(12 /δ ) and c 4 = 6 K √ log(12 /δ ) . The notation N p i denotes the effective dimension associated to the distribution p k .

Specifically, when the effective dimension N satisfies, for some c ≥ 0 ,

either N ρ i ( σ 2 ) ≤ cσ 2 -γ for some γ ∈ (0 , 1) ,

or N ρ i ( σ 2 ) ≤ log(1 + c/σ 2 ) /β, for some β > 0 .

Then, choosing the subsample size m to be

h i = m 1 / (2 -γ ) i log( m i /δ ) in the first case

or h i = √ m i log( √ m i max(1 /δ, c/ (6 K 2 )) in the second case,

we get Err ( ρ i ,m i ,h i ) = O (1 / √ m i )

B Proofs of Section 4

B.1 Exact Kernel Uncertainty GP Formulating

Following the same notation in Section 4, now we can construct a Gaussian process GP (0 , ˆ k ) modelling functions over P . This GP model can then be applied to learn ˆ f from a given set of observations D n = { ( P i , y i ) } n i =1 . Under zero mean condition, the value of ˆ f ( P ∗ ) for a given P ∗ ∈ P follows a Gaussian posterior distribution with

where y n := [ y 1 , · · · , y n ] T , ˆ k n ( P ∗ ) := [ ˆ k ( P ∗ , P 1 ) , · · · , ˆ k ( P ∗ , P n )] T and [ ˆ K n ] ij = ˆ k ( P i , P j ) .

Now we restrict our Gaussian process in the subspace P X = { P x , x ∈ X} ⊂ P . We assume the observation y i = f ( x i ) + ζ i with the noise ζ i . The input-induced noise is defined as ∆ f p x i := f ( x i ) -E P x i [ f ] = f ( x i ) -ˆ f ( P x i ) . Then the total noise is y i -E P x i [ f ] = ζ i +∆ f p x i . To formulate the regret bounds, we introduce the information gain and estimated information gain given any { P t } n t =1 ⊂ P :

and the maximum information gain is defined as ˆ γ n := sup R∈P X ; |R| = n ˆ I ( y n ; ˆ f n |R ) . Here ˆ f n := [ ˆ f ( p 1 ) , · · · , ˆ f ( p n )] T .

We define the sub-Gaussian condition as follows:

Definition 1. For a given σ ξ > 0 , a real-valued random variable ξ is said to be σ ξ -sub-Gaussian if:

Now we can state the lemma for bounding the uncertain-inputs regret of exact kernel evaluations, which is originally stated in Theorem 5 in [26].

Lemma 2. Let δ ∈ (0 , 1) , f ∈ H k , and the corresponding ∥ ˆ f ∥ ˆ k ≤ b . Suppose the observation noise ζ i = y i -f ( x i ) is conditionally σ ζ -sub-Gaussian. Assume that both k and P x satisfy the conditions for ∆ f P x to be σ E -sub-Gaussian, for a given σ E > 0 . Then, we have the following results:

The following holds for all x ∈ X and t ≥ 1 :

Running with upper confidence bound (UCB) acquisition function α ( x |D n ) = ˆ µ n ( P x ) + ˆ β n ˆ σ n ( P x ) where

and set σ 2 = 1 + 2 /n , the uncertain-inputs cumulative regret satisfies:

with probability at least 1 -δ .

Note that although the original theorem restricted to the case when ˆ k ( p, q ) = ⟨ ψ P , ψ Q ⟩ k , the results can be easily generated to other kernels over P , as long as its universal w.r.t C ( P ) given that X is compact and the mean map ψ is injective [11, 22].

B.2 Error Estimates for Inexact Kernel Approximation

Now let us derivative the inference under the introduce of inexact kernel estimations.

Theorem 3. Under the Assumption 1 for ε > 0 , let ˜ µ n , ˜ σ n , ˜ I ( y n ; ˆ f n |{ P t } n t =1 ) as defined in (13) , (14) , (21) respectively, and ˆ µ n , ˆ σ n , ˆ I ( y n ; ˆ f n |{ P t } n t =1 ) as defined in (18) , (19) , (20) . Assume max x ∈X f ( x ) = M , and assume the observation error ζ i = y i -f ( x i ) satisfies | ζ i | < A for all i . Then we have the following error bound holds with probability at least 1 -nε :

Proof. Denote e ( P ∗ , Q ) = ˜ k ( P ∗ , Q ) -ˆ k ( P ∗ , Q ) , e n ( P ∗ ) = [ e ( P ∗ , P 1 ) , · · · , e ( P ∗ .P n )] T , and [ E n ] i,j = e ( P i , P j ) . Now according to the matrix inverse perturbation expansion,

we have

thus

Notic that the following holds with a probability at least 1 -nε , according to the Assumption 1,

here we use the fact that ˆ K n semi-definite (which means ∥ ( ˆ K n + σ 2 I ) -1 ∥ 2 ≤ σ -2 ), ˆ k ( P ∗ , P ∗ ) ≤ 1 , | y i | ≤ M + A . Combining these results, we have that

holds with a probability at least 1 -nε .

Similarly, we can conduct the same estimation to e n ( P ) T ( ˆ K n + σ 2 I ) -1 ˆ k n ( P ∗ ) and ˆ k n ( P ) T ( ˆ K n + σ 2 I ) -1 E n ( ˆ K n + σ 2 I ) -1 ˆ k n ( P ∗ ) , and get

holds with a probability at least 1 -nε .

It remains to estimate the error for estimating the information gain. Notice that, with a probability at least 1 -nε ,

here the second equation uses the fact that det( AB -1 ) = det( A ) det( B ) -1 , and the third and fourth equations use log det( I + A ) = Tr log( I + A ) = Tr ( A -A 2 2 + · · · ) . The last inequality follows from the fact

Tr ( σ 2 I + ˆ K n ) -1 E n ) ≤ ∥ ( σ 2 I + ˆ K n ) -1 ∥ F ∥ E n ∥ F ≤ n 3 / 2 σ -2 e ε and ˆ K n is semi-definite.

With the uncertainty bound given by Lemma 3, let us prove that under inexact kernel estimations, the posterior mean is concentrated around the unknown reward function ˆ f

Theorem 4. Under the former setting as in Theorem 3, with probability at least 1 -δ -nε , let σ ν = √ σ 2 ζ + σ 2 E , taking σ = 1 + 2 n , the following holds for all x ∈ X :

Proof. According to Lemma 2, equation (22), we have with

Notice that

We also have (25), which means

combining (24), (28) and (29), we finally get the result in (27).

B.3 Proofs for Theorem 1

Now we can prove our main theorem 1.

Proof of Theorem 1. Let x ∗ maximize ˆ f ( P x ) over X . Observing that at each round n ≥ 1 , by the choice of x n to maximize the aquisition function ˜ α ( x |D n -1 ) = ˜ µ n -1 ( P x ) + β n -1 ˜ σ n -1 ( P x ) , we have

Here we denote Err ( n, e ε ) := ( β n (1 + n ) + ˜ σ n ( P x ) σ ν n 3 / 4 ) e 1 / 2 ε + ( n + n 2 ) ( M + A ) e ε . The second inequality follows from (27),

and the third inequality follows from the choice of x n :

Thus we have

From Lemma 4 in [9], we have that

and thus

On the other hand, notice that

we find that the e ε term in (30) can be controlled by in (31), thus we immediately get the result.

B.4 Proofs for Theorem 2

Proof. Define the square of the MMD distance between P x 1 , P x 2 as d M ( x 1 , x 2 ) , we have

It is not hard to verify that d M is shift invariant: d M ( x 1 , x 2 ) = d M ( x 1 -x 2 , 0) , and d M has r -th bounded derivatives, thus ˆ k ∗ ( x 1 , x 2 ) := ˆ k ( P x 1 , P x 2 ) = exp( -αd M ( x 1 , x 2 )) is shift invariant with r -th bounded derivatives. Then take µ ( x ) as the Lebesgue measure over X , according to Theorem 4, [17], the integral operator T k,µ : T k,µ f ( x ) = ∫ X K ( x, y ) f ( y ) dµ ( y ) is a symmetric compact operator in L 2 ( X , µ ) , and the spectrum of T k,µ satisfies

Then according to Theorem 5 in [30], we have ˆ γ n = O ( n d ( d +1) r + d ( d +1) log( n )) , which finish the proof.

C Evaluation Details

C.1 Implementation

In our implementation of AIRBO, we design the kernel k used for MMD estimation to be a linear combination of multiple Rational Quadratic kernels as its long tail behavior circumvents the fast decay issue of kernel [6]:

where l i is a learnable lengthscale and a i determines the relative weighting of large-scale and small-scale variations.

Depending on the form of input distributions, the sampling and sub-sampling sizes for Nyström MMD estimator are empirically selected via experiments. Moreover, as the input uncertainty is already modeled in the surrogate, we employ a classic UCB-based acquisition as Eq. 5 with β = 2 . 0 and maximize it via an L-BFGS-B optimizer.

D More Experiments

D.1 Comparing with the Other Models

To compare the modeling performances with the other models, we design the input uncertainty to follow a step-changing Chi-squared distribution: P x = χ 2 ( g ( x ) , σ = 0 . 01) , where g ( x ) = 0 . 5 if x ∈ [0 . 0 , 0 . 6) and g ( x ) = 7 . 0 when x ∈ [0 . 6 , 1 . 0] . Due to this sudden parameter change, the uncertainty at point x = 0 . 6 is expected to be asymmetric: 1) on its left-hand side, as the Chi-squared distribution becomes quite lean and sharp with a small value of g ( x ) = 0 . 5 , the distance from x = 0 . 6 to its LHS points, x lhs ∈ [0 . 0 , 0 . 6) , are relatively large, thus their covariances are small, resulting a fast-growing uncertainty. 2)Meanwhile, when x ∈ [0 . 6 , 1 . 0] , the g ( x ) suddenly increases to 7 . 0 , rendering the input distribution a quite flat one with a long tail. Therefore, the distances between x = 0 . 6 and its RHS points become relatively small, which leads to large covariances and small uncertainties for points in [0 . 6 , 1 . 0] . As a result, we expect to observe an asymmetric posterior uncertainty at x = 0 . 6 .

Several surrogate models are employed in this comparison, including:

MMDGP-nystrom(160/10) is our method with a sampling size m = 160 and sub-sampling size h = 10 . Its complexity is O ( MNmh ) , where M and N are the sizes of training and test samples (Note: all the models in this experiment use the same training and testing samples for a fair comparison).

uGP(40) is the surrogate from [25], which employs an integral kernel with sampling size m = 40 . Due to its O ( MNm 2 ) complexity, we set the sampling size m = 40 to ensure a similar complexity as ours.

uGP(160) is also the surrogate from [25] but uses a much larger sampling size ( m = 160 ). Given the same training and testing samples, its complexity is 16 times higher than MMDGP-nystrom(160/10) .

skl is a robust GP surrogate equipped with a symmetric KL-based kernel, which is described in [20].

ERBF [13] assumes the input uncertainty to be Gaussians and employs a close-form expected RBF kernel.

GP utilizes a noisy Gaussian Process model with a learnable output noise level.

According to Figure 7a, our method, MMDGP-nystrom(160/10), can comprehensively quantify the sudden change of the input uncertainty, evidenced by its abrupt posterior change at x = 0 . 6 . However, Figure 7b shows that uGP(40) with the same complexity fails to model the uncertainty correctly. We suspect this is because uGP requires much larger samples to stabilize its estimation of the integral kernel and thus can perform poorly with insufficient sample size, so we further evaluate the uGP(160) with a much larger sampling size m = 160 in Figure 7c. It does successfully alleviate the issue but also results in a 16 times higher complexity. Apart from this, Figure 7d suggests the noisy GP model with a learnable output noise level is not aware of this uncertainty change at all as it treats the inputs as the exact values instead of random variables. Moreover, Figure 7e and 7f show that both the skl and ERBF fail in this case, this may be due to their misassumption of Gaussian input uncertainty.

D.2 Ablation Test for Nyström Approximation

In this experiment, we aim to examine the effect of Nyström approximation for optimization. To this end, we choose to optimize an RKHS function (Figure 4a) under a beta input distribution: P x = beta ( α = 0 . 4 , β = 0 . 2 , σ = 0 . 1) . Several amortized candidates include:

MMDGP-nystrom(160/10)

xc_Is=0.215,xc_0s=0.723,noise=0.035

F09

2std

Model prediction

40

mean

samples

20

0

20

0.0

0.2

0.4

0.6

0.8

1.0

(a) MMD-GP with a Nystrom estimator, in which the sampling size m = 160 and sub-sampling size h = 10 .

GP

60

2std

gdt

prediction

40

mean

samples

20

Model

0

-20

0.0

0.2

0.4

0.6

0.8

1.0

(d) Conventional noisy GP model.

uGP(40)

xc_Is=0.634,xc_0s=0.787,noise=0.035

60

2std

gdt

Model prediction

40

mean

x

samples

20

0

X

20

0.0

0.2

0.4

0.6

0.8

1.0

(b) uGP model that uses an integral kernel [26] and a sampling size of m = 40 .

skl

F09

2std

prediction

40

X

mean

samples

20

Model

0

X

20

0.0

0.2

0.4

0.6

0.8

1.0

(e) GP model with a symmetric KL-divergence kernel [20].

uGP(160)

60

2std

gdt

prediction

40

mean

samples

20

Model

0

20

0.0

0.2

0.4

0.6

0.8

1.0

(c) uGP with an integral kernel [26] and uses a much larger sampling size ( m = 160 ).

ERBF

60

2std

gdt

prediction

40

mean

samples

20

Model

0

X

-20

0.0

0.2

0.4

0.6

0.8

1.0

(f) Robust GP model with an expected RBF kernel [13]

MMDGP-nystrom

MMDGP-raw-S

MMDGP-nystrom

MMDGP-raw-S

GP

MMDGP-raW-L

GP

MMDGP-raW-L

101

101

Regret(mean)

Regret(std)

100

100

10-1

10-2

10-1

20

40

60

80

100

20

40

60

80

100

step

step

MMDGP-nystrom is our method with Nystrom approximation, in which the sampling size m = 16 and sub-sampling size h = 9 . Its complexity is O ( MNmh ) , where M and N are the sizes of training and test samples respectively, m is the sampling size for MMD estimation, and h indicates the sub-sampling size during the Nystrom approximation.

MMDGP-raw-S does not use the Nystrom approximation but employs an empirical MMD estimator. Due to its O ( MNm 2 ) complexity, we set the sampling size m = 12 to ensure a similar complexity as the MMDGP-nystrom .

MMDGP-raw-L also uses an empirical MMD estimator, but with a larger sampling size ( m = 16 ).

GP utilizes a vanilla GP with a learnable output noise level and optimizes with the upperconfidence-bound acquisition 2 .

According to Figure 8, we find that 1) with sufficient computation power, the MMDGP-raw-L can obtain the best performance by using a large sample size. 2)However, with limited complexity, the performance MMDGP-raw-S degrades obviously while the MMDGP-nystrom performs much better. This suggests that the Nyström approximation can significantly improve the efficiency with a mild cost of performance degradation. 3) All the MMDGP-based methods are better than the vanilla GP-UCB .

60

2.0

100

MMDGP-nystrom

GP

uGP

ERBF

50

MMDGP-nystrom

GP

uGP

ERBF

1.5

80

skl

skl

1.0

Regret(mean)

Regret(std)

40

0.5

60

30

0.0

40

20

0.5

20

10

1.0

1.5

0

0

2.0

0

20

40

60

80

100

0

20

40

60

80

100

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

step

step

Figure 10: The input GMM distribution.

M-GP

M-ERBF

10.5

10.5

9.0

4

D.

9.0

7.5

2

7.5

品

6.0

6.0

0

4.5

4.5

2

3.0

3.0

品

品

1.5

4

1.5

0.0

-6

0.0

中

0

0

M-skI

M-MMDGP

10.5

10.5

9.0

9.0

7.5

7.5

6.0

6.0

0-

4.5

4.5

-2

3.0

3.0

品

1.5

1.5

口

品品

0.0

-6

0.0

D.3 Optimization on 10D Bumped-Bowl Problem

To further evaluate AIRBO's optimization performance on the high-dimensional problem, we employ a 10-dimensional bumped bowl function from [27, 19]:

Here, x i is the i-th dimension of x , x 1:2 represents the first 2 dimensions for the variable, and x 3: indicates the rest dimensions. The input uncertainty is designed to follow a concatenated distribution of a 2D circular distribution( r = 0 . 5 ) and a multivariate normal distribution with a zero mean and diagonal covariance of 0.01.

Figure 9 shows the mean and std values of the optimization regrets. We note that 1)when it comes to a high-dimensional problem and complex input distribution, the misassumption of Gaussian input uncertainty renders the skl and ERBF fail to locate the robust optimum and get stuck at local optimums. 2)Our method outperforms the others and can find the robust optimum efficiently and stably, while the uGP with a similar inference cost suffers the instability caused by insufficient sampling and stumbles over iterations, which can be evidenced by its high std values of optimization regret.

D.4 Robust Robot Pushing

This benchmark is based on a Box2D simulator from [31], where our objective is to identify a robust push configuration, enabling a robot to push a ball to predetermined targets under input randomness. In our experiment, we simplify the task by setting the push angle to r a = arctan r y r x , ensuring the robot is always facing the ball. Also, we intentionally define the input distribution as a two-component Gaussian Mixture Model as follows:

where the covariance matrix Σ is shared among components and w is the weights of mixture components. Meanwhile, as the SKL-UCB and ERBF-UCB surrogates can only accept Gaussian input distributions, we choose to approximate the true input distribution with a Gaussian. As shown in Figure 10, the approximation error is obvious, which explains the performance gap among these algorithms in Figure 5b.

Apart from the statistics of the found pre-images in Figure 6, we also simulate the robot pushes according to the found configurations and visualize the results in Figure 11. In this figure, each black hollow square represents an instance of the robot's initial location, the grey arrow indicates the push direction and duration, and the blue circle marks the ball's ending position after the push. We can find that, as the GP-UCB ignores the input uncertainty, it randomly pushes to these targets and the ball ending positions fluctuate. Also, due to the incorrect assumption of the input distribution, the SKL-UCB and ERBF-UCB fail to control the ball's ending position under input randomness. On the contrary, AIRBO successfully recognizes the twin targets in quadrant I as an optimal choice and frequently pushes to this area. Moreover, all the ball's ending positions are well controlled and centralized around the targets under input randomness.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new method for Bayesian optimization under uncertain inputs, where the distribution of an input can be complex and unknown but can be sampled from. The paper proposes an MMD based kernel between probability distributions and the use of the Nystrom approximation to make the computations tractable. A UCB-type algorithm is used. The paper provides regret bounds accounting for the approximation error due to sampling / Nystrom approximation, and compares the algorithm to some baselines.\n\nSTRENGTHS:\n1. The motivation of requiring to do BO with uncertain inputs with complex distributions that may not be known in closed form is compelling.\n2. The regret bound incorporating approximation error and the corresponding implications on the sampling size $m$ are useful and interesting.\n\nIn general I believe the work has potential given that the technical issues raised in the Weaknesses section are sufficiently addressed.\n\nWEAKNESSES:\n1. **Validity of kernel**. The paper designs an MMD-based kernel between probability distributions $\\hat k = \\eta(\\text{MMD}(P, Q))$. When designing a new kernel, it is important to prove that it is a valid kernel. However, no such proof is given (or references to previous literature that does something similar). $\\eta$ is said to be a \"normalizing function with range $[0, 1]$\". What conditions does $\\eta$ need to fulfill in order for $\\hat k$ to be a valid kernel?\n\n2. **Empirical evaluation**. The empirical evaluation is unconvincing for the following reasons:\n* The Nystrom approximation is adopted for efficiency. It is important to measure how much using this approximation affects performance beyond simply a qualitative comparison by looking at the posteriors in Figure 3. The empirical evaluation should do an ablation study that includes MMDGP without the Nystrom approximation, at a sampling size that results in the same inference time as that with the Nystrom approximation. What if removing the Nystrom approximation results in better performance with no decrease in efficiency?\n* The closest work seems to be Oliveira et. al. (2019). Why isn't their algorithm one of the baselines?\n* The error bars are simply too large and overlap too much. Any performance improvement could be due to randomness; please run with more trials to decrease the error bars so that the performances can be meaningfully compared.\n\n3. **Writing**. The writing has much to be improved:\n* Line 174 \"One important theoretical guarantee to conduct GP model...\" and in Theorem 1 \"...running Gaussian Process with acquisition function...\" I believe you mean \"Bayesian optimization\" in these contexts instead of \"Gaussian process\". The GP is the model, the algorithm is BO which relies on a GP. The thing being run is the algorithm, not the model.\n* Line 151 \"sampling size $N$\" do you mean $m$?\n* Many grammatical and typo errors: the last word in the title is not capitalized; line 178 \"For $\\hat k$ be radial kernels\" and \"For $\\hat k$ be linear kernel\", should be, for example, \"If $\\hat k$ is a radial kernel\"; Lines 201 and 205 are missing periods; Line 256 \"can be more pronounced impact\" should be \"can have\".\n* [24] and [25] point to the same reference.\n\nQUESTIONS:\nNo additional questions.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-03"}

comments: SUMMARY:
This work focuses on the situations where input uncertainty arises and the input values are unobservable, and introduces to measure the distance of uncertain inputs through MMD when training the Gaussian process surrogate. The authors theoretically and empirically demonstrate the effectiveness of the proposed method.

STRENGTHS:
1.	The proposed method is sound and MMD can be a metric to measure the distances of uncertain inputs.

2.	The experiments are extensive to show the effectiveness of the proposed method.

WEAKNESSES:
1.	MMD-GP needs to query m times more samples than GP, which can restrict its application since BO is usually applied to the tasks where evaluating a query can be time-consuming. What about the time cost of this work compared to other methods for input uncertainty?

2.	After sampling m times for one query x, we can also get m rewards/performance. I wonder how to deal with the m rewards/performance? Line 139 shows that $D_n=${$(x_i, y_i)|x_i \sim P_i$}. However, it seems that $y_i$ also should be a distribution.

3.	Other typos:

3.1	Eq. 2 use $\xi$ to denote the noise while line 92 utilize $\epsilon$.

3.2	Line 151: “the computation and space complexities of the empirical MMD estimator scale quadratically with the sampling size N” Should N be m?

3.3	What does the training and testing samples stand for in line 162? Do you mean the number of observations and the number of queries?

QUESTIONS:
Please see the weakness.

LIMITATIONS:
No broader societal impacts are provided.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel Bayesian Optimization (BO) algorithm that explicitly tackles input uncertainty by introducing a new integral probabilistic metric (IPM)-based kernel. The algorithm also utilizes an efficient and stable Nystrom estimator to approximate the Maximum Mean Discrepancy (MMD), which serves as the adopted IPM. Furthermore, the paper extends the GP-UCB framework by incorporating the proposed kernel and derives the corresponding upper bound for the cumulative regret. The empirical study illustrates the effectiveness of the proposed approximation and the BO algorithm.\n\nSTRENGTHS:\n1. The paper is generally well-organized and illustrative with figures and tabular results.\n2. The theoretical result extends the existing UCB work with reasonable approximation to tackle the input uncertainty.\n3. The limitation of the proposed AIRBO on dealing with high-dimensional input and lack of discussion over other IPMs is clearly stated.\n\nWEAKNESSES:\n1. The theoretical bound is arguably sound. Due to the numerical approximation of the Maximum Mean Discrepancy (MMD), it may potentially result in pseudo-metric or even worse outcomes in practice. Therefore, it remains unclear whether the upper bound of the maximum information gain $\\gamma_T$ still holds and guarantees that the UCB cumulative regret is sublinear.\n\n2. The essential justification for the proposed Nystrom estimator lies in assumption 1. While the author claims in lines 210 and 211 that the approximation error can be fairly small, the paper lacks sufficient discussion or substantiation to support this claim.\n\n3. The empirical results are limited. The regret curve lacks statistical significance, and more extensive empirical studies on real-world applications would be desirable.\n\nQUESTIONS:\n1. What is SKL-UCB mentioned in line 262? Is it referring to the algorithm discussed in line 45 from [19]?\n2. What is the corresponding reference for Oliveira et al. mentioned in line 50?\n3. Despite the explicit treatment of input uncertainty, it is unclear to me whether tolerating the input uncertainty by fixing a larger output noise level could provide a simpler solution. The author claims that the GP model overfits (line 222) and justifies the use of MMD-GP. However, one would expect that fixing a larger noise level for the GP model would help mitigate it as well. Additionally, the existing method addressing heteroscedastic noise proposed in [1] potentially aids in mitigating the input uncertainty purely on the output side. It would be valuable to hear the authors' comments on this.\n4. Could the author provide more intermediate quantitative evidence showing the effectiveness of the proposed MMD-based kernel? For example, any evidence showing that the uncertainty quantification is better than the naïve GP model that doesn't explicitly deal with the input uncertainty?\n\nLIMITATIONS:\nLImitations are mentioned in the comments above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the problem of robust Bayesian Optimization (BO) with uncertain inputs, i.e., the input values are deviated from the intended value before evaluation. The paper proposes a new technique, namely AIRBO (Arbitrary Input uncertainty Robust Bayesian Optimization), that can model the uncertain input and incorporate this uncertainty into the surrogate model, and thus can be used to guide the search of the objective function global optimum. The paper further proposes to use Nystrom approximation to reduce the computational cost. Theoretical analysis is conducted to guarantee the performance of the proposed technique. Experiments are conducted on some synthetic and one real-world problem to evaluate the performance of the proposed method.\n\nSTRENGTHS:\n+ The paper’s writing is generally clear and easy to understand. Illustration and figures are plotted nicely to explain the problem setting, as well as the property of the proposed method.\n+ The proposed method of using MMD to construct a kernel that can incorporate uncertainty of the inputs seems to be interesting to me.\n+ Theoretical analysis is conducted to guarantee the convergence property of the proposed method (although note that I have some concerns regarding the theoretical analysis).\n+ The experiments are described very detailed. Various experiments are conducted to understand the performance of each component within the proposed method.\n\nWEAKNESSES:\n+ I think the application of this problem setting should be motivated much better as it’s unclear on the significance of the problem tackled in this paper. In the Introduction, the paper only mentions that this problem is quite common for robotics and process controls, but no references or further explanations are provided.\n+ Some notations are quite confusing, making it hard for me to follow the proposed method. In Eq. (8), the paper shows how to compute MMD(P,Q) from the m samples {x_i}_1^m, {y_i}_1^m – but y is defined as the objective function value. Then how can we compute the kernel \\hat{k}(P_{x_i}, P_{x_j}) in Eqs. (10) and (11)? \n+ I also have some concerns with the theoretical analysis. In the theoretical analysis, there is no formal proof to prove the proposed kernel is actually a valid kernel. There is just one paragraph in Lines 170-173 briefly mentioning about this but no formal proof is given to know if the proposed kernel is valid, and in which scenarios it will be a valid kernel. Assumption 1 is too strong, it assumes that we already know that the error function e(P,Q) can be uniformly upper-bounded. In practice, can this assumption be true? How can we know it can be true? Theorem 1 also doesn’t have much meaning to me. In Theorem 1, \\tilde{\\beta}_n is defined with the maximum information gain \\tilde{I} within the formula. This is not really a standard analysis in BO’s theory, normally \\beta is defined by some constants. Is this maximum information gain \\tilde{I} bounded? \n+ Experiments are mostly conducted with synthetic functions. There is only one real-world problem being used in the experiment, and it’s quite simple to me. It’s just a 3-dim problem.\n+ What is the time cost of the proposed method?\n\nQUESTIONS:\nPlease answer my comments and questions in the Weaknesses section.\n\nLIMITATIONS:\nI can’t find any dedicated section that describes about the limitations of the proposed technique. There are some future work mentioned in the Conclusion, but there are no limitations mentioned there.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}