SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Efficient Robust Bayesian Optimization for Arbitrary Uncertain Inputs
Lin Yang
yanglin33@huawei.com
Junlong Lyu
lyujunlong@huawei.com
Wenlong Lyu
lvwenlong2@huawei.com
Zhitang Chen
chenzhitang2@huawei.com
Bayesian Optimization (BO) is a sample-efficient optimization algorithm widely employed across various applications. In some challenging BO tasks, input uncertainty arises due to the inevitable randomness in the optimization process, such as machining errors, execution noise, or contextual variability. This uncertainty deviates the input from the intended value before evaluation, resulting in significant performance fluctuations in final result. In this paper, we introduce a novel robust Bayesian Optimization algorithm, AIRBO, which can effectively identify a robust optimum that performs consistently well under arbitrary input uncertainty. Our method directly models the uncertain inputs of arbitrary distributions by empowering the Gaussian Process with the Maximum Mean Discrepancy (MMD) and further accelerates the posterior inference via Nyström approximation. Rigorous theoretical regret bound is established under MMD estimation error and extensive experiments on synthetic functions and real problems demonstrate that our approach can handle various input uncertainties and achieve a state-of-the-art performance.
Bayesian Optimization (BO) is a powerful sequential decision-making algorithm for high-cost black-box optimization. Owing to its remarkable sample efficiency and capacity to balance exploration and exploitation, BO has been successfully applied in diverse domains, including neural architecture search [32], hyper-parameter tuning [4, 29, 12], and robotic control [18, 5], among others. Nevertheless, in some real-world problems, the stochastic nature of the optimization process, such as machining error during manufacturing, execution noise of control, or variability in contextual factor, inevitably introduces input randomness, rendering the design parameter  $x$  to deviate to  $x'$  before evaluation. This deviation produces a fluctuation of function value  $y$  and eventually leads to a performance instability of the outcome. In general, the input randomness is determined by the application scenario and can be of arbitrary distribution, even quite complex ones. Moreover, in some cases, we cannot observe the exact deviated input  $x'$  but a rough estimation for the input uncertainty. This is quite common for robotics and process controls. For example, consider a robot control task shown in Figure 1a, a drone is sent to a target location  $x$  to perform a measurement task. However, due to the execution noise caused by the fuzzy control or a sudden wind, the drone ends up at location  $x' \sim P(x)$  and gets a noisy measurement  $y = f(x') + \zeta$ . Instead of observing the exact value of
(a) An example case: drone measurement with execution noise.
Figure 1: Robust Bayesian optimization problem.
(b) Problem formulation.
$x^{\prime}$ , we only have a coarse estimation of the input uncertainty  $P(x)$ . The goal is to identify a robust location that gives the maximal expected measurement under the process randomness.
To find a robust optimum, it is crucial to account for input uncertainty during the optimization process. Existing works [24, 3, 7, 10] along this direction assume that the exact input value, i.e.,  $x'$  in Figure 1b, is observable and construct a surrogate model using these exact inputs. Different techniques are then employed to identify the robust optimum: Nogueira et al. utilize the unscented transform to propagate input uncertainty to the acquisition function [24], while Beland and Nair integrate over the exact GP model to obtain the posterior with input uncertainty [3]. Meanwhile, [7] designs a robust confidence-bounded acquisition and applies min-max optimization to identify the robust optimum. Similarly, [10] constructs an adversarial surrogate with samples from the exact surrogate. These methods work quite well but are constrained by their dependence on observable input values, which may not always be practical.
An alternative approach involves directly modeling the uncertain inputs. A pioneering work by Moreno et al. [20] assumes Gaussian input distribution and employs a symmetric Kullback-Leibler divergence (SKL) to measure the distance of input variables. Dallaire et al. [13] implement a Gaussian process model with an expected kernel and derive a closed-form solution by restricting the kernel to linear, quadratic, or squared exponential kernels and assuming Gaussian inputs. Nonetheless, the applicability of these methods is limited due to their restrictive Gaussian input distribution assumption and kernel choice. To surmount these limitations, Oliveira et al. propose a robust Gaussian process model that incorporates input distribution by computing an integral kernel. Although this kernel can be applied to various distributions and offers a rigorous regret bound, its posterior inference requires a large sampling and can be time-consuming.
In this work, we propose an Arbitrary Input uncertainty Robust Bayesian Optimization algorithm (AIRBO). This algorithm can directly model the uncertain input of arbitrary distribution and propagate the input uncertainty into the surrogate posterior, which can then be used to guide the search for robust optimum. To achieve this, we employ Gaussian Process (GP) as the surrogate and empower its kernel design with the Maximum Mean Discrepancy (MMD), which allows us to comprehensively compare the uncertain inputs in Reproducing Kernel Hilbert Space (RKHS) and accurately quantify the target function under various input uncertainties(Sec. 3.1). Moreover, to stabilize the MMD estimation and accelerate the posterior inference, we utilize Nyström approximation to reduce the space complexity of MMD estimation from  $O(m^2)$  to  $O(mh)$ , where  $h \ll m$  (Sec. 3.2). This can substantially improve the parallelization of posterior inference and a rigorous theoretical regret bound is also established under the approximation error (Sec. 4). Comprehensive evaluations on synthetic functions and real problems in Sec.5 demonstrate that our algorithm can efficiently identify robust optimum under complex input uncertainty and achieve state-of-the-art performance.
In this section, we first formulate the robust optimization problem under input uncertainty then briefly review the intuition behind Bayesian Optimization and Gaussian Processes.
2.1 Optimization with Input Uncertainty
As illustrated in Figure 1b, we consider an optimization of expensive black-box function:  $f(x)$ , where  $x$  is the design parameter to be tuned. At each iteration  $n$ , we select a new query point  $x_{n}$
according to the optimization heuristics. However, due to the stochastic nature of the process, such as machining error or execution noise, the query point is perturbed to  $x_{n}^{\prime}$  before the function evaluation. Moreover, we cannot observe the exact value of  $x_{n}^{\prime}$  and only have a vague probability estimation of its value:  $P_{x_n}$ . After the function evaluation, we get a noisy measurement  $y = f(x_n') + \zeta_n$ , where  $\zeta_n$  is homogeneous measurement noise sampled from  $\mathcal{N}(0,\sigma^2)$ . The goal is to find an optimal design parameter  $x^{*}$  that maximizes the expected function value under input uncertainty:
$$
x ^ {*} = \arg \max  _ {x} \int_ {x ^ {\prime} \sim P _ {x}} f \left(x ^ {\prime}\right) d x ^ {\prime} = \arg \max  _ {x} \mathbb {E} _ {P _ {x}} [ f ] \tag {1}
$$
Depending on the specific problem and randomness source, the input distribution  $P_{x}$  can be arbitrary in general and even become quite complex sometimes. Here we do not place any additional assumption on them, except assuming we can sample from these input distributions, which can be easily done by approximating it with Bayesian methods and learning a parametric probabilistic model [16]. Additionally, we assume the exact values of  $x'$  are inaccessible, which is quite common in some real-world applications, particularly in robotics and process control [25].
2.2 Bayesian Optimization
In this paper, we focus on finding the robust optimum with BO. Each iteration of BO involves two key steps: I) fitting a surrogate model and II) maximizing an acquisition function.
Gaussian Process Surrogate: To build a sample-efficient surrogate, we choose Gaussian Process (GP) as the surrogate model in this paper. Following [34], GP can be interpreted from a weight-space view: given a set of  $n$  observations,  $\mathcal{D}_n = \{(x_i,y_i)|i = 1,\dots,n\}$ . Denote all the inputs as  $X \in \mathbb{R}^{D\times n}$  and all the output vector as  $y \in \mathbb{R}^{n\times 1}$ . We first consider a linear surrogate:
$$
f (x) = x ^ {T} w, y = f (x) + \zeta , \zeta \sim \mathcal {N} (0, \sigma^ {2}), \tag {2}
$$
where  $w$  is the model parameters and  $\zeta$  is the observation noise. This model's capacity is limited due to its linear form. To obtain a more powerful surrogate, we can extend it by projecting the input  $x$  into a feature space  $\phi(x)$ . By taking a Bayesian treatment and placing a zero mean Gaussian prior on the weight vector:  $w \sim \mathcal{N}(0, \Sigma_p)$ , its predictive distribution can be derived as follows (see Section2.1 of [34] for detailed derivation):
$$
\begin{array}{l} f _ {*} | x _ {*}, X, y \sim \mathcal {N} \left(\phi^ {T} (x _ {*}) \Sigma_ {p} \phi (X) (A + \sigma_ {n} ^ {2} I) ^ {- 1} y, \right. \\ \phi^ {T} \left(x _ {*}\right) \Sigma_ {p} \phi \left(x _ {*}\right) - \phi^ {T} \left(x _ {*}\right) \Sigma_ {p} \phi (X) \left(A + \sigma_ {n} ^ {2} I\right) ^ {- 1} \phi^ {T} (X) \Sigma_ {p} \phi \left(x _ {*}\right)), \tag {3} \\ \end{array}
$$
where  $A = \phi^T(X)\Sigma_p\phi(X)$  and  $I$  is a identity matrix. Note the predictive distribution is also a Gaussian and the feature mappings are always in the form of inner product with respect to  $\Sigma_p$ . This implies we are comparing inputs in a feature space and enables us to apply kernel trick. Therefore, instead of exactly defining a feature mapping  $\phi(\cdot)$ , we can define a kernel:  $k(x,x') = \phi(x)^T\Sigma_p\phi(x') = \psi(x)\cdot\psi(x')$ . Substituting it into Eq. 3 gives the vanilla GP posterior:
$$
\begin{array}{l} f _ {*} | X, y, X _ {*} \sim \mathcal {N} \left(\mu_ {*}, \Sigma_ {*}\right), \text {w h e r e} \mu_ {*} = K \left(X _ {*}, X\right) \left[ K (X, X) + \sigma_ {n} ^ {2} I \right] ^ {- 1} y, \\ \Sigma_ {*} = K \left(X _ {*}, X _ {*}\right) - K \left(X _ {*}, X\right) \left[ K (X, X) + \sigma_ {n} ^ {2} I \right] ^ {- 1} K \left(X, X _ {*}\right). \tag {4} \\ \end{array}
$$
From this interpretation of GP, we note that its core idea is to project the input  $x$  to a (possibly infinite) feature space  $\psi(x)$  and compare them in the Reproducing Kernel Hilbert Space (RKHS) defined by kernel.
Acquisition Function Optimization: Given the posterior of GP surrogate model, the next step is to decide a query point  $x_{n}$ . The exploitation and exploration balance is achieved by designing an acquisition function  $\alpha (x|\mathcal{D}_n)$ . Through numerous acquisition functions exist [28], we follow [25, 7] and adopt the Upper Confidence Bound (UCB) acquisition:
$$
\alpha (x \mid \mathcal {D} _ {n}) = \mu_ {*} (x) + \beta \sigma_ {*} (x), \tag {5}
$$
where  $\beta$  is a hyper-parameter to control the level of exploration.
To cope with randomness during the optimization process, we aim to build a robust surrogate that can directly accept the uncertain inputs of arbitrary distributions and propagate the input uncertainty into
the posterior. Inspired by the weight-space interpretation of GP, we empower GP kernel with MMD to compare the uncertain inputs in RKHS. In this way, the input randomness is considered during the covariance computation and naturally reflected in the resulting posterior, which then can be used to guide the search for a robust optimum(Sec. 3.1). To further accelerate the posterior inference, we employ Nyström approximation to stabilize the MMD estimation and reduce its space complexity (Sec. 3.2).
3.1 Modeling the Uncertain Inputs
Assume  $P_x \in \mathcal{P}_X \subset \mathcal{P}$  are a set of distribution densities over  $\mathbb{R}^d$ , representing the distributions of the uncertain inputs. We are interested in building a GP surrogate over the probability space  $\mathcal{P}$ , which requires to measure the difference between the uncertain inputs.
To do so, we turn to the Integral Probabilistic Metric (IPM) [23]. The basic idea behind IPM is to define a distance measure between two distributions  $P$  and  $Q$  as the supremum over a class of functions  $\mathcal{G}$  of the absolute expectation difference:
$$
d (P, Q) = \sup  _ {g \in \mathcal {G}} | \mathbb {E} _ {u \sim P} g (u) - \mathbb {E} _ {v \sim Q} g (v) |, \tag {6}
$$
where  $\mathcal{G}$  is a class of functions that satisfies certain conditions. Different choices of  $\mathcal{G}$  lead to various IPMs. For example, if we restrict the function class to be uniformly bounded in RKHS we can get the MMD [15], while a Lipschitz-continuous  $\mathcal{G}$  realizes the Wasserstein distance [14].
In this work, we choose MMD as the distance measurement for the uncertain inputs because of its intrinsic connection with distance measurement in RKHS. Given a characteristic kernel  $k: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$  and associate RKHS  $\mathcal{H}_k$ , define the mean map  $\psi: \mathcal{P} \to \mathcal{H}_k$  such that  $\langle \psi(P), g \rangle = \mathbb{E}_P[g]$ ,  $\forall g \in \mathcal{H}_k$ . The MMD between  $P, Q \in \mathcal{P}$  is defined as:
$$
\operatorname {M M D} (P, Q) = \sup  _ {\left| \left| g \right| \right| _ {k} \leq 1} \left[ \mathbb {E} _ {u \sim P} g (u) - \mathbb {E} _ {v \sim Q} g (v) \right] = \left| \left| \psi_ {P} - \psi_ {Q} \right| \right|, \tag {7}
$$
Without any additional assumption on the input distributions, except we can get  $m$  samples  $\{u_i\}_{i=1}^m, \{v_i\}_{i=1}^m$  from  $P, Q$  respectively, MMD can be empirically estimated as follows [21]:
$$
\mathrm {M M D} ^ {2} (P, Q) \approx \frac {1}{m (m - 1)} \sum_ {1 \leq i, j \leq m, i \neq j} \left(k \left(u _ {i}, u _ {j}\right) + k \left(v _ {i}, v _ {j}\right)\right) - \frac {2}{m ^ {2}} \sum_ {1 \leq i, j \leq m} k \left(u _ {i}, v _ {j}\right), \tag {8}
$$
To integrate MMD into the GP surrogate, we design an MMD-based kernel over  $\mathcal{P}$  as follows:
$$
\hat {k} (P, Q) = \exp (- \alpha \mathrm {M M D} ^ {2} (P, Q)), \tag {9}
$$
with a learnable scaling parameter  $\alpha$ . This is a valid kernel, and universal w.r.t.  $C(\mathcal{P})$  under mild conditions (see Theorem 2.2, [11]). Also, it is worth to mention that, to compute the GP posterior, we only need to sample  $m$  points from the input distributions, but do not require their corresponding function values.
With the MMD kernel, our surrogate model places a prior  $\mathcal{GP}(0,\hat{k} (P_x,P_{x^{\prime}}))$  and obtain a dataset  $\mathcal{D}_n = \{(\hat{x}_i,y_i)|\hat{x}_i\sim P_{x_i},i = 1,2,\dots,n)\}$ . The posterior is Gaussian with mean and variance:
$$
\hat {\mu} _ {n} \left(P _ {*}\right) = \hat {\mathbf {k}} _ {n} \left(P _ {*}\right) ^ {T} \left(\hat {\mathbf {K}} _ {n} + \sigma^ {2} \mathbf {I}\right) ^ {- 1} \mathbf {y} _ {n} \tag {10}
$$
$$
\hat {\sigma} _ {n} ^ {2} \left(P _ {*}\right) = \hat {k} \left(P _ {*}, P _ {*}\right) - \hat {\mathbf {k}} _ {n} \left(P _ {*}\right) ^ {T} \left(\hat {\mathbf {K}} _ {n} + \sigma^ {2} \mathbf {I}\right) ^ {- 1} \hat {\mathbf {k}} _ {n} \left(P _ {*}\right), \tag {11}
$$
where  $\mathbf{y}_n\coloneqq [y_1,\dots ,y_n]^T$ $\hat{\mathbf{k}}_n(P_*)\coloneqq [\hat{k} (P_*,P_{x_1}),\dots ,\hat{k} (P_*,P_{x_n})]^T$  and  $[\hat{\mathbf{K}}_n]_{ij} = \hat{k} (P_{x_i},P_{x_j})$
3.2 Boosting posterior inference with Nyström Approximation
To derive the posterior distribution of our robust GP surrogate, it requires estimating the MMD between each pair of inputs. Gretton et al. prove the empirical estimator in Eq. 8 approximates MMD in a bounded and asymptotic way [15]. However, the sampling size  $m$  used for estimation greatly affects the approximation error and insufficient sampling leads to a high estimation variance(ref. Figure 3a).
Such an MMD estimation variance causes numerical instability of the covariance matrix and propagates into the posterior distribution and acquisition function, rendering the search for optimal query point a challenging task. Figure 3b gives an example of MMD-GP posterior with insufficient samples, which produces a noisy acquisition function and impedes the search of optima. Increasing the sampling size can help alleviate this issue. However, the computation and space complexities of the empirical MMD estimator scale quadratically with the sampling size  $m$ . This leaves us with a dilemma that insufficient sampling results in a highly-varied posterior while a larger sample size can occupy significant GPU memory and reduce the ability for parallel computation.
To reduce the space and computation complexity while retaining a stable MMD estimation, we resort to the Nyström approximation [33]. This method alleviates the computational cost of kernel matrix by randomly selecting  $h$  subsamples from the  $m$  samples  $(h \ll m)$  and computes an approximated matrix via  $\tilde{K} = K_{mh}K_h^+ K_m^T$ . Combining this with the MMD definition gives its Nyström estimator:
$$
\begin{array}{l} \mathrm {M M D} ^ {2} (P, Q) = \mathbb {E} _ {u, u ^ {\prime} \sim P} \otimes_ {P} [ k (u, u ^ {\prime}) ] + \mathbb {E} _ {v, v ^ {\prime} \sim Q} \otimes_ {Q} [ k (v, v ^ {\prime}) ] - 2 \mathbb {E} _ {u, v \sim P} \otimes_ {Q} [ k (u, v) ] \\ \approx \frac {1}{m ^ {2}} \mathbf {1} _ {m} ^ {T} U \mathbf {1} _ {m} + \frac {1}{m ^ {2}} \mathbf {1} _ {m} ^ {T} V \mathbf {1} _ {m} - \frac {2}{m ^ {2}} \mathbf {1} _ {m} ^ {T} W \mathbf {1} _ {m} \\ \approx \frac {1}{m ^ {2}} \mathbf {1} _ {m} ^ {T} U _ {m h} U _ {h} ^ {+} U _ {m h} ^ {T} \mathbf {1} _ {n} + \frac {1}{m ^ {2}} \mathbf {1} _ {m} ^ {T} V _ {m h} V _ {h} ^ {+} V _ {m h} ^ {T} \mathbf {1} _ {m} - \frac {2}{m ^ {2}} \mathbf {1} _ {m} ^ {T} W _ {m h} W _ {h} ^ {+} W _ {m h} ^ {T} \mathbf {1} _ {m} \tag {12} \\ \end{array}
$$
where  $U = K(\mathbf{u},\mathbf{u}^{\prime})$ ,  $V = K(\mathbf{v},\mathbf{v}^{\prime})$ ,  $W = K(\mathbf{u},\mathbf{v})$  are the kernel matrices,  $\mathbf{1}_m$  represents a m-by-1 vector of ones,  $m$  defines the sampling size and  $h$  controls the sub-sampling size. Note that this Nyström estimator reduces the space complexity of posterior inference from  $O(MNm^2)$  to  $O(MNmh)$ , where  $M$  and  $N$  are the numbers of training and testing samples,  $m$  is the sampling size for MMD estimation while  $h \ll m$  is the sub-sampling size. This can significantly boost the posterior inference of robust GP by allowing more inference to run in parallel on GPU.
Assume  $x \in \mathcal{X} \subset \mathbb{R}^d$ , and  $P_x \in \mathcal{P}_{\mathcal{X}} \subset \mathcal{P}$  are a set of distribution densities over  $\mathbb{R}^d$ , representing the distribution of the noisy input. Given a characteristic kernel  $k: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$  and associate RKHS  $\mathcal{H}_k$ , we define the mean map  $\psi: \mathcal{P} \to \mathcal{H}_k$  such that  $\langle \psi(P), g \rangle = \mathbb{E}_P[g], \forall g \in \mathcal{H}_k$ .
We consider a more general case. Choosing any suitable functional  $L$  such that  $\hat{k} (P,P^{\prime})\coloneqq L(\psi_P,\psi_{P'})$  is a positive-definite kernel over  $\mathcal{P}$ , for example the linear kernel  $\langle \psi_P,\psi_{P'}\rangle_k$  and radial kernels  $\exp (-\alpha \| \psi_P - \psi_{P'}\| _k^2)$  using the MMD distance as a metric. Such a kernel  $\hat{k}$  is associated with a RKHS  $\mathcal{H}_{\hat{k}}$  containing functions over the space of probability measures  $\mathcal{P}$ .
One important theoretical guarantee to conduct  $\mathcal{GP}$  model is that our object function can be approximated by functions in  $\mathcal{H}_k$ , which relies on the universality of  $\hat{k}$ . Let  $C(\mathcal{P})$  be the class of continuous functions over  $\mathcal{P}$  endowed with the topology of weak convergence and the associated Borel  $\sigma$ -algebra, and we define  $\hat{f} \in C(\mathcal{P})$  such that
$$
\hat {f} (P) := \mathbb {E} _ {P} [ f ], \forall P \in \mathcal {P},
$$
which is just our object function, For  $\hat{k}$  be radial kernels, it has been shown that  $\hat{k}$  is universal w.r.t  $C(\mathcal{P})$  given that  $\mathcal{X}$  is compact and the mean map  $\psi$  is injective [11, 22]. For  $\hat{k}$  be linear kernel which is not universal, it has been shown in Lemma 1, [26] that  $\hat{f} \in \mathcal{H}_{\hat{k}}$  if and only if  $f \in \mathcal{H}$  and further  $\| \hat{f} \|_{\hat{k}} = \| f \|_k$ . Thus, in the remain of this chapter, we may simply assume  $\hat{f} \in \mathcal{H}_{\hat{k}}$ .
Suppose we have an approximation kernel function  $\tilde{k} (P,Q)$  near to the exact kernel function  $\hat{k} (P,Q)$ . The mean  $\hat{\mu}_n(p_*)$  and variance  $\hat{\sigma}_n^2 (p_*)$  are approximated by
$$
\tilde {\mu} _ {n} \left(P _ {*}\right) = \tilde {\mathbf {k}} _ {n} \left(P _ {*}\right) ^ {T} \left(\tilde {\mathbf {K}} _ {n} + \sigma^ {2} \mathbf {I}\right) ^ {- 1} \mathbf {y} _ {n} \tag {13}
$$
$$
\tilde {\sigma} _ {n} ^ {2} \left(P _ {*}\right) = \tilde {k} \left(P _ {*}, P _ {*}\right) - \tilde {\mathbf {k}} _ {n} \left(P _ {*}\right) ^ {T} \left(\tilde {\mathbf {K}} _ {n} + \sigma^ {2} \mathbf {I}\right) ^ {- 1} \tilde {\mathbf {k}} _ {n} \left(P _ {*}\right), \tag {14}
$$
where  $\mathbf{y}_n\coloneqq [y_1,\dots ,y_n]^T$ $\tilde{\mathbf{k}}_n(P_*)\coloneqq [\tilde{k} (P_*,P_1),\dots ,\tilde{k} (P_*,P_n)]^T$  and  $[\tilde{\mathbf{K}}_n]_{ij} = \tilde{k} (P_i,P_j)$
The maximum information gain corresponding to the kernel  $\hat{k}$  is denoted as
$$
\hat {\gamma} _ {n} := \sup _ {\mathcal {R} \in \mathcal {P} _ {\mathcal {X}}; | \mathcal {R} | = n} \hat {I} (\mathbf {y} _ {n}; \hat {\mathbf {f}} _ {n} | \mathcal {R}) = \frac {1}{2} \ln \det (\mathbf {I} + \sigma^ {- 2} \hat {\mathbf {K}} _ {n}),
$$
Denote  $e(P, Q) = \hat{k}(P, Q) - \tilde{k}(P, Q)$  as the error function when estimating the kernel  $\hat{k}$ . We suppose  $e(P, Q)$  has an upper bound with high probability:
Assumption 1. For any  $\varepsilon >0$ ,  $P,Q\in \mathcal{P}_{\mathcal{X}}$ , we may choose an estimated  $\tilde{k} (P,Q)$  such that the error function  $e(P,Q)$  can be upper-bounded by  $e_{\varepsilon}$  with probability at least  $1 - \varepsilon$ , that is,  $\mathbb{P}\left(|e(P,Q)|\leq e_{\varepsilon}\right) > 1 - \varepsilon$
Remark. Note that this assumption is standard in our case: we may assume  $\max_{x\in \mathcal{X}}\| \phi \| _k\leq \Phi$  where  $\phi$  is the feature map corresponding to the  $k$ . Then when using empirical estimator, the error between  $\mathrm{MMD}_{\mathrm{empirical}}$  and MMD is controlled by  $4\Phi \sqrt{2\log(6 / \varepsilon)m^{-1}}$  with probability at least  $1 - \varepsilon$  according to Lemma E.1, [8]. When using the Nyström estimator, the error has a similar form as the empirical one, and under mild conditions, when  $h = O(\sqrt{m}\log (m))$ , we get the error of the order  $O(m^{-1 / 2}\log (1 / \varepsilon))$  with probability at least  $1 - \varepsilon$ . One can check more details in Lemma 1.
Now we restrict our Gaussian process in the subspace  $\mathcal{P}_{\mathcal{X}} = \{P_x,x\in \mathcal{X}\} \subset \mathcal{P}$ . We assume the observation  $y_{i} = f(x_{i}) + \zeta_{i}$  with the noise  $\zeta_{i}$ . The input-induced noise is defined as  $\Delta f_{p_{x_i}}\coloneqq$ $f(x_{i}) - \mathbb{E}_{P_{x_{i}}}[f] = f(x_{i}) - \hat{f} (P_{x_{i}})$ . Then the total noise is  $y_{i} - \mathbb{E}_{P_{x_{i}}}[f] = \zeta_{i} + \Delta f_{p_{x_{i}}}$ . We can state our main result, which gives a cumulative regret bound under inexact kernel calculations,