SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\nabla f (\mathbf {x} _ {t}) \sim \mathcal {N} (\nabla \mu_ {\mathcal {D} _ {t}} (\mathbf {x} _ {t}), \nabla k _ {\mathcal {D} _ {t}} (\mathbf {x} _ {t}, \mathbf {x} _ {t}) \nabla^ {\top}).
$$
Thus, the posterior mean gradient  $\nabla \mu_{\mathcal{D}_t}(\mathbf{x}_t)$  is an approximation of the ground truth gradient  $\nabla f(\mathbf{x}_t)$ , where the approximation error is quantified by the posterior covariance  $\nabla k_{\mathcal{D}_t}(\mathbf{x}_t,\mathbf{x}_t)\nabla^\top$ . For any  $f$ , we emphasize that the posterior mean gradient  $\nabla \mu_{\mathcal{D}_t}(\mathbf{x}_t)$  is therefore biased whenever the posterior covariance is nonzero. Unfortunately, because of noise  $\sigma > 0$ , this is always true for any finite data  $\mathcal{D}_t$ . Thus, the standard analysis of stochastic gradient descent does not apply, as it typically assumes the stochastic gradient is unbiased.
The second challenge is that the noise directly makes numerical gradient estimation difficult. To build intuition, consider a finite differencing rule that approximates the partial derivative
$$
\frac {\partial f}{\partial x _ {i}} = \frac {1}{2 h} \left[ f (\mathbf {x} + h \mathbf {e} _ {i}) - f (\mathbf {x} - h \mathbf {e} _ {i}) \right] + \mathcal {O} \left(h ^ {2}\right)
$$
where  $\mathbf{e}_i$  is the  $i$ -th standard unit vector. In order to reduce the  $\mathcal{O}(h^2)$  gradient estimation error, we need  $h \to 0$ . However, the same estimator under the noisy setting
$$
\frac {\partial f}{\partial x _ {i}} \approx \frac {1}{2 h} \left[ \left(f (\mathbf {x} + h \mathbf {e} _ {i}) + \boldsymbol {\epsilon} _ {1}\right) - \left(f (\mathbf {x} - h \mathbf {e} _ {i}) + \boldsymbol {\epsilon} _ {2}\right) \right] + \mathcal {O} \left(h ^ {2}\right),
$$
where  $\epsilon_1, \epsilon_2$  are i.i.d. Gaussians, diverges to infinity when  $h \to 0$ . Note that the above estimator is biased even with repeated function evaluations because  $h$  cannot go to zero.
We first present a general convergence rate for Algorithm 1 that bounds the gradient norm.
Theorem 2. For  $0 < \delta < 1$ , suppose  $f$  is a GP sample whose smoothness constant is  $L$  w.p. at least  $1 - \delta$ . Algorithm 1 with batch size  $b_{t}$  and step size  $\eta_t = \frac{1}{L}$  produces a sequence satisfying
$$
\min  _ {1 \leq t \leq T} \| \nabla f (\mathbf {x} _ {t}) \| ^ {2} \leq \frac {1}{T} (2 L (f (\mathbf {x} _ {1}) - f ^ {*})) + \frac {1}{T} \sum_ {t = 1} ^ {T} C _ {t} E _ {d, k, \sigma} (b _ {t}) \tag {6}
$$
with probability at least  $1 - 2\delta$ , where  $C_t = 2\log \big((\pi^2 /6)(t^2 /\delta)\big)$ .
The second term  $\frac{1}{T}\sum_{t = 1}^{T}C_{t}E_{d,k,\sigma}(b_{t})$  in the right hand side of (6) is the average cumulative bias of the gradient. To finish the convergence analysis, we must further bound the error function  $E_{d,k,\sigma}(b_t)$ . For the RBF kernel, we obtain the following bound:
Lemma 4 (RBF Kernel). Let  $k(\mathbf{x}_1, \mathbf{x}_2) = \exp \left(-\frac{1}{2} \| \mathbf{x}_1 - \mathbf{x}_2 \|^2\right)$  be the RBF kernel. We have
$$
E _ {d, k, \sigma} (2 m d) \leq d \bigg (1 + W \bigg (- \frac {m}{e (m + \sigma^ {2})} \bigg) \bigg) = \mathcal {O} (\sigma d m ^ {- \frac {1}{2}}),
$$
where  $m \in \mathbb{N}$  and  $W$  denotes the principal branch of the Lambert  $W$  function.
The error function (3) is an infimum over all possible designs, which is intractable for analysis. Instead, we analyze the infimum over a subset of designs of particular patterns (based on finite differencing), which can be solved analytically, resulting the first inequality. The second equality is proved by further bounding the Lambert function by its Taylor expansion at  $-1 / e$ .
In addition, we obtain a similar bound for the Matérn kernel with  $\nu = \frac{5}{2}$  by a slightly different proof.
Lemma 5 (Matern Kernel). Let  $k(\cdot, \cdot)$  be the  $\nu = 2.5$  Matérn kernel. Then, we have
$$
E _ {d, k, \sigma} (2 m d) \lesssim \sigma d m ^ {- \frac {1}{2}} + \sigma^ {\frac {3}{2}} d m ^ {- \frac {3}{4}} = \mathcal {O} (\sigma d m ^ {- \frac {1}{2}}).
$$
Interestingly, Lemma 4 and Lemma 5 end up with the same asymptotic rate. Writing the bound in terms of the batch size  $b$ , we can see that  $E_{d,k,\sigma}(b) = \mathcal{O}(\sigma d^{\frac{3}{2}}b^{-\frac{1}{2}})$  for both the RBF kernel and the  $\nu = 2.5$  Matérn kernel. Coupled with Theorem 2, the above lemmas translate into the following convergence rates, depending on the batch size  $b_{t}$  in each iteration:
Corollary 2. Let  $k(\cdot, \cdot)$  be either the RBF kernel or the  $\nu = 2.5$  Matérn kernel. Under the same conditions as Theorem 2, if
$$
b _ {t} = \left\{ \begin{array}{l l} d \log^ {2} t; \\ d t; \\ d t ^ {2}, \end{array} \right. \quad \text {t h e n} \quad \min  _ {1 \leq t \leq T} \| \nabla f (\mathbf {x} _ {t}) \| ^ {2} = \left\{ \begin{array}{l l} \mathcal {O} (1 / T) + \mathcal {O} (\sigma d); \\ \mathcal {O} \big (\sigma d T ^ {- \frac {1}{2}} \log T \big) = \mathcal {O} \big (\sigma d ^ {\frac {5}{4}} n ^ {- \frac {1}{4}} \log n \big); \\ \mathcal {O} \big (\sigma d T ^ {- 1} \log^ {2} T \big) = \mathcal {O} \big (\sigma d ^ {\frac {4}{3}} n ^ {- \frac {1}{3}} \log^ {2} n \big), \end{array} \right.
$$
with probability at least  $1 - 2\delta$ . Here,  $T$  is the total number of iterations and  $n$  is the total number of samples queried.
<sup>1</sup>We suspect a similar bound holds for the entire Matérén family.
With nearly constant batch size  $b_{t} = d\log^{2}t$ , Algorithm 1 converges to a region where the squared gradient norm is  $\mathcal{O}(\sigma d)$ . With linearly increasing batch size, the algorithm converges to a stationary point with rate  $\mathcal{O}(\sigma d^{1.25}n^{-0.25}\log n)$ , significantly slower than the  $\mathcal{O}(d / n)$  rate in the noiseless setting. The quadratic batch size is nearly optimal up to a logarithm factor — increasing the batch size further slows down the rate (see Appendix D for details).
To achieve convergence to a stationary point using Corollary 2, the batch size  $b_{t}$  must increase as optimization progresses. We note this may not be an artifact of our theory or our specific realization of local BO, but rather a general fact that is likely to be implicitly true for any local BO routine. For example, a practical implementation of Algorithm 1 might use a constant batch size and a line search subroutine where the iterate  $\mathbf{x}_t$  is updated only when the (noisy) objective value decreases — otherwise, the iterate  $\mathbf{x}_t$  does not change and the algorithm queries more candidates to reduce the bias in the gradient estimate. With this implementation, the batch size is increased repeatedly on any iterate while the line search condition fails. As the algorithm converges towards a stationary point, the norm of the ground-truth gradient  $\| \nabla f(\mathbf{x}_t)\|$  decreases and thus requires more accurate gradient estimates. Therefore, the line search condition may fail more frequently with the constant batch size, and the effective batch size increases implicitly.
We provide two additional remarks on convergence in the noisy setting. First, the convergence rate is significantly slower than in the noiseless setting, highlighting the difficulty presented by noise. Second, when the noise is small, the convergence rate is faster. If  $\sigma \to 0$ , the rate is dominated by a lower-order term in the big  $\mathcal{O}$  notation, recovering the  $\mathcal{O}\left(\frac{1}{T}\right)$  rate in the noiseless setting (see Appendix D for details). This is in sharp contrast with the existing analysis of BO algorithms. Existing convergence proofs in the noisy setting rely on analyzing the information gain, which is vacuous when  $\sigma \to 0$ , requiring new tools. It is interesting that no new tools are required here.
Finally, we revisit Assumption 2. In the case when it does not hold, we use a modified update:
$$
\mathbf {x} _ {t + 1} = \operatorname {p r o j} _ {\mathcal {X}} \left(\mathbf {x} _ {t} - \eta_ {t} \nabla \mu_ {\mathcal {D} _ {t}} (\mathbf {x} _ {t})\right), \tag {7}
$$
where the projection operator  $\mathrm{proj}_{\mathcal{X}}(\cdot)$  is defined as  $\mathrm{proj}_{\mathcal{X}}(\mathbf{x}) = \operatorname*{argmin}_{\mathbf{z}\in \mathcal{X}}\| \mathbf{z} - \mathbf{x}\|$ , i.e., the closest feasible point to  $\mathbf{x}$ . When the iterates stay in the interior of the domain  $\mathcal{X}$ , the projection operator is an identity map and the update rule simply reduces to  $\mathbf{x}_{t + 1} = \mathbf{x}_t - \eta_t\nabla \mu_{\mathcal{D}_t}(\mathbf{x}_t)$ .
Define the gradient mapping
$$
G \left(\mathbf {x} _ {t}\right) = \frac {1}{\eta_ {t}} \left(\mathbf {x} _ {t} - \operatorname {p r o j} _ {\mathcal {X}} \left(\mathbf {x} _ {t} - \eta_ {t} \nabla f (\mathbf {x} _ {t})\right)\right),
$$
which is a generalization of the gradient in the constrained setting: when  $\mathbf{x}_t - \eta_t\nabla f(\mathbf{x}_t)$  lies in the interior of  $\mathcal{X}$ , the gradient mapping reduces to the (usual) gradient  $\nabla f(\mathbf{x}_t)$ . The following gives convergence rates of  $\| G(\mathbf{x}_t)\|$ .
Theorem 3. Under the same conditions as Corollary 2, without Assumption 2, using the projected update rule (7), Algorithm 1 obtains the following rates:
$$
i f \quad b _ {t} = \left\{ \begin{array}{l l} d t; \\ d t ^ {2}, \end{array} \right. \quad t h e n \quad \min  _ {1 \leq t \leq T} \| G (\mathbf {x} _ {t}) \| ^ {2} = \left\{ \begin{array}{l} \mathcal {O} \big (\sigma d ^ {\frac {5}{4}} n ^ {- \frac {1}{4}} \log n + \sigma^ {\frac {1}{2}} d ^ {\frac {5}{8}} n ^ {- \frac {1}{8}} \log n \big); \\ \mathcal {O} \big (\sigma d ^ {\frac {4}{3}} n ^ {- \frac {1}{3}} \log^ {2} n + \sigma^ {\frac {1}{2}} d ^ {\frac {2}{3}} n ^ {- \frac {1}{6}} \log n \big), \end{array} \right.
$$
with probability at least  $1 - 2\delta$ . Here,  $n$  is the total number of samples queried.
These rates are slower than Corollary 2. We defer more details to Appendix C.
6 Additional Experiments
In this section, we investigate numerically the bounds in our proofs and study situations where the assumptions are violated. We focus on analytical experiments, because the excellent empirical performance of local BO methods on high dimensional real world problems has been well established in prior work [e.g., 10, 19, 20, 27]. Detailed settings and additional experiments are available in Appendix E. The code is available at https://github.com/kayween/local-bo-convergence.
6.1 How loose are our convergence rates?
This section investigates the tightness of the bounds on the error function  $E_{d,k,\sigma}(b)$  — a key quantity in our convergence rates. We plot in Figure 2 the error function  $E_{d,k,\sigma}$  for the  $\nu = 2.5$  Matérn
(a) fixed dimension, varying batch sizes
(b) fixed batch size, varying dimensions
Figure 2: Compare the error function (3) of the  $\nu = 2.5$  Matérn kernel and our upper bound in Lemma 5. The error function  $E_{d,k,\sigma}(b)$  is approximated by minimizing (3) with L-BFGS. Both plots are in log-log scale. Left: The slope indicates the exponent on  $b$ . Since the slope magnitude of the error function is slightly larger, the error function might decreases slightly faster than  $\mathcal{O}(b^{-\frac{1}{2}})$  asymptotically. Right: The slope indicates the exponent on  $d$ . Since all lines have roughly the same slope, the dependency on the dimension in Lemma 5 seems to be tight.
Figure 3: The performance of random restart on a GP sample path in 100 dimensions. Left: a density plot for the minimum value found on a single restart (compare with Figure 1). Right: the median and a  $90\%$  confidence interval for the best value found after a given number of random restarts.
kernel and our bound  $\mathcal{O}(\sigma d^{\frac{3}{2}}b^{-\frac{1}{2}})$  implied by Lemma 5. The error function is empirically estimated by minimizing the trace of posterior gradient covariance (3) using L-BFGS. Figure 2a uses a fixed dimension  $d = 10$  and varying batch sizes, while Figure 2b uses a fixed batch size  $b = 500$  and varying dimensions. Both plots are in log-log scale. Therefore, the slope corresponds to the exponents on  $b$  and  $d$  in the bound  $\mathcal{O}(\sigma d^{\frac{3}{2}}b^{-\frac{1}{2}})$ , and the vertical intercept corresponds to the constant factor in the big  $\mathcal{O}$  notation.
Figure 2a shows that the actual decay rate of the error function  $E_{d,k,\sigma}(b)$  may be slightly faster than  $\mathcal{O}(b^{-\frac{1}{2}})$ , as the slopes of the (empirical) error function have magnitude slightly larger than  $\frac{1}{2}$ . Interestingly, Figure 2b demonstrates that the dependency  $d^{\frac{3}{2}}$  on the dimension is quite tight — all lines in this plot share a similar slope magnitude.
6.2 What is the effect of multiple restarts?
In §3, we analyze local solutions found by a single run on different GP sample paths. Here, we investigate the impact of performing multiple restarts on the same sample path. In Figure 3, we plot (left) a kernel density estimate of the local solution found for a series of 10000 random restarts, and (right) the best value found after several restarts with a  $90\%$  confidence interval. We make two observations. First, the improvement of running 10-20 restarts over a single restart is still significant: the Gaussian tails involved here render a difference of  $\pm 1$  in objective value relatively large. Second, the improvement of multiple restarts saturates relatively quickly. This matches empirical observations
(a)  $\nabla \mu_{\mathcal{D}}(\mathbf{x}) = (0.039,1.018)$  The ground truth is  $(0,1)$
Figure 4: Estimating the "derivative" at  $\mathbf{x} = (0,1)$  with a Matérn Gaussian process ( $\nu = 2.5$ ) in three different settings. Left:  $f(\mathbf{x}) = \frac{1}{2}\|\mathbf{x}\|^2$ . With  $n = 5$  samples, the posterior mean gradient is close to the ground truth. Middle:  $f(\mathbf{x}) = \|\mathbf{x}\|_1$ . The  $\ell_1$  norm is not differentiable at  $(0,1)$ . With exactly the same samples as the left panel, the posterior mean gradient has higher error. Right:  $f(\mathbf{x}) = \|\mathbf{x}\|_1$ . Increasing the sample size to  $n = 10$  decreases the estimation error.
(b)  $\nabla \mu_{\mathcal{D}}(\mathbf{x}) = (0.005, 0.318)$ . Subgradient is  $[-1, 1] \times \{1\}$ .
(c)  $\nabla \mu_{\mathcal{D}}(\mathbf{x}) = (-0.031, 1.053)$ . Subgradient is  $[-1, 1] \times \{1\}$ .
made when using local BO methods on real world problems, where optimization is often terminated after (at most) a handful of restarts.
6.3 What if the objective function is non-differentiable?
In this section, we investigate what happens when  $f$  is not differentiable — a setting ruled out in our theory by Assumption 1. In this case, what does the posterior mean gradient  $\nabla \mu_{\mathcal{D}}$  learn from the oracle queries? To answer this question, we consider the  $\ell_1$  norm function  $\mathbf{x} \mapsto \| \mathbf{x} \|_1$  in two dimensions, which is non-differentiable when either  $x_1$  or  $x_2$  is zero. The  $\ell_1$  norm is convex and thus its subdifferential is well-defined:
$$
\partial \| \mathbf {x} \| _ {1} = \left\{\mathbf {g} \in \mathbb {R} ^ {2}: \| \mathbf {g} \| _ {\infty} \leq 1, \mathbf {g} ^ {\top} \mathbf {x} = \| \mathbf {x} \| _ {1} \right\}.
$$
In Figure 4, we use the posterior mean gradient  $\nabla \mu_{\mathcal{D}}$  to estimate the (sub)gradient at  $(0,1)$ . As a comparison, we also show the result for the differentiable quadratic function  $\mathbf{x} \mapsto \frac{1}{2}\|\mathbf{x}\|^2$ . Comparing Figure 4a and 4b, we observe that the  $\ell_1$  norm has higher estimation error than the Euclidean norm despite using exactly the same set of queries. This might suggest that non-differentiability increases the sample complexity of learning the "slope" of the function. Increasing the sample size to  $n = 10$  eventually results in a  $\nabla \mu_{\mathcal{D}}(\mathbf{x})$  much closer to the subgradient, as shown in Figure 4c.
7 Discussion and Open Questions
For prototypical functions such as GP sample paths with RBF kernel, the local solutions discovered by local Bayesian optimization routines in high dimension are of surprisingly high quality. This motivates the theoretical study of these routines, which to date has been somewhat neglected. Here we have established the convergence of a recently proposed local BO algorithm, GIBO, in both the noiseless and noisy settings. The convergence rates are polynomial in both the number of samples and the input dimension, supporting the obvious intuitive observation that finding local optima is easier than finding the global optimum.
Our developments in this work leave a number of open questions. It is not clear whether solutions from local optimization perform so well because the landscape is inherently easy (e.g., most stationary points are good approximation of the global minimum) or because the local BO has an unknown algorithmic bias that helps it avoid bad stationary points. This question calls for analysis of the landscape of GP sample paths (or the RKHS). Additionally, we conjecture that our convergence rates are not yet tight; §6.1 suggests that there is likely room for improvement. Further, it would be interesting to establish convergence for trust-region based local BO [e.g. 10, 9].
In practice, while existing lower bounds imply that any algorithm seeking to use local BO as a subroutine to discover the global optimum will ultimately face the same exponential-in-  $d$  sample complexities as other methods, our results strongly suggest that, indeed as explored empirically in the literature, local solutions can not only be found, but can be surprisingly competitive.
KW would like to thank Thomas T.C.K. Zhang and Xinran Zhu for discussions on RKHS. RG is supported by NSF awards OAC-2118201, OAC-1940224 and IIS-1845434. JRG is supported by NSF award IIS-2145644.
[18] Natalie Maus, Kaiwen Wu, David Eriksson, and Jacob Gardner. Discovering many diverse solutions with Bayesian optimization. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206, pages 1779-1798, 2023.
[19] Sarah Müller, Alexander von Rohr, and Sebastian Trimpe. Local policy search with Bayesian optimization. In Advances in Neural Information Processing Systems, volume 34, pages 20708-20720, 2021.
[20] Quan Nguyen, Kaiwen Wu, Jacob Gardner, and Roman Garnett. Local Bayesian optimization via maximizing probability of descent. In Advances in Neural Information Processing Systems, volume 35, pages 13190-13202, 2022.
[21] Jonathan Scarlett. Tight regret bounds for Bayesian optimization in one dimension. In Proceedings of the 35th International Conference on Machine Learning, volume 80, pages 4500-4508, 2018.
[22] Yao Shu, Zhongxiang Dai, Weicong Sng, Arun Verma, Patrick Jaillet, and Bryan Kian Hsiang Low. Zeroth-order optimization with trajectory-informed derivative estimation. In The Eleventh International Conference on Learning Representations, 2023.
[23] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, volume 25, 2012.
[24] Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on Machine Learning, pages 1015-1022, 2010.
[25] Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating Bayesian optimization for biological sequence design with denoising autoencoders. In Proceedings of the 39th International Conference on Machine Learning, volume 162, pages 20459-20478, 2022.
[26] Austin Tripp, Erik Daxberger, and José Miguel Hernández-Lobato. Sample-efficient optimization in the latent space of deep generative models via weighted retraining. In Advances in Neural Information Processing Systems, volume 33, pages 11259-11272, 2020.
[27] Linnan Wang, Rodrigo Fonseca, and Yuandong Tian. Learning search space partition for black-box optimization using Monte Carlo tree search. In Advances in Neural Information Processing Systems, volume 33, pages 19511-19522, 2020.
[28] Holger Wendland. Scattered Data Approximation, volume 17. Cambridge University Press, 2004.
[29] James T Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Marc Peter Deisenroth. Pathwise conditioning of Gaussian processes. The Journal of Machine Learning Research, 22(1):4741-4787, 2021.
Lemma 6. Let  $p_n(x) = \sum_{i=0}^{n} \frac{1}{i!} x^i$  be the degree- $n$  Taylor polynomial of  $e^x$  around  $x = 0$ . Then for any  $k \geq 1$  and any  $x \in \mathbb{R}$ , we have
$$
e ^ {x} \geq p _ {2 k - 1} (x).
$$
Proof. The proof is an induction on  $k$ . The base case  $k = 1$  is trivial:  $e^x \geq 1 + x$ .
Consider the general case  $p_{2k+1}$ . Define  $g(x) = e^x - p_{2k+1}(x)$ . It is easy to see that  $g'(x) = e^x - p_{2k}(x)$  and  $g''(x) = e^x - p_{2k-1}(x)$ . By the induction hypothesis,  $g'' \geq 0$  and therefore  $g$  is convex. Thus, the minimum of  $g$  is given by its stationary points. It is easy to observe that  $x = 0$  is indeed a stationary point. Thus,  $\min_{x \in \mathbb{R}} g(x) = g(0) = 0$ , which finishes the proof.
Lemma 7. Let  $\sigma >0$  and  $W$  be the principal branch of the Lambert  $W$  function. For any  $m\in \mathbb{N}$  we have
$$
1 + W (- \frac {m}{e (m + \sigma^ {2})}) \leq \sqrt {\frac {2 \sigma^ {2}}{m + \sigma^ {2}}}.
$$
Proof. The problem is reduced to proving
$$
1 + W (x) \leq \sqrt {2 (1 + e x)}
$$
for all  $-\frac{1}{e} \leq x \leq 0$ . In fact, the right hand side is exactly the first-order Taylor expansion of the left hand side at  $x = -\frac{1}{e}$  [e.g., 4].
Square both sides. It is equivalent to prove  $(1 + W(x))^2 \leq 2(1 + ex)$  for all  $x \in [-\frac{1}{e}, 0]$ . Define
$$
g (x) = (1 + W (x)) ^ {2} - 2 (1 + e x).
$$
The derivative of  $g$  in  $(-\frac{1}{e}, 0)$  is
$$
g ^ {\prime} (x) = 2 \left(\frac {W (x)}{x} - e\right).
$$
By the definition of the Lambert  $W$  function, we have
$$
e x = W (x) e ^ {1 + W (x)} <   W (x)
$$
since  $-1 < W(x) < 0$  for  $x \in (-\frac{1}{e}, 0)$ . Thus,  $g'(x) < 0$  for  $x \in (-\frac{1}{e}, 0)$  and  $g$  is a decreasing function in  $[- \frac{1}{e}, 0]$ . Observe that  $g(-\frac{1}{e}) = 0$ . Therefore  $g(x) \leq 0$  for all  $x \in [-\frac{1}{e}, 0]$ , which completes the proof.
Next, we present a lemma which states that the error function bounds the posterior covariance trace in each iteration of Algorithm 1.
Lemma 8. In the  $t$ -th iteration of Algorithm 1, we have
$$
\operatorname {t r} \left(\nabla k _ {\mathcal {D} _ {t}} (\mathbf {x} _ {t}, \mathbf {x} _ {t}) \nabla^ {\top}\right) \leq E _ {d, k, \sigma} (b _ {t})
$$
Proof. Without loss of generality, we assume  $\mathbf{x}_t = \mathbf{0}$ . Otherwise, shift the data  $\mathcal{D}_t$  and  $\mathbf{x}_t$  by  $-\mathbf{x}_t$ , which does not change the value of the left hand side because of stationarity of the kernel  $k$ . Let  $\mathbf{Z} \in \mathbb{R}^{b_t \times d}$  be arbitrary candidates. Then, we have
$$
\operatorname {t r} \big (\nabla k _ {\mathcal {D} _ {t - 1} \cup \mathbf {Z}} (\mathbf {0}, \mathbf {0}) \nabla^ {\top} \big) \leq \operatorname {t r} \big (\nabla k (\mathbf {0}, \mathbf {0}) \nabla^ {\top} - \nabla k (\mathbf {0}, \mathbf {Z}) (k (\mathbf {Z}, \mathbf {Z}) + \sigma^ {2} \mathbf {I}) ^ {- 1} k (\mathbf {Z}, \mathbf {0}) \nabla^ {\top} \big).
$$
Because the LHS conditions on both  $\mathcal{D}_{t-1}$  and  $\mathbf{Z}$  but the RHS only conditions on  $\mathbf{Z}$ . Now, we minimize  $\mathbf{Z}$  on both sides.
On the one hand, the LHS becomes  $\mathrm{tr}\bigl (\nabla k_{\mathcal{D}_t}(\mathbf{0},\mathbf{0})\nabla^\top \bigr)$ . This is because  $\mathcal{D}_t$  is the union of  $\mathcal{D}_{t - 1}$  and the minimizer of the acquisition function  $\alpha_{\mathrm{trace}}(\mathbf{0},\mathbf{Z}) = \mathrm{tr}\bigl (\nabla k_{\mathcal{D}_{t - 1}\cup \mathbf{Z}}(\mathbf{0},\mathbf{0})\nabla^\top \bigr)$ .
On the other hand, the RHS becomes  $E_{d,k,\sigma}(b_t)$  by definition of the error function, which completes the proof.