SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Of note, the value of  $\epsilon_{s}$  is used for both (a) calculating the number  $N_{\epsilon_s}$  of queries to the quantum oracle for  $x_{s}$ , and (b) computing the weight  $1 / \epsilon_s^2$  assigned to  $(x_{s},y_{s})$  in the weighted GP regression (3) in the subsequent iterations. Regarding (a), our designs of  $\epsilon_{s}$  and  $N_{\epsilon_s} = \widetilde{\mathcal{O}} (1 / \epsilon_s)$  have an interesting interpretation in terms of the exploration-exploitation trade-off: In the initial stages, the weighted GP posterior standard deviation  $\widetilde{\sigma}_{s - 1}(x_s)$  and hence  $\epsilon_{s}$  are usually large, which leads to a small number  $N_{\epsilon_s}$  of queries for every  $x_{s}$  and hence allows our Q-GP-UCB to favor the exploration of more unique inputs; in later stages,  $\widetilde{\sigma}_{s - 1}(x_s)$  and  $\epsilon_{s}$  usually become smaller, which results in large  $N_{\epsilon_s}$ 's and hence causes our Q-GP-UCB to prefer the exploitation of a small number of unique inputs. Regarding (b), assigning a larger weight  $1 / \epsilon_s^2$  to an input  $x_{s}$  with a smaller  $\epsilon_{s}$  is reasonable, because a smaller  $\epsilon_{s}$  indicates a smaller estimation error for  $y_{s}$  as we explained above, which makes the observation  $y_{s}$  more accurate and reliable for calculating the weighted GP regression (3).
Note that we have modified the original GP-UCB by querying every selected input  $x_{s}$  multiple times, in order to make it amenable to the integration of the QMC subroutine. The recent work of [6] has also
adapted BO to evaluate a small number of unique inputs while querying each of them multiple times. It has been shown [6] that the resulting BO algorithm preserves both the theoretical guarantee (i.e., the regret upper bound) and empirical performance of the original BO, while significantly reducing the computational cost. So, the findings from [6] can serve as justifications for our modification to GP-UCB. Also note that despite this similarity, [6] still focuses on the traditional setting (i.e., their regret upper bound is  $R_{T} = \mathcal{O}(\gamma_{T}\sqrt{T})$ ) and their regret analyses are entirely different from ours.
Throughout our analysis, we condition on the event that  $|f(x_s) - y_s| \leq \epsilon_s, \forall s = 1, \dots, m$ . This event holds with probability of at least  $1 - \delta / 2$ , because we set the error probability in QMC (Lemma 1) to  $\delta / (2\overline{m})$  in which  $\overline{m} \geq m$  (see Theorem 2 for details). For simplicity, we mainly focus on the scenario of bounded noise (i.e., the observations are bounded within [0, 1], the first case of Lemma 1), because the theoretical analyses and insights about the other scenario of noise with bounded variance (i.e., the second case of Lemma 1) are almost identical (more details in Sec. 5.5).
5.1 Weighted Information Gain
To begin with, the following lemma (proof in App. B) gives an upper bound on the sum of the weights in all  $m$  stages, which will be useful for upper-bounding the weighted information gain  $\widetilde{\gamma}_m$ .
Lemma 2. Choose  $\delta \in (0,2 / e]$ , then we have that  $\sum_{s = 1}^{m}1 / \epsilon_s^2\leq T^2$
Denote by  $\gamma_{T}$  the maximum information gain from any set of  $T$  inputs:  $\gamma_{T} \triangleq \max_{\{x_{1}, \ldots, x_{T}\} \subset \mathcal{X}} \frac{1}{2} \log \det(I + \frac{1}{\lambda} K_{T})$ , which is commonly used in the analysis of BO/kernelized bandit algorithms [9, 33]. Denote by  $\widetilde{\gamma}_{m}$  the weighted information gain from all  $m$  selected inputs in all stages:  $\widetilde{\gamma}_{m} \triangleq \frac{1}{2} \log \det(I + \frac{1}{\lambda} \widetilde{K}_{m})$ . Note that in the definition of  $\widetilde{\gamma}_{m}$ , we do not take the maximum over all sets of  $m$  inputs, so  $\widetilde{\gamma}_{m}$  still depends on the selected inputs  $\mathcal{X}_{m} \triangleq \{x_{1}, \ldots, x_{m}\}$ . Also note that  $\{\epsilon_{1}, \ldots, \epsilon_{m}\}$  are known constants conditioned on  $\mathcal{X}_{m}$ . This is because the weighted GP posterior standard deviation  $\widetilde{\sigma}_{s-1}(3)$  does not depend on the output observations, so every  $\epsilon_{s} = \widetilde{\sigma}_{s-1}(x_{s}) / \sqrt{\lambda}$  only depends on  $\{x_{1}, \ldots, x_{s}\}$ . The next theorem gives an upper bound on  $\widetilde{\gamma}_{m}$ .
Theorem 1. (a) Given  $\mathcal{X}_m \triangleq \{x_1, \ldots, x_m\}$ , the growth rate of  $\widetilde{\gamma}_m$  is the same as that of  $\gamma_{\sum_{s=1}^{m} 1/\epsilon_s^2}$ . (b)  $\widetilde{\gamma}_m \leq \gamma_{T^2}$ .
We give the complete statement of result (a) in Theorem 8 (App. C), which presents the concrete growth rate. As stated by result (a), to obtain an upper bound on  $\widetilde{\gamma}_m$ , we simply need to firstly use the upper bound on  $\gamma_T$  from previous works [35], and then replace the  $T$  in this upper bound by  $\sum_{s=1}^{m} 1 / \epsilon_s^2$ . Based on this, the result (b) has further upper-bounded  $\sum_{s=1}^{m} 1 / \epsilon_s^2$  by  $T^2$  through Lemma 2. Note that because  $\epsilon_s = \widetilde{\sigma}_{s-1}(x_s) / \sqrt{\lambda} \leq 1$ , we have that  $1 / \epsilon_s^2 \geq 1$  and hence  $\sum_{s=1}^{m} 1 / \epsilon_s^2 \geq m$ . Therefore, our weighted information gain  $\widetilde{\gamma}_m$  has a looser upper bound than  $\gamma_m$ . This intuitively implies that our weighted covariance matrix  $\widetilde{K}_m$  is expected to contain more information than the original  $K_m$ , whose implication will be discussed again in Sec. 5.2. The proof of Theorem 1 (App. C) has followed the analysis of the information gain from [35]. Specifically, we have leveraged a finite-dimensional projection of the infinite-dimensional RKHS feature space, and hence upper-bounded the information gain in terms of the tail mass of the eigenvalues of the kernel  $k$ . The proof requires carefully tracking the impact of the weights  $1 / \epsilon_s^2$  throughout the analysis. Importantly, our Theorem 1 and its analysis may be of wider independent interest for future works which also adopt weighted GP regression (Sec. 4.1).
The growth rate of  $\gamma_{T}$  has been characterized for commonly used kernels [35]. Based on these, the next corollary provides the growth rates of  $\widetilde{\gamma}_{m}$  for the linear and squared exponential (SE) kernels.
Corollary 1. For the linear and squared exponential (SE) kernels, we have that
$$
\widetilde {\gamma} _ {m} = \mathcal {O} (d \log \sum_ {s = 1} ^ {m} 1 / \epsilon_ {s} ^ {2}) = \mathcal {O} (d \log T) \quad \text {l i n e a r k e r n e l},
$$
$$
\widetilde {\gamma} _ {m} = \mathcal {O} \left(\log^ {d + 1} \sum_ {s = 1} ^ {m} 1 / \epsilon_ {s} ^ {2}\right) = \mathcal {O} \left(\left(\log T\right) ^ {d + 1}\right) \quad S E k e r n e l.
$$
5.2 Upper Bound on The Total Number  $m$  of Stages
The next theorem gives an upper bound on the total number of stages of our Q-GP-UCB algorithm.
Theorem 2. For the linear kernel, our  $Q$ -GP-UCB algorithm has at most  $m = \mathcal{O}(d\log T)$  stages. For the SE kernel, our  $Q$ -GP-UCB algorithm has at most  $m = \mathcal{O}((\log T)^{d + 1})$  stages.
Note that for the linear kernel, our upper bound on  $m$  matches that of the Q-LinUCB algorithm from [38] (see Lemma 2 from [38]). The proof of Theorem 2 is given in App. D, and here we give a brief sketch of the proof. For stage  $s$ , define  $V_{s} \triangleq \lambda I + \sum_{\tau=1}^{s} \left( \frac{1}{\epsilon_{\tau}^{2}} \right) \phi(x_{\tau}) \phi(x_{\tau})^{\top}$ , in which  $\phi(\cdot)$  is the (potentially infinite-dimensional) RKHS feature mapping:  $k(x, x') = \phi(x)^{\top} \phi(x')$ . Intuitively,  $\log \det V_{s}$  represents the amount of information contained in the first  $s$  selected inputs  $\{x_{1}, \ldots, x_{s}\}$ . As the first step in our proof, we prove that thanks to our choice of  $\epsilon_{s}$  (line 3 of Algo. 1) and our design of the weights  $1 / \epsilon_{s}^{2}$ , the amount of information  $\log \det V_{s}$  is doubled after every stage:  $\det V_{\tau+1} = 2 \det V_{\tau}, \forall \tau = 0, \ldots, m-1$ . This allows us to show that  $m \log 2 = \log (\det(V_{m}) / \det(V_{0}))$  where  $V_{0} = \lambda I$ . Next, we show that this term is also related to our weighted information gain:  $\log (\det(V_{m}) / \det(V_{0})) = 2 \widetilde{\gamma}_{m}$ , which allows us to upper-bound  $m$  in terms of  $\widetilde{\gamma}_{m}$  and hence in terms of  $\sum_{s=1}^{m} 1 / \epsilon_{s}^{2}$  (see Corollary 1). This eventually allows us to derive an upper bound on  $m$  by using the fact that the total number of iterations (i.e., queries to the quantum oracles) is no larger than  $T$ . Therefore, the key intuition of the proof here is that as long as we gather a sufficient amount of information in every stage, we do not need a large total number of stages.
5.3 Confidence Ellipsoid
The next theorem proves the concentration of  $f$  around the weighted GP posterior mean (3).
Theorem 3. Let  $\eta \triangleq 2 / T$ ,  $\lambda \triangleq 1 + \eta$ , and  $\beta_{s} \triangleq B + \sqrt{2(\widetilde{\gamma}_{s - 1} + 1 + \log(2 / \delta))}$ . We have with probability of at least  $1 - \delta / 2$  that
$$
\left| f (x) - \widetilde {\mu} _ {s - 1} (x) \right| \leq \beta_ {s} \widetilde {\sigma} _ {s - 1} (x), \quad \forall s \in [ m ], x \in \mathcal {X}.
$$
The proof of Theorem 3 is presented in App. E, and here we give a brief proof sketch. Denote by  $\mathcal{F}_{s-1}$  the  $\sigma$ -algebra  $\mathcal{F}_{s-1} = \{x_1, \ldots, x_s, \zeta_1, \ldots, \zeta_{s-1}\}$  where  $\zeta_k = y_k - f(x_k)$  is the noise. Recall that  $W_s^{1/2} \triangleq \mathrm{diag}(1/\epsilon_1, \ldots, 1/\epsilon_s)$  (Sec. 4.1), and define  $\zeta_{1:s} \triangleq [\zeta_k]_{k \in [s]}$ . In the first part of the proof, we use the RKHS feature space view of the weighted GP posterior (3) (see App. A) to upper-bound  $|f(x) - \widetilde{\mu}_{s-1}(x)|$  in terms of  $\widetilde{\sigma}_{s-1}(x)$  and  $||W_s^{1/2}\zeta_{1:s}||_{((\widetilde{K}_s + \eta I)^{-1} + I)^{-1}}$ . Next, the most crucial step in the proof is to upper-bound  $||W_s^{1/2}\zeta_{1:s}||_{((\widetilde{K}_s + \eta I)^{-1} + I)^{-1}}$  in terms of  $\widetilde{\gamma}_s$  by applying the self-normalized concentration inequality from Theorem 1 of [9] to  $I$ -sub-Gaussian noise. This is feasible because (a) every  $\epsilon_s = \widetilde{\sigma}_{s-1}(x_s) / \sqrt{\lambda}$  is  $\mathcal{F}_{s-1}$ -measurable, and (b) conditioned on  $\mathcal{F}_{s-1}$ , the scaled noise term  $\zeta_s / \epsilon_s$  (in  $W_s^{1/2}\zeta_{1:s}$ ) is  $1$ -sub-Gaussian. The statement (a) follows because  $\epsilon_s$  only depends on  $\{x_1, \ldots, x_s\}$  as we have discussed in Sec. 5.1. The statement (b) follows since our QMC subroutine ensures that every noise term  $\zeta_k$  is bounded within  $[- \epsilon_k, \epsilon_k]$  (Sec. 4.2) because  $|y_k - f(x_k)| \leq \epsilon_k$  (with high probability). This suggests that conditioned on  $\mathcal{F}_{s-1}$  (which makes  $\epsilon_s$  a known constant as discussed above),  $\zeta_s / \epsilon_s$  is bounded within  $[-1, 1]$  and is hence  $1$ -sub-Gaussian.
This critical step in the proof is in fact also consistent with an interesting interpretation about our algorithm. That is, after  $x_{s}$  is selected in stage  $s$ , we adaptively decide the number  $N_{\epsilon_s}$  of queries to the quantum oracle in order to reach the target accuracy  $\epsilon_{s}$  (line 4 of Algo. 1). This ensures that  $|y_{s} - f(x_{s})|\leq \epsilon_{s}$  (with high probability), which guarantees that the noise  $\zeta_s = y_s - f(x_s)$  is (conditionally)  $\epsilon_{s}$ -sub-Gaussian. Moreover, note that the vector of observations  $\widetilde{Y}_s$  used in the calculation of the weighted GP posterior mean (3) is  $\widetilde{Y}_{s} = W_{s}^{1 / 2}Y_{s}$ . So, from the perspective of the weighted GP posterior (3), every noisy observation  $y_{s}$ , and hence every noise  $\zeta_s$ , is multiplied by  $1 / \epsilon_{s}$ . So, the effective noise of the observations  $\widetilde{Y}_s$  used in the weighted GP posterior (i.e.,  $\zeta_s / \epsilon_s$ ) is  $\epsilon_{s} / \epsilon_{s} = 1$ -sub-Gaussian. This shows the underlying intuition as to why our proof of the concentration of the reward function  $f$  using the weighted GP regression can make use of 1-sub-Gaussian noise (e.g., in contrast to the  $\sigma$ -sub-Gaussian noise used in the proof of classical GP-UCB [9]).
Our tight confidence ellipsoid around  $f$  from Theorem 3 is crucial for deriving a tight regret upper bound for our Q-GP-UCB (Sec. 5.4). Moreover, as we will discuss in more detail in Sec. 5.6, our proof technique for Theorem 3, interestingly, can be adopted to obtain a tighter confidence ellipsoid for the Q-LinUCB algorithm from [38] and hence improve its regret upper bound.
5.4 Regret Upper Bound
Here we derive an upper bound on the cumulative regret of our Q-GP-UCB algorithm.
Theorem 4. With probability of at least  $1 - \delta$ , we have that<sup>1</sup>
$$
R _ {T} = \mathcal {O} \big (d ^ {3 / 2} (\log T) ^ {3 / 2} \log (d \log T) \big) \quad \text {l i n e a r k e r n e l},
$$
$$
R _ {T} = \mathcal {O} \big ((\log T) ^ {3 (d + 1) / 2} \log ((\log T) ^ {d + 1}) \big) \qquad S E k e r n e l.
$$
The proof of Theorem 4 is presented in App. F. The proof starts by following the standard technique in the proof of UCB-type algorithms to show that  $r_s = f(x^*) - f(x_s) \leq 2\beta_s \widetilde{\sigma}_{s-1}(x_s)$ . Next, importantly, our design of  $\epsilon_s = \widetilde{\sigma}_{s-1}(x_s) / \sqrt{\lambda}$  allows us to show that  $r_s \leq 2\beta_s \epsilon_s \sqrt{\lambda}$ . After that, the total regrets incurred in a stage  $s$  can be upper-bounded by  $N_{\epsilon_s} \times 2\beta_s \epsilon_s \sqrt{\lambda} = \mathcal{O}\left(\frac{1}{\epsilon_s} \log(\overline{m}/\delta) \times \beta_s \epsilon_s\right) = \mathcal{O}(\beta_s \log(\overline{m}/\delta))$ . Lastly, the regret upper bound follows by summing the regrets from all  $m$  stages. Of note, for the commonly used SE kernel, our regret upper bound is of the order  $\mathcal{O}(\text{poly} \log T)$ , which is significantly smaller than the regret lower bound of  $\Omega(\sqrt{T})$  in the classical setting shown by [31]. This improvement over the classical fundamental limit is mostly attributed to the use of the QMC (Lemma 1), i.e., line 6 of Algo. 1. If the QMC procedure is replaced by classical MC estimation, then in contrast to the  $N_{\epsilon_s} = \mathcal{O}(1/\epsilon_s)$  queries by QMC, every stage would require  $N_{\epsilon_s} = \mathcal{O}(1/\epsilon_s^2)$  queries. As a result, this would introduce an additional factor of  $1/\epsilon_s$  to the total regrets in a stage  $s$  (as discussed above), which would render the derivation of our regret upper bound (Theorem 4) invalid. This verifies that our tight regret upper bound in Theorem 4 is only achievable with the aid of quantum computing. Our Theorem 4 is a demonstration of the immense potential of quantum computing to dramatically improve the theoretical guarantees over the classical setting.
Importantly, when the linear kernel is used, the regret upper bound of our Q-GP-UCB is in fact better than that of the Q-LinUCB algorithm from [38]. This improvement arises from our tight confidence ellipsoid in Theorem 3. In Sec. 5.6, we will give a more detailed discussion of this improvement and adopt our proof technique for Theorem 3 to improve the confidence ellipsoid for Q-LinUCB, after which their improved regret upper bound matches that of our Q-GP-UCB with the linear kernel.
5.5 Regret Upper Bound for Noise with Bounded Variance
Here we analyze the regret of our Q-GP-UCB when the variance of the noise is bounded by  $\sigma^2$ , which encompasses common scenarios including  $\sigma$ -sub-Gaussian noises and Gaussian noises with a variance of  $\sigma^2$ . Consistent with the analysis of Q-LinUCB [38], in order to apply the theoretical guarantee provided by QMC (Lemma 1) for noise with bounded variance, here we need to assume that the noise variance is not too small, i.e., we assume that  $\sigma > 1/4$ . In this case of noise with bounded variance, the following theorem gives an upper bound on the regret of our Q-GP-UCB.
Theorem 5. With probability of at least  $1 - \delta$ , we have that
$$
\begin{array}{l} R _ {T} = \mathcal {O} \left(\sigma d ^ {3 / 2} (\log T) ^ {3 / 2} \log (d \log T) \left(\log_ {2} \sigma T\right) ^ {3 / 2} \log_ {2} \left(\log_ {2} \sigma T\right)\right) \quad \text {l i n e a r k e r n e l}, \tag {4} \\ R _ {T} = \mathcal {O} \big (\sigma (\log T) ^ {3 (d + 1) / 2} \log ((\log T) ^ {d + 1}) (\log_ {2} \sigma T) ^ {3 / 2} \log_ {2} (\log_ {2} \sigma T) \big) \quad S E k e r n e l. \\ \end{array}
$$
The proof of Theorem 5 is given in App. G. Note that same as Theorem 4, the regret for the SE kernel in Theorem 5 is also of the order  $\mathcal{O}(\mathrm{poly}\log T)$ , which is also significantly smaller than the classical regret lower bound of  $\Omega (\sqrt{T})$  [31]. Moreover, Theorem 5 shows that for both the linear kernel and SE kernel, the regret of our Q-GP-UCB is reduced when the noise variance  $\sigma^2$  becomes smaller.
5.6 Improvement to Quantum Linear UCB (Q-LinUCB)
As we have mentioned in Sec. 5.4, the regret upper bound of our Q-GP-UCB with the linear kernel is  $R_{T} = \mathcal{O}(d^{3/2}(\log T)^{3/2}\log (d\log T))$ , which is better than the corresponding  $R_{T} = \mathcal{O}(d^{2}(\log T)^{3/2}\log (d\log T))$  for Q-LinUCB [38] by a factor of  $\mathcal{O}(\sqrt{d})$ . This improvement can be attributed to our tight confidence ellipsoid from Theorem 3. Here, following the general idea of the proof of our Theorem 3, we prove a tighter confidence ellipsoid for the Q-LinUCB algorithm, i.e., we improve Lemma 3 from the work of [38]. Here we follow the notations from [38], and we defer the detailed notations, as well as the proof of Theorem 6 below, to App. H due to space limitation.
<sup>1</sup>We have omitted the dependency on  $\log(1/\delta)$  for simplicity. Refer to App. F for the full expression.
2This is the high-probability regret (rather than expected regret) from [38], which we focus on in this work.
Theorem 6 (Improved Confidence Ellipsoid for [38]). With probability of at least  $1 - \delta$ ,
$$
\theta^ {*} \in \mathcal {C} _ {s} = \left\{\theta \in \mathbb {R} ^ {d}: \left\| \theta - \widehat {\theta} _ {s} \right\| _ {V _ {s}} \leq \left(\sqrt {2 d \log \left(1 + \frac {L ^ {2}}{\lambda} \sum_ {k = 1} ^ {s} \frac {1}{\epsilon_ {k} ^ {2}}\right) \frac {1}{\delta}} + \sqrt {\lambda} S\right) \right\}, \quad \forall s \in [ m ].
$$
Note that the size of the confidence ellipsoid from our Theorem 6 is of the order  $\mathcal{O}(\sqrt{d\log(\sum_{k = 1}^{s}1 / \epsilon_k^2)}) = \mathcal{O}(\sqrt{d\log T})$  (we have used Lemma 2 here), which improves over the  $\mathcal{O}(\sqrt{ds}) = \mathcal{O}(\sqrt{dm}) = \mathcal{O}(\sqrt{d\times d\log T}) = \mathcal{O}(d\sqrt{\log T})$  from Lemma 3 of [38] by a factor of  $\sqrt{d}$ . Plugging in our tighter confidence ellipsoid (Theorem 6) into the regret analysis of Q-LinUCB, the regret upper bound for Q-LinUCB is improved to  $R_{T} = \mathcal{O}\big(d^{3 / 2}\log^{3 / 2}T\log (d\log T)\big)$  (more details at the end of App. H). This improved regret upper bound exactly matches that of our Q-GP-UCB with the linear kernel (Theorem 4). Similar to Theorem 3, the most crucial step in the proof of Theorem 6 is to apply the self-normalized concentration inequality from Theorem 1 of [2] to 1-sub-Gaussian noise. Again this is feasible because  $\epsilon_s = \widetilde{\sigma}_{s - 1}(x_s) / \sqrt{\lambda}$  is  $\mathcal{F}_{s - 1}$ -measurable, and conditioned on  $\mathcal{F}_{s - 1}$ , the scaled noise  $\zeta_s / \epsilon_s$  is 1-sub-Gaussian.
5.7 Regret Upper Bound for the Matérn Kernel
So far we have mainly focused on the linear and SE kernels in our analysis. Here we derive a regret upper bound for our Q-GP-UCB algorithm for the Matérn kernel with smoothness parameter  $\nu$ .
Theorem 7 (Matérn Kernel, Bounded Noise). With probability of at least  $1 - \delta$ , we have that
$$
R _ {T} = \widetilde {\mathcal {O}} \big (T ^ {3 d / (2 \nu + d)} \big).
$$
The proof is in App. I. Note that the state-of-the-art regret upper bound for the Matérn kernel in the classical setting is  $R_{T} = \mathcal{O}(\sqrt{T\gamma_{T}}) = \widetilde{\mathcal{O}} (T^{(\nu +d) / (2\nu +d)})$  [7, 24, 30, 36], which matches the corresponding classical regret lower bound (up to logarithmic factors in  $T$ ) [35]. Therefore, the regret upper bound of our Q-GP-UCB for the Matérn kernel (Theorem 7) improves over the corresponding classical regret lower bound when  $\nu >2d$ , i.e., when the reward function is sufficiently smooth. Also note that similar to the classical GP-UCB [33] (with regret  $R_{T} = \mathcal{O}(\gamma_{T}\sqrt{T}) = \widetilde{\mathcal{O}} (T^{(\nu +3d / 2) / (2\nu +d)})$ ) which requires the reward function to be sufficiently smooth (i.e.,  $\nu >d / 2$ ) to attain sub-linear regrets for the Matérn kernel, our Q-GP-UCB, as the first quantum BO algorithm, also requires the reward function to be smooth enough (i.e.,  $\nu >d$ ) in order to achieve a sub-linear regret upper bound for the Matérn kernel. We leave it to future works to further improve our regret upper bound and hence relax this requirement for smooth functions.
We use the Qiskit python package to implement the QMC algorithm (Lemma 1) following the recent work of [20]. Some experimental details are deferred to App. J due to space limitation.
Synthetic Experiment. Here we use a grid of  $|\mathcal{X}| = 20$  equally spaced points within [0,1] as the 1-dimensional input domain  $\mathcal{X}$  ( $d = 1$ ), and sample a function  $f$  from a GP prior with the SE kernel. The sampled function  $f$  is scaled so that its output is bounded within [0,1], and then used as the reward function in the synthetic experiments. We consider two types of noises: (a) bounded noise (within [0,1]) and (b) Gaussian noise, which correspond to the two types of noises in Lemma 1, respectively. For (a) bounded noise, we follow the practice of [38] such that when an input  $x$  is selected, we treat the function value  $f(x)$  as the probability for a Bernoulli distribution, i.e., we observe an output of 1 with probability of  $f(x)$  and 0 otherwise. For (b) Gaussian noise, we simply add a zero-mean Gaussian noise with variance  $\sigma^2$  to  $f(x)$ . The results for (a) bounded noise and (b) Gaussian noise are shown in Figs. 1 (a) and (b), respectively. The figures show that for both types of noises, our Q-GP-UCB significantly outperforms the classical baseline of GP-UCB. Specifically, although our Q-GP-UCB incurs larger regrets in the initial stage, it is able to leverage the accurate observations provided by the QMC subroutine to rapidly find the global optimum. These results show that the quantum speedup of our Q-GP-UCB in terms of the tighter regret upper bounds (Theorems 4 and 5) may also be relevant in practice. We have additionally compared with linear UCB (LinUCB) [2] and Q-LinUCB [38], which are, respectively, the most representative classical and quantum linear bandit algorithms. The results in Fig. 2 (App. J) show that in these experiments where the reward
(a) synthetic (Bernoulli)
Figure 1: Cumulative regret for synthetic experiments with (a) bounded noise and (b) Gaussian noise. (c) Cumulative regret for the AutoML experiment; (d) additionally includes results for linear bandits.
(b) synthetic (Gaussian)
(c) AutoML
(d) AutoML
function is non-linear, algorithms based on linear bandits severely underperform compared with BO/kernelized bandit algorithms in both the classical and quantum settings.