Title: Quantum Bayesian Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: quantum bandits

kernelized bandits
Abstract: Kernelized bandits, also known as Bayesian optimization (BO), has been a prevalent method for optimizing complicated black-box reward functions. Various BO algorithms have been theoretically shown to enjoy upper bounds on their cumulative regret which are sub-linear in the number $T$ of iterations, and a regret lower bound of $\Omega(\sqrt{T})$ has been derived which represents the unavoidable regrets for any classical BO algorithm. Recent works on quantum bandits have shown that with the aid of quantum computing, it is possible to achieve tighter regret upper bounds better than their corresponding classical lower bounds. However, these works are restricted to either multi-armed or linear bandits, and are hence not able to solve sophisticated real-world problems with non-linear reward functions. To this end, we introduce the quantum-Gaussian process-upper confidence bound (Q-GP-UCB) algorithm. To the best of our knowledge, our Q-GP-UCB is the first BO algorithm able to achieve a regret upper bound of $\mathcal{O}(\text{poly}\log T)$, which is significantly smaller than its regret lower bound of $\Omega(\sqrt{T})$ in the classical setting. Moreover, thanks to our novel analysis of the confidence ellipsoid, our Q-GP-UCB with the linear kernel achieves a smaller regret than the quantum linear UCB algorithm from the previous work. We use simulations, as well as an experiment using a real quantum computer, to verify that the theoretical quantum speedup achieved by our Q-GP-UCB is also potentially relevant in practice.

Quantum Bayesian Optimization

Abstract

Kernelized bandits , also known as Bayesian optimization (BO), has been a prevalent method for optimizing complicated black-box reward functions. Various BO algorithms have been theoretically shown to enjoy upper bounds on their cumulative regret which are sub-linear in the number T of iterations, and a regret lower bound of Ω( √ T ) has been derived which represents the unavoidable regrets for any classical BO algorithm. Recent works on quantum bandits have shown that with the aid of quantum computing, it is possible to achieve tighter regret upper bounds better than their corresponding classical lower bounds. However, these works are restricted to either multi-armed or linear bandits, and are hence not able to solve sophisticated real-world problems with non-linear reward functions. To this end, we introduce the quantum-Gaussian process-upper confidence bound (Q-GP-UCB) algorithm. To the best of our knowledge, our Q-GP-UCB is the first BO algorithm able to achieve a regret upper bound of O ( poly log T ) , which is significantly smaller than its regret lower bound of Ω( √ T ) in the classical setting. Moreover, thanks to our novel analysis of the confidence ellipsoid, our Q-GP-UCB with the linear kernel achieves a smaller regret than the quantum linear UCB algorithm from the previous work. We use simulations, as well as an experiment using a real quantum computer , to verify that the theoretical quantum speedup achieved by our Q-GP-UCB is also potentially relevant in practice.

1 Introduction

Kernelized bandits [9], also named Bayesian optimization (BO) [10, 11, 16, 19, 32], has been an immensely popular method for various applications involving the optimization of complicated blackbox reward functions. For example, BO has been extensively used to optimize the hyperparameters of machine learning (ML) models [12, 14, 22], the parameters of computationally expensive simulators [5], etc. In every iteration t = 1 , . . . , T , BO chooses an arm/input x t and then queries the reward function f for a noisy observation y t = f ( x t ) + ζ t where ζ t is a sub-Gaussian noise. In addition to its impressive practical performance, BO is also equipped with solid theoretical guarantees. A number of commonly adopted BO algorithms have been theoretically shown to enjoy sub-linear upper bounds on their cumulative regret [9, 33], which ensures that they are guaranteed to be able to find the global optimum of the reward function as T (i.e., the total number of function queries) increases. On the other hand, a lower bound of Ω( √ T ) (ignoring additional log factors) on the cumulative regret of BO has been shown [31], which represents the fundamental limit of any BO algorithm (in the classical setting). In other words, no classical BO algorithm can achieve a cumulative regret smaller than Ω( √ T ) . This naturally begs the question: can we leverage more advanced technology to go beyond

√

the classical setting and hence break this fundamental limit of Ω( T ) ? In this work, we give an affirmative answer by showing that this can be achieved with the aid of quantum computing [34].

Quantum bandits, which incorporates quantum computing into bandit algorithms, has been studied by a number of recent works [8, 38, 40, 41]. Notably, the recent work of [38] has introduced quantum variants of classical algorithms for multi-armed bandits (MAB) and stochastic linear bandits (SLB). In the setting of quantum bandits adopted by [38], every query to the reward function f at the arm x t (in the classical setting) is replaced by a chance to access a quantum oracle , which encodes the reward distribution for the arm x t . For every selected arm x t , [38] has proposed to adopt the quantum Monte Carlo (QMC) algorithm [4, 28] as a subroutine to obtain an accurate estimation of f ( x t ) in an efficient way, i.e., using a small number of queries to the quantum oracle (Lemma 1). As a result, [38] has shown that the regrets of the quantum algorithms for both MAB and SLB are significantly improved compared with their classical counterparts and are smaller than their classical lower bounds. However, both MAB and SLB fall short when optimizing complex real-world functions (e.g., optimizing the hyperparameters of ML models). This is because MAB is unable to exploit the correlations among the arms to model the reward function, and the assumption of a linear reward function adopted by SLB is usually too restrictive in practice. Therefore, designing a quantum bandit algorithm capable of optimizing sophisticated non-linear reward functions, which is a critical step towards practically useful quantum bandit algorithms, is still an open problem. In this work, we resolve this open problem by proposing the first algorithm for quantum BO.

Similar to [38], in our quantum BO problem setting, queries to the reward function in the classical setting are replaced by access to a quantum oracle encoding the reward distribution. As discussed in [38, 39], a quantum oracle is available when the learning environment is implemented by a quantum algorithm, which makes the setting of our quantum BO fairly general. For example, our quantum BO algorithm may be used to optimize the hyperparameters of quantum ML algorithms [3], such as quantum support vector machines (SVMs) [29], quantum neural networks (NNs) [1, 18], among others. It may also be used to optimize the parameters of simulators implemented on a quantum computer, or for applications involving quantum systems where the data produced is inherently quantum. Moreover, our quantum BO algorithm could also be applied to classical data and algorithms, because as discussed in [39], any classical computer program can be converted to a quantum circuit, allowing it to serve as a quantum oracle in our quantum BO algorithm.

In this work, we introduce the first quantum BO algorithm: quantum-Gaussian process-upper confidence bound (Q-GP-UCB). For every selected arm x s , our Q-GP-UCB algorithm (Algo. 1) adopts the QMC subroutine (Lemma 1) to efficiently estimate the corresponding reward function value f ( x s ) to satisfy a target estimation error ϵ s . Every arm x s is selected based on our weighted GP posterior distribution, in which every previously selected arm x s is given a weight of 1 /ϵ 2 s which is inversely related to its estimation error. The theoretical analysis of our Q-GP-UCB is faced with non-trivial technical challenges, and our contributions can be summarized as follows:

We analyze the growth rate of the weighted information gain (Sec. 5.1) which arises due to the use of our weighted GP regression (Sec. 4.1), and show its connection with the standard maximum information gain commonly used in the analysis of BO/kernelized bandits [9, 33]. Our analysis here may be of broader independent interest for future works adopting weighted GP regression.

We derive a tight confidence ellipsoid which gives a guarantee on the concentration of the reward function around the weighted GP posterior mean (Sec. 5.3), and discuss the intuition behind our proof which corresponds to an interesting interpretation about our Q-GP-UCB algorithm. A particularly crucial and novel step in our analysis relies on the recognition to apply the concentration inequality for 1-sub-Gaussian noise (Sec. 5.3).

We prove that our Q-GP-UCB achieves a regret upper bound of O ( poly log T ) for the commonly used squared exponential (SE) kernel (Secs. 5.4 and 5.5), which is considerably smaller than the classical regret lower bound of Ω( √ T ) [31] and hence represents a significant quantum speedup.

By using a similar proof technique to the one adopted for our tight confidence ellipsoid (Sec. 5.3), we improve the confidence ellipsoid for the quantum linear UCB (Q-LinUCB) algorithm [38]. This improved analysis leads to a tighter regret upper bound for Q-LinUCB , which matches the regret of our Q-GP-UCB with the linear kernel (Sec. 5.6).

Weuse simulations implemented based on the Qiskit package to verify the empirical improvement of our Q-GP-UCB over the classical GP-UCB and over Q-LinUCB, through a synthetic experiment and an experiment on automated machine learning (AutoML) (Sec. 6). Notably, we have also

performed an experiment using a real quantum computer , in which our Q-GP-UCB still achieves a consistent performance advantage (Fig. 4, App. J)

2 Related Work

A number of recent works have studied bandit algorithms in the quantum setting. The works of [8] and [40] have focused on the problem of pure exploration in quantum bandits. [25] has studied a different problem setting where the learner aims to maximize some property of an unknown quantum state (i.e., the rewards) by sequentially selecting the observables (i.e., actions). The recent work of [38] has introduced algorithms for quantum multi-armed bandits (MAB) and stochastic linear bandits (SLB), and has shown that by incorporating the QMC subroutine into MAB and SLB, tight regret upper bounds can be achieved which are better than the classical lower bounds. More recently, the works of [23] and [41] have followed similar approaches to introduce quantum bandit algorithms for, respectively, stochastic convex bandits and bandits with heavy-tailed reward distributions. In addition to quantum bandits, some recent works have introduced quantum reinforcement learning (RL) algorithms. [39] has assumed access to a generative model of the environment and proved that their quantum RL algorithm achieves significantly better sample complexity over their classical counterparts. More recently, [17, 43] have removed the requirement for a generative model in quantum RL and achieved quantum speedup in terms of the regret. More comprehensive reviews of the related works on quantum RL can be found in recent surveys (e.g., [27]). The problem setting of our quantum BO (Sec. 3.2) is the same as many of these above-mentioned previous works on quantum bandits [38, 41] and quantum RL [17, 39, 43]. However, to the best of our knowledge, none of the existing works on quantum bandits (and quantum RL) is able to handle non-linear reward functions, which we resolve in this work. Over the years, extensive efforts [7, 24, 30, 36] have been made to design BO/kernelized bandit algorithms whose regret upper bounds are small enough to (nearly) match the classical regret lower bound of Ω( √ T ) (ignoring additional log factors). Our work provides an alternative route by proving that with the aid of quantum computing, it is possible to achieve a tight regret upper bound which is significantly smaller than the classical regret lower bound.

3 Problem Setting and Background

3.1 Classical Kernelized Bandits

A BO/kernelized bandit algorithm aims to maximize a reward function f : X → R where X ⊂ R d , i.e., it aims to find x ∗ ∈ arg max x ∈X f ( x ) . Consistent with the previous works on BO/kernelized bandits [9, 37], we assume that f lies in the reproducing kernel Hilbert space (RKHS) associated with a kernel k : f ∈ H k ( X ) . That is, we assume that ∥ f ∥ H k ≤ B for B > 0 , in which ∥·∥ H k denotes the RKHS norm induced by the kernel k . Note that unlike previous works on linear bandits [2], our assumption here allows f to be non-linear w.r.t. the input x . In this work, we mainly focus on the widely used squared exponential (SE) kernel: k ( x, x ′ ) ≜ exp( -∥ ∥ x -x ′ ∥ ∥ 2 2 / (2 l 2 )) in which l is the length scale . We also extend our results to the Matérn kernel in Sec. 5.7. Without loss of generality, we assume that k ( · , · ) ≤ 1 . In the following, we use [ t ] to denote { 1 , . . . , t } for simplicity.

In every iteration t ∈ [ T ] of BO, an input x t is selected, after which a corresponding noisy observation y t = f ( x t ) + ζ t is collected where ζ t is a sub-Gaussian noise (e.g., bounded noise or Gaussian noise). The inputs x t 's are sequentially selected by maximizing an acquisition function , which is calculated based on the Gaussian process (GP) posterior distribution. Specifically, after t iterations, we use the current history D t ≜ { ( x 1 , y 1 ) , . . . , ( x t , y t ) } to calculate the GP posterior predictive distribution at x ∈ X : N ( µ t ( x ) , σ 2 t ( x )) where

in which k t ( x ) ≜ [ k ( x τ , x )] ⊤ τ ∈ [ t ] and Y t ≜ [ y x τ ] ⊤ τ ∈ [ t ] are column vectors, K t ≜ [ k ( x τ , x τ ′ )] τ,τ ′ ∈ [ t ] is the t × t covariance matrix, and λ > 1 is a regularization parameter. Based on (1), a commonly used acquisition function is GP-UCB [33], which selects the next query by: x t +1 = arg max x ∈X µ t ( x ) + ξ t +1 σ t ( x ) where ξ t +1 is a parameter carefully chosen to balance exploration and exploitation. The performance of a BO/kernelized bandit algorithm is usually theoretically analyzed by deriving an upper bound on its cumulative regret: R T = ∑ T t =1 [ f ( x ∗ ) -f ( x t )] , and a tighter regret upper bound is an indicator of a better theoretical convergence.

3.2 Quantum Bandits/BO

A quantum state | x ⟩ in Hilbert space C n can be expressed as a superposition of n basis states (e.g. qubits) via a vector ⃗ x = [ x 1 , . . . , x n ] ⊤ , with | x i | 2 representing the probability of being in the i th basis state. Given two quantum states | x ⟩ ∈ C n and | y ⟩ ∈ C m , we use | x ⟩| y ⟩ = [ x 1 y 1 , . . . , x n y m ] ⊤ to denote their tensor product. A quantum algorithm typically works by applying unitary operators to quantum states, and may have access to input data encoded in unitary operators called quantum oracles (examples have been discussed in Sec. 1). We defer a more detailed introduction to quantum computing to related works dedicated to these topics (e.g., [26]).

Our quantum BO setting follows that of the quantum bandits works in [38, 41]. In every iteration of quantum BO, after an input x is selected, instead of observing a noisy reward as in classical BO/bandits (Sec. 3.1), we get a chance to access a quantum unitary oracle O x and its inverse that encode the noisy reward distribution. Specifically, let P x denote the reward distribution, Ω x denote the finite sample space of P x , and y x : Ω x → R denote the random reward associated with input x . Then, O x is formally defined as:

Quantum mean estimation , which aims to estimate the mean of an unknown distribution with better sample efficiency than classical algorithms, has been a widely studied problem [4, 21]. Consistent with the works of [38, 41], we will make use of the following quantum Monte Carlo (QMC) algorithm:

Lemma 1 (Quantum Monte Carlo (QMC) [28]) . Let y x : Ω x → R denote a random variable, Ω x is equipped with probability measure P x , and the quantum unitary oracle O x encodes P x and y x .

Bounded Noise : If the noisy output observation satisfies y x ∈ [0 , 1] , then there exists a constant C 1 > 1 and a QMC algorithm QMC ( O x , ϵ, δ ) which returns an estimate ˆ y x of E [ y x ] such that P ( | ˆ y x -E [ y x ] | > ϵ ) ≤ δ , using at most C 1 ϵ log(1 /δ ) queries to O x and its inverse.

Noise with Bounded Variance : If the variance of y x is ≤ σ 2 , then for ϵ < 4 σ , there exists a constant C 2 > 1 and a QMC algorithm QMC ( O x , ϵ, δ ) which returns an estimate ˆ y x s.t. P ( | ˆ y x -E [ y x ] | > ϵ ) ≤ δ , using at most C 2 σ ϵ log 3 / 2 2 ( 8 σ ϵ ) log 2 (log 2 8 σ ϵ ) log 1 δ queries to O x and its inverse.

The quantum query complexity of a quantum algorithm is usually measured by the number of queries to the quantum oracle [3, 38]. So, the sample complexities from Lemma 1 can be compared with that of classical Monte Carlo (MC) estimation. In the classical setting, it can be easily shown using concentration inequalities that MC estimation requires ˜ O (1 /ϵ 2 ) samples to reach a target mean estimation error of ϵ . Therefore, the QMC algorithm (Lemma 1), which only needs ˜ O (1 /ϵ ) samples, achieves a quadratic reduction in the required number of samples. This dramatic improvement is crucial for the quantum speedup in terms of regrets achieved by our algorithm (Sec. 5.4).

During our Q-GP-UCB algorithm, after every input x s is selected, we will apply the QMC algorithm from Lemma 1 with the quantum oracle O x s to obtain an estimation y s of its reward f ( x s ) (line 6 of Algo. 1). Of note, the above-mentioned equivalence between a query to the quantum oracle and a sample for classical MC mean estimation implies that a query to the quantum oracle O x in quantum BO/bandits is equivalent to the pulling of an arm x in classical BO/bandits . Therefore, the cumulative regret R T of our Q-GP-UCB algorithm is defined as the regret incurred after T queries to the quantum oracle, which makes it amenable to comparisons with the regrets of classical algorithms.

4 Quantum Bayesian Optimization

In this section, we first introduce weighted GP regression (Sec. 4.1) which will be used by our Q-GP-UCB algorithm to calculate the acquisition function for input selection, and then describe our Q-GP-UCB algorithm (Sec. 4.2) in detail.

4.1 Weighted GP Posterior Distribution

In contrast to standard GP-UCB [33] which uses the standard GP posterior distribution (1) to calculate the acquisition function, our Q-GP-UCB makes use of a weighted GP posterior distribution. That is, we assign a weight to every previous observation. Specifically, after stage s of our Q-GP-UCB (i.e., given D s ≜ { ( x τ , y τ ) } τ ∈ [ s ] ), we define the weight matrix W s ≜ diag (1 /ϵ 2 1 , . . . , 1 /ϵ 2 s ) . W s is

an s × s diagonal matrix, in which the τ th diagonal element represents the weight 1 /ϵ 2 τ given to the τ th observation ( x τ , y τ ) . We will set ϵ τ = ˜ σ τ -1 ( x τ ) / √ λ (Sec. 4.2), i.e., ϵ τ is calculated using the weighted GP posterior standard deviation (3) (conditioned on the first τ -1 observations) at x τ .

Define K s ≜ [ k ( x τ , x τ ′ )] τ,τ ′ ∈ [ s ] which is the s × s -dimensional covariance matrix given the first s observations, and define ˜ K s ≜ W 1 / 2 s K s W 1 / 2 s which is the weighted covariance matrix. Similarly, define k s ( x ) ≜ [ k ( x, x τ )] ⊤ τ ∈ [ s ] and ˜ k s ( x ) ≜ W 1 / 2 s k s ( x ) . Denote the collection of output observations by Y s ≜ [ y τ ] ⊤ τ ∈ [ s ] , and define ˜ Y s ≜ W 1 / 2 s Y s . With these definitions, given D s , our weighted GP posterior distribution at an input x ∈ X is a Gaussian distribution: N ( ˜ µ s ( x ) , ˜ σ 2 s ( x )) , in which

Note that the GP posterior mean ˜ µ s above is equivalently the solution to the following weighted kernel ridge regression problem: ˜ µ s = arg min f ∈ H k ( X ) ∑ s τ =1 1 ϵ 2 τ ( y τ -f ( x τ )) 2 + λ ∥ f ∥ 2 H k . We give a more detailed analysis of the weighted GP posterior (3) in App. A. Weighted GP regression has also been adopted by previous works on BO such as [15]. However, our choice of the weights, algorithmic design and theoretical analyses all require significantly novel treatments.

4.2 Q-GP-UCB Algorithm

Algorithm 1 Q-GP-UCB

1: for stage s = 1 , 2 , . . . do

2: x s = arg max x ∈X ˜ µ s -1 ( x ) + β s ˜ σ s -1 ( x ) (3). √

4: (a) N ϵ s = C 1 ϵ s log( 2 m δ ) (for bounded noise), or

3: ϵ s = ˜ σ s -1 ( x s ) / λ .

(b) N ϵ s = C 2 σ ϵ s log 3 / 2 2 ( 8 σ ϵ s ) log 2 (log 2 8 σ ϵ s ) log 2 m δ (for noise with variance bounded by σ 2 ).

6: Run the QMC ( O x s , ϵ s , δ/ (2 m )) algorithm to query the quantum oracle of x s for the next N ϵ s rounds, and obtain y s as an estimate of f ( x s ) .

5: If ∑ s k =1 N ϵ k > T , terminate the algorithm.

7: Update the weighted GP posterior (3) using ( x s , y s ) .

Our Q-GP-UCB algorithm is presented in Algo. 1. Q-GP-UCB proceeds in stages. In stage s , we first select the next input x s to query by maximizing the GP-UCB acquisition function calculated using the weighted GP posterior distribution (3) (line 2 of Algo. 1). Here β s ≜ B + √ 2( ˜ γ s -1 +1+log(2 /δ )) (more details in Sec. 5.3), in which δ ∈ (0 , 2 /e ] and ˜ γ s -1 ≜ 1 2 log( det ( I + 1 λ ˜ K s -1 )) is the weighted information gain (more details in Sec. 5.1). Next, we calculate ϵ s = ˜ σ s -1 ( x s ) / √ λ (line 3) and N ϵ s (line 4), in which N ϵ s depends on the type of noise and the value of ϵ s . Here m is an upper bound on the total number m of stages which we will analyze in Sec. 5.2. Subsequently, unless the algorithm is terminated (line 5), we run the QMC algorithm (Lemma 1) to estimate f ( x s ) by querying the quantum oracle of x s for N ϵ s rounds (line 6). The QMC procedure returns an estimate y s of the reward function value f ( x s ) , for which the estimation error is guaranteed to be bounded: | y s -f ( x s ) | ≤ ϵ s with probability of at least 1 -δ/ (2 m ) . Lastly, we update the weighted GP posterior (3) using the newly collected input-output pair ( x s , y s ) , as well as its weight 1 /ϵ 2 s (line 7).

Of note, the value of ϵ s is used for both (a) calculating the number N ϵ s of queries to the quantum oracle for x s , and (b) computing the weight 1 /ϵ 2 s assigned to ( x s , y s ) in the weighted GP regression (3) in the subsequent iterations. Regarding (a) , our designs of ϵ s and N ϵ s = ˜ O (1 /ϵ s ) have an interesting interpretation in terms of the exploration-exploitation trade-off: In the initial stages, the weighted GP posterior standard deviation ˜ σ s -1 ( x s ) and hence ϵ s are usually large, which leads to a small number N ϵ s of queries for every x s and hence allows our Q-GP-UCB to favor the exploration of more unique inputs; in later stages, ˜ σ s -1 ( x s ) and ϵ s usually become smaller, which results in large N ϵ s 's and hence causes our Q-GP-UCB to prefer the exploitation of a small number of unique inputs. Regarding (b) , assigning a larger weight 1 /ϵ 2 s to an input x s with a smaller ϵ s is reasonable, because a smaller ϵ s indicates a smaller estimation error for y s as we explained above, which makes the observation y s more accurate and reliable for calculating the weighted GP regression (3).

Note that we have modified the original GP-UCB by querying every selected input x s multiple times, in order to make it amenable to the integration of the QMC subroutine. The recent work of [6] has also

adapted BO to evaluate a small number of unique inputs while querying each of them multiple times. It has been shown [6] that the resulting BO algorithm preserves both the theoretical guarantee (i.e., the regret upper bound) and empirical performance of the original BO, while significantly reducing the computational cost. So, the findings from [6] can serve as justifications for our modification to GP-UCB. Also note that despite this similarity, [6] still focuses on the traditional setting (i.e., their regret upper bound is R T = O ( γ T √ T ) ) and their regret analyses are entirely different from ours.

5 Theoretical Analysis

Throughout our analysis, we condition on the event that | f ( x s ) -y s | ≤ ϵ s , ∀ s = 1 , . . . , m . This event holds with probability of at least 1 -δ/ 2 , because we set the error probability in QMC (Lemma 1) to δ/ (2 m ) in which m ≥ m (see Theorem 2 for details). For simplicity, we mainly focus on the scenario of bounded noise (i.e., the observations are bounded within [0 , 1] , the first case of Lemma 1), because the theoretical analyses and insights about the other scenario of noise with bounded variance (i.e., the second case of Lemma 1) are almost identical (more details in Sec. 5.5).

5.1 Weighted Information Gain

To begin with, the following lemma (proof in App. B) gives an upper bound on the sum of the weights in all m stages, which will be useful for upper-bounding the weighted information gain ˜ γ m .

Denote by γ T the maximum information gain from any set of T inputs: γ T ≜ max { x 1 ,...,x T }⊂X 1 2 log det( I + 1 λ K T ) , which is commonly used in the analysis of BO/kernelized bandit algorithms [9, 33]. Denote by ˜ γ m the weighted information gain from all m selected inputs in all stages: ˜ γ m ≜ 1 2 log det( I + 1 λ ˜ K m ) . Note that in the definition of ˜ γ m , we do not take the maximum over all sets of m inputs, so ˜ γ m still depends on the selected inputs X m ≜ { x 1 , . . . , x m } . Also note that { ϵ 1 , . . . , ϵ m } are known constants conditioned on X m . This is because the weighted GP posterior standard deviation ˜ σ s -1 (3) does not depend on the output observations, so every ϵ s = ˜ σ s -1 ( x s ) / √ λ only depends on { x 1 , . . . , x s } . The next theorem gives an upper bound on ˜ γ m .

We give the complete statement of result (a) in Theorem 8 (App. C), which presents the concrete growth rate. As stated by result (a) , to obtain an upper bound on ˜ γ m , we simply need to firstly use the upper bound on γ T from previous works [35], and then replace the T in this upper bound by ∑ m s =1 1 /ϵ 2 s . Based on this, the result (b) has further upper-bounded ∑ m s =1 1 /ϵ 2 s by T 2 through Lemma 2. Note that because ϵ s = ˜ σ s -1 ( x s ) / √ λ ≤ 1 , we have that 1 /ϵ 2 s ≥ 1 and hence ∑ m s =1 1 /ϵ 2 s ≥ m . Therefore, our weighted information gain ˜ γ m has a looser upper bound than γ m . This intuitively implies that our weighted covariance matrix ˜ K m is expected to contain more information than the original K m , whose implication will be discussed again in Sec. 5.2. The proof of Theorem 1 (App. C) has followed the analysis of the information gain from [35]. Specifically, we have leveraged a finitedimensional projection of the infinite-dimensional RKHS feature space, and hence upper-bounded the information gain in terms of the tail mass of the eigenvalues of the kernel k . The proof requires carefully tracking the impact of the weights 1 /ϵ 2 s throughout the analysis. Importantly, our Theorem 1 and its analysis may be of wider independent interest for future works which also adopt weighted GP regression (Sec. 4.1).

The growth rate of γ T has been characterized for commonly used kernels [35]. Based on these, the next corollary provides the growth rates of ˜ γ m for the linear and squared exponential (SE) kernels.

Corollary 1. For the linear and squared exponential (SE) kernels, we have that

5.2 Upper Bound on The Total Number m of Stages

The next theorem gives an upper bound on the total number of stages of our Q-GP-UCB algorithm.

Theorem 2. For the linear kernel, our Q-GP-UCB algorithm has at most m = O ( d log T ) stages. For the SE kernel, our Q-GP-UCB algorithm has at most m = O ((log T ) d +1 ) stages.

Note that for the linear kernel, our upper bound on m matches that of the Q-LinUCB algorithm from [38] (see Lemma 2 from [38]). The proof of Theorem 2 is given in App. D, and here we give a brief sketch of the proof. For stage s , define V s ≜ λI + ∑ s τ =1 (1 /ϵ 2 τ ) ϕ ( x τ ) ϕ ( x τ ) ⊤ , in which ϕ ( · ) is the (potentially infinite-dimensional) RKHS feature mapping: k ( x, x ′ ) = ϕ ( x ) ⊤ ϕ ( x ′ ) . Intuitively, log det V s represents the amount of information contained in the first s selected inputs { x 1 , . . . , x s } . As the first step in our proof, we prove that thanks to our choice of ϵ s (line 3 of Algo. 1) and our design of the weights 1 /ϵ 2 s , the amount of information log det V s is doubled after every stage: det V τ +1 = 2det V τ , ∀ τ = 0 , . . . , m -1 . This allows us to show that m log 2 = log(det( V m ) / det( V 0 )) where V 0 = λI . Next, we show that this term is also related to our weighted information gain: log(det( V m ) / det( V 0 )) = 2 ˜ γ m , which allows us to upper-bound m in terms of ˜ γ m and hence in terms of ∑ m s =1 1 /ϵ 2 s (see Corollary 1). This eventually allows us to derive an upper bound on m by using the fact that the total number of iterations (i.e., queries to the quantum oracles) is no larger than T . Therefore, the key intuition of the proof here is that as long as we gather a sufficient amount of information in every stage, we do not need a large total number of stages.

5.3 Confidence Ellipsoid

The next theorem proves the concentration of f around the weighted GP posterior mean (3).

Theorem 3. Let η ≜ 2 /T , λ ≜ 1 + η , and β s ≜ B + √ 2( ˜ γ s -1 +1+log(2 /δ )) . We have with -

The proof of Theorem 3 is presented in App. E, and here we give a brief proof sketch. Denote by F s -1 the σ -algebra F s -1 = { x 1 , . . . , x s , ζ 1 , . . . , ζ s -1 } where ζ k = y k -f ( x k ) is the noise. Recall that W 1 / 2 s ≜ diag (1 /ϵ 1 , . . . , 1 /ϵ s ) (Sec. 4.1), and define ζ 1: s ≜ [ ζ k ] k ∈ [ s ] . In the first part of the proof, we use the RKHS feature space view of the weighted GP posterior (3) (see App. A) to upper-bound | f ( x ) -˜ µ s -1 ( x ) | in terms of ˜ σ s -1 ( x ) and || W 1 / 2 s ζ 1: s || (( ˜ K s + ηI ) -1 + I ) -1 . Next, the most crucial step in the proof is to upper-bound || W 1 / 2 s ζ 1: s || (( ˜ K s + ηI ) -1 + I ) -1 in terms of ˜ γ s by applying the self-normalized concentration inequality from Theorem 1 of [9] to 1-sub-Gaussian noise . This is feasible because (a) every ϵ s = ˜ σ s -1 ( x s ) / √ λ is F s -1 -measurable, and (b) conditioned on F s -1 , the scaled noise term ζ s /ϵ s (in W 1 / 2 s ζ 1: s ) is 1 -sub-Gaussian. The statement (a) follows because ϵ s only depends on { x 1 , . . . , x s } as we have discussed in Sec. 5.1. The statement (b) follows since our QMC subroutine ensures that every noise term ζ k is bounded within [ -ϵ k , ϵ k ] (Sec. 4.2) because | y k -f ( x k ) | ≤ ϵ k (with high probability). This suggests that conditioned on F s -1 (which makes ϵ s a known constant as discussed above), ζ s /ϵ s is bounded within [ -1 , 1] and is hence 1-sub-Gaussian.

This critical step in the proof is in fact also consistent with an interesting interpretation about our algorithm. That is, after x s is selected in stage s , we adaptively decide the number N ϵ s of queries to the quantum oracle in order to reach the target accuracy ϵ s (line 4 of Algo. 1). This ensures that | y s -f ( x s ) | ≤ ϵ s (with high probability), which guarantees that the noise ζ s = y s -f ( x s ) is (conditionally) ϵ s -sub-Gaussian. Moreover, note that the vector of observations ˜ Y s used in the calculation of the weighted GP posterior mean (3) is ˜ Y s = W 1 / 2 s Y s . So, from the perspective of the weighted GP posterior (3), every noisy observation y s , and hence every noise ζ s , is multiplied by 1 /ϵ s . So, the effective noise of the observations ˜ Y s used in the weighted GP posterior (i.e., ζ s /ϵ s ) is ϵ s /ϵ s = 1 -sub-Gaussian. This shows the underlying intuition as to why our proof of the concentration of the reward function f using the weighted GP regression can make use of 1-sub-Gaussian noise (e.g., in contrast to the σ -sub-Gaussian noise used in the proof of classical GP-UCB [9]).

Our tight confidence ellipsoid around f from Theorem 3 is crucial for deriving a tight regret upper bound for our Q-GP-UCB (Sec. 5.4). Moreover, as we will discuss in more detail in Sec. 5.6, our proof technique for Theorem 3, interestingly, can be adopted to obtain a tighter confidence ellipsoid for the Q-LinUCB algorithm from [38] and hence improve its regret upper bound.

5.4 Regret Upper Bound

Here we derive an upper bound on the cumulative regret of our Q-GP-UCB algorithm.

Theorem 4. With probability of at least 1 -δ , we have that 1

The proof of Theorem 4 is presented in App. F. The proof starts by following the standard technique in the proof of UCB-type algorithms to show that r s = f ( x ∗ ) -f ( x s ) ≤ 2 β s ˜ σ s -1 ( x s ) . Next, importantly, our design of ϵ s = ˜ σ s -1 ( x s ) / √ λ allows us to show that r s ≤ 2 β s ϵ s √ λ . After that, the total regrets incurred in a stage s can be upper-bounded by N ϵ s × 2 β s ϵ s √ λ = O ( 1 ϵ s log( m/δ ) × β s ϵ s ) = O ( β s log( m/δ )) . Lastly, the regret upper bound follows by summing the regrets from all m stages. Of note, for the commonly used SE kernel, our regret upper bound is of the order O ( poly log T ) , which is significantly smaller than the regret lower bound of Ω( √ T ) in the classical setting shown by [31]. This improvement over the classical fundamental limit is mostly attributed to the use of the QMC (Lemma 1), i.e., line 6 of Algo. 1. If the QMC procedure is replaced by classical MCestimation, then in contrast to the N ϵ s = O (1 /ϵ s ) queries by QMC, every stage would require N ϵ s = O (1 /ϵ 2 s ) queries. As a result, this would introduce an additional factor of 1 /ϵ s to the total regrets in a stage s (as discussed above), which would render the derivation of our regret upper bound (Theorem 4) invalid. This verifies that our tight regret upper bound in Theorem 4 is only achievable with the aid of quantum computing. Our Theorem 4 is a demonstration of the immense potential of quantum computing to dramatically improve the theoretical guarantees over the classical setting.

Importantly, when the linear kernel is used, the regret upper bound of our Q-GP-UCB is in fact better than that of the Q-LinUCB algorithm from [38]. This improvement arises from our tight confidence ellipsoid in Theorem 3. In Sec. 5.6, we will give a more detailed discussion of this improvement and adopt our proof technique for Theorem 3 to improve the confidence ellipsoid for Q-LinUCB, after which their improved regret upper bound matches that of our Q-GP-UCB with the linear kernel.

5.5 Regret Upper Bound for Noise with Bounded Variance

Here we analyze the regret of our Q-GP-UCB when the variance of the noise is bounded by σ 2 , which encompasses common scenarios including σ -sub-Gaussian noises and Gaussian noises with a variance of σ 2 . Consistent with the analysis of Q-LinUCB [38], in order to apply the theoretical guarantee provided by QMC (Lemma 1) for noise with bounded variance, here we need to assume that the noise variance is not too small, i.e., we assume that σ > 1 / 4 . In this case of noise with bounded variance, the following theorem gives an upper bound on the regret of our Q-GP-UCB.

Theorem 5. With probability of at least 1 -δ , we have that

The proof of Theorem 5 is given in App. G. Note that same as Theorem 4, the regret for the SE kernel in Theorem 5 is also of the order O ( poly log T ) , which is also significantly smaller than the classical regret lower bound of Ω( √ T ) [31]. Moreover, Theorem 5 shows that for both the linear kernel and SE kernel, the regret of our Q-GP-UCB is reduced when the noise variance σ 2 becomes smaller.

5.6 Improvement to Quantum Linear UCB (Q-LinUCB)

As we have mentioned in Sec. 5.4, the regret upper bound of our Q-GP-UCB with the linear kernel is R T = O ( d 3 / 2 (log T ) 3 / 2 log( d log T )) , which is better than the corresponding R T = O ( d 2 (log T ) 3 / 2 log( d log T )) for Q-LinUCB [38] 2 by a factor of O ( √ d ) . This improvement can be attributed to our tight confidence ellipsoid from Theorem 3. Here, following the general idea of the proof of our Theorem 3, we prove a tighter confidence ellipsoid for the Q-LinUCB algorithm, i.e., we improve Lemma 3 from the work of [38]. Here we follow the notations from [38], and we defer the detailed notations, as well as the proof of Theorem 6 below, to App. H due to space limitation.

Theorem 6 (Improved Confidence Ellipsoid for [38]) . With probability of at least 1 -δ ,

Note that the size of the confidence ellipsoid from our Theorem 6 is of the order O ( √ d log( ∑ s k =1 1 /ϵ 2 k )) = O ( √ d log T ) (we have used Lemma 2 here), which improves over the O ( √ ds ) = O ( √ dm ) = O ( √ d × d log T ) = O ( d √ log T ) from Lemma 3 of [38] by a factor of √ d . Plugging in our tighter confidence ellipsoid (Theorem 6) into the regret analysis of Q-LinUCB, the regret upper bound for Q-LinUCB is improved to R T = O ( d 3 / 2 log 3 / 2 T log( d log T ) ) (more details at the end of App. H). This improved regret upper bound exactly matches that of our Q-GPUCB with the linear kernel (Theorem 4). Similar to Theorem 3, the most crucial step in the proof of Theorem 6 is to apply the self-normalized concentration inequality from Theorem 1 of [2] to 1-sub-Gaussian noise . Again this is feasible because ϵ s = ˜ σ s -1 ( x s ) / √ λ is F s -1 -measurable, and conditioned on F s -1 , the scaled noise ζ s /ϵ s is 1 -sub-Gaussian.

5.7 Regret Upper Bound for the Matérn Kernel

So far we have mainly focused on the linear and SE kernels in our analysis. Here we derive a regret upper bound for our Q-GP-UCB algorithm for the Matérn kernel with smoothness parameter ν .

Theorem 7 (Matérn Kernel, Bounded Noise) . With probability of at least 1 -δ , we have that

The proof is in App. I. Note that the state-of-the-art regret upper bound for the Matérn kernel in the classical setting is R T = O ( √ Tγ T ) = ˜ O ( T ( ν + d ) / (2 ν + d ) ) [7, 24, 30, 36], which matches the corresponding classical regret lower bound (up to logarithmic factors in T ) [35]. Therefore, the regret upper bound of our Q-GP-UCB for the Matérn kernel (Theorem 7) improves over the corresponding classical regret lower bound when ν > 2 d , i.e., when the reward function is sufficiently smooth. Also note that similar to the classical GP-UCB [33] (with regret R T = O ( γ T √ T ) = ˜ O ( T ( ν +3 d/ 2) / (2 ν + d ) ) ) which requires the reward function to be sufficiently smooth (i.e., ν > d/ 2 ) to attain sub-linear regrets for the Matérn kernel, our Q-GP-UCB, as the first quantum BO algorithm, also requires the reward function to be smooth enough (i.e., ν > d ) in order to achieve a sub-linear regret upper bound for the Matérn kernel. We leave it to future works to further improve our regret upper bound and hence relax this requirement for smooth functions.

6 Experiments

We use the Qiskit python package to implement the QMC algorithm (Lemma 1) following the recent work of [20]. Some experimental details are deferred to App. J due to space limitation.

Synthetic Experiment. Here we use a grid of |X| = 20 equally spaced points within [0 , 1] as the 1 -dimensional input domain X ( d = 1 ), and sample a function f from a GP prior with the SE kernel. The sampled function f is scaled so that its output is bounded within [0 , 1] , and then used as the reward function in the synthetic experiments. We consider two types of noises: (a) bounded noise (within [0 , 1] ) and (b) Gaussian noise, which correspond to the two types of noises in Lemma 1, respectively. For (a) bounded noise, we follow the practice of [38] such that when an input x is selected, we treat the function value f ( x ) as the probability for a Bernoulli distribution, i.e., we observe an output of 1 with probability of f ( x ) and 0 otherwise. For (b) Gaussian noise, we simply add a zero-mean Gaussian noise with variance σ 2 to f ( x ) . The results for (a) bounded noise and (b) Gaussian noise are shown in Figs. 1 (a) and (b), respectively. The figures show that for both types of noises, our Q-GP-UCB significantly outperforms the classical baseline of GP-UCB. Specifically, although our Q-GP-UCB incurs larger regrets in the initial stage, it is able to leverage the accurate observations provided by the QMC subroutine to rapidly find the global optimum. These results show that the quantum speedup of our Q-GP-UCB in terms of the tighter regret upper bounds (Theorems 4 and 5) may also be relevant in practice. We have additionally compared with linear UCB (LinUCB) [2] and Q-LinUCB [38], which are, respectively, the most representative classical and quantum linear bandit algorithms. The results in Fig. 2 (App. J) show that in these experiments where the reward

0

2000

4000

6000

8000

10000

Iterations

0

20

40

60

Cumulative Regret

GP-UCB

Q-GP-UCB

0

2000

4000

6000

8000

10000

Iterations

0

20

40

60

Cumulative Regret

GP-UCB (

=0.3)

GP-UCB (

=0.4)

Q-GP-UCB (

=0.3)

Q-GP-UCB (

=0.4)

0

1000

2000

3000

4000

5000

Iterations

0

10

20

30

40

Cumulative Regret

GP-UCB (

=0.01)

GP-UCB (

=0.05)

Q-GP-UCB (

=0.01)

Q-GP-UCB (

=0.05)

0

1000

2000

3000

4000

5000

Iterations

0

100

200

300

400

Cumulative Regret

GP-UCB (

=0.01)

GP-UCB (

=0.05)

Q-GP-UCB (

=0.01)

Q-GP-UCB (

=0.05)

LinUCB (

=0.05)

=0.05)

(a) synthetic (Bernoulli) (b) synthetic (Gaussian)

(c) AutoML

(d) AutoML

function is non-linear, algorithms based on linear bandits severely underperform compared with BO/kernelized bandit algorithms in both the classical and quantum settings.

AutoML Experiment. In our experiments on automated machine learning (AutoML), we use our Q-GP-UCB algorithm to tune the hyperparameters of an SVM for a classification task. Here we consider a Gaussian noise with a variance of σ 2 since it is more practical and more commonly used in real-world experiments. The results in Fig. 1 (c) show that our Q-GP-UCB again significantly outperforms the classical GP-UCB. Fig. 1 (d) additionally shows the comparisons with LinUCB and Q-LinUCB. The results again corroborate that BO/kernelized bandit algorithms are considerably superior in real-world applications with highly non-linear reward functions in both the classical and quantum settings. These results here further demonstrate the potential of our Q-GP-UCB to lead to quantum speedup in practical applications.

More Realistic Experiments. We have additionally tested the performance of our Q-GP-UCB algorithm after accounting for the effect of quantum noise, by incorporating into our Qiskit simulations a noise model based on the actual performance of real IBM quantum computers. The results (Fig. 3 in App. J) show that although the addition of quantum noise slightly deteriorates the performance of our Q-GP-UCB, it is still able to significantly outperform classical GP-UCB. Notably, we have additionally performed an experiment using a real quantum computer (details in App. J), in which the performance of our Q-GP-UCB, although further worsened compared to the experiment using simulated noise, is still considerably better than classical GP-UCB (Fig. 4, App. J). Also note that the work of [38] has not shown that Q-LinUCB outperforms LinUCB in the presence of simulated quantum noise. Therefore, the consistently superior performances of our Q-GP-UCB over GP-UCB with both simulated quantum noise and a real quantum computer serve as important new support for the potential practical advantages of quantum bandit algorithms.

Discussion. In our experimental results (e.g., Fig. 1), our Q-GP-UCB usually has relatively larger regrets in the initial stages but quickly achieves zero regret thereafter (i.e., the curve plateaus), which has also been observed in [38] for the Q-LinUCB algorithm. This is because in the initial stages, our Q-GP-UCB explores a smaller number of unique arms than GP-UCB. However, after the initial exploration, our Q-GP-UCB quickly starts to perform reliable exploitation, because the accurate reward observations achieved thanks to our QMC subroutines allow us to rapidly learn the reward function and hence find the optimal arm.

7 Conclusion

We have introduced the first quantum BO algorithm, named Q-GP-UCB. Our Q-GP-UCB achieves a regret upper bound of O ( poly log T ) , which is significantly smaller than the classical regret lower bound of Ω( √ T ) . A limitation of our work is that the regret upper bound of our Q-GP-UCB: O ((log T ) 3( d +1) / 2 ) (ignoring polylog factors) has a worse dependency on the input dimension d than the regret of the classical GP-UCB: O ((log T ) d +1 √ T ) . A similar limitation is shared by Q-LinUCB, since its regret has a dependency of O ( d 3 / 2 ) in contrast to the O ( d ) of LinUCB. It is an interesting future work to explore potential approaches to remove this extra dependency on d . Another limitation of our work is that for the Matérn kernel, we require the reward function to be smooth enough in order to achieve a sub-linear regret upper bound (Sec. 5.7). So, another important future work is to further tighten the regret upper bound of our Q-GP-UCB for the Matérn kernel when the reward function is less smooth, as well as for other kernels encompassing non-smooth functions such as the neural tangent kernel which has been adopted by recent works on neural bandits [13, 42, 44]. Moreover, another interesting future work is to derive regret lower bounds in our setting of quantum kernelized bandits, which can help evaluate the tightness of our regret upper bounds.

References

A. Abbas, D. Sutter, C. Zoufal, A. Lucchi, A. Figalli, and S. Woerner. The power of quantum neural networks. Nature Computational Science , 1(6):403-409, 2021.

Y. Abbasi-Yadkori, D. Pál, and C. Szepesvári. Improved algorithms for linear stochastic bandits. In Proc. NeurIPS , volume 24, 2011.

J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd. Quantum machine learning. Nature , 549(7671):195-202, 2017.

G. Brassard, P. Hoyer, M. Mosca, and A. Tapp. Quantum amplitude amplification and estimation. Contemporary Mathematics , 305:53-74, 2002.

S. Cakmak, R. Astudillo Marban, P. Frazier, and E. Zhou. Bayesian optimization of risk measures. In Proc. NeurIPS , volume 33, pages 20130-20141, 2020.

D. Calandriello, L. Carratino, A. Lazaric, M. Valko, and L. Rosasco. Scaling Gaussian process optimization by evaluating a few unique candidates multiple times. In Proc. ICML , pages 2523-2541. PMLR, 2022.

R. Camilleri, K. Jamieson, and J. Katz-Samuels. High-dimensional experimental design and kernel bandits. In Proc. ICML , pages 1227-1237. PMLR, 2021.

B. Casalé, G. Di Molfetta, H. Kadri, and L. Ralaivola. Quantum bandits. Quantum Machine Intelligence , 2:1-7, 2020.

S. R. Chowdhury and A. Gopalan. On kernelized multi-armed bandits. In Proc. ICML , pages 844-853, 2017.

Z. Dai, B. K. H. Low, and P. Jaillet. Federated Bayesian optimization via Thompson sampling. In Proc. NeurIPS , 2020.

Z. Dai, B. K. H. Low, and P. Jaillet. Differentially private federated Bayesian optimization with distributed exploration. In Proc. NeurIPS , volume 34, 2021.

Z. Dai, Y. Shu, B. K. H. Low, and P. Jaillet. Sample-then-optimize batch neural Thompson sampling. In Proc. NeurIPS , 2022.

Z. Dai, Y. Shu, A. Verma, F. X. Fan, B. K. H. Low, and P. Jaillet. Federated neural bandits. In Proc. ICLR , 2023.

Z. Dai, H. Yu, B. K. H. Low, and P. Jaillet. Bayesian optimization meets Bayesian optimal stopping. In Proc. ICML , pages 1496-1506, 2019.

Y. Deng, X. Zhou, B. Kim, A. Tewari, A. Gupta, and N. Shroff. Weighted Gaussian process bandits for non-stationary environments. In Proc. AISTATS , pages 6909-6932. PMLR, 2022.

P. I. Frazier. A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811, 2018.

B. Ganguly, Y. Wu, D. Wang, and V. Aggarwal. Quantum computing provides exponential regret improvement in episodic reinforcement learning. arXiv:2302.08617, 2023.

S. Garg and G. Ramakrishnan. Advances in quantum deep learning: An overview. arXiv preprint arXiv:2005.04316, 2020.

R. Garnett. Bayesian Optimization . Cambridge Univ. Press, 2022.

D. Grinko, J. Gacon, C. Zoufal, and S. Woerner. Iterative quantum amplitude estimation. npj Quantum Information , 7(1):52, 2021.

L. K. Grover. A framework for fast quantum mechanical algorithms. In Proceedings of the thirtieth annual ACM symposium on Theory of computing , pages 53-62, 1998.

F. Hutter, L. Kotthoff, and J. Vanschoren. Automated machine learning: methods, systems, challenges . Springer Nature, 2019.

T. Li and R. Zhang. Quantum speedups of optimizing approximately convex functions with applications to logarithmic regret stochastic convex bandits. In Proc. NeurIPS , 2022.

Z. Li and J. Scarlett. Gaussian process bandit optimization with few batches. In Proc. AISTATS , pages 92-107. PMLR, 2022.

J. Lumbreras, E. Haapasalo, and M. Tomamichel. Multi-armed quantum bandits: Exploration versus exploitation when learning properties of quantum states. Quantum , 6:749, 2022.

A. Messiah. Quantum mechanics . Courier Corporation, 2014.

N. Meyer, C. Ufrecht, M. Periyasamy, D. D. Scherer, A. Plinge, and C. Mutschler. A survey on quantum reinforcement learning. arXiv:2211.03464, 2022.

A. Montanaro. Quantum speedup of Monte Carlo methods. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , 471(2181):20150301, 2015.

P. Rebentrost, M. Mohseni, and S. Lloyd. Quantum support vector machine for big data classification. Physical review letters , 113(13):130503, 2014.

S. Salgia, S. Vakili, and Q. Zhao. A domain-shrinking based Bayesian optimization algorithm with order-optimal regret performance. In Proc. NeurIPS , volume 34, pages 28836-28847, 2021.

J. Scarlett, I. Bogunovic, and V. Cevher. Lower bounds on regret for noisy Gaussian process bandit optimization. In Proc. COLT , pages 1723-1742. PMLR, 2017.

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE , 104(1):148-175, 2016.

N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proc. ICML , pages 1015-1022, 2010.

A. Steane. Quantum computing. Reports on Progress in Physics , 61(2):117, 1998.

S. Vakili, K. Khezeli, and V. Picheny. On information gain and regret bounds in Gaussian process bandits. In Proc. AISTATS , pages 82-90. PMLR, 2021.

M. Valko, N. Korda, R. Munos, I. Flaounas, and N. Cristianini. Finite-time analysis of kernelised contextual bandits. In Proc. UAI , 2013.

A. Verma, Z. Dai, and B. K. H. Low. Bayesian optimization under stochastic delayed feedback. In Proc. ICML , pages 22145-22167. PMLR, 2022.

Z. Wan, Z. Zhang, T. Li, J. Zhang, and X. Sun. Quantum multi-armed bandits and stochastic linear bandits enjoy logarithmic regrets. In Proc. AAAI .

D. Wang, A. Sundaram, R. Kothari, A. Kapoor, and M. Roetteler. Quantum algorithms for reinforcement learning with a generative model. In Proc. ICML , pages 10916-10926. PMLR, 2021.

D. Wang, X. You, T. Li, and A. M. Childs. Quantum exploration algorithms for multi-armed bandits. In Proc. AAAI , volume 35, pages 10102-10110, 2021.

Y. Wu, C. Guan, V. Aggarwal, and D. Wang. Quantum heavy-tailed bandits. arXiv:2301.09680, 2023.

W. Zhang, D. Zhou, L. Li, and Q. Gu. Neural Thompson sampling. In Proc. ICLR , 2021.

H. Zhong, J. Hu, Y. Xue, T. Li, and L. Wang. Provably efficient exploration in quantum reinforcement learning with logarithmic worst-case regret. arXiv:2302.10796, 2023.

D. Zhou, L. Li, and Q. Gu. Neural contextual bandits with UCB-based exploration. In Proc. ICML , pages 11492-11502. PMLR, 2020.

A Analysis of the Weighted GP Posterior (3)

Here we show an alternative view of the weighted GP posterior distribution (3) in the RKHS feature space. The notations in this section will be extensively used in the theoretical analyses in the subsequent sections.

We denote the kernel k by k ( x, x ′ ) = ϕ ( x ) ⊤ ϕ ( x ′ ) , in which ϕ ( x ) is the potentially infinitedimensional feature mapping of x in the RKHS of k . Define Φ s ≜ [ ϕ ( x 1 ) , . . . , ϕ ( x s )] ⊤ which is an s ×∞ matrix. Recall that we have defined W s ≜ diag ( 1 ϵ 2 1 , . . . , 1 ϵ 2 s ) which is an s × s diagonal matrix. Define V s ≜ λI + ∑ s τ =1 1 ϵ 2 τ ϕ ( x τ ) ϕ ( x τ ) ⊤ = λI +Φ ⊤ s W s Φ s which is an ∞×∞ matrix, and let V 0 ≜ λI . With these notations, it can be easily verified that K s = Φ s Φ ⊤ s = [ k ( x τ , x τ ′ )] τ,τ ′ =1 ,...,s , and ˜ K s = W 1 / 2 s Φ s Φ ⊤ s W 1 / 2 s = W 1 / 2 s K s W 1 / 2 s . Moreover, we can also easily show that k s ( x ) = Φ s ϕ ( x ) = [ k ( x, x τ )] τ =1 ,...,s , ˜ k s ( x ) = W 1 / 2 s Φ s ϕ ( x ) = W 1 / 2 s k s ( x ) , and ˜ Y s = W 1 / 2 s Y s .

We start by deriving an alternative expression for the weighted GP posterior mean ˜ µ s ( x ) . To begin with, we have that

Now we multiply both sides by V -1 s on the left and by ( ˜ K s + λI ) -1 on the right, which leads to V -1 s Φ ⊤ s W 1 / 2 s = Φ ⊤ s W 1 / 2 s ( ˜ K s + λI ) -1

Now we can show that

Next, we derive an alternative expression for the weighted GP posterior variance ˜ σ 2 s ( x ) . To begin with, we have that

This allows us to show that

in which we have used the matrix inversion lemma in step ( a ) , and used equation (5) in step ( b ) .

To summarize, we have derived the following alternative expressions for the weighted GP posterior distribution (3):

B Proof of Lemma 2

To begin with, we can lower-bound the total number of iterations (i.e., the total number of queries to the quantum oracles) in m stages by

The first inequality follows since C 1 > 1 (Lemma 1) and log( 2 m δ ) ≥ log( 2 δ ) ≥ 1 because we have chosen δ ∈ (0 , 2 /e ] . Suppose that ∑ m s =1 1 ϵ 2 s > T 2 , then we have that ∑ m s =1 C 1 ϵ s log( 2 m δ ) > T which is a contradiction. Therefore, we have that

C Proof of Theorem 1

Here we prove the growth rate of ˜ γ m , i.e., the weighted information gain. Recall that W m = diag ( 1 ϵ 2 1 , 1 ϵ 2 2 , . . . , 1 ϵ 2 m ) , and ˜ γ m = 1 2 log det ( I + λ -1 ˜ K m ) . Our proof here follows closely the proof of Theorem 3 from the work of [35]. We will omit the subscript m as long as the context is clear.

Denote by K m = Φ m Φ ⊤ m = [ k ( x i , x j )] i,j =1 ,...,m the m × m covariance matrix, and define ˜ K m = W 1 / 2 m Φ m Φ ⊤ m W 1 / 2 m = W 1 / 2 m K m W 1 / 2 m . Following Theorem 1 from [35], our kernel k can be denoted as k ( x, x ′ ) = ∑ ∞ τ =1 λ τ ψ τ ( x ) ψ τ ( x ′ ) , in which { λ τ } τ =1 ,..., ∞ and { ψ τ } τ =1 ,..., ∞ represent, respectively, the eigenvalues and eigenfunctions of the kernel k . We will make use of the projection onto a D -dimensional RKHS ( D < ∞ ), which consists of the first D features corresponding to the D largest eigenvalues of the kernel k . Specifically, define k P as the D -dimensional projection in the RKHS of k , and k O corresponds to the orthogonal element such that k ( · , · ) = k P ( · , · ) + k O ( · , · ) . That is, following Section 3.2 from the work of [35], define

We define K P ≜ [ k P ( x i , x j )] i,j =1 ,...,m and K O ≜ [ k O ( x i , x j )] i,j =1 ,...,m . Similarly, define ˜ K P = W 1 / 2 m K P W 1 / 2 m and ˜ K O = W 1 / 2 m K O W 1 / 2 m . This implies that K m = K P + K O and that ˜ K m = ˜ K P + ˜ K O .

Here to be consistent with [35], we use I m to denote the m × m -dimensional identity matrix. Denote by ˜ I ( y m ; f ) the information gain about the function f from the observations y m = [ y 1 , . . . , y m ] . We have that

The last line follows because det ( AB ) = det( A ) det ( B ) . In the following, we will separately upper-bound the two terms in (10).

We start by upper-bounding the first term in equation (10). Denote by ψ D ( x ) the D -dimensional feature vector corresponding to the first D principle features of the kernel k (i.e., the features corresponding to the largest D eigenvalues). Define Ψ m,D ≜ [ ψ D ( x 1 ) , . . . , ψ D ( x m )] ⊤ which is an m × D matrix. Denote by Λ D = diag ( λ 1 , . . . , λ D ) a D × D -diagonal matrix whose diagonal entries consist of the largest D eigenvalues of the kernel k in descending order. With this definition, we have that K P = Ψ m,D Λ D Ψ ⊤ m,D and

Based on these definitions, we have that

which follows from the Weinstein-Aronszajn identity: det ( I D + AA ⊤ ) = det ( I m + A ⊤ A ) for a D × m matrix A , and we have plugged in A = 1 √ λ Λ 1 / 2 D Ψ ⊤ m,D W 1 / 2 m . In addition, the following inequality will also be useful in the subsequent proof:

in which we have made use of the definition of k P from (9), and our assumption (w.l.o.g.) that k ( x, x ′ ) ≤ 1 , ∀ x, x ′ (Sec. 3.1). In the following, we will denote the trace of a matrix A by Tr ( A ) . Next, we have that

in which ( a ) follows because Ψ ⊤ m,D W m Ψ m,D = ∑ m s =1 1 ϵ 2 s ψ D ( x s ) ψ ⊤ D ( x s ) , ( b ) has made use of the cyclic property of the trace operator, and ( c ) follows from (14) above.

Also define the gram matrix:

Also note that for a positive definite matrix P ∈ R n × n , we have that

Therefore, we have that

The first equality follows from (13), and the inequality results from (16) and (15).

Now we upper-bound the second term in equation (10). To begin with, we have that

which is because the matrix ( I m + 1 λ ˜ K P ) -1 is positive semi-definite (PSD) whose largest eigenvalue is upper-bounded by 1 , and Tr ( P 1 P 2 ) ≤ λ P 1 Tr ( P 2 ) where P 1 and P 2 are PSD matrices and λ P 1 is the largest eigenvalue of P 1 . Next, define δ D ≜ ∑ ∞ τ = D +1 λ τ ψ 2 where | ϕ τ ( x ) | ≤ ψ, ∀ x, τ . Then we immediately have that k O ( x, x ′ ) = ∑ ∞ τ = D +1 λ τ ϕ τ ( x ) ϕ τ ( x ′ ) ≤ δ D , ∀ x, x ′ . Therefore, we have that

As a result, we have that

in which the inequality follows from (18) and (19).

Next, again making use of equation (16) and incorporating the upper bound on the trace from (20), we have that

in which the second last step is because log(1 + z ) ≤ z, ∀ z ∈ R .

Finally, combining the upper bounds from 17 and 21, equation (10) can be upper-bounded by

Therefore, we have that

Also recall that for the standard unweighted maximum information gain γ T , we have that [35]

Therefore, the upper bound on the weighted information gain ˜ γ m is obtained by replacing the T from the upper bound on standard maximum information gain γ T by ∑ m s =1 1 ϵ 2 s . This is formalized by the following Theorem.

Theorem 8. [More formal statement of Theorem 1 (a)] Given X m ≜ { x 1 , . . . , x m } , then we have that

Lastly, note that it has been shown that γ T = O ( d log T ) for the linear kernel and γ T = O ((log T ) d +1 ) for the SE kernel, Therefore, plugging in Lemma 2, which allows us to upper-bound ∑ m s =1 1 /ϵ 2 s by T 2 , leads to Corollary 1.

D Proof of Theorem 2

Our proof here follows the outline of the proof of Lemma 2 from the work of [38]. To begin with, we show that det ( V τ +1 ) = 2 det ( V τ ) , ∀ τ = 0 , . . . , m -1 , i.e., the amount of collected information is doubled after every stage.

Step ( a ) has made use of the matrix determinant lemma: det( I + ab ⊤ ) = det( I + a ⊤ b ) , and ( b ) follows from the expression of the weighted GP posterior variance (6) and our choice of ϵ τ +1 =

˜ σ τ ( x τ +1 ) / √ λ (line 3 of Algo. 1). An immediate consequence of (25) is that det ( V m ) = 2 m det ( V 0 ) . Recall we have defined earlier that V 0 = λI (App. A).

Recall that following the notations of App. A, we have that V m = λI +Φ ⊤ m W m Φ m . Next, we can also show that

in which step ( a ) has again made use of the Weinstein-Aronszajn identity (similar to (13)), and the last two equalities have simply plugged in the expressions for ˜ K s = W 1 / 2 s Φ s Φ ⊤ s W 1 / 2 s (App. A), and ˜ γ m = 1 2 log det( I + λ -1 ˜ K m ) (Sec. 5.1).

According to Corollary 1, for the linear kernel, we have that there exists an absolute constant C > 0 such that ˜ γ m ≤ Cd log ( ∑ m s =1 1 ϵ 2 s ) . Combining equations (25) and (26), we can show that

This allows us to show that

Note that in every stage s , we query the quantum oracle for C 1 ϵ s log( 2 m δ ) times, in which C 1 > 1 , δ ∈ (0 , 2 /e ] and m is an upper bound on the total number of stages. An immediate consequence is that log( 2 m δ ) ≥ log( 2 δ ) ≥ 1 . Therefore, the total number of iterations can be analyzed as

Now we derive an upper bound on m by contradiction. Suppose m> 2 Cd log 2 log T , this immediately implies that

This equation, combined with (29), implies that ∑ m s =1 C 1 ϵ s log( 2 m δ ) > T , which is a contradiction. Therefore, we have that

That is, for the linear kernel, our algorithm runs for at most O ( d log T ) stages, which matches the upper bound on the total number of stages for the quantum linear UCB (Q-LinUCB) algorithm, as proved by Lemma 2 from the work of [38].

Next, for the SE kernel, Corollary 1 shows that there exists an absolute constant C > 0 such that ˜ γ m ≤ C log d +1 ( ∑ m s =1 1 ϵ 2 s ) . Again combining equations (25) and (26), we can show that

This allows us to show that

Now again we derive an upper bound on m by contradiction. Suppose m> 2 C log 2 (2 log T ) d +1 , then this implies that

This, combined with (29), implies that ∑ m s =1 C 1 ϵ s log( 2 m δ ) > T which is a contradiction. Therefore

for the SE kernel. This completes the proof.

E Proof of Theorem 3

Our proof here follows closely the proof of Theorem 2 from [9]. Denote by ζ s the observation noise for the observation in stage s : y s = f ( x s ) + ζ s . Define ζ 1: s ≜ [ ζ 1 , . . . , ζ s ] ⊤ which is an s × 1 vector. Denote by F s the σ -algebra: F s = { x 1 , . . . , x s +1 , ζ 1 , . . . , ζ s } . With this definition, we have that x s is F s -1 -measurable, and ζ s is F s -measurable. Note that we have used ϕ ( · ) to denote the RKHS feature map of the kernel k : k ( x, x ′ ) = ϕ ( x ) ⊤ ϕ ( x ′ ) , and here we let ϕ ( x ) = k ( x, · ) . Then using the reproducing property, we have that f ( x ) = ⟨ ϕ ( x ) , f ⟩ k = ϕ ( x ) ⊤ f , in which the inner product denotes the inner product induced by the RKHS of k .

To begin with, we have that

which follows from noting that (Φ ⊤ s W s Φ s + λI )Φ ⊤ s W 1 / 2 s = Φ ⊤ s W 1 / 2 s ( W 1 / 2 s Φ s Φ ⊤ s W 1 / 2 s + λI ) , and then multiplying by (Φ ⊤ s W s Φ s + λI ) -1 on the left and by ( W 1 / 2 s Φ s Φ ⊤ s W 1 / 2 s + λI ) -1 on the right.

Now let f s ≜ [ f ( x k )] ⊤ k =1 ,...,s = Φ s f which is an s × 1 -dimensional vector, and ˜ f s ≜ W 1 / 2 s f s = W 1 / 2 s Φ s f . Next, we can prove that

in which step ( a ) has made use of (36), and step ( b ) follows from our assumption that ∥ f ∥ k ≤ B (Sec. 3.1) and the expression for the weighted GP posterior variance (6). Next, we have that

in which ( a ) and ( b ) follow from (36). Since ˜ µ s ( x ) = ˜ k ⊤ s ( x )( ˜ K s + λI ) -1 ˜ Y s and ˜ Y s = ˜ f s + W 1 / 2 s ζ 1: s , then the two equations above combine to tell us that

in which the second inequality follows since λ = 1 + η > 1 . Again following [9], we apply a few more steps of transformations. Note that for an invertible K , we have that K ( K + I ) -1 = ( ( K + I ) K -1 ) -1 = ( I + K -1 ) -1 . Substituting K = ˜ K s + ηI , we have that

Next, we have that

This allows us to further re-write (39) as

Recall that W s = diag ( 1 ϵ 2 1 , 1 ϵ 2 2 , . . . , 1 ϵ 2 s ) , therefore, W 1 / 2 s ζ 1: s = [ ζ k 1 ϵ k ] ⊤ k =1 ,...,s . Of note, since we have that | f ( x s ) -y s | ≤ ϵ s , ∀ s = 1 , . . . , m (with high probability, refer to the beginning of Sec. 5), so, the noise ζ s = y s -f ( x s ) is bounded: | ζ s | ≤ ϵ s , ∀ s = 1 , . . . , m . In other words, ζ s is ϵ s -sub-Gaussian.

To begin with, note that ϵ k is F k -1 -measurable. This can be seen recursively: conditioned on { x 1 } , ϵ 1 = ˜ σ 0 ( x 1 ) / √ λ is a deterministic constant (i.e., predictable); conditioned on { x 1 , x 2 } , ˜ σ 1 ( · ) is predictable because it depends on x 1 and ϵ 1 (via the weight 1 ϵ 2 1 ), and hence ϵ 2 = ˜ σ 1 ( x 2 ) / √ λ is predictable conditioned on { x 1 , x 2 } . By induction, conditioned on { x 1 , . . . , x k } , ϵ k = ˜ σ k -1 ( x k ) / ( √ λ ) is predictable. Therefore, ϵ k is F k -1 -measurable, because F k -1 = { x 1 , . . . , x k , ζ 1 , . . . , ζ k -1 } .

Next, note that since ζ k is F k -measurable, we have that ζ k ϵ k is F k -measurable. Moreover, conditioned on F k -1 (i.e., ϵ k is a predictable), ζ k ϵ k is 1 -sub-Gaussian, because ζ k is ϵ k -sub-Gaussian as discussed above. To summarize, we have that every element ζ k ϵ k of this noise vector W 1 / 2 s ζ 1: s is (a) F k -measurable and (b) 1-sub-Gaussian conditionally on F k -1 . Therefore, we can apply Theorem 1 of [9] to 1-sub-Gaussian noise, to show that

with probability of at least 1 -δ . When applying Theorem 1 of [9], we have also replaced the original unweighted covariance matrix K s by the corresponding weighted covariance matrix ˜ K s = W 1 / 2 s K s W 1 / 2 s . This is possible because every ϵ k is F k -1 -measurable. More concretely, in the proof of Theorem 1 of [9], when defining the super-martingale { M t } t which is conditioned on F ∞ , we only need to replace the covariance matrix K t (for the multivariate Gaussian distribution of h ( x 1 ) , . . . , h ( x t ) ) by ˜ K t = W 1 / 2 t K t W 1 / 2 t , because both the original K t and our ˜ K t only require conditioning on { x 1 , . . . , x t } .

Combining the two equations above allows us to show that

with probability of ≥ 1 -δ . Next, we can further upper-bound the log determinant term:

in which ( a ) follows since det( AB ) = det( A ) det( B ) and det( cA ) ≤ c s det( A ) for a scalar c and an s × s -matrix A , ( b ) has made use of log(1 + z ) ≤ z , ( c ) has made use of the definition of γ s , and ( d ) follows because η = 2 /T and s ≤ T . This eventually allows us to show that

Of note, although the work of [15] has also adopted weighted GP regression in a similar way to our (3), our proof technique here cannot be applied in their analysis. This is because the work of [15] has chosen the weight to be w s = η -s for η ∈ (0 , 1) and hence W s = diag [ w 1 , . . . , w s ] . Therefore, every element of the scaled noise vector W 1 / 2 s ζ 1: s (i.e., ζ k √ w k ) is not guaranteed to be sub-Gaussian. As a result, this makes it infeasible for them to apply the self-normalizing concentration inequality from Theorem 1 of [9], i.e., our (41) above does not hold in their case. The work of [15] has come up with other novel proof techniques suited to their setting. Therefore, our proof here only works in our problem setting of quantum BO.

F Proof of Theorem 4

To begin with, we can show that

in which ( a ) and ( c ) follow from Theorem 3, ( b ) results from our policy for selecting x s (line 2 of Algo. 1), and ( d ) follows from our design of ϵ s = ˜ σ s -1 ( x s ) / √ λ (line 3 of Algo. 1). As a result, the total regret in stage s can be upper bounded by:

Then an upper bound on the total cumulative regret can be obtained by summing up the regrets from all m stages:

For the linear kernel, recall from Theorem 3 and Corollary 1 that β m = O ( √ ˜ γ m -1 +log(1 /δ )) = O ( √ d log T +log(1 /δ )) . Also recall that Theorem 2 tells us that m ≤ m = O ( d log T ) . Therefore, for the linear kernel,

in which we have ignored the dependency on log(1 /δ )

in the second step to get a cleaner expression.

For the SE kernel, recall from Theorem 3 and Corollary 1 that β m = O ( √ ˜ γ m -1 +log(1 /δ )) = O ( √ (log T ) d +1 +log(1 /δ )) . Also recall that Theorem 2 tells us that m ≤ m = O ((log T ) d +1 ) . Therefore, for the linear kernel,

in which we have again ignored the dependency on log(1 /δ ) to obtain a cleaner expression.

The error probability of δ results from (a) conditioning on the event that | f ( x s ) -y s | ≤ ϵ s , ∀ s = 1 , . . . , m which holds with probability of at least 1 -δ/ 2 , and (b) Theorem 3 which also holds with probability of at least 1 -δ/ 2 .

G Proof of Theorem 5

Consistent with the analysis of Q-LinUCB [38], in order to apply the theoretical guarantee provided by QMC (Lemma 1) for noise with bounded variance, here we need to assume that the noise variance is not too small: We assume that σ > 1 / 3 . We have chosen the value of 1 / 3 just for convenience, in fact, any σ larger than 1 / 4 can be used in our analysis here because these different values will only alter the constants in our regret upper bound which are absorbed by the O .

Note that ϵ s = ˜ σ s -1 ( x s ) / √ λ ≤ 1 , which follows because ˜ σ 2 s -1 ( x s ) ≤ 1 (which can be easily seen from (3) and our assumption that k ( x, x ′ ) ≤ 1 ) and λ > 1 . As a result of the assumption of σ > 1 / 3 , we have that σ > 1 / 3 > 2 √ 2 / 8 . This, combined with ϵ s ≤ 1 , allows us to show that log 3 / 2 2 ( 8 σ ϵ s ) ≥ 2 3 / 4 and that log 2 (log 2 8 σ ϵ s ) ≥ 1 / 2 .

Next, we derive an upper bound on ∑ m s =1 1 ϵ 2 s which is analogous to (8). Note that in the case of a noise with bounded variance, the number of queries to the quantum oracle in stage s is given by N ϵ s = C 2 σ ϵ s log 3 / 2 2 ( 8 σ ϵ s ) log 2 (log 2 8 σ ϵ s ) log 2 m δ with C 2 > 1 . As a result, we have the following inequality which is in a similar spirit to (7)

Now suppose that ∑ m s =1 1 ϵ 2 s > 9 √ 2 T 2 , then we have that ∑ m s =1 N ϵ s > T which is a contradiction. Therefore, we have that

Next, note that Theorem 1 is unaffected since its proof does not depend on the noise. Moreover, Corollary 1 also stays the same because it has only made use of (8) which gives an upper bound on ∑ m s =1 1 /ϵ 2 s , and compared to (8), its counterpart (51) in the analysis here only introduces an extra constant factor which is absorbed by the bit O notation. Similarly, Theorem 2 is also unaltered for a similar reason, i.e., only an extra constant factor of 3 × 2 1 / 4 will be introduced to the right hand side of (30) and (34), which is absorbed by the big O notation.

Theorem 3 is also unaltered since its proof does not depend on the type of noise. In particular, the key underlying reason why it is unaffected is because for both types of noise, given a particular ϵ s , we choose the corresponding number N ϵ s of queries (to the quantum oracle) depending on the type of noise so that the error guarantee of | y s -f ( x s ) | ≤ ϵ s is satisfied. This allows us to make use of the self-normalizing bound from [9] for 1 -sub-Gaussian noise.

Lastly, we need to modify the proof of Theorem 4. To begin with, the total regret from stage s can now be bounded as

Note that every 1 /ϵ s is upper-bounded by 1 /ϵ s ≤ 3 × 2 1 / 4 T = O ( T ) which can be easily inferred using (51). So, the total regret can be upper-bounded as

As a result, for the linear kernel for which m ≤ m = O ( d log T ) (Theorem 2) and β m = √ ˜ γ m -1 +log(1 /δ ) = O ( √ d log T +log(1 /δ )) (Theorem 3 and Corollary 1),

in which we have ignored all log(1 /δ ) factors for a cleaner expression.

in which we have again ignored all log(1 /δ ) factors for a cleaner expression.

H Proof of Theorem 6

Our proof here follows closely the proof of Theorem 2 from the work of [2]. For consistency, in our proof here, we follow the notations from [38] and [2].

Since we focus on linear bandits here, we assume that the reward function is linear with a groundtruth d -dimensional parameter vector θ ∗ : f ( x ) = x ⊤ θ ∗ , ∀ x . Again following [2] and [38] as well as many other works on linear bandits, we assume that ∥ θ ∗ ∥ 2 ≤ S , and ∥ x ∥ 2 ≤ L, ∀ x . We use X s to denote X s ≜ [ x 1 , . . . , x s ] ⊤ which is an s × d matrix, and denote Y s ≜ [ y 1 , . . . , y s ] ⊤ which is an s × 1 -dimensional vector of observations. We use ζ 1: s ≜ [ ζ 1 , . . . , ζ s ] ⊤ to denote the s × 1 -dimensional vector of noise. These notations allow us to write Y s = X s θ ∗ + ζ 1: s . We denote V s ≜ X ⊤ s W s X s + λI . Following [38], we use ̂ θ s to denote the MLE estimate of the parameter θ ∗ given the observations after the first s stages: ̂ θ s = V -1 s X ⊤ s W s Y s . Given these notations, we firstly have that

Therefore,

This allows us to use the Cauchy-Schwarz inequality to show that

The last inequality follows because ∥ θ ∗ ∥ 2 V -1 s ≤ 1 λ min ( V s ) ∥ θ ∗ ∥ 2 2 ≤ 1 λ ∥ θ ∗ ∥ 2 2 . Note that X ⊤ s W s ζ 1: s = ∑ s k =1 1 ϵ 2 k ζ k x k .

Next, we make use of the self-normalized concentration inequality from Theorem 1 of [2]. Specifically, we will use { ζ s ϵ s } ∞ s =1 as to replace the stochastic process { ζ s } ∞ s =1 (i.e., representing the noise, { η s } ∞ s =1 according to the notations from [2]), and use { 1 ϵ s x s } ∞ s =1 to replace the vector-valued stochastic process { x s } ∞ s =1 (i.e., representing the sequence of inputs). Now we explain why this is feasible.

Similar to the proof of Theorem 3, we again denote the observation noise as ζ s : y s = f ( x s ) + ζ s , and define F s as the σ -algebra F s = σ ( x 1 , x 2 , . . . , x s +1 , ζ 1 , ζ 2 , . . . , ζ s ) . Similarly, an immediate consequence is that x s is F s -1 -measurable, and ζ s is F s -measurable. Following a similar analysis to that used in the proof of Theorem 3 allows us to show that ϵ k is F k -1 -measurable (i.e., ϵ k can be predicted based on F k -1 ). Therefore, we have that 1 ϵ k x k is F k -1 -measurable (i.e., 1 ϵ k x k can be predicted based on F k -1 ) and that ζ k ϵ k is F k -measurable. Moreover, conditioned on F k -1 , ζ k ϵ k is 1 -subGaussian. This is because the QMC subroutine guarantees that | y k -f ( x k ) | ≤ ϵ k , ∀ k = 1 , . . . , m (with high probability), which means that the absolute value of the noise ζ k = y k -f ( x k ) is upper-bounded by ϵ k : | ζ k | ≤ ϵ k . Therefore, conditioned on F k -1 , ζ k ϵ k is 1 -sub-Gaussian.

Given that these conditions are satisfied, we can apply the self-normalized concentration inequality from Theorem 1 of [2] to { ζ s ϵ s } ∞ s =1 and { 1 ϵ s x s } ∞ s =1 with (following the notations from Theorem 1 of [2])

in which we have used V = λI . This allows us to show that

which holds with probability of at least 1 -δ . Next, again making use of the trace-determinant inequality from (16) and using the aforementioned assumption that ∥ x ∥ 2 ≤ L , we can show that det( V s ) ≤ ( λ + L 2 d ∑ s k =1 1 ϵ 2 k ) d . Also note that det( λI ) = λ d . As a result, we can further upperbound the equation above by:

in which we have used d ≥ 1 in the last step to get a cleaner expression.

Therefore, we can re-write (58) as

in which we have also used the above-mentioned assumption of ∥ θ ∗ ∥ 2 ≤ S . Now again following [2], we plug in x = V s ( ̂ θ s -θ ∗ ) , which gives us

Therefore, we have that

This completes the proof of Theorem 6.

Nowwebriefly show how the improved confidence ellipsoid from our Theorem 6 leads to an improved regret upper bound for Q-LinUCB. Here we follow the proof of Theorem 3 from [38], and hence defer the detailed explanations of some of the steps to the proof there. To begin with, we have that

Here for simplicity, we have upper-bounded ∑ s k =1 (1 /ϵ 2 k ) by T 2 (Lemma 2) in the second inequality, and have omitted the dependency on log(1 /δ ) .

Then the total regrets in stage s can be upper-bounded by

Plugging in our tighter confidence ellipsoid (Theorem 6) into the regret analysis of QLinUCB, we have that the regret upper bound of QLinUCB can be analyzed as

which gives the tighter regret upper bound for Q-LinUCB that we have discussed in the main text (Sec. 5.6).

I Regret Upper Bound for the Matérn Kernel (Proof of Theorem 7)

In this section, we modify our analysis to derive a regret upper bound for the Matérn kernel. We focus on bounded noise here (i.e., the first scenario in Lemma 1), since the analysis for noise with bounded variance (i.e., the second scenario in Lemma 1) is similar and follows from the analysis of App. G. In our analysis here, for simplicity, we ignore the logarithmic factors.

To begin with, note that Theorem 1 is not kernel-specific and hence still holds in our analysis here. It has been shown that for the Matérn kernel, the upper bound on the standard maximum information gain is γ T = ˜ O ( T d 2 ν + d ) [35]. Therefore, similar to our Corollary 1, for the Matérn kernel, we can use Theorem 1 to show that

Next, we modify the proof of our Theorem 2 to derive the corresponding upper bound on the total number of stages for the Matérn kernel, i.e., we discuss here how we modify the proof of Theorem 2 in App. D. To begin with, (68) suggests that there exists a C > 0 such that ˜ γ m ≤ C ( ∑ m s =1 1 ϵ 2 s ) d 2 ν + d . Different from our analysis for the linear and SE kernels in App. D (i.e., (27) and (32)), the absolute constant C here also contains logarithmic factors. Similar to (27), we can show that

This naturally leads to

which is analogous to (28). Next, similar to (29), the total number of iterations can be analyzed as

Now suppose m>T 2 d 2 ν + d 2 C log 2 , then this implies that

This equation, combined with (71), suggests that ∑ m s =1 C 1 ϵ s log( 2 m δ ) > T , which is a contradiction. Therefore, we have that

Next, note that our confidence ellipsoid in Theorem 3 does not depend on the choice of the kernel and hence also holds for the Matérn kernel.

Lastly, we can modify the proof of Theorem 4 (App. F). Specifically, we can adopt the result from (47): R T ≤ O ( mβ m log m δ ) . For the Matérn kernel, Theorem 3 and (68) allow us to show that β m = ˜ O ( √ ˜ γ m -1 ) = ˜ O ( √ T 2 d 2 ν + d ) = ˜ O ( T d 2 ν + d ) . Combining this with (73) allows us to show that for the Matérn kernel,

This completes the proof of Theorem 7.

J More Experimental Details, and Experiment on Real Quantum Computer

In our experiments, when implementing both the classical GP-UCB and our Q-GP-UCB algorithms, we use random Fourier features (RFF) approximation to approximate the kernel: k ( x, x ′ ) ≈ ˜ ϕ ( x ) ⊤ ˜ ϕ ( x ′ ) . As a result, we can implement the weighted GP posterior following (6) as discussed in App. A, in which the M -dimensional ( M < ∞ ) random features ˜ ϕ ( · ) are used to replace the infinite-dimensional RKHS features ϕ ( · ) . We have used M = 100 random features in the synthetic experiment and M = 200 in the AutoML experiment. As we have mentioned in the main paper (Sec. 6), we have implemented the QMC algorithm from the recent work of [20] using the Qiskit python package. We have used 6 qubits for the Gaussian noise and 1 qubits for the Bernoulli noise. For each experiment, we perform multiple independent trials ( 10 trials for the synthetic experiment and 5 trials for the AutoML experiment) and have plotted the resulting mean and standard error as the solid lines and error bars in each figure, respectively. For our Q-GP-UCB, we choose β s = 1 + log s, ∀ s ≥ 1 following the order given by the theoretical value of β s (Theorem 3). For classical GP-UCB, we tried different approaches to setting β s : β s = 1 + log s , β s = √ 2 and β s = 1 , all of which are commonly adopted practices in BO; we found that β s = √ 2 and β s = 1 have led to the best performances for GP-UCB in, respectively, the synthetic and AutoML experiments.

For the synthetic experiment, some important experimental details have been discussed in Sec. 6. The reward function f used in the synthetic experiment is sampled from a GP with the SE kernel with a length scale of 0 . 1 . In the AutoML experiment, an SVM is used for diabetes diagnosis. That is, we adopt the diabetes diagnosis dataset which can be found at https://www.kaggle.com/uciml/ pima-indians-diabetes-database , which is under the CC0 license. The task involves using 8 features to predict whether the patient had diabetes or not, which constitutes a two-class classification problem. We use 70% of the dataset as the training set and the remaining dataset as the validation set. We aim to tune two hyperparameters of the SVM: the penalty parameter and the RBF kernel parameter, both within the range of [10 -4 , 1] . For simplicity, for each of the two hyperparameters, we discretize its domain into 5 points with equal spacing. This results in a domain with |X| = 25 .

Here we adopt the SE kernel: k ( x, x ′ ) ≜ σ 2 0 exp( -∥ ∥ x -x ′ ∥ ∥ 2 2 / (2 l 2 )) with σ 2 0 = 0 . 5 . Since we have implemented our algorithms (both classical GP-UCB and our Q-GP-UCB) using RFF approximation as discussed above, we performed a grid search for the length scale l within { 0 . 1 , 0 . 2 , . . . , 1 } for both GP-UCB and Q-GP-UCB when generating the random features. We found that l = 1 . 0 leads to the best performance for GP-UCB and l = 0 . 2 works the best for Q-GP-UCB. This is likely because for our Q-GP-UCB, the effective noise is much smaller (thanks to the accurate estimation by the QMC algorithm), which allows us to use a smaller length scale (which can model more complicated functions) to accurately model the reward function.

0

2000

4000

6000

8000

10000

Iterations

0

2000

4000

6000

8000

10000

Cumulative Regret

GP-UCB

Q-GP-UCB

LinUCB

0

2000

4000

6000

8000

10000

Iterations

0

2000

4000

6000

8000

10000

Cumulative Regret

GP-UCB (

=0.3)

GP-UCB (

=0.4)

Q-GP-UCB (

=0.3)

Q-GP-UCB (

=0.4)

LinUCB (

=0.4)

=0.4)

(a) synthetic, Bernoulli noise

(b) synthetic, Gaussian noise

0

2000

4000

6000

8000

10000

Iterations

0

20

40

60

Cumulative Regret

GP-UCB

Q-GP-UCB

Q-GP-UCB (simulated quantum noise)

0

2000

4000

6000

8000

10000

Iterations

0

20

40

60

Cumulative Regret

GP-UCB (

=0.3)

GP-UCB (

=0.4)

Q-GP-UCB (

=0.3)

Q-GP-UCB (

=0.4)

Q-GP-UCB (

=0.3, simulated quantum noise)

Q-GP-UCB (

=0.4, simulated quantum noise)

0

1000

2000

3000

4000

5000

Iterations

0

10

20

30

40

Cumulative Regret

GP-UCB (

=0.01)

GP-UCB (

=0.05)

Q-GP-UCB (

=0.01)

Q-GP-UCB (

=0.05)

Q-GP-UCB (

=0.01, simulated quantum noise)

Q-GP-UCB (

=0.05, simulated quantum noise)

(a) synthetic, Bernoulli noise

(a) synthetic, Gaussian noise

(c) AutoML

All our experiments are run on a computer with Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz, with 64 CPUs. No GPUs are needed.

More Experimental Results. Fig. 2 additionally shows the cumulative regret for the LinUCB and Q-LinUCB algorithms in the synthetic experiment. Consistent with the results in Fig. 1 (d) in the main paper (Sec. 6) which has included the results for LinUCB and Q-LinUCB in the AutoML experiment, here LinUCB and QLinUCB again underperform significantly in Fig. 2 for both types of noises. These results further verify that in realistic experiments with non-linear reward functions, algorithms based on linear bandits are significantly outperformed by those based on BO/kernelized bandits. Fig. 3 additionally shows the results of our Q-LinUCB after considering quantum noise. As we have discussed in the main paper (Sec. 6), the results show that the quantum noise leads to slightly worse performances for our Q-GP-UCB, yet they are still able to significantly outperform classical GP-UCB.

Results Using A Real Quantum Computer. Here we additionally perform an experiment using a real quantum computer. Specifically, we ran our QMC subroutine on a real IBM quantum computer based on superconducting qubit technology (7 qubit, Falcon r5.11H processor) through the IBM Quantum cloud service. We ran our synthetic experiment with Gaussian noise on the system for only 1 trial (instead of 10 as in other simulations) due to time and resource constraints. The results in Fig. 4 show that although the performance of our Q-GP-UCB on real quantum computers is worse than those obtained without quantum noise and with simulated quantum noise, our Q-GP-UCB run on real quantum computers is still able to significantly outperform classical GP-UCB. This experiment provides a further demonstration for the potential of our Q-GP-UCB algorithm to lead to quantum speedup real BO applications.

0

2000

4000

6000

8000

10000

Iterations

0

20

40

60

Cumulative Regret

GP-UCB (

=0.4)

Q-GP-UCB (

=0.4)

Q-GP-UCB (

=0.4, simulated quantum noise)

Q-GP-UCB (

=0.4, real quantum computer)

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies quantum kernelized bandits or Bayesian optimization (BO). Classically, in every iteration t=1,2,\\ldots,T, a BO algorithm chooses an arm x_t and then queries the reward function f for a noisy observation y_t=f(x_t)+\\zeta_t, where f can be non-linear and \\zeta_t is a sub-Gaussian noise. The goal is to minimize the cumulative regret R_T=\\sum_{t=1}^{T}[f(x^*)-f(x_t)]. In the quantum setting, every query to the reward function f at the arm x_t is replaced by a chance to access a quantum oracle or its inverse, which encodes the reward distribution for the arm x_t. Besides, a bounded noise or a noise with bounded variance are considered. The paper introduces the Q-GP-UCB algorithm which is the first BO algorithm able to achieve a regret bound of  O(poly log T), which is significantly smaller than the classical lower bound of \\Omega(\\sqrt{T}).\n\nSTRENGTHS:\n1. The paper provides the first quantum BO algorithm, which achieves a polylog(T) regret, beats the classical lower bound of \\Omega(\\sqrt{T}), and offers more evidence of quantum advantages over classical computers.\n\n2. The result generalizes the previous quantum speedup for multi-armed bandits (MAB) and stochastic linear bandits (SLB) [32]. Besides, the paper improves the regret bound of SLB in [32] by improving the analysis of the confidence ellipsoid.\n\n2. The paper is overall structured very well. The ingredients of the analysis for the Q-GP-UCB algorithm are listed in a readable way so I can quickly get the ideas behind it. I enjoy reading this paper very much.\n\nWEAKNESSES:\n1. I'm a bit doubtful about the technical contributions of this paper. Apparently, the basic framework of Q-GP-UCB follows from the weighted least squared estimator and the doubling trick in [32]. The key difference from [32] is the design of the weighted GP posterior distribution (see Eq. (3)), which looks very similar to the classical one (see Eq. (1)). Such a combination, of course, can be regarded as the main technical novelty, but the question is, I have no idea whether it raises inherent difficulties in the analysis.\n\nQUESTIONS:\n1. Please explain more about the technical difficulties in analyzing the Q-GP-UCB algorithm.\n\n2. I notice that the authors of [32] showed their Q-LinUCB algorithm has a regret of O(log^{5/2} T) for bounded noise, but this paper said the regret is O(log^{3/2} T) (see line 350) when citing [32]. Is this because the analysis of [32] is not tight? Please confirm it since one of the contributions of this paper is an improvement over Q-LinUCB. If you indeed improve the regret from O(log^{5/2} T) to O(log^{3/2} T), this would increase the contributions of the paper.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the regret attainable for multi-armed bandits with non-linear reward functions when having access to a quantum oracle. For this setting they introduce the Quantum Gaussian Process Upper Confidence Bound (Q-GP-UCB) that with probability at least $\\delta$ achieves regret:\n- $\\mathcal{O}((d\\log{T})^{3/2}\\log{d\\log{T}})$ using the linear kernel,\n-  $\\mathcal{O}((\\log{T})^{3/2\\cdot(d+1)}\\cdot (d+1) \\cdot \\log{\\log{T}})$ using the squared exponential (SE) kernel,\n\nwhen the noise is bounded and $d$ is the dimensionality of the input space. Similar rates are derived for noise with bounded variance.\n\nThis notable improvement over the classical fundamental limit is mostly attributed to the use of the Quantum Monte Carlo (QMC), and the improvement over the work of Wan et. al. 2022 (when instantiated with the linear kernel) is due to the paper's novel and tighter analysis of the confidence ellipsoid. They actually modify the proof of this prior work to attain the same rate as well.\n\nFinally, they run experiments using the Qiskit package that demonstrate their algorithm's superiority over the classical variant, as well as the benefits of using the SE kernel in the more practical setting of AutoML.\n\nSTRENGTHS:\n- Intuitive and well explained use of staging to manage the growing number of samples fed to the QMC subroutine.\n- Their weighing technique is very intuitive and by making the noise 1-sub-Gaussian allow them to use a self-normalized concentration inequality that improves the confidence ellipsoid and allows for better rates (also has nice intuitive meaning as enabling a measuring of weighted information gain).\n- I particularly like that they use plug their improved analysis into the work of Wan et. al. to make their original algorithm match the rate the authors attain with the linear kernel. This is a nice way to both showcase the strengthened analysis, as well as to clarify to the community which things are key algorithmic behavior versus analysis artifacts.\n\nWEAKNESSES:\n- No core majorly novel algorithmic design / concept definition, i.e. results are due to elegant combinations of techniques and careful analyses rather than groundbreaking new concepts/approaches introduced.\n- Minor typo: \"encode\" -> \"encodes\" (line 144).\n\nQUESTIONS:\n- Could you expand more on the empirical behavior of Q-GP-UCB in the initial stage? It is a bit hard to see given the large setting of total time-steps. For example, could you comment on when this may be an issue and when not? \n- Do you have intuition for what would happen with other kernels being used?\n\nLIMITATIONS:\n- As per above, maybe more limitations regarding trade-offs of initial phase (and others) could be discussed in experimental section?", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers kernelized bandits also known as Bayesian optimization under a particular feedback model inspired by quantum computing. Under this model repeating sampling from the same point for N times reduces the noise to the level on $1/N$. That is significantly tighter than the classic setting where sampling the same point for N times reduces the noise to the level on $1/\\sqrt{N}$. The paper introduces a UCB based algorithm, that is very similar to mini-META proposed in [6]. The proposed algorithm is referred to as Q-GP-UCB, and, in the case of SE kernel, obtains a polylogarithmic regret bound in the time horizon $T$ that is a significant improvement compared to $\\sqrt{T}$ regret bound in the classic setting.\n\nSTRENGTHS:\nThe problem is inspired by quantum computing and may be of broader interest. Overall this is an interesting formulation of kernel bandits.\n\nWEAKNESSES:\nThe formulation and analytical techniques are similar to those of [32] in the case of linear bandits. That to some extent limits the novelty and contributions. \n\nIn terms of complexity, the regret bounds seem to scale as $\\tilde{\\gamma}_m^{1.5}$, where the number of unique points is bounded by $\\tilde{\\gamma}_m$ and an additional $\\tilde{\\gamma}_m^{0.5}$ is contributed by the regret on each unique point, due to confidence ellipsoid. Theorem 1 bounds this quantity by \\gamma_{T^2}. For example, in the case of Matern kernels, that leads to a regret bound of $T^{(3d)/(4\\nu+2d)}$ where $\\nu$ is the smoothness of the kernel. This regret bound may be worse than the classic results of $T^{(\\nu+d)/(2\\nu+d)}$ when $d$ is large. I think this is an indication that the regret bounds presented in this work are sub-optimal and can be further improved. Given that there is no lower bound under this setting, that raises some doubts about the tightness of the analysis. The results should be seen as some initial attempt on the problem.\n\nQUESTIONS:\n- Considering the example above about Matern kenrel, could the authors comment on the tightness of the bounds? The results seem to be an improvement only in the case of SE kernel. Why they do not necessarily improve the regret bounds in the case of other kernels. \n\n- If the standard mean and variance are used rather than the weighted ones, which step in the proof would fail? The number of unique points or the regret on each unique point?\n\nLIMITATIONS:\nAlthough the formulation seems interesting, the results do not seem tight. To some extent they follow the case of linear bandits. When it comes to more general kernels the regret bounds seem to even fail to be sublinear in some cases.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies Bayesian optimization with quantum reward oracles where the reward function $f$ lies in an RKHS space with the squared exponential kernel, and at every iteration after input is selected, we can access a quantum unitary oracle and its inverse that encode the noisy reward distribution. In such a setting, the authors introduce the quantum-Gaussian process-upper confidence bound (Q-GP-UCB) which achieves a regret upper bound of $\\mathcal O(\\text{poly log} T)$. To do it, they introduce a weighted GP regression and then analyze the growth rate of the weighted information gain. Next, they derive a tight confidence ellipsoid which gives a guarantee of the concentration of the reward function and the weighted GP posterior mean. They also show that their bound on the confidence ellipsoid improves that of the quantum linear UCB (Q-LinUCB) algorithm [32] over a factor $\\sqrt{d}$, where $d$ is the input dimension. Finally, they show the performance of their proposed algorithm over the classical GP-UCB and Q-LinUCB, through a synthetic experiment and an experiment on automated machine learning\n\nSTRENGTHS:\n- The paper is well-organized and easy to read. The arguments, and comparisons with related works are clear and well-supported.\n- The paper is the first to introduce the first quantum Bayesian optimization algorithm which enjoys a regret upper bound of $\\mathcal O(\\text{poly log} T)$.\n\nWEAKNESSES:\nHowever, I am concerned about the novelty of the used techniques in this paper. They seem to be an unsophisticated combination of common techniques from classical Bayesian optimization (e.g., see [9], [28], [30]), the classical bandits [9], and quantum bandits [32]. It follows that the regret upper bound of $\\mathcal O(\\text{poly log} T)$ for their proposed BO algorithm has the same order as that of [32] which is designed for linear bandits.  \n\nThe authors claim that the recent quantum works on multi-armed or linear bandits are not able to solve sophisticated real-world problems with non-linear reward functions like their setting. However, from the technical point of view, Bayesian optimization under the setting that the reward function $f$ lies in an RKHS space is in fact a kind of linear bandit problem except that the input space is continuous. \n\nHence, I think that this paper is OK but not good enough from a technical point of view.\n\nQUESTIONS:\n- The authors seem not to consider the case of the Materm kernel. Is there any improvement in the regret bound with the aid of quantum computing in this case?\n-  It would be interesting if the authors can provide a discussion on the lower bound of the regret bound of BO in the setting of quantum computing.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}