Quantum Speedups of Optimizing Approximately Convex Functions with Applications to Logarithmic Regret Stochastic Convex Bandits
Abstract
We initiate the study of quantum algorithms for optimizing approximately convex functions.
Given a convex set K ⊆ R n and a function F : R n → R such that there exists a convex function f : K → R satisfying sup x ∈K | F ( x ) − f ( x ) | ≤ (cid:15)/n , our quantum algorithm ﬁnds an x ∗ ∈ K such that F ( x ∗ )
− min x ∈K F ( x ) ≤ (cid:15) using ˜ O ( n 3 ) quantum evaluation queries to F .
This achieves a polynomial quantum speedup compared to the best-known classical algorithms.
As an application, we give a quantum algorithm for zeroth-order stochastic convex bandits with ˜ O ( n 5 log 2 T ) regret, an exponential speedup in T compared to the classical Ω( √ T ) lower bound.
Technically, we achieve quantum speedup in n by exploiting a quantum framework of simulated annealing and adopting a quantum version of the hit-and-run walk.
Our speedup in T for zeroth-order stochastic convex bandits is due to a quadratic quantum speedup in multiplicative error of mean estimation.
Introduction
Optimization theory is a central research topic in computer science, mathematics, operations research, etc.
Currently, many efﬁcient algorithms for optimizing convex functions have been proposed (see for instance [10]), but much less is known for nonconvex optimization.
In this paper, we investigate polynomial-time algorithms for optimizing approximately convex functions.
On the one hand, such algorithms enjoy robustness and cover many natural scenarios including stochastic convex optimization, empirical risk minimization, etc.
On the other hand, approximately convex optimization paves the way of understanding nonconvex optimization in the general case.
Speciﬁcally, let K ⊆ R n be a convex set.
We call F : R n → R as an approximately convex function over K if there is a convex function f : K → R such that
[Equation]
Throughout the paper, we assume that f is L -Lipschitz with respect to (cid:96) ∞ norm, i.e., | f ( x ) − f ( y ) | ≤
L (cid:107) x − y (cid:107) ∞ for any x, y ∈ K .
We assume that B 2 (0 , 1) ⊆
K ⊆ B 2 (0 , R ) , i.e., the convex body
K contains the unit ball centered at 0 and is contained by a ball of radius R centered at 0.
Unless otherwise mentioned, we aim at algorithms with poly(log R ) dependence in R .
It is standard to assume the zeroth-order oracle of F , which returns the function value F ( x ) given an input x .
As far as we know, the state-of-the-art algorithm for ﬁnding an x ∗ ∈ K such that F ( x ∗ ) − min x ∈K F ( x ) ≤ (cid:15) with high probability was proposed by Belloni et al.
[8], which takes ˜ O ( n 4 . 5 )queries to the zeroth-order oracle.
On the other hand, Ref.
[34] proved that if the approximation error in (1) is at least ˜Ω(max { (cid:15)/n, (cid:15) 2 / √ n } ) , there exists a function F which no algorithm can ﬁnd a point x ∗ ∈ K such that F ( x ∗ )
− min x ∈K F ( x ) ≤ (cid:15) using poly( n, 1 /(cid:15) ) queries to F .
In other words, the (cid:15)/n term in (1) is fundamental, and it has been the standard assumption of studying approximately convex optimization in [8, 34].
Based on approximate convex functions, a closely related scenario is stochastic convex functions , where we have a function F : R n → R such that
[Equation]
Here f : R n → R is a convex function and (cid:15) x is a sub-Gaussian random variable with parameter σ , i.e., E [exp( λ(cid:15) x )] ≤ exp (cid:0) σ 2 λ 2 / 2 (cid:1) for any λ .
This implies that
[Equation]
Using this fact, Belloni et al.
[8] gave an algorithm for stochastic convex optimization with ˜
O ( n 7 . 5 /(cid:15) 2 ) queries to a zeroth-order oracle of F .
Contributions.
In this paper, we conduct a systemic study of quantum algorithms for optimization of approximately convex functions, with applications to zeroth-order stochastic convex bandits.
Quantum computing is a rapidly advancing technology, the capability of quantum computers is dramatically increasing and recently reached “quantum supremacy" by Google [6] and USTC [42].
In optimization theory, quantum advantages have been proven for semideﬁnite programs [3, 4, 11, 12], general convex optimization [5, 15], the escaping from saddle point problem in optimization [41], etc.
Nevertheless, as far as we know, quantum algorithms for approximately convex optimization and stochastic convex optimization are widely open.
In this paper, we consider these problems using the quantum zeroth-order evaluation oracle O F , a standard model used in previous quantum computing literature [5, 14, 15, 41]:
[Equation]
Here |·(cid:105) is the Dirac notation, and preliminaries of quantum computing will be covered in Section 2.
Intuitively, Eq.
(3) can take inputs with form (cid:80) mi =1 c i
[Equation]
i
[Equation]
where x i ∈ R n ∀ i ∈ [ m ] and (cid:80) mi =
| c i | 2 = 1 , and if we measure the outcome quantum state, we get F ( x i ) with probability
| c
i | 2 .
In particular, the quantum zeroth-order oracle allows the ability to query different locations in superposition , which is stronger than the classical counterpart (i.e., m = 1 ).
Nevertheless, if the classical zeroth-order oracle can be implemented by explicit arithmetic circuits, the quantum oracle in (3) can be implemented by quantum circuits of the same size up to logarithmic factors.
As for stochastic convex functions, similar to (3), we assume the following oracle:
[Equation]
where for any x ∈ R n , g x ( ξ ) follows a sub-Gaussian distribution as in (2).
Our ﬁrst result is a quantum algorithm for optimizing approximately convex functions:
Theorem 1.
With probability at least 0 .
9 , we can ﬁnd an x ∗ ∈ K such that
[Equation]
using ˜ O ( n 3 ) queries to the quantum evaluation oracle (3).
We remark that the succss probability can be easily boosted up to 1 − δ for any δ ∈ (0 , 1) , by paying an extra log(1 /δ ) factor in the quantum query complexity.
Compared to the best-known classical result by Belloni et al.
[8] with query complexity ˜ O ( n 4 . 5 ) , we achieve a polynomial quantum speedup in terms of n .
Technically, Belloni et al.
’s algorithm is based on the simulated annealing process and using the Hit-and-Run walk to generate samples in each stage.
In this work, we give a user-friendly version of the quantum walk framework (Theorem 3) that can be applied very easily to obtain quantum speedup for the mixing of classical Markov chains.
We then analyze the warmness and the overlap between adjacent Hit-and-Run walks in the simulated annealing process, showing that our quantum walk framework is applicable to each classical Hit-and-Run sampler.
By implementing the random walk quantumly, we improve the query complexity of the sampling procedure from (cid:101) O ( n 3 ) to (cid:101) O ( n 1 . 5 ) (Theorem 4).
We also design a non-destructive rounding procedure (Lemma 4) in each stage of the simulated annealing that simulates the classical rounding procedure in [8] but will not destroy the quantum states of the quantum walks.
Combining them together gives the (cid:101) O ( n 3 )
-query
quantum algorithm for minimizing an approximately convex function.
Our result can also be applied to give an (cid:101) O ( n 3 )
-query
quantum algorithm for optimizing approximately convex functions with decreasing ﬂuctuations.
See Section 3 for more details and the proof of Theorem 1.
Furthermore, to estimate the mean of a random variable up to multiplicative error (cid:15) , classical algorithms need to take 1 /(cid:15) 2 samples due to concentration inequalities such as Chernoff’s bound, whereas quantum algorithms can take roughly 1 /(cid:15)
queries (see Proposition 1).
As a result, we obtain polynomial quantum speedup in both n and 1 /(cid:15) for stochastic convex optimization:
Corollary 1.
With probability at least 0 .
8 , we can ﬁnd an x ∗ ∈ K such that
[Equation]
(6) using ˜ O ( n 5 /(cid:15) )
queries to the quantum stochastic evaluation oracle (4).
We apply Corollary 1 to solve the zeroth-order stochastic convex bandit problem, which is a widely studied bandit model (see e.g., [22, 24, 25]).
The problem is deﬁned as follows.
Let K ⊆ R n be a convex body and f : K → [0 , 1] be a convex function.
Here B 2 (0 , 1) ⊆
K ⊆ B 2 (0 , R ) .
An online learner and environment interact alternatively over T rounds.
In each round t ∈ [ T ] , the learner makes a query to the quantum stochastic evaluation oracle (4), and returns a value x t ∈ K as the current guess.
The learner aims to minimize the regret
[Equation]
where
[Equation]
and the expectation is taken over all randomness.
The classical state-of-the-art algorithm using a classical stochastic evaluation in each round achieves a regret bound of ˜ O ( n 4 . 5 √ T ) [25], and there is a classical lower bound Ω( n √ T ) on the regret [19].
Here we prove:
Theorem 2.
There is a quantum algorithm for which R T =
˜ O ( n 5 log( T ) log( TR )) .
This achieves poly(log T ) regret for zeroth-order stochastic convex bandits, an exponential quantum advantage in terms of T compared to classical zeroth-order stochastic convex bandits.
As far as we know, we give the ﬁrst quantum algorithms with poly-logarithmic regret bound on online learning problems.
An independent work [37] gave quantum algorithms with poly-logarithmic regret for multi-armed bandits and stochastic linear bandits, but these two types of bandits concern reward of discrete objects and linear functions, respectively, which are fundamentally different from the stochastic convex bandits we study.
To achieve this poly(log T ) regret for zeroth-order stochastic convex bandits, we divide the T iterations into (cid:98) log 2 T (cid:99) intervals with doubling length 1 , 2 , 4 , . . . .
We use the quantum queries from a previous interval to run our quantum stochastic convex optimization algorithm and use the output as the guess for all iterations in the next interval.
Since we can achieve linear dependence in 1 /(cid:15) in Corollary 1, it can be calculated that the total regret in each iteration is at most ˜ O ( n 5 log( TR )) , leading to our claim.
The proof details are given in Section 4.
Open questions.
Our work raises several natural questions for future investigation:
• Can we further improve the dimension dependence of our approximate convex optimization algorithm?
The current quantum speedup mainly leverages the quantum hit-and-run walk; it is of general interest to understand whether we can gain quantum advantage at other steps, or whether the convergence analysis of the hit-and-run walk per se can be improved.
• Can we improve the dimension dependence of the regret of our zeroth-order stochastic convex bandits?
It is natural to check whether our n 5 term can be improved by quantizing the classical state-of-the-art [25] and other recent works.
• Can we give fast quantum algorithms for more general nonconvex optimization problems, or with poly-logarithmic regret for more general bandit problems?
Limitations and societal impacts.
This work is purely theoretical.
Researchers working on theoretical aspects of quantum computing and optimization theory may beneﬁt from our results at the moment.
In the long term, once we have fault-tolerant quantum computers, our results can be applied to approximate convex optimization and stochastic bandit scenarios arising in the real world.
As far as we are aware, our work does not have negative societal impacts.
Preliminaries 2.1 Basics of quantum computing
We brieﬂy introduce basic notations and concepts of quantum computing in this section.
More details are covered in standard textbooks, for instance [30].
Quantum computing can be formulated in terms of linear algebra.
Speciﬁcally, the computational basis of space C d can be denoted by { (cid:126)e 1 , . . .
,(cid:126)e d } , where (cid:126)e i = (0 , . . .
. , 0) (cid:62) with the i th entry being 1 and other entries being 0.
These basis vectors are typically written by the Dirac notation , where we denote (cid:126)e i by | i (cid:105) (called a “ket"), and denote (cid:126)e (cid:62) i by (cid:104) i | (called a “bra").
A d -dimensional quantum state is an (cid:96) 2 -norm unit vector in C d , e.g., | v (cid:105) =
( v 1 , . .
, v d ) (cid:62) such that (cid:80) i | v i | 2 = 1 .
For each i , v i is called the amplitude in | i (cid:105) .
The tensor product of quantum states is their Kronecker product:
if | u (cid:105) ∈
C
d 1 and | v (cid:105) ∈ C d 2 , then | u (cid:105) ⊗ | v (cid:105) :
= ( u 1 v 1 , u 1 v 2 , . .
u d 1 v d 2 ) (cid:62) ∈
C d 1 ⊗ C d 2 .
We often omit the operator ⊗ and simply write | u (cid:105) | v (cid:105) or | u, v (cid:105) for being concise.
In general, the deﬁnition of quantum states can be extended to a continuous domain.
For instance,
[Equation]
represents a quantum state as long as (cid:82) R n | v x | 2 d x = 1 .
To keep quantum states normalized in (cid:96)norm, operations in quantum computing are unitary transformations .
2.2 Classical and quantum walks
A classical Markov over the space Ω is a sequence of random variables { X i } i ∈ N such that for any i > 0 and x 0 , . . .
, x i ∈ Ω ,
[Equation]
The Markov chain can be represented by its stochastic transition matrix P such that (cid:80)
y ∈ Ω P ( x, y ) = 1 for any x ∈ Ω .
A distribution π is stationary if it satisﬁes (cid:80) x ∈
Ω π ( x ) P ( x, y )
=
π ( y ) for any y ∈ Ω .
A Markov chain is reversible if its stationary distribution satisﬁes the following detailed balance condition:
[Equation]
The mixing time of a Markov chain with initial distribution π 0 is the number of steps t =
t ( (cid:15) ) ∈ N such that the total variation distance between the time- t distribution and the stationary distribution is at most (cid:15) for any (cid:15) ∈ (0 , 1) , i.e., d TV ( P t π 0 , π ) ≤ (cid:15).
In quantum, we can also deﬁne the discrete-time quantum walk [36], which is a quantum-analogue of classical Markov chain.
More precisely, we use a quantum state to represent a classical probability distribution:
[Equation]
In quantum walk, there are two operations:
• Reﬂect:
An operator R that reﬂects the quantum states with respect to the subspace span {| x (cid:105) | ψ x (cid:105)} x ∈ Ω , where | ψ x (cid:105) =
(cid:80) y ∈ Ω (cid:112) P ( x, y )
| y (cid:105) .
• Swap: An operator S that swap the two quantum registers: S | x (cid:105) | y (cid:105) =
| y (cid:105) | x (cid:105) .
Then, the quantum walk operator W is deﬁned as: W :=
S ◦ R .
Intuitively, the ﬁrst quantum register contains the current position of the random walk, and the second register contains the previous position.
In each step of quantum walk, the R operator makes a superposition in the second register of the next step positions with amplitudes proportional to their transition probabilities.
Then, the
S operator swaps the two quantum registers, using the ﬁrst register to store the new positions and the second register to store the old position.
In this way, we complete one-step of the random walk coherently (in superposition).
The quantum advantage of the quantum walk comes from the spectrum of W .
Note that the state of stationary distribution (cid:80) x ∈ Ω (cid:112) π ( x ) | x (cid:105) | ψ x (cid:105) is invariant under W .
In other words, it is an eigenvector with eigenvalue 1 (eigenphase 0).
On the other hand, the other eigenvectors of W has eigenphase at least √ 2 δ , where δ is the spectral gap 2 of the transition matrix P .
Therefore, applying the quantum phase estimation algorithm using O (1 / √ δ ) calls to W can distinguish the state corresponding to the stationary distribution and other eigenstates.
See [17, 36, 38] for more details.
2.3 Hit-and-Run walk
Hit-and-Run walk was introduced by R.L. Smith [35], and has a long line of reserach [1, 7, 9, 16, 26–29, 40] for its mixing time and applications in sampling, optimization, and volume estimation.
Intuitively, the Hit-and-Run walk is deﬁned as follows:
Pick a uniformly distributed random line (cid:96) through the current point x . 2.
Move to a random point y along the line chosen from the restricted distribution π (cid:96) .
More speciﬁcally, let f : R n → R be a logconcave distribution density.
Then, the distribution induced by restricting f to the line (cid:96) is deﬁned as follows:
[Equation]
The following lemma show the transition probability of the Hit-and-Run walk.
Lemma 1 ([26]) .
Let f be the density of a logconcave distribution.
If the current point of Hit-andRun is u , then the density function of the distribution of the next point x is
[Equation]
where π n = π n/ 2 Γ(1+ n/ 2) and µ f ( u, x ) is the f -measure of the chord through u and x .
Quantum Algorithm for Optimizing Approximately Convex Functions
We give a polynomial quantum speedup for minimizing an approximately convex functions using the quantum walk algorithm [36, 38].
More speciﬁcally, we consider the quantum walk in continuous space.
3 Let P denote the (column) stochastic transition density of a reversible Markov chain, i.e.,
[Equation]
and let π ( x ) denote the density of the stationary distribution.
Then, we can implement a quantum walk unitary W ( P ) such that its unique eigenvector with eigenvalue 1 (or equivalently eigenphase 0) is:
[Equation]
where | ψ x (cid:105) :
= (cid:82) R n (cid:112) P ( x, y )
| y (cid:105)
d y mixes all the points that can be moved from x , with amplitudes proportional to the transition probabilities.
This quantum state can be considered as a coherent encoding of the classical distribution π .
The advantage of quantum walk comes from the fact that
W ( P ) has phase gap √ δ , where δ is the spectral gap of P .
Therefore, by the quantum phase estimation algorithm [23], quantum walk can achieve quadratic speedup in δ .
In general, quantum walk algorithm can quadratically speedup the hitting time of classical Markov chain.
For the mixing time, it requires some additional complicated constraints on the Markov chain and distributions (see e.g., [2, 13, 31, 39]).
Based on previous studies on quantum walk mixing, we propose the following user-friendly quantum walk framework:
Theorem 3 (User-friendly quantum walk framework) .
Let M 0 be the initial Markov chain with stationary distribution π 0 , M 1 be the target
Markov chain with stationary distribution π 1 .
Suppose
M 0 , M 1 satisfy the following properties:
• Mixing time: d TV ( P t 0 1 · π 0 , π 1 ) ≤ (cid:15) and d TV ( P t 1 0 ·
π 1 , π 0 ) ≤ (cid:15) .
• Warmness: (cid:107) π 0 /π 1 (cid:107) =
O (1) and (cid:107) π
1 /π
(cid:107) =
O (1) , where (cid:107) π/σ (cid:107) :
= (cid:82) R n π ( x ) σ ( x ) π ( x )d x . • Overlap: |(cid:104) π 0 |
π 1 (cid:105)| =
(cid:82) R n (cid:112) π 0 ( x ) π 1 ( x )d x = Ω(1) .
Furthermore, suppose we have access to a unitary U that prepares the initial state | π 0 (cid:105) =
(cid:82)
R n (cid:112) π 0 ( x ) | x (cid:105)
d x .
Then, we can obtain a
state | (cid:101) π 1 (cid:105) with (cid:107) | (cid:101)
π 1 (cid:105) − |
π 1 (cid:105) (cid:107) 2 ≤ (cid:15) using O (cid:0) √ t 0 + t 1 log 2 (1 /(cid:15) ) (cid:1)
calls to the quantum walk operators.
More details and proofs are deferred to Appendix A.
Next, we can use Theorem 3 to speed-up the best-known classical algorithm for optimizing approximately convex functions [8], which has the following three levels:
• High level: Perform a simulated annealing with K stages.
At the i -th stage, the target distribution π g i has density
∝ g i ( x ) =
e − F ( x ) /T
i , where T i :=
(1 − 1 / √ n ) i .
• Middle level: Use N samples from π g i to construct a linear transformation
Σ i , rounding the distribution to near-isotropic position.
• Low level: Run the hit-and-run walk to evolve the distribution from π g i − 1 to π g i .
Here, each step of the hit-and-run walk picks a uniformly random direction at the current point, and then walks on the 1-dimensional chord intersected by the direction and K with probability density proportional to the logconcave density.
We formally state the hit-and-run walk in Algorithm 3.
We focus on speeding-up the Low level using Theorem 3.
Hence, we need to show that π g i and π g
i +1 satisfy the properties therein.
First of all, it has been proved in [8] that (cid:107)
π
g
i /π
g i +1 (cid:107) =
O (1) (Lemma 11).
We prove the following lemma with proof deferred to Appendix C.1.
Lemma 2 (Informal version of Lemma 12) .
Let π g
i be a distribution with density proportional to g i ( x ) =
exp( − F ( x ) /T
i ) , where F ( x ) is β -approximately convex.
Then, for any 0 ≤ i ≤ K − 1 , (cid:107)
[Equation]
/π g i (cid:107) ≤ 8 exp(2 β/T i +1 ) ≤ O (1) .
Therefore, the warmness property is satisﬁed.
We also prove that the Markov chains in this annealing schedule are slowly evolving:
Lemma 3 (Informal version of Lemma 13) .
Let π g
i be a distribution with density proportional to g i ( x ) =
exp( − F ( x ) /T
i ) , where F ( x ) is β -approximately convex.
Then, for any 0 ≤ i ≤ K − 1 , |(cid:104) π i | π i +1 (cid:105)| ≥ exp( − ( β/T
i +1 + 1) / 2) = Ω(1) .
Hence, the overlap property is also satisﬁed.
With the warmness and classical analysis of the hit-and-run walk (Theorem 7), we get that the classical mixing time from π g i to π g i +
and vice versa can be bounded by (cid:101) O ( n 3 ) .
Moreover, we also show in Appendix C.1 and Lemma 14 that each call to the quantum walk operator can be implemented by querying the evaluation oracle
O (1) times.
Thus, by Theorem 3, we get the following theorem:
Theorem 4 (Low level quantum speedup, informal version of Theorem 11) .
Let γ ∈ (0 , 1 /e ) .
Let g i ( x ) =
exp( − F ( x ) /T
i ) be the density of π g i with F ( x ) being β/ 2 -approximately convex.
Let
T i =
(1 − 1 / √ n ) i for 0 ≤ i ≤ K .
Then, for each 0 ≤ i ≤ K − 1 , given a state | π g i (cid:105) , we can produce a state | ˆ σ ( m ) i (cid:105) such that
[Equation]
using m = (cid:101) O ( n 1 . 5 ) calls for the evaluation oracle of F .
In the Middle level, we need to use N independent samples from π g i to construct a linear transformation Σ i for rounding.
However, we cannot directly measure the state |
π g i (cid:105) , since it will destroy the quantum coherence.
Instead, we propose a non-destructive approach to construct the linear transformation (the proof is deferred to Appendix C.2):
Lemma 4 (Non-destructive rounding, informal version of Lemma 15) .
For each i ∈ [ K ] , the linear transformation Σ i can be obtained using (cid:101) O ( N ) copies of the states (cid:12)(cid:12)
π g i − 1 (cid:11) , with query complexity (cid:101) O
( N · n 1 . 5 ) .
Moreover, the states (cid:12)(cid:12)
π g i − 1 (cid:11) will be recovered with high probability.
Now, we can put all the components together and obtain a quantum algorithm for optimizing approximately convex function with (cid:101) O ( n 1 . 5 ) quantum query complexity (Algorithm 1).
We sketch the proof in below and the formal proof is given in Appendix C.3:
Proof sketch of Theorem 1.
Observe that at each annealing stage, the sample distribution is the same as the classical algorithm.
Thus, by the classical analysis (Theorem 9) in [8], the same optimization guarantee still holds for the quantum algorithm.
Thus, if we take K = √ n log( n/(cid:15) ) and N = (cid:101) O ( n ) , the output x ∗ of QS IM A NNEALING procedure satisﬁes:
[Equation]
with high probability.
Then, consider the query complexity.
There are K stages in the annealing process, where each stage maintains (cid:101)
O
( N ) samples (quantum states).
By Theorem 4, evolving each state takes (cid:101) O ( n 1 . 5 ) queries.
Therefore, the total query complexity is
[Equation]
Here, we assume that the convex body K is known, e.g., K = R n or S n .
However, even if K is unknown, we can call its membership oralce to run our algorithm.
More speciﬁcally, in constructing the initial state | π 0 (cid:105) , we need to query the membership oracle for (cid:101) O
(1) times.
And since we prepare
N copies, this step takes (cid:101)
O
( N ) queries in total.
Then, in each step of the quantum walk, we need to query the membership oracle for (cid:101) O
(1) times to determine the intersection point for the hit-and-run process.
Thus, the number of queries to the membership oracle of K is the same as the the number of evaluation oracle queries.
Hence, our algorithm will query the membership oracle for (cid:101) O ( n 3 ) times in all.
As for the number of qubits to implement our algorithm, each state | π 0 (cid:105) uses (cid:101) O ( n ) qubits.
Thus, (cid:101) O
( n 2 ) qubits are used to store all states.
And we need O ( n ) ancilla qubits for the quantum walk unitaries.
Therefore, our algorithm uses (cid:101) O ( n 2 ) qubits in total.
Optimization of approximately convex functions with decreasing ﬂuctuations.
Beyond the (cid:96) ∞ -norm assumption for all x ∈ K in Eq.
(1), it is also possible to give efﬁcient algorithms for optimizing other types of approximately convex functions.
Speciﬁcally, Belloni et al.
[8, Section 7] studied approximately convex functions with decreasing ﬂuctuations.
1: procedure QS IM A NNEALING (cid:46) Theorem2:
N ← (cid:101) O ( n ) , K ← √ n log( n/(cid:15) ) 3:
Prepare N (approximately) copies of | π 0 (cid:105) , denoted as | (cid:101) π (1)0 (cid:105) , . .
, | (cid:101) π
( N )(cid:105) 4: for i ← 1 , . . .
, K do 5:
Use {| (cid:101) π ( j )
(cid:105)} j ∈ [ N ] to nondestructively construct Σ i (cid:46)
Lemma6:
Apply quantum walk to evolve the states | (cid:101) π ( j ) i − 1 (cid:105) to | (cid:101) π
( j ) i (cid:105) (cid:46) Theorem
: end for 8: x jK ← measure the ﬁnal state | (cid:101) π ( j ) K (cid:105) for j ∈ [ N ] 9: return arg min j ∈ [ N ] F ( x jK ) 10: end procedure
Suppose that the function f in (1) is 1-Lipschtiz and α -strongly convex with minimum at x min ∈ K :
[Equation]
(cid:107)
[Equation]
Deﬁne a measure of "non-convexity" of F w.r.t to f in an n -dimensional ball of radius r near x min :
[Equation]
x ) | .
We can call Theorem 1 iteratively.
Suppose that at the start of the t th iteration we have a ball
B 2 ( x t − 1 , 2 r t − 1 ) satisfying
[Equation]
( x min , r t − 1 ) ⊂
[Equation]
2 ( x t −
[Equation]
r t −
[Equation]
⊂
[Equation]
( x
[Equation]
t − 1 ) .
After executing Theorem 1 in this iteration with (cid:101) O
( n 3 ) quantum queries, we reach a point x t such that with high probability
[Equation]
for some global constant C > 0 .
Due to strong convexity, this gives a new radius r t recursively:
[Equation]
When ∆( r ) =
cr p for some c > 0 , p ∈ (0 , 2) , the iteration stops when
[Equation]
, and when
[Equation]
for some c, d > 0 , the iteration stops at r ∗ satisfying
[Equation]
The total number of quantum queries is still (cid:101) O ( n 3 ) .
Quantum Algorithm for Zeroth-Order Stochastic Convex Bandits
We ﬁrst prove Corollary 1 using quantum mean estimation with a Gaussian tail:
Proposition 1 (Adapted from [20, Theorem 4.2]) .
Suppose that X is a random variable on a probability space (Ω , p ) with mean µ and variance σ 2 .
Suppose we have a unitary oracle U satisfying
U | 0 (cid:105) =
(cid:82) x ∈ Ω (cid:112) p ( x ) | x (cid:105) d x .
Then, for any ∆ ∈ (0 , 1) and τ ∈ N such that τ ≥ log(1 / ∆) , there is a quantum algorithm that outputs a mean estimate ˜ µ such that
[Equation]
using O ( τ log 3 / 2 ( τ ) log log( τ )) queries to U .
Proof of Corollary 1.
We follow Section 6 of [8] while use Theorem 1 and Proposition 1.
Speciﬁcally, for a parameter 0 < α < 1 , we let N α be a box grid of K with side length α .
In other words,
N α is α -net of K in (cid:96) ∞ norm.
Since K ⊆ B 2 (0 , R ) , |N α | ≤ ( R/α ) n .
Note that for the sub-Gaussian random variable (cid:15) x in (2), it has variance at most 4 σ 2 because
[Equation]
Upon a query x (cid:48) ∈ K , we deﬁne an oracle O τ,αf which returns f ( x ) + ˜ (cid:15) x for x ∈ N α which is closest to x (cid:48) , and the ˜ (cid:15) x here is obtained by applying Proposition 1 with the unitary oracle O f in
Eq.
(4) to estimate the function value f ( x ) .
Speciﬁcally, with ∆ = exp (cid:0) − t 2 (cid:1) where t is a parameter determined later, we have
[Equation]
We note that in our algorithm based on the hit-and-run walk, with probability 1 we do not revisit the same point.
As a result, O τ,αf is no more powerful than O f since the learner only obtains information on N α , and in the rest of the proof we assume O τ,αf as the oracle we use.
We take
[Equation]
Note that the value of t promises that exp (cid:0) − t 2 (cid:1) ( R/α ) n ≤ 0 . 1 .
In other words, with probability at least 0 .
9 , we promise that
[Equation]
Finally, we take τ such that the RHS of (9) equals to (cid:15)/ 2 n , which is equivalent to
[Equation]
This will ﬁnally promise that
[Equation]
meeting the condition of Theorem 1.
Consequently, with probability at least 0 . 9 · 0 . 9 > 0 . 8 , we can ﬁnd an x ∗ ∈ K such that f ( x ∗ ) − min x ∈K f ( x ) ≤ (cid:15) using ˜ O ( n 3 ) ·
τ = ˜
O ( n 5 /(cid:15) ) queries to the quantum stochastic evaluation oracle (4).
Proof of Theorem 2.
We prove that Algorithm 2 satisﬁes Theorem 2.
Intuitively, we divide the T rounds into m + 1 intervals where m ← (cid:98) log 2 T (cid:99) , such that [ T ] = (cid:83) m
+ i
=1 T i
and
T
i := { 2 i − 1 , 2 i − 1 + 1 , . . .
, 2 i − 1 } for each i ∈ [ m ] .
When executing in the interval
T i , the output required by the online learner is always x t = x 2 i − 1 , the x at the end of the last interval.
On the other hand, the queries in the current interval are applied to running the quantum stochastic convex optimization algorithm in Corollary 1 and output a nearly-optimal solution with probability at least 1 − O (1 /T ) .
With |T i | = 2 i − 1 queries at hand, we divide them into log( TR ) repeats of
Corollary 1, each using 2 i − 1 / log( TR ) queries in the quantum algorithm.
As a result, each repeat j ∈ [log( TR )] outputs a value ˜ x 2 i ,j such that
[Equation]
with probability at least 0.8.
We take x 2 i :=
arg min j ∈ [log T ] f
(˜ x 2 i ,j ) .
With probability at least 1 − 0 .
8 log( TR ) = 1 − O (1 /TR ) , we have
[Equation]
Going through all i ∈ [ m + 1] intervals, by the union bound, with probability at least
[Equation]
1: procedure QB ANDITS ( T ) 2: m ← (cid:98) log 2 T (cid:99) , K ← (cid:98) log 2 ( TR ) (cid:99) 3: x 1 ←4: for i ← 1 , 2 , . . .[-, m + 1 do 5: t ← 2 i −
 if t ≤ m then 7: L ← 2 i −(cid:46) T i := { 2 i − 1 , 2 i − 1 + 1 , . . .-]
, m + 1 do 5: t ← 2 i {+−
+}
if t ≤ m then 7:
L ← 2 i −(cid:46) T i := { 2 i − 1 , 2 i − 1 + 1 , . . .
, 2 i − 1 } 8: else 9:
[-| ψ x (cid:105) (cid:104) x | (cid:104) 
-]
L ← T − 2 m +(cid:46) T m +1 :
[-| 
-]
= { 2 m , 2 m + 1 , . .
, T } 10: end if 11: for j ← 1 , 2 , . .
[-• (cid:101) R | ψ 0 (cid:105) | 
-]
[-| 
-]
[-| ψ j (cid:105) | 
-]
, K do 12: y j ← QM IN S TOC C ONV ( O τ,αf , L/K ) (cid:46) Corollary 1 with | T i | /K queries 13: for l ← 0 , 1 , . . .
[-By assumption, we know that π 0 mixes in t (cid:48) 
-]
[-Similarly, π 1 mixes in t (cid:48) 
-]
[-Then, by Lemma 7 with ∆ = Ω( t (cid:48) 
-]
[-1 log(1 /(cid:15) 1 )) calls to controlled- W (cid:48) 
-]
, L/K do 14: x t + l ← x 2 i −(cid:46)
[-1 /π 
-]
Onliner learner’s output at time t + l 15: end for 16: if f ( y j ) < f
( x 2 i ) then 17: x 2 i ← y j 18: end if 19: end for 20: end for 21: end procedure we have
[Equation]
[-Lemma 
-]
In all, we get that the regret bound as desired:
R
[Equation]
(cid:34)
[Equation]
[-+
-]
[-g i +
 /π-]t
[Equation]
( f ( x
[Equation]
f
[Equation]
) (cid:35) ≤ (cid:18)
[Equation]
O (cid:18) log T
[-i +
-]
TR (cid:19)(cid:19) · m +1 [-(cid:107) , (cid:107) π g i +
-](cid:88)
[Equation]
Acknowledgement
We thank anonymous referees for valuable comments.
TL was supported by a startup fund from
Peking University.
RZ was supported by the University Graduate Continuing Fellowship from UT
[-i +
 = T i (1 − 1 / √ n ) . Therefore, [Equation]-]Austin
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[-π g i +
 /π g i (cid:107) are upper-bounded by some constants.-][N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[N/A] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[-, | (cid:101) π ( N ) i (cid:105) (cid:46) Theorem
-]
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]