SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Classically, Problem 3 can be solved by averaging  $\mathcal{O}(L^2/\hat{\sigma}^2)$  samples of  $X$ ; this query complexity is optimal among classical algorithms [64]. However, if we have access to the quantum sampling oracle defined in Definition 1, [25] showed that a (possibly) biased estimate  $\hat{\mu}$  with error  $\|\hat{\mu} - \mu\| \leq \hat{\sigma}$  can be computed using  $\widetilde{\mathcal{O}}(L\sqrt{d}\hat{\sigma}^{-1})$  queries.
Lemma 1 ([25, Theorem 3.5]). Given access to the quantum sampling oracle  $O_{X}$ , for any  $\hat{\sigma},\delta \geq 0$  there is a procedure QuantumMeanEstimation  $(X,\hat{\sigma},\delta)$  that uses  $\widetilde{\mathcal{O}} (L\sqrt{d}\log (1 / \delta) / \hat{\sigma})$  queries and outputs an estimate  $\hat{\mu}$  of the expectation  $\mu$  of any  $d$ -dimensional random variable  $X$  satisfying  $\operatorname {Var}[X]\leq L^2$  with error  $\| \hat{\mu} -\mu \| \leq \hat{\sigma}\leq L$  and success probability  $1 - \delta$ .
QuantumMeanEstimation [25] proceeds by introducing a directional mean function that reduces a multivariate mean estimation problem to a series of univariate mean estimation problems through quantum Fourier transform, which in the bounded norm case can be solved by quantum algorithms with a quadratic speedup using phase estimation. In terms of error rate, QuantumMeanEstimation in Lemma 1 improves over any classical sub-Gaussian estimator when  $\frac{\hat{\sigma}}{L} \geq \frac{1}{\sqrt{d}}$ . However, its bias hinders its combination with various optimization algorithms assuming unbiased inputs, see e.g., [57, 2, 29, 30]. In this work we show how to carefully combine their algorithms with a classic multi-level Monte-Carlo (MLMC) technique from [10, 7] to obtain an unbiased estimate  $\hat{\mu}$  and success probability 1 with the same rate as [25] and prove the following theorem.
Theorem 4. Algorithm 1 solves Problem 3 using an expected  $\tilde{\mathcal{O}}(Ld^{1/2}\hat{\sigma}^{-1})$  queries.
In the following Table 2 we provide a comparison between our result and previous works and in Section 6 we prove that our algorithm is optimal up to a poly-logarithmic factor. Notably, our algorithm does not depend on the detailed implementation of QuantumMeanEstimation but only its query complexity. Hence, we believe that our approach may also be useful in removing the bias of other quantum mean estimation algorithms with similar expressions of query complexities, e.g., quantum phase estimation [55, 24] and quantum amplitude estimation [14].
Our algorithm consists of two components. First, we show that we can obtain a variant of the quantum mean estimation algorithm, denoted as QuantumMeanEstimation $^{+}$ $(X,\hat{\sigma})$ , that outputs a low variance estimate with probability 1. This procedure compares the outcomes of the quantum estimator and a classical estimate, and in the event of significant disparity, generates a new independent classical estimate.
Queries	Bias	Variance	Method
σ-2	0	σ2	Classical Variance Reduction
d1/2σ-1	σ	σ2	Quantum Multivariate Mean estimation [25]
log2(1/γ) σ-1	γ	σ2	One-dimensional Quantum Mean estimation [24]
d1/2σ-1	0	σ2	Our Result (Theorem 4)
Table 2: Comparison between different methods for variance reduction in the case of  $L = 1$  and  $\hat{\sigma} \in (0,1)$ . The  $\tilde{\mathcal{O}}$  symbol was omitted in the "Queries" column.
Second, we use the MLMC technique [41], specifically a variant of the methods described in [10, 7], to carefully invoke the biased subroutine QuantumMeanEstimation $^{+}$  and compute the unbiased estimate. The algorithm, Algorithm 1, simply invokes QuantumMeanEstimation $^{+}$  for three randomly chosen accuracies and combines them to obtain the result. Though there are an infinite number of possible accuracies chosen, we show that the expectation, the variance, and expected number of queries are all suitable to prove Theorem 4, whose proof is deferred to Appendix B.
Algorithm 1: Quantum variance reduction
Input: Random variable  $X$ , target variance  $\hat{\sigma}^2$   
Output: An unbiased estimate  $\hat{\mu}$  of  $\mathbb{E}[X]$  with variance at most  $\hat{\sigma}^2$
1 Set  $\tilde{\mu}_0\gets$  QuantumMeanEstimation+  $(\bar{X},\hat{\sigma} /10)$
2 Randomly sample  $j\sim \mathrm{Geom}\left(\frac{1}{2}\right)\in \mathbb{N}$
3  $\tilde{\mu}_{j}\gets \mathrm{QuantumMeanEstimation}^{+}(X,2^{-3j / 4}\hat{\sigma} /10)$
4  $\tilde{\mu}_{j - 1}\leftarrow$  QuantumMeanEstimation  $^+$ $(X,2^{-3(j - 1) / 4}\hat{\sigma} /10)$
5  $\hat{\mu}\gets \tilde{\mu}_0 + 2^j (\tilde{\mu}_j - \tilde{\mu}_{j - 1})$
6 return  $\hat{\mu}$
3 Quantum accelerated stochastic approximation
In this section, we present our  $\widetilde{\mathcal{O}}(d^{5/8}\epsilon^{-3/2})$  query quantum algorithm for Problem 1. Our approach builds upon the framework proposed by Duchi et al. [28, 15], which involves performing a Gaussian convolution on the objective function  $f$  and then optimizing the resulting smooth convoluted function. Compared to their algorithm, our algorithm differs by replacing the variance reduction step by our quantum variance reduction technique (Algorithm 1).
As with a variety of prior work on parallel and private SCO [28, 33, 15], we consider the smooth function  $F_{r}$  that is the Gaussian convolution of the objective function  $f$ :
$$
F _ {r} (\mathbf {x}) := \int_ {\mathbb {R} ^ {d}} \gamma_ {r} (\mathbf {y}) f (\mathbf {x} - \mathbf {y}) d \mathbf {y}, \quad \text {w h e r e} \quad \gamma_ {r} (\mathbf {y}) := \frac {1}{(\sqrt {2 \pi} r) ^ {d}} \exp \left(- \frac {\| \mathbf {y} \| ^ {2}}{2 r ^ {2}}\right). \tag {5}
$$
As shown in Lemma 4, when the radius  $r$  of the convolution is sufficiently small,  $F_{r}$  closely approximates  $f$  pointwise. Consequently, to find an  $\epsilon$ -optimal point of  $f$  it suffices to find an  $\epsilon / 4$ -optimal point of  $F_{r}$  for  $r = \frac{\epsilon}{4\sqrt{dL}}$ . Moreover, Lemma 4 shows that the stochastic gradient  $\mathbf{g}_{F}$  of  $F_{r}$  can be defined and obtained based on the stochastic gradient  $\mathbf{g}$  of  $f$  as follows:
$$
\mathbf {g} _ {F} (\mathbf {x}) = \mathbf {g} (\mathbf {x} - \mathbf {y}), \quad \mathbf {y} \sim \gamma_ {r}, \tag {6}
$$
which satisfies
$$
\underset {\mathbf {y} \sim \gamma_ {r}} {\mathbb {E}} \mathbf {g} _ {F} (\mathbf {x}) = \nabla F _ {r} (\mathbf {x}) \quad \text {a n d} \quad \| \mathbf {g} _ {F} (\mathbf {x}) \| \leq L, \quad \forall \mathbf {x}.
$$
$$
\| \nabla F _ {r} (\mathbf {x}) \| \leq \int_ {\mathbf {y} \sim \gamma_ {r}} \| \mathbf {g} _ {F} (\mathbf {x}) \| d \mathbf {y} \leq L
$$
indicating that  $F_{r}$  is also  $L$ -Lipschitz.
To optimize this smooth convex function  $F_{r}$ , we leverage the accelerated stochastic approximation (AC-SA) algorithm introduced in [57], which applies an accelerated proximal descent method on the objective function using unbiased estimates of gradients. Our algorithm given in Algorithm 2 is a specialization of the AC-SA algorithm, where we implement those unbiased estimates of gradients using quantum variance reduction (Algorithm 1). In the classical setting, one query to the stochastic gradient  $\mathbf{g}_F(\mathbf{x})$  of  $F$  can be implemented by a random sampling a vector  $\mathbf{y} \in \mathbb{R}^d$  from the Gaussian  $\gamma_r$  followed by a query to the SGO  $C_{\mathbf{g}}$  (Definition 2) at  $\mathbf{x} - \mathbf{y}$ . Similarly, we can show that one query to a QSGO of  $F_{r}$  (Definition 3) can also be implemented by one query to the QSGO of  $f$ . The subsequent theorem presents the query complexity of Algorithm 2.
Algorithm 2: Quantum accelerated stochastic approximation (Q-AC-SA)
Input: Function  $f\colon \mathbb{R}^d\to \mathbb{R}$  , precision  $\epsilon$    
Parameters: Domain Size  $R$  total iteration budget  $\mathcal{T} = \frac{4d^{1 / 4}LR}{\epsilon}$  target variance  $\hat{\sigma} = \frac{d^{1 / 8}}{8}\sqrt{\frac{L\epsilon}{R}}$  convolution radius  $r = \frac{\epsilon}{4\sqrt{d}L},\gamma = \frac{R\sqrt{6\ell_F}}{(T + 2)^{3 / 2}\hat{\sigma}}$    
Output: an  $\epsilon$  -optimal point of  $F$    
1 Denote  $F_{r}(\mathbf{x})\coloneqq \int_{\mathbb{R}^{d}}\gamma_{r}(\mathbf{y})f(\mathbf{x} - \mathbf{y})\mathrm{d}\mathbf{y}$  as in (5)   
2 Set  $\mathbf{x}_1\gets \mathbf{0},\mathbf{x}_1^{ag}\gets \mathbf{x}_1$    
3 for  $t = 1,2,\ldots ,T$  do   
4  $\begin{array}{rl} & {\beta_t\leftarrow \frac{t + 1}{2},\gamma_t\leftarrow \frac{t + 1}{2}\gamma}\\ & {\mathbf{x}_t^{md}\leftarrow \beta_t^{-1}\mathbf{x}_t + (1 - \beta_t^{-1})\mathbf{x}_t^{ag}}\\ & {\mathrm{Call~Algorithm~1~for~an~unbiased~estimate~}\tilde{\mathbf{g}}_t\mathrm{~of~}\nabla F_r(\mathbf{x}_t^{md})\mathrm{~with~variance~at~most~}\hat{\sigma}^2}\\ & {\mathbf{x}_{t + 1}\leftarrow \arg \min \left\{\gamma_t(\tilde{\mathbf{g}}_t,\mathbf{z} - \mathbf{x}_t^{md}) + L\| \mathbf{x}_t^{md} - \mathbf{z}\| ^2 /(2r)\right\}}\\ & {\mathbf{x}_{t + 1}^{ag} = \beta_t^{-1}\mathbf{x}_{t + 1} + (1 - \beta_t^{-1})\mathbf{x}_t^{ag}} \end{array}$    
9 return xag
Theorem 5 (Formal version of Theorem 1, Part 1). Algorithm 2 solves Problem 1 using an expected  $\tilde{\mathcal{O}}(d^{5/8}(LR/\epsilon)^{3/2})$  queries.
The proof of Theorem 5 is deferred to Appendix C.
4 Quantum stochastic cutting plane method (Q-SCP)
In this section, we develop our  $\widetilde{\mathcal{O}}(d^{3/2}LR/\epsilon)$  query algorithm for Problem 1 which is based on a stochastic version of the cutting plane method. We introduce the key properties and related concepts of cutting plane methods, and then provide a procedure for efficiently post-processing the outcomes obtained from the stochastic cutting plane method using quantum variance reduction (Algorithm 1). Then, we analyze the overall query complexity for solving Problem 1. Technically, it is possible to obtain the results of this section using the quantum mean estimation routine of [25], rather than quantum variance reduction, however using quantum variance reduction facilitates our presentation.
We begin by introducing some notation and concepts on cutting plane methods. Cutting plane methods solve the feasibility problem defined as follows. Note that this problem is slightly easier to solve than the one in, e.g., [48], however it is simple suffices for our purposes.
Problem 4 (Feasibility Problem). We are given query access to a separation oracle for a set  $K \subset \mathbb{R}^d$  such that on query  $\mathbf{x} \in \mathbb{R}^d$  the oracle outputs a vector  $\mathbf{c}$  and either  $\mathbf{c} = \mathbf{0}$ , in which case  $\mathbf{x} \in K$ , or  $\mathbf{c} \neq \mathbf{0}$ , in which case  $H := \{\mathbf{z} : \mathbf{c}^\top \mathbf{z} \leq \mathbf{c}^\top \mathbf{x}\} \supset K$ . The goal is to query a point  $\mathbf{x} \in K$ .
[48] showed that Problem 4 can be solved by cutting plane method using  $\mathcal{O}(d\log (dR / r))$  queries to a separation oracle where  $R$  and  $r$  denote bounds on  $K$ .
Lemma 2 ([48, Theorem 1.1]). There is a cutting plane method which solves Problem 4 using at most  $C \cdot d\log (dR / r)$  queries for some constant  $C$ , given that the set  $K$  is contained in the ball of radius  $R$  centered at the origin and it contains a ball of radius  $r$ .
[67, 58] demonstrated that, running cutting plane method on a convex function  $f$  with the separation oracle being its gradient yields a sequence of points where at least one of them is an  $\epsilon$ -optimal point of  $f$ . This follows from the fact that there exists a ball of radius  $\mathcal{O}(\epsilon)$  around  $\mathbf{x}^*$  such that every point in this ball is  $\epsilon$ -optimal. In the stochastic setting, although we cannot access the precise gradient, we show that it suffices to use an  $\mathcal{O}(\epsilon / R)$ -approximate gradient oracle of  $f$  (formally defined in Appendix D.1) as the separation oracle. Specifically, we prove the following result.
Proposition 1. For any  $0 \leq \epsilon \leq LR$ , with success probability at least  $5/6$  we can obtain  $\mathcal{T} = \mathcal{O}(d\log(dLR/\epsilon))$  points  $\mathbf{x}_1, \ldots, \mathbf{x}_{\mathcal{T}} \in \mathbb{B}_R(\mathbf{0})$  such that one of the  $x_i$  is  $\epsilon$ -optimal using  $\widetilde{\mathcal{O}}(d^{3/2}LR/\epsilon)$  queries to the QSGO  $O_{\mathbf{g}}$  defined in Definition 3.
The proof of Proposition 1 is deferred to Appendix D.1. After applying Proposition 1, it is not clear which query  $\mathbf{x}_i$  is an  $\mathcal{O}(\epsilon)$ -optimal point. This difficulty arises because we lack access to the function value of  $f$ , which sets our problem apart from the feasibility problem discussed in [48], where there is a clear indication when a query successfully lies within the feasible region. Consequently, we next focus on identifying the optimal solution within the finite set  $\Gamma$  of points using access to the QSGO. We conceptualize this task as the approximately best point problem, formulated as follows.
Problem 5 (Approximately best point). For a  $L$ -Lipschitz convex function  $f \colon \mathbb{R}^d \to \mathbb{R}$  and  $\mathcal{T}$  points  $\mathbf{x}_1, \ldots, \mathbf{x}_{\mathcal{T}} \in \mathbb{B}_R(\mathbf{0})$ , find a convex combination  $\hat{\mathbf{x}} \in \mathbb{R}^d$  of the points satisfying
$$
f (\hat {\mathbf {x}}) \leq \min  _ {t \in \mathcal {T}} f (\mathbf {x} _ {t}) + \epsilon .
$$
In this work, we develop an algorithm that solves Problem 5 by making pairwise comparisons in a hierarchical order, where each pairwise comparison is computed by running binary search on the segment along the segment between the two points. Formally, we prove the following result.
Proposition 2. For any accuracy parameter  $\epsilon >0$ , with success probability at least  $5 / 6$  Algorithm 5 solves Problem 5 using  $\tilde{\mathcal{O}} (RLT / \epsilon)$  queries to an  $L$ -bounded QSGO  $O_{\mathrm{g}}$  defined in Definition 3.
The proof of Proposition 2 and the corresponding quantum algorithm can be found in Appendix D.2. Next, we present the main result of this section, which describes the query complexity of solving Problem 1 using quantum stochastic cutting plane method.
Corollary 1 (Formal version of Theorem 1, Part 2). With success probability at least  $2/3$ , Problem 1 can be solved using an expected  $\widetilde{\mathcal{O}}\left(d^{3/2} LR / \epsilon\right)$  queries.
The proof of Corollary 1 is deferred to Appendix D.3.
5 Quantum stochastic non-convex optimization
In this section, we present our quantum algorithms for Problem 2 in the bounded-variance setting and the mean-squared smoothness setting, respectively, using our quantum variance reduction technique.
To solve Problem 2 in the bounded variance setting, we leverage the randomized SGD method introduced in [40], which is a variant of SGD where the number of iterations is randomized. Our algorithm is a specialization of the randomized stochastic gradient algorithm, where we replace the classical variance reduction step by quantum variance reduction (Algorithm 1). The query complexity of our quantum algorithm is given in the following theorem.
Theorem 6 (Formal version of Theorem 3, bounded variance setting). For any  $\epsilon >0$ , Algorithm 6 solves Problem 2 in the bounded variance setting using an expected  $\tilde{\mathcal{O}} (\Delta \ell \sigma \sqrt{d}\epsilon^{-3})$  queries.
The proof of Theorem 6 and the corresponding quantum algorithm can be found in Appendix E.
To solve Problem 2 in the mean-squared smoothness setting, we leverage the SPIDER algorithm introduced in [29], which is a variance reduction technique that allows us to estimate the gradient of a function with lower cost by utilizing the smoothness structure and reuse the stochastic gradient samples at nearby points. Our algorithm is a specialization of the SPIDER algorithm, where we replace the classical variance reduction step by quantum variance reduction (Algorithm 1). The query complexity of our quantum algorithm is given in the following theorem.
Theorem 7 (Formal version of Theorem 3, mean-squared smoothness setting). For any  $0 \leq \epsilon \leq \sigma$ , Algorithm 7 solves Problem 2 in the mean-squared smoothness setting using an expected  $\widetilde{\mathcal{O}}\left(\ell \Delta \sqrt{d\sigma} \epsilon^{-2.5}\right)$  number of queries.
The proof of Theorem 7 and the corresponding quantum algorithm can be found in Appendix F.
In this section we present two quantum lower bounds for solving quantum variance reduction (Problem 3) and stochastic convex optimization (Problem 1), respectively.
We first establish the following quantum lower bound for the variance reduction problem (Problem 3) which shows that our Algorithm 1 is optimal up to a poly-logarithmic factor when  $\hat{\sigma} = \mathcal{O}\left(d^{-1/2}\right)$ . By Markov's inequality, Proposition 3 equivalently states that any quantum algorithm that solves Problem 3 must make an expected  $\Omega(L\sqrt{d}\hat{\sigma}^{-1})$  queries. This matches our algorithmic result provided in Theorem 4, up to a poly-logarithmic factor.
Proposition 3. There is a constant  $\alpha$  such that for any  $\hat{\sigma} \leq \frac{\sigma}{\alpha \sqrt{d}}$ , any quantum algorithm that solves Problem 3 with success probability at least  $2/3$  must make at least  $\Omega(L \sqrt{d} \hat{\sigma}^{-1})$  queries in the worst case.
The proof of Proposition 3 is deferred to Appendix G.1
Next, we establish the following quantum lower bounds for stochastic convex optimization (Problem 1) in the low-dimension regime and the high-dimension regime, respectively, which show that our quantum stochastic cutting plane method in Section 4 is optimal up to a poly-logarithmic factor when the dimension  $d$  is a constant, and there is no quantum speedup over SGD when  $d \geq \Omega(\epsilon^{-2})$ .
Theorem 8. For any  $\epsilon \leq \frac{RL}{100\sqrt{d}}$ , any quantum algorithm that solves Problem 1 with success probability at least  $2/3$  must make at least  $\Omega(\sqrt{d} RL/\epsilon)$  queries in the worst case. For any  $\frac{RL}{100\sqrt{d}} \leq \epsilon \leq 1$ , any quantum algorithm that solves Problem 1 with success probability at least  $2/3$  must make at least  $\Omega(R^2 L^2/\epsilon^2)$  queries in the worst case.
The proof of Theorem 8 is deferred to Appendix G.2.
We presented improved quantum algorithms for stochastic optimization. We developed a new technical tool which we call quantum variance reduction and show how to use it to improve upon the query complexity for stochastic convex optimization and for critical point computation in smooth, stochastic, non-convex functions. Further, we provided lower bounds which establish both the optimality of our quantum variance reduction technique and of one of our stochastic convex optimization algorithms in low dimensions. A natural open problem suggested by our work is to establish the optimal complexity of the problems we study, e.g., stochastic convex optimization and stochastic non-convex optimization with quantum oracle access, in higher dimensions. We hope this paper fuels further study of these problems.
We thank Adam Bouland, Yair Carmon, András Gilyén, Yujia Jin, and Tongyang Li for helpful discussions. A.S. was supported in part by a Microsoft Research Faculty Fellowship, NSF CAREER Award CCF-1844855, NSF Grant CCF-1955039, a PayPal research award, and a Sloan Research Fellowship. C.Z. was supported in part by the Shoucheng Zhang Graduate Fellowship.
[1] Naman Agarwal, Zeyuan Allen-Zhu, Brian Bullins, Elad Hazan, and Tengyu Ma, Finding approximate local minima faster than gradient descent, Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pp. 1195-1199, 2017, arXiv:1611.01146.
[2] Zeyuan Allen-Zhu, Katyusha: The first direct acceleration of stochastic gradient methods, The Journal of Machine Learning Research 18 (2017), no. 1, 8194-8244, arXiv:1603.05953.
[84] Dongruo Zhou, Pan Xu, and Quanquan Gu, Stochastic variance-reduced cubic regularized Newton methods, International Conference on Machine Learning, pp. 5990-5999, PMLR, 2018, arXiv:1802.04796.
[85] Dongruo Zhou, Pan Xu, and Quanquan Gu, Stochastic nested variance reduction for nonconvex optimization, Journal of Machine Learning Research 21 (2020), no. 103, 1-63, arXiv:1806.07811.
Stochastic convex optimization. Stochastic convex optimization is a broad, well-studied area of research. Beyond the works mentioned in the introduction, for additional references and discussion of research in the area, see [11, 47, 68] for detailed overviews of stochastic convex optimization methods and [19] for a presentation of convex optimization in the online learning setting. Additionally, see [26, 51] for a discussion of the closely related problem of finite-sum optimization.
Non-convex optimization. Non-convex optimization is a rapidly advancing research area in optimization theory. These advances are motivated in part by the fact that the landscapes of various modern machine learning problems are typically non-convex, including deep neural networks, principal component analysis, tensor decomposition, etc. In general, finding a global optimum of a non-convex function is NP-hard [66, 69]. Hence, many theoretical works instead focus on finding local minima rather than a global one, given existing empirical and theoretical evidence that local minima can be as good as global minima in certain machine learning problems [8, 34, 36, 37, 39, 46]. The first step of finding local minima would be to find stationary points, which has been discussed in many previous works [1, 9, 17, 16, 70, 71]. An important line of work focuses on first-order (stochastic) gradient-based algorithms for finding critical points and local minima since higher-order derivatives are often not accessible in practical scenarios [3, 17, 29, 30, 34, 49, 50, 62, 78, 81]. There are also results on non-convex optimization that study different settings [38, 61, 79, 85, 35, 83, 84].
Quantum Monte Carlo methods. Since [65], quantum Monte Carlo methods have been broadly investigated. In particular, [59] discussed quantum Monte-Carlo methods for entropy estimation, and [45] initiated the study of quantum mean estimation problem with multiplicative error, which is followed by [44, 25, 24, 56]. Multilevel Monte Carlo methods are also widely used in quantum algorithms, see e.g., [4, 60]. Specifically, the particular multilevel Monte Carlo strategies we use for quantum variance reduction have roots in the classic computation work of [10, 7].
Algorithm 3: QuantumMeanEstimation  $^+$ $(X,\hat{\sigma})$
Input: Random variable  $X$ , target variance  $\hat{\sigma}^2 \leq L^2$   
Parameters:  $\delta = \hat{\sigma}^6 / (4L)^6$ ,  $D = \frac{\hat{\sigma}}{4} + \frac{16L^3}{\hat{\sigma}^2}$   
Output: An estimate  $\hat{\mu}$  of  $\mu = \mathbb{E}[X]$  satisfying  $\mathbb{E}||\hat{\mu} - \mu||^2 \leq \hat{\sigma}^2$   
1 Set  $X_1 \leftarrow \text{QuantumMeanEstimation}(X, \hat{\sigma}/4, \delta)$   
2 Randomly draw one classical sample  $X_2$  of  $X$   
3 if  $\|X_1 - X_2\| \leq D$  then return  $X_1$   
4 else  
5 Randomly draw one classical sample  $X_3$  of  $X$   
6 return  $X_3$
We begin by presenting a lemma showing how two unbiased, bounded variance random variables  $(X_{2}$  and  $X_{3}$  in the lemma) can be used to control the expected  $\ell_2$  error of a random variable  $(X_{1}$  in the lemma) that is close to the expectation of  $X_{2}$  and  $X_{3}$  with some probability. In Algorithm 3  $X_{1}$  corresponds to the output of QuantumMeanEstimation and then  $X_{2}$  and  $X_{3}$  are obtained by classic sampling.
Lemma 3. Let  $X_{1}, X_{2}, X_{3} \in \mathbb{R}^{d}$  be independent random variables where  $\mathbb{E}X_{2} = \mathbb{E}X_{3} = \mu$ ,  $\operatorname{Var}[X_2] \leq L^2$ , and  $\operatorname{Var}[X_3] \leq L^2$ , and  $\|X_{1}-\mu\| \leq \hat{\sigma}$  with probability at least  $1-\delta$ . Further, let  $Z$  be the random variable that is  $X_{1}$  if  $\|X_{1}-X_{2}\| \leq D$  and  $X_{3}$  otherwise where  $D > \hat{\sigma}$ . Then,
$$
\mathbb {E} \| Z - \mu \| ^ {2} \leq \hat {\sigma} ^ {2} + L \left(\frac {L ^ {2} - \hat {\sigma} ^ {2}}{D - \hat {\sigma}}\right) + \delta (2 D ^ {2} + 3 L ^ {2}).
$$