SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This subsection is devoted to building a minimax optimal estimator of the  $q$ -th moment under the assumption that  $\frac{s}{d} > \frac{1}{p} - \frac{1}{2q}$ , which guarantees that functions in the space  $W^{s,p}$  are sufficiently smooth. From the Sobolev Embedding theorem, we know that the sufficient smoothness assumption implies  $W^{s,p}(\Omega) \subset L^{p^*}(\Omega) \subset L^{2q}(\Omega)$ , where  $\frac{1}{p^*} = \frac{1}{p} - \frac{s}{d}$ . Given any function  $f \in W^{s,p}(\Omega)$  along with  $n$  uniformly sampled quadrature points  $\{x_i\}_{i=1}^n$  and corresponding observations  $\{y_i = f(x_i)\}_{i=1}^n$  of  $f$ , the key idea behind the construction of our estimator  $\hat{H}_C^q$  is to build a nonparametric estimation  $\hat{f}$  of  $f$  based on a sub-dataset and use  $\hat{f}$  as a control variate for Monte Carlo simulation. Consequently, it takes three steps to compute the numerical estimation of  $I_f^q$  for any estimator  $\hat{H}_C^q: \Omega^n \times \mathbb{R}^n \to \mathbb{R}$ . The first step is to divide the observed data into two subsets  $S_1 := \{(x_i, y_i)\}_{i=1}^n$ ,  $S_2 := \{(x_i, y_i)\}_{i=\frac{n}{2}+1}$  of equal size and use a machine learning algorithm to compute a nonparametric estimation  $\hat{f}_{1:\frac{n}{2}}$  of  $f$  based on  $S_1$ . Without loss of generality, we may assume that the number of data points is even. Secondly, we treat  $\hat{f}_{1:\frac{n}{2}}$  as a control variate and compute the  $q$ -th moment  $I_f^q$ . Using the other dataset  $S_2$ , we may obtain a Monte Carlo estimate of  $I_f^q - I_{\hat{f}_{1:\frac{n}{2}}}^q$  as follows:  $I_f^q - I_{\hat{f}_{1:\frac{n}{2}}}^q \approx \frac{2}{n} \sum_{i=\frac{n}{2}+1}^n \left(y_i^q - \hat{f}_{1:\frac{n}{2}}^q(x_i)\right)$ . Finally, combining the estimation of the  $q$ -th moment  $I_{\hat{f}_{1:\frac{n}{2}}}^q = \int_{\Omega} \hat{f}_{1:\frac{n}{2}}^q(x) dx$  with the estimation of  $I_f^q - I_{\hat{f}_{1:\frac{n}{2}}}^q$  gives us the numerical estimation returned by  $\hat{H}_C^q$ :
$$
\hat {H} _ {C} ^ {q} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}\right) := \int_ {\Omega} \hat {f} _ {1: \frac {n}{2}} ^ {q} (x) d x + \frac {2}{n} \sum_ {i = \frac {n}{2} + 1} ^ {n} \left(y _ {i} ^ {q} - \hat {f} _ {1: \frac {n}{2}} ^ {q} \left(x _ {i}\right)\right). \tag {3.1}
$$
We assume that our function estimation  $\hat{f}$  is obtained from an  $\frac{n}{2}$ -oracle  $K_{\frac{n}{2}}: \Omega^{\frac{n}{2}} \times \mathbb{R}^{\frac{n}{2}} \to W^{s,p}(\Omega)$  satisfying Assumption 3.1. For example, there are lines of research [49, 50, 59, 60, 61] considering how the moving least squares method [66, 67] can achieve the convergence rate in (3.2).
Assumption 3.1 (Optimal Function Estimator as an Oracle) Given any function  $f \in W^{s,p}(\Omega)$  and  $n \in \mathbb{N}$ , let  $\{x_i\}_{i=1}^n$  be  $n$  data points sampled independently and identically from the uniform distribution on  $\Omega$ . Assume that for  $s > \frac{2dq - dp}{2pq}$ , there exists an oracle  $K_n: \Omega^n \times \mathbb{R}^n \to W^{s,p}(\Omega)$  that estimates  $f$  based on the  $n$  points  $\{x_i\}_{i=1}^n$  along with the  $n$  observed function values  $\{f(x_i)\}_{i=1}^n$  and satisfies the following bound for any  $r$  satisfying  $\frac{1}{r} \in \left( \max \left\{ \frac{d - sp}{pd}, 0 \right\}, \max \left\{ \frac{1}{p}, \mathbb{1}\{s > \frac{d}{p}\} \right\} \right]$ :
$$
\left(\mathbb {E} _ {\{x _ {i} \} _ {i = 1} ^ {n}} \left[ \| K _ {n} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{f (x _ {i}) \right\} _ {i = 1} ^ {n}\right) - f \| _ {L ^ {r} (\Omega)} ^ {r} \right]\right) ^ {\frac {1}{r}} \lesssim n ^ {- \frac {s}{d} + \left(\frac {1}{p} - \frac {1}{r}\right) +}. \tag {3.2}
$$
A construction of the desired oracle and a complete proof of the upper bound above (up to logarithm factors) is deferred to Appendix E. Based on the oracle above, we can obtain the following upper bound that matches the information-theoretic lower bound in Theorem 2.1.
Theorem 3.1 (Upper Bound on Moment Estimation with Sufficient Smoothness) Assume that  $p > 2$ ,  $q < p < 2q$  and  $s > \frac{2dq - dp}{2pq}$ . Let  $\{x_i\}_{i=1}^n$  be  $n$  quadrature points independently and identically sampled from the uniform distribution on  $\Omega$  and  $\{y_i := f(x_i)\}_{i=1}^n$  be the corresponding  $n$  observations of  $f \in W^{s,p}(\Omega)$ . Then the estimator  $\hat{H}_C^q$  constructed in (3.1) above satisfies
$$
\mathbb {E} _ {\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}} \left[ \left| \hat {H} _ {C} ^ {q} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}\right) - I _ {f} ^ {q} \right| \right] \lesssim n ^ {\max  \left\{- q \left(\frac {s}{d} - \frac {1}{p}\right) - 1, - \frac {s}{d} - \frac {1}{2} \right\}}, \tag {3.3}
$$
Proof Sketch Given a non-parametric estimator  $\hat{f}$  of the function  $f$ , we may bound the variance of the Monte Carlo process by  $(f^q - \hat{f}^q)^2$  and further upper bound it by the sum of the following two terms:
$$
\left| f ^ {q} - \hat {f} ^ {q} \right| ^ {2} \lesssim \underbrace {\left| f ^ {q - 1} (f - \hat {f}) \right| ^ {2}} _ {\text {s e m i - p a r a m e t r i c i n f l u n c e}} + \underbrace {\left| (f - \hat {f}) ^ {q} \right| ^ {2}} _ {\text {e s t i m a t i o n e r r o r p r o p a g a t i o n}}. \tag {3.4}
$$
The first term above represents the semi-parametric influence part of the problem, as  $qf^{q - 1}$  is the influence function for the estimation of the  $q$ -th moment  $f^q$ . The second term characterizes how function estimation affects functional estimation. If we consider the special case of estimating the mean instead of a general  $q$ -th moment, i.e.,  $q = 1$ , the semi-parametric influence term will disappear. Consequently, the convergence rate won't transit from  $n^{-\frac{1}{2} -\frac{s}{d}}$  to  $n^{-q(\frac{s}{d} -\frac{1}{p}) - 1}$  in the special case.
Although the algorithm remains unchanged in the sufficient smooth regime, we need to consider three separate cases to obtain an upper bound on the integral of the semi-parametric influence term  $|f^{q - 1}(f - \hat{f})|^2$  in (3.4). An illustration of the three cases is given in Figure 2.
From Hölder's inequality, we know that  $\int_{\Omega} f^{2q-2}(x) (f(x) - \hat{f}(x))^2 dx$  can be upper bounded by  $\| f^{2q-2} \|_{L^{r'}(\Omega)} \| (f - \hat{f})^2 \|_{L^{r_*}(\Omega)}$ , where  $\| \cdot \|_{L^{r'}(\Omega)}$  and  $\| \|_{L^{r_*}(\Omega)}$  are dual norms. Therefore, the main difficulty here is to embed the function  $f$  in different spaces via the Sobolev Embedding Theorem under different assumptions on the smoothness parameter  $s$ . When the function is smooth enough, i.e.  $s > \frac{d}{p}$ , we embed the function  $f$  in  $L^\infty(\Omega)$  and evaluate the estimation error  $f - \hat{f}$  under the  $L^2$  norm. Then our assumption on the oracle (3.2) gives us an upper bound of magnitude  $n^{-\frac{2s}{d}}$  on  $\| f - \hat{f} \|_{L^2(\Omega)}^2$ , which helps us further upper bound the semi-parametric influence part  $\int_{\Omega} f^{2q-2}(x) (f(x) - \hat{f}(x))^2 dx$  by  $n^{-\frac{2s}{d}}$  up to constants. When  $\frac{d(2q-p)}{p(2q-2)} < s < \frac{d}{p}$ , we embed the function  $f$  in  $L^{\frac{2pq-2p}{p-2}}(\Omega) \subseteq L^{\frac{pd}{d-sp}}(\Omega)$  and evaluate the estimation error  $f - \hat{f}$  under the  $L^p$  norm. Applying our assumption on the oracle (3.2) again implies that the semi-parametric influence part  $\int_{\Omega} f^{2q-2}(x) (f(x) - \hat{f}(x))^2 dx$  can be upper bounded by  $n^{-\frac{2s}{d}}$  up to constants. When  $\frac{d(2q-p)}{2pq} < s < \frac{d(2q-p)}{p(2q-2)}$ , we embed the function  $f$  in  $L^{p^*}$  and evaluate the error of the oracle in  $L^{\frac{2p^*}{p^*+2-2q}}$ , where  $\frac{1}{p^*} = \frac{1}{p} - \frac{s}{d}$ . Similarly, we can use (3.2) to upper bound the semi-parametric influence part  $\int_{x \in \Omega} f^{2q-2}(x) (f(x) - \hat{f}(x))^2 dx$  by  $n^{2q(\frac{1}{p} - \frac{s}{d}) - 1}$ .
The upper bound on the propagated estimation error  $\int_{x\in \Omega} (f(x) - \hat{f}(x))^{2q} dx$  in (3.4) can be derived by evaluating the error of the oracle under the  $L^{2q}$  norm. i.e, by picking  $r = 2q$  in (3.2) above, which yields an upper bound of magnitude  $n^{2q(\frac{1}{p} - \frac{s}{d}) - 1}$ .
The obtained upper bounds on the semi-parametric influence part and the propagated estimation error above provide us with a clear view of the upper bound on the variance of  $f^q - \hat{f}^q$ , which is the random variable we aim to simulate via Monte-Carlo in the second stage. Using the standard Monte-Carlo algorithm to simulate the expectation of  $f^q - \hat{f}^q$  then gives us an extra  $n^{-\frac{1}{2}}$  factor for the convergence rate, which helps us attain the final upper bounds given in (3.3). A complete proof of Theorem 3.1 is given in Appendix C.1.
3.2 Beyond the Sufficient Smoothness Regime: Truncated Monte Carlo
In this subsection, we study the case when the sufficient smoothness assumption breaks, i.e.  $\frac{s}{d} < \frac{1}{p} - \frac{1}{2q}$ . According to the Sobolev Embedding theorem, we have that  $W_{s}^{p}$  is embedded in  $L^{\frac{dp}{d - sp}}$ . Since  $\frac{1}{p} - \frac{s}{d} > \frac{1}{2q}$  implies  $\frac{dp}{d - sp} < 2q$ , the underlying function  $f$  is not guaranteed to have bounded  $L^{2q}$  norm, which indicates the existence of rare and extreme events. Consequently, the Monte Carlo estimate of  $f$ 's  $q$ -th moment must have infinite variance, which makes it hard to simulate. Here we present a truncated version of the Monte Carlo algorithm that can achieve the minimax optimal convergence rate. For any fixed parameter  $M > 0$ , our estimator is designed as follows:
$$
\hat {H} _ {M} ^ {q} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}\right) := \frac {1}{n} \sum_ {i = 1} ^ {n} \max  \left\{\min  \left\{y _ {i}, M \right\}, - M \right\} ^ {q}. \tag {3.5}
$$
In Theorem 3.2, we provide the convergence rate of the estimator (3.5) by choosing the truncation parameter  $M$  in an optimal way.
Theorem 3.2 (Upper Bound on Moment Estimation without Sufficient Smoothness) Assuming that  $p > 2$ ,  $q < p < 2q$  and  $s < \frac{2dq - dp}{2pq}$ , we pick  $M = \Theta(n^{\frac{1}{p} - \frac{s}{d}})$ . Let  $\{x_i\}_{i=1}^n$  be  $n$  quadrature points independently and identically sampled from the uniform distribution on  $\Omega$  and  $\{y_i := f(x_i)\}_{i=1}^n$  be the corresponding  $n$  observations of  $f \in W^{s,p}(\Omega)$ . Then we have that the estimator  $\hat{H}_M^q$  constructed in (3.5) above satisfies
$$
\mathbb {E} _ {\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}} \left[ \left| \hat {H} _ {M} ^ {q} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}\right) - I _ {f} ^ {q} \right| \right] \lesssim n ^ {- q \left(\frac {s}{d} - \frac {1}{p}\right) - 1}. \tag {3.6}
$$
Proof Sketch The error can be decomposed into bias and variance parts. The bias part is caused by the truncation in our algorithm, which is controlled by the parameter  $M$  and can be bounded by  $\int_{\{x:|f(x)| > M\}} |f|^q dx$ . According to the Sobolev Embedding Theorem,  $W^{s,p}(\Omega)$  can be embedded in the space  $L^{p^*}$ , where  $\frac{1}{p^*} = \frac{1}{p} - \frac{s}{d}$ . As  $|f(x)| > M$  implies  $|f(x)|^q < M^{q-p^*}|f(x)|^{p^*}$ , the bias can be upper bounded by  $M^{q-p^*}$ . Similarly, the variance is controlled by  $M$  and can be upper bounded by  $M^{q-\frac{p^*}{2}}$ . Combining the bias and variance bound, we can bound the final error as  $M^{q-p^*} + \frac{M^{q-\frac{p^*}{2}}}{\sqrt{n}}$ . By selecting  $M = \Theta(n^{\frac{1}{p^*}}) = \Theta(n^{\frac{1}{p}-\frac{s}{d}})$ , we obtain the final convergence rate  $n^{-q(\frac{s}{d}-\frac{1}{p})-1}$ . A complete proof of Theorem 3.2 is given in Appendix C.2.
Remark 3.1 [64] has shown that the convergence rate of the optimal non-parametric regression-based estimation is  $n^{-\frac{s}{d} + \frac{1}{p} - \frac{1}{q}}$ , which is slower than the convergence rate of the truncated Monte Carlo estimator that we show above.
4 Adapting to the Noise Level: a Case Study for Linear Functional
In this section, we study how the regression-adjusted control variate adapts to different noise levels. Here we consider the linear functional, i.e. estimating a function's definite integral via low-noise observations at random points.
Problem Setup We consider estimating  $I_{f} = \int_{\Omega} f(x) dx$ , the integral of  $f$  over  $\Omega$ , for a fixed function  $f \in C^{s}(\Omega)$  with uniformly sampled quadrature points  $\{x_{i}\}_{i=1}^{n} \subset \Omega$ . On each quadrature point  $x_{i} (i = 1, \dots, n)$ , we have a noisy observation  $y_{i} := f(x_{i}) + \epsilon_{i}$ . Here the  $\epsilon_{i}$ 's are independently and identically distributed Gaussian noises sampled from  $\mathcal{N}(0, n^{-2\gamma})$ , where  $\gamma \in [0, \infty]$ .
4.1 Information-Theoretic Lower Bound on Mean Estimation
In this subsection, we present a minimax lower bound (Theorem 4.1) for all estimators  $\hat{H}:\Omega^n\times \mathbb{R}^n\to \mathbb{R}$  of the integral  $I_{f}$  of a function  $f\in C^{s}(\Omega)$  when one can only access noisy observations.
Theorem 4.1 (Lower Bound for Integral Estimation) Let  $\mathcal{H}_n^f$  denote the class of all the estimators that use  $n$  quadrature points  $\{x_i\}_{i=1}^n$  and noisy observations  $\{y_i = f(x_i) + \epsilon_i\}_{i=1}^n$  to estimate the integral of  $f$ , where  $\{x_i\}_{i=1}^n$  and  $\{\epsilon_i\}_{i=1}^n$  are independently and identically sampled from the uniform distribution on  $\Omega$  and the normal distribution  $\mathcal{N}(0, n^{-2\gamma})$  respectively. Assuming that  $\gamma \in [0, \infty]$  and  $s > 0$ , we have
$$
\inf  _ {\hat {H} \in \mathcal {H} _ {n} ^ {f}} \sup  _ {f \in C ^ {s} (\Omega)} \mathbb {E} _ {\{x _ {i} \} _ {i = 1} ^ {n}, \{y _ {i} \} _ {i = 1} ^ {n}} \left[ \left| \hat {H} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}\right) - I _ {f} \right| \right] \gtrsim n ^ {\max  \{- \frac {1}{2} - \gamma , - \frac {1}{2} - \frac {s}{d} \}}. \tag {4.1}
$$
Remark 4.1 Functional estimation is a well-studied problem in the literature of nonparametric statistics. However, current information-theoretic lower bounds for functional estimation [51, 52, 53, 56, 57, 58, 65, 68] assume a constant level of noise on the observed function values. One essential idea for proving these lower bounds is to leverage the existence of the observational noise, which enables us to upper bound the amount of information required to distinguish between two reduced hypotheses. In contrast, we provide a minimax lower bound that is applicable for noises at any level by constructing two priors with overlapping support and assigning distinct probabilities to the corresponding Bernoulli random variables, which separates the two hypotheses. A comprehensive proof of Theorem 4.1 is given in Appendix D.2.
4.2 Optimal Nonparametric Regression-Adjusted Quadrature Rule
In the discussion below, we use the nearest-neighbor method as an example. For any  $k \in \{1, 2, \dots, \frac{n}{2}\}$ , the  $k$ -nearest neighbor estimator  $\hat{f}_{k\text{-NN}}(z) \coloneqq \frac{1}{k} \sum_{j=1}^{k} y_{i_j(z)}$ , where  $\{x_{i_j(z)}\}_{j=1}^{\frac{n}{2}}$  is a permutation of the quadrature points  $\{x_i\}_{i=1}^{\frac{n}{2}}$  such that  $\| x_{i_1(z)} - z \| \leq \| x_{i_2(z)} - z \| \leq \dots \leq \| x_{i_{\frac{n}{2}}(z)} - z \|$  holds for any  $z \in \Omega$ . Moreover, we use  $T_{k,z} \coloneqq \{x_{i_j(z)}\}_{j=1}^k$  to denote the collection of the  $k$  nearest neighbors of  $z$  among  $\{x_i\}_{i=1}^{\frac{n}{2}}$  for any  $z \in \Omega$ . For any  $1 \leq i \leq \frac{n}{2}$ , we take  $D_i \subset \Omega$  to be the region formed by all the points whose  $k$  nearest neighbors contain  $x_i$ , i.e.,  $D_i \coloneqq \{z \in \Omega : x_i \in T_{k,z}\}$ . Our estimator  $\hat{H}_{k\text{-NN}}$  can be formally represented as
$$
\hat {H} _ {k \mathrm {- N N}} \Big (\{x _ {i} \} _ {i = 1} ^ {n}, \{y _ {i} \} _ {i = 1} ^ {n} \Big) = \underbrace {\sum_ {i = 1} ^ {\frac {n}{2}} \frac {V (D _ {i})}{k} y _ {i}} _ {\int_ {\Omega} \hat {f} _ {k \mathrm {- N N}} (x) d x} + \underbrace {\frac {2}{n} \sum_ {i = \frac {n}{2} + 1} ^ {n} y _ {i} - \frac {2}{n} \sum_ {i = \frac {n}{2} + 1} ^ {n} \left(\frac {1}{k} \sum_ {j = 1} ^ {\frac {n}{2}} \mathbb {1} \{x _ {i} \in D _ {j} \} y _ {j}\right)} _ {\frac {2}{n} \sum_ {i = \frac {n}{2} + 1} ^ {n} \left(y _ {i} - \hat {f} _ {k \mathrm {- N N}} (x _ {i})\right)}.
$$
In the following theorem, we present an upper bound on the expected risk of the estimator  $\hat{H}_{k\text{-NN}}$ :
Theorem 4.2 (Matching Upper Bound for Integral Estimation) Let  $\{x_{i}\}_{i = 1}^{n}$  be  $n$  quadrature points independently and identically sampled from the uniform distribution on  $\Omega$  and  $\{y_{i}\coloneqq f(x_{i}) + \epsilon_{i}\}_{i = 1}^{n}$  be the corresponding  $n$  noisy observations of  $f\in C^s (\Omega)$ , where  $\{\epsilon_i\}_{i = 1}^n$  are independently and identically sampled from the normal distribution  $\mathcal{N}(0,n^{-2\gamma})$ . Assuming that  $\gamma \in [0,\infty ]$  and  $s\in (0,1)$ , we have that there exists  $k\in \mathbb{N}$  such that the estimator  $\hat{H}_{k - NN}$  constructed above satisfies
$$
\mathbb {E} _ {\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}} \left[ \left| \hat {H} _ {k - N N} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}\right) - I _ {f} \right| \right] \lesssim n ^ {\max  \left\{- \frac {1}{2} - \gamma , - \frac {1}{2} - \frac {s}{d} \right\}}. \tag {4.2}
$$
Remark 4.2 Our upper bound in Theorem 4.2 matches our minimax lower bound in Theorem 4.1, which indicates that the regression-adjusted quadrature rule associated with the nearest neighbor estimator is minimax optimal. When the noise level is high ( $\gamma < \frac{s}{d}$ ), the control variate helps
to improve the rate from  $n^{-\frac{1}{2}}$  (the Monte Carlo rate) to  $n^{-\frac{1}{2} - \gamma}$  via eliminating all the effects of simulating the smooth function. When the noise level is low ( $\gamma > \frac{s}{d}$ ), we show that our estimator  $\hat{H}_{k-NN}$  can achieve the optimal rate of quadrature rules [46]. We defer a complete proof of Theorem 4.2 to Appendix D.3.
In this paper, we have investigated whether a non-parametric regression-adjusted control variate can improve the rate of estimating functionals and its minimax optimality. Using the Sobolev Embedding Theorem, we discover that the existence of infinite variance rare and extreme events will change the answer to this question. We show that when infinite variance rare and extreme events are present, using a non-parametric machine learning algorithm as a control variate does not help to improve the convergence rate, and truncated Monte Carlo is minimax optimal. When the variance of the simulation problem is finite, using a regression-adjusted control variate via an optimal non-parametric estimator is minimax optimal.
The assumptions we made in this paper, such as boundedness of the domain  $\Omega$  and constraints on the parameters  $p$ ,  $q$ , might be too restrictive for some application scenarios. We left relaxations of these assumptions as future work. One other potential direction is to investigate how to combine importance sampling with regression-adjusted control variates. Also, the study of how regression-adjusted control variates adapt to the noise level for non-linear functionals [62, 63] may be of interest. Moreover, another intriguing project is to analyze how the data distribution's information [3, 7] can be used to achieve both better computational trackability and convergence rate [8].
Jose Blanchet is supported in part by the Air Force Office of Scientific Research (AFOSR) under award number FA9550-20-1-0397 and the National Science Foundation (NSF) under award number DMS-1915967. Haoxuan Chen is supported by the T. S. Lo Graduate Fellowship Fund. Yiping Lu is supported by the Stanford Interdisciplinary Graduate Fellowship (SIGF). Lexing Ying is supported by the National Science Foundation (NSF) under award number DMS-2011699 and DMS-2208163.
[1] Søren Asmussen and Peter W Glynn. Stochastic simulation: algorithms and analysis, volume 57. Springer, 2007.
[2] Russell Davidson and James G MacKinnon. Regression-based methods for using control variates in monte carlo experiments. Journal of Econometrics, 54(1-3):203-222, 1992.
[3] Chris Oates and Mark Girolami. Control functionals for quasi-monte carlo integration. In Artificial Intelligence and Statistics, pages 56-65. PMLR, 2016.
[4] Fred J Hickernell, Christiane Lemieux, and Art B Owen. Control variates for quasi-monte carlo. Statistical Science, 20(1):1 - 31, 2005.
[5] Roland Assaraf and Michel Caffarel. Zero-variance principle for monte carlo algorithms. Physical review letters, 83(23):4682, 1999.
[6] Antonietta Mira, Reza Solgi, and Daniele Imparato. Zero variance markov chain monte carlo for bayesian estimators. Statistics and Computing, 23:653-662, 2013.
[7] Chris J Oates, Mark Girolami, and Nicolas Chopin. Control functionals for monte carlo integration. Journal of the Royal Statistical Society. Series B (Statistical Methodology), pages 695-718, 2017.
[8] Chris J Oates, Jon Cockayne, François-Xavier Briol, and Mark Girolami. Convergence rates for a class of estimators based on stein's method. Bernoulli, 25(2):1141 - 1159, 2019.
[9] Leah F South, CJ Oates, A Mira, and C Drovandi. Regularised zero-variance control variates. arXiv preprint arXiv:1811.05073, 2018.