SlowGuess/NeurIPS_2023_OCR huggingface dataset:
When can Regression-Adjusted Control variates Help? Rare Events, Sobolev Embedding and Minimax Optimality
Department of MS&E and ICME
Haoxuan Chen
ICME
haoxuanc@stanford.edu
Yiping Lu
yiping.lu@nyu.edu
Lexing Ying
Department of Mathematics and ICME
lexing@stanford.edu
This paper studies the use of a machine learning-based estimator as a control variate for mitigating the variance of Monte Carlo sampling. Specifically, we seek to uncover the key factors that influence the efficiency of control variates in reducing variance. We examine a prototype estimation problem that involves simulating the moments of a Sobolev function based on observations obtained from (random) quadrature nodes. Firstly, we establish an information-theoretic lower bound for the problem. We then study a specific quadrature rule that employs a nonparametric regression-adjusted control variate to reduce the variance of the Monte Carlo simulation. We demonstrate that this kind of quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under a sufficient smoothness assumption. Due to the Sobolev Embedding Theorem, the sufficient smoothness assumption eliminates the existence of rare and extreme events. Finally, we show that, in the presence of rare and extreme events, a truncated version of the Monte Carlo algorithm can achieve the minimax optimal rate while the control variate cannot improve the convergence rate.
In this paper, we consider a nonparametric quadrature rule on (random) quadrature points based on regression-adjusted control variate [1, 2, 3, 4]. To construct the quadrature rule, we partition our available data into two halves. The first half is used to construct a nonparametric estimator, which is then utilized as a control variate to reduce the variance of the Monte Carlo algorithm implemented over the second half of our data. Traditional and well-known results [1, Chapter 5.2] show that the optimal linear control variate can be obtained via Ordinary Least Squares regression. In this paper, we investigate a similar idea for constructing a quadrature rule [3, 5, 6, 7, 8, 9, 10], which uses a non-parametric machine learning-based estimator as a regression-adjusted control variate. We aim to answer the following two questions:
Is using optimal nonparametric machine learning algorithms to construct control variates an optimal way to improve Monte Carlo methods? What are the factors that determine the effectiveness of the control variate?
Figure 1: According to the Sobolev Embedding Theorem [11], the Sobolev space  $W^{s,p}$  can be embedded in  $L^{p^*}$ , where  $\frac{1}{p^*} = \frac{1}{p} - \frac{s}{d}$ . When  $s$  is large enough, as shown in (a), the smoothness assumption can rule out the existence of rare and extreme events. When  $s$  is not sufficiently large, specifically  $s < \frac{2dq - dp}{2pq}$ , there may exist a peak (a.k.a rare and extreme event) that makes the Monte Carlo simulation hard. Under such circumstances, the function's  $2q$ -th moment is unbounded.
To understand the two questions, we consider a basic but fundamental prototype problem of estimating moments of a Sobolev function from its values observed on (random) quadrature nodes, which has a wide range of applications in Bayesian inference, the study of complex systems, computational physics, and financial risk management [1]. Specifically, we estimate the  $q$ -th moment  $\int_{\Omega} f(x)^{q} dx$  of  $f$  based on values  $f(x_{1}), \dots, f(x_{n})$  observed on  $n$  (random) quadrature nodes  $x_{1}, \dots, x_{n} \in \Omega$  for a function  $f$  in the Sobolev space  $W^{s,p}(\Omega)$ , where  $\Omega \subset \mathbb{R}^d$ . The parameter  $q$  here is introduced to characterize the rare events' extremeness for estimation. To verify the effectiveness of the nonparametric regression adjusted quadrature rule, we first study the statistical limit of the problem by providing a minimax information-theoretic lower bound of magnitude  $n^{\max \left\{(\frac{1}{p} - \frac{s}{d}) q - 1, -\frac{s}{d} - \frac{1}{2}\right\}}$ .
We also provide matching upper bounds for different levels of function smoothness. Under the sufficient smoothness assumption that  $s > \frac{d(2q - p)}{2pq}$ , we find that the non-parametric regression adjusted control variate  $\hat{f}$  can improve the rate of classical Monte Carlo algorithm and help us attain a minimax optimal upper bound. In (3.4) below, we bound variance  $\int_{\Omega}(f^q - \hat{f}^q)^2$  of the Monte Carlo target by the sum of the semi-parametric influence part  $\int_{\Omega} f^{2q-2}(f - \hat{f})^2$  and the propagated estimation error  $\int_{\Omega}(f - \hat{f})^{2q}$ . Although the optimal algorithm in this regime remains the same, we need to consider three different cases to derive an upper bound on the semi-parametric influence part, which is the main contribution of our proof. We propose a new proof technique that embeds the square of the influence function  $(q f^{q-1})^2$  and estimation error  $(f - \hat{f})^2$  in appropriate spaces via the Sobolev Embedding Theorem [11]. The two norms used for evaluating  $(f^{q-1})^2$  and  $(f - \hat{f})^2$  should be dual norms of each other. Also, we should select the norm for evaluating  $(f - \hat{f})^2$  in a way that it's easy to estimate  $f$  under the selected norm, which helps us control the error induced by  $(f - \hat{f})^2$ . A detailed explanation of how to select the proper norms in different cases via the Sobolev Embedding Theorem is exhibited in Figure 2. In the first regime when  $s > \frac{d}{p}$ , we can directly embed  $f$  in  $L^\infty(\Omega)$  and attain a final convergence rate of magnitude  $n^{-\frac{s}{d} - \frac{1}{2}}$ . For the second regime when  $\frac{d(2q - p)}{p(2q-2)} < s < \frac{d}{p}$ , the smoothness parameter  $s$  is not large enough to ensure that  $f \in L^\infty(\Omega)$ . Thus, we evaluate the estimation error  $(f - \hat{f})^2$  under the  $L^{\frac{p}{2}}$  norm and embed the square of the influence function  $(q f^{q-1})^2$  in the dual space of  $L^{\frac{p}{2}}(\Omega)$ . Here the validity of such embedding is ensured by the lower bound  $\frac{d(2q - p)}{p(2q-2)}$  on  $s$ . Moreover, the semi-parametric influence part is still dominant in the second regime, so the final convergence rate is the same as that of the first case. In the third regime, when  $\frac{d(2q - p)}{2pq} < s < \frac{d(2q - p)}{p(2q-2)}$ , the semi-parametric influence no longer dominates and the final converge rate transits from  $n^{-\frac{s}{d} - \frac{1}{2}}$  to  $n^{q(\frac{1}{p} - \frac{s}{d}) - 1}$ .
When the sufficient smoothness assumption breaks, i.e.  $s < \frac{d(2q - p)}{2pq}$ , according to the Sobolev Embedding Theorem [11], the Sobolev space  $W^{s,p}$  is embedded in  $L^{\frac{dp}{d - sp}}$  and  $\frac{dp}{d - sp} < 2q$ . This indicates that rare and extreme events might be present, and they are not even guaranteed to have bounded  $L^{2q}$  norm, which makes the Monte Carlo estimate of the  $q$ -th moment have infinite variance. Under this scenario, we consider a truncated version of the Monte Carlo algorithm, which can be proved to attain the minimax optimal rate of magnitude  $n^q\left(\frac{1}{p} -\frac{s}{d}\right) - 1$ . In contrast, the usage of
regression-adjusted control variates does not improve the convergence rate under this scenario. Our results reveal how the existence of rare events will change answers to the questions raised at the beginning of the section.
We also use the estimation of a linear functional as an example to investigate the algorithm's adaptivity to the noise level. In this paper, we provide minimax lower bounds for estimating the integral of a fixed function with a general assumption on the noise level. Specifically, we consider all estimators that have access to observations  $\{x_{i},f(x_{i}) + \epsilon_{i}\}_{i = 1}^{n}$  of some function  $f$  that is  $s$ -Hölder smooth, where  $x_{i}\stackrel {\mathrm{i.i.d}}{\sim}\mathrm{Uniform}([0,1]^{d})$  and  $\epsilon_i\stackrel {\mathrm{i.i.d}}{\sim}n^{-\gamma}\mathcal{N}(0,1)$  for some  $\gamma >0$ . Based on the method of two fuzzy hypotheses, we present a lower bound of magnitude  $n^{\max}\{-\frac{1}{2} -\gamma , - \frac{1}{2} -\frac{s}{d}\}$ , which exhibits a smooth transition from the Monte Carlo rate to the Quasi-Monte Carlo rate. At the same time, our information-theoretic lower bound also matches the upper bound built for quadrature rules taking use of non-parametric regression-adjusted control variates.
Regression-Adjusted Control Variate The control variate method is a technique used for variance reduction in Monte-Carlo simulation. Consider the task of estimating the expectation  $\mathbb{E}X$  for some random variable  $X$ . The idea of control variate method is to introduce another random variable  $Y$  correlated with the random variable  $X$ , such that the random variable  $X - Y$  has smaller variance than  $X$ . Since  $\mathbb{E}X = \mathbb{E}[X - Y] + \mathbb{E}[Y]$  and  $\mathbb{E}[Y]$  is deterministic, one may obtain a variance reduced estimator of  $\mathbb{E}[X]$  by summing up  $\mathbb{E}[Y]$  and an empirical estimate of  $\mathbb{E}[X - Y]$ . Such a random variable  $Y$  is called a control variate. Regression-adjusted control variate, in particular, refers to the case when  $Y$  is obtained by applying regression methods to observed data samples of  $X$ .
Regression-adjusted control variates have shown both theoretical and empirical improvements in a wide range of applications, including the construction of confidence intervals [12, 13], randomized trace-estimation [14, 15], dimension reduction [16], causal inference [17], light transport simulation [18], MCMC simulation [19], estimation of the normalizing factor [10] and gradient estimation [20, 21]. It is also used as a technique for proving the approximation bounds on two-layer neural networks in the Barron space [22].
Regarding literature most related to our work, we mention [3, 7, 8, 10], which also study the theoretical properties of nonparametric control variate estimator. However, the theoretical analysis in [3, 7] does not provide a specific convergence rate in the Reproducing Kernel Hilbert Space, which requires a high level of smoothness for the underlying function. In contrast to prior work, our research delves into the effectiveness of a non-parametric regression-adjusted control variate in boosting convergence rates across various degrees of smoothness assumptions and identifies the key factor that determines the efficacy of these control variates.
Quadrature Rule There is a long literature on building quadrature rules in the Reproducing Kernel Hilbert Space, including Bayes-Hermite quadrature [23, 24, 25, 26, 27], determinantal point processes [28, 29, 30, 31], Nyström approximation [32, 33], kernel herding[34, 35, 36] and kernel thinning [37, 38, 39]. Nevertheless, the quadrature points chosen in these studies all have the ability to reconstruct the function's information, which results in a suboptimal rate for estimating the moments.
Functional Estimation There are also lines of research that investigated the optimal rates of estimating both linear [8, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50] and nonlinear [51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64] functionals, such as integrals and the  $L^q$  norm. However, as far as the authors know, previous works on this topic have assumed sufficient smoothness, which rules out the existence of rare and extreme events that are hard to simulate. Additionally, existing proof techniques are only applicable in scenarios where there is either no noise or a constant level of noise present. We have developed a novel and unified proof technique that leverages the method of two fuzzy hypotheses, which allows us to account for not only rare and extreme events but also different levels of noise.
- We determine all the regimes when a quadrature rule utilizing a nonparametric estimator as a control variate to reduce the Monte Carlo estimate's variance can boost the convergence rate of estimating the moments of a Sobolev function. Under sufficient smoothness assumption,
Figure 2: We summarize the minimax optimal rates and the corresponding optimal algorithms with respect to the function smoothness here. When the function is smooth enough, regression-adjusted control variates can improve the Monte Carlo rate. However, when there exist rare and extreme events that are hard to simulate, truncating the Monte Carlo estimate directly yields a minimax optimal algorithm. Above the transition point of algorithm selection is  $s = \frac{d(2q - p)}{2pq}$ , while the transition point of the optimal convergence rate is  $s = \frac{d(2q - p)}{p(2q - 2)}$ . To build the optimal convergence guarantee for any algorithm that utilizes a regression-adjusted control variate  $\hat{f}$ , we need to embed the square of the influence function  $(q f^{q - 1})^2$  in an appropriate space via the Sobolev Embedding Theorem and evaluate the estimation error  $(f - \hat{f})^2$  under the dual norm of the norm associated with the chosen space, which allows us to achieve optimal semi-parametric efficiency. Our selections of the metrics in different regimes are shown in this figure.
which rules out the existence of rare and extreme events due to the Sobolev Embedding Theorem, the regression-adjusted control variate improves the convergence rate and achieves the minimax optimal rate. Without the sufficient smoothness assumption, however, there may exist rare and extreme events that are hard to simulate. In this circumstance, we discover that a truncated version of the Monte Carlo method is minimax optimal, while regression-adjusted control variate can't improve the convergence rate.
- As far as the authors know, our paper is the first work considering this problem without assuming that the underlying function  $f$  is uniformly bounded. All previous work assumed that  $s > \frac{d}{p}$ , which implies  $f \in L^{\infty}(\Omega)$  and neglects the possibility of spike functions. As a result, they were unable to discover the transition between the two regimes described above. Under the assumption that  $s > \frac{d(2q - p)}{2pq}$ , the main difficulty in establishing the convergence guarantee lies in determining the right evaluation metric for function estimation. To select a suitable metric, we introduce a new proof technique by embedding the influence function into an appropriate space via the Sobolev Embedding Theorem and evaluating the function estimation in the corresponding dual norm to achieve optimal semi-parametric efficiency. Our selection of the proper embedding metrics is shown in Figure 2.
- To study how the regression adjusted control variate adapts to the noise level, we examine the linear functionals, i.e. the definite integral. We prove that this method is minimax optimal regardless of the level of noise present in the observed data.
Let  $\| \cdot \|$  be the standard Euclidean norm and  $\Omega = [0,1]^d$  be the unit cube in  $\mathbb{R}^d$  for any fixed  $d \in \mathbb{N}$ . Also, let  $\mathbb{1} = \mathbb{1}\{\cdot\}$  denote the indicator function, i.e., for any event  $A$  we have  $\mathbb{1}\{A\} = 1$  if  $A$  is
true and  $\mathbb{1}\{A\} = 0$  otherwise. For any region  $R\subseteq \Omega$ , we use  $V(R)\coloneqq \int_{\Omega}\mathbb{1}\{x\in R\} dx$  to denote the volume of  $R$ . Let  $C(\Omega)$  denote the space of all continuous functions  $f:\Omega \to \mathbb{R}$  and  $\lfloor \cdot \rfloor$  be the rounding function. For any  $s > 0$  and  $f\in C(\Omega)$ , we define the Hölder norm  $\| \cdot \|_{C^s (\Omega)}$  by
$$
\| f \| _ {C ^ {s} (\Omega)} := \max  _ {| k | \leq \lfloor s \rfloor} \| D ^ {k} f \| _ {L ^ {\infty} (\Omega)} + \max  _ {| k | = \lfloor s \rfloor} \sup  _ {x, y \in \Omega , x \neq y} \frac {\left| D ^ {k} f (x) - D ^ {k} f (y) \right|}{\| x - y \| ^ {s - \lfloor s \rfloor}}. \tag {1.1}
$$
The corresponding Hölder space is defined as  $C^s(\Omega) \coloneqq \left\{ f \in C(\Omega) : \|f\|_{C^s(\Omega)} < \infty \right\}$ . When  $s = 0$ , we have that the two norms  $\| \cdot \|_{C^0(\Omega)}$  and  $\| \cdot \|_{L^\infty(\Omega)}$  are equivalent and  $C^0(\Omega) = L^\infty(\Omega)$ . Let  $\mathbb{N}_0 \coloneqq \mathbb{N} \cup \{0\}$  be the set of all non-negative integers. For any  $s \in \mathbb{N}_0$  and  $1 \leq p \leq \infty$ , we define the Sobolev space  $W^{s,p}(\Omega)$  by
$$
W ^ {s, p} (\Omega) := \left\{f \in L ^ {p} (\Omega): D ^ {\alpha} f \in L ^ {p} (\Omega), \forall \alpha \in \mathbb {N} _ {0} ^ {d} \text {s a t i s f y i n g} | \alpha | \leq s \right\}. \tag {1.2}
$$
Let  $(c)_+$  denote  $\max\{c, 0\}$  for any  $c \in \mathbb{R}$ . Fix any two non-negative sequences  $\{a_n\}_{n=1}^{\infty}$  and  $\{b_n\}_{n=1}^{\infty}$ . We write  $a_n \lesssim b_n$ , or  $a_n = O(b_n)$ , to denote that  $a_n \leq C b_n$  for some constant  $C$  independent of  $n$ . Similarly, we write  $a_n \gtrsim b_n$ , or  $a_n = \omega(b_n)$ , to denote that  $a_n \geq c b_n$  for some constant  $c$  independent of  $n$ . We use  $a_n = \Theta(b_n)$  to denote that  $a_n = O(b_n)$  and  $a_n = \omega(b_n)$ .
2 Information-Theoretic Lower Bound on Moment Estimation
Problem Setup To understand how the non-parametric regression-adjusted control variate improves the Monte Carlo estimator's convergence rate, we consider a prototype problem that estimates a function's  $q$ -th moment. For any fixed  $q \in \mathbb{N}$  and  $f \in W^{s,p}(\Omega)$ , we want to estimate the  $q$ -th moment  $I_{f}^{q} \coloneqq \int_{\Omega} f^{q}(x) dx$  with  $n$  random quadrature points  $\{x_{i}\}_{i=1}^{n} \subset \Omega$ . On each quadrature point  $x_{i}$ $(i = 1, \dots, n)$ , we can observe the function value  $y_{i} \coloneqq f(x_{i})$ .
In this section, we study the information-theoretic limit for the problem above via the method of two fuzzy hypotheses [65]. We have the following information-theoretic lower bound on the class  $\mathcal{H}_n^{f,q}$  that contains all estimators  $\hat{H}^q:\Omega^n\times \mathbb{R}^n\to \mathbb{R}$  of the  $q$  -th moment  $I_{f}^{q}$
Theorem 2.1 (Lower Bound on Estimating the Moment) When  $p > 2$  and  $q < p < 2q$ , let  $\mathcal{H}_n^f$  denote the class of all the estimators that use  $n$  quadrature points  $\{x_i\}_{i=1}^n$  and observed function values  $\{y_i = f(x_i)\}_{i=1}^n$  to estimate the  $q$ -th moment of  $f$ , where  $\{x_i\}_{i=1}^n$  are independently and identically sampled from the uniform distribution on  $\Omega$ . Then we have
$$
\inf  _ {\hat {H} ^ {q} \in \mathcal {H} _ {n} ^ {f, q}} \sup  _ {f \in W ^ {s, p} (\Omega)} \mathbb {E} _ {\{x _ {i} \} _ {i = 1} ^ {n}, \{y _ {i} \} _ {i = 1} ^ {n}} \left[ \left| \hat {H} ^ {q} \left(\left\{x _ {i} \right\} _ {i = 1} ^ {n}, \left\{y _ {i} \right\} _ {i = 1} ^ {n}\right) - I _ {f} ^ {q} \right| \right] \gtrsim n ^ {\max  \left\{- q \left(\frac {s}{d} - \frac {1}{p}\right) - 1, - \frac {1}{2} - \frac {s}{d} \right\}}. \tag {2.1}
$$
Proof Sketch Here we give a sketch for our proof of Theorem 2.1. Our proof is based on the method of two fuzzy hypotheses, which is a generalization of the traditional Le Cam's two-point method. In fact, each hypothesis in the generalized method is constructed via a prior distribution. In order to attain a lower bound of magnitude  $\Delta$  via the method of two fuzzy hypotheses, one needs to pick two prior distributions  $\mu_0, \mu_1$  on the Sobolev space  $W^{s,p}(\Omega)$  such that the following two conditions hold. Firstly, the estimators  $I_f^q$  differ by  $\Delta$  with constant probability under the two priors. Secondly, the TV distance between the two corresponding distributions  $\mathbb{P}_0$  and  $\mathbb{P}_1$  of data generated by  $\mu_0$  and  $\mu_1$  is of constant magnitude. In order to prove the two lower bounds given in (2.1), we pick two different pairs of prior distributions as follows:
Below we set  $m = \Theta(n^{\frac{1}{d}})$  and divide the domain  $\Omega$  into  $m^d$  small cubes  $\Omega_1, \Omega_2, \dots, \Omega_{m^d}$ , each of which has side length  $m^{-1}$ . For any  $p \in (0, 1)$ , we use  $v_p, w_p$  to denote the discrete random variables satisfying  $\mathbb{P}(v_p = 0) = \mathbb{P}(w_p = -1) = p$  and  $\mathbb{P}(v_p = 1) = \mathbb{P}(w_p = 1) = 1 - p$ .
(I) For the first lower bound in (2.1), we construct some bump function  $g \in W^{s,p}(\Omega)$  satisfying  $\operatorname{supp}(g) \subseteq \Omega_1$  and  $I_g^q = \int_{\Omega_1} g(x) dx = \Theta(m^{q(-s + \frac{d}{p}) - d})$ . Now let's take some sufficiently small constant  $\epsilon \in (0,1)$  and pick  $\mu_0, \mu_1$  to be discrete measures supported on the two finite sets  $\left\{v_{\frac{1 + \epsilon}{2}}g\right\}$  and  $\left\{v_{\frac{1 - \epsilon}{2}}g\right\}$ . On the one hand, the difference between the  $q$ -th moments under  $\mu_0$  and  $\mu_1$  can be
lower bounded by  $\Theta (n^{q(\frac{1}{p} -\frac{s}{d}) - 1})$  with constant probability. On the other hand,  $KL(\mathbb{P}_0\| \mathbb{P}_1)$  can be upper bounded by the KL divergence between  $v_{\frac{1 + \epsilon}{2}}$  and  $v_{\frac{1 - \epsilon}{2}}$ , which is of constant magnitude.
(II) For the second lower bound in (2.1), we set  $M > 0$  to be some sufficiently large constant and  $\kappa = \Theta\left(\frac{1}{\sqrt{n}}\right)$ . For any  $1 \leq j \leq m^{d}$ , we construct bump functions  $f_{j} \in W^{s,p}(\Omega)$  satisfying  $\mathrm{supp}(f_{j}) \subseteq \Omega_{j}$  and  $I_{f_{j}}^{k} = \int_{\Omega_{j}} f_{j}(x) dx = \Theta(m^{-ks - d})$  for any  $1 \leq j \leq m^{d}$  and  $1 \leq k \leq s$ . Now let's pick  $\mu_{0}, \mu_{1}$  to be discrete measures supported on the two finite sets  $\left\{M + \sum_{j=1}^{m^{d}} w_{j}^{(0)} f_{j}\right\}$  and  $\left\{M + \sum_{j=1}^{m^{d}} w_{j}^{(1)} f_{j}\right\}$ , where  $\{w_{j}^{(0)}\}_{j=1}^{m^{d}}$  and  $\{w_{j}^{(1)}\}_{j=1}^{m^{d}}$  are independent and identical copies of  $w_{\frac{1+\kappa}{2}}$  and  $w_{\frac{1-\kappa}{2}}$  respectively. On the one hand, applying Hoeffding's inequality yields that the  $q$ -th moments under  $\mu_{0}$  and  $\mu_{1}$  differ by  $\Theta(n^{-\frac{s}{d}-\frac{1}{2}})$  with constant probability. On the other hand, note that  $KL(\mathbb{P}_0 \| \mathbb{P}_1)$  can be bounded by the KL divergence between two multivariate discrete distributions  $(w_{j_1}^{(0)}, \dots, w_{j_n}^{(0)})$  and  $(w_{j_1}^{(1)}, \dots, w_{j_n}^{(1)})$ , where  $\{w_{j_i}^{(0)}\}_{i=1}^n$  and  $\{w_{j_i}^{(1)}\}_{i=1}^n$  are independent and identical copies of  $w_{\frac{1+\kappa}{2}}$  and  $w_{\frac{1-\kappa}{2}}$  respectively. Hence,  $KL(\mathbb{P}_0 \| \mathbb{P}_1)$  is of constant magnitude.
Combining the two cases above gives us the minimax lower bound in (2.1). We defer a complete proof of Theorem 2.1 to Appendix B.2.
3 Minimax Optimal Estimators for Moment Estimation
This section is devoted to constructing the minimax optimal estimators of the  $q$ -th moment. We show that under the sufficient smoothness assumption, a regression-adjusted control variate is essential for building the minimax optimal estimators. However, when the given function is not sufficiently smooth, we demonstrate that a truncated version of the Monte Carlo algorithm is the minimax optimal, and control variates cannot give any improvement.
3.1 Sufficient Smoothness Regime: Non-parametric Regression-Adjusted Control Variate