# On a class of recursive estimators for spatially dependent observations

Mohamed El Machkouri, Lucas Reding

<figure><img src="image_1.png" /></figure>

## To cite this version:

Mohamed El Machkouri, Lucas Reding. On a class of recursive estimators for spatially dependent observations. Electronic Journal of Statistics, 2021, 15 (2), 10.1214/21-ejs1878. hal-04368133

HAL Id: hal-04368133

https://hal.science/hal-04368133v1

Submitted on 31 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# On a class of recursive estimators for spatially dependent observations

Mohamed El Machkouri and Lucas Reding

Laboratoire de Mathématiques Raphaël Salem (UMR CNRS 6085),
Université de Rouen Normandie, France

e-mail: mohamed.elmachkouri@univ-rouen.fr; lucas.reding@univ-rouen.fr

**Abstract:** We investigate the mean squared error and the asymptotic normality for a class of recursive kernel estimators based on a sample of spatially dependent observations. Our main result provides sufficient conditions for a spatial version of a recursive estimator introduced by Hall and Patil (1994) to satisfy a central limit theorem. The results are stated for strongly mixing random fields in the sense of Rosenblatt (1956) and for weakly dependent random fields in the sense of Wu (2005).

**MSC2020 subject classifications:** 60G60, 62G20, 62G07, 62G08, 62G05.

**Keywords and phrases:** Recursive estimator, asymptotic normality, quadratic mean error, random fields, strong mixing, weak dependence, density estimation, regression estimation, Lindeberg’s method, *m*-dependence, physical dependence measure.

Received October 2020.

## Contents

<table><tr><td>1 Introduction</td><td>4580</td></tr><tr><td>2 Main results</td><td>4584</td></tr><tr><td>3 Preliminary lemmas</td><td>4587</td></tr><tr><td>4 Proofs of the main results</td><td>4596</td></tr><tr><td>5 Appendix</td><td>4619</td></tr><tr><td>Acknowledgments</td><td>4621</td></tr><tr><td>References</td><td>4622</td></tr></table>

## 1. Introduction

In statistics, the nonparametric estimation of probability density functions of continuous random variables is a basic and central problem. From a given sample of observations, the main goal for a practitioner is to understand the mechanism from which the observations have been generated. In the last several decades, this question has attracted much attention among statisticians since it is of considerable interest in many applied fields such as forecasting, computer vision and machine learning. Among the plethora of nonparametric density estimators is the kernel density estimator introduced by Parzen [31] and Rosenblatt [34]
which received considerable attention in nonparametric estimation for time series. More precisely, if $(X_1, ..., X_n)$ is a sample (observations) drawn from some univariate distribution with an unknown probability density $f$ with respect to the Lebesgue measure on $\mathbb{R}$ then the Parzen-Rosenblatt density estimator of $f$ is defined for any positive integer $n$ and any $x$ in $\mathbb{R}$ by

$$f_n^{PR}(x) = \frac{1}{nh_n} \sum_{i=1}^{n} K\left(\frac{x - X_i}{h_n}\right) \quad (1.1)$$

where $K$ is a density function and the bandwidth $h_n$ is a positive parameter which converges to zero such that $nh_n$ goes to infinity. The bandwidth $h_n$ is the most dominant parameter in the kernel density estimator since it controls its amount of smoothness. In fact, if $h_n$ is small then the variance of the estimator is large while the bias is small. This leads to a nonsmooth estimated density. On the other, if $h_n$ is large then the estimated density will be much smoother (small variance) but with a large bias leading to an unsatisfactory estimation. So, in practice, a trade-of between the variance and the bias must be found and the number of publications which are devoted to this crucial question in the literature is very extensive and is still a subject of many works in the statistic community (see for example [8], [17], [22], [38]). From a theoretical but also practical point of view, it is important to investigate asymptotic properties of density estimators when the number $n$ of observations goes to infinity. For example, the consistency and the asymptotic normality of the estimator are very important in order to get pointwise estimation and confidence intervals for the target density $f$. In his seminal paper, Parzen [31] proved that when the observations $(X_1, ..., X_n)$ are i.i.d. and the bandwidth $h_n$ goes to zero such that $nh_n$ goes to infinity then $(nh_n)^{1/2}(f_n^{PR}(x_0) - E[f_n^{PR}(x_0)])$ converges in distribution to the normal law with zero mean and variance $f(x_0) \int_{\mathbb{R}} K^2(t)dt$ as $n$ goes to infinity and this result was extended by Wu and Mielniczuk [44] for causal linear processes with i.i.d. innovations and by Dedecker and Merlevède [11] for strongly mixing sequences. Previously, Bosq, Merlevède and Peligrad [6] established a central limit theorem for the kernel density estimator $f_n$ when the sequence $(X_i)_{i \in \mathbb{Z}}$ is assumed to be strongly mixing but the bandwidth parameter $h_n$ is assumed to satisfy $h_n \ge Cn^{-1/3} \log n$ (for some positive constant $C$) which is stronger than the bandwidth parameter assumption in [11], [31] and [44].

In many situations, practicians are also interested by the relationship between some predictors and a response. This is a natural question and a very important task in statistics. The objective is to find a relation between a pair of random variables $X$ (predictor) and $Y$ (response) using a given sample $(X_i, Y_i)_{1 \le i \le n}$ drawn from the unknown law of $(X, Y)$. A very popular tool to handle this problem is the kernel regression estimator introduced by Nadaraya [30] and Watson [41]. More formally, let $N$ be a positive integer and assume that $(X_i, Y_i)_{1 \le i \le n}$ are identically distributed $\mathbb{R}^N \times \mathbb{R}$-valued sequence of random variables such that $Y_i = R(X_i, \eta_i)$ where $R$ is an unknown functional and $(\eta_i)_{i \in \mathbb{Z}}$ are i.i.d. $\mathbb{R}^N$-valued random variables with zero mean and finite variance and independent of $(X_i)_{i \in \mathbb{Z}}$. Let $f$ be the marginal density function of $X_0$. If $r$ is the (unknown)
regression function defined for any $x$ in $\mathbb{R}^N$ by $r(x) = \mathbb{E}[R(x, \eta_0)]$ if $f(x) \neq 0$ and $r(x) = \mathbb{E}[Y_0]$ if $f(x) = 0$ then the Nadaraya-Watson regression estimator $r_n^{NW}$ of $r$ is defined for any $x$ in $\mathbb{R}^N$ by

$$r_n^{NW}(x) = \begin{cases} \frac{\sum_{i=1}^n Y_i K((x - X_i)/h_n)}{\sum_{i=1}^n K((x - X_i)/h_n)} & \text{if } \sum_{i=1}^n K((x - X_i)/h_n) \neq 0 \\ n^{-1} \sum_{i=1}^n Y_i & \text{else.} \end{cases} \quad (1.2)$$

The literature on the asymptotic properties of $r_n^{NW}$ for time series is very expansive. One can refer to Lu and Cheng [24], Masry and Fan [26], Robinson [33], Roussas [36] and many references therein. Kernel nonparametric methods are still very popular and fairly well established in the statistical community but despite their power, the data streams problem, which refers to data sets that continuously and rapidly grow over time, present new challenges. In order to handle such data sets, several recursive versions of the Parzen-Rosenblatt estimator (1.1) have been introduced (see for example [3], [12], [20], [43], [47]). For example, if $(w_k)_{k \ge 1}$ is a nonincreasing sequence of positive real numbers satisfying $\sum_{k \ge 1} w_k = \infty$ and $(h_k)_{k \ge 1}$ is a sequence of positive real numbers going to 0 as $n$ goes to infinity (bandwidth parameters) then the recursive kernel density estimator $f_n^{HP}$ of Hall and Patil [20] is defined by

$$f_n^{HP}(x) = \frac{1}{\sum_{k=1}^{n} w_k} \sum_{i=1}^{n} \frac{w_i}{h_i^d} K\left(\frac{x - X_i}{h_i}\right). \quad (1.3)$$

This estimator is recursive in the sense that it satisfies

$$f_{n+1}^{HP}(x) = (1 - \gamma_{n+1}) f_n^{HP}(x) + \gamma_{n+1} \tilde{f}_{n+1}(x) \quad (1.4)$$

where $\gamma_n := \frac{w_n}{\sum_{i=1}^n w_i}$ and $\tilde{f}_{n+1}(x) := \frac{1}{h_{n+1}^d} K\left(\frac{x - X_{n+1}}{h_{n+1}}\right)$.

Such a property endows recursive estimators with a decisive computational advantage because they can be easily updated as new data items arrive over time. More precisely, in order to obtain the estimation $f_{n+1}^{HP}(x)$ at time $n+1$, using the recursive equation (1.4), it is sufficient to combine the estimation $f_n^{HP}(x)$ at time $n$ (which is known at time $n+1$) with the estimation $\tilde{f}_{n+1}(x)$ at time $n+1$ based on the single observation $X_{n+1}$. In fact, a non-recursive estimator must be fully recomputed whenever a new observation is collected. This clearly represents a drawback in a data stream context compared to the recursive approach. The class (1.3) contains the recursive estimators introduced by Wolverton and Wagner [43] and Deheuvels [12] but also a renormalized version of the one introduced by Wegman and Davies [42] and another class of estimators introduced by Amiri [3]. It contains also the (non-recursive) Parzen-Rosenblatt estimator (1.1) when $h_i = h_n$ and $w_i = 1$ for any $1 \le i \le n$. In this work, our aim is to investigate asymptotic properties for a spatial version of the Hall and Patil estimator (1.3) in terms of mean squared error and asymptotic normality under weak and strong dependence conditions. The first studies
that focused on a recursive version of the Parzen-Rosenblatt estimator were presented by [12], [43] and [47] and later by [20], [25], [29] and many others. Actually, many papers in the literature are devoted to the asymptotic properties of recursive kernel density and regression estimators for i.i.d. observations. There are also some published papers on the asymptotic properties of recursive kernel density and regression estimators for dependent (weakly dependent and strongly mixing) data. One can refer for example to [4], [7], [18], [19], [25], [36], [39], [40] and others.

In our context, we deal with spatial data which is modelized using finite realizations of dependent random fields indexed by $\mathbb{Z}^d$ where $d$ is a positive integer. More precisely, let $N$ be a positive integer and let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space. We consider a stationary $\mathbb{R}^N$-valued random field $(X_k)_{k \in \mathbb{Z}^d}$ such that the law $\mu_0$ of $X_0$ is absolutely continuous with respect to the Lebesgue measure $\lambda_N$ on $\mathbb{R}^N$ and we denote by $f$ the probability density function of $\mu_0$ with respect to $\lambda_N$. Given two sub-$\sigma$-algebras $\mathcal{U}$ and $\mathcal{V}$ of $\mathcal{F}$, recall that the $\alpha$-mixing coefficient introduced by Rosenblatt [35] is defined by

$$ \alpha(\mathcal{U}, \mathcal{V}) = \sup\{|\mathbb{P}(A \cap B) - \mathbb{P}(A)\mathbb{P}(B)|, A \in \mathcal{U}, B \in \mathcal{V}\}. $$

Let $p$ be fixed in $[1, +\infty]$. The strong mixing coefficients $(\alpha_{1,p}(n))_{n \ge 0}$ associated to $(X_k)_{k \in \mathbb{Z}^d}$ are defined by

$$ \alpha_{1,p}(n) = \sup \{ \alpha(\sigma(X_k), \mathcal{F}_{\Gamma}), k \in \mathbb{Z}^d, \Gamma \subset \mathbb{Z}^d, |\Gamma| \le p, \rho(\Gamma, \{k\}) \ge n \} $$

where $|\Gamma|$ is the number of elements in $\Gamma$, the collection $\mathcal{F}_{\Gamma}$ is the $\sigma$-algebra $\sigma(X_k; k \in \Gamma)$ and the distance $\rho$ is defined for any subsets $\Gamma_1$ and $\Gamma_2$ of $\mathbb{Z}^d$ by $\rho(\Gamma_1, \Gamma_2) = \min\{|u - v|, u \in \Gamma_1, v \in \Gamma_2\}$ with $|u - v| = \max_{1 \le l \le d} |u_l - v_l|$ for any $u = (u_1, \dots, u_d)$ and $v = (v_1, \dots, v_d)$ in $\mathbb{Z}^d$. We say that the random field $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing if $\lim_{n \to \infty} \alpha_{1,p}(n) = 0$. Moreover, we are going to consider also Bernoulli fields defined for any $k \in \mathbb{Z}^d$ by

$$ X_k = G(\varepsilon_{k-u}; u \in \mathbb{Z}^d) \qquad (1.5) $$

where $G: (\mathbb{R}^m)^{\mathbb{Z}^d} \to \mathbb{R}^N$ is a measurable function, $(\varepsilon_k)_{k \in \mathbb{Z}^d}$ are i.i.d. $\mathbb{R}^m$-valued random variables and $m$ is a positive integer. The class of random fields that (1.5) represents is huge and it includes many commonly used linear and nonlinear processes (see Wu [46] for a review). Let $(\varepsilon'_k)_{k \in \mathbb{Z}^d}$ be an i.i.d. copy of $(\varepsilon_k)_{k \in \mathbb{Z}^d}$ and let $X_k^*$ be the coupled version of $X_k$ defined by $X_k^* = G(\varepsilon_{k-u}^*; u \in \mathbb{Z}^d)$ where $\varepsilon_k^* = \varepsilon_k$ if $k \ne 0$ and $\varepsilon_0^* = \varepsilon_0$. Note that $X_k^*$ is obtained from $X_k$ by replacing $\varepsilon_0$ by its copy $\varepsilon_0'$. For any positive integer $\ell$ and any $\mathbb{R}^\ell$-valued random variable $Z \in \mathbb{L}^p(\Omega, \mathcal{F}, \mathbb{P})$ with $p > 0$, we denote $\|Z\|_p := \mathbb{E}[\|Z\|^p]^{1/p}$ where $\| \cdot \|$ is the Euclidian norm on $\mathbb{R}^\ell$. Following Wu [45] and El Machkouri et al. [16], we define the physical dependence measure coefficient $\delta_{k,p} := \|X_k - X_k^*\|_p$ as soon as $X_k$ is $p$-integrable for $p \ge 2$. Physical dependence measure should be seen as a measure of the dependence of the function $G$ (defined in (1.5)) in the coordinate zero. In some sense, it quantifies the degree of dependence of outputs on inputs in physical systems and provide a natural framework for a limit
theory for stationary random fields (see [16]). In particular, it gives mild and
easily verifiable conditions (see condition (H2)(ii) below) because it is directly
related to the data-generating mechanism.

## 2. Main results

Let $\Lambda_0 = \emptyset$, $s_0 = 0 \in \mathbb{Z}^d$ and $\Lambda_n = \{s_1, \dots, s_n\} \subset \mathbb{Z}^d$ for $n \ge 1$. Let $(w_{s_n})_{n \ge 1}$ and $(h_{s_n})_{n \ge 1}$ be two nonincreasing sequences of positive real numbers such that $(w_{s_n} h_{s_n}^{-N})_{n \ge 1}$ is nondecreasing, $h_{s_n}$ goes to 0 as $n$ goes to infinity and $\sum_{n \ge 1} w_{s_n} = \infty$. Let also $K: \mathbb{R}^N \to \mathbb{R}_+$ be a function (called a kernel) such that $\int_{\mathbb{R}^N} K(t)dt = 1$ and $\sup_{x \in \mathbb{R}^N} K(x) < \infty$. Assume that $K$ is Lipschitz and satisfies $\lim_{\|x\| \to \infty} \|x\|K(x) = 0$, $\int_{\mathbb{R}^N} \|u\|^2 K(u)du < \infty$ where $\|\cdot\|$ is the usual norm on $\mathbb{R}^N$ and $\int_{\mathbb{R}^N} u_i K(u)du = 0$ for any $1 \le i \le N$. Let $\Phi: \mathbb{R} \to \mathbb{R}$ be a measurable function such that $\mathbb{E}[|\Phi(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\Phi(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le Ch_{s_n}^N$ for some $\theta > 0$ and $C > 0$ and assume that $u \mapsto \mathbb{E}[|\Phi(Y_0)|^2|X_0 = u|$ is continuous. One can notice that $\mathbb{E}[|\Phi(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le Ch_{s_n}^N$ is satisfied when $u \mapsto \mathbb{E}[|\Phi(Y_0)|^{2+\theta}|X_0 = u|$ is continuous (see Lemma 2 below). Let $(\eta_k)_{k \in \mathbb{Z}^d}$ be i.i.d. $\mathbb{R}^N$-valued random variables with zero mean and finite variance and independent of $(X_k)_{k \in \mathbb{Z}^d}$ and consider the regression model $Y_{s_i} = R(X_{s_i}, \eta_{s_i})$ for any $1 \le i \le n$ where $R$ is an unknown functional. For any $x \in \mathbb{R}^N$, we denote $f_{\Phi}(x) = r_{\Phi}(x)f(x)$ where $r_{\Phi}(x) = \mathbb{E}[\Phi(Y_0)|X_0 = x] = \mathbb{E}[\Phi(R(x, \eta_0))]$ if $f(x) \ne 0$ and $r_{\Phi}(x) = \mathbb{E}[\Phi(Y_0)]$ if $f(x) = 0$ and we consider the estimator $f_{n,\Phi}$ of $f_{\Phi}$ defined by

$$f_{n,\Phi}(x) = \left(\sum_{i=1}^{n} w_{s_i}\right)^{-1} \sum_{j=1}^{n} w_{s_j} h_{s_j}^{-N} \Phi(Y_{s_j}) K_{s_j}(x, X_{s_j}) \quad (2.1)$$

where $K_{s_j}(x, v) = K((x - v)/h_{s_j})$ for any $v \in \mathbb{R}^N$ and any $1 \le j \le n$. One can notice that if $\Phi(u) = 1$ for any $u \in \mathbb{R}$ then $f_{n,\Phi}$ reduces to the spatial version $f_{n,1}$ of the recursive kernel density estimator of $f$ introduced by Hall and Patil [20] and defined for any $x \in \mathbb{R}^N$ by

$$f_{n,1}(x) = \left(\sum_{i=1}^{n} w_{s_i}\right)^{-1} \sum_{j=1}^{n} w_{s_j} h_{s_j}^{-N} K_{s_j}(x, X_{s_j}). \quad (2.2)$$

Moreover, for particular choices of the weights $(w_{s_n})_{n \ge 1}$, the estimator (2.2) reduces to the recursive estimators introduced by [3], [4], [12] or [43]. In particular, one can check that $f_{n,\Phi}$ satisfies the following recursive equation

$$f_{n,\Phi}(x) = (1 - \rho_n) f_{n-1,\Phi}(x) + \rho_n h_{s_n}^{-N} \Phi(Y_{s_n}) K_{s_n}(x, X_{s_n}) \quad (2.3)$$

where $\rho_n = \frac{w_{s_n}}{\sum_{i=1}^{n} w_{s_i}}$. Equation (2.3) is the spatial version of the recursive equation (1.4). It lays emphasis on that the update of $f_{n,\Phi}$ at time $n$ can be done from $f_{n-1,\Phi}$ at time $n-1$ and the new single observation $X_{s_n}$. This is a definitive advantage over the spatial version of the non-recursive Parzen-Rosenblatt
estimator $f_n^{PR}$ defined by (1.1) since it is necessary to consider the whole sample $(X_{s_1}, ..., X_{s_n})$ in order to compute $f_n^{PR}$ at any time $n$. In this work, we consider also the following class of spatial semi-recursive kernel regression estimator $r_{n,\Phi}$ of $r_{\Phi}$ defined for any $x$ in $\mathbb{R}^N$ by

$$r_{n,\Phi}(x) = \begin{cases} \frac{f_{n,\Phi}(x)}{f_{n,1}(x)} & \text{if } \sum_{j=1}^{n} w_{s_j} h_{s_j}^{-N} K_{s_j}(x, X_{s_j}) \neq 0 \\ n^{-1} \sum_{i=1}^{n} Y_{s_i} & \text{else} \end{cases} \quad (2.4)$$

which contains the first two semi-recursive kernel regression estimators introduced by Ahmad and Lin [2] and Devroye and Wagner [13] for time series (i.e. for $d = 1$) but also the class of semi-recursive kernel regression estimators considered by Amiri [4]. Since $r_{n,\Phi}$ is defined from $f_{n,\Phi}$ and $f_{n,1}$, it inherits the good properties in term of computation time of the recursive estimators $f_{n,\Phi}$ and $f_{n,1}$ and consequently, in a data stream setting, it has a decisive advantage over the spatial version of the non-recursive Nadaraya-Watson estimator defined by (1.2).

Now, we are going to present our main contributions. For $j \in \{2, 4\}$, we adopt the notation

$$\nu_j(\theta) = \mathbb{1}_{\|\Phi\|_{\infty} < \infty} + \frac{\theta}{j + \theta} \mathbb{1}_{\|\Phi\|_{\infty} = \infty} \quad (2.5)$$

and for any sequences $(a_n)_{n \ge 1}$ and $(b_n)_{n \ge 1}$ of real positive numbers, we denote $a_n \le b_n$ if and only if there exists $\kappa > 0$ (not depending on $n$) such that $a_n \le \kappa b_n$. Recall that $(w_{s_n})_{n \ge 1}$ and $(h_{s_n})_{n \ge 1}$ are two nonincreasing sequences of positive real numbers such that $(w_{s_n} h_{s_n}^{-N})_{n \ge 1}$ is nondecreasing, $h_{s_n}$ goes to 0 as $n$ goes to infinity and $\sum_{n \ge 1} w_{s_n} = \infty$ and keep in mind that $K: \mathbb{R}^N \to \mathbb{R}_+$ is a function (kernel) such that $\int_{\mathbb{R}^N} K(t)dt = 1$ and $|K|_{\infty} := \sup_{t \in \mathbb{R}^N} K(t) < \infty$.

For any integer $n \ge 1$ and any $(p, q) \in \mathbb{Z}^2$, we denote also

$$A_{n,p,q} = (nh_{s_n}^p w_{s_n}^q)^{-1} \sum_{i=1}^{n} h_{s_i}^p w_{s_i}^q$$

and we consider the following assumptions:

(H1) There exists $(\beta_{0,1}, \beta_{-N,2}) \in (\mathbb{R}_+^*)^2$ such that $\lim_{n \to \infty} A_{n,0,1} = \beta_{0,1}$ and $\lim_{n \to \infty} A_{n,-N,2} = \beta_{-N,2}$.

(H2) There exist $\theta > 0$ such that $\mathbb{E}[|\Phi(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\Phi(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$, and $\tau \in [1 - \nu_4(\theta), 1]$ such that $\lim_{n \to \infty} nh_{s_n}^{N(1+\frac{d\nu_2(\theta)(\nu_4(\theta)+\tau-1)}{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)})} = \infty$ and $h_{s_n}^{N(1-\tau)} \sum_{i=1}^n w_{s_i}^2 \le nw_{s_n}^2$. Moreover, one of the following condition holds:

(i) $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing and $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d\nu_2(\theta)}{\nu_4(\theta)+\tau-1}} \alpha_{1,\infty}^{\nu_2(\theta)}(|k|) < \infty$,

(ii) $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5) and

$$\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d(N+2\nu_2(\theta)+2N(\nu_4(\theta)+\tau-1))}{2N(\nu_4(\theta)+\tau-1)}} \delta_{k,2}^{\nu_2(\theta)} < \infty.$$
(H3) (i) The function $u \mapsto \mathbb{E}[|\Phi(Y_0)|^2|X_0 = u]$ is continuous.

(ii) The function $f_{\Phi}$ is twice differentiable with bounded second derivatives.

(iii) For any $k \in \mathbb{Z}^d \setminus \{0\}$, the law of $(X_0, X_k)$ is absolutely continuous with respect to the Lebesgue measure on $\mathbb{R}^N \times \mathbb{R}^N$ and there exists $c > 0$ such that $\sup_{k \in \mathbb{Z}^d \setminus \{0\}} |f_{0,k}(x, y) - f(x)f(y)| \le c$ for any $(x, y) \in \mathbb{R}^d \times \mathbb{R}^N$ where $f_{0,k}$ is the joint density function of $(X_0, X_k)$.

Assumptions (H1) and (H3) are classical in the context of recursive kernel estimators (see [4], [25], [28], [42] and many others). In (H2), we assume that the bandwidth parameter $h_{s_n}$ satisfies a condition slightly stronger than the usual minimal condition assumed in the non recursive i.i.d. setting (i.e. $nh_{s_n}^N \to \infty$). However, this fact seems to be inherent to the case of recursive estimators since a condition like $nh_{s_n}^{N(1+\epsilon)} \to \infty$ for some $\epsilon > 0$ is assumed in many contributions for dependent data (see for example [4], [1], [25], [28] or [42]).

For any $x$ in $\mathbb{R}^N$, we denote

$$ \sigma_{\Phi}^{2}(x) := \beta_{0,1}^{-2}\beta_{-N,2}\mathbb{E}[|\Phi(Y_0)|^2|X_0 = x]f(x) \int_{\mathbb{R}^N} K^2(t)dt. \quad (2.6) $$

Our first result gives the asymptotic variance of the estimator $f_{n,\Phi}$ defined by (2.1).

**Proposition 1.** Assume that (H1) and (H3) hold and there exists $\theta > 0$ such that $\mathbb{E}[|\Phi(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\Phi(Y_0)|^{2+\theta}K_{s_n}(x, X_0)] \le h_{s_n}^N$. If there exists $\tau \in ]1 - \nu_4(\theta), 1]$ such that $h_{s_n}^{N(1-\tau)} \sum_{i=1}^n w_{s_i}^2 \le nw_{s_n}^2$ and one of the following conditions is satisfied:

(i) $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing and $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d\nu_2(\theta)}{\nu_4(\theta)+\tau-1}} \alpha_{1,1}^{\nu_2(\theta)} (|k|) < \infty$

(ii) $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5) and

$$ \sum_{k \in \mathbb{Z}^d} |k|^{\frac{d(N+2\nu_2(\theta)+2N(\nu_4(\theta)+\tau-1))}{2N(\nu_4(\theta)+\tau-1)}}} \delta_{k,2}^{\nu_2(\theta)} < \infty $$

where $\nu_2(\theta)$ and $\nu_4(\theta)$ are defined by (2.5) then for any $x \in \mathbb{R}$,

$$ \lim_{n \to \infty} |nh_{s_n}^N \mathbb{V}[f_{n,\Phi}(x)] - \sigma_{\Phi}^2(x)| = 0 \quad (2.7) $$

where $\sigma_{\Phi}^2(x)$ is defined by (2.6).

We obtain also the convergence to zero of the mean square error of $f_{n,\Phi}$.

**Proposition 2.** Assume that $f_{\Phi}$ is twice differentiable with bounded second derivatives.

Then, for any $x \in \mathbb{R}^N$, $|\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x)| \le (\sum_{i=1}^n w_{s_i})^{-1} \sum_{i=1}^n w_{s_i} h_{s_i}^2 = o(1)$. So, if $\max\{A_{n,0,1}^{-1}, A_{n,2,1}\} \le 1$ then $|\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x)| \le h_{s_n}^2$ and, under assumptions of Proposition 1, we get $\mathbb{E}[(f_{n,\Phi}(x) - f_{\Phi}(x))^2] \le n^{-\frac{4}{4+N}}$ for $h_{s_n} = n^{-\frac{1}{4+N}}$.

The main contribution of this paper is the following central limit theorem.
**Theorem 1.** If (H1), (H2) and (H3) hold then for any $x \in \mathbb{R}^N$,

$$ \sqrt{nh_{s_n}^N} (f_{n,\Phi}(x) - \mathbb{E}[f_{n,\Phi}(x)]) \xrightarrow{n \to \infty} \mathcal{N}(0, \sigma_{\Phi}^2(x)) $$

where $\sigma_{\Phi}^2(x)$ is defined by (2.6).

One can notice that Theorem 1 is an extension of Theorem 1 in [1] where the case of strongly mixing time series is considered. In fact, with our notations, if $d = 1$ and $\Phi(u) = 1$ for any $u \in \mathbb{R}$ then $f_{n,\Phi}$ reduces to the recursive kernel density estimator $f_{n,1}$ introduced by Hall and Patil [20]. In this case, we have $\nu_2(\theta) = \nu_4(\theta) = 1$ and (H2)(ii) holds as soon as $\sum_{k>0} k^{1/\tau} \alpha_{1,\infty}(k) < \infty$ and $\lim_{n \to \infty} nh_{s_n}^{N\left(1+\frac{d\nu_2(\theta)(\nu_4(\theta)+\tau-1)}{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)}\right)} = \lim_{n \to \infty} nh_{s_n}^{N(1+\tau)} = \infty$ which are exactly the conditions imposed in Theorem 1 in [1]. Using Theorem 1, we derive the asymptotic normality for the recursive estimator $r_{n,\Phi}$ defined by (2.4).

**Theorem 2.** Assume that (H1), (H2) and (H3) hold. If $f$ is Lipschitz and twice differentiable with bounded second derivatives then for any $x \in \mathbb{R}^N$ such that $f(x) > 0$,

$$ \sqrt{nh_{s_n}^N} \left( r_{n,\Phi}(x) - \frac{\mathbb{E}[f_{n,\Phi}(x)]}{\mathbb{E}[f_{n,1}(x)]} \right) \xrightarrow{n \to \infty} \mathcal{N}(0, \tilde{\sigma}_{\Phi}^2(x)), $$

with $\tilde{\sigma}_{\Phi}^2(x) = \frac{V(x)\beta_{-N,2}}{f(x)\beta_{0,1}^2} \int_{\mathbb{R}^N} K^2(t)dt$ and $V(x) = \mathbb{E}[|\Phi(Y_0)|^2|X_0 = x] - r_{\Phi}^2(x)$.

Theorem 2 is also an extension of Theorem 2.1 in [36] where the asymptotic normality of the semi-recursive kernel regression estimator for time series (i.e. $d = 1$) introduced by Ahmad and Lin [2] is obtained under more restrictive conditions on the bandwidth parameter and the strong mixing coefficients. Using Theorem 2 and Proposition 2, the condition $nh_{s_n}^{N+4} \to 0$ can be imposed for the control of the bias of the estimator and leads immediately to the following result.

**Theorem 3.** Assume that (H1), (H2) and (H3) hold. If $f$ is Lipschitz and twice differentiable with bounded second derivatives, $nh_{s_n}^{N+4} \to 0$ and $A_{n,2,1} \le 1$, then for any $x \in \mathbb{R}^N$ such that $f(x) > 0$,

$$ \sqrt{nh_{s_n}^N} (r_{n,\Phi}(x) - r_{\Phi}(x)) \xrightarrow{n \to \infty} \mathcal{N}(0, \tilde{\sigma}_{\Phi}^2(x)), $$

where $\tilde{\sigma}_{\Phi}^2(x)$ is defined in Theorem 2.

### 3. Preliminary lemmas

This section is devoted to the presentation of several technical lemmas and propositions which are key tools in the proof of the main contributions in section 4. For any real $x$, we also define $[x] = [x] + 1$, where $[x]$ is the largest integer less or equal than $x$.
**Lemma 1.** Let $(a_k)_{k \in \mathbb{Z}^d}$ be a family of real numbers such that $a_{s_n}$ goes to some value $a \in \mathbb{R}$ as $n$ goes to infinity. If $\lim_{n \to \infty} A_{n, -N, 2} = \beta_{-N, 2} \in \mathbb{R}$ then

$$ \lim_{n \to \infty} \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2 a_{s_i}}{h_{s_i}^N} = a\beta_{-N, 2}. $$

*Proof of Lemma 1.* For any positive integers $i$ and $n$, we denote $b_{i,n} = w_{s_i}^2 h_{s_n}^N / (nh_{s_i}^N w_{s_n}^2)$ if $i \le n$ and $b_{i,n} = 0$ otherwise. Since $(h_{s_n})_{n \ge 1}$ is nonincreasing and $(w_{s_n} h_{s_n}^{-N})_{n \ge 1}$ is nondecreasing, for $i \le n$, we have

$$ b_{i,n} = \frac{h_{s_i}^N}{nh_{s_n}^N} \times \frac{w_{s_i}^2/h_{s_i}^{2N}}{w_{s_n}^2/h_{s_n}^{2N}} \le \frac{h_{s_1}^N}{nh_{s_n}^N} \xrightarrow{n \to +\infty} 0. $$

Moreover, $\sum_{i=1}^{+\infty} b_{i,n} = A_{n, -N, 2} \to \beta_{-N, 2} \in \mathbb{R}$ as $n \to +\infty$. So, by Toeplitz's lemma (see Lemma 3 in [25]), we get

$$ \lim_{n \to \infty} \sum_{i=1}^{n} b_{i,n} a_{s_i} = a\beta_{-N, 2}. $$

The proof Lemma 1 is complete. $\square$

<figure><img src="image_3.png" /></figure>

The following lemma will be useful in order to compute the asymptotic variance of the estimator $f_{n, \Phi}$ (see Propositions 1 and 4).

**Lemma 2.** Let $x \in \mathbb{R}^N$ be fixed and let $\Psi_1 : \mathbb{R} \to \mathbb{R}$ and $\Psi_2 : \mathbb{R} \to \mathbb{R}$ be two functions. If $u \mapsto \mathbb{E}[\Psi_1(Y_0)|X_0 = u]$ is continuous and the conditions $\sup_{t \in \mathbb{R}^N} |\Psi_2(K(t))| < \infty$, $\lim_{\|t\| \to \infty} \|t\| |\Psi_2(K(t))| = 0$ and $\int_{\mathbb{R}^N} |\Psi_2(K(t))| dt < \infty$ are satisfied then

$$ \lim_{n \to \infty} h_{s_n}^{-N} \mathbb{E}[\Psi_1(Y_0)\Psi_2(K_{s_n}(x, X_0))] = \mathbb{E}[\Psi_1(Y_0)|X_0 = x]f(x) \int_{\mathbb{R}^N} \Psi_2(K(v)) dv. $$

*Proof of Lemma 2.* Let $x \in \mathbb{R}^N$ and let $n$ be a positive integer. It is obvious that

$$ \mathbb{E}[\Psi_1(Y_0)\Psi_2(K_{s_n}(x, X_0))] = h_{s_n}^N \int_{\mathbb{R}^N} \mathbb{E}[\Psi_1(Y_0) | X_0 = x - v h_{s_n}] \Psi_2(K(v)) f(x - v h_{s_n}) dv. $$

By Theorem 1A in [31], we derive

$$ \lim_{n \to \infty} h_{s_n}^{-N} \mathbb{E}[\Psi_1(Y_0) \Psi_2(K_{s_n}(x, X_0))] = \mathbb{E}[\Psi_1(Y_0) | X_0 = x] f(x) \int_{\mathbb{R}^N} \Psi_2(K(v)) dv. \quad (3.1) $$

The proof of Lemma 2 is complete. $\square$

<figure><img src="image_4.png" /></figure>
For any $\ell \in \{1, 2\}$, any $1 \le i \le n$ and any sequence $(m_n)_{n \ge 1}$ of positive integers, we define

$$ \Delta_{s_i}^{(\ell)} = \frac{\Phi_{\ell}(Y_{s_i}) K_{s_i}(x, X_{s_i}) - \mathbb{E}[\Phi_{\ell}(Y_0) K_{s_i}(x, X_0)]}{h_{s_i}^{N/2}} \quad \text{and} \quad \bar{\Delta}_{s_i}^{(\ell)} = \mathbb{E}[\Delta_{s_i}^{(\ell)} | \mathcal{H}_{i,m_n}] \qquad (3.2) $$

where $\mathcal{H}_{i,m_n} = \sigma(\eta_{s_i}, \varepsilon_{s_i-k}; |k| \le m_n)$ and $\Phi_{\ell}: \mathbb{R} \to \mathbb{R}$ is a measurable function.

**Lemma 3.** Let $\ell \in \{1, 2\}$ and $\theta > 0$ such that $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$ then $\mathbb{E}[|\Phi_{\ell}(Y_0)|^p K_{s_n}(x, X_0)] \le h_{s_n}^N$ for any $0 < p < 2+\theta$. Moreover, if (H3)(iii) holds then $\sup_{1 \le i,j \le n \atop i \ne j} (h_{s_i} h_{s_j})^{-N} \mathbb{E}[K_{s_i}(x, X_{s_i}) K_{s_j}(x, X_{s_j})] \le 1$.

*Proof of Lemma 3.* If $0 < p < 2+\theta$ then

$$ \mathbb{E}[|\Phi_{\ell}(Y_0)|^p K_{s_i}(x, X_0)] \le \mathbb{E}[K_{s_i}(x, X_0)] + \mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta} K_{s_i}(x, X_0)]. $$

Since $\mathbb{E}[|\Phi_{\ell}(Y_0)|^r K_{s_i}(x, X_0)] \le h_{s_n}^N$ for $r \in \{0, 2+\theta\}$, we get $\mathbb{E}[|\Phi_{\ell}(Y_0)|^p K_{s_i}(x, X_0)] \le h_{s_n}^N$. In the other part, using (H3)(iii), for any $1 \le i, j \le n$ such that $i \ne j$ we have

$$ \begin{aligned} \mathbb{E}[K_{s_i}(x, X_{s_i}) K_{s_j}(x, X_{s_j})] &\le \int_{\mathbb{R}^N} K_{s_i}(x, u) du \int_{\mathbb{R}^N} K_{s_j}(x, v) dv \\ &\quad + \mathbb{E}[K_{s_i}(x, X_0)] \mathbb{E}[K_{s_j}(x, X_0)] \\ &\le (h_{s_i} h_{s_j})^N. \end{aligned} $$

The proof of Lemma 3 is complete. $\square$

**Lemma 4.** If $(p, q) \in \{1, 2\}^2$ and $\theta > 0$ such that $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$ for any $\ell \in \{p, q\}$ then

$$ \sup_{1 \le i,j \le n \atop i \ne j} (h_{s_i} h_{s_j})^{-\frac{N\gamma}{2}} \mathbb{E}[|\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}|] \le 1 \qquad (3.3) $$

$$ \text{where } \gamma = \begin{cases} 1 & \text{if } \max(\|\Phi_p\|_{\infty}, \|\Phi_q\|_{\infty}) < \infty \\ \frac{\theta}{4+\theta} & \text{if } \min(\|\Phi_p\|_{\infty}, \|\Phi_q\|_{\infty}) = \infty \\ \frac{\theta}{2+\theta} & \text{else.} \end{cases} \qquad (3.4) $$

*Proof of Lemma 4.* Let $i$ and $j$ be two positive integers such that $i \ne j$ and $(p, q) \in \{1, 2\}^2$ and let $\theta > 0$ such that $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$ and $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta}] < \infty$ for any $\ell \in \{p, q\}$. Keeping in mind the notations $\Delta_{s_i}^{(p)}$ and $\Delta_{s_j}^{(q)}$ defined by (3.2), we have the following bound

$$ (h_{s_i} h_{s_j})^{\frac{N}{2}} \mathbb{E}[|\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}|] \le \mathbb{E}[|\Phi_p(Y_{s_i}) \Phi_q(Y_{s_j})| K_{s_i}(x, X_{s_i}) K_{s_j}(x, X_{s_j})] \\ + 3 \mathbb{E}[|\Phi_p(Y_0)| K_{s_i}(x, X_0)] \mathbb{E}[|\Phi_q(Y_0)| K_{s_j}(x, X_0)]. \qquad (3.5) $$
Note that the second term of the right hand side of (3.5) can be dealt with using Lemma 3. Therefore, we focus on the first part of the right hand side only. Let $L > 1$ be fixed then

$$ 
\begin{aligned}
& \mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})|K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})] \\
&= \mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})| \mathbf{1}_{|\Phi_p(Y_{s_i})| \le L} \mathbf{1}_{|\Phi_q(Y_{s_j})| \le L} K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})] \\
&\quad + \mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})| \mathbf{1}_{|\Phi_p(Y_{s_i})| \le L} \mathbf{1}_{|\Phi_q(Y_{s_j})| > L} K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})] \\
&\quad + \mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})| \mathbf{1}_{|\Phi_p(Y_{s_i})| > L} \mathbf{1}_{|\Phi_q(Y_{s_j})| \le L} K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})] \\
&\quad + \mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})| \mathbf{1}_{|\Phi_p(Y_{s_i})| > L} \mathbf{1}_{|\Phi_q(Y_{s_j})| > L} K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})].
\end{aligned}
$$

Using Cauchy-Schwarz's inequality, we obtain

$$ 
\begin{aligned}
& \mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})|K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})] \\
&\le (L \wedge \|\Phi_p\|_{\infty}) (L \wedge \|\Phi_q\|_{\infty}) \mathbb{E}[K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})] \\
&\quad + \sqrt{\mathbb{E}[|\Phi_p(Y_0)|^2 K_{s_i}^2(x, X_0)]} \sqrt{\mathbb{E}[|\Phi_q(Y_0)|^2 \mathbf{1}_{|\Phi_q(Y_0)| > L} K_{s_j}^2(x, X_0)]} \\
&\quad + \sqrt{\mathbb{E}[|\Phi_p(Y_0)|^2 \mathbf{1}_{|\Phi_p(Y_0)| > L} K_{s_i}^2(x, X_0)]} \sqrt{\mathbb{E}[|\Phi_q(Y_0)|^2 K_{s_j}^2(x, X_0)]} \\
&\quad + \sqrt{\mathbb{E}[|\Phi_p(Y_0)|^2 \mathbf{1}_{|\Phi_p(Y_0)| > L} K_{s_i}^2(x, X_0)]} \sqrt{\mathbb{E}[|\Phi_q(Y_0)|^2 \mathbf{1}_{|\Phi_q(Y_0)| > L} K_{s_j}^2(x, X_0)]}.
\end{aligned}
$$

Since $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$ for any $\ell \in \{p, q\}$, we apply Lemma 3 and we get

$$ 
\begin{aligned}
& \mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})|K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})] \\
&\le (L \wedge \|\Phi_p\|_{\infty}) (L \wedge \|\Phi_q\|_{\infty}) (h_{s_i} h_{s_j})^N + L^{-\theta/2} (h_{s_i} h_{s_j})^{N/2}.
\end{aligned}
\quad (3.6)
$$

Optimizing (3.6) with respect to $L$, we derive

$$ 
\begin{aligned}
& \frac{\mathbb{E}[|\Phi_p(Y_{s_i})\Phi_q(Y_{s_j})|K_{s_i}(x, X_{s_i})K_{s_j}(x, X_{s_j})]}{h_{s_i}^{N/2} h_{s_j}^{N/2}} \\
&\le \begin{cases}
(h_{s_i} h_{s_j})^{\frac{N\theta}{2(2+\theta)}} & \text{if } \|\Phi_p\|_{\infty} < \infty \text{ and } \|\Phi_q\|_{\infty} = \infty \\
(h_{s_i} h_{s_j})^{\frac{N\theta}{2(2+\theta)}} & \text{if } \|\Phi_p\|_{\infty} = \infty \text{ and } \|\Phi_q\|_{\infty} < \infty \\
(h_{s_i} h_{s_j})^{\frac{N}{2}} & \text{if } \|\Phi_p\|_{\infty} < \infty \text{ and } \|\Phi_q\|_{\infty} < \infty \\
(h_{s_i} h_{s_j})^{\frac{N\theta}{2(4+\theta)}} & \text{if } \|\Phi_p\|_{\infty} = \infty \text{ and } \|\Phi_q\|_{\infty} = \infty.
\end{cases}
\quad (3.7)
\end{aligned}
$$

Combining (3.5), (3.7) and Lemma 3, we obtain (3.3). The proof of Lemma 4 is complete. $\square$

**Lemma 5.** Let $(\Xi_k)_{k \in \mathbb{Z}^d}$ be a family of non negative real numbers.

If $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d\ell_2}{\ell_1}} \Xi_k < \infty$ for some positive constants $\ell_1$ and $\ell_2$ then there exists a sequence $(m_n)_{n \ge 1}$ of positive integers satisfying

$$ \lim_{n \to \infty} m_n = +\infty, \quad \lim_{n \to \infty} m_n^d h_{s_n}^{\ell_1} = 0 \quad \text{and} \quad \lim_{n \to \infty} h_{s_n}^{-\ell_2} \sum_{|k| > m_n} \Xi_k = 0. $$
Notice that if $\ell_1 \le N$ and $nh_{s_n}^N \to \infty$ then $m_n^d = o(n)$.

**Proof of Lemma 5.** Let $\ell_1, \ell_2$ and $r$ be positive constants such that $r > \ell_2/\ell_1$ and let $(m_n)_{n\ge 1}$ be the sequence defined for any integer $n \ge 1$ by

$$m_n = \max \left\{ v_n, \left[ h_{s_n}^{-\ell_1/d} \left( \sum_{|k|>v_n} |k|^{\frac{d\ell_2}{\ell_1}} \Xi_k \right)^{\frac{1}{dr}} \right] \right\} \quad \text{and} \quad v_n = \lfloor h_{s_n}^{-\ell_1/(2d)} \rfloor.$$

Since $v_n \to \infty$, we have $m_n \to \infty$ as $n$ goes to infinity. Moreover,

$$m_n^d h_{s_n}^{\ell_1} \le \max \left\{ h_{s_n}^{\ell_1/2}, \left( \sum_{|k|>v_n} |k|^{\frac{d\ell_2}{\ell_1}} \Xi_k \right)^{\frac{1}{r}} + h_{s_n}^{\ell_1} \right\} \xrightarrow{n\to\infty} 0.$$

Since $v_n \le m_n$, we have

$$m_n^d h_{s_n}^{\ell_1} \ge \left( \sum_{|k|>m_n} |k|^{\frac{d\ell_2}{\ell_1}} \Xi_k \right)^{\frac{1}{r}}.$$

Since $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d\ell_2}{\ell_1}} \Xi_k < \infty$ and $r > \ell_2/\ell_1$, we get

$$h_{s_n}^{-\ell_2} \sum_{|k|>m_n} \Xi_k \le (m_n^d h_{s_n}^{\ell_1})^{-\frac{\ell_2}{\ell_1}} \sum_{|k|>m_n} |k|^{\frac{d\ell_2}{\ell_1}} \Xi_k \le \left( \sum_{|k|>m_n} |k|^{\frac{d\ell_2}{\ell_1}} \Xi_k \right)^{1-\frac{\ell_2}{\ell_1 r}} \xrightarrow{n\to\infty} 0.$$

The proof of Lemma 5 is complete. $\square$

<figure><img src="image_5.png" /></figure>

**Lemma 6.** Let $\ell \in \{1, 2\}$ and $\theta > 0$ be fixed such that $\mathbb{E}[|\Phi_\ell(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\Phi_\ell(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$ then $\|\Delta_{s_n}^{(\ell)}\|_{2+\theta}^2 \le h_{s_n}^{\frac{N\theta}{2+\theta}}$ where $\Delta_{s_n}^{(\ell)}$ is given by (3.2).

**Proof of Lemma 6.** Let $\theta > 0$ and $\ell \in \{1, 2\}$ such that $\mathbb{E}[|\Phi_\ell(Y_0)|^{2+\theta}] < \infty$ and let $1 \le i \le n$, we have

$$\|\Delta_{s_i}^{(\ell)}\|_{2+\theta}^2 \le \frac{2 \|\Phi_\ell(Y_0) K_{s_i}(x, X_0)\|_{2+\theta}^2}{h_{s_i}^N} + \frac{2 (\mathbb{E}[|\Phi_\ell(Y_0)| K_{s_i}(x, X_0)])^2}{h_{s_i}^N}.$$

Since $\sup_{t \in \mathbb{R}^N} |K(t)| < \infty$ and $\mathbb{E}[|\Phi_\ell(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$, we get $\mathbb{E}[|\Phi_\ell(Y_0) K_{s_i}(x, X_0)|^{2+\theta}] \le h_{s_i}^N$. Moreover, using Lemma 3, we have $\mathbb{E}[|\Phi_\ell(Y_0)| K_{s_i}(x, X_0)] \le h_{s_i}^N$ and we obtain $\|\Delta_{s_i}^{(\ell)}\|_{2+\theta}^2 \le h_{s_i}^{\frac{\theta N}{2+\theta}}$. The proof of Lemma 6 is complete. $\square$

<figure><img src="image_6.png" /></figure>

**Proposition 3.** Let $M$ be a positive integer and let $x \in \mathbb{R}^N$. If $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5) and $\Phi: \mathbb{R} \to \mathbb{R}$ is a measurable function such that $\|\Phi(Y_0)\|_{2+\theta} < \infty$
for some $\theta \in ]0, +\infty]$ then for any positive integer $n$ and any family $(c_k)_{k \in \Lambda_n}$ of
real numbers and any $(p, q) \in [2, +\infty[ \times ]0, +\infty]$ such that $p+q \le 2+\theta$, we have

$$
\left\| \sum_{i=1}^{n} c_{s_i} W_i \right\|_{p} \leq 8pM^{d} |K|_{\infty}^{\frac{p}{p+q}} C(p,q) \left( \sum_{i=1}^{n} c_{s_i}^{2} \right)^{\frac{1}{2}} h_{s_n}^{\frac{-q}{p+q}} \sum_{|j|>M} \delta_{j,p}^{\frac{q}{p+q}},
$$

where

$$
W_i := \Phi(Y_{s_i}) K_{s_i}(x, X_{s_i}) - \mathbb{E}[\Phi(Y_{s_i}) K_{s_i}(x, X_{s_i}) | \mathcal{H}_{i,M}], \quad (3.8)
$$

$$
C(p,q) = 2^{\frac{2p+q}{p+q}} \|\Phi(Y_0)\|_{p+q} \|K\|_{\text{Lip}}^{\frac{q}{p+q}} \\
\qquad + |K|_{\infty}^{\frac{q}{p+q}} \left\| \sup_{(x,y) \in \mathbb{R}^N \times \mathbb{R}^N \atop x \neq y} \frac{|\Phi(R(x, \eta_0)) - \Phi(R(y, \eta_0))|}{\|x-y\|} \right\|_p
$$

and

$$
\mathcal{H}_{i,M} = \sigma(\eta_{s_i}, \epsilon_{s_i-k}; |k| \le M) \quad \text{and} \quad \|K\|_{\text{Lip}} = \sup_{(x,y) \in \mathbb{R}^N \times \mathbb{R}^N \atop x \neq y} \frac{|K(x) - K(y)|}{\|x - y\|}.
$$

**Proof of Proposition 3.** Let $M$ be a positive integer and let $x \in \mathbb{R}^N$ and $1 \le i \le n$ be fixed. Recall that $Y_{s_i} = R(X_{s_i}, \eta_{s_i})$ and let $W_i = \Phi(Y_{s_i}) K_{s_i}(x, X_{s_i}) - \mathbb{E}[\Phi(Y_{s_i}) K_{s_i}(x, X_{s_i}) | \mathcal{H}_{i,M}]$ where $\mathcal{H}_{i,M} = \sigma(\eta_{s_i}, \epsilon_{s_i-k}; |k| \le M)$. We follow the same lines as in the proof of Proposition 1 in [16]. Let $2 \le p < 2 + \theta$ and denote by $H_i$ the measurable function such that $W_i = H_i(\mathcal{H}_{i,\infty})$ with $\mathcal{H}_{i,\infty} = \sigma(\eta_{s_i}, \epsilon_{s_i-k}; k \in \mathbb{Z}^d)$. Let $\tau$ be a bijection from $\mathbb{Z}$ to $\mathbb{Z}^d$ and $\ell$ in $\mathbb{Z}$ be fixed. We define the projection operator $P_\ell$ by $P_\ell f = \mathbb{E}[f | \mathcal{F}_\ell] - \mathbb{E}[f | \mathcal{F}_{\ell-1}]$ for any integrable function $f$, where $\mathcal{F}_\ell = \sigma(\epsilon_{\tau(j)}; j \le \ell)$. One can notice that the operator $P_\ell$ depends on the bijection $\tau$. The proof of the following technical result is postponed to section 5.

**Lemma 7.** *Almost surely, it holds that* $\mathbb{E}[W_i|\mathcal{F}_{\ell-1}] = \mathbb{E}[H_i(\mathcal{H}_{i,\infty}^{(\ell)})|\mathcal{F}_\ell]$
with $\mathcal{H}_{i,\infty}^{(\ell)} = \sigma(\eta_{s_i}, \varepsilon'_{\tau(\ell)}, \varepsilon_{s_i-k}; k \in \mathbb{Z}^d \setminus \{s_i - \tau(\ell)\})$.

Using Lemma 7, we obtain

$$
\| P_\ell W_i \|_p = \| \mathbb{E}[H_i(\mathcal{H}_{i,\infty}) | \mathcal{F}_\ell] - \mathbb{E}[H_i(\mathcal{H}_{i,\infty}^{(\ell)}) | \mathcal{F}_\ell] \|_p \le \| H_i(\mathcal{H}_{i,\infty}) - H_i(\mathcal{H}_{i,\infty}^{(\ell)}) \|_p. \tag{3.9}
$$

Now, denoting $\mathcal{H}_{i,M}^{(\ell)} = \sigma(\eta_{s_i}, \varepsilon'_{\tau(\ell)}, \varepsilon_{s_i-k}; k \in \mathbb{Z}^d \setminus \{s_i - \tau(\ell)\})$ and $|k| \le M$),
we have

$$
\begin{align*}
H_i(\mathcal{H}_{i,\infty}) &= \Phi(Y_{s_i}) K_{s_i}(x, X_{s_i}) - \mathbb{E}[\Phi(Y_{s_i}) K_{s_i}(x, X_{s_i}) | \mathcal{H}_{i,M} \lor \mathcal{H}_{i,M}^{(\ell)}], \\
H_i(\mathcal{H}_{i,\infty}^{(\ell)}) &= \Phi(Y'_{i,\tau(\ell)}) K_{s_i}(x, X'_{i,\tau(\ell)}) - \mathbb{E}[\Phi(Y'_{i,\tau(\ell)}) K_{s_i}(x, X'_{i,\tau(\ell)}) | \mathcal{H}_{i,M}^{(\ell)} \lor \mathcal{H}_{i,M}]
\end{align*}
$$
where $X'_{i,\tau(\ell)} = G(\varepsilon'_{\tau(\ell)}, \varepsilon_{s_i-k}; k \in \mathbb{Z}^d \setminus \{s_i - \tau(\ell)\})$ and $Y'_{i,\tau(\ell)} = R(X'_{i,\tau(\ell)}, \eta_{s_i})$. So, we derive

$$ \|P_{\ell}W_i\|_p \le 2\|\Phi(Y_{s_i})K_{s_i}(x, X_{s_i}) - \Phi(Y'_{i,\tau(\ell)})K_{s_i}(x, X'_{i,\tau(\ell)})\|_p. $$

So, for any $L > 0$, we get the bound

$$ \|P_{\ell}W_i\|_p \le \frac{2L \|K\|_{\text{Lip}}}{h_{s_i}} \|X_{s_i} - X'_{i,\tau(\ell)}\|_p + \frac{4|K|_{\infty}}{L^{q/p}} \|\Phi(Y_0)\|_{p+q}^{\frac{p+q}{p}} \\ + 2|K|_{\infty} \left\| \sup_{\substack{(x,y) \in \mathbb{R}^N \times \mathbb{R}^N \\ x \neq y}} \frac{|\Phi(R(x, \eta_0)) - \Phi(R(y, \eta_0))|}{\|x-y\|} \right\|_p \|X_{s_i} - X'_{i,\tau(\ell)}\|_p $$

where

$$ \begin{aligned} \|X_{s_i} - X'_{i,\tau(\ell)}\|_p &= \|G(\varepsilon_{s_i-k}; k \in \mathbb{Z}^d) - G(\varepsilon'_{\tau(\ell)}, \varepsilon_{s_i-k}; k \in \mathbb{Z}^d \setminus \{s_i - \tau(\ell)\})\|_p \\ &= \|G(\varepsilon_{s_i-\tau(\ell)-k}; k \in \mathbb{Z}^d) - G(\varepsilon'_{0}, \varepsilon_{s_i-\tau(\ell)-k}; k \in \mathbb{Z}^d \setminus \{s_i - \tau(\ell)\})\|_p \\ &= \|X_{s_i-\tau(\ell)} - X^*_{s_i-\tau(\ell)}\|_p \\ &= \delta_{s_i-\tau(\ell),p}. \end{aligned} $$

Optimizing this last inequality in $L$, we get the following bound

$$ \|P_{\ell}W_i\|_p \le 2|K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_i}^{\frac{-q}{p+q}} \delta_{s_i-\tau(\ell),p}, \quad (3.10) $$

where

$$ C(p,q) = 2^{\frac{2p+q}{p+q}} \|\Phi(Y_0)\|_{p+q} \|K\|_{\text{Lip}}^{\frac{q}{p+q}} \\ + |K|_{\infty}^{\frac{q}{p+q}} \left\| \sup_{\substack{(x,y) \in \mathbb{R}^N \times \mathbb{R}^N \\ x \neq y}} \frac{|\Phi(R(x, \eta_0)) - \Phi(R(y, \eta_0))|}{\|x-y\|} \right\|_p. $$

Now, we are going to obtain another bound for $\|P_{\ell}W_i\|_p$. Let $\ell \ge 0$ and $i \ge 1$ be two integers. We denote by $\Gamma_{i,\ell}$ the set of all $k \in \mathbb{Z}^d$ such that $|s_i - k| = \ell$ and we define

$$ a_{\ell} := \sum_{j=0}^{\ell} |\Gamma_{i,j}| = (2\ell + 1)^d. $$

On the lattice $\mathbb{Z}^d$ we define the lexicographic order as follows: if $u = (u_1, \dots, u_d)$ and $v = (v_1, \dots, v_d)$ are distinct elements of $\mathbb{Z}^d$, the notation $u <_{\text{lex}} v$ means that either $u_1 < v_1$ or for some $k$ in $\{2, \dots, d\}$, $u_k < v_k$ and $u_{\ell} = v_{\ell}$ for $1 \le \ell < k$. We consider the bijection $\tau_i : ]0, +\infty[ \cap \mathbb{Z} \to \mathbb{Z}^d$ defined by $\tau_i(1) = s_i$, $\tau_i(u) \in \Gamma_{i,\ell}$ if $a_{\ell-1} < u \le a_{\ell}$ and $\ell > 0$, and $\tau_i(u) <_{\text{lex}} \tau_i(v)$ if
$a_{\ell-1} < u < v \le a_{\ell}$ and $\ell > 0$. Let $\mathcal{G}_{i,M} = \sigma(\eta_{s_i}, \varepsilon_{\tau_i(j)}; 1 \le j \le M)$ and recall that $\mathcal{H}_{i,M} = \sigma(\eta_{s_i}, \varepsilon_{s_i-k}; |k| \le M)$. Since $1 \le j \le a_M$ if and only if $|s_i - \tau_i(j)| \le M$, we have $\mathcal{G}_{i,a_M} = \mathcal{H}_{i,M}$. Consequently,

$$W_i = \sum_{\ell > a_M} D_{i,\ell}$$

where $D_{i,\ell} = \mathbb{E}[\Phi(Y_{s_i})K_{s_i}(x, X_{s_i})|\mathcal{G}_{i,\ell}] - \mathbb{E}[\Phi(Y_{s_i})K_{s_i}(x, X_{s_i})|\mathcal{G}_{i,\ell-1}]$.

Given that $(D_{i,\ell})_{\ell \ge 1}$ is a martingale difference sequence with respect to the filtration $(\mathcal{G}_{i,\ell})_{\ell \ge 1}$, we apply Burkholder's inequality ([10], remark 6, page 85) and we obtain

$$\|W_i\|_p \le \left( 2p \sum_{\ell > a_M} \|D_{i,\ell}\|_p^2 \right)^{1/2}. \quad (3.11)$$

Since $X'_{i,\tau_i(\ell)} = G(\varepsilon'_{\tau_i(\ell)}, \varepsilon_{s_i-k}; k \in \mathbb{Z}^d \setminus \{s_i-\tau_i(\ell)\})$ and $Y'_{i,\tau_i(\ell)} = R(X'_{i,\tau_i(\ell)}, \eta_{s_i})$, we have $\mathbb{E}[\Phi(Y_{s_i})K_{s_i}(x, X_{s_i})|\mathcal{G}_{i,\ell-1}] = [\Phi(Y'_{i,\tau_i(\ell)})K_{s_i}(x, X'_{i,\tau_i(\ell)})|\mathcal{G}_{i,\ell}]$ and

$$\|D_{i,\ell}\|_p \le \|\Phi(Y_{s_i})K_{s_i}(x, X_{s_i}) - \Phi(Y'_{i,\tau_i(\ell)})K_{s_i}(x, X'_{i,\tau_i(\ell)})\|_p.$$

Arguing as before, for any $L > 0$, we derive

$$\|D_{i,\ell}\|_p \le \frac{L \|K\|_{\text{Lip}}}{h_{s_i}} \|X_{s_i} - X'_{i,\tau_i(\ell)}\|_p + \frac{2|K|_{\infty}}{L^{q/p}} \|\Phi(Y_0)\|_{p+q}^{\frac{p+q}{p}} \\ + |K|_{\infty} \left\| \sup_{(x,y) \in \mathbb{R}^N \times \mathbb{R}^N, x \ne y} \frac{|\Phi(R(x, \eta_0)) - \Phi(R(y, \eta_0))|}{\|x-y\|} \right\|_p \|X_{s_i} - X'_{i,\tau_i(\ell)}\|_p$$

with $\|X_{s_i} - X'_{i,\tau_i(\ell)}\|_p = \delta_{s_i-\tau_i(\ell),p}$. Optimizing this last inequality on $L$ and noting that $s_i - \tau_i(\ell) = -\tau_0(\ell)$, we obtain

$$\|D_{i,\ell}\|_p \le |K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_i}^{-\frac{q}{p+q}} \delta_{-\tau_0(\ell),p}^{-\frac{q}{p+q}}.$$

Consequently, we get

$$\|P_{\ell}W_{i}\|_{p} \le 2\|W_{i}\|_{p} \le 2\sqrt{2p}|K|_{\infty}^{\frac{p}{p+q}} C(p,q)h_{s_i}^{-\frac{q}{p+q}} \sum_{\ell > a_M} \delta_{-\tau_0(\ell),p}^{\frac{q}{p+q}} \\ \le 2\sqrt{2p}|K|_{\infty}^{\frac{p}{p+q}} C(p,q)h_{s_n}^{-\frac{q}{p+q}} \sum_{|k| > M} \delta_{k,p}^{\frac{q}{p+q}}. \quad (3.12)$$

Since $(\sum_{i=1}^n c_{s_i} P_{\ell} W_i)_{\ell \in \mathbb{Z}}$ is a martingale difference sequence with respect to the filtration $(\mathcal{F}_{\ell})_{\ell \in \mathbb{Z}}$, the Burkholder inequality (see [10], remark 6, page 85) implies

$$\left\| \sum_{i=1}^{n} c_{s_i} W_i \right\|_p \le \left( 2p \sum_{\ell \in \mathbb{Z}} \left\| \sum_{i=1}^{n} c_{s_i} P_{\ell} W_i \right\|_p^2 \right)^{\frac{1}{2}}$$
$$ \leq \left( 2p \sum_{\ell \in \mathbb{Z}} \left( \sum_{i=1}^{n} |c_{s_i}| \|P_{\ell} W_i\|_p \right)^2 \right)^{\frac{1}{2}}. \quad (3.13) $$

Moreover, by the Cauchy-Schwarz inequality, we have

$$ \left( \sum_{i=1}^{n} |c_{s_i}| \|P_{\ell} W_i\|_p \right)^2 \leq \sum_{i=1}^{n} c_{s_i}^2 \|P_{\ell} W_i\|_p \times \sum_{j=1}^{n} \|P_{\ell} W_j\|_p. \quad (3.14) $$

Now, keeping in mind that $P_\ell$ is defined from the bijection $\tau$ and using (3.10) and (3.12), we have

$$ \begin{aligned} \sup_{\ell \in \mathbb{Z}} \sum_{i=1}^{n} \|P_{\ell} W_i\|_p &\leq \sup_{\ell \in \mathbb{Z}} \sum_{\substack{1 \leq i \leq n \\ |s_i - \tau(\ell)| \leq M}} \|P_{\ell} W_i\|_p + \sup_{\ell \in \mathbb{Z}} \sum_{\substack{1 \leq i \leq n \\ |s_i - \tau(\ell)| > M}} \|P_{\ell} W_i\|_p \\ &\leq 2\sqrt{2pM^d} |K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_n}^{-\frac{q}{p+q}} \sum_{|k|>M} \delta_{k,p}^{\frac{q}{p+q}} \\ &\quad + 2|K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_n}^{-\frac{q}{p+q}} \sum_{\substack{1 \leq i \leq n \\ |s_i - \tau(\ell)| > M}} \delta_{s_i - \tau(\ell), p}^{\frac{q}{p+q}} \\ &\leq 2(M^d \sqrt{2p} + 1) |K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_n}^{-\frac{q}{p+q}} \sum_{|k|>M} \delta_{k,p}^{\frac{q}{p+q}}. \end{aligned} $$

Similarly, we have also

$$ \begin{aligned} \sup_{1 \leq i \leq n} \sum_{\ell \in \mathbb{Z}} \|P_{\ell} W_i\|_p &\leq \sup_{1 \leq i \leq n} \sum_{\substack{\ell \in \mathbb{Z} \\ |s_i - \tau(\ell)| \leq M}} \|P_{\ell} W_i\|_p + \sup_{1 \leq i \leq n} \sum_{\substack{\ell \in \mathbb{Z} \\ |s_i - \tau(\ell)| > M}} \|P_{\ell} W_i\|_p \\ &\leq 2\sqrt{2pM^d} |K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_n}^{-\frac{q}{p+q}} \sum_{|k|>M} \delta_{k,p}^{\frac{q}{p+q}} \\ &\quad + 2|K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_i}^{-\frac{q}{p+q}} \sum_{\substack{\ell \in \mathbb{Z} \\ |s_i - \tau(\ell)| > M}} \delta_{s_i - \tau(\ell), p}^{\frac{q}{p+q}} \\ &\leq 2(M^d \sqrt{2p} + 1) |K|_{\infty}^{\frac{p}{p+q}} C(p,q) h_{s_n}^{-\frac{q}{p+q}} \sum_{|k|>M} \delta_{k,p}^{\frac{q}{p+q}}. \end{aligned} $$

Combining (3.13) and (3.14) with the last two bound above, we get

$$ \left\| \sum_{i=1}^{n} c_{s_i} W_i \right\|_p \leq \sqrt{2p(2(M^d \sqrt{2p} + 1))^2} |K|_{\infty}^{\frac{p}{p+q}} C(p,q) \left( \sum_{i=1}^{n} c_{s_i}^2 \right)^{\frac{1}{2}} h_{s_n}^{-\frac{q}{p+q}} \sum_{|k|>M} \delta_{k,p}^{\frac{q}{p+q}}. $$

Noting that $\sqrt{2p} + 1 \leq 2\sqrt{2p}$, we obtain

$$ \left\| \sum_{i=1}^{n} c_{s_i} W_i \right\|_p \leq 8pM^d |K|_{\infty}^{\frac{p}{p+q}} C(p,q) \left( \sum_{i=1}^{n} c_{s_i}^2 \right)^{\frac{1}{2}} h_{s_n}^{-\frac{q}{p+q}} \sum_{|k|>M} \delta_{k,p}^{\frac{q}{p+q}}. $$
The proof of Proposition 3 is complete.

<figure><img src="image_7.png" /></figure>

## 4. Proofs of the main results

Now, we denote by $V(Z)$ the variance of any square-integrable $\mathbb{R}$-valued random variable $Z$ and we consider a sequence $(m_n)_{n \ge 1}$ of positive integers. For any $x \in \mathbb{R}$ and any $\ell \in \{1, 2\}$, denote

$$f_n^{(\ell)}(x) = f_{n, \Phi_{\ell}}(x) = \left(\sum_{i=1}^{n} w_{s_i}\right)^{-1} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N} \Phi_{\ell}(Y_{s_i}) K_{s_i}(x, X_{s_i}) \\ \text{and} \\ \bar{f}_n^{(\ell)}(x) = \mathbb{E}[f_n^{(\ell)}(x) | \mathcal{H}_{i,m_n}] \quad (4.1)$$

where $\Phi_{\ell}: \mathbb{R} \to \mathbb{R}$ is a measurable function and $\mathcal{H}_{i,m_n} = \sigma(\eta_{s_i}, \varepsilon_{s_i-k}; |k| \le m_n)$. First, we note that Proposition 1 is a particular case of the following result.

**Proposition 4.** Assume that (H1) and (H3)(iii) hold. Let $(p, q) \in \{1, 2\}^2$ and $x \in \mathbb{R}^N$ be fixed. Let $\theta > 0$ such that $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$ and $\mathbb{E}[|\Phi_{\ell}(Y_0)|^{2+\theta}] < \infty$ for any $\ell \in \{p, q\}$. Assume also that the function $u \mapsto \mathbb{E}[\Phi_p(Y_0)\Phi_q(Y_0)|X_0 = u]$ is continuous. If there exists $\tau \in ]1 - \gamma, 1]$ such that $h_{s_n}^{N(1-\tau)} \sum_{i=1}^n w_{s_i}^2 \le nw_{s_n}^2$ and one of the following conditions is satisfied:

(i) $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing and $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d\tilde{\gamma}}{\gamma+\tau-1}} \alpha_{1,1}^{\tilde{\gamma}} (|k|) < \infty$.

(ii) $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5) and $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d(N+2\tilde{\gamma}+2N(\gamma+\tau-1))}{2N(\gamma+\tau-1)}} \delta_{k,2}^{\tilde{\gamma}} < \infty$.

where $\tilde{\gamma}$ is defined by

$$\tilde{\gamma} = \begin{cases} 1 & \text{if } \max(\|\Phi_p\|_{\infty}, \|\Phi_q\|_{\infty}) < +\infty \\ \frac{\theta}{2+\theta} & \text{else} \end{cases} \quad (4.2)$$

then

$$\lim_{n \to \infty} nh_{s_n}^N \operatorname{Cov} [f_n^{(p)}(x), f_n^{(q)}(x)] \\ = \beta_{0,1}^{-2} \beta_{-N,2} \mathbb{E}[\Phi_p(Y_0)\Phi_q(Y_0)|X_0 = x] f(x) \int_{\mathbb{R}^N} K^2(t)dt.$$

**Proof of Proposition 4.** Let $x \in \mathbb{R}^N$ and let $(p, q) \in \{1, 2\}^2$ be fixed. Using the notations (3.2) and (4.1), for $\ell \in \{p, q\}$, we have

$$f_n^{(\ell)}(x) - \mathbb{E}[f_n^{(\ell)}(x)] = \left(\sum_{i=1}^{n} w_{s_i}\right)^{-1} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} \Delta_{s_i}^{(\ell)}.$$

Keeping in mind that $A_{n,0,1} = (nw_{s_n})^{-1} \sum_{i=1}^n w_{s_i}$, we get

$$nh_{s_n}^N \operatorname{Cov} [f_n^{(p)}(x), f_n^{(q)}(x)]$$
$$ 
\begin{aligned}
&= nh_{s_n}^N \left( \sum_{i=1}^n w_{s_i} \right)^{-2} \mathbb{E} \left[ \left( \sum_{i=1}^n w_{s_i} h_{s_i}^{-N/2} \Delta_{s_i}^{(p)} \right) \left( \sum_{i=1}^n w_{s_i} h_{s_i}^{-N/2} \Delta_{s_i}^{(q)} \right) \right] \\
&= \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \left( \sum_{i=1}^n w_{s_i}^2 h_{s_i}^{-N} \mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_i}^{(q)}] + \sum_{\substack{i,j=1 \\ i \neq j}}^n w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} \mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}] \right).
\end{aligned}
$$

So, we obtain

$$ \left| nh_{s_n}^N \mathrm{Cov}[f_n^{(p)}(x), f_n^{(q)}(x)] - \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{i=1}^n w_{s_i}^2 h_{s_i}^{-N} \mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_i}^{(q)}] \right| \le \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{i,j=1 \\ i \neq j}}^n w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} |\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]|. \quad (4.3) $$

Moreover, for any $1 \le i \le n$,

$$ \mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_i}^{(q)}] = h_{s_i}^{-N} \left( \begin{aligned} & \mathbb{E}[\Phi_p(Y_0)\Phi_q(Y_0)K_{s_i}^2(x, X_0)] \\ & - \mathbb{E}[\Phi_p(Y_0)K_{s_i}(x, X_0)]\mathbb{E}[\Phi_q(Y_0)K_{s_i}(x, X_0)] \end{aligned} \right). $$

Using Lemma 2 and Lemma 3, we derive

$$ \lim_{i \to \infty} |\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_i}^{(q)}] - \mathbb{E}[\Phi_p(Y_0)\Phi_q(Y_0)|X_0 = x]f(x) \int_{\mathbb{R}^N} K^2(t)dt| = 0. $$

So, using Lemma 1 and (H1), we derive

$$ \begin{aligned} & \lim_{n \to \infty} \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{i=1}^n h_{s_i}^{-N} w_{s_i}^2 \mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_i}^{(q)}] \\ & = \beta_{0,1}^{-2} \beta_{-N,2} \mathbb{E}[\Phi_p(Y_0)\Phi_q(Y_0)|X_0 = x] f(x) \int_{\mathbb{R}^N} K^2(t)dt. \end{aligned} \quad (4.4) $$

Now, we are going to prove that

$$ \lim_{n \to \infty} \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{i,j=1 \\ i \neq j}}^n w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} |\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]| = 0. \quad (4.5) $$

Using Lemma 4, we have $|\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]| \le (h_{s_i} h_{s_j})^{\frac{N\gamma}{2}}$ for any $i \ne j$ where $\gamma$ is defined by (3.4). Moreover, since $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d\tilde{\gamma}}{\gamma+\tau-1}} \alpha_{1,1}^{\tilde{\gamma}} (|k|) < \infty$, using Lemma 5, there exists a sequence $(m_n)_{n \ge 1}$ of positive integers such that

$$ \lim_{n \to \infty} m_n^d h_{s_n}^{N(\gamma+\tau-1)} = \lim_{n \to \infty} h_{s_n}^{-N\tilde{\gamma}} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \alpha_{1,1}^{\tilde{\gamma}} (|k|) = 0. \quad (4.6) $$
So, we have

$$
\begin{align*}
& \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{i,j=1 \\ i \neq j}}^n w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} |\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]| \\
&= \frac{2h_{s_n}^N}{nw_{s_n}^2} \sum_{1 \le i < j \le n} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} |\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]| \le E_{1,n} + E_{2,n}
\end{align*}
$$

where

$$
E_{1,n} = \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| < m_n}} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N(1-\gamma)/2}
$$

and

$$
E_{2,n} = \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \ge m_n}} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} |\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]|.
$$

Since $\gamma \le 1$, using the inequality $2ab \le a^2 + b^2$, we have

$$
\begin{align*}
E_{1,n} \le \frac{h_{s_n}^{N\gamma}}{nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| < m_n}} w_{s_i} w_{s_j} &\le \frac{h_{s_n}^{N\gamma}}{2nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| < m_n}} (w_{s_i}^2 + w_{s_j}^2) \\
&\le \frac{m_n^d h_{s_n}^{N\gamma}}{nw_{s_n}^2} \sum_{i=1}^n w_{s_i}^2 \\
&= m_n^d h_{s_n}^{N(\gamma+\tau-1)} \times \frac{h_{s_n}^{N(1-\tau)}}{nw_{s_n}^2} \sum_{i=1}^n w_{s_i}^2.
\end{align*}
$$

Using (4.6) and keeping in mind that $h_{s_n}^{N(1-\tau)} \sum_{i=1}^n w_{s_i}^2 \le n w_{s_n}^2$, we get
$\lim_{n \to \infty} E_{1,n} = 0$. Now, we are going to control the term $E_{2,n}$ when $(X_k)_{k \in \mathbb{Z}^d}$ is
assumed to be strongly mixing. Using Rio's inequality ([32], Theorem 1.1), we
have for any $1 \le i < j \le n$,

$$
|\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]| \le 2 \int_0^{2\alpha_{1,1}(|s_i - s_j|)} Q_{\Delta_{s_i}^{(p)}}(u) Q_{\Delta_{s_j}^{(q)}}(u) du
$$

where $Q_{\Delta_{s_l}^{(p)}}(u) = \inf\{\varepsilon > 0 \mid \mathbb{P}(|\Delta_{s_l}^{(p)}| > \varepsilon) \le u\}$ for any $u \in [0,1]$ and any $l \in \{i,j\}$.

First, we assume that $\|\Phi_p\|_{\infty} = \infty$ or $\|\Phi_q\|_{\infty} = \infty$. In this case, $\tilde{\gamma} = \theta/(2+\theta)$.
Using Lemma 6, we have $\|\Delta_{s_r}^{(\ell)}\|_{2+\theta} \le h_{s_r}^{\frac{-N\tilde{\gamma}}{2}}$ for any $\ell \in \{p,q\}$ and any
$1 \le r \le n$. So, we derive $Q_{\Delta_{s_r}^{(\ell)}}(u) \le u^{-\frac{1}{2+\theta}} h_{s_r}^{\frac{-N\tilde{\gamma}}{2}}$ and

$$
|\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]| \le 2 (h_{s_i} h_{s_j})^{\frac{-N\tilde{\gamma}}{2}} \int_{0}^{2\alpha_{1,1}(|s_i - s_j|)} u^{\frac{-2}{2+\theta}} du
$$
$$ \leq (h_{s_i} h_{s_j})^{\frac{-N\tilde{\gamma}}{2}} \alpha_{1,1}^{\tilde{\gamma}}(|s_i - s_j|). $$

Consequently, we get

$$ E_{2,n} \leq \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{1 \leq i < j \leq n \\ |s_i - s_j| > m_n}} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{\frac{-N(\tilde{\gamma}+1)}{2}} \alpha_{1,1}^{\tilde{\gamma}}(|s_i - s_j|). $$

Using again the inequality $2ab \leq a^2 + b^2$, we derive

$$ \begin{aligned} E_{2,n} &\leq \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{1 \leq i < j \leq n \\ |s_i - s_j| > m_n}} \left( w_{s_i}^2 h_{s_i}^{-N(\tilde{\gamma}+1)} + w_{s_j}^2 h_{s_j}^{-N(\tilde{\gamma}+1)} \right) \alpha_{1,1}^{\tilde{\gamma}}(|s_i - s_j|) \\ &\leq \left( \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^n \frac{w_{s_i}^2}{h_{s_i}^N} \right) h_{s_n}^{-N\tilde{\gamma}} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \alpha_{1,1}^{\tilde{\gamma}}(|k|). \end{aligned} $$

Using (H1) and (4.6), we get

$$ E_{2,n} \leq h_{s_n}^{-N\tilde{\gamma}} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \alpha_{1,1}^{\tilde{\gamma}}(|k|) \xrightarrow{n \to \infty} 0 \quad (4.7) $$

and finally, we obtain (4.5). Now, we deal with the case $||\Phi_p||_\infty < \infty$ and $||\Phi_q||_\infty < \infty$. In this case, we have $\tilde{\gamma} = 1$. So, noting that $|\Delta_{s_r}^{(\ell)}| \leq h_{s_r}^{-N\tilde{\gamma}/2}$ for any $\ell \in \{p, q\}$ and any $1 \leq r \leq n$, we derive $Q_{\Delta_{s_r}^{(\ell)}}(u) \leq h_{s_r}^{-N\tilde{\gamma}/2}$ and $|\mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_j}^{(q)}]| \leq (h_{s_i} h_{s_j})^{-N\tilde{\gamma}/2} \alpha_{1,1}^{\tilde{\gamma}}(|s_i - s_j|)$. Arguing as before, we obtain (4.7) and consequently (4.5) holds.

Finally, combining (4.3), (4.4) and (4.5), we obtain

$$ \lim_{n \to \infty} nh_{s_n}^N \mathrm{Cov}[f_n^{(p)}(x), f_n^{(q)}(x)] = \frac{\beta_{-N,2}}{\beta_{0,1}^2} \mathbb{E}[\Phi_p(Y_0)\Phi_q(Y_0)|X_0 = x]f(x) \int_{\mathbb{R}^N} K^2(t)dt. $$

Now, we assume that $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5). Since $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d(N+2\tilde{\gamma}+2N(\gamma+\tau-1))}{2N(\gamma+\tau-1)}} \delta_{k,2}^{\tilde{\gamma}} < \infty$ implies $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{d(N+2\tilde{\gamma})}{2N(\gamma+\tau-1)}} |k|^d \delta_{k,2}^{\tilde{\gamma}} < \infty$ and using Lemma 5, there exists a sequence $(m_n)_{n \geq 1}$ of positive integers such that

$$ \lim_{n \to \infty} m_n^d h_{s_n}^{N(\gamma+\tau-1)} = \lim_{n \to \infty} h_{s_n}^{-\left(\frac{N}{2}+\tilde{\gamma}\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}} = 0. $$

Keeping in mind (3.2) and (4.1), we have $nh_{s_n}^N \mathrm{Cov}[f_n^{(p)}(x), f_n^{(q)}(x)] = C_{1,n} + C_{2,n} + C_{3,n} + C_{4,n}$ where

$$ C_{1,n} = nh_{s_n}^N \mathrm{Cov}[f_n^{(p)}(x) - \bar{f}_n^{(p)}(x), f_n^{(q)}(x) - \bar{f}_n^{(q)}(x)] $$
$$C_{2,n} = nh_{s_n}^N \operatorname{Cov}[f_n^{(p)}(x) - \bar{f}_n^{(p)}(x), \bar{f}_n^{(q)}(x)]$$

$$C_{3,n} = nh_{s_n}^N \operatorname{Cov}[\bar{f}_n^{(p)}(x), f_n^{(q)}(x) - \bar{f}_n^{(q)}(x)]$$

$$C_{4,n} = nh_{s_n}^N \operatorname{Cov}[\bar{f}_n^{(p)}(x), \bar{f}_n^{(q)}(x)].$$

First, we assume that $\|\Phi_p\|_{\infty} = \infty$ or $\|\Phi_q\|_{\infty} = \infty$. In this case, $\tilde{\gamma} = \theta/(2 + \theta)$. Moreover, we have

$$|C_{1,n}| \le \left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} (\Delta_{s_i}^{(p)} - \bar{\Delta}_{s_i}^{(p)}) \right\|_2 \\ \times \left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} (\Delta_{s_i}^{(q)} - \bar{\Delta}_{s_i}^{(q)}) \right\|_2.$$

Using (H1) and Proposition 3, we obtain for any $\ell \in \{p, q\}$,

$$\left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} (\Delta_{s_i}^{(\ell)} - \bar{\Delta}_{s_i}^{(\ell)}) \right\|_2 \\ \le m_n^d \left( \frac{h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2}{h_{s_i}^N} \right)^{1/2} h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\tilde{\gamma}} \le h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}}. \quad (4.8)$$

So, we derive

$$|C_{1,n}| \le \left( h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}} \right)^2 \underset{n \to \infty}{\longrightarrow} 0.$$

Similarly, we have

$$|C_{2,n}| \le \left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} (\Delta_{s_i}^{(p)} - \bar{\Delta}_{s_i}^{(p)}) \right\|_2 \\ \times \left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} \bar{\Delta}_{s_i}^{(q)} \right\|_2.$$

From (4.8), we know that

$$\left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} (\Delta_{s_i}^{(p)} - \bar{\Delta}_{s_i}^{(p)}) \right\|_2 \le h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}}.$$
Moreover, since $\bar{\Delta}_{s_i}^{(q)}$ and $\bar{\Delta}_{s_j}^{(q)}$ are independent as soon as $|s_i - s_j| > 2m_n$, we
have

$$
\begin{aligned}
& \left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} \bar{\Delta}_{s_i}^{(q)} \right\|_2^2 \\
&= \frac{h_{s_n}^N A_{n,0,1}^{-2}}{n w_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 h_{s_i}^{-N} \|\bar{\Delta}_{s_i}^{(q)}\|_2^2 \\
&\quad + \frac{2h_{s_n}^N A_{n,0,1}^{-2}}{n w_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} w_{s_i} w_{s_j} h_{s_i}^{-N/2} h_{s_j}^{-N/2} \mathbb{E}[\bar{\Delta}_{s_i}^{(q)} \bar{\Delta}_{s_j}^{(q)}]
\end{aligned}
$$

and, keeping in mind $\|\Delta_{s_i}^{(q)}\|_2 \le 1$ (see Lemma 3) and (H1), we have also

$$
\frac{h_{s_n}^N A_{n,0,1}^{-2}}{n w_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 h_{s_i}^{-N} \|\bar{\Delta}_{s_i}^{(q)}\|_2^2 \le \frac{h_{s_n}^N A_{n,0,1}^{-2}}{n w_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 h_{s_i}^{-N} \|\Delta_{s_i}^{(q)}\|_2^2 \le 1.
$$

Denoting $W_i^{(\ell)} = \Phi_\ell(Y_{s_i}) K_{s_i}(x, X_{s_i}) - \mathbb{E}[\Phi_\ell(Y_{s_i}) K_{s_i}(x, X_{s_i}) | \mathcal{H}_{i,m_n}]$ where $\mathcal{H}_{i,m_n} = \sigma(\eta_{s_i}, \varepsilon_{s_i-k}; |k| \le m_n)$ for any $1 \le i \le n$ and any $\ell \in \{p,q\}$ and noting that $\Delta_{s_i}^{(\ell)} - \bar{\Delta}_{s_i}^{(\ell)} = h_{s_i}^{-N/2} W_i^{(\ell)}$, we have

$$
|\mathbb{E}[\bar{\Delta}_{s_i}^{(q)} \bar{\Delta}_{s_j}^{(q)}] - \mathbb{E}[\Delta_{s_i}^{(q)} \Delta_{s_j}^{(q)}]| \le h_{s_j}^{-N/2} \|\Delta_{s_i}^{(q)}\|_2 \|W_j^{(q)}\|_2 + h_{s_i}^{-N/2} \|\Delta_{s_j}^{(q)}\|_2 \|W_i^{(q)}\|_2 \\
\le h_{s_n}^{-N/2} (\|W_j^{(q)}\|_2 + \|W_i^{(q)}\|_2).
$$

Using (3.12), we obtain

$$
|\mathbb{E}[\bar{\Delta}_{s_i}^{(q)} \bar{\Delta}_{s_j}^{(q)}] - \mathbb{E}[\Delta_{s_i}^{(q)} \Delta_{s_j}^{(q)}]| \le h_{s_n}^{-\left(\frac{N}{2} + \tilde{\gamma}\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\tilde{\gamma}}. \quad (4.9)
$$

Consequently, using Lemma 4, we obtain for any $i \neq j$,

$$
|\mathbb{E}[\bar{\Delta}_{s_i}^{(q)} \bar{\Delta}_{s_j}^{(q)}]| \le (h_{s_i} h_{s_j})^{\frac{N\gamma}{2}} + h_{s_n}^{-\frac{N\gamma}{2}+\tilde{\gamma}} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\tilde{\gamma}}
$$

where $\gamma$ is defined by (3.4). Since $\gamma \le 1$ and using the inequality $2ab \le a^2 + b^2$,
we derive

$$
\frac{h_{s_n}^N A_{n,0,1}^{-2}}{n w_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} w_{s_i} w_{s_j} h_{s_i}^{-N/2} h_{s_j}^{-N/2} |\mathbb{E}[\bar{\Delta}_{s_i}^{(q)} \bar{\Delta}_{s_j}^{(q)}]| \\
\le \frac{h_{s_n}^N A_{n,0,1}^{-2}}{n w_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2}
$$
$$
\begin{aligned}
& \times \left( (h_{s_i} h_{s_j})^{N\gamma/2} + h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\tilde{\gamma}} \right) \\
& \le \frac{h_{s_n}^N A_{n,0,1}^{-2}}{2nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} \frac{w_{s_i}^2 + w_{s_j}^2}{h_{s_n}^{N(1-\gamma)}} \\
& \quad + \frac{h_{s_n}^N A_{n,0,1}^{-2}}{2nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} (w_{s_i}^2 h_{s_i}^{-N} + w_{s_j}^2 h_{s_j}^{-N}) h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\tilde{\gamma}}.
\end{aligned}
$$

Using (H1) and keeping in mind that $h_{s_n}^{N(1-\tau)} \sum_{i=1}^n w_{s_i}^2 \le nw_{s_n}^2$, we get

$$
\begin{aligned}
& \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} w_{s_i} w_{s_j} h_{s_i}^{-N/2} h_{s_j}^{-N/2} |\mathbb{E}[\bar{\Delta}_{s_i}^{(q)} \bar{\Delta}_{s_j}^{(q)}]| \\
& \le m_n^d h_{s_n}^{N(\gamma+\tau-1)} + h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}} \xrightarrow{n \to \infty} 0. \quad (4.10)
\end{aligned}
$$

Consequently, we derive

$$
\left\| \frac{h_{s_n}^{N/2} A_{n,0,1}^{-1}}{\sqrt{n} w_{s_n}} \sum_{i=1}^{n} w_{s_i} h_{s_i}^{-N/2} \bar{\Delta}_{s_i}^{(q)} \right\|_2 \le 1
$$

and

$$
|C_{2,n}| \le h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}} \xrightarrow{n \to \infty} 0.
$$

Similarly, one can notice that

$$
|C_{3,n}| \le h_{s_n}^{-(\frac{N}{2}+\tilde{\gamma})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}} \xrightarrow{n \to \infty} 0.
$$

Now, we have to control the last term

$$
C_{4,n} = \frac{A_{n,0,1}^{-2} h_{s_n}^{N}}{nw_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 h_{s_i}^{-N} \mathbb{E}[\bar{\Delta}_{s_i}^{(p)} \bar{\Delta}_{s_i}^{(q)}] \\
+ \frac{2A_{n,0,1}^{-2} h_{s_n}^{N}}{nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} \mathbb{E}[\bar{\Delta}_{s_i}^{(p)} \bar{\Delta}_{s_j}^{(q)}]
$$
We are going to prove that

$$
\begin{align}
& \lim_{n \to \infty} \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 h_{s_i}^{-N} \mathbb{E}[\bar{\Delta}_{s_i}^{(p)} \bar{\Delta}_{s_i}^{(q)}] \nonumber \\
& = \frac{\beta_{-N,2}}{\beta_{0,1}^2} \mathbb{E}[\Phi_p(Y_0) \Phi_q(Y_0) | X_0 = x] f(x) \int_{\mathbb{R}^N} K^2(t) dt \tag{4.11}
\end{align}
$$

and

$$
\lim_{n \to \infty} \frac{2A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} \mathbb{E}[\bar{\Delta}_{s_i}^{(p)} \bar{\Delta}_{s_j}^{(q)}] = 0. \quad (4.12)
$$

Keeping in mind (3.12) and arguing as in (4.9), we have

$$
\begin{align*}
|\mathbb{E}[\bar{\Delta}_{s_i}^{(p)} \bar{\Delta}_{s_i}^{(q)}] - \mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_i}^{(q)}]| &\le h_{s_n}^{-N/2} (\|W_i^{(p)}\|_2 + \|W_i^{(q)}\|_2) \\
&\le h_{s_n}^{-\left(\frac{N}{2} + \tilde{\gamma}\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}}.
\end{align*}
$$

Using (H1), we obtain

$$
\frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 h_{s_i}^{-N} |\mathbb{E}[\bar{\Delta}_{s_i}^{(p)} \bar{\Delta}_{s_i}^{(q)}] - \mathbb{E}[\Delta_{s_i}^{(p)} \Delta_{s_i}^{(q)}]| \le h_{s_n}^{-\left(\frac{N}{2} + \tilde{\gamma}\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}} \\
\xrightarrow{n \to \infty} 0.
$$

Using (4.4), we obtain (4.11). Now, arguing as in (4.10), we get

$$
\begin{align*}
& \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{\substack{1 \le i < j \le n \\ |s_i - s_j| \le 2m_n}} w_{s_i} w_{s_j} (h_{s_i} h_{s_j})^{-N/2} |\mathbb{E}[\bar{\Delta}_{s_i}^{(p)} \bar{\Delta}_{s_j}^{(q)}]| \\
& \le m_n^d h_{s_n}^{N(\gamma+\tau-1)} + h_{s_n}^{-\left(\frac{N}{2}+\tilde{\gamma}\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\tilde{\gamma}} \xrightarrow{n \to \infty} 0.
\end{align*}
$$

So, (4.12) holds.

Finally, if $\|\Phi_p\|_{\infty} < \infty$ and $\|\Phi_q\|_{\infty} < \infty$ then $\tilde{\gamma} = 1$ and the proof follows
exactly the same lines as above. The proof of Proposition 4 is complete. $\square$

*Proof of Proposition 2.* Let $x \in \mathbb{R}^d$ and let $n$ be a positive integer. We have

$$
\mathbb{E}[(f_{n,\Phi}(x) - f_{\Phi}(x))^2] = \mathbb{V}(f_{n,\Phi}(x)) + (\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x))^2.
$$

Moreover,

$$
|\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x)| = \left|\left(\sum_{i=1}^{n} w_{s_i}\right)^{-1} \sum_{i=1}^{n} w_{s_i} \int_{\mathbb{R}^N} K(v) (f_{\Phi}(x - v h_{s_i}) - f_{\Phi}(x)) dv\right|.
$$
Using Taylor's formula, we derive

$$|\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x)| \leq \left(\sum_{i=1}^{n} w_{s_i}\right)^{-1} \sum_{i=1}^{n} w_{s_i} h_{s_i}^2.$$

Since $\max\{A_{n,0,1}^{-1}, A_{n,2,1}\} \leq 1$, we obtain $|\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x)| \leq A_{n,0,1}^{-1} A_{n,2,1} h_{s_n}^2 \leq h_{s_n}^2$. Finally, using Proposition 1, we have $\mathbb{V}[f_{n,\Phi}(x)] \leq (nh_{s_n}^N)^{-1}$ and for $h_{s_n} = n^{\frac{1}{4+N}}$, we get $\mathbb{E}[(f_{n,\Phi}(x) - f_{\Phi}(x))^2] \leq n^{\frac{4}{4+N}}$. The proof of Proposition 2 is complete. □

**Proof of Theorem 1.** We are going to split the proof in two parts. In the first part, we deal simultaneously with the strong mixing case and the weakly mixing case (see (4.15) below) whereas, in the second part, the two dependence conditions are investigated separately.

## First part

Let $n$ be a positive integer and $x \in \mathbb{R}^N$ be fixed. One can notice that

$$\sqrt{nh_{s_n}^N} (f_{n,\Phi}(x) - \mathbb{E}[f_{n,\Phi}(x)]) = \frac{1}{\sqrt{n}} \sum_{i=1}^{n} U_{s_i}$$

where

$$U_{s_i} = \frac{h_{s_n}^{N/2} w_{s_i} \Delta_{s_i}}{h_{s_i}^{N/2} w_{s_n} A_{n,0,1}} \quad \text{and} \quad \Delta_{s_i} = \frac{\Phi(Y_{s_i}) K_{s_i}(x, X_{s_i}) - \mathbb{E}[\Phi(Y_0) K_{s_i}(x, X_0)]}{h_{s_i}^{N/2}}. \qquad (4.13)$$

In the sequel, $(m_n)_{n \geq 1}$ is the sequence defined by Lemma 5 which satisfies $m_n^d h_{s_n}^{N(\nu_4(\theta)+\tau-1)} \to 0$ and either $h_{s_n}^{-N\nu_2(\theta)} \sum_{|k|>m_n} \alpha_{1,\infty}^{\nu_2(\theta)}(|k|) \to 0$ (if $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing) or $h_{s_n}^{-(\nu_2(\theta)+\frac{N}{2})} \sum_{|k|>m_n} |k|^d \delta_{k,2}^{\nu_2(\theta)} \to 0$ (if $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5)) where $\nu_2(\theta) = \mathbb{1}\{\|\Phi\|_{\infty} < \infty\} + \frac{\theta}{2+\theta} \mathbb{1}\{\|\Phi\|_{\infty} = \infty\}$. Moreover, if $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5), we consider the notations

$$\bar{\Delta}_{s_i} = \mathbb{E}[\Delta_{s_i} | \mathcal{H}_{i,m_n}] \quad \text{and} \quad \bar{U}_{s_i} = \mathbb{E}[U_{s_i} | \mathcal{H}_{i,m_n}].$$

where $\mathcal{H}_{i,m_n} = \sigma(\eta_{s_i}, \varepsilon_{s_i-k}; |k| \leq m_n)$.

Note that $\bar{U}_{s_i}$ and $\bar{U}_{s_j}$ are independent if $|s_i - s_j| > 2m_n$. Using (H1) and Proposition 3, we derive

$$\frac{1}{\sqrt{n}} \left\| \sum_{i=1}^{n} (U_{s_i} - \bar{U}_{s_i}) \right\|_2 \leq h_{s_n}^{-(\nu_2(\theta)+\frac{N}{2})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\nu_2(\theta)} \xrightarrow{n \to +\infty} 0. \quad (4.14)$$

From now on, we denote

$$(Z_{s_i}, M_n) = \begin{cases} (U_{s_i}, m_n) & \text{if } (X_i)_{i \in \mathbb{Z}^d} \text{ is strongly mixing} \\ (\bar{U}_{s_i}, 2m_n) & \text{if } (X_i)_{i \in \mathbb{Z}^d} \text{ is of the form (1.5)} \end{cases} \quad (4.15)$$
and it suffices to prove the asymptotic normality of the partial sums $n^{-1/2} \sum_{i=1}^n Z_{s_i}$ as $n$ goes to infinity. Let $(\xi_k)_{k \in \mathbb{Z}^d}$ be independent normal random variables independent of $(X_k)_{k \in \mathbb{Z}^d}$ and $(\eta_k)_{k \in \mathbb{Z}^d}$ and such that $\mathbb{E}[\xi_k] = 0$ and $\mathbb{E}[\xi_k^2] = \mathbb{E}[Z_k^2]$. Let $1 \le i \le n$ and define $T_{s_i} = n^{-1/2} Z_{s_i}$ and $\Xi_{s_i} = n^{-1/2} \xi_{s_i}$. One can notice that $\sum_{i=1}^n \Xi_{s_i}$ is a gaussian random variable with zero mean. If $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing then $Z_{s_i} = U_{s_i}$ and

$$ 
\begin{aligned}
\mathbb{V}\left(\sum_{i=1}^{n} \Xi_{s_i}\right) &= \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[U_{s_i}^2] \\
&= \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2}{h_{s_i}^N} \times \frac{\mathbb{E}[\Phi(Y_0)^2 K_{s_i}^2(x, X_0)] - (\mathbb{E}[\Phi(Y_0)K_{s_i}(x, X_0)])^2}{h_{s_i}^N}.
\end{aligned}
$$

Keeping in mind (2.6) and (H1) and using Lemma 1, Lemma 2 and Lemma 3, we get

$$ \lim_{n \to +\infty} \mathbb{V}\left(\sum_{i=1}^{n} \Xi_{s_i}\right) = \sigma_{\Phi}^2(x). \quad (4.16) $$

If $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5) then $Z_{s_i} = \bar{U}_{s_i}$ and applying (3.12), we get

$$ 
\begin{aligned}
|\mathbb{E}[\bar{U}_{s_i}^2] - \mathbb{E}[U_{s_i}^2]| &\le 2 \|U_{s_i}\|_2 \|U_{s_i} - \bar{U}_{s_i}\|_2 \\
&\le \frac{h_{s_n}^N w_{s_i}^2}{h_{s_i}^N w_{s_n}^2 A_{n,0,1}^2} \times h_{s_n}^{-(\frac{N}{2}+\nu_2(\theta))} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\nu_2(\theta)}.
\end{aligned}
$$

Using (H1), we derive

$$ 
\begin{aligned}
\frac{1}{n} \sum_{i=1}^{n} |\mathbb{E}[\bar{U}_{s_i}^2] - \mathbb{E}[U_{s_i}^2]| &\le \frac{h_{s_n}^N \sum_{i=1}^n h_{s_i}^{-N} w_{s_i}^2}{nw_{s_n}^2 A_{n,0,1}^2} \times h_{s_n}^{-(\nu_2(\theta)+\frac{N}{2})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\nu_2(\theta)} \\
&\le h_{s_n}^{-(\nu_2(\theta)+\frac{N}{2})} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\nu_2(\theta)} \xrightarrow{n \to \infty} 0.
\end{aligned}
$$

So, we get also (4.16) when $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5). Let $\psi$ be any measurable function from $\mathbb{R}$ to $\mathbb{R}$. For any $1 \le i \le j \le n$, we introduce the notation

$$ \psi_{i,j} = \psi\left(\sum_{\ell=1}^{i} T_{s_{\ell}} + \sum_{\ell=j}^{n} \Xi_{s_{\ell}}\right). $$

Let $h: \mathbb{R} \to \mathbb{R}$ be a three times continuously differentiable function such that $\max_{0 \le i \le 3} \|h^{(i)}\|_{\infty} \le 1$. Keeping in mind (4.16), it suffices to prove $\lim_{n \to \infty} |L_n| = 0$, where

$$ L_n := \mathbb{E}\left[h\left(\sum_{i=1}^{n} T_{s_i}\right)\right] - \mathbb{E}\left[h\left(\sum_{i=1}^{n} \Xi_{s_i}\right)\right]. $$
Using Lindeberg's idea [23] (see also [9]), we have

$$ 
\begin{aligned}
L_n &= \mathbb{E}[h_{n,n+1} - h_{0,1}] = \sum_{i=1}^{n} \mathbb{E}[h_{i,i+1} - h_{i-1,i}] \\
&= \sum_{i=1}^{n} \left( \mathbb{E}[h_{i,i+1} - h_{i-1,i+1}] - \mathbb{E}[h_{i-1,i} - h_{i-1,i+1}] \right).
\end{aligned}
$$

Applying Taylor's formula, we get

$$ L_n = \sum_{i=1}^{n} \left( \mathbb{E}[T_{s_i} h'_{i-1,i+1} + \frac{1}{2} T_{s_i}^2 h''_{i-1,i+1} + \beta_i] - \mathbb{E}[\Xi_i h'_{i-1,i+1} + \frac{1}{2} \Xi_{s_i}^2 h''_{i-1,i+1} + \rho_i] \right), $$

where $|\beta_i| \le T_{s_i}^2 (1 \wedge |T_{s_i}|)$ and $|\rho_i| \le \Xi_{s_i}^2 (1 \wedge |\Xi_{s_i}|)$. Since $\Xi_i^2$ and $h''_{i-1,i+1}$ are independent, $\mathbb{E}[\Xi_{s_i} h'_{i-1,i+1}] = 0$ and $\mathbb{E}[\Xi_{s_i}^2] = n^{-1}\mathbb{E}[Z_{s_i}^2]$, we obtain

$$ L_n = \sum_{i=1}^{n} \left( \mathbb{E}[T_{s_i} h'_{i-1,i+1}] + \frac{1}{2} \mathbb{E}[(T_{s_i}^2 - n^{-1}\mathbb{E}[Z_{s_i}^2]) h''_{i-1,i+1}] + \mathbb{E}[\beta_i - \rho_i] \right). $$

Since, for any $1 \le i \le n$, the random variable $\xi_{s_i}$ is gaussian with zero mean and variance $\mathbb{E}[Z_{s_i}^2]$, we have

$$ \mathbb{E}[|\xi_{s_i}|^3] = \sqrt{8/\pi} (\mathbb{E}[Z_{s_i}^2])^{3/2} \le \sqrt{8/\pi} (\mathbb{E}[U_{s_i}^2])^{3/2}. $$

Moreover, since $(w_{s_n} h_{s_n}^{-N})_{n \ge 1}$ is nondecreasing, we get

$$ 
\begin{aligned}
(\mathbb{E}[U_{s_i}^2])^{3/2} &= \frac{h_{s_n}^N w_{s_i}^2}{h_{s_i}^N w_{s_n}^2 A_{n,0,1}^2} \left( \frac{h_{s_n}^N w_{s_i}^2}{h_{s_i}^N w_{s_n}^2 A_{n,0,1}^2} \right)^{1/2} (\mathbb{E}[\Delta_{s_i}^2])^{3/2} \\
&\le \frac{h_{s_n}^N w_{s_i}^2}{h_{s_i}^N w_{s_n}^2 A_{n,0,1}^2} (h_{s_i}^N h_{s_n}^{-N})^{1/2} (\mathbb{E}[\Delta_{s_i}^2])^{3/2}.
\end{aligned}
$$

Using (H1) and $\mathbb{E}[\Delta_{s_i}^2] \le 1$ (see Lemma 3), we get

$$ \sum_{i=1}^{n} \mathbb{E}[|\rho_i|] \le \frac{1}{\sqrt{nh_{s_n}^N}} \times \frac{h_{s_n}^N A_{n,0,1}^{-2}}{w_{s_n}^2 n} \sum_{i=1}^{n} w_{s_i}^2 h_{s_i}^{-N} (\mathbb{E}[\Delta_{s_i}^2])^{3/2} \le (nh_{s_n}^N)^{-1/2} \xrightarrow{n \to +\infty} 0. $$

In the other part, if $\|\Phi\|_{\infty} < \infty$ then $|U_{s_i}| \le h_{s_n}^{-N/2}$ (since $(w_{s_n} h_{s_n}^{-N})_{n \ge 1}$ is nondecreasing) and

$$ 
\begin{aligned}
\sum_{i=1}^{n} \mathbb{E}[|\beta_i|] &\le \frac{1}{n^{3/2}} \sum_{i=1}^{n} \mathbb{E}[|Z_{s_i}|^3] \le \frac{1}{n^{3/2}} \sum_{i=1}^{n} \mathbb{E}[|U_{s_i}|^3] \\
&\le \frac{1}{\sqrt{nh_{s_n}^N}} \times \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[U_{s_i}^2] \le (nh_{s_n}^N)^{-1/2} \xrightarrow{n \to +\infty} 0.
\end{aligned}
$$
Since $(w_{s_n}h_{s_n}^{-N})_{n\ge 1}$ is nondecreasing, if $\|\Phi\|_{\infty} = \infty$ then

$$ \begin{aligned} \mathbb{E}[|U_{s_i}|^{2+\theta}] &= \left( \frac{h_{s_n}^{N/2} w_{s_i}}{h_{s_i}^{N/2} w_{s_n} A_{n,0,1}} \right)^2 \left( \frac{h_{s_n}^{N/2} w_{s_i}}{h_{s_i}^{N/2} w_{s_n} A_{n,0,1}} \right)^{\theta} \mathbb{E}[|\Delta_{s_i}|^{2+\theta}] \\ &\le \frac{h_{s_n}^N w_{s_i}^2}{h_{s_i}^N w_{s_n}^2 A_{n,0,1}^2} \left( h_{s_i}^{N/2} h_{s_n}^{-N/2} \right)^{\theta} h_{s_i}^{-N/2} = h_{s_n}^{-N/2} \frac{h_{s_n}^N w_{s_i}^2}{h_{s_i}^N w_{s_n}^2 A_{n,0,1}^2}. \end{aligned} \quad (4.17) $$

Consequently, if $d_n := (nh_{s_n}^N)^{\frac{\theta}{2(\theta+1)}}$ then

$$ \begin{aligned} \sum_{i=1}^{n} \mathbb{E}[|\beta_i|] &\le \frac{1}{d_n^{\theta} n^{\theta/2}} \times \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[|U_{s_i}|^{2+\theta}] + \frac{d_n}{n} \sum_{i=1}^{n} \mathbb{E}[U_{s_i}^2] \\ &\le d_n^{-\theta} (nh_{s_n}^N)^{-\theta/2} \times \frac{A_{n,0,1}^{-2} h_{s_n}^N}{w_{s_n}^2 n} \sum_{i=1}^{n} \frac{w_{s_i}^2}{h_{s_i}^N} + d_n. \end{aligned} $$

Using (H1), we get

$$ \sum_{i=1}^{n} \mathbb{E}[|\beta_i|] \le \frac{1}{d_n^{\theta} (nh_{s_n}^N)^{\theta/2}} + d_n = 2d_n \xrightarrow{n\to\infty} 0. $$

Now, we have to prove that

$$ \lim_{n \to \infty} \sum_{i=1}^{n} \left( \mathbb{E}[T_{s_i} h'_{i-1, i+1}] + (2n)^{-1} \mathbb{E}[(Z_{s_i}^2 - \mathbb{E}[Z_{s_i}^2]) h''_{i-1, i+1}] \right) = 0. \quad (4.18) $$

For any $1 \le i < j \le n$ and any function $\psi$ from $\mathbb{R}$ to $\mathbb{R}$, we define also

$$ \psi_{i-1,j}^{(M_n)} = \psi \left( \sum_{\substack{l=1 \\ |s_l - s_i| > M_n}}^{i-1} T_{s_l} + \sum_{l=j}^{n} \Xi_{s_l} \right). $$

Using Taylor's formula, we have

$$ T_{s_i} h'_{i-1, i+1} = T_{s_i} h'^{(M_n)}_{i-1, i+1} + T_{s_i} \left( \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} T_{s_l} \right) h''^{(M_n)}_{i-1, i+1} + \beta'_{i} $$

with

$$ |\beta'_{i}| \le 2 \left| T_{s_i} \left( \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} T_{s_l} \right) \left( 1 \wedge \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} T_{s_l} \right) \right|. \quad (4.19) $$

In order to obtain (4.18), we have to prove

$$ \lim_{n \to \infty} \sum_{i=1}^{n} \mathbb{E}[T_{s_i} h'_{i-1, i+1}^{(M_n)}] = 0, \quad (4.20) $$
$$ \lim_{n \to \infty} \sum_{i=1}^{n} \mathbb{E} \left[ T_{s_i} \left( \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} T_{s_l} \right) h''_{i-1, i+1} \right] = 0 \quad (4.21) $$

$$ \lim_{n \to \infty} \sum_{i=1}^{n} \mathbb{E}[|\beta'_{i}|] = 0, \quad (4.22) $$

and

$$ \lim_{n \to \infty} \frac{1}{n} \sum_{i=1}^{n} \mathbb{E} \left[ (Z_{s_i}^2 - \mathbb{E}[Z_{s_i}^2]) h''_{i-1, i+1} \right] = 0. \quad (4.23) $$

## Second part

First, we assume that $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing. We are going to prove (4.20). Since $\Xi$ is independent of $T$, then $\mathbb{E}[T_{s_i} h' (\sum_{r=i+1}^n \Xi_{s_r})] = 0$. So, if we define $E_i^{(n)} = \{1 \le j < i \mid |s_j - s_i| > M_n\}$ and $\pi$ is a one to one map from $[1, |E_i^{(n)}|] \cap \mathbb{Z}$ to $E_i^{(n)}$ such that $|s_{\pi(\ell)} - s_i| \le |s_{\pi(\ell-1)} - s_i|$ then

$$ \begin{aligned} \mathbb{E}[T_{s_i} h'_{i-1, i+1}^{(M_n)}] &= \mathbb{E} \left[ T_{s_i} \left( h'_{i-1, i+1} - h' \left( \sum_{r=i+1}^n \Xi_{s_r} \right) \right) \right] \\ &= \sum_{\ell=1}^{|E_i^{(n)}|} \mathrm{Cov} [T_{s_i}, t_\ell - t_{\ell-1}], \end{aligned} $$

where $t_\ell = h' (\sum_{r=1}^\ell T_{s_{\pi(r)}} + \sum_{r=i+1}^n \Xi_{s_r})$ and $\sum_{r=1}^0 T_{s_{\pi(r)}} = 0$. Since $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing, using Rio's inequality ([32], Theorem 1.1) and keeping in mind that $|s_{\pi(\ell)} - s_i| \le |s_{\pi(\ell-1)} - s_i|$, we get

$$ |\mathbb{E}[T_{s_i} h'_{i-1, i+1}^{(M_n)}]| \le 2 \sum_{\ell=1}^{|E_i^{(n)}|} \int_0^{2\alpha_{1,\infty}(|s_{\pi(\ell)} - s_i|)} Q_{T_{s_i}}(u) Q_{t_\ell - t_{\ell-1}}(u) du. $$

Assume that $\|\Phi\|_\infty < \infty$ and let $u \in [0, 1[$ be fixed. Since $h'$ is Lipschitz and $|U_{s_i}| \le h_{s_n}^{-N/2}$ (because $(w_{s_n} h_{s_n}^{-N})_{n \ge 1}$ is nondecreasing), we have $\max \{Q_{T_{s_i}}(u), Q_{t_\ell - t_{\ell-1}}(u)\} \le (nh_{s_n}^N)^{-1/2}$ and we derive

$$ \sum_{i=1}^{n} |\mathbb{E}[T_{s_i} h'_{i-1, i+1}^{(M_n)}]| \le \frac{1}{nh_{s_n}^N} \sum_{i=1}^{n} \sum_{\ell=1}^{|E_i^{(n)}|} \alpha_{1,\infty} (|s_{\pi(\ell)} - s_i|) \le h_{s_n}^{-N} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \alpha_{1,\infty} (|k|). $$

Using Lemma 6, if $\|\Phi\|_\infty = \infty$ then

$$ Q_{T_{s_i}}(u) \le \frac{u^{-\frac{1}{2+\theta}} \|U_{s_i}\|_{2+\theta}}{\sqrt{n}} \le \frac{u^{-\frac{1}{2+\theta}}}{\sqrt{n} h_{s_n}^{\frac{N\theta}{2(2+\theta)}}} \times \frac{h_{s_n}^{N/2} w_{s_i}}{h_{s_i}^{N/2} w_{s_n} A_{n,0,1}} $$
and

$$Q_{t_{\ell}-t_{\ell-1}}(u) \le \frac{u^{-\frac{1}{2+\theta}} \|U_{s_{\pi(\ell)}}\|_{2+\theta}}{\sqrt{n}} \le \frac{u^{-\frac{1}{2+\theta}}}{\sqrt{n}h_{s_n}^{\frac{N\theta}{2(2+\theta)}}} \times \frac{h_{s_n}^{N/2} w_{s_{\pi(\ell)}}}{h_{s_{\pi(\ell)}}^{N/2} w_{s_n} A_{n,0,1}}.$$

Consequently, using again the inequality $2ab \le a^2 + b^2$, we obtain

$$\begin{align*}
\sum_{i=1}^{n} |\mathbb{E}[T_{s_i} h'_{i-1, i+1}(M_n)]| &\le \frac{h_{s_n}^{-\frac{N\theta}{2+\theta}}}{n} \sum_{i=1}^{n} \sum_{\ell=1}^{|E_i^{(n)}|} \frac{h_{s_n}^N w_{s_i} w_{s_{\pi(\ell)}}}{h_{s_i}^{N/2} h_{s_{\pi(\ell)}}^{N/2} w_{s_n}^2 A_{n,0,1}^{2}} \alpha_{1,\infty}^{\frac{\theta}{2+\theta}} (|s_{\pi(\ell)} - s_i|) \\
&\le h_{s_n}^{-\frac{N\theta}{2+\theta}} \times \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} \sum_{\ell=1}^{|E_i^{(n)}|} \left( \frac{w_{s_i}^2}{h_{s_i}^N} + \frac{w_{s_{\pi(\ell)}}^2}{h_{s_{\pi(\ell)}}^N} \right) \alpha_{1,\infty}^{\frac{\theta}{2+\theta}} (|s_{\pi(\ell)} - s_i|) \\
&= h_{s_n}^{-\frac{N\theta}{2+\theta}} \times \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| > M_n}}^{i-1} \left( \frac{w_{s_i}^2}{h_{s_i}^N} + \frac{w_{s_j}^2}{h_{s_j}^N} \right) \alpha_{1,\infty}^{\frac{\theta}{2+\theta}} (|s_j - s_i|) \\
&\le h_{s_n}^{-\frac{N\theta}{2+\theta}} \times \frac{2A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2}{h_{s_i}^N} \times \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \alpha_{1,\infty}^{\frac{\theta}{2+\theta}} (|k|).
\end{align*}$$

Using (H1), we get

$$\sum_{i=1}^{n} |\mathbb{E}[T_{s_i} h'_{i-1, i+1}(M_n)]| \le h_{s_n}^{-\frac{N\theta}{2+\theta}} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \alpha_{1,\infty}^{\frac{\theta}{2+\theta}} (|k|).$$

Finally, we proved that

$$\sum_{i=1}^{n} |\mathbb{E}[T_{s_i} h'_{i-1, i+1}(M_n)]| \le h_{s_n}^{-N\nu_2(\theta)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \alpha_{1,\infty}^{\nu_2(\theta)} (|k|) \xrightarrow{n \to +\infty} 0$$

where $\nu_2(\theta) = \mathbb{1}_{\{\|\Phi\|_{\infty}<\infty\}} + \frac{\theta}{2+\theta} \mathbb{1}_{\{\|\Phi\|_{\infty}=\infty\}}$. So, (4.20) holds.

The proof of the following lemma is postponed to section 5.

**Lemma 8.** It holds that $\sup_{1 \le i,j \le n} \mathbb{E}[|U_{s_i} U_{s_j}|] \le w_{s_i} w_{s_j} w_{s_n}^{-2} h_{s_n}^{N\nu_4(\theta)}$

where $\nu_4(\theta) = \mathbb{1}_{\{\|\Phi\|_{\infty}<\infty\}} + \frac{\theta}{4+\theta} \mathbb{1}_{\{\|\Phi\|_{\infty}=\infty\}}$.

Since $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing, we have $Z_{s_i} = U_{s_i}$ for any $1 \le i \le n$. Moreover, using (4.19), we have

$$\sum_{i=1}^{n} \mathbb{E}[|\beta'_{i}|] \le 2 \sum_{i=1}^{n} \mathbb{E} \left[ \frac{|Z_{s_i}|}{\sqrt{n}} \left( \sum_{j=1}^{i-1} \frac{|Z_{s_j}|}{\sqrt{n}} \right) \right] = \frac{2}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} \mathbb{E}[|U_{s_i} U_{s_j}|] \quad (4.24)$$
and

$$
\begin{align*}
\sum_{i=1}^{n} \left| \mathbb{E} \left[ T_{s_i} \left( \sum_{\substack{\ell=1 \\ |s_\ell - s_i| \le M_n}}^{i-1} T_{s_\ell} \right) h_{i-1, i+1}^{\prime\prime}(M_n) \right] \right| &\le \sum_{i=1}^{n} \mathbb{E} \left[ \frac{|Z_{s_i}|}{\sqrt{n}} \left( \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} \frac{|Z_{s_j}|}{\sqrt{n}} \right) \right] \\
&= \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} \mathbb{E}[|U_{s_i} U_{s_j}|]. \quad (4.25)
\end{align*}
$$

Using Lemma 8 and the inequality 2ab ≤ a² + b², we get

$$
\frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} \mathbb{E}[|U_{s_i} U_{s_j}|] \le \frac{M_n^d h_{s_n}^{N\nu_4(\theta)}}{nw_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 + \frac{h_{s_n}^{N\nu_4(\theta)}}{nw_{s_n}^2} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} w_{s_j}^2.
$$

Moreover,

$$
\sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} w_{s_j}^2 = \sum_{j=1}^{n-1} w_{s_j}^2 \sum_{\substack{i=j+1 \\ |s_i - s_j| \le M_n}}^{n} 1 \le M_n^d \sum_{i=1}^{n} w_{s_i}^2.
$$

Consequently, using (H1), we obtain

$$
\begin{align*}
\frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} \mathbb{E}[|U_{s_i} U_{s_j}|] &\le \frac{M_n^d h_{s_n}^{N\nu_4(\theta)}}{nw_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 \\
&= M_n^d h_{s_n}^{N(\nu_4(\theta)+\tau-1)} \times \frac{h_{s_n}^{N(1-\tau)}}{nw_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 \\
&\le m_n^d h_{s_n}^{N(\nu_4(\theta)+\tau-1)} \xrightarrow{n\to+\infty} 0. \tag{4.26}
\end{align*}
$$

Combining (4.24), (4.25) and (4.26), we obtain (4.21) and (4.22).

Now, it suffices to prove (4.23). Let $\beta \ge 1$ be a positive integer. For any $1 \le j \le n$, the notation $\mathbb{E}_\beta[Z_{s_j}]$ will stand for the conditional expectation of $Z_{s_j}$ with respect to the $\sigma$-algebra $\sigma(Z_{s_\ell}; \ell < j$ and $|s_\ell - s_j| \ge \beta)$. Then,

$$
\frac{1}{n} \sum_{i=1}^{n} |\mathbb{E}[(Z_{s_i}^2 - \mathbb{E}[Z_{s_i}^2]) h''_{i-1, i+1}]| \le I_1 + I_2,
$$

where

$$
I_1 = \frac{1}{n} \sum_{i=1}^{n} |\mathbb{E}[(Z_{s_i}^2 - \mathbb{E}_{\beta}[Z_{s_i}^2]) h''_{i-1, i+1}]
$$

and

$$
I_2 = \frac{1}{n} \sum_{i=1}^{n} |\mathbb{E}[(\mathbb{E}_{\beta}[Z_{s_i}^2] - \mathbb{E}[Z_{s_i}^2]) h''_{i-1, i+1}]|.
$$

The next result can be found in [27].
**Lemma 9.** Let $\mathcal{U}$ and $\mathcal{V}$ be two $\sigma$-algebras and let $X$ be a random variable which is measurable with respect to $\mathcal{U}$.

If $1 \le p \le r \le \infty$, then $\|\mathbb{E}[X|\mathcal{V}] - \mathbb{E}[X]\|_p \le 2(2^{1/p} + 1) (\alpha(\mathcal{U}, \mathcal{V}))^{\frac{1}{p} - \frac{1}{r}} \|X\|_r$.

Since $(X_k)_{k \in \mathbb{Z}^d}$ is strongly mixing, we have $Z_{s_i} = U_{s_i}$ for any $1 \le i \le n$. If $\|\Phi\|_{\infty} = \infty$ then using Lemma 9 with $p=1$ and $r = (2+\theta)/2$, we derive

$$I_2 \le \frac{1}{n} \sum_{i=1}^{n} \| \mathbb{E}_{\beta}[U_{s_i}^2] - \mathbb{E}[U_{s_i}^2] \|_1 \le \frac{6 \alpha_{1, \infty}^{\frac{\theta}{2+\theta}}(\beta)}{n} \sum_{i=1}^{n} \| U_{s_i} \|_{2+\theta}^2.$$

From Lemma 6, we have

$$\|U_{s_i}\|_{2+\theta}^2 \le \frac{h_{s_n}^{-\frac{N\theta}{2+\theta}} h_{s_n}^N w_{s_i}^2}{h_{s_i}^N w_{s_n}^2 A_{n,0,1}^2}.$$

Consequently, using (H1), we get

$$I_2 \le h_{s_n}^{-\frac{N\theta}{2+\theta}} \alpha_{1,\infty}^{\frac{\theta}{2+\theta}}(\beta) \times \frac{A_{n,0,1}^{-2} h_{s_n}^N}{n w_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2}{h_{s_i}^N} \le h_{s_n}^{-\frac{N\theta}{2+\theta}} \alpha_{1,\infty}^{\frac{\theta}{2+\theta}}(\beta).$$

Similarly, if $\|\Phi\|_{\infty} < \infty$ then $|\Delta_{s_i}| \le h_{s_i}^{-N/2}$ and since $(w_{s_n}h_{s_n}^{-N})_{n \ge 1}$ is nondecreasing, we have $|U_{s_i}| \le h_{s_n}^{-N/2}$ and

$$I_2 \le 6 \alpha_{1,\infty}(\beta) \times \frac{1}{n} \sum_{i=1}^{n} \|U_{s_i}\|_{\infty}^2 \le h_{s_n}^{-N} \alpha_{1,\infty}(\beta).$$

Finally, keeping in mind that $\nu_2(\theta) = 1_{\{\|\Phi\|_{\infty} < \infty\}} + \frac{\theta}{2+\theta} 1_{\{\|\Phi\|_{\infty} = \infty\}}$, it means that

$$I_2 \le \frac{1}{n} \sum_{i=1}^{n} \| \mathbb{E}_{\beta}[U_{s_i}^2] - \mathbb{E}[U_{s_i}^2] \|_1 \le h_{s_n}^{-N\nu_2(\theta)} \alpha_{1,\infty}^{\nu_2(\theta)}(\beta). \quad (4.27)$$

Now, we make the choice

$$\beta = \left[ h_{s_n}^{\frac{-N\nu_2(\theta)(\nu_4(\theta)+\tau-1)}{\nu_4(\theta)+\tau-1}} \right]. \quad (4.28)$$

Consequently, since $\sum_{k \in \mathbb{Z}^d} |k|^{\frac{\nu_2(\theta)}{\nu_4(\theta)+\tau-1}} \alpha_{1,\infty}^{\nu_2(\theta)}(|k|) < \infty$ means $\sum_{\ell=1}^{\infty} \ell^{\frac{\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)}{\nu_4(\theta)+\tau-1}} \alpha_{1,\infty}^{\nu_2(\theta)}(\ell) < \infty$, we derive

$$I_2 \le \beta^{\frac{\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)}{\nu_4(\theta)+\tau-1}} \alpha_{1,\infty}^{\nu_2(\theta)}(\beta) \xrightarrow{n \to \infty} 0.$$

Since $h''_{i-1, i+1}(\beta)$ is $\sigma(\Xi_{s_\ell}, Z_{s_k}; 1 \le k < i, |s_k - s_i| \ge \beta, \ell \in \mathbb{N}^*)$-measurable, we find that $\mathbb{E}[(Z_{s_i}^2 - \mathbb{E}_{\beta}[Z_{s_i}^2])h''_{i-1, i+1}(\beta)] = 0$ and consequently

$$\mathbb{E}[(Z_{s_i}^2 - \mathbb{E}_{\beta}[Z_{s_i}^2]) h''_{i-1, i+1}(\beta)] = \mathbb{E}[(Z_{s_i}^2 - \mathbb{E}_{\beta}[Z_{s_i}^2]) (h''_{i-1, i+1}(\beta) - h''_{i-1, i+1}(\beta))].$$
Keeping in mind that

$$I_1 = \frac{1}{n} \sum_{i=1}^{n} |\mathbb{E}[(Z_{s_i}^2 - \mathbb{E}_{\beta}[Z_{s_i}^2]) h''_{i-1, i+1}|],$$

we obtain

$$I_1 \le \frac{1}{n} \sum_{i=1}^{n} \mathbb{E} \left[ \left( 2 \wedge \left| \sum_{j=1}^{i-1} \frac{Z_{s_j}}{\sqrt{n}} \right| \right) (Z_{s_i}^2 + \mathbb{E}_{\beta}[Z_{s_i}^2]) \right].$$

Let $L > 0$, then

$$I_1 \le \frac{L}{n^{3/2}} \sum_{i=1}^{n} \sum_{|s_j - s_i| \le \beta} \mathbb{E}[|Z_{s_j}| |Z_{s_i}| 1_{|Z_{s_i}| \le L}] + \frac{2}{n} \sum_{i=1}^{n} \mathbb{E}[Z_{s_i}^2 1_{|Z_{s_i}| > L}] \\ + \frac{1}{n} \sum_{i=1}^{n} \mathbb{E} \left[ \left( 2 \wedge \left| \sum_{j=1}^{i-1} \frac{Z_{s_j}}{\sqrt{n}} \right| \right) \left( \mathbb{E}_{\beta}[Z_{s_i}^2] - \mathbb{E}[Z_{s_i}^2] \right) \right] \\ + \frac{1}{n} \sum_{i=1}^{n} \mathbb{E} \left[ \left( 2 \wedge \left| \sum_{j=1}^{i-1} \frac{Z_{s_j}}{\sqrt{n}} \right| \right) \mathbb{E}[Z_{s_i}^2] \right].$$

Since $Z_{s_i} = U_{s_i}$ for any $1 \le i \le n$, we derive

$$I_1 \le \frac{L}{n^{3/2}} \sum_{i=1}^{n} \sum_{|s_j - s_i| \le \beta} \mathbb{E}[|U_{s_i} U_{s_j}|] + \frac{2}{n} \sum_{i=1}^{n} \mathbb{E}[U_{s_i}^2 1_{|U_{s_i}| > L}] \\ + \frac{2}{n} \sum_{i=1}^{n} \|\mathbb{E}_{\beta}[U_{s_i}^2] - \mathbb{E}[U_{s_i}^2]\|_1 + \frac{1}{n} \sum_{i=1}^{n} \left\| \sum_{j=1}^{i-1} \frac{U_{s_j}}{\sqrt{n}} \right\|_2 \mathbb{E}[U_{s_i}^2].$$

Arguing as in (4.26) and Lemma 9, we derive

$$I_1 \le \frac{\beta^d L h_{s_n}^{N(\nu_4(\theta)+\tau-1)}}{\sqrt{n}} + \frac{L^{-\theta}}{n} \sum_{i=1}^{n} \mathbb{E}[|U_{s_i}|^{2+\theta}] + h_{s_n}^{-N\nu_2(\theta)} \alpha_{1,\infty}^{\nu_2(\theta)}(\beta) \\ + \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[U_{s_i}^2] \left\| \sum_{j=1}^{i-1} \frac{U_{s_j}}{\sqrt{n}} \right\|_2.$$
Using (4.17), (4.28) and (H1), we get

$$
I_1 \le \frac{\beta^d L h_{s_n}^{N(\nu_4(\theta)+\tau-1)}}{\sqrt{n}} + L^{-\theta} h_{s_n}^{-\frac{N\theta}{2}} + \beta \frac{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)}{\nu_4(\theta)+\tau-1} \alpha_{1,\infty}^{\nu_2(\theta)}(\beta) \\
+ \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[U_{s_i}^2] \left\| \sum_{\substack{j=1 \\ |s_j-s_i| \le \beta}}^{i-1} \frac{U_{s_j}}{\sqrt{n}} \right\|_2.
$$

Moreover,

$$
\left\| \sum_{\substack{j=1 \\ |s_j - s_i| \le \beta}}^{i-1} \frac{U_{s_j}}{\sqrt{n}} \right\|_2^2 = \frac{1}{n} \left( \sum_{\substack{j=1 \\ |s_j - s_i| \le \beta}}^{i-1} \mathbb{E}[U_{s_j}^2] + \sum_{\substack{j, \ell=1 \\ \max\{|s_j - s_i|, |s_\ell - s_i|\} \le \beta \\ j \ne \ell}}^{i-1} \mathbb{E}[U_{s_j} U_{s_\ell}] \right).
$$

If $\|\Phi\|_{\infty} = \infty$ then using (H1) and (4.17), for $L' = (\beta^{-d}nh_{s_n}^{-\frac{N\theta}{2}})^{\frac{1}{2+\theta}}$, we have

$$
\begin{align*}
\frac{1}{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le \beta}}^{i-1} \mathbb{E}[U_{s_j}^2] &\le \frac{\beta^d L'^2}{n} + \frac{L'^{-\theta}}{n} \sum_{j=1}^{n} \mathbb{E}[|U_{s_j}|^{2+\theta}] \\
&\le \frac{\beta^d L'^2}{n} + L'^{-\theta} h_{s_n}^{-\frac{N\theta}{2}} \\
&= 2 \left( \frac{\beta^d}{nh_{s_n}^N} \right)^{\frac{\theta}{2+\theta}}.
\end{align*}
$$

If $\|\Phi\|_{\infty} < \infty$ then $|U_{s_i}| \le h_{s_n}^{-N/2}$ (since $|\Delta_{s_i}| \le h_{s_i}^{-N/2}$ and $(w_{s_n}h_{s_n}^{-N})_{n\ge 1}$ is nondecreasing) and consequently

$$
\frac{1}{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le \beta}}^{i-1} \mathbb{E}[U_{s_j}^2] \le \frac{\beta^d}{nh_{s_n}^N}.
$$

Finally, it means that

$$
\frac{1}{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le \beta}}^{i-1} \mathbb{E}[U_{s_j}^2] \le \left( \frac{\beta^d}{nh_{s_n}^N} \right)^{\nu_2(\theta)} \\
\le \left( nh_{s_n}^N \left( 1 + \frac{d\nu_2(\theta)(\nu_4(\theta)+\tau-1)}{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)} \right) \right)^{-\nu_2(\theta)} \xrightarrow{n\to\infty} 0. \quad (4.29)
$$

Moreover, we have

$$
\frac{1}{n} \sum_{\substack{j, \ell=1 \\ \max\{|s_j - s_i|, |s_\ell - s_i|\} \le \beta}}^{i-1} |\mathbb{E}[U_{s_j} U_{s_\ell}]| \le \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{\substack{j, \ell=1 \\ \max\{|s_j - s_i|, |s_\ell - s_i|\} \le \beta}}^{i-1} \frac{w_{s_j} w_{s_\ell} |\mathbb{E}[\Delta_{s_j} \Delta_{s_\ell}]|}{(h_{s_j} h_{s_\ell})^{N/2}}
$$
$$
\le \frac{A_{n,0,1}^{-2} h_{s_n}^{N}}{nw_{s_n}^2} \left( \sum_{\substack{j,\ell=1 \\ \max\{|s_j-s_i|,|s_\ell-s_i|\} \le \beta \\ j \ne \ell, |s_\ell-s_j| \le M_n}}^{i-1} \frac{w_{s_j} w_{s_\ell}}{(h_{s_j} h_{s_\ell})^{\frac{N(1-\nu_4(\theta))}{2}}} \right. \\
\left. \qquad + \sum_{\substack{j,\ell=1 \\ \max\{|s_j-s_i|,|s_\ell-s_i|\} \le \beta \\ j \ne \ell, |s_\ell-s_j| > M_n}}^{i-1} \frac{w_{s_j} w_{s_\ell} \alpha_{1,\infty}^{\nu_2(\theta)} (|s_j - s_\ell|)}{(h_{s_j} h_{s_\ell})^{\frac{N(1+\nu_2(\theta))}{2}}} \right) \\
\le \frac{A_{n,0,1}^{-2} h_{s_n}^{N}}{nw_{s_n}^2} \sum_{\substack{j,\ell=1 \\ \max\{|s_j-s_i|,|s_\ell-s_i|\} \le \beta \\ j \ne \ell, |s_\ell-s_j| \le M_n}}^{i-1} \left( \frac{w_{s_j}^2}{h_{s_j}^{N(1-\nu_4(\theta))}} + \frac{w_{s_\ell}^2}{h_{s_\ell}^{N(1-\nu_4(\theta))}} \right) \\
\qquad + \frac{A_{n,0,1}^{-2} h_{s_n}^{N}}{nw_{s_n}^2} \sum_{\substack{j,\ell=1 \\ \max\{|s_j-s_i|,|s_\ell-s_i|\} \le \beta \\ j \ne \ell, |s_\ell-s_j| > M_n}}^{i-1} \left( \frac{w_{s_j}^2}{h_{s_j}^{N(1+\nu_2(\theta))}} + \frac{w_{s_\ell}^2}{h_{s_\ell}^{N(1+\nu_2(\theta))}} \right) \alpha_{1,\infty}^{\nu_2(\theta)} (|s_j - s_\ell|).
$$

Since $\nu_4(\theta) \le 1$, we have

$$
\frac{1}{n} \sum_{\substack{j, \ell=1 \\ \max\{|s_j - s_i|, |s_\ell - s_i|\} \le \beta \\ j \ne \ell}}^{i-1} |\mathbb{E}[U_{s_j} U_{s_\ell}]|
$$
$$
\le \frac{M_n^d h_{s_n}^{N\nu_4(\theta)}}{nw_{s_n}^2 A_{n,0,1}^2} \sum_{i=1}^n w_{s_i}^2 + \left( \frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{j=1}^n \frac{w_{s_j}^2}{h_{s_j}^N} \right) \times h_{s_n}^{-N\nu_2(\theta)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \alpha_{1,\infty}^{\nu_2(\theta)} (|k|).
$$

Using (H1) and (H2), we derive

$$
\frac{1}{n} \sum_{\substack{j, \ell=1 \\ \max\{|s_j - s_i|, |s_\ell - s_i|\} \le \beta \\ j \ne \ell}}^{i-1} |\mathbb{E}[U_{s_j} U_{s_\ell}]| \le m_n^d h_{s_n}^{N(\nu_4(\theta) + \tau - 1)} + h_{s_n}^{-N\nu_2(\theta)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \alpha_{1,\infty}^{\nu_2(\theta)} (|k|).
$$

Consequently, denoting

$$
\epsilon_n^2 := \left( n h_{s_n}^{N} \left( 1 + \frac{d\nu_2(\theta)(\nu_4(\theta)+\tau-1)}{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)} \right) \right)^{-\nu_2(\theta)} + m_n^d h_{s_n}^{N(\nu_4(\theta)+\tau-1)} \\
+ h_{s_n}^{-N\nu_2(\theta)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \alpha_{1,\infty}^{\nu_2(\theta)} (|k|) \xrightarrow{n \to \infty} 0,
$$
we get

$$ \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[U_{s_i}^2] \left\| \sum_{\substack{j=1 \\ |s_j - s_i| \le \beta}}^{i-1} \frac{U_{s_j}}{\sqrt{n}} \right\|_2 \le \underbrace{\frac{A_{n,0,1}^{-2} h_{s_n}^N}{nw_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2 \mathbb{E}[\Delta_{s_i}^2]}{h_{s_i}^N}}_{\le 1} \times \varepsilon_n \le \varepsilon_n $$

and finally,

$$ I_1 \le \frac{\beta^d L h_{s_n}^{N(\nu_4(\theta)+\tau-1)}}{\sqrt{n}} + L^{-\theta} h_{s_n}^{\frac{-N\theta}{2}} + \beta \frac{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)}{\nu_4(\theta)+\tau-1} \alpha_{1,\infty}^{\nu_2(\theta)}(\beta) + \varepsilon_n. $$

Optimizing in L, we get

$$ I_1 \le \frac{\beta^{\frac{d\theta}{1+\theta}} h_{s_n}^{\frac{N\theta(\nu_4(\theta)+\tau-1)}{1+\theta}}}{(nh_{s_n}^N)^{\frac{\theta}{2(1+\theta)}}} + \beta \frac{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)}{\nu_4(\theta)+\tau-1} \alpha_{1,\infty}^{\nu_2(\theta)}(\beta) + \varepsilon_n. $$

Since, $\beta^d \le h_{s_n}^{\frac{-dN\nu_2(\theta)(\nu_4(\theta)+\tau-1)}{d(\nu_4(\theta)+\tau-1)}}$, we derive

$$ I_1 \le (nh_{s_n}^N)^{\frac{-\theta}{2(1+\theta)}} h_{s_n}^{\frac{N\theta(d-1)(\nu_4(\theta)+\tau-1)^2}{(1+\theta)(d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1))}} + \beta \frac{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)}{\nu_4(\theta)+\tau-1} \alpha_{1,\infty}^{\nu_2(\theta)}(\beta) + \varepsilon_n \xrightarrow{n \to +\infty} 0. $$

So, we obtain (4.23).

Now, we assume that $(X_k)_{k \in \mathbb{Z}^d}$ is of the form (1.5). As before, we have to prove (4.20), (4.21), (4.22) and (4.23). Now, we have $Z_{s_i} = \bar{U}_{s_i}$ for any $1 \le i \le n$. Moreover, $\bar{U}_{s_i}$ and $\bar{U}_{s_j}$ are independent as soon as $|s_i - s_j| > M_n$ where $M_n = 2m_n$. So, (4.20) holds since

$$ \mathbb{E}[T_{s_i} h_{i-1, i+1}^{(M_n)}] = n^{-1/2} \mathbb{E} \left[ \bar{U}_{s_i} h' \left( \sum_{\substack{l=1 \\ |s_l - s_i| > M_n}}^{i-1} \frac{\bar{U}_{s_l}}{\sqrt{n}} + \sum_{l=i+1}^{n} \frac{\xi_{s_l}}{\sqrt{n}} \right) \right] = 0. $$

Arguing as in (4.24) and (4.25), we have

$$ \sum_{i=1}^{n} \mathbb{E}[|\beta_i'|] \le \frac{2}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} \mathbb{E}[|\bar{U}_{s_i} \bar{U}_{s_j}|] $$

and

$$ \sum_{i=1}^{n} \left| \mathbb{E} \left[ T_{s_i} \left( \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} T_{s_l} \right) h_{i-1, i+1}^{\prime\prime(M_n)} \right] \right| \le \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{i-1} \mathbb{E}[|\bar{U}_{s_i} \bar{U}_{s_j}|]. $$

The proof of the following lemma is postponed to section 5.
**Lemma 10.** For any positive integer $n$,

$$ \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n \\ i \ne j}}^{n} \mathbb{E}[|\overline{U}_{s_i} \overline{U}_{s_j}|] \\ \le m_n^d h_{s_n}^{N(\nu_4(\theta) + \tau - 1)} + h_{s_n}^{-(\frac{N}{2} + \nu_2(\theta))} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\nu_2(\theta)} \xrightarrow{n \to \infty} 0. $$

Consequently, we obtain (4.21) and (4.22). In order to finish the proof, it suffices to prove (4.23). Let $L > 0$ be fixed. Keeping in mind that $\overline{U}_{s_i}$ and $\overline{U}_{s_j}$ are independent if $|s_i - s_j| > M_n$, we have

$$ \mathbb{E}[(Z_{s_i}^2 - \mathbb{E}[Z_{s_i}^2]) h_{i-1, i+1}^{(M_n)}] \\ = \mathbb{E}\left[\left(\overline{U}_{s_i}^2 - \mathbb{E}[\overline{U}_{s_i}^2]\right) h''\left(\sum_{\substack{\ell=1 \\ |s_\ell - s_i| > M_n}}^{i-1} \frac{\overline{U}_{s_\ell}}{\sqrt{n}} + \sum_{\ell=i+1}^{n} \frac{\xi_{s_\ell}}{\sqrt{n}}\right)\right] = 0. $$

So, we have

$$ \frac{1}{n} \sum_{i=1}^{n} |\mathbb{E}[(Z_{s_i}^2 - \mathbb{E}[Z_{s_i}^2]) h_{i-1, i+1}^{\prime\prime}| \\ = \frac{1}{n} \sum_{i=1}^{n} |\mathbb{E}\left[\left(\overline{U}_{s_i}^2 - \mathbb{E}[\overline{U}_{s_i}^2]\right) (h_{i-1, i+1}^{\prime\prime} - h_{i-1, i+1}^{\prime\prime(M_n)})\right]| \\ \le \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}\left[\left(2 \wedge \left|\sum_{\substack{\ell=1 \\ |s_\ell - s_i| \le M_n}}^{i-1} \frac{\overline{U}_{s_\ell}}{\sqrt{n}}\right|\right) (\overline{U}_{s_i}^2 + \mathbb{E}[\overline{U}_{s_i}^2])\right] \\ \le \frac{L}{n^{3/2}} \sum_{i=1}^{n} \sum_{\substack{\ell=1 \\ |s_\ell - s_i| \le M_n}}^{i-1} \mathbb{E}[|\overline{U}_{s_\ell} \overline{U}_{s_i}|] + \frac{2}{n} \sum_{i=1}^{n} \mathbb{E}[\overline{U}_{s_i}^2 \mathbf{1}_{|\overline{U}_{s_i}| > L}] \\ + \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}\left[2 \wedge \left|\sum_{\substack{\ell=1 \\ |s_\ell - s_i| \le M_n}}^{i-1} \frac{\overline{U}_{s_\ell}}{\sqrt{n}}\right|\right] \mathbb{E}[\overline{U}_{s_i}^2] \\ \le \frac{L}{\sqrt{n}} \times \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{\ell=1 \\ |s_\ell - s_i| \le M_n}}^{n} \mathbb{E}[|\overline{U}_{s_\ell} \overline{U}_{s_i}|] + \frac{2L^{-\theta}}{n} \sum_{i=1}^{n} \mathbb{E}[|U_{s_i}|^{2+\theta}] \\ + \frac{1}{n} \sum_{i=1}^{n} \left\| \sum_{\substack{\ell=1 \\ |s_\ell - s_i| \le M_n}}^{i-1} \frac{\overline{U}_{s_\ell}}{\sqrt{n}} \right\|_2 \mathbb{E}[U_{s_i}^2]. $$
Moreover,

$$ \left\| \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} \frac{\bar{U}_{s_l}}{\sqrt{n}} \right\|_2^2 = \frac{1}{n} \left( \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} \mathbb{E}[\bar{U}_{s_l}^2] + \sum_{\substack{l,j=1 \\ \max(|s_l - s_i|, |s_j - s_i|) \le M_n \\ l \ne j}}^{i-1} \mathbb{E}[\bar{U}_{s_l} \bar{U}_{s_j}] \right). $$

Noting that $\mathbb{E}[\bar{U}_{s_l}^2] \le \mathbb{E}[U_{s_l}^2]$ and using (4.29), we get

$$ \frac{1}{n} \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} \mathbb{E}[\bar{U}_{s_l}^2] \le \left( n h_{s_n}^N \left( 1 + \frac{d\nu_2(\theta)(\nu_4(\theta)+\tau-1)}{d\nu_2(\theta)+(d-1)(\nu_4(\theta)+\tau-1)} \right) \right)^{-\nu_2(\theta)} \xrightarrow{n \to \infty} 0. $$

Using Lemma 10 and keeping in mind that $\bar{U}_{s_l}$ and $\bar{U}_{s_j}$ are independent if $|s_l - s_j| > M_n$, we have

$$ \frac{1}{n} \sum_{\substack{l,j=1 \\ \max(|s_l - s_i|, |s_j - s_i|) \le M_n \\ l \ne j}}^{i-1} |\mathbb{E}[\bar{U}_{s_l} \bar{U}_{s_j}]| \le \frac{1}{n} \sum_{l=1}^{n} \sum_{\substack{j=1 \\ |s_l - s_j| \le M_n \\ l \ne j}}^{n} |\mathbb{E}[\bar{U}_{s_l} \bar{U}_{s_j}]| \xrightarrow{n \to \infty} 0. $$

Consequently,

$$ \lim_{n \to +\infty} \sup_{1 \le i \le n} \left\| \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} \frac{\bar{U}_{s_l}}{\sqrt{n}} \right\|_2 = 0 $$

and

$$ \lim_{n \to +\infty} \frac{1}{n} \sum_{i=1}^{n} \left\| \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} \frac{\bar{U}_{s_l}}{\sqrt{n}} \right\|_2 \mathbb{E}[U_{s_i}^2] \\ \le \lim_{n \to +\infty} \underbrace{\frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2 \mathbb{E}[\Delta_{s_i}^2]}{h_{s_i}^N}}_{\le 1} \times \sup_{1 \le i \le n} \left\| \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{i-1} \frac{\bar{U}_{s_l}}{\sqrt{n}} \right\|_2 = 0. $$

Using (4.17) and (H1), we obtain

$$ \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[(Z_{s_i}^2 - \mathbb{E}[Z_{s_i}^2]) h_{i-1,i+1}''] \le \frac{L}{\sqrt{n}} \times \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{n} \mathbb{E}[|\bar{U}_{s_l} \bar{U}_{s_i}|] + L^{-\theta} h_{s_n}^{\frac{-N\theta}{2(1+\theta)}} + o(1). $$

Optimizing in $L$, we get

$$ \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[(Z_{s_i}^2 - \mathbb{E}[Z_{s_i}^2]) h_{i-1,i+1}''] \le (nh_{s_n}^N)^{\frac{-\theta}{2(1+\theta)}} \left( \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{l=1 \\ |s_l - s_i| \le M_n}}^{n} \mathbb{E}[|\bar{U}_{s_l} \bar{U}_{s_i}|] \right)^{\frac{\theta}{1+\theta}} + o(1). $$
Using Lemma 10, we obtain (4.23). The proof of Theorem 1 is complete. □

In the proof of Theorem 1, the asymptotic normality of the estimator $f_{n,\Phi}$ is obtained using the Lindeberg's method based on the stability of the standard normal law. This approach seems to be superior to the so-called Bernstein's method (see for example [5] and [21]) since it allows us to obtain mild conditions on the weak and strong dependent coefficients of the considered random field. This fact is of theoretical importance and has already been observed in [1], [14] and [15].

*Proof of Theorem 2.* Let $n$ be a positive integer and $x \in \mathbb{R}^N$ such that $f(x) > 0$. Then,

$$r_{n,\Phi}(x) - \frac{\mathbb{E}[f_{n,\Phi}(x)]}{\mathbb{E}[f_{n,\mathbf{1}}(x)]} = \frac{(f_{n,\Phi}(x) - \mathbb{E}[f_{n,\Phi}(x)])\mathbb{E}[f_{n,\mathbf{1}}(x)] - (f_{n,\mathbf{1}}(x) - \mathbb{E}[f_{n,\mathbf{1}}(x)])\mathbb{E}[f_{n,\Phi}(x)]}{f_{n,\mathbf{1}}(x)\mathbb{E}[f_{n,\mathbf{1}}(x)]}.$$

Using Proposition 1 and Proposition 2, we obtain that $f_{n,\mathbf{1}}(x)$ converges in probability to $f(x)$ and $\frac{\mathbb{E}[f_{n,\Phi}(x)]}{\mathbb{E}[f_{n,\mathbf{1}}(x)]}$ converges to $r_{\Phi}(x)$ as $n \to \infty$. So, using Slutsky's lemma, it is sufficient to prove

$$\lambda_1 \sqrt{nh_{s_n}^N} (f_{n,\Phi}(x) - \mathbb{E}[f_{n,\Phi}(x)]) + \lambda_2 \sqrt{nh_{s_n}^N} (f_{n,\mathbf{1}}(x) - \mathbb{E}[f_{n,\mathbf{1}}(x)]) \xrightarrow[n \to \infty]{\text{Law}} \mathcal{N}(0, \rho_{\lambda_1, \lambda_2}^2(x))$$

where

$$\rho_{\lambda_1, \lambda_2}^2(x) = (\lambda_1^2 \mathbb{E}[|\Phi(Y_0)|^2 | X_0 = x] + 2\lambda_1\lambda_2 r_{\Phi}(x) + \lambda_2^2) \beta_{0,1}^{-2} \beta_{-N,2} f(x) \int_{\mathbb{R}^N} K^2(t) dt$$

for any $(\lambda_1, \lambda_2) \in \mathbb{R}^2$.

Let $(\lambda_1, \lambda_2) \in \mathbb{R}^2$ be fixed. Then,

$$\lambda_1 \sqrt{nh_{s_n}^N} (f_{n,\Phi}(x) - \mathbb{E}[f_{n,\Phi}(x)]) + \lambda_2 \sqrt{nh_{s_n}^N} (f_{n,\mathbf{1}}(x) - \mathbb{E}[f_{n,\mathbf{1}}(x)]) = \sqrt{nh_{s_n}^N} (f_{n,\bar{\Phi}}(x) - \mathbb{E}[f_{n,\bar{\Phi}}(x)])$$

where $\bar{\Phi}(x) = \lambda_1\Phi(x) + \lambda_2$ for any $x$ in $\mathbb{R}$. Since $u \mapsto \mathbb{E}[|\Phi(Y_0)|^2 | X_0 = u]$ is continuous, one can notice that $u \mapsto \mathbb{E}[|\bar{\Phi}(Y_0)|^2 | X_0 = u]$ is continuous. Moreover, since $\mathbb{E}[|\Phi(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\Phi(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$, we have $\mathbb{E}[|\bar{\Phi}(Y_0)|^{2+\theta}] < \infty$ and $\mathbb{E}[|\bar{\Phi}(Y_0)|^{2+\theta} K_{s_n}(x, X_0)] \le h_{s_n}^N$. Consequently, using Theorem 1, we get the result. The proof of Theorem 2 is complete. □

One can notice that the asymptotic normality of the regression estimator $r_{n,\Phi}$ obtained in the proof of Theorem 2 is a direct consequence of Theorem 1.
In some sense, it means that Theorem 1 is quite a deep result since it contains both the asymptotic normality of the kernel density estimator $f_{n,1}$ and that of the regression estimator $r_{n,1}$.

**Proof of Theorem 3.** Let $x \in \mathbb{R}^N$ such that $f(x) > 0$. Then, according to Theorem 2, we have

$$ \sqrt{nh_{s_n}^N} \left( r_{n,\Phi}(x) - \frac{\mathbb{E}[f_{n,\Phi}(x)]}{\mathbb{E}[f_{n,1}(x)]} \right) \xrightarrow[n \to \infty]{\text{Law}} \mathcal{N}(0, \tilde{\sigma}_{\Phi}^2(x)), $$

where $\tilde{\sigma}_{\Phi}^2(x) = \frac{V(x)\beta_{-N,2}}{f(x)\beta_{0,1}^2} \int_{\mathbb{R}^N} K^2(t)dt$ and $V(x) = \mathbb{E}[|\Phi(Y_0)|^2|X_0 = x] - r_{\Phi}^2(x)$. Applying Proposition 2, we have

$$ |\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x)| \le h_{s_n}^2 \quad \text{and} \quad |\mathbb{E}[f_{n,1}(x)] - f(x)| \le h_{s_n}^2. $$

Recall that $r_{\Phi}(x) = \frac{f_{\Phi}(x)}{f(x)}$. Then, for $n$ sufficiently large, we have

$$ \left| \frac{\mathbb{E}[f_{n,\Phi}(x)]}{\mathbb{E}[f_{n,1}(x)]} - r_{\Phi}(x) \right| = \frac{|(\mathbb{E}[f_{n,\Phi}(x)] - f_{\Phi}(x))f(x) - (\mathbb{E}[f_{n,1}(x)] - f(x))f_{\Phi}(x)|}{f(x)\mathbb{E}[f_{n,1}(x)]} \le h_{s_n}^2. $$

Finally, using Slutsky's lemma, we obtain

$$ \sqrt{nh_{s_n}^N} (r_{n,\Phi}(x) - r_{\Phi}(x)) \xrightarrow[n \to \infty]{\text{Law}} \mathcal{N}(0, \tilde{\sigma}_{\Phi}^2(x)). $$

The proof of Theorem 3 is complete. □

## 5. Appendix

**Proof of Lemma 7.** First, we note that $\mathbb{E}[W_i|\mathcal{F}_{\ell-1}]$ is $\mathcal{F}_{\ell}$-measurable. It suffices to show that for every $A \in \mathcal{F}_{\ell}$, we have $\mathbb{E}[\mathbb{E}[W_i|\mathcal{F}_{\ell-1}] \mathbf{1}_A] = \mathbb{E}[H_i(\mathcal{H}_{i,\infty}^{(\ell)}) \mathbf{1}_A]$. One can notice that the collections $\mathcal{P} = \{A \cap B | A \in \mathcal{F}_{\ell-1}, B \in \sigma(\epsilon_{\tau(\ell)})\}$ and $\Lambda = \{A \in \mathcal{F}_{\ell} | \mathbb{E}[\mathbb{E}[W_i|\mathcal{F}_{\ell-1}] \mathbf{1}_A] = \mathbb{E}[H_i(\mathcal{H}_{i,\infty}^{(\ell)}) \mathbf{1}_A]\}$ are respectively a $\pi$-system and a $\lambda$-system which satisfy $\sigma(\mathcal{P}) = \mathcal{F}_{\ell}$. Since $(\epsilon_{\tau(j)})_{j \in \mathbb{Z}}$ and $(\epsilon_{\tau(j)}^{(\ell)})_{j \in \mathbb{Z}}$ are identically distributed where $\epsilon_{\tau(j)}^{(\ell)} = \epsilon_{\tau(j)}$ if $j \ne \ell$ and $\epsilon_{\tau(\ell)}^{(\ell)} = \epsilon_{\tau(\ell)}^{\prime}$, it holds for every $C \in \mathcal{F}_{\ell-1}$ that $\mathbb{E}[\mathbf{1}_C W_i] = \mathbb{E}[\mathbf{1}_C H_i(\mathcal{H}_{i,\infty}^{(\ell)})]$. Then, if $A = A_1 \cap A_2 \in \mathcal{P}$ with $A_1 \in \mathcal{F}_{\ell-1}$ and $A_2 \in \sigma(\epsilon_{\tau(\ell)})$, we obtain

$$ \begin{aligned} \mathbb{E}[\mathbb{E}[W_i|\mathcal{F}_{\ell-1}] \mathbf{1}_A] &= \mathbb{E}[\mathbb{E}[\mathbf{1}_{A_1} W_i|\mathcal{F}_{\ell-1}] \mathbf{1}_{A_2}] \\ &= \mathbb{E}[\mathbf{1}_{A_1} H_{i,\infty}(\mathcal{H}_{i,\infty}^{(\ell)}) \mathbb{E}[\mathbf{1}_{A_2}] \\ &= \mathbb{E}[\mathbf{1}_{A_1} H_{i,\infty}(\mathcal{H}_{i,\infty}^{(\ell)}) \mathbf{1}_{A_2}] \\ &= \mathbb{E}[H_{i,\infty}(\mathcal{H}_{i,\infty}^{(\ell)}) \mathbf{1}_A]. \end{aligned} $$
So, we obtain $A \in \Lambda$ and finally $\mathcal{P} \subset \Lambda$. Applying Dynkin's lemma, we get the desired result. The proof of Lemma 7 is complete. $\square$

**Proof of Lemma 8.** Let $1 \le i, j \le n$ such that $i \ne j$. From Lemma 4, we have $\mathbb{E}[|\Delta_{s_i} \Delta_{s_j}|] \le (h_{s_i} h_{s_j})^{\frac{N\nu_4(\theta)}{2}}$. Keeping in mind (4.13), we have

$$ \mathbb{E}[|U_{s_i} U_{s_j}|] = \frac{h_{s_n}^N w_{s_i} w_{s_j} \mathbb{E}[|\Delta_{s_i} \Delta_{s_j}|]}{(h_{s_i} h_{s_j})^{\frac{N}{2}} w_{s_n}^2 A_{n,0,1}^2} \le \frac{A_{n,0,1}^{-2} h_{s_n}^N w_{s_i} w_{s_j}}{w_{s_n}^2 (h_{s_i} h_{s_j})^{N(1-\nu_4(\theta))/2}} $$

Since $\nu_4(\theta) \le 1$ and using (H1), we derive $\mathbb{E}[|U_{s_i} U_{s_j}|] \le w_{s_i} w_{s_j} w_{s_n}^{-2} h_{s_n}^{N\nu_4(\theta)}$. The proof of Lemma 8 is complete. $\square$

**Proof of Lemma 10.** For any $1 \le i, j \le n$ such that $i \ne j$, we have

$$ |\mathbb{E}[|\bar{U}_{s_i} \bar{U}_{s_j}|] - \mathbb{E}[|U_{s_i} U_{s_j}|]| \le \|U_{s_j}\|_2 \|U_{s_i} - \bar{U}_{s_i}\|_2 + \|U_{s_i}\|_2 \|U_{s_j} - \bar{U}_{s_j}\|_2 $$

and for any $1 \le \ell \le n$, using (3.12), we get

$$ \|U_{s_{\ell}}\|_2 = \frac{h_{s_n}^{N/2} w_{s_{\ell}} \|\Delta_{s_{\ell}}\|_2}{h_{s_{\ell}}^{N/2} w_{s_n} A_{n,0,1}} \le \frac{h_{s_n}^{N/2} w_{s_{\ell}}}{h_{s_{\ell}}^{N/2} w_{s_n} A_{n,0,1}} $$

and

$$ \|U_{s_{\ell}} - \bar{U}_{s_{\ell}}\|_2 = \frac{h_{s_n}^{N/2} w_{s_{\ell}} \|\Delta_{s_{\ell}} - \bar{\Delta}_{s_{\ell}}\|_2}{h_{s_{\ell}}^{N/2} w_{s_n} A_{n,0,1}} \le \frac{h_{s_n}^{N/2} w_{s_{\ell}} h_{s_n}^{-(\frac{N}{2} + \nu_2(\theta))}}{h_{s_{\ell}}^{N/2} w_{s_n} A_{n,0,1}} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \delta_{k,2}^{\nu_2(\theta)} $$

Consequently, we get

$$ \begin{aligned} & \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n \\ i \ne j}}^{n} |\mathbb{E}[|\bar{U}_{s_i} \bar{U}_{s_j}|] - \mathbb{E}[|U_{s_i} U_{s_j}|]| \\ & \le \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n \\ i \ne j}}^{n} \frac{w_{s_i} w_{s_j}}{h_{s_i}^{N/2} h_{s_j}^{N/2}} \times h_{s_n}^{-(\frac{N}{2} + \nu_2(\theta))} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \delta_{k,2}^{\nu_2(\theta)} \\ & \le \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n}}^{n} \left( \frac{w_{s_i}^2}{h_{s_i}^N} + \frac{w_{s_j}^2}{h_{s_j}^N} \right) \times h_{s_n}^{-(\frac{N}{2} + \nu_2(\theta))} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \delta_{k,2}^{\nu_2(\theta)} \\ & \le 2M_n^d \left( \frac{h_{s_n}^N A_{n,0,1}^{-2}}{nw_{s_n}^2} \sum_{i=1}^{n} \frac{w_{s_i}^2}{h_{s_i}^N} \right) h_{s_n}^{-(\frac{N}{2} + \nu_2(\theta))} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > M_n}} \delta_{k,2}^{\nu_2(\theta)} \end{aligned} $$
Using (H1), we obtain

$$
\begin{align*}
\frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_j - s_i| \le M_n \\ i \ne j}}^{i-1} |\mathbb{E}[|\bar{U}_{s_i} \bar{U}_{s_j}|] - \mathbb{E}[|U_{s_i} U_{s_j}|] | &\le m_n^d h_{s_n}^{-\left(\frac{N}{2} + \nu_2(\theta)\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} \delta_{k,2}^{\nu_2(\theta)} \\
&\le h_{s_n}^{-\left(\frac{N}{2} + \nu_2(\theta)\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\nu_2(\theta)}.
\end{align*}
$$

Using Lemma 8, we obtain

$$
\begin{align*}
& \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_i - s_j| \le M_n \\ i \ne j}}^{n} \mathbb{E}[|\bar{U}_{s_i} \bar{U}_{s_j}|] \\
& \le \frac{h_{s_n}^{N\nu_4(\theta)}}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_i - s_j| \le M_n \\ i \ne j}}^{n} \frac{w_{s_i} w_{s_j}}{w_{s_n}^2} + h_{s_n}^{-\left(\frac{N}{2} + \nu_2(\theta)\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\nu_2(\theta)} \\
& \le \frac{h_{s_n}^{N\nu_4(\theta)}}{nw_{s_n}^2} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_i - s_j| \le M_n}}^{n} (w_{s_i}^2 + w_{s_j}^2) + h_{s_n}^{-\left(\frac{N}{2} + \nu_2(\theta)\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\nu_2(\theta)} \\
& \le \frac{m_n^d h_{s_n}^{N\nu_4(\theta)}}{nw_{s_n}^2} \sum_{i=1}^{n} w_{s_i}^2 + h_{s_n}^{-\left(\frac{N}{2} + \nu_2(\theta)\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\nu_2(\theta)}.
\end{align*}
$$

Using (H2), we derive

$$
\begin{align*}
& \frac{1}{n} \sum_{i=1}^{n} \sum_{\substack{j=1 \\ |s_i - s_j| \le M_n \\ i \ne j}}^{n} \mathbb{E}[|\bar{U}_{s_i} \bar{U}_{s_j}|] \\
& \le m_n^d h_{s_n}^{N(\nu_4(\theta) + \tau - 1)} + h_{s_n}^{-\left(\frac{N}{2} + \nu_2(\theta)\right)} \sum_{\substack{k \in \mathbb{Z}^d \\ |k| > m_n}} |k|^d \delta_{k,2}^{\nu_2(\theta)} \xrightarrow{n \to \infty} 0.
\end{align*}
$$

The proof of Lemma 10 is complete.

<figure><img src="image_8.png" /></figure>

Acknowledgments

The authors are grateful to the referees and the associate editor for many helpful comments.
References

[1] ABOUBACAR, A. and EL MACHKOURI, M. (2020). Recursive kernel density estimation for time series. *IEEE Trans. Inform. Theory* **66** 6378–6388.
MR4173545

[2] AHMAD, I. A. and LIN, P.-E. (1976). Nonparametric sequential estimation of a multiple regression function. *Bull. Math. Statist.* **17** 63–75. MR0436439

[3] AMIRI, A. (2010). *Estimateurs fonctionnels récursifs et leurs applications à la prévision*. Thèse de Doctorat.

[4] AMIRI, A. (2012). Recursive regression estimators with application to non-parametric prediction. *J. Nonparametr. Stat.* **24** 169–186. MR2885832

[5] BIAU, G. and CADRE, B. (2004). Nonparametric spatial prediction. *Stat. Inference Stoch. Process.* **7** 327–349. MR2111294

[6] BOSQ, D., F., M. and PELIGRAD, M. (1999). Asymptotic normality for density kernel estimators in discrete and continuous time. *J. Multivariate Anal.* **68** 78–95. MR1668915

[7] CHEN, X., HUANG, Y. and WU, W. B. (2013). Recursive Nonparametric Estimation For Time Series. *IEEE Trans. Inform. Theory* **60** 1301–1312.
MR3164976

[8] COMTE, F. and MARIE, N. (2020). Bandwidth selection for the Wolverton–Wagner estimator. *Journal of Statistical Planning and Inference* **207** 198–214. MR4066131

[9] DEDECKER, J. (1998). A central limit theorem for stationary random fields. *Probab. Theory Relat. Fields* **110** 397–426. MR1616496

[10] DEDECKER, J. (2001). Exponential inequalities and functional central limit theorems for random fields. *ESAIM: Probability and Statistics* **5** 77–104.
MR1875665

[11] DEDECKER, J. and MERLEVÈDE, F. (2002). Necessary and sufficient conditions for the conditional central limit theorem. *Annals of Probability* **30** 1044–1081. MR1920101

[12] DEHEUVELS, P. (1973). Sur l'estimation séquentielle de la densité. *C. R. Acad. Sci. Paris Sér. A-B* **276** A1119–A1121. MR0334393

[13] DEVROYE, L. P. and WAGNER, T. J. (1980). On the $L_1$ convergence of kernel estimators of regression functions with applications in discrimination. *Z. Wahrsch. Verw. Gebiete* **51** 15–25. MR0566104

[14] EL MACHKOURI, M., ES-SEBAIY, K. and OUASSOU, I. (2017). On local linear regression for strongly mixing random fields. *J. Multivariate Anal.* **156** 103–115. MR3624688

[15] EL MACHKOURI, M., FAN, X. and REDING, L. (2020). On the Nadaraya-Watson kernel regression estimator for irregularly spaced spatial data. *J. Statist. Plann. Inference* **205** 92–114. MR4011625

[16] EL MACHKOURI, M., VOLNÝ, D. and WU, W. B. (2013). A central limit theorem for stationary random fields. *Stochastic Process. Appl.* **123** 1–14.
MR2988107

[17] GOLDENSHLUGER, A. and LEPSKI, O. (2011). Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality.
The Annals of Statistics 39 1608–1632. MR2850214

[18] GYORFI, L. and MASRY, E. (1987). Strong consistency and rates for recursive probability density estimators of stationary processes. *J. Multivariate Anal.* **22** 79–93. MR0890884

[19] GYORFI, L. and MASRY, E. (1990). The $L_1$ and $L_2$ strong consistency of recursive kernel density estimation from dependent samples. *IEEE Trans. Inform. Theory.* **36** 531–539. MR1053846

[20] HALL, P. and PATIL, P. (1994). On the efficiency of on-line density estimators. *IEEE Trans. Information Theory* **40** 1504–1512.

[21] HALLIN, M., LU, Z. and TRAN, L. T. (2004). Local linear spatial regression. *Ann. Statist.* **32** 2469–2500. MR2153992

[22] LACOUR, C., MASSART, P. and RIVOIRARD, V. (2017). Estimator selection: a new method with applications to kernel density estimation. *Sankhya A* **79** 298–335. MR3707423

[23] LINDEBERG, J. W. (1922). Eine neue Herleitung des Exponentialgezetzes in der Wahrscheinlichkeitsrechnung. *Mathematische Zeitschrift* **15** 211–225. MR1544569

[24] LU, Z. and CHENG, P. (1997). Distribution-free strong consistency for nonparametric kernel regression involving nonlinear time series. *J. Statist. Plann. Inference* **65** 67–86. MR1619667

[25] MASRY, E. (1986). Recursive probability density estimation for weakly dependent stationary processes. *IEEE Trans. Inform. Theory* **32** 254–267. MR0838413

[26] MASRY, E. and FAN, J. (1997). Local polynomial estimation of regression functions for mixing processes. *Scand. J. Statist.* **24** 165–179. MR1455865

[27] McLEISH, D. L. (1975). A maximal inequality and dependent strong laws. *Ann. Probab.* **3** 829–839. MR0400382

[28] MEZHOUD, K., MOHDEB, Z. and LOUHICHI, S. (2014). Recursive kernel estimation of the density under $\eta$-weak dependence. *J. Korean Statist. Soc.* **43** 403–414. MR3233079

[29] MOKKADEM, A., PELLETIER, M. and SLAOUI, Y. (2009). The stochastic approximation method for the estimation of a multivariate probability density. *J. Statist. Plann. Inference* **139** 2459–2478. MR2508006

[30] NADARAYA, E. A. (1964). On a regression estimate. *Teor. Verojatnost. i Primenen.* **9** 157–159. MR0166874

[31] PARZEN, E. (1962). On the estimation of a probability density and the mode. *Ann. Math. Statist.* **33** 1965–1976. MR0143282

[32] RIO, E. (1993). Covariance inequalities for strongly mixing processes. *Ann. Inst. H. Poincaré Probab. Statist.* **29** 587–597. MR1251142

[33] ROBINSON, P. M. (1983). Nonparametric estimators for time series. *J. Time Ser. Anal.* **4** 185–207. MR0732897

[34] ROSENBLATT, M. (1956). Remarks on some nonparametric estimates of a density function. *Ann. Math. Statist.* **27** 832–837. MR0079873

[35] ROSENBLATT, M. (1956). A central limit theorem and a strong mixing condition. *Proc. Nat. Acad. Sci. USA* **42** 43–47. MR0074711

[36] ROUSSAS, G. and TRAN, L. T. (1992). Asymptotic normality of the re-
cursive kernel regression estimate under dependence conditions. Ann. Stat 20 98-120. MR1150336

[37] ROUSSAS, G. G. (1988). Nonparametric estimation in mixing sequences of random variables. *J. Statist. Plann. Inference* **18** 135–149. MR0922204

[38] TSYBAKOV, A. B. (2008). *Introduction to nonparametric estimation*. Springer Science & Business Media. MR2724359

[39] WALK, H. and YAKOWITZ, S. (2002). Iterative nonparametric estimation of a log-optimal portfolio selection function. *IEEE Trans. Inform. Theory* **48** 324–333. MR1872189

[40] WANG, L. and LIANG, H. (2004). Strong uniform convergence of the recursive regression estimator under $\phi$-mixing conditions. *Metrika* **59** 245?261. MR2066728

[41] WATSON, G. S. (1964). Smooth regression analysis. *Sankhyā Ser. A* **26** 359–372. MR0185765

[42] WEGMAN, E. J. and DAVIES, H. I. (1979). Remarks on some recursive estimators of a probability density. *Ann. Statist.* **7** 316–327. MR0520242

[43] WOLVERTON, C. T. and WAGNER, T. J. (1969). Recursive estimates of probability densities. *IEEE Trans. Syst. Man. Cybern.* **5** 307–308.

[44] WU, W. B. and MIELNICZUK, J. (2002). Kernel density estimation for linear processes. *Ann. Statist.* **30** 1441–1459. MR1936325

[45] WU, W. B. (2005). Nonlinear system theory: another look at dependence. *Proc. Natl. Acad. Sci. USA* **102** 14150–14154 (electronic). MR2172215

[46] WU, W. B. (2011). Asymptotic theory for stationary processes. *Statistics and Its Interface* **0** 1–20. MR2812816

[47] YAMATO, H. (1971). Sequential estimation of a continuous probability density function and mode. *Bull. Math. Statist* **14** 1–12. MR0381187