<figure><img src="image_1.png" /></figure>

HAL
open science

# Weak convergence of the conditional single index U-statistics for locally stationary functional time series

Salim Bouzebda

► To cite this version:

Salim Bouzebda. Weak convergence of the conditional single index $U$-statistics for locally stationary functional time series. AIMS Mathematics, 2024, 9 (6), pp.14807-14898. 10.3934/math.2024720. hal-04558676

HAL Id: hal-04558676

https://hal.science/hal-04558676v1

Submitted on 21 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

Research article

Uniform in number of neighbors consistency and weak convergence of $k$NN empirical conditional processes and $k$NN conditional $U$-processes involving functional mixing data*

Salim Bouzebda* and Amel Nezzal

LMAC (Laboratory of Applied Mathematics of Compiègne), Université de technologie de Compiègne, France

* Correspondence: Email: salim.bouzebda@utc.fr.

**Abstract:** $U$-statistics represent a fundamental class of statistics arising from modeling quantities of interest defined by multi-subject responses. $U$-statistics generalize the empirical mean of a random variable $X$ to sums over every $m$-tuple of distinct observations of $X$. Stute [182] introduced a class of so-called conditional $U$-statistics, which may be viewed as a generalization of the Nadaraya-Watson estimates of a regression function. Stute proved their strong pointwise consistency to: $r^{(m)}(\varphi, t) := \mathbb{E}[\varphi(Y_1, \dots, Y_m)|(X_1, \dots, X_m) = t]$, for $t \in \lambda^m$. In this paper, we are mainly interested in the study of the $k$NN conditional $U$-processes in a functional mixing data framework. More precisely, we investigate the weak convergence of the conditional empirical process indexed by a suitable class of functions and of the $k$NN conditional $U$-processes when the explicative variable is functional. We treat the uniform central limit theorem in both cases when the class of functions is bounded or unbounded satisfying some moment conditions. The second main contribution of this study is the establishment of a sharp almost complete Uniform consistency in the Number of Neighbors of the constructed estimator. Such a result allows the number of neighbors to vary within a complete range for which the estimator is consistent. Consequently, it represents an interesting guideline in practice to select the optimal bandwidth in nonparametric functional data analysis. These results are proved under some standard structural conditions on the Vapnik-Chervonenkis classes of functions and some mild conditions on the model. The theoretical results established in this paper are (or will be) key tools for further functional data analysis developments. Potential applications include the set indexed conditional $U$-statistics, Kendall rank correlation coefficient, the discrimination problems and the time series prediction from a continuous set of past values.

**Keywords:** conditional $U$-statistic; functional data analysis; functional regression; Kolmogorov's entropy; small ball probability; $k$NN Kernel-type estimators; empirical processes; $U$-processes; weak convergence; VC-classes

<footnote>*Both authors contributed equally to this work.</footnote>
# 1. Introduction and motivations

U-statistics were first introduced by [115] in connection with unbiased estimators, following initial work by [106]. In brief, U-statistics of order m and kernel h based on a sequence {$X_i$}$_{i=1}^\infty$ of random variables with values in a measurable space $(S, \mathcal{S})$ and a measurable function $f: S^m \to \mathbb{R}$ are given by

$$U_n(h) = \frac{(n-m)!}{n!} \sum_{(i_1, \dots, i_m) \in I_n^m} h(X_{i_1}, \dots, X_{i_m}), \quad n \ge m,$$

where

$$I_n^m = \{(i_1, \dots, i_m) : i_j \in \mathbb{N}, 1 \le i_j \le n, i_j \ne i_k \text{ if } j \ne k\}.$$

$U_n(h)$ is the nonparametric uniformly minimum variance estimator of $\theta = \mathbb{E}(h(X_1, \dots, X_m))$. It is the minimizer with respect to $\alpha$ of

$$\sum_{1 \le i_1 < \dots < i_m \le n} (h(X_{i_1}, \dots, X_{i_m}) - \alpha)^2.$$

Empirical variance, Gini's mean difference, and Kendall's rank correlation coefficient are common examples of estimators based on U-statistics. The Wilcoxon signed rank test for the hypothesis of the location at zero is a classical test based on U-statistics, as discussed by [190] in Example 12.4. Asymptotic results for the case of independent and identically distributed underlying random variables were first provided by [115], who also referred to related work by [57, 83, 172, 176]. Similar results were obtained for V-statistics by [94, 198]. Extensive literature on the theory of U-statistics has been developed, as reviewed by [10, 138, 176], and others. A detailed review and major historical developments in this field can be found in the book by [20]. U-processes are sets of U-statistics indexed by a family of kernels, which can be viewed as infinite-dimensional variants of U-statistics with a single kernel or as stochastic processes that are nonlinear extensions of empirical processes. U-processes have been applied to solve complex statistical problems such as density estimation, nonparametric regression tests, and goodness-of-fit tests. Considering a large group of statistics instead of a single statistic is more statistically interesting, and ideas from the theory of empirical processes can be used to construct limit or approximation theorems for U-processes. However, obtaining results for U-processes is not easy and requires significant effort and distinct methodologies. Generalizing from empirical processes to U-processes is particularly difficult, especially in the stationary setting. U-processes appear in statistics in many instances, such as the components of higher-order terms in von Mises expansions, and play a role in analyzing estimators (including function estimators) with varying degrees of smoothness. For instance, the product limit estimator for truncated data is analyzed in [183] using a.s. uniform bounds for P-canonical U-processes. In addition, [11] introduce two new tests for normality based on U-processes, while [173] use weighted L₁-distances between the standard normal density and local U-statistics based on standardized observations to propose new tests for normality, utilizing the results of [101]. Moreover, in [122], the median-of-means approach, which is based on
U-statistics, is introduced to estimate the mean of multivariate functions in case of possibly heavy-tailed distributions. U-processes play a significant role in various statistical applications, including testing for qualitative features of functions in nonparametric statistics [1, 100], cross-validation for density estimation [157], and establishing limiting distributions of M-estimators [10, 68, 177]. [10] provide necessary and sufficient conditions for the law of large numbers and sufficient conditions for the central limit theorem for U-processes. For further references on U-statistics and U-processes, interested readers may refer to [28, 40, 46, 47, 48, 54, 138, 179], while a comprehensive insight into the U-processes theory is provided by [68]. U-statistics are also naturally found in other contexts, such as the theory of random graphs, where they count occurrences of specific subgraphs like triangles, as presented in [120]. In machine learning, U-statistics arise naturally in various problems such as clustering, image recognition, ranking, and learning on graphs, where natural risk estimates take the form of U-statistics, as discussed in [63]. For instance, the empirical ranking error of any given prediction rule is a U-statistic of order 2, as stated in [62]. For U-statistics with random kernels of diverging orders, readers may refer to [97, 112, 178, 180]. Infinite-order U-statistics are also useful for constructing simultaneous prediction intervals that quantify the uncertainty of ensemble methods like subbagging and random forests, as presented in [161]. The MeanNN approach estimation for differential entropy, introduced by [90], is a particular application of the U-statistic. Additionally, [143] proposed a new test statistic for goodness-of-fit tests using U-statistics. Moreover, [65] have explored a model-free approach for clustering and classifying genetic data based on U-statistics, leading to alternative ways of addressing genetic problems. Their motivation was based on the versatility and adaptability of U-statistics to various genetic problems and different data types. [140] proposed using the U-statistics, in a natural way, for analyzing random compressed sensing matrices in the non-asymptotic regime. Extending the above exploration to conditional U-processes is practically useful and technically more challenging.

We first introduce Stute's estimators. Let us consider regular sequence of random elements $\{(\mathbf{X}_i, Y_i), i \in \mathbb{N}^*\}$ with $\mathbf{X}_i \in \mathbb{R}^d$ and $Y_i \in \mathcal{Y}$ some polish space and $\mathbb{N}^* = \mathbb{N}\{0\}$. Let $\varphi: \mathcal{Y}^m \to \mathbb{R}$ be a measurable function. In this paper, we are primarily concerned with the estimation of the conditional expectation, or regression function, for $\mathbf{t} \in \mathbb{R}^{dm}$,

$$r^{(m)}(\varphi, \mathbf{t}) = \mathbb{E}(\varphi(Y_1, \dots, Y_m) | (\mathbf{X}_1, \dots, \mathbf{X}_m) = \mathbf{t}), \quad (1.1)$$

whenever it exists, i.e.,

$$\mathbb{E}(|\varphi(Y_1, \dots, Y_m)|) < \infty.$$

We now introduce a kernel function $K: \mathbb{R}^d \to \mathbb{R}$ with support contained in $[-B, B]^d$, $B > 0$, satisfying:

$$\sup_{\mathbf{x} \in \mathbb{R}^d} |K(\mathbf{x})| =: \kappa < \infty \quad \text{and} \quad \int K(\mathbf{x}) d\mathbf{x} = 1. \qquad (1.2)$$

[182] introduced a class of estimators for $r^{(m)}(\varphi, \mathbf{t})$, called conditional U-statistics, which is defined for each $\mathbf{t} \in \mathbb{R}^{dm}$ to be:

$$\widehat{r}_n^{(m)}(\varphi, \mathbf{t}; h_K) = \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} \varphi(Y_{i_1}, \dots, Y_{i_m}) K\left(\frac{\mathbf{t}_1 - \mathbf{X}_{i_1}}{h_K}\right) \cdots K\left(\frac{\mathbf{t}_m - \mathbf{X}_{i_m}}{h_K}\right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K\left(\frac{\mathbf{t}_1 - \mathbf{X}_{i_1}}{h_K}\right) \cdots K\left(\frac{\mathbf{t}_m - \mathbf{X}_{i_m}}{h_K}\right)}, \quad (1.3)$$
where

$$I(m, n) = \{ \mathbf{i} = (i_1, \dots, i_m) : 1 \le i_j \le n \text{ and } i_j \ne i_r \text{ if } j \ne r \},$$

is the set of all m-tuples of different integers between 1 and n and {$h_K := h_n$}$_{n \ge 1}$ is a sequence of positive constants converging to zero at the rate $nh_K^m \to \infty$. In the particular case m = 1, the $r^{(m)}(\varphi, t)$ is reduced to

$$r^{(1)}(\varphi, t) = \mathbb{E}(\varphi(\mathbf{Y})|\mathbf{X} = t),$$

and Stute's estimator becomes the Nadaraya-Watson [153, 200] estimator of $r^{(1)}(\varphi, t)$, refer to [50] for details. The work of [175] was devoted to estimating the rate of the uniform convergence in $\mathbf{t}$ of $\hat{r}_n^{(m)}(\varphi, \mathbf{t}; h_K)$ to $r^{(m)}(\varphi, \mathbf{t})$. In [165], the limit distributions of $\hat{r}_n^{(m)}(\varphi, \mathbf{t}; h_K)$ are analyzed and compared to those derived by Stute. Under proper mixing settings, [111] extended the results of [182] to weakly dependent data and applied their findings to validate the Bayes risk consistency of the corresponding discrimination rules. As alternatives to the conventional kernel-type estimators, [186] offered symmetrized closest neighbor conditional U-statistics. [98] evaluated the functional conditional U-statistic and determined its finite-dimensional asymptotic normality. Nonparametric estimate of the conditional U-statistics in a functional data context has gotten comparatively little attention despite the subject's significance. Recent developments are described in [42, 44, 45, 52, 56], in which the authors examine challenges associated with the uniform in bandwidth consistency in general settings. [119] evaluated the test of independence in the functional framework using the Kendall statistics, which may be viewed as special cases of the U-statistics. [14] introduced a comprehensive framework for clustering within multiple groups, employing a U-statistics-based approach specifically designed for high-dimensional datasets. This method classifies data into three groups while evaluating the significance of these partitions. In a related context, [128] focused on dimension-agnostic inference, devising methods whose validity remains independent of assumptions regarding dimension versus sample size. Their approach utilized variational representations of existing test statistics, incorporating sample splitting and self-normalization to yield a refined test statistic with a Gaussian limiting distribution. This modification of degenerate U-statistics involved dropping diagonal blocks and retaining off-diagonal blocks. Exploring further, [59] delved into U-statistics-based empirical risk minimization, while [121] examined asymmetric U-statistics based on a stationary sequence of m-dependent variables, with applications motivated by pattern matching in random strings and permutations. Additionally, [188] developed innovative U-statistics considering left truncation and right censoring. As an application, they proposed a straightforward non-parametric test for assessing the independence between time to failure and the cause of failure in competing risks, particularly when observations are subject to left truncation and right censoring. In a different context, [136] investigated the quadruplet U-statistic, offering applications in statistical inference for network analysis. It will be interesting to find connection of the U-statistics with the problems investigated in [187, 201, 204, 205]. The extension of the preceding investigation to conditional empirical U-processes is both practically beneficial and technically difficult.

Recently, there has been a growing interest in regression models in which the response variable is real-valued and the explanatory variable is represented by smooth functions that vary arbitrarily between repeated observations or measurements. This form of data, known as functional data, appears in numerous disciplines, such as climatology (hourly concentration of pollutants), medicine (the knee angle of children as functions of time), economics, linguistics, etc. Functional time series are commonly encountered in practice, for example, when a long continuous time process is divided into
smaller natural units, such as days. In this instance, every intraday curve is a functional random variable. This paper focuses mostly on the instance of functional data and the theory of the U-processes. We give an excerpt from [6]: Functional data analysis (FDA) is a branch of statistics that focuses on the study of variables having an unlimited number of dimensions, such as curves, sets, and images. It has had spectacular growth over the past two decades, fueled partly by technological advances that have led to the “Big Data” revolution. FDA is today one of the most active and significant disciplines of research in data science, despite its reputation at the turn of the century as a fairly obscure area of study. The reader is recommended to the works of [167, 168], [91] for an introduction to this field. These sources offer a variety of case studies in numerous fields, including criminology, economics, archaeology, and neurophysiology, as well as basic analysis techniques. It should be noted that the extension of probability theory to random variables with values in normed vector spaces (such as Banach and Hilbert spaces), as well as extensions of certain classical asymptotic limit theorems, predates the recent literature on functional data; the reader is referred to [7]. [99] examined density and mode estimates for data occupying a normed vector space. This study examines the topic of the curse of dimensionality for functional data and proposes solutions to the problem. In the context of regression estimation, [91] examined nonparametric models. We may also refer to [21, 116]. [130] provided a nice mix of foundational material, accessible theory, and practical examples. Recently, the contemporary theory was used in the analysis of functional data. [93], who has provided the consistency rates of several functionals of the conditional distribution, such as the regression function, the conditional cumulative distribution, the conditional density, and others, uniformly over a subset of the explanatory variable. [125] established the consistency rates for some functionals nonparametric models, such as the regression function, the conditional distribution, the conditional density, and the conditional hazard function, uniformly in bandwidth (UIB consistency). [35] extended these results to the ergodic setting. [12] examined the issue of local linear estimation of the regression function when the regressor is functional and demonstrated strong convergence (with rates) uniformly in bandwidth parameters. [141] examined the *k*-nearest neighbours (kNN) estimate of the nonparametric regression model for strong mixing functional time series data and demonstrated the uniform nearly complete convergence rate of the *k*NN estimator under several moderate conditions. [30] provided several limiting law results for the conditional mode in the functional setting for ergodic data; for more recent references, see [3, 4, 5, 53, 55, 151].

The primary objective of this study is to examine a generic framework and characterize the weak convergence and the uniform convergence of *k*NN conditional *U*-processes based on a regular sequence of random functions. This is motivated by the fact that the *k*-NN method is a fundamental statistical method presenting several advantages. Recall that the *k*-NN method considers the *k* neighbors of *X*ᵢ nearest to *x* with respect to some distance *d*(·, ·). Although the local bandwidth of the *k*-NN is random and depends on the data *X*ᵢ respecting the local structure of the data, which is essential in the infinite dimension. Historically, the *k*-NN was first introduced by [95]-see also [96]-in the context of nonparametric discrimination, and further investigated by [144], for more details, we refer to [17]. It is commonly used in practice (see [91]) and is simple to handle because the user has only one parameter to control the number *k* of nearest neighbors, valued in a finite set. In addition, it allows us to build a neighbor adapted to the data at any point. The *k*-NN method is widely studied if the explanatory variable is an element of a finite-dimensional space, for instance, see [16, 64, 77, 104, 145]. In an infinite dimensional space, i.e., a functional framework, there are three different approaches for the
$k$-NN regression estimation. The first, published by [135], examines a $k$-NN kernel estimate when the functional variable is an element of a separable Hilbert space $H$. In this approach, [135] established a weak consistency result. The strategy of [135] is to reduce the infinite dimension of $H$ by using a projection on a finite dimension subspace by considering only the first $m$ coefficients of an expansion of $X$ in an orthonormal system of $H$ and then applying the multivariate techniques on the projected data to perform the $k$-NN regression. The second approach is based on the $k$-NN procedure and the functional local linear estimation, the consistency with the convergence rate is obtained in [60] and [142].

More precisely, in this paper, we are interested in establishing the a.co uniform consistency and the a.co uniform in the number of neighbors (UINN) consistency of the nonparametric functional regression estimator and also the functional conditional $U$-processes (statistics). [157] were the first to introduce the notion of uniform in bandwidth consistency for kernel density estimators and they applied empirical process methods in their study. This is motivated by a series of papers, among many others, [15, 25, 33, 36, 37, 38, 42, 44, 51, 52, 54, 71, 78, 87, 88] the authors established uniform in bandwidth (UIB) consistency results for such estimators in the i.i.d. finite-dimensional setting, where $h_n$ varies within suitably chosen intervals indexed by $n$. In the FDA, several authors have been interested in studying non-parametric functional estimators. For example, [93] provided the consistency rates of some functionals of the conditional distribution, including the regression function, the conditional cumulative distribution, the conditional density, and some others, uniformly over a certain subset of the explicative variable. [125] established the uniform consistency rate for some conditional models, including the regression function, the conditional distribution, the conditional density, and the conditional hazard function. The last mentioned paper is extended by [42]. [124] and [3] established the almost complete convergence of the $k$-nearest neighbors ($k$-NN) estimators, which are uniform in the number of neighbors, under some classical assumptions on the kernel and on the small ball probabilities of the functional variable in connection with the entropy condition controlling the space complexity. [12] considered the problem of local linear estimation of the regression function when the covariate is functional and proved the strong uniform-in-bandwidth (UIB) convergence. [141] investigated the $k$-NN estimation of the nonparametric regression model for strong mixing functional time series data and established the uniform a.co convergence rate of the $k$-NN estimator under some mild conditions. [158] stated some new uniform asymptotic results for kernel estimates in the functional single-index model. Most of this literature focuses on UIB or UINN consistency or on uniform consistency on some functional subset but never the both together, which was investigated in [44] in the independent framework. We aim to fill this gap in the literature by combining results from the FDA and the empirical processes theory in the dependent setting. The second problem for the weak convergence that we investigate is not simple, and the main merits of our contribution are the control of the asymptotic equi-continuity under minimal conditions in this general setting, which constitutes a fundamentally unresolved open problem in the literature. We intend to fill this gap in the literature by integrating the results of [8] and [27] with the strategies described in [147] and [43] for handling functional data. But, as will be demonstrated in the following section, the challenge requires much more than “just” merging concepts from the current outcomes. In reality, intricate mathematical derivations will be necessary to deal with the typical functional data in our framework. This necessitates the effective application of large sample theory tools, which were established for dependent empirical processes and for which we have used results from the work of
[8, 27, 43]. Even with i.i.d. functional data, no weak convergence for the $kNN$ conditional $U$-processes has been proven up to the present.

## 1.1. Paper contribution

The current paper looks into the challenges high-dimensional functional data presents and presents important findings that are applicable to high-dimensional data models. This research expands the classical kernel estimator beyond samples that are independent and identically distributed to include stationary random processes. In addition, the main focus is on the $kNN$ kernel estimator. It specifically examines four fundamental aspects that pertain to the $kNN$ kernel estimator for regression and its uniform consistency. The study investigates the UINN and UIB scenarios to guarantee consistent results within the domain of functional regression (the UIB in Theorem 3.3 and the UINN in Theorem 3.1). The outcomes include the prediction of relative error, which enhances the overall comprehension of the $kNN$ kernel estimator's performance (the UIB in Corollary 3.6 and the UINN in Corollary 3.5). The paper includes the $kNN$ method's for functional conditional $U$-statistics with consistent results, thereby extending the application area of these estimators. The study additionally investigates the uniform consistency and UINN consistency of functional conditional $U$-statistics, offering a comprehensive assessment of their limiting behaviour (the UIB in Theorems 3.8–3.10 and Corollary 3.12 and the UINN in Theorems 3.14, 3.16 and Corollary 3.18). In addition, the study makes an advanced contribution to the field by establishing a uniform central limit theorem for classes of functions that, subject to specific moment conditions, are either bounded or unbounded (for the conditional process the Normality is provided in Theorem 4.1 and the equicontinuity in Theorem 4.2; for the conditional $U$-process the Normality is provided in Theorems 4.5, 4.6 and the equicontinuity in Theorem 4.7). The process of establishing our main results uses advanced methodologies, including the k-nearest neighbours ($kNN$) method, covering number, small-ball probability, the Hoeffding decomposition, the decoupling methods, and the modern theory of the empirical process indexed by functions. At this stage, we mention that the Hoeffding decomposition cannot be used directly in our setting and needs some intricate preparation. All these results are established under fairly general conditions on function classes and underlying distributions, the majority of which are derived from prior works, thus guaranteeing their feasibility. The obtained results have the potential to be utilised in numerous statistical domains, such as time series prediction, set-indexed conditional $U$-statistics, and the Kendall rank correlation coefficient. Key technical tools in the proofs are the maximal moment inequalities for $U$-processes, and [84]'s results on $\beta$-mixing.

## 1.2. Paper Organization

The layout of the present article is as follows. Section 2 is devoted to introducing the functional framework and the definitions that we need in our work, we give the assumptions used in our asymptotic analysis with a short discussion. Section 3 is devoted to the strong uniform convergence with rate. Section 4.1 provides the weak convergence of empirical processes in the functional framework. Section 4.2 gives the main results of the paper concerning the uniform TCL for the conditional $U$-processes. In Section 5, we collect some potential applications, including the set indexed conditional $U$-statistics in Section 5.1, Kendall rank correlation coefficient in Section 5.2, the discrimination problems in Section 5.3 and the time series prediction from a continuous set of past
values in Section 5.4. We discuss a bandwidth choice for practical use in Section 6. Some concluding remarks and possible future developments are relegated to Section 7. To prevent interrupting the flow of the presentation, all proofs, based upon modern empirical process theory, are gathered in Section 8. Due to the lengthiness of the proofs, we limit ourselves to the most important arguments. A few relevant technical results are given in the Appendix.

## 2. The functional framework

### 2.1. Generality on the model

Let $\{(X_i, Y_i) : i \ge 1\}$ be a sequence of stationary$^\dagger$ random copies of the random vector [rv] $(X, Y)$, where $X$ takes its values in some abstract space $\mathcal{X}$ and $Y$ in the abstract space $\mathcal{Y}$. Suppose that $\mathcal{X}$ is endowed with a semi-metric $d(\cdot, \cdot)^{\ddagger}$ defining a topology to measure the proximity between two elements of $\mathcal{X}$ and which is disconnected from the definition of $X$ to avoid measurability problems. We are mainly interested in establishing the weak convergence of the conditional $U$-process based on the following $U$-statistic in the $k$-NN setting introduced in [44] by

$$ \widehat{r}_n^{(m)}(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t})) = \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} \varphi(Y_{i_1}, \dots, Y_{i_m}) K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}, \quad (2.1) $$

as an estimator for the multivariate regression function

$$ r^{(m)}(\varphi, \mathbf{t}) := \mathbb{E}(\varphi(Y_1, \dots, Y_m) | X_1, \dots, X_m = \mathbf{t}), \quad (2.2) $$

where

$$ \tilde{K}(\mathbf{v}) = \prod_{i=1}^{m} K(v_i), \quad \mathbf{v} = (v_1, \dots, v_m) \in \mathbb{R}_+^m, \quad (2.3) $$

and $\varphi : \mathcal{Y}^m = \mathcal{Y} \times \dots \times \mathcal{Y} \to \mathbb{R}$ is a symmetric measurable function belonging to some class of functions $\mathcal{F}_m$, and $\mathbf{h}_{n,k}(\mathbf{t}) = (H_{n,k}(t_1), \dots, H_{n,k}(t_m))$ is a vector of positive random variables that depend on $(X_1, \dots, X_n)$ such that, for all $\mathbf{x} = (x_1, \dots, x_m) \in \mathcal{X}^m$ and $j = 1, \dots, m$:

$$ H_{n,k}(x_j) = \min \left\{ h \in \mathbb{R}^+ : \sum_{i=1}^{n} 1_{B(x_j,h)}(X_i) = k \right\}, \quad (2.4) $$

where $B(t, r) = \{z \in \mathcal{X} : d(z, t) \le r\}$ is a ball in $\mathcal{X}$ with the center $t \in \mathcal{X}$ and radius $r$, and $1_A$ is the indicator function of the set $A$. In fact, this $k$-NN estimate can be considered as an extension to the

$$ \mathbb{E}((X_s - \mu) \otimes (X_t - \mu)) = \mathbb{E}((X_{s-t} - \mu) \otimes (X_0 - \mu)), $$

<footnote>$^\dagger$In the case of the Hilbert space valued elements not necessarily strictly stationary is needed, a second order stationarity suffices. An Hilbert space valued sequence $\{X_t\}_{t \in \mathbb{Z}}$ is second-order (or weakly) stationary if $\mathbb{E}\|X_t\|^2 < \infty$, $\mathbb{E}X_t = \mu$, and</footnote>

<footnote>for all $s, t \in \mathbb{Z}$. We say that $\{X_t\}_{t \in \mathbb{Z}}$ is strictly stationary if the joint distribution of $\{X_{t_1}, \dots, X_{t_n}\}$ and the joint distribution of $\{X_{t_{1+h}}, \dots, X_{t_{n+h}}\}$ coincide, for all $t_1, \dots, t_n \in \mathbb{Z}$, $n \ge 1$, and $h \ge 1$.</footnote>

<footnote>$^{\ddagger}$A semi-metric (sometimes called pseudo-metric) $d(\cdot, \cdot)$ is a metric which allows $d(x_1, x_2) = 0$ for some $x_1 \neq x_2$.</footnote>
random and locally adaptive neighbor of the functional conditional U-statistics estimate of $r^{(m)}(\varphi, \mathbf{t})$ defined for all $t \in \mathcal{X}^m$ as :

$$ \widehat{\mathcal{R}}_{n}^{(m)}(\varphi, \mathbf{t}, h_K(\mathbf{t})) = \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} \varphi(Y_{i_1}, \dots, Y_{i_m}) K\left(\frac{d(t_1, X_{i_1})}{h_K(t_1)}\right) \dots K\left(\frac{d(t_m, X_{i_m})}{h_K(t_m)}\right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K\left(\frac{d(t_1, X_{i_1})}{h_K(t_1)}\right) \dots K\left(\frac{d(t_m, X_{i_m})}{h_K(t_m)}\right)}, \quad (2.5) $$

where $h_K(\mathbf{t}) = (h_{K,n}(t_1), \dots, h_{K,n}(t_m)) =: (h_K(t_1), \dots, h_K(t_m))$ are positive real numbers decreasing to zero as $n$ goes to infinity. At this stage, we highlight that kernel estimation is popular since the classical Akaike-Parzen-Rosenblatt kernel density estimation [refer to [2, 160, 171]]. However, the first appearance of kernel estimators is likely to be [95]: as the original technical report is difficult to find, it has been re-published as [96]. [160] has shown, under some assumptions on $K(\cdot)$, that $f_n(\cdot)$ is an asymptotically unbiased and consistent estimator for $f(\cdot)$ whenever $h_n \to 0$, $nh_n^d \to \infty$ and $\tilde{x}$ is a continuity point of $f(\cdot)$. Under some additional assumptions on $f(\cdot)$ and $h_n$, he obtained an asymptotic normality result, too. The kernel estimators have been extensively studied in the literature, see, e.g., [28, 31, 34, 49, 73, 75, 76, 85, 86, 108, 154, 174, 199] and the references therein. The k-NN method is a fundamental statistical tool with various advantages. Generally, the procedure is computationally efficient and requires minimal parameter adjustment. In addition, the k-NN approaches are nonparametric, which allows them to automatically adapt to any continuous underlying distributions without relying on any particular models. The k-NN techniques were proven consistent for various significant statistical problems, including density estimation, classification, and regression, provided that a suitable k is chosen. It should be noted that since our objective is to generalize the results obtained for the estimator defined in (2.5), and given the fact that one of the main differences is that the smoothing parameter, $h_{n,k}(\mathbf{t})$ is a vector of random variables instead of a univariate parameter $h_K$ our first course of action would be to extend the results of [42, 43] to the multivariate setting. First, we need to introduce some notation. Let $\mathcal{F}_m = \{\varphi : \mathcal{Y}^m \to \mathbb{R}\}$ denote a pointwise measurable class of real-valued symmetric functions on $\mathcal{Y}^m$ with a measurable envelope function

$$ F(\mathbf{y}) \geq \sup_{\varphi \in \mathcal{F}_m} |\varphi(\mathbf{y})|, \quad \text{for } \mathbf{y} \in \mathcal{Y}^m. \qquad (2.6) $$

For a kernel function $K(\cdot)$ and a subset $S_X \subset \mathcal{X}$, we define the pointwise measurable class of functions, for $1 \le m \le n$:

$$ \mathcal{H}^m := \left\{ (x_1, \dots, x_m) \mapsto \prod_{i=1}^{m} K\left(\frac{d(x_i, t_i)}{h_i}\right), (h_1, \dots, h_m) \in \mathbb{R}_+^m \setminus \{0\} \text{ and } (\mathbf{t}_1, \dots, \mathbf{t}_m) \in S_X^m \right\}. $$

Statistical observations are not always independent but are often close to being so. Dependence may lead to severe repercussions on statistical inference if it is not taken into consideration. The notion of mixing quantifies how close to independence a sequence of random variables is, allowing us to extend standard results for independent sequences to weakly dependent or mixing sequences. Let us specify the dependence that will be the focus of this study. Let $\mathbf{Z}_1, \mathbf{Z}_2, \dots$ be a stationary sequence of random variables on some probability space $(\Omega, \mathcal{D}, \mathbb{P})$ and let $\sigma_i^j$ be the $\sigma$-field generated by $\mathbf{Z}_i, \dots, \mathbf{Z}_j$, for
i, j ≥ 1. The sequence $\mathbf{Z}_1, \mathbf{Z}_2, \dots$ is said $\beta$-mixing or absolute regular, refer to [170, 197], if :

$$ \beta(s) := \mathbb{E} \sup_{l \ge 1} \left\{ \left| \mathbb{P} (A | \sigma_1^l) - \mathbb{P} (A) \right| : A \in \sigma_{l+s}^{\infty} \right\} \to 0 \quad \text{as } s \to \infty. $$

It should be noted that [118] obtained a complete description of stationary Gaussian processes satisfying the last property. Throughout the sequel, we assume tacitly that the sequence of random elements $\{(X_i, Y_i), i \in \mathbb{N}^*\}$ is absolutely regular. The Markov chains, for instance, are $\beta$-mixing under the milder Harris recurrence condition if the underlying space is finite [19, 46, 66, 180]. We also need to introduce some concepts that are related to the topological structure of functional spaces. First, we define the small-ball probability for a fixed $t \in \mathcal{X}$ and for all $r > 0$ by

$$ \mathbb{P}(X \in B(t, r)) := \phi_t(r), \qquad (2.7) $$

this notion is widely used in nonparametric functional data analysis to avoid introducing density assumptions on the functional variable $X$ and address the issues associated with the infinite-dimensional nature of the functional spaces. At this point, we can refer to [91, 99, 147]. We also need to deal with the VC-subgraph classes ("VC" for Vapnik and Chervonenkis, for instance, see [132, 193, 194]).

**Definition 2.1.** A class of subsets $C$ on a set $C$ is called a VC-class if there exists a polynomial $\mathfrak{P}(\cdot)$ such that, for every set of $N$ points in $C$, the class $C$ picks out at most $\mathfrak{P}(N)$ distinct subsets.

**Definition 2.2.** A class of functions $\mathcal{F}$ is called a VC-subgraph class if the graphs of the functions in $\mathcal{F}$ form a V-C class of sets, that is, if we define the subgraph of a real-valued function $f$ on $S$ as the following subset $\mathcal{G}_f$ on $S \times \mathbb{R}$

$$ \mathcal{G}_f = \{(s, t) : 0 \le t \le f(s) \text{ or } f(s) \le t \le 0\}, $$

the class $\{\mathcal{G}_f : f \in \mathcal{F}\}$ is a VC-class of sets on $S \times \mathbb{R}$. Informally, a VC-class of functions is characterized by having a polynomial covering number (the minimal number of required functions to make a covering on the entire class of functions).

**Definition 2.3.** Let $S_{\mathcal{E}}$ be a subset of a semi-metric space $\mathcal{E}$ and $N_{\mathcal{E}}$ a positive integer, a finite set of points $\{e_1, \dots, e_{N_{\mathcal{E}}}\} \subset \mathcal{E}$ is called, for a given $\varepsilon > 0$, a $\varepsilon$-net of $S_{\mathcal{E}}$ if :

$$ S_{\mathcal{E}} \subseteq \bigcup_{j=1}^{N_{\mathcal{E}}} B(e_j, \varepsilon). $$

If $N_{\mathcal{E}}(S_{\mathcal{E}})$ is the cardinality of the smallest $\varepsilon$-net (the minimal number of open balls of radius $\varepsilon$) in $\mathcal{E}$, needed to cover $S_{\mathcal{E}}$, then we call Kolmogorov's entropy (metric entropy) of the set $S_{\mathcal{E}}$, the quantity

$$ \psi_{S_{\mathcal{E}}} (\varepsilon) := \log N_{\mathcal{E}}(S_{\mathcal{E}}). $$

From its name, one can figure that this concept of metric entropy was introduced by Kolmogorov [131] and was studied subsequently for numerous metric spaces. This concept was used by [80] to give sufficient conditions for continuity of Gaussian processes, and was the basis for striking generalizations of Donsker's theorem on the weak convergence of the empirical process. Suppose that $\mathcal{B}_{\mathcal{X}}$ and $S_{\mathcal{X}}$ are
two subsets of the semi-metric space $\mathcal{X}$ with Kolmogorov's entropy (for the radius $\varepsilon$) $\psi_{\mathcal{B}_{\mathcal{X}}}(\varepsilon)$ and $\psi_{S_{\mathcal{X}}}(\varepsilon)$ respectively, then the Kolomogorov entropy for the subset $\mathcal{B}_{\mathcal{X}} \times S_{\mathcal{X}}$ of the semi-metric space $\mathcal{X}^2$ by:

$$ \psi_{\mathcal{B}_{\mathcal{X}} \times S_{\mathcal{X}}}(\varepsilon) = \psi_{\mathcal{B}_{\mathcal{X}}}(\varepsilon) + \psi_{S_{\mathcal{X}}}(\varepsilon). $$

Hence, $m\psi_{S_{\mathcal{X}}}(\varepsilon)$ is the Kolmogorov entropy of the subset $S_{\mathcal{X}}^m$ of the semi-metric space $\mathcal{X}^m$. Noting that if we designate by $d$ the semi-metric on $\mathcal{X}$, then, we can define the semi metric on $\mathcal{X}^m$ by:

$$ d_{\mathcal{X}^m}(\mathbf{x}, \mathbf{y}) := \frac{1}{m}d(x_1, y_1) + \cdots + \frac{1}{m}d(x_m, y_m) $$

for

$$ \mathbf{x} = (x_1, \dots, x_m), \mathbf{y} = (y_1, \dots, y_m) \in \mathcal{X}^m. $$

Notice that the semi-metric plays an important role in this kind of study. The reader will find useful discussions about how to choose the semi-metric in [91] (see Chapters 3 and 13).

## 2.2. Conditions and comments

Let us present the conditions that we need in our analysis.

### (C.1.) On the distributions/small-ball probabilities

(C.1.1) For $\mathbf{t} = (t_1, \dots, t_m) \in \mathcal{X}^m$ and $\mathbf{h}(\mathbf{t}) = (h_1(t_1), \dots, h_m(t_m)) \in \mathbb{R}_+^m \setminus \{0\}$, we have

$$ \phi_t(\mathbf{h}(\mathbf{t})) := \mathbb{P}(X_1 \in B(t_1, h_1(t_1)), \dots, X_m \in B(t_m, h_m(t_m))) $$

$$ 0 < C_1 \tilde{\phi}(\mathbf{h}) f_1(\mathbf{t}) \le \phi_t(\mathbf{h}(\mathbf{t})) \le C_2 \tilde{\phi}(\mathbf{h}) f_1(\mathbf{t}) < \infty, $$

where $f_1(\mathbf{t})$ is a non-negative functional in $\mathbf{t} = (t_1, \dots, t_m) \in \mathcal{X}^m$, $\tilde{\phi}(\mathbf{h}) := \prod_{j=1}^m \phi(h_j(t_j))$ and $\phi(0) = 0$ and $\phi(u)$ is an invertible function absolutely continuous in a neighbor of the origin.

(C.1.2) For $i \neq j$, let $\mathbf{X}_i = (X_{i_1}, \dots, X_{i_m})$, $\mathbf{X}_j = (X_{j_1}, \dots, X_{j_m})$ and $\mathbf{t} = (t_1, \dots, t_m) \in \mathcal{X}^m$, we have

$$ \sup_{i \neq j} \mathbb{P} \left\{ \mathbf{X}_i \in \prod_{i=1}^m B(t_i, h_i(t_i)), \mathbf{X}_j \in \prod_{i=1}^m B(t_i, h_i(t_i)) \right\} \le \tilde{\Psi}(\mathbf{h}(\mathbf{t})) f_2(\mathbf{t}), $$

where $f_2(\mathbf{t})$ is a non-negative function, $\tilde{\Psi}(\mathbf{h}(\mathbf{t})) := \prod_{i=1}^m \psi(h_i(t_i))$ and $\tilde{\Psi}(\mathbf{h}(\mathbf{t})) \to 0$ as $n \to \infty$, satisfying $\tilde{\Psi}(\mathbf{h}(\mathbf{t})) / \tilde{\phi}^2(\mathbf{h}(\mathbf{t}))$ is bounded.

### (C.2) On the smoothness of the model

(C.2.1) The regression satisfies for $\varphi(\cdot) \in \mathcal{F}_m$, and for $1 \le m \le n$:

$$ \exists \gamma > 0, \forall \mathbf{t}_1, \mathbf{t}_2 \in S_{\mathcal{X}}^m : |r^{(m)}(\varphi, \mathbf{t}_1) - r^{(m)}(\varphi, \mathbf{t}_2)| \le C_3 d_{\mathcal{X}^m}^{\gamma}(\mathbf{t}_1, \mathbf{t}_2), \text{ for some } \gamma > 0, $$
where for $1 \le i \le 2 : \mathbf{t}_i := (t_{i_1}, \dots, t_{i_m})$ and the semi metric $d_{\mathcal{X}^m}(\cdot, \cdot)$ on $\mathcal{X}^m$:

$$ d_{\mathcal{X}^m}(\mathbf{x}, \mathbf{y}) := \frac{1}{m}d(x_1, \varphi(Y)) + \cdots + \frac{1}{m}d(x_m, y_m) $$

$$ \text{for } \mathbf{x} = (x_1, \dots, x_m), \mathbf{y} = (y_1, \dots, y_m) \in \mathcal{X}^m. $$
(C.2.2) The conditional variance, defined for $u \in \mathcal{X}^m$, $\mathrm{Var}[\varphi(\mathbf{Y})|\mathbf{X}=\mathbf{u}] =: g_2(\mathbf{u}, \varphi)$ is continuous in some neighborhood of $\mathbf{t}$

$$ \sup_{d_{\chi^m}(\mathbf{t}, \mathbf{u}) \le h_n} |g_2(\mathbf{t}, \varphi) - g_2(\mathbf{u}, \varphi)| = o(1) \text{ as } n \to \infty. $$

Further, assume that for some $p > 2$, $\mathbb{E}|F(\mathbf{Y})|^p < \infty$, and $\varphi(\cdot) \in \mathcal{F}_m$,

$$ g_p(\mathbf{t}, \mathbf{u}, \varphi) := \mathbb{E}(|\varphi(\mathbf{Y}) - r^{(m)}(\varphi, \mathbf{t})|^p | \mathbf{X} = \mathbf{u}), $$

is continuous in some neighborhood of $\mathbf{t}$.

(C.2.3) For $\mathbf{u}, \mathbf{v} \in \mathcal{X}^m$, the function $g_\varphi(\mathbf{t}, \mathbf{u}, \mathbf{v})$ does not depend on $\mathbf{i}, \mathbf{j}$ and is continuous in some neighborhood of $(\mathbf{t}, \mathbf{t})$

$$ g_{\varphi}(\mathbf{t}, \mathbf{u}, \mathbf{v}) := \mathbb{E}((\varphi(\mathbf{Y}_i) - r^{(m)}(\varphi, \mathbf{t})) (\varphi(\mathbf{Y}_j) - r^{(m)}(\varphi, \mathbf{t})) | \mathbf{X}_i = \mathbf{u}, \mathbf{X}_j = \mathbf{v}). $$

### (C.3) On the kernel function

(C.3.1) The kernel functions $K(\cdot)$ is supported within $[0, 1]$ and there exists some constants $0 < \kappa_1 \le \kappa_2 < \infty$, such that

$$ \int_0^1 K(x)dx = 1, $$

and

$$ 0 < \kappa_1 1_{[0,1]}(\cdot) \le K(\cdot) \le \kappa_2 1_{[0,1]}(\cdot). $$

(C.3.2) The kernel $K(\cdot)$ is a positive function and differentiable function on $[0, 1]$ with derivative $K'(\cdot)$ such that

$$ -\infty < \kappa_3 < K'(\cdot) < \kappa_4 < 0. \qquad (2.8) $$

### (C.4) On the classes of functions

(C.4.1) The class of functions $\mathcal{F}_m$ is bounded and its envelope function satisfies for some $0 < M < \infty$:

$$ F(\mathbf{y}) \le M, \quad \mathbf{y} \in \mathcal{Y}^m $$

(C.4.2) The class of functions $\mathcal{F}_m$ is unbounded and its envelope function satisfies for some $p > 2$:

$$ \theta_p := \sup_{\mathbf{t} \in S_m^\mathcal{X}} \mathbb{E}(F^p(\mathbf{Y})|\mathbf{X} = \mathbf{t}) < \infty. $$

(C.4.3) The metric entropy of the class $\mathcal{F}_m \mathcal{H}^m$ satisfies, for some $1 \le p < \infty$:

$$ \int_0^\infty (\log N(u, \mathcal{F}_m \mathcal{H}^m, \|\cdot\|_p))^{\frac{1}{2}} du < \infty, $$

where

$$ \mathcal{F}_m \mathcal{H}^m = \{fg : f \in \mathcal{F}_m, g \in \mathcal{H}^m\}. $$
(C.4.4) The class of functions $\mathcal{F}_m\mathcal{H}^m$ is supposed to be of VC-type with envelope function previously defined. Hence, there are two finite constants $b$ and $\nu$ such that:

$$N(\epsilon, \mathcal{F}_m\mathcal{H}^m, \|\cdot\|_{L_2(Q)}) \leq \left( \frac{b\|F\kappa^m\|_{L_2(Q)}}{\epsilon} \right)^\nu$$

for any $\epsilon > 0$ and each probability measure such that $Q(F)^2 < \infty$.

(C.5) On the dependence of the random variables

(C.5.1) Absolute regularity

$$\sum_{s=1}^{\infty} s^{\delta} (\log(s))^{\delta(1-1/p)} (\beta(s))^{1-2/p} < \infty,$$

for some $p > 2$ and $\delta > 1 - 2/p$.

(C.5.2) There is a sequence of positive integers $\{s_n\}_{n \in \mathbb{N}^*}$ such that, as $n \to \infty$,

$$s_n \to \infty, \quad s_n = o\left(\sqrt{n\tilde{\phi}(\mathbf{h}(t))}\right), \quad \left(\frac{n}{\tilde{\phi}(\mathbf{h}(t))}\right)^{1/2} \beta(s_n) \to 0.$$

(C.6.) On the entropy

For $n$ large enough and for some $\omega > 1$, the Kolomogorov's entropy satisfies :

$$\frac{(\log n)^2}{n\phi(h_K)} \leq m\psi_{S_X}\left(\frac{\log n}{n}\right) \leq \frac{n}{\log n}\phi(h_K), \qquad (2.9)$$

$$\sum_{n=1}^{\infty} \exp \left\{ m(1-\omega)\psi_{S_X}\left(\frac{\log n}{n}\right) \right\} < \infty, \qquad (2.10)$$

where $\psi_{S_X}(\epsilon) := \log N_\epsilon(S_X)$, and $N_\epsilon(S_X)$ is the minimal number of open balls of radius $\epsilon$ in $X$, needed to cover $S_X$.

(C.7.) The sequences $\{\tilde{h}_n\}$ and $\{\tilde{h}'_n\}$ (resp. $\{h_{n,1}\}$ and $\{h_{n,2}\}$) verify

$$\tilde{h}'_n \to 0 \text{ and } \frac{(\log n/n)}{\min\{\tilde{h}_n^2, \phi^2(\tilde{h}_n)\}} \to 0 \text{ as } n \to \infty. \qquad (2.11)$$

(C.8.) There exist sequences $\{\rho_{n,1}, \dots, \rho_{n,m}\} \subset (0, 1)^m$, $\{k_{1,n}\} \subset \mathbb{Z}^+$ and $\{k_{2,n}\} \subset \mathbb{Z}^+$ ($k_{1,n} \le k \le k_{2,n}$) and constants $\mu = (\mu_1, \dots, \mu_m)$ and $\nu = (\nu_1, \dots, \nu_m)$, such that

$$0 < \mu_j \le \nu_j < \infty, \text{ for all } j = 1, \dots, m \text{ (and we note } \mu \le \nu),$$

and

$$\mu_j \phi^{-1} \left( \frac{\rho_{n,j} k_{1,n}}{n} \right) \le \phi_{t_j}^{-1} \left( \frac{\rho_{n,j} k_{1,n}}{n} \right) \text{ and } \phi_{t_j}^{-1} \left( \frac{k_{2,n}}{\rho_{n,j} n} \right) \le \nu_j \phi^{-1} \left( \frac{k_{2,n}}{\rho_{n,j} n} \right), \qquad (2.12)$$
$$ \phi^{-1}\left(\frac{k_{2,n}}{\rho_{n,j}n}\right) \to 0, \qquad (2.13) $$

$$ \min \left\{ \frac{1 - \rho_{n,j}}{4} \frac{k_{1,n}}{\ln n}, \frac{(1 - \rho_{n,j})^2 k_{1,n}}{4 \rho_{n,j} \ln n} \right\} > 2, \qquad (2.14) $$

$$ \frac{(\log n/n)}{\min\left\{\mu_j\phi^{-1}\left(\frac{\rho_{n,j}k_{1,n}}{n}\right), \phi\left(\mu_j\phi^{-1}\left(\frac{\rho_{n,j}k_{1,n}}{n}\right)\right)\right\}} \to 0. \qquad (2.15) $$

## Additional/alternative conditions

(C.1') For $\mathbf{X}_i = (X_{i_1}, \dots, X_{i_m})$, $\mathbf{X}_j = (X_{j_1}, \dots, X_{j_m})$ and $\mathbf{t} = (t_1, \dots, t_m) \in \mathcal{X}^m$:

(C.1'.1) We have:

$$ \phi_t(\mathbf{x}) = \tilde{\phi}(\mathbf{x})f_1(\mathbf{t}) < \infty \text{ as } \mathbf{x} \to \infty. $$

with $\mathbf{x} = (x_1, \dots, x_m) \in \mathbb{R}_+^m$, $\phi(0) = 0$, and $\phi(u)$ is absolutely continuous in a neighborhood of the origin.

(C.1'.2) we have:

$$ \sup_{i \neq j} \mathbb{P} \left\{ \mathbf{X}_i \in \prod_{i=1}^m B(t_i, h_i(t_i)), \mathbf{X}_j \in \prod_{i=1}^m B(t_i, h_i(t_i)) \right\} \le \tilde{\Psi}(\mathbf{h}(\mathbf{t})) f_2(\mathbf{t}), $$

where $f_2(\mathbf{t})$ is a non-negative function, $\tilde{\Psi}(\mathbf{h}(\mathbf{t})) := \prod_{i=1}^m \psi(h_i(t_i))$, and $\tilde{\Psi}(\mathbf{h}(\mathbf{t})) \to 0$ as $n \to \infty$, satisfying $\tilde{\Psi}(\mathbf{h}(\mathbf{t})) / \tilde{\phi}^2(\mathbf{h}(\mathbf{t}))$ is bounded.

(C.3') The kernel functions $K(\cdot)$ is supported within $[0, 1]$ and there exists some constants $0 < \kappa_2$, $0 < \kappa'_1 \le \kappa'_2 < \infty$, such that for $j = 1, 2$:

$$ \int_0^1 K(x)dx = 1, \quad K(\cdot) \le \kappa_2 \mathbb{1}_{[0,1]}(\cdot), \quad \frac{h_n}{\phi(h_K(t))} \int_0^1 K^j(x)\phi'(vh_K)dv \to \kappa'_j \text{ as } n \to \infty. $$

## Comments

In our nonparametric functional regression model, we deal with a complex theoretical challenge. Mainly establishing functional central limit theorems for the conditional empirical processes and the conditional U-process under functional absolute regular data. We also use random (or data-dependent) bandwidths based on the $k$ nearest neighbors (kNN) approaches. Although standard statistical methods cannot be utilized in the functional setting, most imposed conditions overlap with some characteristics of the infinite-dimensional spaces, such as the topological structure of $\mathcal{X}^m$, the probability distribution of $X$, and the measurability concept for the classes $\mathcal{F}_m$ and $\mathcal{K}^m$. It is worth mentioning that most of the conditions that we will be using throughout this paper are inspired by [43, 91, 99, 141, 147]. Let us start with assumption (C.1.1), which was adapted from [147], who in turn was inspired by [99]. As explained by [147], if $\mathcal{X}^m = \mathbb{R}^m$, then condition (C.1.1) coincides with the fundamental axioms of probability calculus. Furthermore, if $\mathcal{X}^m$ is an infinite-dimensional Hilbert space, then $\phi(h_K)$ can
converge to 0 exponentially as $n \to \infty$. Condition (C.1.1) can be considered a standard condition on the small ball probability, which is used to control the behavior of $\phi_t(\cdot)$ around zero. It shows that we can approximately write the small ball probability as a product of two independent functions $\tilde{\phi}(\cdot)$ and $f_1(\cdot)$; see, for instance, [148] for the diffusion process, [18] for a Gaussian measure, and [139] for a general Gaussian process. The most frequent result available in the literature is of the form $\varphi_t(\varepsilon) \sim g(t)\phi(\varepsilon)$ where $\phi(\varepsilon) = \varepsilon^\gamma \exp(-C/\varepsilon^p)$ with $\gamma \ge 0$ and $p \ge 0$. It corresponds to the Ornstein-Uhlenbeck and general diffusion processes (for such processes, $p = 2$ and $\gamma = 0$) and the fractal processes (for such processes, $\gamma > 0$ and $p = 0$). For more examples, refer to [92]. It is worth noting that, in general, when we deal with functional data, we need some information about the variability of the small-ball probability to adapt to the bias of nonparametric estimators; this information is usually obtained by supposing that:

(C.1.1)

$$ \forall u \in [0, 1] : \quad \lim_{r \to \infty} \frac{\phi_t ur}{\phi_t(r)} = \lim_{r \to \infty} \mathbb{P} \left( d(X, t) \le ur \mid d(X, t) \le r \right) =: \tau_t(u) < \infty. $$

Condition (C.2) concerns the regularity of the model; it consists of mild conditions on the continuity of certain conditional moments, in addition to the standard Lipschitz assumption on the regression ((C.2.1). Assumption (C.3) is another classical condition in nonparametric estimation models which concerns the kernel function $K(\cdot)$. It should be noted that condition (C.3.1) can be replaced with condition (C.3') to find an expression for the asymptotic variance. We use condition (C.4.1) when dealing with bounded functions. However, our interest also extends to conditional $U$-processes indexed by an unbounded class of functions. In this case, we replace (C.4.1) by (C.4.2). Keep in mind that there is a trade-off between the moment order $p$ in (C.4.2) and the decay rate of the mixing coefficient $\beta(s)$ imposed in (C.5): the larger $p$ is, the weaker the decay of $\beta(s)$. Also note that if $\beta(s) = e^{-as}$, i.e, $\beta(s)$ decays exponentially fast, then (C.5.) is automatically satisfied. Furthermore, this condition is indispensable in our work since studying the weak convergence of the empirical processes entails establishing asymptotic equi-continuity. For Assumption (C.4.4), see [163, Examples 26 and 38], [157, Lemma 22], [82, §4.7.], [193, Theorem 2.6.7], [132, §9.1] provide a number of sufficient conditions under which (C.4.4) holds, we may also refer to [70, §3.2] for further discussions. For instance, it is satisfied, for general $d \ge 1$, whenever $g(x) = \phi(p(x))$, with $p(x)$ being a polynomial in $d$ variables and $\phi(\cdot)$ being a real-valued function of bounded variation, we refer the reader to [88, p. 1381]. We also mention that the class of function is assumed to be in general pointwise measurable, that is satisfied whenever $K(\cdot)$ is of bounded variation on $\mathbb{R}^d$ (in the sense of Hardy and Kauser [110, 133, 195], see, e.g., [61, 114, 156, 196]). The condition (C.6.) takes into account the topological considerations by controlling the Kolmogorov entropy§ of the set $S_X^m$, which is standard in nonparametric models when we study the uniform consistency and the uniform in bandwidth consistency, we refer to [93] and [134] for discussions. As mentioned in [134], there are special cases of functional spaces $\mathcal{X}$ and subsets $S_X$ where $\psi_{S_X}(\log(n)/n) \gg \log(n)$. Some examples are the closed ball in a Sobolev space, the unit ball of the Cameron-Martin space, and a compact subset in a Hilbert space with a projection semi-metric (see [93, 131, 192], respectively, for further details). In all these cases it is easy to see that (C.6.) is verified

<footnote>§The concept of metric entropy was introduced by Kolmogorov (cf. [131]) and studied subsequently for numerous metric spaces. This concept was used by [80] to give sufficient conditions for the continuity of Gaussian processes, and was the basis for striking generalizations of Donsker's theorem on the weak convergence of the empirical process, refer to [191] for the connection of this notion with Le Cam's work.</footnote>
as soon as $\beta > 2$. Assumption (C.7.) is essential to establish the rates of convergence (consistency) of the estimator defined in (2.5), while assumption (C.8.) adapts condition (C.7.) to the case of the functional conditional U-statistics in the k-NN setting.

**Remark 2.4.** Note that the condition (C.4.2) may be replaced by more general hypotheses upon moments of Y as in [70]. That is

(M.1)" We denote by {$\mathcal{M}(x) : x \ge 0$} a nonnegative continuous function, increasing on $[0, \infty)$, and such that, for some $s > 2$, ultimately as $x \uparrow \infty$,

$$ (i) x^{-s} \mathcal{M}(x) \downarrow; (ii) x^{-1} \mathcal{M}(x) \uparrow. \qquad (2.16) $$

For each $t \ge \mathcal{M}(0)$, we define $\mathcal{M}^{inv}(t) \ge 0$ by $\mathcal{M}(\mathcal{M}^{inv}(t)) = t$. We assume further that:

$$ \mathbb{E}(\mathcal{M}(|F(\mathbf{Y})|)) < \infty. $$

The following choices of $\mathcal{M}(\cdot)$ are of particular interest:

(i) $\mathcal{M}(x) = x^p$ for some $p > 2$;

(ii) $\mathcal{M}(x) = \exp(sx)$ for some $s > 0$.

### 3. Uniform consistency

For simplicity reasons, the condition (C.1.1) on the small ball probability will be replaced by:

(H.1) For $\mathbf{h} \in \mathbb{R}_+^m \setminus \{0\}$ and $\mathbf{t} \in \mathcal{X}^m$

$$ 0 < C_1 \tilde{\phi}(\mathbf{h}) \le \phi_t(\mathbf{h}(\mathbf{t})) \le C_2 \tilde{\phi}(\mathbf{h}) < \infty, \qquad (3.1) $$

this standard condition can be considered an extension of the multivariate case where we assume that the density function of the variable $\mathbf{X}$ is strictly positive. Also, it is worth mentioning that, if in particular, we denote $\mathbf{h}_K := (h_K, \dots, h_K) \in (\tilde{h}_{n,1}, \tilde{h}_{n,2})^m$, we can find two positive constants $C'_1, C'_2$, such that

$$ 0 < C'_1 \phi(h_K) \le \phi_t(\mathbf{h}_K) \le C'_2 \phi(h_K) < \infty, \qquad (3.2) $$

which is similar to condition (C.1) used in [42, 181], so we will deal with $\phi(h_K)$ instead of $\phi^m(h_K)$, (whenever we encounter a similar situation in the proofs). This approach is not only for notational purposes but also to make it easier to bridge the UIB and the UINN results.

#### 3.1. Uniform consistency of the kNN kernel estimator for regression

In this section, we consider the uniform consistency of the functional regression operator in its general form, which is given for all $t \in \mathcal{X}$, by

$$ \widehat{r}_n^{*(1)}(\varphi, \mathbf{t}, H_{n,k}(t)) = \frac{\sum_{i=1}^{n} \varphi(Y_i) K \left( \frac{d(t, X_i)}{H_{n,k}(t)} \right)}{\sum_{i=1}^{n} K \left( \frac{d(t, X_i)}{H_{n,k}(t)} \right)}, \qquad (3.3) $$
where $k = k_n$ depending on $n$ and

$$H_{n,k}(t) = \min \left\{ h \in \mathbb{R}^{+} \text{ such that } \sum_{i=1}^{n} 1_{B(t,h)}(X_i) = k \right\}.$$

In fact, the k-NN operator presented in (3.3) can be considered as a generalization of the usual kernel regression

$$\hat{r}_n^{(1)}(\varphi, t, h_K) = \frac{\sum_{i=1}^{n} \varphi(Y_i) K\left(\frac{d(t, X_i)}{h_K}\right)}{\sum_{i=1}^{n} K\left(\frac{d(t, X_i)}{h_K}\right)}, \quad \text{for each } t \in \mathcal{X}, \quad (3.4)$$

where the bandwidth $h_K \in \mathbb{R}_+^*$ depends on $n$ (but does not depend on $t$).

### 3.1.1. UIB consistency for functional regression

Recall the bandwidths $h_{n,1}$ and $h_{n,2}$ given in the condition (C.7). The following theorem will play an instrumental role in the sequel.

**Theorem 3.1.** Under the assumptions (H.1), (C.2.1), (C.3.1), (C.4.1), (C.4.3), (C.6.) and (C.7.) (for $m=1$), we have, as $n \to \infty$,

$$\sup_{\varphi K \in \mathcal{H}} \sup_{h_{n,1} \le h_K \le h_{n,2}} \sup_{t \in S_X} |\hat{r}_n^{(1)}(\varphi, t; h_K) - r^{(1)}(\varphi, t)| = O(h_{n,2}^{\gamma}) + O_{a,co} \left( \sqrt{\frac{\psi_{S_X}\left(\frac{\log n}{n}\right)}{n\phi(h_{n,1})}} \right). \quad (3.5)$$

The following result gives uniform consistency when the class of functions is unbounded.

**Corollary 3.2.** Under the assumptions (H.1), (C.2.1), (C.3.1), (C.4.2), (C.4.3), (C.6.) and (C.7.) (for $m=1$), we have

$$\sup_{\varphi K \in \mathcal{H}} \sup_{h_{n,1} \le h_K \le h_{n,2}} \sup_{t \in S_X} |\hat{r}_n^{(1)}(\varphi, t; h_K) - r^{(1)}(\varphi, t)| = O(h_{n,2}^{\gamma}) + O_{a,co} \left( \sqrt{\frac{\psi_{S_X}\left(\frac{\log n}{n}\right)}{n\phi(h_{n,1})}} \right). \quad (3.6)$$

$$\epsilon > 0, \sum_{n=1}^{\infty} \mathbb{P}(|z_n| > \epsilon) < \infty.$$

Moreover, we say that the rate of the almost-complete convergence of $(z_n)$ toward zero is of order $u_n$ (with $u_n \to 0$) and we write $z_n = O_{a,co.}(u_n)$ if, and only if, there exists $\epsilon > 0$ such that

$$\sum_{n=1}^{\infty} \mathbb{P}(|z_n| > \epsilon u_n) < \infty.$$

This kind of convergence implies both the almost-sure convergence and the convergence in probability.

<footnote>¶Let $(z_n)$ for $n \in \mathbb{N}$, be a sequence of real r.v.'s. We say that $(z_n)$ converges almost-completely (a.co.) toward zero if, and only if, for all</footnote>
### 3.1.2. UINN consistency for functional regression

Now, we can state the main results of this section concerning the k-NN functional regression. Recall the bandwidths $k_{1,n}$ and $k_{2,n}$ given in the condition (C.8).

**Theorem 3.3.** Under the assumptions (H.1), (C.2.1), (C.3.1), (C.4.1), (C.4.3), (C.6.) and (C.7.) (for $m = 1$), if, in addition, condition (C.8.) is satisfied, then, we have

$$ \sup_{\varphi K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} \left| \hat{r}_n^{(1)}(\varphi, t; h_{n,k}(t)) - r^{(1)}(\varphi, t) \right| = O \left( \phi^{-1} \left( \frac{k_{2,n}}{\rho_n n} \right)^\gamma \right) + O_{a.co} \left( \sqrt{ \frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)} } \right). $$

The following result gives uniform consistency when the class of functions is unbounded.

**Corollary 3.4.** Under the assumptions (H.1), (C.2.1), (C.3.1), (C.4.2), (C.4.3), (C.6.) and (C.7.) (for $m = 1$), and if condition (C.8.) is satisfied, then we have

$$ \sup_{\varphi K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} \left| \hat{r}_n^{(1)}(\varphi, t; h_{n,k}(t)) - r^{(1)}(\varphi, t) \right| = O \left( \phi^{-1} \left( \frac{k_{2,n}}{\rho_n n} \right)^\gamma \right) + O_{a.co} \left( \sqrt{ \frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)} } \right). \quad (3.7) $$

### 3.2. Relative-error prediction

Recall that the operator $m$ is usually estimated by minimizing the expected squared loss function $E[(Y - m(X))^2 | X]$. Nonetheless, this loss function, which is regarded as a measure of prediction performance, may be inappropriate in certain circumstances. In fact, the application of least-squares regression translates to giving all variables in the study equal weight. Consequently, the prevalence of outliers can render results irrelevant. In this paper, we, therefore, circumvent the limitations of classical regression by estimating the operator $m$ with regard to the minimization of the mean squared relative error (MSRE):

$$ E\left[\left(\frac{Y - m(X)}{Y}\right)^2 | X\right] \quad \text{for } Y > 0 \text{ a.s.} \quad (3.8) $$

This criterion is clearly a more meaningful measure of the prediction performance than the least square error, in particular, when the range of predicted values is large. Moreover, the solution of (3.8) can be explicitly expressed by the ratio of the first two conditional inverse moments of $Y$ given $X$. In fact, in order to construct the regression estimator allowing the best MSRE prediction, we assume that the first two conditional inverse moments of $Y$ given $X$, that is $g_\gamma(x) := E(Y^{-\gamma} | X = x)$ for $\gamma = 1, 2$, exist and are finite almost-surely (a.s.). Then, one can show easily, cf. [74, 123, 159], that the best mean squared relative error predictor of $Y$ given $X$ is:

$$ \check{r}(t) = E(Y^{-1} | X = t) / E(Y^{-2} | X = t) = g_1(t) / g_2(t), \quad \text{a.s.} $$
Thus, we estimate the regression operator $\check{r}(\cdot)$, which minimizes the MSRE by:

$$ \check{r}_n^{(1)}(t, h_K) = \frac{\sum_{i=1}^{n} Y_i^{-1} K \left( \frac{d(t, X_i)}{h_K} \right)}{\sum_{i=1}^{n} Y_i^{-2} K \left( \frac{d(t, X_i)}{h_K} \right)}, \quad \text{for each } t \in \mathcal{X}, \qquad (3.9) $$

and

$$ \check{r}_n^{*(1)}(t, h_K) = \frac{\sum_{i=1}^{n} Y_i^{-1} K \left( \frac{d(t, X_i)}{H_{n,k}(t)} \right)}{\sum_{i=1}^{n} Y_i^{-2} K \left( \frac{d(t, X_i)}{H_{n,k}(t)} \right)}, \quad \text{for each } t \in \mathcal{X}. \qquad (3.10) $$

By considering the special cases $\varphi(y) = y^{-1}$ and $\varphi(y) = y^{-2}$ in Corollaries 3.2 and 3.4, we obtain the following results complementing the work of [22, 23, 24, 74].

**Corollary 3.5.** Under the assumptions (H.1), (C.2.1), (C.3.1), (C.4.2), (C.4.3), (C.6.) and (C.7.) (for $m=1$), we have

$$ \sup_{K \in \mathcal{K}} \sup_{h_{n,1} \le h_K \le h_{n,2}} \sup_{t \in S_X} |\check{r}_n^{(1)}(t, h_K) - \check{r}(t)| = O(h_{n,2}^{\gamma}) + O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(h_{n,1})}} \right). \qquad (3.11) $$

The following result is not considered in the literature.

**Corollary 3.6.** Under the assumptions (H.1), (C.2.1), (C.3.1), (C.4.2), (C.4.3), (C.6.) and (C.7.) (for $m=1$), and if condition (C.8.) is satisfied, then we have

$$ \sup_{K \in \mathcal{K}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |\check{r}_n^{*(1)}(t, h_K) - \check{r}(t)| = O\left(\phi^{-1}\left(\frac{k_{2,n}}{\rho_n n}\right)^\gamma\right) + O_{a,co}\left(\sqrt{\frac{\psi_{S_X}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right)\right)}}\right). \qquad (3.12) $$

### 3.3. Uniform consistency of the kNN functional conditional U-statistics

In addition to the conditions imposed before, the following assumptions are essential to obtain exponential inequalities for dependent data later in the proofs:

(A1) Assume $\{X_i\}_{i \in \mathbb{N}^*}$ is strictly stationary, and there exists an absolute constant $\delta \ge 1$ such that for any $n \ge 1$, we have the $\beta$-mixing coefficient, corresponding to $\{X_i\}_{i \in \mathbb{N}^*}$, satisfies $\beta(n) \le n^{-\delta}$.
(A2) Assume, uniformly, for any integer $J$ such that $1 \le J \le m-1$ and arbitrary $1 \le i_1 < \dots < i_J \le n$, conditional on $X_{i_1}, \dots, X_{i_J}$, the sequence $\{X_i\}_{i=i_1+1}^\infty$ satisfies, for the $\alpha$-mixing coefficient corresponding to it,

$$ \alpha(n; X_{i_1}, \dots, X_{i_J}) := \sup_{j \ge i_j+1} \alpha \left( \sigma_{i_j+1}^j, \sigma_{j+n}^\infty; \mathbb{P}(\cdot | X_{i_1}, \dots, X_{i_J}) \right) \\ \lesssim \exp(-\gamma n), \text{ a.s.} $$

where $\mathbb{P}(\cdot | X_{i_1}, \dots, X_{i_J})$ stands for the conditional probability. In particular, we have, for the $\alpha$-mixing coefficient corresponding to $\{X_i\}_{i \in \mathbb{N}^*}$ itself

$$ \alpha(n) \lesssim \exp(-\gamma n). $$

The $\beta$-mixing condition (A1) is typically necessary to obtain asymptotic normality for $U$-statistics in the absence of a strict Lipchitz-continuity assumption for the kernel functions, for instance, see [72, 203] and Remarks 2.2 and 2.3 of [107]. Assumption (A2) is often less restrictive than the $\phi$-mixing condition. As will be shown in [107], finite-state and vector-valued absolutely continuous data sequences of exponentially $\phi$-mixing decaying rate satisfy (A2). We note that this is more restrictive than the polynomially mixing decaying rate. [203]. This is because we need to calculate higher moments of the $U$-statistics to obtain sharp concentration inequality. The “exponentially decaying rate” condition is routine in the literature of deriving concentration inequalities for weakly dependent data, see [149] and Remarks 2.4 and 2.5 in [107].

This section considers the uniform consistency, the UIB, and the UINN consistency of the functional conditional $U$-statistic given by (2.1). First, let's introduce some notation. For some interval $\mathcal{H}_n^{(m)} \subset \mathbb{R}_+^m \setminus \{0\}$, we denote

$$ \mathcal{H}_n^{(m)} := \prod_{j=1}^{m} (h_{n,j}, h'_{n,j}), $$

where

$$ 0 < h_{n,j} < h'_{n,j} \text{ and } \lim_{n \to \infty} h_{n,j} = \lim_{n \to \infty} h'_{n,j} = 0, \forall j = 1, \dots, m. $$

In the sequel, we denote (unless stated otherwise)

$$ \tilde{h}_n = \min_{1 \le j \le m} h_{n,j} \quad \text{and} \quad \tilde{h}'_n = \max_{1 \le j \le m} h'_{n,j}. $$

For all $\mathbf{b} = (b_1, \dots, b_m) \in (0, 1)^m$, let us denote

$$ \mathcal{H}_0^{(m)} := \prod_{j=1}^{m} (h_{n,j}, b_j), $$

and

$$ \tilde{b}_0 := \max_{1 \le j \le m} b_j. $$

For notational convenience, in the case of $m=1$, we denote $\mathcal{H}_n^{(1)}$ and $\mathcal{H}_0^{(1)}$ simply $\mathcal{H}_n$ and $\mathcal{H}_0$. The same goes for other similar notation unless stated otherwise. Set

$$ \mathbf{X} := (X_1, \dots, X_m) \in \mathcal{X}^m, \quad \mathbf{Y} := (Y_1, \dots, Y_m) \in \mathcal{Y}^m, $$
$$
\mathbf{X}_i := (X_{i_1}, \dots, X_{i_m}), \quad \mathbf{Y}_i := (Y_{i_1}, \dots, Y_{i_m}),
$$

$$
\mathbf{h}_{n,k}(\mathbf{t}) := (H_{n,k}(t_1), \dots, H_{n,k}(t_m)) \quad \text{for} \quad \mathbf{t} = (t_1, \dots, t_m) \in S_{\mathcal{X}}^m,
$$

$$
G_{\varphi, t, h}(\mathbf{x}, \mathbf{y}) := \frac{\varphi(y_1, \dots, y_m) \prod_{i=1}^{m} K\left(\frac{d(x_i, t_i)}{h_i(t_i)}\right)}{\prod_{i=1}^{m} \mathbb{E}\left[K\left(\frac{d(X_i, t_i)}{h_i(t_i)}\right)\right]} \quad \text{for } \mathbf{t}, \mathbf{x} \in \mathcal{X}^m, \mathbf{y} \in \mathcal{Y}^m;
$$

$$
\mathcal{G} := \{ G_{\varphi, t, h}(\cdot, \cdot) \varphi \in \mathcal{F}_m, \mathbf{t} = (t_1, \dots, t_m) \in \mathcal{X}^m \},
$$

$$
\mathcal{G}^{(p)} := \{ \pi_{p,m} G_{\varphi,t,h}(\cdot, \cdot), \varphi \in \mathcal{F}_m, \mathbf{t} = (t_1, \dots, t_m) \},
$$

$$
u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t})) = u_n^{(m)}(G_{\varphi, \mathbf{t}, \mathbf{h}_{n,k}}) := \frac{(n-m)!}{n!} \sum_{i \in I(m,n)} G_{\varphi, \mathbf{t}, \mathbf{h}_{n,k}}(\mathbf{X}_i, \mathbf{Y}_i),
$$

and for some symmetric measurable function $f(\cdot)$ define the $\mathbb{P}$-canonical function $\pi_{p,m}, p = 1, \dots, m,$
see [10] and [68] (we replace the index $k$ with $p$ to avoid confusing it with the smoothing parameter $k$),
by

$$
\pi_{p,m} f(t_1, \dots, t_p) := (\delta_{t_1} - \mathbb{P}) \cdots (\delta_{t_p} - \mathbb{P}) \mathbb{P}^{m-p} f,
$$

where for measures $Q_i$ on $S$ we let

$$
Q_1 \cdots Q_m h = \int_{S^m} h(x_1, \dots, x_m) dQ_1(x_1) \cdots dQ_m(x_m),
$$

and $\delta_x$ denote Dirac measure at point $x \in \mathcal{X}$. This decomposition follows easily by expanding

$$
f(\mathbf{x}_1, \dots, \mathbf{x}_m) = \delta_{\mathbf{x}_1} \times \dots \times \delta_{\mathbf{x}_m} f = ((\delta_{\mathbf{x}_1} - \mathbb{P}) + \mathbb{P}) \times \dots \times ((\delta_{\mathbf{x}_m} - \mathbb{P}) + \mathbb{P}) f,
$$

into terms of the form

$$
(\delta_{x_{i_1}} - \mathbb{P}) \times \cdots \times (\delta_{x_{i_m}} - \mathbb{P}) \times \mathbb{P}^{m-\ell} f.
$$

It is very simple to check that $f(\cdot)$ symmetric is $\mathbb{P}$-degenerate of order $r-1 \| $ if $r = \min \{\ell > 0 : \pi_\ell f \neq 0\}$.
For example,

$$
\pi_{1,m}h(x) = \mathbb{E}(h(X_1, \dots, X_m) | X_1 = x) - \mathbb{E}h(X_1, \dots, X_m).
$$

**Definition 3.7.** A $\mathbb{P}^m$-integrable symmetric function of $m$ variables, $f : S^m \to \mathbb{R}$, is $\mathbb{P}$-degenerate of order $r-1$, $1 < r \le m$, if
$$ \int f(\mathbf{x}_1, \dots, \mathbf{x}_m) d\mathbb{P}^{m-r+1}(\mathbf{x}_r, \dots, \mathbf{x}_{r-1}) = \int f d\mathbb{P}^m \text{ for all } \mathbf{x}_1, \dots, \mathbf{x}_{r-1} \in S \text{ whereas} $$

$$
\int f(\mathbf{x}_1, \dots, \mathbf{x}_m) d\mathbb{P}^{m-r}(\mathbf{x}_{r+1}, \dots, \mathbf{x}_m)
$$

is not a constant function. If $f$ is $\mathbb{P}^m$-centered and is $\mathbb{P}$-degenerate of order $m-1$, that is, if

$$
\int f(\mathbf{x}_1, \dots, \mathbf{x}_m) d\mathbb{P}(\mathbf{x}_1) = 0 \quad \text{for all } \mathbf{x}_2, \dots, \mathbf{x}_m \in S,
$$

then $f(\cdot)$ is said to be canonical or completely degenerate with respect to $\mathbb{P}$. If $f(\cdot)$ is not degenerate of any positive order, we say it is nondegenerate or degenerate of order zero.
It's clear that, for all $\varphi \in \mathcal{F}_m$:

$$ \widehat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t})) = \frac{u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t}))}{u_n(1, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t}))}, $$

and $u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t}))$ is a classical $U$-statistic with the $U$-kernel $G_{\varphi, \mathbf{t}, \mathbf{h}_{n,k}}(\mathbf{x}, \mathbf{y})$. However, the study of the uniform consistency of $\widehat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h})$ to $r^{(m)}(\varphi, \mathbf{t})$ can not be done with a straightforward approach due to the randomness of the bandwidth vector $\mathbf{h}_{n,k}(\mathbf{t})$ which poses some technical problems. To circumvent this, our strategy is first to study the uniform consistency of $\widehat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h})$, where $\mathbf{h} = (h_1, \dots, h_m) \in \mathcal{H}_n^{(m)}$ is a multivariate bandwidth that does not depend on $\mathbf{t}$ and $k$. Hence, we study the uniform consistency and the UIB consistency of $u_n(\varphi, \mathbf{t}, \mathbf{h})$ to $\mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}))$ when $\varphi \in \mathcal{F}_m$ and when $\varphi \equiv 1$, and we shall consider an appropriate centering factor than the expectation $\mathbb{E}(\widehat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}))$, hence we define:

$$ \widehat{\mathbb{E}}(\widehat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h})) = \frac{\mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}))}{\mathbb{E}(u_n(1, \mathbf{t}, \mathbf{h}))}. \quad (3.13) $$

The second step will be the use a general lemma given in [44], adapted to our setting, similar to that of [134] (see Subsection 8.1.1) to derive the results for the bandwidth $\mathbf{h}_{n,k}(\mathbf{t})$.

### 3.3.1. Uniform consistency and UIB consistency for a multivariate bandwidth

Next, we will give the UIB results for all $\mathbf{t} \in S_X^m$ and $\mathbf{h} \in \mathcal{H}_n^{(m)}$. We first start with announcing the result concerning the uniform derivation of the estimate $u_n(\varphi, \mathbf{t}, \mathbf{h})$ with respect to $\mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}))$ when the class of functions is bounded.

**Theorem 3.8.** Suppose that the conditions (H.1.), (C.3.1), (C.4.1), (C.4.4), (C.6.) and (C.7.) are fulfilled, we infer that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} |u_n(\varphi, \mathbf{t}, \mathbf{h}) - \mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}))| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right). \quad (3.14) $$

The following result covers the uniform derivation of the estimate $u_n(\varphi, \mathbf{t}, \mathbf{h})$ with respect to $\mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}))$ when the class of functions is unbounded satisfying general moments condition.

**Theorem 3.9.** Suppose that the conditions (H.1.), (C.3.1), (C.4.2), (C.4.4), (C.6.) and (C.7.) are fulfilled. For all $0 < \tilde{b}_0 < 1$, we infer that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m} \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} |u_n(\varphi, \mathbf{t}, \mathbf{h}) - \mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}))| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right). \quad (3.15) $$

The following result handles the uniform deviation of the estimate $\widehat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h})$ with respect to $\mathbb{E}(\widehat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}))$ in both situations, where the class of functions is bounded or unbounded satisfying a general moment condition.
**Theorem 3.10.** Suppose that the conditions (H.1.), (C.3.1), (C.4.1), (C.4.4), (C.6.) and (C.7.) (or the following (H.1.), (C.3.1), (C.4.2), (C.4.4), (C.6.) and (C.7.)) are fulfilled. For all $0 < \tilde{b}_0 < 1$, we infer, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}) - \mathbb{E} \left( \tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}) \right) \right| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right). \quad (3.16) $$

**Theorem 3.11.** Suppose that the conditions (H.1.), (C.2.1), (C.3.1) and (C.6.) are fulfilled. For all $0 < \tilde{h}'_n < 1$, $\tilde{h}'_n \to 0$, we infer, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \widehat{\mathbb{E}} \left( \tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}) - r^{(m)}(\varphi, \mathbf{t}) \right) \right| = O \left( \tilde{h}_n^{\gamma} \right). \quad (3.17) $$

**Corollary 3.12.** Under the assumptions of Theorems 3.10 and 3.11 it follows that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}) - r^{(m)}(\varphi, \mathbf{t}) \right| = O \left( \tilde{h}_n^{\gamma} \right) + O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right). \quad (3.18) $$

**Remark 3.13.** As in [9, 27, 43], we will divide the U-statistics into different parts: some parts can be approximated by U-statistics of independent blocks, while by conditioning on one block, others are empirical processes of independent blocks. To prove the nonlinear terms are negligible, we will also need some symmetrization and maximal inequalities, we refer to [10, 67].

### 3.4. Uniform consistency and UINN consistency of functional conditional U-statistics

Let $0 < \mu^* \le \nu^* < \infty$ be some constants and $\rho_n^* \in (0, 1)$ is a sequence chosen in such a way that

$$ \min_{1 \le j \le m} \mu_j \phi^{-1} \left( \frac{\rho_{n,j} k_{1,n}}{n} \right) = \mu^* \phi^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) $$

and

$$ \max_{1 \le j \le m} \nu_j \phi^{-1} \left( \frac{k_{2,n}}{\rho_{n,j} n} \right) = \nu^* \phi^{-1} \left( \frac{k_{2,n}}{\rho_n^* n} \right). $$

The following result deals with the uniform deviation of the estimate $u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t}))$ with respect to $\mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t})))$ when the class of functions is bounded.

**Theorem 3.14.** Suppose that the conditions (H.1.), (C.3.1), (C.4.1), (C.4.4), (C.6.) and (C.7.) are fulfilled. If in addition assumption (C.8.) holds, we infer that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_X^m} \left| u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t})) - \mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t}))) \right| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi \left( \mu^* \phi^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) \right)}} \right). \quad (3.19) $$
The following result deals with the uniform deviation of the estimate $u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t}))$ with respect to $\mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t})))$ when the class of functions is unbounded satisfying general moments condition.

**Theorem 3.15.** Suppose that the conditions (H.1.), (C.3.1), (C.4.1), (C.4.4), (C.6.) and (C.7.) are fulfilled. If in addition assumption (C.8.) holds, we infer that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_X^m} \left| u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t})) - \mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t}))) \right| \\ = O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu^* \phi^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) \right)}} \right). \quad (3.20) $$

The next results give uniform consistency when the class of functions is bounded or unbounded.

**Theorem 3.16.** Suppose that the conditions (H.1.), (C.3.1), (C.4.1), (C.4.4), (C.6.) and (C.7.) (or the following (H.1.), (C.3.1), (C.4.2), (C.4.4), (C.6.) and (C.7.)) are fulfilled. If in addition assumption (C.8.) holds, we infer that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_X^m} \left| \tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t})) - \hat{\mathbb{E}}(\tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t}))) \right| \\ = O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu^* \phi^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) \right)}} \right). \quad (3.21) $$

**Theorem 3.17.** Suppose that the conditions (H.1.), (C.2.1), (C.3.1) and (C.6.) are fulfilled. If in addition assumption (C.8.) holds, we infer that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_X^m} \left| \hat{\mathbb{E}}(\tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t}))) - r^{(m)}(\varphi, \mathbf{t}) \right| = O \left( \phi^{-1} \left( \frac{k_{2,n}}{\rho_n^* n} \right)^\gamma \right). \quad (3.22) $$

**Corollary 3.18.** Under the assumptions of Theorems 3.16 and 3.17 it follows that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_X^m} \left| \tilde{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t})) - r^{(m)}(\varphi, \mathbf{t}) \right| \\ = O \left( \phi^{-1} \left( \frac{k_{2,n}}{\rho_n^* n} \right)^\gamma \right) + O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu^* \phi^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) \right)}} \right). \quad (3.23) $$

**Remark 3.19.** The choice of the parameters $\mu^*$ and $\rho_n^*$ defined in a similar way as in condition (C.8.) affects the rate of convergence of the k-NN estimator. We can choose these parameters depending on the small ball probability function $\phi_t(h)$.
**Remark 3.20.** The present work largely extends and completes the work of [42, 44] in several ways. There are basically no restrictions on the choice of the kernel function in our setup, apart from satisfying some mild conditions that we will give after. The selection of the bandwidth or the number of neighbors, however, is more problematic. It is worth noticing that the choice of bandwidth is crucial to obtain a good rate of consistency, for example, it has a big influence on the size of the estimate's bias. In general, we are interested in the selection of bandwidth and neighbors that produce an estimator that has a good balance between the bias and the variance of the considered estimators. It is then more appropriate to consider the bandwidth and neighbors varying according to the criteria applied and to the available data and location which cannot be achieved by using the classical methods. The interested reader may refer to [34, 146] for more details and discussion on the subject. In the present section, we have provided a response to this delicate problem in the FDA associated with the dependent data setting. In the present setting, we divide the U-statistics into different parts: some parts can be approximated by U-statistics of independent blocks, while by conditioning on one block, others are empirical processes of independent blocks. This decomposition is the key tool but makes the proof very involved which is the price of the extension to the dependent framework. This, allows us to use, in a nontrivial way, the techniques used for independent variables and mostly we will be using the results of [44]. We highlight that in the present paper, we have used a novel exponential inequality of [107] tailored to the dependent framework.

# 4. Uniform central limit theorems

## 4.1. kNN conditional empirical process

We define the functional conditional empirical process for univariate bandwidth $h_K$ by:

$$ \{v_n(\psi; h_K | t) = \sqrt{k}(\hat{r}_n^{(1)}(\psi, t; h_K) - r^{(1)}(\psi, t)), \psi \in \mathcal{F}\mathcal{K}\}, \quad (4.1) $$

where $\hat{r}_n^{(1)}(\psi, t; h_K)$ designates (2.5) when $m = 1$, and $r^{(1)}(\psi, t)$ refers to the regression function (2.2), with

$$ \psi(\cdot, \cdot) \in \mathcal{F}\mathcal{K} := \mathcal{F}_1\mathcal{K}^1 = \left\{ \varphi(\cdot)K\left(\frac{d(\cdot, t)}{h_K}\right) : \varphi \in \mathcal{F}_1, K\left(\frac{d(\cdot, t)}{h_K}\right) \in \mathcal{K}^1 \right\}. $$

If, for $\mathbb{P}\psi = \int \psi d\mathbb{P}$, where $\mathbb{P}$ is the probability measure and, for each $(\mathbf{x}, \mathbf{y})$,

$$ \sup_{\psi \in \mathcal{F}\mathcal{K}} |\psi(\mathbf{x}, \mathbf{y}) - \mathbb{P}\psi| < \infty, $$

then $\{v_n(\psi | t) : \psi \in \mathcal{F}\mathcal{K}\}$ is a random element with values in $l_\infty(\mathcal{F}\mathcal{K})$, consisting of all functional $v_\infty$ on $\mathcal{F}\mathcal{K}$ such that

$$ \sup_{\psi \in \mathcal{F}\mathcal{K}} |v_\infty(\psi)| < \infty. $$

Then, it will be important to investigate the following weak convergence

$$ \{v_n(\psi | t) : \psi \in \mathcal{F}\mathcal{K}, t \in \mathbb{I}\} \xrightarrow{w} \{\mathbb{G}(\psi) : \psi \in \mathcal{F}\mathcal{K}\} \text{ in } l_\infty(\mathcal{F}\mathcal{K}). $$

It is known that the weak convergence to a Gaussian limit with a version of uniformly bounded and uniformly continuous paths (with respect to the $||\cdot||_2$) is equivalent to the finite-dimensional convergence
and the existence of pseudo-metric $d_{p.m}$ on $\mathcal{F}\mathcal{H}$ such that $(\mathcal{F}\mathcal{H}, d_{p.m})$ is totally bounded pseudo-metric space and

$$ \lim_{r \to 0} \limsup_{n \to \infty} \mathbb{P}^* \left\{ \sup_{d_{p.m}(\psi_1, \psi_2) \le r} |v_n((\psi_1 - \psi_2) |t)| > \varepsilon \right\} = 0. \quad (4.2) $$

Below, we write $Z \stackrel{d}{=} \mathcal{N}(\mu, \Sigma^2)$ whenever the random vector $Z$ follows a normal law with vector expectation $\mu$ and matrix variance $\Sigma^2$, $\stackrel{d}{\to}$ denotes the convergence in distribution. The following theorem is adapted from [147] to the setting of the $k$-NN estimators. The main objective of this section is to investigate the central limit theorems for the functional conditional empirical process defined by

$$ \{v_n(\psi; H_{n,k}(t)) | t\} = \sqrt{k} (\hat{r}_n^{(1)}(\psi, t; H_{n,k}(t)) - r^{(1)}(\psi, t)), \psi \in \mathcal{F}\mathcal{H} \quad (4.3) $$

**Theorem 4.1.** Let consider the class of functions $\mathcal{F}\mathcal{H}$, suppose that conditions (C.1'), (C.1.2), (C.2.1), (C.2.2), (C.3'), (C.5.) and (C.8.) hold. and if the smoothing parameter $k$ satisfies for all $\gamma > 0$

$$ k \left( \phi^{-1} \left( \frac{k}{n} \right) \right)^{2\gamma} \to 0 \text{ as } n \to \infty, $$

we get for: $l \ge 1: \psi_1, \dots, \psi_l \in \mathcal{F}\mathcal{H}$,

$$ \{v_n(\psi_i; H_{n,k}(t)) | t\} : i = 1, \dots, l \stackrel{d}{\to} \mathcal{N}(0, \Sigma), $$

where $\Sigma := (\sigma_{i,j})_{i,j=1,\dots,l}$ is the covariance matrix with:

$$ \sigma_{i,j} := \frac{\kappa_2' r^{(1)}(\varphi_i \varphi_j, t) - r^{(1)}(\varphi_i, t) r^{(1)}(\varphi_j, t)}{\kappa_1' f_1(t)}. $$

**Theorem 4.2.** Suppose that the conditions (C.3.1), (C.4.2)–(C.5.1), and (C.8.) hold and for each $\varphi \in \mathcal{F}$,

$$ \mathbb{E}(\varphi^2(Y_1)) < \infty. $$

Then, we have

$$ \lim_{b \to 0} \limsup_{n \to \infty} \mathbb{P} \left\{ \sup_{\substack{\|\psi_1 - \psi_2\|_p \le b \\ \psi_1, \psi_2 \in \mathcal{F}\mathcal{H}}} |v_n((\psi_1 - \psi_2) |t)| > \varepsilon \right\} = 0. $$

The two previous theorems can be summarized as follows:

**Theorem 4.3.** Under conditions (C.1'), (C.1.2), (C.2.), (C.3.1), (C.3'), (C.4.4) (C.5.1), (C.5.2) and (C.8.), then the process, as $n \to \infty$,

$$ \{v_n(\psi; H_{n,k}(t)) | t\} = \sqrt{k} (\hat{r}_n^{(1)}(\psi, t; H_{n,k}(t)) - r^{(1)}(\psi, t)) : \psi \in \mathcal{F}\mathcal{H}, $$

converges in law to a Gaussian process $\{G_n(\psi) : \psi \in \mathcal{F}\mathcal{H}\}$ that admits a version with uniformly bounded and uniformly continuous paths with respect to $\|\cdot\|_2$-norm.

**Remark 4.4.** We mention that other types of applications can be obtained from Theorem 4.3 including the conditional distribution, conditional density, and conditional hazard function. This, and other applications of interest, will not be considered here due to lack of space.
## 4.2. kNN conditional U-processes

In this section, we are interested in studying the weak convergence of conditional U-processes under absolute regular observations. Recall the class of functions considered is $\mathcal{F}_m\mathcal{H}^m$ given in Section 2. The conditional U-process indexed by $\mathcal{F}_m\mathcal{H}^m$:

$$ \left\{ U_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) := \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} (\tilde{r}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) - r^{(m)}(\varphi, \mathbf{t})) \right\}_{\mathcal{F}_m\mathcal{H}^m}, \quad (4.4) $$

The U-empirical process is defined by

$$ \mu_n(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) := \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \{u_n(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) - \mathbb{E}(u_n(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})))\}. $$

It should be noted that to establish the weak convergence of (4.4) it is first necessary to go through that of (4.6), below. Indeed, we will develop some details that will be used later. Because condition (C.6.) is satisfied, for each $\lambda > 0$, we have

$$ \begin{aligned} G_{\varphi, \mathbf{t}, \mathbf{h}}(\mathbf{x}, \mathbf{y}) &= G_{\varphi, \mathbf{t}, \mathbf{h}}(\mathbf{x}, \mathbf{y}) \mathbb{1}_{\{\kappa^m F(y) \le \lambda(n/\tilde{\phi}(\mathbf{h}))^{1/2(p-1)}\}} \\ &\quad + G_{\varphi, \mathbf{t}, \mathbf{h}}(\mathbf{x}, \mathbf{y}) \mathbb{1}_{\{\kappa^m F(y) > \lambda(n/\tilde{\phi}(\mathbf{h}))^{1/2(p-1)}\}} \\ &=: G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}(\mathbf{x}, \mathbf{y}) + G_{\varphi, \mathbf{t}, \mathbf{h}}^{(R)}(\mathbf{x}, \mathbf{y}). \end{aligned} \quad (4.5) $$

We can write the U-statistic as follows

$$ \begin{aligned} \mu_n(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) &= \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \{u_n^{(m)}(G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) - \mathbb{E}(u_n^{(m)}(G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}))\} \\ &\quad + \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \{u_n^{(m)}(G_{\varphi, \mathbf{t}, \mathbf{h}}^{(R)}) - \mathbb{E}(u_n^{(m)}(G_{\varphi, \mathbf{t}, \mathbf{h}}^{(R)}))\} \\ &=: \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \{u_n^{(T)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t})) - \mathbb{E}(u_n^{(T)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t})))\} \\ &\quad + \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \{u_n^{(R)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t})) - \mathbb{E}(u_n^{(R)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t})))\} \\ &=: \mu_n^{(T)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) + \mu_n^{(R)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})). \end{aligned} \quad (4.6) $$

We call the first term of the right side of (4.6) $\mu_n^{(T)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t}))$ truncated part and the second $\mu_n^{(R)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t}))$ remainder part. First we are interested in $\mu_n^{(T)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t}))$. An application of Hoeffding's decomposition gives

$$ \begin{aligned} u_n^{(T)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t})) &= \sum_{p=0}^{m} \frac{m!}{(m-p)!} u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) \\ &= \mathbb{E} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}(\mathbf{X}', \mathbf{Y}') + \sum_{p=1}^{m} \frac{m!}{(m-p)!} u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}), \end{aligned} \quad (4.7) $$

where $\{(\mathbf{X}_i', \mathbf{Y}_i')\}_{i \in \mathbb{N}}$ is a sequence of i.i.d. r.v. with $\mathcal{L}(\mathbf{X}_i', \mathbf{Y}_i') = \mathcal{L}(\mathbf{X}_i, \mathbf{Y}_i)$ for each $i$, and $\mathbf{X}'$ and $\mathbf{Y}'$ are respectively defined as $\mathbf{X}$ and $\mathbf{Y}$. In view of (4.7), we have

$$ \mu_n^{(T)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) = \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \left\{ \mathbb{E} G_{\varphi, t}^{(T)}(\mathbf{X}', \mathbf{Y}') + \sum_{p=1}^{m} \frac{m!}{(m-p)!} u_n^{(p)}(\pi_{p,m} G_{\varphi, t, \mathbf{h}}^{(T)}) - \mathbb{E}(u_n^{(T)}(\varphi, \mathbf{t})) \right\}, $$
the stationarity assumption and some algebras show that

$$ \mathbb{E}(u_n^{(T)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t}))) = \mathbb{E}G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}(\mathbf{X}', \mathbf{Y}'). $$

Therefore,

$$ \begin{aligned} \mu_n^{(T)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) &= \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \left\{ \sum_{p=1}^{m} \frac{m!}{(m-p)!} u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) \right\} \\ &= \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \left\{ m u_n^{(1)}(\pi_{1,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) + \sum_{p=2}^{m} \frac{m!}{(m-p)!} u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) \right\}. \end{aligned} \quad (4.8) $$

By the fact that $\pi_{p,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}$ is $\mathbb{P}$-canonical, we have to show that

$$ \left( \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \sum_{p=2}^{m} u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) \right) \xrightarrow{\mathbb{P}} 0. $$

So that to establish the weak convergence of the $U$-process $\{\mu_n^{(T)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t}))\}_{\mathcal{F}_m, \mathcal{H}_m}$, it is enough to show

$$ m \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} u_n^{(1)}(\pi_{1,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) \xrightarrow{w} \mathbb{G}(\varphi) \text{ in } \ell_\infty(m\mathcal{G}^{(1)}), $$

where $\{\mathbb{G}(\varphi)\}_{m\mathcal{G}^{(1)}}$ is a Gaussian process indexed by $m\mathcal{G}^{(1)}$, and for $2 \le p \le m$

$$ \left\| \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}, \mathbf{h}}^{(T)}) \right\|_{\mathcal{F}_m, \mathcal{H}_m} \xrightarrow{\mathbb{P}} 0. $$

We have to prove after, that the remaining part is negligible, in the sense that

$$ \left\| \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} \{u_n^{(R)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t})) - \mathbb{E}(u_n^{(R)}(\varphi, \mathbf{t}, \mathbf{h}(\mathbf{t})))\} \right\|_{\mathcal{F}_m, \mathcal{H}_m} \xrightarrow{\mathbb{P}} 0. $$

Nevertheless, when we have to deal with finite-dimensional convergence, the truncation does not matter. Which means that establishing the finite-dimensional convergence of $\mu_n(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t}))$ is equivalent to establishing that of $\mu_n^{(T)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t}))$.

**Theorem 4.5. (a) Under conditions (C.1'.), (C.1.2), (C.2.), (C.3'.), (C.5.1), (C.5.2) and if $r^{(m)}(\varphi, \mathbf{t})$ is continuous at $\mathbf{t}$, then, as $n \to \infty$,

$$ \sqrt{n\tilde{\phi}(\mathbf{h}(\mathbf{t}))} (\hat{r}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})) - \mathbb{E}(u_n(\varphi, \mathbf{t}; \mathbf{h}(\mathbf{t})))) \xrightarrow{d} \mathcal{N}(0, \rho^2), \quad (4.9) $$

where

$$ \rho^2 := m^2 \left( \sigma_t^2(\varphi, \varphi) - 2r^{(m)}(\varphi, \mathbf{t})\sigma_t^2(\varphi, 1) + (r^{(m)}(\varphi, \mathbf{t}))^2 \sigma_t^2(1, 1) \right), \quad (4.10) $$

$$ \sigma_t^2(\varphi_i, \varphi_j) := \lim_{n \to \infty} \tilde{\phi}(\mathbf{h}) \mathbb{E}(\pi_{1,m} G_{\varphi_i, \mathbf{t}, \mathbf{h}}(\mathbf{t}), \pi_{1,m} G_{\varphi_j, \mathbf{t}, \mathbf{h}}(\mathbf{t})). $$
(b) If, in addition, the smoothing parameter k satisfies the condition (C.8), then we have, as $n \to \infty$,

$$ \sqrt{n^{-m+1}k^m} (\hat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t})) - \mathbb{E}(u_n(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t}))) \xrightarrow{d} \mathcal{N}(0, \rho^2), \quad (4.11) $$

where $\rho^2$ is defined as in (4.10), with

$$ \sigma_{\mathbf{t}}^2(\varphi_i, \varphi_j) := \lim_{n \to \infty} \left(\frac{k}{n}\right)^m \mathbb{E}\left(\pi_{1,m} G_{\varphi_i, \mathbf{t}, \mathbf{h}}, \pi_{1,m} G_{\varphi_j, \mathbf{t}, \mathbf{h}}\right). $$

**Corollary 4.6.** Under conditions (C.1'), (C.1.2), (C.2.), (C.3'), (C.5.) and if $n\tilde{h}_n^{2\gamma}\tilde{\phi}(\mathbf{h}(\mathbf{t})) \to 0$ as $n \to \infty$, then we infer that:

$$ \sqrt{n^{-m+1}k^m} \{\hat{r}_n^{*(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t})) - r^{(m)}(\varphi, \mathbf{t})\} \xrightarrow{d} \mathcal{N}(0, \rho^2). \quad (4.12) $$

**Theorem 4.7.** Under conditions (C.1'), (C.1.2), (C.2.), (C.3'), (C.5.1), (C.5.2), (C.8.) and $n\tilde{h}_n^{2\gamma}\tilde{\phi}(\mathbf{h}(\mathbf{t})) \to 0$ as $n \to \infty$. Let $\mathcal{F}_m, \mathcal{H}^m$ be a measurable VC-subgraph class of functions from $(\mathcal{X}^m, \mathcal{Y}^m) \to \mathbb{R}$ such as condition (C.4.2) is satisfied and, if the $\beta$-coefficients of the mixing stationary sequence $\{(X_i, Y_i)\}_{i \in \mathbb{N}^*}$ fulfill:

$$ \beta_s s^r \to 0, \text{ as } s \to \infty, \quad (4.13) $$

for some $r > 1$, then $\{U_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t}))\}_{\mathcal{F}_m, \mathcal{H}^m}$ converges in law to a Gaussian process $\{\mathbb{G}(\varphi)\}_{\mathcal{F}_m, \mathcal{H}^m}$ which has a version with uniformly bounded and uniformly continuous paths with respect to $\|\cdot\|_2$-norm.

**Remark 4.8.** It is worth noting what is the price to pay by the nice features of the k-NN-based estimators: remembering that $H_{n,k}(X)$ is a random variable (which depends on $(X_1, \dots, X_n)$, one should expect that additional technical difficulties will appear along the proofs of asymptotic properties. To fix the idea on this point, note that the random elements involved in (2.1), $m = 1$, can not be decomposed as sums of independent variables (as it is the case for instance with kernel-based estimators), and hence its treatment will need more sophisticated probabilistic developments than standard limit theorems for sums of i.i.d. variables. Also, the Hoeffding decomposition can not be applied directly to (2.1), which is the main tool for the study U-statistics. The first step in proving Theorem 4.7 is the extension of [27, 43] to the multivariate bandwidth problem. In addition, we have considered new applications: the set indexed conditional U-statistic, Kendall rank correlation coefficient and time series prediction from a continuous set of past values. Another delicate problem lies in the fact that some maximal inequalities and symmetrisation techniques of [10, 67] are not applicable directly in our framework, making the proof quit lengthy, in particular, the equicontinuity of the empirical processes.

**Remark 4.9.** It is straightforward to modify the proofs of our results to show that it remains true when the entropy condition is substituted by the bracketing condition: For some $C_0 > 0$ and $v_0 > 0$,

$$ N_{[1]}(\epsilon, \mathcal{F}_m \mathcal{H}^m, L_2(\mathbb{P})) \le C_0 \epsilon^{-v_0}, \quad 0 < \epsilon < 1. $$

Refer to p. 270 of [190] for the definition of $N_{[1]}(\epsilon, \mathcal{F}_m \mathcal{H}^m, L_2(\mathbb{P}))$.
# 5. Some potential applications

Although only four examples will be given here, they stand as archetypes for a variety of problems that can be investigated in a similar way.

## 5.1. Set indexed conditional U-statistics

We aim to study the links between $X$ and $Y$, by estimating functional operators associated to the conditional distribution of $Y$ given $X$ such as the regression operator, for $C_1 \times \cdots \times C_m := \tilde{\mathbb{C}}$ in a is a class of sets $\mathcal{C}^m$,

$$ \mathbb{G}^{(m)}(C_1 \times \cdots \times C_m | \mathbf{t}) = \mathbb{E} \left( \prod_{i=1}^{m} 1_{\{Y_i \in C_i\}} | (X_1, \dots, X_m) = \mathbf{t} \right) \text{ for } \mathbf{t} \in \mathcal{X}^m. $$

We define metric entropy with the inclusion of the class of sets $\mathcal{C}$. For each $\varepsilon > 0$, the covering number is defined as :

$$ \mathcal{N}(\varepsilon, \mathcal{C}, \mathbb{G}^{(1)}(\cdot | x)) = \inf\{n \in \mathbb{N} : \exists C_1, \dots, C_n \in \mathcal{C} \text{ such that } \forall C \in \mathcal{C} \exists 1 \le i, j \le n \\ \text{with } C_i \subset C \subset C_j \text{ and } \mathbb{G}^{(1)}(C_j \setminus C_i | x) < \varepsilon\}, $$

the quantity $\log(\mathcal{N}(\varepsilon, \mathcal{C}, \mathbb{G}^{(1)}(\cdot | x)))$ is called metric entropy with inclusion of $\mathcal{C}$ with respect to $\mathbb{G}^{(1)}(\cdot | x)$. The quantity $\log \mathcal{N}(\varepsilon, \mathcal{C}, \mathbb{G}^{(1)}(\cdot | x))$ is called metric entropy with inclusion of $\mathcal{C}$ with respect to $\mathbb{G}(\cdot | x)$. Estimates for such covering numbers are known for many classes, (see, e.g., [81]). We will often assume below that either $\log \mathcal{N}(\varepsilon, \mathcal{C}, \mathbb{G}^{(1)}(\cdot | x))$ or $\mathcal{N}(\varepsilon, \mathcal{C}, \mathbb{G}^{(1)}(\cdot | x))$ behave like powers of $\varepsilon^{-1}$: we say that the condition $(R_\gamma)$ holds if

$$ \log \mathcal{N}(\varepsilon, \mathcal{C}, \mathbb{G}^{(1)}(\cdot | x)) \le H_{\gamma}(\varepsilon), \text{ for all } \varepsilon > 0, \quad (5.1) $$

where

$$ H_{\gamma}(\varepsilon) = \begin{cases} \log(A\varepsilon) & \text{if } \gamma = 0, \\ A\varepsilon^{-\gamma} & \text{if } \gamma > 0, \end{cases} $$

for some constants $A, r > 0$. As in [164], it is worth noticing that the condition (5.1), $\gamma = 0$, holds for intervals, rectangles, balls, ellipsoids, and for classes which are constructed from the above by performing set operations union, intersection and complement finitely many times. The classes of convex sets in $\mathbb{R}^d$ ($d \ge 2$) fulfill the condition (5.1), $\gamma = (d - 1)/2$. This and other classes of sets satisfying (5.1) with $\gamma > 0$ can be found in [81]. As a particular case of (2.5), we estimate $\mathbb{G}^{(m)}(C_1 \times \cdots \times C_m | \mathbf{t})$

$$ \widehat{\mathbb{G}}_{n}^{(m)}(\tilde{\mathbb{C}}, \mathbf{t}) = \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} \prod_{j=1}^{m} 1_{\{Y_{i_j} \in C_j\}} K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}. \quad (5.2) $$

One can apply Corollary 3.18 to infer that

$$ \sup_{\tilde{\mathbb{C}} \times \tilde{\mathbb{K}} \in \mathcal{C}^m, \mathcal{H}^m} \sup_{k_1, \dots, k_{2,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_{\mathcal{X}}^m} \left| \widehat{\mathbb{G}}_n^{(m)}(\tilde{\mathbb{C}}, \mathbf{t}) - \mathbb{G}^{(m)}(\tilde{\mathbb{C}} | \mathbf{t}) \right| \to 0 \quad \text{a.co.} \quad (5.3) $$
**Remark 5.1.** Another point of view is to consider the following situation, for a compact $\mathbf{J} \subset \mathbb{R}^{dm}$,

$$ \mathbf{G}^{(m)}(y_1, \dots, y_m | \mathbf{t}) = \mathbb{E} \left( \left( \prod_{i=1}^{m} 1_{\{Y_i \le y_i\}} \mid (X_1, \dots, X_m) = \mathbf{t} \right) \right) \quad \text{for } \mathbf{t} \in \mathcal{X}^m, (y_1, \dots, y_m) \in \mathbf{J}. $$

Let $L(\cdot)$ be a distribution in $\mathbb{R}^d$ and $\tilde{H}_{n,k}(t_i)$ is the number of neighborhoods associated with $Y_i$s. One can estimate $\mathbf{G}^{(m)}(y_1, \dots, y_m | \mathbf{t}) = \mathbf{G}^{(m)}(\mathbf{y} | \mathbf{t})$ by

$$ \widehat{\mathbf{G}}_{n}^{(m)}(\mathbf{y}, \mathbf{t}) := \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} L\left(\frac{t_1 - Y_{i_1}}{\tilde{H}_{n,k}(t_1)}\right) \cdots L\left(\frac{t_m - Y_{i_m}}{\tilde{H}_{n,k}(t_m)}\right) K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}. $$

One can use Corollary 3.18 to infer that, as $n \to \infty$,

$$ \sup_{\tilde{K} \in \mathcal{H}^m} \sup_{k_1, n \le k, k \le k_2, n} \sup_{\mathbf{t} \in S_X^m} \sup_{\mathbf{y} \in \mathbf{J}} \left| \widehat{\mathbf{G}}_n^{(m)}(\mathbf{y}, \mathbf{t}) - \mathbf{G}^{(m)}(\mathbf{y} | \mathbf{t}) \right| \to 0 \quad \text{a.co.} \quad (5.4) $$

## 5.2. Kendall rank correlation coefficient

To test the independence of one-dimensional random variables $Y_1$ and $Y_2$ [127] proposed a method based on the U-statistic $K_n$ with the kernel function :

$$ \varphi((s_1, t_1), (s_2, t_2)) = 1_{\{(s_2-s_1)(t_2-t_1)>0\}} - 1_{\{(s_2-s_1)(t_2-t_1)\le0\}}. \quad (5.5) $$

Its rejection on the region is of the form $\{\sqrt{n}K_n > \gamma\}$, for more general tests, refer [29, 32]. In this example, we consider a multivariate case. To test the conditional independence of $\xi, \eta : Y = (\xi, \eta)$ given $X$, we propose a method based on the conditional U-statistic :

$$ \widehat{r}_n^{(2)}(\varphi, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t})) = \frac{\sum_{i \neq j}^{n} \varphi(Y_i, Y_j) K\left(\frac{d(t_1, X_i)}{H_{n,k}(t_1)}\right) K\left(\frac{d(t_2, X_j)}{H_{n,k}(t_2)}\right)}{\sum_{i \neq j}^{n} K\left(\frac{d(t_1, X_i)}{H_{n,k}(t_1)}\right) K\left(\frac{d(t_2, X_j)}{H_{n,k}(t_2)}\right)}, $$

where $\mathbf{t} = (t_1, t_2) \in \mathbb{I} \subset \mathbb{R}^2$ and $\varphi(\cdot)$ is Kendall's kernel (5.5). Suppose that $\xi$ and $\eta$ are $d_1$ and $d_2$-dimensional random vectors respectively and $d_1 + d_2 = d$. Furthermore, suppose that $Y_1, \dots, Y_n$ are observations of $(\xi, \eta)$, we are interested in testing :

$$ H_0 : \xi \text{ and } \eta \text{ are conditionally independent given } X. \text{ vs } H_a : H_0 \text{ is not true.} \quad (5.6) $$

Let $\mathbf{a} = (\mathbf{a}_1, \mathbf{a}_2) \in \mathbb{R}^d$ such as $\|\mathbf{a}\| = 1$ and $\mathbf{a}_1 \in \mathbb{R}^{d_1}, \mathbf{a}_2 \in \mathbb{R}^{d_2}$, and $F(\cdot), G(\cdot)$ be the distribution functions of $\xi$ and $\eta$ respectively. Suppose $F^{a_1}(\cdot)$ and $G^{a_2}(\cdot)$ to be continuous for any unit vector $\mathbf{a} = (\mathbf{a}_1, \mathbf{a}_2)$ where
$F^{a_1}(t) = \mathbb{P}(\mathbf{a}_1^\top \boldsymbol{\xi} < t)$ and $G^{a_2}(t) = \mathbb{P}(\mathbf{a}_2^\top \boldsymbol{\eta} < t)$ and $\mathbf{a}_1^\top$ means the transpose of the vector $\mathbf{a}_i$, $1 \le i \le 2$. For $n=2$, let $Y^{(1)} = (\boldsymbol{\xi}^{(1)}, \boldsymbol{\eta}^{(1)})$ and $Y^{(2)} = (\boldsymbol{\xi}^{(2)}, \boldsymbol{\eta}^{(2)})$ such as $\boldsymbol{\xi}^{(i)} \in \mathbb{R}^{d_1}$ and $\boldsymbol{\eta}^{(i)} \in \mathbb{R}^{d_2}$ for $i=1,2$, and :

$$ \varphi^a (Y^{(1)}, Y^{(2)}) = \varphi ((\mathbf{a}_1^\top \boldsymbol{\xi}^{(1)}, \mathbf{a}_2^\top \boldsymbol{\eta}^{(1)}), (\mathbf{a}_1^\top \boldsymbol{\xi}^{(2)}, \mathbf{a}_2^\top \boldsymbol{\eta}^{(2)})). $$

An application of Corollary 3.18 gives, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_2, \mathcal{X}^2} \sup_{k_1,n \le k \le k_2,n} \sup_{\mathbf{t} \in S_X^2} |\tilde{r}_n^{*(2)}(\varphi^a, \mathbf{t}; \mathbf{h}_{n,k}(\mathbf{t})) - r^{(2)}(\varphi^a, \mathbf{t})| \to 0, \quad \text{a.co.} \quad (5.7) $$

### 5.3. Discrimination problems

Now, we apply the results to the problem of discrimination described in Section 3 of [185], refer to also to [184]. We will use a similar notation and setting. Let $\varphi(\cdot)$ be any function taking at most finitely many values, say $1, \dots, M$. The sets

$$ A_j = \{(\mathbf{y}_1, \dots, \mathbf{y}_k) : \varphi(\mathbf{y}_1, \dots, \mathbf{y}_k) = j\}, \quad 1 \le j \le M, $$

then yield a partition of the feature space. Predicting the value of $\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k)$ is tantamount to predicting the set in the partition to which $(\mathbf{Y}_1, \dots, \mathbf{Y}_k)$ belongs. For any discrimination rule $g$, we have

$$ \mathbb{P}(g(\mathbf{X}) = \varphi(\mathbf{Y})) \le \sum_{j=1}^{M} \int_{\mathbf{t}:g(\mathbf{t})=j} \max \mathfrak{M}^j(\mathbf{t}) d\mathbb{P}(\mathbf{t}), $$

where

$$ \mathfrak{M}^j(\mathbf{t}) = \mathbb{P}(\varphi(\mathbf{Y}) = j | \mathbf{X} = \mathbf{t}), \quad \mathbf{t} \in \mathcal{X}^m. $$

The above inequality becomes equality if

$$ g_0(\mathbf{t}) = \arg \max_{1 \le j \le M} \mathfrak{M}^j(\mathbf{t}). $$

$g_0(\cdot)$ is called the Bayes rule, and the pertaining probability of error

$$ \mathbf{L}^* = 1 - \mathbb{P}(g_0(\mathbf{X}) = \varphi(\mathbf{Y})) = 1 - \mathbb{E} \left\{ \max_{1 \le j \le M} \mathfrak{M}^j(\mathbf{t}) \right\}, $$

is called the Bayes risk. Each of the above unknown functions $\mathfrak{M}^j$'s can be consistently estimated by one of the methods discussed in the preceding sections. Let, for $1 \le j \le M$,

$$ \mathfrak{M}_n^j(\mathbf{t}) = \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} 1\{\varphi(Y_{i_1}, \dots, Y_{i_m}) = j\} K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}. \quad (5.8) $$

Set

$$ g_{0,n}(\mathbf{t}) = \arg \max_{1 \le j \le M} \mathfrak{M}_n^j(\mathbf{t}). $$
Let us introduce

$$L_n^* = \mathbb{P}(g_{0,n}(\mathbf{X}) \neq \varphi(\mathbf{Y})).$$

The discrimination rule $g_{0,n}(\cdot)$ is asymptotically Bayes' risk consistent, as $n \to \infty$,

$$L_n^* \to L^*.$$

This follows from Corollary 3.18 and the obvious relation

$$|L^* - L_n^*| \le 2\mathbb{E}\left[\max_{1 \le j \le M} \left|\mathfrak{M}_n^j(\mathbf{X}) - \mathfrak{M}^j(\mathbf{X})\right|\right].$$

## 5.4. Time series prediction from a continuous set of past values

Let $\{Z_n(t), t \in \mathbb{R}\}_{n \ge 1}$ denote a sequence of processes with value in $\mathbb{R}$. Let $s$ denote a fixed positive real number. In this model, we suppose that the process is observed from $t=0$ until $t=t_{\max}$, and assume without loss of generality that $t_{\max} = nT + s < \tau$. The method ensures splitting the observed process into $n$ fixed-length segments. Let us denote each piece of the process by

$$X_i = \{Z(t), (i-1)T \le t < iT\}.$$

The response value is therefore $Y_i = Z(iT + s)$, and this can be formulated as a regression problem:

$$\varphi(Z_1(\tau + s), \dots, Z_k(\tau + s)) = r^{(k)}(Z_1(t), \dots, Z_k(t)), \quad \text{for } \tau - T \le t < \tau. \qquad (5.9)$$

provided that we assume that a function of this kind, $r$, does not depend on $i$ (which is satisfied if the process is stationary, for example). Because of this, when we get to time $\tau$, we can use the following predictor, which is directly derived from our estimator, to predict the value that will be at time $\tau + s$

$$\hat{r}_n^{(k)}(\varphi, \mathbf{z}; m_n) = \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} \varphi(Z_{i_1}(\tau+s), \dots, Z_{i_m}(\tau+s)) K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K\left(\frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)}\right)},$$

where $\mathbf{z} = (z_1, \dots, z_k) = \{(Z_1(t), \dots, Z_k(t)), \tau - T \le t < \tau\}$. Corollary 3.12 provides mathematical support for this nonparametric functional predictor and extends previous results in numerous ways in [56, 91]. Notice that this modelization encompasses a wide variety of practical applications, as this procedure allows for the consideration of a large number of past process values without being affected by the curse of dimensionality. We believe that our findings will find applications beyond the scope of this work, in particular, because many popular measures of dependence, such as distance covariance and the Hilbert-Schmidt independence criterion can be estimated using U-statistics.

In the next section, we provide more details about how some of the methodologies of a number of neighbor choices in the literature can be combined with our results.
## 6. The bandwidth selection criterion

Many methods have been established and developed to construct, in asymptotically optimal ways, bandwidth selection rules for nonparametric kernel estimators especially for Nadaraya-Watson regression estimator we quote among them [15, 44, 45, 50, 105, 109, 166]. This parameter has to be selected suitably, either in the standard finite-dimensional case or in the infinite-dimensional framework to ensure good practical performances. However, according to our knowledge, such studies do not presently exist for treating a such general functional conditional U-statistic (unless the real case we could find in the paper of [79] a paragraph devoted to the selection of the number k). Nevertheless an extension of the leave-one-out cross-validation procedure allows to define, for any fixed $\mathbf{i} = (i_1, \dots, i_m) \in I(m, n)$:

$$ \hat{r}_{n, \mathbf{i}}^{*(m)}(\varphi, \mathbf{t}, \mathbf{h}_{n,k}(\mathbf{t})) = \frac{\sum_{\mathbf{j} \in I_n^m(\mathbf{i})} \varphi(Y_{j_1}, \dots, Y_{j_m}) K\left(\frac{d(t_1, X_{j_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{j_m})}{H_{n,k}(t_m)}\right)}{\sum_{\mathbf{j} \in I_n^m(\mathbf{i})} K\left(\frac{d(t_1, X_{j_1})}{H_{n,k}(t_1)}\right) \cdots K\left(\frac{d(t_m, X_{j_m})}{H_{n,k}(t_m)}\right)}, \quad (6.1) $$

where

$$ I_n^m(\mathbf{i}) := \{\mathbf{j} \in I(m, n) \text{ and } \mathbf{j} \neq \mathbf{i}\} = I(m, n) \setminus \{\mathbf{i}\}. $$

The Eq (6.1) represents the leave-out-$(\mathbf{X}_i, \mathbf{Y}_i)$ estimator of the functional regression and also could be considered as a predictor of $\varphi(\mathbf{Y}_i)$. In order to minimize the quadratic loss function, we introduce the following criterion, we have for some (known) non-negative weight function $\mathcal{W}(\cdot)$:

$$ CV(\varphi, k) := \sum_{i \in I(m,n)} \left( \varphi(\mathbf{Y}_i) - \hat{r}_{n,i}^{*(m)}(\varphi, \mathbf{X}_i, \mathbf{h}_{n,k}(\mathbf{X}_i)) \right)^2 \widetilde{\mathcal{W}}(\mathbf{X}_i), \quad (6.2) $$

where

$$ \widetilde{\mathcal{W}}(\mathbf{t}) := \prod_{i=1}^{m} \mathcal{W}(t_i). $$

Following the ideas developed by [166], a natural way for choosing the bandwidth is to minimize the precedent criterion, so let's choose $\tilde{k} \in [k_{1,n}, k_{2,n}]$ minimizing among $k \in [k_{1,n}, k_{2,n}]$:

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m \times \mathcal{H}^m} CV(\varphi, k), $$

we can conclude, by Corollary 3.18, that, as $n \to \infty$,

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m \times \mathcal{H}^m} \sup_{\mathbf{t} \in S_m^m} \left| \hat{r}_{n, \mathbf{i}}^{*(m)}(\varphi, \mathbf{t}, \mathbf{h}_{n, \tilde{k}}(\mathbf{t})) - r^{(m)}(\varphi, \mathbf{t}) \right| \to 0, \quad \text{a.co.} $$

The main interest of our results is the possibility of deriving asymptotics for any automatic data-driven parameters. Let $K'(\cdot)$ be a density function in $\mathbb{R}^d$ and $H'_{n,k'}(t_i)$ is the number of neighborhoods associated with $Y_i$. One can estimate the conditional density $\mathbf{f}^{(m)}(y_1, \dots, y_m | \mathbf{t}) = \mathbf{f}^{(m)}(\mathbf{y} | \mathbf{t})$ by

$$ \hat{\mathbf{f}}_n^{(m)}(\mathbf{y}, \mathbf{t}) $$
$$ := \frac{\sum_{(i_1, \dots, i_m) \in I(m,n)} K' \left( \frac{t_1 - Y_{i_1}}{H'_{n,k'}(t_1)} \right) \cdots K' \left( \frac{t_m - Y_{i_m}}{H_{n,k'}(t_m)} \right) K \left( \frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)} \right) \cdots K \left( \frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)} \right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K \left( \frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)} \right) \cdots K \left( \frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)} \right)}. $$

Hence, the leave-one-out estimator is given by

$$ \widehat{\mathbf{f}}_{n,i}^{(m)}(\mathbf{y}, \mathbf{t}) := \frac{\sum_{(j_1, \dots, j_m) \in I_n^m(i)} K' \left( \frac{t_1 - Y_{j_1}}{H'_{n,k'}(t_1)} \right) \cdots K' \left( \frac{t_m - Y_{j_m}}{H_{n,k'}(t_m)} \right) K \left( \frac{d(t_1, X_{j_1})}{H_{n,k}(t_1)} \right) \cdots K \left( \frac{d(t_m, X_{j_m})}{H_{n,k}(t_m)} \right)}{\sum_{(i_1, \dots, i_m) \in I(m,n)} K \left( \frac{d(t_1, X_{i_1})}{H_{n,k}(t_1)} \right) \cdots K \left( \frac{d(t_m, X_{i_m})}{H_{n,k}(t_m)} \right)}. $$

While the cross-validation procedures described above aim to approximate quadratic errors of estimation, alternative ways for choosing smoothing parameters could be introduced aiming rather to optimize the predictive power of the method. The criterion is given by

$$ (\check{k}, \check{k}') = \arg \min_{k_{1,n} \le k \le k_{2,n}, \ k'_{1,n} \le k' \le k'_{2,n}} \sum_{i \in I(m,n)} \left( \varphi(\mathbf{Y}_i) - \arg \max_{\mathbf{y} \in \mathbf{J}} \widehat{\mathbf{f}}_{n,i}^{(m)}(\mathbf{y}, \mathbf{t}) \right)^2. $$

## 7. Concluding remarks

In this paper, we consider the kNN kernel type estimator for conditional U-statistics, with the Nadaraya-Watson estimator as a special case, in a functional setting with regular datasets. To obtain our results, we need some regularity on the conditional U-statistics and conditional moments, decay rates on the probability of variables belonging to shrinking open balls, and convenient decreasing rates on mixing coefficients. In particular, the conditional moment assumption allows unbounded classes of functions to be considered. The proof of weak convergence adheres to a standard method: convergence of finite dimensions and the intricate equicontinuity of conditional U-processes. Approaching independence with a block decomposition technique, and then proving a central limit theorem for independent variables leads to finite-dimensional convergence. The equicontinuity requires more intricate control, and the details are lengthy due to the general and complex framework we have considered and will be presented in the following section. Observe that mixing is a type of asymptotic independence assumption that is commonly used to seek simplicity, but can be implausible when there is a strong dependence between the data. In [69] it is argued that β-mixing is the weakest mixing assumption that allows for a "complete" empirical process theory that incorporates maximal inequalities and uniform central limit theorems. There exist explicit upper bounds for β-mixing coefficients for Markov chains (cf. [113]) and for so-called V-geometric mixing coefficients (cf. [169]). For several stationary time series models like linear processes (cf. [202] for α-mixing), ARMA (cf. [189]), nonlinear AR (cf. [126]). A common assumption in these results is that the observed process or, more often, the innovations of the corresponding process, have a continuous distribution. This is a crucial assumption to handle the relatively complicated mixing coefficients defined over a
supremum over two different sigma-algebras. A relaxation of β-mixing coefficients was investigated by ([103], Theorem 1) and is specifically designed for the analysis of the EDF, for more details, refer to [162]. The application of non-parametric functional concepts to general dependence structure is a relatively underdeveloped field. Notably, the ergodic framework eschews the commonly employed strong mixing condition and its variants for measuring dependence, as well as the extremely involved probabilistic calculations that this condition necessitates. It would be interesting to extend our work to the case of functional ergodic data, but this would require nontrivial mathematics and is well outside the scope of this paper. The primary obstacle lies in the necessity of formulating novel probabilistic results, as the ones employed in our current work, as demonstrated in [8], are tailored specifically for β-mixing samples. Another direction is to consider reducing the predictor's dimensionality by employing a Single Functional Index Model (SFIM) to estimate the regression [53]. SFIM has demonstrated its effectiveness in enhancing the consistency of the regression operator estimator. Change-point detection is widely employed to pinpoint positions within a data sequence where a stochastic system undergoes abrupt external influences. This method finds application across various scientific disciplines. The identification of these changes is crucial for exploring their diverse causes and enables appropriate responses. The challenge of detecting disruptions in a sequence of random variables has a rich historical background, refer to [26, 39, 41]. It would be of interest to find applications of our results in this direction.

# 8. Mathematical developments

This section is devoted to the proof of our results. The aforementioned notation is also used in what follows. The proof of Theorems are quite involved and will decomposed in several lemmas proved in Section A.

## 8.1. Proofs of uniform consistency results

### 8.1.1. General Lemma

We present Lemma 8.1 in a general setting, for instance, see [44], which could be useful in many other situations than ours; this is a generalization of a result obtained in [58]. More generally, this technical tool could be useful for dealing with random bandwidths.

Let $(A_i, B_i)_{n \ge 1}$ be $n$ random vectors valued in $(\Omega^m \times \Omega'^m, \mathcal{A} \times \mathcal{B})$, a general space. Let $S_\Omega$ be a fixed subset of $\Omega$ and we note that $G: \mathbb{R} \times (S_\Omega \times \Omega) \to \mathbb{R}^+$ a function such that, $\forall t \in S_\Omega$, $G(\cdot, (t, \cdot))$ is measurable and $\forall x, x' \in \mathbb{R}$:

$$ (L_0) \qquad x \le x' \Rightarrow G(x, \mathbf{z}) \le G(x', \mathbf{z}), \forall \mathbf{z} \in S_\Omega \times \Omega. $$

We define the pointwise measurable class of functions, for $1 \le m \le n$:

$$ \mathcal{G}^m := \left\{ (x_1, \dots, x_m) \mapsto \prod_{i=1}^m G(h_i, (x_i, t_i)), (h_1, \dots, h_m) \in \mathbb{R}_+^m \setminus \{0\} \text{ and } (t_1, \dots, t_m) \in S_\Omega^m \right\}. $$

Let $(\mathbf{D}_{n,k}(\mathbf{t}))_{n \in \mathbb{N}}$ be a sequence of random real vectors (r.r.v.) in such a way that for all $\mathbf{t} = (t_1, \dots, t_m) \in S_\Omega^m$, $\mathbf{D}_{n,k}(\mathbf{t}) = (D_{n,k}(t_1), \dots, D_{n,k}(t_m))$, and $\varphi: \Omega'^m \to \mathbb{R}$ be a measurable function belonging to some
class of functions $\mathcal{F}_m$ and let $\mathfrak{M}^{(m)} : \mathcal{F}_m \times S_{\Omega}^m \to \mathbb{R}$ be a nonrandom function such that,

$$ \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{t} \in S_{\Omega}^m} |\mathfrak{M}^{(m)}(\varphi, \mathbf{t})| < \infty. $$

Now, for all $\mathbf{t} \in S_{\Omega}^m$, $\varphi \in \mathcal{F}_m$ and $n \in \mathbb{N} \setminus \{0\}$ we define

$$ \mathfrak{M}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}) = \frac{\sum_{i \in I(m,n)} \varphi(B_{i_1}, \dots, B_{i_m}) \prod_{j=1}^{m} G(H_j, (t_j, A_{i,j}))}{\sum_{i \in I(m,n)} \prod_{j=1}^{m} G(H_j, (t_j, A_{i,j}))}, $$

where $\mathbf{h} = (H_1, \dots, H_m) \in \mathbb{R}_+^m$, and $\tilde{G} = \prod_{i=1}^m G(h_i, (x_i, t_i))$, $\mathbf{t} = (t_1, \dots, t_m) \in S_{\Omega}^m$.

**Lemma 8.1.** Let $(\mathcal{U}_n)_{n \in \mathbb{N}}$ be a decreasing positive sequence such that $\lim_{n \to \infty} \mathcal{U}_n = 0$. If, for all increasing sequence $\xi_n \in (0, 1)$ with $\xi_n - 1 = O(\mathcal{U}_n)$, there exist two sequences of r.r.v. $(D_{n,k}^{-}(\xi_n, \mathbf{t}))_{n \in \mathbb{N}}$ and $(D_{n,k}^{+}(\xi_n, \mathbf{t}))_{n \in \mathbb{N}}$ such that

$$ (L_1) \qquad \forall n \in \mathbb{N} \text{ and } \mathbf{t} \in S_{\Omega}^m, D_{n,k}^{-}(\xi_n, t_j) \le D_{n,k}^{+}(\xi_n, t_j), \forall j = 1, \dots, m, $$

$$ (L_2) \qquad \prod_{j=1}^{m} 1_{\{D_{n,k}^{-}(\xi_n, t_j) \le D_{n,k}^{+}(t_j) \le D_{n,k}^{+}(\xi_n, t_j)\}} \to 1, \quad \mathit{a.co}, \quad \forall \mathbf{t} \in S_{\Omega}^{m}. $$

$$ (L_3) \qquad \sup_{\tilde{G} \in \mathcal{G}^m} \sup_{\mathbf{t} \in S_{\Omega}^m} \left| \frac{\sum_{i \in I(m,n)} \prod_{j=1}^{m} G(D_{n,k}^{-}(\xi_n, t_j), (t_j, A_{i,j}))}{\sum_{i \in I(m,n)} \prod_{j=1}^{m} G(D_{n,k}^{+}(\xi_n, t_j), (t_j, A_{i,j}))} - \xi_n \right| = O_{a.co}(\mathcal{U}_n), $$

$$ (L_4) \qquad \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{t} \in S_{\Omega}^m} \left| \mathfrak{M}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{D}_{n,k}^{-}(\xi_n, \mathbf{t})) - \mathfrak{M}^{(m)}(\varphi, \mathbf{t}) \right| = O_{a.co}(\mathcal{U}_n), $$

$$ (L_5) \qquad \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{t} \in S_{\Omega}^m} \left| \mathfrak{M}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{D}_{n,k}^{+}(\xi_n, \mathbf{t})) - \mathfrak{M}^{(m)}(\varphi, \mathbf{t}) \right| = O_{a.co}(\mathcal{U}_n). $$

Then, as $n \to \infty$,

$$ \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{t} \in S_{\Omega}^m} \left| \mathfrak{M}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{D}_{n,k}(\mathbf{t})) - \mathfrak{M}^{(m)}(\varphi, \mathbf{t}) \right| = O_{a.co}(\mathcal{U}_n). \quad (8.1) $$

We refer to [44] for proof of this lemma.
**Proof of Theorem 3.1**

In order to establish the convergence rates, the following notation is necessary. For all $t \in S_X$, set

$$
\begin{align*}
\Delta_i(t; h_K(t)) &:= K(d(X_i, t)/h_K(t)), \\
\hat{r}_{n,2}^{(1)}(\varphi, t; h_K(t)) &:= \frac{1}{n\mathbb{E}(\Delta_1(t; h_K(t)))} \sum_{i=1}^{n} \varphi(Y_i)\Delta_i(t; h_K(t)), \\
\hat{r}_{n,1}^{(1)}(1, t; h_K(t)) &:= \frac{1}{n\mathbb{E}(\Delta_1(t; h_K(t)))} \sum_{i=1}^{n} \Delta_i(t; h_K(t)).
\end{align*}
$$

This allows us to write

$$
\hat{r}_{n}^{(1)}(\varphi, t; h_{K}(t)) = \hat{r}_{n,2}^{(1)}(\varphi, t; h_{K}(t)) / \hat{r}_{n,1}^{(1)}(1, t; h_{K}(t)).
$$

Now, let us consider the following decomposition

$$
\begin{align*}
\hat{r}_n^{(1)}(\varphi, t; h_K(t)) - r^{(1)}(\varphi, t) &= \frac{1}{\hat{r}_{n,1}^{(1)}(\varphi, t; h_K(t))} \left\{ \hat{r}_{n,2}^{(1)}(\varphi, t; h_K(t)) - \mathbb{E}\left[\hat{r}_{n,2}^{(1)}(\varphi, t; h_K(t))\right] \right\} \\
&\quad + \frac{1}{\hat{r}_{n,1}^{(1)}(\varphi, t; h_K(t))} \left\{ \mathbb{E}\left[\hat{r}_{n,2}^{(1)}(\varphi, t; h_K(t))\right] - r^{(1)}(\varphi, t) \right\} \\
&\quad + \frac{r^{(1)}(\varphi, t)}{\hat{r}_{n,1}^{(1)}(\varphi, t; h_K(t))} \left\{ 1 - \hat{r}_{n,1}^{(1)}(\varphi, t; h_K(t)) \right\}.
\end{align*}
$$

Therefore, the proof of (8.2) is based on the following lemmas.

**Lemma 8.2.** *Under assumptions (C.1.1), (C.3.1), (C.4.1), (C.5.1), (C.5.2'), (C.6.) and (C.7.), we have, as $n \to \infty$,*

$$
\sup_{\varphi K \in \mathcal{F} \mathcal{H}} \sup_{h_{n,1} \le h_K \le h_{n,2}} \sup_{t \in S_X} \left| \hat{r}_{n,2}^{(1)}(\varphi, t; h_K(t)) - \mathbb{E} \left( \hat{r}_{n,2}^{(1)}(\varphi, t; h_K(t)) \right) \right| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(h_{n,1})}} \right), \quad (8.2)
$$

and

$$
\sup_{\varphi K \in \mathcal{F} \mathcal{H}} \sup_{h_{n,1} \le h_K \le h_{n,2}} \sup_{t \in S_X} \left| \hat{r}_{n,1}^{(1)}(\varphi, t; h_K(t)) - 1 \right| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(h_{n,1})}} \right). \quad (8.3)
$$

This lemma gives us the rates of consistency of the stochastic part when the class of functions $\mathcal{F}$ is bounded. The following lemma will give us the result when the class of functions is unbounded.
**Lemma 8.3.** Under assumptions the (C.1.1), (C.3.1), (C.4.2), (C.5.1), (C.5.2'), (C.6.) and (C.7.), we have, as $n \to \infty$,

$$ \sup_{\varphi K \in \mathcal{F}\mathcal{H}} \sup_{h_{n,1} \le h_K \le h_{n,2}} \sup_{t \in S_X} \left| \widehat{r}_{n,2}^{(1)}(\varphi, t, h_K(t)) - \mathbb{E}(\widehat{r}_{n,2}^{(1)}(\varphi, t, h_K(t))) \right| = O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(h_{n,1})}} \right). \quad (8.4) $$

Finally, we only need the following result for the bias term. This lemma can be obtained similar way as in [147], where more details are given.

**Lemma 8.4.** Under the condition (C.2.1), we have, as $n \to \infty$,

$$ \sup_{\varphi K \in \mathcal{F}\mathcal{H}} \sup_{h_{n,1} \le h_K \le h_{n,2}} \sup_{t \in S_X} \left| \mathbb{E} \left[ \widehat{r}_{n,2}^{(1)}(\varphi, t; h_K(t)) \right] - r^{(1)}(\varphi, t) \right| = O(h_{n,2}^{\gamma}). \quad (8.5) $$

## Proof of Theorem 3.3

Similar to [44], To prove Theorem 3.3 we need to check the conditions of Lemma 8.1 in the case of $m = 1$. For that, we first identify the variables as follows: $S_\Omega = S_X$, $A_i = X_i$, $\varphi(B_i) = \varphi(Y_i)$,

$$ \begin{aligned} G(H, (t, A_i)) &= K(H^{-1}d(t, X_i)), \\ D_{n,k}(t) &= H_{n,k}(t), \\ \mathfrak{M}_n^{(1)}(\varphi, t; H_{n,k}(t)) &= \widehat{r}_n^{(1)}(\varphi, t; H_{n,k}(t)), \\ \mathfrak{M}(\varphi, t) &= r^{(1)}(\varphi, t). \end{aligned} $$

Choosing $D_{n,k}^-(\xi_n, t)$ and $D_{n,k}^+(\xi_n, t)$ such that

$$ \phi_t(D_{n,k}^-(\xi_n, t)) = \frac{\sqrt{\xi_n k}}{n}, \quad (8.6) $$

$$ \phi_t(D_{n,k}^+(\xi_n, t)) = \frac{k}{n\sqrt{\xi_n}}. \quad (8.7) $$

We denote $h^-(t) = D_{n,k}^-(\xi_n, t)$, $h^+(t) = D_{n,k}^+(\xi_n, t)$ and

$$ \mathcal{U}_n = \phi^{-1} \left( \frac{k_{2,n}}{\sqrt{\xi_n n}} \right)^\gamma + \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi \left( \mu\phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)}} $$

for all increasing sequence $\xi_n \in (0, 1)$ such that $\xi_n - 1 = O(\mathcal{U}_n)$. Note that for all $\xi_n \in (0, 1)$, $t \in S_X$ and $k_{1,n} \le k \le k_{2,n}$ we have

$$ \phi_t^{-1} \left( \frac{\sqrt{\xi_n k_{1,n}}}{n} \right) \le h^-(t) \le \phi_t^{-1} \left( \frac{\sqrt{\xi_n k_{2,n}}}{n} \right), $$
$$ \phi_t^{-1} \left( \frac{k_{1,n}}{n \sqrt{\xi_n}} \right) \le h^+(t) \le \phi_t^{-1} \left( \frac{k_{2,n}}{n \sqrt{\xi_n}} \right), $$

using the condition (2.12) we can easily deduce that the bandwidths $h^-(t)$ and $h^+(t)$ both belong to the interval

$$ [h_{n,1}, h_{n,2}] = \left[ \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right), \nu \phi^{-1} \left( \frac{k_{2,n}}{\rho_n n} \right) \right]. $$

Checking the conditions (L$_4$) and (L$_5$)

Let us start with checking (L$_4$). The fact that $\xi_n$ is bounded by 1, and the local bandwidth $h^-(t)$ satisfies the conditions of Theorem 3.1 gives

$$
\begin{aligned}
& \sup_{\varphi K \in \mathcal{F} \mathcal{H}} \sup_{h_{n,1} \le h^-(t) \le h_{n,2}} \sup_{t \in S_X} \left| \mathfrak{M}_n^{(1)} (\varphi, t; D_{n,k}^-(\xi_n, t)) - \mathfrak{M}^{(1)} (\varphi, t) \right| \\
& = \sup_{\varphi K \in \mathcal{F} \mathcal{H}} \sup_{h_{n,1} \le h^-(t) \le h_{n,2}} \sup_{t \in S_X} \left| \frac{\sum_{i=1}^n \varphi(Y_i) K \left( \frac{d(t, X_i)}{h^-(t)} \right)}{\sum_{i=1}^n K \left( \frac{d(t, X_i)}{h^-(t)} \right)} - r^{(1)}(\varphi, t) \right| \\
& = \sup_{\varphi K \in \mathcal{F} \mathcal{H}} \sup_{h_{n,1} \le h^-(t) \le h_{n,2}} \sup_{t \in S_X} \left| \hat{r}_n^{(1)}(\varphi, t; h^-(t)) - r^{(1)}(\varphi, t) \right| \\
& = O_{a.co} \left( h_{n,2}^\gamma + O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(h_{n,1})}} \right) \right),
\end{aligned}
$$

which is equivalent to

$$
\begin{aligned}
& \sup_{\varphi K \in \mathcal{F} \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} \left| \hat{r}_n^{(1)}(\varphi, t; h^-(t)) - r^{(1)}(\varphi, t) \right| \\
& = O_{a.co} \left( \phi^{-1} \left( \frac{k_{2,n}}{\sqrt{\xi_n n}} \right)^\gamma + \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)}} \right) \\
& = O_{a.co}(\mathcal{U}_n).
\end{aligned}
$$

We use the same reasoning to check (L$_5$) and we readily obtain

$$ \sup_{\varphi K \in \mathcal{F} \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} \left| \mathfrak{M}_n(\varphi, t; D_{n,k}^+(\xi_n, t)) - \mathfrak{M}(\varphi, t) \right| = O_{a.co}(\mathcal{U}_n). $$

Hence, (L$_4$) and (L$_5$) are checked.
Checking the condition (L₂)

To check (L₂) we show that for all $t \in S_X$ and $\varepsilon_0 > 0$,

$$ \sum_{n \ge 1} \mathbb{P} \left\{ \left| \mathbf{1}_{\left\{ D_{n,k}^{-}(\xi_n, t) \le H_{n,k}(t) \le D_{n,k}^{+}(\xi_n, t) \right\}} - 1 \right| > \varepsilon_0 \right\} < \infty. \quad (8.8) $$

Let $\varepsilon_0 > 0$ be fixed. Let $\{t_1, \dots, t_{N_\varepsilon(S_X)}\}$ be an $\varepsilon$-net for $S_X$, for all $t \in S_X$, we have

$$ \begin{aligned} & \mathbb{P}\left\{\left|\mathbf{1}_{\left\{D_{n,k}^{-}(\xi_n, t) \le H_{n,k}(t) \le D_{n,k}^{+}(\xi_n, t)\right\}} - 1\right| > \varepsilon_0\right\} \\ & \le \mathbb{P}\left(H_{n,k}(t) \le \phi_t^{-1}\left(\frac{\sqrt{\xi_n}k_{1,n}}{n}\right)\right) + \mathbb{P}\left(H_{n,k}(t) \ge \phi_t^{-1}\left(\frac{k_{2,n}}{\sqrt{\xi_n}n}\right)\right) \\ & \le \sum_{\ell=1}^{N_{\varepsilon_n}(S_X)} \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(t_\ell) \le \phi_t^{-1}\left(\frac{\sqrt{\xi_n}k_{1,n}}{n}\right)\right) \\ & \qquad + \sum_{\ell=1}^{N_{\varepsilon_n}(S_X)} \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(t_\ell) \ge \phi_t^{-1}\left(\frac{k_{2,n}}{\sqrt{\xi_n}n}\right)\right) \\ & \le N_{\varepsilon_n}(S_X) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(t_\ell) \le \phi_t^{-1}\left(\frac{\sqrt{\xi_n}k_{1,n}}{n}\right)\right) \\ & \qquad + N_{\varepsilon_n}(S_X) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(t_\ell) \ge \phi_t^{-1}\left(\frac{k_{2,n}}{\sqrt{\xi_n}n}\right)\right). \end{aligned} $$

Now, we use a lemma similar to [124] (see Lemma B.5). For completeness, we give their proof. Making use of Lemma B.5, we infer that

$$ \begin{aligned} \mathbb{P}\left\{H_{n,k}(t) \le \phi_t^{-1}\left(\frac{\alpha k_{1,n}}{n}\right)\right\} &= \mathbb{P}\left\{\sum_{i=1}^{n} 1_{B(t, \phi_t^{-1}\left(\alpha^{\frac{k_{1,n}}{n}}\right)}(X_i) > k\right\} \\ &= \mathbb{P}\left\{\sum_{i=1}^{n} 1_{B(t, \phi_t^{-1}\left(\alpha^{\frac{k_{1,n}}{n}}\right)}(X_i) > \frac{k}{\alpha k_{1,n}}\right\} \\ &\le \exp\left\{-\left(k - \alpha k_{1,n}\right)/4\right\}. \end{aligned} \quad (8.9) $$

This implies that

$$ \begin{aligned} N_{\varepsilon_n}(S_X) \sum_{k=k_{1,n}}^{k_{2,n}} \mathbb{P}\left\{H_{n,k}(t) \le \phi_t^{-1}\left(\frac{\alpha k_{1,n}}{n}\right)\right\} &\le N_{\varepsilon_n}(S_X) k_{2,n} \exp\left\{-\left(1-\alpha\right)k_{1,n}/4\right\} \\ &\le N_{\varepsilon_n}(S_X) n^{1-\left(\left(1-\alpha\right)/4\right)^{\frac{k_{1,n}}{\ln n}}} \end{aligned} $$

In a similar way, we obtain

$$ \mathbb{P}\left\{H_{n,k}(t) \ge \phi_t^{-1}\left(\frac{k_{2,n}}{\alpha n}\right)\right\} \le \exp\left\{-\frac{(k_{2,n} - \alpha k)^2}{2\alpha k_{2,n}}\right\}. \quad (8.10) $$
It follows that

$$
\begin{aligned}
& N_{\varepsilon_n}(S_X) \sum_{k=k_{1,n}}^{k_{2,n}} \mathbb{P} \left\{ H_{n,k}(t) \ge \phi_t^{-1} \left( \frac{k_{2,n}}{\alpha n} \right) \right\} \le N_{\varepsilon_n}(S_X) k_{2,n} \exp \left\{ -(1-\alpha)k_{1,n}/2\alpha \right\} \\
& \le N_{\varepsilon_n}(S_X) n^{1-\left\{ \frac{(1-\alpha)}{2\alpha} \right\} \frac{k_{2,n}}{\ln n}}.
\end{aligned}
$$

Therefore, by that fact that $k_{i,n}/\ln n \to \infty$, $i=1,2$, we obtain

$$
N_{\epsilon_n}(S_X) \sum_{k=k_{1,n}}^{k_{2,n}} \mathbb{P} \left\{ H_{n,k}(t) \le \phi_t^{-1} \left( \frac{\alpha k_{1,n}}{n} \right) \right\} \le N_{\epsilon_n}(S_X) n^{1-\left\{\frac{(1-\alpha)}{4}\right\} \frac{k_{1,n}}{\ln n}} < \infty, \quad (8.11)
$$

$$
N_{\epsilon_n}(S_X) \sum_{k=k_{1,n}}^{k_{2,n}} \mathbb{P} \left\{ H_{n,k}(t) \ge \phi_t^{-1} \left( \frac{k_{2,n}}{\alpha n} \right) \right\} \le N_{\epsilon_n}(S_X) n^{1-\left\{ \frac{(1-\alpha)}{2\alpha} \right\} \frac{k_{2,n}}{\ln n}} < \infty. \quad (8.12)
$$

Checking the condition (L<sub>3</sub>)

We consider the following quantities:

$$
Q_{n1} := \frac{\Delta_1(t, D_n^-(\xi_n, t))}{\Delta_1(t, D_n^+(\xi_n, t))},
$$

$$
Q_{n2} := \frac{\hat{r}_{n,1}^{(1)}(\varphi, t; D_n^-(\xi_n, t))}{\hat{r}_{n,1}^{(1)}(\varphi, t; D_n^+(\xi_n, t))} - 1,
$$

$$
Q_{n3} := \frac{\Delta_1(t, D_n^+(\xi_n, t))}{\Delta_1(t, D_n^-(\xi_n, t))} \xi_n - 1.
$$

The condition (L<sub>3</sub>) can be written as

$$
\left| \frac{\sum_{i=1}^{n} K \left( \frac{d(t, X_i)}{D_n^-(\xi_n, t)} \right)}{\sum_{i=1}^{n} K \left( \frac{d(t, X_i)}{D_n^+(\xi_n, t)} \right)} - \xi_n \right| \le |Q_{n1}| |Q_{n2}| + |Q_{n1}| |Q_{n3}|.
$$

Hence, by the fact that $\xi_n \to 1$, our claimed result is

$$
\sup_{K \in \mathcal{K}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} \left| \frac{\sum_{i=1}^{n} K \left( \frac{d(t, X_i)}{D_n^-(\xi_n, t)} \right)}{\sum_{i=1}^{n} K \left( \frac{d(t, X_i)}{D_n^+(\xi_n, t)} \right)} - \xi_n \right| = O_{a,co}(\mathcal{U}_n). \quad (8.13)
$$

The proof of (8.13) is based on the following results

$$
\sup_{K \in \mathcal{K}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_{n1}| \le C, \quad (8.14)
$$
$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_{n2}| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)}} \right), \quad (8.15) $$

$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_{n3}| = O \left( \phi^{-1} \left( \frac{k_{2,n}}{\rho_n n} \right)^\gamma \right). \quad (8.16) $$

### Proof of (8.14)

Using the condition (C.3.1) one has

$$ \mathbb{E} \left( K \left( \frac{d(X_1, t)}{h(t)} \right) \right) \le \kappa_2 \phi_t(h(t)). $$

Now using the condition (C.1.1) we directly obtain

$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_{n1}| \le C. \quad (8.17) $$

### Proof of (8.15)

We have

$$ \begin{aligned} \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_{n2}| &= \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} \left| \frac{Q_n(K, t, h^-(t))}{Q_n(K, t, h^+(t))} - 1 \right| \\ &\le \frac{1}{\inf_{K \in \mathcal{H}} \inf_{k_{1,n} \le k \le k_{2,n}} \inf_{t \in S_X} |Q_n(K, t, h^+(t))|} \left( \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_n(K, t, h^-(t)) - 1| \right. \\ &\qquad \left. + \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_n(K, t, h^+(t)) - 1| \right). \end{aligned} \quad (8.18) $$

To prove this, we use Lemma 8.2, which gives

$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_n(K, t, h^-(t)) - 1| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)}} \right), \quad (8.19) $$

and

$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_n(K, t, h^+(t)) - 1| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)}} \right). \quad (8.20) $$

Moreover, combining (8.19), (8.20) with the fact that

$$ \sum_{n \ge 1} \mathbb{P} \left( \inf_{K \in \mathcal{H}} \inf_{t \in S_X} \tilde{r}_{n,1}^{(1)}(\varphi, t; D_n^+(\xi_n, t)) < C \right) < \infty, \quad (8.21) $$
it follows that

$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_{n2}| = O_{a,co} \left( \sqrt{\frac{\psi_{S_X}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right)\right)}} \right). \quad (8.22) $$

Proof of (8.16)

[134] use in their proof of part the proof of Lemma 1 in [89], on the other, we will use some computations similar to the steps of the proof of Lemma 5.7.0.3 in [42]. Let us consider the following quantity:

$$ \begin{aligned} \hat{B}(t, \varphi K) &= \frac{\mathbb{E}[\hat{r}_{n,2}^{(1)}(\varphi, t; D_n^+(\xi_n, t))]}{\mathbb{E}[\hat{r}_{n,1}^{(1)}(\varphi, t; D_n^-(\xi_n, t))]} - r^{(1)}(\varphi, t), \\ |\hat{B}(t, \varphi K)| &= \left| \frac{\phi_t(D_n^-(\xi_n, t))}{\phi_t(D_n^+(\xi_n, t))} \frac{\mathbb{E}\left[K\left(\frac{d(t, X_1)}{D_n^+(\xi_n, t)}\right)\mathbb{E}[\varphi(Y_1)|X_1]\right]}{\mathbb{E}\left[K\left(\frac{d(t, X_1)}{D_n^-(\xi_n, t)}\right)\right]} - r^{(1)}(\varphi, t) \right| \\ &\le \frac{1}{\mathbb{E}\left[K\left(\frac{d(t, X_1)}{D_n^-(\xi_n, t)}\right)\right]} \left| \frac{\phi_t(D_n^-(\xi_n, t))}{\phi_t(D_n^+(\xi_n, t))} \mathbb{E}\left[K\left(\frac{d(t, X_1)}{D_n^+(\xi_n, t)}\right)\left(\mathbb{E}[\varphi(Y_1)|X_1] - r^{(1)}(\varphi, t)\right)\right] \right|, \end{aligned} $$

using the fact that $\frac{\phi_t(D_n^-(\xi_n, t))}{\phi_t(D_n^+(\xi_n, t))} = \xi_n$ and supposing that the condition (C.2.1) holds which means

$$ |r^{(1)}(X, t) - r^{(1)}(\varphi, t)| \le C_3 d^{\gamma}(X, t), $$

and assuming that the conditions (C.1.1) and (C.3.1) to be satisfied, then for all $t \in S_X$, and $D_n^-(\xi_n, t)$ and $D_n^+(\xi_n, t)$ in $[h_{n,1}, h_{n,2}]$, one gets

$$ \begin{aligned} \left| \frac{\mathbb{E}[\hat{r}_{n,2}^{(1)}(\varphi, t; D_n^+(\xi_n, t))]}{\mathbb{E}[\hat{r}_{n,1}^{(1)}(\varphi, t; D_n^-(\xi_n, t))]} - r^{(1)}(\varphi, t) \right| &\le \frac{C_3 \kappa_2 \xi_n}{\kappa_1 \phi_t(D_n^-(\xi_n, t))} \left[ \mathbb{E}1_{B(t,D_n^+(\xi_n,t))}(X) d^\gamma(X,t) \right] \\ &\le \frac{C_3 \kappa_2 \xi_n}{\kappa_1} \frac{\phi_t(D_n^+(\xi_n, t))}{\phi_t(D_n^-(\xi_n, t))} (D_n^+(\xi_n, t))^{\gamma} \le \frac{C_2 C_3 \kappa_2 \xi_n}{C_1 \kappa_1} (D_n^+(\xi_n, t))^{\gamma} \\ &\le C(D_n^+(\xi_n, t))^{\gamma}. \end{aligned} $$

Keeping in mind the condition (C.3.1) and the fact that $\xi_n \to 1$, we obtain

$$ \sup_{\varphi K \in \mathcal{H}} \sup_{(h^{-}(t), h^{+}(t)) \in [h_{n,1}, h_{n,2}]^2} \sup_{t \in S_X} \left| \frac{\mathbb{E}[\hat{r}_{n,2}^{(1)}(\varphi, t; D_n^+(\xi_n, t))]}{\mathbb{E}[\hat{r}_{n,1}^{(1)}(\varphi, t; D_n^-(\xi_n, t))]} - r^{(1)}(\varphi, t) \right| \le C' h_{n,2}^{\gamma}. \quad (8.23) $$

Finally, rewriting (8.23) with $\varphi \equiv 1$ gives us

$$ \sup_{\varphi K \in \mathcal{H}} \sup_{(h^{-}(t), h^{+}(t)) \in [h_{n,1}, h_{n,2}]^2} \sup_{t \in S_X} \left| \frac{\mathbb{E}\left[K\left(\frac{d(t, X_1)}{D_n^+(\xi_n, t)}\right)\right]}{\mathbb{E}\left[K\left(\frac{d(t, X_1)}{D_n^-(\xi_n, t)}\right)\right]} \xi_n - 1 \right| \le C' h_{n,2}^{\gamma}, $$
which is equivalent to

$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} |Q_{n3}| = O\left(\phi^{-1}\left(\frac{k_{2,n}}{\rho_n n}\right)^\gamma\right). \quad (8.24) $$

Combining the results of (8.17), (8.22) and (8.24) and the fact that $\xi_n \to 1$, implies that

$$ \sup_{K \in \mathcal{H}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{t \in S_X} \left| \frac{\sum_{i=1}^{n} K\left(\frac{d(t, X_i)}{D_n^-(\xi_n, t)}\right)}{\sum_{i=1}^{n} K\left(\frac{d(t, X_i)}{D_n^+(\xi_n, t)}\right)} - \xi_n \right| = O_{a,co}(\mathcal{U}_n). $$

Hence, $(L_3)$ is checked. Note that $(L_0)$ is obviously satisfied by (C.3.1), and that $(L_1)$ is also trivially satisfied by construction of $D_n^-(\xi_n, t)$ and $D_n^+(\xi_n, t)$. So one can apply Lemma 8.1, and (8.1) with $m=1$ is exactly the result of Theorem 3.3.

## Preliminaries of the proofs

This part is mainly dedicated to the study of the functional conditional U-statistics. Just like in the case of $m=1$, where $S_X$ is covered by

$$ \bigcup_{\ell=1}^{N_{\varepsilon}(S_X)} B(t_{\ell}, \varepsilon), $$

for some radius $\varepsilon$. Hence, for each $\mathbf{t} \in S_X^m$, there exists $\ell(\mathbf{t}) = (\ell(t_1), \dots, \ell(t_m))$ where $\forall 1 \le i \le m, 1 \le \ell(t_i) \le N_{\varepsilon}(S_X)$ such that

$$ \mathbf{t} \in \prod_{i=1}^{m} B(t_{\ell(t_i)}, \varepsilon) \text{ and } d(t_i, t_{\ell(t_i)}) = \underset{1 \le l \le \ell(t_i) \le N_{\varepsilon}(S_X)}{\operatorname{argmin}} d(t_i, t_{\ell}). $$

So for each $\mathbf{t} \in S_X^m$, the closest center is $t_{\ell(t)}$ and the ball with the closest center

$$ \prod_{i=1}^{m} B(t_{\ell(t_i)}, \varepsilon) := B(t_{\ell(t)}, \varepsilon). $$

The proofs of the UIB consistency for the multivariate bandwidth will follow the same lines as the proofs of the UIB consistency for the univariate smoothing parameter in [42, 44]. Furthermore, as in the proof of Theorem 3.1, we divide the sequence $\{(\mathbf{X}_i, \mathbf{Y}_i)\}$ into $\nu_n$ alternate blocks, here the sizes $a_n, b_n$ are different satisfying

$$ b_n \ll a_n, \quad (\nu_n - 1)(a_n + b_n) < n \le \nu_n(a_n + b_n), \quad (8.25) $$

and set, for $1 \le j \le \nu_n - 1$:

$$ H_j^{(U)} = \{i : (j-1)(a_n + b_n) + 1 \le i \le (j-1)(a_n + b_n) + a_n\}, $$

$$ T_j^{(U)} = \{i : (j-1)(a_n + b_n) + a_n + 1 \le i \le (j-1)(a_n + b_n) + a_n + b_n\}, $$

$$ H_{\nu_n}^{(U)} = \{i : (\nu_n - 1)(a_n + b_n) + 1 \le i \le n \wedge (\nu_n - 1)(a_n + b_n) + a_n\}, $$

$$ T_{\nu_n}^{(U)} = \{i : (\nu_n - 1)(a_n + b_n) + a_n + 1 \le i \le n\}. $$
**Proof of Theorem 3.8**

In this section, we consider a bandwidth $\mathbf{h} = (h_1, \dots, h_m) \in \mathcal{H}_n^{(m)}$. To prove Theorem 3.8, we can write the U-statistic for each $\mathbf{t} \in S_X^m$ as follows

$$|u_n(\varphi, \mathbf{t}; \mathbf{h}) - \mathbb{E}[u_n(\varphi, \mathbf{t}; \mathbf{h})]| \le |u_n(\varphi, \mathbf{t}; \mathbf{h}) - u_n(\varphi, \mathbf{t}_{\ell(\mathbf{t})}; \mathbf{h})| \quad (8.26)$$

$$+ |\mathbb{E}[u_n(\varphi, \mathbf{t}_{\ell(\mathbf{t})}; \mathbf{h})] - \mathbb{E}[u_n(\varphi, \mathbf{t}; \mathbf{h})]| + |u_n(\varphi, \mathbf{t}_{\ell(\mathbf{t})}; \mathbf{h}) - \mathbb{E}[u_n(\varphi, \mathbf{t}_{\ell(\mathbf{t})}; \mathbf{h})]|$$

$$\le |u_{n,11}(\varphi, \mathbf{t}; \mathbf{h})| + |u_{n,12}(\varphi, \mathbf{t}; \mathbf{h})| + |u_{n,13}(\varphi, \mathbf{t}; \mathbf{h})|. \quad (8.27)$$

Let us begin with the term $|u_{n,11}(\varphi, \mathbf{t}; \mathbf{h})|$, we have

$$|u_{n,11}(\varphi, \mathbf{t}; \mathbf{h})| \le \frac{\mathbb{C}(n-m)!}{n!\tilde{\phi}(\mathbf{h})} \sum_{i \in I(m,n)} \left| \varphi(Y_{i_1}, \dots, Y_{i_m}) \left\{ \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \right\} \right|$$

By applying the Telescoping binomial, we get

$$\prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \quad (8.28)$$

$$= \sum_{j=1}^{m} \left[ \left\{ K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \right\} \right] \\ \times \prod_{q=1}^{j-1} K\left(\frac{d(X_{iq}, t_q)}{h_q}\right) \prod_{p=j+1}^{m} K\left(\frac{d(X_{ip}, t_{\ell(t_p)})}{h_p}\right). \quad (8.29)$$

From condition (C.4.1), we could claim that

$$\prod_{q=1}^{j-1} K\left(\frac{d(X_{iq}, t_q)}{h_q}\right) \le \kappa_2^{j-1} \prod_{q=1}^{j-1} 1_{B(t_q, h_q)}(X_{iq}),$$

similarly, we have

$$\prod_{p=j+1}^{m} K\left(\frac{d(X_{ip}, t_{\ell(t_p)})}{h_p}\right) \le \kappa_2^{m-j} \prod_{p=j+1}^{m} 1_{B(t_{\ell(t_p)}, h_p)}(X_{ip}).$$

So, (8.28) satisfies :

$$\prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \\ \prod_{q=1}^{j-1} 1_{B(t_q, h_q)}(X_{iq}) \prod_{p=j+1}^{m} 1_{B(t_{\ell(t_p)}, h_p)}(X_{ip}) \\ \le \kappa_2^{m-1} \sum_{j=1}^{m} \left[ \left\{ K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \right\} 1_{B(t_j, h_j) \cup B(t_{\ell(t_j)}, h_j)}(X_{ij}) \right]$$
$$
\prod_{q=1}^{j-1} \mathbf{1}_{B(t_q, h_q)}(X_{iq}) \prod_{p=j+1}^{m} \mathbf{1}_{B(t_{\ell(p)}, h_p)}(X_{ip}) \\
= \sum_{j=1}^{m} \mathbf{K}_{i_j, h_j}(t_j, t_{\ell(t_j)}),
$$

where

$$
\mathbf{K}_{i_j, h_j}^{(\ell)}(t_j, t_{\ell(t_j)}) = \kappa_2^{m-1} \left\{ K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \right\} \mathbf{1}_{B(t_j, h_j) \cup B(t_{\ell(t_j)}, h_j)}(X_{ij}) \\
\times \prod_{q=1}^{j-1} \mathbf{1}_{B(t_q, h_q)}(X_{iq}) \prod_{p=j+1}^{m} \mathbf{1}_{B(t_{\ell(p)}, h_p)}(X_{ip}).
$$

Therefore, we infer that

$$
|u_{n,11}(\varphi, \mathbf{t}; \mathbf{h})| \le \frac{\mathbb{C}(n-m)!}{n! \tilde{\phi}(\mathbf{h})} \kappa_2^{m-1} \sum_{i \in I(m,n)} \left| \varphi(Y_{i_1}, \dots, Y_{i_m}) \sum_{j=1}^{m} \mathbf{K}_{i_j, h_j}^{(\ell)}(\mathbf{t}_j, \mathbf{t}_{\ell(\mathbf{t}_j)}) \right| \\
\le \frac{\mathbb{C}(n-m)!}{n! \tilde{\phi}(\mathbf{h})} M \kappa_2^{m-1} \sum_{i \in I(m,n)} \sum_{j=1}^{m} \left| \left\{ K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - K\left(\frac{d(X_{ij}, t_{\ell(\mathbf{t}_j)})}{h_j}\right) \right\} \right| \\
\times \mathbf{1}_{B(t_j, h_j) \cup B(t_{\ell(\mathbf{t}_j)}, h_j)}(X_{ij}) \prod_{q=1}^{j-1} \mathbf{1}_{B(t_q, h_q)}(X_{iq}) \prod_{p=j+1}^{m} \mathbf{1}_{B(t_{\ell(p)}, h_p)}(X_{ip}) \tag{8.30}
$$

$$
\le \frac{(n-m)!}{n!} \mathbb{C}mM\kappa_2^{m-1} \sum_{i \in I(m,n)} \frac{1}{m} \sum_{j=1}^{m} \left[ \frac{\varepsilon_n}{\tilde{\phi}(\mathbf{h})h_{n,j}} \mathbf{1}_{B(t_j,h_j) \cup B(t_{\ell(t_j)},h_j)}(X_{ij}) \right. \\
\left. \times \prod_{q=1}^{j-1} \mathbf{1}_{B(t_q,h_q)}(X_{iq}) \prod_{p=j+1}^{m} \mathbf{1}_{B(t_{\ell(p)},h_p)}(X_{ip}) \right]. \tag{8.31}
$$

The transition from Eq (8.30) to (8.31) is done thanks to the fact that the kernel function K(·) is Lipschitz. Uniformly on **t** ∈ S<sub>λ</sub><sup>m</sup> and **h** ∈ H<sub>n</sub><sup>(m)</sup>, we get

$$
\sup_{\varphi K \in \mathcal{F}_m \mathcal{H}^m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} |u_{n,11}(\varphi, \mathbf{t}; \mathbf{h})|
$$
$$
\le \sup_{\varphi K \in \mathcal{F}_m \mathcal{H}^m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \frac{(n-m)!}{n!} \sum_{i \in I(m,n)} \frac{1}{m} \sum_{j=1}^m \left[ \frac{\mathbb{C}mM\kappa_2^{m-1}\varepsilon_n}{\tilde{\phi}(\mathbf{H}_n)h_{n,j}} \mathbf{1}_{B(t_j,h_j) \cup B(t_{\ell(t_j)},h_j)}(X_{ij}) \right. \\
\left. \times \prod_{q=1}^{j-1} \mathbf{1}_{B(t_q,h_q)}(X_{iq}) \prod_{p=j+1}^{m} \mathbf{1}_{B(t_{\ell(p)},h_p)}(X_{ip}) \right]
$$
$$
\le \sup_{\varphi K \in \mathcal{F}_m \mathcal{H}^m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \frac{(n-m)!}{n!} \sum_{i \in I(m,n)} \frac{1}{m} \sum_{j=1}^m \left[ \frac{\mathbb{C}mM\kappa_2^{m-1}\varepsilon_n}{\tilde{\phi}(\tilde{\mathbf{H}}_n)\tilde{h}_n} \mathbf{1}_{B(t_j,h_j) \cup B(t_{\ell(t_j)},h_j)}(X_{ij}) \right. \\
\left. \times \prod_{q=1}^{j-1} \mathbf{1}_{B(t_q,h_q)}(X_{iq}) \prod_{p=j+1}^{m} \mathbf{1}_{B(t_{\ell(p)},h_p)}(X_{ip}) \right]
$$
$$
\le \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathcal{H}_n^{\mathcal{H}^m}} \sup_{\mathbf{h} \in \mathcal{H}_n^{\mathcal{H}^m}} \frac{(n-m)!}{n!} \sum_{i \in I(m,n)} \frac{1}{m} \sum_{j=1}^{m} \left[ \frac{\mathbb{C}mM\kappa_2^{m-1}\varepsilon_n}{C_1'\phi(\tilde{h}_n)\tilde{h}_n} 1_{B(t_j,h_j) \cup B(t_{\ell(t_j)},h_j)}(X_{ij}) \right. \\
\left. \times \prod_{q=1}^{j-1} 1_{B(t_q,h_q)}(X_{iq}) \prod_{p=j+1}^{m} 1_{B(t_{\ell(p)},h_p)}(X_{ip}) \right],
$$

by (3.2), where $\mathbf{H}_n = (h_{n,1}, \dots, h_{n,m})$ and $\tilde{\mathbf{H}}_n = (\tilde{h}_n, \dots, \tilde{h}_n)$, with $\tilde{\mathbf{H}}_n \le \mathbf{H}_n$ component by component.
The idea is to apply Lemma B.6 on the function

$$
f_{\mathbf{t},\mathbf{h}}(\mathbf{X}_i) = \frac{1}{m} \sum_{j=1}^{m} \left[ \frac{\mathbb{C}mM\kappa_2^{m-1}\varepsilon_n}{C_1'\phi(\tilde{h}_n)h_{n,j}} 1_{B(t_j,h_j) \cup B(t_{\ell(t_j)},h_j)}(X_{ij}) \prod_{q=1}^{j-1} 1_{B(t_q,h_q)}(X_{iq}) \prod_{p=j+1}^{m} 1_{B(t_{\ell(p)},h_p)}(X_{ip}) \right],
$$

which satisfies for all $\mathbf{t} \in S_X^m$:

$$
0 \le \sup_{\mathbf{t} \in S_X^m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} |f_{\mathbf{t},\mathbf{h}}(\mathbf{X})| \le \frac{\mathbb{C}mM\kappa_2^{m-1}\varepsilon_n}{C_1'\phi(\tilde{h}_n)\tilde{h}_n} \le Cte = C_7.
$$

Notice that the existence of the constant $C_7$ on the last right side of the preceding inequality is deduced
from the condition (2.11). Now, we can apply Lemma B.6 with $x = \sqrt{\frac{\psi_{S_X}(\varepsilon_n)}{n\phi(\tilde{h}_n)} - \frac{\varepsilon_n}{\tilde{h}_n}}$, which gives us

$$
\mathbb{P} \left\{ \sup_{\varphi K \in \mathcal{F}_m} \sup_{\mathcal{H}_n^{\mathcal{H}^m}} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} |u_{n,11}(\varphi, \mathbf{t}; \mathbf{h})| \ge \sqrt{\frac{\psi_{S_X}(\varepsilon_n)}{n\phi(\tilde{h}_n)}} \right\} \quad (8.32)
$$

$$
\le \mathbb{P} \left\{ |u_n^{(m)}(f) - \theta| \ge \sqrt{\frac{\psi_{S_X}(\epsilon_n)}{n\phi(\tilde{h}_n)}} - \frac{\epsilon_n}{\tilde{h}_n} + C_{\gamma,m} / \sqrt{n} \right\} \quad (8.33)
$$

$$
\le 2 \exp \left\{ - \frac{C'_{\gamma,m} x^2 n}{C_7^2 + C_7 x (\log n)(\log_2(n))} \right\} \le n^{-c\epsilon_0}, \quad (8.33)
$$

such that $c\epsilon_0 > 1$. By developing the computation while respecting the imposed conditions, mainly
(C.6.) and (C.7.), we get

$$
\sum_{n \ge 1} \mathbb{P} \left\{ \sup_{\varphi K \in \mathcal{F}_m} \sup_{\mathcal{H}_n^{\mathcal{H}^m}} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} |u_{n,11}(\varphi, \mathbf{t}; \mathbf{h})| \ge \sqrt{\frac{\psi_{S_X}(\epsilon_n)}{n\phi(\tilde{h}_n)}} \right\} < \infty. \quad (8.34)
$$

The study of the term $u_{n,12}$ is deduced from the previous one. In fact:

$$
\begin{align}
|u_{n,12}(\varphi, \mathbf{t}; \mathbf{h})| & = |\mathbb{E}[u_n(\varphi, \mathbf{t}; \mathbf{h}) - u_n(\varphi, \mathbf{t}; \mathbf{h})]| \nonumber \\
& \le \frac{\mathbb{C}(n-m)!}{n! \tilde{\phi}(\mathbf{h})} \left| \sum_{i \in I(m,n)} \mathbb{E} \left( \varphi(Y_{i_1}, \dots, Y_{i_m}) \left\{ \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \right\} \right) \right| \tag{8.35}
\end{align}
$$

$$
\le \frac{\mathbb{C}(n-m)!}{n! \tilde{\phi}(\mathbf{h})} \sum_{i \in I(m,n)} \mathbb{E} \left| \varphi(Y_{i_1}, \dots, Y_{i_m}) \left\{ \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_j)}{h_j}\right) - \prod_{j=1}^{m} K\left(\frac{d(X_{ij}, t_{\ell(t_j)})}{h_j}\right) \right\} \right|. \quad (8.36)
$$
To pass from (8.35) to (8.36), we apply Jensen's inequality in connection with some properties of the absolute value function. Then following the same way already taken, we get

$$|u_{n,12}(\varphi, \mathbf{t}; \mathbf{h})| \leq \mathbb{C} |u'_{n,12}(\varphi, \mathbf{t}; \mathbf{h})|,$$

where

$$|u'_{n,12}(\varphi, \mathbf{t}; \mathbf{h})| := \frac{(n-m)!}{n! \tilde{\phi}(\mathbf{h})} \sum_{i \in I(m,n)} \mathbb{E} \left| \varphi(Y_{i_1}, \dots, Y_{i_m}) \sum_{j=1}^{m} \mathbf{K}_{i_j, h_j}^{(\ell)}(t_j, t_{\ell(t_j)}) \right|.$$

Notice that we have

$$
\begin{aligned}
|u'_{n,12}(\varphi, \mathbf{t}; \mathbf{h})| & \le \frac{(n-m)!}{n! \tilde{\phi}(\mathbf{h})} M \kappa_2^{m-1} \sum_{i \in I(m,n)} \sum_{j=1}^{m} \mathbb{E} \left| \left\{ K \left( \frac{d(X_{ij}, t_j)}{h_j} \right) - K \left( \frac{d(X_{ij}, t_{\ell(t_j)})}{h_j} \right) \right\} \right. \\
& \qquad \left. \times 1_{B(t_j, h_j) \cup B(t_{\ell(t_j)}, h_j)}(X_{ij}) \prod_{q=1}^{j-1} 1_{B(t_q, h_q)}(X_{iq}) \prod_{p=j+1}^{m} 1_{B(t_{\ell(p)}, h_p)}(X_{ip}) \right| \\
& \le \frac{(n-m)!}{n!} m M \kappa_2^{m-1} \sum_{i \in I(m,n)} \frac{1}{m} \sum_{j=1}^{m} \left[ \frac{\varepsilon_n}{\tilde{\phi}(\mathbf{H}_n) h_{n,j}} \mathbb{E} \left( 1_{B(t_j, h_j) \cup B(t_{\ell(t_j)}, h_j)}(X_{ij}) \right. \right. \\
& \qquad \left. \left. \times \prod_{q=1}^{j-1} 1_{B(t_q, h_q)}(X_{iq}) \prod_{p=j+1}^{m} 1_{B(t_{\ell(p)}, h_p)}(X_{ip}) \right) \right] \\
& \le \frac{(n-m)!}{n!} m M \kappa_2^{m-1} \sum_{i \in I(m,n)} \frac{1}{m} \sum_{j=1}^{m} \left[ \frac{\varepsilon_n}{C'_1 \tilde{\phi}(\tilde{h}_n) h_{n,j}} \mathbb{E} \left( 1_{B(t_j, h_j) \cup B(t_{\ell(t_j)}, h_j)}(X_{ij}) \right. \right. \\
& \qquad \left. \left. \times \prod_{q=1}^{j-1} 1_{B(t_q, h_q)}(X_{iq}) \prod_{p=j+1}^{m} 1_{B(t_{\ell(p)}, h_p)}(X_{ip}) \right) \right].
\end{aligned}
$$

That implies

$$
\begin{aligned}
\sup_{\varphi K \in \mathcal{F}_{m,\mathcal{H}^m}} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in \mathcal{S}_X^m} |u_{n,12}(\varphi, \mathbf{t}; \mathbf{h})| & \le m M \kappa_2^{m-1} \frac{\log(n)}{\tilde{n}\tilde{h}_n} \le C'_7 \frac{\log(n)}{n\phi(\tilde{h}_n)} \\
& = O\left(\sqrt{\frac{\psi_{S_X}(\varepsilon_n)}{n\phi(\tilde{h}_n)}}\right).
\end{aligned}
$$

This gives that

$$\sup_{\varphi K \in \mathcal{F}_{m,\mathcal{H}^m}} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in \mathcal{S}_X^m} |u_{n,12}(\varphi, \mathbf{t}; \mathbf{h})| = O\left(\sqrt{\frac{\psi_{S_X}(\varepsilon_n)}{n\phi(\tilde{h}_n)}}\right). \quad (8.37)$$

Continue, now with $u_{n,13}$,

$$|u_{n,13}(\varphi, \mathbf{t}; \mathbf{h})| = |u_n(\varphi, \mathbf{t}_\ell; \mathbf{h}) - \mathbb{E}[u_n(\varphi, \mathbf{t}_\ell; \mathbf{h})]|.$$

Supposing that the kernel function $G_{\varphi, t, h}(\cdot)$ is symmetric, we need to decompose our U-statistic according to [115] decomposition, we have

$$u_{n,13}(\varphi, \mathbf{t}; \mathbf{h}) := u_n(\varphi, \mathbf{t}_\ell; \mathbf{h}) - \mathbb{E}[u_n(\varphi, \mathbf{t}_\ell; \mathbf{h})]$$
$$
\begin{align}
&= \sum_{p=1}^{m} \frac{m!}{(m-p)!} u_n^{(p)}(\pi_{p,m}(G_{\varphi, t_l, h})) \nonumber \\
&= m u_n^{(1)}(\pi_{1,m}(G_{\varphi, t_l, h})) + \sum_{p=2}^{m} \frac{m!}{(m-p)!} u_n^{(p)}(\pi_{p,m}(G_{\varphi, t_l, h})). \tag{8.38}
\end{align}
$$

Define new classes of functions, for **h** = (h₁, ..., hₘ) ∈ H<sub>n</sub><sup>(m)</sup>, **l** ∈ {1, ..., N<sub>εn</sub>(S<sub>χ</sub>)}², and 1 ≤ p ≤ m :

$$
(\mathcal{F}_m \mathcal{H}^m)^{(p)} := \{ \phi_{t_l}(\mathbf{h}) \pi_{p,m}(G_{\varphi,t_l,\mathbf{h}})(\cdot, \cdot) \text{ for } \mathbf{h} \in \mathcal{H}_0^{(m)} \text{ and } \varphi \in \mathcal{F}_m \}.
$$

These classes are VC-type classes of functions with the same characteristics and the envelope function
$F_p$ satisfying

$F_p \leq 2^p \kappa_2^m \| F \|_{\infty}.$

Let us start with the linear term of (8.38), which is

$$
mu_n^{(1)}(\pi_{1,m}(G_{\varphi,t_l,h})) = \frac{m}{n} \sum_{j=1}^{m} \pi_{1,m}(G_{\varphi,t_l,h})(X_i, Y_i).
$$

From Hoeffding’s projection, we have

$$
\begin{align*}
\pi_{1,m}(G_{\varphi,t_l,h})(x,y) &= \left\{ \mathbb{E} \left[ G_{\varphi,t_l,h} \left( (x, X_2, \dots, X_m), (y, Y_2, \dots, Y_m) \right) \right] - \mathbb{E} \left[ G_{\varphi,t_l,h}(\mathbf{X}, \mathbf{Y}) \right] \right\} \\
&= \left\{ \mathbb{E} \left[ G_{\varphi,t_l,h}(\mathbf{X}, \mathbf{Y}) \mid (X_1, Y_1) = (x, y) \right] - \mathbb{E} \left[ G_{\varphi,t_l,h}(\mathbf{X}, \mathbf{Y}) \right] \right\}.
\end{align*}
$$

One can see that

$$
mu_n^{(1)}(\pi_{1,m}(G_{\varphi,t_l,h})) := \frac{1}{\sqrt{n}} \alpha_n(S_{1,t_l,h}),
$$

is an empirical process based on a VC-type class of functions contained in $m(\mathcal{F}_m \mathcal{H}^m)^{(1)}$ with the same
characteristics and the elements are defined by:

$$
S_{1,t_l,h}(x,y) = m \phi_{t_l}(h) \mathbb{E}[G_{\varphi,t_l,h}(X,Y) | (X_1, Y_1) = (x,y)].
$$

Hence, the proof of this part is similar to that of the Lemma 8.2 and then:

$$
\sup_{t \in S_X^m} \sup_{\tilde{h}_n \le h_l \le b_0} \sup_{\varphi \in \mathcal{F}_m} \left| u_n^{(1)}(\pi_{1,m}(G_{\varphi,t_l,h})) \right| = O_{a \cdot co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(\tilde{h}_n)}} \right).
$$

Pass now to the nonlinear terms. The purpose is to prove that, for 2 ≤ p ≤ m :

$$
\sup_{\varphi K \in \mathcal{F}_m \mathcal{H}^m} \sup_{\mathbf{h} \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \frac{\binom{m}{p} \sqrt{n\phi(\tilde{h}_n)} |u_n^{(p)}(\pi_{p,m}(G_{\varphi,\mathbf{t},\mathbf{h}}))|}{\sqrt{\psi_{S_X} \left(\frac{\log n}{n}\right)}} = O_{a \cdot c o}(1), \quad (8.39)
$$
to do that, we need to decompose the interval $\prod_{j=1}^{m}(h_{n,j}, h'_{n,j})$ into smaller intervals. First, let us consider the intervals $(h_{n,j}, b_j)$ for all $j = 1, \dots, m$ and $b_j \in (0, 1)$, we note

$$ \mathcal{H}_{ij} = [h_{ij}, h'_{ij}], $$

where $h_{ij} = 2^{i-1}h_{n,j}$ and $h'_{ij} = 2^i h_{n,j}$ and we set $L_j(n) = \max\{i: h'_{ij} \le 2b_j\}$, and $\mathcal{I}_j = \{i_j: 1 \le i_j \le L_j(n)\}$. We can observe that

$$ \prod_{j=1}^{m} (h_{n,j}, b_j) \subseteq \bigcup_{(i_1, \dots, i_m) \in \mathcal{I}_1 \times \dots \times \mathcal{I}_m} \prod_{j=1}^{m} \mathcal{H}_{ij}, $$

and

$$ L_j(n) \sim \frac{\log(b_j/h_{n,j})}{\log(2)} \le L(n) := \max L_j(n) \text{ for } 1 \le j \le m. \quad (8.40) $$

Now, we set the following new classes, for $\mathbf{h} = (h_1, \dots, h_m) \in \mathcal{H}_n^{(m)}$ and $\ell \in \{1, \dots, N_{\varepsilon_n}(S_X)\}^m$, $1 \le j \le m$, $1 \le i \le L(n)$ and $2 \le p \le m$:

$$ (\mathcal{F}_m \mathcal{H}^m)_{i_j, \ell} := \left\{ \phi_{\mathbf{t}_\ell}(\mathbf{h}) G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}(\cdot, \cdot) \text{ where } \varphi \in \mathcal{F}_m \text{ and } \mathbf{h} \in \prod_{j=1}^m \mathcal{H}_{i_j} \right\}, $$

$$ (\mathcal{F}_m \mathcal{H}^m)_{i_j, \ell}^{(p)} := \left\{ \phi_{\mathbf{t}_\ell}(\mathbf{h}) \pi_{p,m}(G_{\varphi, \mathbf{t}_\ell, \mathbf{h}})(\cdot, \cdot) \text{ where } \varphi \in \mathcal{F}_m \text{ and } \mathbf{h} \in \prod_{j=1}^m \mathcal{H}_{i_j} \right\}. $$

Thus, to prove (8.39), we need to prove that for $1 \le j \le m$ and $\ell = (\ell_1, \dots, \ell_m)$:

$$ \max_{1 \le l_j \le N_{\varepsilon_n}(S_X)} \max_{1 \le i \le L(n)} \sup_{\mathbf{h} \in \prod_{j=1}^m \mathcal{H}_{i_j}} \sup_{\varphi \in \mathcal{F}_m \mathcal{H}^m} \sup_{\mathbf{t} \in S_X^m} \frac{\binom{m}{p} \sqrt{n\phi(\tilde{h}_n)} |u_n^{(p)}(\pi_{p,m}(G_{\varphi, \mathbf{t}_\ell, \mathbf{h}})|}{\sqrt{\psi_{S_X} \left(\frac{\log n}{n}\right)}} = O_{a,co}(1). $$

Notice that for each $\epsilon_0$, $1 \le j \le m$ and $\ell = (\ell_1, \dots, \ell_m)$, we have

$$ \mathbb{P} \left\{ \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} \sup_{\varphi \in \mathcal{F}_m \mathcal{H}^m} \frac{\binom{m}{p} \sqrt{n\phi(\tilde{h}_n)} |u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}})|}{\sqrt{\psi_{S_X} \left(\frac{\log n}{n}\right)}} \ge \epsilon_0 \right\} \\ \le \sum_{1 \le l_j \le N_{\varepsilon_n}(S_X)} \sum_{1 \le i \le L(n)} \mathbb{P} \left\{ \sup_{\mathbf{h} \in \prod_{j=1}^m \mathcal{H}_{i_j}} \sup_{\varphi \in \mathcal{F}_m \mathcal{H}^m} \frac{\binom{m}{p} \sqrt{n\phi(\tilde{h}_n)} |u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}})|}{\sqrt{\psi_{S_X} \left(\frac{\log n}{n}\right)}} \ge \epsilon_0 \right\} $$
$$
\le L(n) N_{\epsilon_n}^m(S_\chi) \max_{1 \le l_j \le N_{\epsilon_n}(S_\chi)} \max_{1 \le i \le L(n)} \\
\times \mathbb{P} \left\{ \left\| \sum_{p=2}^m \binom{m}{p} u_n^{(p)}(\pi_{p,m} G_{\varphi, \mathbf{t}_l(t), \mathbf{h}}) \right\|_{(\mathcal{F}_m, \mathcal{H}^m)_{i,j,l}} \ge \epsilon_0 \sqrt{\frac{\psi_{S_\chi}\left(\frac{\log n}{n}\right)}{n\phi(\tilde{h}_n)}} \right\}.
$$

At this stage, we will focus on studying the above equation for $m = 2$ to simplify the proof (the same steps remain valid for $m > 2$). We have

$$
u_n^{(2)}(\pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}) = \frac{1}{n(n-1)} \sum_{1 \le i < j \le n} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, Y_i), (X_j, Y_j)),
$$

and

$$
\begin{align}
\sum_{i \neq j} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j), (Y_i, Y_j))
&= \sum_{p \neq q}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \nonumber \\
&\quad + \sum_{p=1}^{v_n} \sum_{i \neq j, i, j \in H_p^{(U)}} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \nonumber \\
&\quad + 2 \sum_{p=1}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{q: |q-p| \ge 2} \sum_{j \in T_q^{(U)}} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \nonumber \\
&\quad + 2 \sum_{p=1}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{q: |q-p| \le 1} \sum_{j \in T_q^{(U)}} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \nonumber \\
&\quad + \sum_{p \neq q}^{v_n} \sum_{i \in T_p^{(U)}} \sum_{j \in T_q^{(U)}} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \nonumber \\
&\quad + \sum_{p=1}^{v_n} \sum_{i \neq j, i, j \in T_p^{(U)}} \pi_{2,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \tag{8.41} \\
&=: I + \mathbb{II} + \mathbb{III} + \mathbb{IV} + \mathbb{V} + \mathbb{VI}. \nonumber
\end{align}
$$

For $m=3$ the formula becomes cumbersome and given by

$$
\begin{align*}
& \sum_{i \neq j \neq k}^{n} \pi_{\ell,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&= \sum_{p_1 \neq p_2 \neq p_3}^{v_n} \sum_{i \in H_{p_1}^{(U)}} \sum_{j \in H_{p_2}^{(U)}} \sum_{k \in H_{p_3}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&\quad + 3 \sum_{p_1 \neq p_2}^{v_n} \sum_{i \neq j, i, j \in H_{p_1}^{(U)}} \sum_{k \in H_{p_2}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_l, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k))
\end{align*}
$$
$$
\begin{aligned}
&+ \sum_{p_1=1}^{v_n} \sum_{i \neq j \neq k, i,j,k \in H_{p_1}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1 \neq p_2}^{v_n} \sum_{i \in H_{p_1}^{(U)}} \sum_{j \in H_{p_2}^{(U)}} \sum_{p_3: |p_3-p_1| \ge 2, i=1,2}^{v_n} \sum_{k \in T_{p_3}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1 \neq p_2}^{v_n} \sum_{i \in H_{p_1}^{(U)}} \sum_{j \in H_{p_2}^{(U)}} \sum_{p_3: |p_3-p_1| \ge 2, |p_3-p_2| \le 1}^{v_n} \sum_{k \in T_{p_3}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1=1}^{v_n} \sum_{i \in H_{p_1}^{(U)}} \sum_{p_3: |p_1-p_1| \ge 2, i=2,3}^{v_n} \sum_{j \in T_{p_2}^{(U)}} \sum_{k \in T_{p_3}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1=1}^{v_n} \sum_{i \in H_{p_1}^{(U)}} \sum_{p_3: |p_3-p_1| \ge 2, |p_2-p_1| \le 1}^{v_n} \sum_{j \in T_{p_2}^{(U)}} \sum_{k \in T_{p_3}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1=1}^{v_n} \sum_{i,j \in H_{p_1}^{(U)}} \sum_{p_2: |p_2-p_1| \ge 2}^{v_n} \sum_{k \in T_{p_2}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1=1}^{v_n} \sum_{i \in H_{p_1}^{(U)}} \sum_{p_2: |p_2-p_1| \ge 2}^{v_n} \sum_{j,k \in T_{p_2}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1=1}^{v_n} \sum_{i,j \in H_{p_1}^{(U)}} \sum_{p_2: |p_2-p_1| \le 1}^{v_n} \sum_{k \in T_{p_2}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+ \sum_{p_1 \neq p_2 \neq p_3}^{v_n} \sum_{i \in T_{p_1}^{(U)}} \sum_{j \in T_{p_2}^{(U)}} \sum_{k \in T_{p_3}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+3 \sum_{p_1 \neq p_2}^{v_n} \sum_{i \in T_{p_1}^{(U)}} \sum_{j,k \in T_{p_2}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)) \\
&+ \sum_{p_1=1}^{v_n} \sum_{i \neq j \neq k, i,j,k \in T_{p_1}^{(U)}} \pi_{\ell,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j, X_k), (Y_i, Y_j, Y_k)).
\end{aligned}
$$

Let us start by considering the term $\mathbb{I}$. Suppose that the sequence of independent blocks $\{\xi_i = (\zeta_i, \bar{\zeta}_i)\}_{i \in \mathbb{N}^*}$ is of size $a_n$. An application of (A.1), shows that

$$
P \left\{ \left\| \frac{1}{n(n-1)} \sum_{p \neq q}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} \pi_{2,m} G_{\varphi, \mathbf{t}_\ell, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \right\|_{(\mathcal{F}_2, \mathcal{H}^2)_{i,j,\ell}} \ge \epsilon_0 \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(\tilde{h}_n)}} \right\}
$$
$$
\le \mathbb{P} \left\{ \left\| \frac{1}{n(n-1)} \sum_{p \ne q}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} \pi_{2,m} G_{\varphi, t, h}((\varsigma_i, \varsigma_j), (\zeta_i, \zeta_j)) \right\|_{(\mathcal{F}_2, \mathcal{H}^2)_{i,j,l}} \ge \epsilon_0 \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(\tilde{h}_n)}} \right\} \\
+ 2v_n \beta_{b_n}.
$$

We keep the choice of $v_n$ and $b_n$ such that $v_n b_n^r \le 1$, which implies that $2v_n \beta_{b_n} \to 0$ as $n \to \infty$, so the term to consider is the second summand. the key idea is to apply Lemma B.4. We see clearly that the class of functions $(\mathcal{F}_m \mathcal{H}^m)_{i,j,l}$ is uniformly bounded, i.e.,

$$
\sup_{\varphi \tilde{K} \in (\mathcal{F}_2, \mathcal{H}^2)_{i,j,l}} |\varphi(\cdot) \tilde{K}(\cdot)| \le M \kappa_2^2.
$$

Moreover, by applying Proposition 2.6 of [10] we have for each $(x_i, y_i) \in S_X \times \mathcal{Y}$ and Rademacher variables $\epsilon_i$:

$$
\begin{align}
& \mathbb{E} \left\| n^{-1} \sum_{p \ne q}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} G_{\varphi, t, h}((\varsigma_i, \varsigma_j), (\zeta_i, \zeta_j)) \right\|_{(\mathcal{F}_m, \mathcal{H}^m)_{i,j,l}} \nonumber \\
& \le c_2 \mathbb{E} \left\| n^{-1} \sum_{p \ne q}^{v_n} \epsilon_p \epsilon_q \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} G_{\varphi, t, h}((\varsigma_i, \varsigma_j), (\zeta_i, \zeta_j)) \right\|_{(\mathcal{F}_m, \mathcal{H}^m)_{i,j,l}} \nonumber \\
& \le c_2 \mathbb{E} \int_0^{D_{2,n}} \log N \left( \epsilon, (\mathcal{F}_2, \mathcal{H}^2)_{i,j,l}, d_{2,n} \right) d\epsilon, \tag{8.42}
\end{align}
$$

where

$$
D_{2,n} := \left\| \mathbb{E}_{\epsilon}^{1/2} \left\{ n^{-2} \sum_{p \ne q}^{v_n} \left( \epsilon_p \epsilon_q \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} G_{\varphi, t, h}((\varsigma_i, \varsigma_j), (\zeta_i, \zeta_j)) \right)^2 \right\} \right\|_{(\mathcal{F}_m, \mathcal{H}^m)_{i,j,l}}
$$

and

$$
d_{2,n} := \mathbb{E}_{\epsilon}^{1/2} \left\{ n^{-2} \sum_{p \ne q}^{v_n} \left( \epsilon_p \epsilon_q \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} G_{\varphi, t, h}((\varsigma_i, \varsigma_j), (\zeta_i, \zeta_j)) \right)^2 \right\}.
$$

We see that

$$
D_{2,n} \le n^{-1} \frac{n!}{(n-2)!} M \kappa_2^2 \le n M \kappa_2^2.
$$

So, using the fact that $\mathcal{F}_2\mathcal{H}^2$ is a VC-type class of functions satisfying (C.4.4) which implies that the class $(\mathcal{F}_2\mathcal{H}^2)_{i,j,l}$ is also a VC-type class of functions with the same characteristics as $\mathcal{F}_2\mathcal{H}^2$, then,

$$
\mathbb{E} \left\| n^{-1} \sum_{p \ne q}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} G_{\varphi, t, h}((\varsigma_i, \varsigma_j), (\zeta_i, \zeta_j)) \right\|_{(\mathcal{F}_2, \mathcal{H}^2)_{i,j,l}}
$$
$$ \begin{aligned} & \le c_2 \mathbb{E} \int_0^{D_{2,n}} \log N \left( \epsilon, \left( \mathcal{F}_2 \mathcal{H}^2 \right)_{i_j, \ell}, d_{2,n} \right) d\epsilon \\ & \le c_2 n MC^2. \end{aligned} \quad (8.43) $$

All the conditions of Lemma B.4 are fulfilled, so a direct application gives for each $\epsilon_0$,

$$ \begin{aligned} & \mathbb{P} \left\{ \left\| \frac{1}{n(n-1)} \sum_{p \ne q}^{v_n} \sum_{i \in H_p^{(U)}} \sum_{j \in H_q^{(U)}} \pi_{2,m} G_{\varphi, t_l, h} \left( (X_i, X_j), (Y_i, Y_j) \right) \right\|_{(\mathcal{F}_2 \mathcal{H}^2)_{i_j, \ell}} \ge \epsilon_0 \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right\} \\ & \le 2 \exp \left( - \frac{\epsilon_0 \sqrt{\frac{2\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}}}{2^7 2^3 c_2 M^2 \kappa_2^4} \right) \\ & \le n^{-\epsilon_0 C'_2}, \end{aligned} \quad (8.44) $$

where

$$ \epsilon_n = \frac{\log n}{n}, \quad C'_2 > 0. $$

Next, let us study the same blocks $\mathbb{I}$, we have

$$ \begin{aligned} & \mathbb{P} \left\{ \left\| \frac{1}{n(n-1)} \sum_{p=1}^{v_n} \sum_{i \ne j, i, j \in H_p^{(U)}} \pi_{2,m} G_{\varphi, t_l, h} \left( (X_i, X_j), (Y_i, Y_j) \right) \right\|_{(\mathcal{F}_2 \mathcal{H}^2)_{i_j, \ell}} \ge \epsilon_0 \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right\} \\ & \le \mathbb{P} \left\{ \left\| \frac{1}{n(n-1)} \sum_{p=1}^{v_n} \sum_{i \ne j, i, j \in H_p^{(U)}} \pi_{2,m} G_{\varphi, t_l, h} \left( (X_i, X_j), (Y_i, Y_j) \right) \right\|_{(\mathcal{F}_2 \mathcal{H}^2)_{i_j, \ell}} \ge \epsilon_0 \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right\} \\ & \qquad + 2\nu_n \beta_{b_n}. \end{aligned} $$

Following the same argument as the blocks $\mathbb{I}$, we obtain

$$ \begin{aligned} & \mathbb{E} \left\| n^{-1} \sum_{p=1}^{v_n} \sum_{i \ne j, i, j \in H_p^{(U)}} G_{\varphi, t_l, h} \left( (X_i, X_j), (Y_i, Y_j) \right) \right\|_{(\mathcal{F}_2 \mathcal{H}^2)_{i_j, \ell}} \\ & \le c_2 \mathbb{E} \left\| n^{-1} \sum_{p=1}^{v_n} \epsilon_p \sum_{i \ne j, i, j \in H_p^{(U)}} G_{\varphi, t_l, h} \left( (X_i, X_j), (Y_i, Y_j) \right) \right\|_{(\mathcal{F}_m \mathcal{H}^m)_{i_j, \ell}} \end{aligned} $$
$$ \begin{aligned} & \le c_2 \mathbb{E} \int_0^{D_{2,n}^{(2)}} \log N \left( \epsilon, \left( \mathcal{F}_2 \mathcal{H}^2 \right)_{i,j,\ell}, d_{2,n}^{(2)} \right) d\epsilon \\ & \le c_2 n MC'^2, \end{aligned} \quad (8.45) $$

where

$$ D_{2,n}^{(2)} := \left\| \mathbb{E}_{\epsilon}^{1/2} \left\{ n^{-2} \left( \sum_{p=1}^{\nu_n} \epsilon_p \sum_{i \neq j, i, j \in H_p^{(U)}} G_{\varphi, t, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \right)^2 \right\} \right\|_{(\mathcal{F}_m \mathcal{H}^m)_{i,j,\ell}} $$

and

$$ d_{2,n}^{(2)} := \mathbb{E}_{\epsilon}^{1/2} \left\{ n^{-2} \left( \sum_{p=1}^{\nu_n} \epsilon_p \sum_{i \neq j, i, j \in H_p^{(U)}} G_{\varphi, t, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \right)^2 \right\}. $$

Again, using Lemma B.4, we readily obtain

$$ \begin{aligned} & \mathbb{P} \left\{ \left\| \frac{1}{n(n-1)} \sum_{p=1}^{\nu_n} \sum_{i \neq j, i, j \in H_p^{(U)}} \pi_{2,m} G_{\varphi, t, \mathbf{h}}((X_i, X_j), (Y_i, Y_j)) \right\|_{(\mathcal{F}_2 \mathcal{H}^2)_{i,j,\ell}} \ge \epsilon_0 \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(\tilde{h}_n)}} \right\} \\ & \le 2 \exp \left( - \frac{\epsilon_0 \sqrt{\frac{2 \psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(\tilde{h}_n)}}}{2^7 2^3 c_2 M^2 \kappa_2^4} \right) \\ & \le n^{-\epsilon_0 C'_3}. \end{aligned} \quad (8.46) $$

The results for the remaining blocks can be obtained by following the same strategy above. Consequently, we have

$$ \begin{aligned} & \sum_{n \ge 1} \mathbb{P} \left\{ \sup_{\mathbf{h} \in H_0^{(m)}} \sup_{t \in S_X^m} \sup_{\varphi, \tilde{K} \in \mathcal{F}_m \mathcal{H}^m} \frac{\binom{m}{p} \sqrt{n \phi(\tilde{h}_n)} |u_n^{(p)}(\pi_{p,m} G_{\varphi, t, \mathbf{h}})|}{\sqrt{\psi_{S_X} \left( \frac{\log n}{n} \right)}} \ge \epsilon_0 \right\} \\ & \le \sum_{n \ge 1} L(n) n^{-\epsilon_0 C'_m} \\ & \le \infty, \end{aligned} $$

where

$$ \epsilon_0 = \frac{\log n}{n}, \quad C'_m > 0. $$

This completes the proof of the theorem.
### Proof of Theorem 3.10:

Notice that

$$ 
\begin{aligned}
& \left| \hat{\mathbf{r}}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}) - \hat{\mathbb{E}}(\hat{\mathbf{r}}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h})) \right| \\
& = \left| \frac{u_n(\varphi, \mathbf{t}, \mathbf{h})}{u_n(1, \mathbf{t}, \mathbf{h})} - \frac{\hat{\mathbb{E}}(u_n(\varphi, \mathbf{t}, \mathbf{h}))}{\hat{\mathbb{E}}(u_n(1, \mathbf{t}, \mathbf{h}))} \right| \\
& \le \frac{|u_n(\varphi, \mathbf{t}, \mathbf{h}) - \hat{\mathbb{E}}(u_n(\varphi, \mathbf{t}, \mathbf{h}))|}{|u_n(1, \mathbf{t}, \mathbf{h})|} \\
& \qquad + \frac{|\hat{\mathbb{E}}(u_n(\varphi, \mathbf{t}, \mathbf{h}))| \cdot |u_n(1, \mathbf{t}, \mathbf{h}) - \hat{\mathbb{E}}(u_n(1, \mathbf{t}, \mathbf{h}))|}{|u_n(1, \mathbf{t}, \mathbf{h})| \cdot |\hat{\mathbb{E}}(u_n(1, \mathbf{t}, \mathbf{h}))|} \\
& =: \mathbf{I} + \mathbf{II}.
\end{aligned}
$$

Under the imposed hypothesis and the previously obtained results, and for some $c'_1, c'_2$, we get that

$$ 
\begin{aligned}
& \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} |u_n(1, \mathbf{t}, \mathbf{h})| = c'_1 \quad \text{a.co} \\
& \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} |\hat{\mathbb{E}}(u_n(1, \mathbf{t}, \mathbf{h}))| = c'_2 \\
& \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} |\hat{\mathbb{E}}(u_n(\varphi, \mathbf{t}, \mathbf{h}))| = O(1).
\end{aligned}
$$

Therefore, we can now apply Theorem 3.8 to handle **II** and Theorems 3.8 and 3.9 to handle **I**, depending on whether the class $\mathcal{F}_m$ satisfies (C.4.1) or (C.4.2), we get, for some $c'' > 0$ with probability 1:

$$ 
\begin{aligned}
& \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} \frac{\sqrt{n\phi(\tilde{h}_n)} |\hat{\mathbf{r}}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}) - \hat{\mathbb{E}}(\hat{\mathbf{r}}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{h}))|}{\sqrt{\psi_{S_X}(\varepsilon_n)}} \\
& \le \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{h} \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} \frac{\sqrt{n\phi(\tilde{h}_n)} (\mathbf{I})}{\sqrt{\psi_{S_X}(\varepsilon_n)}} \\
& \qquad + \sup_{\varphi \in \mathcal{F}_m} \sup_{\mathbf{h}(\mathbf{t}) \in \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} \frac{\sqrt{n\phi(\tilde{h}_n)} (\mathbf{II})}{\sqrt{\psi_{S_X}(\varepsilon_n)}} \\
& \le c''.
\end{aligned}
$$

Hence, the proof is complete.

### Proof of Theorem 3.11:

Under the conditions (C.3.1), we have

$$ 
\begin{aligned}
& \left| \frac{\hat{\mathbb{E}}[u_n(\varphi, \mathbf{t}; \mathbf{h})]}{\hat{\mathbb{E}}[u_n(1, \mathbf{t}; \mathbf{h})]} - r^{(m)}(\varphi, \mathbf{t}) \right| \\
& \le |\kappa_1^{-m} C_1^{-1} \hat{\mathbb{E}}[u_n(\varphi, \mathbf{t}; \mathbf{h})] - r^{(m)}(\varphi, \mathbf{t})|
\end{aligned}
$$
$$
\begin{align*}
&\leq \left| \kappa_1^{-m} C_1^{-1} \mathbb{E} \left( \frac{(n-m)!}{n!} \sum_{i \in I(m,n)} \frac{1}{\phi_t(\mathbf{h})} \varphi(Y_{i_1}, \dots, Y_{i_m}) \prod_{j=1}^{m} K\left(\frac{d(X_{i_j}, t_j)}{h_j}\right) \right) \right. \\
&\qquad \left. - r^{(m)}(\varphi, \mathbf{t}) \right| \\
&\leq \left| \kappa_1^{-m} C_1^{-1} \frac{1}{\phi_t(\mathbf{h})} \mathbb{E} \left( \varphi(Y_1, \dots, Y_m) \prod_{i=1}^{m} K\left(\frac{d(X_i, t_i)}{h_i}\right) \right) - r^{(m)}(\varphi, \mathbf{t}) \right| \\
&\leq \frac{1}{\phi_t(\mathbf{h}) \kappa_1^m C_1} \mathbb{E} \left( \prod_{i=1}^{m} K\left(\frac{d(X_i, t_i)}{h_i}\right) \left| r^{(m)}(\varphi, \mathbf{X}) - r^{(m)}(\varphi, \mathbf{t}) \right| \right).
\end{align*}
$$

Taking in consideration the hypotheses (H.1), (C.2.1), (C.3.1) and (C.6.), we get $\forall \mathbf{h} = (h_1, \dots, h_m) \in \prod_{j=1}^{m} (h_{n,j}, b_j)$ and $\forall \mathbf{t} \in S_{\chi}^m$:

$$
\begin{align*}
|\kappa_1^{-m} C_1^{-1} \mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h})) - r^{(m)}(\varphi, \mathbf{t})| &\le \frac{C_3}{\phi_t(\mathbf{h}) \kappa_1^m C_1} \mathbb{E} \left( \prod_{i=1}^m K \left( \frac{d(X_i, t_i)}{h_i} \right) d_{X^m}^\gamma (\mathbf{X}, \mathbf{t}) \right) \\
&\le \frac{C_3}{m} (d(X_1, t_1) + \dots + d(X_m, t_m))^\gamma \\
&\le \frac{C_3}{m} (h_1 + \dots + h_m)^\gamma \\
&= \frac{C_3}{m} (b_1 + \dots + b_m)^\gamma \le C_3 \tilde{b}_0^\gamma,
\end{align*}
$$

where

$$
\tilde{b}_0 := \max_{1 \le j \le m} b_j.
$$

This completes the proof of the theorem.

Proof of Corollary 3.18:

In this section, we will prove Corollary 3.18 using Lemma 8.1. Following the same reasoning as
the case of the functional regression, we use the notation: $S_{\Omega} = S_{\mathcal{X}}$, $\mathbf{A}_i = \mathbf{X}_i$, $\varphi(\mathbf{B}_i) = \varphi(\mathbf{Y}_i)$,

$$
\prod_{i=1}^{m} G(H_i, (t, A_i)) = \prod_{i=1}^{m} K(H_i^{-1} d(t, A_i)),
$$

$$
D_{n,k}(t_j) = H_{n,k}(t_j), \quad \forall \mathbf{t} = (t_1, \dots, t_m) \in S_{\chi}^{m}, \text{ and } j = 1, \dots, m,
$$

$$
\mathfrak{M}_n^{(m)}(\varphi, t; \mathbf{h}_{n,k}(\mathbf{t})) = \widehat{r}_n^{*(m)}(\varphi, t; \mathbf{h}_{n,k}(\mathbf{t})),
$$

$$
\mathfrak{M}(\varphi, t) = r^{(m)}(\varphi, t).
$$

Choosing $D_{n,k}^{-}(\xi_n, t)$ and $D_{n,k}^{+}(\xi_n, t)$ such that for all $j = 1, \dots, m$,

$$
\phi_{t_j}(D_{n,k}^{-}(\xi_n, t_j)) = \frac{\sqrt{\xi_{n,j}k}}{n},
$$
$$ \phi_{t_j}(D_{n,k}^+(\xi_n, t_j)) = \frac{k}{n\sqrt{\xi_{n,j}}}, $$

where $\xi_{n,j}$ are increasing sequences that belong to $(0, 1)$, and

$$ \xi_n = \prod_{j=1}^{m} \xi_{n,j}. $$

We denote $\mathbf{h}^{-}(\mathbf{t}) = (h_{1}^{-}(t_{1}), \dots, h_{m}^{-}(t_{m}))$ and $\mathbf{h}^{+}(\mathbf{t}) = (h_{1}^{+}(t_{1}), \dots, h_{m}^{+}(t_{m}))$, where

$$ h_j^{-}(t_j) = D_{n,k}^{-}(\xi_n, t_j) \text{ and } h_j^{+}(t_j) = D_{n,k}^{+}(\xi_n, t_j), \text{ for all } j = 1, \dots, m. $$

We can easily see that, for all $j = 1, \dots, m$:

$$ \phi_{t_j}^{-1} \left( \frac{\sqrt{\xi_{n,j}} k_{1,n}}{n} \right) \le h^{-}(t_j) \le \phi_{t_j}^{-1} \left( \frac{\sqrt{\xi_{n,j}} k_{2,n}}{n} \right), \quad (8.47) $$

$$ \phi_{t_j}^{-1} \left( \frac{k_{1,n}}{n \sqrt{\xi_{n,j}}} \right) \le h^{+}(t_j) \le \phi_{t_j}^{-1} \left( \frac{k_{2,n}}{n \sqrt{\xi_{n,j}}} \right). \quad (8.48) $$

Using the condition (2.12) one gets, for all $j = 1, \dots, m$, there exist constants $0 < \mu_j \le \nu_j < \infty$, such that

$$ \mu_j \phi^{-1} \left( \frac{\rho_{n,j} k_{1,n}}{n} \right) \le \phi_{t_j}^{-1} \left( \frac{\rho_{n,j} k_{1,n}}{n} \right) \text{ and } \phi_{t_j}^{-1} \left( \frac{k_{2,n}}{\rho_{n,j} n} \right) \le \nu_j \phi^{-1} \left( \frac{k_{2,n}}{\rho_{n,j} n} \right), $$

we put $\rho_{n,j} = \sqrt{\xi_{n,j}}$, $h_{n,j} = \mu_j \phi^{-1} \left( \frac{\rho_{n,j} k_{1,n}}{n} \right)$ and $h'_{n,j} = \nu_j \phi^{-1} \left( \frac{k_{2,n}}{\rho_{n,j} n} \right)$, thus $\mathbf{h}^{-}(\mathbf{t})$ and $\mathbf{h}^{+}(\mathbf{t})$ belong to the interval

$$ \mathcal{H}_n^{(m)} := \prod_{j=1}^{m} (h_{n,j}, h'_{n,j}). $$

We denote $\tilde{h}_n = \min_{1 \le j \le m} h_{n,j}$ and $\tilde{h}'_n = \max_{1 \le j \le m} h'_{n,j}$, therefore,

$$ h_j(t_j) \in (\tilde{h}_n, \tilde{h}'_n), \forall j = 1, \dots, m. $$

We also note for all $\mathbf{b} = (b_1, \dots, b_m) \in (0, 1)^m$:

$$ \mathcal{H}_0^{(m)} := \prod_{j=1}^{m} (h_{n,j}, b_j), $$

and

$$ \tilde{b}_0 := \max_{1 \le j \le m} b_j. $$

Finally, we can choose constants $0 < \mu^* < \nu^* < \infty$ and a sequence $\{\rho_n^*\} \in (0, 1)$, while respecting the condition (C.8), in a way that makes $\tilde{h}_n = \mu^* \phi^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right)$, $\tilde{h}'_n = \nu^* \phi^{-1} \left( \frac{k_{2,n}}{\rho_n^* n} \right)$ and

$$ \mathcal{U}_n = \phi^{-1} \left( \frac{k_{2,n}}{\rho_n^* n} \right)^\gamma + \sqrt{\frac{\psi_{S,N} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu^* \phi^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) \right)}} $$
It is clear that ($L_0$) is satisfied due to the condition (C.3.1), and from (8.47) and (8.48), we can easily verify that the construction of $\mathbf{h}^{-}(\mathbf{t})$ and $\mathbf{h}^{+}(\mathbf{t})$ satisfies the condition ($L_1$).

### Checking the conditions ($L_4$) and ($L_5$)

A direct application of Corollary 3.12 gives

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{\mathbf{h}^{-}(\mathbf{t}) \in \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \mathfrak{M}_n(\varphi, \mathbf{t}; \mathbf{D}_{n,k}^{-}(\xi_n, \mathbf{t})) - \mathfrak{M}(\varphi, \mathbf{t}) \right| \\ = O(\tilde{h}'_n^\gamma) + O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(\tilde{h}_n)}} \right), $$

which is equivalent to

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_1, n \le k \le k_2, n} \sup_{\mathbf{t} \in S_X^m} \left| \mathfrak{M}_n(\varphi, \mathbf{t}; \mathbf{D}_{n,k}^{-}(\xi_n, \mathbf{t})) - \mathfrak{M}(\varphi, \mathbf{t}) \right| \\ = O_{a.co} \left\{ \phi^{-1} \left( \frac{k_{2,n}}{\rho_n^* n} \right)^\gamma + \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu^* \phi_t^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) \right)}} \right\}. \quad (8.49) $$

Applying the same reasoning with $\mathbf{h}^{+}(\mathbf{t})$, we obtain

$$ \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_1, n \le k \le k_2, n} \sup_{\mathbf{t} \in S_X^m} \left| \mathfrak{M}_n(\varphi, \mathbf{t}; \mathbf{D}_{n,k}^{+}(\xi_n, \mathbf{t})) - \mathfrak{M}(\varphi, \mathbf{t}) \right| \\ = O_{a.co} \left\{ \phi^{-1} \left( \frac{k_{2,n}}{\rho_n^* n} \right)^\gamma + \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu^* \phi_t^{-1} \left( \frac{\rho_n^* k_{1,n}}{n} \right) \right)}} \right\}. \quad (8.50) $$

Thus, the conditions ($L_4$) and ($L_5$) are checked.

### Checking the condition ($L_2$)

To check ($L_2$) we show that for all $\varepsilon_0 > 0$ and $\mathbf{t} = (t_1, \dots, t_m) \in S_X^m$:

$$ \sum_{n \ge 1} \mathbb{P} \left\{ \left| \prod_{j=1}^{m} 1 \left\{ D_{n,k}^{-}(\xi_n, t_j) \le H_{n,k}(t_j) \le D_{n,k}^{+}(\xi_n, t_j) \right\} - 1 \right| > \varepsilon_0 \right\} < \infty. \quad (8.51) $$

We have

$$ \mathbb{P} \left\{ \left| \prod_{j=1}^{m} 1 \left\{ D_{n,k}^{-}(\xi_n, t_j) \le H_{n,k}(t_j) \le D_{n,k}^{+}(\xi_n, t_j) \right\} - 1 \right| > \varepsilon_0 \right\} $$
$$
\begin{align*}
&= \sum_{j=1}^{m} \mathbb{P} \{ H_{n,k}(t_j) \notin (D_{n,k}^{-}(\xi_n, t_j), D_{n,k}^{+}(\xi_n, t_j)) \} \\
&\le \sum_{j=1}^{m} \mathbb{P} \left( H_{n,k}(t_j) \le \phi_{t_j}^{-1} \left( \frac{\sqrt{\xi_{n,j}} k_{1,n}}{n} \right) \right) + \sum_{j=1}^{m} \mathbb{P} \left( H_{n,k}(t_j) \ge \phi_{t_j}^{-1} \left( \frac{k_{2,n}}{\sqrt{\xi_{n,j}} n} \right) \right) \\
&\le \sum_{\ell=1}^{N_{\varepsilon}(S, \chi)} \sum_{k=k_{1,n}}^{k_{2,n}} \sum_{j=1}^{m} \mathbb{P} \left( H_{n,k}(t_{\ell}(t_j)) \le \phi_{t_{\ell}(t_j)}^{-1} \left( \frac{\sqrt{\xi_{n,j}} k_{1,n}}{n} \right) \right) \\
&\quad + \sum_{\ell=1}^{N_{\varepsilon}(S, \chi)} \sum_{k=k_{1,n}}^{k_{2,n}} \sum_{j=1}^{m} \mathbb{P} \left( H_{n,k}(t_{\ell}(t_j)) \ge \phi_{t_{\ell}(t_j)}^{-1} \left( \frac{k_{2,n}}{\sqrt{\xi_{n,j}} n} \right) \right),
\end{align*}
$$

now, using

$$
\mathbb{P} \left\{ H_{n,k}(t_{\ell(t_j)}) \le \phi_{t_{\ell(t_j)}}^{-1} \left( \frac{\alpha k_{1,n}}{n} \right) \right\} \le \exp \left\{ -\frac{k - \alpha k_{1,n}}{4} \right\}, \quad (8.52)
$$

and

$$
\mathbb{P} \left\{ H_{n,k}(t_{\ell(t_j)}) \ge \phi_{t_{\ell(t_j)}}^{-1} \left( \frac{k_{2,n}}{\alpha n} \right) \right\} \le \exp \left\{ - \frac{(k_{2,n} - \alpha k)^2}{2\alpha k_{2,n}} \right\}. \quad (8.53)
$$

Consequently, we obtain

$$
\begin{align}
& \sum_{\ell=1}^{N_{\varepsilon}(S, \chi)} \sum_{k=k_{1,n}}^{k_{2,n}} \sum_{j=1}^{m} \mathbb{P} \left\{ H_{n,k}(t_{\ell(t_j)}) \le \phi_{t_{\ell(t_j)}}^{-1} \left( \frac{\alpha k_{1,n}}{n} \right) \right\} \nonumber \\
& \le m N_{\varepsilon_n}(S, \chi) k_{2,n} \exp \left\{ -(1-\alpha)k_{1,n}/4 \right\} \nonumber \\
& \le m N_{\varepsilon_n}(S, \chi) n^{1-\left\{(1-\alpha)/4\right\} \frac{k_{1,n}}{\ln n}} < \infty, \tag{8.54}
\end{align}
$$

and

$$
\begin{align}
& \sum_{\ell=1}^{N_{\varepsilon}(S, \chi)} \sum_{k=k_{1,n}}^{k_{2,n}} \sum_{j=1}^{m} \mathbb{P} \left( H_{n,k}(t_{\ell(t_j)}) \ge \phi_{t_{\ell(t_j)}}^{-1} \left( \frac{k_{2,n}}{\sqrt{\xi_{n,j} n}} \right) \right) \nonumber \\
& \le N_{\varepsilon_n}(S, \chi) k_{2,n} \exp \left\{ -(1-\alpha)k_{1,n}/2\alpha \right\} \nonumber \\
& \le N_{\varepsilon_n}(S, \chi) n^{1-\left\{(1-\alpha)/2\alpha\right\} \frac{k_{2,n}}{\ln n}} < \infty. \tag{8.55}
\end{align}
$$

Thus (L₂) is checked.

Checking the condition (L₃)

Notice that

$$
\left| \mathfrak{M}_{n-}^{(m)}(\varphi, \mathbf{t}; \xi_n) - \xi_n \mathfrak{M}^{(m)}(\varphi, \mathbf{t}) \right|
= \left| \frac{\sum_{i \in I(m,n)} \varphi(Y_{i_1}, \dots, Y_{i_m}) \prod_{j=1}^{m} K\left(\frac{d(X_{i_j}, t_j)}{h_j^-(t_j)}\right)}{\sum_{i \in I(m,n)} \prod_{j=1}^{m} K\left(\frac{d(X_{i_j}, t_j)}{h_j^+(t_j)}\right)} - \frac{\phi_t(\mathbf{h}^-(\mathbf{t}))}{\phi_t(\mathbf{h}^+(\mathbf{t}))} r^{(m)}(\varphi, \mathbf{t}) \right|
$$
$$
\begin{align}
&= \frac{\phi_t(\mathbf{h}^{-}(\mathbf{t}))}{\phi_t(\mathbf{h}^{+}(\mathbf{t}))} \left| \frac{\sum_{i \in I(m,n)} \phi(Y_{i_1}, \dots, Y_{i_m}) \prod_{j=1}^{m} K\left(\frac{d(X_{i_j}, t_j)}{h_j^{-}(t_j)}\right)}{\phi_t(\mathbf{h}^{-}(\mathbf{t})) \sum_{i \in I(m,n)} \prod_{j=1}^{m} K\left(\frac{d(X_{i_j}, t_j)}{h_j^{+}(t_j)}\right)} - r^{(m)}(\varphi, \mathbf{t}) \right| \nonumber \\
&= \xi_n \left| \frac{u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))}{u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))} - r^{(m)}(\varphi, \mathbf{t}) \right|. \tag{8.56}
\end{align}
$$

The study of (8.56) is similar to the proofs of Theorems 3.10 and 3.11, as we can clearly see that

$$
\left| \frac{u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))}{u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))} - r^{(m)}(\varphi, \mathbf{t}) \right| \le \left| \frac{u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))}{u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))} - \frac{\mathbb{E}[u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))]}{\mathbb{E}[u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))]} \right| \quad (8.57)
$$

$$
+ \left| \frac{\mathbb{E}[u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))]}{\mathbb{E}[u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))]} - r^{(m)}(\varphi, \mathbf{t}) \right|. \quad (8.58)
$$

Let us start with (8.57), we have

$$
\begin{align*}
& \left| \frac{u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))}{u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))} - \frac{\mathbb{E}[u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))]}{\mathbb{E}[u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))]} \right| \\
& \le \frac{|u_n(\varphi, \mathbf{t}, \mathbf{h}^{-}(\mathbf{t})) - \mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}^{-}(\mathbf{t})))|}{|u_n(1, \mathbf{t}, \mathbf{h}^{+})|} \\
& \quad + \frac{|\mathbb{E}(u_n(\varphi, \mathbf{t}, \mathbf{h}^{-}(\mathbf{t})))| \cdot |u_n(1, \mathbf{t}, \mathbf{h}^{+}) - \mathbb{E}(u_n(1, \mathbf{t}, \mathbf{h}^{+}(\mathbf{t})))|}{|u_n(1, \mathbf{t}, \mathbf{h}^{+}(\mathbf{t}))| \cdot |\mathbb{E}(u_n(1, \mathbf{t}, \mathbf{h}^{+}(\mathbf{t})))|} \\
& =: \mathbf{I} + \mathbf{II}.
\end{align*}
$$

Applying the same calculation as in the proof of Theorem 3.10, it follows that:

$$
\sup_{\varphi \tilde{K} \in \mathcal{F}_m \times \mathcal{H}^m} \sup_{(\mathbf{h}^{-}(\mathbf{t}), \mathbf{h}^{+}(\mathbf{t})) \in \mathcal{H}_0^{(m)} \times \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \frac{u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))}{u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))} - r^{(m)}(\varphi, \mathbf{t}) \right| \\
= O_{a,co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n \phi(\tilde{h}_n)}} \right).
$$

Now, for (8.58), using the fact that

$$
\mathbb{E}[u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))] \geq \kappa_1^m C_1,
$$

therefore,

$$
\left| \frac{u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))}{u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))} - r^{(m)}(\varphi, \mathbf{t}) \right| \le \kappa_1^{-m} C_1^{-1} \left| u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t})) - r^{(m)}(\varphi, \mathbf{t}) \right|,
$$

following the same steps as in Theorem 3.11, we can easily conclude that

$$
\sup_{\varphi \tilde{K} \in \mathcal{F}_m \times \mathcal{H}^m} \sup_{(\mathbf{h}^{-}(\mathbf{t}), \mathbf{h}^{+}(\mathbf{t})) \in \mathcal{H}_n^{(m)} \times \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \frac{\mathbb{E}[u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))]}{\mathbb{E}[u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))]} - r^{(m)}(\varphi, \mathbf{t}) \right| = O(\tilde{h}_n^\gamma).
$$
Consequently, we have

$$
\begin{align*}
& \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{(\mathbf{h}^{-}(\mathbf{t}), \mathbf{h}^{+}(\mathbf{t})) \in \mathcal{H}_n^{(m)} \times \mathcal{H}_n^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \frac{u_n(\varphi, \mathbf{t}; \mathbf{h}^{-}(\mathbf{t}))}{u_n(1, \mathbf{t}; \mathbf{h}^{+}(\mathbf{t}))} - r^{(m)}(\varphi, \mathbf{t}) \right| \\
& = O(\tilde{h}'_n^\gamma) + O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right).
\end{align*}
$$

Since $\xi_n \to 1$, we can also conclude that

$$
\begin{align*}
& \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{(\mathbf{h}^{-}(\mathbf{t}), \mathbf{h}^{+}(\mathbf{t})) \in \mathcal{H}_0^{(m)} \times \mathcal{H}_0^{(m)}} \sup_{\mathbf{t} \in S_X^m} \left| \frac{\sum_{i \in I(m,n)} \varphi(Y_{i_1}, \dots, Y_{i_m}) \prod_{j=1}^m K\left(\frac{d(X_{i_j}, t_j)}{h_j^{-}(t_j)}\right)}{\sum_{i \in I(m,n)} \prod_{j=1}^m K\left(\frac{d(X_{i_j}, t_j)}{h_j^{+}(t_j)}\right)} - \xi_n r^{(m)}(\varphi, \mathbf{t}) \right| \\
& = O(\tilde{h}'_n^\gamma) + O_{a.co} \left( \sqrt{\frac{\psi_{S_X} \left( \frac{\log n}{n} \right)}{n\phi(\tilde{h}_n)}} \right),
\end{align*}
$$

which implies that

$$
\begin{align}
& \sup_{\varphi \tilde{K} \in \mathcal{F}_m, \mathcal{H}^m} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_X^m} \left| \frac{\sum_{i \in I(m,n)} \varphi(Y_{i_1}, \dots, Y_{i_m}) \prod_{j=1}^m K\left(\frac{d(X_{i_j}, t_j)}{h_j^{-}(t_j)}\right)}{\sum_{i \in I(m,n)} \prod_{j=1}^m K\left(\frac{d(X_{i_j}, t_j)}{h_j^{+}(t_j)}\right)} - \xi_n r^{(m)}(\varphi, \mathbf{t}) \right| \nonumber \\
& = O\left(\phi^{-1}\left(\frac{k_{2,n}}{\rho_n^* n}\right)^\gamma\right) + O_{a.co}\left(\sqrt{\frac{\psi_{S_X}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu^* \phi_t^{-1}\left(\frac{\rho_n^* k_{1,n}}{n}\right)\right)}}\right). \tag{8.59}
\end{align}
$$

Finally, by putting $\varphi \equiv 1$ in (8.59), we get

$$
\begin{align}
& \sup_{\tilde{K} \in \mathcal{H}^m} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{\mathbf{t} \in S_X^m} \left| \frac{\sum_{i \in I(m,n)} \prod_{j=1}^m K\left(\frac{d(X_{i_j}, t_j)}{h_j^{-}(t_j)}\right)}{\sum_{i \in I(m,n)} \prod_{j=1}^m K\left(\frac{d(X_{i_j}, t_j)}{h_j^{+}(t_j)}\right)} - \xi_n \right| \nonumber \\
& = O\left(\phi^{-1}\left(\frac{k_{2,n}}{\rho_n^* n}\right)^\gamma\right) + O_{a.co}\left(\sqrt{\frac{\psi_{S_X}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu^* \phi_t^{-1}\left(\frac{\rho_n^* k_{1,n}}{n}\right)\right)}}\right). \tag{8.60}
\end{align}
$$
Hence, ($L_3$) is checked. Now, with all the conditions of Lemma 8.1.1 satisfied, it follows that

$$ \sup_{\varphi \in \mathcal{F}_m} \sup_{\varphi \in \mathcal{G}^m} \sup_{\mathbf{t} \in S_{\Omega}^m} \left| \mathfrak{M}_n^{(m)}(\varphi, \mathbf{t}; \mathbf{D}_{n,k}(\mathbf{t})) - \mathfrak{M}^{(m)}(\varphi, \mathbf{t}) \right| = O_{a,co}(\mathcal{U}_n), $$

which is exactly the desired result, hence, the proof is completed.

## 8.2. Proofs of weak convergence results

### Preliminaries of the proofs

As mentioned before, a straightforward approach does not work when dealing with random bandwidths. Therefore, we often use some general lemmas (see, for example, [44]) to be able to use the results of the non-random bandwidths. In this section, we present the results of [147] and [43] obtained for some positive bandwidth $h_K$. These results are key instrumental in the proofs. We denote the bias term and the centered variate respectively the following quantities

$$ \begin{aligned} B_n(t; h_K) &:= \frac{\mathbb{E}(\hat{r}_{n,2}^{(1)}(\varphi, t; h_K) - r^{(1)}(\varphi, t)\mathbb{E}(\hat{r}_{n,1}^{(1)}(1, t; h_K)))}{\mathbb{E}(\hat{r}_{n,1}^{(1)}(1, t; h_K))}, \\ Q_n(t; h_K) &:= (\hat{r}_{n,2}^{(1)}(\varphi, t; h_K) - \mathbb{E}(\hat{r}_{n,2}^{(1)}(\varphi, t; h_K))) \\ &\quad - r^{(1)}(\varphi, t)(\hat{r}_{n,1}^{(1)}(1, t; h_K) - \mathbb{E}(\hat{r}_{n,1}^{(1)}(1, t; h_K))). \end{aligned} \quad (8.61) $$

The decomposition (8.61) plays a key role in our proof. Indeed, following the method adopted by [147], we will show that convergences in quadratic mean to 1, and that the bias satisfies

$$ B_n(t; h_K) = o(1) \text{ as } n \to \infty. $$

**Lemma 8.5.** Under conditions (C.1), (C.3.1), and (C.5.1) (or equivalently (C.1'), (C.3') and (C.5.1)), and if $n\phi(h_K) \to \infty$ as $n \to \infty$, then, we have for each $t \in \mathcal{X}$:

$$ \hat{r}_{n,1}^{(1)}(1, t; h_K) \xrightarrow{p.s.} 1. \quad (8.62) $$

Before we present the next result, the following notation is needed:

$$ \mu_{n,1}(t; h_K) := \mathbb{E}[(\varphi(Y_1) - r^{(1)}(\varphi, t))\Delta_1(t; h_K)], \quad (8.63) $$

$$ Z_{n,i}(h_K) := (\varphi(Y_i) - r^{(1)}(\varphi, t; h_K))\Delta_i(t; h_K) - \mu_{n,1}(t; h_K). \quad (8.64) $$

Set

$$ Q_n(t; h_K) = \frac{1}{n\mathbb{E}[\Delta_1(t; h_K)]} \sum_{i=1}^{n} Z_{n,i}(h_K), $$

and

$$ \sigma_{n,0}^2(t; h_K) := \frac{1}{\mathbb{E}^2[\Delta_1(t; h_K)]} \mathrm{Var}[Z_{n,1}(h_K)]. $$

**Lemma 8.6.** Under conditions (C.1), (C.2), (C.3.1) and (C.5.1) (with $p = \infty$ and $\delta > 1$ for condition (C.5.1)), we have for $n\phi(h_K) \to \infty$, $t \in \mathcal{X}$ and positive constants $C_4, C_5$:
$$C_4 \frac{g_2(t, \varphi)}{f_1(t)} \le \phi(h_K) \sigma_{n,0}^2(t) \le C_5 \frac{g_2(t, \varphi)}{f_1(t)}, \qquad (8.65)$$

whenever $f_1(t) > 0$. We have

$$\frac{1}{n\mathbb{E}^2[\Delta_1(t; h_K)]} \sum_{i=1}^{n} \sum_{\substack{i=1 \\ i \neq j}}^{n} \mathrm{Cov}[Z_{n,i}, Z_{n,j}] = o(\sigma_{n,0}^2(t)). \qquad (8.66)$$

$$\mathrm{Var}[Q_n(t)] = \sigma_{n,0}^2(t)(1 + o(1))/n. \qquad (8.67)$$

**Lemma 8.7.** Under conditions (C.1'), (C.2.), (C.3'.) and (C.5.1) (with $p = \infty$ and $\delta > 1$ for condition (C.5.1)), we have for $n\phi(h_K) \to \infty$ as $n \to \infty$ and $t \in \mathcal{X}$:

$$\phi(h_K) \sigma_{n,0}^2(t) \to \frac{\kappa_1' g_2(t, \varphi)}{\kappa_2' f_1(t)}, \qquad (8.68)$$

whenever $f_1(t) > 0$, and $\kappa_1'$, $\kappa_2'$ are constants specified previously.

$$\frac{1}{n\mathbb{E}^2[\Delta_1(t; h_K)]} \sum_{i=1}^{n} \sum_{\substack{i=1 \\ i \neq j}}^{n} \mathrm{Cov}[Z_{n,i}, Z_{n,j}] = o(\sigma_{n,0}^2(t)), \qquad (8.69)$$

$$n\phi(h_K)\mathrm{Var}[Q_n(t)] \to \frac{\kappa_1' g_2(t, \varphi)}{\kappa_2' f_1(t)}. \qquad (8.70)$$

To unburden the notation a bit and for simplicity, we denote

$$\sigma^2(t) := (\kappa_1' g_2(t, \varphi)) / (\kappa_2' f_1(t)).$$

**Lemma 8.8.** Under conditions (C.1'.), (C.2.), (C.3'.), (C.5.1) and (C.5.2), if $n\phi(h_K) \to \infty$ as $n \to \infty$, then we have for $t \in \mathcal{X}$, as $n \to \infty$,

$$(n\phi(h_K))^{1/2} Q_n(t) \to \mathcal{N}(0, \sigma^2(t)). \qquad (8.71)$$

**Lemma 8.9.** Under conditions (C.1.), (C.2.), (C.3.1) and (C.5.1), and if

$$(n\phi(h_K) / \log \log n) \to \infty, \qquad (8.72)$$

then we have, as $n \to \infty$,

$$\left( \frac{n\phi(h_K)}{\log \log n} \right)^{1/2} \left[ \tilde{r}_n^{(1)}(\varphi, t) - r^{(1)}(\varphi, t) - B_n(t) \right] \xrightarrow{\mathbb{P}} 0. \qquad (8.72)$$
## Proof of Theorem 4.1

Using the Cramér-Wold device, it is sufficient to prove the convergence of one-dimensional distribution in order to prove Theorem 4.1. Indeed, by the linearity of $v_n(\psi; k_K | t)$ if suffices to show that

$$v_n(\Phi; H_{n,k}(t) | t) \rightarrow N(0, \sigma^2(\Phi, t)),$$

for all $\Phi$ of the form

$$\Phi = \sum_{p=1}^{L} c_p \psi_p, \quad c_1, \dots, c_L \in \mathbb{R}, \quad \psi_1, \dots, \psi_L \in \mathcal{F} \mathcal{K}.$$

Therefore, we shall only demonstrate convergence in a single dimension. Remember that we're dealing with

$$\begin{aligned} v_n(\psi; H_{n,k}(t) | t) &= \sqrt{k}(\hat{r}_n^{(1)}(\psi, t; H_{n,k}(t)) - r^{(1)}(\psi, t)) \\ &= \sqrt{k}\left(\frac{\sum_{i=1}^n \varphi(Y_i)K(d(X_i, t)/H_{n,k}(t))}{\sum_{i=1}^n K(d(X_i, t)/H_{n,k}(t))} - r^{(1)}(\psi, t)\right). \end{aligned} \quad (8.73)$$

Set

$$\varphi_M(Y) := \varphi(Y)1_{\{F(Y)<M\}}.$$

To obtain the desired result, we write

$$\begin{aligned} & \hat{r}_n^{(1)}(\varphi, t; H_{n,k}(t)) - r^{(1)}(\varphi, t) \\ &= \frac{\hat{r}_{n,2}^{(1)}(\varphi, t; H_{n,k}(t))}{\hat{r}_{n,1}^{(1)}(1, t; H_{n,k}(t))} - r^{(1)}(\varphi, t) \\ &= \frac{1}{\hat{r}_{n,1}^{(1)}(1, t; H_{n,k}(t))} \left[ \begin{aligned} & (\hat{r}_{n,2}^{(1)}(\varphi, t; H_{n,k}(t)) - \hat{r}_{n,2}^{(1)}(\varphi_M, t; H_{n,k}(t))) \\ & - (r^{(1)}(\varphi, t) - \mathbb{E}(\hat{r}_{n,2}^{(1)}(\varphi, t; H_{n,k}(t)))] \\ & + (\mathbb{E}(\hat{r}_{n,2}^{(1)}(\varphi_M, t; H_{n,k}(t))) - \mathbb{E}(\hat{r}_{n,2}^{(1)}(\varphi, t; H_{n,k}(t))) \\ & + (\hat{r}_{n,2}^{(1)}(\varphi_M, t; H_{n,k}(t)) - \mathbb{E}(\hat{r}_{n,2}^{(1)}(\varphi_M, t; H_{n,k}(t))) - r^{(1)}(\varphi, t)(\hat{r}_{n,1}^{(1)}(1, t; H_{n,k}(t)) - 1) \end{aligned} \right] \\ &= \frac{1}{\hat{r}_{n,1}^{(1)}(1, t; H_{n,k}(t))} [I_1(t; H_{n,k}) + I_2(t; H_{n,k})], \end{aligned}$$

where

$$I_1(t; H_{n,k}) = \underbrace{(\hat{r}_{n,2}^{(1)}(\varphi, t; H_{n,k}(t)) - \hat{r}_{n,2}^{(1)}(\varphi_M, t; H_{n,k}(t)))}_{S_1(t; H_{n,k})} - \mathbb{E}(S_1(t; H_{n,k})) \\ + \underbrace{\mathbb{E}(\hat{r}_{n,2}^{(1)}(\varphi, t; H_{n,k}(t))) - r^{(1)}(\varphi, t)}_{S_2(t; H_{n,k})}, \quad (8.74)$$