# k-Nearest Neighbour Estimation of the Conditional Set-Indexed Empirical Process for Functional Data: Asymptotic Properties

Youssouf Souddi, Salim Bouzebda

► To cite this version:

Youssouf Souddi, Salim Bouzebda. k-Nearest Neighbour Estimation of the Conditional Set-Indexed Empirical Process for Functional Data: Asymptotic Properties. Axioms, 2025, 14 (2), pp.76. 10.3390/axioms14020076. hal-04905156

HAL Id: hal-04905156

https://hal.science/hal-04905156v1

Submitted on 5 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# k-Nearest Neighbour Estimation of the Conditional Set-Indexed Empirical Process for Functional Data: Asymptotic Properties

Youssouf Souddi<fnref n="1" /> and Salim Bouzebda<fnref n="2" />,*,

**Abstract:** The main aim of this paper is to improve the existing limit theorems for set-indexed conditional empirical processes involving functional strong mixing random variables. To achieve this, we propose using the *k*-nearest neighbor approach to estimate the regression function, as opposed to the traditional kernel method. For the first time, we establish the weak consistency, asymptotic normality, and density of the proposed estimator. Our results are derived under certain assumptions about the richness of the index class *C*, specifically in terms of metric entropy with bracketing. This work builds upon our previous papers, which focused on the technical performance of empirical process methodologies, and further refines the prior estimator. We highlight that the *k*-nearest neighbor method outperforms the classical approach due to several advantages.

**Keywords:** conditional distribution; small ball probability; empirical process; mixing functional data; semi-metric space; covering number; *k*-nearest neighbour estimation

**MSC:** 62G20; 62G05; 62G32; 62G08; 62G35; 62G07; 62E20

<figure><img src="image_2.png" /></figure>

### ✓ check for updates

Academic Editor: Stefano De Marchi

Received: 24 December 2024

Revised: 7 January 2025

Accepted: 16 January 2025

Published: 21 January 2025

**Citation:** Souddi, Y.; Bouzebda, S. *k*-Nearest Neighbour Estimation of the Conditional Set-Indexed Empirical Process for Functional Data: Asymptotic Properties. *Axioms* **2025**, *14*, 76. https://doi.org/10.3390/axioms14020076

**Copyright:** © 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## 1. Introduction

The framework of empirical processes has long been recognized as a fundamental component of statistical methodology, underpinning theoretical developments and applications in areas such as testing for independence, parameter estimation, and bootstrap procedures. Furthermore, this framework has offered insights into the asymptotic properties of semi-parametric mixture models and various other sophisticated methods. For detailed expositions of both theoretical and applied dimensions of empirical process theory, the reader is referred to [1,2]. Over approximately the last three decades, empirical process theory in finite contexts has undergone marked expansion, as reflected by numerous contributions focusing on the case of independent and identically distributed (i.i.d.) observations.

An early milestone in this line of inquiry was provided by [3], who examined the asymptotic distribution of the standard empirical process when sample sizes are random. The work of [3] established weak convergence for continuous empirical processes within suitably chosen metric spaces. Subsequently, ref. [4] investigated the classes C of sets for which the Glivenko-Cantelli theorem holds (up to measurability), thus solidifying the basis for uniform convergence of empirical distribution functions. Building on these results, ref. [5] introduced both strong and weak approximations of empirical processes by leveraging Gaussian processes, ultimately highlighting associated invariance principles. In a related direction, ref. [6] studied empirical processes pertaining to particular classes

<footnote n="1">¹ Laboratory of Stochastic Models, Statistics and Applications, University of Saida-Dr. Moulay Tahar, P.O. Box 138 EN-NASR, Saida 20000, Algeria; souddiyoucef@yahoo.fr</footnote>

<footnote n="2">² Laboratoire de Mathématiques Appliquées de Compiègne (L.M.A.C.), Université de Technologie de Compiègne, 60200 Compiègne, France</footnote>

<footnote>* Correspondence: salim.bouzebda@utc.fr</footnote>
of sets, laying out the conditions under which weak convergence to Gaussian processes emerges in terms of metric entropy with inclusion. This analysis extends well beyond the classical framework of Donsker's theorem and accommodates independent random variables. Additional seminal results are available in [7–9], among other contributions, underscoring the continued advancement of empirical process theory.

Over subsequent years, attention turned toward mixing processes and their conditional aspects in separable metric spaces. ref. [10] provided broad generalizations of the central limit theorem for three varieties of mixing variables defined on the interval [0, 1]. ref. [11] later investigated weak convergence under $\phi$-mixing conditions, while ref. [12] addressed specific regularity constraints leading to the asymptotic normality of parameter estimators. Extending these ideas further, ref. [13] examined set-indexed empirical processes in $\alpha$-mixing settings to establish results involving weak convergence in probability and asymptotic normality, as well as deriving variance expressions and continuity properties vital to asymptotic investigations. Subsequent research by [14] refined these theoretical insights. For comprehensive treatments and related studies on mixing empirical processes, see [15–20]. A parallel body of work addresses the need to analyze infinite-dimensional data—commonly referred to as functional data or functional variables—across fields such as environmental science, chemometrics, biometrics, medicine, and econometrics. These functional data structures often capture intricate phenomena more effectively than finite-dimensional representations, which is particularly evident in research areas like finance and biology. Comprehensive overviews of functional data analysis are provided by [21–26]. In recent years, this domain has experienced rapid expansion, as evidenced by a substantial increase in scholarly work and specialized monographs, highlighting the evolving and dynamic nature of functional data analysis.

Despite the significant advancements in functional data analysis, the integration of functional data within the framework of set-indexed empirical processes remains relatively underexplored. Notably, recent work by [27] made a substantial contribution by extending the results of [13], thereby proving the invariance principle for the set-indexed conditional empirical process. ref. [27] further applied these theoretical advancements to the problem of conditional independence testing, demonstrating the practical utility of the extended framework. This line of research was further developed in subsequent works by [28,29], which addressed the complexities associated with ergodic data. More recently, ref. [30] incorporated the presence of missing data into their analysis, leveraging the strengths of the k-nearest neighbors (k-NN) method for estimation purposes. The k-NN method has enjoyed widespread application within finite frameworks, with foundational contributions from [8], ref. [31], as well as adaptations for functional data models. Notably, ref. [32] investigated the asymptotic properties of the k-NN kernel estimator for regression tasks, providing comparative analyses with traditional kernel methods. ref. [33] offered asymptotic results for k-NN generalized regression estimators in abstract spaces. For further exploration of k-NN methods in the context of infinite-dimensional data, researchers can consult the works of [34–37].

## 1.1. *Contribution*

Despite the substantial rise in research on empirical processes and the application of k-NN methods to functional data, studies that examine the asymptotic properties of empirical processes unifying functional data with k-NN methods are still relatively scarce. Nonetheless, recent contributions have begun to address this gap. In particular, refs. [28,38,39] offered important theoretical and practical insights by establishing both the uniform consistency and weak convergence of k-NN empirical conditional processes and k-NN conditional U-processes adapted to functional mixing data. These developments hold considerable promise for set-
indexed conditional U-statistics, Kendall's rank correlation coefficient, classification tasks, and the prediction of time series. Building upon these findings, the present paper introduces an innovative methodological framework aimed at efficiently tackling the challenges inherent in set-indexed conditional empirical processes.

The main distinction between this work and [28] or [29] lies in our consideration of a more intricate data-driven bandwidth structure. In addition, we establish novel results concerning the uniform consistency with respect to the number of neighborhoods. Compared with [28,38,39], our results are derived under bracketing numbers that capture the complexity of the relevant sets in a manner that differs from the entropy-based techniques explored in our previous research. Furthermore, the strong mixing condition assumed here is more general than the regular processes context adopted in those earlier works to prove weak convergence.

## 1.2. Organization of the Paper

The structure of this article is meticulously organized to facilitate a coherent and comprehensive exposition of the subject matter. Section 2 delineates the notation and fundamental definitions essential for understanding the ensuing discussions. It also introduces the k-NN set-indexed conditional empirical process, laying the groundwork for the theoretical developments that follow. Section 3 is dedicated to presenting the main results of this study, highlighting the key theoretical contributions and their implications. In Section 4, an application of these theoretical results to the domain of statistical classification is discussed, demonstrating the practical relevance and utility of the proposed methods. Section 3.3 addresses the pragmatic aspects related to the selection of the parameter k, which is pivotal for the performance of the k-NN method. This section provides guidance on optimal bandwidth selection, balancing bias and variance considerations. The concluding remarks, along with prospective directions for future research, are provided in Section 5, with all proofs collected in Appendix A.

# 2. Presentation of the k-NN Set-Indexed Conditional Empirical Process

We consider a sequence of random pairs $(X_1, Y_1), \dots, (X_n, Y_n)$, where each pair is a realization of the random variable $(X, Y)$, which takes values in the space $\mathcal{E} \times \mathbb{R}^d$. The functional space $\mathcal{E}$ is equipped with a semi-metric $d_{\mathcal{E}}(\cdot, \cdot)$ (A semi-metric, or pseudo-metric, allows $d(x_1, x_2) = 0$ even if $x_1 \neq x_2$). Our goal is to examine the relationship between $X$ and $Y$ by estimating functional operators related to the conditional distribution of $Y$ given $X$, such as the regression operator. For any measurable set $C$ in a class of sets $\mathcal{C}$, the conditional probability is defined as:

$$ \Gamma(C | x) = \mathbb{E}[1_{\{Y \in C\}} | X = x]. $$

To model this relationship, we use a Nadaraya-Watson estimator [40,41] and present a k-nearest neighbors (k-NN) version of the conditional empirical distribution. Specifically, we define:

$$ \Gamma_n(C, x) = \frac{\sum_{i=1}^{n} 1_{\{Y_i \in C\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}, \quad (1) $$

where $K(\cdot)$ is a real-valued kernel function defined on $[0, \infty)$, and $H_{n,k}(x)$ is a smoothing parameter that satisfies $H_{n,k}(x) \to 0$ as $n \to \infty$. Here, $C$ is a measurable set, and $x \in \mathcal{E}$. When $C = (-\infty, z]$ for some $z \in \mathbb{R}^d$, the estimator reduces to the conditional empirical distribution function $F_n(z | x) = \Gamma_n((-\infty, z], x)$. Hence, the corresponding class of sets is $\mathcal{C} = \{(-\infty, z] : z \in \mathbb{R}^d\}$. We further define $H_{n,k}(x)$ as the smallest value $h \in \mathbb{R}^+$ such that:
$$ \sum_{i=1}^{n} 1_{B(x,h)}(X_i) = k, $$

where $B(x, h)$ represents the ball of radius $h$ around $x$. It is clear that $H_{n,k}(x)$ is a positive random variable with respect to the semi-metric topology on $\mathcal{E}$. To relate this work to the existing literature and to highlight the differences between the k-NN and classical kernel approaches, we recall the functional Nadaraya-Watson estimator from [23] and refer to [27,29,30] for more information on the asymptotic properties of nonparametric functional distribution estimators. The classical kernel-based estimator is defined as:

$$ \hat{\Gamma}_n(C, x) = \frac{\sum_{i=1}^{n} 1_{\{Y_i \in C\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{h_n}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{h_n}\right)}, \quad (2) $$

where $h_n$ is a smoothing parameter. To further clarify the notation, we use:

$$ B(x, t) = \{x_1 \in \mathcal{E} : d_{\mathcal{E}}(x_1, x) \le t\} $$

to represent the ball in $\mathcal{E}$ with center $x$ and radius $t$ and define the small ball probability function as:

$$ F(t; x) = \mathbb{P}(d_{\mathcal{E}}(x, X_i) \le t) = \mathbb{P}(X_i \in B(x, t)) = \mathbb{P}(D_i \le t), $$

where $D_i = d_{\mathcal{E}}(x, X_i)$. The behavior of $F(t; x)$ as $t \to 0$ is of particular interest, and [42] assumed that $F(h; x) = \phi(h_n)f_1(x)$ as $h \to 0$, where $f_1(x)$ represents the probability density (functional). When $\mathcal{H} = \mathbb{R}^m$, the small ball probability function takes the form $F(h; x) = \mathbb{P}(\|x - X_i\| \le h)$, where $\phi(h_n) = C(m)h^n$ is the volume of a unit ball in $\mathbb{R}^m$. This result motivates the assumption (H4) (i) – (ii), as discussed in [43].

In statistics, observations are often weakly dependent rather than independent. Ignoring this dependence can have serious consequences for statistical inference. The concept of dependence quantifies the extent to which a sequence of random variables deviates from independence and is useful for extending classical results to weakly dependent or mixing sequences. We refer to [44] for further details on dependence structures. For our analysis, we assume that the sequence $\{(X_i, Y_i)\}_{i=1}^n$ is strongly mixing.

**Definition 1.** A sequence $\{\zeta_k, k \ge 1\}$ is said to be $\alpha$-mixing if the $\alpha$-mixing coefficient:

$$ \alpha(n) = \sup_{k \ge 1} \sup \left\{ |\mathbb{P}(A \cap B) - \mathbb{P}(A)\mathbb{P}(B)| : A \in \mathcal{F}_{n+k}^{\infty}, B \in \mathcal{F}_1^k \right\} $$

converges to zero as $n \to \infty$, where $\mathcal{F}_1^m = \sigma(\zeta_1, \zeta_{1+1}, \dots, \zeta_m)$ denotes the $\sigma$-algebra generated by $\zeta_1, \zeta_{1+1}, \dots, \zeta_m$ with $l \le m$. A sequence is geometrically strong mixing if, for some $a > 0$ and $\beta > 1$,

$$ \alpha(j) \le aj^{-\beta}, $$

and exponentially strong mixing if, for some $b > 0$ and $0 < \gamma < 1$,

$$ \alpha(k) \le b\gamma^k. $$

In the exploration of mixing conditions, $\alpha$-mixing stands out as a somewhat lenient criterion that is nonetheless broadly applicable across various stochastic processes, including many time series models. Works by [45,46] pinpointed the necessary conditions for $\alpha$-mixing in linear processes. These studies showed that both linear autoregressive
models and the more complex bilinear time series models exhibit robust mixing properties characterized by mixing coefficients that decrease exponentially. Additionally, the study [47] shed light on the importance of α-mixing, including aspects like geometric ergodicity, in understanding nonlinear time series models, as further discussed in [48–50]. Similarly, ref. [51] illustrated that functional autoregressive processes can achieve geometric ergodicity when certain conditions are met. Moreover, the studies [52,53] showed that with only minimal assumptions, both autoregressive conditional heteroscedastic processes and nonlinear additive autoregressive models that include exogenous variables maintain stationarity and α-mixing.

**Definition 2.** A class $\mathcal{C}$ of subsets of a set C is called a VC-class if there exists a polynomial $\mathfrak{B}(\cdot)$ such that, for every set of N points in C, the class $\mathcal{C}$ can select at most $\mathfrak{B}(N)$ distinct subsets.

**Definition 3.** A class of functions $\mathcal{F}$ is called a VC-subgraph class if the graphs of the functions in $\mathcal{F}$ form a VC-class of sets. Specifically, the subgraph of a real-valued function $f$ on a set $S$ is defined as

$$ \mathcal{G}_f = \{(s,t) : 0 \le t \le f(s) \text{ or } f(s) \le t \le 0\}, $$

and the collection $\{\mathcal{G}_f : f \in \mathcal{F}\}$ is a VC-class of sets on $S \times \mathbb{R}$.

**Definition 4.** Let $S_{\mathcal{E}}$ be a subset of a semi-metric space $\mathcal{E}$, and let $N_{\mathcal{E}}$ be a positive integer. A finite set $\{e_1, \dots, e_{N_{\mathcal{E}}}\} \subset \mathcal{E}$ is called an $\varepsilon$-net of $S_{\mathcal{E}}$ if:

$$ S_{\mathcal{E}} \subseteq \bigcup_{j=1}^{N_{\mathcal{E}}} B(e_j; \mathcal{E}), $$

where $B(e_j; \mathcal{E})$ is the ball of radius $\varepsilon$ around $e_j$. The Kolmogorov entropy (or metric entropy) of $S_{\mathcal{E}}$ is defined as

$$ \psi_{S_{\mathcal{E}}}(\mathcal{E}) = \log \mathcal{N}_{\mathcal{E}}(S_{\mathcal{E}}), $$

where $\mathcal{N}_{\mathcal{E}}(S_{\mathcal{E}})$ is the cardinality of the smallest $\mathcal{E}$-net required to cover $S_{\mathcal{E}}$.

The concept of semi-metrics plays a central role in this study. For further discussions on how to choose the appropriate semi-metric, we refer the reader to [23] (see Chapters 3 and 13).

## 2.1. Notations and Hypothesis

Throughout this paper, we fix $x$ as an element of the functional space $\mathcal{E}$. We introduce the notion of metric entropy with inclusion, which quantifies the complexity or richness of the class of sets $\mathcal{C}$. For each $\varepsilon > 0$, the covering number is defined as follows:

$$ \mathcal{N}(\varepsilon, \mathcal{C}, \Gamma(\cdot | x)) = \inf\{n \in \mathbb{N} : \exists C_1, \dots, C_n \in \mathcal{C} \text{ such that for all } C \in \mathcal{C}, \exists 1 \le i, j \le n \text{ with } C_i \subset C \subset C_j \\ \text{and } \Gamma(C_j \setminus C_i | x) < \varepsilon\}, $$

The term $\log(\mathcal{N}(\varepsilon, \mathcal{C}, \Gamma(\cdot | x)))$ is called the metric entropy with the inclusion of $\mathcal{C}$ with respect to $\Gamma(\cdot | x)$. Estimates for such covering numbers are available for many classes; see, for example, [54]. We will often assume below that either $\log(\mathcal{N}(\varepsilon, \mathcal{C}, \Gamma(\cdot | x)))$ or $\mathcal{N}(\varepsilon, \mathcal{C}, \Gamma(\cdot | x))$ behaves like powers of $\varepsilon^{-1}$. We say that the condition $(R_{\gamma})$ holds if

$$ \log \mathcal{N}(\varepsilon, \mathcal{C}, \Gamma(\cdot | x)) \le H_{\gamma}(\varepsilon), \quad \text{for all } \varepsilon > 0, \qquad (3) $$
where

$$H_{\gamma}(\varepsilon) = \begin{cases} \log(A\varepsilon) & \text{if } \gamma = 0, \\ A\varepsilon^{-\gamma} & \text{if } \gamma > 0, \end{cases}$$

for some constants $A, r > 0$. As in [13], it is worth noting that the condition (3), for $\gamma = 0$, holds for intervals, rectangles, balls, ellipsoids, and for classes constructed from these by performing finitely many set operations such as union, intersection, and complement. The classes of convex sets in $\mathbb{R}^d$ ($d \ge 2$) satisfy the condition (3), where $\gamma = (d-1)/2$. Other classes of sets satisfying (3) with $\gamma > 0$ can be found in [54].

In this section, we derive the almost complete convergence and central limit theorems for the conditional empirical process $\{\Lambda_n(C, x) : C \in \mathcal{C}\}$, defined by

$$\Lambda_n(C, x) := \sqrt{n\phi(h_n)}(\Gamma_n(C, x) - \Gamma(C, x)). \quad (4)$$

Next, we list the conditions required for our analysis:

### (H1) On the distributions/small-ball probabilities

(H1)(i) For $h \in \mathbb{R}^+$ and $x \in \mathcal{C}$, and positive constants $C_1, C_2$, we assume that

$$0 < C_1\phi(h_n) \le \phi_x(h_n) \le C_2\phi(h_n) < \infty. \quad (5)$$

(H1)(ii) For all $i \ge 1$,

$$0 < c_5\phi(h_n)f_1(x) \le \mathbb{P}(X_i \in B(x, h)) = F(h; x) \le c_6\phi(h_n)f_1(x),$$

where $\phi(h_n) \to 0$ as $h \to 0$ and $f_1(x)$ is a nonnegative functional in $x \in \mathcal{C}$.

(H1)(iii) We have

$$\sup_{i \neq j} \mathbb{P}((X_i, X_j) \in B(x, h) \times B(x, h)) = \sup_{i \neq j} \mathbb{P}(D_i \le h, D_j \le h) \le \psi(h)f_2(x),$$

where $\psi(h) \to 0$ as $h \to 0$ and $f_2(x)$ is a nonnegative functional in $x \in \mathcal{C}$.
We assume that the ratio $\psi(h)/\phi^2(h)$ is bounded.

### (H2) On the smoothness of the model

(H2)(i) There exist constants $\beta > 0$ and $\eta_1 > 0$ such that for all $x_1, x_2 \in N_x$, a neighborhood of $x$, we have

$$|\Gamma(C | x_1) - \Gamma(C | x_2)| \le \eta_1 d_{\mathcal{C}}^{\beta}(x_1, x_2).$$

(H2)(ii) Let $g_2(u) = \text{Var}(1_{\{Y_i \in C\}} | X_j = u)$ for $u \in \mathcal{C}$. Assume that $g_2(u)$ is independent of $j$ and continuous in some neighborhood of $x$, as $h \to 0$,

$$\sup_{\{u:d(x,u) \le h\}} |g_2(u) - g_2(x)| = o(1),$$

and assume that

$$g_v(u) = \mathbb{E}(|1_{\{Y_i \in C\}} - \Gamma(C | x)|^\nu | X_i = u)$$

is continuous in some neighborhood of $x$.

(H2)(iii) Define, for $i \neq j, u, v \in \mathcal{C}$,

$$g(u, v; x) = \mathbb{E}((1_{\{Y_i \in C\}} - \Gamma(C | x))(1_{\{Y_j \in C\}} - \Gamma(C | x)) | X_i = u, X_j = v).$$
Assume that $g(u, v; x)$ does not depend on $i, j$ and is continuous in some neighborhood of $(x, x)$.

### (H3) On the kernel function

(H3)(i) The kernel function $K(\cdot)$ is supported within $[0, 1]$, and there exist constants $0 < \eta_1 \le \eta_2 < \infty$ such that:

$$ \int_0^1 K(x)dx = 1, $$

and

$$ 0 < \eta_1 1_{(0,1)}(\cdot) \le K(\cdot) \le \eta_2 1_{(0,1)}(\cdot). $$

(H3)(ii) The kernel $K(\cdot)$ is a positive, differentiable function on $[0, 1]$ with derivative $K'(\cdot)$ such that

$$ -\infty < \eta_3 < K'(\cdot) < \eta_4 < 0. $$

### (H4) On the classes of sets

(H4)(i) The metric entropy of the class $\mathcal{C}$ satisfies, for some $1 \le p < \infty$,

$$ \int_0^\infty \left( \log \mathcal{N}(u, \mathcal{C}_\sigma, \|\cdot\|_p) \right)^{\frac{1}{2}} du < \infty, $$

where

$$ \mathcal{C}_\sigma = \{C_1, C_2 \in \mathcal{C} : \Gamma(C_1 \Delta C_2 | X = x) \le \sigma\}. $$

(H4)(ii) The class of sets $\mathcal{C}$ is assumed to be of VC-type with the envelope function previously defined. Hence, there are two finite positive constants $a$ and $p$ such that:

$$ \mathcal{N}(\epsilon, \mathcal{C}, \|\cdot\|_{L_2(\mathcal{Q})}) \le \left( \frac{b \|F_K\|_{L_2(\mathcal{Q})}}{\epsilon} \right)^a, $$

for any $\epsilon > 0$ and each probability measure such that $Q(F)^2 < \infty$.

### (H5) On the dependence of the random variables

(H5)(i) For some $v > 2$ and $\zeta > 1 - \frac{2}{v}$, we have

$$ \sum_{\ell=1}^{\infty} \ell^{\zeta} [\alpha(\ell)]^{1-\frac{2}{v}} < \infty. $$

(H5)(ii) There exists a sequence of positive integers $\{s_n\}_{n \in \mathbb{N}^*}$ such that, as $n \to \infty$,

$$ s_n \to \infty, \quad s_n = \left( \sqrt{n\phi(h(x))} \right), $$

and

$$ \left( \frac{n}{\phi(h(x))} \right)^{\frac{1}{2}} \alpha(s_n) \to 0. $$

(H6) On the entropy For large enough $n$ and some $\omega > 1$, Kolmogorov's entropy satisfies:

$$ \frac{(\log n)^2}{n\phi(h_K)} \le m\psi_{S_{\mathcal{E}}} \frac{\log n}{n} \le \frac{n}{\log n}\phi(h_K), \quad (6) $$

$$ \sum_{i=1}^{\infty} \exp \left( m(1 - \omega) \psi_{S_{\mathcal{E}}} \frac{\log n}{n} \right) \quad (7) $$
where $\psi_{S_{\mathcal{E}}} (\varepsilon) := \log \mathcal{N}_{\varepsilon}(S_{\mathcal{E}})$, and $\mathcal{N}_{\varepsilon}(S_{\mathcal{E}})$ is the minimal number of open balls of radius $\varepsilon$ of $\mathcal{E}$ needed to cover $S_{\mathcal{E}}$.

(H7) Condition on the smoothing parameter The smoothing parameter $h_n$ satisfies:

$$h_n \to 0 \quad \text{and} \quad \frac{\log n}{n \min(h_n^2, \phi^2(h_n))} \to 0 \quad \text{as} \quad n \to \infty.$$

(H8) There exist sequences $\rho_n \subset (0, 1)$, $\{k_{1,n}\} \subset \mathbb{Z}^+$ such that

$$0 < \mu \le \nu < \infty,$$

and the conditions

$$\mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \le \phi_x^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right), \quad (8)$$

and

$$\phi_x^{-1} \left( \frac{k_{2,n}}{\rho_n n} \right) \to 0, \quad (9)$$

and

$$\min \left\{ \frac{1 - \rho_n}{4} \frac{k_{1,n}}{\log n'}, \frac{(1 - \rho_n)^2}{4\rho_n} \frac{k_{1,n}}{\log n} \right\} > 2, \quad (10)$$

and

$$\frac{\log n/n}{\min\{\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right), \phi\left(\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right)\right)\}}. \quad (11)$$

Additional/Alternative conditions

(C1) For all $t > 0$, we have $\phi(t) > 0$. For all $t \in (0, 1)$, the function $\tau_0(t)$ exists, where

$$\tau_0(t) = \lim_{r \to 0} \frac{\phi(rt)}{\phi(r)} = \lim_{r \to 0} \mathbb{P}(d_{\mathcal{E}}(x, X) \le rt \mid \mathbb{P}(d_{\mathcal{E}}(x, X) \le t)) < \infty.$$

(C2) The kernel function $K(\cdot)$ is supported within $[0, 1]$, and there exist constants $0 < \vartheta_2$ and $0 < \vartheta'_1 \le \vartheta'_2 < \infty$ such that for $j = 1, 2$:

$$\int_0^1 K(x)dx = 1, \quad K(\cdot) \le \vartheta_2 \mathbb{1}_{[0,1]}(\cdot), \quad \frac{h_n}{\phi(h_K(x))} \int_0^1 K^j(x)\phi'(vh_K)dv \to \vartheta'_2, \text{ as } n \to \infty.$$

2.2. Comments on the Hypothesis

We address a complex theoretical problem within the framework of a nonparametric functional regression model, focusing on functional central limit theorems for set-indexed conditional empirical processes under functionally strong mixing data. Specifically, we utilize $k$-nearest neighbors (k-NN)-based random (or data-dependent) bandwidths. The investigation begins with Assumption (H1), which serves as a standard condition on the small-ball probability. This condition, adapted from [43], pertains to the case where $\mathcal{E}$ is an infinite-dimensional space. Under this assumption, $\phi(h_k)$ converges to 0 exponentially as $n \to \infty$, and the function $\phi_x(\cdot)$ is employed to control behavior around zero. Notably, the small-ball probability can be approximated as the product of two independent functions, $\tilde{\phi}(\cdot)$ and $f_1(\cdot)$. For related examples, see [55] for diffusion processes, ref. [56] for Gaussian measures, and ref. [57] for general Gaussian processes. To adjust for the bias in nonparametric estimators, some understanding of the variability of the small-ball probability is typically required when dealing with functional data. This is achieved by assuming:

$$\tau_0(t) = \lim_{r \to 0} \frac{\phi(rt)}{\phi(r)} = \lim_{r \to 0} \mathbb{P}(d_{\mathcal{E}}(x, X) \le rt \mid \mathbb{P}(d_{\mathcal{E}}(x, X) \le t)) < \infty.$$
Assumption (H2) pertains to the model’s regularity, encompassing mild requirements such as the continuity of certain conditional moments and the usual Lipschitz condition for regression ((H2)(i)). (H3) is a standard condition in functional nonparametric estimation, primarily concerning the kernel function K(·). Notably, (H3)(i) can be substituted with hypothesis (C2) to derive the asymptotic variance. Assumption (H4) is invoked for bounded sets. For (H4)(iii), see examples in [58], [59] (§4.7), [60] (Theorem 2.6.7), and [61] (§9.1), which provide necessary conditions. For further discussions, see [62] (§3.2). The class of functions is generally assumed to be pointwise measurable, a condition satisfied when K(·) is defined on $\mathbb{R}^d$. See, for instance, [63–65]. Assumption (H5) concerns the decay rate of the mixing coefficient $\alpha(n)$, a critical requirement for studying the weak convergence of empirical processes and establishing asymptotic equicontinuity. (H6) addresses topological considerations by controlling the Kolmogorov entropy of the sets $S_\mathcal{E}$. This is standard in nonparametric models, aiding in the uniform convergence and consistency of estimators. It quantifies the complexity of function classes using tools such as covering numbers or $\epsilon$-nets. Examples include closed balls in Sobolev spaces, unit balls in Cameron-Martin spaces, and compact subsets in Hilbert spaces with projection semi-metrics. See [33,66,67] for further details. For specific functional spaces $\mathcal{E}$ and subsets $S_\mathcal{E}$, it is noted in [33] that $\psi_{S_\mathcal{E}}(\log(n)/n) \gg \log(n)$. Assumption (H7) is fundamental, as convergence would not hold without it. This condition adapts (H8) to the context of functional conditional processes in the k-NN framework.

### 3. Main Results

We adopt the notation $Z \stackrel{D}{=} \mathcal{N}(\mu, \sigma^2)$ to denote that the random variable Z follows a normal distribution with mean $\mu$ and variance $\sigma^2$. Convergence in distribution is denoted by $\stackrel{D}{\to}$, while convergence in probability is indicated by $\stackrel{P}{\to}$. Let $(z_n)_{n \in \mathbb{N}}$ be a sequence of real random variables. We say that $z_n$ converges almost completely (a.co.) to zero if and only if:

$$ \forall \epsilon > 0, \sum_{n=1}^{\infty} \mathbb{P}(|z_n| > \epsilon) < \infty. $$

Moreover, for a sequence $(u_n)_{n \in \mathbb{N}^*}$ of positive real numbers, we say $z_n = O_{a.co}(u_n)$ if and only if:

$$ \exists \epsilon > 0, \sum_{n=1}^{\infty} \mathbb{P}(|z_n| > \epsilon u_n) < \infty. $$

This type of convergence implies both almost-sure convergence and convergence in probability.

#### 3.1. Uniform in the Number of Neighbors Consistency

Now, we can state the main results of this section concerning the k-NN functional estimator. Recall the bandwidths $k_{1,n}$ and $k_{2,n}$ given in the condition (H8).

**Theorem 1.** Consider the hypotheses (H1)(i), (H2)(i), (H3)(i), (H4)(i), (H4)(ii), (H6), (H7), and (H8). Let $(X_t, Y_t)$ be a geometrically strong mixing sequence with $\beta > 2$. Furthermore, assume that for every $C \in \mathcal{C}$, as $n \to \infty$, we have:

$$ \sup_{C \in \mathcal{C}} \sup_{k_{1,n} \le k \le k_{2,n}} \sup_{x \in S_{\mathcal{E}}} |\Gamma_n(C, x) - \Gamma(C, x)| = O\left(\phi^{-1}\left(\frac{k_{2,n}}{\rho_n n}\right)\right) + O_{a.co}\left(\sqrt{\frac{\psi_{S_x}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right)\right)}}\right). $$
## 3.2. Uniform Central Limit Theorems

The main objective of this section is to investigate the central limit theorems for the functional conditional empirical process defined by (4).

**Theorem 2.** Assume that hypotheses (H1)(ii), (H1)(iii), (H2)(i), (H2)(ii), (C2), (H5), and (H8) hold, and that $(X_i, Y_j)$ is geometrically strong mixing with $\beta > 2$. Furthermore, suppose that $n\phi(h_n) \to \infty$ as $n \to \infty$. For $m \ge 1$ and $C_1, \dots, C_m \in \mathcal{C}$, if the smoothing parameter $k$ satisfies the condition:

$$k\left(\phi^{-1}\left(\frac{k}{2}\right)\right)^{2\gamma} \to 0 \quad \text{as} \quad n \to \infty,$$

then the sequence

$$\{\Lambda(C_i, x)\}_{i=1,\dots,m} \xrightarrow{D} \mathcal{N}(0, \Sigma),$$

where $\Sigma = \sigma_{ij}(x)$ for $i, j = 1, \dots, m$ is the covariance matrix given by:

$$\sigma_{ij}(x) = \frac{\vartheta_2' \Gamma(C_i C_j, x) - \Gamma(C_i, x) \Gamma(C_j, x)}{\vartheta_1' f_1(x)},$$

whenever $f_1(x) > 0$.

We limit our discussion to the beta mixing setting in the theorems presented below.

**Theorem 3.** Assume that conditions (H3)(i), (H4)(i)–(H5)(i), and (H8) are satisfied. Then the family $\{\Lambda_n(C) : C \in \mathcal{C}\}$ is dense in the space $(\mathcal{L}^\infty(\mathcal{C}), \|\cdot\|_\mathcal{C})$, which can be expressed as

$$\lim_{\sigma \to 0} \lim_{n \to \infty} \sup_{\mathcal{P}} \mathbb{P} \left( \sup_{C \in \mathcal{C}_\sigma} |\Lambda_n(C)| > \epsilon \right) = 0,$$

where

$$\mathcal{C}_\sigma = \{C_1, C_2 \in \mathcal{C} : \Gamma(C_1 \Delta C_2 | X = x) \le \sigma\}.$$

**Theorem 4.** Under the conditions of Theorems 2 and 3, and with the additional conditions (H2) and (H4)(iii), the process:

$$\{\Lambda_n(C, x) : C \in \mathcal{C}\}$$

converges in law to a Gaussian process:

$$\{\Lambda(C, x) : C \in \mathcal{C}\},$$

which admits a version with uniformly bounded and uniformly continuous paths with respect to the $\|\cdot\|_2$-norm, with covariance $\sigma_{ij}(x)$ given in Theorem 2.

**Remark 1.** An important consideration when working with infinite-dimensional data, particularly functional data, is the need to account for the local structures inherent in the data. However, the bandwidth (denoted by $h$) used in kernel-based estimators is fixed and does not depend on the covariate component $x$, meaning it fails to capture such local features. An alternative and widely used approach in nonparametric and semiparametric statistics incorporates ideas from k-nearest neighbors (kNN). This method is especially advantageous when the data exhibit local structures, as kNN-based estimators use a location-dependent bandwidth (i.e., a bandwidth that varies with $x$), allowing them to effectively capture local features. In other words, kNN provides location-adaptive methods. The kNN-based estimators offer at least two key practical advantages over kernel-based methods. First, although the number of neighbors $k$ is fixed, the bandwidth $H_{n,k}(X)$ varies with $x$, which provides the local-adaptive property of kNN estimators, thus allowing adaptation to
heterogeneous designs. Second, the computational cost associated with selecting the smoothing parameter k is lower than that of selecting h, since k takes values from the finite set {1, 2, ..., n}. However, the theoretical analysis of kNN statistics presents greater challenges, primarily because $H_{n,k}(X)$ is a random variable that depends on $X_i$ (for i = 1, ..., n), introducing additional technical complexities in the proofs. It is important to note that this paper extends the work of [29].

### 3.3. The Parameter k Selection Criterion

Various methods have been developed to establish bandwidth selection rules for nonparametric kernel estimators, with a focus on achieving asymptotic optimality, especially for the Nadaraya-Watson regression estimator. Notable contributions in this area include [68–70]. This parameter has been appropriately chosen either in finite-dimensional settings or infinite-dimensional settings, aiming to highlight good practical performance. To define the leave-out-$(X_i, Y_i)$ estimator for the regression function, we use the following expression:

$$ \Gamma_n(C, x, k) = \frac{\sum_{i=1, i \neq j}^{n} \mathbb{1}_{\{Y_i \in C\}} K\left(\frac{d_{\mathcal{G}}(x, X_i)}{H_{n,k}(x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{G}}(x, X_i)}{H_{n,k}(x)}\right)}, \quad (12) $$

To minimize the quadratic loss function, we consider the following criterion for some known non-negative weight function $\mathcal{W}$:

$$ CV(C, k) = \frac{1}{n} \sum_{j=1}^{n} \left( \mathbb{1}_{\{Y_j \in C\}} - \Gamma_n(C, X_j, k) \right)^2 \mathcal{W}(X_j). \quad (13) $$

This criterion is based on the practical approach developed by [69], where the smoothing parameter is chosen by minimizing the above criterion. The goal is to select $\hat{k}_n$ that minimizes the following expression:

$$ \sup_{C \in \mathcal{C}} CV(C, k). $$

In our analysis, we aim to derive the asymptotic properties of our estimate, even when the bandwidth parameter is treated as a random variable, as suggested in the previous equation. We can refine (13) by compensating with the following:

$$ CV(C, k) = \frac{1}{n} \sum_{j=1}^{n} \left( \mathbb{1}_{\{Y_j \in C\}} - \Gamma_n(C, X_j, k) \right)^2 \hat{W}(X_j, x). \quad (14) $$

In practice, the global uniform weights $\mathcal{W}(X_j) = 1$ are typically used for $j = 1, ..., n$, and the local weights are defined as:

$$ \hat{W}(X_j, x) = \begin{cases} 1 & \text{if } d(X_j, x) \le h, \\ 0 & \text{otherwise.} \end{cases} $$

In this work, we adopt the popular method of cross-validation to select the bandwidth; though this approach can be extended to other bandwidth selection methods, such as those based on Bayesian ideas, as discussed in [71].

**Remark 2.** Establishing the asymptotic theory for a random, cross-validated bandwidth requires a specialized stochastic equicontinuity argument. Cross-validation, as employed by [72], is used to investigate whether two functions (both unconditional and conditional) are equal in mixed
categorical and continuous settings. Although cross-validation performs optimally for estimation, its optimality may not extend to nonparametric kernel testing. In the context of testing a parametric model for the conditional mean function against a nonparametric alternative, ref. [73] introduced an adaptive-rate-optimal rule. An alternative approach to bandwidth selection is proposed by [74], who leverage the Edgeworth expansion of the test’s asymptotic distribution to identify the bandwidth that maximizes test power while controlling its size.

# 4. Application

## 4.1. Statistical Classification

In this section, we apply the findings obtained in the previous sections to the problem of statistical classification. Given a sample of random elements $(X_1, Y_1), \dots, (X_n, Y_n)$ drawn from the joint distribution of $(X, Y)$, where $X$ takes values in a space $\mathcal{E}$ and $Y$ in $\mathbb{R}^d$. In our classification, the objective is to predict the integer-valued label $Y$ based on the covariate vector $X$. More formally, we aim to find a function (classifier) $\theta: \mathcal{E} \to \mathbb{R}^d$ for which the probability of misclassification error (incorrect prediction), i.e., $\mathbb{P}(\theta(X) \neq Y)$, is minimized. Let

$$ \tau_k(x) = \mathbb{P}(Y = k | X = x), \quad x \in \mathcal{E}, 1 \le k \le n. $$

Demonstrating that the optimal classifier, i.e., the one with the minimum probability of error, is given by

$$ \theta_B(x) = \arg \max_{1 \le k \le n} \tau_k(x), $$

i.e., the best classifier, $\theta_B$ satisfies

$$ \max_{1 \le k \le n} \tau_k(x) = \tau_{\theta_B(x)}(x). $$

As $\theta_B$ is unknown, the data are utilized to construct estimates of $\theta_B$. Specifically, let $D_n = (X_1, Y_1), \dots, (X_n, Y_n)$ represent a random sample from the distribution of $(X, Y)$, where each $(X_i, Y_i)$ is fully observable. Let $\hat{\theta}_n$ be any sample-based classifier. In other words, $\hat{\theta}_n(X)$ is the predicted value of $Y$, based on $D_n$ and $X$. Let

$$ L_n(\hat{\theta}_n) = \mathbb{P}(\hat{\theta}_n(X) \neq Y | D_n), $$

be the conditional probability of error of the sample-based classifier $\hat{\theta}_n$. Then $\hat{\theta}_n$ is said to be consistent if $L_n(\hat{\theta}_n) \to L_n(\theta_n) = \mathbb{P}(\theta_B(X) \neq Y)$ as $n \to \infty$, for $k = 1, \dots, n$. Let $\hat{\tau}_k(x)$ be any sample-based estimator of $\tau_k(x) = \mathbb{P}(Y = k | X = x)$ and define the classification rule $\hat{\theta}_n$ by

$$ \hat{\theta}_n(x) = \arg \max_{1 \le k \le n} \hat{\tau}_k(x). $$

In other words, $\hat{\theta}_n$ satisfies

$$ \max_{1 \le k \le n} \hat{\tau}_k(x) = \hat{\tau}_{\hat{\theta}_n(x)}(x), $$

To show

$$ L_n(\hat{\theta}_n) - L_n(\theta_B) \to 0, $$

we put for all $\hat{\tau}_k(x)$, it is sufficient to show that

$$ \hat{\tau}_k(x) - \tau_k(x) \to 0 $$

We can rewrite the conditional distribution as
$$ \Gamma_n(C, x) = \frac{\sum_{i=1}^{n} \hat{\tau}_k(x) \mathbb{1}_{\{Y_i \in C\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}. \quad (15) $$

**Theorem 5.** Under the conditions of Theorem 1, we have the convergence

$$ L_n(\hat{\theta}_n) - L_n(\theta_B) \to 0. $$

## 4.2. Testing Conditional Independence

Conditional independence is an essential concept that unifies many diverse aspects of statistical inference; see [75]. The tasks of quantifying and detecting conditional dependence underlie a wide range of statistical problems, including limit theorems, Markov chain theory, sufficiency, and causality [76], among others. Conditional independence also plays a pivotal role in graphical models [77], causal inference [78], and artificial intelligence [79]; see also [80] for more recent developments. The notion of treating conditional independence as an abstract concept with a dedicated calculus was first introduced by [76], who demonstrated that many results and theorems related to statistical ideas (such as ancillarity, sufficiency, and causality) can be viewed as specific applications of the fundamental properties of conditional independence, extended to include both stochastic and non-stochastic variables. For more details, see [81].

Let $\mathcal{C}_1, \mathcal{C}_2$ be two classes of sets. In this section, we focus on a sample of random elements.

$$ (X_1, Y_{1,1}, Y_{1,2}), \dots, (X_n, Y_{n,1}, Y_{n,2}), $$

which are i.i.d. copies of $(X, Y_1, Y_2)$ taking values in $\mathcal{E} \times \mathbb{R}^{d_1} \times \mathbb{R}^{d_2}$. For $(C_1, C_2) \in \mathcal{C}_1 \times \mathcal{C}_2$, define

$$ \mathbb{G}_n(C_1 \times C_2, x) = \frac{\sum_{i=1}^{n} \mathbb{1}_{\{Y_{i,1} \in C_1\}} \mathbb{1}_{\{Y_{i,2} \in C_2\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}, \quad (16) $$

$$ \mathbb{G}_{n,1}(C_1, x) = \frac{\sum_{i=1}^{n} \mathbb{1}_{\{Y_{i,1} \in C_1\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}, \quad (17) $$

$$ \mathbb{G}_{n,2}(C_2, x) = \frac{\sum_{i=1}^{n} \mathbb{1}_{\{Y_{i,2} \in C_2\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)}. \quad (18) $$

We investigate the following empirical processes for $(C_1, C_2) \in \mathcal{C}_1 \times \mathcal{C}_2$:

$$ \hat{v}_n(C_1, C_2, x) = \sqrt{n \phi(h_n)} \left( \mathbb{G}_n(C_1 \times C_2, x) - \mathbb{E}[\mathbb{G}_n(C_1, x)] \mathbb{E}[\mathbb{G}_n(C_2, x)] \right), \quad (19) $$

$$ \check{v}_n(C_1, C_2, x) = \sqrt{n \phi(h_n)} \left( \mathbb{G}_n(C_1 \times C_2, x) - \mathbb{G}_{n,1}(C_1, x) \mathbb{G}_{n,2}(C_2, x) \right). \quad (20) $$
Observe that

$$
\begin{aligned}
\tilde{v}_n(C_1, C_2, x) = & \sqrt{n \phi(h_n)} (\mathbb{G}_n(C_1 \times C_2, x) - \mathbb{E}[\mathbb{G}_n(C_1, x)] \mathbb{E}[\mathbb{G}_n(C_2, x)]) \\
& + \sqrt{n \phi(h_n)} \mathbb{E}[\mathbb{G}_n(C_2, x)] (\mathbb{G}_n(C_1, x) - \mathbb{E}[\mathbb{G}_n(C_1, x)]) \\
& - \sqrt{n \phi(h_n)} \mathbb{G}_n(C_1, x) (\mathbb{G}_n(C_2, x) - \mathbb{E}[\mathbb{G}_n(C_2, x)]).
\end{aligned}
$$

Hence,

$$
\begin{aligned}
\tilde{v}_n(C_1, C_2, x) &\doteq \sqrt{n \phi(h_n)} (\mathbb{G}_n(C_1 \times C_2, x) - \mathbb{E}[\mathbb{G}_n(C_1, x)] \mathbb{E}[\mathbb{G}_n(C_2, x)]) \\
&\quad + \sqrt{n \phi(h_n)} \mathbb{E}[\mathbb{G}_n(C_2, x)] (\mathbb{G}_n(C_1, x) - \mathbb{E}[\mathbb{G}_n(C_1, x)]) \\
&\quad - \sqrt{n \phi(h_n)} \mathbb{E}[\mathbb{G}_n(C_1, x)] (\mathbb{G}_n(C_2, x) - \mathbb{E}[\mathbb{G}_n(C_2, x)]) \\
&= \hat{v}_n(C_1, C_2, x) + \mathbb{E}[\mathbb{G}_n(C_2, x)] \tilde{v}_n(C_1, x) - \mathbb{E}[\mathbb{G}_n(C_1, x)] \tilde{v}_n(C_2, x), \quad (21)
\end{aligned}
$$

where $\tilde{v}_n(C_k, x)$ is defined analogously.

It can be shown that for $(A_1, B_1), (A_2, B_2) \in \mathcal{C}_1 \times \mathcal{C}_2$,

$$
\begin{aligned}
& \operatorname{cov}(\hat{v}_n(A_1, B_1, x), \hat{v}_n(A_2, B_2, x)) \\
&= \frac{C_2}{C_1^2 f_1(x)} \left( \mathbb{E}[1_{\{Y \in A_1 \cap A_2\}} | X=x] - \mathbb{E}[1_{\{Y \in A_1\}} | X=x] \mathbb{E}[1_{\{Y \in A_2\}} | X=x] \right) \\
&\quad \times \left( \mathbb{E}[1_{\{Y \in B_1 \cap B_2\}} | X=x] - \mathbb{E}[1_{\{Y \in B_1\}} | X=x] \mathbb{E}[1_{\{Y \in B_2\}} | X=x] \right), \quad (22)
\end{aligned}
$$

provided $f_1(x) > 0$. Although the decomposition in (21) suggests the structure of $\tilde{v}_n(C_1, C_2, x)$, its covariance is more delicate to derive. Let $\{\hat{v}(C_1, C_2, x) : (C_1, C_2) \in \mathcal{C}_1 \times \mathcal{C}_2\}$ be a Gaussian process whose covariance is given by (22). Define the following limiting process for $(C_1, C_2) \in \mathcal{C}_1 \times \mathcal{C}_2$:

$$
\tilde{v}(C_1, C_2, x) = \hat{v}(C_1, C_2, x) + \mathbb{G}(C_2, x) \tilde{v}(C_1, x) - \mathbb{G}(C_1, x) \tilde{v}(C_2, x).
$$

We aim to test the null hypothesis

$\mathcal{H}_0: Y_1$ and $Y_2$ are conditionally independent given $X=x$,

against the alternative

$\mathcal{H}_1: Y_1$ and $Y_2$ are conditionally dependent.

To this end, one may use the following test statistics:

$$
S_{1,n} = \sup_{(C_1,C_2) \in \mathcal{C}_1 \times \mathcal{C}_2} \left| \hat{v}_n(C_1,C_2,x) \right|, \quad (23)
$$

$$
S_{2,n} = \sup_{(C_1,C_2) \in \mathcal{C}_1 \times \mathcal{C}_2} \left| \tilde{v}_n(C_1,C_2,x) \right|. \quad (24)
$$

By combining Theorem 4 with the continuous mapping theorem, we obtain the following result.
**Theorem 6.** Under the conditions of Theorem 4, as $n \to \infty$, it holds that

$$S_{1,n} \to \sup_{(C_1,C_2) \in \mathcal{C}_1 \times \mathcal{C}_2} |\hat{v}(C_1, C_2, x)|, \qquad (25)$$

$$S_{2,n} \to \sup_{(C_1,C_2) \in \mathcal{C}_1 \times \mathcal{C}_2} |\check{v}(C_1, C_2, x)|. \qquad (26)$$

## 5. Concluding Remarks

In this paper, we present new limit theorems for an improved version of the set-indexed conditional empirical process, with particular attention to the interplay between mixing conditions and functional aspects of the data. A central feature of our contribution is the incorporation of the widely recognized k-nearest neighbors (k-NN) approach for estimating the regression function. Building on the results in [29], we establish analogous theorems on weak convergence in probability and asymptotic normality. Furthermore, our findings bolster existing research by refining the density properties inherent to the process under consideration.

The k-NN algorithm, valued for its non-parametric nature and relative simplicity, is well-suited to both classification and regression tasks. One principal advantage of applying k-NN in time series settings lies in its capacity to function effectively without strict assumptions regarding the underlying data distribution, thus making it particularly appropriate for non-stationary data. In addition to its methodological adaptability, k-NN is notable for its computational tractability and straightforward implementation. Consequently, the insights provided in this study hold the potential to guide future methodological developments. We also advocate for the utility of bootstrap procedures in functional data contexts, with the functional form of the wild bootstrap illustrating a promising avenue for selecting smoothing parameters. Nevertheless, the theoretical underpinnings that justify this bandwidth selection strategy via the functional wild bootstrap remain an open question.

It is widely recognized that bias constitutes a key challenge in kernel smoothing. Although local linear fitting offers a theoretical avenue for bias mitigation, there is a pressing need for more extensive investigation within the realm of nonparametric functional data analysis in order to comprehensively address bias-related issues. Empirical evidence suggests that bias correction strategies can markedly improve estimation accuracy. At the same time, further theoretical rigor is essential for understanding and controlling the asymptotic properties of these enhanced approaches. To the best of our knowledge, such formal examinations are still underrepresented in nonparametric functional data analysis, particularly in the context of locally stationary processes. Against this backdrop, a promising objective is to refine estimation procedures through the integration of local linear smoothing with established bias-correction techniques. Contributions from [82] and related foundational works [83] offer a natural basis for developing such integrative methods. Extending our current framework to simultaneously incorporate k-NN and single index models could generate more robust and practically relevant outcomes, thereby paving the way for subsequent empirical investigations.

It is worth noting that the notion of mixing, while commonly adopted due to its relative tractability, may be ill-suited for data exhibiting substantial dependence. Extending non-parametric functional methodologies to more general dependence structures remains an underexplored area. In this regard, ergodic frameworks avoid the stringent requirements of strong mixing assumptions and the intricate probabilistic arguments they necessitate. Nonetheless, adapting our approach to accommodate functional ergodic data would involve considerable mathematical complexity and thus exceeds the present scope.
**Author Contributions:** Conceptualization, S.B.; methodology, S.B.; validation, S.B. and Y.S.; formal analysis, S.B. and Y.S.; investigation, S.B. and Y.S.; original draft preparation, S.B. and Y.S.; writing—review and editing, S.B. and Y.S.; supervision, S.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Data Availability Statement:** Data are contained within the article.

**Acknowledgments:** The authors would like to thank the Editor-in-Chief, an Associate-Editor, and the three referees for their extremely helpful remarks, which resulted in a substantial improvement of the original form of the work and a presentation that was more sharply focused.

**Conflicts of Interest:** The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Appendix A

We present Lemma A1 in a more general context, as seen in [28]. This generalization extends a result found in [32] and may prove useful in a broader range of scenarios than the one we discuss here. More generally, this technique is valuable for handling random bandwidths.

Let $(A_i, B_i)_{n \ge 1}$ be a sequence of random vectors valued in $(\Omega, \Omega'; \mathcal{A}, \mathcal{B})$, a general space. Let $S_\Omega$ be a fixed subset of $\Omega$, and define the function $G: \mathbb{R} \times (S_\Omega \times \Omega) \to \mathbb{R}^+$ such that, for all $t \in S_\Omega$, the function $G(\cdot, (t, \cdot))$ is measurable. Moreover, for all $x, x' \in \mathbb{R}$, the following condition holds:

$$ (L_0) \qquad x \le x' \Rightarrow G(x, \mathbf{z}) \le G(x', \mathbf{z}), \quad \forall \mathbf{z} \in S_\Omega \times \Omega. $$

Next, let $(D_n(t))_{n \in \mathbb{N}}$ be a sequence of random real vectors (r.r.v.) such that for every $t \in S_\Omega$ and for each measurable function $1_{\{\cdot\}}: \Omega' \to \mathbb{R}$ belonging to a class of sets $\mathcal{C}$, define the nonrandom function $\mathfrak{M}: \mathcal{C} \times S_\Omega \to \mathbb{R}$ with the condition:

$$ \sup_{C \in \mathcal{C}} \sup_{t \in S_\Omega} |\mathfrak{M}(C, t)| < \infty. $$

For all $t \in S_\Omega$ and $n \in \mathbb{N} \setminus \{0\}$, we define the sequence

$$ \mathfrak{M}_n(C, t) = \frac{\sum_{i=1}^{n} 1_{B_i} G(t, A_i)}{\sum_{i=1}^{n} G(t, A_i)}. $$

**Lemma A1.** Let $(\mathcal{U}_n)_{n \in \mathbb{N}}$ be a decreasing sequence of positive numbers such that $\lim_{n \to \infty} \mathcal{U}_n = 0$. For every increasing sequence $\xi_n \in (0, 1)$ with $\xi_n - 1 = O(\mathcal{U}_n)$, there exist two sequences of random real vectors $(D_{n,k}^-(\xi_n, t))_{n \in \mathbb{N}}$ and $(D_{n,k}^+(\xi_n, t))_{n \in \mathbb{N}}$ satisfying the following conditions:

$$ (L_1) \qquad \forall n \in \mathbb{N}, \quad \forall t \in S_\Omega, \quad D_{n,k}^-(\xi_n, t) \le D_{n,k}^+(\xi_n, t). $$

$$ (L_2) \qquad 1_{D_{n,k}^-(\xi_n, t) \le D_{n,k}(t) \le D_{n,k}^+(\xi_n, t)} \to 1, \quad \text{a.s.,} \quad \forall t \in S_\Omega. $$

$$ (L_3) \qquad \sup_{C \in \mathcal{C}} \sup_{t \in S_\Omega} \left| \frac{\sum_{i=1}^{n} G(D_{n,k}^-(\xi_n, t), (t, A_i))}{\sum_{i=1}^{n} G(D_{n,k}^+(\xi_n, t), (t, A_i))} - \xi_n \right| = O_{\text{a.s.}}(\mathcal{U}_n), $$
$$ (L_4) \qquad \sup_{C \in \mathcal{C}} \sup_{t \in S_{\Omega}} \left| \mathfrak{M}_n(C, t; D_{n,k}^{-}(\xi_n, t)) - \mathfrak{M}_n(C, t) \right| = O_{\text{a.s.}}(\mathcal{U}_n), $$

$$ (L_5) \qquad \sup_{C \in \mathcal{C}} \sup_{t \in S_{\Omega}} \left| \mathfrak{M}_n(C, t; D_{n,k}^{+}(\xi_n, t)) - \mathfrak{M}_n(C, t) \right| = O_{\text{a.s.}}(\mathcal{U}_n). $$

Then, as $n \to \infty$, the following holds:

$$ \sup_{C \in \mathcal{C}} \sup_{t \in S_{\Omega}} \left| \mathfrak{M}_n(C, t; D_{n,k}(t)) - \mathfrak{M}_n(C, t) \right| = O_{\text{a.s.}}(\mathcal{U}_n). \qquad (A1) $$

For the proof of Lemma A1, we refer to [28].

Here is a reformulation of the lemma with the original content preserved:

**Lemma A2** (Lemma 6.1 in [34], p.186)). Let $X_1, \dots, X_n$ be independent Bernoulli random variables with $\mathbb{P}(X_i = 1) = p$ for each $i = 1, \dots, n$. Define

$$ U = X_1 + X_2 + \dots + X_n $$

and $\mu = pn$. Then, for any $\omega \ge 1$, the following bound holds:

$$ \mathbb{P}(U \ge (1 + \omega)\mu) \le \exp\left(-\frac{\mu \min(\omega^2, \omega)}{4}\right), $$

and if $\omega \in (0, 1)$, we have

$$ \mathbb{P}(U \le (1 - \omega)\mu) \le \exp\left(-\frac{\mu\omega^2}{2}\right). $$

The following notation is necessary for determining the convergence rates:

$$ \Delta_i(x, h_K(x)) = K \left( \frac{d_{\mathcal{E}}(x, X_i)}{h_K(x)} \right), $$

$$ \Gamma_{n,2}(C, x; h_k(x)) = \frac{1}{n\mathbb{E}(\Delta_1(x; h_k(x)))} \sum_{i=1}^{n} 1_{\{Y_i \in C\}} \Delta_i(x, h_K(x)), $$

$$ \Gamma_{n,1}(C, x; h_k(x)) = \frac{1}{n\mathbb{E}(\Delta_1(x; h_k(x)))} \sum_{i=1}^{n} \Delta_i(x, h_K(x)). $$

This leads to the expression:

$$ \Gamma_n(C, x; h_k(x)) = \frac{\Gamma_{n,2}(C, x; h_k(x))}{\Gamma_{n,1}(C, x; h_k(x))}. $$

Now, consider the following decomposition:

$$ \begin{aligned} & \Gamma_n(C, x; h_k(x)) - \Gamma(C, x) \\ = & \frac{1}{\Gamma_{1,2}(C, x; h_k(x))} \{ \Gamma_{n,2}(C, x; h_k(x)) - \mathbb{E}[\Gamma_{n,2}(C, x; h_k(x))] \} \\ & + \frac{1}{\Gamma_{n,1}(C, x; h_k(x))} \{ \mathbb{E}[\Gamma_{n,2}(C, x; h_k(x))] - \Gamma(C, x) \} \\ & + \frac{\Gamma(C, x)}{\Gamma_{n,2}(C, x; h_k(x))} \{ 1 - \Gamma_{n,2}(C, x; h_k(x)) \}. \end{aligned} $$
Appendix A.1. Proof of Theorem 1

As in [28], we proceed to prove Theorem 1. To ascertain the condition in (A1), we first identify the variables as follows: $S_{\Omega} = S_{\mathcal{E}}$, $A_i = X_i$, $1_{\{B_i \in C\}} = 1_{\{Y_i \in C\}}$,

$$G(H, (x; A_i)) = K(H^{-1}d_{\mathcal{E}}(x, X_i));$$

$$D_{n,k}(x) = H_{n,k}(x);$$

$$\mathfrak{M}_n(C, x; H_{n,k}(x)) = \Gamma_n(C, x; H_{n,k}(x));$$

$$\mathfrak{M}(C, x) = \Gamma(C, x).$$

Next, choose $D_{n,k}^{-}(\xi_n, x)$ and $D_{n,k}^{+}(\xi_n, x)$:

$$\phi_t(D_{n,k}^{-}(\xi_n, x)) = \frac{\sqrt{\xi_n}k}{n}, \qquad (A2)$$

$$\phi_t(D_{n,k}^{+}(\xi_n, x)) = \frac{k}{n\sqrt{\xi_n}}. \qquad (A3)$$

We define $h^{-}(x) = D_{n,k}^{-}(\xi_n, x)$, $h^{+}(x) = D_{n,k}^{+}(\xi_n, x)$, and consider an increasing sequence $\xi_n \in (0,1)$ such that $\xi_n - 1 = O(\mathcal{U}_n)$. For all $\xi_n \in (0,1)$ and $x \in S_{\mathcal{E}}$, with $k_{1,n} \le k \le k_{2,n}$, we have:

$$\phi_x^{-1}\left(\frac{\sqrt{\xi_n}k_{1,n}}{n}\right) \le h^{-}(x) \le \phi_x^{-1}\left(\frac{\sqrt{\xi_n}k_{2,n}}{n}\right),$$

$$\phi_x^{-1}\left(\frac{k_{1,n}}{n\sqrt{\xi_n}}\right) \le h^{+}(x) \le \phi_x^{-1}\left(\frac{k_{2,n}}{n\sqrt{\xi_n}}\right).$$

Using the condition (8), we can deduce that the bandwidths $h^{-}(x)$ and $h^{+}(x)$ both belong to the interval:

$$[h_{n,1}, h_{n,2}] = \left[ \mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right), \nu\phi^{-1}\left(\frac{k_{1,n}}{n\sqrt{\xi_n}}\right) \right].$$

**Checking conditions (L4) and (L5)** To check (L4), we begin by noting that since $\xi_n$ is bounded by 1, the local bandwidth $h^{-}(x)$ leads to:

$$\sup_{C \in \mathcal{C}} \sup_{h_{n,1} \le h^{-}(x) \le h_{n,2}} \sup_{x \in S_{\mathcal{E}}} \left| \mathfrak{M}_n(C, x; D_{n,k}^{-}(\xi_n, x)) - \mathfrak{M}_n(C, x) \right| \\
= \sup_{h_{n,1} \le h^{-}(x) \le h_{n,2}} \sup_{x \in S_{\mathcal{E}}} \left| \frac{\sum_{i=1}^{n} 1_{\{Y_i \in C\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{h^{-}(x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{h^{-}(x)}\right)} - \Gamma(C, x) \right| \\
= \sup_{h_{n,1} \le h^{-}(x) \le h_{n,2}} \sup_{x \in S_{\mathcal{E}}} \left| \Gamma_n(C, x; h^{-}(x)) - \Gamma(C, x) \right| \\
= O_{a,co}\left\{ h_{n,2}^{\gamma} + O_{a,co}\left\{ \sqrt{\frac{\psi_{S_x}\left(\frac{\log n}{n}\right)}{n\phi(h_{n,1})}} \right\} \right\}.$$

This is equivalent to:

$$\sup_{C \in \mathcal{C}} \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{E}}} \left| \Gamma_n(C, x; h^{-}(x)) - \Gamma(C, x) \right| \\
= O_{a,co} \left( \phi^{-1} \left( \frac{k_{2,n}}{\sqrt{\xi_n n}} \right)^{\gamma} + \sqrt{ \frac{\psi_{S_x} \left( \frac{\log n}{n} \right)}{n\phi \left( \mu\phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)} } \right) \\
= O_{a,co}(\mathcal{U}_n).$$
We use similar reasoning to verify condition (L5), and we obtain:

$$ \sup_{C \in \mathcal{C}} \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{C}}} \left| \mathfrak{M}_n(C, x; D_{n,k}^{-}(\xi_n, x)) - \mathfrak{M}_n(C, x) \right| = O_{a,co}(\mathcal{U}_n). $$

Thus, conditions (L4) and (L5) are verified.

**Verifying condition (L2)** To verify condition (L2), we show that for all $x \in S_{\mathcal{C}}$ and $\epsilon_0$,

$$ \sum_{n \ge 1} \mathbb{P} \left\{ \left| \mathbb{1}_{D_{n,k}^{-}(\xi_n, x)} \le H_{n,k}(x) \le D_{n,k}^{+}(\xi_n, x) - 1 \right| > \epsilon_0 \right\} < \infty. \quad (A4) $$

For fixed $\epsilon_0$, let $\{x_1, \dots, x_{N_{\mathcal{C}}}\}$ be an $\epsilon$-net for $S_{\mathcal{C}}$. For each $x \in S_{\mathcal{C}}$, we have:

$$ \begin{aligned} & \mathbb{P}\left\{\left|\mathbb{1}_{D_{n,k}^{-}(\xi_n, x)} \le H_{n,k}(x) \le D_{n,k}^{+}(\xi_n, x) - 1\right| > \epsilon_0\right\} \\ & \le \mathbb{P}\left(H_{n,k}(x) \le \phi_x^{-1}\left(\frac{\sqrt{\xi_n}k_{1,n}}{n}\right)\right) + \mathbb{P}\left(H_{n,k}(x) \ge \phi_x^{-1}\left(\frac{k_{2,n}}{\sqrt{\xi_n}n}\right)\right) \\ & \le \sum_{\ell=1}^{N_{\epsilon n}(S_{\mathcal{C}})} \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(x_{\ell}) \le \phi_x^{-1}\left(\frac{k_{2,n}}{\sqrt{\xi_n}n}\right)\right) \\ & \qquad + \sum_{\ell=1}^{N_{\epsilon n}(S_{\mathcal{C}})} \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(x_{\ell}) \ge \phi_x^{-1}\left(\frac{\sqrt{\xi_n}k_{1,n}}{n}\right)\right) \\ & \le N_{\epsilon n}(S_{\mathcal{C}}) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(x_{\ell}) \le \phi_x^{-1}\left(\frac{\sqrt{\xi_n}k_{1,n}}{n}\right)\right) \\ & \qquad + N_{\epsilon n}(S_{\mathcal{C}}) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(x_{\ell}) \ge \phi_x^{-1}\left(\frac{k_{2,n}}{\sqrt{\xi_n}n}\right)\right). \end{aligned} $$

We begin by stating a lemma similar to the one presented in [34] (see Lemma A2). For the sake of completeness, we include the proof here.

By applying Lemma A2, we deduce that

$$ \begin{aligned} & \mathbb{P}\left\{H_{n,k}(x) \le \phi_x^{-1}\left(\frac{\alpha k_{1,n}}{n}\right)\right\} \\ &= \mathbb{P}\left\{\sum_{i=1}^{n} \mathbb{1}_{B\left(x, \phi_x^{-1}\left(\frac{\alpha k_{1,n}}{n}\right)\right)}(X_i) > k\right\} \\ &= \mathbb{P}\left\{\sum_{i=1}^{n} \mathbb{1}_{B\left(x, \phi_x^{-1}\left(\frac{\alpha k_{1,n}}{n}\right)\right)}(X_i) > \frac{k}{\alpha k_{1,n}}\right\} \\ &\le \exp\left\{-\frac{k - \alpha k_{1,n}}{4}\right\}. \end{aligned} \quad (A5) $$

From this, we obtain the following inequality:

$$ \begin{aligned} & N_{\epsilon n}(S_{\mathcal{C}}) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P}\left(H_{n,k}(x) \le \phi_x^{-1}\left(\frac{k_{1,n}}{\alpha n}\right)\right) \\ & \le N_{\epsilon n}(S_{\mathcal{C}}) k_{2,n} \exp\left\{-\frac{1 - \alpha k_{1,n}}{4}\right\} \\ & \le N_{\epsilon n}(S_{\mathcal{C}}) n^{1 - \frac{(1-\alpha)k_{1,n}}{4 \ln n}}. \end{aligned} $$

Similarly, we derive the bound

$$ \mathbb{P}\left\{H_{n,k}(x) \ge \phi_x^{-1}\left(\frac{k_{2,n}}{\alpha n}\right)\right\} \le \exp\left\{-\frac{(k_{2,n} - \alpha k)^2}{2\alpha K_{2,n}}\right\}. \quad (A6) $$
Thus, we conclude that

$$
\begin{aligned}
\mathcal{N}_{\varepsilon n}(S_{\mathcal{E}}) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P} \left( H_{n,k}(x) \ge \phi_x^{-1} \left( \frac{k_{2,n}}{\alpha n} \right) \right) &\le \mathcal{N}_{\varepsilon n}(S_{\mathcal{E}}) k_{2,n} \exp \left\{ -\frac{1 - \alpha k_{1,n}}{4} \right\} \\
&\le \mathcal{N}_{\varepsilon n}(S_{\mathcal{E}}) n^{1 - \frac{(1-\alpha) k_{2,n}}{4 \ln n}}.
\end{aligned}
$$

Therefore, as $k_{i,n} / \ln n \to \infty$ for $i=1,2$, we conclude

$$
\mathcal{N}_{\epsilon n}(S_{\mathcal{E}}) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P} \left( H_{n,k}(x) \le \phi_x^{-1} \left( \frac{k_{1,n}}{\alpha n} \right) \right) \le \mathcal{N}_{\epsilon n}(S_{\mathcal{E}}) n^{1 - \frac{(1-\alpha) k_{1,n}}{4 \ln n}}, \quad (A7)
$$

$$
\mathcal{N}_{\epsilon n}(S_{\mathcal{E}}) \sum_{k=k_{1,n}}^{k=k_{2,n}} \mathbb{P} \left( H_{n,k}(x) \ge \phi_x^{-1} \left( \frac{k_{2,n}}{\alpha n} \right) \right) \le \mathcal{N}_{\epsilon n}(S_{\mathcal{E}}) n^{1 - \frac{(1-\alpha) k_{2,n}}{4 \ln n}}. \quad (A8)
$$

Next, to verify the condition (L3), we introduce the following quantities:

$$
Q_{n1} := \frac{\Delta_1(x, D_n^-(\xi_n, x))}{\Delta_1(x, D_n^+(\xi_n, x))},
$$

$$
Q_{n2} := \frac{\Gamma_{n,1}(x, D_n^-(\xi_n, x))}{\Gamma_{n,1}(C, x; D_n^+(\xi_n, x))} - 1,
$$

$$
Q_{n3} := \frac{\Delta_1(x, D_n^-(\xi_n, x))}{\Delta_1(x, D_n^+(\xi_n, x))} \xi_n - 1.
$$

The condition (L3) is then expressed as

$$
\left| \frac{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{D_n^-(\xi_n, x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{D_n^+(\xi_n, x)}\right)} - \xi_n \right| \le |Q_{n1}| |Q_{n2}| + |Q_{n1}| |Q_{n3}|.
$$

Given that $\xi_n \to 1$, the desired result follows, leading to

$$
\sup_{K_{n,1} \le K \le K_{n,2}} \sup_{x \in S_{\mathcal{E}}} \left| \frac{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{D_n^-(\xi_n, x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{D_n^+(\xi_n, x)}\right)} - \xi_n \right| = O_{a,co}(\mathcal{U}_n). \quad (A9)
$$

**Proof of (A9).** We use the following results:

$$
\sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{E}}} |Q_{n1}| \le C, \quad (A10)
$$

$$
\sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{E}}} |Q_{n2}| = O_{a,co} \left( \sqrt{\frac{\psi_{S_x} \left( \frac{\log n}{n} \right)}{n \phi \left( \mu \phi^{-1} \left( \frac{\rho_n k_{1,n}}{n} \right) \right)}} \right), \quad (A11)
$$

$$
\sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{E}}} |Q_{n3}| = O\left(\phi^{-1}\left(\frac{k_{2,n}}{\rho_n n}\right)\right). \quad (A12)
$$

<figure><img src="image_3.png" /></figure>

**Proof of (A10).** We use the condition(H3(i)) to conclude that

$$
\mathbb{E}\left(K\left(\frac{d_{\mathcal{E}}(X_1, 1)}{h(x)}\right)\right) \leq \eta_2\phi(h(x)).
$$
Then, by applying condition (H1(ii)), we immediately obtain

$$ \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} |Q_{n1}| \le C. \quad (A13) $$

<figure><img src="image_4.png" /></figure>

**Proof of (A11).** We begin with the following expression:

$$ \begin{aligned} \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} |Q_{n2}| &= \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} \left| \frac{Q_n(k, x, h^-(x))}{Q_n(k, x, h^+(x))} - 1 \right| \\ &\le \frac{1}{\inf_{k_{n,1} \le k \le k_{n,2}} \inf_{x \in S_{\mathcal{G}}} |Q_n(K, x, h^+(x))|} \left( \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} \left| Q_n(K, x, h^-(x)) - 1 \right| + \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} \left| Q_n(K, x, h^+(x)) - 1 \right| \right). \end{aligned} \quad (A14) $$

To establish this inequality, which provides the following results:

$$ \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} |Q_n(K, x, h^-(x)) - 1| = O_{a,co} \left( \sqrt{\frac{\psi_{S_x}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right)\right)}} \right), \quad (A15) $$

and

$$ \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} |Q_n(K, x, h^+(x)) - 1| = O_{a,co} \left( \sqrt{\frac{\psi_{S_x}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right)\right)}} \right). \quad (A16) $$

Additionally, combining (A15), (A16) with the fact that

$$ \sum_{n \ge 1} \mathbb{P} \left( \inf_{k_{n,1} \le k \le k_{n,2}} \inf_{x \in S_{\mathcal{G}}} \Gamma_{n,1}(C, x; D_n^+(\xi_n, x)) < C \right) < \infty, \quad (A17) $$

it follows that

$$ \sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{G}}} |Q_{n2}| = O_{a,co} \left( \sqrt{\frac{\psi_{S_x}\left(\frac{\log n}{n}\right)}{n\phi\left(\mu\phi^{-1}\left(\frac{\rho_n k_{1,n}}{n}\right)\right)}} \right). \quad (A18) $$

<figure><img src="image_5.png" /></figure>

**Proof of (A12).** As used in [33], part of the proof follows from Lemma 1 in [84]. Additionally, we will rely on computations similar to those in the proof of Lemma 5.7.0.3 in [85]. Consider the following quantity:

$$ \hat{B}(x, CK) = \frac{\mathbb{E}[\Gamma_{n,2}(C, x; D_n^+(\xi_n, x))]}{\mathbb{E}[\Gamma_{n,1}(C, x; D_n^-(\xi_n, x))]} - \Gamma(C, x). $$

Now, analyzing the quantity:
$$
\begin{aligned}
& \left| \hat{B}(x, CK) \right| \\
&= \left| \frac{\phi_x(D_n^-(\xi_n, x))}{\phi_x(D_n^+(\xi_n, x))} \frac{\mathbb{E}\left[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^+(\xi_n, x)}\right)\mathbb{E}\left[\mathbf{1}_{\{Y_1 \in C\}} | X_1\right]\right]}{\mathbb{E}\left[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^-(\xi_n, x)}\right)\right]} - \Gamma(C, x) \right| \\
&\le \frac{1}{\mathbb{E}\left[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^-(\xi_n, x)}\right)\right]} \left| \frac{\phi_x(D_n^-(\xi_n, x))}{\phi_x(D_n^+(\xi_n, x))} \mathbb{E}\left[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^+(\xi_n, x)}\right)\mathbb{E}\left[\mathbf{1}_{\{Y_1 \in C\}} | X_1\right] - \Gamma(C, x)\right] \right|.
\end{aligned}
$$

Using the fact that $\frac{\phi_x(D_n^-(\xi_n, x))}{\phi_x(D_n^+(\xi_n, x))} = \xi_n$, and assuming the condition (H2(i)) holds, i.e.,

$$|\Gamma(X | x) - \Gamma(C | x)| \leq \eta_1 d_{\mathcal{E}}^{\gamma}(X, x),$$

and that conditions (H1(ii)) and (H3(i)) are satisfied, we then have for all $x \in S_{\mathcal{E}}$, $D_n^-(\xi_n, x)$, and $D_n^+(\xi_n, x)$:

$$
\begin{aligned}
\left| \frac{\mathbb{E}[\Gamma_{n,2}(C, x; D_n^+(\xi_n, x))] - \Gamma(C, x)}{\mathbb{E}[\Gamma_{n,1}(C, x; D_n^+(\xi_n, x))]} \right| &\le \frac{\eta_3 \kappa_2 \xi_n}{\eta_1 \phi_x(D_n^-(\xi_n, x))} \left[ \mathbb{E} \mathbf{1}_{B(x, D_n^+(\xi_n, x))} (X) d_{\mathcal{E}}^{\gamma}(X, x) \right] \\
&\le \frac{\eta_1 \eta_2 \xi_n \phi_x(D_n^-(\xi_n, x))}{\eta_1 \phi_x(D_n^+(\xi_n, x))} (D_n^+(\xi_n, x))^{\gamma} \\
&\le \frac{C_2 C_3 \eta_2 \xi_n}{C_1 \eta_1} (D_n^+(\xi_n, x))^{\gamma} \\
&\le C (D_n^+(\xi_n, x))^{\gamma}.
\end{aligned}
$$

Taking into account the condition (H3(i)) and the fact that $\xi_n \to 1$, we conclude:

$$\sup_{(h^{-}(x), h^{+}(x)) \in [h_{n,1}, h_{n,2}]^{2}} \sup_{x \in S_{\mathcal{E}}} \left| \frac{\mathbb{E}[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^+(\xi_n, x)}\right)]}{\mathbb{E}[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^-(\xi_n, x)}\right)]} \xi_n - 1 \right| \le C' h_{n,2}^{\gamma}. \quad (A19)$$

Finally, rewriting (A19) with $C \equiv 1$, we obtain:

$$\sup_{(h^{-}(x), h^{+}(x)) \in [h_{n,1}, h_{n,2}]^{2}} \sup_{x \in S_{\mathcal{E}}} \left| \frac{\mathbb{E}[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^+(\xi_n, x)}\right)]}{\mathbb{E}[K\left(\frac{d_{\mathcal{E}}(x, X_1)}{D_n^-(\xi_n, x)}\right)]} \xi_n - 1 \right| \le C' h_{n,2}^{\gamma}.$$

This is equivalent to

$$\sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{E}}} |Q_{n3}| = O\left(\phi^{-1}\left(\frac{k_{2,n}}{\rho_n n}\right)\right). \quad (A20)$$

Combining the results from (A10), (A18), and (A20), and noting that $\xi_n \to 1$, we conclude:

$$\sup_{k_{n,1} \le k \le k_{n,2}} \sup_{x \in S_{\mathcal{E}}} \left| \frac{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{D_n^-(\xi_n, x)}\right)}{\sum_{i=1}^{n} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{D_n^+(\xi_n, x)}\right)} - \xi_n \right| = O_{a,co}(U_n).$$

Thus, condition (L3) is satisfied. It is clear that condition (L0) is satisfied by (H3(i)), and condition (L1) is trivially satisfied due to the construction of $D_n^-(\xi_n, x)$ and $D_n^+(\xi_n, x)$, as well as the application of (A1). This completes the proof of Theorem 1. □
## Appendix A.2. Proof Theorem 2

To prove Theorem 2, we use the Cramér-Wold device, which means that it is enough to establish the convergence of the one-dimensional distribution. Moreover, due to the linearity of $\Lambda(C; k_K | x)$, it suffices to show that

$$ \Lambda(\Phi; H_{n,k}(x) | x) \rightarrow \mathcal{N}(0, \sigma^2(\Phi, x)), $$

for all $\Phi$ of the form

$$ \Phi = \sum_{p=1}^{L} c_p 1_{C_p}, \quad c_1, \dots, c_L \in \mathbb{R}, \quad C_1, \dots, C_L \in \mathcal{C}. $$

Thus, our task reduces to proving convergence in one dimension. Recall that we are working with

$$ \begin{aligned} \Lambda(C; H_{n,k}(x) | x) &= \sqrt{n\phi(h_n)} (\Gamma_n(C, x; H_{n,k}(x)) - \Gamma(C, x)) \\ &= \sqrt{n\phi(h_n)} \left( \frac{\sum_{i=1}^n 1_{\{Y_i \in C\}} K\left(\frac{d_{\mathcal{G}}(x, X_i)}{H_{n,k}(x)}\right)}{\sum_{i=1}^n K\left(\frac{d_{\mathcal{G}}(x, X_i)}{H_{n,k}(x)}\right)} - \Gamma(C, x) \right), \end{aligned} $$

To achieve the desired result, we decompose as follows:

$$ \begin{aligned} & \Gamma_n(C, x; H_{n,k}(x)) - \Gamma(C, x) \\ &= \frac{\Gamma_{n,2}(C, x; H_{n,k}(x))}{\Gamma_{n,1}(1, x; H_{n,k}(x))} - \Gamma(C, x) \\ &= \frac{1}{\Gamma_{n,1}(1, x; H_{n,k}(x))} \left[ \begin{aligned} & (\Gamma_{n,2}(C, x; H_{n,k}(x)) - \Gamma_{n,2}(C_M, x; H_{n,k}(x))) \\ & - (\Gamma(C, x) - \mathbb{E}(\Gamma_{n,2}(C, x; H_{n,k}(x))))) \\ & + (\mathbb{E}(\Gamma_{n,2}(C_M, x; H_{n,k}(x))) - \mathbb{E}(\Gamma_{n,2}(C, x; H_{n,k}(x)))) \\ & + (\Gamma_{n,2}(C_M, x; H_{n,k}(x))) - \mathbb{E}(\Gamma_{n,2}(C_M, x; H_{n,k}(x))) - \Gamma(C, x) \\ & \cdot (\Gamma_{n,1}(1, x; H_{n,k}(x)) - 1) \end{aligned} \right] \\ &= \frac{1}{\Gamma_{n,1}(1, x; H_{n,k}(x))} [I_1(x; H_{n,k}) + I_2(x; H_{n,k})], \end{aligned} $$

where

$$ \begin{aligned} I_1(x; H_{n,k}) = & (\Gamma_{n,2}(C, x; H_{n,k}(x)) - \Gamma_{n,2}(C_M, x; H_{n,k}(x))) \\ & - \mathbb{E}(\Gamma_{n,2}(C, x; H_{n,k}(x)) - \Gamma_{n,2}(C_M, x; H_{n,k}(x))) \\ & + \mathbb{E}(\Gamma_{n,2}(C_M, x; H_{n,k}(x))) - \Gamma(C, x), \end{aligned} \quad (A21) $$

and

$$ \begin{aligned} I_2(x; H_{n,k}) = & (\Gamma_{n,2}(C_M, x; H_{n,k}(x)) - \mathbb{E}(\Gamma_{n,2}(C_M, x; H_{n,k}(x)))) \\ & - \Gamma(C, x)(\Gamma_{n,1}(1, x; H_{n,k}(x)) - 1). \end{aligned} $$

**Lemma A3.** Under the assumptions (H2)(i), (H2)(ii), (H3)(ii), (H4)(i), and (H8), we have

$$ \sqrt{n\phi(h_k)} I_1(x; H_{n,k}(x)) \xrightarrow{\mathbb{P}} 0 \quad \text{as} \quad n \rightarrow \infty. $$
Appendix A.3. Proof of Lemma A3

Let us consider the term $I_1(x, H_{n,k}(x))$, where $1_{D_n^-} \le H_{n,k}(x) \le 1_{D_n^+}$ holds as $\frac{k}{n} \to 0$. By applying condition (H3) (ii), we obtain the inequality:

$$ \Gamma_{n,2}(C, x; D_n^+) \le \Gamma_{n,2}(C, x; H_{n,k}(x)) \le \Gamma_{n,2}(C, x; D_n^-). \quad (A22) $$

Next, we have:

$$ \sqrt{n\phi(h_k)}|S_1(x, H_{n,k}(x)) - \mathbb{E}(S_1(x, H_{n,k}(x)))| \le \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C, x, H_{n,k}(x)) - \mathbb{E}(\Gamma_{n,2}(C, x, H_{n,k}(x)))| \quad (A23) $$

$$ + \sqrt{n\phi(h_k)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, H_{n,k}(x))) - \Gamma_{n,2}(C_M, x, H_{n,k}(x))| \quad (A24) \\ = I_{11}(x, H_{n,k}(x)) + I_{12}(x, H_{n,k}(x)). $$

From Equation (A23), we have:

$$ \begin{aligned} |I_{11}(x, H_{n,k}(x))| &= \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C, x, H_{n,k}(x)) - \mathbb{E}(\Gamma_{n,2}(C, x, H_{n,k}(x)))| \\ &\le \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C, x, D_n^+))| \\ &\le \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C, x, D_n^-))| \end{aligned} \quad (A25) $$

$$ + \sqrt{n\phi(h_k)}\mathbb{E}(\Gamma_{n,2}(C, x, D_n^-)) + \mathbb{E}(\Gamma_{n,2}(C, x, D_n^+)). \quad (A26) $$

Similarly, from Equation (A24), we have:

$$ \begin{aligned} |I_{12}(x, H_{n,k}(x))| &= \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C_M, x, H_{n,k}(x)) - \mathbb{E}(\Gamma_{n,2}(C_M, x, H_{n,k}(x)))| \\ &\le \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C_M, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-))| \\ &\quad + \sqrt{n\phi(h_k)}\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) + \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+)). \end{aligned} \quad (A27) $$

We can apply the following decomposition to handle Equation (A25):

$$ \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C, x, D_n^-))| \le \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C, x, D_n^-) - \Gamma_{n,2}(C_M, x, D_n^-)| \quad (A28) $$

$$ + \sqrt{n\phi(h_k)}|\Gamma_{n,2}(C_M, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-))| \quad (A29) $$

$$ + \sqrt{n\phi(h_k)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) - \mathbb{E}(\Gamma_{n,2}(C, x, D_n^-))|. \quad (A30) $$

For some $\zeta_n \in (0, 1)$, recall that $D_n^-$ and $D_n^+$ are defined in Equations (A2) and (A3), respectively. Thus, observing:

$$ \begin{aligned} &|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) - \mathbb{E}(\Gamma_{n,2}(C, x, D_n^-))| \\ &= \frac{1}{n\mathbb{E}(\Delta_1(x, D_n^-))}\left|\mathbb{E}\left(\sum_{i=1}^n 1_{\{Y_i \in C\}} 1_{\{|F(Y_i)| > \delta_n\}} \Delta_i(x, D_n^-)\right)\right| \\ &\le \mathbb{E}\left(|1_{\{Y \in C\}} 1_{\{|F(Y)| > \delta_n\}} \Delta_1(x, D_n^-)\right) (\mathbb{E}(\Delta_1(x, D_n^-)))^{-1}. \end{aligned} $$
Under conditions (H8), using Hölder's inequality with $\beta_1 = \frac{p}{2}$ and $\beta_2$ such that $\frac{1}{\beta_1} + \frac{1}{\beta_2} = 1$, we can write:

$$
\begin{aligned}
& \sqrt{n\phi(h_k)} |\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) - \mathbb{E}(\Gamma_{n,2}(C, x, D_n^-))| \\
& \le \frac{\sqrt{n\phi(h_k)}}{\phi(D_n^-)} \mathbb{E}^{1/\beta_1} [|F(Y)|^p \mathbf{1}_{\{|F(Y)|>\delta_n\}}] \mathbb{E}^{1/\beta_2} [\Delta_1(x, D_n^-)^{\beta_2}] \\
& \le \frac{\sqrt{n\phi(h_k)}}{\phi(D_n^-)} \delta_n^{-1} \mathbb{E}^{1/\beta_1} [|F(Y)|^p |X|] \phi^{1/\beta_2}(D_n^-) \\
& \le \frac{C \sqrt{n\phi(h_k)}}{\delta_n \phi^{2/p}(D_n^-)} \\
& \le \frac{C}{\zeta^{1/p} \sqrt{n} \delta_n} \left(\frac{k}{n}\right)^{1/2-2/p} \to 0 \quad \text{as } n \to \infty.
\end{aligned}
$$

To establish convergence in probability for Equation (A28), it is necessary to use assumptions (H8), and Markov's inequality. Thus, for all $\varepsilon > 0$, we obtain:

$$
\begin{aligned}
& \mathbb{P}\left(\sqrt{n\phi(h_k)}|\Gamma_{n,2}(C_M, x, D_n^-) - \Gamma_{n,2}(C, x, D_n^-)| > \sqrt{n\phi(h_k)}\varepsilon\right) \\
& \le \frac{C\phi^{(1-\beta_2)/\beta_2}(D_n^-)}{\varepsilon\delta_n\sqrt{n\phi(h_k)}} \\
& \le \frac{C}{\varepsilon\delta_n}\frac{\phi^{-2/p}(D_n^-)}{\sqrt{n\phi(h_k)}} \to 0 \quad \text{as } n \to \infty.
\end{aligned}
$$

For the second term (A27) and (Lemma 8.2 in [38]), we have

$$ |\Gamma_{n,2}(C_M, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-))| \to 0 \quad \text{as } n \to \infty. \quad (A31) $$

Using the fact that $k(\phi^{-1}(\frac{k}{n}))^{2\gamma} \to 0$ and condition (H2)(i), we have for $u = u_n = \phi^{-1}(\frac{k}{n})$:

$$
\begin{aligned}
& \sqrt{n\phi(h_k)} |\mathbb{E}(\Gamma_{n,2}(C, x, D_n^-)) - \mathbb{E}(\Gamma_{n,2}(C, x, D_n^+))| \\
& \le \sqrt{n\phi(h_k)} |\mathbb{E}(\Gamma_{n,2}(C, x, D_n^-)) - \Gamma(C, x)| \\
& \quad + \sqrt{n\phi(h_k)} |\mathbb{E}(\Gamma_{n,2}(C, x, D_n^+)) - \Gamma(C, x)| \\
& \le \sqrt{n\phi(h_k)} \left|\mathbb{E}\left(\frac{1}{\mathbb{E}\Delta_1(x, D_n^-)} \Delta_1(x, D_n^-) \mathbf{1}_{\{Y_i \in C\}}\right) - \Gamma(C, x)\right| \\
& \quad + \sqrt{n\phi(h_k)} \left|\mathbb{E}\left(\frac{1}{\mathbb{E}\Delta_1(x, D_n^+)} \Delta_1(x, D_n^+) \mathbf{1}_{\{Y_i \in C\}}\right) - \Gamma(C, x)\right| \quad (A32) \\
& \le \sqrt{n\phi(h_k)} \sum_{u=D_n^-, D_n^+} \left| \frac{1}{\mathbb{E}\Delta_1(x, u)} \mathbb{E}'\left(\Delta_1(x, u) \mathbf{1}_{\{B(x,u)\}} (\Gamma(C, X_1) - \Gamma(C, x))\right) \right| \\
& \le (C \mathbf{1}_{\{B(x,D_n^-)\}} + C' \mathbf{1}_{\{B(x,D_n^+)\}}) d_{\mathcal{E}}(x, X_1)^{\gamma} \\
& \le \frac{2C}{\zeta^{\gamma}} \sqrt{k \left(\phi^{-1}\left(\frac{k}{n}\right)\right)^{2\gamma}} \to 0 \quad \text{as } n \to \infty.
\end{aligned}
$$

On the other hand, recall that $\mathbf{1}_{\{Y \in C_M\}} := \mathbf{1}_{\{Y \in C\}} \mathbf{1}_{\{F(Y) < M\}}$, and under the given conditions, using the fact that the regression function satisfies the Lipschitz condition, we obtain:

$$
\begin{aligned}
& \sqrt{n\phi(h_k)} |\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+))| \\
& \le \sqrt{n\phi(h_k)} |\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) - \Gamma(C_M, x)|
\end{aligned}
$$
$$
\begin{align}
&+ \sqrt{n\phi(h_k)} |\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+)) - \Gamma(C_M, x)| \nonumber \\
\le {} & \sqrt{n\phi(h_k)} \left|\mathbb{E}\left(\frac{1}{\mathbb{E}\Delta_1(x, D_n^-)} \Delta_1(x, D_n^-) \mathbf{1}_{\{Y \in C_M\}}\right) - \Gamma(C_M, x)\right| \nonumber \\
&+ \sqrt{n\phi(h_k)} \left|\mathbb{E}\left(\frac{1}{\mathbb{E}\Delta_1(x, D_n^+)} \Delta_1(x, D_n^+) \mathbf{1}_{\{Y \in C_M\}}\right) - \Gamma(C_M, x)\right| \tag{A33} \\
\le {} & \sqrt{n\phi(h_k)} \sum_{u=D_n^-, D_n^+} \left|\frac{1}{\mathbb{E}\Delta_1(x, u)} \mathbb{E}'\left(\Delta_1(x, u) \mathbf{1}_{\{B(x, u)\}} (\Gamma(C_M, X_1) - \Gamma(C_M, x))\right)\right| \nonumber \\
\le {} & (C \mathbf{1}_{\{B(x, D_n^-)\}} + C' \mathbf{1}_{\{B(x, D_n^-)\}}) d_{\mathcal{E}}(x, X_1)^\gamma \nonumber \\
\le {} & \frac{2C}{\zeta^\gamma} \sqrt{k \left(\phi^{-1}\left(\frac{k}{n}\right)\right)^{2\gamma}} \to 0 \quad \text{as } n \to \infty. \nonumber
\end{align}
$$

For the second term on the right-hand side of Equation (A21), we have:

$$
|S_2(x, H_{n,k}(x))| = |\mathbb{E}(\Gamma_{n,2}(C, x, H_{n,k}(x)) - \Gamma(C, x))| \quad (A34)
$$

$$
\leq |\mathbb{E}(\Gamma_{n,2}(C, x, D_n^-) - \Gamma(C, x))|. \quad (A35)
$$

Using Lemma A1, we readily deduce:

$$
\sqrt{n\phi(h_k)} S_2(x, H_{n,k}(x)) = o\left(\sqrt{\zeta^{\gamma} k \left(\phi^{-1}\left(\frac{k}{n}\right)\right)^{2\gamma}}\right). \quad (A36)
$$

The proof of $S_2(x, H_{n,k}(x))$ is now complete. Therefore, combining the results from Equations (A21), (A31), (A32), (A33) and (A36), we conclude the convergence of $I_1(x, H_{n,k}(x))$ to 0.

**Lemma A4.** Under the assumptions (H1)(ii), (H2)(i), (H2)(ii), (C2) and (H5) and if

$$
k \left( \phi^{-1} \left( \frac{k}{2} \right) \right)^{2\gamma} \to 0 \quad \text{as} \quad n \to \infty
$$

we have

$$
\sqrt{n\phi(h_k)} I_2(C, x, H_{n,k}(x)) \to \mathcal{N}(0, \sigma^2(x)) \quad \text{as} \quad n \to \infty
$$

Appendix A.4. Proof of Lemma A4

Recall that

$$
\begin{align*}
I_2(x, u) \\
&= \Gamma_{n,2}(C_M, x, u) - \mathbb{E}(\Gamma_{n,2}(C_M, x, u)) - \Gamma(C, x)(\Gamma_{n,1}(1, x, u) - 1) \\
&= [(\Gamma_{n,2}(C_M, x, u)) - \Gamma(C, t)\Gamma_{n,1}(1, x, u)] - \mathbb{E}[\Gamma_{n,2}(C_M, x, u) - \Gamma(C, t)\Gamma_{n,1}(1, x, u)].
\end{align*}
$$

Thus, the decomposition below is an essential step in proving this lemma:

$$
\begin{align*}
& \sqrt{n\phi(h_K)} I_2(C, x, H_{n,k}(x)) \\
&= \sqrt{n\phi(h_K)} (\Gamma_{n,2}(C_M, x, H_{n,k}(x)) - \Gamma(C,t)\Gamma_{n,1}(1,x, H_{n,k}(x)) \\
&\quad - \mathbb{E}(\Gamma_{n,2}(C_M, x, H_{n,k}(x)) - \Gamma(C,t)\Gamma_{n,1}(1,x, H_{n,k}(x)))) \\
&= \sqrt{n\phi(h_K)} [\Gamma_{n,2}(C_M, x, H_{n,k}(x)) - \Gamma_{n,2}(C_M, x, D_n^+)] \\
&\quad + \sqrt{n\phi(h_K)} (\Gamma_{n,2}(C_M, x, D_n^+) - \Gamma(C,x)\Gamma_{n,1}(1,x, D_n^+))
\end{align*}
$$
$$
\begin{aligned}
& -\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+) - \Gamma(C, x)\Gamma_{n,1}(1, x, D_n^+)) \\
& + \sqrt{n\phi(h_K)}[\Gamma(C, x)(\Gamma_{n,1}(1, x, D_n^+) - \mathbb{E}(\Gamma_{n,1}(1, x, D_n^+)))] \\
& + \sqrt{n\phi(h_K)}[\Gamma(C, x)(\Gamma_{n,1}(1, x, D_n^+) - \mathbb{E}(\Gamma_{n,1}(1, x, D_n^+)))] \\
& + \sqrt{n\phi(h_K)}[\Gamma(C, x)(\mathbb{E}(\Gamma_{n,1}(1, x, H_{n,k}(x))) - \Gamma_{n,1}(1, x, H_{n,k}(x)))] \\
& + \sqrt{n\phi(h_K)}[\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+) - \mathbb{E}(\Gamma_{n,2}(C_M, x, H_{n,k}(x))))] \\
& = J_1(t) + J_2(t) + J_3(t) + J_4(t) + J_5(t).
\end{aligned}
\tag{A37}
$$

We apply the same arguments used in the proof of $I_1(C, x, H_{n,k}(x))$, which gives us:

$$
\begin{aligned}
|J_1(t)| &= \sqrt{n\phi(h_K)}|\Gamma_{n,2}(C_M, x, H_{n,k}(x)) - \Gamma_{n,2}(C_M, x, D_n^+)| \\
&\le \sqrt{n\phi(h_K)}|\Gamma_{n,2}(C_M, x, D_n^-) - \Gamma_{n,2}(C_M, x, D_n^+)| \\
&\le \sqrt{n\phi(h_K)}|\Gamma_{n,2}(C_M, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-))| \\
&\quad + \sqrt{n\phi(h_K)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+)) - \Gamma_{n,2}(C_M, x, D_n^+)| \\
&\quad + \sqrt{n\phi(h_K)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+))|,
\end{aligned}
\tag{A38}
$$

and

$$
\begin{aligned}
|J_5(t)| &= \sqrt{n\phi(h_K)}\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+) - \mathbb{E}(\Gamma_{n,2}(C_M, x, H_{n,k}(x)))) \\
&\le \sqrt{n\phi(h_K)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+)) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+))| \\
&\quad + \sqrt{n\phi(h_K)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+)) - \mathbb{E}(\Gamma_{n,2}(C_M, x, H_{n,k}(x)))| \quad (A39)
\end{aligned}
$$

$$
\le 2\sqrt{n\phi(h_K)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+)) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+))|. \quad (A40)
$$

By combining Equations (A38) and (A39) and using similar arguments as in the analysis of $I_{11}(C_M, x, H_{n,k}(x))$ and $I_{12}(C_M, x, H_{n,k}(x))$, we obtain:

$$
\begin{aligned}
|J_1(t) + J_5(t)| &\le \sqrt{n\phi(h_K)}|\Gamma_{n,2}(C_M, x, D_n^-) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-))| \\
&\quad +3\sqrt{n\phi(h_K)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+)) - \Gamma_{n,2}(C_M, x, D_n^+)| \\
&\quad +\sqrt{n\phi(h_K)}|\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^-)) - \mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+))|.
\end{aligned}
$$

Thus, as $n \to \infty$, we find that $|J_1(t) + J_5(t)| \to 0$. Now, consider the term $J_2(t)$ on the right-hand side of (A37). We introduce the following sum:

$$
J_2(t) = \sum_{i=1}^{n} \mathbb{Z}_{ni},
$$

where

$$
\mathbb{Z}_{ni} = \frac{\sqrt{n\phi(h_K)}}{\sqrt{n\mathbb{E}(\Delta_1(x, D_n^+))}} \left[ \left( \mathbb{1}_{\{Y_i \in C_M\}} - \Gamma(C_M, x) \right) \Delta_i(x, D_n^+) \right. \\
\left. - \mathbb{E} \left( \left( \mathbb{1}_{\{Y_i \in C_M\}} - \Gamma(C_M, x) \right) \Delta_i(x, D_n^+) \right) \right],
$$

and

$$
J_2(t) = \sqrt{n\phi(h_K)}[\Gamma_{n,2}(C_M, x, D_n^+) - \Gamma(C, x)\Gamma_{n,2}(1, x, D_n^+)]
$$
$$-\mathbb{E}(\Gamma_{n,2}(C_M, x, D_n^+) - \Gamma(C, x)\Gamma_{n,2}(1, x, D_n^+))].$$

Thus, the result we claim is:

$$J_2(t) \to \mathcal{N}(0, \sigma^2(x)).$$

The asymptotic normality of $J_2(t)$ was showed in (Lemma 8.8 in [38]) by choosing the bandwidth parameters as $u = D_n^+$. For $J_3(t)$ and $J_4(t)$, we obtain by (Lemma 8.2 in [38]) and the fact that $\mathbb{E}(\Gamma_{n,1}(1, x, u)) = 1$ with $u = D_n^-$ or $D_n^+$:

$$\begin{aligned} & |J_3(t) + J_4(t)| \\ &= \sqrt{n\phi(h_K)}|\Gamma(C, x)[\Gamma_{n,1}(1, x, D_n^+) - \mathbb{E}(\Gamma_{n,1}(1, x, D_n^+))]| \\ & \quad + \sqrt{n\phi(h_K)}|\Gamma(C, x)[\mathbb{E}(\Gamma_{n,1}(1, x, H_{n,k}(x))) - \Gamma_{n,1}(1, x, H_{n,k}(x))]| \\ & \le \sqrt{n\phi(h_K)}|\Gamma(C, x)(\Gamma_{n,1}(1, x, D_n^+) - 1)| \\ & \quad + \sqrt{n\phi(h_K)}|1 - \Gamma_{n,1}(1, x, D_n^+)| \\ & \le 2\sqrt{n\phi(h_K)}|\Gamma(C, x)(\Gamma_{n,1}(1, x, D_n^+) - 1)|. \end{aligned}$$

We find that

$$\sqrt{n\phi(h_K)}|\Gamma_{n,1}(1, x, D_n^+) - 1| \to 0 \quad \text{as} \quad n \to \infty.$$

Finally, we conclude that

$$|J_3(t) + J_4(t)| \to 0.$$

Hence, the proof is complete.

**Lemma A5.** Under the assumptions (H1)(ii), (H2)(i), (H2)(ii), (C2), and (H5), and provided that

$$k\left(\phi^{-1}\left(\frac{k}{2}\right)\right)^{2\gamma} \to 0 \quad \text{as} \quad n \to \infty,$$

it follows that for each $x \in \mathcal{C}$:

$$\sqrt{n\phi(h_k)\Gamma_{n,1}(1, x; H_{n,k}(x)) - 1} \xrightarrow{\mathbb{P}} 0 \quad \text{as} \quad n \to \infty.$$

## Appendix A.5. Proof of Lemma A5

To prove this lemma, it is sufficient to apply the results from [32] in inequality (A22), along with Chebyshev's inequality. For any $\varepsilon > 0$, we derive the following:

$$\begin{aligned} & \mathbb{P}(|\Gamma_{n,1}(1, x, H_{n,k}(x)) - 1| > \delta) \\ & \le \mathbb{P}(|\Gamma_{n,1}(1, x, D_n^-) - \mathbb{E}(\Gamma_{n,1}(1, x, D_n^-))| > \varepsilon) \\ & \le \frac{\mathrm{Var}(\Gamma_{n,1}(1, x, D_n^-))}{\varepsilon^2}. \end{aligned}$$

Using the fact that

$$\mathrm{Var}(\Gamma_{n,1}(1, x, D_n^-)) = o\left(\frac{1}{n\phi(x, D_n^-)}\right),$$

we ultimately obtain the result

$$\Gamma_{n,1}(1, x; H_{n,k}(x)) - 1 \xrightarrow{p} 0 \quad \text{as} \quad n \to \infty.$$

Thus, the proof is complete.
**Lemma A6.** Given the conditions (H1)(ii), (H3), and (H5)(i), along with the assumption that $n\phi(h_K) \to \infty$, for each $x \in \mathcal{C}$, we have

$$ \Gamma_{n,1}(C, x; H_K(x)) \xrightarrow{p.s.} \mathbb{E}(\Gamma_{n,1}(C, x; H_K(x))), \quad \text{as } n \to \infty. $$

## Appendix A.6. Proof of Lemma A6

To prove Lemma A6, we first express the following:

$$ \begin{aligned} & \Gamma_{n,2}(C, x, h_K(x)) - \mathbb{E}(\Gamma_{n,2}(C, x, h_K(t))) \\ &= \frac{1}{n\mathbb{E}[\Delta_1(x, h_K(x))]} \sum_{i=1}^{n} \left\{ \mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x)) - \mathbb{E}(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x))) \right\} \\ &= \frac{1}{n\mathbb{E}[\Delta_1(x, h_K(x))]} \sum_{i=1}^{n} Z_i(x, h_K(x)), \end{aligned} $$

where

$$ Z_i(x, h_K(x)) = \mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x)) - \mathbb{E}(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x))). $$

Taking into account assumption **H4**, and using Hölder's inequality with $2 < q < p$ such that $\frac{1}{p} + \frac{1}{q} = 1$, we obtain for $i \neq j$:

$$ \begin{aligned} & \mathbb{E}[\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x; h_K(x)) \mathbb{1}_{\{Y_j \in C\}} \Delta_j(x; h_K(x))] \\ &= \mathbb{E}[\Delta_i(x; h_K(x)) \Delta_j(x; h_K(x)) \mathbb{E}[\mathbb{1}_{\{Y_i \in C\}} \mathbb{1}_{\{Y_j \in C\}}]] \\ &\le \mathbb{E}[\Delta_i(x; h_K(x)) \Delta_j(x; h_K(x)) \mathbb{E}^{1/p}[F^p(Y)|X] \mathbb{E}^{1/q}[F^q(Y)|X]] \\ &\le C \mathbb{E}[\Delta_i(x; h_K(x)) \Delta_j(x; h_K(x))] \\ &\le C \sup_{i \neq j} \mathbb{P}\{(X_i, X_j) \in B(x, h_K(x)) \times B(x, h_K(x))\} \\ &\le C \Psi(h_K(x)) f_2(x), \end{aligned} \quad (A41) $$

where the last inequality follows from condition **H1**(iii). Next, we examine the variance of the sum:

$$ \begin{aligned} & \mathrm{Var}\left(\frac{1}{n\mathbb{E}[\Delta_1(x, h_K(x))]} \sum_{i=1}^{n} Z_i(x, h_K(x))\right) \\ &= \frac{1}{n^2 \mathbb{E}^2[\Delta_1(x, h_K(x))]} \sum_{i=1}^{n} \sum_{j=1}^{n} \mathrm{Cov}\left(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x)); \mathbb{1}_{\{Y_j \in C\}} \Delta_j(x, h_K(x))\right) \\ &= \frac{1}{n^2 \mathbb{E}^2[\Delta_1(x, h_K(x))]} \mathrm{Var}\left(\mathbb{1}_{\{Y_1 \in C\}} \Delta_1(x, h_K(x))\right) \\ &\quad + \frac{1}{n^2 \mathbb{E}^2[\Delta_1(x, h_K(x))]} \sum_{i \neq j} \sum_{j=1}^{n} \mathrm{Cov}\left(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x)); \mathbb{1}_{\{Y_j \in C\}} \Delta_j(x, h_K(x))\right) \\ &=: V_1 + V_2. \end{aligned} $$

We begin with the term $V_1$, considering assumptions **H1**(ii) and **H2**(i), we have the following bounds:

$$ c_5 \eta_1 \phi(h_K(x)) f_1(x) \le \mathbb{E}(\Delta_1^j(x, h_K(x))) \le c_6 \eta_2 \phi(h_K(x)) f_1(x), \quad (A42) $$

and for the variance:

$$ \mathrm{Var}(\mathbb{1}_{\{Y_1 \in C\}} \Delta_1(x; h_K(x))) = \mathbb{E}[\mathbb{1}_{\{Y_1 \in C\}}^2 \Delta_1^2(x, h_K(x))] - \mathbb{E}^2[\mathbb{1}_{\{Y_1 \in C\}} \Delta_1(x, h_K(x))] $$
$$
\begin{align*}
&= \mathbb{E}[\mathbb{1}_{\{Y_1 \in C\}} \Delta_1^2(x, h_K(x))] \\
&= \mathbb{E}[\mathbb{E}[F^2(Y)|X] \Delta_1^2(x, h_K(x))] \\
&= \mathbb{E}^{2/p}[F^p(Y)|X] \mathbb{E}^{2/q}[\Delta_1^q(x, h_K(x))] \\
&= C^{2/p} C_2^{2/q} \eta_2^2 \phi^2(h_K(x)) f_1^2(x).
\end{align*}
$$

Hence, combining this with (A42) gives:

$$
\frac{\text{const}_1}{nf_1(x)\phi(h_K(x))} \le V_1 \le \frac{\text{const}_2}{nf_1(x)\phi(h_K(x))}, \quad (A43)
$$

for $f_1(x) > 0$ and constants $\text{const}_1 < \text{const}_2$.

Next, consider $V_2$:

$$
\begin{align}
V_2 &= \frac{1}{n^2 \mathbb{E}^2[\Delta_1(x, h_K(x))]} \left\{ \sum_{0<i-j\le\omega_n} \sum_{j=1}^n \operatorname{Cov}(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x)); \mathbb{1}_{\{Y_j \in C\}} \Delta_j(x, h_K(x))) \right. \nonumber \\
&\qquad \left. + \sum_{|i-j|\ge\omega_n} \sum_{j=1}^n \operatorname{Cov}(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x, h_K(x)); \mathbb{1}_{\{Y_j \in C\}} \Delta_j(x, h_K(x))) \right\} \nonumber \\
&:= V_{2,1} + V_{2,2}, \tag{A44}
\end{align}
$$

where $\omega_n$ satisfies $\omega_n = o(n)$. Using conditions **H1**(iii) and **H3**(i), we deduce that:

$$
\begin{align*}
& \operatorname{Cov}(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x; h_K(x)), \mathbb{1}_{\{Y_j \in C\}} \Delta_j(x; h_K(x))) \\
& \le C \Psi(h_K(x)) f_2(x) - C^{2/p} C_2^{2/q} \eta_2^2 \phi^2(h_K(x)) f_1^2(x).
\end{align*}
$$

This yields the bound:

$$
\begin{align*}
V_{2,1} &\le \frac{\Psi(h_K(x))f_2(x) - C^{2/p}C_2^{2/q}\eta_2^2\phi^2(h_K(x))f_1^2(x)}{n^2\mathbb{E}^2[\Delta_1(x, h_K(x))]} n\omega_n \\
&= \frac{\eta_2^2 f_2(x)\Psi(h_K)\omega_n}{n\mathbb{E}^2[\Delta_1(x, h_K(x))]} - C^{2/p}C_2^{2/q}\eta_2^2\phi^2(h_K(x))f_1^2(x)\frac{\omega_n}{n}.
\end{align*}
$$

Using Equation (A42), we obtain:

$$
V_{2,1} \le \frac{\text{const}' f_2(x) \Psi(h_K) \omega_n}{n f_1^2(x) \phi^2(h_K)} - \text{const}'' \frac{\omega_n}{n} \phi^2(h_K) f_1^2(x).
$$

This, in combination with (A43), implies:

$$
\begin{align*}
\frac{V_{2,1}}{V_1} &\le \text{const}' \frac{f_2(x)}{f_1(x)} \frac{\Psi(h_K)\omega_n}{\phi(h_K)} - \omega_n \phi(h_K) f_1(x) C^{2/p} C_2^{2/q} \eta_2^2 \phi^2(h_K(x)) f_1^2(x) \\
&\le \text{const}' \frac{f_2(x)}{f_1(x)} \frac{\Psi(h_K)\omega_n}{\phi(h_K)} - \text{const}'' \omega_n \phi^3(h_K) f_1^2(x).
\end{align*}
$$

By choosing $\omega_n$ such that the above bound tends to 0 as $n \to \infty$, we conclude that $V_{2,1}$ vanishes. Now consider $V_{2,2}$. By applying Davydov's lemma for strong mixing sequences and using the hypothesis **H5**(i), we have:

$$
\begin{align*}
& \operatorname{Cov}(\mathbb{1}_{\{Y_i \in C\}} \Delta_i(x; h_K(x)), \mathbb{1}_{\{Y_j \in C\}} \Delta_j(x; h_K(x))) \\
& \le 8 \left\{ \mathbb{E} \left| \mathbb{1}_{\{Y_i \in C\}} \Delta_i(x; h_K(x)) \right|^p \right\}^{2/p} [\alpha(|i-j|)]^{1-2/p}
\end{align*}
$$
$$ \leq 8 \{ \mathbb{E}[\mathbb{E}[|F(Y_i)|^p | X] | \Delta_i(x; h_K(x))|^p] \}^{2/p} [\alpha(|i-j|)]^{1-2/p} $$

$$ \leq 8C \{ \mathbb{E}[|\Delta_i(x; h_K(x))|^p] \}^{2/p} [\alpha(|i-j|)]^{1-2/p}. $$

Thus, we obtain:

$$ V_{2,2} \leq \frac{\text{const} f_1^{2/p}(x) \{\phi(h_K)\}^{2/p}}{n^2 \mathbb{E}^2[\Delta_1(x, h_K(x))]} \sum_{|i-j| \geq \omega_n} [\alpha(|i-j|)]^{1-2/p}. $$

By using (A42) and reducing the double sum, we get:

$$ V_{2,2} \leq \frac{\text{const}}{n \omega_n^\delta f_1^{2(1-1/p)}(x) \{\phi(h_K)\}^{2(1-1/p)}} \sum_{k=\omega_n+1}^{\infty} [k^\delta \alpha(k)]^{1-2/p}. $$

Combining the above with the bound on $V_1$, we get:

$$ \frac{V_{2,2}}{V_1} \leq \frac{\text{const}}{\omega_n^\delta (\log(\omega_n))^{\delta(1-1/p)} f_1^{1-2/p}(x) \{\phi(h_K)\}^{(1-2/p)}} \sum_{k=\omega_n+1}^{\infty} k^\delta [\log(k)]^{1-2/p} (\alpha(k))^{1-2/p}. \quad (A45) $$

By choosing $\omega_n = \{\phi(h_K)\}^{-(1-2/p)/\delta}$ and using assumption **H5(i)**, we get:

$$ \frac{V_{2,2}}{V_1} \leq \frac{\text{const}}{\left[-\frac{1}{\delta}\left(1 - \frac{2}{p}\log(\phi(h_K))\right)\right]^{\delta(1-1/p)} f_1^{1-2/p}(x)} \sum_{k=\omega_n+1}^{\infty} k^{\delta} [\log(k)]^{1-2/p} (\alpha(k))^{1-2/p} \quad (A46) \\ \to 0 \quad \text{as} \quad n \to \infty. $$

Thus, by the choice of $\omega_n$ and the boundedness of $\frac{\Psi(h_K)}{\phi^2(h_K)}$, we conclude that:

$$ \frac{V_{2,2}}{V_1} \leq \frac{\text{const} f_2(x) \Psi(h_K)}{f_1(x) \{\phi(h_K)\}^2} \omega_n \phi(h_K) + \{\phi(h_K)\}^{1-(1-2/p)/\delta}. $$

Since both terms tend to 0 as $n \to \infty$, the proof is complete.

## Appendix A.7. Proof of Theorem 3

Here, we will apply the same technique as in [20] and earlier work [86], utilizing the blocking approach. This method involves breaking down a strictly stationary sequence $(X_1, \dots, X_n)$ into $2v_n$ equal-sized blocks, each of length $n - 2v_n a_n$, while keeping in mind the notation. The goal is to establish the asymptotic equi-continuity of the conditional empirical process:

$$ \left\{ \Lambda_n(C, x) := \sqrt{n\phi(h_n)} (\Gamma_n(C, x; H_{n,k}(x)) - \Gamma(C, x)), C \in \mathcal{C}K, C \in \mathcal{C} \right\}. $$

Let us define, for any $CK \in \mathcal{C}K$ and $x \in \mathcal{E}$:

$$ W_n(x, C; H_{n,k}(x)) = \sum_{i=1}^{n} \mathbb{1}_{\{Y_i \in C\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right) - n\mathbb{E}\left\{\mathbb{1}_{\{Y_1 \in C\}} K\left(\frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)}\right)\right\}, \quad (A47) $$

and

$$ \begin{aligned} \Lambda(C | x) &= \sqrt{n\phi(h_n)} (\Gamma_n(C, x; H_{n,k}(x)) - \Gamma(C, x)) \\ &= \sqrt{n\phi(h_n)} (\Gamma_n(C, x; H_{n,k}(x)) - \Gamma(C, x)). \end{aligned} \quad (A48) $$
Thus, we have:

$$
\begin{align*}
\Lambda(C | x) &= \sqrt{n\phi(h_n)}(\Gamma_n(C, x; H_{n,k}(x)) - \Gamma(C, x)) \\
&= \sqrt{n\phi(h_n)} \left( \frac{\sum_{i=1}^n 1_{\{Y_i \in C\}} K\left( \frac{d_{\mathcal{G}}(x, X_i)}{H_{n,k}(x)} \right)}{\sum_{i=1}^n K\left( \frac{d_{\mathcal{G}}(x, X_i)}{H_{n,k}(x)} \right)} - \Gamma(C, x) \right) \\
&= \frac{1}{\check{\Gamma}_{n,1}(C, x; H_{n,k}(x))} \frac{1}{\sqrt{n\phi(h_n)}} \mathcal{W}_n(x, C; H_{n,k}(x)) \\
&\quad - \frac{\mathbb{E}(\check{\Gamma}_{n,2}(C, x; H_{n,k}(x)))}{\check{\Gamma}_{n,1}(C, x)\mathbb{E}(\check{\Gamma}_{n,1}(C, x; H_{n,k}(x)))} \frac{1}{\sqrt{n\phi(h_n)}} \mathcal{W}_n(x, 1; H_{n,k}(x)) \\
&\quad - \sqrt{n\phi(h_n)}(x; H_{n,k}(x)),
\end{align*}
$$

where for $h_K > 0$, we define:

$$
\begin{align*}
\check{\Gamma}_{n,2}(C, x; h_k) &:= \frac{1}{n\phi(h_K(x))} \sum_{i=1}^{n} 1_{\{Y_i \in C\}} \Delta_i(x, h_K), \\
\check{\Gamma}_{n,1}(1, x; h_k) &:= \frac{1}{n\phi(h_K(x))} \sum_{i=1}^{n} \Delta_i(x, h_K).
\end{align*}
$$

We now study the asymptotic equi-continuity of each of the terms above. For a class of functions $\mathcal{G}$, let $\alpha_n(\cdot)$ be an empirical process based on the sample $(X_1, Y_1), \dots, (X_n, Y_n)$, indexed by $\mathcal{G}$:

$$
\alpha_n(g) = \frac{1}{\sqrt{n}} \sum_{i=1}^{n} \{g(X_i, Y_i) - \mathbb{E}(g(X_i, Y_i))\}, \quad \|\alpha_n(g)\|_{\mathcal{G}} = \sup_{g \in \mathcal{G}} |\alpha_n(g)|,
$$

and for a measurable function $1_{\{\cdot \in C\}}$ and $x \in \mathcal{E}$, define:

$$
\eta_{n,x,C,K}(u,v,h_K) = 1_{\{v \in C\}} K\left(\frac{d_{\mathcal{E}}(u,x)}{h_K}\right), \quad \text{for } u,v \in \mathcal{E}.
$$

Thus, we have:

$$
\frac{1}{\sqrt{n\phi(h_K(x))}} \mathcal{W}_n(x, C; H_{n,k}(x)) = \frac{1}{\sqrt{\phi(h_K(x))}} \alpha_n(\eta_{n,x,C,K}(u,v,h_K)).
$$

Recall that $1_{\{D_n^- \le H_{n,k}(x) \le D_n^+\}} \xrightarrow{a.c.} 1$ as $\frac{k}{n} \to 0$. We aim to prove the asymptotic equi-continuity of:

$$
\left\{ \sqrt{\frac{n}{k}} \alpha_n (\eta_{n,x,C,K}) : \eta_{n,x,C,K} \in \mathcal{E} K \right\},
$$

which means, for every $\varepsilon > 0$, we need to show:

$$
\lim_{b \to 0} \lim_{n \to \infty} \sup \mathbb{P} \left\{ \sqrt{\frac{n}{k}} \| \alpha_n (\eta_{n,x,C,K}) \|_{\mathcal{E} K_{(b,\|\cdot\|)}} > \varepsilon \right\} = 0,
$$

where

$$
\mathcal{E} K_{(b, \|\cdot\|)} = \{ \eta_{n,x,C,K_1} - \eta_{n,x,C,K_2} : \| \eta_{n,x,C,K_1} - \eta_{n,x,C,K_2} \|_p < b, \eta_{n,x,C,K_1}, \eta_{n,x,C,K_2} \in \mathcal{E} K \}.
$$

The idea is to work with the independent block sequence $\{\zeta_j = (\zeta_{j,1}, \zeta_{j,2})\}_{j=1}^{\infty}$ rather than focusing on the dependent one, ((A1) proof of Lemma 8.2 in [38]) we have:
$$
\begin{equation} \tag{A49}
\begin{aligned}
& \mathbb{P} \left\{ \left\| k^{-1/2} \sum_{j=1}^{n} \left( 1_{\{Y_i \in C\}} K \left( \frac{d_{\mathcal{E}}(x, X_i)}{H_{n,k}(x)} \right) - \mathbb{P}(\eta_{n,x,C,K_1}(H_{n,k}(x))) \right) \right\|_{\mathcal{E}K_{(b,\|\cdot\|)}} > \delta \right\} \\
& \le 2 \mathbb{P} \left\{ \left\| k^{-1/2} \sum_{j=1}^{v_n} \sum_{i \in H_j} \left( 1_{\{Y_i \in C\}} K \left( \frac{d_{\mathcal{E}}(\zeta_i, x)}{H_{n,k}(x, \zeta)} \right) - \mathbb{P}(\eta_{n,x,C,K_1}(H_{n,k}(x, \zeta))) \right) \right\|_{\mathcal{E}K_{(b,\|\cdot\|)}} > \delta' \right\} \\
& \quad + 2(v_n - 1) \alpha_{a_n},
\end{aligned}
\end{equation}
$$

where $H_{n,k}(x, \zeta)$ is defined as:

$$
H_{n,k}(x, \zeta) = \min \left\{ h \in \mathbb{R}^{+} : \sum_{i=1}^{n} 1_{B(x,h)}(\zeta_i) = k \right\}. \quad (A50)
$$

We choose:

$$
a_n = \left[ (\log n)^{-1} \left( n^{p-2} \phi(h_K) \right)^{\frac{1}{2(p-2)}} \right] \quad \text{and} \quad v_n = \left\lfloor \frac{n}{2a_n} \right\rfloor.
$$

Note that $a_n$ in our setting is equivalent to:

$$
(\log n)^{-1} \left( n^{-2} k^p \right)^{\frac{1}{2(p-2)}}.
$$

Using assumption (H5)(i), we obtain $(v_n - 1)\alpha_{a_n} \to 0$ as $n \to 0$. Therefore, it remains to analyze the right-hand side term of (A49). Let us begin with the blocks being independent. We symmetrize using a sequence of i.i.d. Rademacher variables $\{\epsilon_j\}_{j \in \mathbb{N}^*}$, where $\mathbb{P}(\epsilon_j = 1) = \mathbb{P}(\epsilon_j = -1) = 1/2$. Importantly, for all $\delta > 0$, we have:

$$
\lim_{b \to 0} \lim_{n \to \infty} \mathbb{P} \left\{ \left\| k^{-1/2} \sum_{j=1}^{v_n} \epsilon_j \sum_{i \in H_j} 1_{\{Y_i \in C\}} K \left( \frac{d_{\mathcal{E}}(\zeta_i, x)}{H_{n,k}(x, \zeta)} \right) \right\|_{\mathcal{E}K_{(b, \|\cdot\|)}} > \delta \right\} = 0.
$$

Finally, using the fact that $1_{\{D_n^- \le H_{n,k}(x) \le D_n^+\}} \xrightarrow{a.c.} 1$ as $\frac{k}{n} \to 0$, and ((A9) proof of Lemma 8.10 in [38]) it suffices to prove:

$$
\lim_{b \to 0} \lim_{n \to \infty} \mathbb{P} \left\{ \left\| k^{-1/2} \sum_{j=1}^{v_n} \epsilon_j \sum_{i \in H_j} 1_{\{Y_i \in C\}} K \left( \frac{d_{\mathcal{E}}(\zeta_i, x)}{D_n^-} \right) \right\|_{\mathcal{E}K_{(b, \|\cdot\|)}} > \delta \right\} = 0.
$$

Since the *p*-th conditional moment, we can truncate and get, for each $\lambda > 0$ as $n \to \infty$:

$$
\begin{align}
& k^{-1/2} \sum_{j=1}^{n} \mathbb{E} \left( \eta_2 F(\zeta_i) 1_{\left\{ F(\zeta_i) \ge \lambda(M_n)^{1/2(p-1)} \right\}} \right) \nonumber \\
&= k^{-1/2} \int_{0}^{\infty} \mathbb{P} \left( \eta_2 F 1_{\left\{ F \ge \lambda(M_n)^{1/2(p-1)} \right\}} \ge t \right) dt \nonumber \\
&= k^{-1/2} \int_{0}^{\lambda(M_n)^{1/2(p-1)}} \mathbb{P} \left( F \ge \lambda(M_n)^{1/2(p-1)} \right) dt \nonumber \\
&\quad + k^{-1/2} \int_{0}^{\lambda(M_n)^{1/2(p-1)}} \mathbb{P} \left( F \ge x \right) dt \nonumber \\
&\xrightarrow{n \to \infty} 0. \tag{A51}
\end{align}
$$
Hence, there exists a sequence $\lambda_n \to 0$ as $n \to \infty$ such that:

$$k^{-1/2} \mathbb{E} \left( \eta_2 F 1_{\{F(\xi_i) \ge \lambda(M_n)^{1/2(p-1)}\}} \right) \to 0 \quad \text{as} \quad n \to \infty.$$

Next, we aim to demonstrate the following:

$$\lim_{b \to 0} \lim_{n \to \infty} \mathbb{P} \left( \left\| k^{-1/2} \sum_{j=1}^{v_n} \epsilon_j \sum_{i \in H_j} \left( 1_{\{Y_i \in C\}} K \left( \frac{d_{\mathcal{E}}(\xi_i, x)}{D_n^{-}} \right) \right) 1_{\{F(\xi_i) \ge \lambda(M_n)^{1/2(p-1)}\}} \right\|_{\mathcal{E} K_{(b, \|\cdot\|_p)}} > \delta \right) = 0.$$

We now define:

$$\Lambda_n^{(2)}(\eta_{n,x,C,K}) = k^{-1/2} \sum_{j=1}^{v_n} \epsilon_j \sum_{i \in H_j} \left( 1_{\{Y_i \in C\}} K \left( \frac{d_{\mathcal{E}}(\xi_i, x)}{D_n^{-}} \right) \right) 1_{\{F(\xi_i) \ge \lambda(M_n)^{1/2(p-1)}\}}.$$

The chaining approach from [86] uses the choice $b_q = 2^{-q}, q = 0, \dots, q_n$, where $q_n$ satisfies:

$$2^{-1} \lambda_n (\log(n))^{-1} \le b_q^2 \le \lambda_n (\log(n))^{-1}.$$

For the class $\mathcal{C}K_q$ of measurable sets, we have the covering number $N_q = \mathcal{N}(b_q, \mathcal{C}K_q, \|\cdot\|_p)$, and the following bound on the minimal distance:

$$\sup_{\eta_{n,x,C_1,K_1} \in \mathcal{C}K} \min_{\eta_{n,x,C_2,K_2} \in \mathcal{C}K} \|\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}\|_p \le b_q.$$

There exists a map $\pi_q: \mathcal{C}K \to \mathcal{C}K_q$ that maps each $\eta_{n,x,C,K} \in \mathcal{C}K$ to its closest set in $\mathcal{C}K_q$, such that:

$$\|\eta_{n,x,C,K} - \pi_q(\eta_{n,x,C,K})\|_p \le b_q.$$

By utilizing the chaining technique, we get the following inequality:

$$\begin{aligned} & \sup_{\substack{\eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C}K \\ \|\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}\|_p \le b}} \Lambda_n^{(2)}(\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}) \\ & \le \sup_{\substack{\eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C}K \\ \|\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}\|_p \le b_{q_n}}} \Lambda_n^{(2)}(\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}) \\ & + 2 \sum_{q=1}^{q_n} \sup_{\substack{\eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C}K_{q-1} \\ \|\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}\|_p \le 3b_q}} \Lambda_n^{(2)}(\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}) \\ & + \sup_{\substack{\eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C}K_0 \\ \|\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}\|_p \le 2b}} \Lambda_n^{(2)}(\eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2}). \end{aligned}$$

Let $\delta_q$ be defined as:

$$\delta_q = (b_q)^{1/2} \lor (3b_q(8 + c_{p,\alpha}^2)^{1/2}(\log \mathcal{N}_q)^{1/2}).$$

Let $\Gamma$ be chosen such that:

$$2 \sum_{q=1}^{+\infty} \delta_q \le \delta.$$
From the above, we deduce that:

$$
\mathbb{P} \left( \sup_{ \substack{ \eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C} K \\ \| \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} \|_p \le b } } \Lambda_n^{(2)} ( \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} ) \ge 3\delta \right) \\
\le \mathbb{P} \left( \sup_{ \substack{ \eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C} K \\ \| \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} \|_p \le b_{q_n} } } \Lambda_n^{(2)} ( \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} ) \ge \delta \right) \\
+ 2 \sum_{q=1}^{q_n} \mathbb{P} \left( \sup_{ \substack{ \eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C} K_{q-1} \\ \| \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} \|_p \le 3b_q } } \Lambda_n^{(2)} ( \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} ) \ge \delta_q \right) \\
+ \mathbb{P} \left( \sup_{ \substack{ \eta_{n,x,C_1,K_1}, \eta_{n,x,C_2,K_2} \in \mathcal{C} K_0 \\ \| \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} \|_p \le 2b } } \Lambda_n^{(2)} ( \eta_{n,x,C_1,K_1} - \eta_{n,x,C_2,K_2} ) \right).
$$

By the boundedness of the terms in $\Lambda_n^{(2)}(\eta_{n,x,C,K})$, we apply Bernstein's inequality to obtain:

$$
\mathbb{B} \le 2 \sum_{q=1}^{q_n} \exp \left( 2 \log \mathcal{N}_q - \frac{\delta_q^2 k}{nb_q^2 c_{p,\alpha} + (4/3)\delta_q a_n \lambda_n n^{p/2}(p-1)\phi(h_K)^{(p-2)/2(p-1)}} \right).
$$

Using the expression for $b_q$, we have:

$$
\begin{align*}
\delta_q a_n \lambda_n n^{p/2(p-1)} \phi(h_K)^{(p-2)/2(p-1)}
&= (4/3) \delta_q \lambda_n k (\log(n))^{-1} \\
&\le (8/3) n b_q^2 \delta_q \\
&\le 8 n b_q^2.
\end{align*}
$$

Thus:

$$
\begin{align*}
\mathbb{B} &\le 2 \sum_{q=1}^{q_n} \exp \left( 2 \log \mathcal{N}_q - \frac{\delta_q^2}{(8 + c_{p,\alpha}^2) b_q^2} \right) \\
&\le 2 \sum_{q=1}^{q_n} \exp \left( - \frac{\delta_q^2}{2(8 + c_{p,\alpha}^2) b_q^2} \right) \\
&\le 2 \sum_{q=1}^{q_n} \exp \left( - \frac{2^q}{2(8 + c_{p,\alpha}^2) b} \right) \to 0 \quad \text{as} \quad b \to 0.
\end{align*}
$$

Finally, we deduce that for each $\delta > 0$:

$$
\lim_{n \to \infty} \mathbb{P} \left( \left\| \Lambda_n^{(2)} (\eta_{n,x,C,K}) \right\|_{\mathcal{C} K_{(\lambda_n^{1/2} (\log(n)))^{-1/2}, \|\cdot\|_p}} \ge \delta \right) = 0.
$$

By applying the square root trick (Lemma 5.2 in [87]), similar methods as in [88], and
following the arguments in [86], we obtain:

$$
\mathbb{A}_2 \to 0.
$$

Thus, the theorem is proved.
## References

1. Pollard, D. *Empirical Processes: Theory and Applications*; NSF-CBMS Regional Conference Series in Probability and Statistics; Institute of Mathematical Statistics: Hayward, CA, USA; American Statistical Association: Alexandria, VA, USA, 1990; Volume 2, p. viii+86.

2. Shorack, G.R.; Wellner, J.A. *Empirical Processes with Applications to Statistics*; Classics in Applied Mathematics; Society for Industrial and Applied Mathematics (SIAM): Philadelphia, PA, USA, 2009; Volume 59, p. xli+956. [CrossRef]

3. Pyke, R. The weak convergence of the empirical process with random sample size. *Proc. Camb. Philos. Soc.* **1968**, *64*, 155-160.
[CrossRef]

4. Vapnik, V.N.; Červonenkis, A.J. The uniform convergence of frequencies of the appearance of events to their probabilities. *Teor. Verojatnost. Primenen.* **1971**, *16*, 264-279. [CrossRef]

5. Csörgő, M. Invariance principles for empirical processes. *Handb. Stat.* **1984**, *4*, 431-462. [CrossRef]

6. Dudley, R.M. Central limit theorems for empirical measures. *Ann. Probab.* **1978**, *6*, 899-929. [CrossRef]

7. Pollard, D. A central limit theorem for empirical processes. *J. Aust. Math. Soc.* **1982**, *33*, 235-248. [CrossRef]

8. Stute, W. Conditional empirical processes. *Ann. Statist.* **1986**, *14*, 638-647. [CrossRef]

9. Ossiander, M. A central limit theorem under metric entropy with $L_2$ bracketing. *Ann. Probab.* **1987**, *15*, 897-919. [CrossRef]

10. Withers, C.S. Convergence of empirical processes of mixing rv's on $[0, 1]$. *Ann. Statist.* **1975**, *3*, 1101-1108. [CrossRef]

11. Philipp, W. Invariance principles for sums of mixing random elements and the multivariate empirical process. *Colloq. Math. Soc. János Bolyai* **1984**, *36*, 843-873.

12. Yoshihara, K.i. Conditional empirical processes defined by $\phi$-mixing sequences. *Comput. Math. Appl.* **1990**, *19*, 149-158.
[CrossRef]

13. Polonik, W.; Yao, Q. Set-indexed conditional empirical and quantile processes based on dependent data. *J. Multivar. Anal.* **2002**, *80*, 234-255. [CrossRef]

14. Poryvaǐ, D.V. An invariance principle for conditional empirical processes formed by dependent random variables. *Izv. Math. 2005*, *69*, 129-148. [CrossRef]

15. Yu, B. Rates of convergence for empirical processes of stationary mixing sequences. *Ann. Probab.* **1994**, *22*, 94-116. [CrossRef]

16. Doukhan, P.; Massart, P.; Rio, E. Invariance principles for absolutely regular empirical processes. *Ann. l'IHP Probab. Stat.* **1995**, *31*, 393-427.

17. Eberlein, E. Weak convergence of partial sums of absolutely regular sequences. *Statist. Probab. Lett.* **1984**, *2*, 291-293. [CrossRef]

18. Bouzebda, S. Weak convergence of the conditional single index $U$-statistics for locally stationary functional time series. *AIMS Math.* **2024**, *9*, 14807-14898. [CrossRef]

19. Bouzebda, S.; Nemouchi, B. Central limit theorems for conditional empirical and conditional $U$-processes of stationary mixing sequences. *Math. Methods Statist.* **2019**, *28*, 169-207. [CrossRef]

20. Bouzebda, S.; Nemouchi, B. Weak-convergence of empirical conditional processes and conditional $U$-processes involving functional mixing data. *Stat. Inference Stoch. Process.* **2023**, *26*, 33-88. [CrossRef]

21. Ramsay, J.O.; Silverman, B.W. *Functional Data Analysis*, 2nd ed.; Springer Series in Statistics; Springer: New York, NY, USA, 2005; p. xx+426.

22. Bosq, D. *Linear Processes in Function Spaces: Theory and Applications*; Lecture Notes in Statistics; Springer: New York, NY, USA, 2000; Volume 149, p. xiv+283. [CrossRef]

23. Ferraty, F.; Vieu, P. *Nonparametric Functional Data Analysis: Theory and Practice*; Springer Series in Statistics; Springer: New York, NY, USA, 2006; p. xx+258.

24. Horváth, L.; Kokoszka, P. *Inference for Functional Data with Applications*; Springer Series in Statistics, Springer: New York, NY, USA, 2012; p. xiv+422. [CrossRef]

25. Agua, B.M.; Bouzebda, S. Single index regression for locally stationary functional time series. *AIMS Math.* **2024**, *9*, 36202-36258.
[CrossRef]

26. Bouzebda, S.; Taachouche, N. Oracle inequalities and upper bounds for kernel conditional $U$-statistics estimators on manifolds and more general metric spaces associated with operators. *Stochastics* **2024**, *96*, 2135-2198. [CrossRef]

27. Bouzebda, S.; Madani, F.; Souddi, Y. Some asymptotic properties of the conditional set-indexed empirical process based on dependent functional data. *Int. J. Math. Stat* **2022**, *22*, 77-105.

28. Bouzebda, S.; Nezzal, A. Uniform consistency and uniform in number of neighbors consistency for nonparametric regression estimates and conditional $U$-statistics involving functional data. *Jpn. J. Stat. Data Sci.* **2022**, *5*, 431-533. [CrossRef]

29. Souddi, Y.; Madani, F.; Bouzebda, S. Some characteristics of the conditional set-indexed empirical process involving functional ergodic data. *Bull. Inst. Math. Acad. Sin. (N.S.)* **2021**, *16*, 367-399. [CrossRef]

30. Bouzebda, S.; Souddi, Y.; Madani, F. Weak Convergence of the Conditional Set-Indexed Empirical Process for Missing at Random Functional Ergodic Data. *Mathematics* **2024**, *12*, 448. [CrossRef]

31. Horváth, L.; Yandell, B.S. Asymptotics of conditional empirical processes. *J. Multivariate Anal.* **1988**, *26*, 184-206. [CrossRef]
32. Burba, F.; Ferraty, F.; Vieu, P. *k*-nearest neighbour method in functional nonparametric regression. *J. Nonparametr. Stat.* **2009**, *21*, 453–469. [CrossRef]

33. Kudraszow, N.L.; Vieu, P. Uniform consistency of kNN regressors for functional variables. *Statist. Probab. Lett.* **2013**, *83*, 1863–1870. [CrossRef]

34. Kara, L.Z.; Laksaci, A.; Rachdi, M.; Vieu, P. Data-driven kNN estimation in nonparametric functional data analysis. *J. Multivariate Anal.* **2017**, *153*, 176–188. [CrossRef]

35. Almanjahie, I.M.; Bouzebda, S.; Chikr Elmezouar, Z.; Laksaci, A. The functional kNN estimator of the conditional expectile: Uniform consistency in number of neighbors. *Stat. Risk Model.* **2022**, *38*, 47–63. [CrossRef]

36. Bouzebda, S.; Laksaci, A.; Mohammedi, M. The *k*-nearest neighbors method in single index regression model for functional quasi-associated time series data. *Rev. Mat. Complut.* **2023**, *36*, 361–391. [CrossRef]

37. Almanjahie, I.M.; Chikr Elmezouar, Z.; Laksaci, A.; Rachdi, M. kNN local linear estimation of the conditional cumulative distribution function: Dependent functional data case. *C. R. Math. Acad. Sci. Paris* **2018**, *356*, 1036–1039. [CrossRef]

38. Bouzebda, S.; Nezzal, A. Uniform in number of neighbors consistency and weak convergence of kNN empirical conditional processes and kNN conditional U-processes involving functional mixing data. *AIMS Math.* **2024**, *9*, 4427–4550. [CrossRef]

39. Bouzebda, S. Uniform in Number of Neighbor Consistency and Weak Convergence of k-Nearest Neighbor Single Index Conditional Processes and k-Nearest Neighbor Single Index Conditional U-Processes Involving Functional Mixing Data. *Symmetry* **2024**, *16*, 1576. [CrossRef]

40. Nadaraja, E.A. On a regression estimate. *Teor. Verojatnost. Primenen.* **1964**, *9*, 157–159.

41. Watson, G.S. Smooth regression analysis. *Sankhyā Ser. A* **1964**, *26*, 359–372.

42. Gasser, T.; Hall, P.; Presnell, B. Nonparametric estimation of the mode of a distribution of random curves. *J. R. Stat. Soc. Ser. B Stat. Methodol.* **1998**, *60*, 681–691. [CrossRef]

43. Masry, E. Nonparametric regression estimation for dependent functional data: Asymptotic normality. *Stochastic Process. Appl.* **2005**, *115*, 155–177. [CrossRef]

44. Bradley, R.C. *Introduction to Strong Mixing Conditions*. Vol. 3; Kendrick Press: Heber City, UT, USA, 2007; p. xii+597.

45. Gorodecki, V.V. The strong mixing property for linearly generated sequences. *Teor. Verojatnost. Primenen.* **1977**, *22*, 421–423.

46. Withers, C.S. Conditions for linear processes to be strong-mixing. *Z. Wahrsch. Verw. Gebiete* **1981**, *57*, 477–480. [CrossRef]

47. Auestad, B.; Tjøstheim, D. Identification of nonlinear time series: First order characterization and order determination. *Biometrika* **1990**, *77*, 669–687. [CrossRef]

48. Bouzebda, S.; Didi, S. Some asymptotic properties of kernel regression estimators of the mode for stationary and ergodic continuous time processes. *Rev. Matemática Complut.* **2021**, *34*, 811–852. [CrossRef] [PubMed]

49. Bouzebda, S.; Didi, S. Additive regression model for stationary and ergodic continuous time processes. *Comm. Statist. Theory Methods* **2017**, *46*, 2454–2493. [CrossRef]

50. Bouzebda, S.; Didi, S. Multivariate wavelet density and regression estimators for stationary and ergodic discrete time processes: Asymptotic results. *Commun.-Stat.-Theory Methods* **2017**, *46*, 1367–1406. [CrossRef]

51. Chen, R.; Tsay, R.S. Functional-coefficient autoregressive models. *J. Am. Stat. Assoc.* **1993**, *88*, 298–308. [CrossRef]

52. Masry, E.; Tjøstheim, D. Nonparametric estimation and identification of nonlinear ARCH time series. *Econom. Theory* **1995**, *11*, 258–289. [CrossRef]

53. Masry, E.; Tjøstheim, D. Additive nonlinear ARX time series and projection estimates. *Econom. Theory* **1997**, *13*, 214–252. [CrossRef]

54. Dudley, R.M. A course on empirical processes. In *École d'été de probabilités de Saint-Flour, XII—1982*; Lecture Notes in Mathematics; Springer: Berlin/Heidelberg, Germany, 1984; Volume 1097, pp. 1–142. [CrossRef]

55. Mayer-Wolf, E.; Zeitouni, O. The probability of small Gaussian ellipsoids and associated conditional moments. *Ann. Probab.* **1993**, *21*, 14–24. [CrossRef]

56. Bogachev, V.I. *Gaussian Measures*; Mathematical Surveys and Monographs; American Mathematical Society: Providence, RI, USA, 1998; Volume 62, p. xii+433. [CrossRef]

57. Li, W.V.; Shao, Q.M. Gaussian processes: Inequalities, small ball probabilities and applications. *Handb. Stat.* **2001**, *19*, 533–597. [CrossRef]

58. Pollard, D. *Convergence of Stochastic Processes*; Springer Series in Statistics; Springer: New York, NY, USA, 1984; p. xiv+215. [CrossRef]

59. Dudley, R.M. *Uniform Central Limit Theorems*, 2nd ed.; Cambridge Studies in Advanced Mathematics; Cambridge University Press: New York, NY, USA, 2014; Volume 142, p. xii+472.

60. van der Vaart, A.W.; Wellner, J.A. *Weak Convergence and Empirical Processes: With Applications to Statistics*; Springer Series in Statistics; Springer: New York, NY, USA, 1996; p. xvi+508. [CrossRef]

61. Kosorok, M.R. *Introduction to Empirical Processes and Semiparametric Inference*; Springer Series in Statistics; Springer: New York, NY, USA, 2008; p. xiv+483. [CrossRef]
62. Deheuvels, P. One bootstrap suffices to generate sharp uniform bounds in functional estimation. *Kybernetika* **2011**, *47*, 855–865.

63. Hardy, G.H. On double Fourier series, and especially those which represent the double zeta-function with real and incommensurable parameters. *Quart. J. Math* **1906**, *37*, 53–79.

64. Krause, M. Über mittelwertsätze im gebiete der doppelsummen und doppelintegrale. *Leipz. Ber* **1903**, *55*, 239–263.

65. Volkonski, V.A.; Rozanov, Y.A. Some limit theorems for random functions. I. *Theor. Probab. Appl.* **1959**, *4*, 178–197. [CrossRef]

66. Kolmogorov, A.N.; Tihomirov, V.M. $\varepsilon$-entropy and $\varepsilon$-capacity of sets in functional space. *Am. Math. Soc. Transl.* **1961**, *17*, 277–364.

67. van der Vaart, A.; van Zanten, H. Bayesian inference with rescaled Gaussian process priors. *Electron. J. Stat.* **2007**, *1*, 433–448. [CrossRef]

68. Härdle, W.; Marron, J.S. Optimal bandwidth selection in nonparametric regression function estimation. *Ann. Statist.* **1985**, *13*, 1465–1481. [CrossRef]

69. Rachdi, M.; Vieu, P. Nonparametric regression for functional data: Automatic smoothing parameter selection. *J. Stat. Plan. Inference* **2007**, *137*, 2784–2801. [CrossRef]

70. Bouzebda, S.; El-hadjali, T. Uniform convergence rate of the kernel regression estimator adaptive to intrinsic dimension in presence of censored data. *J. Nonparametric Stat.* **2020**, *32*, 864–914. [CrossRef]

71. Shang, H.L. Bayesian bandwidth estimation for a functional nonparametric regression model with mixed types of regressors and unknown error density. *J. Nonparametric Stat.* **2014**, *26*, 599–615. [CrossRef]

72. Li, Q.; Maasoumi, E.; Racine, J.S. A nonparametric test for equality of distributions with mixed categorical and continuous data. *J. Econom.* **2009**, *148*, 186–200. [CrossRef]

73. Horowitz, J.L.; Spokoiny, V.G. An adaptive, rate-optimal test of a parametric mean-regression model against a nonparametric alternative. *Econometrica* **2001**, *69*, 599–631. [CrossRef]

74. Gao, J.; Gijbels, I. Bandwidth selection in nonparametric kernel testing. *J. Am. Statist. Assoc.* **2008**, *103*, 1584–1594. [CrossRef]

75. Dawid, A.P. Conditional independence for statistical operations. *Ann. Statist.* **1980**, *8*, 598–617. [CrossRef]

76. Dawid, A.P. Conditional independence in statistical theory. *J. R. Stat. Soc. Ser. B Stat. Methodol.* **1979**, *41*, 1–31. [CrossRef]

77. Koller, D.; Friedman, N. *Probabilistic Graphical Models*; Adaptive Computation and Machine Learning Series; MIT Press: Cambridge, MA, USA, 2009.

78. Pearl, J. *Causality*, 2nd ed.; Models, Reasoning, and Inference; Cambridge University Press: Cambridge, UK, 2009; p. xx+464. [CrossRef]

79. Zhang, K.; Peters, J.; Janzing, D.; Schölkopf, B. Kernel-based Conditional Independence Test and Application in Causal Discovery. *arXiv* **2012**, arXiv:1202.3775. [CrossRef]

80. Zhou, Y.; Liu, J.; Zhu, L. Test for conditional independence with application to conditional screening. *J. Multivar. Anal.* **2020**, *175*, 104557. [CrossRef]

81. Bouzebda, S. General tests of conditional independence based on empirical processes indexed by functions. *Jpn. J. Stat. Data Sci.* **2023**, *6*, 115–177. [CrossRef]

82. Yao, W. A bias corrected nonparametric regression estimator. *Statist. Probab. Lett.* **2012**, *82*, 274–282. [CrossRef]

83. Karlsson, A. Bootstrap methods for bias correction and confidence interval estimation for nonlinear quantile regression of longitudinal data. *J. Stat. Comput. Simul.* **2009**, *79*, 1205–1218. [CrossRef]

84. Ezzahrioui, M.; Ould-Saïd, E. Asymptotic normality of a nonparametric estimator of the conditional mode function for functional data. *J. Nonparametric Stat.* **2008**, *20*, 3–18. [CrossRef]

85. Bouzebda, S.; Nemouchi, B. Uniform consistency and uniform in bandwidth consistency for nonparametric regression estimates and conditional *U*-statistics involving functional data. *J. Nonparametric Stat.* **2020**, *32*, 452–509. [CrossRef]

86. Arcones, M.A.; Yu, B. Central limit theorems for empirical and *U*-processes of stationary mixing sequences. *J. Theor. Probab.* **1994**, *7*, 47–71. [CrossRef]

87. Giné, E.; Zinn, J. Some limit theorems for empirical processes. *Ann. Probab.* **1984**, *12*, 929–998. With discussion, [CrossRef]

88. Le Cam, L. A remark on empirical measures. In *A Festschrift for Erich Lehmann in Honor of His Sixty-Fifth Birthday*; Wadsworth: Belmont, CA, USA, 1983; pp. 305–327.

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.