# Nonparametric conditional U-statistics on Lie groups with measurement errors

Salim Bouzebda, Nourelhouda Taachouche

► To cite this version:

Salim Bouzebda, Nourelhouda Taachouche. Nonparametric conditional U-statistics on Lie groups with measurement errors. Journal of Complexity, 2025, pp.1–86. 10.1016/j.jco.2025.101944. hal-05018104

HAL Id: hal-05018104

https://hal.science/hal-05018104v1

Submitted on 7 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Nonparametric conditional *U*-statistics on Lie groups with measurement errors

<figure><img src="image_2.png" /></figure>

Salim Bouzebda *, Nourelhouda Taachouche

Université de Technologie de Compiègne, LMAC, Compiègne, France

## ARTICLE INFO

**Article history:**

Received 17 June 2024

Accepted 27 March 2025

Available online 31 March 2025

Communicated by M. Zani

**MSC:**

60F05

60G15

62G08

62G07

62G20

**Keywords:**

Conditional *U*-statistic

Deconvolution

Errors-in-variables

Manifold-valued data

Lie groups

Non-Euclidean data

## ABSTRACT

This study presents a comprehensive framework for conditional *U*-statistics of a general order in the context of Lie group-valued predictors affected by measurement errors. Such situations arise in a variety of modern statistical problems. Our approach is grounded in an abstract harmonic analysis on Lie groups, a setting relatively underexplored in statistical research. In a unified study, we introduce an innovative deconvolution method for conditional *U*-statistics and investigate its convergence rate and asymptotic distribution for the first time. Furthermore, we explore the application of conditional *U*-statistics to variables that combine, in a nontrivial way, Euclidean and non-Euclidean elements subject to measurement errors, an area largely uncharted in statistical research. We derive general asymptotic properties, including convergence rates across various modes and the asymptotic distribution. All results are established under fairly general conditions on the underlying models. Additionally, our results are used to derive the asymptotic confidence intervals derived from the asymptotic distribution of the estimator. We also discuss applications of the general approximation results and give new insights into the Kendall rank correlation coefficient and discrimination problems.

© 2025 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

<footnote>* Corresponding author.</footnote>

<footnote>E-mail addresses: salim.bouzebda@utc.fr (S. Bouzebda), nourelhouda.taachouche@outlook.fr (N. Taachouche).</footnote>
# 1. Introduction

U-statistics, introduced by Hoeffding [68] within the framework of unbiased estimators, build on earlier work by Halmos [61], and represent a fundamental class of statistical measures. Essentially, U-statistics of order k with kernel $\varphi(\cdot)$ are constructed from a sequence $\{\mathbf{X}_i\}_{i=1}^\infty$ of random variables taking values in a measurable space $(S, \mathcal{S})$, and a measurable function $\varphi: S^k \to \mathbb{R}$. They are defined as follows:

$$U_n(\varphi) = \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \varphi(\mathbf{X}_{i_1}, \dots, \mathbf{X}_{i_k}), \quad n \ge k,$$

where

$$I_n^k = \{(i_1, \dots, i_k) : i_j \in \mathbb{N}, 1 \le i_j \le n, i_j \ne i_\ell \text{ if } j \ne \ell\}.$$

$U_n(\varphi)$ serves as the nonparametric uniformly minimum variance estimator of $\theta = \mathbb{E}(\varphi(\mathbf{X}_1, \dots, \mathbf{X}_k))$, minimizing the expression

$$\sum_{1 \le i_1 < \dots < i_k \le n} (\varphi(\mathbf{X}_{i_1}, \dots, \mathbf{X}_{i_k}) - \alpha)^2,$$

where $\alpha$ is the minimizer. Notable examples of estimators based on U-statistics include empirical variance, Gini's mean difference, and Kendall's rank correlation coefficient. The Wilcoxon signed rank test for the hypothesis of a zero location parameter is a classical test based on U-statistics, as illustrated by van der Vaart [137] in Example 12.4. Asymptotic results for U-statistics based on independent and identically distributed random variables were first presented by Hoeffding [68], with references to related works by Rubin and Vitale [121], Serfling [125], Dynkin and Mandelbaum [42], and Bretagnolle [26]. Parallel results for V-statistics were obtained by von Mises [138] and Filipova [53]. An extensive body of literature on the theory of U-statistics has emerged, thoroughly reviewed by Serfling [125], Arcones and Giné [5], and Lee [96], among others. Comprehensive reviews and historical perspectives are available in the book by Borovskikh [11]. The significance of U-statistics extends to testing for qualitative features of functions in nonparametric statistics [59,1], cross-validation for density estimation [115], and establishing limiting distributions of M-estimators [5,127,38]. Arcones and Giné [5] provide necessary and sufficient conditions for the law of large numbers and sufficient conditions for the central limit theorem for U-processes. Recent applications include tests for normality by Arcones and Wang [6], utilizing U-processes, and new normality tests by Schick et al. [122], employing weighted L1-distances between the standard normal density and local U-statistics based on standardized observations. The median-of-means approach, based on U-statistics, is introduced by Joly and Lugosi [80] for estimating the mean of multivariate functions in potentially heavy-tailed distributions. For further references on U-statistics and U-processes, readers may consult Bouzebda and Soukarieh [17], Bouzebda and Soukarieh [16], Bouzebda et al. [25], Bouzebda and Taachouche [22], Soukarieh and Bouzebda [129], Bouzebda and Soukarieh [18], and Bouzebda and Ferfache [12], while de la Peña and Giné [38] provides a comprehensive insight into the theory of U-statistics.

U-statistics are prevalent across diverse fields, including the theory of random graphs, where they facilitate the counting of specific subgraphs, such as triangles, as demonstrated by Janson [73]. In machine learning, U-statistics naturally arise in various applications, including clustering, image recognition, ranking, and graph-based learning problems. Notably, natural risk estimates in these areas often take the form of U-statistics, as discussed by Clémençon et al. [35]. For instance, the empirical ranking error of a prediction rule is a U-statistic of order 2, as highlighted by Clémençon et al. [34]. Moreover, U-statistics remain pertinent even when dealing with random kernels of diverging orders. For comprehensive discussions, readers can refer to Frees [55], Heilig and Nolan [65], Song et al. [128], and Soukarieh and Bouzebda [130]. Infinite-order U-statistics have applications in constructing simultaneous prediction intervals, offering a means to quantify the uncertainty associated with ensemble
methods like subbagging and random forests, as elucidated by Peng et al. [117]. Specific applica-
tions include the MeanNN approach for estimating differential entropy, introduced by Faivishevsky
and Goldberger [46]. Additionally, Liu et al. [101] proposed a novel test statistic for goodness-of-fit
tests employing U-statistics. In genetic data analysis, Cybis et al. [36] explored a model-free approach
for clustering and classification, leveraging the versatility and adaptability of U-statistics to various
genetic problems and diverse data types. The work of Lim and Stojanovic [99] applied U-statistics
to analyze random compressed sensing matrices in the non-asymptotic regime. Jadhav and Ma [72]
assessed the test of independence in the functional framework using Kendall statistics, which can
be viewed as special cases of U-statistics. In the realm of clustering within multiple groups, Bello
et al. [8] introduced a comprehensive framework utilizing a U-statistics-based approach tailored for
high-dimensional datasets, classifying data into three groups while assessing the significance of these
partitions. Similarly, Kim and Ramdas [85] focused on dimension-agnostic inference, developing meth-
ods whose validity remains independent of assumptions about dimension versus sample size. This
approach employed variational representations of existing test statistics, incorporating sample splitting
and self-normalization to yield a refined test statistic with a Gaussian limiting distribution. The modi-
fication of degenerate U-statistics involved dropping diagonal blocks and retaining off-diagonal blocks.
Further exploration by Chen et al. [32] delved into U-statistics-based empirical risk minimization,
while Janson [74] examined asymmetric U-statistics based on a stationary sequence of m-dependent
variables, with applications in pattern matching in random strings and permutations. Additionally,
Sudheesh et al. [136] developed innovative U-statistics considering left truncation and right censor-
ing. As an application, they proposed a non-parametric test for assessing the independence between
time to failure and the cause of failure in competing risks, particularly when observations are subject
to left truncation and right censoring. In a different context, Le Minh [95] investigated the quadruplet
U-statistic, offering applications in statistical inference for network analysis. Extending this investiga-
tion to conditional U-statistics in the setting of measurement errors presents both practical benefits
and technical challenges.

We introduce [132] estimators in the context of independent and identically distributed random variables {$\mathbf{X}_i, \mathbf{Y}_i$}, $i \in \mathbb{N}^*$, where $\mathbf{X}_i \in \mathbb{R}^d$ and $\mathbf{Y}_i \in \mathcal{Y}$, a Polish space, with $\mathbb{N}^* = \mathbb{N} \setminus \{0\}$. Consider a measurable function $\varphi: \mathcal{Y}^k \to \mathbb{R}$. Our objective is to estimate the conditional expectation, or regression function, denoted as $r^{(k)}(\varphi, \tilde{\mathbf{t}})$ for $\tilde{\mathbf{t}} = (\mathbf{t}_1, \dots, \mathbf{t}_k) \in \mathbb{R}^{dk}$, given by:

$$
r^{(k)}(\varphi, \tilde{\mathbf{t}}) = \mathbb{E} \left( \varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) \mid (\mathbf{X}_1, \dots, \mathbf{X}_k) = \tilde{\mathbf{t}} \right),
$$

whenever it exists, i.e., $\mathbb{E}(|\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k)|) < \infty$. Let $K: \mathbb{R}^d \to \mathbb{R}$ be a kernel function with support contained in $[-B, B]^d$, $B > 0$, satisfying:

$$
\sup_{\mathbf{x} \in \mathbb{R}^d} |K(\mathbf{x})| := \kappa < \infty \quad \text{and} \quad \int K(\mathbf{x}) \, d\mathbf{x} = 1.
$$

Stute's class of estimators for $r^{(k)}(\varphi, \cdot)$, known as conditional U-statistics, is defined for each $\tilde{\mathbf{t}} \in \mathbb{R}^{dk}$ as follows:

$$
r_n^{(k)}(\varphi, \tilde{\mathbf{t}}; h_K) = \frac{\sum_{(i_1, \dots, i_k) \in I_n^k} \varphi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) K\left(\frac{\mathbf{t}_1 - \mathbf{X}_{i_1}}{h_K}\right) \dots K\left(\frac{\mathbf{t}_k - \mathbf{X}_{i_k}}{h_K}\right)}{\sum_{(i_1, \dots, i_k) \in I_n^k} K\left(\frac{\mathbf{t}_1 - \mathbf{X}_{i_1}}{h_K}\right) \dots K\left(\frac{\mathbf{t}_k - \mathbf{X}_{i_k}}{h_K}\right)},
$$

where {$h_K := h_n$}$_{n \ge 1}$ is a sequence of positive constants converging to zero at the rate $nh_K^{k} \to \infty$. In the special case $k=1$, $r^{(k)}(\varphi, \tilde{\mathbf{t}})$ reduces to $r^{(1)}(\varphi, \mathbf{t}) = \mathbb{E}(\varphi(\mathbf{Y}) | \mathbf{X}=\mathbf{t})$, and Stute's estimator becomes the Nadaraya-Watson estimator of $r^{(1)}(\varphi, \mathbf{t})$ [113,141], as detailed in [20,22,19]. Sen [124] focused on estimating the rate of uniform convergence in $\mathbf{t}$ of $r_n^{(k)}(\varphi, \mathbf{t}; h_K)$ to $r^{(k)}(\varphi, \tilde{\mathbf{t}})$. Subsequent work by Prakasa Rao and Sen [118] analyzed the limit distributions of $r_n^{(k)}(\varphi, \mathbf{t}; h_K)$, comparing the results with those obtained by Stute [132]. Under appropriate mixing conditions, Harel and Puri [63]
extended Stute [132]'s results to weakly dependent data, using them to validate the Bayes risk consistency of corresponding discrimination rules. As alternatives to conventional kernel-type estimators, Stute [135] introduced symmetrized closest neighbor conditional U-statistics. Fu [56] evaluated the functional conditional U-statistic and determined its finite-dimensional asymptotic normality. Recent developments are discussed in [13, 23, 14, 15, 24], where challenges related to uniform bandwidth consistency in general settings are examined.

The significance of regression analysis in statistics cannot be overstated, as it provides valuable insights into the relationships among variables of interest. However, complications arise when certain variables are not accurately observed due to measurement errors. Applying conventional methods designed for error-free variables in such cases leads to inaccurate inferences. The issue of measurement error in covariates has been thoroughly examined within traditional regression frameworks, where covariate details are commonly presented in vector form. The Cox proportional hazards model was examined by [69] in the context of covariates measured with errors. Addressing the challenge of fitting a parametric model to the nonparametric component in partially linear regression models, [91] tackled cases where covariates in both parametric and nonparametric segments are prone to Berkson measurement errors. Introducing a corrected loss function for partially functional linear quantile regression to mitigate the impact of errors on the estimation process, [146] presented an alternative approach. In the realm of testing for parametric regression models in the presence of measurement errors in covariates, [90] conducted a study. For analyzing longitudinal data with covariate measurement errors and outliers, [147] proposed a robust estimating equation method. In the domain of a general measurement error model with heteroscedastic error, [98] introduced a novel method applicable to parametric models with unspecified heteroscedastic measurement error variance structures. Examining semiparametric regression models with censored failure time data when covariates undergo measurement error, [93] delved into the analysis. In the scenario of censored quantile regression with covariate measurement errors, [144] suggested smoothed and corrected estimating equations for obtaining consistent estimators. [142] proposed a method for consistent linear quantile estimation in the presence of covariate measurement error. Addressing the estimation of regression coefficients in general nonlinear models with unspecified link functions and predictors measured with error, [27] provided insights. [58] tackled the estimation problem for linear regression and complemented the work of [120,58]. Highlighting the known phenomenon that errors significantly impede the rate of regression estimation, [44] presented a sharp constant result. Analyzing error-contaminated matrix-variate data, [51] focused on matrix-variate logistic measurement error models. In the realm of semiparametric censored linear regression with differential measurement error for time-to-event data and validation samples, [145] proposed an estimation equation correction method. Lastly, [106] explored the estimation of multivariate regression functions of stationary random processes with errors-in-variables, with additional insights from [49]. For an introduction to the errors-in-Euclidean-variables problems, we refer to [108], [39], [83], [28,62], [29,143], [107,105,104]. Examining non-Euclidean data has become a crucial endeavor in contemporary statistics, given the swift emergence of such data across diverse fields. To explore the current trend in the analysis of non-Euclidean data, one can refer to [103] and the associated references. Notably, substantial attention has been dedicated to functional data analysis in recent years ([100], [126], [111]). However, there is a noticeable scarcity of literature addressing situations where a functional predictor in functional regression is influenced by a genuinely functional measurement error ([41], [31], [71]). Manifold-valued data encounter measurement errors similar to those observed in Euclidean and functional data. Consider wind directions at specific times, challenging to precisely measure due to the swift speed of the wind. Similarly, periodic time variables like daily event times or annual event dates may harbor measurement errors unless consistently observed. These instances exemplify circular data. In astronomy, precise positioning of sunspots on the sun or determining the oriented directions of distant astronomical objects poses challenges due to their rapid movements, constituting examples of spherical data. The orbits of comets or asteroids, convertible to SO(3)-valued data ([123]), can also be impacted by measurement errors, where SO(p) represents the space of p × p orthogonal matrices with a unit determinant. Moreover, the achievable orientations of robot arms within SO(3) may exhibit measurement errors, as discussed in [102]. Another category involves hyperspherical data derived from Euclidean sources. This occurs when observed Euclidean vectors are normalized to maintain a unit Euclidean norm, ensuring that
data analysis relies solely on the relative orders of vector elements rather than their absolute orders. If the original Euclidean data contains measurement errors, the resulting hyperspherical data will likewise be affected.

Analyzing manifold-valued data presents significant challenges due to the absence of a vector space structure on most manifolds, unlike Banach or Hilbert spaces where functional data are typically situated. Numerous studies have addressed density estimation (e.g., [10], [57], [60]) and regression analysis (e.g., [7], [30], [33]) for error-free manifold-valued data. Specifically, [30] investigates scenarios where both the response and predictor are spherical variables, with the response symmetrically distributed around the product of an unknown orthogonal matrix and the predictor. However, literature is sparse regarding manifold-valued variables contaminated by manifold-valued measurement errors, with existing works primarily focusing on density estimation. This paper addresses a novel scenario involving conditional U-statistics where the response variable is real-valued, and the predictor resides within a compact and connected Lie group $G$. Lie groups represent significant classes of non-Euclidean spaces, combining manifold and algebraic group structures. Examples include tori (with the unit circle as a specific instance), special orthogonal groups, special unitary groups, unitary groups, compact symplectic groups, metaplectic groups, and their composite spaces. Thus, our framework encompasses various data types, such as torus-valued data (including circular data), special orthogonal matrix-valued data, and certain hyperspherical data. We tackle the situation where the Lie group-valued predictor is subject to Lie group-valued measurement error, establishing a theoretical foundation for this conditional U-statistic framework. This paper can be seen as a nonlinear extension of [76]. Beyond establishing convergence rates, we derive the asymptotic distribution for a new deconvolution regression estimator and formulate asymptotic confidence intervals. Achieving these results is highly challenging and unconventional, relying extensively on harmonic analysis on Lie groups and incorporating various novel tools and facts previously unknown in non-Euclidean contexts. The derivation of these results significantly deviates from approaches used in Euclidean cases, such as in [132]. We extend some arguments from [76] to handle conditional U-statistics. Additionally, we consider a data domain of the form $\mathbb{R}^d \times G$ to analyze general contaminated data, where $d \in \mathbb{N}$. This framework covers many types of data, such as linear-circular data on the cylinder. Our approach even accommodates cases where some or all of the Euclidean and non-Euclidean variables are error-free. In this paper, we do not assume that our target density is the product of a marginal density on $\mathbb{R}^d$ and a marginal density on $G$, which could otherwise be estimated by the product of two marginal density estimators. We derive various asymptotic properties of our novel conditional U-statistics, following a similar spirit as in [75]. The intricacies lie in the details, and extending the previous results from the Nadaraya-Watson estimator ([76]) to conditional U-statistics involves significantly different proofs and requires substantial novel techniques. To the best of our knowledge, there are no available results for the Nadaraya-Watson estimator based on mixed data, and our results in this setting are the first established in full generality.

This paper is organized as follows. Section 2 provides essential notation, definitions, and fundamental properties related to the Fourier analysis on topological groups, including examples to enhance understanding of our framework. Section 3 offers a comprehensive overview of the problem setting, detailing our methodology and the technical assumptions underlying our approach. Sections 4 and 5 present the primary results, including findings related to variables that are a mix of Euclidean and non-Euclidean types. Specifically, Section 4 focuses on the rate of convergence, while Section 5 discusses asymptotic distribution results. Section 6 explores various practical applications, such as the construction of asymptotic confidence intervals in §6.1, Kendall's tau in §6.2, and discrimination problems in §6.3. Section 8 compiles all proofs to maintain a coherent narrative flow. The Appendix contains additional technical properties and lemmas.

## 2. Preliminaries and notation

Before properly presenting our results and providing a strong base for our discussion, we will need to introduce some notation and fundamental definitions
## 2.1. Preliminaries

We begin by clarifying the definitions of certain unfamiliar statistical concepts essential for introducing our methodology. A set $\mathcal{G}$ equipped with an operation $\circ$ is termed a group if it satisfies the following conditions: (i) there exists an (identity) element $e \in \mathcal{G}$ such that $e \circ g = g \circ e = g$ for all $g \in \mathcal{G}$; (ii) for each $g \in \mathcal{G}$, there exists an (inverse) element $g^{-1} \in \mathcal{G}$ such that $g \circ g^{-1} = g^{-1} \circ g = e$; (iii) $g_1 \circ (g_2 \circ g_3) = (g_1 \circ g_2) \circ g_3$ for all $g_1, g_2, g_3 \in \mathcal{G}$. A group $\mathcal{G}$ equipped with a group operation $\circ$ is called a Lie group if it is a finite-dimensional smooth manifold such that the map $f: \mathcal{G} \times \mathcal{G} \to \mathcal{G}$ defined by $f(g_1, g_2) = g_1 \circ g_2^{-1}$ is smooth ($\mathcal{C}^\infty$).

For a separable complex Hilbert space $\mathcal{H}$, we let $\text{Inv}(\mathcal{H})$ denote the space of all bounded linear invertible operators from $\mathcal{H}$ to itself, and $\langle \cdot, \cdot \rangle_{\mathcal{H}}$ denote an inner product of $\mathcal{H}$. For a Lie group $\mathcal{G}$ and a separable complex Hilbert space $\mathcal{H}$, a homomorphism $\sigma: \mathcal{G} \to \text{Inv}(\mathcal{H})$ is called a strongly continuous unitary representation of $\mathcal{G}$ if (i) for each $h \in \mathcal{H}$ the map $g \mapsto \sigma(g)(h)$ is continuous and (ii) for each $g \in \mathcal{G}$ the map $\sigma(g): \mathcal{H} \to \mathcal{H}$ is a unitary operator, that is, $\langle h_1, h_2 \rangle_{\mathcal{H}} = \langle \sigma(g)(h_1), \sigma(g)(h_2) \rangle_{\mathcal{H}}$ for all $h_1, h_2 \in \mathcal{H}$. Such $\mathcal{H}$ is called the representation space of $\sigma$ and is denoted by $\mathcal{H}_{\sigma}$. Also, $\dim(\mathcal{H}_{\sigma})$ is called the dimension of $\sigma$. Hereafter, we call strongly continuous unitary representation simply representation.

For a representation $\sigma: \mathcal{G} \to \text{Inv}(\mathcal{H}_{\sigma})$, a closed linear subspace $\mathcal{W}_{\sigma}$ of $\mathcal{H}_{\sigma}$ is called invariant for $\sigma$ if $\sigma(g)(\mathcal{W}_{\sigma}) \subset \mathcal{W}_{\sigma}$ for all $g \in \mathcal{G}$. A representation $\sigma: \mathcal{G} \to \text{Inv}(\mathcal{H}_{\sigma})$ is called irreducible if $\mathcal{H}_{\sigma} \neq \{0_{\sigma}\}$ and the only invariant subspaces for $\sigma$ are $\mathcal{H}_{\sigma}$ and $\{0_{\sigma}\}$, where $0_{\sigma}$ is the zero vector of $\mathcal{H}_{\sigma}$. It is known that (i) if $\mathcal{G}$ is compact, then every irreducible representation of $\mathcal{G}$ is of finite dimension, and (ii) if $\mathcal{G}$ is Abelian, that is, $g_1 \circ g_2 = g_2 \circ g_1$ for all $g_1, g_2 \in \mathcal{G}$, then every irreducible representation of $\mathcal{G}$ is of one dimension. Two irreducible representations $\sigma: \mathcal{G} \to \text{Inv}(\mathcal{H}_{\sigma})$ and $\tau: \mathcal{G} \to \text{Inv}(\mathcal{H}_{\tau})$ are said equivalent, denoted by $\sigma \sim \tau$, if $\mathcal{H}_{\sigma}$ and $\mathcal{H}_{\tau}$ are isomorphic and there exists a bounded linear unitary operator $T: \mathcal{H}_{\sigma} \to \mathcal{H}_{\tau}$ satisfying $T(\sigma(g)(h)) = \tau(g)(T(h))$ for all $g \in \mathcal{G}$ and $h \in \mathcal{H}_{\sigma}$. The relation $\sim$ between irreducible representations of $\mathcal{G}$ is an equivalence relation. We take a representative for each equivalence class $[\sigma] = \{\tau: \tau$ is an irreducible representation of $\mathcal{G}$ such that $\tau \sim \sigma\}$ and write the space of all representatives by $\hat{\mathcal{G}}$. It is known that $\hat{\mathcal{G}}$ is countable if $\mathcal{G}$ is compact.

A Borel measure $\mu$ on a Lie group $\mathcal{G}$ is a left Haar measure if $\mu(g \circ A) = \mu(A)$ for all $g \in \mathcal{G}$, and Borel sets $A \subset \mathcal{G}$. A unique normalized Haar measure exists for compact $\mathcal{G}$. In Fourier analysis of topological groups, analogous definitions hold, with additional considerations for continuity and topological properties. For a topological group $\mathcal{G}$, the Fourier transform of an integrable function $f: \mathcal{G} \to \mathbb{C}$ relative to a Haar measure $\mu$ involves matrix-valued integrals, leading to results like the Fourier inversion formula and Plancherel's theorem. We highlight that Lie groups, being smooth manifolds, are locally compact and Hausdorff. Examples include $\mathbb{R}^d$, $\mathbb{T}^q$, $\text{SO}(3)$, and $\text{SU}(2)$. The product of two Lie groups forms another Lie group, with the dimension being the sum of the dimensions of the factors.

**Example 2.1.** We illustrate the aforementioned concepts across various topological groups. Here, we denote $\iota = \sqrt{-1}$ and $\langle \cdot, \cdot \rangle$ as the inner product in Euclidean spaces, as referenced in [75].

(i) **Euclidean Spaces:** Consider $\mathcal{G} = \mathbb{R}^d$ with $d \in \mathbb{N}$ and $\circ$ representing conventional addition. We define $\hat{\mathcal{G}} = \{\sigma_t: t \in \mathbb{R}^d\}$, $\mathcal{H}_{\sigma_t} = \mathbb{C}$, and $\dim(\mathcal{H}_{\sigma_t}) = 1$. Here, $\sigma_t(g)(h) = h \cdot e^{-i \langle t, g \rangle}$ for $g \in \mathbb{R}^d$ and $h \in \mathbb{C}$, as detailed in Chapter 2.1 of [3].

(ii) **Toruses:** Let $\mathbb{T}^q = \{g = (g_1, \dots, g_q) \in \mathbb{C}^q : |g_j| = 1, j \in \{1, \dots, q\}\}$ for $q \in \mathbb{N}$. For $\mathcal{G} = \mathbb{T}^q$ with $\circ$ denoting componentwise multiplication in $\mathbb{C}^q$, we have $\hat{\mathcal{G}} = \{\sigma_\ell : \ell = (\ell_1, \dots, \ell_q) \in \mathbb{Z}^q\}$, $\mathcal{H}_{\sigma_\ell} = \mathbb{C}$, and $\dim(\mathcal{H}_{\sigma_\ell}) = 1$. The expression $\sigma_\ell(g)(h) = h \cdot \prod_{j=1}^q g_j^\ell$ applies, with $h \in \mathbb{C}$; see Chapter 2.1 of [3] for more details.

(iii) **Rotation Group:** Let $O(3)$ denote the space of $3 \times 3$ real orthogonal matrices, and $\text{SO}(3) = \{g \in O(3) : \det(g) = 1\}$. For $\mathcal{G} = \text{SO}(3)$ with $\circ$ symbolizing matrix multiplication, we identify $\hat{\mathcal{G}} = \{\sigma_\ell : \ell \in \{0\} \cup \mathbb{N}\}$ and $\dim(\mathcal{H}_{\sigma_\ell}) = 2\ell + 1$. Detailed forms of $\sigma_\ell$ and $\mathcal{H}_{\sigma_\ell}$ are provided in Chapter 7.6 of [52].
(iv) **Special Unitary Group of Degree 2:** Define $SU(2) = \{g = \begin{pmatrix} g_{11} & g_{12} \\ -g_{12} & g_{11} \end{pmatrix} : g_{11}, g_{12} \in \mathbb{C}, |g_{11}|^2 + |g_{12}|^2 = 1\}$ with $\bar{a}$ representing the conjugate of $a \in \mathbb{C}$. For $\mathcal{G} = SU(2)$ with $\circ$ denoting matrix multiplication, we have $\hat{\mathcal{G}} = \{\sigma_\ell : \ell \in \{0\} \cup \mathbb{N}\}$ and $\dim(\mathcal{H}_{\sigma_\ell}) = \ell + 1$. Further details are available in Chapter 7.5 of [45].

(v) **Product Groups:** Let $G_1$ and $G_2$ be topological groups with group operations $\circ_1$ and $\circ_2$, respectively. The product space $G = G_1 \times G_2$ with group operation $\circ$ defined by $(g_1, g_2) \circ (g_1^*, g_2^*) = (g_1 \circ_1 g_1^*, g_2 \circ_2 g_2^*)$ for $g_1, g_1^* \in G_1$ and $g_2, g_2^* \in G_2$, forms a topological group. Each finite-dimensional irreducible representation of $G$ is given by $\sigma_1 \otimes \sigma_2 : G \to \text{Inv}(\mathcal{H}_{\sigma_1} \otimes \mathcal{H}_{\sigma_2})$, where $\mathcal{H}_{\sigma_1} \otimes \mathcal{H}_{\sigma_2}$ denotes the tensor product of $\mathcal{H}_{\sigma_1}$ and $\mathcal{H}_{\sigma_2}$, as detailed in Theorem 3.10.2 of [45].

We call $G$ a (locally) compact group if it is a topological group that is (locally) compact and Hausdorff. An Abelian group, denoted by $\mathcal{G}$, satisfies the condition $g_1 \circ g_2 = g_2 \circ g_1$ for any $g_1, g_2 \in \mathcal{G}$. Notably, $\mathbb{T}^d$ is a locally compact Abelian group, while $\mathbb{T}^q$ is a compact Abelian group. Additionally, the product of two (locally) compact (Abelian) groups is also a (locally) compact (Abelian) group. A Borel measure $\mu(\cdot)$ on a locally compact group $G$ is called left-translation-invariant if $\mu(g \circ A) = \mu(A)$ for every $g \in G$ and Borel set $A \subset G$, where $g \circ A = \{g \circ a : a \in A\}$. Similarly, a right-translation-invariant Borel measure is defined with $g \circ A$ replaced by $A \circ g = \{a \circ g : a \in A\}$. If $G$ is a locally compact Abelian group or a compact group, there exists a Radon measure that is both left- and right-translation-invariant. Such a Radon measure is termed a Haar measure. Specifically, if $G$ is a compact group, there exists a unique Haar measure $\mu(\cdot)$ such that $\mu(G) = 1$, known as the normalized Haar measure.

Now, let us consider the scenario where $G$ is a locally compact Abelian group. In this context, $\sigma^M(g)$ is a complex number such that $|\sigma^M(g)| = 1$ for all $g \in G$. The space $\tilde{\mathcal{G}} = \{\sigma^M : \sigma \in \hat{\mathcal{G}}\}$ consists of functions $\sigma^M : G \to \mathbb{T}^1$ and forms a locally compact Abelian group. The group operation $\tilde{o}$ is defined by $\sigma^M \tilde{o} \tau^M = (\sigma \otimes \tau)^M$, where the irreducible representation $\sigma \otimes \tau : G \to \text{Inv}(\mathbb{C})$ is given by $\sigma \otimes \tau(g)(h) = \sigma(g)(h) \cdot \tau(g)(h)$. Consequently, there exists a Haar measure $\tilde{\mu}(\cdot)$ on $\tilde{\mathcal{G}}$. If $f : G \to \mathbb{C}$ is integrable with respect to $\mu(\cdot)$, then the function $\phi^f : \tilde{\mathcal{G}} \to \mathbb{C}$, mapping $\sigma^M$ to $\phi^f(\sigma^M)$, is integrable with respect to $\tilde{\mu}(\cdot)$.

## 2.2. Notation

Let $\| \cdot \|_2$ denote the norm on $L^2((\mathcal{G}, \mu), \mathbb{C})$ induced by the inner product $\langle \cdot, \cdot \rangle_2$. The Fourier transform of an integrable function $f : \mathcal{G} \to \mathbb{C}$ at $\sigma^M$ is written as $\phi^f(\sigma^M)$, defined by

$$ \phi^f(\sigma^M) = \int_{\mathcal{G}} \sigma^M(g^{-1}) f(g) d\mu(g), $$

with its $(i, j)$-th element denoted by $\phi_{ij}^f(\sigma^M)$. With this notation, $\phi^{f \vee}(\sigma^M) = \phi^{\vee}(\sigma^M)$ for $\mathbb{V} = \mathbb{X}, \mathbb{X}', \mathbb{U}, \mathbb{U}', \mathbb{Z},$ and $\mathbb{Z}'$. Let $\text{Im}(a)$ represent the imaginary part of a complex number $a \in \mathbb{C}$. Note that $[a] = \min\{b \in \mathbb{N} : b > a\}$. We denote by $\mathbb{C}$ positive constants that may vary at each occurrence. Throughout this paper, we use the following notation for $\mathbb{V}$, which can be any of $\mathbb{X}, \mathbb{X}', \mathbb{Z},$ or $\mathbb{Z}'$:

$$ \mathbb{V} = (\mathbf{V}_1, \dots, \mathbf{V}_k) \in \mathbb{G}^k, \quad \text{and} \quad \mathbb{V}_i = (\mathbf{V}_{i1}, \dots, \mathbf{V}_{i\ell}) \in \mathbb{G}^\ell, \quad \mathbf{i} = (i_1, \dots, i_\ell) \in I(\ell, n), $$

$$ \mathbb{Y} = (\mathbf{Y}_1, \dots, \mathbf{Y}_k) \in \mathbb{R}^{dk}, \quad \text{and} \quad \mathbb{Y}_i = (\mathbf{Y}_{i1}, \dots, \mathbf{Y}_{i\ell}) \in \mathbb{R}^{d\ell}, \quad \mathbf{i} = (i_1, \dots, i_\ell) \in I(\ell, n). $$

## 3. Model and estimation

Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space and $\mathcal{G}$ be a compact and connected Lie group equipped with a group operation $\circ$. We consider a sequence of independent and identically distributed random vectors $\{(\mathbf{X}_i, \mathbf{Y}_i) : i \in \mathbb{N}^*\}$ with $\mathbf{X}_i \in \mathcal{G}$ and $\mathbf{Y}_i \in \mathbb{R}^d$. In our analysis, we will explore the case where the variable $\mathbf{X}$ is affected by an unobservable measurement error $\mathbb{U} = (\mathbf{U}_1, \dots, \mathbf{U}_k) : \Omega^k \to \mathbb{R}^k$ satisfying $\mathbb{U} \perp (\mathbf{X}, \mathbb{Y})$ so that we only observe $\mathbb{Z} := \mathbb{U} \circ \mathbf{X}$. Here, $\perp$ stands for statistical independence.
Let $(\mathbf{Y}_1, \mathbf{Z}_1), \dots, (\mathbf{Y}_n, \mathbf{Z}_n)$ be n i.i.d. copies of $(\mathbf{Y}, \mathbf{Z})$ that we actually observe. Let $\varphi : \mathbb{R}^{dk} \to \mathbb{R}$ be a measurable function. In this paper, we are primarily concerned with the estimation of the conditional expectation, or regression function of $\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k)$ evaluated at $(\mathbf{Z}_1, \dots, \mathbf{Z}_k) = \tilde{\mathbf{t}}$, given by

$$r^{(k)}(\varphi, \tilde{\mathbf{t}}) = \mathbb{E} (\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) | (\mathbf{Z}_1, \dots, \mathbf{Z}_k) = \tilde{\mathbf{t}}), \quad \text{for } \tilde{\mathbf{t}} \in \mathbb{G}^k, \qquad (3.1)$$

whenever it exists, i.e., $\mathbb{E}(|\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k)|) < \infty$. In describing our procedure for estimating the regression $r^{(k)}(\varphi, \cdot)$, we denote $\tilde{f}_{\mathbf{X}}(\cdot)$ and $\tilde{f}_{\mathbf{U}}(\cdot)$ as the respective square integrable densities of $\mathbf{X}$ and $\mathbf{U}$ with respect to the normalized Haar measure $\mu \in \mathbb{G}$. It can be readily verified that the density $\tilde{f}_{\mathbf{Z}}(\cdot)$ of $\mathbb{Z}$ with respect to $\mu(\cdot)$ exists and is determined by the following convolution formula:

$$\tilde{f}_{\mathbb{Z}}(\tilde{\mathbf{z}}) := \prod_{j=1}^{k} f_{\mathbb{Z}}(\mathbf{z}_j) = \prod_{j=1}^{k} (f_{\mathbf{U}} * f_{\mathbf{X}})(\mathbf{z}_j) := \prod_{j=1}^{k} \int f_{\mathbf{U}}(\mathbf{u}_j) f_{\mathbf{X}}(\mathbf{u}_j^{-1} \circ \mathbf{z}_j) d\mu(\mathbf{u}_j), \\ \tilde{\mathbf{z}} = (\mathbf{z}_1, \dots, \mathbf{z}_k) \in \mathbb{G}^k, \qquad (3.2)$$

where $\mathbf{u}_j^{-1}$ is the inverse element of $\mathbf{u}_j \in \mathbb{G}$. For a given $\sigma \in \hat{\mathbb{G}}$ and its dimension $d_\sigma \in \mathbb{N}$, let $\{e_i^\sigma : 1 \le i \le d_\sigma\}$ be an orthonormal basis of the representation space $\mathbb{H}_\sigma$ of $\sigma$. Also, for each $g \in \mathbb{G}$, let $\sigma^M(g)$ denote the $d_\sigma \times d_\sigma$ complex matrix whose $(i, j)$th element equals $\sigma_{ij}^M(g) := \langle \sigma(g) (e_i^\sigma), e_i^\sigma \rangle_{\mathbb{H}_\sigma} = \overline{\langle e_i^\sigma, \sigma(g) (e_j^\sigma) \rangle_{\mathbb{H}_\sigma}}$. We call $\sigma^M(g)$ the matrix form of $\sigma(g)$ with respect to $\{e_i^\sigma : 1 \le i \le d_\sigma\}$. Then, for $\mathbb{V} = (\mathbf{V}_1, \dots, \mathbf{V}_k)$ being any of $\mathbb{X}, \mathbb{U}$ and $\mathbb{Z}$, the Fourier transform $\phi^{\mathbb{V}}(\sigma^M)$ of $\tilde{f}_{\mathbb{V}}$ at $\sigma^M$ is given by the following matrix-valued integral:

$$\phi(\mathbf{V}_1, \dots, \mathbf{V}_k) (\sigma^M) := \prod_{j=1}^{k} \int \sigma^M(g_j^{-1}) f_{\mathbf{V}}(g_j) d\mu(g_j) = \prod_{j=1}^{k} \mathbb{E} (\sigma^M ((\mathbf{V}_j)^{-1})). \qquad (3.3)$$

Also, the following convolution identity holds (e.g., (5.1.7) in [54]):

$$\phi(\mathbf{Z}_1, \dots, \mathbf{Z}_k) (\sigma^M) = \phi(\mathbf{X}_1, \dots, \mathbf{X}_k) (\sigma^M) \phi(\mathbf{U}_1, \dots, \mathbf{U}_k) (\sigma^M), \quad \sigma \in \hat{\mathbb{G}}, \qquad (3.4)$$

where the right-hand side is to be understood as the matrix multiplication. Let $L^2((\mathbb{G}, \mu), \mathbb{C})$ be the space of complex-valued functions $f : \mathbb{G} \to \mathbb{C}$ such that $\int_{\mathbb{G}} f(g) \overline{f(g)} d\mu(g) < \infty$, endowed with the inner product $\langle f_1, f_2 \rangle_2 = \int_{\mathbb{G}} f_1(g) \overline{f_2(g)} d\mu(g)$. According to Lemma 1 in [76], $\tilde{f}_{\mathbf{X}}(\cdot) := \prod_{j=1}^{k} f_{\mathbf{X}}(\cdot)$ admits the following Fourier series expansion:

$$\tilde{f}_{\mathbf{X}}(\cdot) = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^{k} d_{\sigma_j} \sum_{i, \ell=1}^{d_{\sigma_j}} \left\langle f_{\mathbf{X}}(\cdot), (\sigma_{i\ell}^M)_{j\ell}(\cdot) \right\rangle_2 (\sigma_{i\ell}^M)_{j\ell}(\cdot) \\ = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^{k} d_{\sigma_j} \mathrm{Tr} \left( \phi^{\mathbb{X}}(\sigma_j^M) \sigma_j^M(\cdot) \right), \qquad (3.5)$$

where the equalities hold in $L^2((\mathbb{G}, \mu), \mathbb{C})$-sense and $\mathrm{Tr}(A)$ for a square matrix A denotes the trace of A. From (3.4) and (3.5), we get the following Fourier series expansion:

$$\tilde{f}_{\mathbf{X}}(\cdot) = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^{k} d_{\sigma_j} \mathrm{Tr} \left( \phi^{\mathbb{Z}}(\sigma_j^M) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \sigma_j^M(\cdot) \right), \qquad (3.6)$$

if the following condition holds:
(K2) $\phi^{\mathbf{U}_j}(\sigma_j^M)$ are invertible for all $\sigma_j^M \in \hat{\mathbb{G}}$.

We take a finite sum over $(\sigma_1, \dots, \sigma_k) \in \hat{\mathbb{G}}$ with a truncation point determined by the Casimir spectrum of $\mathbb{G}$ and a smoothing parameter $T_n > 0$ such that $\lim_{n \to \infty} T_n = \infty$. The Casimir spectrum is an unbounded set $\{k_{\sigma_j} : \sigma_j \in \hat{\mathbb{G}}\}$ with $k_{\sigma_j} \ge 0$ such that $-k_{\sigma_j}$ for each $\sigma_j$ is the eigenvalue of the Laplace-Beltrami operator $\Delta$ (twice differential operator acting on twice continuously differentiable functions mapping $\mathbb{G}$ to $\mathbb{C}$), corresponding to $(\sigma_{i\ell}^M)_j(\cdot)$ as eigenfunctions, that is, $\Delta((\sigma_{i\ell}^M)_j) = -k_{\sigma_j} \cdot (\sigma_{i\ell}^M)_j$ for all $1 \le i, \ell \le d_{\sigma_j}$ and $1 \le j \le k$. We cut off the infinite sum $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} d_{\sigma_j} \text{Tr}(\sigma_j^M (\mathbf{z}_i^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \sigma_j^M(\cdot))$ at $\sigma_j$ where the Casimir spectrum $k_{\sigma_j}$ exceeds $T_n$. In this way, we obtain a finite sum since the set $\{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}} : k_{\sigma_j} < T_n\}$ is finite for each $n$. The Casimir spectrum also has the property $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}} : k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \asymp T_n^{k \dim(\mathbb{G})/2}$, which is useful in controlling the strength of the variability of the resulting finite sum by adjusting $T_n$. The truncation gives a density product estimator, for $\tilde{\mathbf{z}} = (\mathbf{z}_1, \dots, \mathbf{z}_k)$, $\tilde{\mathbf{t}} = (\mathbf{t}_1, \dots, \mathbf{t}_k) \in \mathbb{G}^k$,

$$ \hat{f}_n^{(k)}(\tilde{\mathbf{t}}) = \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k}), \\ \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \tilde{\mathbf{z}}) = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}} : k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \text{Tr} \left( \sigma_j^M (\mathbf{z}_j^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \sigma_j^M(\mathbf{t}_j) \right). \quad (3.7) $$

The function $\tilde{K}_{T_n}(\cdot, \cdot)$ mentioned above serves as the equivalent of the Euclidean “deconvolution kernel” utilized in [131] and [50], which studied the cases of Euclidean measurement errors when $k=1$. Notably, the $L^2$ error rates of $\hat{f}_n^{(k)}(\cdot)$ were studied by [86] for $\mathbb{G}$-valued measurement errors, also in the context where $k=1$. We define the following novel deconvolution product density estimator

$$ \tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) = \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \text{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})), \quad (3.8) $$

where $\text{Re}(a)$ stands for the real part of a complex number $a \in \mathbb{C}$. Taking the real part of $\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})$ for each $\tilde{\mathbf{t}}$ and $\mathbf{Z}_{i_j}$ as in (3.8) is natural since the target $\hat{f}_{\mathbb{X}}(\cdot)$ is real-valued. It actually gives an estimator with a smaller error since $|\text{Re}(a) - b| \le |a - b|$ for all $a \in \mathbb{C}$ and $b \in \mathbb{R}$. Including the imaginary part gives $\hat{f}_n^{(k)}(\cdot)$ and an estimator of $r^{(k)}(\varphi, \cdot)$ whose asymptotic distributions are supported on $\mathbb{C}$, which would mystify the construction of the asymptotic confidence intervals on $\mathbb{R}$ for the targets $\hat{f}_{\mathbb{X}}(\cdot)$ and $r^{(k)}(\varphi, \cdot)$. In this context, [132] presented a class of estimators for $r^{(k)}(\varphi, \cdot)$, called the conditional U-statistics, which is defined for each $\tilde{\mathbf{t}} \in \mathbb{G}^k$ as follows:

$$ \hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \frac{\sum_{(i_1, \dots, i_k) \in I_n^k} \varphi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \text{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k}))}{\sum_{(i_1, \dots, i_k) \in I_n^k} \text{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k}))}. \quad (3.9) $$

The evaluation of $\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})$ in the estimators (3.8) and (3.9) needs the knowledge of $k_{\sigma_j}$, $\sigma_j^M$ and $\phi^{\mathbf{U}_j}$. In the particular case $k=1$, the $r^{(k)}(\varphi, \tilde{\mathbf{t}})$ is reduced to $r^{(1)}(\varphi, \mathbf{t}) = \mathbb{E}(\varphi(\mathbf{Y}) | \mathbf{Z} = \mathbf{t})$ and Stute's estimator becomes the Nadaraya-Watson estimator of $r^{(1)}(\varphi, \mathbf{t})$ of [76] given by:

$$ \hat{r}_n^{(1)}(\varphi, T_n, \mathbf{t}) = \frac{\sum_{i=1}^n \varphi(\mathbf{Y}_i) \text{Re}(K_{T_n}(\mathbf{t}, \mathbf{Z}_i))}{\sum_{i=1}^n \text{Re}(K_{T_n}(\mathbf{t}, \mathbf{Z}_i))}. $$
Consider the U-kernel

$$
G_{\varphi, T_n, \tilde{\mathbf{t}}}(\tilde{\mathbf{z}}, \tilde{\mathbf{y}}) := \varphi(\tilde{\mathbf{y}}) \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \tilde{\mathbf{z}}), \quad \tilde{\mathbf{t}}, \tilde{\mathbf{z}} \in \mathbb{G}^k, \quad \tilde{\mathbf{y}} \in \mathbb{R}^{dk}.
$$

For the sample (Z₁, Y₁), . . . , (Zn, Yn), define

$$
\begin{align}
\tilde{U}_n(\varphi, T_n, \tilde{\mathbf{t}}) := \tilde{U}_n^{(k)}(G_{\varphi, T_n, \tilde{\mathbf{t}}}) &= \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} G_{\varphi, T_n, \mathbf{t}}(\mathbb{Z}_{\mathbf{i}}, \mathbb{Y}_{\mathbf{i}}) \nonumber \\
&= \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \varphi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sum_{\sigma_1, \dots, \sigma_k \in \mathbb{G}: k\sigma_j < T_n} \nonumber \\
&\qquad \times \prod_{j=1}^k d_{\sigma_j} \operatorname{Tr} \left( \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \sigma_j^M (\mathbf{t}_j) \right). \tag{3.10}
\end{align}
$$

**Remark 3.1.** It is noteworthy that the estimator represented in (3.9) resembles a Stute-type estimator, which is well-known for Euclidean data. This presentation aims to help the reader draw parallels between the estimator for Lie group data and its Euclidean counterpart. However, the theoretical framework for the estimator (3.9) differs significantly from those for Euclidean Stute-type estimators.

**Remark 3.2.** The estimators $\tilde{f}_n^{(k)}(\cdot)$ and $\hat{r}_n^{(k)}(\cdot, \cdot)$ are independent of the choice of an orthonormal basis of $\mathbb{H}_\sigma^k$. If we denote two distinct orthonormal bases of $\mathbb{H}_\sigma^k$ as $\{e_i^\sigma : 1 \le i \le d_\sigma\}$ and $\{v_i^\sigma : 1 \le i \le d_\sigma\}$ respectively, and represent the matrix forms of $\sigma(g)$ as $\sigma^{M,e}(g)$ and $\sigma^{M,v}(g)$, then, it's demonstrated that $\sigma^{M,v}(g) = A\sigma^{M,e}(g)A^*$ for all $g \in \mathbb{G}$, where $A$ denotes a $d_\sigma \times d_\sigma$ unitary matrix determined by the orthonormal bases $\{e_i^\sigma : 1 \le i \le d_\sigma\}$ and $\{v_i^\sigma : 1 \le i \le d_\sigma\}$, and $A^*$ denotes the conjugate transpose of $A$. Because $\operatorname{Tr}(A_1A_2) = \operatorname{Tr}(A_2A_1)$ holds for all complex square matrices $A_1$ and $A_2$, it follows that the deconvolution kernels $K_{T_n}(\cdot, \cdot)$ at (3.7), derived from $\{e_i^\sigma : 1 \le i \le d_\sigma\}$ and $\{v_i^\sigma : 1 \le i \le d_\sigma\}$, are identical.

**Proposition 3.3 ([76]).** $\int_{\mathbb{G}} K_{T_n}(\mathbf{t}, \mathbf{z}) d\mu(\mathbf{t}) = 1$ for any $\mathbf{z} \in \mathbb{G}$, so that $\int_{\mathbb{G}} \operatorname{Re}(K_{T_n}(\mathbf{t}, \mathbf{z})) d\mu(\mathbf{t}) = 1$ for any $\mathbf{z} \in \mathbb{G}$ and $\int_{\mathbb{G}^k} \tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) d\mu(\tilde{\mathbf{t}}) = 1$.

Noting that $\operatorname{Re}(K_{T_n}(\cdot, \cdot))$ plays the role of the Euclidean deconvolution kernel.

An essential functionality that $\operatorname{Re}(\tilde{K}_{T_n}(\cdot, \cdot))$ must have is the ability to remove the effect of $\mathcal{U}$ that might be of a nonnegligible order if

$$
\tilde{K}_{T_n}^0(\tilde{\mathbf{t}}, \tilde{\mathbf{v}}) = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \operatorname{Tr} \left( \sigma_j^M (\mathbf{v}_j^{-1}) \sigma_j^M (\mathbf{t}_j) \right), \quad \tilde{\mathbf{t}}, \tilde{\mathbf{v}} \in \mathbb{G}^k \tag{3.11}
$$

is used, instead of $\operatorname{Re}(\tilde{K}_{T_n}(\cdot, \cdot))$, with the noisy $\mathbf{Z}_{i_j}$. In view of (3.5) and considering that $\prod_{j=1}^k \mathbb{E}(\sigma_j^M (\mathbf{X}_j^{-1})) = \prod_{j=1}^k \sigma_j^M (\mathbf{X}_j)$, If the true predictor values $\mathbf{X}_j$ are observed, one could use $\tilde{K}_{T_n}^0(\cdot, \cdot)$ to estimate $\tilde{f}_{\mathbb{X}}$. For example, one may estimate $\tilde{f}_{\mathbb{X}}$ by

$$
\frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \tilde{K}_{T_n}^0 (\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k}) \quad \text{or} \quad \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \operatorname{Re} (\tilde{K}_{T_n}^0 (\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})),
$$

where the former has been studied by [66] for the error-free case, in the case where $k=1$.

The following proposition gives that $\hat{f}_n(\cdot)$ integrates to one, which is a desired property for density estimators.

The following proposition gives a relationship between $\tilde{K}_{T_n}^0(\cdot, \cdot)$ and $\tilde{K}_{T_n}(\cdot, \cdot)$.
**Proposition 3.4.** $\prod_{j=1}^k \mathbb{E}(\sigma_j^M(Z^{-1}) | \mathbf{X}) = \prod_{j=1}^k \sigma_j^M(X^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1}$, so that $\mathbb{E}(\mathrm{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}))) | \mathbf{X}) = \mathrm{Re}(\tilde{K}_{T_n}^0(\tilde{\mathbf{t}}, \mathbf{X}))$, for all $\tilde{\mathbf{t}} \in \mathbb{G}^k$.

The conclusion asserted in the above proposition, termed as unbiased scoring, implies that $\prod_{j=1}^k \mathbb{E}(\sigma_j^M(Z^{-1})\varphi(\mathbf{Y}))\phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} = \phi^{r^{(k)}\cdot \tilde{\mathbf{f}}_X}(\sigma_j^M)$, where $\phi^{r^{(k)}\cdot \tilde{\mathbf{f}}_X}(\sigma_j^M)$ is the Fourier transform of $r^{(k)}(\varphi, \cdot) \cdot \tilde{\mathbf{f}}_X(\cdot)$ at $\sigma_j^M$ given by $\prod_{j=1}^k \int_{\mathbb{G}^k} \sigma_j^M(g_j^{-1})r^{(k)}(g_j)f_X(g_j)d\mu(g_j)$. Indeed, it holds that

$$ 
\begin{aligned}
& \prod_{j=1}^{k} \mathbb{E}(\sigma_j^M(Z^{-1}) \varphi(\mathbf{Y})) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \\
& = \prod_{j=1}^{k} \mathbb{E}(\mathbb{E}(\sigma_j^M(Z^{-1}) \varphi(\mathbf{Y}) | \mathbf{X})) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \\
& = \prod_{j=1}^{k} \mathbb{E}(\sigma_j^M(X^{-1}) \mathbb{E}(\sigma_j^M(U^{-1}) | \mathbf{X}) r^{(k)}(\mathbf{X})) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \\
& = \prod_{j=1}^{k} \mathbb{E}(\sigma_j^M(X^{-1}) r^{(k)}(\mathbf{X})) \\
& = \phi^{r^{(k)}\cdot \tilde{\mathbf{f}}_X}(\sigma_j^M),
\end{aligned}
$$

where the second and third equalities follow from $U \perp (X, Y)$. The above result is useful in various places of our asymptotic analysis. In particular, the unbiased scoring property gives that

$$ 
\begin{aligned}
\mathbb{E}(\tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) | \mathbf{X}_1, \dots, \mathbf{X}_n) &= \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \mathrm{Re}(\tilde{K}_{T_n}^0(\tilde{\mathbf{t}}, \mathbf{X}_{i_1}, \dots, \mathbf{X}_{i_k})) \\
\mathbb{E}(r^{(k)}(\varphi, \tilde{\mathbf{t}}) \tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) | \mathbf{X}_1, \dots, \mathbf{X}_n) &= \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \mathrm{Re}(\tilde{K}_{T_n}^0(\tilde{\mathbf{t}}, \mathbf{X}_{i_1}, \dots, \mathbf{X}_{i_k})) r^{(k)}(\varphi, \mathbf{X}_{i_k}).
\end{aligned}
$$

The above identities basically tell that the “cut-off spectral kernel” $\tilde{K}_{T_n}$ defined at (3.7) deconvolves efficiently the influence of measurement errors. They indicate that the biases of $\tilde{f}_n^{(k)}(\cdot)$ and $r^{(k)}(\varphi, \cdot)$ based on the contaminated observations $\mathbf{Z}_{i_j}$ are the same as the respective biases of the corresponding estimators based on the unobservable original $\mathbf{X}_{i_j}$.

**Remark 3.5.** The density estimator $\tilde{f}_n^{(k)}(\cdot)$ defined at (3.7) can also be constructed by replacing the Fourier transform $\phi^{\mathbb{Z}}(\sigma_j^M)$ in (3.6) with $\prod_{j=1}^k \int_{\mathbb{G}^k} \sigma_j^M(g_j^{-1}) \tilde{f}_n^{(k,0)}(g_j)(g_j)d\mu(g_j)$. Here, $\tilde{f}_n^{(k,0)}(\tilde{\mathbf{g}}) = \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \tilde{K}_{T_n}^0(\tilde{\mathbf{g}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})$ is the estimator of $\tilde{f}_{\mathbb{Z}}(\tilde{\mathbf{g}})$, which generalizes the estimator introduced in [66] for cases without measurement error. This construction automatically reduces the infinite sum $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}}$ in (3.6) to the finite sum $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k \sigma_j < T_n}$ as shown in the expression of $\hat{f}_n^{(k)}(\tilde{\mathbf{t}})$. Alternatively, one might use a kernel density estimator of $\tilde{f}_{\mathbb{Z}}$ such as

$$ \tilde{f}_{\mathbb{Z}}(\tilde{\mathbf{g}}) := \frac{(n-k)!}{n!h^k} \sum_{(i_1, \dots, i_k) \in I_n^k} \prod_{j=1}^k (\Theta_{\mathbf{Z}_{i_j}}(g_j))^{-1} K(d(g_j, \mathbf{Z}_{i_j})/h), $$

which is a generalization of the estimator introduced in [116], instead of $\tilde{f}_n^{(k,0)}(\tilde{\mathbf{g}})$. Here, $K$ is a baseline kernel function, $h$ is the bandwidth, $\Theta_{\mathbf{Z}_{i_j}}$ is the volume density function at $\mathbf{Z}_{i_j}$ and $d$ is the
geodesic distance. However, density and regression estimators based on $\tilde{f}_{\mathbb{Z}}(\tilde{g})$ may include the infinite sum $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}}$ because the kernel weighting scheme does not automatically reduce it to a finite sum. This leads to high variability. One can take an arbitrary finite sum, which would introduce another tuning parameter in addition to the bandwidth $h$. Moreover, estimators based on $\tilde{f}_{\mathbb{Z}}(\tilde{g})$ do not have the unbiased scoring property that removes the impact of measurement errors from the bias.

### 3.1. Assumptions

#### Assumptions 3.6.

(A1) For some $\hat{m} > \dim(\mathbb{G})/4$ with $\hat{m} \in \mathbb{N}$, (i) $\tilde{f}_{\mathbb{X}}(\cdot) \in C^{2\hat{m}}(\mathbb{G}^k)$ and (ii) $r^{(k)}(\varphi, \cdot) \in C^{2\hat{m}}(\mathbb{G}^k)$;

(A2) $\tilde{f}_{\mathbb{X}}(\cdot)$ is bounded away from zero on $\mathbb{G}^k$.

**Assumptions 3.7.** The function $\varphi(\cdot)$ is such that:

* $\varphi(\cdot)$ is bounded, i.e., for some $0 < \mathbb{M} < \infty$:

$$ \sup_{(\mathbf{y}_1, \dots, \mathbf{y}_k) \in \mathbb{R}^{dk}} |\varphi(\mathbf{y}_1, \dots, \mathbf{y}_k)| \le \mathbb{M}; \quad (\text{F.i}) $$

* $\varphi(\cdot)$ is unbounded and satisfies, for some $\delta > 0$,

$$ \theta_{\delta} := \sup_{(\mathbf{t}_1, \dots, \mathbf{t}_k) \in \mathbb{G}^k} \mathbb{E} \left( |\varphi(\mathbf{Y})|^{2+\delta} \mid \mathbf{X} = (\mathbf{t}_1, \dots, \mathbf{t}_k) \right) < \infty. \quad (\text{F.ii}) $$

We assume the same level of smoothness for $\tilde{f}_{\mathbb{X}}(\cdot)$ and $r^{(k)}(\varphi, \cdot)$ in (A1) in Assumption 3.6 because the asymptotic analysis of $\tilde{U}_n(\varphi, T_n, \tilde{\mathbf{t}})$ in $\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}})$ needs the Fourier series of $r^{(k)}(\varphi, \cdot) \cdot \tilde{f}_{\mathbb{X}}(\cdot)$ to be expressed in terms of $\Delta^r (r^{(k)}(\varphi, \cdot) \cdot \tilde{f}_{\mathbb{X}}(\cdot))$. Here, $\Delta^r$ refers to the $\hat{m}$-times composition of the Laplace-Beltrami operator $\Delta$. This operator $\Delta$ acts on twice continuously differentiable functions mapping $\mathbb{G}$ to $\mathbb{C}$. Condition (A2) is a regularity condition similar to those typically assumed in non-parametric statistics for the Euclidean case.

### 4. Error rate analysis

In this section, we delve into a detailed explanation of the convergence rates for our estimator, which is represented in (3.9). The problems we are investigating in this section are far from straightforward, and the key merits of our contribution lie in achieving control over the $L_2$ errors under minimal conditions within this broad setting. This achievement represents a long-standing unresolved issue in the literature, pushing the boundaries of statistical research and paving the way for new research directions.

#### 4.1. Smoothness of measurement error distribution

In the Euclidean context, two primary smoothness classes are commonly distinguished for the distributions of measurement errors: ordinary-smoothness and supersmoothness scenarios, as well-discussed in [48]. For instance, the Laplace distribution represents an example of an ordinary-smoothness scenario in Euclidean spaces, while the Gaussian distribution exemplifies supersmoothness. We focus on the extended notions of smoothness for the present non-Euclidean setting, as introduced by [86]. To outline these smoothness scenarios, let us define:

$$ \|A\|_{\text{op}} = \sup \left\{ \|Av\|_{C^{d_{\sigma}}} : v \in C^{d_{\sigma}}, \|v\|_{C^{d_{\sigma}}} = 1 \right\}. $$
where A is a $d_{\sigma} \times d_{\sigma}$ complex matrix and $\| \cdot \|_{C^{d_{\sigma}}}$ is the complex norm on $C^{d_{\sigma}}$. We present the smoothness scenarios as follows

(S1) Non-Euclidean ordinary-smoothness class of order $\beta \ge 0$: There exist constants $c_1, c_2 > 0$ such that, for all $\sigma_j \in \hat{\mathbb{G}}$ with $k_{\sigma} > 0$,

$$ (i) \quad \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} \le c_1 k_{\sigma}^{\beta}, $$

$$ (ii) \quad \left\| \phi_{\mathbf{U}_j} (\sigma_j^M) \right\|_{\text{op}} \le c_2 k_{\sigma}^{-\beta}; $$

(S2) Non-Euclidean supersmoothness class of order $\beta > 0$: There exist constants $c_1, c_2, \gamma > 0$ and $\alpha \in \mathbb{R}$ such that, for all $\sigma_j \in \hat{\mathbb{G}}$ with $k_{\sigma} > 0$,

$$ (i) \quad \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} \le c_1 k_{\sigma}^{\alpha} \exp(\gamma \cdot k_{\sigma}^{\beta}), $$

$$ (ii) \quad \left\| \phi_{\mathbf{U}_j} (\sigma_j^M) \right\|_{\text{op}} \le c_2 k_{\sigma}^{-\alpha} \exp(-\gamma \cdot k_{\sigma}^{\beta}); $$

(S3) Non-Euclidean log-supersmoothness class of order $\beta > 0$: There exist constants $c_1, c_2, \gamma > 0$ and $\alpha, \xi_1, \xi_2 \in \mathbb{R}$ such that, for all $\sigma_j \in \hat{\mathbb{G}}$ with $k_{\sigma} > 0$,

$$ (i) \quad \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} \le c_1 k_{\sigma}^{\alpha} \exp(\gamma \cdot k_{\sigma}^{\beta} (\log k_{\sigma} - \xi_1)), $$

$$ (ii) \quad \left\| \phi_{\mathbf{U}_j} (\sigma_j^M) \right\|_{\text{op}} \le c_2 k_{\sigma}^{-\alpha} \exp(-\gamma \cdot k_{\sigma}^{\beta} (\log k_{\sigma} - \xi_2)). $$

We provide some examples of measurement error distributions within the above scenarios. As a trivial case, consider the case of no measurement error where $\mathbb{P}(\mathbf{U}_j = e_{\mathbb{G}}) = 1$ with $e_{\mathbb{G}}$ being the identity element of $\mathbb{G}$. We let $I_{d_{\sigma_j}}$ denote the $d_{\sigma_j} \times d_{\sigma_j}$ identity matrix. In that case, $\phi_{\mathbf{U}_j} (\sigma_j^M) = \sigma_j^M (e_{\mathbb{G}}^{-1}) = I_{d_{\sigma_j}}$ and hence it satisfies (S1) with $\beta = 0$. Another example of semi-ordinary-smooth distribution is the general Laplace distribution on $\mathbb{G}$ whose Fourier transform is of the form $\phi_{\mathbf{U}_j} (\sigma_j^M) = c (1 + \lambda \cdot k_{\sigma_j})^{-1} I_{d_{\sigma_j}}$, where $c \in \mathbb{C}$ is a nonzero constant and $\lambda > 0$ is a parameter. It satisfies (S1) with $\beta = 1$. An important example of semi-super-smooth distribution is the general Gaussian distribution on $\mathbb{G}$ whose Fourier transform is of the form $\phi_{\mathbf{U}_j} (\sigma_j^M) = c \exp(-\lambda \cdot k_{\sigma_j}) I_{d_{\sigma_j}}$. It satisfies (S2) with $\beta = 1, \gamma = \lambda$ and $\alpha = 0$. Examples of distributions satisfying (S3) include the von Mises distribution on $\mathbb{T}^1$ and von Mises-Fisher distribution on SO(3). For more details, we refer to [97,102,114,119].

For the next lemma, depending on the smoothness scenarios for the distribution of $\mathbf{U}_j$, we define, for $s > 0$

$$ A_n(s) = \begin{cases} T_n^{sk\beta + \frac{k\dim(\mathbb{G})}{2}} & \text{for (S1)-(i),} \\ T_n^{sk\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp(s \cdot k \cdot \gamma \cdot T_n^{\beta}) & \text{for (S2)-(i),} \\ T_n^{sk\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp(s \cdot k \cdot \gamma \cdot T_n^{\beta} (\log T_n - \xi_1)) & \text{for (S3)-(i).} \end{cases} $$

**Lemma 4.1.** For each $s > 0$ and smoothness scenario for the distribution of $\mathbf{U}_j$, it holds that

$$ \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}} : k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}}^{s} = O(A_n(s)). $$

**Proof.** First, we examine the situation of ordinary smoothness as described in (S1)-(i). We have
$$
\begin{align}
\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}}^S &\le 1 + c_1^k \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; 0 < k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 k_{\sigma_j}^{S\beta} \nonumber \\
&\le 1 + c_1^k T_n^{Sk\beta} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; 0 < k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 \nonumber \\
&\le C \cdot T_n^{Sk\beta + \frac{k \dim(\mathbb{G})}{2}}, \tag{4.1}
\end{align}
$$

where the first inequality follows from the fact that $k_{\sigma_j} = 0$ if and only if $\sigma_j$ is the trivial representation such that $\mathbb{H}_{\sigma_j} = \mathbb{C}$ and $\sigma_j(g)(h) = 1$ for all $g \in \mathbb{G}$ and $h \in \mathbb{C}$ (page 45 in [3]), which give $d_{\sigma_j} = 1$ and $\left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} = 1$. The last inequality follows from the corollary in [109]. For the supersmoothness scenario (S2)-(i), we obtain that

$$
\begin{align}
& \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}}^S \nonumber \\
& \le 1 + c_1^k \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; 0 < k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 k_{\sigma_j}^{S\beta} \exp(s \cdot \gamma \cdot k_{\sigma_j}^{\beta}) \nonumber \\
& \le 1 + c_1^k T_n^{Sk\alpha} \exp \left( s \cdot k \cdot \gamma \cdot T_n^{\beta} \right) \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; 0 < k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 \nonumber \\
& \le C \cdot T_n^{Sk\alpha + \frac{k \dim(\mathbb{G})}{2}} \exp \left( s \cdot k \cdot \gamma \cdot T_n^{\beta} \right). \tag{4.2}
\end{align}
$$

Similarly, we get the desired result for the log-supersmoothness scenario (S3)-(i). □

## 4.2. Rates of convergence

Firstly, we begin by providing the uniform consistency of the density product estimator, which has not been investigated even for the complex-valued version of $\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})$. In the particular case where $k=1$, it is noteworthy that [86] only achieved the $L^2$ rates of convergence for the complex-valued of $\hat{f}_n^{(1)}(\mathbf{t})$. The uniform consistency of $\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})$ is necessary for the main results concerning our conditional U-statistics $\hat{T}_n^{(k)}(\varphi, T_n, \cdot)$ and is also significant in its own right. For each smoothness scenario, we consider $T_n$ increasing to infinity as $n \to \infty$ at the following specified speed:

(T1) Ordinary-smoothness: $n^{-k^2/2} T_n^{k\beta+k \dim(\mathbb{G})/2} = o(1)$.

(T2) Supersmoothness: $n^{-k^2/2} T_n^{k\alpha+k \dim(\mathbb{G})/2} \exp(k \cdot \gamma \cdot T_n^\beta) = o(1)$.

(T3) Log-supersmoothness: $n^{-k^2/2} T_n^{k\alpha+k \dim(\mathbb{G})/2} \exp(k \cdot \gamma \cdot T_n^\beta (\log T_n - \xi_1)) = o(1)$.

**Proposition 4.2.** Assume that the series in (3.5) converges uniformly to $\tilde{f}_{\mathbb{X}}(\cdot)$. Then, under either of the conditions (S1)-(i)+(T1), (S2)-(i)+(T2) and (S3)-(i)+(T3), it holds that

$$
\sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \left| \tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) \right| = o_P(1). \qquad (4.3)
$$
A sufficient condition for $\tilde{f}_{\mathcal{X}}(\cdot)$ to satisfy the assumption in Proposition 4.2 is that $\tilde{f}_{\mathcal{X}}(\cdot)$ is $2 \times [\dim(\mathbb{G})/4]$-times continuously differentiable on $\mathbb{G}^k$; see, for instance, Theorem 3.3.1 in [3] for the case where $k=1$. The required conditions are much weaker for some groups $\mathbb{G}$. For example, if $\mathbb{G}=\mathbb{T}^1$, the uniform convergence of the series in (3.5) holds if $\tilde{f}_{\mathcal{X}}(\cdot)$ is Hölder continuous with a positive exponent. In the case where $\mathbb{G}=SU(2)$, uniform convergence is ensured by the Lipschitz continuity of $\tilde{f}_{\mathcal{X}}(\cdot)$. For more details, refer to [112]. The forthcoming theorem is the main achievement of this section, which specifies the rate of convergence of $\hat{r}_n^{(k)}(\varphi, T_n, \cdot)$ for each smoothness class. It is important to note that the error rate of this estimator depends on the level of smoothness of both $\tilde{f}_{\mathcal{X}}(\cdot)$ and $r^{(k)}(\varphi, \cdot)$.

**Theorem 4.3.** Assume that the conditions (A1) and (A2) hold. Then, the following results are valid:

1. Under (S1)(i), (T1) and (F.i), it holds that

$$ \int_{\mathbb{G}^k} |\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) = O_P \left( T_n^{-2k\hat{m}} + n^{-k^2} T_n^{2k\beta + \frac{k \dim(\mathbb{G})}{2}} \right). $$

Moreover, if instead of (F.i), and assume that (F.ii) is satisfied, we get

$$ \int_{\mathbb{G}^k} |\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) = O_P \left( T_n^{-2k\hat{m}} + n^{-k^2} T_n^{2k\beta + \frac{k \dim(\mathbb{G})}{2}} \right). $$

2. Under (S2)(i), (T2) and (F.i), it holds that

$$ \int_{\mathbb{G}^k} |\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) \\ = O_P \left( T_n^{-2k\hat{m}} + n^{-k^2} T_n^{2k\alpha + \frac{k \dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^{\beta} \right) \right). $$

Moreover, if instead of (F.i), and assume that (F.ii) is satisfied, we obtain

$$ \int_{\mathbb{G}^k} |\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) \\ = O_P \left( T_n^{-2k\hat{m}} + n^{-k^2} T_n^{2k\alpha + \frac{k \dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^{\beta} \right) \right). $$

3. Under (S3)(i), (T3), and (F.i), it holds that

$$ \int_{\mathbb{G}^k} |\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) \\ = O_P \left( T_n^{-2k\hat{m}} + n^{-k^2} T_n^{2k\alpha + \frac{k \dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^{\beta} (\log T_n - \xi_1) \right) \right). $$

Moreover, if instead of (F.i), and assume that (F.ii) is satisfied, we have

$$ \int_{\mathbb{G}^k} |\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) \\ = O_P \left( T_n^{-2k\hat{m}} + n^{-k^2} T_n^{2k\alpha + \frac{k \dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^{\beta} (\log T_n - \xi_1) \right) \right). $$
**Remark 4.4.** We may show that $\int_{\mathbb{G}^k} |\tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k)$ attains the same rates specified in Theorem 4.3 under assumption (A1)(i). Furthermore, Proposition 4.2 and Theorem 4.3 remain valid for the complex-valued versions of $\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})$ and $\tilde{\tau}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}})$, respectively, under their corresponding conditions.

We can optimize the $L^2$ error rate for each smoothness class in Theorem 4.3 by choosing an appropriate speed for $T_n \to \infty$. We consider the following speeds:

(T1') Ordinary-smoothness: $T_n \asymp n^{2/(4\hat{m}+4\beta+\dim(\mathbb{G}))}$;

(T2') Supersmoothness: $T_n = C \cdot (\log n)^{1/\beta}$ for $0 < C < (2\gamma)^{-1/\beta}$;

(T3') Log-supersmoothness: $T_n = C \cdot (\log n / \log \log n)^{1/\beta}$ for $0 < C < (2\gamma/\beta)^{-1/\beta}$.

The speed of $T_n$ in the ordinary-smoothness case actually balances the two terms of the orders $T_n^{-2k\hat{m}}$ and $T_n^{2k\beta + \frac{k\dim(\mathbb{G})}{2}}$, so that it optimizes the $L^2$ error rate. In the cases of supersmoothness and log-supersmoothness, however, no such thing exists that makes the corresponding two terms be of the same order. This is because $T_n$ also appears in the exponents of $\exp(2 \cdot k \cdot \gamma \cdot T_n^\beta)$ and $\exp(2 \cdot k \cdot \gamma \cdot T_n^\beta (\log T_n - \xi_1))$, respectively. The choices of $T_n$ given in (T2') and (T3') have specific constant factors $C$ with constraints. The upper bounds of $C$, in conditions (T2') and (T3'), are actually the thresholds, beyond which $n^{-k^2} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp(2 \cdot k \cdot \gamma \cdot T_n^\beta)$ and $n^{-k^2} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp(2 \cdot k \cdot \gamma \cdot T_n^\beta (\log T_n - \xi_1))$, respectively, diverge to infinity, while they are dominated by $T_n^{-2kr}$ for $C$ smaller than the thresholds. Thus, the choices of $T_n$ are optimal in the sense that they minimize the $L^2$ error rates in the respective scenarios. We note that, in the Euclidean case, similar constraints are imposed on the constant factors of the bandwidths that take the role of $T_n$ here.

**Corollary 4.5.** Assume that the conditions (A1) and (A2) hold. Then the following results are valid:

1. Under (S1)(i), (T1') and (F.i) or (F.ii), it holds that

$$ \int_{\mathbb{G}^k} |\hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) = O_P \left( n^{-4k\hat{m}/(4\hat{m}+4\beta+\dim(\mathbb{G}))} \right). $$

2. Under (S2)(i), (T2') and (F.i) or (F.ii), it holds that

$$ \int_{\mathbb{G}^k} |\hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) = O_P \left( (\log n)^{-2k\hat{m}/\beta} \right). $$

3. Under (S3)(i), (T3'), and (F.i) or (F.ii), it holds that

$$ \int_{\mathbb{G}^k} |\hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) = O_P \left( (\log n / \log \log n)^{-2k\hat{m}/\beta} \right). $$

### 4.3. Rates of convergence for mixed data

Now, we let $\mathcal{X}' = (\mathcal{X}'_1, \dots, \mathcal{X}'_k)$ be a random vector taking values in $\mathbb{R}^{dk}$. We note that $\mathbb{R}^{dk}$ is a $dk$-dimensional locally compact and connected Lie group that is not compact. Denote $\tilde{f}_{\mathbb{X}}(\cdot)$ as the density of $\mathcal{X}' = (\mathcal{X}'_1, \mathcal{X})$, where $\mathcal{X} \in \mathbb{G}^k$, with respect to the product measure $\text{Leb}_d \otimes \mu$, where $\text{Leb}_d$ is the Lebesgue measure on $\mathbb{R}^d$ and $\mu$ is the normalized Haar measure on $\mathbb{G}^k$. We suppose that $\mathcal{X}'$ is not observable but only $\mathcal{Z} = (\mathcal{Z}'_1, \mathcal{Z}) = (\mathcal{U}' + \mathcal{X}', \mathcal{U} \circ \mathcal{X}) \in \mathbb{R}^{dk} \times \mathbb{G}^k$ is observable, where
$\overline{\mathbb{U}} = (\mathbb{U}', \mathbb{U}) \in \mathbb{R}^{dk} \times \mathbb{G}^k$ is an unobservable measurement error. We will assume that $\overline{\mathbb{X}} \perp \overline{\mathbb{U}}$ and $\mathbb{U}' \perp \mathbb{U}$. We also assume that the density $\tilde{f}_{\mathbb{U}'}(\cdot)$ of $\mathbb{U}'$ with respect to $\mathbb{U}$ is known and the density $\tilde{f}_{\mathbb{U}}(\cdot)$ of $\mathbb{U}$ with respect to $\mu$ is known. Here, our main focus is on estimating the following regression function

$$ \bar{r}^{(k)}(\varphi, \tilde{\mathbf{t}}) = \mathbb{E} \left( \varphi \left( \mathbf{Y}_1, \dots, \mathbf{Y}_k \right) \mid \left( \bar{\mathbf{Z}}_1, \dots, \bar{\mathbf{Z}}_k \right) = \tilde{\mathbf{t}} \right), \quad \text{for } \tilde{\mathbf{t}} \in \mathbb{R}^{dk} \times \mathbb{G}^k. \qquad (4.4) $$

The literature on Euclidean measurement error problems is prevalently based on the assumption of a known Euclidean measurement error distribution, as evidenced by references such as [9,48,47,131]. Likewise, studies focusing on non-Euclidean measurement error problems commonly assume known non-Euclidean measurement error distributions, as seen in references like [43,64,70,102]. Our focus is on scenarios where the measurement error distributions are known, and we aim to establish a strong theoretical framework for this new problem. If $\tilde{f}_{\mathbb{U}}(\cdot)$ or $\tilde{f}_{\mathbb{U}',}(\cdot)$ are unknown, they can be estimated from repeated measurements, as discussed in Sections 3 and 4 and also in [40,81], particularly for $k=1$. This estimation involves using a random sample of the measurement error distributions as detailed in Sections 3 and 4 and also discussed in [78,77,37,81], specifically for $k=1$.

For an arbitrary random element $\mathcal{L}$, we let $\tilde{f}_{\mathcal{L}}(\cdot)$ denote the density of $\mathcal{L}$ and $\phi^{\mathcal{L}}(\sigma^M)$ denote the Fourier transform of $\tilde{f}_{\mathcal{L}}(\cdot)$ at $\sigma^M$. When $\mathcal{L}$ is an Euclidean variable, we write $\phi(\tilde{\mathbf{w}})$ instead of $\phi^{\mathcal{L}}(\sigma^M)$. We note that

$$ \phi^{\mathcal{L}}(\tilde{\mathbf{w}}) = \prod_{j=1}^{k} \int_{\mathbb{R}^{dk}} \exp(t \langle \mathbf{r}_j, \mathbf{w}_j \rangle) f_{\mathcal{L}}(\mathbf{r}_j) d\mathbf{r}_j, $$

when $\mathcal{L}$ is $dk$-dimensional. Let $\mathbf{X}_j, \mathbf{U}'_j$ and $\mathbf{Z}'_j$ denote the respective jth element of $\mathbb{X}', \mathbb{U}'$ and $\mathbb{Z}',$ for $1 \le j \le d$. Suppose that we observe $n$ i.i.d. observations $\{\tilde{\mathbf{Z}}_i = (\mathbf{Z}'_i, \mathbf{Z}_i) = (\mathbf{Z}'_1, \dots, \mathbf{Z}'_d, \mathbf{Z}_i) : 1 \le i \le n\}$. So, we define the product kernel estimators as follows, for $\tilde{\mathbf{t}} = (\tilde{\mathbf{t}}', \tilde{\mathbf{t}}) \in \mathbb{R}^{dk} \times \mathbb{G}^k$,

$$ \hat{f}_n^{(k)}(T_n, \tilde{\mathbf{t}}) = \sum_{(i_1, \dots, i_k) \in I_n^k} \mathrm{Re} \left( \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k}) \right) K_h^D \left( \frac{\mathbf{t}'_1 - \mathbf{Z}'_{i_1}}{h} \right) \cdots K_h^D \left( \frac{\mathbf{t}'_k - \mathbf{Z}'_{i_k}}{h} \right), \quad (4.5) $$

and we introduce our estimator of the regression function $r^{(k)}(\varphi, \cdot)$ as,

$$ \tilde{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \frac{\sum_{(i_1, \dots, i_k) \in I_n^k} \varphi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \mathrm{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})) K_h^D \left( \frac{\mathbf{t}'_1 - \mathbf{Z}'_{i_1}}{h} \right) \cdots K_h^D \left( \frac{\mathbf{t}'_k - \mathbf{Z}'_{i_k}}{h} \right)}{\sum_{(i_1, \dots, i_k) \in I_n^k} \mathrm{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})) K_h^D \left( \frac{\mathbf{t}'_1 - \mathbf{Z}'_{i_1}}{h} \right) \cdots K_h^D \left( \frac{\mathbf{t}'_k - \mathbf{Z}'_{i_k}}{h} \right)}, \quad (4.6) $$

where

$$ K_h^D(\mathbf{u}) = (2\pi)^{-d} \int_{\mathbb{R}^d} e^{-i\langle \mathbf{s}, \mathbf{u} \rangle} \cdot \phi^{K_h^D}(\mathbf{s}) \cdot (\phi^{\mathcal{U}}(\mathbf{u}/h))^{-1} d\mathbf{s}, $$

and $\phi^{K_h^D}(\cdot)$ is the Fourier transform of $K_h^D(\cdot)$. We remark that $K_h^D(\cdot)$ is well-defined if the following condition is satisfied:

(K1) (i) $\phi^{\mathcal{U}}(\mathbf{v}) \neq 0$ for all $\mathbf{v} \in \mathbb{R}^d$; (ii) $\phi^{K_h^D}(\cdot)/\phi^{\mathbb{U}}(\cdot/h)$ is integrable.
Let

$$
\bar{K}_{T_n} (\tilde{\mathbf{t}}, \tilde{\mathbf{z}}) := \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{z}_1, \dots, \mathbf{z}_k) \prod_{j=1}^{k} K_h^D \left( \frac{\mathbf{t}'_j - \mathbf{z}'_j}{h} \right), \quad \tilde{\mathbf{t}} = (\tilde{\mathbf{t}}', \tilde{\mathbf{t}}) \in \mathbb{R}^{dk} \times \mathbb{G}^k,
$$

denote the product kernel. Consider the U-kernel

$$
\bar{G}_{\varphi, T_n, \tilde{\mathbf{t}}} (\tilde{\mathbf{z}}, \tilde{\mathbf{y}}) := \varphi(\tilde{\mathbf{y}}) \bar{K}_{T_n} (\tilde{\mathbf{t}}, \tilde{\mathbf{z}}), \quad \tilde{\mathbf{z}} = (\tilde{\mathbf{z}}', \tilde{\mathbf{z}}) \in \mathbb{R}^{dk} \times \mathbb{G}^k, \quad \tilde{\mathbf{y}} \in \mathbb{R}^{dk}.
$$

For the sample $(\bar{\mathbf{z}}_1, \mathbf{Y}_1), \dots, (\bar{\mathbf{z}}_n, \mathbf{Y}_n)$, define

$$
\begin{align}
\bar{U}_n (\varphi, T_n, \tilde{\mathbf{t}}) &:= \bar{U}_n^{(k)} (\bar{G}_{\varphi, T_n, \tilde{\mathbf{t}}}) = \frac{(n-k)!}{n! h^{dk}} \sum_{\mathbf{i} \in I_n^k} \bar{G}_{\varphi, T_n, \tilde{\mathbf{t}}} (\bar{Z}_{\mathbf{i}}, \mathbf{Y}_{\mathbf{i}}) \nonumber \\
&= \frac{(n-k)!}{n! h^{dk}} \sum_{\mathbf{i} \in I_n^k} \varphi (\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k}) \prod_{j=1}^k K_h^D \left( \frac{\mathbf{t}'_j - \mathbf{Z}'_{i_j}}{h} \right). \tag{4.7}
\end{align}
$$

**Remark 4.6.** We can define another estimator of the regression function $\tilde{r}^{(k)}(\varphi, \cdot)$ as follows

$$
\tilde{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \frac{\sum_{(i_1, \dots, i_k) \in I_n^k} \varphi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \operatorname{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})) \tilde{K}_h^D \left(\frac{\mathbf{t}'_1 - \mathbf{Z}'_{i_1}}{h}\right) \dots \tilde{K}_h^D \left(\frac{\mathbf{t}'_k - \mathbf{Z}'_{i_k}}{h}\right)}{\sum_{(i_1, \dots, i_k) \in I_n^k} \operatorname{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})) \tilde{K}_h^D \left(\frac{\mathbf{t}'_1 - \mathbf{Z}'_{i_1}}{h}\right) \dots \tilde{K}_h^D \left(\frac{\mathbf{t}'_k - \mathbf{Z}'_{i_k}}{h}\right)},
$$

where $\tilde{K}_h^D(\tilde{\mathbf{s}}) = \prod_{\ell=1}^d K_{\ell,h}^D(\mathbf{s}_\ell)$ and

$$
K_{\ell,h}^{D}(\mathbf{s}_{\ell}) = \frac{1}{2\pi} \int_{\mathbb{R}} e^{-i\mathbf{u}_{\ell}\cdot\mathbf{s}_{\ell}} \frac{\phi^{K_{\ell}}(\mathbf{u}_{j})}{\phi^{U'_{\ell}}(\mathbf{u}_{\ell}/h)} d\mathbf{u}_{\ell}. \quad (4.8)
$$

This kernel is considered by [105] and [107]. Here, $\phi^{K_\ell}$ are the characteristic functions of univariate
kernels $K_\ell: \mathbb{R} \rightarrow \mathbb{R}$. The estimator $\tilde{r}_n^{(k)}(\varphi, T_n, \cdot)$ is a special case of $\tilde{r}_n^{(k)}(\varphi, T_n, \cdot)$, obtained by using
the product kernel $K(\tilde{\mathbf{u}}) = \prod_{\ell=1}^d K_\ell(\mathbf{u}_\ell)$ and assuming that $(\mathbf{U}'_\ell: 1 \le \ell \le d)$ are mutually independent.
While we primarily discuss kernel conditions in the context of product kernels, our arguments will
use a general integrable kernel $K$. We will not assume the mutual independence of $(\mathbf{U}'_\ell: 1 \le \ell \le d)$,
enabling us to discover more general conditions for each result in this multivariate setting.

Before diving into the presentation of our results, it is essential to introduce some fundamental assumptions. We begin by stating similar smoothness conditions for the distribution of $\mathbf{U}'_j$ as those for $\mathbf{U}_j$ in Section 4.1, as follows

(S'1) Euclidean semi-ordinary-smooth scenario of order $\beta^* \ge 0$: There exist constants $c_1^* > 0$ such that,

$$
|\phi^{\mathbf{U}'_j}(\mathbf{u})^{-1}| \le c_1^* (1 + \|\mathbf{u}\|)^{\beta^*}, \quad \text{for all } \mathbf{u} \in \mathbb{R}^d.
$$

(S'2) Euclidean semi-super-smooth scenario of order $\beta^* > 0$: There exist constants $c_1^*, \gamma > 0$ such that,

$$
|\phi^{\mathbf{U}'_j}(\mathbf{u})^{-1}| \le c_1^* \exp(\gamma' \cdot \|\mathbf{u}\|)^{\beta^*}, \quad \text{for all } \mathbf{u} \in \mathbb{R}^d.
$$
We note that the above scenarios imply the condition (K1)-(i). An important example of semi-
ordinary-smooth distribution is the distribution satisfying $\mathbb{P}(\mathbf{U}'_j = 0_d) = 1$ where $0_d$ is the zero
vector in $\mathbb{R}^d$. In the case, $\phi^{\mathbf{U}_j} \equiv 1$ which satisfies (S'1) with $\beta^* = 0$. This indicates that (S'1) cov-
ers the error-free case. Another example is the symmetric multivariate Laplace distribution with
positive definite covariance matrix $\Sigma$ as discussed in [89]. Its characteristic function at $\mathbf{w}$ is given
by $(1 + \mathbf{w}^\top \Sigma \mathbf{w})^{-1}$ which satisfies (S'1) with $\beta^* = 2$. A more general example is the multivariate
Linnik distribution whose characteristic function at $\mathbf{w}$ is given by $(1 + (\sum_{l=1}^L \mathbf{w}^\top \Sigma_l \mathbf{w})^\eta)^{-1}$, where
$L \in \mathbb{N}, 0 < \eta \le 1$ and $\Sigma_l$ are $d \times d$ positive definite matrices with no two of them being propor-
tional as detailed in [2]. It satisfies (S'1) with $\beta^* = 2\eta$. Another example of semi-ordinary-smooth
distribution is the bivariate gamma distribution whose characteristic function at $\mathbf{w} = (\mathbf{w}_1, \mathbf{w}_2) \in \mathbb{R}^2$
is given by $(\det(I_2 - \iota \Sigma \operatorname{diag}(\mathbf{w}_1, \mathbf{w}_2)))^{-\eta}$, where $I_2$ is the $2 \times 2$ identity matrix, $\Sigma$ is the correla-
tion matrix of $(\mathbf{U}'_1, \mathbf{U}'_2)$ and $\eta > 0$ as discussed in [92]. It satisfies (S'1) with $\beta^* = 2\eta$. An example
of semi-super-smooth distribution is the multivariate normal distribution with mean zero and posi-
tive definite covariance matrix $\Sigma$ whose characteristic function at $\mathbf{w}$ is given by $\exp(-\mathbf{w}^\top \Sigma \mathbf{w}/2)$. It
satisfies (S'2) with $\beta^* = 2$ and $\gamma = \max_{1 \le j \le d} \lambda_j/2$, where $\lambda_j > 0$ are the eigenvalues of $\Sigma$. Another
example is the symmetric multivariate Cauchy distribution with positive definite scale matrix $\Sigma$ as
in [87]. Its characteristic function at $\mathbf{w}$ is given by $\exp(-(\mathbf{w}^\top \Sigma \mathbf{w})^{1/2})$. Hence, it satisfies (S'2) with
$\beta^* = 1$ and $\gamma = \max_{1 \le j \le d} \sqrt{\lambda_j}$, where $\lambda_j > 0$ are the eigenvalues of $\Sigma$ as detailed in [88]. Before
presenting the results, we state the conditions for the pointwise rate at $\tilde{\mathbf{t}} = (\tilde{\mathbf{t}}', \tilde{\mathbf{t}}) \in \mathbb{R}^{dk} \times \hat{\mathbb{G}}^k$.

**Assumptions 4.7.**

(A'1) $\tilde{f}_{\bar{\mathbb{X}}} (\cdot)$ is bounded on $\mathbb{R}^{dk} \times \mathbb{G}^k$;

(A'2) For some $m \in \mathbb{N}$,

(i) $\tilde{f}_{\bar{\mathbb{X}}}(\cdot, \mathbf{g}) : \mathbb{R}^d \to \mathbb{R}$ are $m$-times differentiable for all $\mathbf{g} \in \mathbb{G}^k$;

(ii) $\int_{\mathbb{G}^k} \left( \sup_{\tilde{\mathbf{w}} \in \mathbb{R}^{dk}} \left| \frac{\partial^{m_j} \tilde{f}_{\bar{\mathbb{X}}}}{\partial \mathbf{t}_{1_1} \dots \partial \mathbf{t}_{1_m}} (\tilde{\mathbf{w}}, \mathbf{u}_j) \frac{\partial^{\tilde{m}_j} r^{(k)}}{\partial \mathbf{t}_{1_1} \dots \partial \mathbf{t}_{\tilde{m}_j}} (\varphi, (\tilde{\mathbf{w}}, \tilde{\mathbf{u}})) \right|^2 d\mu(\mathbf{u}_j) < \infty$.

(A'3) For some $\hat{m} \in \mathbb{N}$ with $r > \dim(\mathbb{G})/4$,

(i) $\tilde{f}_{\bar{\mathbb{X}}}(\tilde{\mathbf{t}}', \cdot) \in C^{2\hat{m}}(\mathbb{G}^k)$ and $\tilde{r}^{(k)}(\varphi, \cdot) \in C^{2\hat{m}}(\mathbb{G}^k)$;

(ii) the Fourier series of $\Delta^{\hat{m}}(\tilde{f}_{\bar{\mathbb{X}}}(\tilde{\mathbf{t}}', \cdot)) : \mathbb{G}^k \to \mathbb{R}$ is absolutely convergent;

(ii*) The Fourier series of $\Delta^{\hat{m}}(\tilde{f}_{\bar{\mathbb{X}}}(\tilde{\mathbf{t}}', \cdot)) : \mathbb{G}^k \to \mathbb{R}$ is uniformly absolutely-convergent.

(iii*) The Fourier series of $\Delta^{\hat{m}}(\tilde{f}_{\bar{\mathbb{X}}}(\cdot, \tilde{\mathbf{r}}^{(k)}(\varphi, (\tilde{\mathbf{t}}', \cdot))) : \mathbb{G}^k \to \mathbb{R}$ is uniformly absolutely-convergent.

(A'4) (i) $K_h^D(\cdot)$ is continuous, $\int_{\mathbb{R}^d} K_h^D(\mathbf{u})d\mathbf{u} = 1$.

(ii) $\phi_h^{K_h^D}(\cdot) = 0$ on $\mathbb{R}^d \setminus B(0_d, a)$ for some $a > 0$.

(iii) $K_h^D(\cdot)$ is of order $m$, that is,

$$
\int_{\mathbb{R}^{dk}} \left| \sum_{\ell=0}^{m-1} \binom{m-1}{\ell} \sum_{m_1+\cdots+(m-1)_k=\ell} \tilde{m}_{1+\cdots+\tilde{m}_k} = (m-1-\ell) \binom{\ell}{m-1} \binom{m-1-\ell}{\tilde{m}_{1:k}} \right. \\
\left. \times \prod_{j=1}^k \sum_{i_1, \ldots, i_{m_j}=1} \tilde{t}_{i_1} \cdots \tilde{t}_{i_{m_j}} = 1 \quad K_h^D(\mathbf{w}_j) \, d\mathbf{w}_j = 0, \right| \quad (4.9)
$$

and
$$ \int_{\mathbb{R}^{dk}} \left| \sum_{\ell=0}^{m} \binom{m}{\ell} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=m-\ell} \binom{\ell}{m_{1:k}} \binom{m-\ell}{\tilde{m}_{1:k}} \times \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1} \sum_{\tilde{t}_{1}, \dots, \tilde{t}_{\tilde{m}_j}=1} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{t}_{1}} \dots \mathbf{w}_{\tilde{t}_{\tilde{m}_j}} \right| K_h^D(\mathbf{w}_j) \, d\mathbf{w}_j \le \infty. \quad (4.10) $$

**Assumptions 4.8.** Suppose that the regression function satisfies for $\varphi(\cdot)$ and $1 \le k \le n$:

$$ \exists \mathfrak{R} > 0, \forall \tilde{\mathbf{t}}_1, \tilde{\mathbf{t}}_2 \in \mathbb{R}^{dk}: \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} |\tilde{r}^{(k)}(\varphi, (\tilde{\mathbf{t}}_1, \tilde{\mathbf{t}}_2)) - \tilde{r}^{(k)}(\varphi, (\tilde{\mathbf{t}}_2, \tilde{\mathbf{t}}_1))| \le \|\tilde{\mathbf{t}}_1 - \tilde{\mathbf{t}}_2\|^2_{\mathfrak{R}}, \quad (4.11) $$

where for $1 \le i \le 2$: $\tilde{\mathbf{t}}_i := (\mathbf{t}'_{i_1}, \dots, \mathbf{t}'_{i_k})$.

Condition (A'1) is a standard assumption in nonparametric statistics. Conditions (A'2) and (A'3) introduce smoothness conditions on $\tilde{f}_{\mathbb{X}}(\cdot)$ and $\tilde{f}_{\mathbb{X}}(\cdot)^{r^{(k)}}(\varphi, \cdot)$. Condition (A'2) extends the typical smoothness conditions in $\mathbb{R}$ to our broader data domain. Specifically, (A'2)-(ii) is satisfied if $\sup_{\tilde{w} \in \mathbb{R}^{dk}} \left| \frac{\partial^{m_j} \tilde{f}_{\mathbb{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{m_j}}} (\tilde{w}, \cdot) \frac{\partial^{\tilde{m}_j} \tilde{f}_{\mathbb{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{w}, \cdot)) \right|$ is bounded on $\mathbb{G}^k$ since $\mu(\mathbb{G}) < \infty$. Condition (A'3)-(i) ensures the Fourier series of $\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}', \cdot)$ and $\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}', \cdot)^{\tilde{r}^{(k)}}(\varphi, (\tilde{\mathbf{t}}', \cdot))$ are uniformly absolutely convergent. Likewise, (A'3)-(ii) and (A'3)-(ii*) hold if $\Delta^{\tilde{m}}(\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}', \cdot))$ and $\Delta^{\tilde{m}}(\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}', \cdot)^{\tilde{r}^{(k)}}(\varphi, (\tilde{\mathbf{t}}', \cdot)))$ are $2\tilde{m}^*$-times continuously differentiable for some $\tilde{m}^* \in \mathbb{N}$ with $\tilde{m}^* > \dim(\mathbb{G})/4$, implying the uniform absolute convergence of their Fourier series. However, (A'3)-(ii) holds under much weaker conditions for various $\mathbb{G}^k$. For example, if $\mathbb{G}^k = \mathbb{T}^k$ and $\Delta^{\tilde{m}}(\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}', \cdot))$ is Hölder continuous with exponent greater than $1/2$, then condition (A'3)-(ii) is satisfied (e.g., [82]). Some other weak sufficient conditions can be found in Theorem 3 of [139] for $\mathbb{G} = \mathbb{T}^q$ with $q > 1$ and in [112] for $\mathbb{G} = \mathbb{SU}(2)$. Further, parametric distributions typically satisfy the smoothness conditions. For instance, parametric distributions on $\mathbb{R}^d \times \mathbb{T}^q$, as explored in [79], and those defined over $\mathbb{R}^d \times \mathbb{SO}(3)$, as discussed in [140], satisfy them. Additionally, the class of possibly nonparametric distributions on the cylinder $\mathbb{R} \times \mathbb{T}^1$ described in Theorem 5 of [79] may also fulfill these conditions. Condition (A'3)-(ii*) is slightly stronger than Condition (A'3)-(ii). As we noted, a sufficient condition for (A'3)-(ii*) is that $\Delta^{\tilde{m}}(\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}', \cdot))$ is $2r^*$-times continuously differentiable on $\mathbb{G}^k$, for some $r^* \in \mathbb{N}$ with $r^* > \dim(\mathbb{G})/4$. Condition (A'4) is a multivariate version of the standard kernel condition in univariate deconvolution kernel estimation. Condition (A'4)-(i) is classical in the non-parametric kernel estimations. Under conditions (K1)-(i) and (A'4)-(ii), the condition (K1)-(ii) is automatically satisfied since $\phi_{h}^{K_h^D}(\cdot)/\phi_{h}^{U_h}(\cdot/h)$ is continuous. Suppose that we take the product kernel $K_h^D(\tilde{w}) = \prod_{j=1}^d K_{h,j}^D(\mathbf{w}_j)$ with continuous univariate kernels $K_{h,j}^D: \mathbb{R} \to \mathbb{R}$ satisfying $\phi_{h,j}^{K_h^D}(\cdot) = 0$ on $\mathbb{R}\setminus[-1, 1]$, $\int_{\mathbb{R}} K_{h,j}^D(\mathbf{w}_j) d\mathbf{w}_j = 1$, $\int_{\mathbb{R}} |K_{h,j}^D(\mathbf{w}_j)| |\mathbf{w}_j|^m d\mathbf{w}_j < \infty$ and $\int_{\mathbb{R}} K_{h,j}^D(\mathbf{w})\mathbf{w}_j^{\alpha_j} d\mathbf{w}_j = 0$ for all $1 \le \alpha_j \le m-1$. Then, one can easily check that $K_h^D(\cdot)$ satisfies (A'4)-(ii) with $a = \sqrt{d}$ and (A'4)-(iii). Finally, imposing $\tilde{r}^{(k)}(\varphi, \cdot)$ to be Lipschitz, as stated in Assumption 4.8, is a classical condition that enables us to determine the exact rate of convergence. We now state the pointwise rate for $\tilde{r}_n^{(k)}(\varphi, T_n, \cdot)$.

**Theorem 4.9.** Assume that Conditions (K1)-(i), (K2), (A'1)-(A'4) and (F.i) hold. Then, it holds that

$$ \begin{aligned} & \tilde{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}}) \\ & = O_P \left( h^m T_n^{k \dim(\mathbb{G})/4} + T_n^{-k \tilde{m}} + n^{-k^2/2} h^{-dk/2} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j: \|\mathbf{w}_j\| \le ah^{-1}} \left| \phi_{\mathbf{U}_j}^{\mathbf{U}_j}(\mathbf{w}_j)^{-1} \right| \right) \right) \end{aligned} $$
$$ \times \left( \sum_{\sigma_1, \dots, \sigma_k \in \mathbb{G}: k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j}^2 \| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \|_{\text{op}}^2 \right)^{1/2} . $$

Moreover, if instead of (F.i), and assume that (F.ii) is satisfied, we get

$$ \begin{aligned} \tilde{\mathring{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}}) & = O_P \left( h^m T_n^{k \dim(\mathbb{G})/4} + T_n^{-k\hat{m}} + n^{-k^2/2} h^{-dk/2} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j: \|\mathbf{w}_j\| \le ah^{-1}} |\phi_{\mathbf{U}_j}(\mathbf{w}_j)^{-1}| \right) \right. \\ & \qquad \left. \times \left( \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \|_{\text{op}}^2 \right)^{1/2} \right) . \end{aligned} $$

The term $h^m T_n^{k \dim(\mathbb{G})/4} + T_n^{-k\hat{m}}$ is from the bias part and the other term is from the variance part. The factor $T_n^{k \dim(\mathbb{G})/4}$ in the asymptotic bias arises from the term $\int_{\mathbb{G}^k} |\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k)| d\mu(\mathbf{u}_j)$ found in the bias part. This term is known to have a rate that is at least as fast as $T_n^{k \dim(\mathbb{G})/4}$. Now, we aim to derive the partial uniform rate $\mathbb{G}$.

**Theorem 4.10.** Assume that Conditions (K1)-(i), (K2), (A'1)-(A'4), (A3)-(i), (A3)-(ii*) and (F.i) are satisfied. Then, for $\tilde{\mathbf{t}} = (\tilde{\mathbf{t}}', \tilde{\mathbf{t}}) \in \mathbb{R}^{dk} \times \mathbb{G}^k$, we can establish that

$$ \begin{aligned} & \mathbb{E} \left( \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} |\tilde{\mathring{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}})| \right) \\ & = O \left( h^m T_n^{k \dim(\mathbb{G})/4} + T_n^{-k\hat{m}} + n^{-k^2/2} h^{-dk/2} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j: \|\mathbf{w}_j\| \le ah^{-1}} |\phi_{\mathbf{U}_j}(\mathbf{w}_j)^{-1}| \right) \right. \\ & \qquad \left. \times \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \|_{\text{op}} \right) . \end{aligned} $$

Additionally, if we replace (F.i) with the assumption that (F.ii) holds, then we obtain

$$ \begin{aligned} & \mathbb{E} \left( \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} |\tilde{\mathring{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}})| \right) \\ & = O \left( h^m T_n^{k \dim(\mathbb{G})/4} + T_n^{-k\hat{m}} + n^{-k^2/2} h^{-dk/2} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j: \|\mathbf{w}_j\| \le ah^{-1}} |\phi_{\mathbf{U}_j}(\mathbf{w}_j)^{-1}| \right) \right. \\ & \qquad \left. \times \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \|_{\text{op}} \right) . \end{aligned} $$

We note that the only difference between the rates of convergence in Theorems 4.9 and 4.10 is that the former rate includes the term $\left( \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \|_{\text{op}}^2 \right)^{1/2}$, while the
latter rate includes the term $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\text{op}}$. Considering that $T_n \to \infty$, we can conclude that the latter rate is generally slower than the former. If the asymptotic bias $h^m T_n^{k \dim(\hat{\mathbb{G}})/4} + T_n^{-k\hat{m}}$ in Theorems 4.9 and 4.10, then both error rates can be the same since the terms $\left( \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\text{op}}^2 \right)^{1/2}$ and $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\text{op}}$ contribute to the asymptotic variances. Such an exceptional case arises with optimal speeds of $h$ and $T_n$ in all smoothness scenarios except when conditions (S'1) and (S1) are satisfied simultaneously.

Now, let us explore the uniform rate and $L^2$ rate over $\mathbb{R}^{dk} \times \mathbb{G}^k$. In both cases, we analyze the case where $\mathbb{G}^k = \mathbb{T}^{qk}$ for some $q \in \mathbb{N}$. We recall that $\mathbb{R}^{dk} \times \mathbb{T}^{qk}$ is a locally compact Abelian group due to its nature as an Abelian Lie group. It is known that any finite-dimensional connected Abelian Lie group is isomorphic to $\mathbb{R}^{dk} \times \mathbb{T}^{qk}$ for some $d$ and $q$. Therefore, investigating the scenario of $\mathbb{R}^{dk} \times \mathbb{T}^{qk}$ essentially covers all finite-dimensional connected Abelian Lie groups. Initially, we begin by studying the uniform rate over $\mathbb{R}^{dk} \times \mathbb{T}^{qk}$. We recall the definition $\sigma_{\tilde{\mathbf{v}}}^M(\mathbf{g}) = \prod_{j=1}^d (g_j)^{v_j}$ for $\tilde{\mathbf{v}} = (v_1, \dots, v_q) \in \mathbb{Z}^q$ and $\mathbf{g} = (g_1, \dots, g_q) \in \mathbb{T}^q$. We impose the following conditions:

(A3*) For some $\hat{m} \in \mathbb{N}$ with $\hat{m} > q/4$,

(i) $\tilde{f}_{\bar{\mathbb{X}}}(\tilde{\mathbf{t}}', \cdot) : \mathbb{T}^{qk} \to \mathbb{R}$ are $2\hat{m}$-times continuously differentiable for all $\tilde{\mathbf{t}}' \in \mathbb{R}^{dk}$;

(ii) $\sup_{(\tilde{\mathbf{t}}', \mathbf{g}) \in \mathbb{R}^{dk} \times \mathbb{T}^{qk}} \sum_{v_1, \dots, v_k \in \mathbb{Z}^{qk}: \|v_j\|^2 \ge T_n} |\phi_{\Delta\hat{m}}(\tilde{f}_{\bar{\mathbb{X}}}(\tilde{\mathbf{t}}', \cdot))(\sigma_{\tilde{\mathbf{v}}}^M) \cdot \sigma_{\tilde{\mathbf{v}}}^M(\mathbf{g})| = 0(1).$

Condition (A3*) is an extension of (A3) for the case where $\mathbb{G}^k = \mathbb{T}^{qk}$. Condition (A3*)-(ii) is about some uniform absolute convergence of the Fourier series $\sum_{v_1, \dots, v_k \in \mathbb{Z}^{qk}} \phi_{\Delta\hat{m}}(\tilde{f}_{\bar{\mathbb{X}}}(\tilde{\mathbf{t}}', \cdot))(\sigma_{\tilde{\mathbf{v}}}^M) \cdot \sigma_{\tilde{\mathbf{v}}}^M(\mathbf{g})$ of $\Delta\hat{m}(\tilde{f}_{\bar{\mathbb{X}}}(\tilde{\mathbf{t}}', \cdot))$.

**Theorem 4.11.** Under the assumptions that Conditions (K1)-(i), (K2), (A2), (A3*), (A4), and (F.i) are satisfied, alongside the integrability of $K_h^D(\cdot)$, it follows that

$$ 
\begin{aligned}
& \mathbb{E} \left( \sup_{\tilde{\mathbf{t}} \in \mathbb{R}^{dk} \times \mathbb{T}^{qk}} \left| \tilde{\dot{r}}_n^{(k)} (\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)} (\varphi, \tilde{\mathbf{t}}) \right| \right) \\
& = 0 \left( h^m T_n^{qk/4} + T_n^{-k\hat{m}} + n^{-k^2/2} h^{-dk} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j: \|\mathbf{w}_j\| \le ah^{-1}} \left| \phi^{\mathbf{U}_j}(\mathbf{w}_j)^{-1} \right| \right) \right. \\
& \qquad \left. \times \sum_{v_1, \dots, v_k \in \mathbb{Z}^q: \|v_j\|^2 < T_n} \prod_{j=1}^k \left| \phi^{\mathbf{U}_j} \left( (\sigma_{v_j}^M)^{-1} \right) \right| \right).
\end{aligned}
$$

Furthermore, if we assume that (F.ii) is satisfied instead of (F.i), we obtain

$$ 
\begin{aligned}
& \mathbb{E} \left( \sup_{\tilde{\mathbf{t}} \in \mathbb{R}^{dk} \times \mathbb{T}^{qk}} \left| \tilde{\dot{r}}_n^{(k)} (\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)} (\varphi, \tilde{\mathbf{t}}) \right| \right) \\
& = 0 \left( h^m T_n^{qk/4} + T_n^{-k\hat{m}} + n^{-k^2/2} h^{-dk} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j: \|\mathbf{w}_j\| \le ah^{-1}} \left| \phi^{\mathbf{U}_j}(\mathbf{w}_j)^{-1} \right| \right) \right. \\
& \qquad \left. \times \sum_{v_1, \dots, v_k \in \mathbb{Z}^q: \|v_j\|^2 < T_n} \prod_{j=1}^k \left| \phi^{\mathbf{U}_j} \left( (\sigma_{v_j}^M)^{-1} \right) \right| \right).
\end{aligned}
$$
It is observed that the rate mentioned in Theorem 4.11 tends to be slower compared to that in Theorem 4.10, mainly because of the additional factor $h^{-dk/2}$ in the asymptotic variance. A similar additional factor $h^{-k/2}$ is also present in the uniform rate of the conditional U-statistics estimator.

**Remark 4.12.** By employing similar reasoning as in the proof of Theorem 4.11, we can also obtain the same rate for

$$ \mathbb{E} \left( \sup_{\tilde{\mathbf{t}}' \in \mathbb{R}^{dk}} \left| \tilde{r}_n^{(k)} (\varphi, T_n, \tilde{\tilde{\mathbf{t}}}) - \bar{r}^{(k)} (\varphi, \tilde{\tilde{\mathbf{t}}}) \right| \right) $$

with fixed $\tilde{\mathbf{t}} \in \mathbb{G}^k$, under the same conditions except that (A3*)-(ii) is replaced by
$$ \sup_{\tilde{\mathbf{t}}' \in \mathbb{R}^{dk}} \sum_{v_1, \dots, v_k \in \mathbb{Z}^{qk}; \|v_j\|^2 \ge T_n} \left| \phi_{\Delta \tilde{m}}(\bar{f}_{\mathbb{X}}(\tilde{\mathbf{t}}', \cdot)) (\sigma_{\tilde{\mathbf{v}}}^M) \cdot \sigma_{\tilde{\mathbf{v}}}^M(\tilde{\mathbf{t}}) \right| = O(1). $$

We now present the $L^2$ rate over $\mathbb{R}^{dk} \times \mathbb{T}^{qk}$. To do so, we consider the following conditions:

(A5) (i) $\tilde{f}_{\mathbb{X}}(\cdot)$ is square integrable on $\mathbb{R}^{dk} \times \mathbb{T}^{qk}$,

(ii) $\sum_{v_1, \dots, v_k \in \mathbb{Z}^q; \|v_j\|^2 < T_n} \prod_{j=1}^k \int_{\mathbb{R}^{dk}} |\phi^{\mathbb{X}}((\sigma_{\tilde{\mathbf{w}}} \otimes \sigma_{\tilde{\mathbf{v}}})^M)|^2 \|\mathbf{w}_j\|^{2m} d\mathbf{w}_1 \cdots d\mathbf{w}_k < \infty$ for some $m \in \mathbb{N}$;

(A6) For $\hat{m}$ satisfying (A3*)-(i), the map $(\tilde{\mathbf{w}}, \mathbf{g}) \mapsto \Delta \hat{m}(\bar{f}_{\mathbb{X}}(\tilde{\mathbf{w}}, \cdot))(\mathbf{g})$ is square integrable on $\mathbb{R}^{dk} \times \mathbb{T}^{qk}$;

(A7) $\sup_{\mathbf{w}_j \neq 0_d} (\|\mathbf{w}_j\|^{-2m} |\phi^{K_h^D}(\mathbf{w}_j) - 1|) < \infty$ for $m$ in (A5)-(ii).

Condition (A5) and (A6) impose smoothness conditions on $\tilde{f}_{\mathbb{X}}$. Condition (A5)-(i) is relatively weak. If it holds, then

$$ \sum_{v_1, \dots, v_k \in \mathbb{Z}^q; \|v_j\|^2 < T_n} \prod_{j=1}^k \int_{\mathbb{R}^{dk}} |\phi^{\mathbb{X}}((\sigma_{\tilde{\mathbf{w}}} \otimes \sigma_{\tilde{\mathbf{v}}})^M)|^2 \|\mathbf{w}_j\|^{2m} d\mathbf{w}_1 \cdots d\mathbf{w}_k < \infty. $$

Condition (A5)-(ii) is a variation of (A5)-(i) and extends the Sobolev condition $\int_{\mathbb{R}} |\phi^{\mathbf{X}}(\sigma_{\mathbf{w}_j})|^2 \times |\mathbf{w}_j|^{2m} d\mathbf{w}_j < \infty$. This Sobolev condition is sometimes assumed in deconvolution kernel density estimation on $\mathbb{R}$; see (2.30) in [78]. Condition (A6) is a variation of (A3*)-(ii). For condition (A7), suppose that $K_h^D = \prod_{j=1}^d K_{h,j}^D$ and $\phi_{K_{h,j}^D}(\mathbf{w}_j) = 1 + O(\mathbf{w}_j^m)$ as $\mathbf{w}_j \to 0$ for all $1 \le j \le d$. Then, $\phi_{K_h^D}(\mathbf{w}) - 1 = \prod_{j=1}^d \phi_{K_{h,j}^D}(\mathbf{w}_j) - 1 = O(\prod_{j=1}^d \mathbf{w}_j^m)$ as $\mathbf{w} \to 0_d$ and it satisfies (A7). This condition on $K_{h,j}^D(\cdot)$ is sometimes assumed in the literature of deconvolution kernel density estimation on $\mathbb{R}$, see (2.40) in [108] for example.

**Theorem 4.13.** Under the assumptions that Conditions (K1)-(i), (K2), (A2), (A3*), (A4), and (F.i) are satisfied, with the integrability of $K_h^D(\cdot)$, it holds that

$$ \left( \int_{\mathbb{R}^{dk} \times \mathbb{T}^{qk}} \mathbb{E} \left( \left| \tilde{r}_n^{(k)} (\varphi, T_n, \tilde{\tilde{\mathbf{t}}}) - \bar{r}^{(k)} (\varphi, \tilde{\tilde{\mathbf{t}}}) \right|^2 \right) d \text{Leb}_d \otimes \mu(\tilde{\mathbf{t}}_1) \cdots d \text{Leb}_d \otimes \mu(\tilde{\mathbf{t}}_k) \right)^{1/2} \\ = O \left( h^m + T_n^{-k\hat{m}} + n^{-k^2/2} h^{-dk/2} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j; \|\mathbf{w}_j\| \le ah^{-1}} |\phi^{U_j}(\mathbf{w}_j)^{-1}| \right) \right) $$
$$ \times \sum_{v_1, \dots, v_k \in \mathbb{Z}^q : \|v_j\|^2 < T_n} \prod_{j=1}^{k} \left| \phi^{\mathbf{u}_j} \left( \left( \sigma_{v_j}^{M} \right)^{-1} \right) \right|^2 \quad \frac{1}{2} $$

Furthermore, if we assume that (F.ii) is satisfied instead of (F.i), we obtain

$$ \left( \int_{\mathbb{R}^{dk} \times \mathbb{T}^{qk}} \mathbb{E} \left( \left| \tilde{\bar{r}}_n^{(k)}(\varphi, T_n, \tilde{\bar{\mathbf{t}}}) - \bar{r}^{(k)}(\varphi, \tilde{\mathbf{t}}) \right|^2 \right) d \mathrm{Leb}_d \otimes \mu(\bar{\mathbf{t}}_1) \cdots d \mathrm{Leb}_d \otimes \mu(\bar{\mathbf{t}}_k) \right)^{1/2} \\ = O \left( h^m + T_n^{-k\hat{m}} + n^{-k^2/2} h^{-dk/2} \prod_{j=1}^k \left( \sup_{\mathbf{w}_j : \|\mathbf{w}_j\| \le ah^{-1}} \left| \phi^{\mathbf{u}_j}(\mathbf{w}_j)^{-1} \right| \right) \right) \\ \times \sum_{v_1, \dots, v_k \in \mathbb{Z}^q : \|v_j\|^2 < T_n} \prod_{j=1}^k \left| \phi^{\mathbf{u}_j} \left( \left( \sigma_{v_j}^{M} \right)^{-1} \right) \right|^2 \quad \frac{1}{2} $$

The rate in Theorem 4.13 is generally faster than the rate in Theorem 4.9 since $T_n^{k \dim(\mathbb{G})/4}$ in the latter rate does not appear in the former rate. This is due to the absence of the term $\int_{\mathbb{T}^{qk}} |\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k)| d\mu(\tilde{\mathbf{u}})$ in the bias part of the $L^2$ error.

**Remark 4.14.** We observe that the term $\prod_{j=1}^k \left( \sup_{\mathbf{w}_j : \|\mathbf{w}_j\| \le ah^{-1}} \left| \phi^{\mathbf{u}_j}(\mathbf{w}_j)^{-1} \right| \right)$, along with the terms involving $\phi^{\mathbf{u}_j}(\cdot)$ in the error rates of Theorems 4.9-4.13, represent the effects of the smoothness of $\tilde{f}_{\mathbb{U}}(\cdot)$ and $\tilde{f}_{\mathbb{U}}(\cdot)$. The former term with $d=1$ and $k=1$ is a standard term found in the pointwise, uniform and $L^2$ error rates of the univariate Euclidean deconvolution density estimator, see Lemma 2 in [50] and Chapters 2.4.2 and 2.4.3 in [108]. The terms involving $\phi^{\mathbf{u}_j}(\cdot)$ in Theorems 4.9 and 4.13 appear in the $L^2$ error rates of $\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})$ and the corresponding terms in Theorems 4.10 and 4.11 can be found in the uniform consistency of $\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})$.

# 5. Asymptotic distributions

## 5.1. Asymptotic distributions for certain Lie groups

In this section, we explore the asymptotic distributions of our conditional U-statistics defined in (3.9). Due to our work in a nonstandard setting, our proofs are more intricate than those in the existing literature, highlighting the value of our contribution. We will consider the following Lie groups:

(G1) $\mathbb{G} = \mathbb{T}^D$ for some $D \ge 1$;

(G2) $\mathbb{G}$ is either SU(2) or SO(3), and $\phi^{\mathbf{u}_j}(\sigma_l^M) = s_l \cdot I_{d_{\sigma_l}}$ for some $0 \ne s_l \in \mathbb{R}$ and for all $l \in \{0\} \cup \mathbb{N}$.

It is worth noting that all compact and connected Abelian Lie groups are algebraically and topologically isomorphic to $\mathbb{T}^D$ for some $D \ge 1$. In other words, a group isomorphism exists between any two compact and connected Abelian Lie groups of the same dimension, which is also a homeomorphism (e.g., Chapter 4.4.2 in [112]). Since all properties of a Lie group are governed by its algebraic and topological structure, addressing case (G1) essentially encompasses all Abelian cases. Like for the case (G2), the assumption $\phi^{\mathbf{u}_j}(\sigma_l^M) = s_l \cdot I_{d_{\sigma_l}}$ is not overly restrictive; it covers significant distributions such as the Laplace and Gaussian families introduced in Section 4.1, as well as scenarios involving no measurement error. Laplace (Gaussian) distributions on $\mathbb{G} = \mathrm{SU}(2)$ and $\mathbb{G} = \mathrm{SO}(3)$ are referred to as hyperspherical Laplace (Gaussian) distributions and rotational Laplace (Gaussian) distributions, respectively. Before properly presenting our results for the asymptotic distribution of $\hat{\tau}_n^{(k)}(\varphi, \cdot)$, let
us remember an essential tool in analyzing U-statistics: Hoeffding's decomposition, which we recall from [38] for easy reference. With the aid of the notation $\mathbb{P}_1 \times \dots \times \mathbb{P}_k L = \int L d(\mathbb{P}_1 \times \dots \times \mathbb{P}_k)$, the Hoeffding [68] projections of $L: S^k \to \mathbb{R}$ are defined by

$$ \pi_{\ell} L(x_1, \dots, x_{\ell}) := \pi_{\ell,k}^{\mathbb{P}} f(x_1, \dots, x_{\ell}) := (\delta_{x_1} - \mathbb{P}) \times \dots \times (\delta_{x_{\ell}} - \mathbb{P}) \times \mathbb{P}^{k-\ell} L, $$

for $x_i \in S$ and $0 \le \ell \le k$. It is important to note that $\pi_0 L = \mathbb{P}^k L$, and for $\ell > 0$, $\pi_\ell L$ becomes a completely degenerate function of $\ell$ variables. When considering an integrable function $L(\cdot)$, these projections lead to a decomposition of the U-statistic

$$ U_n(L) := U_n^{(k)}(L) := U_n^{(k)}(L, \mathbb{P}) := \frac{1}{\binom{n}{k}} \sum_{1 \le i_1 < \dots < i_k \le n} L(X_{i_1}, \dots, X_{i_k}), $$

into a sum of U-statistics of orders $\ell \le k$, which are orthogonal if $\mathbb{P}^k L^2 < \infty$ and whose kernels are completely degenerate, namely, the Hoeffding decomposition:

$$ U_n^{(k)}(L) = \sum_{\ell=0}^{k} \binom{k}{\ell} U_n^{(\ell)}(\pi_{\ell} L). $$

This decomposition readily follows from the following expansion:

$$ L(x_1, \dots, x_k) = \delta_{x_1} \times \dots \times \delta_{x_k} = ((\delta_{x_1} - \mathbb{P}) + \mathbb{P}) \times \dots \times ((\delta_{x_k} - \mathbb{P}) + \mathbb{P}) L, $$

into terms of the form

$$ (\delta_{x_{i_1}} - \mathbb{P}) \times \dots \times (\delta_{x_{i_\ell}} - \mathbb{P}) \times \mathbb{P}^{k-\ell} L. $$

It is simple to check that $L$ symmetric is $\mathbb{P}$-degenerate of order $r-1$ iff $r = \min\{\ell > 0 : \pi_{\ell,k}^{\mathbb{P}} L \ne 0\}$. Thus, $L(\cdot)$ is degenerate of order $r-1 \ge 0$ if and only if its Hoeffding expansion, except for the constant term, starts at term $r$, that is,

$$ U_n(L) - \mathbb{P}^k L = \sum_{\ell=r}^{k} \binom{k}{\ell} U_n^{(\ell)}(\pi_{\ell} L). $$

Considering $(\mathbf{X}_i, \mathbf{Y}_i), i \ge 1$, i.i.d.- $\mathbb{P}$ and assuming $L$ is in $L_2(\mathbb{P}^k)$, this is an orthogonal decomposition and

$$ \mathbb{E}[\pi_{\ell} L(\mathbf{X}_{\ell}, \mathbf{Y}_{\ell}) | (\mathbf{X}_2, \mathbf{Y}_2), \dots, (\mathbf{X}_{\ell}, \mathbf{Y}_{\ell})] = 0, \quad \ell \ge 1, $$

where we denote $\mathbf{X}_\ell$ and $\mathbf{Y}_\ell$ for $(\mathbf{X}_1, \dots, \mathbf{X}_\ell)$ and $(\mathbf{Y}_1, \dots, \mathbf{Y}_\ell)$, respectively. Thus, the kernels $\pi_\ell L$ are canonical for $\mathbb{P}$ (or completely degenerate, or completely centered). Also, $\pi_\ell, \ell \ge 1$, are nested projections, that is, $\pi_\ell \circ \pi_l = \pi_\ell$ if $\ell \le l$, and

$$ \mathbb{E}[(\pi_{\ell} L)^2 (\mathbf{X}_{\ell}, \mathbf{Y}_{\ell})] \le \mathbb{E}[(L - \mathbb{E}L)^2 (\mathbf{X}, \mathbf{Y})] \le \mathbb{E}L^2 (\mathbf{X}, \mathbf{Y}). \quad (5.1) $$

Making use of the decomposition of [68], we get that

$$ \begin{aligned} \tilde{\mu}_n(\varphi, T_n, \tilde{\mathbf{t}}) &:= \tilde{U}_n(\varphi, T_n, \tilde{\mathbf{t}}) - \theta_n = \sum_{\ell=1}^{k} \binom{k}{\ell} \tilde{U}_n^{(\ell)}(\pi_{\ell} G_{\varphi, T_n, \tilde{\mathbf{t}}}) \\ &= k \tilde{U}_n^{(1)}(\pi_1 G_{\varphi, T_n, \tilde{\mathbf{t}}}) + \sum_{\ell=2}^{k} \binom{k}{\ell} \tilde{U}_n^{(\ell)}(\pi_{\ell} G_{\varphi, T_n, \tilde{\mathbf{t}}}), \end{aligned} \quad (5.2) $$
where $\theta_n$ denotes the expectation of $\tilde{U}_n$ which equals to

$$ \theta_n = \int_{\mathbb{G}^k} r^{(k)}(\mathbf{s}_1, \dots, \mathbf{s}_k) \prod_{j=1}^k \mathrm{Re} (K_{T_n}(\mathbf{t}_j, \mathbf{u}_j)) f_X(\mathbf{s}_j, \mathbf{u}_j) d\mu(\mathbf{u}_j). \quad (5.3) $$

Based on the definition of the Hoeffding projections and considering the i.i.d. sample $(\mathbf{Z}_1, \mathbf{Y}_1), \dots, (\mathbf{Z}_n, \mathbf{Y}_n)$, for all $(\mathbf{z}, \mathbf{y}) \in \mathbb{G} \times \mathbb{R}$, we have:

$$ \begin{aligned} \pi_1 G_{\varphi, T_n, \tilde{\mathbf{t}}}(\mathbf{z}, \mathbf{y}) &= \mathbb{E} \left[ G_{\varphi, T_n, \tilde{\mathbf{t}}} \left( (\mathbf{z}, \mathbf{Z}_2, \dots, \mathbf{Z}_k), (\mathbf{y}, \mathbf{Y}_2, \dots, \mathbf{Y}_k) \right) \right] - \mathbb{E} G_{\varphi, T_n, \tilde{\mathbf{t}}}(\mathbf{Z}, \mathbf{Y}) \\ &= \mathbb{E} \left[ G_{\varphi, T_n, \tilde{\mathbf{t}}} (\mathbf{Z}, \mathbf{Y}) \mid (\mathbf{Z}_1, \mathbf{Y}_1) = (\mathbf{z}, \mathbf{y}) \right] - \mathbb{E} G_{\varphi, T_n, \tilde{\mathbf{t}}} (\mathbf{Z}, \mathbf{Y}). \end{aligned} $$

Let us introduce the following function on $\mathbb{G} \times \mathbb{R}$

$$ S_{\varphi, T_n, \tilde{\mathbf{t}}} : \mathbb{G} \to \mathbb{R} \\ (\mathbf{z}, \mathbf{y}) \mapsto k \mathbb{E} \left[ G_{\varphi, T_n, \tilde{\mathbf{t}}} (\mathbf{Z}, \mathbf{Y}) \mid (\mathbf{Z}_1, \mathbf{Y}_1) = (\mathbf{z}, \mathbf{y}) \right]. $$

With this notation, it is straightforward to express:

$$ k\pi_1 G_{\varphi, T_n, \tilde{\mathbf{t}}} = S_{\varphi, T_n, \tilde{\mathbf{t}}} - \mathbb{E} S_{\varphi, T_n, \tilde{\mathbf{t}}}. \quad (5.4) $$

Additionally, we can see that

$$ \begin{aligned} & \mathbb{E} \left[ G_{\varphi, T_n, \tilde{\mathbf{t}}} (\mathbf{Z}, \mathbf{Y}) \mid (\mathbf{Z}_1, \mathbf{Y}_1) = (\mathbf{z}, \mathbf{y}) \right] \\ &= \mathrm{Re} (K_{T_n}(\mathbf{t}_1, \mathbf{z}_1)) \int_{\mathbb{R}^{dk}} \int_{\mathbb{G}^{k-1}} \varphi(\mathbf{y}, \mathbf{y}_2, \dots, \mathbf{y}_k) \prod_{j=2}^k \mathrm{Re} (K_{T_n}(\mathbf{t}_j, \mathbf{z}_j)) f_{\mathbf{X}\mathbf{Y}}(\mathbf{z}_j, \mathbf{y}_j) d\mu(\mathbf{z}_j) d\mathbf{y}_j \\ &=: \mathrm{Re} (K_{T_n}(\mathbf{t}_1, \mathbf{z}_1)) \tilde{\varphi}_n(\mathbf{y}). \end{aligned} $$

Let

$$ w_n(\mathbf{t}_1, \mathbf{z}_1, \mathbf{y}) = \mathrm{Re} (K_{T_n}(\mathbf{t}_1, \mathbf{z}_1)) \left( \tilde{\varphi}_n(\mathbf{y}) - \prod_{j=1}^{k-1} f_X(\mathbf{t}_j) r^{(k)}(\varphi, \tilde{\mathbf{t}}) \right). \quad (5.5) $$

Before presenting the result concerning the asymptotic distribution of $\hat{\Gamma}_n^{(k)}(\varphi, T_n, \cdot)$ for the ordinary-smooth scenario, Let us recall that $\Delta$ represents the Laplace-Beltrami operator associated with $\mathbb{G}$, and $\Delta\hat{m}$ denotes the $\hat{m}$-times composition of $\Delta$. It is notable that if a function $f: \mathbb{G} \to \mathbb{C}$ is $2\hat{m}$-times continuously differentiable on $\mathbb{G}$, then the function $\Delta(f): \mathbb{G} \to \mathbb{C}$ is well defined and is continuous on $\mathbb{G}$.

**Theorem 5.1.** Under the assumption that either (G1) or (G2) holds, along with the absolute convergence of the Fourier series of $\Delta\hat{m}(\tilde{f}_{\mathbf{X}}(\cdot))$ and $\Delta\hat{m}(r^{(k)}(\varphi, \cdot) \cdot \tilde{f}_{\mathbf{X}}(\cdot))$ on $\mathbb{G}^k$, and given that conditions (S1), (A1), (A2), (A3), and (T1') are satisfied, we have that for all $\tilde{\mathbf{t}} \in \mathbb{G}^k$:

$$ \sqrt{n} \cdot \frac{\hat{\Gamma}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}}) - \mathbb{E}(w_n(\mathbf{t}_1, \mathbf{Z}, \mathbf{Y})) / \prod_{j=1}^k f_X(\mathbf{t}_j)}{\sqrt{\mathrm{Var}(w_n(\mathbf{t}_1, \mathbf{Z}, \mathbf{Y})) / \prod_{j=1}^k f_X(\mathbf{t}_j)}} \xrightarrow{d} N(0, 1). \quad (5.6) $$

Concerning the absolute convergence of the Fourier series as stated in Theorem 5.1, it is worth noting that if a function $f: \mathbb{G} \to \mathbb{C}$ is $2 \times [\dim(\mathbb{G})/4]$-times continuously differentiable, then its Fourier series converges absolutely (Theorem 3.1.3 in [3]). Indeed, in certain cases, much weaker conditions
are sufficient. For instance, the Fourier series of a function on $\mathbb{T}^1$ is absolutely convergent if the function is Hölder continuous with an exponent greater than 1/2; see [82]. For general toruses, sufficient conditions can be found in Theorem 3 of [139]. Additionally, for SU(2), some weak conditions are outlined in [112].

**Remark 5.2.** It is important to remark that the asymptotic distribution derived in this theorem does not hold under the supersmooth and log-supersmooth scenarios. For more understanding, please refer to Remark 8.1, which is included in its proof.

## 5.2. Extension to general Lie groups

In this section, we expand the asymptotic distribution results presented in Section 5.1 to general $\mathbb{G}$ under certain high-level conditions. To state the conditions, for any two positive sequences $a_n$ and $b_n$, we denote $a_n \gtrsim b_n$ (or $a_n \lesssim b_n$) to indicate the existence of a constant $c > 0$ such that for all $n$, $a_n \ge c \cdot b_n$ (or $a_n \le c \cdot b_n$). The constants $\alpha$ and $\gamma$ mentioned in the conditions below correspond to those appearing in the smoothness scenarios (S2) and (S3).

(B1) Ordinary-smoothness: There exists some constant $0 \le q \le \dim(\mathbb{G})/2$ such that, for each $\mathbf{t}_1 \in \mathbb{G}$,

$$ \mathbb{E} \left( \left( \mathrm{Re} \left( K_{T_n} (\mathbf{t}_1, \mathbf{Z}_1) \right) \right)^2 \right) \gtrsim T_n^{2\beta+q}; $$

(B2) Super smoothness: There exists some constant $0 \le q \le \dim(\mathbb{G})/2$ such that, for each $\mathbf{t}_1 \in \mathbb{G}$ and $\eta \in (0, 1)$,

$$ \mathbb{E} \left( \left( \mathrm{Re} \left( K_{T_n} (\mathbf{t}_1, \mathbf{Z}_1) \right) \right)^2 \right) \gtrsim T_n^{2\alpha+q} \exp \left( 2 \cdot \gamma \cdot (\eta \cdot T_n)^\beta \right); $$

(B3) Log-supersmoothness: There exist some constants $0 \le q \le \dim(\mathbb{G})/2$ and $\zeta \in \mathbb{R}$ such that, for each $\mathbf{t}_1 \in \mathbb{G}$ and $\eta \in (0, 1)$,

$$ \mathbb{E} \left( \left( \mathrm{Re} \left( K_{T_n} (\mathbf{t}_1, \mathbf{Z}_1) \right) \right)^2 \right) \gtrsim T_n^{2\alpha+q} \exp \left( 2 \cdot \gamma \cdot (\eta \cdot T_n)^\beta (\log T_n - \zeta) \right). $$

Notably, the lower bounds for $\mathbb{E} \left( \mathrm{Re} \left( K_{T_n} (\mathbf{t}_1, \mathbf{Z}_1) \right)^2 \right)$ in (B1)-(B3) are inspired by the upper bounds, which are of the order

$$ T_n^{2\beta + \frac{\dim(\mathbb{G})}{2}}, \quad T_n^{2\alpha + \frac{\dim(\mathbb{G})}{2}} \exp \left( 2 \cdot \gamma \cdot T_n^\beta \right) \quad \text{or} \quad T_n^{2\alpha + \frac{\dim(\mathbb{G})}{2}} \exp \left( 2 \cdot \gamma \cdot T_n^\beta (\log T_n - \xi_1) \right). $$

These lower bounds depend on the smoothness scenarios. Notice that $q = \dim(\mathbb{G})/2$ is maximal in view of the upper bounds. Indeed, cases (G1) and (G2) satisfy (B1)-(B3) with $q = \dim(\mathbb{G})/2$, as mentioned in Lemma 9.3. Additionally, we analyze general diverging speeds of $T_n$ instead of the specific ones outlined in (T1')-(T3') according to (B1)-(B3). We highlight that the following ranges cover the optimal speeds in (T1')-(T3'):

(T1'') $T_n \asymp n^p$ for some $0 < p < 1/(\dim(\mathbb{G}) - q)$, where $q$ is the constant in (B1);

(T2'') $T_n \lesssim (\log n)^{1/\beta}$ for $\beta$ in (S2);

(T3'') $T_n \lesssim (\log n / \log \log n)^{1/\beta}$ for $\beta$ in (S3);

(T1'') $T_n \asymp n^p$ for some $1/(2k\beta + 4k\hat{m} + k \dim(\mathbb{G})/2) < p < 1/(2k\beta + k \dim(\mathbb{G}))$.

The above range of $p$ remains valid since $r^{(k)}(\phi, \cdot)$ in (A1) is larger than $\dim(\mathbb{G})/4$. The upper bound $1/(2\beta + \dim(\mathbb{G}))$ is necessary for $T_n$ to fulfill (T1). We also note that the range in (T1'') covers the optimal speed in (T1'). Now, we present the asymptotic distribution of $\hat{r}_n^{(k)}(\phi, \cdot)$ for general $\mathbb{G}$ and $T_n$.

**Theorem 5.3.** Assume that the Fourier series of $\Delta\hat{m}(\tilde{f})(\cdot)$ and of $\Delta\hat{m}(r^{(k)} \cdot \tilde{f})(\cdot)$ converge absolutely on $\mathbb{G}$ and that the conditions (S1)(i), (A1), (A2), (A3), (B1) with $q = \dim(\mathbb{G})/2$, (T1''), (T2''), (T3'') and (T1''') hold. Then, the asymptotic distribution provided in Theorem 5.1 remains valid for all $\tilde{t} \in \mathbb{G}^k$.
We can prove that the conclusion of Theorem 5.3 holds for any $0 \le q \le \dim(\mathbb{G})/2$ with more complex versions of (A1) and (T1''). However, for simplicity, we state it with $q = \dim(\mathbb{G})/2$. Verifying the high-level conditions (B1)-(B3) is challenging because determining the exact size of $\mathbb{E}((\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)))^2)$ is difficult using the existing Taylor expansions on manifolds; see [67] and [94] in the case when $k=1$. A simple yet rough lower bound for $\mathbb{E}((\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)))^2)$ is $[\mathbb{E}(\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)))]^2$. However, we only know that this lower bound remains bounded away from zero. Below, we present a method for verifying (B1)-(B3) with $q = \dim(\mathbb{G})/2$, which provides the sharpest lower bounds. One can similarly verify (B1)-(B3) with $q=0$ by following the same arguments as in the case of $q = \dim(\mathbb{G})/2$ using Proposition 9.6 in the Appendix. To explore $\mathbb{E}(|K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)|^2)$ instead of $\mathbb{E}((\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)))^2)$ and demonstrate that it achieves the lower bounds specified in (B1)-(B3) with $q = \dim(\mathbb{G})/2$, we introduce a condition. Let $\|\cdot\|_{\mathrm{HS}}$ denote the Hilbert-Schmidt norm on complex matrices. For a square complex matrix $A$, it is defined as $\|A\|_{\mathrm{HS}} = (\mathrm{Tr}(AA^*))^{1/2}$, where $A^*$ represents the conjugate transpose of $A$.

(B4) There exists a constant $c > 0$ such that, for all $\sigma_j \in \hat{\mathbb{G}}$, $c \cdot d_{\sigma_j}^{1/2} \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\mathrm{op}} \le \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\mathrm{HS}}$.

It is important to note that $\left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\mathrm{op}} \le \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\mathrm{HS}}$, always holds. Therefore, if $\sup_{\sigma_j \in \hat{\mathbb{G}}} d_{\sigma_j} < \infty$, then condition (B4) is satisfied with $c = (\sup_{\sigma \in \hat{\mathbb{G}}} d_{\sigma_j})^{-1/2}$. An equivalent condition to $\sup_{\sigma_j \in \hat{\mathbb{G}}} d_{\sigma_j} < \infty$ can be found in Theorem 1 of [110]. This implies that for any distribution of $\mathcal{U}$ on Abelian $\mathbb{G}$, where $d_{\sigma_j} = 1$, (B4) holds with $c = 1$. Additionally, since $\|I_{d_{\sigma_j}}\|_{\mathrm{op}} = 1$ and $\|I_{d_{\sigma_j}}\|_{\mathrm{HS}} = d_{\sigma_j}^{1/2}$, condition (B4) with $c = 1$ is satisfied for the Laplace and Gaussian distribution families introduced in Section 4.1, as well as in the case of no measurement error. It is noteworthy that cases (G1) and (G2) also satisfy (B4) with $c = 1$.

**Lemma 5.4.** Assume that the conditions (A2) and (B4) hold. Then, for each $j \in \{1, 2, 3\}$, under (Sj)(ii) $\mathbb{E}(|K_{T_n}(\mathbf{t}_1, \mathbf{Z})|^2)$ attains the lower bound given in (Bj) with $q = \dim(\mathbb{G})/2$.

Now, we present a version of Lemma 5.4 for $\mathbb{E}((\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)))^2)$. This last is not directly derived from Lemma 5.4 since

$$ \mathbb{E}(|K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)|^2) \ge \mathbb{E}((\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)))^2). $$

Therefore, we consider the case where

$$ \int_{\mathbb{G}} |K_{T_n}(\mathbf{t}_1, \mathbf{u})|^2 d\mu(\mathbf{u}) \asymp \int_{\mathbb{G}} \mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{u}))^2 d\mu(\mathbf{u}). \quad (5.7) $$

Since $\int_{\mathbb{G}} |K_{T_n}(\mathbf{t}_1, \mathbf{u})|^2 d\mu(\mathbf{u})$ achieves the same lower bounds as $\mathbb{E}(|K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)|^2)$ as shown in Lemma 5.4, (5.7) with (A2) implies $\mathbb{E}(|K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)|^2) \asymp \mathbb{E}((\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)))^2)$. Consequently, (B1)-(B3) follow with $q = \dim(\mathbb{G})/2$ from Lemma 5.4.

**Lemma 5.5.** Assume that the conditions (A2) and (B4) hold, and that (5.7) is satisfied. Then, for each $j \in \{1, 2, 3\}$, under (Sj)(ii), (Bj) holds with $q = \dim(\mathbb{G})/2$.
### 5.3. Asymptotic distributions for mixed data

In this section, we delve into the asymptotic distribution of $\tilde{\tilde{r}}_n^{(k)}(\varphi, T_n, \cdot)$, as defined in (4.6), for a general $\mathbb{G}$. This exploration demands advanced analytical methods. For this purpose, we consider the case where

$$ \tilde{K}_h^D(\tilde{\mathbf{u}}) := K_h^D(\tilde{\mathbf{u}}) = \prod_{\ell=1}^d K_{\ell,h}^D(\mathbf{u}_\ell) \text{ and } (\tilde{\mathbf{u}}' : 1 \le \ell \le d), $$

are considered mutually independent to simplify the conditions and avoid complexity. Here, $K_{\ell,h}^D(\mathbf{u}_\ell)$ is defined as follows

$$ K_{\ell,h}^{D}(\mathbf{u}_{\ell}) = \frac{1}{2\pi} \int_{\mathbb{R}} e^{-i\mathbf{s}_{\ell}\mathbf{u}_{\ell}} \frac{\phi K_{\ell,h}^{D}(\mathbf{s}_{\ell})}{\phi \tilde{\mathbf{u}}'_{\ell}(\mathbf{s}_{j}/h)} d\mathbf{s}_{\ell}. \quad (5.8) $$

In this case, $\tilde{\tilde{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}})$ simplifies to

$$ \bar{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \frac{\sum_{(i_1, \dots, i_k) \in I_n^k} \varphi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \operatorname{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})) \tilde{K}_h^D \left(\frac{\mathbf{t}'_1 - \mathbf{Z}'_{i_1}}{h}\right) \dots \tilde{K}_h^D \left(\frac{\mathbf{t}'_k - \mathbf{Z}'_{i_k}}{h}\right)}{\sum_{(i_1, \dots, i_k) \in I_n^k} \operatorname{Re}(\tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k})) \tilde{K}_h^D \left(\frac{\mathbf{t}'_1 - \mathbf{Z}'_{i_1}}{h}\right) \dots \tilde{K}_h^D \left(\frac{\mathbf{t}'_k - \mathbf{Z}'_{i_k}}{h}\right)}. \quad (5.9) $$

In order to achieve our result, we will use Hoeffding's decomposition, which would be the same as the previous decomposition in Subsection §5.1 and proceed through similar steps. Thus, we have

$$ \begin{aligned} \bar{\mu}_n(\varphi, T_n, \tilde{\bar{\mathbf{t}}}) &:= \bar{U}_n(\varphi, T_n, \tilde{\bar{\mathbf{t}}}) - \bar{\theta}_n = \sum_{\ell=1}^k \binom{k}{\ell} \bar{U}_n^{(\ell)}(\pi_\ell G_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}) \\ &= k \bar{U}_n^{(1)}(\pi_1 \bar{G}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}) + \sum_{\ell=2}^k \binom{k}{\ell} \bar{U}_n^{(\ell)}(\pi_\ell \tilde{\bar{G}}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}), \quad (5.10) \end{aligned} $$

where $\bar{\theta}_n$ represents the expectation of $\bar{U}_n(\varphi, T_n, \tilde{\bar{\mathbf{t}}})$ and this last is defined in (4.7). According to the Hoeffding projections, we get

$$ \pi_1 \bar{G}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}(\bar{\mathbf{z}}, \mathbf{y}) = \mathbb{E}[\bar{G}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}(\bar{\mathbf{z}}, \mathbf{Y}) | (\bar{\mathbf{z}}_1, \mathbf{Y}_1) = (\bar{\mathbf{z}}, \mathbf{y})] - \mathbb{E}\bar{G}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}(\bar{\mathbf{z}}, \mathbf{Y}), \quad \bar{\mathbf{z}} \in \mathbb{R} \times \mathbb{G}, \mathbf{y} \in \mathbb{R}. $$

We introduce the following function on $\mathbb{R} \times \mathbb{G} \times \mathbb{R}$

$$ \begin{aligned} \bar{S}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}} : \mathbb{R} \times \mathbb{G} &\to \mathbb{R} \\ (\bar{\mathbf{z}}, \mathbf{y}) &\mapsto k \mathbb{E} [\bar{G}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}(\bar{\mathbf{z}}, \mathbf{Y}) | (\bar{\mathbf{z}}_1, \mathbf{Y}_1) = (\bar{\mathbf{z}}, \mathbf{y})]. \end{aligned} $$

Using this notation, we write

$$ k\pi_1 \bar{G}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}} = \bar{S}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}} - \mathbb{E} \bar{S}_{\varphi, T_n, \tilde{\bar{\mathbf{t}}}}. \quad (5.11) $$

Furthermore, we have that
$$
\begin{align*}
& \mathbb{E} \left[ \bar{G}_{\varphi, T_n, \tilde{\mathbf{t}}}(\bar{Z}, \mathbf{Y}) \middle| (\bar{Z}_1, \mathbf{Y}_1) = (\bar{z}, \mathbf{y}) \right] \\
&= \frac{1}{h^d} K_h^D \left( \frac{\mathbf{t}'_1 - \mathbf{z}'_1}{h} \right) \mathrm{Re} \left( K_{T_n}(\mathbf{t}_1, \mathbf{z}_1) \right) \int_{\mathbb{R}^{dk}} \int_{\mathbb{R}^{d(k-1) \times \mathbb{G}^{k-1}}} \varphi(\mathbf{y}, \mathbf{y}_2, \dots, \mathbf{y}_k) \\
&\quad \times \frac{1}{h^{d(k-1)}} \prod_{j=2}^k K_h^D \left( \frac{\mathbf{t}'_j - \mathbf{z}'_j}{h} \right) \mathrm{Re} \left( K_{T_n}(\mathbf{t}_j, \mathbf{z}_j) \right) f_{\bar{\mathbf{X}}\mathbf{Y}}(\bar{z}_j, \mathbf{y}_j) dz'_j d\mu(\mathbf{z}_j) d\mathbf{y}_j \\
&=: \frac{1}{h^d} K_h^D \left( \frac{\mathbf{t}'_1 - \mathbf{z}'_1}{h} \right) \mathrm{Re} \left( K_{T_n}(\mathbf{t}_1, \mathbf{z}_1) \right) \bar{\varphi}_n(\mathbf{y}).
\end{align*}
$$

Let us introduce

$$
\tilde{\tilde{w}}_n(\bar{\mathbf{t}}_1, \bar{\mathbf{z}}, \mathbf{y}) = \frac{1}{h^d} K_h^D \left( \frac{\mathbf{t}'_1 - \mathbf{z}'_1}{h} \right) \mathrm{Re} \left( K_{T_n}(\mathbf{t}_1, \mathbf{z}_1) \right) \left( \bar{\varphi}_n(\mathbf{y}) - \prod_{j=1}^{k-1} f_{\bar{\mathbf{X}}}(\bar{\mathbf{t}}_j) \bar{r}^{(k)} \left( \varphi, \tilde{\tilde{\mathbf{t}}} \right) \right). \quad (5.12)
$$

Before presenting the asymptotic distribution result, we impose the following conditions:

(A8) $K_{\ell,h}^D(\cdot)$ are continuous, $\phi_{\ell,h}^{K_{\ell,h}^D} = 0$ on $\mathbb{R}^k \setminus [-1, 1]^k$ and $\phi_{\ell,h}^{K_{\ell,h}^D}(\cdot)$ are continuously differentiable for all $1 \le \ell \le d$;

(A9) There exists a constant $\beta_0 \ge 0$ such that, for all $1 \le \ell \le d$,

(i) $\lim_{|\mathbf{u}_\ell| \to \infty} \mathbf{u}_\ell^{\beta_0} \phi^{\mathbf{u}_\ell}(\mathbf{u}_\ell) = a_\ell$ for some constants $0 \ne a_\ell \in \mathbb{C}$,

(ii) $\phi^{\mathbf{u}_\ell}$ are continuously differentiable and $h^{k(\beta_0-1)} \sup_{\mathbf{u}_\ell \in \mathbb{R}: |\mathbf{u}_\ell| \le h^{-1}} |((\phi^{\mathbf{u}_\ell})^{-1})'(\mathbf{u}_\ell)|^k = O(1)$ for all $k \in [2, 2+\epsilon)$ for some $\epsilon > 0$;

(A10) $h \asymp n^{-p}$ and $T_n \asymp n^{p*}$ for some constants $p, p^* > 0$ satisfying $2dp + \dim(\mathbb{G}) p^* < 2$;

(A11) (i) $\tilde{f}_{\mathbb{X}}$ is bounded on $\mathbb{G}^k$;

(ii) $f_{\bar{\mathbf{X}}}(\tilde{\mathbf{t}}', \cdot)$ is bounded away from zero on $\mathbb{G}^k$;

(iii) $\int_{\mathbb{G}} (\mathrm{Re} (K_{T_n}(\mathbf{t}_1, \mathbf{z})))^2 d\mu(\mathbf{z}) \ge C_t \cdot T_n^{2\beta^*+\dim(\mathbb{G})/2}$ for some constant $C_t > 0$ and the constant $\beta^* \ge 0$ in (S1*).

Condition (A8) represents a weak kernel condition. Condition (A9) is a multivariate adaptation of con-
dition $A_{m,0}$-(i) from [47], adjusted for cases where $\phi_{\ell,h}^{K_{\ell,h}^D}$ have compact supports. Condition (K1)-(i),
along with the mutual independence of $(\mathbf{u}_j^\prime : 1 \le j \le d)$ and (A9)-(i), implies (S'1) with $\beta^* = d\beta_0^*$.
Many semi-ordinary-smooth distributions on $\mathbb{R}$ satisfy (A9). For example, the Laplace distribution
with parameter $\lambda > 0$, having $\phi_j^*(\mathbf{u}_j) = (1 + \lambda \mathbf{u}_j^2)^{-1}$, satisfies (A9) with $\beta_0^* = 2$ and $a_j = \lambda^{-1}$. Ad-
ditionally, the exponential distribution with parameter $\lambda > 0$, having $\phi_j^*(\mathbf{u}_j) = \lambda/(\lambda - i\mathbf{u}_j)$, satisfies
(A9) with $\beta_0^* = 1$ and $a_j = \lambda i$. Moreover, the gamma distribution with shape parameter $s > 0$ and
scale parameter $\lambda > 0$, having $\phi_j^*(\mathbf{u}_j) = (1 - \lambda i \mathbf{u}_j)^{-s}$, satisfies (A9) with $\beta_0^* = s$ and $a_j = (-\lambda i)^{-s}$.
Condition (A10) assumes polynomial rates for the smoothing parameters. Conditions (A11)-(i) and
(A11)-(ii) are weak given that $\mathbb{G}$ is compact. Condition (A11)-(iii) is a high-level condition regarding
the lower bound of $\int_{\mathbb{G}} (\mathrm{Re} (K T_n(\mathbf{t}, \mathbf{z})))^2 d\mu(\mathbf{z})$. This is satisfied if the following condition holds, along
with (G1) and (G2),

(RS1*) There exists a constant $c_2^*$ such that $\left\| \phi_{\mathbf{u}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} \ge c_2^* \cdot k_{\sigma_j}^\beta$, for all $\sigma_j \in \hat{\mathbb{G}}$ with $k_\sigma > 0$.

To clarify, condition (RS1*) is a reverse form of (S1) and covers all the semi-ordinary-smooth distri-
butions illustrated in Section 4.1. We now state the asymptotic distribution of $\hat{\mathbf{r}}_n^{(k)} (\varphi, T_n, \cdot)$.
**Theorem 5.6.** Under the assumptions that Conditions (K1)-(i), (S1), (A1), (A8)-(A10), and (A11)-(i) are satisfied. For $\tilde{\mathbf{t}} = (\tilde{\mathbf{t}}', \tilde{\mathbf{t}}) \in \mathbb{R}^{dk} \times \mathbb{G}^k$, then it follows that

$$ \sqrt{n} \cdot \frac{\tilde{\mathcal{V}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \bar{\mathcal{R}}^{(k)}(\varphi, \tilde{\mathbf{t}}) - \mathbb{E}(\tilde{\mathcal{W}}_n(\tilde{\mathbf{t}}_1, \bar{\mathbf{Z}}_1, \mathbf{Y}) / \prod_{j=1}^k f_{\bar{\mathbf{Z}}}(\tilde{\mathbf{t}}_j))}{\sqrt{\mathrm{Var}(\tilde{\mathcal{W}}_n(\tilde{\mathbf{t}}_1, \bar{\mathbf{Z}}_1, \mathbf{Y}) / \prod_{j=1}^k f_{\bar{\mathbf{Z}}}(\tilde{\mathbf{t}}_j))}} \xrightarrow{d} N(0, 1). $$

Now, we explore the asymptotic distribution of $\tilde{\mathcal{V}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}})$ under the conditions where (S2) and (S1*) are satisfied. To do so, we will consider modifications to Conditions (A8)-(A10) as follows.

(A8*) (i) $\varphi^{K_\ell}(\cdot)$ are real-valued for all $1 \le \ell \le d$,

(ii) there exist constants $c, k, \varrho > 0$ such that, for all $1 \le j \le d$ and $1 - \varrho \le |\mathbf{s}_\ell| \le 1$, $\varphi^{K_\ell}(\mathbf{s}_\ell) \ge c(1 - \mathbf{s}_\ell)^k$;

(A9*) (i) $\varphi^{\mathcal{U}_\ell}(\cdot)$ are real-valued for all $1 \le \ell \le d$,

(ii) there exist constants $C_1, C_2, \beta_0, \gamma_0 > 0$ such that, for all $1 \le \ell \le d$ and $\mathbf{s}_\ell \in \mathbb{R}$,

$$ C_1 \exp(\gamma_0 |\mathbf{s}_\ell|^{\beta_0}) \le |\varphi^{\mathcal{U}_\ell}(\mathbf{s}_\ell)^{-1}| \le C_2 \exp(\gamma_0 |\mathbf{s}_\ell|^{\beta_0}); $$

(A10*) $h \asymp (\log n)^{-1/\beta_0}$ for some constant $C > 0$ and the constant $\beta_0$ in (A9*), and $T_n = O(n^{p^*})$ for some constant $0 < p^* < 2/\mathrm{dim}(\mathbb{G})$.

Condition (A8*) is a weak condition. Specifically, (A8*)-(i) holds if $K_\ell$ are symmetric. Condition (A9*) is not overly restrictive either, as it covers various distributions on $\mathbb{R}$ like mean zero normal distributions and Cauchy distributions. Notably, Condition (A9*)-(i) is met by all symmetric distributions on $\mathbb{R}$. It's worth mentioning that Condition (A9*) implies $\varphi^{\mathcal{U}_\ell}(\mathbf{w}_j) > 0$ for every $\mathbf{w}_j \in \mathbb{R}$. Consequently, Condition (K1)-(i) is fulfilled. Additionally, Condition (A9*) along with the mutual independence of $(\varphi^{\mathcal{U}_\ell}: 1 \le \ell \le d)$ implies (S'2) with $\beta^* = \beta_0^*$ and $\gamma = \sqrt{d} \cdot \gamma_0$. Lastly, Condition (A10*) covers the optimal speeds of $h$ and $T_n$ for Theorem 4.9.

**Theorem 5.7.** Assume that the conditions (S1), (A1), and (A8*)-(A10*) are satisfied. For $\tilde{\mathbf{t}} = (\tilde{\mathbf{t}}', \tilde{\mathbf{t}}) \in \mathbb{R}^{dk} \times \mathbb{G}^k$, with $\tilde{\mathbf{t}}'$ satisfying (A11)-(ii) and $\tilde{\mathbf{t}}$ satisfying (A11)-(iii), then it holds that

$$ \sqrt{n} \cdot \frac{\tilde{\mathcal{V}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \bar{\mathcal{R}}^{(k)}(\varphi, \tilde{\mathbf{t}}) - \mathbb{E}(\tilde{\mathcal{W}}_n(\tilde{\mathbf{t}}_1, \bar{\mathbf{Z}}_1, \mathbf{Y}) / \prod_{j=1}^k f_{\bar{\mathbf{Z}}}(\tilde{\mathbf{t}}_j))}{\sqrt{\mathrm{Var}(\tilde{\mathcal{W}}_n(\tilde{\mathbf{t}}_1, \bar{\mathbf{Z}}_1, \mathbf{Y}) / \prod_{j=1}^k f_{\bar{\mathbf{Z}}}(\tilde{\mathbf{t}}_j))}} \xrightarrow{d} N(0, 1). $$

# 6. Applications

While only the following examples will be provided here, they serve as representative models for various problems that can be explored similarly.

## 6.1. Asymptotic confidence intervals

We will establish the asymptotic confidence intervals for $\hat{\mathcal{U}}_n^{(k)}(\varphi, T_n, \cdot)$. To achieve this, we will directly estimate the bias and the variance without delving into their leading terms. The natural approach is to estimate the bias component

$$ \mathbb{E} \left( \mathrm{Re} (K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)) \left( \tilde{\varphi}_n(\mathbf{Y}) - \prod_{j=1}^{k-1} f_{\mathbf{X}}(\mathbf{t}_j) r^{(k)}(\varphi, \tilde{\mathbf{t}}) \right) \right), $$
as shown in Theorem 5.1, as zero. When we substitute

$$n^{-1} \sum_{i=1}^{n} \mathrm{Re} (K_{T_n}(\mathbf{t}_1, \mathbf{Z}_i)) (\tilde{\phi}_n(\mathbf{Y}_i) - \hat{f}_n^{(k-1)}(\tilde{\mathbf{t}}) \hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}))$$

into $\mathbb{E}(\mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_1)) (\tilde{\phi}_n(\mathbf{Y}) - \prod_{j=1}^{k-1} f_X(\mathbf{t}_j) r^{(k)}(\varphi, \tilde{\mathbf{t}})))$, we obtain zero estimates. Moreover,

$$\tilde{\phi}_n(\mathbf{y}) = \iint \varphi(\mathbf{y}, \mathbf{y}_2, \dots, \mathbf{y}_k) \prod_{j=2}^{k} \mathrm{Re}(K_{T_n}(\mathbf{t}_j, \mathbf{z}_j)) d\mathbb{P}_n(\mathbf{z}_j, \mathbf{y}_j), \quad (6.1)$$

where $\mathbb{P}_n$ represents the empirical measure. It is crucial to recall from our proof that the variance dominates the squared bias to justify the zero estimation of biases in constructing confidence intervals based on Theorem 5.1. For the variance $V_n(\mathbf{t}_1, \varphi) = \mathrm{Var}(w_n(\mathbf{t}_1, \mathbf{Z}_1, \mathbf{Y}))$, we estimate it as

$$V_n(\mathbf{t}_1, \varphi) = n^{-1} \sum_{i=1}^{n} \left[ \mathrm{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_i)) (\tilde{\phi}_n(\mathbf{Y}_i) - \hat{f}_n^{(k-1)}(\tilde{\mathbf{t}}) \hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}})) \right]^2.$$

We derive the following consequence from Theorem 5.1.

**Theorem 6.1.** Assuming either (G1) or (G2) are fulfilled, alongside the absolute convergence of the Fourier series of $\Delta^{\hat{m}}(\hat{f}_{\mathbb{X}}(\cdot))$ and $\Delta^{\hat{m}}(r^{(k)}(\varphi, \cdot) \cdot \hat{f}_{\mathbb{X}}(\cdot))$ on $\mathbb{G}^k$, and given that conditions (S1), (A1), (A2), (A3), and (T1') are satisfied, we have that for all $\tilde{\mathbf{t}} \in \mathbb{G}^k$:

$$\sqrt{n} \cdot \frac{\hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})}{\sqrt{V_n(\mathbf{t}_1, \varphi)} / \hat{f}_n^{(k)}(\tilde{\mathbf{t}})} \xrightarrow{d} N(0, 1). \quad (6.2)$$

Hence, a $(1 - \alpha) \times 100\%$ asymptotic confidence interval for $\hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \cdot)$ is given by

$$\left( \hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - z_{\alpha/2} \cdot \frac{(V_n(\mathbf{t}_1, \varphi))^{1/2} / \hat{f}_n^{(k)}(\tilde{\mathbf{t}})}{\sqrt{n}}, \hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) + z_{\alpha/2} \cdot \frac{(V_n(\mathbf{t}_1, \varphi))^{1/2} / \hat{f}_n^{(k)}(\tilde{\mathbf{t}})}{\sqrt{n}} \right),$$

where $z_{\alpha}$ represents the upper $\alpha$-quantile of the standard normal distribution.

## 6.2. Kendall tau

To test the independence of one-dimensional random variables $\mathbf{Y}_1$ and $\mathbf{Y}_2$ [84] proposed a method based on the U-statistic $K_n$ with the following kernel function:

$$\varphi((\mathbf{s}_1, \mathbf{t}_1), (\mathbf{s}_2, \mathbf{t}_2)) = \mathbb{1}_{\{(\mathbf{s}_2 - \mathbf{s}_1)(\mathbf{t}_2 - \mathbf{t}_1) > 0\}} - \mathbb{1}_{\{(\mathbf{s}_2 - \mathbf{s}_1)(\mathbf{t}_2 - \mathbf{t}_1) \le 0\}}. \quad (6.3)$$

The rejection region for this test is of the form $\{\sqrt{n}K_n > \gamma\}$. In extending this method to the multivariate case, we consider testing the conditional independence of $\xi, \eta: Y = (\xi, \eta)$ given $X$. We propose a method based on the conditional U-statistic:

$$\hat{\mathbf{r}}_n^{(2)}(\varphi, \tilde{\mathbf{t}}; h) = \frac{\sum_{i \neq j}^{n} \varphi(\mathbf{Y}_i, \mathbf{Y}_j) K_{T_n}(\mathbf{t}_1, \mathbf{Z}_i) \mathrm{Re} K_{T_n}(\mathbf{t}_2, \mathbf{Z}_j)}{\sum_{i \neq j}^{n} K_{T_n}(\mathbf{t}_1, \mathbf{Z}_i) \mathrm{Re} K_{T_n}(\mathbf{t}_2, \mathbf{Z}_j)},$$

where $\tilde{\mathbf{t}} = (\mathbf{t}_1, \mathbf{t}_2) \in \mathbb{G}^2$ and $\varphi(\cdot)$ is Kendall's kernel (6.3). Assume that $\xi$ and $\eta$ are $k_1$ and $k_2$-dimensional random vectors respectively and $k_1 + k_2 = k$. Also, suppose that $\mathbf{Y}_1, \dots, \mathbf{Y}_n$ are observations of $(\xi, \eta)$, we aim to test the hypothesis:
$H_0: \xi$ and $\eta$ are conditionally independent given $X$. vs $H_a: H_0$ is not true. (6.4)

Let $\mathbf{a} = (\mathbf{a}_1, \mathbf{a}_2) \in \mathbb{R}^k$ such that $\|\mathbf{a}\| = 1$, with $\mathbf{a}_1 \in \mathbb{R}^{k_1}$, $\mathbf{a}_2 \in \mathbb{R}^{k_2}$. Let $F(\cdot), G(\cdot)$ be the distribution functions of $\xi$ and $\eta$ respectively. Assume that $F^{\mathbf{a}_1}(\cdot)$ and $G^{\mathbf{a}_2}(\cdot)$ are continuous for any unit vector $\mathbf{a} = (\mathbf{a}_1, \mathbf{a}_2)$ where $F^{\mathbf{a}_1}(t) = \mathbb{P}(\mathbf{a}_1^\top \xi < t)$ and $G^{\mathbf{a}_2}(t) = \mathbb{P}(\mathbf{a}_2^\top \eta < t)$, with $\mathbf{a}_1^\top$ denoting the transpose of the vector $\mathbf{a}_i, 1 \le i \le 2$. For $n=2$, let $\mathbf{Y}^{(1)} = (\xi^{(1)}, \eta^{(1)})$ and $\mathbf{Y}^{(2)} = (\xi^{(2)}, \eta^{(2)})$ such as $\xi^{(i)} \in \mathbb{R}^{k_1}$ and $\eta^{(i)} \in \mathbb{R}^{k_2}$ for $i=1,2$, define:

$$ \varphi^{\mathbf{a}}(\mathbf{Y}^{(1)}, \mathbf{Y}^{(2)}) = \varphi\left((\mathbf{a}_1^{\top}\xi^{(1)}, \mathbf{a}_2^{\top}\eta^{(1)}), (\mathbf{a}_1^{\top}\xi^{(2)}, \mathbf{a}_2^{\top}\eta^{(2)})\right). $$

From this setup, applying Corollary 4.5 yields the following result.

**Corollary 6.2.** Assume that the conditions (A1) and (A2) hold. Then the following results are valid:

1. Under (S1)(i), (T1') and (F.i) or (F.ii), it holds that

$$ \int_{\mathbb{G}^2} \hat{r}_n^{(2)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}}) |^2 d\mu(\mathbf{t}_1)d\mu(\mathbf{t}_2) = O_P \left( n^{-8\hat{m}/(4\hat{m}+4\beta+\dim(\mathbb{G}))} \right). $$

2. Under (S2)(i), (T2') and (F.i) or (F.ii), it holds that

$$ \int_{\mathbb{G}^2} \hat{r}_n^{(2)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}}) |^2 d\mu(\mathbf{t}_1)d\mu(\mathbf{t}_2) = O_P \left( (\log n)^{-4\hat{m}/\beta} \right). $$

3. Under (S3)(i), (T3'), and (F.i) or (F.ii), it holds that

$$ \int_{\mathbb{G}^2} \hat{r}_n^{(2)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}}) |^2 d\mu(\mathbf{t}_1)d\mu(\mathbf{t}_2) = O_P \left( (\log n / \log \log n)^{-4\hat{m}/\beta} \right). $$

## 6.3. Discrimination problems

We now apply the results to the discrimination problem as outlined in Section 3 of [134], also referring to [132]. We'll maintain similar notation and settings. Let $\varphi(\cdot)$ be any function that takes at most finitely many values, say $1, \dots, M$. The sets

$$ A_j = \{ (\mathbf{y}_1, \dots, \mathbf{y}_k) : \varphi(\mathbf{y}_1, \dots, \mathbf{y}_k) = j \}, \quad 1 \le j \le M, $$

form a partition of the feature space. Predicting the value of $\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k)$ is equivalent to predicting the set in the partition to which $(\mathbf{Y}_1, \dots, \mathbf{Y}_k)$ belongs. For any discrimination rule $g$, we have

$$ \mathbb{P}(g(\mathbf{X}) = \varphi(\mathbf{Y})) \le \sum_{j=1}^{M} \int_{\tilde{\mathbf{x}}:g(\tilde{\mathbf{x}})=j} \max \mathfrak{M}^j(\tilde{\mathbf{t}}) d\mathbb{P}(\tilde{\mathbf{t}}), $$

where

$$ \mathfrak{M}^j(\tilde{\mathbf{x}}) = \mathbb{P}(\varphi(\mathbf{Y}) = j | \mathbf{X} = \tilde{\mathbf{t}}), \quad \tilde{\mathbf{t}} \in \mathbb{G}^k. $$

The inequality above becomes equality if

$$ g_0(\tilde{\mathbf{x}}) = \arg \max_{1 \le j \le M} \mathfrak{M}^j(\tilde{\mathbf{t}}), $$

where $g_0(\cdot)$ is called the Bayes rule, and the corresponding probability of error
$$ \mathbf{L}^* = 1 - \mathbb{P}(g_0(\mathbf{X}) = \varphi(\mathbf{Y})) = 1 - \mathbb{E} \left\{ \max_{1 \le j \le M} \mathfrak{M}^j(\tilde{\mathbf{t}}) \right\}, $$

is known as the Bayes risk. Each of the unknown functions $\mathfrak{M}^j$'s can be consistently estimated by one of the methods discussed in the preceding sections. Let, for $1 \le j \le M$,

$$ \mathfrak{M}_n^j(\tilde{\mathbf{t}}) = \frac{\sum_{(i_1, \dots, i_k) \in I(k,n)} 1\{\varphi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) = j\} \operatorname{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_{i_1})) \cdots \operatorname{Re}(K_{T_n}(\mathbf{t}_k, \mathbf{Z}_{i_k}))}{\sum_{(i_1, \dots, i_k) \in I_n^k} \operatorname{Re}(K_{T_n}(\mathbf{t}_1, \mathbf{Z}_{i_1})) \cdots \operatorname{Re}(K_{T_n}(\mathbf{t}_k, \mathbf{Z}_{i_k}))}, \quad (6.5) $$

and set

$$ g_{0,n}(\tilde{\mathbf{t}}) = \arg \max_{1 \le j \le M} \mathfrak{M}_n^j(\tilde{\mathbf{t}}). $$

Let's define

$$ \mathbf{L}_n^* = \mathbb{P}(g_{0,n}(\mathbf{X}) \neq \varphi(\mathbf{Y})). $$

The discrimination rule $g_{0,n}(\cdot)$ is asymptotically consistent with Bayes' risk:

$$ \mathbf{L}_n^* \to \mathbf{L}^*. $$

This directly follows from the relation

$$ |\mathbf{L}^* - \mathbf{L}_n^*| \le 2\mathbb{E} \left[ \max_{1 \le j \le M} \left| \mathfrak{M}_n^j(\mathbf{X}) - \mathfrak{M}^j(\mathbf{X}) \right| \right]. $$

## 7. Concluding remarks

This paper introduces a robust framework for conditional U-statistics, focusing on Lie group-valued predictors influenced by measurement errors. Our methodology, based on harmonic analysis of Lie groups, explores a largely untapped area in statistical research. We present an innovative deconvolution technique for conditional U-statistics, providing a detailed analysis of its convergence rate and asymptotic distribution. Additionally, we extend our study to include conditional U-statistics for variables that encompass both Euclidean and non-Euclidean components, addressing measurement errors—an area scarcely covered in existing literature. Our comprehensive analysis reveals the asymptotic properties of these conditional U-statistics, including their convergence rates in various modes and asymptotic distributions. These findings are applicable under broad conditions, demonstrating their versatility and robustness. We also establish asymptotic confidence intervals derived from the estimator's asymptotic distribution, highlighting potential applications such as the Kendall rank correlation coefficient and discrimination problems.

However, the optimal selection of smoothing parameters remains an open question and warrants further investigation. Future research could extend our methodology to incomplete data settings, which involve complex mathematical challenges beyond the current scope. Additionally, applying our findings to serially dependent data, particularly under mixing or ergodic conditions, presents a promising direction for further exploration. Change-point detection, essential for identifying abrupt changes in stochastic systems, represents an intriguing yet unexplored application for conditional U-statistics. Investigating this could significantly enhance our understanding of underlying processes, opening exciting new avenues for future research. U-statistics of infinite order have garnered renewed attention in recent statistics and machine learning literature, particularly in the context of uncertainty quantification for Breiman's bagging and random forests. In these applications, tree-based prediction rules are conceptualized as U-statistics with deterministic and random kernels, corresponding to the sub-sample size of the training data. For robust predictions, the sub-sample size k used to construct each tree must increase with the total sample size n. A promising but challenging direction for future
research involves exploring conditional U-statistics of infinite order. This endeavor will necessitate a
novel methodology distinct from that employed in the present study, leaving this complex problem
open for future investigation. In our investigation [21], we have developed oracle inequalities and
upper limits for kernel-based estimators of conditional U-statistics, applicable across a wide range
of metric spaces. The natural question is to obtain the oracle inequalities in the framework of the
present paper.

8. Mathematical developments

This section is devoted to proving our results. The notation previously presented continues to be
used in the following.

8.1. *Proof of Proposition* 4.2

Let us first recall

$$
\hat{f}_n^{(k)}(\tilde{\mathbf{t}}) = \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in \mathcal{I}_n^k} \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{Z}_{i_1}, \dots, \mathbf{Z}_{i_k}).
$$

Denote that $\tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) = \mathrm{Re}(\hat{f}_n^{(k)}(\tilde{\mathbf{t}}))$ and

$$
\begin{align*}
\mathbb{E} (\hat{f}_n^{(k)} (\tilde{\mathbf{t}})) &= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j} \operatorname{Tr} \left( \prod_{j=1}^{k} \mathbb{E} (\sigma_j^M (\mathbb{Z}^{-1})) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \sigma_j^M (\mathbf{t}_j) \right) \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j} \operatorname{Tr} (\phi^{\mathbf{U}_j} (\sigma_j^M) \sigma_j^M (\mathbf{t}_j)).
\end{align*}
$$

This implies that, taking into consideration that the Fourier series of $\tilde{f}_{\mathbb{X}}(\cdot)$ converges uniformly,

$$
\sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \left| \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) - \mathbb{E}(\hat{f}_n^{(k)}(\tilde{\mathbf{t}})) \right| = \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \left| \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^k d_{\sigma_j} \operatorname{Tr} \left( \phi^{\mathbb{X}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j) \right) \right| = o(1). \quad (8.1)
$$

Now, we note that

$$
\begin{align*}
& \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \left| \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) - \mathbb{E}(\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})) \right| \\
& \le \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \left| \operatorname{Tr} \left( \left( \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in \mathcal{I}_n^k} \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \right. \right. \\
& \qquad \left. \left. - \mathbb{E}(\sigma_j^M (\mathbb{Z}^{-1})) \right) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \sigma_j^M (\mathbf{t}_j) \right| \\
& \le \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \left\| \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \sigma_j^M (\mathbf{t}_j) \right\|_{\text{HS}}
\end{align*}
$$
$$ 
\begin{aligned}
& \left\| \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \sigma_j^M (\mathbf{z}_{i_j}^{-1}) - \mathbb{E} (\sigma_j^M (\mathbb{Z}^{-1})) \right\|_{\text{HS}} \\
& \le \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^{3/2} \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} \\
& \qquad \cdot \left\| \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \sigma_j^M (\mathbf{z}_{i_j}^{-1}) - \mathbb{E} (\sigma_j^M (\mathbb{Z}^{-1})) \right\|_{\text{HS}}.
\end{aligned}
$$

Taking expectation on both sides, we obtain that

$$ 
\begin{aligned}
& \mathbb{E} \left( \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \left| \tilde{f}_{\mathbf{X}}(\tilde{\mathbf{t}}) - \mathbb{E} (\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})) \right| \right) \\
& \le \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^{3/2} \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} \\
& \qquad \times \left( \mathbb{E} \left( \left\| \frac{(n-k)!}{n!} \sum_{(i_1, \dots, i_k) \in I_n^k} \sigma_j^M (\mathbf{z}_{i_j}^{-1}) - \mathbb{E} (\sigma_j^M (\mathbb{Z}^{-1})) \right\|_{\text{HS}}^2 \right) \right)^{1/2} \\
& \le \frac{(n-k)!}{n!} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^{3/2} \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}} \left( \mathbb{E} \left( \left\| \sigma_j^M (\mathbb{Z}^{-1}) \right\|_{\text{HS}}^2 \right) \right)^{1/2} \\
& = \frac{(n-k)!}{n!} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi_{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}}.
\end{aligned}
$$

Combining the above result and (8.1) with Lemma 4.1, we get that

$$ \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \left| \hat{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbf{X}}(\tilde{\mathbf{t}}) \right| = o_P(1), $$

for the choices of $T_n$ satisfying the assumption of the proposition. $\square$

## 8.2. Proof of Theorem 4.3

### 8.2.1. Bounded case

Recall the definition $\tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}}) := r^{(k)}(\varphi, \tilde{\mathbf{t}}) \tilde{f}_{\mathbf{X}}(\tilde{\mathbf{t}})$. Making use of the condition (F.i), we denote

$$ 
\begin{aligned}
\tilde{r}_B^{(k)} (\varphi, \tilde{\mathbf{t}}) &:= r_B^{(k)} (\varphi, \tilde{\mathbf{t}}) \tilde{f}_{\mathbf{X}}(\tilde{\mathbf{t}}) := \mathbb{E} (\Psi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) | (\mathbf{X}_1, \dots, \mathbf{X}_k) = \tilde{\mathbf{t}}) \tilde{f}_{\mathbf{X}}(\tilde{\mathbf{t}}) \\
&= \mathbb{E} (\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) \mathbb{1}_{\{\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) \le \mathbf{M}\}} | (\mathbf{X}_1, \dots, \mathbf{X}_k) = \tilde{\mathbf{t}}) \tilde{f}_{\mathbf{X}}(\tilde{\mathbf{t}}).
\end{aligned}
\quad (8.2) $$

For $\varphi: \mathbb{R} \to \mathbb{R}$ measurable and bounded and with $\mathbb{P}\varphi = 0$, let $\varphi_k^\phi (x_1, \dots, x_k) = \varphi (x_1) \cdots \varphi (x_k)$, $k \in \mathbb{N}$. So we obtain that for any symmetric $\varphi$ there are simple functions $\psi_{lj}(\cdot)$ and real numbers $t_{lj}$ such that

$$ \varphi = \lim_{l \to \infty} \sum_{j=1}^{s_l} t_{lj} \varphi_k^{\psi_{lj}} \text{ in } L_2 (\mathbb{P}^k). $$
Let $\phi_{lj} = \psi_{lj} - \mathbb{P}\psi_{lj}$. If moreover $\varphi(\cdot)$ is $\mathbb{P}$-canonical, then $\varphi = \pi_k\varphi = \lim_{l \to \infty} \sum_{j=1}^{s_l l_j} \varphi_k^{\phi_{lj}}$ in $L_2(\mathbb{P}^k)$ because $\pi_k$ is an operator of norm 1 on $L_2$, at this point we refer to [4] and the proof of Lemma 4.1 [133]. By Assumption 3.6, it holds that

$$ \tilde{\tau}_B^{(k)}(\varphi, \tilde{\mathbf{t}}) = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr} \left( \phi_B^{\tilde{\tau}_B^{(k)}} (\sigma_j^M) \sigma_j^M(\mathbf{t}_j) \right). \quad (8.3) $$

According to (8.3), we get that

$$
\begin{align*}
\tilde{U}_n^{(k)}(\Psi, T_n, \tilde{\mathbf{t}}) - \tilde{\tau}_B^{(k)}(\varphi, \tilde{\mathbf{t}}) \\
&= \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in l_n^k} \Psi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr} \left( \sigma_j^M(\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \sigma_j^M(\mathbf{t}_j) \right) \\
&\quad - \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr} \left( \phi_B^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j) \right) \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \\
&\quad \times \mathrm{Tr} \left( \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in l_n^k} \Psi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M(\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} - \phi_B^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) \right) \sigma_j^M(\mathbf{t}_j) \right) \\
&\quad - \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr} \left( \phi_B^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j) \right) \\
&:= \Phi_1(\tilde{\mathbf{t}}) - \Phi_2(\tilde{\mathbf{t}}).
\end{align*}
$$

It's evident that $\Phi_1$ is square integrable. Also, $\Phi_2$ is square integrable because it is the sum of two square-integrable functions $\tilde{\tau}_B^{(k)}(\varphi, \cdot)$ and $\sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr}(\phi_B^{\tilde{\tau}_B^{(k)}}(\sigma_j^M)\sigma_j^M(\cdot))$. We start by studying the following inner product. So, we have

$$
\begin{align*}
\langle \Phi_1, \Phi_2 \rangle_2 &= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \\
&\quad \times \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in l_n^k} \Psi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M(\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} - \phi_B^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) \right)_{j'k'} \\
&\quad \times \left\langle \left( \sigma_j^M \right)_{k'j'}, \sum_{\tau_1, \dots, \tau_k \in \hat{\mathbb{G}}: k_{\tau_j} \ge T_n} \prod_{j=1}^k d_{\tau_j} \sum_{j''=1}^{d_{\tau_j}} \sum_{k''=1}^{d_{\tau_j}} \phi_{k''j''}^{\tilde{\tau}_B^{(k)}}(\tau_j^M) \left( \tau_j^M \right)_{k''j''} \right\rangle_2 \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}
\end{align*}
$$
$$ 
\begin{aligned}
& \times \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \Psi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M(\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} - \phi^{\tilde{\mathbf{r}}_B^{(k)}}(\sigma_j^M) \right)_{j'k'} \\
& \times \sum_{\tau_1, \dots, \tau_k \in \hat{\mathbb{G}}: k_{\tau_j} \ge T_n} \prod_{j=1}^k d_{\tau_j} \sum_{j''=1}^{d_{\tau_j}} \sum_{k''=1}^{d_{\tau_j}} \bar{\phi}_{k''j''}^{\tilde{\tau}_B^{(k)}}(\tau_j^M) \left\langle (\sigma_j^M)_{k'j'}(\cdot), (\tau_j^M)_{k''j''}(\cdot) \right\rangle_2
\end{aligned}
\quad (8.4) $$

where the second equality follows from the fact that $\langle v, \sum_{c=1}^{\infty} v_c \rangle = \sum_{l=1}^{\infty} \langle v, v_c \rangle$ for $v$ and $v_c$ in an inner product space $(V, \langle \cdot, \cdot \rangle)$ such that $\sum_{c=1}^{\infty} v_c \in V$. Therefore, it holds that

$$ \| \tilde{\mathbf{r}}_n^{(k)} - \tilde{\mathbf{r}}_B^{(k)} \|_2^2 = \| \Phi_1 \|_2^2 + \| \Phi_2 \|_2^2. $$

Now, we approximate $\mathbb{E}(\|\Phi_1\|_2^2)$ and $\|\Phi_2\|_2^2$. According to Proposition 3.4, we can see that

$$ 
\begin{aligned}
& \prod_{j=1}^{k} \mathbb{E} \left[ \sigma_j^M \left( \mathbf{Z}_j^{-1} \right) \Psi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) \right] \\
&= \prod_{j=1}^{k} \mathbb{E} \left[ \mathbb{E} \left( \sigma_j^M \left( \mathbf{Z}_i^{-1} \right) \Psi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) \mid (\mathbf{X}_1, \dots, \mathbf{X}_k) \right) \right] \\
&= \prod_{j=1}^{k} \mathbb{E} \left[ \mathbb{E} \left( \sigma_j^M \left( \mathbf{Z}_i^{-1} \right) \mid (\mathbf{X}_1, \dots, \mathbf{X}_k) \right) \tilde{\mathbf{r}}_B^{(k)}(\varphi, \tilde{\mathbf{t}}) \right] \\
&= \prod_{j=1}^{k} \mathbb{E} \left[ \sigma_j^M \left( \mathbf{X}_i^{-1} \right) \tilde{\mathbf{r}}_B^{(k)}(\varphi, \tilde{\mathbf{t}}) \right] \phi^{\mathbf{U}}(\sigma_j^M) = \phi^{\tilde{\mathbf{r}}_B^{(k)}}(\sigma_j^M) \phi^{\mathbf{U}_j}(\sigma_j^M). \quad (8.5)
\end{aligned}
$$

Therefore, making use of (8.5), we infer that

$$ 
\begin{aligned}
\mathbb{E}(\|\Phi_1\|_2^2) &= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \\
&\quad \times \mathbb{E} \left( \left\| \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \Psi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M(\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} - \phi^{\tilde{\mathbf{r}}_B^{(k)}}(\sigma_j^M) \right\|_{\text{HS}}^2 \right) \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \\
&\quad \times \operatorname{Var} \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \Psi(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M(\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \right) \\
&\le \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \mathbb{E} \left( \|\sigma_j^M(\mathbf{Z}^{-1})\phi^{\mathbf{U}_j}(\sigma_j^M)^{-1}\|_{\text{HS}}^2 \Psi^2(\mathbf{Y}) \right) \\
&= \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \mathbb{E} \left( \left\| (\sigma_j^M(\mathbf{Z}^{-1}))^* (\phi^{\mathbf{U}_j}(\sigma_j^M)^{-1})^* \right\|_{\text{HS}}^2 \Psi^2(\mathbf{Y}) \right)
\end{aligned}
$$
$$
\begin{align*}
&\le \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \\
&\qquad \times \left\| \left(\phi_{\mathbf{U}_j}(\sigma_j^M)^{-1}\right)^* \right\|_{\mathrm{op}}^2 \mathbb{E} \left( \left\| \left(\sigma_j^M(Z^{-1})\right)^* \right\|_{\mathrm{HS}}^2 \Psi^2(Y) \right) \\
&= \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \left\| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\mathrm{op}}^2 \mathbb{E} \left( \left\| \sigma_j^M(Z^{-1}) \right\|_{\mathrm{HS}}^2 \Psi^2(Y) \right) \\
&= \frac{(n-k)!^k}{n!^k} \left( \mathbb{E}(\Psi^2(Y)) \right)^k \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\mathrm{op}}^2 \\
&\le \frac{(n-k)!^k}{n!^k} \mathbb{M}^{2k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi_{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\mathrm{op}}^2, \tag{8.6}
\end{align*}
$$

where the first equality is a consequence of Lemma 9.1, the third and fourth equalities follow from the
invariance of $\| \cdot \|_{\text{op}}$ and $\| \cdot \|_{\text{HS}}$ concerning the conjugate transpose, the second inequality arises from
the fact that $\|AB\|_{\text{HS}} \le \|A\|_{\text{op}}\|B\|_{\text{HS}}$ for all complex square matrices A and B, and the last equality
follows from Lemma 9.2-(ii). Now, we approximate $\|\Phi_2\|_2^2$. According to Lemma 1 in [76], we have

$$
\begin{align*}
\|\Phi_2\|_2^2 &= \left\langle \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j \ge T_n} \prod_{j=1}^k d_{\sigma_j} \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \phi_{j'k'}^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) (\sigma_j^M)_{k'j'}(\cdot), \sum_{\tau_1, \dots, \tau_k \in \hat{\mathbb{G}}: k\tau_j \ge T_n} \prod_{j=1}^k d_{\tau_j} \right. \\
&\qquad \left. \times \sum_{j''=1}^{d_{\tau_j}} \sum_{k''=1}^{d_{\tau_j}} \phi_{j''k''}^{\tilde{\tau}_B^{(k)}}(\tau_j^M) (\tau_j^M)_{k''j''}(\cdot) \right\rangle_2 \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j \ge T_n} \prod_{j=1}^k d_{\sigma_j} \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \sum_{\tau_1, \dots, \tau_k \in \hat{\mathbb{G}}: k\tau_j \ge T_n} \prod_{j=1}^k d_{\tau_j} \\
&\qquad \times \sum_{j''=1}^{d_{\tau_j}} \sum_{k''=1}^{d_{\tau_j}} \phi_{j'k'}^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) \bar{\phi}_{j''k''}^{\tilde{\tau}_B^{(k)}}(\tau_j^M) \left\langle (\sigma_j^M)_{k'j'}(\cdot), (\tau_j^M)_{k''j''}(\cdot) \right\rangle_2 \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j \ge T_n} \prod_{j=1}^k d_{\sigma_j} \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \sum_{\tau_1, \dots, \tau_k \in \hat{\mathbb{G}}: k\tau_j \ge T_n} \prod_{j=1}^k d_{\tau_j} \\
&\qquad \times \sum_{j''=1}^{d_{\tau_j}} \sum_{k''=1}^{d_{\tau_j}} \phi_{j'k'}^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) \bar{\phi}_{j''k''}^{\tilde{\tau}_B^{(k)}}(\tau_j^M) \int_{\mathbb{G}^k} (\sigma_j^M)_{k'j'}(\mathbf{t}_j) (\bar{\tau}_j^M)_{k''j''}(\mathbf{t}_j) d\mu(\mathbf{t}_j) \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k\sigma_j \ge T_n} \prod_{j=1}^k d_{\sigma_j} \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \phi_{j'k'}^{\tilde{\tau}_B^{(k)}}(\sigma_j^M) \bar{\phi}_{j''k''}^{\tilde{\tau}_B^{(k)}}(\sigma_j^M)
\end{align*}
$$
$$
\begin{align}
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}} : k\sigma_j \ge T_n} \prod_{j=1}^k d_{\sigma_j} \left\| \phi^{\tilde{r}_B^{(k)}} (\sigma_j^M) \right\|_{\text{HS}}^2 \nonumber \\
&= T_n^{-2k\hat{m}} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}} : k\sigma_j \ge T_n} \prod_{j=1}^k d_{\sigma_j} \frac{T_n^{2r}}{k^{2\hat{m}} \sigma_j} \left\| (-1)^{\hat{m}} k^{\hat{m}} \phi^{\tilde{r}_B^{(k)}} (\sigma_j^M) \right\|_{\text{HS}}^2 \nonumber \\
&\le T_n^{-2k\hat{m}} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}} : k\sigma_j \ge T_n} \prod_{j=1}^k d_{\sigma_j} \left\| \phi^{\Delta^{\hat{m}}(\tilde{r}_B^{(k)})} (\sigma_j^M) \right\|_{\text{HS}}^2 \nonumber \\
&\le T_n^{-2k\hat{m}} \left( \left\| \Delta^{\tilde{r}_B^{(k)}} (\sigma_j^M) \right\|_2^2 \right)^k = O \left( T_n^{-2k\hat{m}} \right), \tag{8.7}
\end{align}
$$

where the sixth equality follows from (3.1.5) in [3] and the second inequality follows from Theorem 6.4.2 in [52]. Combining (8.5) and (8.7) with Lemma 4.1, we arrive at

$$
\begin{align}
& \left\| \tilde{U}_n^{(k)} (\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}_B^{(k)} (\varphi, \tilde{\mathbf{t}}) \right\|_2^2 \nonumber \\
& = \begin{cases}
O_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)^{tk}}{n!^k} T_n^{2k\beta + \frac{k\dim(\mathbb{G})}{2}} \right), & \text{for (S1)-(i),} \\
O_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)^{tk}}{n!^k} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^\beta \right) \right), & \text{for (S2)-(i),} \\
O_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)^{tk}}{n!^k} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^\beta (\log T_n - \xi_1) \right) \right), & \text{for (S3)-(i).}
\end{cases} \tag{8.8}
\end{align}
$$

Now, we define $\hat{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \mathrm{Re}(\tilde{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}))$ and note that

$$
\hat{r}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r_n^{(k)}(\varphi, \tilde{\mathbf{t}}) = \frac{\hat{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}})}{\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})} - \frac{\tilde{r}_B^{(k)}(\tilde{\mathbf{t}})}{\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})}. \quad (8.9)
$$

Notice that

$$
\left\| \hat{U}_n^{(k)} (\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}_B^{(k)} (\varphi, \tilde{\mathbf{t}}) \right\|_2^2 \le \left\| \tilde{U}_n^{(k)} (\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}_B^{(k)} (\varphi, \tilde{\mathbf{t}}) \right\|_2^2. \quad (8.10)
$$

So, it's evident that orders for $\|\tilde{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}})\|_2^2$, represented in (8.8), also hold true for $\|\hat{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}})\|_2^2$. Likewise, we can demonstrate their validity for $\|\tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})\|_2^2$.
Now, we note that

$$
\begin{align*}
\inf_{\tilde{\mathbf{t}} \in \mathbb{G}^k} |\tilde{f}_n^{(k)}(\tilde{\mathbf{t}})| &\ge \inf_{\tilde{\mathbf{t}} \in \mathbb{G}^k} (\tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) - |\tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})|) \\
&\ge \inf_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) - \sup_{\tilde{\mathbf{t}} \in \mathbb{G}^k} |\tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})|.
\end{align*}
$$

Making use Proposition 4.2 and the fact that $\inf_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) > 0$, there exists a constant $C > 0$ such that $\inf_{\tilde{\mathbf{t}} \in \mathbb{G}^k} \tilde{f}_n^{(k)}(\tilde{\mathbf{t}}) > C$, with probability tending to one. So we arrive at

$$
\int_{\mathbb{G}^k} \left| \hat{r}_n^{(k)} (\varphi, T_n, \tilde{\mathbf{t}}) - r_n^{(k)} (\varphi, \tilde{\mathbf{t}}) \right|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k)
$$
$$
\begin{align}
&\le 2 \int_{\mathbb{G}^k} \frac{|\hat{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}})|^2}{|\hat{f}_n^{(k)}(\tilde{\mathbf{t}})|^2} + \frac{r_B^{(k)}(\varphi, \tilde{\mathbf{t}})^2 |\hat{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})|^2}{|\hat{f}_n^{(k)}(\tilde{\mathbf{t}})|^2} d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) \nonumber \\
&\le C \cdot \left( \left\| \hat{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}}) \right\|_2^2 + \left\| \hat{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) \right\|_2^2 \right), \tag{8.11}
\end{align}
$$

with probability tending to one. This is enough to complete the proof for the bounded case.

8.2.2. *Unbounded case*

Now, we shall prove the Theorem 4.3 under assumption (F.ii). To do this, we must truncate our conditional U-statistic. Let $\gamma_n$ represent an increasing sequence of positive numbers such that $\gamma_n \to \infty$ as $n \to \infty$. We set, for any function $\varphi$ and $\tilde{\mathbf{t}} \in \mathbb{G}^k$,

$$
G_{\varphi, T_n, \mathbf{t}}(\tilde{\mathbf{z}}, \tilde{\mathbf{y}}) = G_{\varphi, T_n, \mathbf{t}}(\tilde{\mathbf{z}}, \tilde{\mathbf{y}}) 1_{\{\varphi(\tilde{\mathbf{y}}) \le \gamma_n\}} + G_{\varphi, T_n, \mathbf{t}}(\tilde{\mathbf{z}}, \tilde{\mathbf{y}}) 1_{\{\varphi(\tilde{\mathbf{y}}) > \gamma_n\}} =: G_{\varphi, T_n, \mathbf{t}}^{(T)}(\tilde{\mathbf{z}}, \tilde{\mathbf{y}}) + G_{\varphi, T_n, \mathbf{t}}^{(R)}(\tilde{\mathbf{z}}, \tilde{\mathbf{y}}),
$$

which means that we truncate the function $\varphi$ as follows:

$$
\varphi(\tilde{\mathbf{y}}) = \varphi(\mathbf{y}) 1_{\{\varphi(\tilde{\mathbf{y}}) \le \gamma_n\}} + \varphi(\mathbf{y}) 1_{\{\varphi(\tilde{\mathbf{y}}) > \gamma_n\}} = \varphi^{(T)}(\tilde{\mathbf{y}}) + \varphi^{(R)}(\mathbf{y}). \quad (8.12)
$$

According to (8.12), we can see that

$$
\begin{align}
\tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}}) &= \mathbb{E} \left( \varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) 1_{\{\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) \le \gamma_n\}} \middle| (\mathbf{X}_1, \dots, \mathbf{X}_k) = \tilde{\mathbf{t}} \right) \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) \nonumber \\
&\quad + \mathbb{E} \left( \varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) 1_{\{\varphi(\mathbf{Y}_1, \dots, \mathbf{Y}_k) > \gamma_n\}} \middle| (\mathbf{X}_1, \dots, \mathbf{X}_k) = \tilde{\mathbf{t}} \right) \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}}) \nonumber \\
&=: \tilde{r}^{(T)}(\varphi, \tilde{\mathbf{t}}) + \tilde{r}^{(R)}(\varphi, \tilde{\mathbf{t}}). \tag{8.13}
\end{align}
$$

We can also see, $\tilde{U}_n^{(k)}(G_{\varphi, T_n, \tilde{\mathbf{t}}})$, described in (3.10), can be decomposed as follows,

$$
\tilde{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \tilde{U}_n^{(k)}(G_{\varphi, T_n, \tilde{\mathbf{t}}}^{(T)}) + \tilde{U}_n^{(k)}(G_{\varphi, T_n, \tilde{\mathbf{t}}}^{(R)}) =: \tilde{U}_n^{(T)}(\varphi, T_n, \tilde{\mathbf{t}}) + \tilde{U}_n^{(R)}(\varphi, T_n, \tilde{\mathbf{t}}). \quad (8.14)
$$

We call the term of the right side, $\tilde{r}_n^{(T)}(\varphi, T_n, \tilde{\mathbf{t}})$, truncated part and the second one, $\tilde{r}_n^{(R)}(\varphi, T_n, \tilde{\mathbf{t}})$, remainder part. First, we start with $\tilde{r}_n^{(T)}(\varphi, T_n, \tilde{\mathbf{t}})$.

Truncated part

According to Assumption 3.6, it is established that

$$
\tilde{r}^{(T)}(\varphi, \tilde{\mathbf{t}}) = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^{k} d_{\sigma_j} \operatorname{Tr} \left( \phi^{\tilde{r}^{(T)}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j) \right). \quad (8.15)
$$

Making use of (8.15), it is plain to see that

$$
\begin{align*}
\tilde{U}_n^{(T)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(T)}(\varphi, \tilde{\mathbf{t}}) \\
&= \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \varphi^{(T)}(\mathbf{Y}_{i1}, \dots, \mathbf{Y}_{i_k}) \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \operatorname{Tr} \left( \sigma_j^M (\mathbf{Z}_{ij}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \sigma_j^M(\mathbf{t}_j) \right) \\
&\quad - \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^k d_{\sigma_j} \operatorname{Tr} \left( \phi^{\tilde{r}^{(T)}}(\sigma_j^M) \sigma_j^M(\tilde{\mathbf{t}}) \right) \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \operatorname{Tr} \left( \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \varphi^{(T)}(\mathbf{Y}_{i1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M (\mathbf{Z}_{ij}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \right) \right).
\end{align*}
$$
$$ -\phi^{\tilde{r}^{(T)}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j) - \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr}(\phi^{\tilde{r}^{(T)}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j)) \\ =: \Phi_1^{(T)}(\tilde{\mathbf{t}}) - \Phi_2^{(T)}(\tilde{\mathbf{t}}). $$

In the same manner, as (8.4), we can demonstrate that $\langle \Phi_1^{(T)}, \Phi_2^{(T)} \rangle_2 = 0$. This leads to

$$ \| \tilde{U}_n^{(T)} - \tilde{r}^{(T)} \|_2^2 = \| \Phi_1^{(T)} \|_2^2 + \| \Phi_2^{(T)} \|_2^2. $$

Now, we aim to approximate $\mathbb{E}(\|\Phi_1^{(T)}\|_2^2)$ and $\|\Phi_2^{(T)}\|_2^2$. We start by dealing with $\mathbb{E}(\|\Phi_1^{(T)}\|_2^2)$.

Similarly to (8.5), we can observe that

$$ \begin{aligned} \mathbb{E} \left[ \prod_{j=1}^{k} \sigma_j^M (\mathbf{Z}_j^{-1}) \varphi^{(T)} (\mathbf{Y}_1, \dots, \mathbf{Y}_k) \right] &= \mathbb{E} \left[ \mathbb{E} \left( \sigma_j^M (\mathbf{Z}_i^{-1}) \varphi^{(T)} (\mathbf{Y}_1, \dots, \mathbf{Y}_k) \mid (\mathbf{X}_1, \dots, \mathbf{X}_k) \right) \right] \\ &= \mathbb{E} \left[ \mathbb{E} \left( \sigma_j^M (\mathbf{Z}_i^{-1}) \mid (\mathbf{X}_1, \dots, \mathbf{X}_k) \right) \tilde{r}^{(T)} (\varphi, \tilde{\mathbf{t}}) \right] \\ &= \phi^{\tilde{r}^{(T)}} (\sigma_j^M) \phi^{\mathbb{U}} (\sigma_j^M). \end{aligned} \quad (8.16) $$

Thus, making use of (8.16) and Lemmas 9.1 and 9.2-(ii), we get that

$$ \begin{aligned} \mathbb{E} \left( \| \Phi_1^{(T)} \|_2^2 \right) &= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \\ &\quad \times \mathbb{E} \left( \left\| \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \varphi^{(T)} (\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} - \phi^{\tilde{r}^{(T)}} (\sigma_j^M) \right\|_{\mathrm{HS}}^2 \right) \\ &= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \mathrm{Var} \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \varphi^{(T)} (\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \right) \\ &\le \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \mathbb{E} \left( \| \sigma_j^M (\mathbf{Z}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \|_{\mathrm{HS}}^2 (\varphi^{(T)} (\mathbf{Y}))^2 \right) \\ &\le \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \left\| (\phi^{\mathbf{U}_j} (\sigma_j^M)^{-1})^* \right\|_{\mathrm{op}}^2 \\ &\quad \times \mathbb{E} \left( \left\| (\sigma_j^M (\mathbf{Z}^{-1}))^* \right\|_{\mathrm{HS}}^2 (\varphi^{(T)} (\mathbf{Y}))^2 \right) \\ &= \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \left\| \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\mathrm{op}}^2 \mathbb{E} \left( \left\| \sigma_j^M (\mathbf{Z}^{-1}) \right\|_{\mathrm{HS}}^2 (\varphi^{(T)} (\mathbf{Y}))^2 \right) \end{aligned} $$
$$ 
\begin{aligned}
&= \frac{(n-k)!^k}{n!^k} \left( \mathbb{E} \left[ \left( \varphi^{(T)}(\mathbf{Y}) \right)^2 \right] \right)^k \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\text{op}}^2 \\
&\le \frac{(n-k)!^k}{n!^k} \theta_{\delta}^{\frac{2k}{2+\delta}} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \right\|_{\text{op}}^2.
\end{aligned} 
\quad (8.17) 
$$

Next, we approximate $\|\Phi_2^{(T)}\|_2^2$. In a similar way as (8.7), we infer that, by utilizing Lemma 1 in [76],

$$ 
\begin{aligned}
& \| \Phi_2^{(T)} \|_2^2 \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^k d_{\sigma_j} \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \sum_{\tau_1, \dots, \tau_k \in \hat{\mathbb{G}}: k_{\tau_j} \ge T_n} \\
& \qquad \times \prod_{j=1}^k d_{\tau_j} \sum_{j''=1}^{d_{\tau_j}} \sum_{k''=1}^{d_{\tau_j}} \phi_{j'k'}^{\tilde{\tau}^{(T)}}(\sigma_j^M) \bar{\phi}_{j''k''}^{\tilde{\tau}^{(T)}}(\tau_j^M) \langle (\sigma_j^M)_{k'j'}(\cdot), (\tau_j^M)_{k''j''}(\cdot) \rangle_2 \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^k d_{\sigma_j} \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \sum_{\tau_1, \dots, \tau_k \in \hat{\mathbb{G}}: k_{\tau_j} \ge T_n} \\
& \qquad \times \prod_{j=1}^k d_{\tau_j} \sum_{j''=1}^{d_{\tau_j}} \sum_{k''=1}^{d_{\tau_j}} \phi_{j'k'}^{\tilde{\tau}^{(T)}}(\sigma_j^M) \bar{\phi}_{j''k''}^{\tilde{\tau}^{(T)}}(\tau_j^M) \int_{\hat{\mathbb{G}}^k} (\sigma_j^M)_{k'j'}(\mathbf{t}_j) (\bar{\tau}_j^M)_{k''j''}(\mathbf{t}_j) d\mu(\mathbf{t}_j)
\end{aligned}
$$

$$ = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^k d_{\sigma_j} \sum_{j'=1}^{d_{\sigma_j}} \sum_{k'=1}^{d_{\sigma_j}} \phi_{j'k'}^{\tilde{\tau}^{(T)}}(\sigma_j^M) \bar{\phi}_{j''k''}^{\tilde{\tau}^{(T)}}(\sigma_j^M) $$

$$ = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^{k} d_{\sigma_j} \left\| \phi^{\tilde{\tau}^{(T)}}(\sigma_j^M) \right\|_{\text{HS}}^2 $$

$$ = T_n^{-2k\hat{m}} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^{k} d_{\sigma_j} \frac{T_n^{2\hat{m}}}{k^{2\hat{m}} \sigma_j} \left\| (-1)^{\hat{m}} k \hat{m}_{\sigma_j} \phi^{\tilde{\tau}^{(T)}}(\sigma_j^M) \right\|_{\text{HS}}^2 $$

$$ \le T_n^{-2k\hat{m}} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^{k} d_{\sigma_j} \left\| \phi^{\Delta^{\hat{m}}(\tilde{\tau}^{(T)})}(\sigma_j^M) \right\|_{\text{HS}}^2 $$

$$ \le T_n^{-2k\hat{m}} \left( \left\| \Delta^{\tilde{\tau}^{(T)}}(\sigma^M) \right\|_2^2 \right)^k = O \left( T_n^{-2k\hat{m}} \right). \quad (8.18) $$

It is essential to know that the fifth equality in (8.18) is a consequence of (3.1.5) in [3] whereas the second inequality is derived from Theorem 6.4.2 in [52]. By integrating (8.16) and (8.18) with Lemma 4.1, we obtain that

$$ \| \tilde{U}_n^{(T)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{\tau}^{(T)}(\varphi, \tilde{\mathbf{t}}) \|_2^2 $$
$$ = \begin{cases} \mathbb{O}_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)!k}{n!k} T_n^{2k\beta + \frac{k\dim(\mathbb{G})}{2}} \right), & \text{for (S1)-(i),} \\ \mathbb{O}_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)!k}{n!k} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^{\beta} \right) \right), & \text{for (S2)-(i),} \\ \mathbb{O}_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)!k}{n!k} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp \left( 2 \cdot k \cdot \gamma \cdot T_n^{\beta} (\log T_n - \xi_1) \right) \right), & \text{for (S3)-(i).} \end{cases} \quad (8.19) $$

Hence, the proof for the truncated part is complete.

## Remainder part

Under Assumption 3.6, as (8.15), it holds that

$$ \tilde{\Gamma}^{(R)}(\varphi, \tilde{\mathbf{t}}) = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^{k} d_{\sigma_j} \mathrm{Tr} \left( \phi^{\tilde{\Gamma}^{(R)}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j) \right). \quad (8.20) $$

By using (8.20), we have

$$ \begin{aligned} & \tilde{U}_n^{(R)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{\Gamma}^{(R)}(\varphi, \tilde{\mathbf{t}}) \\ &= \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \varphi^{(R)}(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr} \left( \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \sigma_j^M(\mathbf{t}_j) \right) \\ & \quad - \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}} \prod_{j=1}^k d_{\sigma_j} \mathrm{Tr} \left( \phi^{\tilde{\Gamma}^{(R)}}(\sigma_j^M) \sigma_j^M(\tilde{\mathbf{t}}) \right) \end{aligned} $$

$$ = \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} < T_n} \prod_{j=1}^{k} d_{\sigma_j} \mathrm{Tr} \left( \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in I_n^k} \varphi^{(R)}(\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j}(\sigma_j^M)^{-1} \right. \right. \\ & \quad \left. - \phi^{\tilde{\Gamma}^{(R)}}(\sigma_j^M) \right) \sigma_j^M(\mathbf{t}_j) \\ & \quad - \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}: k_{\sigma_j} \ge T_n} \prod_{j=1}^{k} d_{\sigma_j} \mathrm{Tr} \left( \phi^{\tilde{\Gamma}^{(R)}}(\sigma_j^M) \sigma_j^M(\mathbf{t}_j) \right) $$

$$ =: \Phi_1^{(R)}(\tilde{\mathbf{t}}) - \Phi_2^{(R)}(\tilde{\mathbf{t}}). $$

Similarly, following the approach used for (8.4), we can prove that $\langle \Phi_1^{(R)}, \Phi_2^{(R)} \rangle_2 = 0$, thereby leading to

$$ \| \tilde{U}_n^{(R)} - \tilde{\Gamma}^{(R)} \|_2^2 = \| \Phi_1^{(R)} \|_2^2 + \| \Phi_2^{(R)} \|_2^2. $$

So, our objective is to approximate $\mathbb{E}(\|\Phi_1^{(R)}\|_2^2)$ and $\|\Phi_2^{(R)}\|_2^2$. Let us start with $\mathbb{E}(\|\Phi_1^{(R)}\|_2^2)$. Analogously to (8.5), we can show that

$$ \mathbb{E}[\sigma_j^M ((\mathbf{Z}_j)^{-1}) \varphi^{(R)} (\mathbf{Y}_1, \dots, \mathbf{Y}_k)] = \phi^{\tilde{\Gamma}^{(R)}} (\sigma_j^M) \phi^{\mathbb{U}} (\sigma_j^M). \quad (8.21) $$

Notice that from (8.16) and Lemmas 9.1 and 9.2-(ii), we infer that by taking Advantage of the Condition (F.ii),
$$
\begin{align*}
& \mathbb{E} \left( \left\| \Phi_1^{(R)} \right\|_2^2 \right) \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \\
&\quad \times \mathbb{E} \left( \left\| \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in \hat{\mathbb{I}}_n^k} \varphi^{(R)} (\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} - \phi^{\tilde{r}^{(T)}} (\sigma_j^M) \right\|_{\text{HS}}^2 \right) \\
&= \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \text{Var} \left( \frac{(n-k)!}{n!} \sum_{\mathbf{i} \in \hat{\mathbb{I}}_n^k} \varphi^{(R)} (\mathbf{Y}_{i_1}, \dots, \mathbf{Y}_{i_k}) \sigma_j^M (\mathbf{Z}_{i_j}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \right) \\
&\le \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \mathbb{E} \left( \left\| \sigma_j^M (\mathbf{Z}^{-1}) \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{HS}}^2 (\varphi^{(R)} (\mathbf{Y}))^2 \right) \\
&\le \frac{(n-k)!^k}{n!^k} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j} \left\| (\phi^{\mathbf{U}_j} (\sigma_j^M)^{-1})^* \right\|_{\text{op}}^2 \\
&\quad \times \mathbb{E} \left( \left\| (\sigma_j^M (\mathbf{Z}^{-1}))^* \right\|_{\text{HS}}^2 (\varphi^{(R)} (\mathbf{Y}))^2 \right) \\
&= \frac{(n-k)!^k}{n!^k} \left( \mathbb{E} \left[ (\varphi^{(R)} (\mathbf{Y}))^2 \right] \right)^k \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}}^2 \\
&= \frac{(n-k)!^k}{n!^k} \theta_{\delta}^{\frac{2k}{2+\delta}} \sum_{\sigma_1, \dots, \sigma_k \in \hat{\mathbb{G}}; k \sigma_j < T_n} \prod_{j=1}^k d_{\sigma_j}^2 \left\| \phi^{\mathbf{U}_j} (\sigma_j^M)^{-1} \right\|_{\text{op}}^2 . \tag{8.22}
\end{align*}
$$

Similarly, for $\|\Phi_2^{(R)}\|_2^2$, following the same reasoning as for (8.18), we obtain that

$$
\| \Phi_2^{(R)} \|_2^2 = O \left( T_n^{-2k\hat{m}} \right). \tag{8.23}
$$

Thus, an application of Lemma 4.1 in combination with (8.22) and (8.23), we readily obtain that

$$
\begin{align}
& \| \tilde{U}_n^{(R)} (\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(R)} (\varphi, \tilde{\mathbf{t}}) \|_2^2 \nonumber \\
&= \begin{cases}
O_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)!^k}{n!^k} T_n^{2k\beta + \frac{k\dim(\mathbb{G})}{2}} \right), & \text{for (S1)-(i),} \\
O_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)!^k}{n!^k} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp \left(2 \cdot k \cdot \gamma \cdot T_n^\beta \right) \right), & \text{for (S2)-(i),} \\
O_{\mathbb{P}} \left( T_n^{-2k\hat{m}} + \frac{(n-k)!^k}{n!^k} T_n^{2k\alpha + \frac{k\dim(\mathbb{G})}{2}} \exp \left(2 \cdot k \cdot \gamma \cdot T_n^\beta (\log T_n - \xi_1) \right) \right), & \text{for (S3)-(i).}
\end{cases} \tag{8.24}
\end{align}
$$

Recall that $\hat{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \text{Re}(\tilde{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}))$. Therefore, making use of the decompositions (8.9) and (8.10), we get that
$$
\begin{align*}
& \int_{\mathbb{G}^k} |\hat{\mathbf{r}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - r^{(k)}(\varphi, \tilde{\mathbf{t}})|^2 d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) \\
& \le 2 \int_{\mathbb{G}^k} \frac{|\hat{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}})|^2}{|\hat{f}_n^{(k)}(\tilde{\mathbf{t}})|^2} + \frac{r^{(k)}(\varphi, \tilde{\mathbf{t}})^2 |\hat{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})|^2}{|\hat{f}_n^{(k)}(\tilde{\mathbf{t}})|^2} d\mu(\mathbf{t}_1) \cdots d\mu(\mathbf{t}_k) \\
& \le C \cdot \left( \|\tilde{U}_n^{(T)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(T)}(\varphi, \tilde{\mathbf{t}})\|_2^2 + \|\tilde{U}_n^{(R)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(R)}(\varphi, \tilde{\mathbf{t}})\|_2^2 \right. \\
& \qquad \left. + \|\hat{f}_n^{(k)}(\tilde{\mathbf{t}}) - \tilde{f}_{\mathbb{X}}(\tilde{\mathbf{t}})\|_2^2 \right),
\end{align*}
$$

with probability tending to one. Hence, the proof of the theorem is complete. □

8.3. *Proof of Theorem 4.9*

First, define $\tilde{\mathcal{U}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) = \mathrm{Re}(\overline{U}_n(\varphi, T_n, \tilde{\mathbf{t}}))$. We find the rate of $\overline{U}_n(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}})$, where $\tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}}) := r^{(k)}(\varphi, \tilde{\mathbf{t}}) \prod_{j=1}^k f_{\tilde{\mathbf{X}}}(\tilde{\mathbf{t}}_j)$ and $\overline{U}_n(\varphi, T_n, \tilde{\mathbf{t}})$ is described in (4.7). Subsequently, the same rate applies to $\tilde{\mathcal{U}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}})$ since

$$
|\tilde{\mathcal{U}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}})| + |\mathrm{Im}(\tilde{\mathcal{U}}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}))| = |\overline{U}_n^{(k)}(\varphi, T_n, \tilde{\mathbf{t}}) - \tilde{r}^{(k)}(\varphi, \tilde{\mathbf{t}})|. \quad (8.25)
$$

8.3.1. *Bounded case*

Making use of the condition (F.i), we first determine the rate of the bias $\mathbb{E}(\overline{U}_n^{(k)}(\Psi, T_n, \tilde{\mathbf{t}})) - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}})$. So, we have

$$
\begin{align*}
& \mathbb{E} \left( \overline{U}_n^{(k)} (\Psi, T_n, \tilde{\mathbf{t}}) \right) - \tilde{r}_B^{(k)} (\varphi, \tilde{\mathbf{t}}) \\
&= \frac{1}{h^{dk}} \int_{\mathbb{G}^k \times \mathbb{R}^{dk}} \int_{\mathbb{R}^{dk}} \Psi(\mathbf{s}_1, \dots, \mathbf{s}_k) \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^k K_h^D \\
&\quad \times \left( \frac{\mathbf{t}'_j - \mathbf{v}_j}{h} \right) f_{\tilde{\mathbf{X}}Y}((\mathbf{v}_j, \mathbf{u}_j), \mathbf{s}_j) d\mathbf{v}_j d\mu(\mathbf{u}_j) d\mathbf{s}_j - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}}) \\
&= \frac{1}{h^{dk}} \int_{\mathbb{G}^k \times \mathbb{R}^{dk}} \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^k K_h^D \\
&\quad \times \left( \frac{\mathbf{t}'_j - \mathbf{v}_j}{h} \right) \tilde{r}_B^{(k)}(\varphi, (\tilde{\mathbf{v}}, \tilde{\mathbf{u}})) f_{\tilde{\mathbf{X}}} (\mathbf{v}_j, \mathbf{u}_j) d\mathbf{v}_j d\mu(\mathbf{u}_j) - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}}) \\
&= \int_{\mathbb{G}^k \times \mathbb{R}^{dk}} \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^k K_h^D (\mathbf{w}_j) \tilde{r}_B^{(k)}(\varphi, (\tilde{\mathbf{t}} - h\tilde{\mathbf{w}}, \tilde{\mathbf{u}})) (f_{\tilde{\mathbf{X}}} (\mathbf{t}_j - h\mathbf{w}_j, \mathbf{u}_j) \\
&\quad - f_{\tilde{\mathbf{X}}} (\mathbf{t}_j, \mathbf{u}_j)) d\mathbf{w}_j d\mu(\mathbf{u}_j) \\
&\quad + \int_{\mathbb{G}^k} \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \tilde{r}_B^{(k)}(\varphi, (\tilde{\mathbf{v}}, \tilde{\mathbf{u}})) f_{\tilde{\mathbf{X}}} (\mathbf{t}_j, \mathbf{u}_j) d\mu(\mathbf{u}_j) - \tilde{r}_B^{(k)}(\varphi, \tilde{\mathbf{t}}). \tag{8.26}
\end{align*}
$$
We begin with the first term on the right-hand side of (8.26). We remark that

$$
\left| \int_{\mathbb{G}^{k \times \mathbb{R}^{dk}}} \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^{k} K_h^D(\mathbf{w}_j) \tilde{r}_B^{(k)}(\varphi, (\tilde{\mathbf{t}} - h\tilde{\mathbf{w}}, \tilde{\mathbf{u}})) (f_{\bar{\mathbf{X}}}(\mathbf{t}_j - h\mathbf{w}_j, \mathbf{u}_j) - f_{\bar{\mathbf{X}}}(\mathbf{t}_j, \mathbf{u}_j)) d\mathbf{w}_j d\mu(\mathbf{u}_j) \right| \\
= \left| \int_{\mathbb{G}^{k \times \mathbb{R}^{dk}}} \tilde{K}_{T_n}(\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^{k} K_h^D(\mathbf{w}_j) \tilde{r}_B^{(k)}(\varphi, (\tilde{\mathbf{t}} - h\tilde{\mathbf{w}}, \tilde{\mathbf{u}})) (\Theta_{\mathbf{t}, \mathbf{w}}(1) - \Theta_{\mathbf{t}, \mathbf{w}}(0)) d\mathbf{w}_j d\mu(\mathbf{u}_j) \right|, \quad (8.27)
$$

where $\Theta_{\mathbf{t},\mathbf{w}}(s) := \prod_{j=1}^k f_{\bar{\mathbf{x}}}(\mathbf{t}_j - s\mathbf{h}\mathbf{w}_j, \mathbf{u}_j) - f_{\bar{\mathbf{x}}}(\mathbf{t}_j, \mathbf{u}_j)$ for $s \in [-1, 1]$. Then, we have

$$
\begin{align}
\Theta_{\mathbf{t}, \mathbf{w}}^{(\alpha)}(0) &= \sum_{m_1+\dots+m_k=\alpha} \binom{\alpha}{m_1:k} \prod_{j=1}^{k} \frac{\partial^{m_j} (f_{\bar{\mathbf{x}}}(\mathbf{t}_j - s\mathbf{h}\mathbf{w}_j, \mathbf{u}_j))}{\partial s^{m_j}}(0) \nonumber \\
&= \sum_{m_1+\dots+m_k=\alpha} \binom{\alpha}{m_1:k} \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1}^{k} (-1)^{m_j} h^{m_j} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \nonumber \\
&\qquad \times \frac{\partial^{m_j} f_{\bar{\mathbf{x}}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{m_j}}} (\mathbf{t}_j - s_{\mathbf{t},\mathbf{w}} h \mathbf{w}_j, \mathbf{u}_j), \tag{8.28}
\end{align}
$$

where

$$
\binom{\alpha}{m_1:k} := \alpha! / \left( \prod_{j=1}^{k} (m_j!) \right),
$$

is the multinomial coefficient. In the same manner, denote

$$
\Re_{\mathbf{t},\mathbf{w}}(s) := r_B^{(k)}(\varphi, (\tilde{\mathbf{t}} - s\tilde{\mathbf{w}}, \tilde{\mathbf{u}})),
$$

and we have

$$
\begin{align*}
\Re_{\mathbf{v},\mathbf{w}}^{(\alpha)}(0) &= \sum_{m_1+\dots+m_k=\alpha} \binom{\alpha}{m_1:k} \prod_{j=1}^{k} \frac{\partial^{m_j} \left( r_B^{(k)}(\varphi, (\tilde{\mathbf{t}} - s_{\mathbf{t},\mathbf{w}} h \tilde{\mathbf{w}}, \tilde{\mathbf{u}})) \right)}{\partial s^{m_j}}(0) \\
&= \sum_{m_1+\dots+m_k=\alpha} \binom{\alpha}{m_1:k} \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1}^{k} (-1)^{m_j} h^{m_j} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \\
&\qquad \times \frac{\partial^{m_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{m_j}}} (\varphi, (\tilde{\mathbf{t}} - s_{\mathbf{t},\mathbf{w}} h \tilde{\mathbf{w}}, \tilde{\mathbf{u}})).
\end{align*}
$$

Let $\vartheta(s) := \Re_{\mathbf{t},\mathbf{w}}(s)\Re_{\mathbf{t},\mathbf{w}}(s)$. By using Leibniz's rule, we obtain

$$
\vartheta(\tilde{m}) (0) = \sum_{\ell=0}^{\tilde{m}} \binom{\tilde{m}}{\ell} \Re_{\mathbf{t},\mathbf{w}}^{(\ell)}(0) \Theta_{\mathbf{t},\mathbf{w}}^{(\tilde{m}-\ell)}(0)
$$
$$
\begin{align*}
&= \sum_{\ell=0}^{\tilde{m}} \binom{\tilde{m}}{\ell} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=\tilde{m}-\ell} \binom{\ell}{m_1:k} \binom{\tilde{m}-\ell}{\tilde{m}_1:k} \\
&\quad \times \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1} \sum_{\tilde{i}_1, \dots, \tilde{i}_{\tilde{m}_j}=1} (-1)^{m_j} (-1)^{\tilde{m}_j} h^{m_j} h^{\tilde{m}_j} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{i}_1} \dots \mathbf{w}_{\tilde{i}_{\tilde{m}_j}} \\
&\quad \times \frac{\partial^{m_j} f_{\bar{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\mathbf{t}_j - s_{\mathbf{t},\mathbf{w}} \mathbf{w}_j, \mathbf{u}_j) \frac{\partial^{\tilde{m}_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{t} - s_{\mathbf{t},\mathbf{w}} \mathbf{w} \tilde{\mathbf{w}}, \tilde{\mathbf{u}})) \\
&= \sum_{\ell=0}^{\tilde{m}} \binom{\tilde{m}}{\ell} (-1)^{\tilde{m}} h^{\tilde{m}} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=\tilde{m}-\ell} \binom{\ell}{m_1:k} \binom{\tilde{m}-\ell}{\tilde{m}_1:k} \\
&\quad \times \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1} \sum_{\tilde{i}_1, \dots, \tilde{i}_{\tilde{m}_j}=1} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{i}_1} \dots \mathbf{w}_{\tilde{i}_{\tilde{m}_j}} \frac{\partial^{m_j} f_{\bar{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\mathbf{t}_j - s_{\mathbf{t},\mathbf{w}} \mathbf{w}_j, \mathbf{u}_j) \\
&\quad \times \frac{\partial^{\tilde{m}_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{t} - s_{\mathbf{t},\mathbf{w}} \mathbf{w} \tilde{\mathbf{w}}, \tilde{\mathbf{u}})).
\end{align*}
$$

Accordingly, if the function $\vartheta(\cdot)$ is $m$-times differentiable, and by a Taylor-Lagrange expansion, (8.27)
implies that,

$$
\begin{align*}
& \left| \int_{\mathbb{G}^{k} \times \mathbb{R}^{dk}} \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^{k} K_h^D (\mathbf{w}_j) r_B^{(k)} (\varphi, (\tilde{\mathbf{t}} - h\tilde{\mathbf{w}}, \tilde{\mathbf{u}})) (f_{\bar{X}} (\mathbf{t}_j - h\mathbf{w}_j, \mathbf{u}_j) \\
& \qquad - f_{\bar{X}} (\mathbf{t}_j, \mathbf{u}_j)) d\mathbf{w}_j d\mu(\mathbf{u}_j) \right| \\
&= \left| \int_{\mathbb{G}^{k} \times \mathbb{R}^{dk}} \left( \sum_{\tilde{m}=1}^{m-1} \frac{1}{\tilde{m}!} \vartheta^{(\tilde{m})} (0) + \frac{1}{m!} \vartheta^{(m)} (s) \right) \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^{k} K_h^D (\mathbf{w}_j) d\mathbf{w}_j d\mu(\mathbf{u}_j) \right| \\
&\le \left| \int_{\mathbb{G}^{k} \times \mathbb{R}^{dk}} \sum_{\tilde{m}=1}^{m-1} \frac{1}{\tilde{m}!} \vartheta^{(\tilde{m})} (0) \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^{k} K_h^D (\mathbf{w}_j) d\mathbf{w}_j d\mu(\mathbf{u}_j) \right| \\
&\quad + \left| \int_{\mathbb{G}^{k} \times \mathbb{R}^{dk}} \frac{1}{m!} \vartheta^{(m)} (s) \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \prod_{j=1}^{k} K_h^D (\mathbf{w}_j) d\mathbf{w}_j d\mu(\mathbf{u}_j) \right| \\
&\le h^{\tilde{m}} \left| \int_{\mathbb{G}^{k} \times \mathbb{R}^{dk}} \sum_{\tilde{m}=1}^{m-1} \frac{1}{\tilde{m}!} \sum_{\ell=0}^{\tilde{m}} \binom{\tilde{m}}{\ell} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=\tilde{m}-\ell} \binom{\ell}{m_1:k} \binom{\tilde{m}-\ell}{\tilde{m}_1:k} \right. \\
& \qquad \times \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1} \sum_{\tilde{i}_1, \dots, \tilde{i}_{\tilde{m}_j}=1} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{i}_1} \dots \mathbf{w}_{\tilde{i}_{\tilde{m}_j}} \frac{\partial^{m_j} f_{\bar{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\mathbf{t}_j - s_{\mathbf{t},\mathbf{w}} \mathbf{w}_j, \mathbf{u}_j) \\
& \qquad \left. \times \frac{\partial^{\tilde{m}_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{t} - s_{\mathbf{t},\mathbf{w}} \mathbf{w} \tilde{\mathbf{w}}, \tilde{\mathbf{u}})) \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) K_h^D (\mathbf{w}_j) d\mathbf{w}_j d\mu(\mathbf{u}_j) \right|
\end{align*}
$$
$$
\begin{align*}
&+ \frac{h^m}{m!} \left| \int_{\mathbb{G}^{k \times \mathbb{R}^{dk}}} \sum_{\ell=0}^{m} \binom{m}{\ell} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=m-\ell} \binom{\ell}{m_1:k} \binom{m-\ell}{\tilde{m}_1:k} \right. \\
&\qquad \left. \times \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1}^{k} \sum_{\tilde{i}_1, \dots, \tilde{i}_{\tilde{m}_j}=1}^{k} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{i}_1} \dots \mathbf{w}_{\tilde{i}_{\tilde{m}_j}} \left( \frac{\partial^{m_j} f_{\bar{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{m_j}}} (\mathbf{s}_j, \mathbf{u}_j) \right. \right. \\
&\qquad \left. \left. \times \frac{\partial^{\tilde{m}_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{\mathbf{s}}, \tilde{\mathbf{u}})) \bigg|_{\mathbf{s}_j = p(\mathbf{t}_j, h\mathbf{w}_j, \mathbf{u}_j)} \right) \right\} \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) K_h^D (\mathbf{w}_j) \, d\mathbf{w}_j d\mu(\mathbf{u}_j) \right| \\
&\le \frac{h^m}{m!} \left| \int_{\mathbb{G}^{k \times \mathbb{R}^{dk}}} \sum_{\ell=0}^{m} \binom{m}{\ell} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=m-\ell} \binom{\ell}{m_1:k} \binom{m-\ell}{\tilde{m}_1:k} \right. \\
&\qquad \left. \times \prod_{j=1}^{k} \sum_{i_1, \dots, i_{m_j}=1}^{k} \sum_{\tilde{i}_1, \dots, \tilde{i}_{\tilde{m}_j}=1}^{k} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{i}_1} \dots \mathbf{w}_{\tilde{i}_{\tilde{m}_j}} \left( \frac{\partial^{m_j} f_{\bar{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{m_j}}} (\mathbf{s}_j, \mathbf{u}_j) \right. \right. \\
&\qquad \left. \left. \times \frac{\partial^{\tilde{m}_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{\mathbf{s}}, \tilde{\mathbf{u}})) \bigg|_{\mathbf{s}_j = p(\mathbf{t}_j, h\mathbf{w}_j, \mathbf{u}_j)} \right) \right\} \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) K_h^D (\mathbf{w}_j) \, d\mathbf{w}_j d\mu(\mathbf{u}_j) \\
&\le \frac{h^m}{m!} \sum_{\ell=0}^{m} \binom{m}{\ell} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=m-\ell} \binom{\ell}{m_1:k} \binom{m-\ell}{\tilde{m}_1:k} \\
&\qquad \times \int_{\mathbb{R}^{dk}} \prod_{j=1}^{k} \left| K_h^D (\mathbf{w}_j) \right| \sum_{i_1, \dots, i_{m_j}=1}^{k} \sum_{\tilde{i}_1, \dots, \tilde{i}_{\tilde{m}_j}=1}^{k} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{i}_1} \dots \mathbf{w}_{\tilde{i}_{\tilde{m}_j}} \, d\mathbf{w}_j \\
&\qquad \times \int_{\mathbb{G}^k} \left| \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \right| \sup_{\tilde{\mathbf{w}} \in \mathbb{R}^{dk}} \left| \frac{\partial^{m_j} f_{\bar{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{m_j}}} (\tilde{\mathbf{w}}, \mathbf{u}_j) \frac{\partial^{\tilde{m}_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{\mathbf{w}}, \tilde{\mathbf{u}})) \right| d\mu(\mathbf{u}_j) \\
&\le \frac{h^m}{m!} \sum_{\ell=0}^{m} \binom{m}{\ell} \sum_{m_1+\dots+m_k=\ell} \sum_{\tilde{m}_1+\dots+\tilde{m}_k=m-\ell} \binom{\ell}{m_1:k} \binom{m-\ell}{\tilde{m}_1:k} \\
&\qquad \times \int_{\mathbb{R}^{dk}} \prod_{j=1}^{k} \left| K_h^D (\mathbf{w}_j) \right| \sum_{i_1, \dots, i_{m_j}=1}^{k} \sum_{\tilde{i}_1, \dots, \tilde{i}_{\tilde{m}_j}=1}^{k} \mathbf{w}_{i_1} \dots \mathbf{w}_{i_{m_j}} \mathbf{w}_{\tilde{i}_1} \dots \mathbf{w}_{\tilde{i}_{\tilde{m}_j}} \, d\mathbf{w}_j \\
&\qquad \times \left( \int_{\mathbb{G}^k} \left| \tilde{K}_{T_n} (\tilde{\mathbf{t}}, \mathbf{u}_1, \dots, \mathbf{u}_k) \right|^2 d\mu(\mathbf{u}_j) \right)^{1/2} \\
&\qquad \times \left( \int_{\mathbb{G}^k} \sup_{\tilde{\mathbf{w}} \in \mathbb{R}^{dk}} \left| \frac{\partial^{m_j} f_{\bar{X}}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{m_j}}} (\tilde{\mathbf{w}}, \mathbf{u}_j) \frac{\partial^{\tilde{m}_j} r_B^{(k)}}{\partial \mathbf{t}_{i_1} \dots \partial \mathbf{t}_{i_{\tilde{m}_j}}} (\varphi, (\tilde{\mathbf{w}}, \tilde{\mathbf{u}})) \right|^2 d\mu(\mathbf{u}_j) \right)^{1/2} \\
&= O(h^m T_n^{k \dim(\mathbb{G})/4}). \tag{8.29}
\end{align*}
$$

Here, $p(\mathbf{t}_j, h\mathbf{w}_j, \mathbf{u}_j)$ represents a point on the line segment between $\mathbf{t}_j$ and $\mathbf{t}_j - h\mathbf{w}_j$ which depends on $\mathbf{t}_j, h\mathbf{w}_j$ and $\mathbf{u}_j$. The second inequality is a consequence of Assumption (A'4)-(iii), and the last