In addition, note that the following holds for every $h > 0$, $\delta > 0$, and $y_1, y_2 \in B_d(\tilde{m}_{h,1}(\boldsymbol{x}), \delta)$.

$$
\begin{aligned}
\left| U_{h,1}(\boldsymbol{z}, w, y_1) - U_{h,1}(\boldsymbol{z}, w, y_2) \right| & \leq |\tilde{W}_{\boldsymbol{x},h}(\boldsymbol{z})| |D_{h,1}(\boldsymbol{x}, w, y_1) - D_{h,1}(\boldsymbol{x}, w, y_2)| \\
& = |\tilde{W}_{\boldsymbol{x},h}(\boldsymbol{z})| \left| d^2(w, y_1) - d^2(w, y_2) \right| \\
& \leq \frac{d(y_1, y_2)}{\delta} H_{h,\delta}(\boldsymbol{z}, w), \quad \forall \boldsymbol{z} \in \mathbb{S}^{d-1}, \forall w \in \Omega
\end{aligned}
\quad (4.149)
$$

Therefore, by Lemma 3.7 and Lemma 3.8 with (4.148) and (4.149), there exist constants $L_2 > 0$ and $N_2 \in \mathbb{N}$ such that the following holds for every $\delta > 0$.

$$
\begin{aligned}
n \ge N_2 & \Rightarrow \mathbb{E} \left[ \sup_{y \in B_d(\tilde{m}_{h,1}(\boldsymbol{x}), \delta)} \left| n^{-1} \sum_{i=1}^n U_{h,1}(\boldsymbol{X}_i, Y_i, y) - \mathbb{E} \left[ U_{h,1}(\boldsymbol{X}, Y, y) \right] \right| \right] \\
& \le L_2 \delta n^{-\frac{1}{2}} h^{-\frac{d-1}{2}} \int_0^1 \sqrt{1 + \log N(\delta \epsilon/2, B_d(\tilde{m}_{h,1}(\boldsymbol{x}), \delta), d)} \mathrm{d}\epsilon \\
& = 2L_2 \delta n^{-\frac{1}{2}} h^{-\frac{d-1}{2}} \int_0^1 \sqrt{1 + \log N(\delta \epsilon, B_d(\tilde{m}_{h,1}(\boldsymbol{x}), \delta), d)} \mathrm{d}\epsilon \\
& \le 2L_2 \delta n^{-\frac{1}{2}} h^{-\frac{d-1}{2}} \int_0^1 \sqrt{1 + \log N(\delta \epsilon, B_d(\tilde{m}_{h,1}(\boldsymbol{x}), \delta), d)} \mathrm{d}\epsilon
\end{aligned}
\quad (4.150)
$$

Note that by the condition R2, there exist constants $L_3 > 0$ and $\delta_1 > 0$ such that the following holds.

$$0 < \delta < \delta_1 \quad \Rightarrow \quad \sup_{w \in \Omega} \int_0^1 \sqrt{1 + \log N(\delta \epsilon, B_d(w, \delta), d)} \mathrm{d}\epsilon \le L_3 \quad (4.151)$$

Therefore, by (4.150) and (4.149), the following holds.

$$
\begin{aligned}
n \ge N_2, \quad 0 < \delta < \delta_1 & \Rightarrow \mathbb{E} \left[ \sup_{y \in B_d(\tilde{m}_{h,1}(\boldsymbol{x}), \delta)} \left| n^{-1} \sum_{i=1}^n U_{h,1}(\boldsymbol{X}_i, Y_i, y) - \mathbb{E} \left[ U_{h,1}(\boldsymbol{X}, Y, y) \right] \right| \right] \\
& \le 2L_2 L_3 \delta n^{-\frac{1}{2}} h^{-\frac{d-1}{2}}
\end{aligned}
\quad (4.152)
$$

Now define constants $N_3 := \max\{N_1, N_2\}$ and $L_4 := 2\mathrm{diam}(\Omega)L_1 + 2L_2L_3$. Then the following holds by (4.141) and (4.152), where $\mathbb{1}$ is the characteristic function.

$$n \ge N_3, \quad 0 < \delta < \delta_1 \quad \Rightarrow \quad \mathbb{E} \left[ \sup_{y \in B_d(\tilde{m}_{h,1}(\boldsymbol{x}), \delta)} \left| \hat{T}_{h,1}(\boldsymbol{x}, y) - \hat{T}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \right| \mathbb{1}_{F_n} \right] \le L_4 \delta n^{-\frac{1}{2}} h^{-\frac{d-1}{2}} \quad (4.153)$$

Suppose constants $\tilde{h}_1 > 0$, $\tilde{\eta}_1 > 0$, $\tilde{C}_1 > 0$, $\tilde{\beta}_1 > 0$ are from the condition M3-1. since $\sum_{k=1}^{\infty} 4^{-k(\tilde{\beta}_1-1)/\tilde{\beta}_1}$ converges, there exists a constant $M \in \mathbb{N}$ such that the following holds.

$$\frac{4L_4}{\tilde{C}_1} \sum_{k=M}^{\infty} \left( 4^{-\frac{\tilde{\beta}_1-1}{\tilde{\beta}_1}} \right)^k < \frac{\epsilon}{3} \quad (4.154)$$
Also, by Theorem 4.3 with the conditions L1, D1, D2, B1, R1, and M1-1, there exists $N_4 \in \mathbb{N}$ such that the following holds.

$$n \ge N_4 \Rightarrow \mathbb{P}\left(d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x})) > \frac{\min\{\tilde{\eta}_1, \delta_1\}}{2}\right) < \frac{\epsilon}{3}, \quad 0 < h < \tilde{h}_1 \quad (4.155)$$

For the convenience, define $t_n := (nh^{d-1})^{\tilde{\beta}_1/(4\tilde{\beta}_1-4)}$ for every $n \in \mathbb{N}$. Also, we define events $E_n$, $A_{n,k}$, $B_n$ as the following for every $n, k \in \mathbb{N}$. Note that $n \ge N_4$ implies $\mathbb{P}(B_n^c) < \epsilon/3$.

$$E_n := \left\{ d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x})) > 2^{\frac{2M}{\tilde{\beta}_1} (nh^{d-1})^{\frac{-1}{2(\tilde{\beta}_1-1)}}} \right\} = \left\{ t_n d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x}))^{\frac{\tilde{\beta}_1}{2}} > 2^M \right\}, \\ A_{n,k} := \left\{ 2^{k-1} < t_n d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x}))^{\frac{\tilde{\beta}_1}{2}} \le 2^k \right\}, \quad B_n := \left\{ d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x})) \le \frac{\min\{\tilde{\eta}_1, \delta_1\}}{2} \right\}$$

Since $E_n = \cup_{k=M+1}^{\infty} A_{n,k}$ for every $n \in \mathbb{N}$, note that the following holds by the union bound for every $n \in \mathbb{N}$.

$$\mathbb{P}(E_n) \le \mathbb{P}(E_n \cap B_n \cap F_n) + \mathbb{P}(B_n^c \cup F_n^c) \le \sum_{k=M+1}^{\infty} \mathbb{P}(A_{n,k} \cap B_n \cap F_n) + \mathbb{P}(B_n^c \cup F_n^c) \quad (4.156)$$

Also, define $r_{n,k} := \min\{4^{k/\tilde{\beta}_1} t_n^{-2/\tilde{\beta}_1}, \delta_1/2\}$ for every $n, k \in \mathbb{N}$. Suppose $n \ge \max\{N_3, N_4\}$ and $k \ge M+1$. On the event $A_{n,k} \cap B_n$, note that the following holds.

$$\begin{aligned} \tilde{C}_1 \frac{4^{k-1}}{t_n^2} &\le \tilde{C}_1 d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x}))^{\tilde{\beta}_1} \le \tilde{M}_{h,1}(\boldsymbol{x}, \hat{m}_{h,1}(\boldsymbol{x})) - \tilde{M}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \\ &\le \left[ \tilde{M}_{h,1}(\boldsymbol{x}, \hat{m}_{h,1}(\boldsymbol{x})) - \tilde{M}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \right] - \left[ \hat{M}_{h,1}(\boldsymbol{x}, \hat{m}_{h,1}(\boldsymbol{x})) - \hat{M}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \right] \\ &\le \left| \hat{T}_{h,1}(\boldsymbol{x}, \hat{m}_{h,1}(\boldsymbol{x})) - \hat{T}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \right| \le \sup_{y \in B_d(\tilde{m}_{h,1}(\boldsymbol{x}), r_{n,k})} \left| \hat{T}_{h,1}(\boldsymbol{x}, y) - \hat{T}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \right| \end{aligned}$$

Therefore, by Markov's inequality and (4.153), the following holds for every $n, k \in \mathbb{N}$ such that $n \ge \max\{N_3, N_4\}$ and $k \ge M+1$.

$$\begin{aligned} \mathbb{P}(A_{n,k} \cap B_n \cap F_n) &\le \mathbb{P}\left( \left\{ \sup_{y \in B_d(\tilde{m}_{h,1}(\boldsymbol{x}), r_{n,k})} \left| \hat{T}_{h,1}(\boldsymbol{x}, y) - \hat{T}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \right| \ge \tilde{C}_1 \frac{4^{k-1}}{t_n^2} \right\} \cap F_n \right) \\ &\le \frac{t_n^2}{\tilde{C}_1 4^{k-1}} \mathbb{E} \left[ \sup_{y \in B_d(\tilde{m}_{h,1}(\boldsymbol{x}), r_{n,k})} \left| \hat{T}_{h,1}(\boldsymbol{x}, y) - \hat{T}_{h,1}(\boldsymbol{x}, \tilde{m}_{h,1}(\boldsymbol{x})) \right| \mathbb{1}_{F_n} \right] \\ &\le \frac{L_4 t_n^2}{\tilde{C}_1 4^{k-1}} r_{n,k} (nh^{d-1})^{-\frac{1}{2}} \le \frac{4L_4 t_n^2}{\tilde{C}_1 4^k} \left( \frac{4^k}{t_n^2} \right)^{\frac{1}{\tilde{\beta}_1}} (nh^{d-1})^{-\frac{1}{2}} = \frac{4L_4}{\tilde{C}_1} \left( 4^{-\frac{\tilde{\beta}_1-1}{\tilde{\beta}_1}} \right)^k \quad (4.157) \end{aligned}$$

Now define $N := \max\{N_3, N_4\}$. Then by combining (4.154), (4.155), (4.56), and (4.157), we get the following. Note that $n \ge N$ implies $\mathbb{P}(B_n^c \cup F_n^c) \le \mathbb{P}(F_n^c) + \mathbb{P}(F_n^c) < 2\epsilon/3$.

$$n \ge N \Rightarrow \mathbb{P}\left(d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x})) > 2^{\frac{2M}{\tilde{\beta}_1} (nh^{d-1})^{\frac{-1}{2(\tilde{\beta}_1-1)}}}\right)$$
$$ \leq \frac{4L_4}{\tilde{C}_1} \sum_{k=M+1}^{\infty} 4^{-\frac{k(\tilde{\beta}_1-1)}{\tilde{\beta}_1}} + \mathbb{P}(B_n^c \cup F_n^c) < \epsilon \quad (4.158) $$

Since $\epsilon > 0$ is arbitrary, (4.158) implies $d(\hat{m}_{h,1}(\boldsymbol{x}), \tilde{m}_{h,1}(\boldsymbol{x}))^{\tilde{\beta}_1-1} = O_p(n^{-\frac{1}{2}}h^{-\frac{d-1}{2}})$. $\square$

Finally, by Theorem 4.2 and Theorem 4.4, we can get the convergence rate of $\hat{m}_{h,1}(\boldsymbol{x})$ for every $\boldsymbol{x} \in \mathbb{S}^{d-1}$. Also, we can get the optimal bandwidth sequence for $h = h_n > 0$.

**Theorem 4.5.** Suppose the conditions L1, D1, D2, C1, B1, R1, R2, M1-1, M2, and M3-1 hold. Then the following holds for every $\boldsymbol{x} \in \mathbb{S}^{d-1}$, where $\beta_{\oplus} > 1$ and $\tilde{\beta}_1 > 1$ are the constants defined in M2 and M3-1 respectively.

$$ d(\hat{m}_{h,1}(\boldsymbol{x}), m_{\oplus}(\boldsymbol{x})) = O(h^{\frac{2}{\beta_{\oplus}-1}}) + O_p((nh^{d-1})^{\frac{-1}{2(\tilde{\beta}_1-1)}}) \quad \text{as} \quad n \to \infty $$

Also, among bandwidth sequences $h = h_n = n^{-\gamma}$ for any constant $\gamma > 0$, the optimal sequence is obtained when $\gamma = \frac{\beta_{\oplus}-1}{4(\tilde{\beta}_1-1)+(d-1)(\beta_{\oplus}-1)}$, and the following holds for every $\boldsymbol{x} \in \mathbb{S}^{d-1}$ in this case.

$$ d(\hat{m}_{h,1}(\boldsymbol{x}), m_{\oplus}(\boldsymbol{x})) = O_p(n^{\frac{-2}{4(\tilde{\beta}_1-1)+(d-1)(\beta_{\oplus}-1)}}) \quad \text{as} \quad n \to \infty $$

## References

Z.D. Bai, C.Radhakrishna Rao, and L.C. Zhao. Kernel estimators of density function of directional data. *Journal of Multivariate Analysis*, 27(1):24–39, 1988.

Marco Di Marzio, Stefania Fensore, Agnese Panzera, and Charles C. Taylor. Local binary regression with spherical predictors. *Statistics & Probability Letters*, 144:30–36, 2019. ISSN 0167-7152. doi: https://doi.org/10.1016/j.spl.2018.07.019. URL https://www.sciencedirect.com/science/article/pii/S0167715218302633. Advances in statistical methods and applications for Climate change and Environment.

Eduardo García-Portugués, Rosa M. Crujeiras, and Wenceslao González-Manteiga. Kernel density estimation for directional–linear data. *Journal of Multivariate Analysis*, 121:152–175, 2013.

Peter Hall, G. S. Watson, and Javier Cabrera. Kernel density estimation with spherical data. *Biometrika*, 74(4):751–762, 1987.

C. J. Im, J. M. Jeon, and B. U. Park. Fréchet regression for circular predictors. *Manuscript*, 1: 1–43, 2024.
Agnese Panzera Marco Di Marzio and Charles C. Taylor. Nonparametric regression for spherical data. *Journal of the American Statistical Association*, 109(506):748-763, 2014. doi: 10.1080/01621459.2013.866567.

Agnese Panzera Marco Di Marzio and Charles C. Taylor. Nonparametric rotations for sphere-sphere regression. *Journal of the American Statistical Association*, 114(525):466-476, 2019. doi: 10.1080/01621459.2017.1421542.

Alexander Petersen and Hans-Georg Müller. Fréchet regression for random objects with Euclidean predictors. *The Annals of Statistics*, 47(2):691-719, 2019.

A. van der Vaart and J.A. Wellner. *Weak Convergence and Empirical Processes: With Applications to Statistics*. Springer Series in Statistics. Springer, 1996.