SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We conclude this paper by proposing several promising directions for future research. First, it would be worthwhile to explore the large sample theory for selecting the optimal threshold parameter  $\lambda$  in the proposed SVT method, in order to characterize the theoretical phase transition of the bias-variance trade-off in the regularized (global) Fréchet regression. Second, we believe that our framework could be extended to errors-in-variables Fréchet regression for response variables in a broader class of metric spaces, e.g., by leveraging the quadruple inequality proposed by Schötz [44, 45]. Lastly, investigating the asymptotic distribution of the proposed SVT estimator would be highly appealing in the statistical literature, as it would enable us to make statistical inferences on the conditional Fréchet mean in non-Euclidean spaces [6, 8] with errors-in-variables covariates.
DS acknowledges support from the National Science Foundation under Grant No. CCF-2212326.
[1] Biban Afsari. Riemannian  $L^p$  center of mass: Existence, uniqueness, and convexity. Proceedings of the American Mathematical Society, 139(2):655-673, 2011.
[2] Anish Agarwal, Devavrat Shah, Dennis Shen, and Dogyoon Song. On robustness of principal component regression. Journal of the American Statistical Association, 116(536):1731-1745, 2021.
[3] Adil Ahidar-Coutrix, Thibaut Le Gouic, and Quentin Paris. Convergence rates for empirical barycenters in metric spaces: Curvature, convexity and extendable geodesics. Probability Theory and Related Fields, 177(1):323-368, 2020.
[4] Gustavo Amorim, Ran Tao, Sarah Lotspeich, Pamela A Shaw, Thomas Lumley, and Bryan E Shepherd. Two-phase sampling designs for data validation in settings with covariate measurement error and continuous outcome. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(4):1368-1389, 2021.
[5] Marc Arnaudon and Laurent Miclo. Means in complete manifolds: Uniqueness and approximation. ESAIM: Probability and Statistics, 18:185-206, 2014.
[6] Dennis Barden, Huiling Le, and Megan Owen. Central limit theorems for Fréchet means in the space of phylogenetic trees. Electronic Journal of Probability, 18(25):1-25, 2013.
[7] Alexandre Belloni, Mathieu Rosenbaum, and Alexandre B Tsybakov. Linear and conic programming estimators in high dimensional errors-in-variables models. Journal of the Royal Statistical Society. Series B (Statistical Methodology), pages 939–956, 2017.
[8] Rabi Bhattacharya and Lizhen Lin. Omnibus CLTs for Fréchet means and nonparametric inference on non-Euclidean spaces. Proceedings of the American Mathematical Society, 145(1):413-428, 2017.
[9] Norman E Breslow and Thomas Lumley. Semiparametric models and two-phase samples: Applications to Cox regression. IMS Collections, 9:65-77, 2013.
[10] Louis Capitaine, Jérémie Bigot, Rodolphe Thiébaut, and Robin Genuer. Fréchet random forests for metric space valued regression with non-Euclidean predictors. arXiv preprint arXiv:1906.01741, 2019.
[11] Raymond J Carroll, Helmut Kuchenhoff, Frederick Lombard, and Leonard A Stefanski. Asymptotics for the SIMEX estimator in nonlinear measurement error models. Journal of the American Statistical Association, 91(433):242-250, 1996.
[12] Raymond J Carroll, David Ruppert, Ciprian M Crainiceanu, Tor D Tosteson, and Margaret R Karagas. Nonlinear and nonparametric regression and instrumental variables. Journal of the American Statistical Association, 99(467):736-750, 2004.
[13] Raymond J Carroll, David Ruppert, Leonard A Stefanski, and Ciprian M Crainiceanu. Measurement Error in Nonlinear Models: A Modern Perspective. Chapman and Hall/CRC, 2006.
[14] Benjamin Charlier. Necessary and sufficient condition for the existence of a Fréchet mean on the circle. ESAIM: Probability and Statistics, 17:635-649, 2013.
[15] Yan Mei Chen, Xiao Shan Chen, and Wen Li. On perturbation bounds for orthogonal projections. Numerical Algorithms, 73(2):433-444, 2016.
[16] John R Cook and Leonard A Stefanski. Simulation-extrapolation estimation in parametric measurement error models. Journal of the American Statistical Association, 89(428):1314-1328, 1994.
[38] Po-Ling Loh and Martin J Wainwright. High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity. Advances in Neural Information Processing Systems, 24, 2011.
[39] James Stephen Marron and Ian L Dryden. Object Oriented Data Analysis. CRC Press, 2021.
[40] Alexander Petersen and Hans-Georg Müller. Fréchet regression for random objects with Euclidean predictors. The Annals of Statistics, 47(2):691–719, 2019.
[41] James O Ramsay and Bernard W Silverman. Functional Data Analysis. Springer, 2nd edition, 2005.
[42] Philip T Reiss and R Todd Ogden. Functional principal component regression and functional partial least squares. Journal of the American Statistical Association, 102(479):984-996, 2007.
[43] Susanne M Schennach. Instrumental variable estimation of nonlinear errors-in-variables models. Econometrica, 75(1):201-239, 2007.
[44] Christof Schötz. Convergence rates for the generalized Fréchet mean via the quadruple inequality. Electronic Journal of Statistics, 13:4280-4345, 2019.
[45] Christof Schötz. Nonparametric regression in nonstandard spaces. Electronic Journal of Statistics, 16(2):4679-4741, 2022.
[46] Damla Şentürk and Hans-Georg Müller. Covariate-adjusted regression. Biometrika, 92(1):75–89, 2005.
[47] Donna Spiegelman, Aidan McDermott, and Bernard Rosner. Regression calibration method for correcting measurement-error bias in nutritional epidemiology. The American Journal of Clinical Nutrition, 65(4):1179S-1186S, 1997.
[48] Florian Steinke and Matthias Hein. Non-parametric regression between manifolds. Advances in Neural Information Processing Systems, 21, 2008.
[49] Florian Steinke, Matthias Hein, and Bernhard Scholkopf. Nonparametric regression between general Riemannian manifolds. SIAM Journal on Imaging Sciences, 3(3):527-563, 2010.
[50] Gilbert W Stewart. On the perturbation of pseudo-inverses, projections and linear least squares problems. SIAM Review, 19(4):634-662, 1977.
[51] Karl-Theodor Sturm. Probability measures on metric spaces of nonpositive curvature. Heat Kernels and Analysis on Manifolds, Graphs, and Metric Spaces: Lecture Notes from a Quarter Program on Heat Kernels, Random Walks, and Analysis on Manifolds and Graphs, 338:357, 2003.
[52] Renata Talska, Alessandra Menafoglio, Jitka Machalova, Karel Hron, and E Fisserova. Compositional regression with functional response. Computational Statistics & Data Analysis, 123:66-85, 2018.
[53] Kristin M Tolle, D Stewart W Tansley, and Anthony JG Hey. The fourth paradigm: Data-intensive scientific discovery [point of view]. Proceedings of the IEEE, 99(8):1334-1337, 2011.
[54] Danielle C Tucker, Yichao Wu, and Hans-Georg Müller. Variable selection for global Fréchet regression. Journal of the American Statistical Association, pages 1–15, 2021.
[55] Aad W Vaart and Jon A Wellner. Weak Convergence and Empirical Processes. Springer, 2nd edition, 2023.
[56] Roman Vershynin. High-dimensional Probability: An Introduction with Applications in Data Science, volume 47. Cambridge University Press, 2018.
[57] Cinzia Viroli. On matrix-variate regression analysis. Journal of Multivariate Analysis, 111:296-309, 2012.
[58] Takumi Yokota. Convex functions and barycenter on CAT(1)-spaces of small radii. Journal of the Mathematical Society of Japan, 68(3):1297-1323, 2016.
[59] Y Zemel and V M Panaretos. Fréchet means and procrustes analysis in Wasserstein space. Bernoulli, 25(2):932-976, 2019.
A Verification of the model assumptions
A.1 Additional background
Definition 6 ( $\varepsilon$ -net and covering number). Let  $(\mathcal{M}, d)$  be a metric space. Let  $S \subseteq T$  be a subset and let  $\varepsilon > 0$ . A subset  $\mathcal{N} \subseteq S$  is called an  $\varepsilon$ -net of  $S$  if every point in  $S$  is within distance  $\varepsilon$  of some point  $\mathcal{N}$ , i.e.,
$$
\forall x \in S, \exists x _ {0} \in \mathcal {N} \text {s u c h} d (x, x _ {0}) \leq \varepsilon .
$$
The  $\varepsilon$ -covering number of  $S$ , denoted by  $\mathfrak{N}(S, \varepsilon)$ , is the smallest possible cardinality of an  $\varepsilon$ -net of  $S$ , i.e.,
$$
\mathfrak {N} (S, \varepsilon) := \min  \left\{k \in \mathbb {N}: \exists y _ {1}, \dots , y _ {k} \in \mathcal {M} s u c h t h a t S \subseteq \bigcup_ {i = 1} ^ {k} B _ {d} \left(y _ {i}, \varepsilon\right) \right\}, \tag {16}
$$
where  $B_{d}(y,\varepsilon) = \{y^{\prime}\in \mathcal{M}:d(y,y^{\prime})\leq \varepsilon \}$  denotes the closed  $\varepsilon$  -ball centered at  $y\in \mathcal{M}$
Let  $B_2^r(0,1) \coloneqq \{x \in \mathbb{R}^r : \|x\|_2 \leq 1\}$  denote the unit  $\ell^2$ -norm ball in  $\mathbb{R}^r$ . It is well known that for any  $\varepsilon > 0$ ,
$$
\left(\frac {1}{\varepsilon}\right) ^ {r} \leq \Re \left(B _ {2} ^ {r} (0, 1), \varepsilon\right) \leq \left(\frac {2}{\varepsilon} + 1\right) ^ {r}. \tag {17}
$$
A.2 Example 1: Euclidean space
Proposition 1. The space  $(\mathcal{H}, d_{\mathrm{HS}})$  defined in Example 1 satisfies Assumptions (C0), (C1), and (C2).
Proof of Proposition 1. For any probability distribution  $\nu$  and any  $\lambda \in \mathbb{R}_+$ , let  $y_{\nu}^{(\lambda)} := \mathbb{E}_{\nu}\left[w_{\nu}^{(\lambda)}(X,x)\cdot Y\right]$ . Then we observe that
$$
\begin{array}{l} R _ {\nu} ^ {(\lambda)} (y; x) = \mathbb {E} _ {\nu} \left[ w _ {\nu} ^ {(\lambda)} (X, x) \cdot d ^ {2} (Y, y) \right] \\ = \mathbb {E} _ {\nu} \left[ w _ {\nu} ^ {(\lambda)} (X, x) \cdot \| Y - y \| ^ {2} \right] \\ = \mathbb {E} _ {\nu} \left[ w _ {\nu} ^ {(\lambda)} (X, x) \cdot \| Y - y _ {\nu} ^ {(\lambda)} \| ^ {2} \right] + \| y - y _ {\nu} ^ {(\lambda)} \| _ {\mathrm {H S}} ^ {2} \\ + 2 \left\langle \underbrace {\mathbb {E} _ {\nu} \left[ w _ {\nu} ^ {(\lambda)} (X , x) \cdot \left(Y - y _ {\nu} ^ {(\lambda)}\right) \right]} _ {= 0}, y _ {\nu} ^ {(\lambda)} - y \right\rangle \\ = R _ {\nu} ^ {(\lambda)} (y _ {\nu} ^ {(\lambda)}; x) + \| y - y _ {\nu} ^ {(\lambda)} \| _ {\mathrm {H S}} ^ {2}. \\ \end{array}
$$
As  $R_{\nu}^{(\lambda)}(y;x)$  is a strictly convex and coercive function, there exists a unique minimizer,  $\varphi_{\nu}^{(\lambda)}$ . Thus, Condition (C0) is proved. Furthermore, Condition (C1) is also satisfied with  $D_{\mathrm{g}} = \infty$ ,  $C_{\mathrm{g}} = 1$ , and  $\alpha = 2$ .
Lastly, for any  $y \in \mathcal{H}$  and any  $\varepsilon > 0$ ,
$$
\mathfrak {N} \big (B _ {d _ {\mathrm {H S}}} (y, \delta), \delta \varepsilon \big) = \mathfrak {N} \big (B _ {d _ {\mathrm {H S}}} (y, 1), \varepsilon \big) \leq \left(\frac {2}{\varepsilon} + 1\right) ^ {r} \leq C \cdot \varepsilon^ {- r}
$$
where  $r = \dim \mathcal{H}$  and  $C > 1$  is a constant that depends on  $r$  only; see the covering number upper bound in (17). Thus, the integral (11) is bounded as follows:
$$
\begin{array}{l} \int_ {0} ^ {1} \sqrt {1 + \log \Re (B _ {d} (\varphi (x) , \delta) , \delta \varepsilon)} d \varepsilon \leq \int_ {0} ^ {1} \sqrt {1 + \log C - r \log \varepsilon} d \varepsilon \\ \leq \sqrt {1 + \log C} + \sqrt {r} \int_ {0} ^ {1} \sqrt {- \log \varepsilon} d \varepsilon \\ = \sqrt {1 + \log C} + \sqrt {r} \int_ {0} ^ {\infty} e ^ {- t} \sqrt {t} d t \\ = \sqrt {1 + \log C} + \frac {\sqrt {r \pi}}{2} \\ \end{array}
$$
See [56, Corollary 4.2.13] for example.
using the change of variable  $t = -\log \varepsilon$ . Therefore, Assumption (C2) holds with  $C_{\mathrm{e}} = \sqrt{1 + \log C} + \frac{\sqrt{r\pi}}{2}$ .
A.3 Example 2: Set of probability distributions
Proposition 2. The space  $(\mathcal{W},d_{\mathcal{W}})$  defined in Example 2 satisfies Assumptions (C0), (C1), and (C2).
Proof of Proposition 2. For a probability distribution function  $y \in \mathcal{W}$  defined on  $\mathbb{R}$ , let  $Q = Q(\mathcal{W}) \coloneqq \{Q(y) : y \in \mathcal{W}\}$  denote the collection of corresponding quantile functions, where  $(Q(y))(u) = y^{-1}(u)$  for  $u \in [0,1]$ .
We note that  $f \mapsto \mathbb{E}_{\nu} \left[ w_{\nu}^{(\lambda)}(X, x) \langle Q(Y), f \rangle \right]$  is a bounded linear functional defined on  $L^2[0,1]$  because  $\mathbb{E}_{\nu} |w_{\nu}^{(\lambda)}(X, x)|^2 \leq 2 + 2p \| (x - \mu_{\nu}) \|_{\Sigma_{\nu}}^2$  implies that  $\mathbb{E}_{\nu} \left[ w_{\nu}^{(\lambda)}(X, x) \right] \cdot \| Q(Y) \|_2$  is bounded. It follows from the Riesz representation theorem that there exists  $f_x^{(\lambda)} \in L^2[0,1]$  such that
$$
\mathbb {E} _ {\nu} \left[ w ^ {(\lambda)} (X, x) \langle Q (Y), g \rangle_ {2} \right] = \langle f _ {x} ^ {(\lambda)}, g \rangle_ {2} \tag {18}
$$
for all  $g\in L^2 [0,1]$  . Then, we have
$$
R _ {\nu} ^ {(\lambda)} (y; x) = \mathbb {E} _ {\nu} \left[ w _ {\nu} ^ {(\lambda)} (X, x) \| Q (Y) - f _ {x} ^ {(\lambda)} \| _ {2} ^ {2} \right] + \| Q (y) - f _ {x} ^ {(\lambda)} \| _ {2} ^ {2}, \tag {19}
$$
$$
\varphi_ {\nu} ^ {(\lambda)} (x) = Q ^ {- 1} \left(\underset {\bar {Q} \in \mathcal {Q}} {\arg \min } \| Q - f _ {x} ^ {(\lambda)} \| _ {2} ^ {2}\right). \tag {20}
$$
The condition (C0) follows from the convexity of  $(\mathcal{Q},\| \cdot \| _2)$ . Moreover, the convexity also gives  $\left\langle Q(y) - Q(\varphi_{\nu}^{(\lambda)}(x)),f_x^{(\lambda)}(x) - Q(\varphi_\nu^{(\lambda)}(x))\right\rangle_2\leq 0$  for all  $y\in \mathcal{W}$ , so that
$$
\begin{array}{l} R _ {\nu} ^ {(\lambda)} (y; x) - R _ {\nu} ^ {(\lambda)} (\varphi^ {(\lambda)} (x); x) \\ = \| Q (y) - f _ {x} ^ {(\lambda)} (x) \| _ {2} ^ {2} - \| Q \left(\varphi_ {\nu} ^ {(\lambda)} (x)\right) - f _ {x} ^ {(\lambda)} (x) \| _ {2} ^ {2} \\ = \| Q (y) - Q \left(\varphi_ {\nu} ^ {(\lambda)} (x)\right) \| _ {2} - 2 \left\langle Q (y) - Q \left(\varphi_ {\nu} ^ {(\lambda)} (x)\right), f _ {x} ^ {(\lambda)} (x) - Q \left(\varphi_ {\nu} ^ {(\lambda)} (x)\right) \right\rangle_ {2} \tag {21} \\ \geq \| Q (y) - Q \left(\varphi_ {\nu} ^ {(\lambda)} (x)\right) \| _ {2} \\ = d _ {W} ^ {2} (y, \varphi_ {\nu} ^ {(\lambda)} (x)). \\ \end{array}
$$
Therefore, the condition (C1) holds for any arbitrary constant  $D_{\mathbf{g}} > 0$  with  $C_{\mathbf{g}} = 1$  and  $\alpha = 2$ .
Finally, we refer to [40, Proposition 1] to ensure that for any  $\delta > 0$  and any  $\varepsilon > 0$ ,
$$
\sup  _ {y \in \mathcal {W}} \log \mathfrak {N} \left(B _ {d _ {W}} (y, \delta), D _ {\mathrm {e}} \varepsilon\right) \leq \sup  _ {Q \in \mathcal {Q}} \log \mathfrak {N} \left(B _ {d _ {2}} (Q, \delta), \delta \varepsilon\right) \leq C \cdot \varepsilon^ {- 1} \tag {22}
$$
holds with an absolute constant  $C > 0$ . Technically, this fact comes from the covering number bound for a class of uniformly bounded and monotone functions in  $L^2$ . This confirms that the entropy condition (C2) holds.
A.4 Example 3: Set of correlation matrices
Proposition 3. The space  $(\mathcal{M},d_F)$  defined in Example 3 satisfies Assumptions (C0), (C1), and (C2).
Proof of Proposition 3. First of all, we note that  $\mathcal{M}$  is a convex subset of  $S^r \coloneqq \{X \in \mathbb{R}^{r \times r} : X = X^\top\}$ , which is the set of  $r \times r$  symmetric matrices. It is because  $\mathcal{M} = S_+^r \cap H$  where  $S_+^r$  denotes the cone of  $r \times r$  positive semidefinite matrices and  $H \coloneqq \{X \in S^r : X_{ii} = 1, \forall i \in [r]\}$  denotes an affine subspace of  $S^r$ , both of which are convex.
Next, we observe that  $S^r$  equipped with the Frobenius metric  $d_F$  is isometrically isomorphic to  $\mathbb{R}^{r(r + 1) / 2}$  equipped with the  $\ell^2$ -metric. Hence,  $(\mathcal{M}, d_F)$  satisfies Assumptions (C0), (C1), and (C2), inheriting these properties from the ambient space  $S^r$ , which is established by Proposition 1. We note that the inheritance of (C0), (C1) relies on the convexity of  $\mathcal{M}$ , while (C2) is inherited simply based on the inclusion  $\mathcal{M} \subset S^r$ .
A.5 Example 4: Bounded Riemannian manifold
Proposition 4. The space  $(\mathcal{M}, d_g)$  defined in Example 4 satisfies Assumption (C2) provided that the Riemannian metric is equivalent to the ambient Euclidean metric.
Furthermore, let  $T_y\mathcal{M}$  be the tangent space of  $\mathcal{M}$  at  $y$ , and  $\mathrm{Exp}_y: T_y\mathcal{M} \to \mathcal{M}$  be the manifold exponential map at  $y$ . Let  $g_{\nu}^{(\lambda)}(u; y, x) := R_{\nu}^{(\lambda)}(\mathbb{E}_y(u), x)$  for  $u \in T_y\mathcal{M}$  If (C0) holds and the Hessian of  $g_{\nu}^{(\lambda)}(u; \varphi_{\nu}^{(\lambda)}(x), x)$  is positive definite, then (C1) for some  $D_g > 0$ .
Proof of Proposition 3. Since  $\mathcal{M}$  has finite dimension and is bounded, the bounded entropy condition (C2) follows from the metric equivalence.
Suppose that (C0) holds, and let  $\delta > 0$  be the injectivity radius at  $\varphi_{\nu}^{(\lambda)}(x)$ . Consider  $y \in \mathcal{M}$  such that  $d(y, \varphi_{\nu}^{(\lambda)}(x)) < \delta$ , and let  $u_y = \operatorname{Log}_{\varphi_{\nu}^{(\lambda)}(x)}(y)$ . Then we have
$$
R _ {\nu} ^ {(\lambda)} (y; x) - R _ {\nu} ^ {(\lambda)} \big (\varphi_ {\nu} ^ {(\lambda)} (x); x \big) = g _ {\nu} ^ {(\lambda)} \big (u _ {y}; \varphi_ {\nu} ^ {(\lambda)} (x), x \big) - g _ {\nu} ^ {(\lambda)} \big (0; \varphi_ {\nu} ^ {(\lambda)} (x), x \big) = u _ {y} ^ {\top} \nabla^ {2} g _ {\nu} ^ {(\lambda)} (\bar {u} _ {y}) u _ {y}
$$
for some  $\bar{u}_y$  between 0 and  $u_y$ . Since  $u_y^\top u_y = d\big(y, \varphi_\nu^{(\lambda)}(x)\big)^2$  and  $g_\nu^{(\lambda)}$  is continuous, the positive definiteness of  $\nabla^2 g_\nu^{(\lambda)}(\bar{u}_y)$  implies (C1) with  $\alpha = 1$ .
Proof of Theorem 1. Recall from Definition 4, cf. (9), that for any probability distribution  $\nu$  on  $\mathbb{R}^p$ , any  $\lambda \in \mathbb{R}_+$ , and any  $x \in \mathbb{R}^p$ , the  $\lambda$ -regularized Fréchet regression function evaluated at  $x$  is given as the minimizer of a function  $R_{\nu}^{(\lambda)}$  as
$$
\varphi_{\nu}^{(\lambda)}(x) = \operatorname *{arg  min}_{y\in \mathcal{M}}R_{\nu}^{(\lambda)}(y;x)
$$
$$
R _ {\nu} ^ {(\lambda)} (y; x) = \mathbb {E} _ {(X, Y) \sim \nu} \left[ w _ {\nu} ^ {(\lambda)} (X, x) \cdot d ^ {2} (Y, y) \right] \quad \text {a n d}
$$
$$
w _ {\nu} ^ {(\lambda)} \left(x ^ {\prime}, x\right) = 1 + \left(x ^ {\prime} - \mu_ {\nu}\right) ^ {\top} \left[ \mathrm {S V T} ^ {(\lambda)} \left(\Sigma_ {\nu}\right) \right] ^ {\dagger} \left(x - \mu_ {\nu}\right).
$$
In this proof, we follow a similar strategy to that in the proof of [40, Theorem 1]. Specifically, it suffices to show  $\sup_{y\in \mathcal{M}}\left|R_{\mathcal{D}_n}^{(\lambda)}(y;x) - R_{\nu^*}^{(0)}(y;x)\right|$  converges to zero in probability, due to [55, Corollary 3.2.3]. To this end, we show  $R_{\mathcal{D}_n}^{(\lambda)}(y;x)$  weakly converges to  $R_{\nu^*}^{(0)}(y;x)$  in the  $\ell^{\infty}(\mathcal{M})$  sense, and then apply [55, Theorem 1.3.6]. Again, according to [55, Theorem 1.5.4], this weak convergence can be proved by showing that
(S1)  $R_{\mathcal{D}_n}^{(\lambda)}(y;x) - R_{\nu^*}^{(0)}(y;x) = o_p(1)$  for all  $y \in \mathcal{M}$ , and
(S2)  $R_{\mathcal{D}_n}^{(\lambda)}$  is asymptotically equicontinuous in probability, i.e., for any  $\varepsilon, \eta > 0$ , there exists  $\delta > 0$  such that
$$
\limsup_{n}P\left(\sup_{y_{1},y_{2}\in \mathcal{M}:d(y_{1},y_{2}) <   \delta}\left|R^{(\lambda)}_{\mathcal{D}_{n}}(y_{1};x) - R^{(\lambda)}_{\mathcal{D}_{n}}(y_{2};x)\right| > \varepsilon\right) <   \eta .
$$
In what follows, we prove these two statements, (S1) and (S2), thereby completing the proof of Theorem 1.
Step 1: proof of (S1). First of all, we observe that
$$
R _ {\mathcal {D} _ {n}} ^ {(\lambda)} (y; x) - R _ {\nu^ {*}} ^ {(0)} (y; x) = \underbrace {\left(R _ {\mathcal {D} _ {n}} ^ {(\lambda)} (y ; x) - R _ {\mathcal {D} _ {n}} ^ {(0)} (y ; x)\right)} _ {=: T _ {1}} + \underbrace {\left(R _ {\mathcal {D} _ {n}} ^ {(0)} (y ; x) - R _ {\nu^ {*}} ^ {(0)} (y ; x)\right)} _ {=: T _ {2}}. \tag {23}
$$
We separately analyze the two terms  $T_{1}$  and  $T_{2}$  below to show  $T_{1} = o_{p}(1)$  and  $T_{2} = o_{p}(1)$  as  $n \to \infty$ .
(i)  $T_{1} = o_{p}(1)$
Let  $\mathcal{D}_n = \{(X_i,Y_i):i\in [n]\}$ , and we re-write
$$
T _ {1} = \frac {1}{n} \sum_ {i = 1} ^ {n} \left(w _ {\mathcal {D} _ {n}} ^ {(\lambda)} (X _ {i}, x) - w _ {\mathcal {D} _ {n}} ^ {(0)} (X _ {i}, x)\right) \cdot d ^ {2} (Y _ {i}, y).
$$
Letting  $\widehat{\mu}_n = \mu_{\mathcal{D}_n}$ ,  $\widehat{\Sigma}_n = \Sigma_{\mathcal{D}_n}$ , and  $\widehat{\Sigma}_n^{(\lambda)} = \mathrm{SVT}^{(\lambda)}(\widehat{\Sigma}_n)$  for shorthand, we observe that
$$
w _ {\mathcal {D} _ {n}} ^ {(\lambda)} (X _ {i}, x) - w _ {\mathcal {D} _ {n}} ^ {(0)} (X _ {i}, x) = \left(X _ {i} - \widehat {\mu} _ {n}\right) ^ {\top} \left[ \widehat {\Sigma} _ {n} ^ {(\lambda), \dagger} - \widehat {\Sigma} _ {n} ^ {\dagger} \right] (x - \widehat {\mu} _ {n}).
$$
Let  $\mathbf{X} = [X_1\quad \dots \quad X_n]^\top \in \mathbb{R}^{n\times p}$ , and note that  $\widehat{\Sigma}_n = \frac{1}{n}\bigl (\mathbf{X} - \mathbf{1}_n\widehat{\mu}_n^\top \bigr)^\top \bigl (\mathbf{X} - \mathbf{1}_n\widehat{\mu}_n^\top \bigr)$ . Then it follows that
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} (X _ {i} - \widehat {\mu} _ {n}) ^ {\top} \left[ \widehat {\Sigma} _ {n} ^ {(\lambda), \dagger} - \widehat {\Sigma} _ {n} ^ {\dagger} \right] = \frac {1}{n} \mathbf {1} _ {n} ^ {\top} \left(\boldsymbol {X} - \mathbf {1} _ {n} \widehat {\mu} _ {n} ^ {\top}\right) \left[ \widehat {\Sigma} _ {n} ^ {(\lambda), \dagger} - \widehat {\Sigma} _ {n} ^ {\dagger} \right]
$$
Consider a singular value decomposition of  $X - \mathbf{1}_n\widehat{\mu}_n^\top$ , namely,
$$
\boldsymbol {X} - \mathbf {1} _ {n} \widehat {\boldsymbol {\mu}} _ {n} ^ {\top} = \sum_ {i = 1} ^ {\min \{n, p \}} s _ {i} \cdot u _ {i} v _ {i} ^ {\top},
$$
and observe that  $\widehat{\Sigma}_n = \sum_{i=1}^{\min\{n,p\}} s_i^2 \cdot v_i v_i^\top$  is an eigenvalue decomposition of  $\widehat{\Sigma}_n$ . Letting  $\mathcal{V}_n^{(\lambda)} := \operatorname{span}\left\{v_i : i \in [p], 0 < s_i \leq \sqrt{\lambda}\right\}$  be a subspace of  $\mathbb{R}^p$  spanned by the eigenvectors of  $\widehat{\Sigma}_n$  corresponding to the nonzero eigenvalues no greater than  $\lambda$ , we have
$$
\begin{array}{l} \widehat {\Sigma} _ {n} ^ {(\lambda), \dagger} - \widehat {\Sigma} _ {n} ^ {\dagger} = \sum_ {i = 1} ^ {p} \frac {1}{s _ {i} ^ {2}} \cdot \mathbb {1} \{s _ {i} > \sqrt {\lambda} \} \cdot v _ {i} v _ {i} ^ {\top} - \sum_ {i = 1} ^ {p} \frac {1}{s _ {i} ^ {2}} \cdot \mathbb {1} \{s _ {i} > 0 \} \cdot v _ {i} v _ {i} ^ {\top} \\ = \sum_ {i = 1} ^ {p} \frac {1}{s _ {i} ^ {2}} \cdot \mathbb {1} \left\{0 <   s _ {i} \leq \sqrt {\lambda} \right\} \cdot v _ {i} v _ {i} ^ {\top} \\ = \widehat {\Sigma} _ {n} ^ {\dagger} \cdot \Pi_ {\mathcal {V} _ {n} ^ {(\lambda)}} \\ = n \cdot \left(\boldsymbol {X} - \mathbf {1} _ {n} \widehat {\mu} _ {n} ^ {\top}\right) ^ {\dagger} \left(\boldsymbol {X} - \mathbf {1} _ {n} \widehat {\mu} _ {n} ^ {\top}\right) ^ {\dagger , \top} \cdot \Pi_ {\nu_ {n} ^ {(\lambda)}} \tag {24} \\ \end{array}
$$
where  $\Pi_{\mathcal{V}_n^{(\lambda)}}$  denotes the projection matrix onto the subspace  $\mathcal{V}_n^{(\lambda)}$ . Note that  $\Pi_{\mathcal{V}_n^{(\lambda)}} = 0$  if and only if  $\min \left\{i\in [p]:0 < s_i\leq \sqrt{\lambda}\right\} = \emptyset$ .
$$
\begin{array}{l} T _ {1} = \frac {1}{n} \sum_ {i = 1} ^ {n} \left(w _ {\mathcal {D} _ {n}} ^ {(\lambda)} (X _ {i}, x) - w _ {\mathcal {D} _ {n}} ^ {(0)} (X _ {i}, x)\right) \cdot d ^ {2} (Y _ {i}, y) \\ \leq \frac {\mathrm {d i a m} (\mathcal {M}) ^ {2}}{n} \mathbf {1} _ {n} ^ {\top} \big (\mathbf {X} - \mathbf {1} _ {n} \widehat {\boldsymbol {\mu}} _ {n} ^ {\top} \big) \left[ \widehat {\boldsymbol {\Sigma}} _ {n} ^ {(\lambda), \dagger} - \widehat {\boldsymbol {\Sigma}} _ {n} ^ {\dagger} \right] (x - \widehat {\boldsymbol {\mu}} _ {n}) \\ = \operatorname {d i a m} (\mathcal {M}) ^ {2} \cdot \mathbf {1} _ {n} ^ {\top} \left(\boldsymbol {X} - \mathbf {1} _ {n} \widehat {\mu} _ {n} ^ {\top}\right) ^ {\dagger , \top} \cdot \Pi_ {\mathcal {V} _ {n} ^ {(\lambda)}} \cdot (x - \widehat {\mu} _ {n}) \quad \because \tag {24} \\ = o _ {p} (1). \\ \end{array}
$$