SlowGuess/NeurIPS_2023_OCR huggingface dataset:
When  $\mathcal{D}_n = \{(X_i,Y_i)\in \mathbb{R}^p\times \mathcal{M}:i\in [n]\}$  is an IID sample from  $P_{X,Y}$ , the  $\lambda$ -regularized estimator  $\varphi_{\mathcal{D}_n}^{(\lambda)}$  subsumes the sample-analogue estimator  $\widehat{\varphi}_{\mathcal{D}_n}$  in (4) as a special case where  $\lambda = 0$ .
Connection to principal component regression. Here we remark that when  $\mathcal{M}$  is a Euclidean space, the regularized Fréchet regression function  $\varphi_{\nu}^{(\lambda)}$  effectively reduces to the principal component regression. Suppose that  $\mathcal{M} = \mathbb{R}$  and  $\mathcal{D}_n = \{(x_i,y_i)\in \mathbb{R}^p\times \mathbb{R}:i\in [n]\}$  is a given dataset. Then  $\varphi_{\mathcal{D}_n}^{(\lambda)}(x) = \overline{y} +\hat{\beta}_{\lambda}^\top (x - \mu_{\mathcal{D}_n})$  where  $\overline{y} = \frac{1}{n}\sum_{i = 1}^{n}y_{i}$  and  $\hat{\beta}_{\lambda} = [\mathrm{SVT}^{(\lambda)}(\Sigma_{\mathcal{D}_n})]^{\dagger}\cdot \left[\frac{1}{n}\sum_{i = 1}^{n}(x_i - \mu_{\mathcal{D}_n})\cdot (y_i - \overline{y})\right]$ . Observe that  $\hat{\beta}_{\lambda}$  is exactly the regression coefficient of principal component regression applied to the centered dataset  $\mathcal{D}_n^{\mathrm{ctr}} = \{(x_i - \mu_{\mathcal{D}_n},y_i - \overline{y}):i\in [n]\}$  using  $k$  principal components with  $k = \max_{a\in [p]}\{\sigma_a(\Sigma_{\mathcal{D}_n^{\mathrm{ctr}}})\geq \lambda \}$ .
In this section, we investigate properties of  $\varphi_{\nu}^{(\lambda)}$  for  $\lambda \geq 0$ , with a focus on two cases:  $\nu = \mathcal{D}_n$  and  $\nu = \widetilde{\mathcal{D}}_n$ , cf. Section 3.1. By denoting the true distribution that generates  $(X,Y)$  as  $\nu^*$ , we can express  $\varphi_{\mathrm{glo}}$  as  $\varphi_{\nu^*}^{(0)}$ . To analyze the discrepancy between the estimator  $\varphi_{\nu}^{(\lambda)}(x)$  and  $\varphi_{\mathrm{glo}}(x)$ , we examine the relationships depicted in the schematic in Figure 1. Our theoretical findings can be summarized as follows: Even in the presence of covariate noises,  $\varphi_{\widetilde{\mathcal{D}}_n}^{(\lambda)}$  with a suitable  $\lambda > 0$  can effectively eliminate the noise in  $Z$  to estimate  $X$ , thereby reducing the error in estimating  $\varphi_{\mathrm{glo}}$ .
Figure 1: A schematic for the relationship between the regularized Fréchet regression estimators.
4.1 Model assumptions and examples
We impose the following assumptions for our analysis.
(C0) (Existence) For any probability distribution  $\nu$  and any  $\lambda \in \mathbb{R}_+$ , the object  $\varphi_{\nu}^{(\lambda)}(x)$  exists (almost surely) and is unique. In particular,  $\inf_{y \in \mathcal{M}: d(y, \varphi_{\mathrm{glo}}(x)) > \varepsilon} R(y; x) > R(\varphi_{\mathrm{glo}}(x); x)$  for all  $\varepsilon > 0$ , where  $R(y; x) := R_{\nu^*}^{(0)}(y; x)$ .
(C1) (Growth) There exist  $D_{\mathbf{g}} > 0$ ,  $C_{\mathbf{g}} > 0$  and  $\alpha > 1$ , possibly depending on  $x$ , such that for any probability distribution  $\nu$  and any  $\lambda \in \mathbb{R}_{+}$ ,
$$
\left\{ \begin{array}{l l} d \left(y, \varphi_ {\nu} ^ {(\lambda)} (x)\right) <   D _ {\mathrm {g}} & \Longrightarrow R _ {\nu} ^ {(\lambda)} (y; x) - R _ {\nu} ^ {(\lambda)} \left(\varphi_ {\nu} ^ {(\lambda)} (x); x\right) \geq C _ {\mathrm {g}} \cdot d \left(y, \varphi_ {\nu} ^ {(\lambda)} (x)\right) ^ {\alpha}, \\ d \left(y, \varphi_ {\nu} ^ {(\lambda)} (x)\right) \geq D _ {\mathrm {g}} & \Longrightarrow R _ {\nu} ^ {(\lambda)} (y; x) - R _ {\nu} ^ {(\lambda)} \left(\varphi_ {\nu} ^ {(\lambda)} (x); x\right) \geq C _ {\mathrm {g}} \cdot D _ {\mathrm {g}} ^ {\alpha}. \end{array} \right. \tag {10}
$$
(C2) (Bounded entropy) There exists  $C_{\mathbf{e}} > 0$ , possibly depending on  $y$ , such that
$$
\lim  _ {\delta \rightarrow 0} \sup  _ {0} ^ {1} \sqrt {1 + \log \mathfrak {N} \left(B _ {d} (y , \delta) , \delta \varepsilon\right)} d \varepsilon \leq C _ {\mathrm {e}}, \tag {11}
$$
where  $B_{d}(y,\delta)\coloneqq \{y^{\prime}\in \mathcal{M}:d(y,y^{\prime})\leq \delta \}$  and  $\mathfrak{N}(S,\varepsilon)$  is the  $\varepsilon$  -covering number of  $S$
2The formal definition of covering number is provided in Appendix A; see Definition 6.
Assumption (C0) is common to establish the consistency of an M-estimator [55, Chapter 3.2]; in particular, it ensures the weak convergence of the empirical process  $R_{\mathcal{D}_n}^{(\lambda)}$  to the population process  $R_{\nu^*}^{(\lambda)}$  implying convergence of their minimizers. Furthermore, the conditions on the curvature (C1) and the covering number (C2) control the behavior of the objectives near the minimum in order to obtain rates of convergence; it is worth mentioning that (C2) corresponds to a (locally) bounded entropy for every  $y \in \mathcal{M}$ , while (P1) in [40] requires the same condition only with  $y = \varphi_{\mathrm{glo}}(x)$ . These conditions arise from empirical process theory and are also commonly adopted [40, 44, 45].
Here we provide several examples of the space  $\mathcal{M}$ , in which the conditions (C0), (C1) and (C2) are satisfied. We verify the conditions in Appendix A; see Propositions 1, 2, 3, and 4.
Example 1. Let  $\mathcal{M} = (\mathcal{H}, d_{\mathrm{HS}})$  be a finite-dimensional Hilbert space  $\mathcal{H}$  equipped with the Hilbert-Schmidt metric  $d_{\mathrm{HS}}(y_1, y_2) = \langle y_1 - y_2, y_1 - y_2 \rangle^{1/2}$ , e.g.,  $\mathcal{M} = (\mathbb{R}^r, d_2)$  where  $d_2$  is the  $\ell^2$ -metric.
Example 2. Let  $\mathcal{M}$  be  $\mathcal{W}$ , the set of probability distributions  $G$  on  $\mathbb{R}$  such that  $\int_{\mathbb{R}} x^2 \, dG(x) < \infty$ , equipped with the Wasserstein metric  $d_W$  defined as
$$
d _ {W} \left(G _ {1}, G _ {2}\right) ^ {2} = \int_ {0} ^ {1} \left(G _ {1} ^ {- 1} (t) - G _ {2} ^ {- 1} (t)\right) ^ {2} d t,
$$
where  $G_1^{-1}$  and  $G_2^{-1}$  are the quantile functions of  $G_1$  and  $G_2$ , respectively. See [40, Section 6].
Example 3. Let  $\mathcal{M} = \left\{M \in \mathbb{R}^{r \times r} : M = M^T, M \succeq 0 \text{ and } M_{ii} = 1, \forall i \in [r]\right\}$  be the set of correlation matrices of size  $r$ , equipped with the Frobenius metric,  $d_F(M, M') = \|M - M'\|_F$ .
Example 4. Let  $\mathcal{M}$  be a (bounded) Riemannian manifold of dimension  $r$ , and let  $d_g$  be the geodesic distance induced by the Riemannian metric.
4.2 Theorem statements
4.2.1 Noiseless covariate setting
We first verify the consistency of the  $\lambda$ -regularized Fréchet regression function as follows.
Theorem 1 (Consistency). Suppose that Assumption (C0) holds. If  $\mathrm{diam}(\mathcal{M}) < \infty$ , then for any  $\lambda \in \mathbb{R}$  such that  $0 \leq \lambda < \min \{\sigma_i(\Sigma_{\nu^*}) : \sigma_i(\Sigma_{\nu^*}) > 0\}$ , and any  $x \in \mathbb{R}^p$ ,
$$
d \left(\varphi_ {\mathcal {D} _ {n}} ^ {(\lambda)} (x), \varphi_ {\nu^ {*}} ^ {(0)} (x)\right) = o _ {P} (1) \quad a s n \rightarrow \infty . \tag {12}
$$
If  $\lambda < \sigma^{(0)}(\Sigma_{\nu^*}) = \min \{\sigma_i(\Sigma_{\nu^*}):\sigma_i(\Sigma_{\nu^*}) > 0\}$ , then the regularized estimator  $\varphi_{\mathcal{D}_n}^{(\lambda)}(x)$  effectively reduces to the same as the sample-analog estimator  $\widehat{\varphi}_{\mathcal{D}_n}(x)$  in (4) in the limit  $n\to \infty$ . Thus,  $\varphi_{\mathcal{D}_n}^{(\lambda)}(x)$  inherits the consistency of  $\widehat{\varphi}_{\mathcal{D}_n}$ . We provide a detailed proof of Theorem 1 in Appendix B.
In addition to the consistency of  $\varphi_{\mathcal{D}_n}^{(\lambda)}$  in the small  $\lambda$  limit, we present an analysis for the convergence rate of  $\varphi_{\mathcal{D}_n}^{(\lambda)}(x)$  in the following theorem.
Definition 5. The Mahalanobis seminorm of  $x$  induced by a positive semidefinite matrix  $\Sigma$  is  $\| x\|_{\Sigma} \coloneqq (x^{\top}\Sigma^{\dagger}x)^{1 / 2}$ .
Theorem 2 (Rate of convergence). Suppose that Assumptions (C0)-(C2) hold. If  $\mathrm{diam}(\mathcal{M}) < \infty$ , then for any  $\lambda \in \mathbb{R}_+$  and  $x \in \mathbb{R}^p$  such that  $\| x - \mu_{\nu^*}\|_{\Sigma_{\nu^*}} \leq \frac{C_g \cdot D_g^\alpha}{\mathrm{diam}(\mathcal{M})^2 \cdot \sqrt{\mathrm{rank} \Sigma_{\nu^*}}}$ ,
$$
d \left(\varphi_ {\mathcal {D} _ {n}} ^ {(\lambda)} (x), \varphi_ {\nu^ {*}} ^ {(0)} (x)\right) = O _ {P} \left(b _ {\lambda} (x) ^ {\frac {1}{\alpha - 1}} + n ^ {- \frac {1}{2 (\alpha - 1)}}\right) \quad a s \quad n \rightarrow \infty , \tag {13}
$$
where  $b_{\lambda}(x) = \mathrm{rank}\left(\Sigma_{\nu^{*}} - \Sigma_{\nu^{*}}^{(\lambda)}\right)^{\frac{1}{2}}\cdot \| x - \mu_{\nu^{*}}\|_{\Sigma_{\nu^{*}} - \Sigma_{\nu^{*}}^{(\lambda)}}$
We obtain Theorem 2 by showing a "bias" upper bound  $d\big(\varphi_{\nu^{*}}^{(\lambda)}(x), \varphi_{\nu^{*}}^{(0)}(x)\big) = O\big(\mathsf{b}_{\lambda}(x)^{\frac{1}{\alpha - 1}}\big)$  and a "variance" bound  $d\big(\varphi_{\mathcal{D}_n}^{(\lambda)}(x), \varphi_{\nu^*}^{(\lambda)}(x)\big) = O_P\big(n^{-\frac{1}{2(\alpha - 1)}}\big)$ ; see Lemmas 1 and 2 in Appendix C. Here we remark that  $\mathsf{b}_{\lambda}(x)$  is a monotone non-decreasing function of  $\lambda$ , and if  $\lambda < \sigma^{(0)}(\Sigma_{\nu^{*}})$  then  $\mathsf{b}_{\lambda}(x) = 0$ . Also, the condition on  $\| x - \mu_{\nu^{*}} \|_{\Sigma_{\nu^{*}}}$  is introduced for a technical reason, and can be removed when  $D_{\mathbf{g}} = \infty$ . Note that Condition (C1) holds with  $D_{\mathbf{g}} = \infty$  and  $\alpha = 2$  for Examples 1, 2 and 3. Thus, we have  $d\big(\varphi_{\mathcal{D}_n}^{(\lambda)}(x), \varphi_{\nu^{*}}^{(0)}(x)\big) = O_P\big(\mathsf{b}_{\lambda}(x) + n^{-\frac{1}{2}}\big)$  as  $n \to \infty$ .
4.2.2 Error-prone covariate setting
Given a set  $\mathcal{D}_n = \{(x_i, y_i) : i \in [n]\}$ , let  $X_{\mathcal{D}_n} := [x_1 \cdots x_n]^\top \in \mathbb{R}^{n \times p}$ . We let  $\pmb{X} = \pmb{X}_{\mathcal{D}_n}$  and  $\pmb{Z} = \pmb{X}_{\widetilde{\mathcal{D}}_n}$  for shorthand, and further, we let  $\pmb{X}_{\mathrm{ctr}} = \left(I_n - \frac{1}{n}\pmb{1}_n\pmb{1}_n^\top\right)\pmb{X}$  and  $\pmb{Z}_{\mathrm{ctr}} = \left(I_n - \frac{1}{n}\pmb{1}_n\pmb{1}_n^\top\right)\pmb{Z}$  denote the 'row-centered' matrices.
Theorem 3 (De-noising covariates). Suppose that Assumptions (C0) and (C1) hold. Then there exists a constant  $C > 0$  such that for any  $\lambda \in \mathbb{R}_+$ , if  $x \in \mu_{\mathcal{D}_n} + \mathrm{rowsp} X_{\mathrm{ctr}}$  and
$$
\left\| x - \mu_ {\mathcal {D} _ {n}} \right\| _ {\Sigma_ {\mathcal {D} _ {n}}} \leq \frac {1}{2} \left(\frac {C _ {g} \cdot D _ {g} ^ {\alpha}}{2 \operatorname {d i a m} (\mathcal {M})} \cdot \frac {\sigma^ {(\lambda)} \left(\boldsymbol {X} _ {\mathrm {c t r}}\right) \wedge \sigma^ {(\lambda)} \left(\boldsymbol {Z} _ {\mathrm {c t r}}\right)}{\| \boldsymbol {Z} - \boldsymbol {X} \|} - 1\right), \tag {14}
$$
$$
d \left(\varphi_ {\widetilde {\mathcal {D}} _ {n}} ^ {(\lambda)} (x), \varphi_ {\mathcal {D} _ {n}} ^ {(\lambda)} (x)\right) \leq C \cdot \left(\frac {\| \boldsymbol {Z} - \boldsymbol {X} \|}{\sigma^ {(\lambda)} \left(\boldsymbol {X} _ {\mathrm {c t r}}\right) \wedge \sigma^ {(\lambda)} \left(\boldsymbol {Z} _ {\mathrm {c t r}}\right)} \cdot \frac {2 \cdot \left\| x - \mu_ {\mathcal {D} _ {n}} \right\| _ {\Sigma_ {\mathcal {D} _ {n}}} + 1}{C _ {\mathbf {g}}}\right) ^ {\frac {1}{\alpha}}. \tag {15}
$$
Note that the condition on  $\| x - \mu_{\nu^{*}}\|_{\Sigma_{\nu^{*}}}$  in (14) can be removed when  $D_{g} = \infty$ . We highlight that the quantity  $\frac{\|\pmb{Z} - \pmb{X}\|}{\sigma^{(\lambda)}(\pmb{X}_{\mathrm{ctr}})\wedge\sigma^{(\lambda)}(\pmb{Z}_{\mathrm{ctr}})}$  acts as the reciprocal of the signal-to-noise ratio. Here,  $\| Z - X\|$  quantifies the magnitude of the "noise" in the covariates, while  $\min \left\{\sigma^{(\lambda)}(\pmb{X}_{\mathrm{ctr}}),\sigma^{(\lambda)}(\pmb{Z}_{\mathrm{ctr}})\right\}$  measures the strength of the "signal" retained in the  $\lambda$ -SVT of the design matrix. We observe that the error bound (15) increases proportionally to the normalized deviation of  $x$  from the mean,  $\mu_{\mathcal{D}_n}$ , which is a reasonable outcome. For the complete version of Theorem 3 and its proof, refer to Appendix D.
Remarks on Theorem 3. We avoid imposing distributional assumptions on the noise  $\varepsilon = Z - X$ , to ensure broad applicability of the result. Also, the inequality (15) is sharp, as there is a worst-case noise instance that attains equality (up to a multiplicative constant). Despite its generality, this upper bound highlights effective error mitigation in specific scenarios. For instance, consider well-balanced, effectively low-rank covariates  $X \in \mathbb{R}^{n \times p}$  such that  $|X_{ij}| = \Omega(1)$  for all  $i, j$  and  $\sigma_1(X) \asymp \sigma_r(X) \gg \sigma_{r+1}(X) \asymp \sigma_{n \wedge p}(X) = O(1)$ , where  $r \ll n \wedge p$  is the effective rank of  $X$ . Then  $\sigma_1(X)^2 \asymp \sigma_r(X)^2 \asymp \|X\|_F^2 / r \gtrsim np / r$ . Additionally, if  $Z = X + E$  where  $E$  is a random matrix with independent sub-Gaussian rows, then  $\|Z - X\| \lesssim \sqrt{n} + \sqrt{p}$  with high probability. In the random design scenario where the rows of  $X$  and the test point  $x$  are drawn IID from the same distribution,  $\|x - \mu_{D_n}\|_{\Sigma} \approx 1$  with high probability. Consequently, the upper bound in (15) is bounded by  $\sqrt{r / p} + \sqrt{r / n}$ , which diminishes to 0 when  $r \ll n \wedge p$ .
4.3 Proof sketches
Proof of Theorem 1. We show that  $R_{\mathcal{D}_n}^{(\lambda)}(y;x)$  weakly converges to  $R_{\nu^*}^{(0)}(y;x)$  in the  $\ell^\infty (\mathcal{M})$ -sense. According to [55, Theorem 1.5.4], it suffices to show that (1)  $R_{\mathcal{D}_n}^{(\lambda)}(y;x) - R_{\nu^*}^{(0)}(y;x) = o_p(1)$  for all  $y \in \mathcal{M}$ , and (2)  $R_{\mathcal{D}_n}^{(\lambda)}$  is asymptotically equicontinuous in probability.
Proof of Theorem 2. We prove upper bounds for the bias and the variance separately.
To control the bias (Appendix C, Lemma 1), we show an upper bound for  $R\big(\varphi^{(\lambda)}(x);x\big) - R\big(\varphi (x);x\big)$  and convert it to restrain the distance between the minimizers  $d\big(\varphi^{(\lambda)}(x),\varphi (x)\big)$  using the Growth condition (C1). In this conversion, we employ the "peeling technique" in empirical process theory.
To control the variance (Appendix C, Lemma 2), we follow a similar strategy as in Lemma 1, but with additional technical considerations. Defining the 'fluctuation variable'  $Z_{n}^{(\lambda)}(y;x) \coloneqq R_{\mathcal{D}_{n}}^{(\lambda)}(y;x) - R_{\nu^{*}}^{(\lambda)}(y;x)$  parameterized by  $y \in \mathcal{M}$ , we derive a probabilistic upper bound for  $R_{\nu^{*}}^{(\lambda)}(\varphi_{\mathcal{D}_{n}}^{(\lambda)}(x);x) - R_{\nu^{*}}^{(\lambda)}(\varphi_{\nu^{*}}^{(\lambda)}(x);x)$  by establishing a uniform upper bound for  $Z_{n}^{(\lambda)}(y;x) - Z_{n}^{(\lambda)}(\varphi(x);x)$ ; here, the Entropy condition (C2) is used. Again, we use the Growth condition (C1) and the peeling technique to obtain a probabilistic upper bound for the distance  $d(\varphi_{\mathcal{D}_{n}}^{(\lambda)}(x),\varphi_{\nu^{*}}^{(\lambda)}(x))$ .
Proof of Theorem 3. Expressing the difference in the weights  $w_{\widetilde{\mathcal{D}}_n}^{(\lambda)}(y;x) - w_{\mathcal{D}_n}^{(\lambda)}(y;x)$  in terms of  $X$  and  $Z$ , we utilize classical matrix perturbation theory to control  $R_{\widetilde{\mathcal{D}}_n}^{(\lambda)}(y;x) - R_{\mathcal{D}_n}^{(\lambda)}(y;x)$ , and transform it to an upper bound on the distance  $d(\varphi_{\widetilde{\mathcal{D}}_n}^{(\lambda)}(x),\varphi_{\mathcal{D}_n}^{(\lambda)}(x))$  using the Growth condition (C1).
In this section, we present numerical simulation results to validate and support our theoretical findings. We focus on global Fréchet regression analysis for one-dimensional distribution functions (Example 2). These simulations cover various conditions, allowing us to evaluate and compare our methodology's performance with alternative approaches. For a summary of the experimental results, please refer to Figure 2 and Table 1. Further details about simulation settings and additional results are provided in Appendix E.
Experimental setup. We consider combinations of  $p \in \{150, 300, 600\}$  and  $n \in \{100, 200, 400\}$ . The datasets  $\mathcal{D}_n = \{(X_i, Y_i) : i \in [n]\}$  and  $\widetilde{\mathcal{D}}_n = \{(Z_i, Y_i) : i \in [n]\}$  are generated as follows. Let  $X_i \sim \mathcal{N}_p(\mathbf{0}_p, \Sigma)$  be IID multivariate Gaussian with mean  $\mathbf{0}_p$  and covariance  $\Sigma$  such that  $\operatorname{spec}(\Sigma) = \{\kappa_j > 0 : j \in [p]\}$  is an exponentially decreasing sequence such that  $\operatorname{tr}(\Sigma) = \sum_{j=1}^{p} \kappa_j = p$  and  $\kappa_1 / \kappa_p = 10^3$ . Note that  $\sum_{j=1}^{\lfloor p/3 \rfloor} \kappa_j / \sum_{j'=1}^{p} \kappa_{j'} \approx 0.9$ , and thus,  $\Sigma$  is effectively low-rank. We generate  $Z_i$  following (5) under two scenarios  $\varepsilon_{ij} \stackrel{IID}{\sim} \mathcal{N}(0, \sigma_\varepsilon^2)$  and  $\varepsilon_{ij} \stackrel{IID}{\sim} \mathrm{Laplace}(0, \sigma_\varepsilon)$ , respectively. Lastly, given  $X = x$ , let  $Y$  be the distribution function of  $\mathcal{N}\big(\mu_{\alpha,\beta}(x) + \eta, \tau^2\big)$ , where (i)  $\mu_{\alpha,\beta}(x) = \alpha + \beta^\top x$  with  $\alpha = 1$  and  $\beta = p^{-1/2} \cdot \mathbf{1}_p$ ; (ii)  $\eta \sim \mathcal{N}(0, \sigma_\eta^2)$ ; and (iii)  $\tau^2 \sim \mathcal{IG}(s_1, s_2)$ , an inverse gamma distribution with shape  $s_1$  and scale  $s_2$ . We performed  $B = 500$  Monte Carlo experiments by drawing  $\mathcal{D}_n^{(b)}$  and  $\widetilde{\mathcal{D}}_n^{(b)}$  as independent copies of  $\mathcal{D}_n$  and  $\widetilde{\mathcal{D}}_n$ , respectively, for  $b \in [B]$ .
Performance evaluation. We assess the in-sample and out-of-sample performance of the Fréchet regression function estimator by using the mean squared error (MSE) and the mean squared prediction error (MSPE). To this end, we create a "test set"  $\mathcal{D}_N^{\mathrm{new}} = \{(X_i^{\mathrm{new}},Y_i^{\mathrm{new}}):i\in [N]\}$ , with  $N = 1000$ . The MSE and the MSPE are computed as the average of squared metric-distance residuals from the observed responses in the "training set"  $\mathcal{D}_n$  and in the "test set"  $\mathcal{D}_N^{\mathrm{new}}$ , respectively:
$$
\mathrm {M S E} (\varphi_ {\nu} ^ {(\lambda)}) = \frac {1}{n} \sum_ {i = 1} ^ {n} d _ {W} \big (Y _ {i}, \varphi_ {\nu} ^ {(\lambda)} (X _ {i}) \big) ^ {2} \mathrm {a n d} \mathrm {M S P E} (\varphi_ {\nu} ^ {(\lambda)}) = \frac {1}{N} \sum_ {i = 1} ^ {N} d _ {W} \big (Y _ {i} ^ {\mathrm {n e w}}, \varphi_ {\nu} ^ {(\lambda)} (X _ {i} ^ {\mathrm {n e w}}) \big) ^ {2}.
$$
We report the MSE averaged over  $B = 500$  random trials:  $\mathrm{MSE}(\varphi_{\nu}^{(\lambda)}) = B^{-1}\sum_{b = 1}^{B}\mathrm{MSE}(\varphi_{\nu^{(b)}}^{(\lambda)})$ , and likewise for MSPE. Furthermore, we evaluate the accuracy and efficiency of the estimator using bias and variance, with detailed definitions deferred to Appendix E.
Simulation results. Our numerical study demonstrates that the proposed SVT method consistently improves both estimation and prediction performance, especially in the errors-in-variables setting. Figure 2 highlights how the SVT estimator outperforms the naive errors-in-variables (EIV) estimator, which corresponds to SVT with  $\lambda = 0$ . The naive EIV suffers from an intrinsic model bias, called the attenuation effect [13], as it regresses responses on error-prone covariates. This leads to a misrepresentation of the association between responses and true covariates, potentially leading to statistical inference based on a mis-specified model.
Figure 2: Comparison of the prediction performance of  $\varphi_{\mathcal{D}_n}^{(0)}$  (REF),  $\varphi_{\widetilde{\mathcal{D}}_n}^{(0)}$  (EIV), and  $\varphi_{\widetilde{\mathcal{D}}_n}^{(\lambda)}$  (SVT) (left), and the trade-off between the bias and the variance (right) for  $B = 500$ ,  $p = 50$  and  $n = 100$ .
Table 1: Average performance of  $\varphi_{\mathcal{D}_n}^{(0)}$  (REF),  $\varphi_{\widetilde{\mathcal{D}}_n}^{(0)}$  (EIV), and  $\varphi_{\widetilde{\mathcal{D}}_n}^{(\lambda)}$  (SVT) in various settings (boldface indicates the best). The choice of threshold  $\lambda$  for SVT is detailed in Appendix E.
Error distribution	Sample size	n = 100	n = 200	n = 400
p	Criterion	REF	EIV	SVT	REF	EIV	SVT	REF	EIV	SVT
Gaussian N(0,0.052)	150	Bias	0.107	0.164	0.312	0.040	0.087	0.189	0.020	0.078	0.136
√Var	1.147	0.973	0.531	0.969	0.857	0.414	0.432	0.381	0.295
MSE	0.000	0.154	0.205	0.075	0.334	0.232	0.190	0.255	0.267
MSPE	1.613	1.267	0.686	1.245	1.042	0.513	0.492	0.456	0.412
300	Bias	0.317	0.352	0.485	0.112	0.170	0.327	0.039	0.091	0.207
√Var	0.805	0.731	0.523	1.137	0.970	0.512	0.964	0.845	0.405
MSE	0.000	0.033	0.217	0.000	0.152	0.235	0.075	0.327	0.237
MSPE	1.045	0.956	0.808	1.602	1.267	0.667	1.232	1.025	0.509
600	Bias	0.568	0.592	0.619	0.311	0.353	0.451	0.104	0.155	0.298
√Var	0.663	0.631	0.598	0.799	0.735	0.589	1.135	0.956	0.510
MSE	0.000	0.016	0.067	0.000	0.036	0.112	0.000	0.153	0.208
MSPE	1.075	1.062	1.054	1.039	0.968	0.858	1.602	1.242	0.657
Laplacian DE(0,0.05)	150	Bias	0.102	0.112	0.286	0.045	0.053	0.171	0.019	0.025	0.121
√Var	1.155	1.113	0.549	0.971	0.949	0.421	0.431	0.416	0.297
MSE	0.000	0.026	0.187	0.075	0.151	0.218	0.189	0.205	0.253
MSPE	1.633	1.538	0.688	1.254	1.207	0.513	0.489	0.477	0.406
300	Bias	0.321	0.325	0.491	0.105	0.117	0.344	0.043	0.048	0.198
√Var	0.805	0.794	0.525	1.140	1.099	0.511	0.960	0.929	0.413
MSE	0.000	0.003	0.233	0.000	0.025	0.227	0.076	0.148	0.229
MSPE	1.049	1.034	0.814	1.610	1.521	0.677	1.226	1.169	0.514
600	Bias	0.566	0.568	0.608	0.312	0.317	0.443	0.102	0.109	0.282
√Var	0.664	0.661	0.614	0.800	0.792	0.606	1.134	1.094	0.521
MSE	0.000	0.001	0.074	0.000	0.004	0.157	0.000	0.025	0.200
MSPE	1.073	1.071	1.059	1.045	1.035	0.872	1.602	1.515	0.662
Remarkably, the SVT estimator achieved a smaller MSPE even compared to the oracle estimator (REF) obtained from the error-free sample. Although the REF estimator had the smallest MSE due to its small bias, we observed its overfitting to the training sample, resulting in poor prediction performance. Notably, even the naïve EIV estimator outperformed the REF estimator in MSPE. We believe this is mainly because the true covariate matrix was nearly singular in our simulation setup, causing multicollinearity issues for the REF. In contrast, measurement errors introduced non-ignorable minimum singular values in the EIV covariate matrix, unintentionally mitigating multicollinearity for the naïve EIV and causing it to behave like ridge regression.
This paper has addressed errors-in-variables regression of non-Euclidean response variables through the (global) Fréchet regression framework enhanced by low-rank approximation of covariates. Specifically, we introduce a novel regularized (global) Fréchet regression framework (Section 3), which combines the Fréchet regression with principal component regression. We also provide a comprehensive theoretical analysis in three main theorems (Section 4), and validate our theory through numerical experiments on simulated datasets. Moreover, our numerical experiments demonstrate empirical evidence of the effectiveness and superiority of our approach, reinforcing its practical relevance and potential impact in non-Euclidean regression analysis.