SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(d) Entry  $i = 1, j = 5000$ , an entry in the upper-right block
(e) Entry  $i = 1, j = 2$ , a non-diagonal entry in the upper-left block
Figure 6: Illustration of (42).  $\mathbf{X}$  is diagonal matrix from the eigenvalues of a Wishart matrix with aspect ratio  $1/2$  and  $\mathbf{Y}$ ,  $\mathbf{Z}$  are Gaussian matrices with  $N = 2000$ ,  $M = 3000$ . The empirical estimate of  $G_{S}(z)$  (dashed blue line) is computed for  $z = \gamma_{i} - \mathrm{i}\sqrt{\frac{1}{2N}}$  for  $1 \leq i \leq N$ . The Theoretical estimate (solid orange line) is computed from the rhs of (42) with parameters obtained from the generated matrix. Note that, the theoretical estimate has also fluctuations because the parameters  $\zeta_1^*, \zeta_3^*$  are given by the numerical estimate of  $G_{\bar{\mu}_S}(z)$ .
(a) Wigner  $\pmb{X}$
(b) Square root Wishart  $\pmb{X}$  with  $\alpha^{\prime} = 1 / 4$
Figure 7: Computation of the rescaled overlap. Both  $\mathbf{Y}$  and  $\mathbf{W}$  are  $N\times M$  matrices with i.i.d. Gaussian entries of variance  $1 / N$ , and aspect ratio  $N / M = 1 / 2$ . The simulation results are averaged over 1000 experiments with fixed  $\mathbf{X}$ , and  $N = 1000$ ,  $M = 2000$ . Some of the simulation points are dropped for clarity.
$$
p _ {2} ^ {*} = \frac {1}{z - \zeta_ {2} ^ {*}} = \frac {\zeta_ {1} ^ {*}}{\zeta_ {1} ^ {*} z - \frac {z}{\zeta_ {1} ^ {*}} \mathcal {G} _ {\rho_ {X ^ {2}}} \left(\frac {z}{\zeta_ {1} ^ {*}} - 1\right) + 1}
$$
$$
\begin{array}{l} \zeta_ {1} ^ {*} = \frac {1}{\alpha} p _ {2} ^ {*} \Longrightarrow \zeta_ {1} ^ {*} z - \frac {z}{\zeta_ {1} ^ {*}} \mathcal {G} _ {\rho_ {X ^ {2}}} \left(\frac {z}{\zeta_ {1} ^ {*}} - 1\right) + 1 = \frac {1}{\alpha} \\ \Rightarrow \mathcal {G} _ {\rho_ {X ^ {2}}} \Big (\frac {z}{\zeta_ {1} ^ {*}} - 1 \Big) = \zeta_ {1} ^ {* 2} + \Big (1 - \frac {1}{\alpha} \Big) \frac {\zeta_ {1} ^ {*}}{z} \\ \Rightarrow \frac {z}{\zeta_ {1} ^ {*}} - 1 = \mathcal {G} _ {\rho_ {X ^ {2}}} ^ {- 1} \left(\zeta_ {1} ^ {* 2} + \left(1 - \frac {1}{\alpha}\right) \frac {\zeta_ {1} ^ {*}}{z}\right) \tag {54} \\ \Rightarrow \frac {z}{\zeta_ {1} ^ {*}} - 1 - \frac {1}{\zeta_ {1} ^ {* 2} + \left(1 - \frac {1}{\alpha}\right) \frac {\zeta_ {1} ^ {*}}{z}} = \mathcal {R} _ {\rho_ {X ^ {2}}} \left(\zeta_ {1} ^ {* 2} + \left(1 - \frac {1}{\alpha}\right) \frac {\zeta_ {1} ^ {*}}{z}\right) \\ \end{array}
$$
Thus,  $\zeta_1^*$  is the solution to (54). For each example, we solve this equation and compare the obtained theoretical overlap against the average over the experiments.
Wigner  $X$ . Let  $X \in \mathbb{R}^{N \times N}$  be a Wigner matrix, then  $\mathcal{R}_{\rho_{X^2}}(z) = \frac{1}{1 - z}$ . Solving (54), we can compute the overlap using (48). In Fig. 7a, we compare the theoretical computation with simulations for  $N = 1000$ ,  $M = 2000$ . As in previous cases  $\bar{\mu}_S(\gamma)$  is approximated using a Cauchy kernel [58].
Square root Wishart  $X$ . Let  $X \in \mathbb{R}^{N \times N}$  be the square root of a Wishart matrix  $X = \sqrt{\frac{1}{N} \mathbf{H} \mathbf{H}^{\mathrm{T}}}$  with  $\mathbf{H} \in \mathbb{R}^{N \times N'}$  having i.i.d. Gaussian entries. Then  $\mathcal{R}_{\rho_{X^2}}(z) = \frac{1}{\alpha'} \frac{1}{1 - z}, \alpha' = N / N'$ . Solving (54), we can compute the overlap using (48). In Fig. 7b, we compare the theoretical computation with simulations for  $N = 1000$ ,  $N' = 4000$ ,  $M = 2000$ .
C.3.3 RIE performance
In this section, we investigate the performance of our proposed estimators for  $X$ . We compare performances of the optimal RIE (49) with the one of Oracle estimator (3). Moreover, we illustrate the performance of the estimator for  $X^2$  (50), and the sub-optimal estimator of  $X$  derived from it, see remark 4.
For  $\mathbf{Y}$ ,  $\mathbf{W}$  with Gaussian i.i.d. entries, (51) simplifies to:
$$
\begin{array}{l} \widehat {\xi_ {x ^ {2} i} ^ {*}} = \frac {1}{\kappa} \frac {1}{\pi \bar {\mu} _ {S} (\gamma_ {i})} \operatorname {I m} \lim  _ {z \rightarrow \gamma_ {i} - \mathrm {i} 0 ^ {+}} \frac {z - \zeta_ {1} ^ {*}}{\zeta_ {3} ^ {*}} \mathcal {G} _ {\bar {\mu} _ {S}} (z) - \frac {1}{\zeta_ {3} ^ {*}} \\ = \frac {1}{\kappa} \frac {1}{\pi \bar {\mu} _ {S} (\gamma_ {i})} \operatorname {I m} \lim  _ {z \rightarrow \gamma_ {i} - \mathrm {i} 0 ^ {+}} \frac {z}{\zeta_ {1} ^ {*}} \mathcal {G} _ {\bar {\mu} _ {S}} (z) - \mathcal {G} _ {\bar {\mu} _ {S}} - \frac {1}{\zeta_ {1} ^ {*}} \\ = \frac {1}{\kappa} \frac {1}{\pi \bar {\mu} _ {S} (\gamma_ {i})} \operatorname {I m} \lim  _ {z \rightarrow \gamma_ {i} - \mathrm {i} 0 ^ {+}} \frac {z}{\mathcal {G} _ {\bar {\mu} _ {S}} (z) + \frac {1 - \alpha}{\alpha} \frac {1}{z}} \mathcal {G} _ {\bar {\mu} _ {S}} (z) - \mathcal {G} _ {\bar {\mu} _ {S}} (z) - \frac {1}{\mathcal {G} _ {\bar {\mu} _ {S}} (z) + \frac {1 - \alpha}{\alpha} \frac {1}{z}} \\ = \frac {1}{\kappa} \frac {1}{\pi \bar {\mu} _ {S} (\gamma_ {i})} \operatorname {I m} \left\{\frac {\gamma_ {i}}{\pi \mathsf {H} [ \bar {\mu} _ {S} ] (\gamma_ {i}) + \pi \mathrm {i} \bar {\mu} _ {S} (\gamma_ {i}) + \frac {1 - \alpha}{\alpha} \frac {1}{\gamma_ {i}}} \big (\pi \mathsf {H} [ \bar {\mu} _ {S} ] (\gamma_ {i}) + \pi \mathrm {i} \bar {\mu} _ {S} (\gamma_ {i}) \big) \right. \\ \left. - \left(\pi \mathsf {H} [ \bar {\mu} _ {S} ] (\gamma_ {i}) + \pi \mathsf {i} \bar {\mu} _ {S} (\gamma_ {i})\right) - \frac {1}{\pi \mathsf {H} [ \bar {\mu} _ {S} ] (\gamma_ {i}) + \pi \mathsf {i} \bar {\mu} _ {S} (\gamma_ {i}) + \frac {1 - \alpha}{\alpha} \frac {1}{\gamma_ {i}}} \right\} \\ = \frac {1}{\kappa} \frac {1}{\pi \bar {\mu} _ {S} (\gamma_ {i})} \pi \bar {\mu} _ {S} (\gamma_ {i}) \bigg (- 1 + \frac {1}{\alpha \left(\pi^ {2} \bar {\mu} _ {S} (\gamma_ {i}) ^ {2} + \left(\pi \mathsf {H} [ \bar {\mu} _ {S} ] (\gamma_ {i}) + \frac {- 1 + \frac {1}{\alpha}}{\gamma_ {i}}\right) ^ {2}\right)} \bigg) \\ = \frac {1}{\kappa} \left[ - 1 + \frac {1}{\alpha \left(\pi^ {2} \bar {\mu} _ {S} \left(\gamma_ {i}\right) ^ {2} + \left(\pi H [ \bar {\mu} _ {S} ] \left(\gamma_ {i}\right) + \frac {- 1 + \frac {1}{\alpha}}{\gamma_ {i}}\right) ^ {2}\right)} \right] \tag {55} \\ \end{array}
$$
For our first example, we consider two priors for  $X$ :
Shifted Wigner  $X$ . We consider  $\mathbf{X} = \mathbf{F} + c\mathbf{I}$  where  $\mathbf{F} = \mathbf{F}^{\intercal} \in \mathbb{R}^{N \times N}$  has i.i.d. entries with variance  $1/N$ , and  $c \neq 0$  is a real number. Then, the spectrum of  $\mathbf{X}$  is a shifted version of the Wigner law
$$
\rho_ {X} (\lambda) = \frac {\sqrt {4 - (\lambda - c) ^ {2}}}{2 \pi}, \quad \text {f o r} c - 2 <   \lambda <   c + 2,
$$
and the Stieltjes transform reads:
$$
\mathcal {G} _ {\rho_ {X}} (z) = \frac {z - c - \sqrt {(z - 2 - c) (z + 2 - c)}}{2}
$$
Wishart  $X$ . Take  $\mathbf{X} = \frac{1}{N}\mathbf{H}\mathbf{H}^{\intercal}$  with  $\mathbf{H} \in \mathbb{R}^{N \times N'}$  having i.i.d. Gaussian entries, with  $N/N' = \alpha' \leq 1$ . Then, the spectrum of  $\mathbf{X}$  is the renowned Marchenko-Pastur distribution:
$$
\rho_ {X} (\lambda) = \frac {\sqrt {\left[ \lambda - \left(\frac {1}{\sqrt {\alpha^ {\prime}}} - 1\right) ^ {2} \right] \left[ \left(\frac {1}{\sqrt {\alpha^ {\prime}}} + 1\right) ^ {2} - \lambda \right]}}{2 \pi \lambda}, \quad \mathrm {f o r} \left(\frac {1}{\sqrt {\alpha^ {\prime}}} - 1\right) ^ {2} <   \lambda <   \left(\frac {1}{\sqrt {\alpha^ {\prime}}} + 1\right) ^ {2},
$$
$$
\mathcal {G} _ {\rho_ {X}} (z) = \frac {z - \left(\frac {1}{\alpha^ {\prime}} - 1\right) - \sqrt {\left[ z - \left(\frac {1}{\sqrt {\alpha^ {\prime}}} - 1\right) ^ {2} \right] \left[ z - \left(\frac {1}{\sqrt {\alpha^ {\prime}}} + 1\right) ^ {2} \right]}}{2 z}
$$
In Figure 8, the MSE of Oracle estimator, RIE (49), and  $\sqrt{X^2}$ -RIE is illustrated for shifted Wigner  $X$  with  $c = 3$ , and Wishart with aspect-ratio  $\alpha' = 1/4$ . We see that the performance of RIE is close to the one of Oracle estimator, which implies the optimality of the proposed estimator (49). Moreover, we observe the sub-optimality of estimating  $X$  using  $\sqrt{\widehat{\Xi_{X^2}^*}(S)}$ . Note that, in the low-SNR regime, the estimated eigenvalues  $\widehat{\xi_{x^2 i}^*}$  might be negative which makes the estimator  $\sqrt{\widehat{\Xi_{X^2}^*}(S)}$  undefined, so the MSE is not depicted in this case.
In Figure 9, the MSE of estimating  $X^2$  is shown. We see that in the high-SNR regimes the RIE (55) has the same performance as the Oracle estimator.
(a) Shifted Wigner,  $c = 3$
(b) Wishart,  $\alpha^{\prime} = 1 / 4$
Figure 8: Estimating  $\mathbf{X}$ . The MSE is normalized by the norm of the signal,  $\| \mathbf{X} \|_{\mathrm{F}}^2$ . Both  $\mathbf{Y}$  and  $\mathbf{W}$  are  $N \times M$  matrices with i.i.d. Gaussian entries of variance  $1/N$ , and aspect ratio  $N/M = 1/2$ . The RIE is applied to  $N = 2000$ ,  $M = 4000$ , and the results are averaged over 10 runs (error bars are invisible). Average relative error between RIE  $\widehat{\Xi}_X^*(S)$  and Oracle estimator is also reported.
(a) Shifted Wigner,  $c = 3$
Figure 9: Estimating  $\mathbf{X}^2$ . The MSE is normalized by the norm of the signal,  $\| \mathbf{X}^2 \|_{\mathrm{F}}^2$ . Both  $\mathbf{Y}$  and  $\mathbf{W}$  are  $N \times M$  matrices with i.i.d. Gaussian entries of variance  $1 / N$ , and aspect ratio  $N / M = 1 / 2$ . The RIE is applied to  $N = 2000$ ,  $M = 4000$ , and the results are averaged over 10 runs (error bars are invisible). Average relative error between RIE  $\widehat{\Xi_X^*}(S)$  and Oracle estimator is also reported.
Bernoulli spectral distribution. In this case, the matrix  $\mathbf{X}$  is constructed as  $\mathbf{X} = \mathbf{U}_X \boldsymbol{\Lambda} \mathbf{U}_X^\top$  with  $\mathbf{U}_X$  a  $N \times N$  orthogonal matrix distributed according to Haar measure on orthogonal matrices, and  $\boldsymbol{\Lambda} = \mathrm{diag}(\boldsymbol{\lambda})$  where  $\boldsymbol{\lambda}$  has i.i.d. Bernoulli elements. Thus,  $\rho_X = p\delta_0 + (1 - p)\delta_{+1}$  for  $p \in (0,1)$ , and the Stieltjes transform is:
$$
\mathcal {G} _ {\rho_ {X}} (z) = p \frac {1}{z} + (1 - p) \frac {1}{z - 1}
$$
For this prior, we have that  $\mathbf{X} = \mathbf{X}^2$ , so both estimators  $\widehat{\Xi_X^*}(S)$  and  $\widehat{\Xi_{X^2}^*}(S)$  should have the same performance. However, note that  $\widehat{\Xi_{X^2}^*}(S)$  does not use any knowledge of  $\rho_X$ . In Figure 10, the MSE is illustrated for these two estimators for two sparsity parameter,  $p = 0.5$  and 0.9. We observe that, except for the low-SNR regimes, both estimators have the same MSE. The poor performance of  $\widehat{\Xi_{X^2}^*}(S)$  in the low-SNR regimes might be due to the fact that, some of the estimated eigenvalues  $\widehat{\xi_{x^2i}^*}$  are negative although the true eigenvalue is 0. This makes the estimation more difficult for the sparser prior, see Figure 10b. However, this problem is resolved in  $\widehat{\Xi_X^*}(S)$  by taking the knowledge of  $\mathcal{G}_{\rho_X}(z)$  into account.
Effect of aspect-ratio  $\alpha$ . In Figure 11, we consider  $X$  to be shifted Wigner with  $c = 3$ , and the MSE is depicted for various values of the aspect-ratio  $\alpha$ . As expected, as  $M$  increases ( $\alpha$  decreases) and we have more observation or more data samples, the estimation error decreases.
(b) Estimating  $X^2$
Figure 10: Estimating  $\mathbf{X}$  and  $\mathbf{X}^2$  with Bernoulli spectral prior distribution. The MSE is normalized by the norm of the signal,  $\| \mathbf{X} \|_{\mathrm{F}}^2 = \| \mathbf{X}^2 \|_{\mathrm{F}}^2$ . Both  $\mathbf{Y}$  and  $\mathbf{W}$  are  $N \times M$  matrices with i.i.d. Gaussian entries of variance  $1/N$ , and aspect ratio  $N / M = 1 / 2$ . The RIE is applied to  $N = 2000$ ,  $M = 4000$ , and the results are averaged over 10 runs (error bars are invisible). Average relative error between RIE  $\widehat{\Xi}_{X}^{*}(S)$  and Oracle estimator is also reported.
Figure 11: MSE of estimating  $\mathbf{X}$  as a function of aspect-ratio  $\alpha$ , prior on  $\mathbf{X}$  is shifted Wigner with  $c = 3$ , and  $\kappa = 5$ . MSE is normalized by the norm of the signal,  $\| \mathbf{X} \|_{\mathrm{F}}^2$ . Both  $\mathbf{Y}$  and  $\mathbf{W}$  are  $N \times M$  matrices with i.i.d. Gaussian entries of variance  $1 / N$ . The RIE is applied to  $N = 2000$ ,  $M = 1 / \alpha N$ , and the results are averaged over 10 runs (error bars are invisible). Average relative error between RIE  $\widehat{\Xi}_{X}^{*}(\mathbf{S})$  and Oracle estimator is also reported.
D Estimating Y
In this section, we present the derivation of the optimal RIE for  $\mathbf{Y}$ . For simplicity, the SNR parameter in (1) is absorbed into  $\mathbf{Y}$ , so the model is  $S = \mathbf{X}\mathbf{Y} + \mathbf{W}$ . Therefore, the final estimator should be divided by  $1 / \sqrt{\kappa}$  to give an estimate of the original  $\mathbf{Y}$ .
The optimal singular values are constructed as  $\xi_{y_i}^* = \sum_{j=1}^{N} \sigma_j (\pmb{u}_i^\top \pmb{y}_j^{(l)}) (\pmb{v}_i^\top \pmb{y}_j^{(r)})$ . We assume that, for large  $N$ ,  $\xi_{y_i}^*$  can be approximated by its expectation:
$$
\widehat {\xi} _ {y _ {i}} ^ {*} \approx \sum_ {j = 1} ^ {N} \sigma_ {j} \mathbb {E} \left[ \left(\boldsymbol {u} _ {i} ^ {\intercal} \boldsymbol {y} _ {j} ^ {(l)}\right) \left(\boldsymbol {v} _ {i} ^ {\intercal} \boldsymbol {y} _ {j} ^ {(r)}\right) \right]
$$
where the expectation is over the singular vectors of the observation matrix  $S$ . Therefore, to compute the optimal singular values, we need to find the mean overlap  $\mathbb{E}\left[\left(\boldsymbol{u}_i^\top \boldsymbol{y}_j^{(l)}\right)\left(\boldsymbol{v}_i^\top \boldsymbol{y}_j^{(r)}\right)\right]$  between singular vectors of  $Y$  and singular vectors of  $S$ . In the following we will see that (a rescaling of) this quantity can be expressed in terms of  $i$ -th singular value of  $S$  and  $j$ -th singular value of  $Y$  (and the limiting measures, indeed). Thus, we will use the notation  $O_{Y}(\gamma_{i},\sigma_{j})\coloneqq N\mathbb{E}\left[\left(\boldsymbol{u}_{i}^{\top}\boldsymbol{y}_{j}^{(l)}\right)\left(\boldsymbol{v}_{i}^{\top}\boldsymbol{y}_{j}^{(r)}\right)\right]$  in what follows. In the nest section, we discuss how the overlap can be computed from the resolvent of the Hermitized matrix of  $S$ .
D.1 Relation between overlap and the resolvent
By Theorem 7.3.3 in [59],  $\mathcal{S}$  has the following eigen-decomposition:
$$
\boldsymbol {S} = \left[\begin{array}{c c c}\hat {\boldsymbol {U}} _ {S}&\hat {\boldsymbol {U}} _ {S}&\mathbf {0}\\\hat {\boldsymbol {V}} _ {S} ^ {(1)}&- \hat {\boldsymbol {V}} _ {S} ^ {(1)}&\boldsymbol {V} _ {S} ^ {(2)}\end{array}\right]\left[\begin{array}{c c c}\boldsymbol {\Gamma} _ {N}&\mathbf {0}&\mathbf {0}\\\mathbf {0}&- \boldsymbol {\Gamma} _ {N}&\mathbf {0}\\\mathbf {0}&\mathbf {0}&\mathbf {0}\end{array}\right]\left[ \right.\begin{array}{c c c}\hat {\boldsymbol {U}} _ {S}&\hat {\boldsymbol {U}} _ {S}&\mathbf {0}\\\hat {\boldsymbol {V}} _ {S} ^ {(1)}&- \hat {\boldsymbol {V}} _ {S} ^ {(1)}&\boldsymbol {V} _ {S} ^ {(2)}\end{array}] ^ {\intercal} \tag {56}
$$
with  $V_{S} = \left[ \begin{array}{cc}V_{S}^{(1)} & V_{S}^{(2)} \end{array} \right]$  in which  $V_{S}^{(1)}\in \mathbb{R}^{M\times N}$ . And,  $\hat{V}_S^{(1)} = \frac{1}{\sqrt{2}} V_S^{(1)}$ ,  $\hat{U}_{S} = \frac{1}{\sqrt{2}} U_{S}$ . Eigenvalues of  $\pmb{S}$  are signed singular values of  $\pmb{S}$ , therefore the limiting eigenvalue distribution of  $\pmb{S}$  (ignoring zero eigenvalues) is the same as the limiting symmetrized singular value distribution of  $\pmb{S}$ .
Define the resolvent of  $\pmb{S}$
$$
\boldsymbol {G} _ {\mathcal {S}} (z) = \left(z \boldsymbol {I} - \boldsymbol {\mathcal {S}}\right) ^ {- 1}
$$
Denote the eigenvectors of  $\mathcal{S}$  by  $\mathbf{s}_i\in \mathbb{R}^{M + N}$ ,  $i = 1,\ldots ,M + N$ . For  $z = x - \mathrm{i}\epsilon$  with  $x\in \mathbb{R}$  and  $\epsilon \gg \frac{1}{N}$ , we have:
$$
\pmb {G} _ {\mathcal {S}} (x - \mathrm {i} \epsilon) = \sum_ {k = 1} ^ {2 N} \frac {x + \mathrm {i} \epsilon}{(x - \tilde {\gamma} _ {k}) ^ {2} + \epsilon^ {2}} \mathbf {s} _ {k} \mathbf {s} _ {k} ^ {\intercal} + \frac {x + \mathrm {i} \epsilon}{x ^ {2} + \epsilon^ {2}} \sum_ {k = 2 N + 1} ^ {N + M} \mathbf {s} _ {k} \mathbf {s} _ {k} ^ {\intercal}
$$
where  $\tilde{\gamma}_k$  are the eigenvalues of  $S$ , which are in fact the (signed) singular values of  $S$ ,  $\tilde{\gamma}_1 = \gamma_1, \dots, \tilde{\gamma}_N = \gamma_N$ ,  $\tilde{\gamma}_{N+1} = -\gamma_1, \dots, \tilde{\gamma}_{2N} = -\gamma_N$ .
Define the vectors  $\boldsymbol{r}_i = \left[ \begin{array}{c} \boldsymbol{0}_N \\ \boldsymbol{y}_i^{(r)} \end{array} \right], \boldsymbol{l}_i = \left[ \begin{array}{c} \boldsymbol{y}_i^{(l)} \\ \boldsymbol{0}_M \end{array} \right]$  for  $\boldsymbol{y}_i^{(r)}, \boldsymbol{y}_i^{(l)}$  right/left singular vectors of  $\boldsymbol{Y}$ , we have
$$
\boldsymbol {r} _ {i} ^ {\intercal} \left(\operatorname {I m} \boldsymbol {G} _ {\mathcal {S}} (x - \mathrm {i} \epsilon)\right) \boldsymbol {l} _ {i} = \sum_ {k = 1} ^ {2 N} \frac {\epsilon}{(x - \tilde {\gamma} _ {k}) ^ {2} + \epsilon^ {2}} \left(\boldsymbol {r} _ {i} ^ {\intercal} \mathbf {s} _ {k}\right) \left(\boldsymbol {l} _ {i} ^ {\intercal} \mathbf {s} _ {k}\right) + \frac {x + \mathrm {i} \epsilon}{x ^ {2} + \epsilon^ {2}} \sum_ {k = 2 N + 1} ^ {N + M} \left(\boldsymbol {r} _ {i} ^ {\intercal} \mathbf {s} _ {k}\right) \left(\boldsymbol {l} _ {i} ^ {\intercal} \mathbf {s} _ {k}\right) \tag {57}
$$
Given the structure of  $\mathbf{s}_k$ 's in (56), we have:
$$
\left(\boldsymbol {r} _ {i} ^ {\intercal} \mathbf {s} _ {k}\right) \left(\boldsymbol {l} _ {i} ^ {\intercal} \mathbf {s} _ {k}\right) = \left\{ \begin{array}{l l} \frac {1}{2} \left(\boldsymbol {u} _ {k} ^ {\intercal} \boldsymbol {y} _ {i} ^ {(l)}\right) \left(\boldsymbol {v} _ {k} ^ {\intercal} \boldsymbol {y} _ {i} ^ {(r)}\right) & \text {f o r} 1 \leq k \leq N \\ - \frac {1}{2} \left(\boldsymbol {u} _ {k - N} ^ {\intercal} \boldsymbol {y} _ {i} ^ {(l)}\right) \left(\boldsymbol {v} _ {k - N} ^ {\intercal} \boldsymbol {y} _ {i} ^ {(r)}\right) & \text {f o r} N + 1 \leq k \leq 2 N \\ 0 & \text {f o r} 2 N + 1 \leq k \leq N + M \end{array} \right.
$$
In the limit of large  $N$ , the latter quantity is also self-averaging, due to the fact that as  $N \to \infty$ , these overlaps exhibit asymptotic independence, enabling the law of large numbers to be applied here. We can thus state that:
$$
\boldsymbol {r} _ {i} ^ {\intercal} \left(\operatorname {I m} \boldsymbol {G} _ {\mathcal {S}} (x - \mathrm {i} \epsilon)\right) \boldsymbol {l} _ {i} \xrightarrow {N \rightarrow \infty} \int_ {\mathbb {R}} \frac {\epsilon}{(x - t) ^ {2} + \epsilon^ {2}} O _ {Y} (t, \sigma_ {i}) \bar {\mu} _ {\mathcal {S}} (t) d t \tag {58}
$$
where the overlap function  $O_{Y}(t,\lambda_{i})$  is extended (continuously) to arbitrary values within the support of  $\bar{\mu}_{S}$  with the property that  $O_{Y}(-t,\lambda_{i}) = -O_{Y}(t,\lambda_{i})$  for  $t\in \mathrm{supp}(\mu_S)$ . Sending  $\epsilon \to 0$ , we find
$$
\boldsymbol {r} _ {i} ^ {\top} \left(\operatorname {I m} \boldsymbol {G} _ {S} (x - \mathrm {i} \epsilon)\right) \boldsymbol {l} _ {i} \approx \pi \bar {\mu} _ {S} (x) O _ {Y} (x, \sigma_ {i}) \tag {59}
$$
In the next section, we establish a connection between the resolvent  $G_{\mathcal{S}}(z)$  and the signal  $Y$ , which enables us to determine the overlap and consequently the optimal singular values values  $\widehat{\xi}_{y_i}^*$  in terms of the singular values of the observation matrix  $S$ .
D.2 Resolvent relation for  $Y$
In this section, we consider estimating  $\mathbf{Y}$ , and treat both  $\mathbf{X}$  and  $\mathbf{W}$  as noise. We consider the model to be:
$$
\boldsymbol {S} = \boldsymbol {O X O ^ {\intercal}} \boldsymbol {Y} + \boldsymbol {U W V ^ {\intercal}} \tag {60}
$$
where  $\mathbf{X} = \mathbf{X}^{\intercal} \in \mathbb{R}^{N \times N}$ ,  $\mathbf{W} \in \mathbb{R}^{N \times M}$  are fixed matrices with limiting eigenvalue/singular value distribution  $\rho_{X}, \mu_{W}$ , and  $\mathbf{O}, \mathbf{U} \in \mathbb{R}^{N \times N}$ ,  $\mathbf{V} \in \mathbb{R}^{M \times M}$  are independent random Haar matrices. For simplicity of notation, we use  $\mathbf{T} \equiv \mathbf{OXO}^{\intercal}\mathbf{Y}$ , and  $\mathcal{T} \in \mathbb{R}^{(N + M) \times (N + M)}$  the hermitization of  $\mathbf{T}$ . And  $\widetilde{\mathcal{W}}$  denotes the hermitization of the matrix  $U\mathbf{W}\mathbf{V}^{\intercal}$ .
As in the case for  $X$ , we express the entries of  $G(z) \equiv G_S(z)$  using Gaussian integral representation, and after applying the replica trick (28), we find:
$$
\begin{array}{l} \left\langle G _ {i j} (z) \right\rangle = \lim  _ {n \rightarrow \infty} \int \Big (\prod_ {k = 1} ^ {N + M} \prod_ {\tau = 1} ^ {n} d \eta_ {k} ^ {(\tau)} \Big) \eta_ {i} ^ {(1)} \eta_ {j} ^ {(1)} \Big \langle \exp \left\{- \frac {1}{2} \sum_ {\tau = 1} ^ {n} \boldsymbol {\eta} ^ {(\tau) \mathsf {T}} (z \boldsymbol {I} - \boldsymbol {\mathcal {S}}) \boldsymbol {\eta} ^ {(\tau)} \right\} \Big \rangle_ {\boldsymbol {O}, \boldsymbol {U}, \boldsymbol {V}} \\ = \lim  _ {n \rightarrow \infty} \int \left(\prod_ {k = 1} ^ {N + M} \prod_ {\tau = 1} ^ {n} d \eta_ {k} ^ {(\tau)}\right) \eta_ {i} ^ {(1)} \eta_ {j} ^ {(1)} \exp \left\{- \frac {z}{2} \sum_ {\tau = 1} ^ {n} \boldsymbol {\eta} ^ {(\tau) \top} \boldsymbol {\eta} ^ {(\tau)} \right\} \\ \times \left\langle \exp \left\{\frac {1}{2} \sum_ {\tau = 1} ^ {n} \boldsymbol {\eta} ^ {(\tau) \mathrm {T}} \boldsymbol {\mathcal {T}} \boldsymbol {\eta} ^ {(\tau)} \right\} \right\rangle_ {O} \left\langle \exp \left\{\frac {1}{2} \sum_ {\tau = 1} ^ {n} \boldsymbol {\eta} ^ {(\tau) \mathrm {T}} \widetilde {\boldsymbol {\mathscr {W}}} \boldsymbol {\eta} ^ {(\tau)} \right\} \right\rangle_ {U, V} \tag {61} \\ \end{array}
$$
Split each replica  $\pmb{\eta}^{(\tau)}$  into two vectors  $\pmb{a}^{(\tau)} \in \mathbb{R}^N$ ,  $\pmb{b}^{(\tau)} \in \mathbb{R}^M$ ,  $\pmb{\eta}^{(\tau)} = \left[ \begin{array}{c}\pmb{a}^{(\tau)}\\ \pmb{b}^{(\tau)} \end{array} \right]$ . The exponent in the first bracket in (61) can be written as:
$$
\begin{array}{l} \eta^ {(\tau) ^ {\intercal}} \mathcal {T} \eta^ {(\tau)} = a ^ {(\tau) ^ {\intercal}} O X O ^ {\intercal} Y b ^ {(\tau)} + b ^ {(\tau) ^ {\intercal}} Y ^ {\intercal} O X O ^ {\intercal} a ^ {(\tau)} \\ = \operatorname {T r} O X O ^ {\intercal} \left(\underbrace {\boldsymbol {Y} \boldsymbol {b} ^ {(\tau)} \boldsymbol {a} ^ {(\tau) ^ {\intercal}} + \boldsymbol {a} ^ {(\tau)} \boldsymbol {b} ^ {(\tau) ^ {\intercal}} \boldsymbol {Y} ^ {\intercal}} _ {\tilde {\boldsymbol {Y}} ^ {(\tau)}}\right) \tag {62} \\ \end{array}
$$
where  $\tilde{\pmb{Y}}^{(\tau)}$  is a symmetric  $N\times N$  matrix with two non-zero eigenvalues  $\pmb{a}^{(\tau)\mathsf{T}}\pmb{Y}\pmb{b}^{(\tau)}\pm$ $\| \pmb{a}^{(\tau)}\| \| \pmb{Y}\pmb{b}^{(\tau)}\|$  by lemma 3.
Using the formula for the spherical integral [21] (see Theorem 1 in H.1), we find:
$$
\begin{array}{l} \left\langle \exp \left\{\frac {1}{2} \sum_ {\tau = 1} ^ {n} \operatorname {T r} O X O ^ {\intercal} \tilde {Y} ^ {(\tau)} \right\} \right\rangle_ {O} \approx \exp \left\{\frac {N}{2} \sum_ {\tau = 1} ^ {n} \mathcal {P} _ {\rho_ {X}} \left(\frac {1}{N} \left(a ^ {(\tau) \intercal} Y b ^ {(\tau)} + \| a ^ {(\tau)} \| \| Y b ^ {(\tau)} \|\right)\right) \right. \\ \left. + \mathcal {P} _ {\rho_ {X}} \left(\frac {1}{N} \left(\boldsymbol {a} ^ {(\tau) \mathrm {T}} \boldsymbol {Y} \boldsymbol {b} ^ {(\tau)} - \| \boldsymbol {a} ^ {(\tau)} \| \| \boldsymbol {Y} \boldsymbol {b} ^ {(\tau)} \|\right)\right) \right\} \tag {63} \\ \end{array}
$$
By the same computation as previous section, for the second bracket we have:
$$
\left\langle \exp \left\{\sum_ {\tau = 1} ^ {n} \operatorname {T r} \boldsymbol {b} ^ {(\tau)} \boldsymbol {a} ^ {(\tau) ^ {\intercal}} \boldsymbol {U} \boldsymbol {W} \boldsymbol {V} ^ {\intercal} \right\} \right\rangle_ {\boldsymbol {U}, \boldsymbol {V}} \approx \exp \left\{\frac {N}{2} \sum_ {\tau = 1} ^ {n} \mathcal {Q} _ {\mu_ {W}} ^ {(\alpha)} \left(\frac {1}{N M} \| \boldsymbol {a} ^ {(\tau)} \| ^ {2} \| \boldsymbol {b} ^ {(\tau)} \| ^ {2}\right) \right\} \tag {64}
$$
From (61), (63), (64), we find: