SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\lim  _ {N \rightarrow \infty} \frac {1}{N} \ln \mathcal {J} _ {N} (\boldsymbol {A}, \boldsymbol {B}) = \int_ {0} ^ {\theta} \frac {\mathcal {C} _ {\mu_ {B}} ^ {(\alpha)} (t ^ {2})}{t} d t = \frac {1}{2} \int_ {0} ^ {\theta^ {2}} \frac {\mathcal {C} _ {\mu_ {B}} ^ {(\alpha)} (t)}{t} d t \equiv \frac {1}{2} \mathcal {Q} _ {\mu_ {B}} ^ {(\alpha)} (\theta^ {2}) \tag {110}
$$
In our derivation, we use a generalization of this formula, namely when  $A$  has higher (but fixed) rank, the limit is the sum over singular values of the expression on the rhs of (110). Although we are not aware if this generalization has been proved, we believe that the ideas found in [63] can be applied to show it holds.
Remark 6. It is known that additional terms may be present on the rhs of (109) and (110) when the parameter  $\theta$  is "large". This has been rigorously proved at least in the case of symmetric  $\mathbf{A}$  and  $\mathbf{B}$  (see theorem 6 in [21]). In the replica calculation the order of magnitude of this parameter is determined by the solutions of the saddle point equations, but it is difficult to fully control its order of magnitude. However the numerics show very good agreement between our explicit RIEs and the Oracle estimator, which strongly suggests it is sound to use (109) and (110).
H.2 Matrix analysis tools
Proposition 3 (Inverse of a block matrix, Bernstein [61]). For a block matrix  $\mathbf{F} = \left[ \begin{array}{cc} \mathbf{A} & \mathbf{B} \\ \mathbf{C} & \mathbf{D} \end{array} \right]$  with  $\mathbf{A} \in \mathbb{R}^{N \times N}$ ,  $\mathbf{B} \in \mathbb{R}^{N \times M}$ ,  $\mathbf{C} \in \mathbb{R}^{M \times N}$ ,  $\mathbf{D} \in \mathbb{R}^{M \times M}$ , if  $\mathbf{A}$  and  $\mathbf{D} - \mathbf{C}\mathbf{A}^{-1}\mathbf{B}$ , are non-singular, then,
$$
\boldsymbol {F} ^ {- 1} = \left[ \begin{array}{c c} \boldsymbol {A} ^ {- 1} + \boldsymbol {A} ^ {- 1} \boldsymbol {B} (\boldsymbol {D} - \boldsymbol {C A} ^ {- 1} \boldsymbol {B}) ^ {- 1} \boldsymbol {C A} ^ {- 1} & - \boldsymbol {A} ^ {- 1} \boldsymbol {B} (\boldsymbol {D} - \boldsymbol {C A} ^ {- 1} \boldsymbol {B}) ^ {- 1} \\ - (\boldsymbol {D} - \boldsymbol {C A} ^ {- 1} \boldsymbol {B}) ^ {- 1} \boldsymbol {C A} ^ {- 1} & (\boldsymbol {D} - \boldsymbol {C A} ^ {- 1} \boldsymbol {B}) ^ {- 1} \end{array} \right]
$$
Block structure of  $G_{S}(z)$  The matrix  $G_{S}(z)$  is:
$$
\boldsymbol {G} _ {\mathcal {S}} (z) = \left(z \boldsymbol {I} - \boldsymbol {\mathcal {S}}\right) ^ {- 1} = \left[ \begin{array}{c c} z \boldsymbol {I} _ {N} & - \boldsymbol {S} \\ - \boldsymbol {S} ^ {\intercal} & z \boldsymbol {I} _ {M} \end{array} \right] ^ {- 1}
$$
Using Proposition 3, first we need to compute the inverse matrix  $\left(z\boldsymbol{I}_M - (-\boldsymbol{S}^\intercal)(z\boldsymbol{I}_N)^{-1}(-\boldsymbol{S})\right)^{-1}$  which simply reads:
$$
\left(z \boldsymbol {I} _ {M} - \frac {1}{z} \boldsymbol {S} ^ {\intercal} \boldsymbol {S}\right) ^ {- 1} = z \left(z ^ {2} \boldsymbol {I} _ {M} - \boldsymbol {S} ^ {\intercal} \boldsymbol {S}\right) ^ {- 1} = z \boldsymbol {G} _ {S \intercal S} (z ^ {2})
$$
Consequently, we find:
$$
\boldsymbol {G} _ {\mathcal {S}} (z) = \left[ \begin{array}{c c} \frac {1}{z} \boldsymbol {I} _ {N} + \frac {1}{z} \boldsymbol {S} \boldsymbol {G} _ {S \tau S} \left(z ^ {2}\right) \boldsymbol {S} ^ {\intercal} & \boldsymbol {S} \boldsymbol {G} _ {S \tau S} \left(z ^ {2}\right) \\ \boldsymbol {G} _ {S \tau S} \left(z ^ {2}\right) \boldsymbol {S} ^ {\intercal} & z \boldsymbol {G} _ {S \tau S} \left(z ^ {2}\right) \end{array} \right] \tag {111}
$$
Inverse of  $C_X^*$  For  $C_X^*$  since the blocks  $B, C$  are zero, the inverse is simply:
$$
\begin{array}{l} \boldsymbol {C} _ {X} ^ {*} ^ {- 1} = \left[ \begin{array}{c c} {\left[ (z - \zeta_ {1} ^ {*}) \boldsymbol {I} _ {N} - \zeta_ {3} ^ {*} \boldsymbol {X} ^ {2} \right] ^ {- 1}} & \boldsymbol {0} \\ \boldsymbol {0} & {\left[ (z - \zeta_ {2} ^ {*}) \boldsymbol {I} _ {M} \right] ^ {- 1}} \end{array} \right] \\ = \left[ \begin{array}{c c} \frac {1}{\zeta_ {3} ^ {*}} \left[ \frac {z - \zeta_ {1} ^ {*}}{\zeta_ {3} ^ {*}} \boldsymbol {I} _ {N} - \boldsymbol {X} ^ {2} \right] ^ {- 1} & \boldsymbol {0} \\ \boldsymbol {0} & \frac {1}{z - \zeta_ {2} ^ {*}} \boldsymbol {I} _ {M} \end{array} \right] \tag {112} \\ = \left[ \begin{array}{c c} \frac {1}{\zeta_ {3} ^ {*}} \boldsymbol {G} _ {X ^ {2}} \big (\frac {z - \zeta_ {1} ^ {*}}{\zeta_ {3} ^ {*}} \big) & \boldsymbol {0} \\ \boldsymbol {0} & \frac {1}{z - \zeta_ {2} ^ {*}} \boldsymbol {I} _ {M} \end{array} \right] \\ \end{array}
$$
Inverse of  $C_Y^*$  Let the block structure of  $C_Y^*$  be as in Proposition 3, then
$$
\begin{array}{l} (\pmb {D} - \pmb {C A} ^ {- 1} \pmb {B}) ^ {- 1} = \left((z - \beta_ {2} ^ {*}) \pmb {I} _ {M} - \beta_ {3} ^ {*} \pmb {Y} ^ {\intercal} \pmb {Y} - \frac {\beta_ {4} ^ {* 2}}{z - \beta_ {1} ^ {*}} \pmb {Y} ^ {\intercal} \pmb {Y}\right) ^ {- 1} \\ = \left(\left(z - \beta_ {2} ^ {*}\right) \boldsymbol {I} _ {M} - \left(\beta_ {3} ^ {*} + \frac {\beta_ {4} ^ {* 2}}{z - \beta_ {1} ^ {*}}\right) \boldsymbol {Y} ^ {\intercal} \boldsymbol {Y}\right) ^ {- 1} \\ = \left(z - \beta_ {1} ^ {*}\right) \left(Z _ {1} (z) \boldsymbol {I} _ {M} - Z _ {2} (z) \boldsymbol {Y} ^ {\intercal} \boldsymbol {Y}\right) ^ {- 1} \\ = \frac {z - \beta_ {1} ^ {*}}{Z _ {2} (z)} \left(\frac {Z _ {1} (z)}{Z _ {2} (z)} \boldsymbol {I} _ {M} - \boldsymbol {Y} ^ {\intercal} \boldsymbol {Y}\right) ^ {- 1} \\ = \frac {z - \beta_ {1} ^ {*}}{Z _ {2} (z)} G _ {Y \tau Y} \left(\frac {Z _ {1} (z)}{Z _ {2} (z)}\right) \\ \end{array}
$$
where  $G_{Y\tau Y}$  is the resolvent of the matrix  $\mathbf{Y}^{\top}\mathbf{Y}$ . So, we have
$$
\begin{array}{r} \pmb {C} _ {Y} ^ {*} ^ {- 1} = \left[ \begin{array}{c c} (z - \beta_ {1} ^ {*}) ^ {- 1} \pmb {I} _ {N} + \frac {\beta_ {4} ^ {* 2}}{(z - \beta_ {1} ^ {*}) Z _ {2} (z)} \pmb {Y} \pmb {G} _ {Y \tau Y} \big (\frac {Z _ {1} (z)}{Z _ {2} (z)} \big) \pmb {Y} ^ {\intercal} & \frac {\beta_ {4} ^ {*}}{Z _ {2} (z)} \pmb {Y} \pmb {G} _ {Y \tau Y} \big (\frac {Z _ {1} (z)}{Z _ {2} (z)} \big) \\ \frac {\beta_ {4} ^ {*}}{Z _ {2} (z)} \pmb {G} _ {Y \tau Y} \big (\frac {Z _ {1} (z)}{Z _ {2} (z)} \big) \pmb {Y} ^ {\intercal} & \frac {z - \beta_ {1} ^ {*}}{Z _ {2} (z)} \pmb {G} _ {Y \tau Y} \big (\frac {Z _ {1} (z)}{Z _ {2} (z)} \big) \end{array} \right] \end{array}
$$
Lemma 3. Consider two vectors  $\mathbf{x}, \mathbf{y} \in \mathbb{R}^N$ . The symmetric matrix  $\mathbf{x}\mathbf{y}^{\top} + \mathbf{y}\mathbf{x}^{\top}$  has rank at most two with non-zero eigenvalues  $\mathbf{x}^{\top}\mathbf{y} \pm \| \mathbf{x} \| \| \mathbf{y} \|$ .
Proof. Construct the matrices  $\mathbf{A} \in \mathbb{R}^{2 \times N}$ ,  $\mathbf{B} \in \mathbb{R}^{N \times 2}$  as follows:
$$
\boldsymbol {A} = \left[ \begin{array}{l} \boldsymbol {x} ^ {\intercal} \\ \boldsymbol {y} ^ {\intercal} \end{array} \right], \quad \boldsymbol {B} = \left[ \begin{array}{l l} \boldsymbol {y} & \boldsymbol {x} \end{array} \right]
$$
Then, we have that  $\boldsymbol{x}\boldsymbol{y}^{\intercal} + \boldsymbol{y}\boldsymbol{x}^{\intercal} = \boldsymbol{B}\boldsymbol{A}$ . Using the lemma 4, we have that:
$$
z ^ {2} \det  \left(z \boldsymbol {I} _ {N} - \boldsymbol {B} \boldsymbol {A}\right) = z ^ {N} \det  \left(z \boldsymbol {I} _ {2} - \boldsymbol {A} \boldsymbol {B}\right)
$$
So, the characteristic polynomial of  $\boldsymbol{x}\boldsymbol{y}^{\mathsf{T}} + \boldsymbol{y}\boldsymbol{x}^{\mathsf{T}}$  is  $z^{N - 2}\operatorname*{det}\left(zI_2 - AB\right)$ , which implies that the  $\boldsymbol{x}\boldsymbol{y}^{\mathsf{T}} + \boldsymbol{y}\boldsymbol{x}^{\mathsf{T}}$  has eigenvalue 0 with multiplicity  $N - 2$ , plus the eigenvalues of the  $2\times 2$  matrix  $\boldsymbol{AB}$ . The matrix  $\boldsymbol{AB}$  is:
$$
\boldsymbol {A} \boldsymbol {B} = \left[ \begin{array}{c c} \boldsymbol {x} ^ {\intercal} \boldsymbol {y} & \| \boldsymbol {x} \| ^ {2} \\ \| \boldsymbol {y} \| ^ {2} & \boldsymbol {x} ^ {\intercal} \boldsymbol {y} \end{array} \right]
$$
which has two eigenvalues  $\pmb{x}^{\mathrm{T}}\pmb{y}\pm \| \pmb {x}\| \| \pmb {y}\|$
Lemma 4. For matrices  $\mathbf{A} \in \mathbb{R}^{M \times N}$ ,  $\mathbf{B} \in \mathbb{R}^{N \times M}$ , we have:
$$
z ^ {M} \det  \left(z \boldsymbol {I} _ {N} - \boldsymbol {B} \boldsymbol {A}\right) = z ^ {N} \det  \left(z \boldsymbol {I} _ {M} - \boldsymbol {A} \boldsymbol {B}\right)
$$
Proof. Construct the matrices  $\pmb{C}, \pmb{D} \in \mathbb{R}^{(M + N) \times (M + N)}$  as follows:
$$
\boldsymbol {C} = \left[ \begin{array}{c c} z \boldsymbol {I} _ {M} & \boldsymbol {A} \\ \boldsymbol {B} & \boldsymbol {I} _ {N} \end{array} \right], \quad \boldsymbol {D} = \left[ \begin{array}{c c} \boldsymbol {I} _ {M} & \boldsymbol {0} _ {M \times N} \\ - \boldsymbol {B} & z \boldsymbol {I} _ {N} \end{array} \right]
$$
$$
\det  \boldsymbol {C} \boldsymbol {D} = z ^ {N} \det  (z \boldsymbol {I} _ {M} - \boldsymbol {A} \boldsymbol {B}), \quad \det  \boldsymbol {D} \boldsymbol {C} = z ^ {M} \det  (z \boldsymbol {I} _ {N} - \boldsymbol {B} \boldsymbol {A})
$$
The result follows from the fact that  $\operatorname{det} CD = \operatorname{det} DC$ .