SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We now apply Lemma 8 with  $\mathbf{M}_0 = \widehat{\boldsymbol{\Lambda}}^{(m)}$ . We set  $\gamma$  and  $\varphi(x)$  as before and verify Assumptions B1-B4 in the same way. By analogous calculations to the above, we obtain the bound
$$
\left\| \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2, \infty} \stackrel {{\mathbb {P}}} {{\lesssim}} \frac {\mu \sqrt {n d} \lambda_ {\max } \log n}{\delta}.
$$
The final bound is shown in the same way, requiring the additional observation that  $\kappa \{\gamma \vee \varphi (\gamma)\} \lesssim 1$  which follows from Assumption 3, and we obtain
$$
\begin{array}{l} \pi_ {1} \| \widehat {\mathbf {U}} _ {0} \mathbf {O} - \mathbf {U} _ {0} \| _ {2, \infty} \vee \pi_ {2} \| \widehat {\mathbf {V}} _ {0} \mathbf {O} - \mathbf {V} _ {0} \| _ {2, \infty} \\ \stackrel {\mathbb {P}} {\lesssim} \left[ \kappa_ {0} \left\{\kappa_ {0} + \varphi (1) \right\} \left\{\gamma + \varphi (\gamma) \right\} + \varphi (1) \right] \left(\pi_ {1} \| \mathbf {U} _ {0} \| _ {2, \infty} \vee \pi_ {2} \| \mathbf {V} _ {0} \| _ {2, \infty}\right) + \gamma \left(\pi_ {1} \left\| \bar {\boldsymbol {\Lambda}} \right\| _ {2, \infty} \vee \pi_ {2} \left\| \bar {\boldsymbol {\Lambda}} ^ {\top} \right\| _ {2, \infty}\right) / \delta_ {0} \\ \lesssim \varphi (1) (\pi_ {1} \| \mathbf {U} _ {0} \| _ {2, \infty} \vee \pi_ {2} \| \mathbf {V} _ {0} \| _ {2, \infty}) + \gamma (\pi_ {1} \| \bar {\boldsymbol {\Lambda}} \| _ {2, \infty} \vee \pi_ {2} \| \bar {\boldsymbol {\Lambda}} ^ {\top} \| _ {2, \infty}) / \delta_ {0} \\ \lesssim \frac {\mu \sqrt {n d} \lambda_ {\mathrm {m a x}} \log n}{\delta} \\ \end{array}
$$
G.7 Proof of Proposition 8
We show (15) using a simple application of Wedin's inequality:
$$
\begin{array}{l} \left\| \widehat {\mathbf {U}} ^ {(m)} \left(\widehat {\mathbf {U}} ^ {(m)}\right) ^ {\top} - \bar {\mathbf {U}} \bar {\mathbf {U}} ^ {\top} \right\| _ {2} = \left\| \sin \Theta \left(\widehat {\mathbf {U}} ^ {(m)}, \bar {\mathbf {U}}\right) \right\| _ {2} \lesssim \frac {\left\| \widehat {\boldsymbol {\Lambda}} ^ {(m)} - \bar {\boldsymbol {\Lambda}} \right\| _ {2}}{\sigma_ {d} (\bar {\boldsymbol {\Lambda}}) - \sigma_ {d + 1} (\bar {\boldsymbol {\Lambda}})} \leq \frac {\left\| \widehat {\boldsymbol {\Lambda}} - \bar {\boldsymbol {\Lambda}} \right\| _ {2}}{\sigma_ {d} (\bar {\boldsymbol {\Lambda}}) - \sigma_ {d + 1} (\bar {\boldsymbol {\Lambda}})} \\ \stackrel {\mathbb {P}} {\lesssim} \frac {\sqrt {M n \lambda_ {\operatorname* {m a x}}}}{\sqrt {M} \delta} = \frac {\sqrt {n \lambda_ {\operatorname* {m a x}}}}{\delta}. \\ \end{array}
$$
The proof of (16) requires a more delicate argument. We apply Wedin's theorem to obtain
$$
\left\| \widehat {\mathbf {U}} ^ {(m)} \left(\widehat {\mathbf {U}} ^ {(m)}\right) ^ {\top} - \widehat {\mathbf {U}} \widehat {\mathbf {U}} ^ {\top} \right\| _ {2} = \left\| \sin \Theta \left(\widehat {\mathbf {U}} ^ {(m)}, \widehat {\mathbf {U}}\right) \right\| _ {2} \lesssim \frac {\left\| \left(\widehat {\boldsymbol {\Lambda}} ^ {(m)} - \widehat {\boldsymbol {\Lambda}}\right) ^ {\top} \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2} \vee \left\| \left(\widehat {\boldsymbol {\Lambda}} ^ {(m)} - \widehat {\boldsymbol {\Lambda}}\right) \widehat {\mathbf {V}} ^ {(m)} \right\| _ {2}}{\sigma_ {d} (\widehat {\boldsymbol {\Lambda}}) - \sigma_ {d + 1} (\widehat {\boldsymbol {\Lambda}})} \tag {23}
$$
By Weyl's inequality
$$
\sigma_ {d} \left(\widehat {\boldsymbol {\Lambda}}\right) \geq \sigma_ {d} \left(\bar {\boldsymbol {\Lambda}}\right) + \left\| \widehat {\boldsymbol {\Lambda}} - \bar {\boldsymbol {\Lambda}} \right\| _ {2} \stackrel {{\mathbb {P}}} {\gtrsim} \sigma_ {d} \left(\bar {\boldsymbol {\Lambda}}\right).
$$
$$
\sigma_ {d + 1} \left(\widehat {\boldsymbol {\Lambda}}\right) \leq \sigma_ {d + 1} \left(\bar {\boldsymbol {\Lambda}}\right) - \left\| \widehat {\boldsymbol {\Lambda}} - \bar {\boldsymbol {\Lambda}} \right\| _ {2} \stackrel {{\mathbb {P}}} {\lesssim} \sigma_ {d + 1} \left(\bar {\boldsymbol {\Lambda}}\right).
$$
$$
\sigma_ {d} (\widehat {\boldsymbol {\Lambda}}) - \sigma_ {d + 1} (\widehat {\boldsymbol {\Lambda}}) \stackrel {\mathbb {P}} {\gtrsim} \sigma_ {d} (\bar {\boldsymbol {\Lambda}}) - \sigma_ {d + 1} (\bar {\boldsymbol {\Lambda}}) = \sqrt {M} \left(\sigma_ {d} ^ {1 / 2} (\boldsymbol {\Sigma}) - \sigma_ {d + 1} ^ {1 / 2} (\boldsymbol {\Sigma})\right) = \sqrt {M} \delta . \tag {24}
$$
We now focus our attention on obtaining a bound for  $\| (\widehat{\pmb{\Lambda}}^{(m)} - \widehat{\pmb{\Lambda}})\widehat{\mathbf{U}}^{(m)}\|_{\mathrm{F}}$ . Let
$$
\mathcal {N} _ {m} = \left\{m + (\ell - 1) n, \ell \in [ M ] \right\}.
$$
The  $ij$ th entry of  $\widehat{\pmb{\Lambda}} -\widehat{\pmb{\Lambda}}^{(m)}$  is
$$
\left(\widehat {\boldsymbol {\Lambda}} ^ {(m)} - \widehat {\boldsymbol {\Lambda}}\right) _ {i j} = \left(\widehat {\boldsymbol {\Lambda}} ^ {(m)} - \bar {\boldsymbol {\Lambda}}\right) _ {i j} \mathbb {I} (i = m, j \in \mathcal {N} _ {m}),
$$
and so  $\widehat{\mathbf{\Lambda}}^{(m)} - \widehat{\mathbf{\Lambda}}$  is independent of  $\widehat{\mathbf{\Lambda}}^{(m)}$  and hence  $\widehat{\mathbf{\Lambda}}^{(m)} - \widehat{\mathbf{\Lambda}}$  is independent of  $\widehat{\mathbf{U}}^{(m)}$ . We can then write
$$
\begin{array}{l} \left\| \left(\widehat {\boldsymbol {\Lambda}} ^ {(m)} - \widehat {\boldsymbol {\Lambda}}\right) ^ {\top} \widehat {\mathbf {U}} ^ {(m)} \right\| _ {\mathrm {F}} ^ {2} = \sum_ {\ell \notin \mathcal {N} _ {m}} \left(\widehat {\boldsymbol {\Lambda}} _ {m, \ell} - \bar {\boldsymbol {\Lambda}} _ {m, \ell}\right) ^ {2} \left\| \widehat {\mathbf {U}} _ {\ell , \cdot} ^ {(m)} \right\| _ {2} ^ {2} \\ + \left\| \sum_ {\ell \in \mathcal {N} _ {m}} \sum_ {i = 1} ^ {n} \left(\widehat {\mathbf {A}} _ {i \ell} - \bar {\mathbf {A}} _ {i \ell}\right) \widehat {\mathbf {U}} _ {\ell , \cdot} ^ {(m)} \right\| _ {2} ^ {2} \\ =: \zeta_ {1} + \zeta_ {2}. \\ \end{array}
$$
The (square root of the) first term is easily bounded as
$$
\begin{array}{l} \zeta_ {1} ^ {1 / 2} \leq \left\| \widehat {\mathbf {A}} - \bar {\mathbf {A}} \right\| _ {2, \infty} \left\| \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2, \infty} \\ \leq \left\| \widehat {\mathbf {A}} - \bar {\mathbf {A}} \right\| _ {2} \left\| \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2, \infty} \\ \stackrel {\mathbb {P}} {\lesssim} \sqrt {M n \lambda_ {\max }} \cdot \frac {\mu \sqrt {n d} \lambda_ {\max } \log n}{\delta} \\ = \frac {\sqrt {M d} n \lambda_ {\max } ^ {3 / 2} \mu \log n}{\delta} \\ \end{array}
$$
and to bound the second term, we employ Lemma 4 to obtain
$$
\begin{array}{l} \zeta_ {2} ^ {1 / 2} \stackrel {\mathbb {P}} {\lesssim} M \log^ {2} n \left\| \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2, \infty} + \sqrt {M \lambda_ {\max } \log n} \left\| \widehat {\mathbf {U}} ^ {(m)} \right\| _ {\mathrm {F}} \\ \leq M \log^ {2} n \left\| \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2, \infty} + \sqrt {M \lambda_ {\max } n \log n} \left\| \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2, \infty} \\ \lesssim \sqrt {M \lambda_ {\max } n \log n} \| \widehat {\mathbf {U}} ^ {(m)} \| _ {2, \infty} \\ \lesssim \sqrt {M \lambda_ {\max} n \log n} \cdot \frac {\mu \sqrt {n d} \lambda_ {\max} \log n}{\delta} \\ = \frac {\sqrt {M} d n \lambda_ {\max } ^ {3 / 2} \mu \log^ {3 / 2} n}{\delta} \\ \end{array}
$$
where we used Assumption 4 in the third inequality. Therefore
$$
\left\| \left(\widehat {\boldsymbol {\Lambda}} ^ {(m)} - \widehat {\boldsymbol {\Lambda}}\right) ^ {\top} \widehat {\mathbf {U}} ^ {(m)} \right\| _ {2} \leq \left\| \left(\widehat {\boldsymbol {\Lambda}} ^ {(m)} - \widehat {\boldsymbol {\Lambda}}\right) ^ {\top} \widehat {\mathbf {U}} ^ {(m)} \right\| _ {\mathrm {F}} \leq \zeta_ {1} ^ {1 / 2} + \zeta_ {2} ^ {1 / 2} \stackrel {\mathbb {P}} {\lesssim} \frac {\sqrt {M d} n \lambda_ {\max } ^ {3 / 2} \mu \log^ {3 / 2} n}{\delta}.
$$
Similar analysis yields an analogous bound for  $\| (\widehat{\pmb{\Lambda}}^{(m)} - \widehat{\pmb{\Lambda}})\widehat{\mathbf{V}}^{(m)}\| _2$ , and combining this, with (23) and (24) we have
$$
\begin{array}{l} \left\| \hat {\mathbf {U}} ^ {(m)} \left(\hat {\mathbf {U}} ^ {(m)}\right) ^ {\top} - \hat {\mathbf {U}} \hat {\mathbf {U}} ^ {\top} \right\| _ {2} \lesssim \frac {\left\| \left(\hat {\boldsymbol {\Lambda}} ^ {(m)} - \hat {\boldsymbol {\Lambda}}\right) ^ {\top} \hat {\mathbf {U}} ^ {(m)} \right\| _ {2} \vee \left\| \left(\hat {\boldsymbol {\Lambda}} ^ {(m)} - \hat {\boldsymbol {\Lambda}}\right) \hat {\mathbf {V}} ^ {(m)} \right\| _ {2}}{\sigma_ {d} (\hat {\boldsymbol {\Lambda}}) - \sigma_ {d + 1} (\hat {\boldsymbol {\Lambda}})} \\ \stackrel {\mathbb {P}} {\lesssim} \frac {\sqrt {d} n \lambda_ {\max } ^ {3 / 2} \mu \log^ {3 / 2} n}{\delta^ {2}} \\ \end{array}
$$