SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \left\| \frac {Z ^ {(i) \top} Z ^ {(i)} + E ^ {\top} Z ^ {(i)} + Z ^ {(i) \top} E}{\mu^ {2}} \right\| _ {2} \leq \frac {1}{\mu^ {2}} \left(\left\| Z ^ {(i) \top} Z ^ {(i)} \right\| _ {2} + \left\| E ^ {\top} Z ^ {(i)} + Z ^ {(i) \top} E \right\| _ {2}\right) \\ \leq C r \log d \cdot \left(4 \eta^ {2} w _ {i} ^ {* 2} L ^ {2} r \rho^ {4} + 8 \eta w _ {i} ^ {*} \frac {L \rho^ {2}}{\sqrt {C}}\right) \\ \leq \frac {1 6 \sqrt {C} L}{K} \leq 1 \text {w i t h s u f f i c i e n t l y l a r g e} K. \\ \end{array}
$$
Claim B.3.  $\left\| \frac{1}{n}\sum_{i = 1}^{n}Z^{(i)\top}Z^{(i)}\right\| _2\leq \frac{4L^2}{K\log d}\cdot \frac{\eta}{nr}.$
$$
\begin{array}{l} \sum_ {i = 1} ^ {n} \left\| Z ^ {(i) \top} Z ^ {(i)} \right\| _ {2} = \sum_ {i = 1} ^ {n} \eta^ {2} w _ {i} ^ {* 2} \left\| \left(u _ {i} A _ {i} - S ^ {*}\right) ^ {\top} \left(u _ {i} A _ {i} - S ^ {*}\right) \right\| _ {2} \\ \leq 2 \sum_ {i = 1} ^ {n} \eta^ {2} w _ {i} ^ {* 2} \left(u _ {i} ^ {2} \left\| A _ {i} ^ {\top} A _ {i} \right\| _ {2} + \left\| S ^ {* \top} S ^ {*} \right\| _ {2}\right) \\ \leq 2 \eta \sum_ {i = 1} ^ {n} \eta w _ {i} ^ {*} \left(w _ {i} ^ {*} u _ {i} ^ {2} \rho^ {2} + w _ {i} ^ {*} L ^ {2}\right) \\ \leq 2 \eta \frac {1}{K r \rho^ {2} \log d} (L \rho^ {2} + L ^ {2}) \\ \leq \frac {4 L ^ {2}}{K \log d} \frac {\eta}{r} \\ \Rightarrow \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} Z ^ {(i) \top} Z ^ {(i)} \right\| _ {2} \leq \frac {1}{n} \sum_ {i = 1} ^ {n} \left\| Z ^ {(i) \top} Z ^ {(i)} \right\| _ {2} \\ \leq \frac {4 L ^ {2}}{K \log d} \cdot \frac {\eta}{n r}. \\ \end{array}
$$
Claim B.4.  $\left\| \frac{1}{n}\sum_{i = 1}^{n}E^{\top}Z^{(i)} + Z^{(i)\top}E\right\| _2\leq \frac{4}{\sqrt{CK}}\cdot \frac{\eta}{nr}.$
$$
\begin{array}{l} \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} E ^ {\top} Z ^ {(i)} + Z ^ {(i) \top} E \right\| _ {2} = \frac {1}{n} \left\| E ^ {\top} \sum_ {i = 1} ^ {n} Z ^ {(i)} + \sum_ {i = 1} ^ {n} Z ^ {(i) \top} E \right\| _ {2} \\ \leq \frac {2}{n} \left\| E ^ {\top} \sum_ {i = 1} ^ {n} Z ^ {(i)} \right\| _ {2} \\ \leq \frac {2}{n} \| E \| _ {2} \cdot \left\| \sum_ {i = 1} ^ {n} Z ^ {(i)} \right\| _ {2} \\ = \frac {2}{n} \| E \| _ {2} \cdot \| \eta G ^ {*} - \eta S ^ {*} \| _ {2} \\ \leq \frac {2}{n} \| E \| _ {2} \cdot \eta \| E ^ {*} \| _ {2} \\ \leq \frac {2}{n} \cdot \frac {2}{\sqrt {C r}} \cdot \frac {\eta}{K \sqrt {r}} \\ = \frac {4}{\sqrt {C} K} \frac {\eta}{n r}. \\ \end{array}
$$
Claim B.5.  $\frac{2}{\mu^2}\left\| \frac{1}{n}\sum_{i = 1}^{n}(Z^{(i)\top}Z^{(i)})^2\right\| _2\leq O\left(\frac{CL^4}{K^3r\rho^2\log^2d}\right)\cdot \frac{\eta}{nr}.$
$$
\begin{array}{l} \left\| \sum_ {i = 1} ^ {n} \left(Z ^ {(i) \top} Z ^ {(i)}\right) ^ {2} \right\| _ {2} \leq 8 \sum_ {i = 1} ^ {n} \eta^ {4} w _ {i} ^ {* 4} \left(u _ {i} ^ {4} \cdot \left\| \left(A _ {i} ^ {\top} A _ {i}\right) ^ {2} \right\| _ {2} + \left\| \left(S ^ {* \top} S ^ {*}\right) ^ {2} \right\| _ {2}\right) (L e m m a B. 5) \\ \leq 8 \eta \sum_ {i = 1} ^ {n} \eta^ {3} w _ {i} ^ {* 2} \left(w _ {i} ^ {* 2} u _ {i} ^ {4} \rho^ {4} + w _ {i} ^ {* 2} L ^ {4}\right) \\ \leq 8 \eta \cdot \frac {1}{K ^ {3} r ^ {3} \rho^ {6} \log^ {3} d} \left[ \sum_ {i = 1} ^ {n} w _ {i} ^ {* 2} \rho^ {4} u _ {i} ^ {4} + \sum_ {i = 1} ^ {n} w _ {i} ^ {* 2} L ^ {4} \right] \\ \leq 8 \eta \cdot \frac {1}{K ^ {3} r ^ {3} \rho^ {6} \log^ {3} d} \left[ \left(\sum_ {i = 1} ^ {n} w _ {i} ^ {*} \rho^ {2} u _ {i} ^ {2}\right) ^ {2} + \left(\sum_ {i = 1} ^ {n} w _ {i} ^ {*} L ^ {2}\right) ^ {2} \right] \\ \leq 8 \eta \cdot \frac {1}{K ^ {3} r ^ {3} \rho^ {6} \log^ {3} d} (\rho^ {4} L ^ {2} + L ^ {4}) \\ \leq \frac {1 6 L ^ {4}}{K ^ {3} r ^ {2} \rho^ {2} \log^ {3} d} \cdot \frac {\eta}{r} \\ \leq O \left(\frac {L ^ {4}}{K ^ {3} r ^ {2} \rho^ {2} \log^ {3} d}\right) \cdot \frac {\eta}{r}. \\ \Rightarrow \frac {2}{\mu^ {2}} \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} \left(Z ^ {(i) \top} Z ^ {(i)}\right) ^ {2} \right\| _ {2} \leq O \left(\frac {C L ^ {4}}{K ^ {3} r \rho^ {2} \log^ {2} d}\right) \cdot \frac {\eta}{n r} \\ \end{array}
$$
Claim B.6.  $\frac{2}{\mu^2}\left\| \frac{1}{n}\sum_{i = 1}^{n}(E^\top Z^{(i)} + Z^{(i)\top}E)^2\right\| _2\leq \frac{96L^2}{K}\cdot \frac{\eta}{nr}.$
$$
\begin{array}{l} \frac {2}{\mu^ {2}} \left\| \frac {1}{n} \sum_ {i = 1} ^ {n} \left(E ^ {\top} Z ^ {(i)} + Z ^ {(i) \top} E\right) ^ {2} \right\| _ {2} \leq \frac {2}{\mu^ {2}} \frac {1}{n} \sum_ {i = 1} ^ {n} \left\| \left(E ^ {\top} Z ^ {(i)} + Z ^ {(i) \top} E\right) ^ {2} \right\| _ {2} \\ \leq \frac {2}{\mu^ {2}} \frac {1}{n} \sum_ {i = 1} ^ {n} 4 \left\| E ^ {\top} Z ^ {(i)} Z ^ {(i) ^ {\top}} E \right\| _ {2} \\ \leq \frac {2}{\mu^ {2}} \frac {1}{n} \sum_ {i = 1} ^ {n} 4 \| E \| _ {2} ^ {2} \left\| Z ^ {(i) \top} Z ^ {(i)} \right\| _ {2} \\ \leq \frac {8}{\mu^ {2}} \| E \| _ {2} ^ {2} \cdot \frac {1}{n} \sum_ {i = 1} ^ {n} \left\| Z ^ {(i) \top} Z ^ {(i)} \right\| _ {2} \\ \leq \frac {8}{\mu^ {2}} \cdot \frac {3}{C r} \cdot \frac {4 L ^ {2}}{K \log d} \frac {\eta}{n r} (\text {C l a i m B . 3}) \\ = \frac {9 6 L ^ {2}}{K} \cdot \frac {\eta}{n r} \\ \end{array}
$$
Lemma B.6 (Golden-Thompson Inequality [Tho65]). For two  $n \times n$  Hermitian matrices  $A$  and  $B$ :
$$
\operatorname {t r} \left(\exp (A + B)\right) \leq \operatorname {t r} \left(\exp (A) \exp (B)\right).
$$
Proof. Algorithm 3 has a nested for loop that's repeated for  $N' \times N = O(n\log d\log \frac{1}{\delta} r\rho^2)$  times. The major step in the loop is line 11:  $s_t \gets \operatorname{argmax}_{s \in [0,\eta]} \Phi(w_t + se_i)$ , which is equivalent to  $\operatorname{argmin}_{s \in [0,\eta]} Cr\Phi_{\mathrm{op}}(w + se_i) + \frac{s}{4L} - su_i^2$ . Recall that  $\Phi_{\mathrm{op}}(w) = \min_{\|S\|_F \leq L\|w\|_1} \left( \mu^2\log[F(G_w - S)] \right)$  where  $F(E) = \operatorname{tr}\exp\left(\frac{E^\top E}{\mu^2}\right)$ . Note that  $\mu^2\log[F(E)]$  is convex in  $E$ .
First we show that  $\Phi_{\mathrm{op}}(w)$  is convex in  $w$ , i.e., given  $w_1, w_2, \Phi_{\mathrm{op}}\left(\frac{1}{2}(w_1 + w_2)\right) \leq \frac{1}{2}\left(\Phi_{\mathrm{op}}(w_1) + \Phi_{\mathrm{op}}(w_2)\right)$ .
Let  $w_{3} = \frac{1}{2}(w_{1} + w_{2})$ , and  $G_{k} = \sum_{i=1}^{n}(w_{k})_{i}u_{i}A_{i}$  for  $k = 1,2,3$ . Suppose  $S_{1},S_{2}$  attain the minimum for  $\Phi_{\mathrm{op}}(w_1),\Phi_{\mathrm{op}}(w_2)$  respectively, i.e.,  $\Phi_{\mathrm{op}}(w_1) = \mu^2\log [F(G_1 - S_1)]$  and  $\Phi_{\mathrm{op}}(w_2) = \mu^2\log [F(G_2 - S_2)]$ .
Let  $S_{3} = \frac{1}{2} (S_{1} + S_{2})$ . Notice that  $G_{3} = \frac{1}{2} (G_{1} + G_{2})$ , so  $G_{3} - S_{3} = \frac{1}{2} (G_{1} - S_{1} + G_{2} - S_{2})$ . Since  $\| S_3\| _F\leq \frac{1}{2} (\| S_1\| _F + \| S_2\| _F)\leq \frac{1}{2} L(\| w_1\| _1 + \| w_2\| _1) = L\| w_3\| _1$ ,  $S_{3}$  is a valid argument for  $\Phi_{\mathrm{op}}(w_3)$ , therefore:
$$
\begin{array}{l} \Phi_ {\mathrm {o p}} (w _ {3}) \leq \mu^ {2} \log [ F (G _ {3} - S _ {3}) ] \leq \frac {1}{2} \left(\mu^ {2} \log [ F (G _ {1} - S _ {1}) ] + \mu^ {2} \log [ F (G _ {2} - S _ {2}) ]\right) \\ = \frac {1}{2} \left(\Phi_ {\mathrm {o p}} \left(w _ {1}\right) + \Phi_ {\mathrm {o p}} \left(w _ {2}\right)\right). \\ \end{array}
$$
Line 11 is equivalent to minimizing  $Cr\Phi_{\mathrm{op}}(w + se_i) + \frac{s}{4L} - su_i^2$ , which is convex in  $s$  for a fixed  $w$ , over a bounded interval  $[0, \eta]$ , so the minimization needs to evaluate  $\Phi_{\mathrm{op}}(w + se_i)$  for  $O(\mathrm{polylog}(d))$  different values of  $s$ . Evaluating  $\Phi_{\mathrm{op}}$  is also a minimization which can be solved by computing SVD on  $G_w$  and evaluating  $F(G_w - S)$  in time  $O(d^\omega)$  for polylog(d) various constructions of  $S$ . Overall finding the optimal value of  $s$  takes time  $O(d^\omega \cdot \mathrm{polylog}(d))$ , and the algorithm's total running time is  $O(nd^\omega \mathrm{polylog}(d)\log \frac{1}{\delta} r\rho^2)$ .
C Omitted proofs: From wRIP to dRIP condition
Here we show that the dRIP Condition 2.3 is implied by the wRIP Condition 2.2, given proper choices of parameters within a constant factor. Notice that in the wRIP condition, we have a low-rank constraint on the input matrix, i.e.,  $\mathrm{rank}(X) \leq r$ , and in dRIP we have a norm constraint instead, i.e.,  $\|V\|_F \in [\frac{1}{4}, 1]$ ,  $\|V\|_* \leq 2\sqrt{2r}$ . To make use of the wRIP condition of  $\mathcal{A}$ , we will decompose  $V$  into low-rank matrices, so that wRIP condition applies to each of the low-rank matrices. Though the rank of  $V$  is arbitrary, we can still upper bound its numerical rank based on the norm constraint.
First we will introduce a low-rank decomposition, and an upper bound on the sum of their Frobenius norms. This is the matrix version of the shelling-decomposition in Lemma 15 for vectors in  $\mathrm{[KLL^{+}23]}$
Lemma C.1 (Low-rank Decomposition). Given  $V \in \mathbb{R}^{d_1 \times d_2}$  with  $\mathrm{Rank}_{\mathrm{n}}(V) = \frac{\|V\|_{*}^{2}}{\|V\|_{F}^{2}} = \nu$ , and let  $V = \sum \sigma_i u_i v_i^\top$  be its SVD with  $\sigma_i$  in descending order. Decompose  $V$  into sum of rank- $r$  matrices, i.e., write  $V = \sum_{\ell=1}^{\ell=k} V^{(\ell)}$  where  $V^{(\ell)} = \sum_{i=(\ell-1)r+1}^{i=\ell r} \sigma_i u_i v_i^\top$ . Then we have:  $\sum_{\ell=2}^{k} \|V^{(\ell)}\|_F \leq \sqrt{\frac{\nu}{r}} \|V\|_F$ .
Proof. Note that  $V^{(1)}$  is the rank-  $r$  approximation of  $V$ , and  $V^{(\ell)}$ 's are constructed using disjoint singular values and vectors in groups of size  $r$ , and are orthogonal to each other.
Denote  $\sigma_{i}\left(V^{(\ell)}\right)$  as the  $i^{\mathrm{th}}$  largest singular value of  $V^{(\ell)}$ .
$$
\begin{array}{l} \left\| V ^ {(\ell + 1)} \right\| _ {F} \leq \sqrt {r} \cdot \sigma_ {1} \left(V ^ {(\ell + 1)}\right) \leq \sqrt {r} \cdot \sigma_ {r} \left(V ^ {(\ell)}\right) \leq \sqrt {r} \cdot \frac {\left\| V ^ {(\ell)} \right\| _ {*}}{r}, \\ \sum_ {\ell = 2} ^ {k} \left\| V ^ {(\ell)} \right\| _ {F} \leq \frac {\sqrt {r}}{r} \cdot \sum_ {\ell = 1} ^ {k - 1} \left\| V ^ {(\ell)} \right\| _ {*} \leq \frac {\sqrt {r}}{r} \cdot \sum_ {\ell = 1} ^ {k} \left\| V ^ {(\ell)} \right\| _ {*} = \frac {\sqrt {r}}{r} \cdot \| V \| _ {*}, \\ \frac {\| V \| _ {*} ^ {2}}{\| V \| _ {F} ^ {2}} \leq \nu \Rightarrow \sum_ {\ell = 2} ^ {k} \left\| V ^ {(\ell)} \right\| _ {F} \leq \sqrt {\frac {\nu}{r}} \| V \| _ {F}. \\ \end{array}
$$
Now we are ready to prove Lemma 5.1, which states that wRIP implies dRIP condition. The proof uses similar techniques as in the second part of Lemma 17 [KLL  $^{+}23]$  for vector recovery.
Proof. Boundedness property is satisfied by assumption  $\| A_i\| _2\leq \rho \forall i$
Isometry property:
Consider  $V \in \mathbb{R}^{d_1 \times d_2}$  s.t.  $\| V \|_F \in [\frac{1}{4}, 1]$ ,  $\| V \|_* \leq 2\sqrt{2r}$ . Need to show  $\frac{1}{L} \leq \sum_{i=1}^{n} w_i^* \langle A_i, V \rangle^2 \leq L$ .
Let  $L = 25L^{\prime}$ $K\geq 1$  and  $r = \frac{r'}{12800L^2K^2}$
$\nu = \mathrm{Rank}_{\mathrm{n}}(V) = \frac{\|V\|_{*}^{2}}{\|V\|_{F}^{2}} \leq 128r$ . By Lemma C.1, decompose  $V$  into rank- $r'$  matrices so that we can apply the  $(r', L')$ -wRIP property of  $\mathcal{A}$ .
$$
\sum_ {\ell = 2} ^ {k} \left\| V ^ {(\ell)} \right\| _ {F} \leq \sqrt {\frac {\nu}{r ^ {\prime}}} \| V \| _ {F} \leq \frac {1}{1 0 L K} \| V \| _ {F} \leq \frac {1}{1 0} \| V \| _ {F},
$$
$$
\| V \| _ {F} \geq \left\| V _ {(r ^ {\prime})} \right\| _ {F} = \left\| V ^ {(1)} \right\| _ {F} = \left\| V - \sum_ {\ell = 2} ^ {k} V ^ {(\ell)} \right\| _ {F} \geq \| V \| _ {F} - \sum_ {\ell = 2} ^ {k} \left\| V ^ {(\ell)} \right\| _ {F} \geq \frac {9}{1 0} \| V \| _ {F}.
$$
Let  $B_{i} = \sqrt{w_{i}^{*}} A_{i}$ , so that  $\sum_{i=1}^{n} w_{i}^{*}\langle A_{i}, V \rangle^{2} = \sum_{i=1}^{n}\langle B_{i}, V \rangle^{2} = \|\sum_{i=1}^{n}\langle B_{i}, V \rangle e_{i}\|_{2}^{2}$ .
Lower bound:
$$
\begin{array}{l} \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, V \rangle e _ {i} \right\| _ {2} \geq \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, V ^ {(1)} \rangle e _ {i} \right\| _ {2} - \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, \sum_ {\ell = 2} ^ {k} V ^ {(\ell)} \rangle e _ {i} \right\| _ {2} \\ \geq \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, V ^ {(1)} \rangle e _ {i} \right\| _ {2} - \sum_ {\ell = 2} ^ {k} \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, V ^ {(\ell)} \rangle e _ {i} \right\| _ {2} \\ = \sqrt {\sum_ {i = 1} ^ {n} \langle B _ {i} , V ^ {(1)} \rangle^ {2}} - \sum_ {\ell = 2} ^ {k} \sqrt {\sum_ {i = 1} ^ {n} \langle B _ {i} , V ^ {(\ell)} \rangle^ {2}} \\ \geq \sqrt {\frac {1}{L ^ {\prime}} \cdot \left\| V ^ {(1)} \right\| _ {F} ^ {2}} - \sum_ {\ell = 2} ^ {k} \sqrt {L ^ {\prime} \cdot \left\| V ^ {(\ell)} \right\| _ {F} ^ {2}} \\ \geq \frac {0 . 9}{\sqrt {L ^ {\prime}}} \cdot \| V \| _ {F} - \frac {0 . 1}{L} \sqrt {L ^ {\prime}} \cdot \| V \| _ {F} \\ = \frac {4 . 5}{\sqrt {L}} \cdot \| V \| _ {F} - \frac {0 . 0 2}{\sqrt {L}} \cdot \| V \| _ {F}. \\ \end{array}
$$
Taking the square:  $\sum_{i=1}^{n} w_i^* \langle A_i, V \rangle^2 \geq \frac{4.48^2}{L} \cdot \|V\|_F^2 \geq \frac{4.48^2}{16L} \geq \frac{1}{L}$ .
Upper bound:
$$
\begin{array}{l} \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, V \rangle e _ {i} \right\| _ {2} \leq \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, V ^ {(1)} \rangle e _ {i} \right\| _ {2} + \sum_ {\ell = 2} ^ {k} \left\| \sum_ {i = 1} ^ {n} \langle B _ {i}, V ^ {(\ell)} \rangle e _ {i} \right\| _ {2} \\ = \sqrt {\sum_ {i = 1} ^ {n} \langle B _ {i} , V ^ {(1)} \rangle^ {2}} + \sum_ {\ell = 2} ^ {k} \sqrt {\sum_ {i = 1} ^ {n} \langle B _ {i} , V ^ {(\ell)} \rangle^ {2}} \\ \leq \sqrt {L ^ {\prime}} \cdot \left\| V ^ {(1)} \right\| _ {F} + \sum_ {\ell = 2} ^ {k} \sqrt {L ^ {\prime}} \cdot \left\| V ^ {(\ell)} \right\| _ {F} \\ \leq \sqrt {L ^ {\prime}} \cdot \| V \| _ {F} ^ {2} + \frac {0 . 1 \sqrt {L ^ {\prime}}}{L} \cdot \| V \| _ {F} \\ = \frac {\sqrt {L}}{5} \cdot \| V \| _ {F} + \frac {0 . 0 2}{\sqrt {L}} \cdot \| V \| _ {F}. \\ \end{array}
$$
Taking the square:  $\sum_{i=1}^{n} w_i^* \langle A_i, V \rangle^2 \leq \frac{L}{25} \cdot \|V\|_F^2 + \frac{0.02^2}{L} \cdot \|V\|_F^2 + 0.008 \|V\|_F^2 \leq L \cdot \|V\|_F^2 \leq L$ .
Combining the lower bound and upper bound:  $\frac{1}{L} \leq \sum_{i=1}^{n} w_i^* \langle A_i, V \rangle^2 \leq L$ .
Decomposition property:
Let  $S = G_{(r')}$ , the rank- $r'$  approximation of  $G = \sum_{i=1}^{n} w_i^* \langle A_i, V \rangle A_i$ . Let  $E = G - S$ . Suffices to show  $\| S \|_F \leq L$  and  $\| E \|_2 \leq \frac{1}{K\sqrt{r}}$ . We have
$$
\begin{array}{l} \| S \| _ {F} ^ {2} = \langle S, S \rangle = \langle G, S \rangle = \left\langle \sum_ {i = 1} ^ {n} w _ {i} ^ {*} \langle A _ {i}, V \rangle A _ {i}, S \right\rangle = \left\langle \sum_ {i = 1} ^ {n} \langle B _ {i}, V \rangle B _ {i}, S \right\rangle = \sum_ {i = 1} ^ {n} \langle B _ {i}, V \rangle \langle B _ {i}, S \rangle \\ = \left\langle \sum_ {i = 1} ^ {n} \left\langle B _ {i}, V \right\rangle e _ {i}, \sum_ {j = 1} ^ {n} \left\langle B _ {j}, S \right\rangle e _ {j} \right\rangle \leq \left\| \sum_ {i = 1} ^ {n} \left\langle B _ {i}, V \right\rangle e _ {i} \right\| _ {2} \cdot \left\| \sum_ {i = 1} ^ {n} \left\langle B _ {i}, S \right\rangle e _ {i} \right\| _ {2} \\ = \sqrt {\sum_ {i = 1} ^ {n} \langle B _ {i} , V \rangle^ {2}} \cdot \sqrt {\sum_ {i = 1} ^ {n} \langle B _ {i} , S \rangle^ {2}} = \sqrt {\sum_ {i = 1} ^ {n} w _ {i} ^ {*} \langle A _ {i} , V \rangle^ {2}} \cdot \sqrt {\sum_ {i = 1} ^ {n} w _ {i} ^ {*} \langle A _ {i} , S \rangle^ {2}} \\ \leq \sqrt {L} \cdot \sqrt {L ^ {\prime}} \cdot \| S \| _ {F} \leq \frac {L}{5} \| S \| _ {F} \\ \end{array}
$$
which implies  $\| S\| _F\leq \frac{L}{5}\leq L$  , and consequently,
$$
\left\| E \right\| _ {2} = \sigma_ {r ^ {\prime} + 1} (G) \leq \sigma_ {r ^ {\prime}} (G) = \sigma_ {r ^ {\prime}} (S) \leq \sqrt {\frac {\left\| S \right\| _ {F} ^ {2}}{r ^ {\prime}}} \leq \frac {L}{5 \sqrt {r ^ {\prime}}} = \frac {L}{4 0 0 \sqrt {2} L K \sqrt {r}} \leq \frac {1}{K \sqrt {r}}.
$$