SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition 2.2 (Weyl's inequality). For all  $A, E \in \mathbb{R}^{n \times n}$ , we have  $|\lambda_i(A) - \lambda_i(A + E)| \leq \|E\|_2$ . Proposition 2.3 (Norm of a random matrix [Vu18]). There is a constant  $C_0 > 0$ . Let  $E$  be a symmetric matrix whose upper diagonal entries  $e_{ij}$  are independent random variables where  $e_{ij} = 1 - p_{ij}$  or  $-p_{ij}$  with probabilities  $p_{ij}$  and  $1 - p_{ij}$  respectively, where  $p_{ij} \in [0,1]$ . Let  $\sigma^2 \coloneqq \max_{ij} \{p_{ij}(1 - p_{ij})\}$ . If  $\sigma^2 \geq C_0 \log n/n$ , then  $\mathbf{Pr}[\|E\|_2 \geq C_0 \sigma n^{1/2}] \leq n^{-3}$ .
Proposition 2.4 (Projection of a random vector, lemma 2.1 in [Vu18]). There exists a constant  $C_1$  such that the following holds. Let  $X = (\xi_{1},\dots ,\xi_{n})$  be a random vector in  $\mathbb{R}^n$  whose coordinates  $\xi_{i}$  are independent random variables with mean 0 and variance at most  $\sigma^2\leqslant 1$ . Assume furthermore that the  $\xi_{i}$  are bounded by 1 in absolute value. Let  $H$  be a subspace of dimension  $d$  and let  $\Pi_H\xi$  be the length of the orthogonal projection of  $\xi$  onto  $H$ . Then  $\mathbf{Pr}\left[\Pi_H X\geq \sigma \sqrt{d} +C_1\sqrt{\log n}\right]\leq n^{-3}$ .
Proposition 2.5. For  $a \in [0,2]$  and  $r \in \mathbb{N}$ , if  $|a - 1| \leq \delta < \frac{1}{2r}$ , then  $|a^r - 1| \leq 2r\delta$ .
3 Analysis of Vanilla SVD Algorithm
Write  $s_i \stackrel{\text{def}}{=} |V_i|$ . We say the partition  $V_1, \ldots, V_k$  is balanced if  $\left(1 - \frac{1}{16\log n}\right)\frac{n}{k} \leq s_i \leq \left(1 + \frac{1}{16\log n}\right)\frac{n}{k}, \forall i \in [k]$ . By Chernoff bound, the partition  $V_1, \ldots, V_k$  is balanced with probability at least  $1 - n^{-1}$ ; hence, we assume that the partition is balanced in the following argument. Since  $\sigma^2 \geq C\log n / n$ , the event  $\| E \| = O(\sqrt{n})$  holds with high probability (see Proposition 2.3).
Recall the decomposition into deviation term and noise term in Equation (1). We first state our upper bound of the deviation term, which readily follows from the argument in Section 1.4, and the complete proof is in Appendix B.
Lemma 3.1 (Upper bound of deviation term). Let  $C_0$  be the constant in Proposition 2.3. If the partition is balanced and  $n \geq 10^4 \cdot C_0^{2\frac{k^2\sigma^2}{(p - q)^2}}$ , then with probability at least  $1 - n^{-3}$  we have  $\left\| (P_{\widehat{G}_k} - I)G_u\right\| \leq 0.04(p - q)\sqrt{n / k}, \forall u \in V$ .
Section 3.1 and Section 3.2 lead to an upper bound of the noise term, and Section 3.3 is the proof of main theorem.
3.1 An Approximation of  $P_{G_k}$  and  $P_{\widehat{G}_k}$
In order to give some intuition on the choice of  $\varphi$ , we first analyze the spectrum of  $G$ , and the result is summed up in Theorem 3.1.
The eigenvalues of  $G$  Note that  $G = H + q\mathbf{1}_n\mathbf{1}_n^\top$  , where  $H = \left( \begin{array}{cccc}(p - q)J_{s_1} & \mathbf{0} & \mathbf{0} & \mathbf{0}\\ \mathbf{0} & (p - q)J_{s_2} & \mathbf{0} & \mathbf{0}\\ \vdots & \ddots & \ddots \\ \mathbf{0} & \mathbf{0} & \mathbf{0} & (p - q)J_{s_k} \end{array} \right).$
Without loss of generality, assume that  $s_1 \geq s_2 \geq \dots \geq s_k$ . It is easy to see that the eigenvalues of  $H$  are  $(p - q)s_1, \ldots, (p - q)s_k, 0$ . Viewing  $G$  as a rank-one perturbation of  $H$ , we have the following theorem that characterizes eigenvalues of  $G$ . Its proof, in Appendix C, readily follows from a theorem in [BNS79], which studies eigenvalues under rank-one perturbation.
Theorem 3.1. Write  $s_i \stackrel{\text{def}}{=} |V_i|$  and assume that  $s_1 \geq s_2 \geq \dots \geq s_k$ . Define  $\delta_i \stackrel{\text{def}}{=} \lambda_i(G) - (p - q)s_i$ , then (1)  $\delta_i \geq 0$  and  $\sum_{i=1}^k \delta_i = nq$ ; (2)  $\lambda_1(G) \geq nq + (p - q)\frac{n}{k}$ , and hence  $\sum_{i=2}^k \delta_i \leq (p - q)(s_1 - \frac{n}{k})$ .
The choice of the polynomial  $\varphi$  Let  $\mu \stackrel{\mathrm{def}}{=} (p - q) \frac{n}{k}$ , and let  $\psi(t)$  be the quadratic polynomial such that  $\psi(\lambda_1(G)) = \psi(\mu) = 1$ ,  $\psi(0) = 0$ , i.e.,  $\psi(t) \stackrel{\mathrm{def}}{=} -\frac{1}{\lambda_1(G)\mu} (t - \lambda_1(G))(t - \mu) + 1 \stackrel{\mathrm{def}}{=} At^2 + Bt$ , where  $A = -\frac{1}{\lambda_1(G)\mu}$ ,  $B = \frac{1}{\lambda_1(G)} + \frac{1}{\mu}$ . Finally, let  $\varphi(t) \stackrel{\mathrm{def}}{=} (\psi(t))^r$  where  $r \stackrel{\mathrm{def}}{=} \log n$ .
Here we give some intuition for the choice of  $\varphi$ . Let  $\widehat{G} = \sum_{i=1}^{n} \widehat{\lambda}_i v_i v_i^\top$  be the spectral decomposition of  $\widehat{G}$ . Then  $\varphi(\widehat{G}) = \sum_{i=1}^{n} \varphi(\widehat{\lambda}_i) v_i v_i^\top$ ,  $P_{\widehat{G}_k} = \sum_{i=1}^{k} v_i v^\top$ . The spectral decomposition of  $\varphi(\widehat{G}) - P_{\widehat{G}_k}$  is  $\varphi(\widehat{G}) - P_{\widehat{G}_k} = \sum_{i=1}^{k} (\varphi(\widehat{\lambda}_i) - 1) v_i v^\top + \sum_{i=k+1}^{n} \varphi(\lambda_i) v_i v^\top$ . Hence,
$$
\left\| \varphi (\widehat {G}) - P _ {\widehat {G} _ {k}} \right\| _ {2} = \max  \{| \varphi (\widehat {\lambda} _ {1}) - 1 |, \dots , | \varphi (\widehat {\lambda} _ {k}) - 1 |, | \varphi (\widehat {\lambda} _ {k + 1}) |, \dots , | \varphi (\widehat {\lambda} _ {n}) | \}. \tag {3}
$$
Recall that  $\widehat{\lambda}_i - \lambda_i(G)$  is bounded by Weyl's inequality. Plus, when the partition is balanced, Theorem 3.1 shows that the eigenvalues of  $G$  is nicely distributed: Except for  $\lambda_1(G)$ , other eigenvalues are all close to  $\mu$ . Hence, our choice of  $\varphi$  makes  $\left\| \varphi (\widehat{G}) - P_{\widehat{G}_k}\right\| _2$  small, and thus  $\varphi (\widehat{G})$  is a good approximation of  $P_{\widehat{G}_k}$ . Formally, we have the following lemma.
Lemma 3.2 (Polynomial approximation). Assume that the partition is balanced and  $n \geq 10^4 \cdot C_0^2 \cdot \frac{k^2 \cdot p \cdot \log n}{(p - q)^2}$ , where  $C_0$  is the constant in Proposition 2.3. Then with probability at least  $1 - n^{-3}$ , it holds that for all  $x \in \mathbb{R}^n$ ,  $\frac{1}{2} \left\| P_{\widehat{G}_k}x \right\| \leq \left\| \varphi (\widehat{G})x \right\| \leq \frac{3}{2} \left\| P_{\widehat{G}_k}x \right\| + \| x \| / n^{\log \log n}$ , and  $\frac{1}{2} \| P_{G_k}x \| \leq \| \varphi (G)x \| \leq \frac{3}{2} \| P_{G_k}x \|$ .
Proof. Let  $G = \sum_{i=1}^{k} \lambda_i u_i u_i^\top$  (resp.,  $\widehat{G} = \sum_{i=1}^{n} \widehat{\lambda}_i v_i v_i^\top$ ) be the spectral decomposition of  $G$  (resp.,  $\widehat{G}$ ). We shall use the following claim.
Claim 3.1. The following holds with probability  $1 - n^{-3}$  (over the choice of  $E$ ): for every  $i \in [k]$ ,  $\left|\varphi(\widehat{\lambda}_i) - 1\right| < \frac{1}{2}, |\varphi(\lambda_i) - 1| < \frac{1}{2}$ ; and for every  $i = k + 1, \ldots, n$ ,  $\left|\varphi(\widehat{\lambda}_i)\right| < n^{-\log \log n}$ .
Fix  $x\in \mathbb{R}^n$  . On the one hand,  $\frac{1}{2}\leq \varphi (\widehat{\lambda}_i)\leq \frac{3}{2},\forall i\in [k],$  and hence
$$
\left\| \varphi (\widehat {G}) x \right\| ^ {2} = \sum_ {i = 1} ^ {n} \varphi (\widehat {\lambda} _ {i}) ^ {2} \langle x, v _ {i} \rangle^ {2} \geq \sum_ {i = 1} ^ {k} \varphi (\widehat {\lambda} _ {i}) ^ {2} \langle x, v _ {i} \rangle^ {2} \geq \sum_ {i = 1} ^ {k} \frac {1}{4} \langle x, v _ {i} \rangle^ {2} = \frac {1}{4} \left\| P _ {\widehat {G} _ {k}} x \right\| ^ {2},
$$
which means  $\left\| \varphi (\widehat{G})x\right\| \geq \frac{1}{2}\left\| P_{\widehat{G}_k}x\right\|$ . On the other hand,
$$
\left\| \varphi (\widehat {G}) x \right\| ^ {2} = \sum_ {i = 1} ^ {n} \varphi (\widehat {\lambda} _ {i}) ^ {2} \langle x, v _ {i} \rangle^ {2} \leq \sum_ {i = 1} ^ {k} \left(\frac {3}{2}\right) ^ {2} \langle x, v _ {i} \rangle^ {2} + \sum_ {i = k + 1} ^ {n} \frac {\langle x , v _ {i} \rangle^ {2}}{n ^ {2 \log \log n}} \leq \frac {9}{4} \left\| P _ {\widehat {G} _ {k}} x \right\| ^ {2} + \frac {\| x \| ^ {2}}{n ^ {2 \log \log n}}.
$$
Since  $\sqrt{a + b} \leq \sqrt{a} + \sqrt{b}$ , we have  $\left\| \varphi(\widehat{G}) x \right\| \leq \frac{3}{2} \left\| P_{\widehat{G}_k} x \right\| + \frac{\|x\|}{n^{\log \log n}}$ . This establishes the first part.
Note that  $\| \varphi (G)x\| = \sqrt{\sum_{i = 1}^{k}\varphi(\lambda_i)^2\langle x,u_i\rangle^2}$  and we also have  $\frac{1}{2}\leq \varphi (\lambda_i)\leq \frac{3}{2},\forall i\in [k]$ , and thus similar argument goes for  $G$ . This finishes the proof of Lemma 3.2.
It remains to prove Claim 3.1. The claim readily follows from the choice of  $\varphi$  and the fact that  $\lambda_{i},\widehat{\lambda}_{i}$  are close. A complete proof can be found in Appendix C.
3.2 The Upper Bound of the Noise Term
According to Equation (1), in order to derive an upper bound of  $\| P_{G_k}E_u\|$ , it remains to bound  $\left\| \left(\varphi (\widehat{G}) - \varphi (G)\right)E_u\right\|$  from above. This is done by the following lemma.
Lemma 3.3. Let  $C_0$  be the constant in Proposition 2.3. Assume that the partition is balanced and  $n \geq (100 + C_0)^2 \cdot \frac{k^2 \cdot p \cdot \log^{12} n}{(p - q)^2}$ . For every  $u \in V$ , it holds that
$$
\mathbf {P r} _ {E} \left[ \left\| \left(\varphi (\widehat {G}) - \varphi (G)\right) E _ {u} \right\| \leq C _ {2} (\sqrt {k p} \log^ {2} n) + \frac {1}{\log n}) \right] \geq 1 - O (n ^ {- 2}),
$$
where  $C_2 \stackrel{\mathrm{def}}{=} 7 \cdot 10^6$  is a constant.
Combining Lemma 3.2 and Proposition 2.4, we get an upper bound of the noise term:
Lemma 3.4 (Upper bound of noise term). Let  $C_0$  be the constant in Proposition 2.3. Assume that  $n \geq (100 + C_0)^2 \cdot \frac{k^2 \cdot p \cdot \log^{12} n}{(p - q)^2}$ . Then with probability at least  $1 - O(n^{-1})$ , we have  $\left\| P_{\widehat{G}_k} E_u \right\| \leq C_3 (\sqrt{kp} \log^2 n + \sqrt{\log n})$  for all  $u \in V$ , where  $C_3$  is a constant.
The proof of Lemma 3.3 is deferred to Section 4. We use it to prove Lemma 3.4 here.
Proof of Lemma 3.4. It follows from Lemma 3.2 that
$$
\begin{array}{l} \left\| P _ {\widehat {G} _ {k}} E _ {u} \right\| \leq 2 \left\| \varphi (\widehat {G}) E _ {u} \right\| \leq 2 \left\| \left(\varphi (\widehat {G}) - \varphi (G)\right) E _ {u} \right\| + 2 \| \varphi (G) E _ {u} \| \\ \leq 2 \left\| \left(\varphi (\widehat {G}) - \varphi (G)\right) E _ {u} \right\| + 3 \| P _ {G _ {k}} E _ {u} \|. \\ \end{array}
$$
By Proposition 2.4, with high probability at least  $1 - n^{-1}$ ,  $\| P_{G_k}E_u\|$  is bounded by  $\sigma \sqrt{k} + C_1\sqrt{\log n}$  where  $C_1$  is a universal constant. Meanwhile, by Lemma 3.3 and union bound over all  $u$ , with probability at least  $1 - O(n^{-1})$ ,  $\left\| \left(\varphi (\widehat{G}) - \varphi (G)\right)E_u\right\| \leq 7\cdot 10^6 (\sqrt{kp}\log^2 n + 1 / \log n)$  for every  $u \in V$ . Therefore, with probability  $1 - O(n^{-1})$ , it holds that  $\left\| P_{\widehat{G}_k}E_u\right\| \leq 1.4\times 10^7\sqrt{kp}\log^2 n + 3\sigma \sqrt{k} + 3C_1\sqrt{\log n}$  for all  $u \in V$ . Setting  $C_3 \stackrel{\mathrm{def}}{=} (1.4\times 10^7 + 3 + 3C_1)$ , we have the desired result.
3.3 Putting It Together
Now we are well-equipped to prove Theorem 1.1.
Proof of Theorem 1.1. Let  $C \stackrel{\text{def}}{=} (100 + 100C_0 + 100C_3)^2$ , where  $C_0, C_3$  are the constants in Proposition 2.3 and Lemma 3.4. By our assumption on  $n$ , we have  $(p - q)\sqrt{n / k} > 100C_3(\sqrt{kp}\log^6 n + \sqrt{\log n})$ . It is easy to verify  $n$  satisfies the conditions in Lemma 3.4 and Lemma 3.1.
Write  $\Delta \stackrel{\mathrm{def}}{=} 0.8(p - q)\sqrt{n / k}$ . We aim to show that for every  $u, v \in V$ : if  $u, v \in V_{\ell}$  for some  $\ell$ , then  $\left\| P_{\widehat{G}_k} \widehat{G}_u - P_{\widehat{G}_k} \widehat{G}_v \right\| \leq \Delta / 4$ ; otherwise,  $\left\| P_{\widehat{G}_k} \widehat{G}_u - P_{\widehat{G}_k} \widehat{G}_v \right\| \geq \Delta$ . Then by calling ClusterByDistance, alg. 1 recovers all large clusters correctly.
Let  $\varepsilon(u) \stackrel{\mathrm{def}}{=} \left\| P_{\widehat{G}_k} \widehat{G}_u - G_u \right\|$ . According to the argument in Section 1.4, it suffices to show that  $\varepsilon(u) \leq 0.1(p - q)\sqrt{n / k}$  for all  $u \in V$ . We further decompose  $\varepsilon(u)$  into noise term and deviation term, i.e.,  $\varepsilon(u) \leq \text{noise}(u) + \text{dev}(u)$ , where  $\text{noise}(u) \stackrel{\mathrm{def}}{=} \| P_{\widehat{G}} E_u \|$  and  $\text{dev}(u) \stackrel{\mathrm{def}}{=} \| (P_{\widehat{G}} - I) G_u \|$ . By Lemma 3.4 and Lemma 3.1, with probability at least  $1 - O(n^{-1})$ , the following hold for all  $u \in V$ : (1)  $\text{noise}(u) \leq C_3 (\sqrt{k p} \log^2 n + \sqrt{\log n}) \leq 0.01(p - q) \sqrt{n / k}$ ; (2)  $\text{dev}(u) \leq 0.04(p - q) \sqrt{n / k}$ . Therefore, with probability at least  $1 - O(n^{-1})$ , we indeed have  $\varepsilon(u) \leq 0.1(p - q) \sqrt{n / k}, \forall u \in V$ . This completes the proof.
4 Proof of Lemma 3.3: Entrywise Analysis
This section is dedicated to proving Lemma 3.3.
Since both  $(\varphi(\widehat{G}) - \varphi(G))$  and  $E$  are symmetric, we have  $\left\| (\varphi(\widehat{G}) - \varphi(G)) E_u \right\| \leq \left\| E(\varphi(\widehat{G}) - \varphi(G)) \right\|_{2 \to \infty}$ . The high-level idea is to write  $E(\varphi(\widehat{G}) - \varphi(G))$  as a sum of matrices, where each matrix is of the form  $E^t S Q$  such that  $\| Q \|_2 = O(1)$ . This way, we have  $\| E^t S Q \|_{2 \to \infty} \leq \| E^t S \|_{2 \to \infty} \cdot O(1)$ , and  $\| E^t S \|_{2 \to \infty}$  is bounded by a lemma from [MZ22b].
Let  $D \stackrel{\text{def}}{=} \psi(\widehat{G}) - \psi(G) = A(EG + GE + E^2) + BE$  and write  $F \stackrel{\text{def}}{=} \psi(G), \widehat{F} \stackrel{\text{def}}{=} \psi(\widehat{G})$ . Then
$$
\begin{array}{l} \varphi (\widehat {G}) - \varphi (G) = \psi (\widehat {G}) ^ {r} - \psi (G) ^ {r} = (F + D) ^ {r} - F ^ {r} \\ = \underbrace{F^{r - 1}D + F^{r - 2}D\widehat{F} + \cdots + FD\widehat{F}^{r - 2}}_{\stackrel{\mathrm{def}}{= M}} + D\widehat{F}^{r - 1}, \\ \end{array}
$$
where the last step is a decomposition based on the first location of  $D$  in the product terms. And
$$
D\widehat{F}^{r - 1} = D(D + F)^{r - 1} = D^{r} + \underbrace{DF\widehat{F}^{r - 2} + D^{2}F\widehat{F}^{r - 3} + \cdots + D^{r - 1}F}_{\stackrel{\text{def}}{= M^{\prime}}}.
$$
That is,  $E(\varphi(\widehat{G}) - \varphi(G)) = EM + ED^r + EM'$ . We bound the three terms respectively.
Here we first list some definitions and estimations of the quantities involved.
- According to Proposition 2.3, with probability at least  $1 - n^{-3}$ , we have  $\|E\|_2 \leq C_0 \sigma \sqrt{n}$ , where  $C_0$  is a constant. In the following argument, we always assume this holds.
-  $\mu \stackrel{\mathrm{def}}{=} (p - q)n / k$ . By our assumption on  $n$ , we have  $\mu \geq (100 + C_0)\sqrt{np}\log^6 n$ .
-  $A = -\frac{1}{\lambda_1(G)\mu}, B = \left(\frac{1}{\lambda_1(G)} + \frac{1}{\mu}\right), r = \log n; \lambda_1(G) > \mu,$  and thus  $B \leq \frac{2}{\mu}, |A| \leq \frac{1}{\mu^2}$ .
- By Claim 3.1,  $\| F\| _2\leq 1 + \frac{1}{4\log n},\left\| \widehat{F}\right\| _2\leq 1 + \frac{1}{4\log n}.$  By Proposition 2.5,  $\| F\| _2^t,\left\| \widehat{F}\right\| _2^t\leq$ $2,\forall t\leq \log n$
Upper bound of  $\| EM\|_{2\to \infty}$  Note that  $\| EF^t D\|_{2\to \infty}\leq \| EF\|_{2\to \infty}\| F\|_2^{t - 1}\| D\|_2$ , and  $\| F\|_2^{t - 1}\leq 2$  for all  $t\leq r$ . Moreover,  $\| D\|_2\leq |A|(2\| E\|_2\| G\|_2 + \| E\|_2^2) + B\| E\|_2\leq 3\frac{\|E\|_2}{\mu} +\frac{\|E\|_2^2}{\mu^2} +\leq 4\frac{\|E\|_2}{\mu}\leq 4(\log^6 n)^{-1} < \frac{1}{\log^3n}$ . And the following lemma gives an upper bound of  $\| EF\|_{2\to \infty}$ .
Lemma 4.1.  $\mathbf{Pr}_E\left[\| EF\|_{2\to \infty}\leq 10(\sqrt{kp\log n} +\sqrt{\log n})\right]\geq 1 - 2n^{-2}.$
Therefore, by union bound, we have the following holds with probability at least  $1 - n^{-1}$ :
$$
\left\| E M \right\| _ {2 \rightarrow \infty} \leq r \cdot 1 0 \left(\sqrt {k p \log n} + \sqrt {\log n}\right) \cdot 2 \cdot \frac {1}{\log^ {3} n} \leq \frac {4 0 (\sqrt {k p} + 1)}{\log n}. \tag {4}
$$
Upper bound of  $\|ED^r\|_{2\to \infty}$  Since  $\|D\|_2 < \frac{1}{\log^3 n}$ , we have
$$
\left\| E D ^ {r} \right\| _ {2 \rightarrow \infty} \leq \left\| E \right\| _ {2 \rightarrow \infty} \left\| D \right\| _ {2} ^ {r} \leq \sqrt {n} \cdot \left(\log^ {3} n\right) ^ {- \log n} <   \frac {1}{n}. \tag {5}
$$
Lemma 4.2 (Upper bound of  $\| EM' \|_{2 \to \infty}$ ). With probability  $1 - O(n^{-2})$  (over the choice of  $E$ ), we have  $\| EM' \|_{2 \to \infty} \leq 6C_2\sqrt{kp}\log^2 n$ , where  $C_2 = 10^6$  is a constant.
Finally, combining Equation (4), Equation (5), and the above lemma, we conclude that with probability at least  $1 - O(n^{-2})$
$$
\left\| E (\varphi (\widehat {G}) - \varphi (G)) \right\| _ {2 \rightarrow \infty} \leq \frac {4 0 (\sqrt {k p} + 1)}{\log n} + \frac {1}{n} + 6 C _ {2} \sqrt {k p} \log^ {2} n \leq 7 C _ {2} (\sqrt {k p} \log^ {2} n + \frac {1}{\log n}).
$$
This establishes Lemma 3.3.
Proofs of Lemma 4.1 and Lemma 4.2 are deferred to Appendix D.
We are grateful to anonymous NeurIPS reviewers for their helpful comments. This research is supported by NSF CAREER award 2141536.
[AFWZ20] Emmanuel Abbe, Jianqing Fan, Kaizheng Wang, and Yiqiao Zhong. Entrywise eigenvector analysis of random matrices with low expected rank. Ann. Statist., 48(3):1452-1474, 2020.
[ARS+04] Paolo Antonelli, HE Revercomb, LA Sromovsky, WL Smith, RO Knuteson, DC Tobin, RK Garcia, HB Howell, H-L Huang, and FA Best. A principal component noise filter for high spectral resolution infrared measurements. Journal of Geophysical Research: Atmospheres, 109(D23), 2004.
[BNS79] James R. Bunch, Christopher P. Nielsen, and Danny C. Sorensen. Rank-one modification of the symmetric eigenproblem. Numer. Math., 31(1):31-48, 1978/79.
[Col19] Sam Cole. Recovering nonuniform planted partitions via iterated projection. Linear Algebra and its Applications, 576:79-107, 2019.
[CTP19] Joshua Cape, Minh Tang, and Carey E Priebe. The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics. The Annals of Statistics, 47(5):2405-2439, 2019.
[EBW18] Justin Eldridge, Mikhail Belkin, and Yusu Wang. Unperturbed: spectral analysis beyond Davis-Kahan. In Algorithmic learning theory 2018, volume 83 of Proc. Mach. Learn. Res. (PMLR), page 38. Proceedings of Machine Learning Research PMLR, [place of publication not identified], 2018.
[GM05] Joachim Giesen and Dieter Mitsche. Reconstructing many partitions using spectral techniques. In International Symposium on Fundamentals of Computation Theory, pages 433-444. Springer, 2005.
[HLL83] Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic block-models: First steps. Social networks, 5(2):109-137, 1983.
[KAH19] Vladimir Yu Kiselev, Tallulah S Andrews, and Martin Hemberg. Challenges in unsupervised clustering of single-cell rna-seq data. Nature Reviews Genetics, 20(5):273-282, 2019.
[LR15] Jing Lei and Alessandro Rinaldo. Consistency of spectral clustering in stochastic block models. The Annals of Statistics, 43(1):215-237, 2015.
[LZK22] Boris Landa, Thomas TCK Zhang, and Yuval Kluger. Bi whitening reveals the rank of a count matrix. SIAM Journal on Mathematics of Data Science, 4(4):1420-1446, 2022.
[McS01] Frank McSherry. Spectral partitioning of random graphs. In 42nd IEEE Symposium on Foundations of Computer Science (Las Vegas, NV, 2001), pages 529-537. IEEE Computer Soc., Los Alamitos, CA, 2001.
[MZ22a] Chandra Sekhar Mukherjee and Jiapeng Zhang. Compressibility: Power of pca in clustering problems beyond dimensionality reduction. arXiv preprint arXiv:2204.10888, 2022.
[MZ22b] Chandra Sekhar Mukherjee and Jiapeng Zhang. Detecting hidden communities by power iterations with connections to vanilla spectral algorithms, 2022.
$\left[\mathrm{PPV}^{+}19\right]$  Carey E Priebe, Youngser Park, Joshua T Vogelstein, John M Conroy, Vince Lyzinski, Minh Tang, Avanti Athreya, Joshua Cape, and Eric Bridgeford. On a two-truths phenomenon in spectral graph clustering. Proceedings of the National Academy of Sciences, 116(13):5995-6000, 2019.
[SEK04] Michael Steinbach, Levent Ertöz, and Vipin Kumar. The challenges of clustering high dimensional data. In New directions in statistical physics, pages 273-309. Springer, 2004.
[Vu18] Van Vu. A simple SVD algorithm for finding hidden partitions. Combin. Probab. Comput., 27(1):124-140, 2018.
[ZLWZ09] Lei Zhang, Rastislav Lukac, Xiaolin Wu, and David Zhang. Pca-based spatially adaptive denoising of cfa images for single-sensor digital cameras. IEEE transactions on image processing, 18(4):797-812, 2009.
A Useful inequalities
Proposition A.1 (Chernoff bound). Let  $X_{1}, \ldots, X_{m}$  be i.i.d random variables that can take values in  $\{0, 1\}$ , with  $\mathbf{E}[X_i] \leq p$  for  $1 \leq i \leq m$ . Then it holds that
$$
\mathbf {P r} \left[ \left| \sum_ {i = 1} ^ {n} X _ {i} - m p \right| \geq t \right] \leq \exp \left(- \frac {3 t ^ {2}}{m p}\right).
$$
Proposition A.2 (Hoeffding bound). Let  $X_{1}, \ldots, X_{m}$  be independent random variables such that  $a_{i} \leq X_{1} \leq b_{i}$ , and write  $S \stackrel{\mathrm{def}}{=} \sum_{i=1}^{m} X_{i}$ . Then it holds that
$$
\mathbf {P r} \left[ | S - \mathbf {E} [ S ] | > t \right] \leq 2 \exp \left(- \frac {2 t ^ {2}}{\sum_ {i = 1} ^ {m} (b _ {i} - a _ {i}) ^ {2}}\right).
$$
Definition A.1. Let  $X$  be a Bernoulli random variable with parameter  $p$ , i.e.,  $\mathbf{Pr}[X = 1] = p$ ,  $\mathbf{Pr}[X = 0] = 1 - p$ . The random variable  $Y \stackrel{\mathrm{def}}{=} X - p = X - \mathbf{E}[X]$  is called centered Bernoulli random variable with parameter  $p$ .
Proposition A.3 (Adapted from [MZ22b]). Let  $S \in \mathbb{R}^{n \times n}$ , and let  $E = (\xi_{ij})$  be an  $n \times n$  symmetric random matrix, where
$$
\left\{\xi_ {i j}: 1 \leq i \leq j \leq n \right\}
$$
are independent, centered Bernoulli random variables with parameter at most  $\alpha$  for all  $i,j$ . Suppose that every entry of  $S$  takes value in  $[- \beta, \beta]$ , and each column of  $S$  has at most  $\gamma$  non-zero entries. Then for every  $t \in [\log n]$ , it holds that
$$
\mathbf {P r} \left[ \big | (E ^ {t} S) _ {i j} \big | > (\log n) ^ {5 t} C _ {t} \right] = O (n ^ {- 4}), \forall i, j \in [ n ],
$$
$$
C _ {t} \stackrel {\text {d e f}} {=} 5 0 0 \beta \sqrt {\alpha} \sqrt {\gamma} \cdot (1 0 0 \sqrt {n \alpha}) ^ {t - 1}.
$$
By union bound,
$$
\Pr \left[\left\| E ^ {t} S \right\| _ {2 \rightarrow \infty} > \sqrt {n} (\log n) ^ {5 t} C _ {t} \right] = O (n ^ {- 2}).
$$
Remark A.1. The parameter  $\alpha$  is determined by  $E$ , which equals to  $p$  in our case. The above bound is particularly useful when  $\beta, \gamma$  are small, that is, we want the matrix  $S$  to have either small entries or sparse columns.
Proposition A.4 (Proposition 2.5 restated). For  $a \in [0,2]$  and  $r \in \mathbb{N}$ , if  $|a - 1| \leq \delta < \frac{1}{2r}$ , then  $|a^r - 1| \leq 2r\delta$ .
Proof. Let  $x = a - 1 \in [-\delta, \delta]$ . If  $0 \leq a \leq 1$ , we have  $1 \geq a^r = (1 + x)^r \geq 1 + rx \geq 1 - r\delta$ . If  $1 < a < 1 + 1/r$ , then  $0 < x < 1/r$  and hence
$$
1 \leq a ^ {r} = (1 + x) ^ {r} = \sum_ {i = 0} ^ {r} {\binom {r} {i}} x ^ {i} \leq \sum_ {i = 0} ^ {r} r ^ {i} x ^ {i} <   \sum_ {i = 0} ^ {\infty} (r x) ^ {i} = \frac {1}{1 - r x} = 1 + \frac {r x}{1 - r x} \leq 1 + 2 r \delta .
$$
B Bounding the Deviation Term
Proof of Lemma 3.1. Our assumption on  $n$  implies that  $(p - q)n / k > 100C_0\sigma \sqrt{n}$ . By Proposition 2.3, with probability at least  $1 - n^{-3}$ , we have
$$
\left\| E \right\| _ {2} \leq C _ {0} \sigma \sqrt {n} \leq 0. 0 1 (p - q) n / k.
$$
According to Equation (2) and  $s_u \geq \frac{n}{2k}$ , we have
$$
\left| \left| \left(P _ {\widehat {G}} - I\right) G _ {u} \right| \right| \leq \frac {2 \| E \| _ {2}}{\sqrt {s _ {u}}} \leq 0. 0 4 (p - q) n / k.
$$
C Polynomial Approximation
The proof of Theorem 3.1 relies on the following result on rank-one perturbation.