SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition C.1 (Eigenvalues under rank-one perturbation, Theorem 1 in [BNS79]). Let  $C = D + \rho zz^T$ , where  $D$  is diagonal,  $\|z\|_2 = 1$ . Let  $d_1 \geq d_2 \geq \dots \geq d_n$  be the eigenvalues of  $D$ , and let  $\widetilde{d}_1 \geq \widetilde{d}_2 \geq \dots \geq \widetilde{d}_n$  be the eigenvalues of  $C$ . Then
$$
\widetilde {d _ {i}} = d _ {i} + \rho \mu_ {i}, \quad 1 \leq i \leq n,
$$
where  $\sum_{i=1}^{n} \mu_i = 1$  and  $0 \leq \mu_i \leq 1$ .
Proof of Theorem 3.1. Let  $\chi_i \in \{0,1\}^V$  be the indicator vector for  $V_i$ , i.e.,  $\chi_i(u) = 1$  iff  $u \in V_i$ . It is easy to see that the eigenvectors of  $H$  are  $\frac{1}{\sqrt{s_1}}\chi_1, \ldots, \frac{1}{\sqrt{s_k}}\chi_k$ . Write  $U = \left(\frac{1}{\sqrt{s_1}}\chi_1, \ldots, \frac{1}{\sqrt{s_k}}\chi_k\right) \in \mathbb{R}^{V \times V}$ ,  $D = \mathrm{diag}((p - q)s_1, \ldots, (p - q)s_k, 0, \ldots, 0)$ , then we have  $H = UDU^\top$ . Note that  $\mathbf{1}_n = U(\sqrt{s_1}, \ldots, \sqrt{s_n})^\top$ , and hence
$$
G = H + q \mathbf {1} _ {n} \mathbf {1} _ {n} ^ {\top} = U (D + \rho z z ^ {\top}) U ^ {\top},
$$
where  $\rho = nq, z = \frac{1}{\sqrt{n}} (\sqrt{s_1}, \ldots, \sqrt{s_n})^\top$ . This means the eigenvalues of  $G$  are the same as those of  $D + \rho z z^\top$ . Since  $\|z\| = 1$ , Item 1 follows directly from Proposition C.1. To see Item 2, we use the Rayleigh quotient characterization of the largest eigenvalue:
$$
\begin{array}{l} \lambda_ {1} (G) = \max  _ {v} \frac {v ^ {\top} G v}{\| v \| ^ {2}} \geq \frac {\mathbf {1} _ {n} ^ {\top} G \mathbf {1} _ {n}}{n} = \frac {\sum_ {u , v \in X} G _ {u v}}{n} = \frac {n ^ {2} q + (p - q) \cdot \left(s _ {1} ^ {2} + \cdots + s _ {k} ^ {2}\right)}{n} \\ \geq n q + (p - q) \frac {n}{k}. \\ \end{array}
$$
where the last inequality follows from  $\frac{n}{k} = \frac{1}{k}\sum_{i = 1}^{k}s_i\leq \sqrt{\sum_{i = 1}^{k}s_i^2 / k},$
Proof of Claim 3.1. The assumption on  $n$  in Lemma 3.2 implies that  $\mu = (p - q)n / k \geq 100C_0\sigma \sqrt{n} \cdot \log n$ . By Weyl's inequality, we have
$$
\left| \widehat {\lambda} _ {i} - \lambda_ {i} \right| \leq \| E \| _ {2} \leq C _ {0} \sigma \sqrt {n} \leq \frac {\mu}{1 0 0 \log n}, \forall i \in [ n ].
$$
Meanwhile, by Theorem 3.1,
$$
\left| \lambda_ {i} - \mu \right| \leq \left| \lambda_ {i} (G) - (p - q) s _ {i} \right| + \left| (p - q) s _ {i} - \mu \right| \leq \frac {(p - q) n / k}{1 6 \log n} + \frac {(p - q) n / k}{1 6 \log n} \leq \frac {\mu}{8 \log n}.
$$
for  $i = 2,3,\ldots ,k$ . Hence, write  $\varepsilon \stackrel{\mathrm{def}}{=} \frac{\mu}{6 \log n}$ , we have
1.  $\left|\widehat{\lambda}_1 - \lambda_1\right| \leq \varepsilon$
2.  $\lambda_2,\ldots ,\lambda_k,\widehat{\lambda}_2,\ldots ,\widehat{\lambda}_k\in [\mu -\varepsilon ,\mu +\varepsilon ];$
3. for every  $i \geq k + 1$ ,  $\left|\widehat{\lambda}_i\right| \leq \varepsilon$ .
First,  $\psi (\lambda_1) = 1$  according to the definition of  $\psi$ , and hence  $\varphi (\lambda_1) = 1$ . As for  $\widehat{\lambda}_1$ ,
$$
\begin{array}{l} \left| \psi (\widehat {\lambda} _ {1}) - 1 \right| = \left| \psi (\widehat {\lambda} _ {i}) - \psi (\lambda_ {1}) \right| \leq | A | \varepsilon^ {2} + | 2 A \lambda_ {1} + B | \varepsilon \quad \text {(b y d e f i n i t i o n o f} \psi) \\ \leq \frac {\varepsilon^ {2}}{\lambda_ {1} \mu} + \frac {\varepsilon}{\mu} \quad (\text {s i n c e} 2 A \lambda_ {1} + B = \frac {1}{\lambda_ {1}} - \frac {1}{\mu} \geq - \frac {1}{\mu}) \\ \leq \frac {1}{3 6 \log^ {2} n} + \frac {1}{6 \log n} \quad (\text {a s} \frac {\varepsilon}{\mu} = \frac {1}{6 \log n}) \\ <   \frac {1}{4 \log n}. \\ \end{array}
$$
Consequently,  $|\varphi (\widehat{\lambda}_1) - 1| < \frac{2r}{4\log n}\leq 1 / 2$  by Proposition 2.5.
Next, for  $a \in \left\{\lambda_2, \dots, \lambda_k, \widehat{\lambda}_2, \dots, \widehat{\lambda}_k\right\}$ , the argument is similar:
$$
| \psi (a) - 1 | = | \psi (a) - \psi (\mu) | \leq | A | \varepsilon^ {2} + | 2 A \mu + B | \varepsilon \leq \frac {\varepsilon^ {2}}{\lambda_ {1} \mu} + \frac {\varepsilon}{\mu} <   \frac {1}{4 \log n},
$$
where the second inequality follows from  $2A\mu + B = \frac{1}{\mu} - \frac{1}{\lambda_1} \leq \frac{1}{\mu}$ . This yields  $|\varphi(a) - 1| \leq 1/2$  by Proposition 2.5.
Finally, for  $i \geq k + 1$ , it holds that
$$
\left| \psi (\widehat {\lambda} _ {i}) \right| \leq | A | \varepsilon^ {2} + B \varepsilon = \frac {\varepsilon^ {2}}{\lambda_ {1} \mu} + \frac {\varepsilon}{\mu} <   \frac {1}{4 \log n},
$$
which means  $\left|\varphi (\widehat{\lambda}_i)\right|^r = \left|\psi (\widehat{\lambda}_i)\right|^r < \left(\frac{1}{4\log n}\right)^{\log n} < n^{-\log \log n}.$
D Bounding the Noise Term
Proof of Lemma 4.1. The lemma readily follows from the following entrywise bound and Chernoff bound.
Claim D.1 (Entries of  $\psi(G)$ ). For every  $u, v \in X$ , if  $u, v \in V_{\ell}$  for some  $\ell$ , then  $0 \leq F_{uv} \leq \frac{5k}{n}$ ; otherwise,  $|F_{uv}| \leq \frac{10}{n}$ .
We decompose  $F = F' + F''$ , where  $F'$  is the intra-cluster part, i.e.,  $F_{uv}' = F_{uv}$  if  $u, v \in V_{\ell}$  for some  $\ell$ , and  $F_{uv}' = 0$  otherwise. Since for every column of  $F_{v}'$ , its non-zero entries are identical and at most  $5k / n$  by the above claim. Hence, every entry of  $EF'$  equals to the sum of at most  $2n / k$  independent, centered Bernoulli variables with parameter  $p$ , scaled by some factor at most  $\frac{5k}{n}$ . By Chernoff bound,  $\mathbf{Pr}_E\left[|(EF')_{uv}| > 10\sqrt{kp\log n / n}\right] \leq n^{-4}, \forall u, v \in V$ , and we have  $\mathbf{Pr}_E\left[\| EF'\|_{2\to \infty} \leq 10\sqrt{kp\log n}\right] \geq 1 - n^{-2}$  by union bound. Analogously, by Hoeffding bound,  $\mathbf{Pr}_E\left[\| EF''\|_{2\to \infty} \leq 10\sqrt{\log n}\right] \geq 1 - n^{-2}$ . Since  $\| EF\|_{2\to \infty} \leq \| EF'\|_{2\to \infty} + \| EF''\|_{2\to \infty}$ , the lemma follows from the above two inequalities and union bound.
Proof of Claim D.1. Write  $\lambda = \lambda_1(G)$  and recall that (i)  $(p - q)s_u\leq 2\mu$  for all  $u$  (ii)  $nq + \mu \leq \lambda \leq$ $nq + (p - q)s_{1} <   nq + 2\mu$  , (iii)  $\lambda >p\cdot \mu$  . Assume that  $u,v\in V_{\ell}$  for some  $\ell$  . Then
$$
\begin{array}{l} F _ {u v} = A G _ {u} ^ {\top} G _ {v} + B G _ {u v} = - \frac {n q ^ {2} + (p ^ {2} - q ^ {2}) s _ {u}}{\lambda \mu} + \left(\frac {1}{\lambda} + \frac {1}{\mu}\right) p \\ = \frac {- n q ^ {2} - (p ^ {2} - q ^ {2}) s _ {u} + (p - q) (\lambda + \mu) + q (\lambda + \mu)}{\lambda \mu} \\ = \frac {q (\mu + \lambda - n q) + (p - q) (\lambda + \mu - (p + q) s _ {u})}{\lambda \mu}. \\ \end{array}
$$
Since  $\lambda - nq \geq \mu$ , the numerator is at least
$$
2 q \mu + (p - q) (\lambda + \mu - (p + q) s _ {u}) = (p - q) (2 q n / k + \lambda + \mu - (p + q) s _ {u}).
$$
Because  $s_u \leq 2n / k, \lambda \geq nq + (p - q)n / k$ , we have
$$
2 q n / k + \lambda + \mu - (p + q) s _ {u} > 2 q n / k + n q + (p - q) 2 n / k - (p + q) 2 n / k = (n - 2 n / k) q \geq 0,
$$
which means  $F_{uv}\geq 0$ . Meanwhile,
$$
F _ {u v} \leq \frac {q (\mu + \lambda - n q)}{\lambda \mu} + \frac {(p - q) (\lambda + \mu)}{\lambda \mu},
$$
where the first term is at most  $\frac{3q}{\lambda} \leq \frac{3}{n}$  by (ii); second term is at most  $\frac{2(p - q)}{\mu} \leq \frac{2k}{n}$ . Therefore,  $|F_{uv}| \leq \frac{5k}{n}$ .
For the second part, assume that  $u, v$  are not in the same cluster. Then
$$
F _ {u v} = A G _ {u} ^ {\top} G _ {v} + B G _ {u v} = - \frac {n q ^ {2} + (p q - q ^ {2}) (s _ {u} + s _ {v})}{\lambda \mu} + \left(\frac {1}{\lambda} + \frac {1}{\mu}\right) q.
$$
$$
\left| F _ {u v} \right| \leq \left| \frac {q (\lambda + \mu - n q)}{\lambda \mu} \right| + \left| \frac {q (p - q) \left(s _ {u} + s _ {v}\right)}{\lambda \mu} \right|.
$$
By (ii), the first term is at most  $\frac{3q}{\lambda} < \frac{3}{n}$ ; by (i), the second term is at most  $\frac{4q}{\lambda} \leq \frac{4}{n}$ ; hence,  $|F_{uv}| \leq \frac{10}{n}$ .
Upper bound of  $\| EM' \|_{2 \to \infty}$  (Proof of Lemma 4.2) Write  $L \stackrel{\text{def}}{=} A(EG + GE)$ ,  $R \stackrel{\text{def}}{=} AE^2 + BE$ . Then  $D^t F = (L + R)^t F = R^t F + R^{t-1} LF + R^{t-2} LDF + \dots + RLD^{t-2} + LD^{t-1} F$ . It suffices to derive a good upper bound of  $\| E^\eta L \|_{2 \to \infty}$  and  $\| E^\eta F \|_{2 \to \infty}$ , as  $R^w$  can be further expressed as sum of powers of  $E$ . This is done by the following lemma:
Lemma D.1. The following holds with probability  $1 - O(n^{-2})$  over the choice of  $E$ : for all  $\eta \leq \log n$ , it holds that
$\| E^{\eta}L\|_{2\to \infty}\leq C_2\sqrt{kp} (100\sqrt{np})^{\eta -1}\log^{5\eta}n,$
$\| E^{\eta}F\|_{2\to \infty}\leq C_2\sqrt{kp} (100\sqrt{np})^{\eta -1}\log^{5\eta}n,$
where  $C_2 \stackrel{\mathrm{def}}{=} 10^6$  is an absolute constant.
$$
\begin{array}{l} E D ^ {t} F = E R ^ {t} F + \sum_ {i = 0} ^ {t - 1} E R ^ {i} L D ^ {t - 1 - i} F \\ = \underbrace{E\sum_{j = 0}^{t}\binom{t}{j}A^{j}B^{t - j}E^{j + t}F}_{\stackrel{\mathrm{def}}{=}M_{t}}\quad +\underbrace{\sum_{i = 0}^{t - 1}E\sum_{j = 0}^{i}\binom{i}{j}A^{j}B^{i - j}E^{i + j}LD^{t - 1 - i}F}_{\stackrel{\mathrm{def}}{=}N_{t}}. \\ \end{array}
$$
Note that  $|A^j B^{w - j}| \leq 2^w \cdot \mu^{-(w + j)} \leq 2^w \cdot (100\sqrt{np}\log^6 n)^{-(w + j)}$ . It follows that for every  $t \in [r - 1]$ ,
$$
\begin{array}{l} \| M _ {t} \| _ {2 \to \infty} \leq \sum_ {j = 0} ^ {t} \binom {t} {j} | A ^ {j} B ^ {t - j} |   \left\| E ^ {t + j + 1} F \right\| _ {2 \to \infty} \\ \leq \sum_ {j = 0} ^ {t} \binom {t} {j} 2 ^ {t} \cdot (1 0 0 \sqrt {n p} \log^ {6} n) ^ {- (t + j)} \cdot C _ {2} \sqrt {k p} \cdot (1 0 0 \sqrt {n p}) ^ {t + j} \log^ {5 (t + j)} n \\ \leq C _ {2} \sqrt {k p} \cdot 2 ^ {t} \sum_ {j = 0} ^ {t} \binom {t} {j} (\log n) ^ {- (t + j)} \\ = C _ {2} \sqrt {k p} \cdot 2 ^ {t} (\log n) ^ {- t} \cdot \left(1 + \frac {1}{\log n}\right) ^ {t} \leq C _ {2} \sqrt {k p}, \\ \end{array}
$$
where the second inequality is by Lemma D.1, and the last step follows from Proposition 2.5. Similarly, for every  $t \in [r - 1]$ ,
$$
\begin{array}{l} \| N_{t}\|_{2\to \infty}\leq \sum_{i = 0}^{t - 1}\sum_{j = 0}^{i}\binom {i}{j}|A^{j}B^{i - j}|\left\| E^{i + j + 1}L\right\|_{2\to \infty}\left\| D^{t - 1 - i}F\right\|_{2} \\ \leq 2\sum_{i = 0}^{t - 1}\sum_{j = 0}^{i}\binom {i}{j}|A^{j}B^{i - j}|\left\| E^{i + j + 1}L\right\|_{2\to \infty} \\ \leq 2\sum_{i = 0}^{t - 1}\sum_{j = 0}^{i}\binom {i}{j}2^{i}\cdot (100\sqrt{np}\log^{6}n)^{-(i + j)}\cdot C_{2}\sqrt{kp}\cdot (100\sqrt{np})^{i + j}\log^{5(i + j)}n \\ \leq 2C_{2}\sqrt{kp}\sum_{i = 0}^{t - 1}2^{i}\sum_{j = 0}^{i}\binom {t}{j}(\log n)^{-(i + j)} \\ \leq 2 C _ {2} \sqrt {k p} \cdot t \leq 2 C _ {2} \sqrt {k p} \cdot \log n, \\ \end{array}
$$
where the second inequality follows from  $\left\| D^{t - 1 - i}F\right\| _2\leq 2$  , and the third inequality is by Lemma D.1. In sum,
$$
\left\| E M ^ {\prime} \right\| _ {2 \rightarrow \infty} \leq \sum_ {t = 1} ^ {r - 1} \left(\left\| M _ {t} \right\| _ {2 \rightarrow \infty} + \left\| N _ {t} \right\| _ {2 \rightarrow \infty}\right)\left\| \widehat {F} \right\| _ {2} ^ {r - 1 - t} \leq 6 C _ {2} \sqrt {k p} \log^ {2} n, \tag {6}
$$
where in the last inequality we also use  $\left\| \widehat{F}\right\| _2^t\leq 2$
It remains to prove Lemma D.1; the proof draws on the entrywise bound in Proposition A.3.
Proof of Lemma D.1. Fix an  $\eta \leq \log n$ . Write  $s^* \stackrel{\mathrm{def}}{=} n / k$  for the ease of notation. Observe that  $E^\eta L = A(E^{\eta + 1}G + E^\eta GE) = A(E^{\eta + 1}H + E^\eta HE) + Aq(E^{\eta + 1}J_n + E^\eta J_nE)$ . We can apply Proposition A.3 to  $E^{\eta + 1}H$  and  $E^\eta H$ , with  $\alpha = p, \beta = p - q, \gamma = 2s^*$ . That is, with probability at least  $1 - O(n^{-2})$ ,
$$
\left\| E ^ {j} H \right\| _ {2 \rightarrow \infty} \leq 5 0 0 (\log n) ^ {5 j} \sqrt {n} (p - q) \sqrt {p} \sqrt {2 s ^ {*}} \cdot (1 0 0 \sqrt {n p}) ^ {j - 1} \mathrm {f o r} j = \eta , \eta + 1.
$$
Our assumption on  $n$  yields  $\mu \geq C\sqrt{np}\log^6 n$ ; moreover,  $|A|(p - q) \leq \frac{p - q}{\mu^2} \leq 1 / s^* \cdot \frac{1}{\mu} \leq 1 / s^* \cdot (C\sqrt{np}\log^6 n)^{-1}$ . Therefore,
$$
\begin{array}{l} \left\| A \left(E ^ {\eta + 1} H + E ^ {\eta} H E\right)\right\| _ {2 \rightarrow \infty} \\ \leq A \left(\left\| E ^ {\eta + 1} H \right\| _ {2 \rightarrow \infty} + \left\| E ^ {\eta} H \right\| _ {2 \rightarrow \infty} \| E \| _ {2}\right) \\ \leq \frac {1}{s ^ {*}} \cdot \left(C \sqrt {n p} \log^ {6} n\right) ^ {- 1} \cdot 5 0 0 (\log n) ^ {5 \eta + 5} \cdot \sqrt {n} \cdot \sqrt {p} \cdot \sqrt {2 s ^ {*}} \left(\left(1 0 0 \sqrt {n p}\right) ^ {\eta} + \left(1 0 0 \sqrt {n p}\right) ^ {\eta - 1} C _ {0} \sigma \sqrt {n}\right) \\ \leq 5 0 0 0 0 0 \sqrt {n p / s ^ {*}} (\log n) ^ {5 \eta} (1 0 0 \sqrt {n p}) ^ {\eta - 1}. \tag {7} \\ \end{array}
$$
Similarly, by applying Proposition A.3 to  $E^{j}J_{n}$  with  $\alpha = p, \beta = 1, \gamma = n$ , we have, with probability at least  $1 - O(n^{-2})$ ,
$$
\left\| E ^ {j} J _ {n} \right\| _ {2 \rightarrow \infty} \leq 5 0 0 (\log n) ^ {5 j} \cdot \sqrt {p} \cdot n \cdot (1 0 0 \sqrt {n p}) ^ {j - 1}, j = \eta , \eta + 1.
$$
Since  $|Aq| = \frac{q}{\lambda} \cdot \frac{1}{\mu} \leq \frac{1}{n} \cdot (C\sqrt{n}\log^6 n)^{-1}$ , we have
$$
\begin{array}{l} \left\| A q \left(E ^ {\eta + 1} J _ {n} + E ^ {\eta} J _ {n} E\right)\right\| _ {2 \rightarrow \infty} \\ \leq \left| A q \right|\left(\left\| E ^ {\eta + 1} J _ {n} \right\| _ {2 \rightarrow \infty} + \left\| E ^ {\eta} J _ {n} \right\| _ {2 \rightarrow \infty} \| E \| _ {2}\right) \\ \leq \frac {1}{n} \cdot \left(C \sqrt {n p} \log^ {6} n\right) ^ {- 1} \cdot 5 0 0 \cdot (\log n) ^ {5 \eta + 5} \cdot n \cdot \left(\left(1 0 0 \sqrt {n p}\right) ^ {\eta} + \left(1 0 0 \sqrt {n p}\right) ^ {\eta - 1} C _ {0} \sigma \sqrt {n}\right) \\ \leq 5 0 0 0 0 0 \sqrt {p} (\log n) ^ {5 \eta} (1 0 0 \sqrt {n p}) ^ {\eta - 1}. \tag {8} \\ \end{array}
$$
Combining Equation (7) and Equation (8), we have, with probability at least  $1 - O(n^{-2})$ ,  $\| E^{\eta}L\|_{2\to \infty}\leq C_2\sqrt{np / s^{*}} (\log n)^{5\eta}(100\sqrt{np})^{\eta -1}$  where  $C_2\stackrel {\mathrm{def}}{=}10^6$ .
For the second part, we decompose  $F = F' + F''$ , where  $F'$  is the intra-cluster part, i.e.,  $F_{uv}' = F_{uv}$  if  $u, v \in V_\ell$  for some  $\ell$ ; and  $F_{uv}' = 0$  otherwise. Equipped with Claim D.1, we can apply Proposition A.3 on  $E^\eta F'$  (with  $\alpha = p, \beta = 10 / s^*, \gamma = 2s^*$ ), and  $E^\eta F''$  (with  $\alpha = p, \beta = \frac{5}{n}, \gamma = n$ ):
$$
\begin{array}{l} \| E ^ {\eta} F \| _ {2 \rightarrow \infty} \leq \| E ^ {\eta} F ^ {\prime} \| _ {2 \rightarrow \infty} + \| E ^ {\eta} F ^ {\prime \prime} \| _ {2 \rightarrow \infty} \\ \leq 2 0 0 0 0 \log^ {5 \eta} \sqrt {n} \sqrt {p} \left(\frac {1 0}{s ^ {*}} \cdot \sqrt {2 s ^ {*}} + \frac {1 0}{n} \cdot \sqrt {n}\right) (1 0 0 \sqrt {n}) ^ {\eta - 1} \\ \leq C _ {2} (\log n) ^ {5 \eta} \sqrt {n p / s ^ {*}} (1 0 0 \sqrt {n}) ^ {\eta - 1}, \\ \end{array}
$$
where the second inequality holds with probability at least  $1 - O(n^{-2})$ . The lemma follows from a union bound over all  $\eta \leq \log n$ .