SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Here  $D(x||y)$  is the KL divergence of  $x$  and  $y$ . We recall the KL divergence between Bernoulli random variables  $x, y$ .  $D(x||y) = x\ln (x / y) + (1 - x)\ln ((1 - x) / (1 - y))$ . It is easy to see that If  $x \geq y$ , then  $D(x||y) \geq \frac{(x - y)^2}{2x}$ , and  $D(x||y) \geq \frac{(x - y)^2}{2y}$  otherwise.
We also note down a random projection Lemma that we use in our proof.
Lemma A.3 (Expected random projection [28]). Let  $P_{\widehat{A}_{k'}}$  be a  $k'$ -dimensional projection matrix, and  $e_u$  be an  $n$  dimensional random vector where each entry is 0 mean and has a variance of at most  $\sigma^2$ . Then we have  $\operatorname{E}[\| P_{\widehat{A}_{k'}}(e_u) \|^2] \leq \sigma^2 \cdot k'$ .
B Deferred Proofs from Section 2
We first give a general concentration bound concerning neighbors of vertices in the different partitions.
Lemma B.1. Let  $V$  be a set of  $n$  vertices sampled according to the  $SBM(n,k,p,q)$  model. Let  $V' \subset V$  where the vertices in  $V'$  are selected independently of each other. Let  $V_i$  be a latent cluster with  $V_i' = V_i \cap V'$ . We denote by  $N_{V'}(u)$  the number of neighbors of  $u$  in  $V'$ . Then with probability  $1 - \mathcal{O}(n^{-7})$  we have for every  $u \in V_i'$ ,
$$
\begin{array}{l} q \left| V ^ {\prime} \right| + (p - q) \left| V ^ {\prime} \cap V _ {i} \right| - 1 6 \cdot \sqrt {p} \cdot \sqrt {n} \log n \\ \leq N _ {V ^ {\prime}} (u) \leq q | V ^ {\prime} | + (p - q) | V ^ {\prime} \cap V _ {i} | + 4 8 \cdot \sqrt {p} \cdot \sqrt {n} \log n. \\ \end{array}
$$
Proof of Lemma B.1. We look at two different sums of random variables. The first is  $N_{V_i'}(u)$  which is the sum of  $|V_i \cap V'|$  many random 0-1 variables with probability of 1 being  $p$ . The second is  $N_{V' \setminus V_i'}(u)$ , which is the sum of  $|V' \setminus V'|$  variables with probability of 1 being  $q$ .
Then we have  $\operatorname{E}[N_{V_i'}(u)] = p|V_i \cap V'$  and  $\operatorname{E}[N_{V' \setminus V_i'}(u)] = q|V' \setminus V'$ . Finally the Chernoff bound implies,
1.  $\mathbf{Pr}\left(\frac{N_{V_i'}(u)}{|V_i'|} < p - \alpha\right) \leq e^{-D(p - \alpha ||p)|V_i'|$ . We fix  $\alpha = \frac{8\sqrt{p}\sqrt{n}\log n}{|V_i'|}$  and then the term  $D(p - \alpha ||p)|V_i'$  evaluates to
$$
D (p - \alpha | | p) | V _ {i} ^ {\prime} | \geq \frac {\alpha^ {2} | V _ {i} ^ {\prime} |}{2 p} \geq \frac {8 \cdot p \cdot n \cdot 2 \log n \cdot | V _ {i} ^ {\prime} |}{| V _ {i} ^ {\prime} | ^ {2} \cdot 2 p} \geq \frac {8 \cdot \log n \cdot n}{| V _ {i} ^ {\prime} |} \geq 8 \log n.
$$
$$
\Pr \left(N _ {V _ {i} ^ {\prime}} (u) <   p \left| V _ {i} ^ {\prime} \right| - 8 \sqrt {p} \sqrt {n} \log n\right) \leq n ^ {- 8} \tag {1}
$$
2.  $\mathbf{Pr}\left(\frac{N_{V'\setminus V_i'}(u)}{|V'\setminus V_i'|} < q - \beta\right) \leq e^{-D(q - \beta ||q)|V'\setminus V_i'|$ . We fix  $\beta = \frac{8\sqrt{p}\sqrt{n}\log n}{|V'\setminus V_i'|\}$  and the term  $D(q - \beta ||q)|V'\setminus V_i'$  evaluates to
$$
D (q - \beta | | q) | V ^ {\prime} \setminus V _ {i} ^ {\prime} | \geq \frac {\beta^ {2}}{2 q} \geq \frac {8 \cdot p \cdot n \cdot 2 \log n}{| V ^ {\prime} \setminus V _ {i} ^ {\prime} | 2 q} \geq \frac {p \cdot 8 \log n}{q} \cdot \frac {n}{| V ^ {\prime} \setminus V _ {i} ^ {\prime} |} \geq 8 \log n.
$$
$$
\Pr \left(N _ {V ^ {\prime} \backslash V _ {i} ^ {\prime}} (u) <   q \mid V ^ {\prime} \backslash V _ {i} ^ {\prime} \mid - 8 \sqrt {p} \sqrt {n} \log n\right) \leq n ^ {- 8} \tag {2}
$$
Combining Equation (1) and (2) gives us
$$
\begin{array}{l} \Pr \left(N _ {V _ {i} ^ {\prime}} (u) + N _ {V ^ {\prime} \backslash V _ {i} ^ {\prime}} (u) <   p \mid V _ {i} ^ {\prime} | - 8 \sqrt {p} \sqrt {n} \log n + q \mid V ^ {\prime} \backslash V _ {i} ^ {\prime} | - 8 \sqrt {p} \sqrt {n} \log n\right) \leq 2 n ^ {- 8} \\ \Rightarrow \mathbf {P r} \left(N _ {V ^ {\prime}} (u) <   q | V ^ {\prime} | + (p - q) | V _ {i} ^ {\prime} | - 1 6 \sqrt {p} \sqrt {n} \log n\right) \leq 2 n ^ {- 8}. \\ \end{array}
$$
Now we study the event  $N_{V^{\prime}}(u)\geq q|V^{\prime}| + (p - q)|V^{\prime}\cap V_{i}| + 48\cdot \sqrt{p}\cdot \sqrt{n}\log n$  again by breaking into two terms.
The probability bounds for two terms  $N_{V_i'}(u)$  and  $N_{V' \setminus V_i'}$  are  $e^{-D(p + 3\alpha ||p)|V_i'|}$  and  $e^{-D(q + 3\beta ||q)|V' \setminus V_i'|}$  respectively. Here note that we use  $3\alpha$  instead of  $\alpha$ , to make calculations easier.
For the first case we have  $D(p + 3\alpha ||p)|V_i'| \geq \frac{9\alpha^2|V_i'|}{2(p + 3\alpha)}$ . If  $p \geq \alpha$  then  $D(p + \alpha ||p)|V_i'| \geq 9\alpha^2 |V_i'|8p$  which implies we get the same bound as above. If  $p < \alpha$  then  $D(p + \alpha ||p)|V_i'| \geq \frac{9\alpha^2|V_i'|}{8\alpha} \geq \alpha |V_i'|$ . Now we have  $\alpha = \frac{8\sqrt{p}\sqrt{n}\log n}{|V_i'|}$ . Since  $p = \Omega (\log n / n)$  we have  $\alpha |V_i'| \geq 8\log n$ . Combining we get that  $e^{-D(p + 2\alpha ||p)|V_i' |} \leq n^{-8}$ .
Next we analyze  $D(q + 3\beta ||q)|V' \setminus V_i'| \geq \frac{9\beta^2|V' \setminus V_i'|}{2(q + 3\beta)}$ . As before, if  $q \geq \beta$  we have  $D(q + 3\beta ||q)|V' \setminus V_i'| \geq \frac{9\beta^2|V_i|}{8q}$  and the result follows as before. Otherwise  $D(q + 3\beta ||q)|V' \setminus V_i'| \geq \frac{9\beta^2|V' \setminus V_i'|}{8\beta} \geq \beta |V' \setminus V_i'| \geq 8\sqrt{p}\sqrt{n} \log n \geq 8\log n$ , which completes the proof.
Then Lemma 2.1 can be proved as follows.
Proof of Lemma 2.1. We know that  $s_{\max} \geq \frac{2^{13} \cdot \sqrt{p(1 - q)} \sqrt{n} \log n}{p - q}$  from the problem definition. Let the corresponding cluster be  $V_i$ . Then a simple application of Hoeffding bounds gives us that with probability  $1 - n^{-8}$ ,  $0.51 \cdot s_{\max} \geq |V_i \cap W| \geq 0.49 \cdot s_{\max}$ .
Furthermore, we are interested in the regime where  $p \leq 3/4$  so  $\sqrt{1 - q} \geq 1/2$ .
Then if we sample  $u \in |V_i \cap Y_2|$ , Lemma B.1 states that with probability  $1 - n^{-8}$
$$
\begin{array}{l} | N _ {W} (u) - q | W | - (p - q) | V _ {i} \cap W | | \leq 4 8 \sqrt {p} \sqrt {n} \log n \leq \frac {(p - q) \cdot 9 6 \cdot \sqrt {p (1 - q)} \sqrt {n} \log n}{(p - q)} \\ \Rightarrow | N _ {W} (u) - q | W | - (p - q) | V _ {i} \cap W | | \leq \frac {(p - q) | V _ {i} \cap W |}{1 0 0} \\ \end{array}
$$
This implies with probability  $1 - n^{-8}$ ,  $q|W| + 1.01|V_i \cap W| \geq N_W(u) \geq q|W| + 0.99|V_i \cap W|$  which coupled with the fact  $0.49 \leq \frac{|V_i \cap W|}{|V_i|} \leq 0.51$  implies that if we are able to sample a vertex from the largest cluster, we get an estimate of  $s_{\max}$  as described.
Since  $|V_i \cap Y_2| \geq 100\sqrt{n} \log n$ , if we sample  $\sqrt{n} \log n$  vertices, we sample a vertex  $u$  from  $V_i$  with probability  $1 - n^{-8}$ . Now, for vertices belonging to smaller clusters, the same bounds apply, and this implies that as long as we are able to sample a vertex from the largest cluster, we get an estimate of  $s_{\max}$  between a factor of 0.48 and 0.52.
Proof of Lemma 2.2. These results follow directly from Vu [28] with some minor modifications. In their paper, Vu decomposes the matrix into  $Y$  and  $Z$ . In comparison, we decompose the matrix to  $U$  and  $W$  first, and then  $U$  is decomposed into  $Y$  and  $Z$ . Thus the size of  $Y$  and  $Z$  in our framework is roughly half as compared to [28]. However, since the size of the clusters we are concerned about are all larger than  $128 \cdot \sqrt{n} \log n$ , the results follow in the same way with a change of a factor of 2.
Now we describe the results and how we deviate from Vu's analysis to get our result. For the first part, in [28, page 132] it was proved that for any fixed  $\hat{\mathbf{u}}\in \hat{B}$
$$
\begin{array}{l} \left\| P _ {\widehat {A} _ {k}} (\widehat {\mathbf {u}}) - \mathbf {u} \right\| = \left\| P _ {\widehat {A} _ {k}} (\widehat {\mathbf {u}} - \mathbf {u}) + (P _ {\widehat {A} _ {k}} - I) \mathbf {u} \right\| \\ \leq \| P _ {\widehat {A} _ {k}} (e _ {u}) \| + \| \left(P _ {\widehat {A} _ {k}} - I\right) \boldsymbol {u} \| \leq \| P _ {\widehat {A} _ {k}} (e _ {u}) \| + \frac {1}{\sqrt {s _ {u}}} \left\| \left(P _ {\widehat {A} _ {k}} - I\right) A \right\|. \\ \end{array}
$$
Furthermore, it was proven (also in page 132 of [28]) that
$$
\left\| \right.\left( \right.P _ {\widehat {A} _ {k}} - I) A \left. \right\| = \left\| \right.\left( \right.P _ {\widehat {A} _ {k}} - I) \hat {A} - \left(P _ {\widehat {A} _ {k}} - I\right) E \left. \right\|.
$$
It was observed that  $\left\| (P_{\widehat{A}_k} - I)\hat{A}\right\| \leq \lambda_{k + 1}(\hat{A})\leq \lambda_{k + 1}(A) + \| E\| = \| E\|$  as  $A$  has rank at most  $k$ ; and  $\left\| (P_{\widehat{A}_k} - I)E\right\| \leq \| E\|$ . Then from Lemma 2.2 in [28] we have that with probability  $1 - \mathcal{O}(n^{-3})$ $\| E\| \leq C_2\sigma n^{1 / 2}$  for some constant  $C_2 > 0$ .
Next, we observe that for any  $k' \geq 1$ , it still holds that
$$
\begin{array}{l} \left\| P _ {\widehat {A} _ {k ^ {\prime}}} (\widehat {\mathbf {u}}) - \mathbf {u} \right\| = \left\| P _ {\widehat {A} _ {k ^ {\prime}}} (\widehat {\mathbf {u}} - \mathbf {u}) + \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) \mathbf {u} \right\| \\ \leq \| P _ {\widehat {A} _ {k ^ {\prime}}} (e _ {u}) \| + \| \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) \boldsymbol {u} \| \leq \| P _ {\widehat {A} _ {k ^ {\prime}}} (e _ {u}) \| + \| \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) A \| / \sqrt {s _ {u}}. \\ \end{array}
$$
$$
\begin{array}{l} \left\| \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) A \right\| = \left\| \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) \hat {A} - \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) E \right\| \\ \leq \left\| \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) \widehat {A} \right\| + \left\| \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) E \right\| \\ \leq \lambda_ {k ^ {\prime} + 1} (\hat {A}) + \| E \| \\ \leq \lambda_ {k ^ {\prime} + 1} (A) + 2 \| E \| \\ \end{array}
$$
Again, with probability at least  $1 - 1/n^3$ ,  $\|E\| \leq C_2 \sigma \sqrt{n}$ , which further implies that
$$
\left\| \left(P _ {\widehat {A} _ {k ^ {\prime}}} - I\right) A \right\| \leq 2 C _ {2} \sigma \sqrt {n} + \lambda_ {k ^ {\prime} + 1} (A).
$$
This is a simple but crucial step that removes our dependency on  $k$ , and allows us to treat all clusters of size  $o(\sqrt{n})$  as noise.
Now, we analyze the first term. From Lemma A.3 we have  $\operatorname{E}[\| P_{\widehat{A}_{k'}}(e_u) \|^2] \leq \sigma^2 k'$  for any  $u \in Y_2$ .
Then for any  $u$ , Markov's inequality gives us  $\mathbf{Pr}\left(\| P_{\widehat{A}_{k'}}(e_u)\| \geq \frac{\sigma\sqrt{k'}}{\varepsilon}\right) \leq \varepsilon$ .
Now let us consider any set  $V' \subset Y_2$  such that  $|V'| \geq 16\sqrt{n}\log n$ . For any  $u \in V'$  we define  $X_u$  to be the indicator random variable that gets 1 if  $\| P_{\widehat{A}_{k'}}(e_u)\| \leq \frac{\sigma\sqrt{k'}}{\varepsilon}$ , and 0 otherwise. Then  $\operatorname{E}[X_u] \geq 1 - \varepsilon$ . Now, since  $V' \subset Y_2$ , the variables  $X_u$  are independent of each other (as  $e_u$  are independent of each other). Then, using the fact that  $|V'| \geq \frac{4\log n}{\varepsilon^2}$ , the Chernoff bound gives us
$$
\Pr \left(\sum_ {u \in V ^ {\prime}} X _ {u} \leq (1 - \varepsilon) | V ^ {\prime} | - \varepsilon | V ^ {\prime} |\right) \leq e ^ {- \frac {2 \varepsilon^ {2} | V ^ {\prime} | ^ {2}}{| V ^ {\prime} |}} \leq n ^ {- 8}
$$
That is, with probability at least  $1 - n^{-8}$ , for at least  $(1 - 2\varepsilon)$  fraction of the points  $u\in V^{\prime}$ ,  $\| P_{\widehat{A}_{k^{\prime}}}(e_u)\| \leq \frac{1}{\varepsilon}\sigma \sqrt{k^{\prime}}$ .
Proof of Lemma 2.3. Let there be  $k$  many clusters  $V_{1},\ldots ,V_{k}$  in the SBM problem. Then we define  $a_{i} = |V_{i}\cap Z|$  and  $b_{i} = |V_{i}\cap Y_{1}|$ . Then we have that  $A$  is an  $n_1\times n_2$  matrix where  $n_1 = \sum_{i = 1}^k a_i$  and  $n_2 = \sum_{i = 1}^k b_i$ . The matrix  $A$  can be then written as a sum of  $k + 1$  many rank 1 matrices:
$$
A = \sum_ {i = 1} ^ {k} (p - q) M _ {i} + q M _ {0}
$$
Here  $M_0$  is the all 1 matrix, and  $M_i$  is a block matrix with 1's in a  $a_i \times b_i$  sized diagonal block. Since  $M_i$  is a  $a_i \times b_i$  block diagonal matrix of rank 1, with each entry being  $(p - q)$ , its singular value is  $(p - q)\sqrt{a_i b_i}$ . Now we define  $A_1 = \sum_{i=1}^{k} (p - q)M_i$ . As  $M_i$ 's are non-overlapping block diagonal matrices, the singular vectors of  $M_i$  are also singular vectors of  $A_1$ , with the same singular values.
Thus, the sum of singular values of  $A_{1}$  is  $(p - q)\sum_{i = 1}^{k}\sqrt{a_{i}b_{i}}\leq (p - q)\sqrt{n_{1}n_{2}}\leq \frac{(p - q)n}{2}$ , where the first inequality follows from the Cauchy-Schwarz inequality. Thus, for any  $t\geq 1$
$$
t \cdot \lambda_ {t} (A _ {1}) \leq \lambda_ {1} (A _ {1}) + \dots \lambda_ {t} (A _ {1}) \leq \frac {(p - q) n}{2},
$$
which gives  $\lambda_t(A_1) \leq \frac{(p - q)n}{2t}$ . Since  $M_0$  has rank 1,  $\lambda_2(q \cdot M_0) = 0$  and thus for  $t > 1$  we have
$$
\lambda_ {t + 1} (A) \leq \lambda_ {t} (A _ {1}) + \lambda_ {2} (q \cdot M _ {0}) \leq \frac {(p - q) n}{2 t} \leq \frac {(p - q) n}{t + 1},
$$
where the first inequality follows from the Weyl's inequality.
The guarantee of obtaining a plural set is a consequence of Lemma B.2.
Lemma B.2. Let  $k' = \frac{(p - q)\sqrt{n}}{\sqrt{p(1 - q)}}$  and  $\varepsilon = 0.002$ . Let  $u \in Y_2$  be a good center belonging to  $V_i$ , then
1. There is a set  $\hat{V_i'} \subset Y_2 \cap V_i$  such that  $|V_i'| \geq (1 - 2\varepsilon)|Y_2 \cap V_i|$  so that for all  $\mathbf{v} \in V_i'$ , we have  $\| P_{\widehat{A}_{k'}}(\mathbf{u} - \mathbf{v}) \| \leq \frac{L_\varepsilon}{30}$  with probability  $1 - \mathcal{O}(n^{-3})$ .
2. For any  $V_{j} \neq V_{i}$  which is a  $\varepsilon$ -large cluster, there is a set  $V_{j}^{\prime} \subset V_{j} \cap Y_{2}$  s.t  $|V_j^{\prime}| \geq (1 - 2\varepsilon)|V_j \cap Y_2|$  so that for all  $\pmb{v} \in V_{j}^{\prime}$  we have  $\| P_{\hat{A}_{k^{\prime}}}(\pmb{u} - \pmb{v}) \| \geq \frac{L_{\varepsilon}}{6}$  with probability  $1 - \mathcal{O}(n^{-3})$ .
Proof of Lemma B.2. First note that  $L_{\varepsilon} \geq \sqrt{2\varepsilon} \cdot 2^{11} \cdot \frac{(p - q) \cdot (p(1 - q))^{1/4} \cdot n^{1/4} \cdot \log^{1/2} n}{(p - q)^{1/2}} \geq 132,000 \cdot (p - q)^{1/2} \cdot (p(1 - q))^{1/4} \cdot n^{1/4} \cdot \sqrt{\log n}$ .
When  $\mathbf{u}$  and  $\mathbf{v}$  belong to the same cluster we have  $\| P_{\widehat{A}_{k'}}(\widehat{\mathbf{u}} - \widehat{\mathbf{v}}) \| \leq \| P_{\widehat{A}_{k'}}(\widehat{\mathbf{u}}) - \mathbf{u} \| + \| P_{\widehat{A}_{k'}}(\widehat{\mathbf{v}}) - \mathbf{v} \|$ .
Now, since  $\pmb{u}$  is a good center, from Lemma 2.2 we have
$$
\begin{array}{l} \left\| P _ {\widehat {A} _ {k ^ {\prime}}} (\widehat {\mathbf {u}}) - \boldsymbol {u} \right\| \leq \frac {1}{\varepsilon} \sigma \sqrt {k ^ {\prime}} + \frac {1}{\sqrt {s _ {u}}} \left(2 C _ {2} \sigma \sqrt {n} + \lambda_ {k ^ {\prime} + 1} (\hat {A})\right) \\ \leq \frac {1}{\varepsilon} \sqrt {p (1 - q)} \sqrt {k ^ {\prime}} + \frac {1}{\sqrt {s _ {u}}} \left(2 C _ {2} \sqrt {p (1 - q)} \sqrt {n} + \lambda_ {k ^ {\prime} + 1} (A) + \| E \|\right) \\ \leq \frac {1}{\varepsilon} \sqrt {p (1 - q)} \sqrt {k ^ {\prime}} + \frac {1}{\sqrt {s _ {u}}} \left(2 C _ {2} \sqrt {p (1 - q)} \sqrt {n} + \lambda_ {k ^ {\prime} + 1} (A) + C _ {2} \sqrt {p (1 - q)} \sqrt {n}\right) \\ \leq \frac {1}{\varepsilon} \sqrt {p (1 - q)} \sqrt {k ^ {\prime}} + \frac {1}{\sqrt {s _ {u}}} \left(3 C _ {2} \sqrt {p (1 - q)} \sqrt {n} + \frac {(p - q) n}{k ^ {\prime}}\right) [ \text {S u b s t i t u t i n g} \lambda_ {k ^ {\prime} + 1} (A) \text {f r o m L e m m a 2 . 3} ] \\ \leq \frac {1}{\varepsilon} (p (1 - q)) ^ {1 / 4} (p - q) ^ {1 / 2} n ^ {1 / 4} + 4 C _ {2} (\varepsilon) ^ {- 1 / 2} (p - q) ^ {1 / 2} (p (1 - q)) ^ {1 / 4} n ^ {1 / 4} \log^ {- 1 / 2} n \\ \leq \frac {2}{\varepsilon} (p (1 - q) ^ {1 / 4} (p - q) ^ {1 / 2} n ^ {1 / 4} \\ \leq \frac {1 0 , 0 0 0 L _ {\varepsilon}}{1 3 2 , 0 0 0 \log^ {1 / 2} n} \leq \frac {L _ {\varepsilon}}{6 0}, \quad \text {f o r} n \geq 6 4 \\ \end{array}
$$
Now from Lemma 2.2 we also know that at least  $(1 - 2\varepsilon)$  fraction of the vertices  $v\in V_i\cap Y_2$  are also "good centers" with probability  $1 - n^{-8}$ . For all such vertices  $\| P_{\widehat{A}_{k^{\prime}}}(\widehat{\mathbf{u}} -\widehat{\mathbf{v}})\| \leq 2\| P_{\widehat{A}_{k^{\prime}}}(\widehat{\mathbf{u}} -\mathbf{u})\| \leq \frac{L_{\varepsilon}}{30}$  with probability  $1 - n^{-3}$ .
On the other hand when they belong to different clusters we have
$$
\left\| P _ {\widehat {A} _ {k ^ {\prime}}} \left(\widehat {\mathbf {u}} - \widehat {\mathbf {v}}\right) \right\| \geq \left\| \boldsymbol {u} - \boldsymbol {v} \right\| - \left\| P _ {\widehat {A} _ {k ^ {\prime}}} \left(\widehat {\mathbf {u}} - \boldsymbol {u}\right) \right\| - \left\| P _ {\widehat {A} _ {k ^ {\prime}}} \left(\widehat {\mathbf {v}} - \boldsymbol {v}\right) \right\|.
$$
Since  $V_{j}$  is a  $\varepsilon$ -large cluster,  $|V_{i}| \geq 256\sqrt{n}\log n$  and thus  $|V_{j} \cap Y_{2}| \geq 16\sqrt{n}\log n$  with probability  $1 - \mathcal{O}(n^{-8})$ . In that case for at least  $1 - 2\varepsilon$  fraction of points  $\pmb{v} \in V_{j} \cap Y_{2}$  we have  $P_{\hat{A}_{k'}}(\widehat{\pmb{v}} - \pmb{v}) \leq \frac{L_{\varepsilon}}{60}$ .
Now  $\| \pmb{u} - \pmb{v} \| \geq (p - q) \sqrt{s_u + s_v} \geq \frac{\sqrt{2\varepsilon} \cdot (p - q) \sqrt{s_{\max}}}{6} \geq \frac{\sqrt{2\varepsilon} \cdot (p - q) \sqrt{s}}{\sqrt{0.52} \cdot 6} \geq \frac{L_\varepsilon}{5}$  with probability  $1 - n^{-8}$  from Lemma 2.1. Thus with probability  $1 - n^{-3}$  we get
$$
\| P _ {\hat {A} _ {k ^ {\prime}}} (\boldsymbol {u} - \boldsymbol {v}) \| \geq \| \boldsymbol {u} - \boldsymbol {v} \| - \| P _ {\hat {A} _ {k ^ {\prime}}} (\widehat {\boldsymbol {u}} - \widehat {\boldsymbol {v}}) \| - \| P _ {\hat {A} _ {k ^ {\prime}}} (\widehat {\boldsymbol {v}} - \boldsymbol {v}) \| \geq \frac {L _ {\varepsilon}}{5} - \frac {L _ {\varepsilon}}{6 0} - \frac {L _ {\varepsilon}}{6 0} \geq \frac {L _ {\varepsilon}}{6}
$$
for  $1 - 2\varepsilon$  fraction of points  $v \in Y_2 \cap V_j$  for any  $\varepsilon$ -large cluster  $V_j$ . This completes the proof.
Proof of Lemma 2.6. By Lemma 2.1, we have  $|V_i| \geq \frac{s_{\max}}{4} \geq \frac{\overline{s}}{2.1}$  with probability  $1 - n^{-8}$ . Then the following events happen.
1. Since  $u \in V_i \cap Y_2$  is a good center, for  $1 - 2\varepsilon$  fraction of points  $v$  in  $V_i \cap Y_2$ ,  $\| \mathbf{p_u} - \mathbf{p_v} \| \leq L_{\varepsilon} / 30$  with probability  $1 - \mathcal{O}(n^{-3})$  as per Lemma B.2. All such points are selected to  $S$ . Furthermore,  $|V_i \cap Y_2|$  is lower bounded by  $|V_i| / 9$  with probability  $1 - n^{-8}$ . Therefore, with probability  $1 - \mathcal{O}(n^{-3})$ , we have
$$
\left| V _ {i} \cap S \right| \geq (1 - 2 \varepsilon) \left| V _ {i} \cap Y _ {2} \right| \geq (1 - 2 \varepsilon) \left| V _ {i} \right| / 9 \geq (1 - 2 \varepsilon) \bar {s} / 2 0 \geq \bar {s} / 2 1.
$$
2. For other clusters  $V_{\ell}$ , if  $|V_{\ell}| \geq \varepsilon \cdot s_{\max}$ , we have  $\|\mathbf{p}_{\mathbf{u}} - \mathbf{p}_{\mathbf{v}}\| \leq \frac{L_{\varepsilon}}{6}$  for only  $2\varepsilon$  fraction of points  $v$  in  $V_{\ell} \cap Y_{2}$  from Lemma B.2. Thus  $|S \cap V_{\ell}| \leq 2\varepsilon |V_{\ell} \cap Y_{2}|$ . On the other hand  $|V_{\ell} \cap Y_{2}| \leq |V_{\ell}| / 6$ . Thus, with probability  $1 - n^{-8}$  we have
$$
| S \cap V _ {\ell} | \leq 2 \varepsilon | V _ {\ell} | / 6 \leq \frac {2 \cdot \varepsilon \cdot s _ {\max}}{6} \leq \frac {2 \cdot \varepsilon \cdot \bar {s}}{0 . 4 8 \cdot 6} \leq 0. 7 \varepsilon \cdot \bar {s}
$$
3. Otherwise, if  $V_{\ell}$  is such that  $\varepsilon \cdot s_{\max} \geq |V_{\ell}| \geq \frac{\varepsilon}{2} \cdot s_{\max}$ , then  $|V_{\ell} \cap Y_2| \leq |V_{\ell}| / 6$  with probability  $1 - n^{-8}$ . Then  $|S \cap V_{\ell}| \leq \varepsilon s_{\max} / 6 \leq \varepsilon \overline{s}$ .
4. Otherwise, if  $|V_{\ell}| \leq \frac{\varepsilon}{2} \cdot s_{\max}$  then  $|S \cap V_{\ell}| \leq |V_{\ell}| \leq \frac{\varepsilon}{2} \cdot s_{\max} \leq \frac{\varepsilon \overline{s}}{2 \cdot 0.48} \leq \frac{\varepsilon \overline{s}}{0.96} \leq 1.05 \cdot \varepsilon \cdot \overline{s}$ .
Now, note that for any  $V_{\ell}$  with  $\ell \neq i$ , it holds with probability  $1 - \mathcal{O}(n^{-3})$  that  $|S \cap V_{\ell}| \leq 1.05 \cdot \varepsilon \cdot \overline{s} \leq (21 \cdot 1.05 \cdot \varepsilon) \cdot \frac{\overline{s}}{21} \leq 0.05 \cdot \frac{\overline{s}}{21}$ .
Proof of Lemma 2.7. We first show that if  $S$  is a  $V_{i}$ -plural set with  $V_{i} \geq s_{\max} / 4$ , then  $T_{1} = V_{i} \cap W$  where  $T_{1}$  is the outcome of IDENTIFYCLUSTER(S, W,  $\overline{s}$ ). Since  $S$  is a  $V_{i}$  plural set, for any vertex  $v \in W \cap V_{i}$ , from Lemma B.1 we have that with probability  $1 - \mathcal{O}(n^{-3})$ ,
$$
\begin{array}{l} N _ {S} (v) \geq q | S | + (p - q) \left| V _ {i} \cap S \right| - 4 8 \sqrt {p} \sqrt {n} \log n \\ \Rightarrow N _ {S} (v) \geq q | S | + (p - q) \cdot \frac {\overline {{s}}}{2 1} - \frac {4 8}{\sqrt {1 - q}} \cdot \frac {(p - q) \cdot \sqrt {p (1 - q)} \sqrt {n} \log n}{(p - q)} \\ \Rightarrow N _ {S} (v) \geq q | S | + (p - q) \cdot \frac {\bar {s}}{2 1} - (p - q) \cdot \frac {9 6 \cdot \sqrt {p (1 - q)} \sqrt {n} \log n}{(p - q)} \\ \Longrightarrow N _ {S} (v) \geq q | S | + (p - q) \cdot \frac {\bar {s}}{2 1} - (p - q) \cdot \frac {9 6 \cdot \bar {s}}{2 ^ {1 3}} \\ \Longrightarrow N _ {S} (v) \geq q | S | + (p - q) \cdot \frac {\overline {{s}}}{2 8} \\ \end{array}
$$
Now, let us consider the case when  $v \in V_j \cap W$  where  $j \neq i$ . Then we know from Lemma 2.6  $|S \cap V_j| \leq 1.05\varepsilon \overline{s} \leq \frac{\overline{s}}{210}$ . Then using Lemma B.1 we have that with probability  $1 - \mathcal{O}(n^{-3})$
$$
\begin{array}{l} N _ {S} (v) \leq q | S | + (p - q) | V _ {j} \cap S | + 2 4 \sqrt {p} \sqrt {n} \log n \\ \Longrightarrow N _ {S} (v) \leq q | S | + (p - q) \frac {\overline {{s}}}{2 1 0} + (p - q) \frac {2 4 \overline {{s}}}{2 ^ {1 3}} \\ \Longrightarrow N _ {S} (v) \leq q | S | + (p - q) \frac {\bar {s}}{1 2 8} \\ \end{array}
$$
Now note that in the IDENTIFYCLUSTER(S, W,  $\overline{s}$ ) algorithm, we select all vertices from  $W$  that have  $q|S| + (p - q) \cdot \frac{\overline{s}}{56}$  neighbors in  $S$ . Thus, the above analysis implies with probability  $1 - \mathcal{O}(n^{-3})$ $T_{1} = \text{IDENTIFYCLUSTER}(S, W, \overline{s}) = V_{i} \cap W$ . Furthermore, since  $|V_{i}| \geq s_{\max} / 4$ , we have  $|V_{i} \cap W| \geq \frac{s_{\max}}{2.2} \geq \frac{\overline{s}0.48}{2.2} \geq \overline{s} / 6$ .
We then use  $T_{1}$  as a plural set to recover  $V_{i} \cap U$  so that we are able to recover all the vertices of  $V_{i}$ , but now  $T_{1}$  and  $U$  are not completely independent and thus we cannot proceed simply as before.
We overcome this by an union bound based argument. Let's consider  $T_{i}^{\prime} = V_{i}\cap W$  for any  $i$  such that  $T^{\prime}\geq \overline{s} /6$ . Then we have the following facts.
1. Let  $u \in U \cap V_i$ . Then  $\operatorname{E}[N_{T_i'}(u)] = p|T'|$ . Then Lemma B.1 shows that  $\mathbf{Pr}(N_T(u) \leq q|T'| + 0.99(p - q)|T'|) \leq n^{-10}$ .
2. Similarly, let  $u \in U \cap V_j$ . Then  $\mathbf{Pr}(N_{T_i'}(u) \geq q|T'| + 0.01(p - q)|T'|) \leq n^{-10}$ .
If either of this is true for a vertex  $u \in U$  then we call it a bad vertex w.r.t  $T_{i}^{\prime}$ . Then a union bound over all  $V_{i}$  and all  $u \in U$  gives us that no vertex  $u \in U$  is bad w.r.t any  $T_{i}^{\prime}$  with probability  $1 - n^{-8}$ .
Then we can make this argument for  $T_{i}^{\prime} = T_{1}$ . Since  $|V_{i}| > s_{\max} / 4$ , we have  $|V_{i}\cap W|\geq V_{i} / 3$  with probability  $1 - n^{-8}$ . Then with probability  $1 - n^{-8}$  no vertex  $u\in U$  is bad w.r.t  $T_{1}$ .