SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Then applying Lemma B.1 to  $T_{1}$  w.r.t vertices in  $U$  we get, with probability  $1 - \mathcal{O}(n^{-3})$
1. If  $v \in V_i \cap U$ , then  $N_{T_1}(v) \geq q|T_1| + (p - q)|T_1| - (p - q)\frac{T_1}{96}$ .
2. If  $v \in V_j \cap U$ , then  $N_{T_1}(v) \leq q|T_1| + (p - q)\frac{T_1}{96}$ .
Thus  $\mathsf{IDENTIFYCLUSTER}(T_1,U,\overline{s})$  only selects the set of vertices  $T_{2}$  in  $V_{i}\cap U$ . Then taking the union of  $T_{1}$  and  $T_{2}$  gives us  $V_{i}$ .
Proof of Lemma 2.8. Since  $|V_i| > 256\sqrt{n}\log n$ , and every vertex of  $V_i$  will be assigned to  $W$  with probability  $1/2$ , we have that  $|V_i \cap W| \geq \frac{|V_i|}{2.5} \geq \frac{s_{\max}}{10} \geq \frac{\overline{s}}{0.52 \cdot 10} \geq \frac{\overline{s}}{6}$  with probability  $1 - n^{-8}$ . Furthermore if  $|V_i| \geq \frac{s_{\max}}{4}$ , then Lemma 2.7 shows  $T_1 = V_i \cap W$  and  $|T_1| \geq \frac{\overline{s}}{6}$ .
Furthermore, for any vertex  $u \in T_1 \cap \{v\}$ , we can calculate  $N_{T_1}(u)$  in the following way.
We have  $\operatorname{E}[N_{T_1}(u)] = p|T_1|$ . Then a simple application of Lemma B.1 gives us that with probability  $1 - n^{-8}$ ,  $N_{T_1}(u) \geq p|T_1| - (p - q)\frac{T_1}{96} \geq (0.9p + 0.1q)|T_1|$ .
Similarly, since  $T_{1} = V_{i}\cap W$  for any vertex  $u\in W\cap T_1$ , we have  $\operatorname {E}[N_{T_1}(u)] = q|T_1|$  and Lemma B.1 implies that with probability  $1 - n^{-8}$
$$
N _ {T _ {1}} (u) \leq q | T _ {1} | + (p - q) \frac {| T _ {1} |}{9 6} \leq \frac {p | T _ {1} |}{3} + \frac {2 q | T _ {1} |}{3} \leq (0. 3 3 p + 0. 6 7 q) | T _ {1} | <   (0. 9 p + 0. 1 q) | T _ {1} |.
$$
Proof of Corollary 2.9. If  $T_{1}$  is a pure subset of some  $V_{i}$ , such that  $|V_{i}| \leq s_{\max} / 7$ , then with probability  $1 - \mathcal{O}(n^{-8})$ ,  $|Y_{2} \cap V_{i}| \leq \overline{s} / 6$ . If  $|T_{1}| < \frac{\overline{s}}{6}$ , the first condition is satisfied.
Otherwise if  $|T_1| \geq \frac{\overline{s}}{6}$  and  $T_1$  is not a pure set, there exists  $V_j$  such that  $|T_1 \cap V_j| \leq \frac{|T_1|}{2}$ . In that case for any vertex  $v \in V_j \cap T_1$  we have  $\operatorname{E}[N_{T_1}(v)] \leq q |T_1| + (p - q) \frac{T_1}{2}$  and Lemma B.1 implies that with probability  $1 - n^{-8}$ ,
$$
N _ {T _ {1}} (u) \leq q | T _ {1} | + (p - q) \frac {| T _ {1} |}{2} + (p - q) \frac {| T _ {1} |}{9 6} \leq (0. 5 + 1 / 9 6) p | T _ {1} | + (0. 5 - 1 / 9 6) q | T _ {1} | <   (0. 9 p + 0. 1 q) | T _ {1} |.
$$
Finally if  $T_{1} \subset V_{i}$  is a large pure set and  $T_{1} \neq V_{i} \cap W$ , then for a vertex  $v \in V_{i} \cap (W \setminus T_{1})$  we have  $N_{T_{1}}(v) \geq (0.9p + 0.1q)|T_{1}|$ .
C An improved algorithm in the balanced case
Our algorithm is built upon [28] and [25]. However, even in the balanced case, our algorithm improves a result of [28] on partially recovering clusters in the SBM. More precisely, we can use Theorem 1.2 to prove the following theorem.
Theorem C.1. Let  $G = (V, E)$  be sampled from  $\mathrm{SBM}(n, k, p, q)$  for  $\sigma^2 = \Omega(\log n / n)$  where size of each cluster is  $\Omega(n / k)$ . Then there exists a polynomial time algorithm that exactly recovers all clusters if  $(p - q)\sqrt{\frac{n}{k}} > C'\sigma\sqrt{k}\log n$  for some constant  $C'$ .
In [28] (see Lemma 1.4 therein),  $\mathrm{Vu}$  gave an algorithm that partially recovers all the clusters in the sense that with probability at least  $1 - \varepsilon$ , each cluster output by the algorithm contains at  $1 - \varepsilon$  fraction of any one underlying communities, for any constant  $\varepsilon > 0$ . For the balanced case, his result holds under the assumption that  $\sigma^2 > C \log n / n$ , and  $(p - q) \sqrt{\frac{n}{k}} > C \sigma \sqrt{k}$ . In comparison, we obtain a full recovery of all the clusters under  $\mathrm{Vu}$ 's partial recovery assumptions at the cost of an extra  $\log n$  factor in the tradeoff of parameters.
Proof of Theorem C.1. We have  $(p - q)\sqrt{n / k} > C'\sigma \sqrt{k}\log n$ . Let  $C' = 2C$ . Since  $p \leq 3/4$ , we have  $1 - p \geq 1/4$  and then  $\sigma \geq \frac{\sqrt{p(1 - q)}}{2}$ . Thus  $(p - q)\sqrt{n / k} > C\sqrt{p(1 - q)}\sqrt{k}\log n$ . This implies  $k < \frac{(p - q)\sqrt{n}}{C\sqrt{p(1 - q)}\log n}$  and  $n / k > \frac{C\cdot\sqrt{p(1 - q)}\sqrt{n}\cdot\log n}{p - q}$ . That is the size of each cluster is at least  $s^*$ . Then we can run Algorithm 1 to recover one such cluster. Now, since the size of each cluster is same, we can run this iteratively  $k$  times, recovering a cluster at each round with probability  $1 - \mathcal{O}(n^{-2})$ . Using union bound we get that we are able to recover all clusters with probability  $1 - \mathcal{O}(kn^{-2}) = 1 - \mathcal{O}(n^{-1})$ .
D Lower bounds
First, we show that our algorithm is optimal up to logarithmic factors when  $p$  and  $q$  are constant. To do so, we make use of the well-known planted clique conjecture.
Conjecture D.1 (Planted clique hardness). Given an Erdős-Rényi random graph  $G(n,q)$  with  $q = 1/2$ , if we plant in  $G(n,q)$  a clique of size  $t$  where  $t \in [3 \cdot \log n, o(\sqrt{n})]$ , then there exists no polynomial time algorithm to recover the largest clique in this planted model.
Under the planted clique conjecture, we note that there is no polynomial time algorithm for the SBM problem that recovers clusters of size  $o(\sqrt{n})$  irrespective of the number  $k$  of clusters present in the graph, for any constants  $p$  and  $q$ . This can be seen by defining the partition of  $V$  as  $V = \cup_{i=1}^{k} V_i$ , where  $V_1$  is a clique of size  $t = o(\sqrt{n})$ , and  $V_2, \dots, V_k$  are singleton vertices,  $k = n - t$ . Finally, let  $p = 1$ ,  $q = \frac{1}{2}$ . Then an algorithm for finding a cluster of size  $o(\sqrt{n})$  in a graph  $G$  that is sampled from the SBM with the above partition solves the planted clique problem.
Thus, the dependency of our algorithm in Theorem 1.2 on  $n$  is optimal under the planted clique conjecture up to logarithmic factors.
The following result was given in [23], we give a proof here for the sake of completeness.
Theorem D.2 ([23]). Let  $A$  be a polynomial time algorithm in the faulty oracle model with parameters  $n, k, \delta$ . Suppose that  $A$  finds a cluster of size  $t$  irrespective of the value of  $k$ . Then under the planted clique conjecture, it holds that  $t = \Omega(\sqrt{n})$ .
Proof. Let  $G$  be a graph generated from the planted clique problem with parameter  $t$ . Note that each potential edge in the size- $t$  clique, say  $K$ , appears with probability 1, and each of the remaining potential edges appear with probability  $\frac{1}{2}$ . Now we delete each edge in  $G$  with probability  $\frac{1}{3}$ . Then the resulting graph can be viewed as an instance generated from the faulty oracle model with parameters  $n$ ,  $k = n - t + 1$  and  $\delta = \frac{1}{3}$ : there are  $k$  clusters, one being  $H$ , and  $n - t$  clusters being singleton vertices. Furthermore, each intra-cluster edge is removed with probability  $\frac{1}{3}$  and each inter-cluster is added with probability  $\frac{1}{2} \cdot (1 - \frac{1}{3}) = \frac{1}{3}$ . If there is a polynomial time algorithm that recovers the cluster  $H$ , no matter how many queries it performs, then it also solves the planted clique problem with clique size  $t$ . Under the planted clique conjecture,  $t = \Omega(\sqrt{n})$ .
E High-level ideas of the algorithm for the faulty oracle
Discussion about the previous algorithm in the faulty oracle model One crucial limitation of all the previous polynomial-time algorithms that make sublinear number of queries is that they cannot recover large clusters, if there are at least  $\tilde{\Omega} (n^{2 / 5})$  small clusters. The reason is that the query complexities of all these algorithms are at least  $\Omega (k^5)$ , and if there are  $\tilde{\Omega} (n^{2 / 5})$  small clusters, then  $k = \tilde{\Omega} (n^{2 / 5})$ , which further implies that these polynomial time algorithms have to make  $\Omega (k^5) = \Omega (n^2)$  queries.
Main ideas of our algorithm Now we apply our algorithm in the SBM to the faulty oracle model. Consider the faulty oracle model with and parameters  $n, k, \delta$ . Assume that the oracle  $\mathcal{O}$  outputs  $+$  to indicate the queried two vertices belong to the same cluster, and  $-$  otherwise.
Observe that if we make queries on all pairs  $u, v \in V$ , then the graph  $G$  that is obtained by adding all + edges answered by the oracle  $\mathcal{O}$  is exactly the graph that is generated from the  $\mathrm{SBM}(n, k, p, q)$  with parameters  $n, k, p = \frac{1}{2} + \frac{\delta}{2}$  and  $q = \frac{1}{2} - \frac{\delta}{2}$ . However, the goal is to recover the clusters by making sublinear number of queries, i.e., without seeing the whole graph.
We now describe our algorithm NOISYCLUSTERING (i.e., Algorithm 5) for clustering with a faulty oracle. Let  $V$  be the items which contains  $k$  latent clusters  $V_{1},\ldots ,V_{k}$  and  $\mathcal{O}$  be the faulty oracle. Following the idea of [27], we first sample a subset  $T\subseteq V$  of appropriate size and query  $\mathcal{O}(u,v)$  for all pairs  $u,v\in T$ . Then apply our SBM clustering algorithm (i.e. Algorithm 1 CLUSTER) on the graph induced by  $T$  to obtain clusters  $X_{1},\ldots ,X_{t}$  for some  $t\leq k$ . We can show that each of these sets is a subcluster of some large cluster  $V_{i}$ . Then we can use a majority voting to find all other vertices that belong to  $X_{i}$ , for each  $i\leq t$ . That is, for each  $X_{i}$  and  $v\in V$ , we check if the number of neighbors of  $v$  in  $X_{i}$  is at least  $\frac{|X_i|}{2}$ . In this way, we can identify all the large clusters  $V_{i}$  corresponding to  $X_{i}$ ,  $1\leq i\leq t$ . Furthermore, we note that we can choose a small subset of  $X_{i}$  of size  $O(\frac{\log n}{\delta^2})$  for majority voting to reduce query complexity. Then we can remove all the vertices in  $V_{i}$ 's and remove all the edges incident to them from both  $V$  and  $T$  and then we can use the remaining subsets  $T$  and  $V$  and corresponding subgraphs to find the next sets of large clusters. The algorithm NOISYCLUSTERING then recursively find all the large clusters until we reach a point where the recovery condition on the current graph no longer holds.
F The algorithm in faulty oracle model
Now we turn to the faulty oracle model and give the corresponding algorithm Algorithm 5.
To analyze the algorithm NOISYCLUSTERING (i.e., Algorithm 5), we first describe two results.
Lemma F.1. Let  $|V| = n$  and  $V_{i} \subset V: |V_{i}| = s \geq \frac{C\sqrt{n} \cdot \log^{2}n}{\delta}$  for some constant  $C > 1$ . If a set  $T \subset V$  of size  $\frac{16C^2n^2\log n}{\delta^2s^2}$  is sampled randomly, then with probability  $1 - n^{-8}$ , we have  $|T \cap V_{i}| \geq \frac{C\sqrt{|T|}\log|T|}{4\delta} \geq \frac{C\log n}{\delta^{2}}$ .
Algorithm 5 NOISYCLUSTERING  $(V,\delta ,s)$  : recover all clusters of size more than  $s\geq s^*$
1:  $V^{\prime}\gets V;t^{\prime}\gets 0$    
2: Randomly sample a subset  $T\subset V^{\prime}$  of size  $|T| = \frac{C^2n^2\log^2n}{s^2\delta^2}$    
3: Query all pairs  $u,v\in T$  and let  $G[T]$  be graph on vertex set  $T$  with only positive edges from the query answers   
4: for each  $\ell$  from 1 to  $\lfloor n / s\rfloor$  do   
5: Apply CLUSTER  $(G[T],\frac{1}{2} +\delta ,\frac{1}{2} -\delta)$  to obtain a cluster  $T_{\ell}$    
6: if  $T_{\ell} = \emptyset$  then   
7: continue   
8: else   
9:  $t^\prime \gets t^\prime +1$    
10: Find an arbitrary subset  $T_{\ell}^{\prime}\subseteq T_{\ell}$  of size  $\frac{4\log n}{\delta^2}$    
11:  $C_{t^{\prime}}^{\prime}\gets \{v\in V^{\prime}\backslash T:N_{T_{\ell}^{\prime}}(v)\geq |T_{\ell}^{\prime}| / 2\}$    
12:  $C_{t^{\prime}}\gets T_{\ell}\cap C_{t^{\prime}}^{\prime}$    
13:  $T\gets T\setminus T_{\ell}$    
14:  $V^{\prime}\gets V\setminus C_{t^{\prime}}$    
15: Return  $C_1,\dots ,C_{t'}$
Proof. We use Hoeffding bound to obtain these bounds. We have  $|T| \geq \frac{16C^2n^2\log^2n}{\delta^2s^2} \geq 16\log^2 n$ . For every vertex  $u \in T$ , we define  $X_{u}$  as the indicator random variable which is 1 if  $u \in V_i$ .
Then  $\operatorname{E}[X_u] = |V_i| / |V|$ . Thus applying Hoeffding bound we get
$$
\Pr \left(\sum_ {u \in T} X _ {u} \leq \frac {0 . 5 \cdot | T | | V _ {i} |}{| V |}\right) \leq e ^ {- 8 \log n} \leq n ^ {- 8}
$$
Now, substituting value of  $|T|$  we get  $\frac{0.5 \cdot |T||V_i|}{|V|} \geq \frac{8 \cdot C^2 \cdot n^2 \log^2 n \cdot s}{s^2 \cdot \delta^2 \cdot n} \geq \frac{4C \cdot n \cdot \log n}{s \cdot \delta} \cdot \frac{C \cdot \log n}{\delta} \geq \frac{C \cdot \sqrt{|T|} \cdot \log n}{\delta} \geq \frac{C \sqrt{|T|} \cdot \log |T|}{\delta}$ . Furthermore, the last equation shows  $\frac{0.5 \cdot |T||V_i|}{|V|} \geq \frac{C \sqrt{|T|} \cdot \log |T|}{\delta} \geq \frac{Cn \log n}{s \cdot \delta \cdot \delta} \geq \frac{C \log n}{\delta^2}$ . Now the proof follows by noting that  $|T \cap V_i| = \sum_{u \in T} X_u$ .
Lemma F.2. Let  $V$  be partitioned into two sets  $U$  and  $W$ , where each vertex  $v \in V$  is independently assigned to either set with equal probability. Let  $S \subset V_i \cap U$  be a set such that  $|S| \geq \frac{4\log n}{\delta^2}$ . Then with probability  $1 - \mathcal{O}(n^{-8})$ , we have  $N_S(u) \geq \frac{|S|}{2}$  for all  $u \in V_i \cap W$ , and  $N_S(u) < \frac{|S|}{2}$  for all  $u \in V_j \cap W$  for any  $j \neq i$ .
Proof. Let  $u \in V_i \cap W$ . Then  $\operatorname{E}[N_S(u)] = (0.5 + \delta) \cdot |S|$ . Then
$\mathbf{Pr}(N_S(u)\leq (0.5 + \delta)\cdot |S| - \delta |S|) = \mathbf{Pr}(N_S(u)\leq 0.5|S|)\leq e^{-2\delta^2 |S|^2 /|S|}\leq e^{-2\delta^2 |S|}\leq e^{-8\log n}$
The last inequality holds  $|S| \geq 4\log n / \delta^2$ . Thus if  $u \in V_i \cap W$  then  $N_S(u) \geq 0.5|S|$  with probability  $1 - n^{-8}$ .
Similarly, if  $u \notin V_i$ , then with probability  $1 - n^{-8}$  we have  $N_S(u) \leq 0.5|S|$ .
F.1 Proof of Theorem 1.6
Given  $s$ , first we randomly sample  $n' = \frac{C^2n^2\log^2n}{s^2\delta^2}$  many vertices from  $V$ , and denote this set as  $T$ .
Then Lemma F.2 proves that for any cluster  $V_{i}:|V_{i}|\geq s^{*}$ , we have  $|T_{i}| = |T\cap V_{i}|\geq \frac{C\sqrt{n^{\prime}}\log n^{\prime}}{\delta}$  with probability  $1 - n^{-8}$ . For any underlying cluster  $V_{i}$ , we denote  $T_{i} = T\cap V_{i}$ .
Next we query all the pair of edges for vertices in  $T$ , which amounts  $\mathcal{O}\left(\frac{n^4\log^2n}{\delta^4s^4}\right)$  queries. The resultant graph  $G'$  is an SBM graph on  $n'$  vertices with  $p = 0.5 + \delta$  and  $q = 0, .5 - \delta$ .
Thus, if we run Algorithm 1 with parameters  $G'$ ,  $0.5 + \delta$ ,  $0.5 - \delta$ , then Theorem 1.2 implies that we recover a cluster  $T_i$  such that  $|T_i| \geq \frac{C n' \log n'}{\delta}$  with probability  $1 - n^{-2}$ .
Once we get such a set  $T_{i}$ , we can take  $4\log n / \delta^{2}$  many vertices from it, calling it a set  $S$ . Then for every vertex  $v\in V\setminus T$ , we obtain  $N_{S}(v)$ , which requires  $|S|$  many queries, and select all vertices such that  $N_{S}(u)\geq 0.5|S|$ . Lemma F.1 shows that we recover  $V_{i}\cap (V\setminus T)$  with probability  $1 - n^{-8}$ , together recovering  $V_{i}$ . Thus this step requires  $4n\log n / \delta^2$  queries for each iteration.
Once we have recovered  $V_{i}$ , we can then remove  $T_{i}$  from  $T$  and run Algorithm 1 again on the residual graph, followed by the sample-and recovery step of Line 10. Note that once we remove a recovered cluster, all sets  $T_{j}$  that satisfied the recovery requirement of Theorem 1.2 in the graph  $G'$  defined on  $T$ , also satisfies it on the graph  $G''$  defined on  $T \setminus T_{i}$ , and we do not need to sample any more edges.
Finally, there are at most  $\delta^2\sqrt{T}$  many clusters  $T_i \in T$  such that  $|T_i| \geq \sqrt{|T|}\log |T| / \delta^2$ . Here we have  $\delta^2\sqrt{T} = \frac{Cn\log n}{s}$ . This upper bounds the number of iterations and thus the number of times the voting system on Line 10 is applied.
Thus the query complexity is  $\mathcal{O}\left(\frac{n^4\log^2n}{\delta^4s^4} +\frac{n\log n}{s}\cdot \frac{4n\log n}{\delta^2}\right) = \mathcal{O}\left(\frac{n^4\log^2n}{s^4\cdot\delta^4} +\frac{n^2\log^2n}{s\cdot\delta^2}\right)$ . This finishes the proof of Theorem 1.6.