Title: Recovering Unbalanced Communities in the Stochastic Block Model with Application to Clustering with a Faulty Oracle
Venue: NeurIPS 2023
Accepted: yes
Keywords: SBM

Unbalanced SBM

Spectral algorithms

Small cluster barrier
Abstract: The stochastic block model (SBM) is a fundamental model for studying graph clustering or community detection in networks. It has received great attention in the last decade and the balanced case, i.e., assuming all clusters have large size, has been well studied. 
However, our understanding of SBM with unbalanced communities (arguably, more relevant in practice) is still limited. In this paper, we provide a simple SVD-based algorithm for recovering the communities in the SBM with communities of varying sizes.
We improve upon a result of Ailon, Chen and Xu [ICML 2013; JMLR 2015] by removing the assumption that there is a large interval such that the sizes of clusters do not fall in, and also remove the dependency of the size of the recoverable clusters on the number of underlying clusters. We further complement our theoretical improvements with experimental comparisons.
Under the planted clique conjecture, the size of the clusters that can be recovered by our algorithm is nearly optimal (up to poly-logarithmic factors) when the probability parameters are constant. 

As a byproduct, we obtain an efficient clustering algorithm with sublinear query complexity in a faulty oracle model, which is capable of detecting all clusters larger than $\tilde{\Omega}({\sqrt{n}})$, even in the presence of $\Omega(n)$ small clusters in the graph. In contrast, previous efficient algorithms that use a sublinear number of queries are incapable of recovering any large clusters if there are more than $\tilde{\Omega}(n^{2/5})$ small clusters.

Recovering Unbalanced Communities in the Stochastic Block Model with Application to Clustering with a Faulty Oracle ∗

Abstract

The stochastic block model (SBM) is a fundamental model for studying graph clustering or community detection in networks. It has received great attention in the last decade and the balanced case, i.e., assuming all clusters have large size, has been well studied. However, our understanding of SBM with unbalanced communities (arguably, more relevant in practice) is still limited. In this paper, we provide a simple SVD-based algorithm for recovering the communities in the SBM with communities of varying sizes. We improve upon a result of Ailon, Chen and Xu [ICML 2013; JMLR 2015] by removing the assumption that there is a large interval such that the sizes of clusters do not fall in, and also remove the dependency of the size of the recoverable clusters on the number of underlying clusters. We further complement our theoretical improvements with experimental comparisons. Under the planted clique conjecture, the size of the clusters that can be recovered by our algorithm is nearly optimal (up to poly-logarithmic factors) when the probability parameters are constant.

As a byproduct, we obtain an efficient clustering algorithm with sublinear query complexity in a faulty oracle model, which is capable of detecting all clusters larger than ˜ Ω( √ n ) , even in the presence of Ω( n ) small clusters in the graph. In contrast, previous efficient algorithms that use a sublinear number of queries are incapable of recovering any large clusters if there are more than ˜ Ω( n 2 / 5 ) small clusters.

1 Introduction

Graph clustering (or community detection) is a fundamental problem in computer science and has wide applications in many domains, including biology, social science, and physics. Among others, the stochastic block model (SBM) is one of the most basic models for studying graph clustering, offering both a theoretical arena for rigorously analyzing the performance of different types of clustering algorithms, and synthetic benchmarks for evaluating these algorithms in practice. Since the 1980s (e.g., [19, 8, 15, 7]), there has been much progress towards the understanding of the statistical and computational tradeoffs for community detection in SBM with various parameter regimes. We refer to the recent survey [1] for a list of such results.

In this paper, we focus on a very basic version of the stochastic block model.

glyph[negationslash]

Definition 1.1 (The SBM( n, k, p, q ) model) . In this model, given an n -vertex set V with a hidden partition V = ∪ k i =1 V i such that V i ∩ V j = ∅ for all i = j , we say a graph G = ( V, E ) is sampled from SBM( n, k, p, q ) , if for all pairs of vertices v i , v j ∈ V , (1) an edge ( v i , v j ) is added independently with probability p , if v i , v j ∈ V glyph[lscript] for some glyph[lscript] ; (2) an edge ( v i , v j ) is added independently with probability q , otherwise.

We are interested in the problem of fully recovering all or some of the clusters, given a graph G that is sampled from SBM( n, k, p, q ) . A cluster V i is said to be fully recovered if the algorithm outputs a set S that is exactly V i . Most of the previous algorithms on the full recovery of SBM either just work for the nearly balanced case (i.e., each cluster has size Ω( n k ) ) when k is small, say k = o (log n ) (see e.g. [2]), or only work under the following assumption:

All of the latent clusters are sufficiently large 4 , i.e., for each j , | V j | = ˜ Ω( √ n ) (see e.g., [25, 6, 10, 9, 1, 28, 11]).

From a practical perspective, many real-world graphs may have many communities of different sizes, that is, large and small clusters co-exist in these graphs. This motivates us to investigate how to recover the communities in SBM if the latent communities have very different sizes. In particular, we are interested in efficiently recovering all the large clusters in the presence of small clusters. However, such a task can be quite difficult, as those small clusters may be confused with noisy edges. Indeed, most previous algorithms try to find all the k -clusters in one shot, which always computes some structures/information of the graph that are sensitive to noise (and small clusters). For example, the classical SVD-based algorithms (e.g., [25, 28]) first compute the first k singular vectors of some matrix associated with the graph and then use these k vectors to find clusters. Such singular vectors are sensitive to edge insertions or deletions (e.g. [13]). In general, this difficulty was termed by Ailon et al. [3] as ' small cluster barrier ' for graph clustering.

To overcome such a barrier, Ailon et al. [3, 4] proposed an algorithm that recovers all large latent clusters in the presence of small clusters under the following assumptions (see [4]),

there exists a large cluster, say of size at least Υ := Θ ( max { √ p (1 -q ) n p -q , k log n ( p -q ) 2 }) .

none of the cluster sizes falls in the interval ( α/c, α ) for a number α ∼ Θ ( √ p (1 -q ) n p -q ) and c > 1 is some universal constant;

The algorithm in [4] then has to exhaustively search for such a gap, and then apply a convex programbased algorithm to find a large cluster of size at least Υ . As we discuss in the Appendix D. the assumption of the recoverable cluster being larger than Ω( √ p (1 -q ) n/ ( p -q )) is (relatively) natural as any polynomial time algorithm can only recover clusters of size Ω( √ n ) , under the planted clique conjecture. Still, two natural questions that remain are

Can we break the small cluster barrier without making the first assumption on the existence of a gap between the sizes of some clusters?

Can we remove the dependency of the size of the recoverable cluster on the number k of clusters? In particular, when k glyph[greatermuch] √ n , can we still recover a cluster of size ˜ Ω( √ n ) ?

The above questions are inherently related to the clustering problem under the faulty oracle model which was recently proposed by Mazumdar and Saha [23], as an instance from the faulty oracle model is exactly the graph that is sampled from SBM with corresponding parameters. Thus, it is natural to ask if one can advance the state-of-the-art algorithm for recovering large clusters for the graph instance from the faulty oracle model using an improved algorithm for the SBM?

1.1 Our contributions

We affirmatively answer all three questions mentioned above. Specifically, we demonstrate that clusters of size ˜ Ω( √ n ) can be successfully recovered in both the standard SBM and the faulty oracle model, regardless of the number of clusters present in the graph. This guarantee surpasses any previous achievements in related studies. The practical implications of this finding are significant

since real-world networks often exhibit a substantial number of clusters (see e.g. [29]), varying in size from large to small.

1.1.1 Recovering large clusters in the SBM

We first provide a singular value decomposition (SVD) based algorithm, without assuming there is a gap between the sizes of some clusters, for recovering large latent clusters. Furthermore, the recoverability of the largest cluster is unaffected by the number of underlying clusters.

Theorem 1.2 (Recovering one large cluster) . Let G be a graph that is generated from the SBM( n, k, p, q ) with σ = max ( √ p (1 -p ) , √ q (1 -q ) ) . If both of the following conditions are satisfied: (1) the size of the largest cluster, denoted by s max , is at least s ∗ := 2 13 · √ p (1 -q ) · n · log n ( p -q ) ; (2) σ 2 = Ω(log n/n ) . There exists a polynomial time algorithm that exactly recovers a cluster of size at least s max 7 with probability 1 -1 n 2 .

We have the following remarks about Theorem 1.2. (1) By the assumption that σ 2 = Ω(log n/n ) , we obtain that p = Ω( log n n ) , which further implies that the expected degrees are at least logarithmic in n . This is necessary as exact recovery in SBM requires the node degrees to be at least logarithmic even in the balanced case (i.e. when all the clusters have the same size; see e.g. [1]). (2) In contrast to the work [4], our algorithm breaks the small cluster barrier and improves upon the result of [4] in the following sense: we do not need to assume there is a large interval such that the sizes of clusters do not fall in, nor do our bounds get affected with increasing number of small clusters. (3) As a byproduct of Theorem 1.2, we give an algorithm that improves a result of [28] on partially recovering clusters in the SBM in the balanced case. We refer to Appendix C for details.

In addition, the tradeoff of the parameters in our algorithm in Theorem 1.2 is nearly optimal up to polylogarithmic factors for constant p and q under the planted clique conjecture (see Appendix D).

Recovering more clusters . We can apply the above algorithm to recover even more clusters, using a 'peeling strategy' (see [3]). That is, we first recover the largest cluster (under the preconditions of Theorem 1.2), say V 1 . Then we can remove V 1 and all the edges incident to them and obtain the induced subgraph of G on the vertices V ′ := V \ { V 1 } , denoting it as G ′ . Note that G ′ is a graph generated from SBM( n ′ , k -1 , p, q ) where n ′ = n -| V 1 | . Then we can invoke the previous algorithm on G ′ to find the largest cluster again. We can repeat the process until the we reach a point where the recovery conditions no longer hold on the residual graph. Formally, we introduce the following definition of prominent clusters.

Definition 1.3 (Prominent clusters) . Let V 1 , . . . , V k be the k latent clusters and s 1 , . . . , s k be the size of the clusters. WLOG we assume s 1 ≥ ··· ≥ s k . Let k ′ ≥ 0 be the smallest integer such that one of the following is true. (1) s k ′ +1 < 2 13 · √ p (1 -q ) √∑ k i = k ′ +1 s i ( p -q ) , (2) σ 2 < log( ∑ k i = k ′ +1 s i ) / ( ∑ k i = k ′ +1 s i ) . We call V 1 , . . . , V k ′ prominent clusters of V .

By the above definition, Theorem 1.2, and the aforementioned algorithm, which we call RECURSIVECLUSTER, we can efficiently recover all these prominent clusters.

Corollary 1.4 (Recovering all the prominent communities) . Let G be a graph that is generated from the SBM( n, k, p, q ) model. Then there exists a polynomial time algorithm RECURSIVECLUSTER that correctly recovers all the prominent clusters of G , with probability 1 -o n (1) .

Experimental Comparisons. We evaluate the performance of our algorithm in the simulation settings outlined in [4] and confirm its effectiveness. Moreover, the experiments conducted in [4] established that their gap constraint is an observable phenomenon. We demonstrate that our algorithm can accurately recover clusters even without this gap constraint. Specifically, we succeed in identifying large clusters in scenarios where there were Ω( n ) single-vertex clusters, a situation where the guarantees provided by [4] are inadequate. We observed that simpler spectral algorithms, such as [28], also failed to perform well in this scenario. Furthermore, we observe that the run-time of our algorithm is significantly faster than the SDP based approach of [3, 4]. Finally, we present empirical evidence of the efficacy of our techniques beyond their theoretical underpinnings.

1.1.2 An algorithm for clustering with a faulty oracle

We apply the above algorithm to give an improved algorithm for a clustering problem in a faulty oracle model, which was proposed by [23]. The model is defined as follows:

Definition 1.5. Given a set V = [ n ] := { 1 , · · · , n } of n items which contains k latent clusters V 1 , · · · , V k such that ∪ i V i = V and for any 1 ≤ i < j ≤ k , V i ∩ V j = ∅ . The clusters V 1 , . . . , V k are unknown. We wish to recover them by making pairwise queries to an oracle O , which answers if the queried two vertices belong to the same cluster or not. This oracle gives correct answer with probability 1 2 + δ 2 , where δ ∈ (0 , 1) is a bias parameter. It is assumed that repeating the same question to the oracle O , it always returns the same answer 5 .

Our goal is to recover the latent clusters efficiently (i.e., within polynomial time) with high probability by making as few queries to the oracle O as possible. One crucial limitation of all the previous polynomial-time algorithms ([23, 21, 27, 20, 14]) that make sublinear 6 number of queries is that they cannot recover large clusters, if there are at least ˜ Ω( n 2 / 5 ) small clusters. Now we present our result for the problem of clustering with a faulty oracle.

Theorem 1.6. In the faulty oracle model with parameters n, k, δ , there exists a polynomial time algorithm NOSIYCLUSTERING ( s ) , such that for any n ≥ s ≥ C · √ n log 2 n δ , it recovers all clusters of size larger than s by making O ( n 4 log 2 n δ 4 · s 4 + n 2 log 2 n s · δ 2 ) queries in the faulty oracle model.

We remark that our algorithm works without the knowledge of k , i.e., the number of clusters. Note that Theorem 1.6 says even if there are Ω( n ) small clusters, our efficient algorithm can still find all clusters of size larger than Ω( √ n log n δ ) with sublinear number of queries. We note that the size of clusters that our algorithm can recover is nearly optimal under the planted clique conjecture. Due to space constraints, all the missing algorithms, analyses, and proofs are deferred to Appendix E and F.

1.2 Our techniques

Now we describe our main idea for recovering the largest cluster in a graph G = ( V, E ) that is generated from SBM( n, k, p, q ).

Previous SBM algorithms The starting point of our algorithm is a Singular Value Decomposition (SVD) based algorithm by [28], which in turn is built upon the seminal work of [25]. The main idea underlying this algorithm is as follows: Given the adjacency matrix A of G , project the columns of A to the space A k , which is the subspace spanned by the first k left singular vectors of A k . Then it is shown that for appropriately chosen parameters, the corresponding geometric representation of the vertices satisfies a separability condition. That is, there exists a number r > 0 such that 1) vertices in the same cluster have a distance at most r from each other; 2) vertices from different clusters have a distance at least 4 r from each other. This is proven by showing that each projected point P u is close to its center, which is point u corresponding to a column in the expected adjacency matrix E[ A ] . There are exactly k centers corresponding to the k clusters. Then one can easily find the clusters according to the distances between the projected points.

The above SVD-based algorithm aims to find all the k clusters at once. Since the distance between two projected points depends on the sizes of the clusters they belong to, the parameter r is inherently related to the size s of the smallest cluster. Slightly more formally, in order to achieve the above separability condition, the work [28] requires that the minimum distance (which is roughly √ s ( p -q ) ) between any two centers is at least Ω( √ n/s ) , which essentially leads to the requirement that the minimum cluster size is large, say Ω( √ n ) , in order to recover all the k clusters.

High-level idea of our algorithm In comparison to the work [28], we do not attempt to find all the k clusters at once. Instead, we focus on finding large clusters, one at a time. As in [28], we first project the vertices to points using the SVD. Then instead of directly finding the 'perfect' clusters from the projected points, we first aim to find a set S that is somewhat close to a latent cluster that is large enough. Formally, we introduce the following definition of V i -plural set.

glyph[negationslash]

Definition 1.7 (Plural set) . We call a set S ⊂ V as a V i -plural set if (1) | S ∩ V i | ≥ 2 13 √ n log n ; (2) For any V j = V i we have | S ∩ V j | ≤ 0 . 1 · | S ∩ V i | .

That is, a plural set contains sufficiently many vertices from one cluster and much fewer vertices from any other cluster.

Recall that s ∗ := C p (1 -q ) · n · log n ( p -q ) for C = 2 13 , and s max ≥ s ∗ . We will find a V i -plural set for any cluster V i that is large enough, i.e., | V i | ≥ s max 7 . To recover large clusters, our crucial observation is that it suffices to separate vertices of one large cluster from other large clusters, rather than trying to separate it from all the other clusters. This is done by setting an appropriate distance threshold L to separate points from any two different and large clusters. Then by refining Vu's analysis, we can show that for any u ∈ V i with | V i | ≥ s max 7 , the set S that consists of all vertices whose projected points belong to the ball surrounding u with radius L is a V i -plural set, for some appropriately chosen L . It is highly non-trivial to find such a radius L . To do so, we carefully analyze the geometric properties of the projected points. In particular, we show that the distances between a point and its projection can be bounded in terms of the k ′ -th largest eigenvalue of the expected adjacency matrix of the graph (see Lemma 2.2), for a carefully chosen parameter k ′ . To bound this eigenvalue, we make use of the fact that A is a sum of many rank 1 matrices and Weyl's inequality (see Lemma 2.3). We refer to Section 2 for more details.

Now suppose that the V i -plural set S is independent of the edges in V × V (which is not true and we will show how to remedy this later). Then given S , we can run a statistical test to identify all the vertices in V i . To do so, for any vertex v ∈ V , observe that the subgraph induced by S ∪ { v } is also sampled from a stochastic block model. For each vertex v ∈ V i , the expected number of its neighbors in S is

glyph[negationslash]

On the other hand, for each vertex u ∈ V j for some different cluster V j = V i , the expected number of its neighbors in S is

glyph[negationslash]

since | S ∩ V j | ≤ 0 . 1 · | S ∩ V i | for any V j = V i . Hence there exists a Θ(( p -q ) · | S ∩ V i | ) gap between them. Thus, as long as | S ∩ V i | is sufficiently large, with high probability, we can identify if a vertex belong to V i or not by counting the number of its neighbors in S .

To address the issue that the set S does depend on the edge set on V , we use a two-phase approach: that is, we first randomly partition V into two parts U, W (of roughly equal size), and then find a V i -plural set S from U , then use the above statistical test to find all the vertices of V i in W (i.e., V \ U ), as described in IDENTIFYCLUSTER( S, W, s ) (i.e. Algorithm 4).

Note that the output, say T 1 , of this test is also V i -plural set. Then we can find all vertices of V i in U by running the statistical test again using T 1 and U , i.e., invoking IDENTIFYCLUSTER( T 1 , U, s ). Then the union of the outputs of these two tests gives us V i . We note that there is correlation between T 1 and U , which makes our analysis a bit more involved. We solve it by taking a union bound over a set of carefully defined bad events; see the proof of Lemma 2.7.

1.3 Other related work

In [11] (which improves upon [12]), the author also gave a clustering algorithm for SBM that recovers a cluster at a time, while the algorithm only works under the assumption that all latent clusters are of size Ω( √ n ) , thus they do not break the 'small cluster barrier'.

The model for clustering with a faulty oracle captures some applications in entity resolution (also known as the record linkage ) problem [16, 24], the signed edges prediction problem in a social network [22, 26] and the correlation clustering problem [5]. A sequence of papers has studied the problem of query-efficient (and computationally efficient) algorithms for this model [23, 21, 27, 20, 14]. We refer to references [23, 21, 27] for more discussions of the motivations for this model.

2 The algorithm in the SBM

We start by giving a high-level view of our algorithm (i.e., Algorithm 1). Let G = ( V, E ) be a graph generated from SBM( n, k, p, q ) . For a vertex v and a set T ⊂ V , we let N T ( v ) denote the number of neighbors of v in T .

We first preprocess (in Line 1) the graph G by invoking Algorithm 2 PREPROCESSING, which randomly partitions V into four subsets Y 1 , Y 2 , Z, W such that each vertex is added to Y 1 , Y 2 , Z, W

with probability 1 / 8 , 1 / 8 , 1 / 4 , 1 / 2 , respectively. Let Y = Y 1 ∪ Y 2 , U = Y ∪ Z . See Figure 1 for a visual presentation of the partition. Let ˆ A (resp. ˆ B ) be the bi-adjacency matrix between Y 1 (resp. Y 2 ) and Z . This part is to reduce the correlation between some random variables in the analysis, similar to in [25] and [28]. Then we invoke (in Line 2) Algorithm 3 ESTMATINGSIZE to estimate the size of the largest cluster. It first samples √ n log n vertices from Y 2 and then counts their number of neighbors in W . These counters allow us to obtain a good approximation s of s max .

We then repeat the following process to find a large cluster (or stop when the number of iterations is large enough). In Line 4-7, we sample a vertex u ∈ Y 2 and consider the column vector ̂ u corresponding to u in the bi-adjacency matrix ˆ A between Y 2 and Z . Then we consider the projection P ̂ A k ′ ̂ u of ̂ u onto the subspace of the first k ′ singular vectors of ˆ A for some appropriately chosen k ′ , and the set S of all vertices v in Y 2 whose projections are within distance L/ 20 from p u , for some parameter L . In Lines 9-15, we give a process that completely recovers a large cluster when S is a plural set. More precisely, we first test if | S | ≥ ¯ s/ 21 and if so, we invoke Algorithm 4 to obtain T 1 = IDENTIFYCLUSTER ( S, W, s ) , which simply defines T 1 to be the set of all vertices v ∈ W with N S ( v ) ≥ q | S | +( p -q ) s 56 . Then we check (Line 10) if the set T 1 satisfies a few conditions to test if u is indeed a good center (so that S is a plural set) and test if T 1 = V 1 ∩ W . If so, we then invoke IDENTIFYCLUSTER ( T 1 , U, s ) to find V 1 ∩ U . Note that we use a two-step process to find V 1 , as N S ( u ) is not a sum of independent events for u ∈ U .

2.1 The analysis

We first show that ESTIMATINGSIZE outputs an estimator s approximating the size of the largest cluster within a factor of 2 with high probability.

Lemma 2.1. Let s be as defined in Line 6 of Algorithm 3. Then with probability 1 -n -8 we have 0 . 48 · s max ≤ s ≤ 0 . 52 · s max .

Recall that ˆ A (resp. ˆ B ) is the bi-adjacency matrix between Y 1 (resp. Y 2 ) and Z . Let A and B be the corresponding matrices of expectations. That is, ˆ A = A + E , where E is a random matrix consisting of independent random variables with 0 means and standard deviations either √ p (1 -p ) or √ q (1 -q ) .

For a vertex u ∈ Y 1 , let ̂ u and u represent the column vectors corresponding to u in the matrices ˆ A and A respectively (We define analogous notations for ˆ B and B when u ∈ Y 2 .). We let e u := ̂ u -u , i.e., e u is the random vector with zero mean in each of its entries. Recall that p u = P ̂ A k ′ ̂ u .

V

U

W

Y

Z

Y

1

Y

2

1: Randomly partitions V into four subsets Y 1 , Y 2 , Z, W such that each vertex is added to Y 1 , Y 2 , Z, W with probability 1 / 8 , 1 / 8 , 1 / 4 , 1 / 2 , respectively.

2: Let Y = Y 1 ∪ Y 2 , U = Y ∪ Z .

3: Let ˆ A (resp. ˆ B ) be the bi-adjacency matrix between Y 1 (resp. Y 2 ) and Z .

4: Return ˆ A, ˆ B,Y 2 , Y 1 , Z, W

Algorithm 3 ESTIMATINGSIZE( G = ( V, E ) , p, q, W, Y 2 ): Estimating the size of the largest cluster

Algorithm 4 IDENTIFYCLUSTER( S, R, s ): Finding a subcluster R ∩ V i using a V i -plural set S

1: T ←∅ 2: for each v ∈ R do 3: if N v,S ≥ q | S | +( p -q ) s 56 then 4: add v to T

5: Return T

Now we bound the distance between P ̂ A k ′ ̂ u and the expectation vector u . We set ε = 0 . 002 in the following.

Lemma 2.2. Follows the setting of Algorithm 2, we fix Y 1 , Y 2 , Z, W . For any vector u ∈ Y 2 and k ′ ≥ 1 we have ‖ P ̂ A k ′ ( ̂ u ) -u ‖ ≤ 1 √ s u ‖ ( P ̂ A k ′ -I ) A ‖ + ‖ P ̂ A k ′ ( e u ) . ‖

Furthermore, for some constant C 2 , and ε as described above we have √

‖ ( P ̂ A k ′ -I ) A ‖ = ‖ ( P ̂ A k ′ -I ) ˆ A -( P ̂ A k ′ -I ) E ‖ ≤ 2 C 2 σ n + λ k ′ +1 ( A ) with probability 1 -O ( n -3 ) for a random ˆ A , where λ t ( A ) is the t -th largest sigular value of A .

For any set V ′ ⊂ Y 2 s.t. | V ′ | ≥ 4 log n ε 2 , with probability 1 -n -8 , we have ‖ P ̂ A k ′ ( e u ) ‖ ≤ 1 ε σ √ k ′ for at least (1 -2 ε ) fraction of the points u ∈ V ′ .

We have the following result regarding the t -th largest singular value λ t ( A ) of A .

Lemma 2.3. For any t > 1 , λ t ( A ) ≤ ( p -q ) n/t .

Now we introduce the a definition of good center, the ball of which induces a plural set.

Definition 2.4 (Good center) . We call a vector ̂ u ∈ ˆ B a good center if it belongs to a cluster V i such that | V i | ≥ s max 4 and ∥ ∥ ∥ P ̂ A k ′ ( e u ) ∥ ∥ ∥ ≤ 1 ε σ √ k ′ .

That is, a good center is a vertex that belongs to a large cluster and has a low glyph[lscript] 2 norm after the projection. Then by Lemma 2.2, we have the following corollary on the number of good centers.

Corollary 2.5. If s max ≥ 16 √ n log n , then with probability 1 -n -8 there are (1 -2 ε ) · s max many good centers in V .

This implies that if we sample 100 n log n s many vertices independently at random, we shall sample a good center with probability 1 -n -8 .

Good center leads to plural set We show that if at line 4 a good center from a cluster V i is chosen, then the set S formed in line 7 is a V i -plural set. Recall that L = √ 0 . 004( p -q ) √ s . Let L ε := L .

glyph[negationslash]

Lemma 2.6. Let u be a good center belonging to V i ∩ Y 2 and S = { v ∈ Y 2 : ‖ p u -p v ‖ ≤ L ε / 20 } . Then it holds with probability 1 -O ( n -3 ) that | V i ∩ S | ≥ s/ 21 and for any other cluster V glyph[lscript] with glyph[lscript] = i , | S ∩ V glyph[lscript] | ≤ 1 . 05 εs . Thus S is a V i plural set as 1 / 21 · 1 / 10 ≥ 1 . 05 ε .

Plural set leads to cluster recovery We now prove that given a plural set for a large cluster V i , we can recover the whole cluster. This is done by two invocations of Algorithm 4.

Lemma 2.7. Let U, W be the random partition as specified in Algorithm 1. Let S ⊆ Y 2 be the V i -plural set where | V i | ≥ s max / 4 . Let T 1 := IDENTIFYCLUSTER ( S, W, s ) and T := T 1 ∪ IDENTIFYCLUSTER ( T 1 , U, s ) . Then with probability 1 - O ( n -3 ) , it holds that T 1 = V i ∩ W , T 1 ≥ s 6 and T = V i .

Testing if T 1 is a sub-cluster Since S may not be a plural set, we show that we can test if T 1 = W ∩ V i for some large cluster V i using the conditions of Line 10 of Algorithm 1.

Lemma 2.8. Let v be a good center from V i ∩ Y 2 such that | V i | ≥ s max 4 and let S = { u ∈ Y 2 : ‖ p u -p v ‖ ≤ L ε 30 } . Let T 1 be the set returned by IDENTIFYCLUSTER ( S, W, s ). Then with probability at least 1 -n -8 , | T 1 | ≥ s 6 and N T 1 ( u ) ≥ (0 . 9 p + 0 . 1 q ) | T 1 | for any u ∈ T 1 and N T 1 ( u ) ≤ (0 . 9 p +0 . 1 q ) | T 1 | for any u ∈ W \ T 1 .

glyph[negationslash]

Finally, we show that if the set T 1 = V i ∩ W for some large cluster V i , then it satisfies one of the conditions at line 10 of Algorithm 1. Together with the previous results this guarantees correct recovery of a large set at every round.

glyph[negationslash]

Corollary 2.9. Let T 1 = IDENTIFYCLUSTER ( S, W, s ) be a set such that T 1 = V i ∩ W for any underlying community V i of size | V i | ≥ s max / 7 . Then with probability 1 -n -8 either | T 1 | ≤ s 6 or there is a vertex u ∈ T 1 such that N T 1 ( u ) ≤ (0 . 9 p +0 . 1 q ) | T 1 | .

Remark 2.10. Note that in Lemma 2.8 and Corollary 2.9, the quantity N T 1 ( u ) for any u ∈ T 1 is a sum of independent events. This is because the event that a vertex in v ∈ W is chosen in T 1 is solely based on N u ( S ) , where S ∩ T = ∅ . Thus, for any u 1 , u 2 ∈ T , there is an edge between them (as per underlying cluster identities) independent of other edges in the graph.

The proofs of the above results are deferred to Appendix 2.1.

Now we are ready to prove Theorem 1.2.

glyph[negationslash]

Proof of Theorem 1.2 By the precondition, we have that s max ≥ s ∗ . First, in Line 2, Lemma 2.1 guarantees that 0 . 48 s max ≤ s ≤ 0 . 52 s max . By Corollary 2.5 and the fact that we iteratively sampled vertices Ω( √ n log n ) times, with probability 1 -n -8 , one such vertex u is a good center. Given such a good center, by Lemma 2.6, we know with probability 1 -O ( n -3 ) , a V i -plural set is recovered on Line 7. Then by Lemma 2.7, given such a V i -plural set, the two invocations of IDENTIFYCLUSTER recovers the cluster V i with probability 1 - O ( n -3 ) . Furthermore, Lemma 2.8 shows that if the sampled vertex v is a good center, then with probability 1 -n -8 none of the conditions of line 10 are satisfied, and we are able to recover a cluster. On the other hand, Corollary 2.9 shows that if T 1 = V i ∩ W for any large cluster V i , ( V i : | V i | ≥ s max / 7 ) then one of the conditions of line 10 is satisfied with probability 1 -n -8 and the algorithm goes to the next iteration to sample a new vertex in line 4. Taking a union bound on all the events for at most O ( √ n log n ) iterations guarantees that algorithm 1 finds a cluster of size s max / 7 with probability 1 - O ( n -2 ) . This completes the correctness of Algorithm 1.

3 The algorithm in the faulty oracle model

We describe the main ideas of our algorithm NOISYCLUSTERING for clustering with a faulty oracle. Let V be the set of items that contains k latent clusters V 1 , . . . , V k and O be the faulty oracle. Following the idea of [27], we first sample a subset T ⊆ V of appropriate size and query O ( u, v ) for all pairs u, v ∈ T . Then apply our SBM clustering algorithm (i.e. Algorithm 1 CLUSTER) on the graph (with all the edges for the pairs that are reported to belong to the same cluster) induced by T to obtain clusters X 1 , . . . , X t for some t ≤ k . We can show that each of these sets is a subcluster of some large cluster V i . Then we can use majority voting to find all other vertices that belong to X i , for each i ≤ t . That is, for each X i and v ∈ V , we check if the number of neighbors of v in X i is at least | X i | 2 . In this way, we can identify all the large clusters V i corresponding to X i , 1 ≤ i ≤ t .

Furthermore, we can just choose a small subset of X i of size O ( log n δ 2 ) for majority voting to reduce query complexity. Then we can remove all the vertices in V i 's and remove all the edges incident to them from both V and T and then we can use the remaining subsets T and V and corresponding subgraphs to find the next sets of large clusters. The algorithm NOISYCLUSTERING then recursively finds all the large clusters until we reach a point where the recovery condition on the current graph no longer holds. The pseudocode and the analysis of NOISYCLUSTERING are deferred to Appendix F.

4 Experiments

Now we exhibit various properties of our algorithms by running it on several unbalanced SBM instantiations and also compare our improvement w.r.t the state-of-the-art. We start by running our algorithm RECURSIVECLUSTER on the instances used by the authors of [4]. WLOG, we assume that | V 1 | ≥ | V 2 | · · · ≥ | V k | . We denote the algorithm in [4] by ACX.

Comparison with ACX In Exp1 (abbreviated for Experiment #1) and Exp2 , our algorithm recovers the largest cluster while ACX recovers all the clusters. This is because we have a large, constant lower bound on the size of the clusters we can recover. If we scale up the size of the clusters by a factor of 20 in those instances, then we are also able to recover all clusters.

Overcoming the gap constraint in practice Exp3 is the 'mid-size-cluster' experiment in [4]. In this case, ACX recovers the largest cluster completely, but only some fraction of the second-largest cluster, which is an incorrect outcome. In [4], the authors used this experiment to emphasize that their 'gap-constraint' is not only a theoretical artifact but also observable in practice. In comparison, we recover the largest cluster while do not make any partial recovery of the rest of the clusters. In Exp4 , we modify the instance in Exp3 by changing the size of the second cluster to 200 . Note that this further reduces the gap, and ACX fails in this case as before. In comparison, we are able to recover both the largest and the second largest cluster. This exhibits that we are indeed able to overcome the experimental impact of the gap constraint observed in [4] in the settings of Table 1.

We then run some more experiments in the settings of Table 2 to describe other properties of our algorithms as well as demonstrate the practical usefulness of our 'plural-set' technique.

Manyclusters Exp5 covers a situation where k = Ω( n ) (specifically n/ 3 ), which can not be handled by ACX, as the size of the recoverable cluster in [4] is lower bounded by k log n/ ( p -q ) 2 > n . In comparison, our algorithm can recover the two main clusters. We also remark, in this setting, the spectral algorithm in [28] with k = 1000 can not geometrically separate the large clusters.

Recovery of small clusters Exp6 describes a situation where the peeling strategy successfully recovers clusters that were smaller than √ n in the original graph. Once the largest cluster is removed, the smaller cluster then becomes recoverable in the residual graph. Finally, we discuss the usefulness of the plural set.

Run-time comparison Here, note that our method is a combination of a ( p -q ) √ n/ √ p (1 -q ) dimensional SVD projection, followed by some majority voting steps. Furthermore, we have

( p -q ) / √ p (1 -q ) ≤ 2 √ p . This implies that the time complexity of our algorithm is O ( 2 √ p · n 2 . 5 ) . In comparison, the central tool used in the algorithms by [3, 4] is an SDP relaxation, which scales as O ( n 3 ) . This implies that the asymptotic time complexity of our method is also an improvement on the state-of-the-art. We also confirm that the difference in the run-time becomes observable even for small values of n . For example, our algorithm recovers the largest cluster in Experiment 1 (with n = 1100 ) of table 1 in 1.4 seconds. In comparison, [3] recovers all 4 clusters, but takes 44 seconds.

On the importance of plural sets Recall that in Algorithm 1 (which is the core part of RECURSIVECLUSTER), we first obtain a plural-set S in the partition Y 2 of V (see Figure 1 to recall the partition). S is not required to be V i ∩ Y 2 for any cluster V i , but the majority of the vertices in S must belong to a large cluster V i (which is the one we try to recover). We have the following observations:

In Exp-3 of Table 1, in the first round we recover a cluster V 1 . Here in our first step, we recover a plural set S , where S ⊂ V 1 ∩ Y 2 . That is, we do not recover all the vertices of V 1 in Y 2 when forming the plural-set.

In Exp-4 of Table 1, in the second iteration we recover a cluster V 2 . However, the plural set S glyph[negationslash]⊂ V 2 , and in fact contains a few vertices from V 4 ! This is in fact the exact situation that motivates the plural-set method.

In both cases, the plural-set is then used to recover S 1 := V 1 ∩ W and V 2 ∩ W respectively, and then S 1 is used to recover the vertices of the corresponding cluster in U . Thus, our technique enables us to completely recover the largest cluster even though in the first round we may have some misclassifications. A more thorough empirical understanding of the Plural sets in different applications is an interesting future work.

We conclude our paper with some more discussion and future directions.

5 Conclusion

In this work, we design a spectral algorithm that recovers large clusters in the SBM model in the presence of arbitrary numbers of small clusters and compared to previous work, we do not require gap constraint in the size of consecutive clusters. Some interesting directions that remain open are as follows.

We note that both our algorithm and [4] require knowledge of the probability parameters p and q ([4] also need the knowledge of k , the number of clusters). Thus, whether parameterfree community recovery algorithms can be designed with similar recovery guarantees is a very interesting question.

Both our result (Algorithm 1) as well as [4] have a multiplicative log n term in our recovery guarantees. In comparison, the algorithm by Vu [28], which is the state-of-the-art algorithm in the dense case when 'all' clusters are large, only has an additive logarithmic term. This raises the question of whether this multiplicative logarithmic factor can be further optimized when recovering large clusters in the presence of small clusters.

Additionally, we note that the constant in our recovery bound is quite large ( 2 13 ), and we have not made efforts to optimize this constant. We believe this constant can be optimized significantly, such as through a more careful calculation of the Chernoff bound in Theorem A.2.

References

Emmanuel Abbe. Community detection and stochastic block models: recent developments. The Journal of Machine Learning Research , 18(1):6446-6531, 2017.

Emmanuel Abbe and Colin Sandon. Community detection in general stochastic block models: Fundamental limits and efficient algorithms for recovery. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science , pages 670-688. IEEE, 2015.

Nir Ailon, Yudong Chen, and Huan Xu. Breaking the small cluster barrier of graph clustering. In International conference on machine learning , pages 995-1003. PMLR, 2013.

Nir Ailon, Yudong Chen, and Huan Xu. Iterative and active graph clustering using trace norm minimization without cluster size constraints. J. Mach. Learn. Res. , 16:455-490, 2015.

Nikhil Bansal, Avrim Blum, and Shuchi Chawla. Correlation clustering. Machine learning , 56(1-3):89-113, 2004.

Béla Bollobás and Alex D Scott. Max cut for random graphs with a planted partition. Combinatorics Probability and Computing , 13(4-5):451-474, 2004.

Ravi B Boppana. Eigenvalues and graph bisection: An average-case analysis. In 28th Annual Symposium on Foundations of Computer Science (sfcs 1987) , pages 280-285. IEEE, 1987.

Thang Nguyen Bui, Soma Chaudhuri, Frank Thomson Leighton, and Michael Sipser. Graph bisection algorithms with good average case behavior. Combinatorica , 7(2):171-191, 1987.

Kamalika Chaudhuri, Fan Chung, and Alexander Tsiatas. Spectral clustering of graphs with general degrees in the extended planted partition model. In Conference on Learning Theory , pages 35-1. JMLR Workshop and Conference Proceedings, 2012.

Yudong Chen, Sujay Sanghavi, and Huan Xu. Clustering sparse graphs. In Proceedings of the 25th International Conference on Neural Information Processing Systems-Volume 2 , pages 2204-2212, 2012.

Sam Cole. Recovering nonuniform planted partitions via iterated projection. Linear Algebra and its Applications , 576(1):79-107, 2019.

Sam Cole, Shmuel Friedland, and Lev Reyzin. A simple spectral algorithm for recovering planted partitions. Special Matrices , 5(1):139-157, 2017.

Chandler Davis and William M Kahan. Some new bounds on perturbation of subspaces. Bulletin of the American Mathematical Society , 75(4):863-868, 1969.

Alberto Del Pia, Mingchen Ma, and Christos Tzamos. Clustering with queries under semirandom noise. arXiv preprint arXiv:2206.04583. To appear at Conference on Learning Theory (COLT) 2022 , 2022.

Martin E. Dyer and Alan M. Frieze. The solution of some random np-hard problems in polynomial expected time. Journal of Algorithms , 10(4):451-489, 1989.

Ivan P Fellegi and Alan B Sunter. A theory for record linkage. Journal of the American Statistical Association , 64(328):1183-1210, 1969.

Sally A Goldman, Michael J Kearns, and Robert E Schapire. Exact identification of circuits using fixed points of amplification functions. In Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science , pages 193-202. IEEE, 1990.

Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association , 58(301):13-30, 1963.

Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social networks , 5(2):109-137, 1983.

Zengfeng Huang Jinghui Xia. Optimal clustering with noisy queries via multi-armed bandit. To appear at the 39th International Conference on Machine Learning (ICML 2022) , 2022.

Kasper Green Larsen, Michael Mitzenmacher, and Charalampos Tsourakakis. Clustering with a faulty oracle. In Proceedings of The Web Conference 2020 , pages 2831-2834, 2020.

Jure Leskovec, Daniel Huttenlocher, and Jon Kleinberg. Predicting positive and negative links in online social networks. In Proceedings of the 19th international conference on World wide web , pages 641-650, 2010.

Arya Mazumdar and Barna Saha. Clustering with noisy queries. In Advances in Neural Information Processing Systems , pages 5788-5799, 2017.

Arya Mazumdar and Barna Saha. A theoretical analysis of first heuristics of crowdsourced entity resolution. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence , pages 970-976, 2017.

Frank McSherry. Spectral partitioning of random graphs. In Proceedings 42nd IEEE Symposium on Foundations of Computer Science , pages 529-537. IEEE, 2001.

Michael Mitzenmacher and Charalampos E Tsourakakis. Predicting signed edges with o ( n 1+ o (1) log n ) queries. arXiv preprint arXiv:1609.00750 , 2016.

Pan Peng and Jiapeng Zhang. Towards a query-optimal and time-efficient algorithm for clustering with a faulty oracle. In Conference on Learning Theory , 2021.

Van Vu. A simple svd algorithm for finding hidden partitions. Combinatorics, Probability & Computing , 27(1):124, 2018.

Jaewon Yang and Jure Leskovec. Defining and evaluating network communities based on ground-truth. In Proceedings of the ACM SIGKDD Workshop on Mining Data Semantics , pages 1-8, 2012.

A Preliminary Notations and Tools

Notations for vectors. Let ˆ M be the adjacency matrix of the graph G = ( V, E ) that is sampled from SBM( n, k, p, q ) . We denote by M the matrix of expectations, where M [ i, j ] = p if the i -th and j -th vertices belong to the same underlying cluster, and M [ i, j ] = q otherwise. Going forward, we shall work with several sub-matrices of ˆ M and for any submatrix M ′ , we denote by ̂ M ′ and M ′ the random matrix and the corresponding matrix of expectations.

We also use the norm operator ‖ · ‖ frequently in this paper. We use the operator both for vectors and matrices. Given a vector x = ( x 1 , . . . , x d ) , we let ‖ x ‖ := √∑ i x 2 i denote its Euclidean norm. When the input is a matrix M , ‖ M ‖ denotes the spectral norm of M , which is its largest singular value.

We describe the well-known Weyl's inequality.

Theorem A.1 (Weyl's inequality) . Let ˆ A = A + E be a matrix. Then λ t +1 ( ˆ A ) ≤ λ t +1 ( A ) + ‖ E ‖ where ‖ · ‖ is the spectral norm operator as described above.

We will make use of the following general Chernoff Hoeffding bound.

Theorem A.2 (Chernoff Hoeffding bound [18]) . Let X 1 , . . . , X n be i.i.d random variables that can take values in { 0 , 1 } , with E[ X i ] = p for 1 ≤ i ≤ n . Then we have

Here D ( x || y ) is the KL divergence of x and y . We recall the KL divergence between Bernoulli random variables x, y D ( x || y ) = x ln( x/y ) + (1 -x ) ln((1 -x ) / (1 -y )) . it is easy to see that If x ≥ y , then D ( x || y ) ≥ ( x -y ) 2 2 x , and D ( x || y ) ≥ ( x -y ) 2 2 y otherwise.

We also note down a random projection Lemma that we use in our proof.

Lemma A.3 (Expected random projection [28]) . Let P ̂ A k ′ be a k ′ -dimensional projection matrix, and e u be an n dimensional random vector where each entry is 0 mean and has a variance of at most σ 2 . Then we have E[ ‖ P ̂ A k ′ ( e u ) ‖ 2 ] ≤ σ 2 · k ′ .

B Deferred Proofs from Section 2

We first give a general concentration bound concerning neighbors of vertices in the different partitions.

Lemma B.1. Let V be a set of n vertices sampled according to the SBM ( n, k, p, q ) model. Let V ′ ⊂ V where the vertices in V ′ are selected independently of each other. Let V i be a latent cluster with V ′ i = V i ∩ V ′ . We denote by N V ′ ( u ) the number of neighbors of u in V ′ . Then with probability 1 -O ( n -7 ) we have for every u ∈ V ′ i ,

Proof of Lemma B.1. We look at two different sums of random variables. The first is N V ′ i ( u ) which is the sum of | V i ∩ V ′ | many random 0 -1 variables with probability of 1 being p . The second is N V ′ \ V ′ i ( u ) , which is the sum of | V ′ \ V ′ i | variables with probability of 1 being q .

Then we have E[ N V ′ i ( u )] = p | V i ∩ V ′ | and E[ N V ′ \ V ′ i ( u )] = q | V ′ \ V ′ i | . Finally the Chernoff bound implies,

This gives us

Pr ( N V ′ \ V ′ i ( u ) | V ′ \ V ′ i | < q -β ) ≤ e -D ( q -β || q ) | V ′ \ V ′ i | . We fix β = 8 √ p √ n log n | V ′ \ V ′ i | and the term D ( q -β || q ) | V ′ \ V ′ | evaluates to

i

This gives us

Combining Equation (1) and (2) gives us

Now we study the event N V ′ ( u ) ≥ q | V ′ | +( p -q ) | V ′ ∩ V i | +48 · √ p · √ n log n again by breaking into two terms.

The probability bounds for two terms N V ′ i ( u ) and N V ′ \ V ′ i are e -D ( p +3 α || p ) | V ′ i | and e -D ( q +3 β || q ) | V ′ \ V ′ i | respectively. Here note that we use 3 α instead of α , to make calculations easier.

For the first case we have D ( p +3 α || p ) | V ′ i | ≥ 9 α 2 | V ′ i | 2( p +3 α ) . If p ≥ α then D ( p + α || p ) | V ′ i | ≥ 9 α 2 | V ′ i | 8 p which implies we get the same bound as above. If p < α then D ( p + α || p ) | V ′ i | ≥ 9 α 2 | V ′ i | 8 α ≥ α | V ′ i | . Now we have α = 8 √ p √ n log n | V ′ i | . Since p = Ω(log n/n ) we have α | V ′ i | ≥ 8 log n . Combining we get that e -D ( p +2 α || p ) | V ′ i | ≤ n -8 .

Next we analyze D ( q + 3 β || q ) | V ′ \ V ′ i | ≥ 9 β 2 | V ′ \ V ′ i | 2( q +3 β ) . As before, if q ≥ β we have D ( q + 3 β || q ) | V ′ \ V ′ i | ≥ 9 β 2 | V i | 8 q and the result follows as before. Otherwise D ( q + 3 β || q ) | V ′ \ V ′ i | ≥ 9 β 2 | V ′ \ V ′ i | 8 β ≥ β | V ′ \ V ′ i | ≥ 8 √ p √ n log n ≥ 8 log n , which completes the proof.

Then Lemma 2.1 can be proved as follows.

Proof of Lemma 2.1. We know that s max ≥ 2 13 · √ p (1 -q ) √ n log n p -q from the problem definition. Let the corresponding cluster be V i . Then a simple application of Hoeffding bounds gives us that with probability 1 -n -8 , 0 . 51 · s max ≥ | V i ∩ W | ≥ 0 . 49 · s max .

Furthermore, we are interested in the regime where p ≤ 3 / 4 so √ 1 -q ≥ 1 / 2 .

Then if we sample u ∈ | V i ∩ Y 2 | , Lemma B.1 states that with probability 1 -n -8 ,

This implies with probability 1 -n -8 , q | W | +1 . 01 | V i ∩ W | ≥ N W ( u ) ≥ q | W | +0 . 99 | V i ∩ W | which coupled with the fact 0 . 49 ≤ | V i ∩ W | | V i | ≤ 0 . 51 implies that if we are able to sample a vertex from the largest cluster, we get an estimate of s max as described.

Since | V i ∩ Y 2 | ≥ 100 √ n log n , if we sample √ n log n vertices, we sample a vertex u from V i with probability 1 -n -8 . Now, for vertices belonging to smaller clusters, the same bounds apply, and this implies that as long as we are able to sample a vertex from the largest cluster, we get an estimate of s max between a factor of 0 . 48 and 0 . 52 .

Proof of Lemma 2.2. These results follow directly from Vu [28] with some minor modifications. In their paper, Vu decomposes the matrix into Y and Z . In comparison, we decompose the matrix to U and W first, and then U is decomposed into Y and Z . Thus the size of Y and Z in our framework is roughly half as compared to [28]. However, since the size of the clusters we are concerned about are all larger than 128 · √ n log n , the results follow in the same way with a change of a factor of 2 .

Now we describe the results and how we deviate from Vu's analysis to get our result. For the first part, in [28, page 132] it was proved that for any fixed ̂ u ∈ ˆ B ,

Furthermore, it was proven (also in page 132 of [28]) that

It was observed that ∥ ∥ ∥ ( P ̂ A k -I ) ˆ A ∥ ∥ ∥ ≤ λ k +1 ( ˆ A ) ≤ λ k +1 ( A ) + ‖ E ‖ = ‖ E ‖ as A has rank at most k ; and ∥ ∥ ∥ ( P ̂ A k -I ) E ∥ ∥ ∥ ≤ ‖ E ‖ . Then from Lemma 2.2 in [28] we have that with probability 1 -O ( n -3 ) ‖ E ‖ ≤ C 2 σn 1 / 2 for some constant C 2 > 0 .

Next, we observe that for any k ′ ≥ 1 , it still holds that

Furthermore,

Again, with probability at least 1 -1 /n 3 , ‖ E ‖ ≤ C 2 σ √ n , which further implies that

This is a simple but crucial step that removes our dependency on k , and allows us to treat all clusters of size o ( √ n ) as noise.

Now, we analyze the first term. From Lemma A.3 we have E[ ‖ P ̂ A k ′ ( e u ) ‖ 2 ] ≤ σ 2 k ′ for any u ∈ Y 2 . Then for any u , Markov's inequality gives us Pr ( ‖ P ̂ A k ′ ( e u ) ‖ ≥ σ √ k ′ ε ) ≤ ε .

Now let us consider any set V ′ ⊂ Y 2 such that | V ′ | ≥ 16 √ n log n . For any u ∈ V ′ we define X u to be the indicator random variable that gets 1 if ‖ P ̂ A k ′ ( e u ) ‖ ≤ σ √ k ′ ε , and 0 otherwise. Then E[ X u ] ≥ 1 -ε . Now, since V ′ ⊂ Y 2 , the variables X u are independent of each other (as e u are independent of each other). Then, using the fact that | V ′ | ≥ 4 log n ε 2 , the Chernoff bound gives us

That is, with probability at least 1 -n -8 , for at least (1 -2 ε ) fraction of the points u ∈ V ′ , ‖ P ̂ A k ′ ( e u ) ‖ ≤ 1 ε σ √ k ′ .

Proof of Lemma 2.3. Let there be k many clusters V 1 , . . . , V k in the SBM problem. Then we define a i = | V i ∩ Z | and b i = | V i ∩ Y 1 | . Then we have that A is an n 1 × n 2 matrix where n 1 = ∑ k i =1 a i and n 2 = ∑ k i =1 b i . The matrix A can be then written as a sum of k +1 many rank 1 matrices:

Here M 0 is the all 1 matrix, and M i is a block matrix with 1 's in a a i × b i sized diagonal block. Since M i is a a i × b i block diagonal matrix of rank 1 , with each entry being ( p -q ) , its singular value is ( p -q ) √ a i b i . Now we define A 1 = ∑ k i =1 ( p -q ) M i . As M i 's are non-overlapping block diagonal matrices, the singular vectors of M i are also singular vectors of A 1 , with the same singular values.

Thus, the sum of singular values of A 1 is ( p -q ) ∑ k i =1 √ a i b i ≤ ( p -q ) √ n 1 n 2 ≤ ( p -q ) n 2 , where the first inequality follows from the Cauchy-Schwarz inequality. Thus, for any t ≥ 1

which gives λ t ( A 1 ) ≤ ( p -q ) n 2 t . Since M 0 has rank 1 , λ 2 ( q · M 0 ) = 0 and thus for t > 1 we have

where the first inequality follows from the Weyl's inequality.

The guarantee of obtaining a plural set is a consequence of Lemma B.2.

√

Lemma B.2. Let k ′ = ( p -q ) n √ p (1 -q ) and ε = 0 . 002 . Let u ∈ Y 2 be a good center belonging to V i , then 1. There is a set V ′ i ⊂ Y 2 ∩ V i such that | V ′ i | ≥ (1 -2 ε ) | Y 2 ∩ V i | so that for all v ∈ V ′ i , we have ‖ P ̂ A k ′ ( u -v ) ‖ ≤ L ε 30 with probability 1 -O ( n -3 ) .

glyph[negationslash]

For any V j = V i which is a ε -large cluster, there is a set V ′ j ⊂ V j ∩ Y 2 s.t | V ′ j | ≥ (1 -2 ε ) | V j ∩ Y 2 | so that for all v ∈ V ′ j we have ‖ P ̂ A k ′ ( u -v ) ‖ ≥ L ε 6 with probability 1 -O ( n -3 ) .

Proof of Lemma B.2. First note that L ε ≥ √ 2 ε · 2 11 · ( p -q ) · ( p (1 -q )) 1 / 4 · n 1 / 4 · log 1 / 2 n ( p -q ) 1 / 2 ≥ 132 , 000 · ( p -q ) 1 / 2 · ( p (1 -q )) 1 / 4 · n 1 / 4 · √ log n .

When u and v belong to the same cluster we have ‖ P ̂ A k ′ ( ̂ u -̂ v ) ‖ ≤ ‖ P ̂ A k ′ ( ̂ u ) -u ‖ + ‖ P ̂ A k ′ ( ̂ v ) -v ‖ .

Now, since u is a good center, from Lemma 2.2 we have

Now from Lemma 2.2 we also know that at least (1 -2 ε ) fraction of the vertices v ∈ V i ∩ Y 2 are also 'good centers' with probability 1 -n -8 . For all such vertices ‖ P ̂ A k ′ ( ̂ u -̂ v ) ‖ ≤ 2 ‖ P ̂ A k ′ ( ̂ u -u ) ‖ ≤ L ε 30 with probability 1 -n -3 .

On the other hand when they belong to different clusters we have

Since V j is a ε -large cluster, | V i | ≥ 256 √ n log n and thus | V j ∩ Y 2 | ≥ 16 √ n log n with probability 1 -O ( n -8 ) . In that case for at least 1 -2 ε fraction of points v ∈ V j ∩ Y 2 we have P ̂ A k ′ ( ̂ v -v ) ≤ L ε 60 .

Now ‖ u -v ‖ ≥ ( p -q ) √ s u + s v ≥ √ 2 ε · ( p -q ) √ s max 6 ≥ √ 2 ε · ( p -q ) √ s √ 0 . 52 · 6 ≥ L ε 5 with probability 1 -n -8 from Lemma 2.1. Thus with probability 1 -n -3 we get

for 1 -2 ε fraction of points v ∈ Y 2 ∩ V j for any ε -large cluster V j . This completes the proof.

Proof of Lemma 2.6. By Lemma 2.1, we have | V i | ≥ s max 4 ≥ s 2 . 1 with probability 1 -n -8 . Then the following events happen.

Since u ∈ V i ∩ Y 2 is a good center, for 1 -2 ε fraction of points v in V i ∩ Y 2 , ‖ p u -p v ‖ ≤ L ε / 30 with probability 1 -O ( n -3 ) as per Lemma B.2. All such points are selected to S . Furthermore, | V i ∩ Y 2 | is lower bounded by | V i | / 9 with probability 1 -n -8 . Therefore, with probability 1 -O ( n -3 ) , we have

For other clusters V glyph[lscript] , if | V glyph[lscript] | ≥ ε · s max , we have ‖ p u -p v ‖ ≤ L ε 6 for only 2 ε fraction of points v in V glyph[lscript] ∩ Y 2 from Lemma B.2. Thus | S ∩ V glyph[lscript] | ≤ 2 ε | V glyph[lscript] ∩ Y 2 | . On the other hand | V glyph[lscript] ∩ Y 2 | ≤ | V glyph[lscript] | / 6 . Thus, with probability 1 -n -8 we have

Otherwise, if V glyph[lscript] is such that ε · s max ≥ | V glyph[lscript] | ≥ ε 2 · s max , then | V glyph[lscript] ∩ Y 2 | ≤ | V glyph[lscript] | / 6 with probability 1 -n -8 . Then | S ∩ V glyph[lscript] | ≤ εs max / 6 ≤ εs .

Otherwise, if | V glyph[lscript] | ≤ ε · s max then | S ∩ V glyph[lscript] | ≤ | V glyph[lscript] | ≤ ε · s max ≤ εs ≤ εs ≤ 1 . 05 · ε · s .

2 2 2 · 0 . 48 0 . 96

glyph[negationslash]

Now, note that for any V glyph[lscript] with glyph[lscript] = i , it holds with probability 1 -O ( n -3 ) that | S ∩ V glyph[lscript] | ≤ 1 . 05 · ε · s ≤ (21 · 1 . 05 · ε ) · s 21 ≤ 0 . 05 · s 21 .

Proof of Lemma 2.7. We first show that if S is a V i -plural set with V i ≥ s max / 4 , then T 1 = V i ∩ W where T 1 is the outcome of IDENTIFYCLUSTER ( S, W, s ) . Since S is a V i plural set, for any vertex v ∈ W ∩ V i , from Lemma B.1 we have that with probability 1 -O ( n -3 ) ,

glyph[negationslash]

Now, let us consider the case when v ∈ V j ∩ W where j = i . Then we know from Lemma 2.6 | S ∩ V j | ≤ 1 . 05 εs ≤ s 210 . Then using Lemma B.1 we have that with probability 1 -O ( n -3 )

Now note that in the IDENTIFYCLUSTER ( S, W, s ) algorithm, we select all vertices from W that have q | S | +( p -q ) · s 56 neighbors in S . Thus, the above analysis implies with probability 1 -O ( n -3 ) T 1 = IDENTIFYCLUSTER ( S, W, s ) = V i ∩ W . Furthermore, since | V i | ≥ s max / 4 , we have | V i ∩ W | ≥ s max 2 . 2 ≥ s 0 . 48 2 . 2 ≥ s/ 6 .

We then use T 1 as a plural set to recover V i ∩ U so that we are able to recover all the vertices of V i , but now T 1 and U are not completely independent and thus we cannot proceed simply as before.

We overcome this by an union bound based argument. Let's consider T ′ i = V i ∩ W for any i such that T ′ ≥ s/ 6 . Then we have the following facts.

Let u ∈ U ∩ V i . Then E[ N T ′ i ( u )] = p | T ′ | . Then Lemma B.1 shows that Pr ( N T ( u ) ≤ q | T ′ | +0 . 99( p -q ) | T ′ | ) ≤ n -10 .

Similarly, let u ∈ U ∩ V j . Then Pr ( N T ′ i ( u ) ≥ q | T ′ | +0 . 01( p -q ) | T ′ | ) ≤ n -10 .

If either of this is true for a vertex u ∈ U then we call it a bad vertex w.r.t T ′ i . Then a union bound over all V i and all u ∈ U gives us that no vertex u ∈ U is bad w.r.t any T ′ i with probability 1 -n -8 .

Then we can make this argument for T ′ i = T 1 . Since | V i | > s max / 4 , we have | V i ∩ W | ≥ V i / 3 with probability 1 -n -8 . Then with probability 1 -n -8 no vertex u ∈ U is bad w.r.t T 1 .

Then applying Lemma B.1 to T 1 w.r.t vertices in U we get, with probability 1 -O ( n -3 )

Thus IDENTIFYCLUSTER ( T 1 , U, s ) only selects the set of vertices T 2 in V i ∩ U . Then taking the union of T 1 and T 2 gives us V i .

Proof of Lemma 2.8. Since | V i | > 256 √ n log n , and every vertex of V i will be assigned to W with probability 1 / 2 , we have that | V i ∩ W | ≥ | V i | 2 . 5 ≥ s max 10 ≥ s 0 . 52 · 10 ≥ s 6 with probability 1 -n -8 . Furthermore if | V i | ≥ s max 4 , then Lemma 2.7 shows T 1 = V i ∩ W and | T 1 | ≥ s 6 .

Furthermore, for any vertex u ∈ T 1 ∩ { v } , we can calculate N T 1 ( u ) in the following way.

We have E[ N T 1 ( u )] = p | T 1 | . Then a simple application of Lemma B.1 give us that with probability 1 -n -8 , N T 1 ( u ) ≥ p | T 1 | -( p -q ) T 1 96 ≥ (0 . 9 p +0 . 1 q ) | T 1 | .

Similarly, since T 1 = V i ∩ W for any vertex u ∈ W ∩ T 1 , we have E[ N T 1 ( u )] = q | T 1 | and Lemma B.1 implies that with probability 1 -n -8

Proof of Corollary 2.9. If T 1 is a pure subset of some V i , such that | V i | ≤ s max / 7 , then with probability 1 -O ( n -8 ) , | Y 2 ∩ V i | ≤ s/ 6 . If | T 1 | < s 6 , the first condition is satisfied.

Otherwise if | T 1 | ≥ s 6 and T 1 is not a pure set, there exists V j such that | T 1 ∩ V j | ≤ | T 1 | 2 . In that case for any vertex v ∈ V j ∩ T 1 we have E[ N T 1 ( v )] ≤ q | T 1 | +( p -q ) T 1 2 and Lemma B.1 implies that with probability 1 -n -8 ,

glyph[negationslash]

Finally if T 1 ⊂ V i is a large pure set and T 1 = V i ∩ W , then for a vertex v ∈ V i ∩ ( W \ T 1 ) we have N T 1 ( v ) ≥ (0 . 9 p +0 . 1 q ) | T 1 | .

C An improved algorithm in the balanced case

Our algorithm is built upon [28] and [25]. However, even in the balanced case, our algorithm improves a result of [28] on partially recovering clusters in the SBM. More precisely, we can use Theorem 1.2 to prove the following theorem.

Theorem C.1. Let G = ( V, E ) be sampled from SBM( n, k, p, q ) for σ 2 = Ω(log n/n ) where size of each cluster is Ω( n/k ) . Then there exists a polynomial time algorithm that exactly recovers all clusters if ( p -q ) √ n k > C ′ σ √ k log n for some constant C ′ .

In [28] (see Lemma 1.4 therein), Vu gave an algorithm that partially recovers all the clusters in the sense that with probability at least 1 -ε , each cluster output by the algorithm contains at 1 -ε fraction of any one underlying communities, for any constant ε > 0 . For the balanced case, his result holds under the assumption that σ 2 > C log n/n , and ( p -q ) √ n k > Cσ √ k . In comparison, we obtain a full recovery of all the clusters under Vu's partial recovery assumptions at the cost of an extra log n factor in the tradeoff of parameters.

Proof of Theorem C.1. We have ( p -q ) √ n/k > C ′ σ √ k log n . Let C ′ = 2 C . Since p ≤ 3 / 4 , we have 1 -p ≥ 1 / 4 and then σ ≥ √ p (1 -q ) 2 . Thus ( p -q ) √ n/k > C √ p (1 -q ) √ k log n . This implies k < ( p -q ) √ n C √ p (1 -q ) log n and n/k > C · √ p (1 -q ) √ n · log n p -q . That is the size of each cluster is at least s ∗ . Then we can run Algorithm 1 to recover one such cluster. Now, since the size of each cluster is same, we can run this iteratively k times, recovering a cluster at each round with probability 1 - O ( n -2 ) . Using union bound we get that we are able to recover all clusters with probability 1 -O ( kn -2 ) = 1 -O ( n -1 ) .

D Lower bounds

First, we show that our algorithm is optimal up to logarithmic factors when p and q are constant. To do so, we make use of the well-known planted clique conjecture.

Conjecture D.1 (Planted clique hardness) . Given an Erd˝ os-Rényi random graph G ( n, q ) with q = 1 / 2 , if we plant in G ( n, q ) a clique of size t where t ∈ [3 · log n, o ( √ n )] , then there exists no polynomial time algorithm to recover the largest clique in this planted model.

Under the planted clique conjecture, we note that there is no polynomial time algorithm for the SBM problem that recovers clusters of size o ( √ n ) irrespective of the number k of clusters present in the graph, for any constants p and q . This can be seen by defining the partition of V as V = ∪ k i =1 V i , where V 1 is a clique of size t = o ( √ n ) , and V 2 , · · · , V k are singleton vertices, k = n -t . Finally, let p = 1 , q = 1 2 . Then an algorithm for finding a cluster of size o ( √ n ) in a graph G that is sampled from the SBM with the above partition solves the planted clique problem.

Thus, the dependency of our algorithm in Theorem 1.2 on n is optimal under the planted clique conjecture up to logarithmic factors.

The following result was given in [23], we give a proof here for the sake of completeness.

Theorem D.2 ([23]) . Let A be a polynomial time algorithm in the faulty oracle model with parameters n, k, δ . Suppose that A finds a cluster of size t irrespective of the value of k . Then under the planted clique conjecture, it holds that t = Ω( √ n ) .

Proof. Let G be a graph generated from the planted clique problem with parameter t . Note that each potential edge in the sizet clique, say K , appears with probability 1 , and each of the remaining potential edges appear with probability 1 2 . Now we delete each edge in G with probability 1 3 . Then the resulting graph can be viewed as an instance generated from the faulty oracle model with parameters n , k = n -t +1 and δ = 1 3 : there are k clusters, one being H , and n -t clusters being singleton vertices. Furthermore, each intra-cluster edge is removed with probability 1 3 and each inter-cluster is added with probability 1 2 · (1 -1 3 ) = 1 3 . If there is a polynomial time algorithm that recovers the cluster H , no matter how many queries it performs, then it also solves the planted clique problem with clique size t . Under the planted clique conjecture, t = Ω( √ n ) .

E High-level ideas of the algorithm for the faulty oracle

Discussion about the previous algorithm in the faulty oracle model One crucial limitation of all the previous polynomial-time algorithms that make sublinear number of queries is that they cannot recover large clusters, if there are at least ˜ Ω( n 2 / 5 ) small clusters. The reason is that the query complexities of all these algorithms are at least Ω( k 5 ) , and if there are ˜ Ω( n 2 / 5 ) small clusters, then k = ˜ Ω( n 2 / 5 ) , which further implies that these polynomial time algorithms have to make Ω( k 5 ) = Ω( n 2 ) queries.

Main ideas of our algorithm Now we apply our algorithm in the SBM to the faulty oracle model. Consider the faulty oracle model with and parameters n, k, δ . Assume that the oracle O outputs '+' to indicate the queried two vertices belong to the same cluster, and '-' otherwise.

Observe that if we make queries on all pairs u, v ∈ V , then the graph G that is obtained by adding all + edges answered by the oracle O is exactly the graph that is generated from the SBM( n, k, p, q ) with parameters n , k , p = 1 2 + δ 2 and q = 1 2 -δ 2 . However, the goal is to recover the clusters by making sublinear number of queries, i.e., without seeing the whole graph.

We now describe our algorithm NOISYCLUSTERING (i.e., Algorithm 5) for clustering with a faulty oracle. Let V be the items which contains k latent clusters V 1 , . . . , V k and O be the faulty oracle. Following the idea of [27], we first sample a subset T ⊆ V of appropriate size and query O ( u, v ) for all pairs u, v ∈ T . Then apply our SBM clustering algorithm (i.e. Algorithm 1 CLUSTER) on the graph induced by T to obtain clusters X 1 , . . . , X t for some t ≤ k . We can show that each of these sets is a subcluster of some large cluster V i . Then we can use a majority voting to find all other vertices that belong to X i , for each i ≤ t . That is, for each X i and v ∈ V , we check if the number of neighbors of v in X i is at least | X i | 2 . In this way, we can identify all the large clusters V i corresponding to X i , 1 ≤ i ≤ t . Furthermore, we note that we can choose a small subset of X i of size O ( log n δ 2 ) for majority voting to reduce query complexity. Then we can remove all the vertices in V i 's and remove all the edges incident to them from both V and T and then we can use the remaining subsets T and V and corresponding subgraphs to find the next sets of large clusters. The algorithm NOISYCLUSTERING then recursively find all the large clusters until we reach a point where the recovery condition on the current graph no longer holds.

F The algorithm in faulty oracle model

Now we turn to the faulty oracle model and give the corresponding algorithm Algorithm 5.

To analyze the algorithm NOISYCLUSTERING (i.e., Algorithm 5), we first describe two results.

Lemma F.1. Let | V | = n and V i ⊂ V : | V i | = s ≥ C √ n · log 2 n δ for some constant C > 1 . If a set T ⊂ V of size 16 C 2 n 2 log n δ 2 s 2 is sampled randomly, then with probability 1 -n -8 , we have | T ∩ V i | ≥ C √ | T | log | T | 4 δ ≥ C log n δ 2 .

Algorithm 5 NOISYCLUSTERING ( V, δ, s ) : recover all clusters of size more than s ≥ s ∗

1: V ′ ← V ; t ′ ← 0

2: Randomly sample a subset T ⊂ V ′ of size | T | = C 2 n 2 log 2 n s 2 δ 2

3: Query all pairs u, v ∈ T and let G [ T ] be graph on vertex set T with only positive edges from the query answers

4: for each glyph[lscript] from 1 to glyph[floorleft] n/s glyph[floorright] do

5: Apply CLUSTER ( G [ T ] , 1 2 + δ, 1 2 -δ ) to obtain a cluster T glyph[lscript]

6: if T glyph[lscript] = ∅ then

7: continue

8: else

9: t ′ ← t ′ +1

11: C ′ t ′ ←{ v ∈ V ′ \ T : N T ′ glyph[lscript] ( v ) ≥ | T ′ glyph[lscript] | / 2 }

10: Find an arbitrary subset T ′ glyph[lscript] ⊆ T glyph[lscript] of size 4 log n δ 2

12: C t ′ ← T glyph[lscript] ∩ C ′ t ′

13: T ← T \ T glyph[lscript] .

14: V ′ ← V \ C t ′

15: Return C 1 , · · · , C t ′

Proof. We use Hoeffding bound to obtain these bounds. We have | T | ≥ 16 C 2 n 2 log 2 n δ 2 s 2 ≥ 16 log 2 n . For every vertex u ∈ T , we define X u as the indicator random variable which is 1 if u ∈ V i .

Then E[ X u ] = | V i | / | V | . Thus applying Hoeffding bound we get

Now, substituting value of | T | we get 0 . 5 ·| T || V i | | V | ≥ 8 · C 2 · n 2 log 2 n · s s 2 · δ 2 · n ≥ 4 C · n · log n s · δ · C · log n δ ≥ C · √ | T |· log n δ ≥ C √ | T |· log | T | δ . Furthermore, the last equation shows 0 . 5 ·| T || V i | | V | ≥ C √ | T |· log | T | δ ≥ Cn log n s · δ · δ ≥ C log n δ 2 . Now the proof follows by noting that | T ∩ V i | = ∑ u ∈ T X u .

glyph[negationslash]

Lemma F.2. Let V be partitioned into two sets U and W , where each vertex v ∈ V is independently assigned to either set with equal probability . Let S ⊂ V i ∩ U be a set such that | S | ≥ 4 log n δ 2 . Then with probability 1 -O ( n -8 ) , we have N S ( u ) ≥ | S | 2 for all u ∈ V i ∩ W , and N S ( u ) < | S | 2 for all u ∈ V j ∩ W for any j = i .

Proof. Let u ∈ V i ∩ W . Then E[ N S ( u )] = (0 . 5 + δ ) · | S | . Then

The last inequality holds | S | ≥ 4 log n/δ 2 . Thus if u ∈ V i ∩ W then N S ( u ) ≥ 0 . 5 | S | with probability 1 -n -8 .

Similarly, if u / ∈ V i , then with probability 1 -n -8 we have N S ( u ) ≤ 0 . 5 | S | .

F.1 Proof of Theorem 1.6

Given s , first we randomly sample n ′ = C 2 n 2 log 2 n s 2 δ 2 many vertices from V , and denote this set as T .

Then Lemma F.2 proves that for any cluster V i : | V i | ≥ s ∗ , we have | T i | = | T ∩ V i | ≥ C √ n ′ log n ′ δ with probability 1 -n -8 . For any underlying cluster V i , we denote T i = T ∩ V i .

Next we query all the pair of edges for vertices in T , which amounts O ( n 4 log 2 n δ 4 s 4 ) queries. The resultant graph G ′ is an SBM graph on n ′ vertices with p = 0 . 5 + δ and q = 0 , . 5 -δ .

Thus, if we run Algorithm 1 with parameters G ′ , 0 . 5 + δ, 0 . 5 -δ , then Theorem 1.2 implies that we recover a cluster T i such that | T i | ≥ Cn ′ log n ′ δ with probability 1 -n -2 .

Once we get such a set T i , we can take 4 log n/δ 2 many vertices from it, calling it a set S . Then for every vertex v ∈ V \ T , we obtain N S ( v ) , which requires | S | many queries, and select all vertices such that N S ( u ) ≥ 0 . 5 | S | . Lemma F.1 shows that we recover V i ∩ ( V \ T ) with probability 1 -n -8 , together recovering V i . Thus this step requires 4 n log n/δ 2 queries for each iteration.

Once we have recovered V i , we can then remove T i from T and run Algorithm 1 again on the residual graph, followed by the sample-and recovery step of Line 10. Note that once we remove a recovered cluster, all sets T j that satisfied the recovery requirement of Theorem 1.2 in the graph G ′ defined on T , also satisfies it on the graph G ′′ defined on T \ T i , and we do not need to sample any more edges.

Finally, there are at most δ 2 √ T many clusters T i ∈ T such that | T i | ≥ √ | T | log | T | /δ 2 . Here we have δ 2 √ T = Cn log n s . This upper bounds the number of iterations and thus the number of times the voting system on Line 10 is applied.

Thus the query complexity is O ( n 4 log 2 n δ 4 s 4 + n log n s · 4 n log n δ 2 ) = O ( n 4 log 2 n s 4 · δ 4 + n 2 log 2 n s · δ 2 ) . This finishes the proof of Theorem 1.6.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a classic problem of recovering clusters in a random graph. Concretely, the authors consider the stochastic block model. Here there is an underlying graph on n nodes. The n nodes are partitioned into k unknown clusters. There is then an edge independently between any two nodes in the same cluster with probability p and between any two in different clusters with probability q < p.\n\nThis is an extensively studied problem and many algorithms have been designed that allow the recovery of all clusters of a reasonable size (somewhat larger than sqrt(n), which is anyway a requirement for computational efficiency under the planted clique conjecture). The previous state of the art allows recovering clusters under two assumptions (here simplified for clarity and brevity):\n\n1. The clusters to recover have size at least max(sqrt(n), k)/(p-q).\n2. There is a number alpha of about sqrt(n)/(p-q) such that no cluster has size in the interval [alpha/C, alpha] for a constant C.\n\nThe assumption that the cluster sizes are at least sqrt(n) for those to be recovered is natural as mentioned above. However, the dependency on k is unfortunate when there are many small clusters. These would prevent the recovery of medium sizes clusters when k >> sqrt(n). Secondly, the assumption about the empty interval is quite unnatural.\n\nThe main contribution of this work is to remove the dependency on k in 1. and to remove the assumption 2. all together.\n\nThe authors also present applications of their algorithm in the related problem of clustering with a faulty oracle. Here one can ask whether two nodes v, w are in the same cluster or not. One is then returned a noise answer. Here the paper also improves over the state of the art in terms of the cardinality of clusters that can be recovered.\n\nSTRENGTHS:\n-The problem studied is fundamental in graph clustering.\n-Removing the dependency on k and the requirement of an empty interval of cluster sizes is significant and the algorithm guarantees of the algorithm much more natural than previously\n-The authors have implemented their algorithm and compared experimentally to previous work. The comparison is overall in favour of the new algorithm.\n\nWEAKNESSES:\n-I know this is a theoretical contribution, and also the authors probably did not attempt to optimize constants that much, but a factor 2^13 in the guarantees is quite severe in practice. Hopefully and probably, this constant is smaller in practice.\n\nQUESTIONS:\n-Could you say a bit about the running time of your algorithm in practice compared to previous work?\n-Can you comment on whether the 2^13 constant can be reduced to a more reasonable constant without too much effort?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies stochastic block models where blocks/clusters can have different sizes. It proposed a simple SVD algorithm which recovers communities in this setting. The main technical improvement of this work is that the assumption is removed which requires there to be a ‘size interval’ where no clusters appear. \nA secondary result is a efficient clustering algorithm with sublinear query complexity.\n\nSTRENGTHS:\n-\tThis work is a clear improvement over the previous state-of-the-art. As I understand it, a key technical contribution of this work that might influence future work is instead of finding $k$ clusters as is done using the SVD approach, the algorithm first aims to find large clusters one-by-one. Although these are not perfect (they form a so-called plural set), using some non-trivial techniques perfect recovery can be obtained. \n- Experiments on synthetic data indicate that the algorithm not only works well in theory but also in practice.\n- The write-up of this work is excellent.\n\nWEAKNESSES:\n-\tGiven that the aim of the studied setting is to look at more realistic settings, I would have expected to find experimental results on real-world datasets as well. Although this work does provide better bounds for SBMs generated with differently sized clusters, SBMs still have a highly symmetric structure compared to real-world graphs. It would be interesting to see the performance of the proposed algorithm on some real-world graphs.\n\nQUESTIONS:\n-\tHow does the algorithm compare with respect to the previous work in terms of running time?\n- In practice the Spectral Clustering algorithm performs well in practice on graphs with clusters of unbalanced size. Even though not many bounds are known of spectral clustering with respect to SBMs, did you try to compare your algorithm experimentally with Spectral Clustering?\n\nLIMITATIONS:\nnone", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

comments: SUMMARY:
The authors consider the problem of perfect recovery in a stochastic block model where the average degree is large and where the groups are not balanced. They provide an algorithm based on singular value decomposition to recover recursively the largest clusters. They provide a few numerical experiments illustrating their claims. The authors apply their results to the problem of clustering with a faulty oracle.

STRENGTHS:
I have little knowledge as to this problem of perfect recovery in a dense SBM and I am not able to assess the correctness of the claims and their relevance.

WEAKNESSES:
The same.

QUESTIONS:
Maybe the authors could precise the complexity of the algorithm 1. In experiment 6 it seems the authors are able to run this algorithm for n substantially larger than the other experiments. The authors could go to higher n and test how tight are their bounds; in particular taking p and q smaller.

A small section to conclude the article and for future work would be appreciable.

Some references are ill-formatted. Eg ref. 27 "svd" –> "SVD".
Inconsistency: plural set vs plural-set.

LIMITATIONS:
The same.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper deals with the problem of community detection for unbalanced community sizes. Specifically, the paper concentrates on the situation where both large (O(\\sqrt{n})) and small communities exist in the network. The paper proposes a stepwise method of recovering the large clusters in the presence of small clusters for planted clique SBM and faulty oracle models.\n\nSTRENGTHS:\nThe main strengths of the paper are as follows - \n\n(1) The paper addresses a gap in the literature on the simultaneous recovery of large and small communities in networks.\n\n(2) The paper deals with the problem of community recovery of large communities in the presence of small communities. The paper provides a stepwise method of recovering large communities in planted clique SBM and faulty oracle models.\n\n(3) The paper provides theoretical results supporting the recovery of large communities by overcoming the \"small cluster barrier\" of the size of the remaining small clusters.\n\n(4) The paper is well-written.\n\nWEAKNESSES:\nThe main weaknesses of the paper are as follows - \n\n(1) The paper misses some relevant literature. Such as - Li, Tianxi, et.al. \"Hierarchical community detection by recursive partitioning.\" Journal of the American Statistical Association 117, no. 538 (2022): 951-968. It describes an algorithm that is very similar to the algorithm proposed in this work.\n\n(2) Algorithms 2 and 3 assumes the knowledge of p and q, which are very strong assumptions. It is not immediately clear how the algorithm can be extended for general SBM.\n\n(3) The stopping criterion of the proposed algorithm is not clear.\n\nQUESTIONS:\n(1) Does the proposed algorithm assume the knowledge of p and q?\n\n(2) Does the proposed algorithm assume the knowledge of the number of communities, or is there a stopping criteria of the proposed algorithm for recovery of the number of large communities?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}