Title: A Sublinear-Time Spectral Clustering Oracle with Improved Preprocessing Time
Venue: NeurIPS 2023
Accepted: yes
Keywords: Sublinear-time algorithms

Spectral Clustering

Graph Clustering

Random Walks
Abstract: We address the problem of designing a sublinear-time spectral clustering oracle for graphs that exhibit strong clusterability. Such graphs contain $k$ latent clusters, each characterized by a large inner conductance (at least $\varphi$) and a small outer conductance (at most $\varepsilon$). Our aim is to preprocess the graph to enable clustering membership queries, with the key requirement that both preprocessing and query answering should be performed in sublinear time, and the resulting partition should be consistent with a $k$-partition that is close to the ground-truth clustering. Previous oracles have relied on either a $\textrm{poly}(k)\log n$ gap between inner and outer conductances or exponential (in $k/\varepsilon$) preprocessing time. Our algorithm relaxes these assumptions, albeit at the cost of a slightly higher misclassification ratio. We also show that our clustering oracle is robust against a few random edge deletions. To validate our theoretical bounds, we conducted experiments on synthetic networks.

A Sublinear-Time Spectral Clustering Oracle with Improved Preprocessing Time

Abstract

We address the problem of designing a sublinear-time spectral clustering oracle for graphs that exhibit strong clusterability. Such graphs contain k latent clusters, each characterized by a large inner conductance (at least φ ) and a small outer conductance (at most ε ). Our aim is to preprocess the graph to enable clustering membership queries, with the key requirement that both preprocessing and query answering should be performed in sublinear time, and the resulting partition should be consistent with a k -partition that is close to the ground-truth clustering. Previous oracles have relied on either a poly ( k ) log n gap between inner and outer conductances or exponential (in k/ε ) preprocessing time. Our algorithm relaxes these assumptions, albeit at the cost of a slightly higher misclassification ratio. We also show that our clustering oracle is robust against a few random edge deletions. To validate our theoretical bounds, we conducted experiments on synthetic networks.

1 Introduction

Graph clustering is a fundamental task in the field of graph analysis. Given a graph G = ( V, E ) and an integer k , the objective of graph clustering is to partition the vertex set V into k disjoint clusters C 1 , . . . , C k . Each cluster should exhibit tight connections within the cluster while maintaining loose connections with the other clusters. This task finds applications in various domains, including community detection [32, 12], image segmentation [11] and bio-informatics [29].

However, global graph clustering algorithms, such as spectral clustering [27], modularity maximization [26], density-based clustering [10], can be computationally expensive, especially for large datasets. For instance, spectral clustering is a significant algorithm for solving the graph clustering problem, which involves two steps. The first step is to map all the vertices to a k -dimensional Euclidean space using the Laplacian matrix of the graph. The second step is to cluster all the points in this k -dimensional Euclidean space, often employing the k -means algorithm. The time complexity of spectral clustering, as well as other global clustering algorithms, is poly ( n ) , where n = | V | denotes the size of the graph. As the graph size increases, the computational demands of these global clustering algorithms become impractical.

Addressing this challenge, an effective approach lies in the utilization of local algorithms that operate within sublinear time. In this paper, our primary focus is on a particular category of such algorithms designed for graph clustering, known as sublinear-time spectral clustering oracles [30, 14]. These algorithms consist of two phases: the preprocessing phase and the query phase, both of which can be executed in sublinear time. During the preprocessing phase, these algorithms sample a subset of vertices from V , enabling them to locally explore a small portion of the graph and gain insights into its cluster structure. In the query phase, these algorithms utilize the cluster structure learned during the preprocessing phase to respond to WHICHCLUSTER( G,x ) queries. The resulting partition

defined by the output of WHICHCLUSTER( G,x ) should be consistent with a k -partition that is close to the ground-truth clustering.

We study such oracles for graphs that exhibit strong clusterability, which are graphs that contain k latent clusters, each characterized by a large inner conductance (at least φ ) and a small outer conductance (at most ε ). Let us assume φ > 0 is some constant. In [30] (see also [8]), a robust clustering oracle was designed with preprocessing time approximately O ( √ n · poly ( k log n ε )) , query time approximately O ( √ n · poly ( k log n ε )) , misclassification error (i.e., the number of vertices that are misclassified with respect to a ground-truth clustering) approximately O ( kn √ ε ) . The oracle relied on a poly ( k ) log n gap between inner and outer conductance. In [14], a clustering oracle was designed with preprocessing time approximately 2 poly ( k ε ) poly (log n ) · n 1 / 2+ O ( ε ) , query time approximately poly ( k log n ε ) · n 1 / 2+ O ( ε ) , misclassification error O (log k · ε ) | C i | for each cluster C i , i ∈ [ k ] and it takes approximately O (poly( k ε ) · n 1 / 2+ O ( ε ) · poly(log n )) space. This oracle relied on a log k gap between inner and outer conductance.

One of our key contributions in this research is a new sublinear-time spectral clustering oracle that offers enhanced preprocessing efficiency. Specifically, we introduce an oracle that significantly reduces both the preprocessing and query time, performing in poly ( k log n ) · n 1 / 2+ O ( ε ) time and reduces the space complexity, taking O (poly( k ) · n 1 / 2+ O ( ε ) · poly(log n )) space. This approach relies on a poly ( k ) gap between the inner and outer conductances, while maintaining a misclassification error of O ( poly ( k ) · ε 1 / 3 ) | C i | for each cluster C i , i ∈ [ k ] . Moreover, our oracle offers practical implementation feasibility, making it well-suited for real-world applications. In contrast, the clustering oracle proposed in [14] presents challenges in terms of implementation (mainly due to the exponential dependency on k/ε ).

We also investigate the sensitivity of our clustering oracle to edge perturbations. This analysis holds significance in various practical scenarios where the input graph may be unreliable due to factors such as privacy concerns, adversarial attacks, or random noises [31]. We demonstrate the robustness of our clustering oracle by showing that it can accurately identify the underlying clusters in the resulting graph even after the random deletion of one or a few edges from a well-clusterable graph.

Basic definitions. Graph clustering problems often rely on conductance as a metric to assess the quality of a cluster. Several recent studies ([8, 9, 30, 14, 22]) have employed conductance in their investigations. Hence, in this paper, we adopt the same definition to characterize the cluster quality. We state our results for d -regular graphs for some constant d ≥ 3 , though they can be easily generalized to graphs with maximum degree at most d (see Appendix B).

Definition 1.1 (Inner and outer conductance) . Let G = ( V, E ) be a d -regular n -vertex graph. For a set S ⊆ C ⊆ V , we let E ( S, C \ S ) denote the set of edges with one endpoint in S and the other endpoint in C \ S . The outer conductance of a set C is defined to be ϕ out ( C, V ) = | E ( C,V \ C ) | d | C | . The inner conductance of a set C is defined to be ϕ in ( C ) = min S ⊆ C, 0 < | S |≤ | C | 2 ϕ out ( S, C ) =

min S ⊆ C, 0 < | S |≤ | C | 2 | E ( S,C \ S ) | d | S | if | C | > 1 and one otherwise. Specially, the conductance of graph G is

Note that based on the above definition, for a cluster C , the smaller the ϕ out ( C, V ) is, the more loosely connected with the other clusters and the bigger the ϕ in ( C ) is, the more tightly connected within C . For a high quality cluster C , we have ϕ out ( C, V ) ≪ ϕ in ( C ) ≤ 1 .

Definition 1.2 ( k -partition) . Let G = ( V, E ) be a graph. A k -partition of V is a collection of disjoint subsets C 1 , . . . , C k such that ∪ k i =1 C i = V .

Based on the above, we have the following definition of clusterable graphs.

Definition 1.3 ( ( k, φ, ε ) -clustering) . Let G = ( V, E ) be a d -regular graph. A ( k, φ, ε ) -clustering of G is a k -partition of V , denoted by C 1 , . . . , C k , such that for all i ∈ [ k ] , ϕ in ( C i ) ≥ φ , ϕ out ( C i , V ) ≤ ε and for all i, j ∈ [ k ] one has | C i | | C j | ∈ O (1) . G is called a ( k, φ, ε ) -clusterable graph if there exists a ( k, φ, ε ) -clustering of G .

Main results. Our main contribution is a sublinear-time spectral clustering oracle with improved preprocessing time for d -regular ( k, φ, ε ) -clusterable graphs. We assume query access to the adjacency list of a graph G , that is, one can query the i -th neighbor of any vertex in constant time.

Theorem 1. Let k ≥ 2 be an integer, φ ∈ (0 , 1) . Let G = ( V, E ) be a d -regular n -vertex graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k , ε φ 2 ≪ γ 3 k 9 2 · log 3 k and for all i ∈ [ k ] , γ n k ≤ | C i | ≤ n γk , where γ is a constant that is in (0 . 001 , 1] . There exists an algorithm that has query access to the adjacency list of G and constructs a clustering oracle in O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γφ )) preprocessing time and takes O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γ )) space. Furthermore, with probability at least 0 . 95 , the following hold:

Using the oracle, the algorithm can answer any WHICHCLUSTER query in O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γφ )) time and a WHICHCLUSTER query takes O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γ )) space.

Let U i := { x ∈ V : WHICHCLUSTER ( G,x ) = i } , i ∈ [ k ] be the clusters recovered by the algorithm. There exists a permutation π : [ k ] → [ k ] such that for all i ∈ [ k ] , | U π ( i ) △ C i | ≤ O ( k 3 2 γ · ( ε φ 2 ) 1 / 3 ) | C i | .

Specifically, for every graph G = ( V, E ) that admits a k -partition C 1 , . . . , C k with constant inner conductance φ and outer conductance ε ≪ O ( 1 poly ( k ) ) , our oracle has preprocessing time ≈ n 1 / 2+ O ( ε ) · poly ( k log n ) , query time ≈ n 1 / 2+ O ( ε ) · poly ( k log n ) , space ≈ O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n )) and misclassification error ≈ O ( poly ( k ) · ε 1 / 3 ) | C i | for each cluster C i , i ∈ [ k ] . In comparison to [30], our oracle relies on a smaller gap between inner and outer conductance (specifically O ( poly ( k ) log n ) ). In comparison to [14], our oracle has a smaller preprocessing time and a smaller space at the expense of a slightly higher misclassification error of O ( poly ( k ) · ε 1 / 3 ) | C i | instead of O (log k · ε ) | C i | and a slightly worse conductance gap of ε ≪ O ( φ 2 / poly( k )) instead of ε ≪ O ( φ 3 / log( k )) . It's worth highlighting that our space complexity significantly outperforms that of [14] (i.e., O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k ε · log n )) ), particularly in cases where k is fixed and ε takes on exceptionally small values, such as ε = 1 n c for sufficiently small constant c > 0 , since the second term in our space complexity does not depend on ε in comparison to the one in [14].

Another contribution of our work is the verification of the robustness of our oracle against the deletion of one or a few random edges. The main idea underlying the proof is that a well-clusterable graph is still well-clusterable (with a slightly worse clustering quality) after removing a few random edges, which in turn is built upon the intuition that after removing a few random edges, an expander graph remains an expander. For the complete statement and proof of this claim, we refer to Appendix E.

Theorem 2 (Informal; Robust against random edge deletions) . Let c > 0 be a constant. Let G 0 be a graph satisfying the similar conditions as stated in Theorem 1. Let G be a graph obtained from G 0 by randomly deleting c edges. Then there exists a clustering oracle for G with the same guarantees as presented in Theorem 1.

1.1 Related work

Sublinear-time algorithms for graph clustering have been extensively researched. Czumaj et al. [8] proposed a property testing algorithm capable of determining whether a graph is k -clusterable or significantly far from being k -clusterable in sublinear time. This algorithm, which can be adapted to a sublinear-time clustering oracle, was later extended by Peng [30] to handle graphs with noisy partial information through a robust clustering oracle. Subsequent improvements to both the testing algorithm and the oracle were introduced by Chiplunkar et al. [6] and Gluchowski et al. [14]. Recently, Kapralov et al. [16, 17] presented a hierarchical clustering oracle specifically designed for graphs exhibiting a pronounced hierarchical structure. This oracle offers query access to a highquality hierarchical clustering at a cost of poly ( k ) · n 1 / 2+ O ( γ ) per query. However, it is important to note that their algorithm does not provide an oracle for flat k -clustering, as considered in our work, with the same query complexity. Sublinear-time clustering oracles for signed graphs have also been studied recently [25].

The field of local graph clustering [33, 1, 3, 2, 34, 28] is also closely related to our research. In this framework, the objective is to identify a cluster starting from a given vertex within a running time

that is bounded by the size of the output set, with a weak dependence on n . Zhu et al. [34] proposed a local clustering algorithm that produces a set with low conductance when both inner and outer conductance are used as measures of cluster quality. It is worth noting that the running times of these algorithms are sublinear only if the target set's size (or volume) is small, for example, at most o ( n ) . In contrast, in our setting, the clusters of interest have a minimum size that is Ω( n/k ) .

Extensive research has been conducted on fully or partially recovering clusters in the presence of noise within the 'global algorithm regimes'. Examples include recovering the planted partition in the stochastic block model with modeling errors or noise [4, 15, 24, 21], correlation clustering on different ground-truth graphs in the semi-random model [23, 5, 13, 20], and graph partitioning in the average-case model [18, 19, 20]. It is important to note that all these algorithms require at least linear time to run.

2 Preliminaries

Let G = ( V, E ) denote a d -regular undirected and unweighted graph, where V := { 1 , . . . , n } . Throughout the paper, we use i ∈ [ n ] to denote 1 ≤ i ≤ n . For any two vectors x , y ∈ R n , we let ⟨ x , y ⟩ = x T y denote the dot product of x and y . For a graph G , we let A ∈ R n × n denote the adjacency matrix of G and let D ∈ R n × n denote a diagonal matrix with D ( i, i ) = deg( i ) , where deg( i ) is the degree of vertex i, i ∈ [ n ] . We denote with L the normalized Laplacian of G where L = D -1 / 2 ( D -A ) D -1 / 2 = I -A d . For L , we use 0 ≤ λ 1 ≤ ... ≤ λ n ≤ 2 [7] to denote its eigenvalues and we use u 1 , . . . , u n ∈ R n to denote the corresponding eigenvectors. Note that the corresponding eigenvectors are not unique, in this paper, we let u 1 , . . . , u n be an orthonormal basis of eigenvectors of L . For any two sets S 1 and S 2 , we let S 1 △ S 2 denote the symmetric difference between S 1 and S 2 .

Due to space constraints, we present only the key preliminaries here; the complete preliminaries will be presented in Appendix A.

Our algorithms in this paper are based on the properties of the dot product of spectral embeddings, so we also need the following definition.

Definition 2.1 (Spectral embedding) . For a graph G = ( V, E ) with n = | V | and an integer 2 ≤ k ≤ n , we use L denote the normalized Laplacian of G . Let U [ k ] ∈ R n × k denote the matrix of the bottom k eigenvectors of L . Then for every x ∈ V , the spectral embedding of x , denoted by f x ∈ R k , is the x -row of U [ k ] , which means f x ( i ) = u i ( x ) , i ∈ [ k ] .

Definition 2.2 (Cluster centers) . Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . The cluster center µ i of C i is defined to be µ i = 1 | C i | ∑ x ∈ C i f x , i ∈ [ k ] .

The following informal lemma shows that the dot product of two spectral embeddings can be approximated in ˜ O ( n 1 / 2+ O ( ε/φ 2 ) · poly ( k )) time.

Lemma 2.1 (Informal; Theorem 2, [14]) . Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k , ε ≤ φ 2 10 5 . Then for every pair of vertices x, y ∈ V , SPECTRALDOTPRODUCT ( G,x,y, 1 / 2 , ξ, D ) computes an output value ⟨ f x , f y ⟩ apx in ˜ O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k )) time such that with high probability: ∣ ∣ ∣ ⟨ f x , f y ⟩ apx -⟨ f x , f y ⟩ ∣ ∣ ∣ ≤ ξ n .

For the completeness of this paper, we will show the formal Lemma 2.1 and algorithm SPECTRALDOTPRODUCT( G,x,y, 1 / 2 , ξ, D ) in Appendix C.

3 Spectral clustering oracle

3.1 Our techniques

We begin by outlining the main concepts of the spectral clustering oracle presented in [14]. Firstly, the authors introduce a sublinear time oracle that provides dot product access to the spectral embedding of graph G by estimating distributions of short random walks originating from vertices in G (as described in Lemma 2.1). Subsequently, they demonstrate that (1) the set of points corresponding to the spectral embeddings of all vertices exhibits well-concentrated clustering around the cluster center µ i (refer to Definition 2.2), and (2) all the cluster centers are approximately orthogonal to each other. The clustering oracle in [14] operates as follows: it initially guesses the k cluster centers from a set of

0.06

0.04

0.02

0.00

0.02

Z

0.04

0.06

↑0.08

↑0.10

0.08

0.06

0.04

0.02

0.0650

57900-

0090°0-

0.0575

X

0.0550

0.0525

0.0500

0.0475

0.08

0.06

0.04

0.02 y

0.00

poly ( k/ε ) sampled vertices, which requires a time complexity of 2 poly ( k/ε ) n 1 / 2+ O ( ε ) . Subsequently, it iteratively employs the dot product oracle to estimate ⟨ f x , µ i ⟩ . If the value of ⟨ f x , µ i ⟩ is significant, it allows them to infer that vertex x likely belongs to cluster C i .

Now we present our algorithm, which builds upon the dot product oracle in [14]. Our main insight is to avoid relying directly on cluster centers in our algorithm. By doing so, we can eliminate the need to guess cluster centers and consequently remove the exponential time required in the preprocessing phase described in [14]. The underlying intuition is as follows: if two vertices, x and y , belong to the same cluster C i , their corresponding spectral embeddings f x and f y will be close to the cluster center µ i . As a result, the angle between f x and f y will be small, and the dot product ⟨ f x , f y ⟩ will be large (roughly on the order of O ( k n ) ). Conversely, if x and y belong to different clusters, their embeddings f x and f y will tend to be orthogonal, resulting in a small dot product ⟨ f x , f y ⟩ (close to 0). We prove that this desirable property holds for the majority of vertices in d -regular ( k, φ, ε ) -clusterable graphs (see Figure 1 for an illustrative example). Slightly more formally, we introduce the definitions of good and bad vertices (refer to Definition 3.1) such that the set of good vertices corresponds to the core part of clusters and each pair of good vertices satisfies the aforementioned property; the rest vertices are the bad vertices. Leveraging this property, we can directly utilize the dot product of spectral embeddings to construct a sublinear clustering oracle.

Based on the desirable property discussed earlier, which holds for d -regular ( k, φ, ε ) -clusterable graphs, we can devise a sublinear spectral clustering oracle. Let G = ( V, E ) be a d -regular ( k, φ, ε ) -clusterable graph that possesses a ( k, φ, ε ) -clustering C 1 , . . . , C k . In the preprocessing phase, we sample a set S of s vertices from V and construct a similarity graph, denoted as H , on S . For each pair of vertices x, y ∈ S , we utilize the dot product oracle from [14] to estimate ⟨ f x , f y ⟩ . If x and y belong to the same cluster C i , yielding a large ⟨ f x , f y ⟩ , we add an edge ( x, y ) to H . Conversely, if x and y belong to different clusters, resulting in a ⟨ f x , f y ⟩ close to 0, we make no modifications to H . Consequently, only vertices within the same cluster C i ( i ∈ [ k ]) can be connected by edges. We can also establish that, by appropriately selecting s , the sampling set S will, with high probability, contain at least one vertex from each C 1 , . . . , C k . Thus, the similarity graph H will have k connected components, with each component corresponding to a cluster in the ground-truth. We utilize these k connected components, denoted as S 1 , . . . , S k , to represent C 1 , . . . , C k .

During the query phase, we determine whether the queried vertex x belongs to a connected component in H . Specifically, we estimate ⟨ f x , f y ⟩ for all y ∈ S . If there exists a unique index i ∈ [ k ] for which ⟨ f x , f u ⟩ is significant (approximately O ( k n ) ) for all u ∈ S i , we conclude that x belongs to cluster C i , associated with S i . If no such unique index is found, we assign x a random index i , where i ∈ [ k ] .

3.2 The clustering oracle

Next, we present our algorithms for constructing a spectral clustering oracle and handling the WHICHCLUSTER queries. In the preprocessing phase, the algorithm CONSTRUCTORACLE( G,k,φ,ε,γ ) learns the cluster structure representation of G . This involves constructing a similarity graph H on a sampled vertex set S and assigning membership labels ℓ to all vertices in S . During the query phase, the algorithm WHICHCLUSTER( G,x ) determines the clustering membership index to which vertex x belongs. More specifically, WHICHCLUSTER( G,x ) utilizes the function SEARCHINDEX( H,ℓ, x )

Algorithm 1: CONSTRUCTORACLE( G,k,φ,ε,γ )

to check whether the queried vertex x belongs to a unique connected component in H . If it does, SEARCHINDEX( H,ℓ, x ) will return the index of the unique connected component in H .

The algorithm in preprocessing phase is given in Algorithm 1 CONSTRUCTORACLE( G,k,φ,ε,γ ).

See Appendix C for algorithm INITIALIZEORACLE and SPECTRALDOTPRODUCTORACLE invoked by CONSTRUCTORACLE( G,k,φ,ε,γ ).

Our algorithms used in the query phase are described in Algorithm 2 SEARCHINDEX( H,ℓ, x ) and Algorithm 3 WHICHCLUSTER( G,x ).

Algorithm 2: SEARCHINDEX( H,ℓ, x )

Algorithm 3: WHICHCLUSTER( G,x

3.3 Analysis of the oracle

We now prove the following property: for most vertex pairs x, y , if x, y are in the same cluster, then ⟨ f x , f y ⟩ is rough O ( k n ) (Lemma 3.4); and if x, y are in the different clusters, then ⟨ f x , f y ⟩ is close to 0 (Lemma 3.5). We make use of the following three lemmas. Due to the limited space, all the missing proofs will be given in Appendix D.

The following lemma shows that for most vertices x , the norm ∥ f x ∥ 2 is small.

Lemma 3.1. Let α ∈ (0 , 1) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε ∈ (0 , 1) . Let G = ( V, E ) be a d -regular ( k, φ, ε ) -clusterable graph with | V | = n . There exists a subset ̂ V ⊆ V with | ̂ V | ≥ (1 -α ) | V | such that for all x ∈ ̂ V , it holds that ∥ f x ∥ 2 ≤ √ 1 α · k n .

We then show that for most vertices x , f x is close to its center µ x of the cluster containing x .

Lemma 3.2. Let β ∈ (0 , 1) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε ∈ (0 , 1) . Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k with | V | = n . There exists a subset ˜ V ⊆ V with | ˜ V | ≥ (1 -β ) | V | such that for all x ∈ ˜ V , it holds that ∥ f x -µ x ∥ 2 ≤ √ 4 kε βφ 2 · 1 n .

The next lemma shows that for most vertices x in a cluster C i , the inner product ⟨ f x , µ i ⟩ is large.

Lemma 3.3. Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Let C i denote the cluster corresponding to the center µ i , i ∈ [ k ] . Then for every C i , i ∈ [ k ] , there exists a subset ˜ C i ⊆ C i with | ˜ C i | ≥ (1 -10 4 ε φ 2 ) | C i | such that for all x ∈ ˜ C i , it holds that ⟨ f x , µ i ⟩ ≥ 0 . 96 ∥ µ i ∥ 2 2 .

For the sake of description, we introduce the following definition.

Definition 3.1 (Good and bad vertices) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular n -vertex graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . We call a vertex x ∈ V a good vertex with respect to α ∈ (0 , 1) and β ∈ (0 , 1) if x ∈ ( ̂ V ∩ ˜ V ∩ ( ∪ k i =1 ˜ C i )) , where ̂ V is the set as defined in Lemma 3.1, ˜ V is the set as defined in Lemma 3.2 and ˜ C i ( i ∈ [ k ]) is the set as defined in Lemma 3.3. We call a vertex x ∈ V a bad vertex with respect to α ∈ (0 , 1) and β ∈ (0 , 1) if it's not a good vertex with respect to α and β .

Note that for a good vertex x with respect to α ∈ (0 , 1) and β ∈ (0 , 1) , the following hold: (1) ∥ f x ∥ 2 ≤ √ 1 α · k n ; (2) ∥ f x -µ x ∥ 2 ≤ √ 4 kε βφ 2 · 1 n ; (3) ⟨ f x , µ x ⟩ ≥ 0 . 96 ∥ µ x ∥ 2 2 . For a bad vertex x with respect to α ∈ (0 , 1) and β ∈ (0 , 1) , it does not satisfy at least one of the above three conditions.

The following lemma shows that if vertex x and vertex y are in the same cluster and both of them are good vertices with respect to α and β ( α and β should be chosen appropriately), then the spectral dot product ⟨ f x , f y ⟩ is roughly 0 . 96 · 1 | C i | .

Lemma 3.4. Let k ≥ 2 , φ ∈ (0 , 1) and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular n -vertex graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Suppose that x, y ∈ V are in the same cluster C i , i ∈ [ k ] and both of them are good vertices with respect to α = 2 √ k · ( ε φ 2 ) 1 / 3 and β = 2 √ k · ( ε φ 2 ) 1 / 3 . Then ⟨ f x , f y ⟩ ≥ 0 . 96 ( 1 -4 √ ε φ ) 1 | C i | -√ k n · ( ε φ 2 ) 1 / 6 .

The following lemma shows that if vertex x and vertex y are in different clusters and both of them are good vertices with respect to α and β ( α and β should be chosen appropriately), then the spectral dot product ⟨ f x , f y ⟩ is close to 0 .

̸

Lemma 3.5. Let k ≥ 2 , φ ∈ (0 , 1) and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular n -vertex graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Suppose that x ∈ C i , y ∈ C j , ( i, j ∈ [ k ] , i = j ) and both of them are good vertices with respect to α = 2 √ k · ( ε φ 2 ) 1 / 3 and β = 2 √ k · ( ε φ 2 ) 1 / 3 , the following holds:

Proof of Theorem 1. Now we prove our main result Theorem 1.

Proof. Let s = 10 k log k γ be the size of sampling set S , let α = β = 2 √ k · ( ε φ 2 ) 1 / 3 . Recall that we call a vertex x a bad vertex, if x ∈ ( V \ ̂ V ) ∪ ( V \ ˜ V ) ∪ ( V \ ( ∪ k i =1 ˜ C i )) , where ̂ V , ˜ V , ˜ C i , i ∈ [ k ] are defined in Lemma 3.1, 3.2, 3.3 respectively. We use B to denote the set of all bad vertices. Then we have | B | ≤ ( α + β + 10 4 ε φ 2 ) · n = (4 √ k · ( ε φ 2 ) 1 / 3 + 10 4 ε φ 2 ) · n . We let κ ≤ 4 √ k · ( ε φ 2 ) 1 / 3 + 10 4 ε φ 2

be the fraction of B in V . Since ε φ 2 < γ 3 4 3 · 10 9 · k 9 2 · log 3 k , we have κ ≤ 4 √ k · ( ε φ 2 ) 1 / 3 + 10 4 ε φ 2 ≤ γ 10 3 k log k + γ 3 4 3 · 10 5 · k 9 2 log 3 k ≤ 2 γ 10 3 k log k = 1 50 s .

Therefore, by union bound, with probability at least 1 -κ · s ≥ 1 -1 50 s · s = 1 -1 50 , all the vertices in S are good (we fixed α = β = 2 √ k · ( ε φ 2 ) 1 / 3 , so we will omit 'with respect to α and β ' in the following). In the following, we will assume all the vertices in S are good.

1 -1 k 10 ≥ 1 -1 50 k , there exists at least one vertex in S that is from C i . Then with probability at least 1 -1 50 , for all k clusters C 1 , . . . , C k , there exists at least one vertex in S that is from C i .

Let ξ = √ γ 1000 . By Lemma 2.1, we know that with probability at least 1 -1 n 100 , for any pair of x, y ∈ V , SPECTRALDOTPRODUCTORACLE( G,x,y, 1 / 2 , ξ, D ) computes an output value ⟨ f x , f y ⟩ apx such that ∣ ∣ ∣ ⟨ f x , f y ⟩ apx - ⟨ f x , f y ⟩ ∣ ∣ ∣ ≤ ξ n . So, with probability at least 1 -s · s n 100 ≥ 1 -1 n 50 , for all pairs x, y ∈ S , SPECTRALDOTPRODUCTORACLE( G,x,y, 1 / 2 , ξ, D ) computes an output value ⟨ f x , f y ⟩ apx such that ∣ ∣ ∣ ⟨ f x , f y ⟩ apx - ⟨ f x , f y ⟩ ∣ ∣ ∣ ≤ ξ n . In the following, we will assume the above inequality holds for any x, y ∈ S .

By Lemma 3.4, we know that if x, y are in the same cluster and both of them are good vertices, then we have ⟨ f x , f y ⟩ ≥ 0 . 96(1 -4 √ ε φ ) 1 | C i | -√ k n · ( ε φ 2 ) 1 / 6 ≥ 0 . 96(1 -4 √ ε φ ) γk n -√ k n ( ε φ 2 ) 1 / 6 since | C i | ≤ n γk . By Lemma 3.5, we know that if x, y are in the different clusters and both of them are good vertices, then we have ⟨ f x , f y ⟩ ≤ √ k n · ( ε φ 2 ) 1 / 6 + √ 2 k 1 / 4 √ n · ( ε φ 2 ) 1 / 3 · √ (1 + 4 √ ε φ ) 1 | C j | + 8 √ ε φ · 1 √ | C i |·| C j | ≤ √ k n · ( ε φ 2 ) 1 / 6 + √ 2 γ k 3 / 4 n √ 1 + 4 √ ε φ · ( ε φ 2 ) 1 / 3 + 8 √ ε φ · k γn since γn k ≤ | C i | for all i ∈ [ k ] .

Recall that ε φ 2 < γ 3 4 3 · 10 9 · k 9 2 · log 3 k and γ ∈ (0 . 001 , 1] . Let θ = 0 . 96(1 -4 √ ε φ ) γk n -√ k n ( ε φ 2 ) 1 / 6 -ξ n , then we have θ > √ γ n · (0 . 96 √ γk -0 . 48 10 9 / 2 · k 5 / 4 log 3 / 2 k -1 2 · 10 3 / 2 · k 1 / 4 log 1 / 2 k -1 1000 ) > 0 . 034 · √ γ n . Let S satisfies that all the vertices in S are good, and S contains at least one vertex from C i for all i = 1 , . . . , k . For any x, y ∈ S , then:

If x, y belong to the same cluster, by above analysis, we know that ⟨ f x , f y ⟩ ≥ 0 . 96(1 -4 √ ε φ ) γk n -√ k n ( ε φ 2 ) 1 / 6 . Then it holds that ⟨ f x , f y ⟩ apx ≥ ⟨ f x , f y ⟩-ξ n ≥ 0 . 96(1 -4 √ ε φ ) γk n -√ k n ( ε φ 2 ) 1 / 6 -ξ n = θ . Thus, an edge ( x, y ) will be added to H (at lines 8 and 9 of Alg.1). √

If x, y belong to two different clusters, by above analysis, we know that ⟨ f x , f y ⟩ ≤ k n · ( ε φ 2 ) 1 / 6 + √ 2 γ k 3 / 4 n √ 1 + 4 √ ε φ · ( ε φ 2 ) 1 / 3 + 8 √ ε φ · k γn . Then it holds that ⟨ f x , f y ⟩ apx ≤ ⟨ f x , f y ⟩ + ξ n ≤ √ k n · ( ε φ 2 ) 1 / 6 + √ 2 γ k 3 / 4 n √ 1 + 4 √ ε φ · ( ε φ 2 ) 1 / 3 + 8 √ ε φ · k γn + ξ n < √ γ n · ( 1 2 · 10 3 / 2 · k 1 / 4 log 1 / 2 k + 1 2 · 10 3 · k 3 / 4 log k + 1 10 9 / 2 · k 5 / 4 log 3 / 2 k + 1 1000 ) < 0 . 027 · √ γ n < θ , since ε φ 2 < γ 3 4 3 · 10 9 · k 9 2 · log 3 k and ξ = √ γ 1000 . Thus, an edge ( u, v ) will not be added to H .

̸

Therefore, with probability at least 1 -1 50 -1 50 -1 n 50 ≥ 0 . 95 , the similarity graph H has following properties: (1) all vertices in V ( H ) (i.e., S ) are good; (2) all vertices in S that belongs to the same cluster C i form a connected components, denoted by S i ; (3) there is no edge between S i and S j , i = j ; (4) there are exactly k connected components in H , each corresponding to a cluster.

Now we are ready to consider a query WHICHCLUSTER( G,x ).

Assume x is good. We use C x to denote the cluster that x belongs to. Since all the vertices in S are good, let y ∈ C x ∩ S , so with probability at least 1 -s n 100 ≥ 1 -1 n 50 , by above analysis, we have ⟨ f x , f y ⟩ apx ≥ ⟨ f x , f y ⟩ -ξ n ≥ θ . On the other hand, for any y ∈ S \ C x , with probability at least 1 -s n 100 ≥ 1 -1 n 50 , by above analysis, we have ⟨ f x , f y ⟩ apx ≤ ⟨ f x , f y ⟩ + ξ n < θ . Thus, WHICHCLUSTER( G,x ) will output the label of y ∈ C x ∩ S as x ′ s label (at line 3 of Alg.2).

Therefore, with probability at least 1 -1 50 -1 50 -1 n 50 -n n 50 ≥ 0 . 95 , all the good vertices will be correctly recovered. So the misclassified vertices come from B . We know that | B | ≤ ( α + β + 10 4 ε φ 2 ) · n = ( 4 √ k · ( ε φ 2 ) 1 / 3 + 10 4 ε φ 2 ) · n. Since | C i | ≥ γn k , we have n ≤ k γ | C i | . So, | B | ≤ (4 √ k · ( ε φ 2 ) 1 / 3 + 10 4 ε φ 2 ) · k γ | C i | ≤ O ( k 3 2 γ · ( ε φ 2 ) 1 / 3 ) | C i | . This implies that there exists a permutation π : [ k ] → [ k ] such that for all i ∈ [ k ] , | U π ( i ) △ C i | ≤ O ( k 3 2 · ( ε 2 ) 1 / 3 ) | C i | .

γ φ

Running time. By Lemma 2.1, we know that INITIALIZEORACLE( G, 1 / 2 , ξ ) computes in time ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 3 · 1 φ 2 a sublinear space data structure D and for every pair of vertices x, y ∈ V , SPECTRALDOTPRODUCTORACLE( G,x,y, 1 / 2 , ξ, D ) computes an output value ⟨ f x , f y ⟩ apx in ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 2 · 1 φ 2 time.

In preprocessing phase, for algorithm CONSTRUCTORACLE( G,k,φ,ε,γ ), it invokes INITIALIZEORACLE one time to construct a data structure D and SPECTRALDOTPRODUCTORACLE s · s times to construct a similarity graph H . So the preprocessing time of our oracle is ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 3 · 1 φ 2 + 100 k 2 log 2 k γ 2 · ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 2 · 1 φ 2 = O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k · log n γφ )) .

In query phase, to answer the cluster index that x belongs to, algorithm WHICHCLUSTER( G,x ) invokes SPECTRALDOTPRODUCTORACLE s times. So the query time of our oracle is 10 k log k γ · ( k ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 2 · 1 2 = O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k · log n )) .

Space. By the proof of Lemma 2.1 in [14], we know that overall both algorithm INITIALIZEORACLE and SPECTRALDOTPRODUCTORACLE take ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · poly (log n ) space. Therefore, overall preprocessing phase takes ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · poly (log n ) = O ( n 1 / 2+ O ( ε/φ 2 ) · poly ( k log n γ )) space (at lines 5 and 7 of Alg.1). In query phase, WHICHCLUSTER query takes ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · poly (log n ) = O ( n 1 / 2+ O ( ε/φ 2 ) · poly ( k log n γ )) space (at line 2 of Alg.2).

4 Experiments

To evaluate the performance of our oracle, we conducted experiments on the random graph generated by the Stochastic Block Model (SBM). In this model, we are given parameters p, q and n, k , where n, k denote the number of vertices and the number of clusters respectively; p denotes the probability that any pair of vertices within each cluster is connected by an edge, and q denotes the probability that any pair of vertices from different clusters is connected by an edge. Setting p q > c for some big enough constant c we can get a well-clusterable graph. All experiments were implemented in Python 3.9 and the experiments were performed using an Intel(R) Core(TM) i7-12700F CPU @ 2.10GHz processor, with 32 GB RAM. Due to the limited space, practical changes to our oracle will be shown in Appendix F.

Misclassification error. To evaluate the misclassification error our oracle, we conducted this experiment. In this experiment, we set k = 3 , n = 3000 , q = 0 . 002 , p ∈ [0 . 02 , 0 . 07] in the SBM, where each cluster has 1000 vertices. For each graph G = ( V, E ) , we run WHICHCLUSTER( G,x ) for all x ∈ V and get a k -partition U 1 , . . . , U k of V . In experiments, the misclassification error of our oracle is defined to be 1 -1 n · max π ∑ k i =1 U π ( i ) ∩ C i , where π : [ k ] → [ k ] is a permutation and C 1 , . . . , C k are the ground-truth clusters of G .

Moreover, we implemented the oracle in [8] to compare with our oracle 2 . The oracle in [8] can be seen as a non-robust version of oracle in [30]. Note that our primary advancement over [8] (also [30]) is evident in the significantly reduced conductance gap we achieve.

We did not compare with the oracle in [14], since implementing the oracle in [14] poses challenges. As described in section 3.1, the oracle in [14] initially approximates the k cluster centers by sampling around O (1 /ε · k 4 log k ) vertices, and subsequently undertakes the enumeration of approximately 2 O (1 /ε · k 4 log 2 k ) potential k -partitions (Algorithm 10 in [14]). This enumeration process is extremely time-intensive and becomes impractical even for modest values of k .

According to the result of our experiment (Table 1), the misclassification error of our oracle is reported to be quite small when p ≥ 0 . 025 , and even decreases to 0 when p ≥ 0 . 035 . The outcomes of our experimentation distinctly demonstrate that our oracle's misclassification error remains notably minimal in instances where the input graph showcases an underlying latent cluster structure. In addition, Table 1 also shows that our oracle can handle graphs with a smaller conductance gap than the oracle in [8], which is consistent with the theoretical results. This empirical validation reinforces the practical utility and efficacy of our oracle beyond theoretical conjecture.

1 In the experiment, we found that the misclassification error of oracle in [8] fluctuated greatly, so for the oracle in [8], for each value of p , we conducted 30 independent experiments and recorded the minimum error, maximum error and average error of oracle in [8].

Robustness experiment. The base graph G 0 = ( V, E ) is generated from SBM with n = 3000 , k = 3 , p = 0 . 05 , q = 0 . 002 . Note that randomly deleting some edges in each cluster is equivalent to reducing p and randomly deleting some edges between different clusters is equivalent to reducing q . So we consider the worst case. We randomly choose one vertex from each cluster; for each selected vertex x i , we randomly delete delNum edges connected to x i in cluster C i . If x i has fewer than delNum neighbors within C i , then we delete all the edges incident to x i in C i . We run WHICHCLUSTER queries for all vertices in V on the resulting graph. We repeated this process for five times for each parameter delNum and recorded the average misclassification error (Table 2). The results show that our oracle is robust against a few number of random edge deletions.

We also conducted experiments to report the query complexity and running time of our oracle. See Appendix F for more details.

5 Conclusion

We have devised a new spectral clustering oracle with sublinear preprocessing and query time. In comparison to the approach presented in [14], our oracle exhibits improved preprocessing efficiency, albeit with a slightly higher misclassification error rate. Furthermore, our oracle can be readily implemented in practical settings, while the clustering oracle proposed in [14] poses challenges in terms of implementation feasibility. To obtain our oracle, we have established a property regarding the spectral embeddings of the vertices in V for a d -bounded n -vertex graph G = ( V, E ) that exhibits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Specifically, if x and y belong to the same cluster, the dot product of their spectral embeddings (denoted as ⟨ f x , f y ⟩ ) is approximately O ( k n ) . Conversely, if x and y are from different clusters, ⟨ f x , f y ⟩ is close to 0. We also show that our clustering oracle is robust against a few random edge deletions and conducted experiments on synthetic networks to validate our theoretical results.

Acknowledgments and Disclosure of Funding

The work is supported in part by the Huawei-USTC Joint Innovation Project on Fundamental System Software, NSFC grant 62272431 and 'the Fundamental Research Funds for the Central Universities'.

References

Reid Andersen, Fan Chung, and Kevin Lang. Local graph partitioning using pagerank vectors. In 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06) , pages 475-486. IEEE, 2006.

Reid Andersen, Shayan Oveis Gharan, Yuval Peres, and Luca Trevisan. Almost optimal local graph clustering using evolving sets. Journal of the ACM (JACM) , 63(2):1-31, 2016.

Reid Andersen and Yuval Peres. Finding sparse cuts locally using evolving sets. In Proceedings of the forty-first annual ACM symposium on Theory of computing , pages 235-244, 2009.

T Tony Cai and Xiaodong Li. Robust and computationally feasible community detection in the presence of arbitrary outlier nodes. The Annals of Statistics , 43(3):1027-1059, 2015.

Yudong Chen, Ali Jalali, Sujay Sanghavi, and Huan Xu. Clustering partially observed graphs via convex optimization. The Journal of Machine Learning Research , 15(1):2213-2238, 2014.

Ashish Chiplunkar, Michael Kapralov, Sanjeev Khanna, Aida Mousavifar, and Yuval Peres. Testing graph clusterability: Algorithms and lower bounds. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS) , pages 497-508. IEEE, 2018.

Fan RK Chung. Spectral graph theory , volume 92. American Mathematical Soc., 1997.

Artur Czumaj, Pan Peng, and Christian Sohler. Testing cluster structure of graphs. In Proceedings of the forty-seventh annual ACM symposium on Theory of Computing , pages 723-732, 2015.

Tamal K Dey, Pan Peng, Alfred Rossi, and Anastasios Sidiropoulos. Spectral concentration and greedy k-clustering. Computational Geometry , 76:19-32, 2019.

Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd , volume 96, pages 226-231, 1996.

Pedro F Felzenszwalb and Daniel P Huttenlocher. Efficient graph-based image segmentation. International journal of computer vision , 59:167-181, 2004.

Santo Fortunato. Community detection in graphs. Physics reports , 486(3-5):75-174, 2010.

Amir Globerson, Tim Roughgarden, David Sontag, and Cafer Yildirim. Tight error bounds for structured prediction. arXiv preprint arXiv:1409.5834 , 2014.

Grzegorz Gluch, Michael Kapralov, Silvio Lattanzi, Aida Mousavifar, and Christian Sohler. Spectral clustering oracles in sublinear time. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 1598-1617. SIAM, 2021.

Olivier Guédon and Roman Vershynin. Community detection in sparse networks via grothendieck's inequality. Probability Theory and Related Fields , 165(3-4):1025-1049, 2016.

Michael Kapralov, Akash Kumar, Silvio Lattanzi, and Aida Mousavifar. Learning hierarchical structure of clusterable graphs. CoRR , abs/2207.02581, 2022.

Michael Kapralov, Akash Kumar, Silvio Lattanzi, and Aida Mousavifar. Learning hierarchical cluster structure of graphs in sublinear time. In Nikhil Bansal and Viswanath Nagarajan, editors, Proceedings of the 2023 ACM-SIAM Symposium on Discrete Algorithms, SODA 2023, Florence, Italy, January 22-25, 2023 , pages 925-939. SIAM, 2023.

Konstantin Makarychev, Yury Makarychev, and Aravindan Vijayaraghavan. Approximation algorithms for semi-random partitioning problems. In Proceedings of the forty-fourth annual ACM symposium on Theory of computing , pages 367-384. ACM, 2012.

Konstantin Makarychev, Yury Makarychev, and Aravindan Vijayaraghavan. Constant factor approximation for balanced cut in the pie model. In Proceedings of the 46th Annual ACM Symposium on Theory of Computing , pages 41-49. ACM, 2014.

Konstantin Makarychev, Yury Makarychev, and Aravindan Vijayaraghavan. Correlation clustering with noisy partial information. In Proceedings of The 28th Conference on Learning Theory , pages 1321-1342, 2015.

Konstantin Makarychev, Yury Makarychev, and Aravindan Vijayaraghavan. Learning communities in the presence of errors. In Conference on Learning Theory , pages 1258-1291, 2016.

Bogdan-Adrian Manghiuc and He Sun. Hierarchical clustering: o (1) -approximation for wellclustered graphs. Advances in Neural Information Processing Systems , 34:9278-9289, 2021.

Claire Mathieu and Warren Schudy. Correlation clustering with noisy input. In Proceedings of the twenty-first annual ACM-SIAM symposium on discrete algorithms , pages 712-728. Society for Industrial and Applied Mathematics, 2010.

Ankur Moitra, William Perry, and Alexander S Wein. How robust are reconstruction thresholds for community detection? In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing , pages 828-841. ACM, 2016.

Stefan Neumann and Pan Peng. Sublinear-time clustering oracle for signed graphs. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , volume 162 of Proceedings of Machine Learning Research , pages 1649616528. PMLR, 2022.

Mark EJ Newman and Michelle Girvan. Finding and evaluating community structure in networks. Physical review E , 69(2):026113, 2004.

Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems , 14, 2001.

Lorenzo Orecchia and Zeyuan Allen Zhu. Flow-based algorithms for local graph clustering. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms , pages 1267-1286. SIAM, 2014.

Alberto Paccanaro, James A Casbon, and Mansoor AS Saqi. Spectral clustering of protein sequences. Nucleic acids research , 34(5):1571-1580, 2006.

Pan Peng. Robust clustering oracle and local reconstructor of cluster structure of graphs. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 2953-2972. SIAM, 2020.

Pan Peng and Yuichi Yoshida. Average sensitivity of spectral clustering. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020 , pages 1132-1140. ACM, 2020.

Mason A Porter, Jukka-Pekka Onnela, Peter J Mucha, et al. Communities in networks. Notices of the AMS , 56(9):1082-1097, 2009.

Daniel A Spielman and Shang-Hua Teng. A local clustering algorithm for massive graphs and its application to nearly linear time graph partitioning. SIAM Journal on computing , 42(1):1-26, 2013.

Zeyuan Allen Zhu, Silvio Lattanzi, and Vahab Mirrokni. A local algorithm for finding wellconnected clusters. In International Conference on Machine Learning , pages 396-404. PMLR, 2013.

A Complete preliminaries

Let G = ( V, E ) denote a d -regular undirected and unweighted graph, where V := { 1 , . . . , n } . Throughout the paper, we use i ∈ [ n ] to denote 1 ≤ i ≤ n and all the vectors will be column vectors unless otherwise specified or transposed to row vectors. For a vertex x ∈ V , let ✶ x ∈ R n denote the indicator of x , which means ✶ x ( i ) = 1 if i = x and 0 otherwise. For a vector x , we let ∥ x ∥ 2 = √∑ i x ( i ) 2 denote its ℓ 2 norm. For a matrix A ∈ R n × n , we use ∥ A ∥ to denote the spectral norm of A , and we use ∥ A ∥ F to denote the Frobenius norm of A . For any two vectors x , y ∈ R n , we let ⟨ x , y ⟩ = x T y denote the dot product of x and y . For a matrix A ∈ R n × n , we use A [ i ] ∈ R n × i to denote the first i columns of A , 1 ≤ i ≤ n .

Let A ∈ R n × n denote the adjacency matrix of G and let D ∈ R n × n denote a diagonal matrix. For the adjacency matrix A , A ( i, j ) = 1 if ( i, j ) ∈ E and 0 otherwise, u, v ∈ [ n ] . For the diagonal matrix D , D ( i, i ) = deg( i ) , where deg( i ) is the degree of vertex i, i ∈ [ n ] . We denote with L the normalized Laplacian of G where L = D -1 / 2 ( D -A ) D -1 / 2 = I -A d . For L , we use 0 ≤ λ 1 ≤ · · · ≤ λ n ≤ 2 [7] to denote its eigenvalues and we use u 1 , . . . , u n ∈ R n to denote the corresponding eigenvectors. Note that the corresponding eigenvectors are not unique, in this paper, we let u 1 , . . . , u n be an orthonormal basis of eigenvectors of L . Let U ∈ R n × n be a matrix whose i -th column is u i , i ∈ [ n ] , then for every vertex x ∈ V , f x = U T [ k ] ✶ x . For any two sets S 1 and S 2 , we let S 1 △ S 2 denote the symmetric difference between S 1 and S 2 .

B d -bounded graphs to d -regular graphs

For a d -bounded graph G ′ = ( V, E ) , we can get a d -regular graph G from G ′ by adding d -deg( x ) self-loops with weight 1 / 2 to each vertex x ∈ V . Note that the lazy random walk on G is equivalent to the random walk on G ′ , with the random walk satisfying that if we are at vertex x , then we jump to a random neighbor with probability 1 2 d and stay at x with probability 1 -deg( x ) 2 d . We use w self ( x ) = ( d -deg( x )) · 1 2 to denote the the weight of all self-loops of x ∈ V .

C Formal statement of Lemma 2.1 and description of dot-product algorithms in [14]

Lemma C.1 (Lemma 2.1, Formal; Theorem 2, [14]) . Let ε, φ ∈ (0 , 1) with ε ≤ φ 2 10 5 . Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Let 1 n 5 < ξ < 1 . Then INITIALIZEORACLE ( G, 1 / 2 , ξ ) computes in time ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 3 · 1 φ 2 a sublinear space data structure D of size ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 3 such that with probability at least 1 -n -100 the following property is satisfied:

For every pair of vertices x, y ∈ V , SPECTRALDOTPRODUCT ( G,x,y, 1 / 2 , ξ, D ) computes an output value ⟨ f x , f y ⟩ apx such that with probability at least 1 -n -100

The running time of SPECTRALDOTPRODUCT ( G,x,y, 1 / 2 , ξ, D ) is ( k ξ ) O (1) · n 1 / 2+ O ( ε/φ 2 ) · (log n ) 2 · 1 φ 2 .

Algorithm 4: RUNRANDOMWALKS( G,R,t,x )

1 Run R random walks of length t starting from x

2 Let ̂ m x ( y ) be the fraction of random walks that ends at y

Appendix

Algorithm 5: ESTIMATETRANSITIONMATRIX( G,I S , R, t ) 1 for each sample x ∈ I S do 2 ̂ m x := RUNRANDOMWALKS( G,R,t,x ) 3 end 4 Let ̂ Q be the matrix whose columns are ̂ m x for x ∈ I S 5 return ̂ Q Algorithm 6: ESTIMATECOLLISIONPROBABILITIES( G,I S , R, t ) 1 for i = 1 to O (log n ) do 2 ̂ Q i := ESTIMATETRANSITIONMATRIX( G,I S , R, t ) 3 ̂ P i := ESTIMATETRANSITIONMATRIX( G,I S , R, t ) 4 G i := 1 2 ( ̂ P T i ̂ Q i + ̂ Q T i ̂ P i ) 5 end 6 Let G be a matrix obtained by taking the entrywise median of G ′ i s 7 return G Algorithm 7: INITIALIZEORACLE( G,δ, ξ ) Need: ε/φ 2 ≤ 1 10 5 1 t := 20 · log n φ 2 2 R init := O ( n 1 -δ +980 · ε/φ 2 · k 17 /ξ 2 ) 3 s := O ( n 480 · ε/φ 2 · log n · k 8 /ξ 2 ) 4 Let I S be the multiset of s indices chosen independently and uniformly at random from { 1 , . . . , n } 5 for i = 1 to O (log n ) do 6 ̂ Q i := ESTIMATETRANSITIONMATRIX( G,I S , R init , t ) 7 end 8 G := ESTIMATECOLLISIONPROBABILITIES( G,I S , R init , t ) 9 Let n s · G := ̂ W ̂ Σ ̂ W T be the eigendecomposition of n s · G if ̂ Σ -1 exists then Ψ := n s · ̂ W [ k ] ̂ Σ -2 [ k ] ̂ W T [ k ] return D := { Ψ , ̂ Q 1 , . . . , ̂ Q O (log n ) } end Algorithm 8: SPECTRALDOTPRODUCTORACLE( G,x,y,δ, ξ, D ) Need: ε/φ 2 ≤ 1 10 5 1 R query := O ( n δ +500 · ε/φ 2 · k 9 /ξ 2 ) 2 for i = 1 to O (log n ) do 3 ̂ m i x := RUNRANDOMWALKS( G,R query , t, x ) 4 ̂ m i y := RUNRANDOMWALKS( G,R query , t, y ) 5 end 6 Let α x be a vector obtained by taking the entrywise median of ( ̂ Q i ) T ( ̂ m i x ) over all runs 7 Let α y be a vector obtained by taking the entrywise median of ( ̂ Q i ) T ( ̂ m i y ) over all runs 8 return ⟨ f x , f y ⟩ apx := α T x Ψ α y

D Deferred proofs

Lemma D.1 (Restatement of Lemma 3.1) . Let α ∈ (0 , 1) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε ∈ (0 , 1) . Let G = ( V, E ) be a d -regular ( k, φ, ε ) -clusterable graph with | V | = n . There exists a subset ̂ V ⊆ V with | ̂ V | ≥ (1 -α ) | V | such that for all x ∈ ̂ V , it holds that ∥ f x ∥ 2 ≤ √ 1 α · k n .

Proof. Recall that u 1 , . . . , u k are an orthonormal basis of eigenvectors of L , so ∥ u i ∥ 2 2 = 1 for all i ∈ [ k ] . So ∑ k i =1 ∥ u i ∥ 2 2 = ∑ n i =1 ∥ f x i ∥ 2 2 = k. Let X be a random variable such that X = ∥ f x i ∥ 2 2 with probability 1 n , for each i ∈ [ n ] . Then we have E [ X ] = 1 n ∑ n i =1 ∥ f x i ∥ 2 2 = k n . Using Markov's inequality, we have Pr[ X ≥ 1 α · E [ X ]] = Pr[ X ≥ 1 α · k n ] ≤ α. This gives us that Pr[ X ≤ 1 α · k n ] ≥ 1 -α , which means that at least (1 -α ) fraction of vertices in V satisfies ∥ f x ∥ 2 2 ≤ 1 α · k n . We define ̂ V := { x ∈ V : ∥ f x ∥ 2 2 ≤ 1 α · k n } , then we have | ̂ V | ≥ (1 -α ) | V | . This ends the proof.

Lemma D.2 (Restatement of Lemma 3.2) . Let β ∈ (0 , 1) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε ∈ (0 , 1) . Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k with | V | = n . There exists a subset ˜ V ⊆ V with | ˜ V | ≥ (1 -β ) | V | such that for all x ∈ ˜ V , it holds that ∥ f x -µ x ∥ 2 ≤ √ 4 kε βφ 2 · 1 n .

The following result will be used in our proof:

LemmaD.3 (Lemma 6, [14]) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε ∈ (0 , 1) . Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Then for all α ∈ R k , with ∥ α ∥ 2 = 1 we have

Proof of Lemma 3.2. By summing over α in an orthonormal basis of R k , we can get

where µ x is the cluster center of the cluster that x belongs to. Define V ∗ = { x ∈ V : ∥ f x -µ x ∥ 2 2 ≥ 4 kε βφ 2 · 1 n } . Then,

So, we can get | V ∗ | ≤ βn . We define ˜ V = V \ V ∗ = { x ∈ V : ∥ f x -µ x ∥ 2 2 ≤ 4 kε βφ 2 · 1 n } . Therefore, we have | ˜ V | ≥ (1 -β ) n = (1 -β ) | V | . This ends the proof.

Lemma D.4 (Restatement of Lemma 3.3) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Let C i denote the cluster corresponding to the center µ i , i ∈ [ k ] . Then for every C i , i ∈ [ k ] , there exists a subset ˜ C i ⊆ C i with | ˜ C i | ≥ (1 -10 4 ε φ 2 ) | C i | such that for all x ∈ ˜ C i , it holds that ⟨ f x , µ i ⟩ ≥ 0 . 96 ∥ µ i ∥ 2 2 .

The following result will be used in our proof:

Lemma D.5 (Lemma 31, [14]) . Let k ≥ 2 , φ ∈ (0 , 1) , and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . If µ ′ i s are cluster means then the following conditions hold. Let S ⊂ { µ 1 , . . . , µ k } . Let Π denote the orthogonal projection matrix on to the span ( S ) ⊥ . Let µ ∈ { µ 1 , . . . , µ k }\ S . Let C denote the cluster corresponding to the center µ . Let

then we have:

Proof of Lemma 3.3. We apply S = ∅ in Lemma D.5 so that Π is an identity matrix and we will have | C i \ ̂ C i | ≤ 10 4 ε φ 2 | C i | , where ̂ C i := { x ∈ V : ⟨ f x , µ i ⟩ ≥ 0 . 96 ∥ µ i ∥ 2 2 } , i ∈ [ k ] . So

We define ˜ C i = C i ∩ ̂ C i , i ∈ [ k ] . Therefore, for every C i , i ∈ [ k ] , there exists a subset ˜ C i ⊆ C i with | ˜ C i | ≥ (1 -10 4 ε φ 2 ) | C i | such that for all x ∈ ˜ C i , it holds that ⟨ f x , µ i ⟩ ≥ 0 . 96 ∥ µ i ∥ 2 2 .

LemmaD.6 (Restatement of Lemma 3.4) . Let k ≥ 2 , φ ∈ (0 , 1) and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular n -vertex graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Suppose that x, y ∈ V are in the same cluster C i , i ∈ [ k ] and both of them are good vertex with respect to α = 2 √ k · ( ε φ 2 ) 1 / 3 and β = 2 √ k · ( ε φ 2 ) 1 / 3 , the following holds:

The following result will be used in our proof:

Lemma D.7 (Lemma 7, [14]) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) , and ε ∈ (0 , 1) . Let G = ( V, E ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Then we have

̸

Proof of Lemma 3.4. According to the distributive law of dot product, we have

By using Cauchy-Schwarz Inequality, we have |⟨ f x , f y -µ i ⟩| ≤ ∥ f x ∥ 2 · ∥ f y -µ i ∥ 2 . Since x and y are both good vertices with respect to α = 2 √ k · ( ε φ 2 ) 1 / 3 and β = 2 √ k · ( ε φ 2 ) 1 / 3 , we have

which gives us that ⟨ f x , f y -µ i ⟩ ≥ -√ k n · ( ε φ 2 ) 1 / 6 . Recall that x is a good vertex, we have ⟨ f x , µ i ⟩ ≥ 0 . 96 ∥ µ i ∥ 2 2 . Hence, it holds that

The last inequality is according to item 1 in Lemma D.7.

̸

LemmaD.8 (Restatement of Lemma 3.5) . Let k ≥ 2 , φ ∈ (0 , 1) and ε φ 2 be smaller than a sufficiently small constant. Let G = ( V, E ) be a d -regular n -vertex graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Suppose that x ∈ C i , y ∈ C j , ( i, j ∈ [ k ] , i = j ) and both of them are good vertex with respect to α = 2 √ k · ( ε φ 2 ) 1 / 3 and β = 2 √ k · ( ε φ 2 ) 1 / 3 , the following holds:

Proof. According to the distributive law of dot product, we have

By Cauchy-Schwarz Inequality, we have and

Since x and y are both good vertices with respect to α = 2 √ k · ( ε φ 2 ) 1 / 3 and β = 2 √ k · ( ε φ 2 ) 1 / 3 , we have

and

So we have

The last inequality is according to item 1 and item 2 in Lemma D.7.

E Formal statement of Theorem 2 and proof

Theorem 2 (Formal; Robust against random edge deletions) . Let k ≥ 2 be an integer, φ ∈ (0 , 1) . Let G 0 = ( V, E 0 ) be a d -regular n -vertex graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k , ε φ 4 ≪ γ 3 k 9 2 · log 3 k and for all i ∈ [ k ] , γ n k ≤ | C i | ≤ n γk , where γ is a constant that is in (0 . 001 , 1] .

Let G be a graph obtained from G 0 by deleting at most c edges in each cluster, where c is a constant. If 0 ≤ c ≤ dφ 2 2 √ 10 , then there exists an algorithm that has query access to the adjacency list of G and constructs a clustering oracle in O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γφ )) preprocessing time and takes O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γ )) space. Furthermore, with probability at least 0 . 95 , the following hold:

1). Using the oracle, the algorithm can answer any WHICHCLUSTER query in O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γφ )) time and a WHICHCLUSTER query takes O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γ )) space.

2). Let U i := { x ∈ V : WHICHCLUSTER ( G,x ) = i } , i ∈ [ k ] be the clusters recovered by the algorithm. There exists a permutation π : [ k ] → [ k ] such that for all i ∈ [ k ] , | U π ( i ) △ C i | ≤ O ( k 3 2 γ · ( ε φ 4 ) 1 / 3 ) | C i | .

Let G be a graph obtained from G 0 by randomly deleting at most O ( kd 2 log k + d ) edges in G 0 . With probability at least 1 -1 k 2 , then there exists an algorithm that has query access to the adjacency list of G and constructs a clustering oracle in O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γφ )) preprocessing time and takes O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γ )) space. Furthermore, with probability at least 0 . 95 , the following hold:

1). Using the oracle, the algorithm can answer any WHICHCLUSTER query in O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γφ )) time and a WHICHCLUSTER query takes O ( n 1 / 2+ O ( ε/φ 2 ) · poly( k log n γ )) space.

2). Let U i := { x ∈ V : WHICHCLUSTER ( G,x ) = i } , i ∈ [ k ] be the clusters recovered by the algorithm. There exists a permutation π : [ k ] → [ k ] such that for all i ∈ [ k ] , | U π ( i ) △ C i | ≤ O ( k 3 2 γ · ( ε φ 4 ) 1 / 3 ) | C i | .

To proof Theorem 2, we need the following lemmas.

Lemma E.1 (Cheeger's Inequality) . In holds for any graph G that

Lemma E.2 (Weyl's Inequality) . Let A,B ∈ R n × n be symmetric matrices. Let α 1 , . . . , α n and β 1 , . . . , β n be the eigenvalues of A and B respectively. Then for any i ∈ [ n ] , we have

where ∥ A -B ∥ is the spectral norm of A -B .

Proof of Theorem 2. Proof of item 1: For any d -bounded graph G ′ = ( V, E ) , we can get a d -regular graph G from G ′ by adding d -deg( x ) self-loops with weight 1 / 2 to each vertex x ∈ V . Then according to [7], the normalized Laplacian of G (denoted as L ) satisfies

̸

̸

Let G 0 = ( V, E 0 ) be a d -regular graph that admits a ( k, φ, ε ) -clustering C 1 , . . . , C k . Now we consider a cluster C i , i ∈ [ k ] . Let C 0 i be a d -regular graph obtained by adding d -deg i ( x ) self-loops to each vertex x ∈ C i , where deg i ( x ) is the degree of vertex x in the subgraph C i , i ∈ [ k ] . Let C j i be a graph obtained from C j -1 i by: (1) randomly deleting one edge ( u, v ) ∈ E ( C j -1 i ) , where E ( C j -1 i ) is a set of edges that have both endpoints in C j -1 i ; (2) turning the result subgraph in (1) be a d -regular graph, i ∈ [ k ] , j ∈ [ c ] . Let L j i be the normalized Laplacian of C j i , i ∈ [ k ] , j = 0 , 1 , . . . , c . Let H j i = L j i -L j -1 i , i ∈ [ k ] , j ∈ [ c ] . Then if u = v , we have

j

and if u = v , H i is a all-zero matrix. Consider the fact that for a symmetric matrix, the spectral norm is less than or equal to its Frobenius norm, we will have ∥ H j i ∥ ≤ ∥ H j i ∥ F = √ 2 · 1 d 2 +2 · 1 4 d 2 = √ 5 2 d 2 = √ 10 2 d for all i ∈ [ k ] , j ∈ [ c ] . Let H i = ∑ c j =1 H j i = L c i -L 0 i , we have ∥ H i ∥ ≤ √ 10 2 d · c, i ∈ [ k ] . Let λ 2 ( L 0 i ) and λ 2 ( L c i ) be the second smallest eigenvalue of L 0 i and L c i respectively. By Lemma E.2, we have | λ 2 ( L c i ) -λ 2 ( L 0 i ) | ≤ ∥ H i ∥ ≤ √ 10 2 d · c , which gives us λ 2 ( L c i ) ≥ λ 2 ( L 0 i ) -√ 10 2 d · c , i ∈ [ k ] . By Lemma E.1 and the precondition that c ≤ dφ 2 2 √ 10 , we have λ 2 ( L 0 i ) ≥ φ 2 2 ≥ √ 10 d · c . Therefore,

Again by Lemma E.1, for graph C c i , we have ϕ in ( C c i ) = ϕ ( C c i ) ≥ 1 2 λ 2 ( L c i ) ≥ 1 4 λ 2 ( L 0 i ) ≥ 1 8 φ 2 , i ∈ [ k ] . Note that we slightly abuse the notion C c i , for ϕ ( C c i ) , we treat C c i as a d -regular graph, and

for ϕ in ( C c i ) , we treat C c i as a cluster obtained by deleting c edges from E ( C i ) . So, for a ( k, φ, ε ) -clusterable graph G 0 = ( V, E 0 ) , if we delete at most c ≤ dφ 2 2 √ 10 edges in each cluster, then the resulting graph G is ( k, 1 8 φ 2 , ε ) -clusterable. Since ε φ 4 ≪ γ 3 k 9 2 · log 3 k , we have ε ( 1 8 φ 2 ) 2 ≪ γ 3 k 9 2 · log 3 k . The statement of item 1 in this theorem follows from the same augments as those in the proof of Theorem 1 with parameter φ ′ = 1 8 φ 2 in G .

Proof of item 2: Let c = dφ 2 2 √ 10 . Since | C i | ≤ n γk for all i ∈ [ k ] , we have | E ( C i ) | ≤ n γk · d 2 , where E ( C i ) is a set of edges that have both endpoints in C i . So | E ( C i ) | | E 0 | ≤ n γk · d 2 nd 2 = 1 γk , i ∈ [ k ] . Since | C i | ≥ γn k and ϕ out ( C i , V ) ≤ ε , we have E ( C i ) ≥ nd 2 k ( γ -ε γ ) . So | E ( C i ) | | E 0 | ≥ nd 2 k ( γ -ε γ ) nd 2 = 1 ( γ -ε ) , i ∈ [ k ] . Combining the above two results, we have 1 ( γ -ε ) ≤ | E ( C i ) | ≤ 1 .

In the following, we use X i to denote the number of edges that are deleted from E ( C i ) . If we randomly delete kc 2 γ ( γ 2 -ε ) 9 log k +2( γ 2 -ε ) c = O ( kd 2 log k + d ) edges from G 0 , then we have 1 k ( γ -ε γ ) · kc 2 γ ( γ 2 -ε ) 9 log k +2( γ 2 -ε ) c ≤ E ( X i ) ≤ 1 γk · kc 2 γ ( γ 2 -ε ) 9 log k +2( γ 2 -ε ) c , which gives us

Chernoff-Hoeffding implies that Pr[ X i > (1+ δ ) · E ( X i )] ≤ e -E ( Xi ) · δ 2 3 . Weset δ = 9 log k +2( γ 2 -ε ) c ( γ 2 -ε ) c -1 , then we have

Using union bound, with probability at least 1 -1 k 3 · k = 1 -1 k 2 , we have that if we randomly delete kc 2 γ ( γ 2 -ε ) 9 log k +2( γ 2 -ε ) c = O ( kd 2 log k + d ) edges from G 0 , there is no cluster that is deleted more than c edges. Therefore, according to item 1 of Theorem 2, with probability at least 1 -1 k 3 · k = 1 -1 k 2 , G is a ( k, 1 8 φ 2 , ε ) -clusterable graph. The statement of item 2 in this theorem also follows from the same augments as those in the proof of Theorem 1 with parameter φ ′ = 1 8 φ 2 in G .

F Experiment Details

Practical changes to our oracle. In order to implement our oracle, we need to make some modifications to the theoretical algorithms. To adapt the dot product oracle parameters (see Algorithm 7 and Algorithm 8 in Appendix C), i.e., t (random walk length), s (sampling set size), and R init , R query (number of random walks), we exploit the theoretical gap between intra-cluster and intercluster dot products in clusterable graphs. Given a clusterable graph G , by constructing the dot product oracle with various parameter settings and calculating some intra-cluster and inter-cluster dot products, we generate density graphs. The setting with the most prominent gap in the density graph is selected (see Figure 2 for an illustrative example).

1200

inter-cluster

intra-cluster

8000

inter-cluster

intra-cluster

7000

1000

6000

800

5000

Density

Density

600

4000

3000

400

2000

200

1000

0

0.002

-0.001

0.000

0.001

0.002

0.003

0

-0.00025.000000.000250.000500.000750.001000.001250.001500.00175

Determining the appropriate threshold θ (at lines 2, 8, 9 of Alg.1 and line 3 of Alg.2) is the next step. By observing the density graph linked to the chosen dot product oracle parameters, we identify the fitting θ (see Figure 2 for an illustrative example).

Determining the appropriate sampling set size s (at line 3 of Alg.1) of our oracle is the final step. Given a graph G = ( V, E ) generated by SBM, for all vertices in V , we know their ground-truth clusters. We can built our clustering oracle for several parameters for s . For each parameter setting, we run WHICHCLUSTER( G,x ) for some x ∈ V and check if x was classified correctly. We pick the parameter setting with the most correct answers.

Query complexity. We conducted an experiment on a SBM graph with k = 3 , n = 15000 , q = 0 . 002 , p = 0 . 2 . We calculate the fraction of edges that have been accessed given a number of invocations of WHICHCLUSTER( G,x ) (Table 3). (Note that there is a trade-off between computational cost and clustering quality. Therefore, it is necessary to point out that the parameters of this experiment are set reasonably and the misclassification error is 0 .) Table 3 shows that as long as the number of WHICHCLUSTER queries is not too large, our algorithm only reads a small portion of the input graph.

The above experiment shows that for a small target misclassification error, our algorithms only require a sublinear amount of data, which is often critical when analyzing large social networks, since one typically does not have access to the entire network.

Running time. To evaluate the running time of our oracle, we conducted this experiment on some random graphs generated by SBM with n = 3000 , k = 3 , q = 0 . 002 and p ∈ [0 . 02 , 0 . 06] . Note that there is a trade-off between running time and clustering quality. In this experiment, we set the experimental parameters the same as those in the misclassification error experiment, which can ensure a small error. We recorded the running time of constructing a similarity graph H as construct-time. For each p , we query all the vertices in the input graph and recorded the average time of the n = 3000 queries as query-time (Table 4).

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of constructing a spectral clustering oracle with sublinear pre-processing and query time complexity. The paper introduces a new algorithm which improves on previous methods with respect to the running time at the expense of a slightly worse approximation guarantee. In contrast with previous methods, the new algorithm is more practical as demonstrated by experiments on synthetic data.\n\nSTRENGTHS:\nOn the theoretical side, this paper introduces a new sublinear-time clustering oracle with guarantees which improve on the previous state of the art in terms of running time. The new proof techniques are introduced in quite a natural way and the intuition is clearly explained. An important contribution of this paper is that the proposed algorithm is practical and admits an implementation.\n\nWEAKNESSES:\nThe theoretical improvements over the previous algorithms are quite small (in particular, the improvement over [30]), however I don't view this as a major weakness given that the new algorithm is more practical.\n\nThe statement of Theorem 1 is quite difficult to follow. For example, the parameter $\\xi$ doesn't seem to be introduced or explained intuitively.\n\nThe experimental evaluation is quite limited - it does not include comparison with any other method, and does not report the running time of the algorithm. If the algorithm cannot be compared with that of [30] or [14], consider stating why this is the case in section 4.\n\nQUESTIONS:\nWhat is the purpose of $\\xi$ in Theorem 1? As far as I can see from the Theorem statement, it only appears in the running times, and always in the denominator - why not always set it to $1$?\n\nIs it possible to experimentally compare the algorithm with [30] and [14]? \n\n### Typos\n* Line 133 - space missing after '.'\n* Line 144 - space missing after reference [7]\n* Line 145 - Noted -> Note\n* Line 246 - dose -> does\n* Line 262 - space missing after 'Theorem'\n\nLIMITATIONS:\nLimitations are adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies oracles for spectral graph clustering, i.e., local algorithms that answer membership queries for single nodes in a spectral clustering of a graph. There is a line of research on testing cluster structure in degree-bounded graphs, and recently, the learning version of the problem studied in this paper has become popular. Besides a result for robust clustering oracles by Peng, this works is closely related to a paper by Gluch et al. Compared to the work of Gluch et al., this submission improves the preprocessing time to $O(n^{1/2 + O(\\epsilon / \\phi^2)})$, which is better by a factor of approximately $2^{poly(k/e)}$ at the expsense of requiring a conductance gap of approximately $\\Omega(1/poly(k))$, which is worse by approximately a $poly(k)/\\log(k)$ factor, and a misclassification error of $O(poly(k) \\epsilon)$, which is worse by approximately a $k / \\log(k)$ factor. The misclassification error is the fraction of vertices that are assigned to the wrong cluster (compared to the ground truth clustering). The query time of the two algorithms is roughly the same. In a nutshell, the result in this submission trades an additional polynomial dependency in conductance gap and misclassification error against the removal of an exponential dependency in preprocessing time. The authors experimentally confirm the misclassification error and query complexity proven in their theorems.\n\nThis work builds up on the dot product oracle introduced by Gluch et al. The algorithm in the latter work estimates the means of the clusters (in an embedding space) and uses the dot product oracle to estimate the closest cluster center (mean) for a query node. The exponential preprocessing time arises from the former part. In the present submission, the authors propose an algorithm that doesn't estimate the cluster means, but compares the dot product between node embedding directly. Intuitively, if two nodes belong to the same cluster, they have a large dot product with their cluster mean, and so they should also have a large dot product with each other. This modification also results in the aforementioned trade off, i.e., the increased misclassification error and the stronger requirement on cluster separation.\n\nThe question answered by this paper arises naturally from the work of Gluch et al.: Do we need to compute the cluster means explicitly? While the answer may not be surprising (no, but there is a trade off), it requires some work to actually prove this, as the formal argument is not simple and obvious. The intention of the experiments is not clear to me, as there is no comparison with other algorithms, or insights how the theoretical algorithm needs to be modified and tuned for applications.\n\nRebuttal: Rating changed from weak accept to accept due to authors' rebuttal responses.\n\nSTRENGTHS:\n* The question explored in this paper is natural.\n* The paper confirms an intuitive concept of spectral embeddings for clustering.\n\nWEAKNESSES:\n* The result in this paper is not very surprising or better than one would expect. It seems more like a reasonable trade off.\n* The experiments seem currently not very useful.\n\nQUESTIONS:\nNote: It seems to me that a comparison of the conductance gap of your result and [14] is missing.\n\nLIMITATIONS:\n-", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a spectral clustering oracle with sublinear pre-processing time and query time. The query is in the form of $(G, x)$ where $G$ is a graph with underlying clusters and $x \\in V$ is a vertex. The goal is to (1) construct the oracle efficiently, (2) report which cluster vertex $x$ belongs to efficiently.\n\nComparing to the previous work, the main contribution is improvement on the pre-processing time, which reduces exponential to polynomial on $O(k/\\varepsilon)$, but blows up the misclassification error from $\\log k$ to $\\text{poly}(k)$, also slightly relaxing an assumption on the gap between inner and outer conductance.  The query time is asym-same. \n\nThe main technique is to replace the exhaustive search for each sampled vertex to decide a vertex $x$ belongs to a cluster with center $\\mu$, with estimating the inner product of their spectral embeddings. It is proved the magnitude of the inner product roughly shows if two vertices belong to the same cluster.\n\nSTRENGTHS:\n- The paper is clearly written and well organized.\n- The result is neat, the proposed algorithms are more easy to implement comparing to previous ones.\n- I think for a theory paper, having experiments is always a plus. However, there is a mentality that either do it well or just don't do it. The experiments can be improved or at least clarified better.\n\nWEAKNESSES:\n- The major concern is that the contribution of the main result is quite limited. Yes, $O(k/\\varepsilon)$ is an important factor, but it still in the $\\tilde{\\Omega}(\\sqrt{n})$ regime, not to mention compromise on others. I actually like the robustness result better.\n- On the experiments, if the authors want to keep and improve the section, I would suggest:\n  - (1) Clarify the evaluation. I believe the theoretical result on the error is the number of query instances? Then the current report does not look like so, is it the fraction?\n  - (2) The issue of query complexity is not mentioned before. It is out of blue. Explain why you do it.\n  - (3) Add the robustness experiments.\n\nQUESTIONS:\nFor the oracle results, there is another measure-of interest on the space. It would be good to report and compare with previous ones on this. \n\nJust some curiosity on the lower bound. Is there any result stating something like: to obtain a small error, at least some space / time is needed?\n\nAnother suggestion is to give some conceptual description of good and bad vertices before the definition, maybe in the main technique part.\n\nLIMITATIONS:\nSee above.\n\nMy reason to give the current assessment is mainly on the technical novelty limit.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}