Estimation of Number of Communities in Assortative Sparse Networks

1 INTRODUCTION . Statistical analysis of network data has now become an extensively studied field within statistics and machine learning ( see ( Goldenberg et al. , 2010 ; Kolaczyk & Csárdi , 2014 ; Newman , 2018 ) for reviews ) . Network datasets show up in several disciplines . Examples include networks originating from biosciences such as gene regulation networks ( Emmert-Streib et al . ( 2014 ) ) , protein-protein interaction networks ( De Las Rivas & Fontanillo ( 2010 ) ) , structural ( Rubinov & Sporns ( 2010 ) ) and functional networks ( Friston ( 2011 ) ) of brain and epidemiological networks ( Reis et al . ( 2007 ) ) ; networks originating from social media such as Facebook , Twitter and LinkedIn ( Faloutsos et al . ( 2010 ) ) ; citation and collaboration networks ( Lehmann et al . ( 2003 ) ) ; information and technological networks such as internet-based networks ( Adamic & Glance ( 2005 ) ) , power networks ( Pagani & Aiello ( 2013 ) ) and cell-tower networks ( Isaacman et al . ( 2011 ) ) . There are several active areas of research in developing statistical methodologies for network data analysis and also deriving the theoretical properties of the statistical methods . In this paper , we focus on networks with community structure and finding the number of communities in networks with arbitrary sparsity level . The last two decades saw a resurgence of interest in a problem popularly known as “ community detection '' . A common problem definition is to partition N nodes in a graph into K communities such that there are differences in edge densities between within and between communities , where K is assumed to be known a priori . Estimating number of communities ( K ) has recently become active in the literature . While the initial focus in the literature for estimating K has been developing algorithms and drawing support from domain-specific intuition and empirical studies using the Stochastic Block Model ( SBM ) , first proposed in Holland et al . ( 1983 ) , ( such as , Saade et al . ( 2014a ) , Yan et al . ( 2018 ) ) , there has been recent progress on attaining theoretical understanding of community numbers . Bickel & Sarkar ( 2015 ) and Lei et al . ( 2016 ) proposed hypothesis testing approaches based on principal eigenvalues or singular values . Some likelihood-based methods using the BIC criterion were proposed by Wang et al . ( 2017 ) and Hu et al . ( 2019 ) . From a Bayesian perspective , Riolo et al . ( 2017 ) discussed priors for number of communities under the SBM and designed an Markov Chain Monte Carlo algorithm , Kemp et al . ( 2006 ) presented a nonparametric Bayesian approach for detecting concept systems , Xu et al . ( 2006 ) introduced an infinite-state latent variable as part of a Dirichlet process mixture model , and Cerqueira & Leonardi ( 2020 ) proposed an estimator based on integrated likelihood for the SBM . Rosvall & Bergstrom ( 2007 ) introduced the concept of the minimum description length ( MDL ) to describe network modularities in partitioning networks , and Peixoto ( 2013 ) employed MDL to detect the number of communities . Chen & Lei ( 2018 ) and Li et al . ( 2020 ) proposed cross-validation based approaches with theoretical guarantees to estimate K. Yan et al . ( 2018 ) proposed a semi-definite programming approach , and Ma et al . ( 2018 ) proposed an estimator based on the loss of binary segmentation using pseudo-likelihood ratio . All of these approaches had theoretical guarantees . However , all the theoretical results were obtained under the assumption that mean density of the networks is greater than log ( N ) . Methods based on the spectrum of a certain class of matrices have become increasingly popular in recent years as non-parametric alternatives that are more computationally efficient and applicable to a wider range of settings . Most notably the non-backtracking matrices ( e.g. , Krzakala et al . ( 2013 ) , Saade et al . ( 2014b ) , Coste & Zhu ( 2019 ) , Bordenave et al . ( 2015 ) , Saade et al . ( 2016 ) ) and the Bethe Hessian matrices ( e.g. , Saade et al . ( 2015b ) , Lelarge ( 2018 ) , Dall ’ Amico et al . ( 2019 ) , Saade et al . ( 2015a ) , Dall ’ Amico et al . ( 2020 ) , Saade et al . ( 2014a ) , Le & Levina ( 2015 ) ) have received much attention due to their non-parametric form and competitive performance in the presence of degree heterogeneity and sparsity . In particular , unlike the non-backtracking operator , the Bethe Hessian is a real symmetric operator and hence offers additional computational advantages . Through simulations , Saade et al . ( 2014a ) demonstrated that the Bethe Hessian outperformed the non-backtracking operator , belief propagation , and the adjacency matrices on clustering on both accuracy and efficiency . Le & Levina ( 2015 ) proved the consistency of the method based on the spectrum of the Bethe Hessian operator in semi-dense regimes , i.e. , with the expected degree d̃ log ( N ) and the scalar parameter chosen from the two values commonly used in the literature based on heuristics for assortative and disassortative networks . However , other than the two candidate values and their variations , there are no other known values for the scalar parameter to ensure the consistency result in any regime . Furthermore , real-world networks are generally much more sparse and there is no theoretical result in the literature that guarantees the effectiveness of the Bethe Hessian operator in more sparse regimes . Our contribution . In this paper , we contribute to the theoretical understanding of the Bethe Hessian operator in estimating K for networks generated from the SBM in any regime regardless of the sparsity . We have three main contributions . • We show that the method of estimating K based on the spectral properties of the Bethe Hessian matrix ( `` spectral method '' ) is statistically consistent , even in regimes more sparse than those previously considered in the literature , with the expected degree 1 d̃ log ( N ) . The precise definition of d̃ is given in §2.1 . • We provide the first-of-its-kind interval of values for the scalar parameter of the Bethe Hessian operator that serves as a sufficient condition for the spectral method to correctly estimate K asymptotically in network data . • Through extensive simulations , we demonstrate that for any value chosen from the interval for the scalar parameter , the spectral method correctly estimates K in networks regardless of sparsity . We also consider the heuristics-based values commonly used in the literature for the scalar parameter in the context of the interval . The paper is arranged as follows . We present the definitions and a formal problem statement in §2 . We present our main theoretical result and a sketch of the proof in §3 , followed by empirical methods in §4 . The simulation results and concluding remarks are given in §5 and §6 , respectively . 2 PRELIMINARIES . 2.1 NOTATION . An adjacency matrix , denoted by A , is a random matrix whose rows and columns are labeled by nodes i , j ∈ [ N ] , where Aij = 1 if there is an edge between nodes i and j and 0 otherwise , and [ N ] denotes the set { 1 , . . . , N } . The mean observed degree is denoted by d̄ : = 1N 1 T NA1N and the expected degree by d̃ : = 1N 1 T NEA1N . λ ↓ ` ( A ) denotes the ` -th largest eigenvalue of A and λ ↑ ` ( A ) denotes the ` -th smallest eigenvalue of A . 2.2 THE STOCHASTIC BLOCK MODEL . The stochastic block model ( SBM ) is a simple generative model for network data that embeds a community structure in an adjacency matrix AN×N of the randomly generated network . SBM has three parameters : ( 1 ) the number of communities K ; ( 2 ) the membership vector z = ( z1 , ... , zN ) that assigns a community label zi ∈ [ K ] to each node i ∈ [ N ] ; and ( 3 ) the connectivity probability matrix BK×K where the elementBab represents the probability of an edge between nodes belonging to community a and b , where a , b ∈ [ K ] . Z ∈ ZN×K > 0 is defined as the community membership matrix such that Zij = 1 if node i belongs to community j and 0 otherwise . We denote the maximum expected degree by dmax : = N maxi ∑N j=1 [ ( ZBZ T ) ij − Diag ( ZBZT ) ij ] and the maximum entry in matrix B by d/N , where d : = N maxa , b∈ [ K ] Bab . λ denotes the smallest eigenvalue of the normalized B matrix , λ : = λ↓K ( N d B ) . Ā is the expectation of A and is computed as Ā = ZBZT − Diag ( ZBZT ) . D̄ is a diagonal matrix whose i-th diagonal entry is the sum of the i-th row of Ā . Let N be the vector of true community sizes and Nmin denotes the number of nodes in the community with the lowest number of nodes in it . A network generated from the SBM with parameters K , B , Z is defined to be assortative if Baa > Bab for all a , b ∈ [ K ] with a 6= b , and if B has all positive eigenvalues ( i.e. , B has full-rankK ) . The existing works in the literature on the spectral method referenced above have considered assortative networks , and we also consider assortative networks in this paper . 2.3 THE BETHE HESSIAN MATRIX . The Bethe Hessian matrix associated with an adjacency matrix A is defined as Hζ : = ( ζ2 − 1 ) IN + D− ζA ( 2.1 ) where ζ > 1 is a real scalar parameter , D : = Diag ( A1N ) is a diagonal matrix whose i-th diagonal entry corresponds to the degree of the i-th node , and IN is an identity matrix of dimension N ×N . As a real symmetric operator , Hζ is analytically tractable and computationally efficient , and has a number of useful properties . Saade et al . ( 2014a ) demonstrated that the community structure in A can be recovered by applying a standard clustering algorithm ( such as k-means clustering ) to the eigenvectors of Hζ corresponding to negative eigenvalues . In the spectral clustering literature , those eigenvalues whose eigenvectors encode the community structure are known as the informative eigenvalues and have been observed to be well-separated from the bulk of the spectrum . In Saade et al . ( 2014a ) , ζ was set to be the square-root of the mean observed degree as a heuristic to render informative ( negative ) eigenvalues of Hζ . Le & Levina ( 2015 ) showed that the number of informative eigenvalues of Hζ directly estimateK in the semi-dense regime ( d̃ log ( N ) ) when ζ is set to be either rm : = ( d1+ · · ·+dN ) −1 ( d21+ · · ·+ d2N ) −1 or ra : = √ ( d1 + · · ·+ dN ) /N . Both rm and ra are obtained based on heuristic arguments and are commonly used in the literature to estimate the radius of the bulk of the spectra . ra was considered in Saade et al . ( 2014a ) and the choice of rm stems from the deep connection between the spectrum of Hζ and that of another matrix which is known as the non-backtracking operator B. Denoting by m the number of edges in A , B is a 2m× 2m matrix indexed by directed edges i→ j and defined Bi→j , k→l = δjk ( 1− δil ) , where δ is the Kronecker delta and m is the number of edges . As in Hζ , the informative eigenvalues of B are well-separated from the bulk of its spectrum and are real , so it also has been used to develop many popular non-parametric methods for clustering ( see e.g. , Saade et al . ( 2014b ) , Coste & Zhu ( 2019 ) , Bordenave et al . ( 2015 ) , Bruna & Li ( 2017 ) , Gulikers et al . ( 2016 ) ) . This deep connection between Hζ and B was noted in Krzakala et al . ( 2013 ) and can be summarized by the phenomenon that , given any eigenvalue ν of B , the determinant of Hν vanishes . However , unlike Hζ , B is non-symmetric and its dimension ( 2m × 2m ) can get quite large . These present analytical and computational challenges when using B , and in turn have popularized Hζ as a tool for clustering . Le & Levina ( 2015 ) showed that in semi-dense regimes with expected degree d̃ log ( N ) , the number of negative eigenvalues of Hζ directly estimate K for ζ ∈ { rm , ra } , where the methods were called BHm and BHa . In addition , it was noted that the number of negative eigenvalues of Hζ tend to underestimate K when networks are unbalanced . Hence , corrections for BHm and BHa were proposed , namely BHmc and BHac , which heuristically estimate K̂ = max { k : tρn−k+1 6 ρn−k } where ρ1 > · · · > ρN are sorted eigenvalues and t > 0 is the hyperparameter . In light of this , we present the following problem we focus on in this paper : Problem Definition : Suppose that we observe one network generated from the SBM , where the parameters K , Z , B satisfy ( i ) assortativity , and ( ii ) the sparsity condition d̃ = o ( log ( N ) ) . For the appropriate choices of ζ , are the negative eigenvalues of the Bethe Hessian matrix Hζ still informative for estimating K ? If so , what are the appropriate choices for ζ ? Can there be other heuristic choices for ζ ? Are the popular heuristic choices of ζ , i.e. , rm and ra as defined above ( hereinafter “ heuristic choices '' ) , appropriate in the above sense ?

Summary: In this paper the authors consider the problem of computing the number of communities K in an arbitrarily sparse graph generated under the Stochastic Block Model (SMB). Previous studies that consider the problem of computing K show theoretical guarantees only in graphs with average degree $\Omega(\log n)$. One of the previous studies (namely, [1]) has shown that  the number of communities equals the number of negative eigenvalues of the Bethe Hessian matrix for graphs with expected average degree $\Omega(\log n)$ under SBM. In this paper the authors show that with an appropriate scalar parameter for the Bethe Hessian matrix of graphs with average degree $o(\log n)$, the same property still holds, and thus obtain a method for computing K in graphs of sublogarithmic density. In particular, the authors give an interval for the choice of the $\zeta$ scalar that depends on several parameters of the underlying SBM distribution.