Estimation of Number of Communities in Assortative Sparse Networks

1 INTRODUCTION . Statistical analysis of network data has now become an extensively studied field within statistics and machine learning ( see ( Goldenberg et al. , 2010 ; Kolaczyk & Csárdi , 2014 ; Newman , 2018 ) for reviews ) . Network datasets show up in several disciplines . Examples include networks originating from biosciences such as gene regulation networks ( Emmert-Streib et al . ( 2014 ) ) , protein-protein interaction networks ( De Las Rivas & Fontanillo ( 2010 ) ) , structural ( Rubinov & Sporns ( 2010 ) ) and functional networks ( Friston ( 2011 ) ) of brain and epidemiological networks ( Reis et al . ( 2007 ) ) ; networks originating from social media such as Facebook , Twitter and LinkedIn ( Faloutsos et al . ( 2010 ) ) ; citation and collaboration networks ( Lehmann et al . ( 2003 ) ) ; information and technological networks such as internet-based networks ( Adamic & Glance ( 2005 ) ) , power networks ( Pagani & Aiello ( 2013 ) ) and cell-tower networks ( Isaacman et al . ( 2011 ) ) . There are several active areas of research in developing statistical methodologies for network data analysis and also deriving the theoretical properties of the statistical methods . In this paper , we focus on networks with community structure and finding the number of communities in networks with arbitrary sparsity level . The last two decades saw a resurgence of interest in a problem popularly known as “ community detection '' . A common problem definition is to partition N nodes in a graph into K communities such that there are differences in edge densities between within and between communities , where K is assumed to be known a priori . Estimating number of communities ( K ) has recently become active in the literature . While the initial focus in the literature for estimating K has been developing algorithms and drawing support from domain-specific intuition and empirical studies using the Stochastic Block Model ( SBM ) , first proposed in Holland et al . ( 1983 ) , ( such as , Saade et al . ( 2014a ) , Yan et al . ( 2018 ) ) , there has been recent progress on attaining theoretical understanding of community numbers . Bickel & Sarkar ( 2015 ) and Lei et al . ( 2016 ) proposed hypothesis testing approaches based on principal eigenvalues or singular values . Some likelihood-based methods using the BIC criterion were proposed by Wang et al . ( 2017 ) and Hu et al . ( 2019 ) . From a Bayesian perspective , Riolo et al . ( 2017 ) discussed priors for number of communities under the SBM and designed an Markov Chain Monte Carlo algorithm , Kemp et al . ( 2006 ) presented a nonparametric Bayesian approach for detecting concept systems , Xu et al . ( 2006 ) introduced an infinite-state latent variable as part of a Dirichlet process mixture model , and Cerqueira & Leonardi ( 2020 ) proposed an estimator based on integrated likelihood for the SBM . Rosvall & Bergstrom ( 2007 ) introduced the concept of the minimum description length ( MDL ) to describe network modularities in partitioning networks , and Peixoto ( 2013 ) employed MDL to detect the number of communities . Chen & Lei ( 2018 ) and Li et al . ( 2020 ) proposed cross-validation based approaches with theoretical guarantees to estimate K. Yan et al . ( 2018 ) proposed a semi-definite programming approach , and Ma et al . ( 2018 ) proposed an estimator based on the loss of binary segmentation using pseudo-likelihood ratio . All of these approaches had theoretical guarantees . However , all the theoretical results were obtained under the assumption that mean density of the networks is greater than log ( N ) . Methods based on the spectrum of a certain class of matrices have become increasingly popular in recent years as non-parametric alternatives that are more computationally efficient and applicable to a wider range of settings . Most notably the non-backtracking matrices ( e.g. , Krzakala et al . ( 2013 ) , Saade et al . ( 2014b ) , Coste & Zhu ( 2019 ) , Bordenave et al . ( 2015 ) , Saade et al . ( 2016 ) ) and the Bethe Hessian matrices ( e.g. , Saade et al . ( 2015b ) , Lelarge ( 2018 ) , Dall ’ Amico et al . ( 2019 ) , Saade et al . ( 2015a ) , Dall ’ Amico et al . ( 2020 ) , Saade et al . ( 2014a ) , Le & Levina ( 2015 ) ) have received much attention due to their non-parametric form and competitive performance in the presence of degree heterogeneity and sparsity . In particular , unlike the non-backtracking operator , the Bethe Hessian is a real symmetric operator and hence offers additional computational advantages . Through simulations , Saade et al . ( 2014a ) demonstrated that the Bethe Hessian outperformed the non-backtracking operator , belief propagation , and the adjacency matrices on clustering on both accuracy and efficiency . Le & Levina ( 2015 ) proved the consistency of the method based on the spectrum of the Bethe Hessian operator in semi-dense regimes , i.e. , with the expected degree d̃ log ( N ) and the scalar parameter chosen from the two values commonly used in the literature based on heuristics for assortative and disassortative networks . However , other than the two candidate values and their variations , there are no other known values for the scalar parameter to ensure the consistency result in any regime . Furthermore , real-world networks are generally much more sparse and there is no theoretical result in the literature that guarantees the effectiveness of the Bethe Hessian operator in more sparse regimes . Our contribution . In this paper , we contribute to the theoretical understanding of the Bethe Hessian operator in estimating K for networks generated from the SBM in any regime regardless of the sparsity . We have three main contributions . • We show that the method of estimating K based on the spectral properties of the Bethe Hessian matrix ( `` spectral method '' ) is statistically consistent , even in regimes more sparse than those previously considered in the literature , with the expected degree 1 d̃ log ( N ) . The precise definition of d̃ is given in §2.1 . • We provide the first-of-its-kind interval of values for the scalar parameter of the Bethe Hessian operator that serves as a sufficient condition for the spectral method to correctly estimate K asymptotically in network data . • Through extensive simulations , we demonstrate that for any value chosen from the interval for the scalar parameter , the spectral method correctly estimates K in networks regardless of sparsity . We also consider the heuristics-based values commonly used in the literature for the scalar parameter in the context of the interval . The paper is arranged as follows . We present the definitions and a formal problem statement in §2 . We present our main theoretical result and a sketch of the proof in §3 , followed by empirical methods in §4 . The simulation results and concluding remarks are given in §5 and §6 , respectively . 2 PRELIMINARIES . 2.1 NOTATION . An adjacency matrix , denoted by A , is a random matrix whose rows and columns are labeled by nodes i , j ∈ [ N ] , where Aij = 1 if there is an edge between nodes i and j and 0 otherwise , and [ N ] denotes the set { 1 , . . . , N } . The mean observed degree is denoted by d̄ : = 1N 1 T NA1N and the expected degree by d̃ : = 1N 1 T NEA1N . λ ↓ ` ( A ) denotes the ` -th largest eigenvalue of A and λ ↑ ` ( A ) denotes the ` -th smallest eigenvalue of A . 2.2 THE STOCHASTIC BLOCK MODEL . The stochastic block model ( SBM ) is a simple generative model for network data that embeds a community structure in an adjacency matrix AN×N of the randomly generated network . SBM has three parameters : ( 1 ) the number of communities K ; ( 2 ) the membership vector z = ( z1 , ... , zN ) that assigns a community label zi ∈ [ K ] to each node i ∈ [ N ] ; and ( 3 ) the connectivity probability matrix BK×K where the elementBab represents the probability of an edge between nodes belonging to community a and b , where a , b ∈ [ K ] . Z ∈ ZN×K > 0 is defined as the community membership matrix such that Zij = 1 if node i belongs to community j and 0 otherwise . We denote the maximum expected degree by dmax : = N maxi ∑N j=1 [ ( ZBZ T ) ij − Diag ( ZBZT ) ij ] and the maximum entry in matrix B by d/N , where d : = N maxa , b∈ [ K ] Bab . λ denotes the smallest eigenvalue of the normalized B matrix , λ : = λ↓K ( N d B ) . Ā is the expectation of A and is computed as Ā = ZBZT − Diag ( ZBZT ) . D̄ is a diagonal matrix whose i-th diagonal entry is the sum of the i-th row of Ā . Let N be the vector of true community sizes and Nmin denotes the number of nodes in the community with the lowest number of nodes in it . A network generated from the SBM with parameters K , B , Z is defined to be assortative if Baa > Bab for all a , b ∈ [ K ] with a 6= b , and if B has all positive eigenvalues ( i.e. , B has full-rankK ) . The existing works in the literature on the spectral method referenced above have considered assortative networks , and we also consider assortative networks in this paper . 2.3 THE BETHE HESSIAN MATRIX . The Bethe Hessian matrix associated with an adjacency matrix A is defined as Hζ : = ( ζ2 − 1 ) IN + D− ζA ( 2.1 ) where ζ > 1 is a real scalar parameter , D : = Diag ( A1N ) is a diagonal matrix whose i-th diagonal entry corresponds to the degree of the i-th node , and IN is an identity matrix of dimension N ×N . As a real symmetric operator , Hζ is analytically tractable and computationally efficient , and has a number of useful properties . Saade et al . ( 2014a ) demonstrated that the community structure in A can be recovered by applying a standard clustering algorithm ( such as k-means clustering ) to the eigenvectors of Hζ corresponding to negative eigenvalues . In the spectral clustering literature , those eigenvalues whose eigenvectors encode the community structure are known as the informative eigenvalues and have been observed to be well-separated from the bulk of the spectrum . In Saade et al . ( 2014a ) , ζ was set to be the square-root of the mean observed degree as a heuristic to render informative ( negative ) eigenvalues of Hζ . Le & Levina ( 2015 ) showed that the number of informative eigenvalues of Hζ directly estimateK in the semi-dense regime ( d̃ log ( N ) ) when ζ is set to be either rm : = ( d1+ · · ·+dN ) −1 ( d21+ · · ·+ d2N ) −1 or ra : = √ ( d1 + · · ·+ dN ) /N . Both rm and ra are obtained based on heuristic arguments and are commonly used in the literature to estimate the radius of the bulk of the spectra . ra was considered in Saade et al . ( 2014a ) and the choice of rm stems from the deep connection between the spectrum of Hζ and that of another matrix which is known as the non-backtracking operator B. Denoting by m the number of edges in A , B is a 2m× 2m matrix indexed by directed edges i→ j and defined Bi→j , k→l = δjk ( 1− δil ) , where δ is the Kronecker delta and m is the number of edges . As in Hζ , the informative eigenvalues of B are well-separated from the bulk of its spectrum and are real , so it also has been used to develop many popular non-parametric methods for clustering ( see e.g. , Saade et al . ( 2014b ) , Coste & Zhu ( 2019 ) , Bordenave et al . ( 2015 ) , Bruna & Li ( 2017 ) , Gulikers et al . ( 2016 ) ) . This deep connection between Hζ and B was noted in Krzakala et al . ( 2013 ) and can be summarized by the phenomenon that , given any eigenvalue ν of B , the determinant of Hν vanishes . However , unlike Hζ , B is non-symmetric and its dimension ( 2m × 2m ) can get quite large . These present analytical and computational challenges when using B , and in turn have popularized Hζ as a tool for clustering . Le & Levina ( 2015 ) showed that in semi-dense regimes with expected degree d̃ log ( N ) , the number of negative eigenvalues of Hζ directly estimate K for ζ ∈ { rm , ra } , where the methods were called BHm and BHa . In addition , it was noted that the number of negative eigenvalues of Hζ tend to underestimate K when networks are unbalanced . Hence , corrections for BHm and BHa were proposed , namely BHmc and BHac , which heuristically estimate K̂ = max { k : tρn−k+1 6 ρn−k } where ρ1 > · · · > ρN are sorted eigenvalues and t > 0 is the hyperparameter . In light of this , we present the following problem we focus on in this paper : Problem Definition : Suppose that we observe one network generated from the SBM , where the parameters K , Z , B satisfy ( i ) assortativity , and ( ii ) the sparsity condition d̃ = o ( log ( N ) ) . For the appropriate choices of ζ , are the negative eigenvalues of the Bethe Hessian matrix Hζ still informative for estimating K ? If so , what are the appropriate choices for ζ ? Can there be other heuristic choices for ζ ? Are the popular heuristic choices of ζ , i.e. , rm and ra as defined above ( hereinafter “ heuristic choices '' ) , appropriate in the above sense ?

Summary: The authors propose a spectral framework using the Bethe Hessian matrix to infer the number of communities in sparse networks. The method relies on the eigendecomposition of the Bethe Hessian matrix for which negative eigenvalues are preserved and the number of such eigenvalues used to define the number of communities. In particular, theoretical guarantees for settings of the scalar of the Bethe Hessian matrix is derived including an associated spectral estimation procedure.