Streaming Belief Propagation for Community Detection

1 Introduction . Given a single realization of a network G = ( V , E ) , the community detection problem requires to find a partition of its vertices into a small number of clusters or ‘ communities ’ [ GN02 , MO04 , JTZ04 , For10 , PKVS12 , JYL+18 ] . Numerous methods have been developed for community detection on static networks [ GN02 , CNM04 , GA05 , RCC+04 , LF09 , VLBB08 , WZCX18 ] . However , the network structure evolves over time in most applications . For instance , in social networks new users can join the network ; in online commerce new products can be listed ; see [ WGKM18 , GLZ08 , KvB+14 ] for other examples . In such dynamic settings , it is desirable to have algorithms that perform only a limited number of operations each time a node joins or leaves , possibly revising the labels of nodes in a neighborhood of the new node . ( These notions will be formalized below . ) Several groups have developed algorithms of this type in the recent past [ HS12 , CSG16 , ZWCY19 , MKV+20 ] . The present paper aims at characterizing the fundamental statistical limits of community detection in the dynamic setting , and proposing algorithms that achieve those limits . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . As usual , establishing fundamental statistical limits requires introducing a statistical model for the networkG = ( V , E ) . In the case of static networks , precise characterizations have only been achieved recently , and almost uniquely for a simple network model , namely the stochastic block model ( SBM ) [ HLL83 , ABFX08 , KN11 , RCY+11 , DKMZ11 , Mas14 , ABH15 , AS15 , MNS18 ] . In this paper we build on these recent advances and study a dynamic generalization of the SBM , which we refer to as streaming SBM ( StSBM ) . The SBM can be defined as follows . Each vertex v is given a label τ ( v ) drawn independently from a fixed distribution over the set [ k ] = { 1 , 2 , . . . , k } . Edges are conditionally independent given vertex labels . Two vertices u , v are connected by an edge with probability Wτ ( u ) , τ ( v ) . The analyst is given a realization of the graph and required to estimate the labels τ . We will assume in addition that the analyst has access to some noisy version of the vertex labels , denoted by τ̃ = ( τ̃ ( v ) ) v∈V . This is a mathematically convenient generalization : the special case in which no noisy observations τ̃ are available can be captured by letting τ̃ be independent of τ . Further , such a generalization is useful to model cases in which covariate information is available at the nodes [ MX16 ] . Informally , the streaming SBM ( StSBM ) is a version of SBM in which nodes are revealed one at a time in random order ( see below for a formal definition ) . In order to model the notion that only a limited number of updates is performed each time a new node joins the network , we introduce a class of ‘ local streaming algorithms. ’ These encompass several algorithms in earlier literature , e.g , [ ZGL03 , CSG16 ] . Our definition is inspired and motivated by the more classical definition of local algorithms for static graphs . Local algorithms output estimates for one vertex based only on a small neighborhood around it , and thus scale well to large graphs ; see [ Suo13 ] for a survey . A substantial literature studies the behavior of local algorithms for sparse graphs [ GT12 , HLS14 , GS14 , Mon15 , MX16 , FM17 ] . Our results focus on the sparse regime in which the graph ’ s average degree is bounded . This is the most challenging regime for the SBM , and it is also relevant for real-world applications where networks are usually sparse . We present the following contributions : Fundamental limitations of local streaming algorithms . We prove that , in the absence of side information , in streaming symmetric SBM ( introduced in Section 2 ) , local streaming algorithms ( introduced in Section 3 ) do not achieve any non-trivial reconstruction : their accuracy is asymptotically the same as random guessing ; see Corollary 1 . This holds despite the fact that there exist polynomial time non-local algorithms that achieve significantly better accuracy . From a practical viewpoint , this indicates that methods with a small ‘ locality radius ’ ( the range over which the algorithm updates its estimates ) are ineffective at aggregating information : they perform poorly unless strong local side information is available . Optimality of streaming belief propagation . On the positive side , in Section 4 we define a streaming version of belief propagation ( BP ) , a local streaming algorithm that we call STREAMBP , parameterized by a locality radius R. We prove that , for any non-vanishing amount of side information , STREAMBP achieves the same reconstruction accuracy as offline BP ; see Theorem 2 . The latter is in turn conjectured to be the optimal offline polynomial-time algorithm [ DKMZ11 , Abb17 , HS17 ] . Under this conjecture , there is no loss of performance in restricting to local streaming algorithms as long as ( 1 ) local side information is available ; ( 2 ) the locality radius is sufficiently large ; and ( 3 ) information is aggregated optimally via STREAMBP . Let us emphasize that we do not claim ( nor do we expect ) STREAMBP to outperform offline BP . We use offline BP as an ‘ oracle ’ benchmark ( as it has the full graph information available , and is not constrained to act in a streaming fashion ) . Implementation and numerical experiments . In Section 5 and Appendix A–C , we validate our results both on synthetic data , generated according to the StSBM , and on real datasets . Our empirical results are consistent with the theory ; in particular , STREAMBP substantially outperforms simple voting methods . However , we observe that it can behave poorly with large locality radius R. In order to overcome this problem , we introduce a ‘ bounded distance ’ version of STREAMBP , called STREAMBP∗ , which appears to be more robust . ( STREAMBP∗ can be shown to enjoy the same theoretical guarantees as STREAMBP . ) 2 Streaming stochastic block model . In this section we present a formal definition of the proposed model . The streaming stochastic block model is a probability distribution StSBM ( n , k , p , W , α ) over triples ( τ , τ̃ , G ) where τ ∈ [ k ] n is a vector of labels ( here [ k ] , { 1 , . . . , k } ) , τ̃ ∈ [ k ] n are noisy observations of the labels τ , and G = ( G ( 0 ) , G ( 1 ) , . . . , G ( n ) ) is a sequence of undirected graphs . Here G ( t ) = ( V ( t ) , E ( t ) ) is a graph over |V ( t ) | = t vertices and , for each 0 ≤ t ≤ n− 1 , V ( t ) ⊆ V ( t+ 1 ) and E ( t ) ⊆ E ( t+ 1 ) . We will assume , without loss of generality , that V ( n ) = [ n ] , and interpret τ ( v ) as the label associated to vertex v ∈ [ n ] . For each 0 ≤ t ≤ n − 1 , G ( t ) is the subgraph induced in G ( t + 1 ) by V ( t ) ; equivalently , all edges in E ( t+ 1 ) \ E ( t ) are incident to the unique vertex in V ( t+ 1 ) \ V ( t ) . The distribution StSBM ( n , k , p , W , α ) is parameterized by a scalar α ∈ [ 0 , k−1k ] , a probability vector p = ( p1 , . . . , pk ) ∈ ∆k , { x ∈ [ 0 , 1 ] k , 〈x , 1〉 = 1 } , and a symmetric matrix W ∈ [ 0 , 1 ] k×k . We draw the coordinates of τ independently with distribution p , and set τ̃ ( v ) = τ ( v ) with probability 1− α , and τ̃ ( v ) ∼ Unif ( [ k ] \ { τ ( v ) } ) otherwise , independently across vertices : P ( τ ( v ) = s ) = ps , P ( τ̃ ( v ) = s1 | τ ( v ) = s0 ) = { 1− α if s1 = s0 , α/ ( k − 1 ) if s1 6= s0 . We then construct G ( n ) by generating conditionally independent edges , given ( τ , τ̃ ) , with P ( ( u , v ) ∈ E ( n ) | τ , τ̃ ) = Wτ ( u ) , τ ( v ) . ( 1 ) Note that the labels τ̃ provide noisy ‘ side information ’ about the true labels τ . This information τ̃ is conditionally independent of the graph G ( n ) given τ . Finally we generate the graph sequence G by choosing a uniformly random permutation of the vertices ( v ( 1 ) , v ( 2 ) , . . . , v ( n ) ) and setting V ( t ) = { v ( 1 ) , . . . , v ( t ) } and G ( t ) to the graph induced by V ( t ) . If v = v ( t ) , then we define t as the arrival order of vertex v. Note that , for each t , G ( t ) is distributed according to a standard SBM with t vertices : G ( t ) ∼ SBM ( t , k , p , W ) . An equivalent description is that ( conditional on τ , τ̃ ) StSBM ( n , k , p , W , α ) defines a Markov chain over graphs . The new graph G ( t + 1 ) is generated from G ( t ) by drawing the vertex v ( t + 1 ) uniformly at random from [ n ] \ V ( t ) , and then the edges ( u , v ( t+ 1 ) ) , u ∈ V ( t ) , independently with probabilities given by Equation ( 1 ) . We are interested in the behavior of large graphs with bounded average degree . In order to focus on this regime , we will consider n → ∞ and W = W ( n ) → 0 with W ( n ) = W0/n for a matrix W0 ∈ Rk×k≥0 independent of n. A case of special interest is the streaming symmetric SBM , StSSBM ( n , k , a , b , α ) , which corresponds to taking p = ( 1/k , . . . , 1/k ) and W0 having diagonal elements a and non-diagonal elements b . Finally , the case α = ( k − 1 ) /k corresponds to pure noise τ̃ : in this case we can drop τ̃ from the observations and we will drop α from the distribution parameters . hhnni矗 鸞 ag.ci Nine nazis hdnet 矗tanto 䎖矗趱 tanto engin tanto

Summary: This paper studies a dynamic version of the stochastic block model, named as streaming stochastic block model (StSBM). The authors consider a sparse regime and analyze the asymptotic behavior of the model. In particular, the authors show that without some side information about the node labels, recovery of the true labels through a R-local streaming algorithm is no better than random guessing (Corollary 1). The authors also propose a streaming R-local belief propagation algorithm (StreamBP), and show their streaming algorithm perform as good as the offline version algorithm (Theorem 2). Empirical evaluations using synthetic and real-world datasets are provided in the paper.