Streaming Belief Propagation for Community Detection

1 Introduction . Given a single realization of a network G = ( V , E ) , the community detection problem requires to find a partition of its vertices into a small number of clusters or ‘ communities ’ [ GN02 , MO04 , JTZ04 , For10 , PKVS12 , JYL+18 ] . Numerous methods have been developed for community detection on static networks [ GN02 , CNM04 , GA05 , RCC+04 , LF09 , VLBB08 , WZCX18 ] . However , the network structure evolves over time in most applications . For instance , in social networks new users can join the network ; in online commerce new products can be listed ; see [ WGKM18 , GLZ08 , KvB+14 ] for other examples . In such dynamic settings , it is desirable to have algorithms that perform only a limited number of operations each time a node joins or leaves , possibly revising the labels of nodes in a neighborhood of the new node . ( These notions will be formalized below . ) Several groups have developed algorithms of this type in the recent past [ HS12 , CSG16 , ZWCY19 , MKV+20 ] . The present paper aims at characterizing the fundamental statistical limits of community detection in the dynamic setting , and proposing algorithms that achieve those limits . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . As usual , establishing fundamental statistical limits requires introducing a statistical model for the networkG = ( V , E ) . In the case of static networks , precise characterizations have only been achieved recently , and almost uniquely for a simple network model , namely the stochastic block model ( SBM ) [ HLL83 , ABFX08 , KN11 , RCY+11 , DKMZ11 , Mas14 , ABH15 , AS15 , MNS18 ] . In this paper we build on these recent advances and study a dynamic generalization of the SBM , which we refer to as streaming SBM ( StSBM ) . The SBM can be defined as follows . Each vertex v is given a label τ ( v ) drawn independently from a fixed distribution over the set [ k ] = { 1 , 2 , . . . , k } . Edges are conditionally independent given vertex labels . Two vertices u , v are connected by an edge with probability Wτ ( u ) , τ ( v ) . The analyst is given a realization of the graph and required to estimate the labels τ . We will assume in addition that the analyst has access to some noisy version of the vertex labels , denoted by τ̃ = ( τ̃ ( v ) ) v∈V . This is a mathematically convenient generalization : the special case in which no noisy observations τ̃ are available can be captured by letting τ̃ be independent of τ . Further , such a generalization is useful to model cases in which covariate information is available at the nodes [ MX16 ] . Informally , the streaming SBM ( StSBM ) is a version of SBM in which nodes are revealed one at a time in random order ( see below for a formal definition ) . In order to model the notion that only a limited number of updates is performed each time a new node joins the network , we introduce a class of ‘ local streaming algorithms. ’ These encompass several algorithms in earlier literature , e.g , [ ZGL03 , CSG16 ] . Our definition is inspired and motivated by the more classical definition of local algorithms for static graphs . Local algorithms output estimates for one vertex based only on a small neighborhood around it , and thus scale well to large graphs ; see [ Suo13 ] for a survey . A substantial literature studies the behavior of local algorithms for sparse graphs [ GT12 , HLS14 , GS14 , Mon15 , MX16 , FM17 ] . Our results focus on the sparse regime in which the graph ’ s average degree is bounded . This is the most challenging regime for the SBM , and it is also relevant for real-world applications where networks are usually sparse . We present the following contributions : Fundamental limitations of local streaming algorithms . We prove that , in the absence of side information , in streaming symmetric SBM ( introduced in Section 2 ) , local streaming algorithms ( introduced in Section 3 ) do not achieve any non-trivial reconstruction : their accuracy is asymptotically the same as random guessing ; see Corollary 1 . This holds despite the fact that there exist polynomial time non-local algorithms that achieve significantly better accuracy . From a practical viewpoint , this indicates that methods with a small ‘ locality radius ’ ( the range over which the algorithm updates its estimates ) are ineffective at aggregating information : they perform poorly unless strong local side information is available . Optimality of streaming belief propagation . On the positive side , in Section 4 we define a streaming version of belief propagation ( BP ) , a local streaming algorithm that we call STREAMBP , parameterized by a locality radius R. We prove that , for any non-vanishing amount of side information , STREAMBP achieves the same reconstruction accuracy as offline BP ; see Theorem 2 . The latter is in turn conjectured to be the optimal offline polynomial-time algorithm [ DKMZ11 , Abb17 , HS17 ] . Under this conjecture , there is no loss of performance in restricting to local streaming algorithms as long as ( 1 ) local side information is available ; ( 2 ) the locality radius is sufficiently large ; and ( 3 ) information is aggregated optimally via STREAMBP . Let us emphasize that we do not claim ( nor do we expect ) STREAMBP to outperform offline BP . We use offline BP as an ‘ oracle ’ benchmark ( as it has the full graph information available , and is not constrained to act in a streaming fashion ) . Implementation and numerical experiments . In Section 5 and Appendix A–C , we validate our results both on synthetic data , generated according to the StSBM , and on real datasets . Our empirical results are consistent with the theory ; in particular , STREAMBP substantially outperforms simple voting methods . However , we observe that it can behave poorly with large locality radius R. In order to overcome this problem , we introduce a ‘ bounded distance ’ version of STREAMBP , called STREAMBP∗ , which appears to be more robust . ( STREAMBP∗ can be shown to enjoy the same theoretical guarantees as STREAMBP . ) 2 Streaming stochastic block model . In this section we present a formal definition of the proposed model . The streaming stochastic block model is a probability distribution StSBM ( n , k , p , W , α ) over triples ( τ , τ̃ , G ) where τ ∈ [ k ] n is a vector of labels ( here [ k ] , { 1 , . . . , k } ) , τ̃ ∈ [ k ] n are noisy observations of the labels τ , and G = ( G ( 0 ) , G ( 1 ) , . . . , G ( n ) ) is a sequence of undirected graphs . Here G ( t ) = ( V ( t ) , E ( t ) ) is a graph over |V ( t ) | = t vertices and , for each 0 ≤ t ≤ n− 1 , V ( t ) ⊆ V ( t+ 1 ) and E ( t ) ⊆ E ( t+ 1 ) . We will assume , without loss of generality , that V ( n ) = [ n ] , and interpret τ ( v ) as the label associated to vertex v ∈ [ n ] . For each 0 ≤ t ≤ n − 1 , G ( t ) is the subgraph induced in G ( t + 1 ) by V ( t ) ; equivalently , all edges in E ( t+ 1 ) \ E ( t ) are incident to the unique vertex in V ( t+ 1 ) \ V ( t ) . The distribution StSBM ( n , k , p , W , α ) is parameterized by a scalar α ∈ [ 0 , k−1k ] , a probability vector p = ( p1 , . . . , pk ) ∈ ∆k , { x ∈ [ 0 , 1 ] k , 〈x , 1〉 = 1 } , and a symmetric matrix W ∈ [ 0 , 1 ] k×k . We draw the coordinates of τ independently with distribution p , and set τ̃ ( v ) = τ ( v ) with probability 1− α , and τ̃ ( v ) ∼ Unif ( [ k ] \ { τ ( v ) } ) otherwise , independently across vertices : P ( τ ( v ) = s ) = ps , P ( τ̃ ( v ) = s1 | τ ( v ) = s0 ) = { 1− α if s1 = s0 , α/ ( k − 1 ) if s1 6= s0 . We then construct G ( n ) by generating conditionally independent edges , given ( τ , τ̃ ) , with P ( ( u , v ) ∈ E ( n ) | τ , τ̃ ) = Wτ ( u ) , τ ( v ) . ( 1 ) Note that the labels τ̃ provide noisy ‘ side information ’ about the true labels τ . This information τ̃ is conditionally independent of the graph G ( n ) given τ . Finally we generate the graph sequence G by choosing a uniformly random permutation of the vertices ( v ( 1 ) , v ( 2 ) , . . . , v ( n ) ) and setting V ( t ) = { v ( 1 ) , . . . , v ( t ) } and G ( t ) to the graph induced by V ( t ) . If v = v ( t ) , then we define t as the arrival order of vertex v. Note that , for each t , G ( t ) is distributed according to a standard SBM with t vertices : G ( t ) ∼ SBM ( t , k , p , W ) . An equivalent description is that ( conditional on τ , τ̃ ) StSBM ( n , k , p , W , α ) defines a Markov chain over graphs . The new graph G ( t + 1 ) is generated from G ( t ) by drawing the vertex v ( t + 1 ) uniformly at random from [ n ] \ V ( t ) , and then the edges ( u , v ( t+ 1 ) ) , u ∈ V ( t ) , independently with probabilities given by Equation ( 1 ) . We are interested in the behavior of large graphs with bounded average degree . In order to focus on this regime , we will consider n → ∞ and W = W ( n ) → 0 with W ( n ) = W0/n for a matrix W0 ∈ Rk×k≥0 independent of n. A case of special interest is the streaming symmetric SBM , StSSBM ( n , k , a , b , α ) , which corresponds to taking p = ( 1/k , . . . , 1/k ) and W0 having diagonal elements a and non-diagonal elements b . Finally , the case α = ( k − 1 ) /k corresponds to pure noise τ̃ : in this case we can drop τ̃ from the observations and we will drop α from the distribution parameters . hhnni矗 鸞 ag.ci Nine nazis hdnet 矗tanto 䎖矗趱 tanto engin tanto

Summary: This paper proposes to do community detection in a variant of the stochastic block model for which nodes arrive in a streaming fashion, and there may exists noisy estimates of the real labels. They prove that if the noise render the side information uninformative, a local streaming algorithm is equivalent to a local algorithm, which cannot achieve non-trivial accuracy. If the side information is informative, they provide a belief propagation algorithm in the streaming setting which achieves comparable accuracy to offline BP asymptotically. Offline BP has been conjectured to be optimal, so their algorithm would also be optimal if the conjecture is proven (it is in special cases). They also provide another algorithm which is in practice more consistent than StreamBP.