Streaming Belief Propagation for Community Detection

1 Introduction . Given a single realization of a network G = ( V , E ) , the community detection problem requires to find a partition of its vertices into a small number of clusters or ‘ communities ’ [ GN02 , MO04 , JTZ04 , For10 , PKVS12 , JYL+18 ] . Numerous methods have been developed for community detection on static networks [ GN02 , CNM04 , GA05 , RCC+04 , LF09 , VLBB08 , WZCX18 ] . However , the network structure evolves over time in most applications . For instance , in social networks new users can join the network ; in online commerce new products can be listed ; see [ WGKM18 , GLZ08 , KvB+14 ] for other examples . In such dynamic settings , it is desirable to have algorithms that perform only a limited number of operations each time a node joins or leaves , possibly revising the labels of nodes in a neighborhood of the new node . ( These notions will be formalized below . ) Several groups have developed algorithms of this type in the recent past [ HS12 , CSG16 , ZWCY19 , MKV+20 ] . The present paper aims at characterizing the fundamental statistical limits of community detection in the dynamic setting , and proposing algorithms that achieve those limits . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . As usual , establishing fundamental statistical limits requires introducing a statistical model for the networkG = ( V , E ) . In the case of static networks , precise characterizations have only been achieved recently , and almost uniquely for a simple network model , namely the stochastic block model ( SBM ) [ HLL83 , ABFX08 , KN11 , RCY+11 , DKMZ11 , Mas14 , ABH15 , AS15 , MNS18 ] . In this paper we build on these recent advances and study a dynamic generalization of the SBM , which we refer to as streaming SBM ( StSBM ) . The SBM can be defined as follows . Each vertex v is given a label τ ( v ) drawn independently from a fixed distribution over the set [ k ] = { 1 , 2 , . . . , k } . Edges are conditionally independent given vertex labels . Two vertices u , v are connected by an edge with probability Wτ ( u ) , τ ( v ) . The analyst is given a realization of the graph and required to estimate the labels τ . We will assume in addition that the analyst has access to some noisy version of the vertex labels , denoted by τ̃ = ( τ̃ ( v ) ) v∈V . This is a mathematically convenient generalization : the special case in which no noisy observations τ̃ are available can be captured by letting τ̃ be independent of τ . Further , such a generalization is useful to model cases in which covariate information is available at the nodes [ MX16 ] . Informally , the streaming SBM ( StSBM ) is a version of SBM in which nodes are revealed one at a time in random order ( see below for a formal definition ) . In order to model the notion that only a limited number of updates is performed each time a new node joins the network , we introduce a class of ‘ local streaming algorithms. ’ These encompass several algorithms in earlier literature , e.g , [ ZGL03 , CSG16 ] . Our definition is inspired and motivated by the more classical definition of local algorithms for static graphs . Local algorithms output estimates for one vertex based only on a small neighborhood around it , and thus scale well to large graphs ; see [ Suo13 ] for a survey . A substantial literature studies the behavior of local algorithms for sparse graphs [ GT12 , HLS14 , GS14 , Mon15 , MX16 , FM17 ] . Our results focus on the sparse regime in which the graph ’ s average degree is bounded . This is the most challenging regime for the SBM , and it is also relevant for real-world applications where networks are usually sparse . We present the following contributions : Fundamental limitations of local streaming algorithms . We prove that , in the absence of side information , in streaming symmetric SBM ( introduced in Section 2 ) , local streaming algorithms ( introduced in Section 3 ) do not achieve any non-trivial reconstruction : their accuracy is asymptotically the same as random guessing ; see Corollary 1 . This holds despite the fact that there exist polynomial time non-local algorithms that achieve significantly better accuracy . From a practical viewpoint , this indicates that methods with a small ‘ locality radius ’ ( the range over which the algorithm updates its estimates ) are ineffective at aggregating information : they perform poorly unless strong local side information is available . Optimality of streaming belief propagation . On the positive side , in Section 4 we define a streaming version of belief propagation ( BP ) , a local streaming algorithm that we call STREAMBP , parameterized by a locality radius R. We prove that , for any non-vanishing amount of side information , STREAMBP achieves the same reconstruction accuracy as offline BP ; see Theorem 2 . The latter is in turn conjectured to be the optimal offline polynomial-time algorithm [ DKMZ11 , Abb17 , HS17 ] . Under this conjecture , there is no loss of performance in restricting to local streaming algorithms as long as ( 1 ) local side information is available ; ( 2 ) the locality radius is sufficiently large ; and ( 3 ) information is aggregated optimally via STREAMBP . Let us emphasize that we do not claim ( nor do we expect ) STREAMBP to outperform offline BP . We use offline BP as an ‘ oracle ’ benchmark ( as it has the full graph information available , and is not constrained to act in a streaming fashion ) . Implementation and numerical experiments . In Section 5 and Appendix A–C , we validate our results both on synthetic data , generated according to the StSBM , and on real datasets . Our empirical results are consistent with the theory ; in particular , STREAMBP substantially outperforms simple voting methods . However , we observe that it can behave poorly with large locality radius R. In order to overcome this problem , we introduce a ‘ bounded distance ’ version of STREAMBP , called STREAMBP∗ , which appears to be more robust . ( STREAMBP∗ can be shown to enjoy the same theoretical guarantees as STREAMBP . ) 2 Streaming stochastic block model . In this section we present a formal definition of the proposed model . The streaming stochastic block model is a probability distribution StSBM ( n , k , p , W , α ) over triples ( τ , τ̃ , G ) where τ ∈ [ k ] n is a vector of labels ( here [ k ] , { 1 , . . . , k } ) , τ̃ ∈ [ k ] n are noisy observations of the labels τ , and G = ( G ( 0 ) , G ( 1 ) , . . . , G ( n ) ) is a sequence of undirected graphs . Here G ( t ) = ( V ( t ) , E ( t ) ) is a graph over |V ( t ) | = t vertices and , for each 0 ≤ t ≤ n− 1 , V ( t ) ⊆ V ( t+ 1 ) and E ( t ) ⊆ E ( t+ 1 ) . We will assume , without loss of generality , that V ( n ) = [ n ] , and interpret τ ( v ) as the label associated to vertex v ∈ [ n ] . For each 0 ≤ t ≤ n − 1 , G ( t ) is the subgraph induced in G ( t + 1 ) by V ( t ) ; equivalently , all edges in E ( t+ 1 ) \ E ( t ) are incident to the unique vertex in V ( t+ 1 ) \ V ( t ) . The distribution StSBM ( n , k , p , W , α ) is parameterized by a scalar α ∈ [ 0 , k−1k ] , a probability vector p = ( p1 , . . . , pk ) ∈ ∆k , { x ∈ [ 0 , 1 ] k , 〈x , 1〉 = 1 } , and a symmetric matrix W ∈ [ 0 , 1 ] k×k . We draw the coordinates of τ independently with distribution p , and set τ̃ ( v ) = τ ( v ) with probability 1− α , and τ̃ ( v ) ∼ Unif ( [ k ] \ { τ ( v ) } ) otherwise , independently across vertices : P ( τ ( v ) = s ) = ps , P ( τ̃ ( v ) = s1 | τ ( v ) = s0 ) = { 1− α if s1 = s0 , α/ ( k − 1 ) if s1 6= s0 . We then construct G ( n ) by generating conditionally independent edges , given ( τ , τ̃ ) , with P ( ( u , v ) ∈ E ( n ) | τ , τ̃ ) = Wτ ( u ) , τ ( v ) . ( 1 ) Note that the labels τ̃ provide noisy ‘ side information ’ about the true labels τ . This information τ̃ is conditionally independent of the graph G ( n ) given τ . Finally we generate the graph sequence G by choosing a uniformly random permutation of the vertices ( v ( 1 ) , v ( 2 ) , . . . , v ( n ) ) and setting V ( t ) = { v ( 1 ) , . . . , v ( t ) } and G ( t ) to the graph induced by V ( t ) . If v = v ( t ) , then we define t as the arrival order of vertex v. Note that , for each t , G ( t ) is distributed according to a standard SBM with t vertices : G ( t ) ∼ SBM ( t , k , p , W ) . An equivalent description is that ( conditional on τ , τ̃ ) StSBM ( n , k , p , W , α ) defines a Markov chain over graphs . The new graph G ( t + 1 ) is generated from G ( t ) by drawing the vertex v ( t + 1 ) uniformly at random from [ n ] \ V ( t ) , and then the edges ( u , v ( t+ 1 ) ) , u ∈ V ( t ) , independently with probabilities given by Equation ( 1 ) . We are interested in the behavior of large graphs with bounded average degree . In order to focus on this regime , we will consider n → ∞ and W = W ( n ) → 0 with W ( n ) = W0/n for a matrix W0 ∈ Rk×k≥0 independent of n. A case of special interest is the streaming symmetric SBM , StSSBM ( n , k , a , b , α ) , which corresponds to taking p = ( 1/k , . . . , 1/k ) and W0 having diagonal elements a and non-diagonal elements b . Finally , the case α = ( k − 1 ) /k corresponds to pure noise τ̃ : in this case we can drop τ̃ from the observations and we will drop α from the distribution parameters . hhnni矗 鸞 ag.ci Nine nazis hdnet 矗tanto 䎖矗趱 tanto engin tanto

Summary: This paper tackles a streaming version of community detection, where the vertices of a graph generated with the classical stochastic block model are revealed sequentially, in a random order.  The authors restrict themselves to what they call $R$-local algorithms, wherein the new information brought by the arrival of a vertex is only propagated at distance at most $R$ of this revealed vertex.  In the absence of side information (i.e. additional information correlated with the ground truth community of each vertex), they show that this class of algorithms is unable to yield meaningful reconstruction when $n$ goes to infinity. In contrast, when side information is present, they provide an algorithm (StreamBP) for community detection, and show that this algorithm matches the performance of the classical belief propagation algorithm (OfflineBP) on the whole graph.   Those results are complemented by experiments on both synthetic and real-world datasets, comparing the StreamBP and OfflineBP algorithms, as well as several alternatives : simple voting algorithms, and a modified version of StreamBP (StreamBP*), modified to better deal with real-world datasets while enjoying the same theoretical guarantees as the original algorithm.