Overlapping Community Detection with Graph Neural Networks

1 INTRODUCTION . Graphs provide a natural way of representing complex real-world systems . For understanding the structure and behavior of these systems , community detection methods are an essential tool . Detecting communities allows us to analyze social networks ( Girvan & Newman , 2002 ) , to detect fraud ( Pinheiro , 2012 ) , to discover functional units of the brain ( Garcia et al. , 2018 ) , and to predict functions of proteins ( Song & Singh , 2009 ) . Over the last decades , this problem has attracted significant attention of the research community and numerous models and algorithms have been proposed ( Xie et al. , 2013 ) . In particular , it is a well known fact that communities in real graphs are in fact overlapping ( Yang & Leskovec , 2014 ) , thus , requiring the development of advanced models to capture this complex structure . In this regard , the advent of deep learning methods for graph-structured data opens new possibilities for designing more accurate and more scalable algorithms . Indeed , deep learning on graphs has already shown state-of-the-art results in s for various graph-related tasks such as semi-supervised node classification and link prediction ( Cai et al. , 2018 ) . Likewise , a few deep learning methods for community detection in graphs have been proposed ( Yang et al. , 2016 ; Chen et al. , 2017 ) . However , they all have one drawback in common : they only focus on the special case of disjoint ( non-overlapping ) communities . Handling overlapping communities , is a requirement not yet met by existing deep learning approaches to community detection . In this paper we propose an end-to-end deep probabilistic model for overlapping community detection in graphs . Our core idea lies in predicting the community affiliations using a graph neural network . Despite its simplicity , our model achieves state-of-the art results in community recovery and significantly outperforms the existing approaches . Moreover , our model is able to perform out-of-sample ( inductive ) community detection for nodes that were not seen at training time . To summarize , our main contributions are : • We propose the Deep Overlapping Community detection ( DOC ) model - a simple , yet effective deep learning model for overlapping community detection in graphs . DOC is one of few methods is able to perform community detection both transductively and inductively . • We introduce 5 new datasets for overlapping community detection , which can act as a benchmark to stimulate future work in this research area . • We perform a thorough experimental evaluation of our model , and show its superior performance when comparing with established methods for overlapping community detection . 2 COMMUNITY DETECTION IN GRAPHS . Assume that we are given an undirected unweighted graph G = ( V , E ) , with N : = |V| nodes and M : = |E| edges , represented by a symmetric adjacency matrix A ∈ { 0 , 1 } N×N . Moreover , each node is associated with D real-valued attributes , that can be represented as a matrix X ∈ RN×D . The goal of overlapping community detection is to assign nodes in the graph into C communities . Such assignment can be represented as a non-negative community affiliation matrix F ∈ RN×C≥0 , where Fuc denotes the strength of node u ’ s membership in community c ( with the notable special case of binary hard-assignment F ∈ { 0 , 1 } N×C ) . There is no single universally definition of community in the literature . However , most recent works tend to agree with the statement that a community is a group of nodes that have a higher probability to form edges with each other than with other nodes in the graph ( Fortunato & Hric , 2016 ) . One can broadly subdivide the existing methods for overlapping community detection into three categories : approaches based on non-negative matrix factorization ( NMF ) , probabilistic inference , or heuristics . Methods based on NMF are trying to recover the community affiliation matrix F by performing a low-rank decomposition of the adjacency matrix A or some other related matrix ( Wang et al. , 2011 ; Kuang et al. , 2012 ) . Probabilistic approaches , such as Yang & Leskovec ( 2013 ) or Zhou ( 2015 ) , treat F as a latent variable in a generative model for the graph , p ( A , F ) . This way the problem of community detection is cast as an instance of probabilistic inference . Lastly , heuristic-based approaches usually define a goodness measure , like within-community edge density ( Galbrun et al. , 2014 ) , and then directly optimize it . All of these approaches can be very generally formulated as an optimization problem min F L ( F ) ( 1 ) for an appropriate choice of the loss function L , be it Frobenius norm L ( F ) = ‖A − FF T ‖F or negative log-likelihood L ( F ) = − log p ( A|F ) of some generative model . Besides these traditional approaches , one can also view the problem of community detection through the lens of representation learning . The community affiliation matrix F can be considered as an embedding of nodes into RC≥0 , with the aim of preserving the community structure . Given the recent success of deep representation learning for graphs ( Cai et al. , 2018 ) , a question arises : ” Can the advances in deep representation learning for graphs be used to design better community detection algorithms ? ” . A very simple idea is to first apply a node embedding approach to the graph , and then cluster the nodes in the embedding space using k-means to obtain communities ( as done in , e.g. , Tsitsulin et al . ( 2018 ) ) . However , such approach is only able to detect disjoint communities , which does not correspond to the structure of communities in real-world graphs ( Yang & Leskovec , 2014 ) . Instead , we argue that an end-to-end deep learning architecture able to detect overlapping communities is preferable . Traditional community detection methods treat F as a free variable , with respect to which optimization is performed ( Equation 1 ) . This is similar to how embeddings are learned in methods like DeepWalk ( Perozzi et al. , 2014 ) and node2vec ( Grover & Leskovec , 2016 ) . In contrast , recent works of Kipf & Welling ( 2016 ) ; Hamilton et al . ( 2017 ) ; Bojchevski & Günnemann ( 2018 ) have adopted the approach of defining the embeddings as a function of node attributes F : = fθ ( X , A ) and solving the optimization problem min θ L ( fθ ( X , A ) ) , ( 2 ) where fθ is defined by a neural network . 1 Such formulation allows to • achieve better performance in downstream tasks like link prediction and node classification ; • naturally incorporate the attributesX without hand-crafting the generative model p ( X , F ) ; • generate embeddings inductively for previously unseen nodes . We propose to use this framework for overlapping community detection , and describe our model in the next section . 1Even if node attributes are not available , this approach can work by , for example , setting X to an identity matrix I or to the adjacency matrix A . 3 THE DOC MODEL . We let the community affiliations be produced by a three-layer graph convolutional neural network ( GCN ) , as defined in Kipf & Welling ( 2017 ) . F = GCNθ ( X , A ) = ReLU ( Â ReLU ( Â ReLU ( ÂXW ( 0 ) ) W ( 1 ) ) W ( 2 ) ) , ( 3 ) where Â = D̃− 1 2 ÃD̃− 1 2 is the normalized adjacency matrix , Ã = A+I , and D̃ the corresponding degree matrix . A ReLU nonlinearity is applied element-wise to the output layer to ensure nonnegativity of the community affiliation matrix F . Any other graph neural network architecture can be used here – we choose GCN because of its simplicity and popularity . Link function . A good F explains well the community structure of the graph . To model this formally , we adopt a probabilistic approach to community detection where we need to define the likelihood p ( A|F ) . A standard assumption in probabilistic community detection is that the edges Auv are conditionally independent given the community memberships F . Thus , once the F matrix is given , every pair of nodes ( u , v ) produces an interaction based on their community affiliations Xuv = FuF T v ∈ R≥0 . For a probabilistic interpretation , this interaction is transformed into an edge probability by means of a link function g : R≥0 → [ 0 , 1 ] . The edge probability is then given by Auv ∼ Bern ( g ( FuF Tv ) ) ( 4 ) We consider two choices for the link function g : Bernoulli-Poisson link and sigmoid link . Bernoulli-Poisson link , defined as ξ ( Xuv ) = 1−exp ( −Xuv ) , is a common probabilistic model for overlapping community detection ( Yang & Leskovec , 2013 ; Zhou , 2015 ; Todeschini et al. , 2016 ) . Note , that under the BP model a pair of nodes that have no communities in common ( i.e . FuF Tv = 0 ) have a zero probability of forming an edge . This is an unrealistic assumption , which can be easily fixed by adding a small offset ε > 0 , that is ξ ( Xuv ) = 1− exp ( −Xuv − ε ) . Sigmoid link , defined as σ ( x ) = ( 1+exp ( −Xuv ) ) −1 , is the standard choice for binary classification problems . It can also be used to convert the edge scores into probabilities in probabilistic models for graphs ( Latouche et al. , 2011 ; Tang et al. , 2015 ; Kipf & Welling , 2016 ) . Since a non-negative F implies that the interactions between every pair of nodes Xuv are at least 0 , the edge probability under the sigmoid model is always above σ ( 0 ) = 0.5 . This can be fixed by introducing an offset : g ( Xuv ) = σ ( Xuv − b ) . The offset b becomes an additional variable to optimize over , closedform expression for which is provided in Schein et al . ( 2003 ) . However , while optimizing over b produces better likelihood scores , we have empirically observed that fixing it to zero leads to the same performance in community recovery ( Section 4.3 ) . Thus , we set b = 0 in our experiments . We consider both link functions , and denote the two variants of our model as DOC-BP and DOCSigmoid for Bernoulli-Poisson and sigmoid link functions respectively . Loss . Once the link function g is chosen , the Equation 4 defines the likelihood function p ( A|F ) . Maximizing the likelihood is equivalent to minimizing the negative log-likelihood , which corresponds to the well-known binary cross-entropy loss function . Since real-world graphs are extremely sparse ( only 10−2 − 10−5 of possible edges are present ) , we are dealing with an extremely imbalanced binary classification problem . A standard way of dealing with this problem is by balancing the contribution from both classes , which corresponds to the following objective function L ( F ) = −E ( u , v ) ∼PE [ log p ( Auv = 1|Fu , Fv ) ] − E ( u , v ) ∼PN [ log p ( Auv = 0|Fu , Fv ) ] , ( 5 ) where PE and PN stand for uniform distributions over edges and non-edges respectively . Evaluating the gradient of the full loss requiresO ( N2 ) operations ( since we need to compute the expectation over ( N 2 ) possible edges/non-edges ) . This is impractical even for moderately-sized graphs . Instead , we optimize the objective using stochastic gradient descent . That is , at every iteration we approximate∇L using S randomly sampled edges , as well as the same number of non-edges . To summarize , we use stochastic gradient descent to optimize the objective min θ L ( GCNθ ( X , A ) ) , ( 6 ) where the parameters θ are the weights of the neural network , { W ( 0 ) , W ( 1 ) , W ( 2 ) } .

Summary: This paper proposes Deep Overlapping Community detection model (DOC), a graph convolutional network (GCN) based community detection algorithm for network data. The model is a simple combination of GCN and existing framework for community detection. The proposed algorithm is compared to baselines on various datasets, and demonstrated to be accurate in many cases.