Weakly Supervised Graph Clustering

Graph Clustering , which clusters the nodes of a graph given its collection of node features and edge connections in an unsupervised manner , has long been researched in graph learning and is essential in certain applications . While this task is common , more complex cases arise in practice—can we cluster nodes better with some graph-level side information or in a weakly supervised manner as , for example , identifying potential fraud users in a social network given additional labels of fraud communities . This triggers an interesting problem which we define as Weakly Supervised Graph Clustering ( WSGC ) . In this paper , we firstly discuss the various possible settings of WSGC , formally . Upon such discussion , we investigate a particular task of weakly supervised graph clustering by making use of the graph labels and node features , with the assistance of a hierarchical graph that further characterizes the connections between different graphs . To address this task , we propose Gaussian Mixture Graph Convolutional Network ( GMGCN ) , a simple yet effective framework for learning node representations under the supervision of graph labels guided by a proposed consensus loss and then inferring the category of each node via a Gaussian Mixture Layer ( GML ) . Extensive experiments are conducted to test the rationality of the formulation of weakly supervised graph clustering . The experimental results show that , with the assistance of graph labels , the weakly supervised graph clustering method has a great improvement over the traditional graph clustering method . 1 INTRODUCTION . With the development of deep graph learning , graph clustering has achieved significant improvement in community detection ( Jin et al. , 2019 ) , text clustering ( Aggarwal & Zhai , 2012 ) and other applications ( Yang et al. , 2010 ) . Such applications all assume that only node/edge information and connections are accessible to assist with graph clustering in an unsupervised manner . However , it is possible that more graph-level side information , such as graph labels , may also be accessible and helpful for clustering the nodes in graphs . Or namely , can we use the global label of the graph to help improve the effectiveness of graph clustering ? This problem is interesting and very common in the real world . For example , point cloud segmentation , which could be re-considered as clustering of nodes in point cloud if we only know the objects contained in each point cloud but not all the node labels in the training process instead of predictions for nodes with known node labels that are solved by supervised graph learning methods ( Ye et al. , 2018 ; Li et al. , 2019 ) . Moreover , in social media , thinking of hot events that are widely discussed in the form of graphs where users are involved as nodes and the mutual following relationship among users as edges , it is interesting to think how to pick out the malicious users from the mass of users with the labels of these event topics such as the authenticity of each topic ( Zhou & Zafarani , 2018 ; Cao et al. , 2018 ) . Actually , this problem is more like graph clustering in a weakly supervised manner ( Zhou , 2018 ) , or named Weakly Supervised Graph Clustering ( WSGC ) , where the available training labels are much coarser-grained than the node labels we want to fit . Unlike traditional graph clustering ( Girvan & Newman , 2002 ) which only conducts clustering based on node features or graph structures , WSGC is better guaranteed by the given label information of graphs , which , we will demonstrate , closely influences the clustering results . Note that WSGC is different from traditional weakly supervised learning problems . Generally , weakly supervised learning is considered to consist of three types of weak supervision problems : incomplete supervision , inaccurate supervision and inexact supervision ( Zhou , 2018 ) . WSGC is different from incomplete supervision where a subset of training nodes is given with labels and inaccurate supervision where the given node labels are not always ground-truth , since WSGC does not contain any node labels for training . It is also different from inexact supervision , whose representative method as Multiple-Instance Learning ( MIL ) ( Carbonneau et al. , 2018 ) adopts global labels of bags to identify the labels of local instances . Because MIL assumes that each instance is i.i.d . and there are no edge connections involved . As far as we know , there are no studies on WSGC even though WSGC is very useful to formulate some real-world problems . Therefore , it is interesting and important to study the WSGC as a new challenge and seek new solutions for it . In this paper , we formally define Weakly Supervised Graph Clustering ( WSGC ) , i.e . a graph clustering task by making use of graph labels , graph structure and node features . To address this particular task , we propose a novel model based on Gaussian mixture model ( GMM ) and graph convolutional network ( GCN ) named as Gaussian Mixture Graph Convolutional Network ( GMGCN ) . The proposed model includes a Gaussian mixture layer ( GML ) and a new hierarchical GAT ( hierGAT ) to update the node and graph hidden representations , respectively . Specifically , we design a consensus loss that plays a key role in improving node clustering with the assistant of graph labels in the training process . Finally , a graph clustering is carried out according to the parameters of GML . The main contributions are as follows : 1 . New problem : We introduce a new problem , i.e . Weakly Supervised Graph Clustering ( WSGC ) , which tries to identify the nodes in graphs based on the labels of graphs , and we take a formal discussion of the variants of this problem to attract more research attention on this problem . 2 . New solution : We propose an effective model called GMGCN based on GMM and GCN to solve WSGC . To the best of our knowledge , this is the first work to achieve graph clustering in graph structure by integrating GMM into GCN . 3 . New loss : We propose a consensus loss function to boost the model training through the principle of ” same attraction , opposite repulsion ” . Experiments validate the effectiveness of this consensus loss . We validate the significant improvement of WSGC over traditional clustering methods on various synthetic and real-world datasets . Compared to the best performing baseline method , GMGCN improves Normalized Mutual Information1 ( NMI ) by an average of 13 % on 5 synthetic datasets , by more than 13 % on PHEME social media datasets , and improves mean intersection-over-union2 ( mIoU ) by more than 18 % on point cloud segmentation datasets . 2 RELATED WORKS . Graph Learning . Recently , there has been an increasing interest in the graph learning domain . Among all the existing works , GCN is one of the most effective convolution models . A typical GCN model is the message passing neural network ( MPNN ) proposed by Gilmer et al . ( Gilmer et al. , 2017 ) which re-generalizes several neural network and graph convolutional network approaches as a general ” message-passing ” architecture . Many kinds of GCN ( Bruna et al. , 2014 ; Defferrard et al. , 2016 ; Kipf & Welling , 2017 ; Chang et al. , 2020 ) deliver different message propagation functions for GCN . Among them , Graph Attention Networks ( GAT ) ( Veličković et al. , 2017 ) first leverages learnable self-attentional layers to aggregate weighted neighbors ’ information . Besides these methods to obtain the appropriate node representation , pooling strategies are proposed to integrate information over the node representations ( Wu et al. , 2020 ) , such as max/min pooling ( Defferrard et al. , 2016 ) , SortPooling ( Zhang et al. , 2018 ) , and so on . In addition , Lin et al . ( Lin et al. , 2017 ) propose a learnable attentive pooling for weighted averaging of node representations . Despite extensive research on node-level and graph-level graph learning tasks , few studies have been conducted on WSGC . Graph Clustering . Graph clustering is a fundamental data analysis task that aims to group similar nodes into the same category . Many real-world applications are cast as graph clustering ( Shi & Ma- 1NMI is a normalization of the Mutual Information score to scale the clustering results between 0 ( no mutual information ) and 1 ( perfect correlation ) . 2IoU is a commonly used metric in semantic segmentation measures similarity between finite sample sets , and is defined as the size of the intersection divided by the size of the union of the sample sets . lik , 2000 ; Hastings , 2006 ) . The major strategy of graph clustering is to perform traditional clustering algorithms such as K-means ( Jain , 2010 ) or GMM ( McLachlan & Basford , 1988 ) on the node features . With the great achievements of deep learning , more graph clustering studies have resorted to graph learning to learn embedding that captures both node features and structural relationships ( Wu et al. , 2020 ) . Researchers employ the stacked sparse autoencoder ( Tian et al. , 2014 ) , the variational autoencoder ( Kipf & Welling , 2016 ) , or the combination of both autoencoder and GCN ( Bo et al. , 2020 ) to obtain graph representations for clustering . Nevertheless , these graph clustering methods do not perform better than WSGC methods due to the lack of attention to graph labels . Multiple-Instance Learning . Another task that has a similar definition to WSGC is MultipleInstance Learning ( MIL ) . MIL is a variant of inductive machine learning , where each learning example consists of a bag of instances instead of a single feature vector ( Foulds & Frank , 2010 ) . When obtaining local instances annotations is costly or not possible , but global labels for bags are available , MIL is utilized to train classifiers using weakly labeled data . It has received a considerable amount of attention due to its applicability to real-world problems ( Andrews et al. , 2002a ; Cheplygina et al. , 2019 ) . Although MIL learns a classifier from the labels of bags to classify instance , it ignore structural information between instances and are therefore not suitable for WSGC . Gaussian Mixture Model . As a core part of the proposed GMGCN model , GMM is a parametric probability density function for a weighted sum of Gaussian component densities ( McLachlan & Basford , 1988 ) . It is commonly used to find underlying clusters in data samples ( Bishop , 2006 ) . Generally , the GMM parameters are estimated using the iterative Expectation-Maximization ( EM ) ( Moon , 1996 ) algorithm from training data . In this paper , we integrate the GMM into GCN to establish a solution for WSGC and update the GMM parameters using stochastic gradient descent . Point Cloud Semantic Segmentation . Semantic segmentation of 3D point clouds has attracted more researchers ’ attentions due to the developments of 3D scanners , such as Light Detection and Ranging ( LIDAR ) , structure-from-motion ( SFM ) techniques , etc . To operate directly on these unstructured point clouds , PointNet ( Qi et al. , 2017a ) is proposed to subdivide the input points into a grid of blocks and process each such block individually , but limited neighborhood context of each point is taken into consideration . To overcome this drawback , many approaches ( Engelmann et al. , 2017 ; Ye et al. , 2018 ; Qi et al. , 2017b ) are built upon PointNet and better capture local spatial structures as well as long dependency context . Recently , some latest works ( Qi et al. , 2017c ; Landrieu & Simonovsky , 2018 ; Wang et al. , 2019 ) re-consider semantic segmentation of unstructured 3D point clouds as predictions for nodes on graph data and incorporate contextual information and topological information on top of 3D points . However , these supervised graph learning methods are not suitable for weakly supervised point cloud semantic segmentation datasets if the point labels are unknown .

Summary: Authors provide the framework that combines Graph Attention Networks (GAT) and Gaussian Mixture Models (GMM) to tackle the weakly supervised graph clustering model. In particular, the consensus loss, a new loss function designed by the authors, is used for the entire framework. Authors also address different kinds of weak supervision settings and show the performance of the proposed framework for each case on synthetic datasets.