Trading Quality for Efficiency of Graph Partitioning: An Inductive Method across Graphs

1 INTRODUCTION . For various complex systems , e.g. , communication networks , graph is a generic model to describe the entities and their relations using a set of nodes and edges . Graph partitioning ( GP ) is a classic inference task that aims to partition the nodes of a graph into several groups ( i.e. , clusters ) with dense linkage distinct from other groups . Since the extracted clusters are believed to correspond to several real-world substructures of a system , e.g. , cells in wireless networks ( Dai & Bai , 2017 ) , many network applications are formulated as GP tasks ( Qin et al. , 2019 ; Patil & Kulkarni , 2021 ) . Mathematically , GP can be described as some NP-hard combinatorial optimization problems , e.g. , modularity maximization ( Newman , 2006 ) and normalized cut ( NCut ) minimization ( Von Luxburg , 2007 ) . Due to the NP-hardness , to balance the quality and efficiency of GP remains a challenge but some real applications have both requirements of high quality and low runtime , e.g. , accurate GP on a wireless cellular network with several thousand nodes in a few seconds ( Dai & Bai , 2017 ) . On the one hand , prior work has demonstrated the ability of machine learning ( ML ) techniques to achieve high GP quality . Typical ML-based methods include non-negative matrix factorization ( Wang et al. , 2011 ) and generative probabilistic models ( Karrer & Newman , 2011 ) . Graph embedding emerges as a promising technique for GP in recent studies . These methods first learn lowdimensional node representations via random walk ( Perozzi et al. , 2014 ) or deep learning ( Wang et al. , 2016 ; Yang et al. , 2016 ) to capture high-order proximities and nonlinear characteristics of a graph . The GP result is derived by feeding the learned embedding to a downstream clustering module ( e.g , KMeans ) . However , most ML-based methods have to use iterative optimization algorithms ( e.g. , gradient descent ) to learn large-scale parameters for each graph with high complexity . On the other hand , to reduce the overall runtime and satisfy real-time constraints of some applications is another major focus . Some fast GP methods apply heuristic strategies to approximate conventional objectives , e.g. , greedy modularity maximization ( Blondel et al. , 2008 ) and multi-level coarsening for graph-cut minimization ( Dhillon et al. , 2007 ) . Several graph embedding approaches also adopt fast approximation to reduce the inference time , e.g. , random projection for high-order proximities ( Zhang et al. , 2018a ) . Despite their high efficiency ( e.g. , low runtime ) , they may suffer from unexpected quality decline due to the inherent information loss of heuristic approximation . The aforementioned state-of-the-art ( SOTA ) GP methods are inherently transductive , which independently optimize the model on each static graph and can only tackle GP on such a unique graph . We try to achieve a better trade-off between quality and efficiency from a new inductive perspective across multiple associated graphs of a system or scenario . An inductive graph partitioning ( IGP ) framework is proposed based on the fact that most real-world complex systems generate a set of graphs via common knowledge in terms of several underlying distributions , e.g. , power-law distributions . This hypothesis is also adopted in the simulation of various network systems ( Wehrle et al. , 2010 ) . The multiple graphs can be ( i ) associated snapshots evolving over time or ( ii ) independent ( sub ) graphs of a scenario without temporal dependency e.g. , ego-nets in social media . IGP first trains a high-quality GP model on historical snapshots in an offline way , which aims to fully capture properties of the system or scenario regardless of time cost . The trained model is then generalized to newly generated graphs for online GP without additional optimization , which significantly reduces the runtime on these new graphs and is believed to have high-quality GP results . For instance , we can conduct the offline training of IGP on existing known ego-nets of a social network and generalize it to new unseen ego-nets . In real applications , it is usually assumed that we have enough time to prepare a high-quality model using historical known data in an offline way , which is also one-timeeffort only . Our focus is to achieve a better trade-off between quality and efficiency of online GP on new graphs after deploying the trained model to a system with model parameters fixed . Technically , IGP follows a graph embedding scheme for unsupervised node-level tasks ( i.e. , GP ) across graphs using the inductive nature of graph neural networks ( GNN ) . Although some inductive GNNs have the potential to be generalized to new unseen nodes and graphs ( Hamilton et al. , 2017 ; Veličković et al. , 2018 ) , they still suffer from the following limitations for GP . In this study , we consider GP on multiple graphs where topology is the only available information source , i.e. , without graph attributes . As some systems allow the addition and deletion of entities , we assume that the number of nodesN is non-fixed for multiple graphs . Although existing inductive GNNs can be applied to graphs with non-fixedN , they were originally designed for attributed graphs and rely on fixed dimensionality of node features . Our preliminary experiments indicate that some classic settings of GNN for the case without node features ( e.g. , using a constant matrix with fixed dimensionality as the feature input ) may suffer from poor GP quality . Some methods extract features via dimensionality reduction ( e.g. , PCA ( Nazi et al. , 2019 ) ) to map the topology ( e.g. , adjacency matrix ) with non-fixed N to a fixed-dimensional feature space , which is still time-consuming . Moreover , different system snapshots can be assigned with different number of clusters K. Some inductive GNN based approaches adopt an end-to-end ( E2E ) framework to achieve a better approximation to classic GP objectives ( e.g. , GAP ( Nazi et al. , 2019 ) for NCut minimization and ClusNet ( Wilder et al. , 2019 ) for modularity maximization ) , with partitioning results derived via a fullyconnected ( FC ) output layer . Despite their high quality , they can only tackle the inductive GP across graphs with fixed K , due to fixed dimensionality of the output layer . These E2E methods still need to be trained from scratch for new graphs with non-fixed K , which is time-consuming . Although prior studies have demonstrated the ability of inductive GNNs to tackle ( semi- ) supervised tasks ( e.g. , node classification ) on new unseen nodes and graphs ( Hamilton et al. , 2017 ; Veličković et al. , 2018 ) , few of them consider unsupervised node-level tasks ( e.g. , GP ) across graphs . Our experiments also indicate that some standard settings of inductive GNNs for unsupervised tasks ( e.g. , using classic unsupervised training loss of graph embedding for GNN ( Hamilton et al. , 2017 ) ) may still lack robustness for the online GP on newly generated graphs . In addition to obtaining a better trade-off between quality and efficiency over conventional transductive GP methods via a novel inductive framework across graphs , we also make the following contributions to address the aforementioned limitations of inductive GNNs . ( i ) To tackle the online GP with non-fixed N , we develop an efficient feature extraction module for inductive GNNs via graph coarsening . ( ii ) In contrast to E2E methods , IGP adopts an inductive graph embedding scheme across graphs which can tackle the online GP with non-fixed K. ( iii ) Note that GP is an unsupervised node-level task , where the cluster labels are permutation invariant , e.g. , label assignments ( l1 , l2 , l3 ) = ( 1 , 2 , 2 ) and ( l1 , l2 , l3 ) = ( 2 , 1 , 1 ) are the same in terms of GP with li as the cluster label of node vi . IGP can further incorporate such permuttion invariant label information ( e.g. , GP ground-truth ) of historical graphs to the offline training by combining several GP objectives ( e.g. , modularity maximization and NCut minimization ) with a novel dual GNN structure which ensures strong robustness for the online GP on new graphs . 2 PROBLEM STATEMENTS AND PRELIMINARIES . In this study , we consider GP on a set of graphs S = { G1 , G2 , · · · , GT } from a common system or scenario . Each graph Gt ∈ S can be represented as Gt = ( Vt , Et ) with Vt = { vt1 , . . . , vtNt } and Et = { ( vti , vtj ) ∣∣vti , vtj ∈ Vt } as the sets of nodes and edges . For each Gt , topology is the only available information source without graph attributes . We use an adjacency matrix At ∈ < Nt×Nt to describe its topology with Nt nodes , where ( At ) ij = ( At ) ji = 1 if ( vti , v t j ) ∈ Et and ( At ) ij = ( At ) ji = 0 otherwise . Since some systems allow addition and deletion of entities , we assume that different snapshots in S can have different node sets , i.e. , ∃Gt , Gs ∈ S s.t . Vt 6= Vs . Since there may be no temporal dependency among { G1 , · · · , GT } in some cases ( e.g. , multiple ego-nets in social media ) , we do not consider the node correspondence among { V1 , · · · , VT } . We follow the hypothesis adopted in the simulation of various network systems ( Wehrle et al. , 2010 ) that graphs in S are independently generated via several underlying distributions of a common system or scenario . Graph Partitioning . Given a graph Gt , GP aims to partition the node set Vt into Kt subsets ( i.e. , clusters ) Ct = { Ct1 , · · · , CtKt } so that ( i ) within each cluster the linkage is dense but ( ii ) between different clusters the linkage is relatively loose . The result Ct also satisfies Ctr ∩Cts = ∅ for ∀r 6= s. Mathematically , GP can be formulated as several NP-hard combinatorial optimization problems , e.g. , NCut minimization ( Von Luxburg , 2007 ) and modularity maximization ( Newman , 2006 ) . Given a graph Gt , NCut minimization aims to get the GP result Ct that minimizes the NCut metric : arg minCt NCut ( Ct ) = 1 2 ∑Kt r=1 [ cut ( Ctr , C̄ t r ) /vol ( C t r ) ] ( 1 ) where C̄tr = Vt − Ctr is the complementary set of Ctr ; cut ( Ctr , C̄tr ) = ∑ vi∈Ctr , vj∈C̄tr ( At ) ij is the cut between Ctr and C̄ t r ; vol ( C t r ) = ∑ vi , vj∈Ctr ( At ) ij is the volume of C t r. The objective ( 1 ) can be equivalently expressed in the following matrix form : arg minHt tr ( H T t LtHt ) s.t.H T t Ht = IKt , ( 2 ) where Lt = INt − D−0.5t AtD−0.5t is the Laplacian matrix of At ; Dt = diag ( dt1 , · · · , dtNt ) is a diagonal matrix with dti = ∑ j ( At ) ij ; IN is anN -dimensional identity matrix . Ht ∈ < Nt×Kt is the membership indicator , where ( Ht ) ir = [ dti · vol ( Ctr ) −1 ] 0.5 if vti ∈ Ctr and ( Ht ) ir = 0 otherwise . Modularity maximization is another conventional NP-hard objective of GP . Given a graph Gt , it aims to find a partition Ct that maximizes the modularity metric : arg minCt Mod ( Ct ) = 1 2e ∑ r ∑ vti , v t j∈Ctr [ ( At ) ij − d t id t j/ ( 2e ) ] , ( 3 ) where e = ∑ i d t i/2 is the number of edges . The objective can also be rewritten in a matrix form : argminHt − tr ( HTt QtHt ) s.t . tr ( HTt Ht ) = Nt , ( 4 ) where Qt ∈ < Nt×Nt is the modularity matrix with ( Qt ) ij = ( At ) ij − dtidtj/ ( 2e ) . Ht ∈ < Nt×Kt is the membership indicator , where ( Ht ) ir = 1 if vti ∈ Ctr and ( Ht ) ir = 0 otherwise . Inductive Graph Embedding . Given a single graph Gt , conventional transductive graph embedding aims to learn a function f : { vti } 7→ { uti ∈ < 1×k } that maps each node vti to a k-dimensional vector uti . The key characteristics of Gt ( e.g. , clustering structure ) should be preserved in the embedding space , where a node pair ( vti , v t j ) with similar properties ( e.g. , in the same cluster ) should have similar representations ( uti , u t j ) ( e.g. , with close distance in the embedding space ) . The learned representations { uti } are used as the input of several downstream tasks on Gt including GP . To alleviate the NP-hard challenge of GP , we consider a novel inductive graph embedding scheme , where model parameters of f are shared by all the graphs in S. We divide S into a training set Dual GNN Structure Clustering Module Model Opt . ( Param . Fixed ) Dual GNN Structure Snapshots New System Historical GP Results ( e.g. , ground-truth ) { } t { } w.r.t . { } t tC Historical System { } t Snapshots New GP Results ( a ) Offline Training ( b ) Online fast high-quality GP ' ' { } w.r.t . { } t tC Figure 1 : IGP across multiple associated graphs . 1 2 3 4 5 6 7 8 Cluster 1 Cluster 2 1 2 3 4 5 6 7 8 Componet 1 Compoent 2 Feature Encoder Aux . Label- Induced Encoder ( Dual GNN w/ shared parameters ) { 1 , 2 , 3 , 4 } { 5 , 6 , 7 , 8 } Original Graph t Aux . Label-Ind . Graph Supergraph * t ( ) g t ( HEM Graph Coarsening ) Adjacency Matrix At Red . Feat . Zt Aux . Label-Ind . Adj . Matrix ( ) g tA Emb . Ut Aux . LabelInd . Emb . ( ) g tU FC Classifier Discriminator Generator In n er P ro d u ct In Label-Ind . Encoder , message passing only occurs in each component ? tU ( ) ? gtU ( i ) Adv . Learning Obj . Reconstru cted Feat . tX ( ii ) Feature Reconstruction Obj . Neigh-Ind . Feature Xt ( min error ) ( Regularizing ) GP Result Ct ( ground-truth ) of t ( iii ) Clustering Reg . Obj . 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 In Feature Encoder , message passing occurs in the original topology G D Figure 2 : A running example of the offline training of our dual GNN structure . Γ ⊂ S and a test set Γ′ = S − Γ , which represent the sets of historical known and newly generated graphs . We first train f on Γ in an offline way to fully capture structural properties of historical graphs . Although graph embedding and GP are unsupervised procedures , we assume the permutation invariant GP result Ct ( i.e. , label information ) of each historical graph Gt ∈ Γ is available in the offline training , which can be used to enhance the offline embedding optimization . In general , the training label information { Ct|Gt ∈ Γ } can be from the ( i ) ground-truth of system applications ( e.g. , cells in wireless cellular network decomposition ) or ( ii ) results of some high-quality ( but usually time-consuming ) GP baselines . It indicates another application that we can use GP results of some strong baselines to regularize the offline training of IGP . After the training , we directly obtain the embedding { ut′i } from f for each new graph Gt′ ∈ Γ′ with model parameters fixed . The online GP on a new graph Gt′ is the downstream task , where we assume the number of clusters Kt′ is given . The GP result of Gt′ is derived by applying a clustering algorithm ( e.g. , KMeans ) to { ut ′ i } .

Summary: The authors consider the problem of inductive graph partitioning, which they formulate as clustering or partitioning multiple snapshots of a time-evolving graph for which we have no node correspondence. In other words, we cannot link the nodes in snapshot $t$ to those in snapshot $t-1$, which prevents incremental or evolutionary clustering algorithms from being applied. The propose a complicated dual graph neural network (GNN) architecture for this problem setting and demonstrate potentially good clustering accuracy with low computation time on simulated and real networks.