Trading Quality for Efficiency of Graph Partitioning: An Inductive Method across Graphs

1 INTRODUCTION . For various complex systems , e.g. , communication networks , graph is a generic model to describe the entities and their relations using a set of nodes and edges . Graph partitioning ( GP ) is a classic inference task that aims to partition the nodes of a graph into several groups ( i.e. , clusters ) with dense linkage distinct from other groups . Since the extracted clusters are believed to correspond to several real-world substructures of a system , e.g. , cells in wireless networks ( Dai & Bai , 2017 ) , many network applications are formulated as GP tasks ( Qin et al. , 2019 ; Patil & Kulkarni , 2021 ) . Mathematically , GP can be described as some NP-hard combinatorial optimization problems , e.g. , modularity maximization ( Newman , 2006 ) and normalized cut ( NCut ) minimization ( Von Luxburg , 2007 ) . Due to the NP-hardness , to balance the quality and efficiency of GP remains a challenge but some real applications have both requirements of high quality and low runtime , e.g. , accurate GP on a wireless cellular network with several thousand nodes in a few seconds ( Dai & Bai , 2017 ) . On the one hand , prior work has demonstrated the ability of machine learning ( ML ) techniques to achieve high GP quality . Typical ML-based methods include non-negative matrix factorization ( Wang et al. , 2011 ) and generative probabilistic models ( Karrer & Newman , 2011 ) . Graph embedding emerges as a promising technique for GP in recent studies . These methods first learn lowdimensional node representations via random walk ( Perozzi et al. , 2014 ) or deep learning ( Wang et al. , 2016 ; Yang et al. , 2016 ) to capture high-order proximities and nonlinear characteristics of a graph . The GP result is derived by feeding the learned embedding to a downstream clustering module ( e.g , KMeans ) . However , most ML-based methods have to use iterative optimization algorithms ( e.g. , gradient descent ) to learn large-scale parameters for each graph with high complexity . On the other hand , to reduce the overall runtime and satisfy real-time constraints of some applications is another major focus . Some fast GP methods apply heuristic strategies to approximate conventional objectives , e.g. , greedy modularity maximization ( Blondel et al. , 2008 ) and multi-level coarsening for graph-cut minimization ( Dhillon et al. , 2007 ) . Several graph embedding approaches also adopt fast approximation to reduce the inference time , e.g. , random projection for high-order proximities ( Zhang et al. , 2018a ) . Despite their high efficiency ( e.g. , low runtime ) , they may suffer from unexpected quality decline due to the inherent information loss of heuristic approximation . The aforementioned state-of-the-art ( SOTA ) GP methods are inherently transductive , which independently optimize the model on each static graph and can only tackle GP on such a unique graph . We try to achieve a better trade-off between quality and efficiency from a new inductive perspective across multiple associated graphs of a system or scenario . An inductive graph partitioning ( IGP ) framework is proposed based on the fact that most real-world complex systems generate a set of graphs via common knowledge in terms of several underlying distributions , e.g. , power-law distributions . This hypothesis is also adopted in the simulation of various network systems ( Wehrle et al. , 2010 ) . The multiple graphs can be ( i ) associated snapshots evolving over time or ( ii ) independent ( sub ) graphs of a scenario without temporal dependency e.g. , ego-nets in social media . IGP first trains a high-quality GP model on historical snapshots in an offline way , which aims to fully capture properties of the system or scenario regardless of time cost . The trained model is then generalized to newly generated graphs for online GP without additional optimization , which significantly reduces the runtime on these new graphs and is believed to have high-quality GP results . For instance , we can conduct the offline training of IGP on existing known ego-nets of a social network and generalize it to new unseen ego-nets . In real applications , it is usually assumed that we have enough time to prepare a high-quality model using historical known data in an offline way , which is also one-timeeffort only . Our focus is to achieve a better trade-off between quality and efficiency of online GP on new graphs after deploying the trained model to a system with model parameters fixed . Technically , IGP follows a graph embedding scheme for unsupervised node-level tasks ( i.e. , GP ) across graphs using the inductive nature of graph neural networks ( GNN ) . Although some inductive GNNs have the potential to be generalized to new unseen nodes and graphs ( Hamilton et al. , 2017 ; Veličković et al. , 2018 ) , they still suffer from the following limitations for GP . In this study , we consider GP on multiple graphs where topology is the only available information source , i.e. , without graph attributes . As some systems allow the addition and deletion of entities , we assume that the number of nodesN is non-fixed for multiple graphs . Although existing inductive GNNs can be applied to graphs with non-fixedN , they were originally designed for attributed graphs and rely on fixed dimensionality of node features . Our preliminary experiments indicate that some classic settings of GNN for the case without node features ( e.g. , using a constant matrix with fixed dimensionality as the feature input ) may suffer from poor GP quality . Some methods extract features via dimensionality reduction ( e.g. , PCA ( Nazi et al. , 2019 ) ) to map the topology ( e.g. , adjacency matrix ) with non-fixed N to a fixed-dimensional feature space , which is still time-consuming . Moreover , different system snapshots can be assigned with different number of clusters K. Some inductive GNN based approaches adopt an end-to-end ( E2E ) framework to achieve a better approximation to classic GP objectives ( e.g. , GAP ( Nazi et al. , 2019 ) for NCut minimization and ClusNet ( Wilder et al. , 2019 ) for modularity maximization ) , with partitioning results derived via a fullyconnected ( FC ) output layer . Despite their high quality , they can only tackle the inductive GP across graphs with fixed K , due to fixed dimensionality of the output layer . These E2E methods still need to be trained from scratch for new graphs with non-fixed K , which is time-consuming . Although prior studies have demonstrated the ability of inductive GNNs to tackle ( semi- ) supervised tasks ( e.g. , node classification ) on new unseen nodes and graphs ( Hamilton et al. , 2017 ; Veličković et al. , 2018 ) , few of them consider unsupervised node-level tasks ( e.g. , GP ) across graphs . Our experiments also indicate that some standard settings of inductive GNNs for unsupervised tasks ( e.g. , using classic unsupervised training loss of graph embedding for GNN ( Hamilton et al. , 2017 ) ) may still lack robustness for the online GP on newly generated graphs . In addition to obtaining a better trade-off between quality and efficiency over conventional transductive GP methods via a novel inductive framework across graphs , we also make the following contributions to address the aforementioned limitations of inductive GNNs . ( i ) To tackle the online GP with non-fixed N , we develop an efficient feature extraction module for inductive GNNs via graph coarsening . ( ii ) In contrast to E2E methods , IGP adopts an inductive graph embedding scheme across graphs which can tackle the online GP with non-fixed K. ( iii ) Note that GP is an unsupervised node-level task , where the cluster labels are permutation invariant , e.g. , label assignments ( l1 , l2 , l3 ) = ( 1 , 2 , 2 ) and ( l1 , l2 , l3 ) = ( 2 , 1 , 1 ) are the same in terms of GP with li as the cluster label of node vi . IGP can further incorporate such permuttion invariant label information ( e.g. , GP ground-truth ) of historical graphs to the offline training by combining several GP objectives ( e.g. , modularity maximization and NCut minimization ) with a novel dual GNN structure which ensures strong robustness for the online GP on new graphs . 2 PROBLEM STATEMENTS AND PRELIMINARIES . In this study , we consider GP on a set of graphs S = { G1 , G2 , · · · , GT } from a common system or scenario . Each graph Gt ∈ S can be represented as Gt = ( Vt , Et ) with Vt = { vt1 , . . . , vtNt } and Et = { ( vti , vtj ) ∣∣vti , vtj ∈ Vt } as the sets of nodes and edges . For each Gt , topology is the only available information source without graph attributes . We use an adjacency matrix At ∈ < Nt×Nt to describe its topology with Nt nodes , where ( At ) ij = ( At ) ji = 1 if ( vti , v t j ) ∈ Et and ( At ) ij = ( At ) ji = 0 otherwise . Since some systems allow addition and deletion of entities , we assume that different snapshots in S can have different node sets , i.e. , ∃Gt , Gs ∈ S s.t . Vt 6= Vs . Since there may be no temporal dependency among { G1 , · · · , GT } in some cases ( e.g. , multiple ego-nets in social media ) , we do not consider the node correspondence among { V1 , · · · , VT } . We follow the hypothesis adopted in the simulation of various network systems ( Wehrle et al. , 2010 ) that graphs in S are independently generated via several underlying distributions of a common system or scenario . Graph Partitioning . Given a graph Gt , GP aims to partition the node set Vt into Kt subsets ( i.e. , clusters ) Ct = { Ct1 , · · · , CtKt } so that ( i ) within each cluster the linkage is dense but ( ii ) between different clusters the linkage is relatively loose . The result Ct also satisfies Ctr ∩Cts = ∅ for ∀r 6= s. Mathematically , GP can be formulated as several NP-hard combinatorial optimization problems , e.g. , NCut minimization ( Von Luxburg , 2007 ) and modularity maximization ( Newman , 2006 ) . Given a graph Gt , NCut minimization aims to get the GP result Ct that minimizes the NCut metric : arg minCt NCut ( Ct ) = 1 2 ∑Kt r=1 [ cut ( Ctr , C̄ t r ) /vol ( C t r ) ] ( 1 ) where C̄tr = Vt − Ctr is the complementary set of Ctr ; cut ( Ctr , C̄tr ) = ∑ vi∈Ctr , vj∈C̄tr ( At ) ij is the cut between Ctr and C̄ t r ; vol ( C t r ) = ∑ vi , vj∈Ctr ( At ) ij is the volume of C t r. The objective ( 1 ) can be equivalently expressed in the following matrix form : arg minHt tr ( H T t LtHt ) s.t.H T t Ht = IKt , ( 2 ) where Lt = INt − D−0.5t AtD−0.5t is the Laplacian matrix of At ; Dt = diag ( dt1 , · · · , dtNt ) is a diagonal matrix with dti = ∑ j ( At ) ij ; IN is anN -dimensional identity matrix . Ht ∈ < Nt×Kt is the membership indicator , where ( Ht ) ir = [ dti · vol ( Ctr ) −1 ] 0.5 if vti ∈ Ctr and ( Ht ) ir = 0 otherwise . Modularity maximization is another conventional NP-hard objective of GP . Given a graph Gt , it aims to find a partition Ct that maximizes the modularity metric : arg minCt Mod ( Ct ) = 1 2e ∑ r ∑ vti , v t j∈Ctr [ ( At ) ij − d t id t j/ ( 2e ) ] , ( 3 ) where e = ∑ i d t i/2 is the number of edges . The objective can also be rewritten in a matrix form : argminHt − tr ( HTt QtHt ) s.t . tr ( HTt Ht ) = Nt , ( 4 ) where Qt ∈ < Nt×Nt is the modularity matrix with ( Qt ) ij = ( At ) ij − dtidtj/ ( 2e ) . Ht ∈ < Nt×Kt is the membership indicator , where ( Ht ) ir = 1 if vti ∈ Ctr and ( Ht ) ir = 0 otherwise . Inductive Graph Embedding . Given a single graph Gt , conventional transductive graph embedding aims to learn a function f : { vti } 7→ { uti ∈ < 1×k } that maps each node vti to a k-dimensional vector uti . The key characteristics of Gt ( e.g. , clustering structure ) should be preserved in the embedding space , where a node pair ( vti , v t j ) with similar properties ( e.g. , in the same cluster ) should have similar representations ( uti , u t j ) ( e.g. , with close distance in the embedding space ) . The learned representations { uti } are used as the input of several downstream tasks on Gt including GP . To alleviate the NP-hard challenge of GP , we consider a novel inductive graph embedding scheme , where model parameters of f are shared by all the graphs in S. We divide S into a training set Dual GNN Structure Clustering Module Model Opt . ( Param . Fixed ) Dual GNN Structure Snapshots New System Historical GP Results ( e.g. , ground-truth ) { } t { } w.r.t . { } t tC Historical System { } t Snapshots New GP Results ( a ) Offline Training ( b ) Online fast high-quality GP ' ' { } w.r.t . { } t tC Figure 1 : IGP across multiple associated graphs . 1 2 3 4 5 6 7 8 Cluster 1 Cluster 2 1 2 3 4 5 6 7 8 Componet 1 Compoent 2 Feature Encoder Aux . Label- Induced Encoder ( Dual GNN w/ shared parameters ) { 1 , 2 , 3 , 4 } { 5 , 6 , 7 , 8 } Original Graph t Aux . Label-Ind . Graph Supergraph * t ( ) g t ( HEM Graph Coarsening ) Adjacency Matrix At Red . Feat . Zt Aux . Label-Ind . Adj . Matrix ( ) g tA Emb . Ut Aux . LabelInd . Emb . ( ) g tU FC Classifier Discriminator Generator In n er P ro d u ct In Label-Ind . Encoder , message passing only occurs in each component ? tU ( ) ? gtU ( i ) Adv . Learning Obj . Reconstru cted Feat . tX ( ii ) Feature Reconstruction Obj . Neigh-Ind . Feature Xt ( min error ) ( Regularizing ) GP Result Ct ( ground-truth ) of t ( iii ) Clustering Reg . Obj . 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 In Feature Encoder , message passing occurs in the original topology G D Figure 2 : A running example of the offline training of our dual GNN structure . Γ ⊂ S and a test set Γ′ = S − Γ , which represent the sets of historical known and newly generated graphs . We first train f on Γ in an offline way to fully capture structural properties of historical graphs . Although graph embedding and GP are unsupervised procedures , we assume the permutation invariant GP result Ct ( i.e. , label information ) of each historical graph Gt ∈ Γ is available in the offline training , which can be used to enhance the offline embedding optimization . In general , the training label information { Ct|Gt ∈ Γ } can be from the ( i ) ground-truth of system applications ( e.g. , cells in wireless cellular network decomposition ) or ( ii ) results of some high-quality ( but usually time-consuming ) GP baselines . It indicates another application that we can use GP results of some strong baselines to regularize the offline training of IGP . After the training , we directly obtain the embedding { ut′i } from f for each new graph Gt′ ∈ Γ′ with model parameters fixed . The online GP on a new graph Gt′ is the downstream task , where we assume the number of clusters Kt′ is given . The GP result of Gt′ is derived by applying a clustering algorithm ( e.g. , KMeans ) to { ut ′ i } .

Summary: This paper considers the problem of solving the graph partitioning problem repeatedly over many different graphs. Graphs are sampled i.i.d. from an unknown but fixed distribution and the goal of the algorithm is to solve the GP problem on each of the graphs. There are two quantities that are opposing; efficiency (on one hand we can solve an NP-hard problem each time but would be prohibitively time consuming) and quality (on the other hand we can generalize from the learnings on the other graph since they are related via the i.i.d. distribution). This paper proposes a NN architecture that uses a subset of the graphs in the family to learn an embedding, and then derives the solution to the other instances by using this embedding via a matrix multiplication. Using a number of simulated and real-world datasets they show that this method works well empirically.