GraphENS: Neighbor-Aware Ego Network Synthesis for Class-Imbalanced Node Classification

1 INTRODUCTION . Node classification for graphs has attracted significant attention as the importance of large-scale graphs analysis increases in various domains such as bioinformatics and commercial graphs to name a few ( Perozzi et al. , 2016 ; Hamilton et al. , 2017 ; Ying et al. , 2018 ; Mohammadrezaei et al. , 2018 ) . For example , in retail services , acquiring the qualitative node representations for items or customers is critical for improving the quality of recommendation systems ( Perozzi et al. , 2016 ; Ying et al. , 2018 ) . Detecting abnormal users in social networks , as another example , is also closely related to classifying the property of each node ( Mohammadrezaei et al. , 2018 ) . Recently , Graph Neural Networks ( GNNs ) have demonstrated their effectiveness on learning node representations ( Hamilton et al. , 2017 ; Kipf & Welling , 2017 ; Velickovic et al. ) . However , the nodes in many real-world graphs are sometimes class-imbalanced ( Mohammadrezaei et al. , 2018 ; Wang et al. , 2020a ) , hence GNNs are prone to be biased toward major classes , as in general class-imbalance tasks . This bias forces networks to poorly classify the nodes of minor classes , resulting in destructive impacts and a large cost to their services . While the peculiar characteristics of imbalanced node classification and specialized solutions suitable for it have hardly been investigated , applying generic imbalance handling methods ( Chawla et al. , 2002 ; Cui et al. , 2019 ; Cao et al. , 2019 ; Kang et al. , 2020 ; Menon et al. , 2021 ) directly to the graph domain has several non-trivial challenges . One of the distinct natures of graph data is that adjacent nodes are involved in constructing the representation of each node , which makes the model more confused to learn the unbiased representation of minor class nodes . Here , we hypothesize that it is in fact more serious to overfitting to neighbors of minor nodes than to overfitting to the node feature itself . This ‘ neighbor memorization ’ is the critical obstacle to naively adopt the class-imbalance approaches of other domains such as re-weighting and re-sampling used in image classification . Specifically , re-weighting approaches ( Cui et al. , 2019 ; Tan et al. , 2020 ; Cao et al. , 2019 ; Menon et al. , 2021 ; Hong et al. , 2021 ) , applying penalties according to the number of data , simply assign large weight to minor nodes , hence there is no change in neighbor sets of minor nodes observed during training . Re-sampling methods ( Chawla et al. , 2002 ; Kang et al. , 2020 ; Zhang et al. , 2021 ; Wang et al. , 2021 ) , sampling data to balance the number of data for each class , are also vulner- able to overfit on minor nodes with their neighbors . Another challenge of the re-sampling method , especially for oversampling variants , is determining how to connect the newly sampled nodes to the original graph . For example , simply connecting an oversampled node with all neighbors of original node will change the edges of neighbor nodes as well and hence significantly alters the message passing to the neighbors , which might impair their class semantics . To mitigate this issue , GraphSMOTE ( Zhao et al. , 2021 ) exploits edge predictor to decide the connectivity with the neighbors of two minor nodes used in oversampling . Nevertheless , GNNs trained with GraphSMOTE still suffer from neighbor memorization when the number of minor nodes is limited . In this paper , we propose GraphENS , a novel augmentation approach that synthesizes the whole ego network for minor class ( minor node and its one-hop neighbors ) by interpolating two different ego networks in the data ; to enlarge the limited neighbor views of minor instances in the data , our method combines the ego network of anchoring minor node with that of randomly selected node from all classes , where it interpolates ego networks based on KL-divergence between model predictions of ego networks in order to keep the semantics of minor classes . Synthesized ego networks are attached to the original graph to construct a class-balanced graph , and GNNs are trained with the enlarged graph . GraphENS enables the model to learn the minor class nodes with feasible neighbors by generating the virtual ego networks . To further prevent the synthesis of deleterious ego networks , we introduce a saliency-based node mixing approach to generate the central node of ego network . Our method separates class-generic node features from class-specific node features by using saliency information of each feature , and exploits only class-generic attributes to combine with node feature of anchoring minor node . We validate our method on various benchmark datasets including citation networks ( Sen et al. , 2008 ) , and Amazon product co-purchasing networks ( Shchur et al. , 2018 ) with diverse architectures such as GCN ( Kipf & Welling , 2017 ) , GAT ( Velickovic et al . ) , and GraphSAGE ( Hamilton et al. , 2017 ) , and confirm that our approach consistently outperforms baseline methods over various settings . In summary , our contribution is threefold : • We demonstrate that in class-imbalanced node classification , GNNs severely overfit to neighbor sets of minor class nodes , rather than to minor nodes themselves . This ‘ neighbor memorization ’ problem becomes severe when the number of minor nodes is extremely small . • Our method effectively alleviates the neighbor memorization problem in class-imbalanced graphs by synthesizing feasible ego networks based on the similarity between source ego networks . We also block the injection of harmful features in generating the mixed nodes using node feature saliency . • Through extensive experiments , we show that our approach consistently outperforms baselines on multiple benchmark datasets including real-world imbalanced datasets . Even in highly imbalanced synthetic graphs , our method exhibits superior performance . 2 RELATED WORK AND PRELIMINARY . 2.1 CLASS IMBALANCE PROBLEM . The goal of class-imbalance handling in classification is to construct an unbiased classifier to the label distribution of the training set . There are three main streams : loss modification , post-hoc correction , and re-sampling approaches . Loss modification methods alter the objective function to assign more weights ( Japkowicz & Stephen , 2002 ; Cui et al. , 2019 ) or margins ( Tan et al. , 2020 ; Cao et al. , 2019 ; Menon et al. , 2021 ) on minor classes . Post-hoc correction strategies ( Kang et al. , 2020 ; Tian et al. , 2020 ; Menon et al. , 2021 ; Hong et al. , 2021 ) remedy logits to compensate minor classes in the inference . Re-sampling approaches augment minor class data by sampling strategies ( Kang et al. , 2020 ; Liu et al. , 2020 ; Ren et al. , 2020 ) or generation ( Chawla et al. , 2002 ; Kim et al. , 2020a ; Chu et al. , 2020 ; Zhang et al. , 2021 ; Wang et al. , 2021 ) . Among minor class generation approaches , SMOTE ( Chawla et al. , 2002 ) is a widely used method to mix minor data with the nearest data of the identical class . Synthesizing minor class data from data of other classes ( Kim et al. , 2020a ; Chu et al. , 2020 ; Wang et al. , 2021 ) is introduced to exploit the rich information of other classes . Kim et al . ( 2020a ) produces new minor class data by taking gradient steps to translate major class data into minor class data . Wang et al . ( 2021 ) synthesizes minor class data by combining features of minor class data with feature displacements of other data . To extend these approaches to the graph domain , structural aspects of graph have to be considered when generating minor instances . In node classification , imbalance handling works ( Zhou et al. , 2018 ; Wang et al. , 2020b ; Shi et al. , 2020 ; Zhao et al. , 2021 ; Qu et al. , 2021 ) are proposed to exploit structural information in graphs . DR-GCN ( Shi et al. , 2020 ) produces the virtual minor nodes generated by additional conditional GAN and regularizes the features of virtual nodes close to adjacent nodes . GraphSMOTE ( Zhao et al. , 2021 ) generates synthetic minor nodes by interpolating two minor class nodes and a ( pretrained ) edge predictor determines the connectivity of synthetized nodes between synthetized nodes and neighbors of two source minor nodes . ImGAGN ( Qu et al. , 2021 ) synthesizes minor nodes by interpolating features among whole minor nodes with the generated weight matrix . Then the synthesized nodes are connected to the original minor nodes if weights in the matrix are larger than a fixed threshold . As GraphSMOTE and ImGAGN only utilize nodes of the identical minor class to generate minor nodes , the sample diversity of synthesized nodes would be significantly constrained . Moreover , ImGAGN mainly targets binary classification and its extension to multi-class classification is non-trivial since an independent generator is required per each class . Compared to these approaches , our GraphENS utilizes whole nodes to synthesize minor nodes , thus our method outperforms baselines when the number of minor classes is low in Section 5.3 ( Table 4 ) . 2.2 GRAPH NEURAL NETWORKS FOR NODE CLASSIFICATION . We briefly introduce GNNs for node classification tasks . Let us first define graph G ( V , E ) where V is the set of nodes and E is the set of edges between two nodes . Let X ∈ R|V |×d be the node features whose the i-th row represents the d-dimensional feature of the i-th node . N ( v ) is the set of adjacent nodes that are directly connected to v : { u ∈ V | { u , v } ∈ E } . In node classification , each node in the graph corresponds to a class y ∈ { 1 , . . . , C } where C is the number of classes . In this paper , we consider several variants of GNNs that consists of three differentiable functions : 1 ) message function ml , 2 ) permutation invariant message aggregation function φl , and 3 ) node update function hl . Let x ( l ) v be the latent vector of node v at layer l. To simplify notations for the recursive definition of GNNs , we use x ( 0 ) v to denote the input node feature . At each GNN layer , node features are updated as x ( l+1 ) v = hl ( x ( l ) v , φl ( { ml ( x ( l ) v , x ( l ) u , ev , u ) |u ∈ N ( v ) } ) to consider adjacent node embeddings . By passing these aggregated node features to a linear classifier as input , we obtain the prediction ŷ = f ( xv ) for node v where ŷc = P ( y = c|xv ) . As a representative example , a layer of Graph Convolutional Network ( GCN ) ( Kipf & Welling , 2017 ) is defined as x ( l+1 ) v = Θ ∑ u∈N ( v ) ∪ { v } ev , u√ d̂ud̂v x ( l ) u where d̂v = 1 + ∑ u∈N ( v ) ev , u with ev , u is the edge weight of edge { u , v } ∈ E and Θ is a filter parameters . There are several variants depending on how these key components are designed , but they are beyond scope of our work .

Summary: This paper investigates the problem of class-imbalanced node classification on graph. To address this, the authors first conduct a case study on a dataset to analyze the underlying reason for inferior performance on minor class nodes. Then a model named GraphENS, which consists of two key components, is proposed to deal with the imbalance issue. Experiments on several datasets show that the proposed model can outperform the baselines.