G-Mixup: Graph Augmentation for Graph Classification

1 INTRODUCTION . Recently deep learning has been widely adopted to graph analysis . Graph Neural Networks ( GNNs ) ( Wu et al. , 2020 ; Zhou et al. , 2020b ; Zhang et al. , 2020 ; Xu et al. , 2018 ) have made many significant breakthroughs on graph classification . Meanwhile , data augmentation ( e.g. , DropEdge ( Rong et al. , 2020 ) , Subgraph ( You et al. , 2020 ) ) has also been adopted to graph analysis by generating synthetic graphs to create more training data for improving the generalization of graph classification models . However , existing graph data augmentation strategies typically aim to augment graphs at a within-graph level by either modifying edges or nodes in individual graph , which limits them to only generating new graphs based on one individual graph . The between-graph augmentation methods ( i.e. , augmenting graphs between graphs ) are still under-explored . In parallel with the development of graph neural networks , Mixup ( Zhang et al. , 2017 ) and its variants ( e.g. , Manifold Mixup ( Verma et al. , 2019 ) ) , as data augmentation methods , have been theoretically and empirically shown to improve the generalization and robustness of deep neural networks in image recognition ( Zhang et al. , 2017 ; Verma et al. , 2019 ; Zhang et al. , 2021 ) and natural language processing ( Guo et al. , 2019 ; Guo , 2020 ) . The basic idea of Mixup is to linearly interpolate continuous values of random sample pairs to generate more synthetic training data . The formal mathematical expression of Mixup is xnew = λxi + ( 1 − λ ) xj , ynew = λyi + ( 1 − λ ) yj , where ( xi , yi ) and ( xj , yj ) are two samples drawn at random from training data and the target y are onehot labels . With graph neural network and mixup in mind , the following question naturally arises , Can we mix up graph data to improve the generalization and robustness of GNNs ? It remains an open and challenging problem to mix up the graph data due to the characteristics of graphs and the requirements of applying Mixup . Typically , Mixup requires that original data instances are regular and well-aligned in Euclidean space , such as image data and table data . However , graph data is distinctly different from image data due to the following characteristics : ( i ) graph data is irregular . The number of nodes in different graphs are typically different to others ; ( ii ) graph data is not well-aligned . The nodes in different graphs can not be aligned well directly ; ( iii ) graph topology between classes are divergent . The topologies of a pair of graphs from different classes are usually different while the topologies of those from the same class are usually similar . thus make it challenging to directly adopt the Mixup strategy to graph data . WI WH WG I = { I1 , I2 , · · · , Ik } with label ( 0.5 , 0.5 ) H = { H1 , H2 , · · · , Hk } with label ( 0 , 1 ) G = { G1 , G2 , · · · , Gk } with label ( 1 , 0 ) … … … 1 ) graphon estimation 3 ) graph sampling WI = 0.5 ⇤ WG + 0.5 ⇤ WH 1 ) graphon estimation 2 ) graphon mixup Figure 1 : An overview of G-Mixup . The task is binary graph classification . We have two classes of graphs G andH with different topologies ( G has two communities whileH has eight communities ) . G and H have different graphons . We mix up the graphons WG and WH to obtain a mixed graphon WI , and then sample new graphs from the mixed graphon . Intuitively , the synthetic graphs have two major communities and each of which has four sub-communities , demonstrating that the generated graphs preserve the structure of original graphs from both classes . To tackle the aforementioned problems , we propose G-Mixup , a class-level graph data augmentation method , to mix up graph data based on graphons . The graphs within one class have the same generator ( i.e. , graphon ) . We mix up the graphons of different classes and then generate synthetic graphs . Informally , a graphon can be thought of as a probability matrix ( e.g. , the matrix WG and WH in Figure 1 ) , where W ( i , j ) represents the probability of edge between node i and j . The realworld graphs can be regraded as generated from graphons . In this way , we can mix two classes of graphs by mixing their generators . Since the graphons of different graphs is regular , well-aligned , and is defined in Euclidean space , it is easy and natural to mix up the graphons and then generate the synthetic graphs therefrom . We also provide theoretical analysis of graphons mixup , which guarantees that the generated graphs will preserve the key characteristics of both original classes . Our proposed method is illustrated in Figure 1 with an example . Given two graph training sets G = { G1 , G2 , · · · , Gk } and H = { H1 , H2 , · · · , Hk } with different labels and distinct topologies ( i.e. , G has two communities while H has eight communities ) , we estimate graphons WG and WH respectively from G and H. We then mix up the two graphons and obtain a mixed graphon WI . After that , we sample synthetic graphs from WI as additional training graphs . The generated synthetic graphs have two major communities and each of them have four sub-communities , which is a mixture of the two graph sets . It thus shows that G-Mixup is capable of mixing up graphs . In summary , our main contributions are three-fold . Firstly , we propose G-Mixup to augment the training graphs for graph classification . Since directly mixing up graphs is intractable , G-Mixup mixes the graphons of different classes of graphs to generate synthetic graphs . Secondly , we theoretically prove that the synthetic graph will be the mixture of the original graphs , where the key topology ( i.e. , discriminative motif ) of source graphs will be preserved . Thirdly , we demonstrate the efficacy of the proposed G-Mixup on various graph neural network backbones and datasets . Extensive experimental results show the proposed G-Mixup substantially improves the performance of graph neural networks in terms of enhancing their generalization and robustness . 2 PRELIMINARIES . In this section , we first go over the notations used in this paper , and then introduce graph related concepts including graph homomorphism and graphons , which will be used for theoretical analysis in this work . Finally , we briefly review the graph neural networks for graph classification . 2.1 NOTATIONS . Given a graph G , we use V ( G ) and E ( G ) to denote its nodes and edges , respectively . The number of nodes is v ( G ) = |V ( G ) | , and the number of edges is e ( G ) = |E ( G ) | . We use G , H , I to denote graphs and G , H , I to denote graph set . yG ∈ RC denotes the label of graph set G , where C is number of classes of graphs . A graph could contain some interesting and frequent patterns and subgraphs which are called motifs . The motifs in graph G is denoted as FG . The set of motifs in graph set G is denoted as FG . WG denotes the graphon of graph set G. W denotes the step function . Unif [ 0,1 ] denotes the uniform distribution between 0 and 1 . Bern ( · ) denotes the Bernoulli distribution . G ( n , W ) denotes the random graph with n nodes based on graphon W . 2.2 GRAPH HOMOMORPHISM AND GRAPHONS . Graph Homomorphism . A graph homomorphism is an adjacency-preserving mapping between two graphs , i.e. , mapping adjacent vertices in one graph to adjacent vertices in the other . Formally , a graph homomorphism φ : F → G is a map from V ( F ) to V ( G ) , where if { u , v } ∈ E ( F ) , then { φ ( u ) , φ ( v ) } ∈ E ( G ) . For two graphs H and G , there could be multiple graph homomorphisms between them . Let hom ( H , G ) denotes the total number of graph homomorphisms from graph H to graph G. For example , hom ( , G ) = |V ( G ) | if graph H is , hom ( , G ) = 2|E ( G ) | if graph H is , and hom ( , G ) is six times the number of triangles in G. There are in total |V ( G ) ||V ( H ) | mappings from H to G , but only some of them are homomorphisms . Thus , we define homomorphism density to measure the relative frequency that the graph H appears in graph G : t ( H , G ) = hom ( H , G ) |V ( G ) ||V ( H ) | . For example , t ( , G ) = |V ( G ) |/n1 = 1 , t ( , G ) = 2|E ( G ) |/n2 . Graphon . A graphon ( Airoldi et al. , 2013 ) is a continuous , bounded and symmetric function W : [ 0 , 1 ] 2 → [ 0 , 1 ] which may be thought of as the weight matrix of a graph with infinite number of nodes . Then , given two points ui , uj ∈ [ 0 , 1 ] , W ( i , j ) represents the probability that nodes i and j be related with an edge . Various quantities of a graph can be calculated as a function of the graphon . For example , the degree of nodes in graphs can be easily extended to a degree distribution function in graphons , which is characterized by its graphon marginal dW ( x ) = ∫ 1 0 W ( x , y ) dy . Similarly , the concept of homomorphism density can be naturally extended from graphs to graphons . Given an arbitrary graph motif F , its homomorphism density with respect to graphon W is defined by t ( F , W ) = ∫ [ 0,1 ] V ( F ) ∏ i , j∈E ( F ) W ( xi , xj ) ∏ i∈V ( F ) dxi . For example , the edge density of graphon W is t ( , W ) = ∫ [ 0,1 ] 2 W ( x , y ) dxdy , and the triangle density of graphon W is t ( , W ) = ∫ [ 0,1 ] 3 W ( x , y ) W ( x , z ) W ( y , z ) dxdydz . 2.3 GRAPH CLASSIFICATION WITH GRAPH NEURAL NETWORKS . Given a set of graphs , graph classification aims to assign a class label for each graph G. Recently , graph neural networks have become the state-of-the-art approach for graph classification . Without loss of generalization , we present the formal expression of a graph convolution network ( GCN ) ( Kipf & Welling , 2016 ) . The forward propagation at k-th layer is described as the following : a ( k ) i = AGG ( k ) ( { h ( k−1 ) j : j ∈ N ( i ) } ) , h ( k ) i = COMBINE ( k ) ( h ( k−1 ) i , a ( k ) i ) , ( 1 ) where h ( k ) i ∈ Rn×dk is the intermediate representation of node i at the k-th layer , N ( i ) denotes the neighbors of node i. AGG ( · ) is an aggregation function to collect embedding representations from neighbors , and COMBINE ( · ) combines neighbor representation and its representation at ( k−1 ) -th layer followed by nonlinear transformation . For graph classification , a graph-level representation is obtained by summarizing all node-level representations in the graph by a readout function : hG = READOUT ( { h ( k ) i : i ∈ E ( G ) } ) , ŷ = softmax ( hG ) , ( 2 ) where READOUT ( · ) is the readout function , which can be a simple function such as average or sophisticated pooling function ( Gao & Ji , 2019 ; Ying et al. , 2018 ) , hG is the representation of graph G , and ŷ ∈ RC is the output estimating the probability that G belongs to each of the C classes .

Summary: This work studied the research problem of graph data augmentation for supervised graph classification. The authors proposed G-mixup that performs mixup on graph data via graphon generators. Due to the irregular characteristics of graphs, instead of directly mixing up the data objects, G-mixup mix up the graph generators.