G-Mixup: Graph Augmentation for Graph Classification

1 INTRODUCTION . Recently deep learning has been widely adopted to graph analysis . Graph Neural Networks ( GNNs ) ( Wu et al. , 2020 ; Zhou et al. , 2020b ; Zhang et al. , 2020 ; Xu et al. , 2018 ) have made many significant breakthroughs on graph classification . Meanwhile , data augmentation ( e.g. , DropEdge ( Rong et al. , 2020 ) , Subgraph ( You et al. , 2020 ) ) has also been adopted to graph analysis by generating synthetic graphs to create more training data for improving the generalization of graph classification models . However , existing graph data augmentation strategies typically aim to augment graphs at a within-graph level by either modifying edges or nodes in individual graph , which limits them to only generating new graphs based on one individual graph . The between-graph augmentation methods ( i.e. , augmenting graphs between graphs ) are still under-explored . In parallel with the development of graph neural networks , Mixup ( Zhang et al. , 2017 ) and its variants ( e.g. , Manifold Mixup ( Verma et al. , 2019 ) ) , as data augmentation methods , have been theoretically and empirically shown to improve the generalization and robustness of deep neural networks in image recognition ( Zhang et al. , 2017 ; Verma et al. , 2019 ; Zhang et al. , 2021 ) and natural language processing ( Guo et al. , 2019 ; Guo , 2020 ) . The basic idea of Mixup is to linearly interpolate continuous values of random sample pairs to generate more synthetic training data . The formal mathematical expression of Mixup is xnew = λxi + ( 1 − λ ) xj , ynew = λyi + ( 1 − λ ) yj , where ( xi , yi ) and ( xj , yj ) are two samples drawn at random from training data and the target y are onehot labels . With graph neural network and mixup in mind , the following question naturally arises , Can we mix up graph data to improve the generalization and robustness of GNNs ? It remains an open and challenging problem to mix up the graph data due to the characteristics of graphs and the requirements of applying Mixup . Typically , Mixup requires that original data instances are regular and well-aligned in Euclidean space , such as image data and table data . However , graph data is distinctly different from image data due to the following characteristics : ( i ) graph data is irregular . The number of nodes in different graphs are typically different to others ; ( ii ) graph data is not well-aligned . The nodes in different graphs can not be aligned well directly ; ( iii ) graph topology between classes are divergent . The topologies of a pair of graphs from different classes are usually different while the topologies of those from the same class are usually similar . thus make it challenging to directly adopt the Mixup strategy to graph data . WI WH WG I = { I1 , I2 , · · · , Ik } with label ( 0.5 , 0.5 ) H = { H1 , H2 , · · · , Hk } with label ( 0 , 1 ) G = { G1 , G2 , · · · , Gk } with label ( 1 , 0 ) … … … 1 ) graphon estimation 3 ) graph sampling WI = 0.5 ⇤ WG + 0.5 ⇤ WH 1 ) graphon estimation 2 ) graphon mixup Figure 1 : An overview of G-Mixup . The task is binary graph classification . We have two classes of graphs G andH with different topologies ( G has two communities whileH has eight communities ) . G and H have different graphons . We mix up the graphons WG and WH to obtain a mixed graphon WI , and then sample new graphs from the mixed graphon . Intuitively , the synthetic graphs have two major communities and each of which has four sub-communities , demonstrating that the generated graphs preserve the structure of original graphs from both classes . To tackle the aforementioned problems , we propose G-Mixup , a class-level graph data augmentation method , to mix up graph data based on graphons . The graphs within one class have the same generator ( i.e. , graphon ) . We mix up the graphons of different classes and then generate synthetic graphs . Informally , a graphon can be thought of as a probability matrix ( e.g. , the matrix WG and WH in Figure 1 ) , where W ( i , j ) represents the probability of edge between node i and j . The realworld graphs can be regraded as generated from graphons . In this way , we can mix two classes of graphs by mixing their generators . Since the graphons of different graphs is regular , well-aligned , and is defined in Euclidean space , it is easy and natural to mix up the graphons and then generate the synthetic graphs therefrom . We also provide theoretical analysis of graphons mixup , which guarantees that the generated graphs will preserve the key characteristics of both original classes . Our proposed method is illustrated in Figure 1 with an example . Given two graph training sets G = { G1 , G2 , · · · , Gk } and H = { H1 , H2 , · · · , Hk } with different labels and distinct topologies ( i.e. , G has two communities while H has eight communities ) , we estimate graphons WG and WH respectively from G and H. We then mix up the two graphons and obtain a mixed graphon WI . After that , we sample synthetic graphs from WI as additional training graphs . The generated synthetic graphs have two major communities and each of them have four sub-communities , which is a mixture of the two graph sets . It thus shows that G-Mixup is capable of mixing up graphs . In summary , our main contributions are three-fold . Firstly , we propose G-Mixup to augment the training graphs for graph classification . Since directly mixing up graphs is intractable , G-Mixup mixes the graphons of different classes of graphs to generate synthetic graphs . Secondly , we theoretically prove that the synthetic graph will be the mixture of the original graphs , where the key topology ( i.e. , discriminative motif ) of source graphs will be preserved . Thirdly , we demonstrate the efficacy of the proposed G-Mixup on various graph neural network backbones and datasets . Extensive experimental results show the proposed G-Mixup substantially improves the performance of graph neural networks in terms of enhancing their generalization and robustness . 2 PRELIMINARIES . In this section , we first go over the notations used in this paper , and then introduce graph related concepts including graph homomorphism and graphons , which will be used for theoretical analysis in this work . Finally , we briefly review the graph neural networks for graph classification . 2.1 NOTATIONS . Given a graph G , we use V ( G ) and E ( G ) to denote its nodes and edges , respectively . The number of nodes is v ( G ) = |V ( G ) | , and the number of edges is e ( G ) = |E ( G ) | . We use G , H , I to denote graphs and G , H , I to denote graph set . yG ∈ RC denotes the label of graph set G , where C is number of classes of graphs . A graph could contain some interesting and frequent patterns and subgraphs which are called motifs . The motifs in graph G is denoted as FG . The set of motifs in graph set G is denoted as FG . WG denotes the graphon of graph set G. W denotes the step function . Unif [ 0,1 ] denotes the uniform distribution between 0 and 1 . Bern ( · ) denotes the Bernoulli distribution . G ( n , W ) denotes the random graph with n nodes based on graphon W . 2.2 GRAPH HOMOMORPHISM AND GRAPHONS . Graph Homomorphism . A graph homomorphism is an adjacency-preserving mapping between two graphs , i.e. , mapping adjacent vertices in one graph to adjacent vertices in the other . Formally , a graph homomorphism φ : F → G is a map from V ( F ) to V ( G ) , where if { u , v } ∈ E ( F ) , then { φ ( u ) , φ ( v ) } ∈ E ( G ) . For two graphs H and G , there could be multiple graph homomorphisms between them . Let hom ( H , G ) denotes the total number of graph homomorphisms from graph H to graph G. For example , hom ( , G ) = |V ( G ) | if graph H is , hom ( , G ) = 2|E ( G ) | if graph H is , and hom ( , G ) is six times the number of triangles in G. There are in total |V ( G ) ||V ( H ) | mappings from H to G , but only some of them are homomorphisms . Thus , we define homomorphism density to measure the relative frequency that the graph H appears in graph G : t ( H , G ) = hom ( H , G ) |V ( G ) ||V ( H ) | . For example , t ( , G ) = |V ( G ) |/n1 = 1 , t ( , G ) = 2|E ( G ) |/n2 . Graphon . A graphon ( Airoldi et al. , 2013 ) is a continuous , bounded and symmetric function W : [ 0 , 1 ] 2 → [ 0 , 1 ] which may be thought of as the weight matrix of a graph with infinite number of nodes . Then , given two points ui , uj ∈ [ 0 , 1 ] , W ( i , j ) represents the probability that nodes i and j be related with an edge . Various quantities of a graph can be calculated as a function of the graphon . For example , the degree of nodes in graphs can be easily extended to a degree distribution function in graphons , which is characterized by its graphon marginal dW ( x ) = ∫ 1 0 W ( x , y ) dy . Similarly , the concept of homomorphism density can be naturally extended from graphs to graphons . Given an arbitrary graph motif F , its homomorphism density with respect to graphon W is defined by t ( F , W ) = ∫ [ 0,1 ] V ( F ) ∏ i , j∈E ( F ) W ( xi , xj ) ∏ i∈V ( F ) dxi . For example , the edge density of graphon W is t ( , W ) = ∫ [ 0,1 ] 2 W ( x , y ) dxdy , and the triangle density of graphon W is t ( , W ) = ∫ [ 0,1 ] 3 W ( x , y ) W ( x , z ) W ( y , z ) dxdydz . 2.3 GRAPH CLASSIFICATION WITH GRAPH NEURAL NETWORKS . Given a set of graphs , graph classification aims to assign a class label for each graph G. Recently , graph neural networks have become the state-of-the-art approach for graph classification . Without loss of generalization , we present the formal expression of a graph convolution network ( GCN ) ( Kipf & Welling , 2016 ) . The forward propagation at k-th layer is described as the following : a ( k ) i = AGG ( k ) ( { h ( k−1 ) j : j ∈ N ( i ) } ) , h ( k ) i = COMBINE ( k ) ( h ( k−1 ) i , a ( k ) i ) , ( 1 ) where h ( k ) i ∈ Rn×dk is the intermediate representation of node i at the k-th layer , N ( i ) denotes the neighbors of node i. AGG ( · ) is an aggregation function to collect embedding representations from neighbors , and COMBINE ( · ) combines neighbor representation and its representation at ( k−1 ) -th layer followed by nonlinear transformation . For graph classification , a graph-level representation is obtained by summarizing all node-level representations in the graph by a readout function : hG = READOUT ( { h ( k ) i : i ∈ E ( G ) } ) , ŷ = softmax ( hG ) , ( 2 ) where READOUT ( · ) is the readout function , which can be a simple function such as average or sophisticated pooling function ( Gao & Ji , 2019 ; Ying et al. , 2018 ) , hG is the representation of graph G , and ŷ ∈ RC is the output estimating the probability that G belongs to each of the C classes .

Summary: This paper proposes the mixup method for graphs which is based on the concept of the graphon. Specifically, G-mixup first estimates the graphon for each class of graphs with the same label then interpolates the graphons of different classes of graphs. Finally, they generate the new graphs from the interpolated graphons to produce the mixed graph with a mixed label.  The authors theoretically show that G-mixup can preserve the different motifs of the two different graphs into one mixed graphon. In experiments, they empirically show that the G-mixup improve the generalization and the performance of existing GNNs on several graph classification task.