Hierarchical Graph Matching Networks for Deep Graph Similarity Learning

1 INTRODUCTION . Learning a general similarity metric between arbitrary pairs of graph-structured objects is one of the key challenges in machine learning . Such learning problems often arise in a variety of applications , ranging from graph similar searching in graph-based database ( Yan & Han , 2002 ) , to Fewshot 3D Action Recognition ( Guo et al. , 2018 ) , unknown malware detection ( Wang et al. , 2019 ) , and promising selection in automatic theory proving ( Wang et al. , 2017 ) , to name just a few . Conceptually , classical exact ( or inexact ) graph matching techniques ( Ullmann , 1976 ; Caetano et al. , 2009 ; Bunke & Allermann , 1983 ; Riesen et al. , 2010 ) provide a strong tool for learning graph similarity . However , these methods usually either require input graphs with similar sizes or consider mainly the graph structures for finding a correspondence between the nodes of different graphs without taking into account the node representations or features . In contrast , in this paper , we consider the graph matching problem of learning a mapping between a pair of graph inputs pG1 , G2q P GˆG and the similarity score y P Y , based on a set of training triplet of structured input pairs and scalar output score pG11 , G21 , y1q , ... , pG1n , G2n , ynq P G ˆ G ˆ Y drawn from some fixed but unknown probability distribution . Recent years have seen a surge of interests in graph neural networks ( GNNs ) , which have been demonstrated to be a powerful class of models for learning node embeddings of graph-structured data ( Bronstein et al. , 2017 ) . Various GNN models have since been developed for learning effective node representations for node classification ( Li et al. , 2016 ; Kipf & Welling , 2016 ; Hamilton et al. , 2017 ; Veličković et al. , 2017 ) , or pooling the learned node embeddings into a graph vector for graph classification ( Ying et al. , 2018 ; Ma et al. , 2019 ) , or combining with variational auto-encoder to learn the graph distribution for graph generation ( Simonovsky & Komodakis , 2018 ; Li et al. , 2018 ; Samanta et al. , 2018 ; You et al. , 2018 ) . However , there is relatively less study on learning graph similarity using GNNs . To learn graph similarity , a simple yet straightforward way is to encode each graph as a vector and combine two vectors of each graph to make a decision . This approach is useful since graphlevel embeddings contain important information of a pair of graphs . One obvious limitation of this approach lies in the fact of the ignorance of more fine-grained interactions among different level embeddings of two graphs . Very recently , a few of attempts have been made to take into account lowlevel interactions either by considering the histogram information of node-wise similarity matrix of node embeddings ( Bai et al. , 2019 ) or improving the node embeddings of one graph by incorporating implicit attentive neighbors of another graphs through a soft attention ( Li et al. , 2019 ) . However , there are two significant challenges making these graph matching models potentially ineffective : i ) how to learn different-level granularity ( global level and local level ) of interactions between a pair of graphs ; ii ) how to effectively learn richer cross-level interactions between parts of a graph and a whole graph . Inspired by these observations , in this paper , we propose a Hierarchical Graph Matching Network ( HGMN ) for computing the graph similarity between any pair of graph-structured objects . Our model jointly learns graph representations and a graph matching metric function for computing graph similarity in an end-to-end fashion . The proposed HGMN model consists of a novel multiperspective node-graph matching network for effectively learning cross-level interactions between parts of a graph and a whole graph , and a siamese graph neural network for learning global-level interactions between two graphs . Our final small prediction networks consume these feature vectors from both cross-level and global-level interactions to perform either graph-graph classification or graph-graph regression tasks , respectively . Recently proposed works only compute graph similarity by considering either graph-graph classification problem ( with labels Y “ t´1 , 1u ) ( Li et al. , 2019 ) , or graph-graph regression problem ( with similarity score Y “ r0 , 1s ) ( Bai et al. , 2019 ) . To demonstrate the effectiveness of our model , we systematically investigate the performance of our HGMN model compared with these recently proposed graph matching models on four datasets for both graph-graph classification and regression tasks . To bridge the gap of the lack of standard graph matching datasets , we also create one new dataset from a real application together with a previously released dataset by ( Xu et al. , 2017 ) for graph-graph classification task 1 . One important aspect is previous works did not consider the impact of the size of two input graphs , which often plays an important role in determining the performance of graph matching . Motivated by this observation , we have considered three different ranges of graph sizes from [ 3 , 200 ] , [ 20,200 ] , and [ 50,200 ] in order to evaluate the robustness of each graph matching model . We highlight our main contributions of this paper as follows : • We propose a hierarchical graph matching network ( HGMN ) for computing the graph similarity between any pair of graph-structured objects . Our HGMN model jointly learns graph representations and a graph matching metric function for computing graph similarity in an end-to-end fashion . • In particular , we propose a multi-perspective node-graph matching network for effectively capturing the cross-level interactions between a node embeddings of a graph and a corresponding attentive graph-level embedding of another graph . • We systematically investigate different factors on the performance of all graph matching models such as the impact of different tasks ( classification and regression ) and the sizes of input graphs . • Our comprehensive experiments demonstrate that our proposed HGMN consistently outperforms state-of-the-art graph matching network baselines for both classification and regression tasks . Compared with previous works , our proposed model HGMN is also more robust when the sizes of the two input graphs increase . 2 PROBLEM FORMULATION . In this section , we briefly introduce the problem formulation . Given a pair of graph inputs pG1 , G2q , the aim of the graph matching problem we consider in this paper is to produce a graph similarity score y “ spG1 , G2q P Y . The graph G1 “ pV 1 , E1q is represented as a set of N nodes vi P V 1 with a feature matrix X1 P RNˆd , edges pvi , vjq P E1 ( binary or weighted ) formulating an adjacency matrix A1 P RNˆN , and a degree matrix D1ii “ ř j A 1 ij . Similarly , the graph G 2 “ 1We release these datasets via this link : https : //github.com/runningoat/hgmn_dataset . Our codes will be released as well upon the acceptance of this paper . pV 2 , E2q is represented as a set of M nodes vi P V 2 with a feature matrix X2 P RMˆd , edges pvi , vjq P E2 ( binary or weighted ) formulating an adjacency matrix A2 P RMˆM , and a degree matrix D2ii “ ř j A 2 ij . Note that , when performing graph-graph classification task the scalar y is the class labels y “ t´1 , 1u ; when performing graph-graph regression task the scalar y is the the measure of the similarity score y P r0 , 1s . We train a graph matching model based on a set of training triplet of structured input pairs and scalar output score pG11 , G21 , y1q , ... , pG1n , G2n , Ynq P G ˆ G ˆ Y drawn from some fixed but unknown probability distribution in real applications . 3 HIERARCHICAL GRAPH MATCHING NETWORKS ARCHITECTURE . In this section , we will introduce two key components of our HGMN architecture - Siamese Graph Neural Networks ( SGNN ) and Multi-Perspective Node-Graph Matching Networks ( MPNGMN ) . We first discuss SGNN for learning the global-level interactions between two graphs and then outline MPNGMN for effectively learning the cross-level node-graph interactions between parts of one graph and one whole graph . Our overall model architecture for HGMN is shown in Fig . 1 . 3.1 SGNN FOR GLOBAL-LEVEL INTERACTION LEARNING . The graph-level embeddings contain important information of a graph . Therefore , learning graphlevel interactions between two graphs could be an important component for learning the graph similarity of two graphs . In order to capture the global-level interactions between two graphs , we employ SGNN which is based on Siamese Networks architecture ( Bromley et al. , 1994 ) , which has achieved great success in many applications such as visual recognition ( Bertinetto et al. , 2016 ; Varior et al. , 2016 ) and sentence similarity ( He et al. , 2015 ; Mueller & Thyagarajan , 2016 ) . Independently , a similar idea using high-order siamese graph neural networks was presented for brain network analysis ( Chaudhuri et al. , 2019 ) . Our SGNN adapts popular Graph Convolution Networks ( GCN ) ( Kipf & Welling , 2016 ) with siamese neural networks for simplicity . Other variants of graph neural networks such as GraphSAGE ( Hamilton et al. , 2017 ) and Gated Graph Neural Networks ( Li et al. , 2016 ) can also be used . Our SGNN consists of three components : 1 ) node embedding layers ; 2 ) graph-level embedding aggregation layers ; 3 ) graph-graph matching and prediction layers . Node Embedding Layers . We utilize three-layer GCN with the siamese networks to generate node embeddings H l “ thliu tN , Mu i “ 1 P RtN , Muˆd 1 of both graphs G1 and G2 , H l “ fpX l , Alq “ ReLU ´ sAl ReLU ´ sAl ReLU ´ sAlX lW p0q ¯ W p1q ¯ W p2q ¯ , l “ t1 , 2u . ( 1 ) where sAl “ p rDlq´ 12 rAlp rDlq´ 12 is the normalized Laplacian matrix for rAl “ Al ` ItN , Mu depending on the graph is G1 or G2 , and W piq , i “ t0 , 1 , 2u are hidden weighted matrices for each layer . Note that the twin networks share the parameters of GCN when training on the pair of graphs pG1 , G2q . The number of GCN layers required depends on the real application graph data . To isolate the effect of overtuning , we choose the three layers after some initial experiments on validation sets . Graph-level Embedding Aggregation Layers . After we compute the resulting node embeddings H l of each graph from GCN , we need to aggregate these node embeddings to formulate their corresponding graph-level embeddings of each graph . hlG “ Aggregate ´ thliu tN , Mu i “ 1 ¯ , l “ t1 , 2u . ( 2 ) We employ different aggregation functions such as element-wise max pooling ( Max ) , elementwise max pooling following a transformation by applying a fully connected layer on Hi ( FCMax ) , element-wise mean pooling ( Avg ) , element-wise mean pooling following a transformation by applying a fully connected layer on Hi ( FCAvg ) , and a sophisticated aggregator based on LSTM architecture ( Hochreiter & Schmidhuber , 1997a ) . Note that , among these aggregation functions , the LSTM aggregator is not permutation invariant on a set of node embeddings although LSTM may admit more expressive ability . We adapt LSTMs to operate on these node embeddings by simply applying the LSTMs to a random permutation of the node embeddings . Graph-Graph Matching and Prediction Layers . After the graph-level embeddings h1G and h2G are computed for the graphs G1 and G2 , we then use the resulting graph embeddings to compute the graph similarity score of pG1 , G2q . Depending on the specific tasks , we have slightly different ways to calculate the final similarity score . For classification tasks , we simply compute the cosine similarity of two graph-level embeddings , ry “ spG1 , G2q “ cosineph1G , h2Gq ( 3 ) where the similarity function s could also be other similarity metric such as Euclidean similarity and dot-product similarity . We find that the cosine similarity function performs generally better across different datasets . For regression tasks , we first concatenate the two aggregated graph embeddings to rh1G , h2Gs and then employ four standard fully connected layers to gradually project the vector of dimension rh1G , h2Gs down to a scalar of the dimension 1 . Since the expected similar score ry should be in range of [ 0,1 ] , we perform sigmoid function to enforce the final score in this range . We therefore compute the similarity score for graph-graph regression task as following , ry “ spG1 , G2q “ sigmoid ´ MLP ´ rh1G , h2Gs ¯¯ . ( 4 ) For both tasks , we train the SGNN model using mean square error loss function to compare the computed similarity score ry with the groud-truth similarity score y , L “ 1 n n ÿ i “ 1 ´ ry ´ y ¯2 . ( 5 )

Summary: In this paper, a hierarchical graph matching network, which considers both graph-graph interaction and node-graph interaction, is proposed. Specifically, the graph-graph interaction is modeled through graph level embeddings learned by GCN with pooling layers. While node-graph interaction is modeled using node embedding learned by GCN and attentive graph embedding aggregated from node embedding.