Neural Maximum Common Subgraph Detection with Guided Subgraph Extraction

1 INTRODUCTION . Graph data are ubiquitous . Due to its flexible and expressive nature , graphs have been used to store data of various domains . In computational biology , atomic networks can represent molecular compounds . In software analysis , program dependence graphs can describe the data and control dependencies . In social science , social networks can represent community structures . Because of graphs ’ unique ability to capture these data , algorithms tackling novel tasks on graphs across different domains have been gaining increased interest in the representation learning community . Graph matching , in particular , is a recently popular task with new approaches such as Zanfir & Sminchisescu ( 2018 ) , Bai et al . ( 2019a ) , and Li et al . ( 2019 ) . These methods either produce a score indicating how much and/or how well two graphs match or a graph alignment indicating how and where two graphs match . The latter case is a far more difficult task than the former one . Current methods perform a special case of graph alignment , namely image matching , where the two input graphs are of spatial structures ( ex . pixel grids , object orientation , etc . ) To account for more general graphs , we extract the Maximum Common Subgraph ( MCS ) ( Bunke & Shearer , 1998 ) , a widely used metric for graph alignment , and perform matching on the extracted subgraphs . MCS is a very useful metric to match two graphs in real-world applications . For example , in drug discovery , identifying compounds sharing similar substructures which tend to share similarity properties can dramatically reduce the amount of molecules that need to be manually tested ( Ehrlich & Rarey , 2011 ) . In addition to molecular science , MCS also has application values in malware detection ( Park et al. , 2013 ) , pattern recognition ( Solnon et al. , 2015 ) , computer-aided circuit design ( Djoko et al. , 1997 ; Li et al. , 2012 ) , etc . Unfortunately , MCS is NP-hard and , to the best of our knowledge , no existing algorithms tackle this problem from a purely machine learning approach . We are among the first to tackle graph matching defined by MCS . This is more challenging than image matching or graph similarity computation , because MCS requires the extraction of the largest connected subgraph that is commonly present in both input graphs . This implies that the two extracted subgraphs must not only be contained in both graphs but also be isomorphic to each other . To capture the MCS definition , our proposed model , NEURALMCS , fundamentally changes the way that the representations are learned . Instead of computing similarity all in one step , we introduce an iterative procedure to match nodes one at a time . By selecting nodes successively , we ensure the extracted subgraphs are connected . We utilize subgraph embeddings to perform a stopping condition check , dealing with subgraph isomorphism and ending the procedure when the connected subgraphs are the largest . By performing this iterative procedure , we both better ensure the isomorphism of the extracted subgraph as well as capture the recurrent relationship between matching pairs of nodes . We experimentally verify NEURALMCS on four real graph datasets . and show that NEURALMCS can achieve 48.1× runtime gain over state-of-the-art exact MCS computation algorithm MCSPLIT ( McCreesh et al. , 2017 ) , and is much more accurate than all the baseline approximate approaches to graph matching . 2 PROBLEM DEFINITION AND PRELIMINARIES . 2.1 PROBLEM DEFINITION . We denote a graph as G = ( V , E ) with node set V and edge setE . We define as an induced subgraph as G′ = ( V ′ , E′ ) where V ′ ⊆ V and E′ ⊆ E and E′ preserves all edges between nodes in V ′ in the original graph G. In other words , for two nodes in V ′ , if there is an edge between them in G , the induced subgraph must also contain the edge . In the rest of the paper , we use the term “ subgraph ” to refer to “ induced subgraph ” . In this work , our goal is to detect the Maximum Common Subgraph ( MCS ) for an input graph pair ( G1 , G2 ) . In general , MCS refers to the largest ( induced ) subgraph that is common to both graphs . In this paper , we make the following qualifications : • Labeled graph . The nodes of G1 and G2 are labeled , and nodes of different labels can not be matched . • Connected subgraph . We require the detected subgraph to be connected , which is a com- mon qualification consistent with McCreesh et al . ( 2017 ) . Our goal is to detect the MCS for an input graph pair G1 , G2 . We adopt the state-of-the-art exact solver , MCSPLIT ( McCreesh et al. , 2017 ) , to provide ground-truth MCS results for a set of training graph pairs . Specifically , MCSPLIT provides not only the nodes that are included in the MCS in both graphs , but also the node-node correspondence in the MCS , as illustrated in Figure 1 . 2.2 NODE REPRESENTATION LEARNING . Among various node embedding methods , GRAPH MATCHING NETWORKS ( GMN ) ( Li et al. , 2019 ) are recent models designed for graph similarity computation . GMN is based on GRAPH CONVOLUTIONAL NETWORKS ( GCN ) ( Kipf & Welling , 2016 ) , but in addition to message passing within a graph ( intra-graph ) , GMN explicitly handles inter-graph information passing between the two input graphs . First , GMN performs explicit cross-graph communication on node embeddings denoted as h ( l ) where initial node features have l = 0 . More specifically , cross-graph communication is achieved through the following attention based mechanism : aj→i = exp ( cos ( h ( l ) i , h ( l ) j ) ∑ j ′ exp ( cos ( h ( l ) i , h ( l ) j ′ ) . Next , a softmax function is applied to the cosine similarity between two node embeddings which makes sure similar node pairs across graphs receive greater attention . This weight is then multiplied with the difference between cross-graph node embeddings to allow all node pairs in the two graphs to communicate with each other . Finally each node embedding resulted from inter-graph message passing is concatenated with embedding from intra-graph neighborhood aggregation and placed through a MLP or a GRU core to complete a single layer update of a hi . 3 THE PROPOSED APPROACH : NEURALMCS . Our proposed approach , NEURALMCS , relies on the learning capacity of the embedding model to generate a good matching matrix for each input graph pair , encoding the likelihood of each nodenode pair being included in the MCS and matched to each other . Therefore , the training process aims to learn a matching matrix for each graph pair that is as close to the ground-truth node-node correspondence ( as illustrated in Figure 1 ) as possible . However , we suppose that a good matching matrix by itself is not enough for an accurate prediction of the MCS , mainly due to the fact that MCS by definition requires the two extracted subgraphs must be isomorphic to each other and both subgraphs must also be connected ( see Section 2.1 ) . To satisfy these two requirements , we propose a novel GUIDED SUBGRAPH EXTRACTION ( GSE ) process that iteratively performs a guided search procedure to enlarge both extracted subgraphs using the matching matrix as guidance . The rest of the section details our proposed approach . 3.1 MATCHING MATRIX GENERATION . Our task fundamentally requires the matching between two graphs . Therefore , the ideal node embeddings should receive information from the nodes of both graphs . Thus , we adopt the state-of-theart node embedding model , GRAPH MATCHING NETWORKS ( GMN ) , as described in Section 2.2 . Specifically , we stack L GMN layers on the input node representations to allow inter-graph message passing at multiple scales for sufficient interaction of the two graphs . We denote the final node representations as U1 ∈ R|V1|×D ( L ) and U2 ∈ R|V2|×D ( L ) . To match nodes from the input graphs , we compute the likelihood of matching each node in G1 to each node in G2 . This likelihood indicates which node pair is most likely to be in the MCS , and should account for cases where some nodes in G1 do not match any nodes in G2 ( and vice-versa ) . We naturally encode these likelihoods into a matching matrix , Y ∈ [ 0 , 1 ] |G1|×|G2| . To compute Y , one can simply calculate the dot product between the node embeddings , U1U > 2 . However , this resulting matrix is simply the similarity score between each pair of nodes in the two graphs , which is in the range of ( − inf , + inf ) and requires further processing to reflect the probability of the node pair matched and being included in the MCS . As this matrix encode the general similarity between nodes , we denote it as the similarity matrix , X , and further normalize it to obtain Y . Specifically , we perform the following transformations . To find the likelihood of each node in one graph matching each node in the other graph , we apply column-wise and row-wise normalization on X . p̃coln ( i , j ) = eXij∑ k e Xik , p̃row ( i , j ) = eXij∑ k e Xkj ( 1 ) To allow for some nodes to go unmatched , we multiply these likelihoods by a value encoding the overall matching score of a node in one graph to nodes in the other graph . pcoln ( i , j ) = σ ( ∑ j Xij |V2| ) · p̃coln ( i , j ) , prow ( i , j ) = σ ( ∑ iXij |V1| ) · p̃row ( i , j ) ( 2 ) We consider both row-wise and column-wise normalization by taking the average of both scores , p̃ ( i , j ) = pcoln ( i , j ) +prow ( i , j ) 2 , and , to ensure we only select nodes with the same labels , we mask out node pairs with different labels to form Y . 3.2 LEARNING OF NEURALMCS . Although there exist multiple points where we can apply our loss function ( before , during , or after GSE ) , we found that applying binary cross entropy loss as early as possible allows for GMN to G1 G2 C C C C S C P C F O C C C S O F NO O O 0 0 1 1 2 2 3 3 4 4 5 56 6 7 7 8 8 9 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 G1 G2 C C C C S C P C F O C C C S O F NO O O 0 0 1 1 2 2 3 3 4 4 5 56 6 7 7 8 8 9 9 G1 G2 C C C C S C P C F O C C C S O F NO O O 0 0 1 2 2 3 3 4 4 5 56 6 7 7 8 8 9 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 G1 G2 C C C C S C P C F O C C C S O F NO O O 0 0 1 2 2 3 3 4 4 5 56 6 7 7 8 8 9 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 G1 G2 C C C C S C P C F O C C C S O F N O O O 0 0 1 2 2 3 3 4 4 5 56 6 7 7 8 8 9 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 G1 G2 C C C C S C P C F O C C C S O F NO O O 0 0 1 2 2 3 3 4 4 5 56 6 7 7 8 8 9 9 ( a ) Matching Matrix Generation ( b ) ( c ) ( d ) ( e ) ( f ) ( k ) ( j ) ( i ) ( h ) ( g ) ( l ) Stopping Condition Reached Figure 2 : A detailed illustration of NEURALMCS using the example graph pair in Figure 1 . For the input graph pair ( G1 , G2 ) , the NEURALMCS first generates a matching matrix encoding the likelihood of each pair of node being matched ( ( a ) and ( b ) ) . The GUIDED SUBGRAPH EXTRACTION ( GSE ) process uses the matching matrix as follows : NEURALMCS selects the initial pair to be included in the predicted MCS , node 2 in G1 and node 6 in G2 ( ( b ) and ( c ) ) . Next , NEURALMCS sets its search frontier as the neighbors of the selected nodes , circled in dashed blue lines . By selecting the pair with the largest matching score which still preserves subgraph isomorphism , node 3 in G1 and node 5 in G2 , the extracted subgraphs in both graphs grow to size 2 ( ( d ) and ( e ) ) . The procedure continues until a stopping condition is reached , which is detailed in Section 3.3. receive a better learning signal . For this reason , our loss function acts directly on the matching matrix . Specifically , we design the following loss function to train the model L ( Y ) = − ∑ i ∑ j Iij log ( Yij ) |V1|·|V2| where Yij denotes our predicted matching matrix and Iij denote an indicator value of whether the node i matches node j in the ground truth . Since for a given graph pair , there can be multiple correct MCSs of equal size , we also employ multiple choice learning ( Guzman-Rivera et al. , 2012 ; Li et al. , 2018 ) . This is done by ensembling various copies of the model and propagating the loss function only through the copy which achieves the lowest loss , as shown by the following : L = minY ∈Y ( L ( Y ) ) .

Summary: This paper proposed a graph net based approach for subgraph matching. The general idea is based on the graph matching network (Li et.al, ICML 2019) that computes the node embeddings of two graphs with co-attentions. The training requires the supervision of ground truth matching. During inference an iterative method with heuristic stopping criteria is used. Experiments on tiny graphs show better results than learning based baselines, but worse results than MCS solver.