Learning to Search for Fast Maximum Common Subgraph Detection

1 INTRODUCTION . Due to the flexible and expressive nature of graphs , designing machine learning approaches to solve graph tasks is gaining increasing attention from researchers . Among various graph tasks detecting the largest subgraph that is commonly present in both input graphs , known as Maximum Common Subgraph ( MCS ) ( Bunke & Shearer , 1998 ) ( as shown in Figure 1 ) , is an important yet particularly hard task . MCS naturally encodes the degree of similarity between two graphs , is domain-agnostic , and thus has occurred in many domains such as software analysis ( Park et al. , 2013 ) , graph database systems ( Yan et al. , 2005 ) and cloud computing platforms ( Cao et al. , 2011 ) . In drug design , the manual testing of the effects of a new drug is known to be a major bottleneck , and the identification of compounds that share common or similar subgraphs which tend to have similar properties can effectively reduce the manual labor ( Ehrlich & Rarey , 2011 ) . MCS detection is NP-hard in its nature and is thus a very challenging task . On one hand , the state-ofthe-art exact MCS detection algorithms based on branch and bound run in exponential time in worst cases ( Liu et al. , 2019 ) . What is worse , they rely on several heuristics on how to explore the search space . For example , MCSP ( McCreesh et al. , 2017 ) uses node degree as its heuristic by choosing high-degree nodes to visit first , but in many cases the true MCS contains small-degree nodes . On the other hand , existing machine learning approaches to graph matching such as Wang et al . ( 2019 ) and Bai et al . ( 2020b ) either do not address the MCS detection task directly or rely on labeled data requiring the pre-computation of MCS results by running exact solvers . In this paper , we present GLSEARCH ( Graph Learning to Search ) , a general framework for MCS detection combining the advantages of search and reinforcement learning . GLSEARCH learns to search by adopting a Deep Q-Network ( DQN ) ( Mnih et al. , 2015 ) to replace the node selection heuristics required by state-of-the-art MCS solvers , leading to faster arrival of the optimal solution for an input graph pair , which is particularly useful when the simpler heuristics fail and graphs are large with a limited search budget . Thanks to the learning capacity of Graph Neural Networks ( GNN ) , our DQN is specially designed for the MCS detection task with a novel reformulation of DQN to better capture the effect of different node selections . Given the large action space incurred by large graph pairs , to enhance the training of DQN , we leverage the search algorithm to not only provide supervised signals in a pre-training stage but also offer guidance during an imitation learning stage . Experiments on real graph datasets that are significantly larger than exisitng datasets adopted by state-of-the-art MCS solvers demonstrate that GLSEARCH outperforms baseline solvers and machine learning models for graph matching in terms of effectiveness by a large margin . Our contributions can be summarized as follows : • We address the challenging yet important task of Maximum Common Subgraph detection for general-domain input graph pairs and propose GLSEARCH as the solution . • The key novelty is the DQN which learns to search . Specifically , it is trained under the reinforcement learning framework to make the best decision at each search step in order to quickly find the best MCS solution during search . The search in turns helps training of DQN in a pre-training stage and an imitation learning stage . • We conduct extensive experiments on medium-size synthetic graphs and very large real- world graphs to demonstrate the effectiveness of the proposed approach compared against a series of string baselines in MCS detection and graph matching . 2 PRELIMINARIES . 2.1 PROBLEM DEFINITION . We denote a graph as G = ( V , E ) where V and E denote the vertex and edge set . An induced subgraph is defined as Gs = ( Vs , Es ) where Es preserves all the edges between nodes in Vs , i.e . ∀i , j ∈ Vs , ( i , j ) ∈ Es if and only if ( i , j ) ∈ E . In this paper , we aim at detecting the Maximum Common induced Subgraph ( MCS ) between an input graph pair , denoted as MCS ( G1 , G2 ) , which is the largest induced subgraph that is contained in both G1 and G2 . In addition , we require MCS ( G1 , G2 ) to be a connected subgraph . We allow the nodes of input graphs to be labeled , in which case the labels of nodes in the MCS must match as in Figure 1 . Graph isomorphism and subgraph isomorphism can be regarded as two special tasks of MCS : |MCS ( G1 , G2 ) | = |V1| = |V2| if G1 are isomorphic to G2 , |MCS ( G1 , G2 ) | = min ( |V1| , |V2| ) when G1 ( or G2 ) is subgraph isomorphic to G2 ( or G1 ) . 2.2 SEARCH ALGORITHM FOR MCS . Among various algorithms for MCS , we adopt the state-of-the-art search-based algorithm in our framework . The basic version , MCSP , is presented in McCreesh et al . ( 2017 ) and the more advanced version , MCSP+RL , is proposed in Liu et al . ( 2019 ) . The whole search algorithm , outlined in Algorithm 11 , is a branch-and-bound algorithm that maintains a best solution found so far throughout the search , which is initialized as empty subgraphs . In each search iteration , denote the current search state as st consisting of G1 , G2 , the current selected subgraphs G1s = ( V1s , E1s ) and G2s = ( V2s , E2s ) as well as their node-node mappings . The algorithm tries to select one node pair , ( vi , vj ) , where vi is from G1 and vj is from G2 , as its action , denoted as at , and either backtracks to the parent search state if the solution is not promising or continues the search otherwise . Various heuristics on node pair selection policy , denoted as “ policy ” , are proposed in MCSP and MCSP+RL . For example , in MCSP , nodes of large degrees are selected before small-degree nodes . Algorithm 1 Branch and Bound for MCS 1 : Input : Input graph pair G1 , G2 . 2 : Output : maxSol . 3 : Initialize stack← new Stack ( ) . 4 : Initialize maxSol← empty solution . 5 : stack.push ( s0 ) ; 6 : while stack 6= Ø do 7 : st ← stack.pop ( ) ; 8 : curSol← st.getCurSol ( ) ; 9 : if |curSol| > |maxSol| then 10 : maxSol← curSol ; 11 : end if 12 : UB t← |curSol| + overestimate ( st ) ; 13 : if UB t ≤ |maxSol| then 14 : continue ; 15 : end if 16 : At ← st.actions ; 17 : at ← policy ( st , At ) ; 18 : st.actions← st.actions \ { at } ; 19 : stack.push ( st ) ; 20 : st+1 ← env.update ( st , At ) ; 21 : stack.push ( st+1 ) ; 22 : end while There are two major limitations of MCSP and MCSP+RL : ( 1 ) Such heuristics-based node pair selection policy can not adapt to different graph structures2 ; ( 2 ) The search may enter a bad state and get “ stuck ” without finding a better ( larger ) solution , maxSol , for many iterations . At each search state , in order to compute the upper bound “ UB t ” and reduce the action space “ At ” , i.e . the candidate node pairs to select from , the concept of “ bidomain ” is introduced . Bidomains partition the nodes in the remaining subgraphs , i.e . outside G1s and G2s , into equivalent classes . Among all bidomains of a given state , D , the k-th bidomain Dk consists of two sets of nodes , 〈V ′k1 , V ′k2〉 where V ′k1 and V ′k2 have the same connectivity pattern with respect to the already matched nodes V1s and V2s . Figure 1 shows an example with three bidomains . Due to the subgraph isomorphism constraint posed by MCS , only nodes in V ′k1 can match to V ′k2 and vice versa . This also guarantees the extracted subgraphs at each state are isomorphic to each other . Thus , each bidomain can contribute at most min ( |V ′k1| , |V ′k2| ) nodes to the future best solution . Therefore , the upper bound can be estimated as ∑ Dk∈Dmin ( |V ′ k1| , |V ′k2| ) . This upper bound computation is consistently used for all the methods in the paper . The major difference is in the policy for node pair selection . 3 PROPOSED METHOD . In this section we formulate the problem of MCS detection as learning an RL agent that iteratively grows the extracted subgraphs by adding new node pairs to the current subgraphs in a graphstructure-aware environment . We first describe the environment setup , then depict our proposed Deep Q-Network ( DQN ) which provides actions for our agent to grow the subgraphs in a search context . We also describe how to leverage supervised data via pre-training and imitation learning . 3.1 LEVERAGING DQN FOR SEARCH . Since graph matching for MCS detection must satisfy a hard constraint that the resulting two subgraphs must be isomorphic to each other , instead of learning to match two graphs in one shot , we design an RL agent which explores the input graph pair and sequentially grows the extracted two subgraphs one node pair at a time . The iterative subgraph extraction process can be described by a 1The original algorithm is recursive . To highlight our novelty , we rewrite into an equivalent iterative version . 2MCSP+RL claims it uses reinforcement learning ( RL ) to compute a score for each node pair . However , the scores for each node are the only learnable parameters , whose update relies on a heuristic . This requires the learning of scores for each new testing graph pair . In contrast , we use continuous embeddings to represent graphs which are fed into a DQN to compute a score trained on many graph pairs . Once trained , GLSEARCH can be applied to any new testing pair without retraining . Markov Decision Process ( MDP ) , where the definitions of state and action are the same as Section 2.2 . The difference is that , for MDP , reward needs to be defined too . for MCS , the immediate reward for transitioning from one state to any next state is rt = +1 since one new node pair is selected . To address the issue that the algorithm may get stuck in a bad state for many iterations without finding a larger solution , we utilize additional information stored in Q-values computed by our learned model . We suppose backtracking to an earlier better state can alleviate such issue in practice , but there lacks a principled measure for MCSP and MCSP+RL to determine which earlier state is better . By design , our node pair selection policy is a learned DQN , so our agent knows not only the quality of immediate actions , but also the values associated with previous states . Therefore , if the best solution found so far does not increase , i.e . we do not enter line 10 of Algorithm 1 for a pre-defined number of iterations , in the next iteration , we find the best state as determined by the Bellman Equation , remove that state , then visit it on line 7 . We refer to this improved search methodology as promise-based search . More details can be found in the supplementary material .

Summary: Given two input graphs G1,G2 the maximum common subgraph detection problem asks to find an induced subgraph of both G1 and G2, with as many vertices as possible. In the recent years, there have been papers that introduce different heuristics for guiding the search of this subgraph within branch & bound algorithms. The main contribution of this paper is a combination of graph neural network embeddings and RL to guide the search more efficiently.  The function used to guide the deep Q-network is given in Equation (3). The paper performs a set of experiments on synthetic and real world pairs of graphs, where it is shown that it performs well in practice. The supplementary material provides more details on the experiments.