L EARNING TO S EARCH FOR F AST M AXIMUM C OMMON S UBGRAPH D ETECTION
Anonymous authors
Paper under double-blind review
A BSTRACT
Detecting the Maximum Common Subgraph (MCS) between two input graphs is fundamental for applications in biomedical analysis, malware detection, cloud computing, etc.
This is especially important in the task of drug design, where the successful extraction of common substructures in compounds can reduce the number of experiments needed to be conducted by humans.
However, MCS computation is NP-hard, and state-of-the-art MCS solvers rely on heuristics in search which in practice cannot ﬁnd good solution for large graph pairs under a limited search budget.
Here we propose GLS EARCH , a Graph Neural Network based model for MCS detection, which learns to search.
Our model uses a state-ofthe-art branch and bound algorithm as the backbone search algorithm to extract subgraphs by selecting one node pair at a time.
In order to make better node selection decision at each step, we replace the node selection heuristics with a novel task-speciﬁc Deep Q-Network (DQN), allowing the search process to ﬁnd larger common subgraphs faster.
To enhance the training of DQN, we leverage the search process to provide supervision in a pre-training stage and guide our agent during an imitation learning stage.
Therefore, our framework allows search and reinforcement learning to mutually beneﬁt each other.
Experiments on synthetic and real-world large graph pairs demonstrate that our model outperforms state-ofthe-art MCS solvers and neural graph matching network models.
I NTRODUCTION
Due to the ﬂexible and expressive nature of graphs, designing machine learning approaches to solve graph tasks is gaining increasing attention from researchers.
Among various graph tasks detecting the largest subgraph that is commonly present in both input graphs, known as Maximum Common
Subgraph (MCS) (Bunke & Shearer, 1998) (as shown in Figure 1), is an important yet particularly hard task.
MCS naturally encodes the degree of similarity between two graphs, is domain-agnostic, and thus has occurred in many domains such as software analysis (Park et al., 2013), graph database systems (Yan et al., 2005) and
cloud computing platforms (Cao et al., 2011).
In drug design, the manual testing of the effects of a new drug is known to be a major bottleneck, and the identiﬁcation of compounds that share common or similar subgraphs which tend to have similar properties can effectively reduce the manual labor (Ehrlich & Rarey, 2011).
MCS detection is NP-hard in its nature and is thus a very challenging task.
On one hand, the state-ofthe-art exact MCS detection algorithms based on branch and bound run in exponential time in worst cases (Liu et al., 2019).
What is worse, they rely on several heuristics on how to explore the search space.
For example, M C S P (McCreesh et al., 2017) uses node degree as its heuristic by choosing high-degree nodes to visit ﬁrst, but in many cases the true MCS contains small-degree nodes.
On the other hand, existing machine learning approaches to graph matching such as Wang et al.
and Bai et al.
(2020b) either do not address the MCS detection task directly or rely on labeled data requiring the pre-computation of MCS results by running exact solvers.
In this paper, we present GLS EARCH ( G raph L earning to S earch), a general framework for MCS detection combining the advantages of search and reinforcement learning.
GLS EARCH learns to search by adopting a Deep Q-Network (DQN) (Mnih et al., 2015) to replace the node selection heuristics required by state-of-the-art MCS solvers, leading to faster arrival of the optimal solution for an input graph pair, which is particularly useful when the simpler heuristics fail and graphs are large
G G
C C
C C
C
O
P
C C
C C
C C
N
S
C
G G
C C
C C
C
O
P
C C
C C
C C
N
S
C            with a limited search budget.
Thanks to the learning capacity of Graph Neural Networks (GNN), our DQN is specially designed for the MCS detection task with a novel reformulation of DQN to better capture the effect of different node selections.
Given the large action space incurred by large graph pairs, to enhance the training of DQN, we leverage the search algorithm to not only provide supervised signals in a pre-training stage but also offer guidance during an imitation learning stage.
Experiments on real graph datasets that are signiﬁcantly larger than exisitng datasets adopted by state-of-the-art MCS solvers demonstrate that GLS EARCH outperforms baseline solvers and machine learning models for graph matching in terms of effectiveness by a large margin.
Our contributions can be summarized as follows:
• We address the challenging yet important task of Maximum Common Subgraph detection for general-domain input graph pairs and propose GLS EARCH as the solution.
• The key novelty is the DQN which learns to search.
Speciﬁcally, it is trained under the reinforcement learning framework to make the best decision at each search step in order to quickly ﬁnd the best MCS solution during search.
The search in turns helps training of DQN in a pre-training stage and an imitation learning stage.
• We conduct extensive experiments on medium-size synthetic graphs and very large realworld graphs to demonstrate the effectiveness of the proposed approach compared against a series of string baselines in MCS detection and graph matching.
P RELIMINARIES 2. P ROBLEM D EFINITION
We denote a graph as G =
( V , E ) where V and E denote the vertex and edge set.
An induced subgraph is deﬁned as G s =
( V s , E s ) where E s preserves all the edges between nodes in V s , i.e. ∀ i, j ∈ V s , ( i, j
) ∈
E s if and only if ( i, j ) ∈ E .
In this paper, we aim at detecting the Maximum Common induced Subgraph (MCS) between an input graph pair, denoted as MCS( G 1 , G 2 ) , which is the largest induced subgraph that is contained in both G 1 and G 2 .
In addition, we require MCS( G 1 , G 2 ) to be a connected subgraph.
We allow the nodes of input graphs to be labeled, in which case the labels of nodes in the MCS must match as in Figure 1.
Graph isomorphism and subgraph isomorphism can be regarded as two special tasks of MCS: | MCS( G 1 , G 2 )
| =
|V 1 | =
|V 2 | if G 1 are isomorphic to G 2 , | MCS( G 1 , G 2 ) | = min ( |V 1 | , |V 2 | ) when G 1 (or G 2 ) is subgraph isomorphic to G 2 (or G 1 ).
2. S EARCH A LGORITHM FOR MCS
Among various algorithms for MCS, we adopt the state-of-the-art search-based algorithm in our framework.
The basic version, M C S P , is presented in McCreesh et al.
(2017) and the more advanced version, M C S P +RL, is proposed in Liu et al.
The whole search algorithm, outlined in
Algorithm 1 1 , is a branch-and-bound algorithm that maintains a best solution found so far throughout the search, which is initialized as empty subgraphs.
In each search iteration, denote the current search state as s t consisting of G 1 , G 2 , the current selected subgraphs G 1 s =
( V 1 s , E 1 s ) and G 2 s =
( V 2 s , E 2 s ) as well as their node-node mappings.
The algorithm tries to select one node pair, ( v i , v j ) , where v i is from G 1 and v j is from G 2 , as its action, denoted as a t , and either backtracks to the parent search state if the solution is not promising or continues the search otherwise.
Various heuristics on node pair selection policy, denoted as “ policy ”, are proposed in M C S P and M C S P +RL.
For example, in
M C S P , nodes of large degrees are selected before small-degree nodes.
do 7: ; 11: end 14: continue ; 15: end t +1 ) ; 22: end while
There are two major limitations of M C S P and
M C S P +RL:
(1) Such heuristics-based node pair selection policy cannot adapt to different graph structures 2 ; (2) The search may enter a bad state and get “stuck” without ﬁnding a better (larger) solution, maxSol , for many iterations.
At each search state, in order to compute the upper bound “ UB t ” and reduce the action space “ A t ”, i.e. the candidate node pairs to select from, the concept of “bidomain” is introduced.
Bidomains partition the nodes in the remaining subgraphs, i.e. outside G 1 s and G 2 s , into equivalent classes.
Among all bidomains of a given state,
D , the k -th bidomain D k consists of two sets of nodes, (cid:104)V (cid:48)
k 1 , V (cid:48)
k 2 (cid:105) where V (cid:48)
k 1 and V (cid:48)
k 2 have the same connectivity pattern with respect to the already matched nodes V 1 s and V 2 s .
Figureshows an example with three bidomains.
Due to the subgraph isomorphism constraint posed by
MCS, only nodes in V (cid:48) k 1 can match to V (cid:48)
k 2 and vice versa.
This also guarantees the extracted subgraphs at each state are isomorphic to each other.
Thus, each bidomain can contribute at most min( |V (cid:48)
k 1 | , |V (cid:48)
k 2 | ) nodes to the future best solution.
Therefore, the upper bound can be estimated as (cid:80) D k ∈D min( |V (cid:48)
k 1 | , |V (cid:48)
k 2 | ) .
This upper bound computation is consistently used for all the methods in the paper.
The major difference is in the policy for node pair selection.
P ROPOSED M ETHOD
In this section we formulate the problem of MCS detection as learning an RL agent that iteratively grows the extracted subgraphs by adding new node pairs to the current subgraphs in a graphstructure-aware environment.
We ﬁrst describe the environment setup, then depict our proposed Deep
Q-Network (DQN) which provides actions for our agent to grow the subgraphs in a search context.
We also describe how to leverage supervised data via pre-training and imitation learning.
3. L EVERAGING DQN FOR S EARCH
Since graph matching for MCS detection must satisfy a hard constraint that the resulting two subgraphs must be isomorphic to each other, instead of learning to match two graphs in one shot, we design an RL agent which explores the input graph pair and sequentially grows the extracted two subgraphs one node pair at a time.
The iterative subgraph extraction process can be described by a
[Figure]
MCS
[Figure]
Markov Decision Process (MDP), where the deﬁnitions of state and action are the same as Section 2.2.
The difference is that, for MDP, reward needs to be deﬁned too.
for MCS, the immediate reward for transitioning from one state to any next state is r t =
+1 since one new node pair is selected.
To address the issue that the algorithm may get stuck in a bad state for many iterations without ﬁnding a larger solution, we utilize additional information stored in Q-values computed by our learned model.
We suppose backtracking to an earlier better state can alleviate such issue in practice, but there lacks a principled measure for M C S P and M C S P +RL to determine which earlier state is better.
By design, our node pair selection policy is a learned DQN, so our agent knows not only the quality of immediate actions, but also the values associated with previous states.
Therefore, if the best solution found so far does not increase, i.e. we do not enter line 10 of Algorithm 1 for a pre-deﬁned number of iterations, in the next iteration, we ﬁnd the best state as determined by the Bellman Equation, remove that state, then visit it on line 7.
We refer to this improved search methodology as promise-based search .
More details can be found in the supplementary material.
3. R EPRESENTATION L EARNING FOR DQN
Since the action space can be large for MCS, we leverage the representation learning capacity of continuous representations for DQN design.
At state s t , for each action a t , our DQN predicts a
Q ( s t , a t ) representing the future reward to go if the action a t =
( i, j ) where i ∈ V 1 and j ∈ V 2 is selected, intuitively corresponding to the largest number of nodes that will be eventually selected starting from the action edge ( s t , a t ) as in tree in Figure 2.
Based on the above insights, one can design a simple DQN leveraging the representation learning power of Graph Neural Networks (GNN) such as Kipf & Welling (2016) and Velickovic et al.
by passing G 1 and G 2 to a GNN to obtain one embedding per node, { h i |∀ i ∈ V 1 } and { h j |∀ j ∈ V 2 } .
Denote CONCAT as concatenation, READOUT as a readout operation that aggregates node-level embeddings into subgraph embeddings h s 1 and h s 1 , and whole-graph embeddings h G 1 and h G 2 .
A state can then be represented as h s t =
CONCAT ( h G 1 , h G 2 , h s 1 , h s 2 ) .
An action can be represented as h a t =
CONCAT ( h i , h j ) .
The Q function can the be designed as:
[Equation]
However, there are several ﬂaws to this simple design of Q function:
(A) h i and h j generated by typical GNNs encode only local neighborhood information, but Q ( s t , a t ) represents the long-term effect of adding ( i, j ) .
What is worse, different node pairs have different embeddings, but their immediate rewards are always +1 in MCS.
(B) Swapping the order of G 1 and G 2 should not cause Q ( s t , a t ) to change, but concatenating embeddings from the two graphs causes the DQN to be sensitive to their ordering.
(C) How to effectively leverage the node-node mappings between G 1 s and G 2 s for predicting Q ( s t , a t ) remains a challenge.
To address these issues, we propose the following improvements over the simple DQN design.
Factoring out Action In order to maximally reﬂect the effect of adding node pair ( i, j ) to G 1 s and
G 2 s ,
we ﬁrst notice that Q ∗ ( s t , a t ) =
r t + γV ∗ ( s t +1 )
= 1 + γV ∗ ( s t +1 ) in MCS, where Q and V are the Q and value functions, respectively, and γ is the discount factor.
Then, in order to compute the effect of a t , we can compute the value associated with s t +1 which does not depend on a t and avoids the use of local h i and h j .
Interaction between Input Graphs To resolve the graph symmetry issue, we ﬁrst construct the interaction between the embeddings from two graphs, i.e. INTERACT ( h x 1 , h x 2 ) , where h x 1 and h xrepresent any embedding from G 1 and G 2 respectively and INTERACT ( · ) is any commutative function to combine the two embeddings (e.g. summation).
This interacted embedding is later concatenated with other useful representations and fed into a ﬁnal MLP to compute the Q score. We describe our INTERACT ( · ) operator in Section B.3.
Bidomain Representations Bidomains are derived from node-node mappings and partition the rest of G 1 and G 2 , which is a more useful signal for predicting the future reward.
In fact, as described in
Section 2.2, bidomains have been adopted to in search-based MCS solvers to estimate the upper bound.
Here, we require the harder prediction of Q ( s t , a t ) for which we propose to use the representation of bidomains.
Denote h D k as the representation for bidomain D k .
Similar to computing the graph-level and subgraph-level embeddings, we compute D k as
[Equation]
Since we require the MCS to be connected subgraphs, we differentiate bidomains D ( c ) that are connected (adjacent) to G 1 s and G 2 s from the single bidomain D 0 disconnected (unconnected) from
G 1 s and G 2 s (e.g. bidomain “00” in Figure 1).
Given all the bidomain embeddings, we compute a single representation for D ( c ) , h D c = READOUT ( { h D k | k ∈ D
[Equation] c ) } ) .
Our ﬁnal DQN has the form:
[Equation]
3. L EVERAGING S EARCH FOR DQN T RAINING
At each state s t , the action space size in the worst case is quadratic to the number of nodes in the remaining subgraphs.
Thus, to enhance the training of our DQN, before the standard training of
DQN (Mnih et al., 2013), we pre-train DQN and guide its exploration with expert trajectories supplied by the search algorithm.
For the pre-training stage, we ﬁrst observe the overall mse loss is ( y t − Q ( s t , a t )) 2 where y t the target for iteration t and Q ( s t , a t ) is the predicted Q ( s t , a t ) .
We then notice that for small training graph pairs, the complete exploration of search space can be performed to obtain the true target for every ( s t , a t ) by ﬁnding the longest sequence starting from s t to a leaf node in the search tree.
For larger graph pairs though, ﬁnding the true target becomes too slow.
In that case, after pre-training, we enter the imitation learning stage where we let the agent mimic the decision made by the state-ofthe-art MCS search algorithm instead of relying on its own predicted Q ( s t , a t ) .
More details can be in found in the supplementary material.
E XPERIMENTS
We evaluate GLS EARCH against two state-of-the-art exact MCS detection algorithms and a series of approximate graph matching methods from various domains.
We conduct experiments on a variety of medium-sized synthetic graph datasets and real-world graph datasets, whose details can be found in the supplementary material.
Among the different baseline models, we ﬁnd no consistent trend.
This indicates the difﬁculty of our task, as existing methods can not ﬁnd a consistent policy that guarantees good performance on datasets from different domains.
Our model can substantially outperform the baselines, highlighting the signiﬁcance of our contributions to learning for search.
[Table]
4. B ASELINE M ETHODS
There are two groups of methods: Exact MCS algorithms including M C S P (McCreesh et al., 2017) and M C S P +RL (Liu et al., 2019), learning based graph matching models including GW - QAP (Xu et al., 2019a), I- PCA (Wang et al., 2019), and N EURAL M CS (Bai et al., 2020b).
All the methods either originally use or are adapted to use the branch and bound search framework in
Section 2.2 with differences in node pair selection policy and training strategies.
GW - QAP performs
Gromov-Wasserstein discrepancy (Peyr´e et al., 2016) based optimization for each graph pair and outputs a matching matrix Y for all node pairs indicating the likelihood of matching which is treated the same way as our q scores, i.e. at each search iteration we index into Y to select a node pair.
I- PCA and N EURAL M CS also output a matching matrix but require supervised training, and thus are trained using the same training data graph pairs as our GLS EARCH but with different loss functions and training signals.
More details on training and setup of baselines can be found in the supplementary material.
During testing, we apply the trained model on all testing graph pairs.
For medium-size synthetic testing graph pairs, each method is given a budget of 500 search iterations.
For large real-world graph pairs, each method is given a budget of 7500 search iterations.
These budgets were chosen based on when the models’ performances stabilized.
Details about performance using other iteration budgets may be found in the Supplementary Material.
To validate the usefulness of the learned DQN, we compare GLS EARCH , our full model, with a randomly initialized model, GLS EARCH -R AND , which replaces the output of our DQN with a completely random scalar.
We show the performance gain of our model through training by substantially outperforming this baseline on all real-world datasets.
4. P ARAMETER S ETTINGS
For I- PCA , N EURAL M CS and GLS EARCH , we utilize 3 layers of Graph Attention Networks (GAT) (Velickovic et al., 2018)
each with 64 dimensions for the embeddings.
The initial node embedding is encoded using the local degree proﬁle (Cai & Wang, 2018).
We use ELU( x ) =
α (exp( x )–1) for x ≤ 0 and x for x > 0 as our activation function where α = 1 .
We run all experiments with Intel i7-6800K CPU and one Nvidia Titan GPU.
For DQN, we use MLP layers to project concatenated embeddings to a scalar.
We use SUM followed by an MLP for READOUT and 1 DCONV + MAXPOOL followed by an MLP for INTERACT .
For training, we set the learning rate to 0.001, the number of training iterations to 10000, and use the Adam optimizer (Kingma & Ba, 2015).
The models were implemented with the PyTorch and PyTorch Geometric libraries (Fey & Lenssen, 2019).
4. R ESULTS
The key property of GLS EARCH is its ability to ﬁnd the best solution in the fewest number of search iterations.
As shown in Table 1, our model outperforms baselines in terms of size of extracted subgraphs on all medium-sized synthetic graph datasets and the small chemical compound dataset
[Table]
N CI 109.
However, baseline solvers are already quite powerful on these datasets.
As it is easy to extract the maximum common subgraph on smaller graph datasets because the total search space grows exponentially with graph size, to truly show the performance advantage of GLS EARCH , we also run experiments on large real-world graph datasets with thousands of nodes.
As shown in Table 2, our model outperforms baselines in terms of the size of the extracted subgraphs on all large real-world datasets.
The exact solvers rely on heuristics for node selection, and consistently ﬁnd smaller subgraphs compared to our results.
Figure 3 compares results by M C S P and GLS EARCH .
Since M C S P selects nodes with large degrees as its heuristic, the selected nodes tend to be conﬁned in one dense cluster of large degree nodes in G 1 .
This implies the subgraph in G 2 matched to this dense cluster must also be dense (isomorphism constraint of MCS).
In contrast, GLS EARCH is able to ﬁnd long chains in G 1 which allows easier matching in G 2 .
In general, there are many cases of large real-world graph pairs where heuristics are not enough to extract large high quality subgraphs.
Due to its leveraging both learning and search, GLS EARCH consistently ﬁnds subgraphs more than double the size of those found by search based baselines for large real-world graph pairs.
[Figure]
Compared with learning based graph matching models, GLS EARCH is the only model which learns a reward that is dependent on both state and action, i.e. Q ( s t , a t ) .
GW - QAP , I- PCA , and N EURAL M CS essentially pre-compute the matching scores for all the node pairs in the input graphs, and therefore at each search step, the scores cannot adapt to the particular state, i.e. the matching scores only depend on G 1 , G 2 .
Notice our state representation includes G 1 , G 2 as well, hence GLS EARCH has more representational power than baselines.
Trained under a reinforcement learning framework guided by search, GLS EARCH also performs the best among learning based baselines. [Table]
4. A BLATION AND P ARAMETER S TUDY
To evaluate the effectiveness of different components proposed in our DQN model [Table] we run ablation studies on all real world datasets.
We ﬁrst measure the importance of each embedding vector fed to our DQN module, as described by Equation 3.
We remove each embedding vector (speciﬁcally: h G = INTERACT ( h G 1 , h G 2 ) ,
h s = INTERACT ( h s 1 , h s 2 ) , h D c , and h D 0 ) individually from the DQN model and retrain the model under the same training settings.
Table 3 is consistent with our conclusion that every embedding vector used by GLS EARCH is critical in capturing the search state’s representation.
Furthermore, we ﬁnd leveraging bidomain representations is very beneﬁcial to our model.
We next measure the importance of interaction to address the symmetry issue of the MCS calculation, where input graph pairs must be order insensitive.
We ﬁrst test the necessity of using more complex interaction functions, by replacing our 1 DCONV + MAXPOOL interaction with simple SUM for interaction (still followed by an MLP).
As shown in Table 3, we see that simpler interaction functions may not be powerful enough to encode the interaction between 2 graphs.
Particularly, this suggests that interaction is quite important to model performance.
Finally, we measure the importance of factoring out actions from our DQN model.
We test this with 2 models.
The ﬁrst utilizes Equation 1 to encode the Q-value, which we refer to as GLS EARCH (unfactored).
Since Equation 1 also suffers from the issue of graph symmetry, we adapt this model to use the same interaction function as GLS EARCH to construct 3 order-invariant embeddings h G = INTERACT ( h G 1 , h G 2 ) , h s =
INTERACT ( h s 1 , h s 2 ) , h a = INTERACT ( h i , h j ) to concatenate and pass to the ﬁnal MLP layer in Equation 1.
We refer to this model as GLS EARCH (unfactored-i).
Our results show that without factoring out the action, our performance is comparable to or worse than M C S P , indicating the signiﬁcant performance boost introduced by maximally reﬂecting the effect of adding node pairs. 4. O VERHEAD OF GLS EARCH
Although in each iteration, GLS EARCH needs to compute a Q score, and is thus more computationally expensive than M C S P and M C S P +RL, the decision GLS EARCH makes is “smarter” so that across many iterations, GLS EARCH can ﬁnd a larger solution. In order to verify this, for each method, in each search iteration, we collect the best solution (largest subgraph) found so far, and plot the best solution size across search iterations. Since we also measure the running time at each iteration, we also plot the best solution size across running time. As shown in Figure 4, across the search, initially GLS EARCH ﬁnds a smaller solution compared to some baseline methods, but, with more iterations and running time, GLS EARCH quickly outperforms more and more baselines, eventually becoming the best method.
Since all the methods follow the same search framework, and the only difference is the node pair selection policy, this experiment veriﬁes that our learned policy can indeed ﬁnd a larger common subgraph.
Given inﬁnite amount of time budget, all the methods would ﬁnd the true MCS which is the same or larger than the current best solution, but the aim of this section is to show the advantage of the learned policy for search allows the ﬁnding of a better solution in fewer iterations and running time.
Section E shows the plots on other datasets and provides additional experimental results.
[Figure]
R ELATED W ORK
MCS detection is HP-hard, with existing methods based on constraint programming (Vismara &
Valery, 2008; McCreesh et al., 2016), branch and bound (McCreesh et al., 2017; Liu et al., 2019), mathematical programming (Bahiense et al., 2012), conversion to maximum clique detection (Levi, 1973; McCreesh et al., 2016), etc.
Closed related to MCS detection is Graph Edit Distance (GED) computation (Bunke, 1983), which in the most general form refers to ﬁnding a series of edit operations that transform one graph to another and has also been adopted in many task where the matching or similarity between graphs is necessary.
There is a growing trend of using machine learning approaches to approximate graph matching and similarity score computation, but these works either do not address MCS detection speciﬁcally and must be adapted Zanﬁr & Sminchisescu (2018); Wang et al.
; Yu et al.
Xu et al.
(2019b;a); Bai et al.
(2019; 2020a)
; Li et al.
; Ling et al.
(2020), or rely on a large amount of labeled MCS instances Bai et al.
(2020b)
C ONCLUSION
We believe the interaction of search and learning is a promising direction for future research, and take a step towards bridging the gap by tackling the NP-hard challenging task, Maximum Common
Subgraph detection.
We have proposed a reinforcement learning method which uniﬁes search and deep Q-learning into a single framework.
By using the search to train our carefully designed DQN, the DQN provides better node selection policy for search to ﬁnd large common subgraph solutions faster, which is experimentally veriﬁed on real-world large graph pairs.
In future, the adaptation of our framework to other NP-hard tasks requiring search can be explored.
A D ATASET D ESCRIPTION
This section describes the datasets used for evaluating our model and baselines.
Section B.1 describes the dataset we use for training GLS EARCH as well as baseline learning based graph matching models.
We use the following real-world datasets for evaluation:
• N CI 109: It is a collection of small-sized chemical compounds (Wale et al., 2008) whose nodes are labeled indicating atom type.
We form 100 graph pairs from the dataset whose average graph size (number of nodes) is 28.73.
• R OAD : The graph is a road network of California whose nodes indicate intersections and endpoints and edges represent the roads connecting the intersections and endpoints (Leskovec et al., 2009).
The graph contains 1965206 nodes, from which we randomly sample a connected subgraph of around 0.05% nodes twice to generate two subgraphs for the graph pair G 1 =
( V 1 , E 1 ) and G 2 =
( V 2 , E 2 ) .
• D B E N , D B Z H , and D BPD : It is a dataset originally used in a work on cross-lingual entity alignment (Sun et al., 2017).
The dataset contains pairs of DBpedia knowledge graphs in different languages.
For D B E N , we use the English knowledge graph and sample 10% nodes twice to generate two graphs for our task.
For D B Z H , we sample around 10% nodes from the knowledge graph in Chinese.
For D BPD , we sample once from the English graph to get G 1 and sample once from the Chinese graph to get G 2 .
Note that although the nodes have features, we do not use them because our task is more about graph structural matching rather than node semantic meanings, and leave the incorporation of continuous node initial representations as future work.
• E NRO :
The graph is an email communication network whose nodes represent email addresses and (undirected) edges represent at least one email sent between the addresses (Klimt & Yang, 2004).
From the total 36692 nodes, we sample around 10% nodes to generate the graph pair.
• C O P R : An Amazon computer product network whose nodes represent goods and edges represent two goods frequently purchased together (Shchur et al., 2018).
The graph contains 703655 nodes from which we sample around 0.5% to get the pair we use.
• C IRC :
This is a graph pair where each graph is a circuit diagram whose nodes represent devices/wires and edges represent the connecting relations between devices and wires.
In other words, each node is either a device or a wire, and the entire graph is bipartite.
The two graphs given are known to be isomorphic and we do not perform any sampling.
Nodes have labels about the type of the device/wire.
In real world, the successful matching of circuit layout diagrams is an essential process in circuit design veriﬁcation.
• HP PI :
It is a human protein-protein interaction network whose nodes represent proteins and edges represent physical interaction between proteins in a human cell (Agrawal et al., 2018).
From the 21557 nodes, we sample around 10% nodes to generate the pair used in experiments.
The details of all the graph pairs can be found in Table 4.
[Table]
[Table] [Table]
[Table]
Email Communication Network
[Table]
[Table] [Table]
purchasing
[Table]
C IRC
Circuit Layout Diagram    1.
[Table]
[Table]
For synthetic datasets, we generate graph pairs using the Barab´asi-Albert (BA) (Barab´asi & Albert, 1999) algorithm (edge density set to 5), the Erd˝os-R´enyi (ER) (Gilbert, 1959)
algorithm
(edge density set to 0.08), and the Watts–Strogatz (WS) (Watts & Strogatz, 1998) algorithm (rewiring probability set to 0.2 and ring density set to 4), respectively.
B D ETAILS ON DQN AND T RAINING GLS EARCH B. T RAINING D ATA P REPARATION : C URRICULUM L EARNING
Curriculum learning (Bengio et al., 2009) is a strategy for training machine learning models whose core idea is to train a model ﬁrst using “easy” examples before moving on to using “hard” ones.
Our goal is to train a general model for MCS detection task which works well on general testing graph pairs from different domains.
Therefore, we employ the idea of curriculum learning in training our
GLS EARCH .
More speciﬁcally, we prepare the training graph pairs in the following way:
• Curriculum 1: The ﬁrst curriculum consists of the easiest graph pairs that are small: (1) We sample 30 graph pairs from A IDS (Zeng et al., 2009), a chemical compound dataset usually for graph similarity computation (Bai et al., 2019) where each graph has less than or equal to 10 nodes; (2) We sample 30 graph pairs from L INUX (Wang et al., 2012), another dataset commonly used for graph matching consisting of small program dependency graphs generated from Linux kernel; (3) So far we have 60 real-world graph pairs.
We then generate 60 graph pairs using popular graph generation algorithms.
Specifcally, we generate 20 graph pairs using the BA algorithm, 20 graph pairs using the ER algorithm, and 20 graph pairs using the Watts–Strogatz WS algorithm, respectively.
Details of the graphs can be found in Table 5.
In summary, the ﬁrst curriculum contains 120 graph pairs in total.
• Curriculum 2: After the ﬁrst curriculum, each next curriculum contains graphs that are larger and harder to match than the previous curriculum.
For the second curriculum, we samplegraph pairs from P TC (Shrivastava & Li, 2014), a collection of chemical compounds,graph paris from I MDB (Yanardag & Vishwanathan, 2015), a collection of ego-networks of movie actors/actresses, and generate 20 graph pairs again using the BA, ER, and WS algorithms but with larger graph sizes.
• Curriculum 3:
For the third curriculum, we sample 30 graph pairs from M UTAG (Debnath et al., 1991), a collection of chemical compounds, 30 graph paris from R EDDIT (Yanardag & Vishwanathan, 2015), a collection of ego-networks corresponding to online discussion threads, and generate 20 even larger graph pairs using the BA, ER, and WS algorithms.
• Curriculum 4:
For the last curriculum, we sample 30 graph pairs from W EB (Riesen & Bunke, 2008), a collection of text document graphs, 30 graph paris from MCSPLAINCONNECTED (McCreesh et al., 2017), a collection of synthetic graph pairs adopted by M C S P , and generate 20 graph pairs again using BA, ER, and WS algorithms but with larger graph sizes.
For each curriculum, we train the model for 2500 iterations before moving on to the next, resulting in 10000 training iterations in total.
B. T RAINING T ECHNIQUES AND D ETAILS B.2. S TAGE 1: P RE - TRAINING
For the ﬁrst 1250 iterations, we pre-train our DQN with the supervised true target y t obtained as follows:
• For each graph pair, we run the complete search,
i
e. we do not perform any pruning for unpromising states.
The entire search space is explored, and the future reward for every action can be found by ﬁnding the longest path starting from the action to a terminal state.
Since graphs are small in the initial stage, such complete search can be affordable.
Using Figure 2 in the main text as an example, for the action that causes state 0 to transition to state 6, the longest path is 0, 6, 7, 11, 12, 13 (or 0, 6, 7, 11, 12, 14).
[Table]
• Given the longest path found for each action, we then compute y t
= 1+ γ + γ
+ ...
+ γ ( L − 1) , where γ is the discount factor set to 1.0, L is the length of the longest path.
In the example above, y t = 5 , intuitively meaning that at state 0, for the action that leads to state 6, in future the best solution will have 5 more nodes.
In contrast, the action 0 to 1 has y t = 4 , meaning the action 0 to 6 is more preferred.
• Given the true target computed for each action, we run the mini-batch gradient descents over the mse loss ( y t − Q ( s t , a t )) 2 , where the batch size (number of sampled actions) is set to 32.
B.2. S TAGE 2: I MITATION L EARNING AND S TAGEFor
stage 2 (2500 iterations) and stage 3 (6250 iterations), we train the DQN using the framework proposed in Mnih et al. (2013).
The difference is that in stage 2, instead of allowing the model to use its own predicted Q ( s t , a t ) at each state, we let the model make a decision using the heuristics by the
M C S P algorithm, which serves as an expert providing trajectories in stage 2.
We aim to outperform
M C S P eventually after training using our own predicted Q ( s t , a t ) in stage 3.
Here we describe the procedure of the training process.
In each training iteration, we sample a graph pair from the current curriculum for which we run the DQN multiple times until a terminal state is reached to collect all the transitions, i.e. 4-tuples in the form of ( s t , a t , r t , s t +1 ) where r t is 1 and y t
= 1 + γ max a (cid:48) Q ( s t +1 , a (cid:48) ) , and store them into a global experience replay buffer, a queue that maintains the most recent L 4-tuples.
In our calculations, L = 1024 .
Afterwards, at the end of the iteration, the agent gets updated by performing the mini-batch gradient descents over the mse loss ( y t − Q ( s t , a t )) 2 , where the batch size (number of sampled transitions from the replay buffer) is set to 32.
To stabilize our training, we adopt a target network which is a copy of the DQN network and use it for computing max a (cid:48) γQ ( s t +1 , a (cid:48) ) .
This target network is synchronized with the DQN periodically, in every 100 iterations.
Since at the beginning of stage 3, the q approximation may still be unsatisfactory, and random behavior may be better, we adopt the epsilon-greedy method by switching between random policy and Q policy using a probability hyperparameter (cid:15) .
Thus, the decision is made as arg max q where q is the our predicted Q ( s t , a t ) for all possible actions ( i, j ) with 1 − (cid:15) probability; With (cid:15) probability, the decision is random.
This probability is tuned to decay slowly as the agent learns to play the game, eventually stabilizing at a ﬁxed probability.
We set the starting epsilon to 0.1 decaying to 0.01.
B. DQN P ARAMETER D ETAILS
In experiments, we use SUM followed by an MLP for READOUT and 1 DCONV + MAXPOOL followed by an MLP for INTERACT .
Speciﬁcally, the MLP has 2 layers down-projecting the node embeddings from 64 to 32 dimensions.
Notice that different types of embeddings require different MLPs, e.g. the
MLP used for aggregating and generating graph-level embeddings is different from the MLP used for aggregating and generating subgraph-level embeddings.
For 1 DCONV + MAXPOOL , we apply a 1-dimensional convolutional neural network to each one of two embeddings being interacted, followed by performing max pooling across each dimension in the two embeddings before feeding into an MLP to generate the ﬁnal interacted embedding.
Speciﬁcally, the 1 DCONV contains a ﬁlter of size 3 and stride being 1.
The MLP afterwards is again a 2-layer
MLP projecting the dimension to 32.
As shown in the main text, such learnable interaction operator brings performance gain compared to simple summation based interaction.
The ﬁnal MLP takes four components, h G = INTERACT ( h G 1 , h G 2 ) ,
h s = INTERACT ( h s 1 , h s 2 ) , h D c , and h D 0 , each with dimension 32.
It consists of 7 layers down-projecting the 128-dimensional input ( 32 × 4 ) to a scalar as the predicted q score.
For every MLP used in experiments, all the layers except the last use the ELU( x ) activation function.
An exception is the ﬁnal MLP, whose last layer uses the ELU( x ) + 1 as the activation function to ensure positive q output.
A subtle point to notice is the necessity of using either nonlinear readout or nonlinear interaction for generating the bidomain representation.
Otherwise, if both operators are a simple summation, the representation for all the connected bidomains ( h D c ) is essentially the global summation of all nodes in all the connected bidomains.
In other words, the nonlinearity of MLP in the readout operation or the interaction operator allows our model to capture the bidomain partitioning information in h D c .
C N OTES ON S EARCH C. C OMPARISON WITH M C S P AND M C S P +RL
The key idea of our model is that under a limited search budget, by exploring the most promising node pairs ﬁrst, search can reach a larger common subgraph solution faster.
In other words, for small graph pairs, all baseline models would obtain the exact MCS result as long as the search algorithm runs to complete, i.e. the stack is eventually empty, meaning no more actions to select and no more states to backtrack to (all states have been visited and fully expanded to all possible next states).
However, for large graph pairs, the task is NP-hard, and the complete search becomes nearly impossible.
Exceptions exist though:
For example, if the pruning condition based on the upper bound estimation is powerful enough to prune many states, i.e. lines 12-14 in Algorithm 1 in the main text, the search may ﬁnish in relatively few iterations.
However, we observe that the state-of-the-art solvers,
M C S P and M C S P +RL, cannot ﬁnish completely for all the graph pairs used in testing.
Instead of trying to improve the upper bound estimation to be more exact, in this paper, our goal is to learn a better node pair selection policy, i.e. line 17 in Algorithm 1, to replace the heuristics used by baseline solvers.
Notice that our focus on node pair selection policy instead of upper bound estimation implies that a better selection policy would mean the search can quickly ﬁnd a larger solution and update its best solution found so far maxSol .
This not only mean when the search budget is used up, the result returned is larger, but also mean that for subsequent iterations (before the iteration limit is reached), more states would be pruned by checking UB t ≤ | maxSol | , thus further helping the search.
In summary, in our framework, the upper bound computation strategy remains unchanged, yet the successful node pair selection policy beneﬁts the search in two major ways.
Since we use M C S P in the imitation learning stage of training our DQN, and compare with M C S P and M C S P +RL in the main text, we describe their node pair selection heuristics.
For M C S P , when entering a new state, it ﬁrst selects the node with the largest node degree in G 1 , and then enumerates through all the nodes in G 2 in descending order of node degrees.
In the original implementation provided by M C S P , this is achieved by recursive function calls.
After all the nodes in G 2 are visited, i.e. the depth-ﬁrst search of all the node pairs ( i, j ) ﬁnishes where i is the largest-degree node in Gand j is every node in G 2 , the algorithm selects the second largest-degree node in G 1 , and repeats the enumeration of nodes in G 2 .
After all node pairs are exhausted, the function returns and the algorithm essentially backtracks to the parent state.
If the current state is the root node in the search tree, the search is complete and the exact MCS is returned.
However, as noted earlier, for large graph pairs it is almost impractical to search exhaustively and a budget on the amount of search conducted has to be applied.
Thus, which node pairs to visit ﬁrst matters a lot for successfully extracting a large solution for large input graphs.
However, as seen in Figure 10 and 11, in many cases the true MCS does not contain large-degree nodes, since large-degree nodes tend to form more complicated subgraphs which are harder to match in the other input graph compared to simpler subgraphs like a chain.
Thus, by visiting large-degree nodes ﬁrst, M C S P may not always yield a large solution fast.
In contrast, M C S P +RL maintains a promising score for each node and iteratively updates the scores as search visits more states.
The update formula is based on the reduction of upper bound for search, where upper bound in an overestimation of future subgraph size.
As search makes progress, the scores are updated in each iteration, and nodes which cause large reduction in upper bound computation get large reward.
This has the limitation that for each new graph pair, the scores associated with each node must be re-initialized to 0 and re-learned, since there is no neural network and the only learnable parameters are the scores for each node.
At the beginning of search, all scores are initialized to 0, and the search has to break the tie using another heuristic, while once trained, our GLS EARCH can be applied to any new testing pair, and at the beginning of search, the learned parameters in GLS EARCH starts to beneﬁt the search.
In other words, the whole design of M C S P +RL can be regarded as a search framework with shallow learning (without neural networks or training via back-propagation).
GLS EARCH is the ﬁrst model to use deep learning for node pair selection.
Another limitation of M C S P +RL is that the scores maintained for nodes reﬂect the potential ability of a node to reduce the upper bound for future iterations in the current search, which is indirect as the MCS aims to ﬁnd the largest common subgraph, not the reduction of upper bound.
Moreover, the upper bound itself is an overestimation of future subgraph size, which may or may not be close enough to the actual best future subgraph size.
In contrast, we aim to predict the q score for actions which directly reﬂect the best future subgraph size.
Overall, the lack of deep learning ability causes
M C S P +RL not only to re-estimate the scores for the nodes for each new graph pair, but also to resort to the upper bound heuristic for updating the scores.
To ensure the budget on search iterations is applied consistently for all the models evaluated in the main text, we adapt the original recursive implementation of M C S P in C++ to an iterative implementation in Python so that all the models compare with each other in the same programming language and the same search backbone algorithm.
To be speciﬁc, we check the iteration count at the beginning of every search iteration and early stop the search if the pre-deﬁned budget is reached.
C. T REE VS S EQUENCE
At this point, having illustrated the differences between GLS EARCH and M C S P and M C S P +RL, it is worth clarifying whether GLS EARCH search yields a tree or sequence in different stages.
For training stage 1 and 2, as described in Section B.2, our model is just randomly initialized and not well trained, so the pre-training and imitation learning stages use the policy of M C S P instead of using its own predicted q scores.
In stage 1, the complete search is performed to provide maximum amount of supervised signals, i.e. y t , but in stage 2, we start using the RL training framework, i.e. experience replay buffer, target network, etc, so we run the agent multiple times until a terminal state is reached, corresponding to a sequence in a tree, which starts from the root node and ends at a leaf node.
For stages 2 and 3, since sequences are generated instead of trees, for each sequence, the upper bound check always passes, because the pruning only happens when backtracking is allowed, i.e. a tree is formed.
To see this more clearly, recall the pruning only happens if UB t ≤ | maxSol | in Algorithmin the main text.
However, during the sequence generation process, maxSol keeps increasing by one each time a new state is reached.
Since UB t ← | curSol | + overestimate ( s t ) , UB t > | curSol | for non-terminal s t , and | curSol | = | maxSol | , and thus UB t > | maxSol | , and thus the pruning never happens.
At the beginning of stage 2, the sequences we collect are usually not long, since the policy is the same as M C S P in stage 2.
However, in stage 3, we start using our predicted Q ( s t , a t ) to get such sequences, and at the end of training, when we apply GLS EARCH to testing pairs during inference, as shown in the main text, we perform better than M C S P and all the other baselines.
In inference, we completely rely on our predicted q scores as the policy, and a search tree is yielded, although the tree is not complete since the graph pairs are large and a search budget is reached.
C. T ERMINAL C ONDITIONS
This section discusses on how a terminal state, i.e. a leaf node in the search tree, is determined.
Notice our deﬁnition of bidomain (equivalence class) does not include node labels, and in each iteration, we allow the matching between nodes in the same bidomain with the same node label.
We consider the node labels as additional pruning on each bidomain, i.e. we further only allow nodes with the same label to match within bidomain when considering actions to be fed into DQN.
Suppose G 1 and G 2 are connected graphs.
There are two cases:
• Case 1: Nodes are unlabeled (or equivalently, all the nodes have the same label).
The terminal condition is that there is no non-empty connected (adjacent) bidomains.
For example, there are still some adjacent bidomains, but for each bidomain (cid:104)V (cid:48)
k 1 , V (cid:48)
k 2 (cid:105) , at least one of V (cid:48)
k 1 and V (cid:48) k 2 is empty (containing no nodes), so there is no nodes to match in each bidomain.
Examples are states 3 and 6 in Figure 6.
• Case 2: Nodes are labeled.
For the terminal condition, there may be still some non-empty connected bidomains, but the node labels do not match causing no more node pairs to select from.
For example, one bidomain contains V (cid:48)
k 1 with C and
labels label.
Then essentially
C. P ROMISE - BASED S EARCH : I MPROVING S EARCH WITH B ACKTRACKING
[Figure]
Because it computes Q-values, GLS EARCH can leverage more information than baselines when exploring the search space.
Speciﬁcally, Q-values may be used to compute a predicted maximum cumulative sum of rewards for every search state, s t , in the search tree: (cid:80) t − 1 j =0 γ j r ( s j , s j +1 )
+ γ t max k Q ( s t , a k ) , where Q ( · , · ) can be the Q-score predicted by GLS EARCH .
This score intuitively encodes maximum subgraph size achievable from every search state, and, for brevity, we will refer call it the promise score.
An example of how to compute this score is shown in Figure 5.
Because promise scores are comparable across all search states, GLS EARCH can consider the best possible state to visit from any search state, whereas baselines can only consider the best possible state to visit from the current search state.
GLS EARCH can also backtrack to more promising search states at any iteration of the search process, whereas past MCS models backtrack only when all future search states from the current state are exhausted.
Unlike M C S P or M C S P +RL, GLS EARCH is optimized to ﬁnd the largest common subgraph in one try, not to prune the search space.
This is because, in practice, even with advanced pruning techniques, it is not practical to exhaust the entire search space for large graphs.
As a consequence,
GLS EARCH may fall into local solutions if it strictly follows the M C S P search algorithm (Algorithm 1 of main text).
Thus, GLS EARCH improves upon M C S P search by backtracking to an earlier state with the highest predicted maximum cumulative sum of rewards when the current best solution is not improved upon within a ﬁxed number of iterations.
In practice, we set this number to 3.
In implementation, whenever a new state, s t +1 is reached from state s t by taking action a ∗ , promise scores for both s t +1 and s t are computed to tell the search algorithm where to backtrack.
We associate the promise value of (cid:80) t − 1 j =0 γ j r ( s j , s j +1 )
+ γ t Q ( s
t , a ∗ ) to s t
+1 and update the promise value of s t with (cid:80) t − 1 j =0 γ j r ( s j , s j +1 )
+ γ t max a ∈ A \{ a ∗ } Q ( s t , a ) , because a ∗ is excluded in future iterations of search from state s t (line 18 of Algorithm 1 of main text).
In addition to pushing states to the stack on line 19 and 21 of Algorithm 1 in the main text, we also push states to an additional priority queue.
Thus, whenever the backtrack condition is satisﬁed, we retrieve states with the highest expected cumulative sum of rewards by popping the priority queue instead of the stack.
C. N OTES ON E QUIVALENT S TATES AND M ULTIPLE G ROUND T RUTHS
[Figure]
It is important to note that states3, 5, 6 are different since their node-node mappings are different.
However, the solutions derived from both states 3 and 6 have the same subgraph size, 3.
In other words, there can be multiple ways to arrive at the same solution size, with different underlying sequential processes to reach the ﬁnal states.
It is well known that for the MCS detection task, there can be multiple ground truth solutions with the same subgraph size.
For example, in Figure 6, both states 3 and 6 correspond to the same subgraph size but the states 3 and 6 are different due to their different node-node mappings.
Our model maintains the node-node mappings for each state, and therefore states 3 and 6 would be reached as different states.
It is important to note that for large graph pairs, reaching both states 3 and 6 usually does not happen, since the search would ﬁrst reach state 3 and need many iterations to backtrack to state 0 and further many iterations to reach state 6.
There is an even more subtle point in Figure 6.
Suppose the search ﬁrst matches node a to node 1, denoted as state 1, then matches node b to node 2, leading to state 2.
After several iterations, it backtracks to state 0 and chooses to match node b to node 2, denoted as state x, then matches node a to node 1, denoted as state y. Although states 1 and x are different, but states 2 and y are equivalent, since both states 2 and y have the same node-node mapping, i.e. a to 1 and b to 2.
Thus, the search maintains an additional set of visited states and at each time a new state is reached, a checking is performed to avoid revisiting the same state twice.
The node-node mapping is is an important component of the deﬁnition of state, not only because it differentiates the otherwise equivalent states, but also because different node-node mappings can lead to different ﬁnal states and future reward (and thus must be considered by the design of DQN).
Suppose the bidomain “01” in state 1 contains more than two nodes, i.e. there are many nodes connected to b in G 1 besides node d and many nodes connected to node 2 in G 2 besides node 3.
Then state 2 is an intuitively more preferred state compared to state 5, since the matching of node b to node
2 allows more node pairs to be matched to each other in future, thus a larger action space in state 2.
The value associated with state 2 thus should be larger than state 5.
C. D EALING WITH L ARGE A CTION S PACE
For large graph pairs, the successful detection of MCS not only depend on the design of our critical
DQN component as well as its training, but also rely on techniques which prune the large action space at each state.
The bidomain partitioning idea has been outlined in the main text which effectively reduces the action space size by only matching nodes in the same bidomain.
However, for extremely large and dense graphs, the bidomains may not split the rest of the graphs enough and the action space may still be too large.
For example, consider two fully connected graphs G 1 and G 2 , i.e. for every two nodes there is an edge.
Then initially, there is no nodes selected, and there is only one bidomain consisting of all the node pairs.
What is worse, at any state, there is always only one bidomain consisting of all the node pairs in the remaining subgraphs.
Therefore, to reduce the action space further, we only compute the q scores for N d nodes at most in each state.
Speciﬁcally, we ﬁrst sort the candidate bidomains by their size in ascending order, and select the ﬁrst N b small bidomains.
Next, we sort the nodes in each bidomain by degree in descending order and select the ﬁrst N d /N b nodes with large node degrees.
For our experiments, N b = 1 and N d = 20 .
We ﬁnd, in practice, these settings do not drastically alter performance.
Denote C 1 = N 2 d , which is a constant representing the most amount of node pairs the DQN looks at per iteration.
Then, in each iteration, the time complexity of GLS EARCH is O ( C 1 ) .
Suppose on average, each one of the C 1 node pairs requires
C 2 amount of computations.
In practice, for a simple implementation, all the C 1 node pairs can be computed sequentially, resulting in C 1 C 2 running time.
However, various techniques can be applied in practice, e.g. batching the C 1 computations, which will further reduce the bounded running time.
In implementation, we apply some optimizations to decrease the running time, e.g. computing the node embeddings at the beginning of search and caching them for the entire search during inference.
However, we believe further optimization of our implementation is possible and leave for future efforts.
In experiments, we set the search budget to 7500 iterations for all baselines to compare and ﬁnd which method can quickly ﬁnd a larger solution in terms of iterations.
In future, additional techniques for pruning the action space and speeding up the computation can be explored.
For example, instead of bounding the computation to be N d , we may perform a hierarchical graph matching by ﬁrst running a clustering algorithm and then matching clusters which will be bounded by the number of clusters.
Another possible direction is to learn an additional Q function learning which node is more promising instead of which node pair, i.e. Q ( s t , a ( i ) t ) for node i and
Q ( s t , a ( j ) t ) for node j in the action a t =
( i, j ) .
We suppose such additional Q function may bring further performance gain.
C. A NALYSIS OF T IME C OMPLEXITY
Overall the branch-and-bound search has exponential worst-case time complexity due to the NP-hard nature of exact MCS detection, and our goal is to use additional overhead per search iteration to make “smarter” decision each iteration so that we can ﬁnd a larger common subgraph faster (in less iterations AND real running time).
Per iteration, our model requires the neural network operations to compute a Q score instead of simply using a degree heuristic which is O (1) .
Here we analyze the time complexity of these neural operations:
• To compute the node embeddings, the complexity is the same as the GNN model, which in our case is O ( |V| + |E| ) for GAT (since nodes must aggregate embeddings from neighbors and attention scores must be computed for each edge).
Notice the node embeddings are computed by local neighborhood aggregation, and will not be updated in search, and therefore we compute the node embeddings only once at the beginning of search, and can be cached for efﬁciency.
• At each iteration, to compute a Q score for a state-action pair, we run Equation 3 which requires computing the whole-graph, subgraph, and bidomain embeddings.
Overall the time complexity is O ( |V| − |V s | ) where V s is the number of nodes in the currently matched subgraph.
The whole-graph embeddings do not change across search, so they only need to be
computed once at the beginning.
The subgraph embeddings can be maintained incrementally, i.e. adding new node embeddings as search grows the subgraph.
The bidomain embeddings are computed via a series of READOUT and INTERACT operations (Equation 2): For
READOUT: We use summation followed by MLP so the runtime is O ( |V| − |V s | ) ; For
INTERACT:
We use a 1D CNN followed by MLP which depends on the embedding dimension set to a constant, and does not depend on the number of nodes in the input graphs.
D B ASELINE D ESCRIPTION AND C OMPARISON
For all the models used in experiments, we evaluate their performance under the same search framework, i.e. with consistent search iteration counting, upper bound estimation, etc.
M C S P and
M C S P +RL use heuristics to select node pairs, which is ineffective as shown in the main text and has been described in Section C.1.
Therefore, this Section focuses on the comparison with the rest baselines, i.e. GW - QAP (Xu et al., 2019a), I- PCA (Wang et al., 2019), and N EURAL M CS (Bai et al., 2020b).
GW - QAP is a state-of-the-art graph matching model for general graph matching.
The task is not about MCS speciﬁcally, but instead about matching two graphs with its own criterion based on the
Gromov-Wasserstein discrepancy (Peyr´e et al., 2016).
Therefore, we suppose the matching matrix Y generated for each graph pair can be used as a guidance for which node pairs should be visited ﬁrst.
In other words, we pre-compute the matching scores for all the node pairs before the search starts, and in iteration, we look up the matching matrix and treat the score as the q score for action selection.
I- PCA and N EURAL M CS essentially compute a matching matrix too, and it is worth mentioning that all the three methods cannot learn a score based on both the state and the action.
They can be regarded as generating the matching scores based on the whole graphs only without being conditioned and dynamically updated on states and actions.
I- PCA is a state-of-the-art image matching model, where each image is turned into a graph with techniques such as Delaunay triangulation (Lee & Schachter, 1980).
It utilizes similarity scores and normalization to perform graph matching.
We adapt the model to our task by replacing these layers with 3 GAT layers, consistent with GLS EARCH .
As the loss is designed for matching image-derived graphs, we alter their loss functions to binary cross entropy loss similar to N EURAL M CS which will be detailed below.
N EURAL M CS is proposed for MCS detection with similar idea from I- PCA that a matching matrix is generated for each graph pair using GNNs.
However, they both require the supervised training signals, i.e. the complete search for training graph pairs must be done to guide the update of I- PCA and N EURAL M CS .
In contrast, GLS EARCH is trained under the RL framework which does not require the complete search (in stage 2 and 3, only sequences are generated as detailed in Section C.2).
This has the beneﬁt of exploring the large action space in a “smarter” way and eventually allows our model to outperform I- PCA and N EURAL M CS .
In implementation, the complete search is not possible for large training graph pairs, so instead we apply a search budget and use the best solution found so far to guide the training of I- PCA and N EURAL M CS .
Regarding the subgraph extraction strategy, for all the baselines, we use the same branch and bound algorithm, which is the state-of-the-art search designed for MCS (McCreesh et al., 2017).
However, as mentioned in Section C.4, only our model is equipped with the ability to backtrack in a principled way.
The main text shows the performance gain to GLS EARCH brought by the backtracking ability.
E M ORE R ESULTS WITH A NALYSIS E. B EST S OLUTION S IZES ACROSS T IME
Figure 7 shows that under the budget we set, for the large real-world graph pairs, all the methods reach a “steady state” where the best solution found so far no longer grows.
This means the search continues but the search cannot ﬁnd a larger solution, illustrating the fact that search gets “stuck”.
In theory, given inﬁnitely many iterations, all the models will eventually ﬁnd the true MCS, which is the largest, but since the task is NP-hard, the search space can be exponential in the worst case (subject to pruning but in all the testing graph pairs the search has not ﬁnished yet), such budget has to be applied to search.
At the end of the 7500 iterations, though, all the models have made “mistakes” (visiting unpromising states) and in order to ﬁnd even larger common subgraphs, the search needs to backtrack potentially many times to ﬁx those “mistakes” (backtracking to a very early state). Admittedly
[Table] there is always the possibility that for more iterations [Table] some baseline method may ﬁnd a larger solution.
Besides, in each iteration GLS EARCH does take more running time overhead as shown in Table 6.
However, the point of our model is to quickly ﬁnd a larger solution in as few iterations as possible, not to ﬁnd a large solution given too many iterations.
In other words, the goal of GLS EARCH is to be smart enough to quickly ﬁnd a large solution instead of purely ﬁnding a good solution.
Figure 7 shows that GLS EARCH not only ﬁnds solutions larger than baselines when the 7500 iterations budget is reached but also ﬁnds larger solutions faster than baselines before the budget is reached [Table]
[Table] In addition to reaching a larger solution is less iterations, GLS EARCH also reaches better solutions with respect to runtime, as shown in Figure 8.
Notice, GLS EARCH ﬁnds the same large subgraph in 10 minutes as in 7500 iterations.
Although per iteration, it is slower than M C S P per iteration,
GLS EARCH ﬁnds a larger solution in usually less than a minute.
Moreover, our implementation can be further optimized.
Note, we adapted all baselines to run on Python for fair comparison, and made sure the search iteration counting is consistent across all baselines and the results shown in the main text.
At this stage, our main goal is to explore the idea of “learning to search” , which has been experimentally veriﬁed to be a promising direction of research, and leave the efforts of implementation optimization using various techniques as future focus.
E. A DDITIONAL A BLATION S TUDY
Table 7 shows that pre-training and imitation learning beneﬁt the performance under four out of the eight datasets.
On E NRO and HP PI , without pre-training, our model performs better, which may be attributed to the fact that they are dense graphs (Table 4) while the training graphs used in stage 1 are relatively small and sparse (Section B.1). training.
[Table]
Table 8 shows that the promised-based search improves the performance under four out of the eight datasets.
For the other four datasets, the performance does not change, indicating that the backtracking to an earlier promising state based on the DQN output at least does not hurt the performance.
In the cases like E NRO and HP PI , whose average node degrees are large, the promise-based search improves the performance by a large amount, showing the usefulness of the proposed strategy [Table]
[Figure] [Figure] [Table] E. S CALABILITY S TUDY
GLS EARCH can scale to very large graphs containing tens of thousands of nodes. To verify this, we generate 6 large synthetic graphs (3 graph pairs) using the BA, ER
[Table]
and WS algorithms, each with 10000 nodes [Table] and ran GLS EARCH against heuristic-based baselines for 7500 iterations. Our model is superior in all instances, as shown in Table
[Table]
[Table]
F E XTENSIONS OF GLS EARCH
GLS EARCH can be extended for a ﬂurry of other MCS deﬁnitions, e.g. approximate MCS, MCS for weighted and directed graphs, etc. via a moderate amount of change to the search and learning components.
In this section, we brieﬂy outline what could be done for these tasks.
For approximate MCS detection, the bidomain constraint must be relaxed.
One method of relaxing this constraint is to allow sets of nodes belonging to different but similar bidomains to match to each other.
For instance, nodes in G 1 from the bidomain of bitstring “00110” could map with nodes in
G 2 from the bidomain of bitstring “00111”, since they are only 1 hamming distance away.
Such relaxations as this can be made stricter or looser based on the application.
The difference would be the search framework, thus the learning part of GLS EARCH can largely stay the same.
Regarding MCS for graphs with non-negative edge weights, assuming our task is to maximize the sum of edge weights in the MCS, instead of deﬁning r t = 1 , we can alter the reward function to be the difference of the sum of edge weights before and after selecting a node pair r t = Σ e ∈ S ( u,v ) t w ( e ) − Σ e ∈ S t w ( e ) where S t is the edges of currently selected subgraph, S ( u,v ) t is the edges of the subgraph after adding node pair, ( u, v ) , and w ( · ) is a function that takes and edge and returns its weight.
As the cumulative sum of rewards at step T is the sum of edge weights Σ t ∈ [1 ,...,T ] r t = Σ e ∈ S T w ( e ) and reinforcement learning aims to maximize the cumulative sum of rewards, we can adapt GLSearch to optimize for MCS problems with weighted edges.
Regarding MCS for directed graphs, the bidomain constraint may be altered such that every bit in the bidomain string representations now has 3 states: ‘0’ for disconnected, ‘1’ for connected by in-edge, and ‘2’ for connected by out-edge.
By considering the inward/outward direction of a bitstring, we can guarantee the isomorphism of directed graphs.
In this case, the search framework would only differ in how bidomains are partitioned.
The learning part of GLSearch would stay the same for this application.
More generally, we believe that there are many more extensions to GLSearch in addition to the ones listed, such as disconnected MCS, network alignment, or subgraph extraction.
Further exploration of these are to be done as future efforts.
We plot the testing graph pairs and the results of M C S P and GLS EARCH in this Section.
For all the ﬁgures, we use two colors for nodes, one for the selected subgraphs by the model, the other for the remaining subgraphs that cannot be further matched within the search budget.
When plotting, we use larger circle size for nodes with larger degrees.
Figure 10 shows the extraction result of M C S P .
Compared with our result in Figure 11, M C S P uses the largest node degree as its node pair selection policy — As a result, it tends to focus on nodes with large degrees and ignore smaller-degree nodes.
On the contrary, our model learns which node pairs should be selected, and as a result, it selects a balance of large and small degree nodes leading to an eventual large subgraph selected in G 1 and G 2 .