Efficient Graph Similarity Computation with Alignment Regularization
Abstract
We consider the graph similarity computation (GSC) task based on graph edit distance (GED) estimation.
State-of-the-art methods treat GSC as a learningbased prediction task using Graph Neural Networks (GNNs).
To capture finegrained interactions between pair-wise graphs, these methods mostly contain a node-level matching module in the end-to-end learning pipeline, which causes high computational costs in both the training and inference stages.
We show that the expensive node-to-node matching module is not necessary for GSC, and highquality learning can be attained with a simple yet powerful regularization technique, which we call the Alignment Regularization (AReg).
In the training stage, the
AReg term imposes a node-graph correspondence constraint on the GNN encoder.
In the inference stage, the graph-level representations learned by the GNN encoder are directly used to compute the similarity score without using AReg again to speed up inference.
We further propose a multi-scale GED discriminator to enhance the expressive ability of the learned representations.
Extensive experiments on real-world datasets demonstrate the effectiveness, efficiency and transferability of our approach.
Introduction
Graph similarity computation (GSC) is a fundamental task in graph databases and plays a critical role in many real-world applications, including drug design [14, 28], program analysis [16] and social group identification [21, 27].
For example, one can search a drug database for a query chemical compound, in order to identify drugs with high similarity in structures or attributes and thus similar curative effects as desired [23].
To measure the similarity between pair-wise graphs, Graph Edit
Distance (GED) [5] has been a major metric due to its generality and many other graph similarity measures have been proven to be its special cases [17].
Unfortunately, computing exact GED is an
NP-hard problem in general [12].
With the provably expressive power in distinguishing graph structures [32, 19, 7, 34], Graph Neural Networks (GNNs) have been adopted for GED approximation and shown to achieve superior performance on accuracy.
Most state-of-the-art GNN-based GSC models [1, 3, 8, 18, 16] contain two sequential submodules in the end-to-end learnable pipeline (left of Fig.
1): (1) a GNN encoder, which is shared across two graphs to embed nodes into representation vectors to capture the intra-graph structure and feature information; (2) a matching model, which computes cross-graph node-level similarity, i.e., how a node in one graph relates to all the nodes in the other graph.
The model outputs a summarized vector that fuses the node-level similarities between two graphs.
Then, the similarity score is predicted based on the summarized vector via a regression head.
The computational cost of such a sequential framework mainly comes from the matching model, which requires computational and memory cost quadratic in the number of nodes and sometimes involves additional parameters such as attention weights [18, 16], leading to heavy time consumption in both the training and inference stages.
Especially in the inference stage, we need to query every testing graph from the database.
The recent approach EGSC [22] speeds up the similarity learning by dropping the matching model from SimGNN [1].
However, since cross-graph node-level interactions are ignored, EGSC cannot capture finer-grained similarity information, and thus results in suboptimal prediction performance.
To overcome the intrinsic tension between predictive accuracy and speed, we propose a separated neural structure (right of Fig. 1) that detaches the matching model from the sequential pipeline, where the matching model only acts as a regularization term in the training stage to help the GNN encoder capture fine-grained similarity information, while in the inference (testing) stage, since the latent cross-graph interactions have been learned by the GNN encoder, the final similarity score can be directly computed based on the output representations of the GNN encoder without invoking the matching model again.
[Figure]
We show that explicitly learning the cross-graph nodeto-node similarity is unnecessary, as the correlation information contained in the features and graph topology, when properly exploited, is sufficient to reflect such cross-graph interactions (see Section 3).
Specifically, the problem of GED computation is equivalent to finding an optimal permutation such that the adjacency matrices of the two graphs can be best aligned.
Hence, by analyzing the necessary conditions under the optimal permutation, we find that the best matching between two graphs can be inferred by minimizing the difference between the intra-graph node-graph similarity and cross-graph node-graph similarity .
It motivates us to design a task-agnostic matching model based on the input data itself, with a novel regularization technique, called the Alignment
Regularization (AReg).
AReg obviates the need of a matching model in the inference stage, thus making the model more efficient.
AReg is also model-agnostic and can be applied to other GNN-based
GSC models.
On the other hand, GNN-based GSC models usually use a single GED discriminator followed by a regression head to fuse graph representations from pair-wise input graphs and output a final similarity prediction score.
We show that a single GED discriminator does not fully capture the dissimilarity between two graphs, while diverse discriminators may provide complementary information to reflect GED more accurately.
Thus, we propose a multi-scale GED discriminator to improve the discriminability of the learned representations.
We call the overall framework ERIC: Efficient gRaph sImilarity Computation, and conduct extensive experiments to verify the efficacy of our design.
Results on several real-world datasets demonstrate that ERIC achieves state-of-the-art performance by significantly outperforming the baselines.
Preliminary
Problem Formulation of GSC.
Given a graph database D and a set of query graphs Q , the problem of graph similarity computation is to produce a similarity score y between ∀ G i ∈ Q and ∀ G j ∈ D , i.e., y = s ( G i , G j ) where s : D × Q → R (0 , 1] is a similarity estimator.
A graph G ∈ D ∪ Q is defined as G =
( V, E ) , where V = { v k } Nk =1 is a node set and E ∈
V × V is an edge set.
In our setting, all the accessible graphs are unweighted and undirected.
V and E jointly formulate an adjacency matrix A ∈ R N × N .
If nodes are accompanied by features (e.g., labels or attribute vectors), they are represented as X ∈ R N × d with dimension d .
Graph Edit Distance (GED).
GED as a graph similarity measure has been popularly adopted on graph search queries, due to its capacity to capture the structural and feature differences between graphs.
As shown in Fig.
2, GED is defined as the number of edit operations in the optimal path that transforms G i into G j , where the possible edit operations under consideration include edge deletion/insertion, node deletion 1 /insertion, and node relabeling.
To well fit the end-to-end regression task, instead of directly estimating the GED between two graphs, we convert the GED to the ground truth similarity score that the learning model aims to approximate.
Following [1], the normalized
GED is defined as nGED( G i , G j ) =
GED( G i ,G j ) ( | V i | + | V j | ) / 2 , and the ground truth similarity score between
G i and G j is defined as the normalized exponential of GED, resulting in a value ranging (0 , 1] , i.e.,
S ij = exp( − nGED( G i , G j )) .
[Figure]
Motivation: Analyzing GED in Embedding Space
Given two graphs G i
=
( A i , X i ) and G j =
( A j , X j ) with the same number of nodes
N (If they have different numbers of nodes, pad the smaller A and X with zeros to make the two graphs equal in size), π ( · ) is a node index permutation that preserves the adjacency matrix.
π ( A ) denotes the adjacency matrix after node index permuting.
We divide the computation of GED between G i and
G j into two steps: (i) finding a permutation for G j , such that
[Equation]
where c is the sum of the absolute values of all elements in matrix A i − π ( A j ) .
We denote the optimal permutation satisfying Eq.
(1) as π ⋆ .
(ii) counting the number of cross-graph node pairs with the same indices yet different features, denoted as m .
Then GED( G i , G j ) =
c 2 + m .
Obviously, if two graphs are topologically isomorphic, there exists π = π ⋆
such that A
i = π
⋆ ( A j ) , i.e., c = 0 .
Hence the GED is only determined by distinct features.
For another example, when the node permutation π assigns indices to G j as shown in Fig.
2, the structure of G i and G j can be best aligned, i.e., c = 4 .
Then, only one pair of nodes with the same index across graphs have different features (e.g., node 4).
Thus, GED( G i , G j ) = 3 .
The core of this two-step method is finding the optimal permutation to best align two graphs, then m is determined under such alignment.
Traversing all possible permutations to find π ⋆ is also NP-hard, so we make some heuristic rules from Eq.
(1) to guide the model design.
Under the optimal permutation π ⋆ , the row similarity between A i and π ⋆ ( A j ) is maximized, so given an injective function f θ ( · ) : R N → R d parameterized by θ to guarantee that nodes with different connectivity can be distinguished, a necessary condition is that the distance between f θ ( A i [ k ]) and f θ ( π ⋆ ( A j )[ k ]) is minimized for all k ∈ { 1 , · · · , N } .
From a global view, the matrix similarity between A i and π ⋆ ( A j ) is also maximized.
Given an injective function g ϕ ( · ) : R N × N → R d parameterized by ϕ , another necessary condition under the optimal permutation is that the distance between g ϕ
( A i ) and g ϕ ( π ⋆ ( A j )) is minimized.
Thus, the optimal permutation π ⋆ in Eq.
(1) also satisfies the following function,
[Equation]
[Equation] , N } , (2) where DIST( · , · ) is a distance metric.
The two terms in Eq.
(2) can reflect GED at node-level and global-level respectively.
Further, we regard { f θ ( A i [ k ]) }
Nk =1 ∪ { f θ ( π ⋆ ( A j )[ k ]) } Nk =1 as 2 N anchors and assume N ≫ d .
When the second term of Eq.
(2) reaches a minimum, it indicates that g ϕ ( A i ) and g ϕ ( π ⋆ ( A j )) have similar distances to all anchors, i.e., the following γ i and γ j take the minimum value when π = π ⋆ ,
[Equation]
[Figure]
[Equation]
On the other hand, the first term of Eq.
(2) is a finer-grained alignment between two graphs, and π ⋆ causes the first term of Eq.
(2) to reach a minimum.
Since the only difference between γ i and γ j is that γ j replaces f θ ( A i [ k ]) in γ i with f θ ( π ( A j )
[ k ]) , the distance between f θ ( A i [ k ]) and f θ ( π ( A j )
[ k ]) can be reflected by the difference between γ i and γ j , i.e., ∥ γ i − γ j ∥ 2 .
Hence, under the optimal permutation π = π ⋆ , G i and G j are best aligned, and so Eq.
(2) is established, which is equivalent to γ i , γ j , and ∥ γ i − γ j ∥ 2 all taking the minimum values.
These terms reflect similarities between pair-wise graphs at both node- and graph- levels, which can be inferred from the graph structure itself without consulting the ground-truth GEDs.
It motivates us to separate the matching model from the end-to-end pipeline.
In other words, instead of directly searching the optimal permutation π ⋆ , we derive the necessary conditions under π ⋆ , which impose additional constraints on the coordinates of nodes in the embedding space.
These conditions are not necessarily sufficient, nevertheless they can provide useful knowledge for the model to learn representations that are better tailored to the
GSC task.
In Appendix A, we further analyze the rationality of this approximation by analyzing the sufficiency of these conditions.
Assuming g ϕ is a permutation-invariant function, then all permutation operators in Eq.
(3) and Eq.
(4) can be removed.
Moreover, instead of computing cross-graph node-to-node similarity to compute a ‘soft’ alignment as done in most related work, γ i and γ j only compute node-to-graph similarity, which is more efficient.
Proposed Model: ERIC
The proposed ERIC framework mainly consists of two submodules: the Alignment Regularization (AReg) module and the Multi-Scale GED Discriminator, as depicted in Fig.
AReg aims to train the shared GNN encoder to enable the GNN to capture underlying alignment information between pair-wise graphs.
The multi-scale GED discriminator trains the same GNN encoder so that the learned representations of two graphs can accurately reflect the GED.
Such a paradigm follows the intuition of the GED definition that when two graphs are best aligned, the difference between them is the GED.
4.1 Alignment Regularization
Learning the optimal alignment between graphs is crucial for GED estimation, however, most existing
GSC models rely on intricate cross-graph node-to-node similarity computation to learn a ‘soft’ alignment.
Furthermore, learning node-to-node similarity inevitably yields a dense similarity matrix, which could introduce more noise.
To address this issue, we introduce Alignment Regularization (AReg) as a part of our framework.
AReg is a model- and task-agnostic regularization term, which can easily be combined with existing embedding-based GSC models in a plug-and-play manner.
The design of AReg is motivated by the analysis of GED in Section 3.
Recall that GED is the difference between two graphs when they are best aligned under the optimal permutation π ⋆ , whose necessary conditions are that γ i , γ j as well as ∥ γ i − γ j ∥ 2 take the minimum values.
Thus, we expect that the learning-based model can couple with the nature of GED as much as possible, i.e., the
[Figure]
GNN encoder can preserve the best alignment, in which case the difference between the learned graph representations can reflect the GED.
To that end, we design the model based on the necessary conditions under π ⋆ .
Specifically, based on Eq.
(3) and Eq.
(4), we instantiate f θ ( · ) as the L -layer
Graph Isomorphism Network (GIN) [32], then at the ℓ -th layer f ( ℓ ) θ ( A i [ k ]) can be represented as:
[Equation]
(cid:17)
[Equation]
(5) where ξ ( ℓ ) is a learnable parameter, H ( ℓ ) i ∈ R N × d ( ℓ ) is the feature matrix of G i at the ℓ -th layer where d ( ℓ ) denotes the feature dimension, H (0) i
= X i .
On the other hand, AReg implements the readout function g ϕ with a one-layer permutation-invariant DeepSets [33] to guarantee injectiveness, taking the form:
[Equation]
where MLP ( ℓ ) ϕ and MLP ( ℓ ) θ have the same output dimension d ( ℓ )MLP .
Since g ϕ ( A j ) =
g ϕ
( π ⋆ ( A j )) , and γ i and γ j are sums over all N nodes, hence they are permutation-invariant and we can remove all operation π ( · ) in Eq.
(3) and Eq.
DIST( · , · ) can be defined as any distance metrics such as cosine similarity.
Let γ ( ℓ ) i and γ ( ℓ ) j be the value of Eq.
(3) and Eq.
(4) at the ℓ -th layer, by considering multi-scale cross-graph interactions, AReg is represented as L AReg where:
[Equation]
Since γ i and γ j induced from Eq.
(2) integrate graph-level alignment and finer-grained node-level alignment, L AReg therefore preserves underlying cross-graph interactions without computing complicated node-to-node similarity, also making the training stage more efficient.
4.2 Multi-Scale GED Discriminator
Now we have L graph-level representations for G i and G j respectively, denoted as { Z (1) i , · · · , Z ( L ) i } and { Z (1) j , · · · , Z ( L ) j } .
We concatenate the layer-wise graph representations for G as: (cid:98) Z = (cid:76) (
cid:8) Z ( ℓ ) (cid:9)
L ℓ =1 , where (cid:76) denotes the concatenation operator along the last dimension to combine the graph representation in each layer, i.e., (cid:98) Z i , (cid:98) Z j ∈ R d ms , and d ms = (cid:80) ℓ d ( ℓ )MLP .
Then they are fed into a GED discriminator that generates score vectors as GED similarity embedding for the graph pair.
Neural Tensor Network (NTN) [26] has demonstrated strong power to model the relation between the graph-level embeddings of two graphs [1, 22, 30] thanks to its capacity of exploring the element-wise dependence among the features.
However, directly using NTN as the discriminator may be expensive when the dimension of (cid:98) Z i and (cid:98) Z j is high.
Thus, we decompose the weight matrix W t ∈ R d ms × d ms into two matrices W t 1 ∈ R d ms × d ′ and W t 2 ∈ R d ′ ×
d ms where d ′ ≪
d ms to reduce the number of parameters.
Then NTN with decomposed weight matrices is used to measure the similarity between (cid:98) Z i and (cid:98)
Z j :
[Equation]
where W 1 ∈ R d ms × d ′ × T , W 2 ∈ R d ′ ×
d ms × T , W 3 ∈ R T × 2 d ms , and b ∈ R T are learnable; T is a hyper-parameter controlling the output dimension, which is assigned as 16 for all datasets in our settings.
[ · ] in Eq.
(8) means computing ( (cid:98)
Z i W t 1 )( W t 2 (cid:98) Z ⊤ j ) for all t ∈ { 1 , · · · , T } and stacking them as a T -dimensional tensor.
δ NTN : R T → R (0 , 1] is a fully-connected neural network with Sigmoid activation as a regression function to project the similarity score vector to the final predicted similarity value.
NTN is an expressive and general similarity discriminator because it can approximate many similarity measures.
The first term of Eq.
(8) can be regarded as a multi-head weighted cosine similarity function.
It can also approximate kernel similarity between (cid:98) Z i and (cid:98) Z j according to the universal approximation theorem [13].
The second term captures the residual knowledge.
However, it is difficult for NTN to approximate high-order Minkowski distance between (cid:98) Z i and (cid:98) Z j , while diverse similarity discriminators may provide complementary information to reflect GED more accurately as shown in Fig.
Hence, we consider an additional similarity discriminator based on exponential p -order
Minkowski distance:
[Equation]
where δ p : R d ms → R (0 , 1] is also a fully-connected neural network with Sigmoid activation.
For simplicity, we uniformly set p = 2 (i.e., ℓ 2 distance) for all datasets, and analyze the sensitivity of the hyper-parameter p in Section 5.4.
After two similarity scores s NTN ( G i , G j ) and s p ( G i , G j ) are obtained, the final estimated similarity score is given by:
[Equation]
where α and β are trainable scalars denoting the weights of two similarity discriminators respectively.
Given a graph database D , the GED discriminator is trained on a set of n training pairs ( G i , G j ) ∈
D × D .
The predicted similarity is compared against the ground-truth similarity S ij based on GEDs with Mean Squared Error (MSE) loss function as:
[Equation]
Model training: Combining AReg and GED discriminator, the training stage aims to minimize the following overall objective function
L =
L GED + λ L AReg , where λ is an adjustable hyper-parameter controlling the strength of the regularization term.
Model inference: Given a set of query graphs Q and a graph database D , in the testing stage, all pairs of graphs ( G i ,
G j ) ∈
Q × D are fed into ERIC, and directly computing a similarity score for each pair based on Eq.
(10) without any node-level interactions.
Complexity:
The time complexity of GIN in AReg is O ( m ) [31] where m is the number of edges.
The cross-graph node-graph interactions have complexity O
(max( N i , N j )) .
The time complexity of the GED discriminator is O
( d ms d ′ T ) .
Thus the complexity of ERIC is O ( m + max( N i , N j ) + d ms d ′ T ) in the training stage, while in the inference stage the complexity is O ( m + d ms d ′ T ) .
Experiments
In this section, we empirically evaluate ERIC on the graph similarity computation task.
[Table]
5.1 Datasets
We conduct experiments on four widely used GSC datasets including AIDS700 , LINUX , IMDB [1], and NCI109 [2].
Following the same splits as [1–3], i.e., 60%, 20%, and 20% of all graphs as training set, validation set, and query set, respectively.
The training set together with the validation set are called the database .
More details of the datasets are presented in Appendix B.1.
5.2 Experimental Settings
Baselines.
We implement two groups of baselines for comparison: (1) Combinatorial search-based
Methods: Beam [20], Hungarian [15], and VJ [9]; (2) GNN-based Methods: SimGNN [1], GMN [16],
GraphSim [3], MGMN [18], and EGSC [22].
We re-implemented all baselines with the same hyperparameters as the original literature provides, or carefully tuned the parameters to get the optimal results when they are not provided.
We give more implementation details of ERIC and baselines in
Appendix B.2.
Evaluation Metric.
To comprehensively evaluate our model on the similarity computation task, following [1, 22] five metrics are adopted to evaluate results for fair comparisons: Mean Squared
Error (MSE) which measures the average squared differences between the predicted and the groundtruth similarity.
Spearman’s Rank Correlation Coefficient ( ρ ) and Kendall’s Rank Correlation
Coefficient ( τ ) which evaluates the ranking correlations between the predicted and the true ranking results.
Precision@ k where k
= 10 , 20 , which computes the interactions of the predicted and ground-truth top-
k results divided by k .
The smaller the MSE, the better performance of models; for ρ , τ , and p @
k , the larger the better.
5.3 Main Results
The results of ERIC and the baselines on our benchmarks are reported in Table 1.
For datasets with relatively small graphs whose ground-truth GEDs are exactly computed by the A ⋆ algorithm,
ERIC consistently achieves state-of-the-art performance across all evaluation metrics as shown in
Table 1.
For datasets with large graphs whose ground-truth GEDs are computed approximately,
ERIC still achieves the best results on NCI109, and comparable results on IMDB.
The suboptimal performance of the methods based on node-to-node similarity (SimGNN, GraphSim, GMN, and
MGMN) demonstrate that overly dense and fine-grained similarity computation may not always bring a beneficial boost.
EGSC also uses GIN as the backbone and totally ignores cross-graph node-level interactions, and it achieves the best results on metrics ρ and τ on the IMDB dataset.
The reason is that EGSC adopts intra-graph attention pooling and NTN in all layers to enhance the expression ability of representation vectors.
However, the attention mechanism and full NTN need additional parameters which increase the burden of learning.
Although EGSC proposed a student model based on knowledge distillation to speed up the inference process, it would sacrifice the prediction accuracy in most cases.
The performance of ERIC over baselines illustrates the importance of fine-grained interactions and the proper design of such interactions.
5.4 Ablation Study
Effectiveness of AReg and Multiple GED Discriminators.
ERIC contains two key components:
AReg and Multi-Scale GED Discriminator.
To glean a deeper insight into how different components help ERIC to achieve highly competitive results, we conduct ablation experiments by removing individual components separately.
Specifically, we alter the loss by removing the AReg term to
[Table]
[Table] [Table]
[Figure]
study the effect of the cross-graph node-graph interaction, with the results reported in ERIC (w/o
AReg) of Table 2.
We find that only using the MSE loss L GED will lead to a performance drop on the extracted three evaluation metrics, which confirms the necessity of AReg.
In our design, the final similarity score is obtained by the weighted average of two similarity scores based on the NTN discriminator s NTN and
the ℓ 2 discriminator s p respectively.
To further investigate the impact of multiple GED discriminators, we remove one of s NTN and s p to study the effect of each discriminator.
As shown in Table 2, both ERIC (w/o NTN) and ERIC (w/o ℓ 2 ) cause a decrease in effectiveness, which demonstrates both of them contribute to the final performance, while the model benefits more from NTN.
Transferability of AReg Further, since AReg is a model-agnostic regularization term, we are interested in the transferability of AReg, so we evaluate the performance of applying AReg to other
GSC models.
We use SimGNN and EGSC as baselines.
For SimGNN, we use AReg to replace the node-to-node similarity computation.
For EGSC, we directly add AReg to the loss function.
Tableshows the effect of AReg on SimGNN and EGSC.
As expected, the advantage of SimGNN+AReg over SimGNN shows that integrating the dense similarity matrix into the final similarity score may bring noise which affects performance.
While the performance on EGSC proves that combining fine-grained similarity in a proper way, i.e., node-graph rather than node-node, can improve the model.
Sensitivity of Order p in δ p
In the multi-scale GED discriminator module, we adopt the exponential p -order Minkowski distance as a similarity measure to further improve the separability of clusters in the graph embedding space.
For simplicity, we directly use the 2-order Minkowski distance, i.e., ℓ 2 distance.
Results in Table 2 show the effectiveness of considering the ℓ 2 distance, and Fig.demonstrates the complementarity of different similarity measures.
To investigate the sensitivity of p in our model, we set p from [2 , 4 , 6 , 8] , and run 10 times for each value of p .Then
the results of mean square error with standard deviation are reported in Fig.
We can observe that the performance increases first and then the error becomes stable when p ≥ 2 .
It shows that using the ℓ 2 distance as the GED discriminator is suitable for our model and higher-order Minkowski distance would not improve the performance.
5.5 Inference Time
In ERIC, the cross-graph alignment only acts as a regularization term in the training stage but is no longer used in the inference stage.
To evaluate the efficiency, we compare the performance of
ERIC with baselines in terms of inference time in Table 4.
In Table 5 of Appendix B.1 we list the number of graph pairs in the inference stage (#Testing pairs), and all experiments are implemented with a single machine with 1 NVIDIA Quadro RTX 8000 GPU.
As can be observed, node-to-node
[Table]
[Table] [Table]
[Table] [Table]
[Figure] [Figure]
(a)~(e) are ranked by the exact normalized GEDs between the query graph and graphs of particular ranks in the database.
For example, 0.571 (10th) means the nGED of the graph that ranks 10th in similarity to the query graph is 0.571.
The color depth in the heatmap represents the similarity of the node pair; the deeper the color, the higher the similarity.
similarity computation is more time-consuming on all datasets.
Our proposed node-graph similarity computation does not incur substantial additional running time in practice.
To summarize, ERIC is faster than all baseline models, while still achieving significantly higher accuracy on all datasets.
5.6 Visualization of Graph Search
The goal of the graph search task is to find k graphs from the dataset that are most similar to the given query graph, which is a routine task in drug discovery [24].
In Fig.
6 we show a case based on the
AIDS700 dataset, where each graph represents a functional group.
Comparing the exact similarity ranking computed by A ⋆ and the estimated one computed by ERIC, we see that the ranking of ERIC has a high consistency with the exact ranking, which proves that ERIC is able to extract graphs that contain the similar substructure from around 700 graphs with varying size and structure and the top-ranked graph has a high degree of isomorphism with the query.
It also demonstrates that ERIC can capture structural patterns shared across graphs.
More results of visualization are provided in
Appendix C.
5.7 Analysis of Node-to-Node Similarity
In Fig.
7, we show the node-to-node similarity between a query graph and the graphs at different ranking positions in terms of normalized GEDs.
It can be found that for graph pairs with a small
GED, the node representations generated by ERIC show a clear correspondence between nodes as shown in Fig.
7 (a).
As the GED increases, the correspondence between nodes across the graph gradually weakens, i.e., cross-graph node-to-node similarity reduces.
Thus, the similarity matrices in
Fig.
7 can guide us to find a better alignment between pair-wise graphs.
Related Work
The goal of graph similarity computation (GSC) is to quantify the similarity between graphs under a specific similarity measure.
Various similarity measures have been well studied in prior works, such as graph edit distance (GED) [5, 25, 20] and maximum common subgraph (MCS) [6, 10].
Among these, GED is the most popular one, and many other similarity measures can be proven to be its special cases [17].
However, computing the exact GED between two graphs is known to be
NP-hard.
In practice, the computation becomes challenging when the number of nodes is more than 16 [4] using exact GED solvers such as the A ⋆ algorithm [25].
Thus, approximate algorithms have been proposed for GED-based GSC.
These approximate methods can be broadly divided into two classes: (1) Combinatorial search-based methods , which aim to exploit combinatorial structures or theoretical lower-bounds to approximate GED.
Beam [20] is a GED estimator based on Beam
Search; Hungarian [15] is proposed based on the famous Hungarian algorithm for weighted graph matching; Hausdorff approximation [11] provides a lower bound for the GED approximation; VJ [9] uses the Volgenant and Jonker algorithm for GED approximation.
However, these methods are highly heuristic-driven and run with either sub-exponential time or cubic time complexity, limiting the scalability as the graphs grow in size.
Also, these methods totally ignore the node feature information, so that the underlying semantic similarity can not be captured.
(2) Learning-based methods , which are data-driven and aim to learn graph similarity from the data itself, hopefully with higher accuracy and far lower time costs compared with search-based methods.
GMN [16] introduces a cross-graph attention layer that allows the nodes in the two graphs to interact with each other and predicts graph similarity using the representation vectors that fuse cross-graph information.
SimGNN [1] relies on a shared GNN encoder, a neural tensor network, and a pairwise node comparison module to compute the similarity between two graphs.
GraphSim [3] extends SimGNN by using convolutional neural networks to capture the multi-scale node-level interactions.
EGSC [22] simplifies SimGNN by ignoring node-level interactions and uses knowledge distillation to accelerate the inference stage.
MGMN [18] employs a node-graph matching network to capture cross-level features between nodes of a graph and the other whole graph, where the cross-graph aggregation weights are computed by node-to-node attention coefficients.
Our proposed ERIC is also a learning-based method.
Unlike the above approaches that either discard node-level interactions entirely, or performs interactions between all node pairs, our method proposes a novel node-graph interaction paradigm that avoids dense similarity computation while preserving fine-grained interactions.
Conclusion
We propose ERIC, a simple yet powerful GNN-based framework for the graph similarity computation task.
Specifically, we first give a deep insight into the graph edit distance, and propose Alignment
Regularization (AReg) which is a separated structure independent of the end-to-end learning pipeline.
AReg frees the model from complicated node-to-node interaction for similarity computation.
Further, we propose a multi-scale GED discriminator to improve the discriminative ability of the learned representations.
We show the effectiveness of our model through a comprehensive set of experiments and analyses.
Acknowledgments and Disclosure of Funding
This work is supported in part by Shenzhen Baisc Research Fund under grant
JCYJ20200109142217397.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[N/A]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section(b)
Did you include complete proofs of all theoretical results?
[Yes] See Section3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Supplemental Material (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Appendix B. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Section 5.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[No] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[No] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[No] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Sufficiency Analysis of AReg
As mentioned in Section 3, GED measures the difference between two graphs when they are best aligned.
In other words, if we perform node permutation on one graph, so that the same-indexed nodes in two graphs are as similar as possible in structure, then the GED can be directly computed by computing the difference between the permuted adjacency matrices.
In the paper, we refer to the permutation that satisfies such a condition as the optimal permutation π ⋆ .
However, directly traversing all possible node permutations to find π ⋆ has complexity O ( N !) .
Alternatively, we rethink this problem in embedding space, and consider necessary conditions that are implicitly required by π ⋆ and that are easy to implement in the model.
To that end, we define γ i , γ j and require both objectives, as well as their difference || γ
i − γ
j
|| 2 to be minimized.
To analyze the accuracy of such approximation, we analyze the sufficiency of these conditions.
Notice that γ
i
[Equation]
[Equation]
|| 2 formally reflects how each node is compatible with the two graphs equally well.
A minimum γ i therefore implies the two graphs are properly aligned.
Similarly, a minimum γ j achieves the same goal from the standpoint of
G j .
As an additional perspective, we view 2 N vectors { f θ ( A i [ k ]) } Nk =1 ∪{ f θ ( π ( A j )
[ k ]) } Nk =1 as a set of anchors, denoted as T .
Since f θ ( · ) ∈ R d , if the linear span of a subset of T can be a space R d , then for all k , a small difference between DIST ( f θ ( A i [ k ]) , g ϕ ( A i )) and DIST ( f θ ( A i [ k ]) , g ϕ ( π ( A j ))) can reflect a small distance between g ϕ
( A i ) and g ϕ ( π ( A j )) , thus we can further get that A i and π ( A j ) are very similar input for the injective function, i.e., A i and π ( A j ) is best aligned, so we have π = π ⋆ .
If || γ i − γ j || 2 is minimized meanwhile, it will lead to the distance between f θ ( A i [ k ]) and f θ ( π ( A j )[ k ]) takes the minimum, which is the node-level embeddings under π ⋆ .
Thus, assuming
N > d , as long as at least d of the 2 N nodes have different neighbor indexes, there must exist a subset of T that can span R d , then our necessary conditions can be sufficient.
In most real-world cases the number of nodes N is large than the embedding dimension d , and the connected structures of most nodes are not exactly the same due to the complexity of the network.
Hence, the sufficiency of our proposed conditions usually can be satisfied in real-world cases.
B More Details of Dataset and Experiments B.1 Datasets
The statistical information of our experimental datasets is shown in Table 5.
The exact GEDs of
AIDS700 and LINUX are computed by the A ∗ [25] algorithm in exponential time.
For datasets with large graphs, following [1, 2] IMDB and NCI109 take the pair-wise smallest distance computed by three approximate algorithms, Beam [20], Hungarian [15], and VJ [9], as ground truth.
[Table]
B.2 Implementation Details
For the AReg submodule, the number of layers of the GNN encoder is set to 4 and the hidden dimension of each layer is selected from [32 , 64] .
The DeepSets function MLP ϕ is a 1-layer MLP with the same output dimension with the GNN encoder in each layer.
We simply set DIST( · , · ) to cosine similarity.
For the GED discriminator submodule, both the hyper-parameter d ′ and the output dimension T of the decomposed NTN are set to 16.
Both δ NTN and δ p are 2-layer MLP with hidden dimension 16.
For SimGNN and GraphSim, the results we re-implement using the official code are different from that reported by original papers.
It is because SimGNN and GraphSim adopt an additional hyper-parameter to scale the ground-truth GEDs, while other baselines do not.
Thus, we do not consider scaling of the ground-truth.
The source code is available https://github.com/JhuoW/ERIC.
C Extra Visualizations of Graph Search
A few more graph search cases are included in Fig.
8 and Fig. 9, which show that the ranking results learned by our ERIC have high consistency with the ground-truth similarity ranking.
We can find that graphs at the same ranking position have similar substructures.
It indicates that ERIC is able to retrieve graphs that best resemble the query
[Figure]
[Table]
[Figure] [Figure]
ERIC.