GREED: A Neural Framework for Learning Graph Distance Functions
Abstract
Among various distance functions for graphs, graph and subgraph edit distances (G ED and S ED respectively) are two of the most popular and expressive measures.
Unfortunately, exact computations for both are NP-hard.
To overcome this computational bottleneck, neural approaches to learn and predict edit distance in polynomial time have received much interest.
While considerable progress has been made, there exist limitations that need to be addressed.
First, the efﬁcacy of an approximate distance function lies not only in its approximation accuracy, but also in the preservation of its properties.
To elaborate, although G ED is a metric, its neural approximations do not provide such a guarantee.
This prohibits their usage in higher order tasks that rely on metric distance functions, such as clustering or indexing.
Second, several existing frameworks for G ED do not extend to S ED due to S ED being asymmetric.
In this work, we design a novel siamese graph neural network called G REED , which through a carefully crafted inductive bias, learns
G ED and S ED in a property-preserving manner.
Through extensive experiments across 10 real graph datasets containing up to 7 million edges, we establish that
G REED is not only more accurate than the state of the art, but also up to 3 orders of magnitude faster.
Even more signiﬁcantly, due to preserving the triangle inequality, the generated embeddings are indexable and consequently, even in a CPU-only environment, G REED is up to 50 times faster than GPU- powered baselines for graph / subgraph retrieval.
Introduction and Related Work
A distance function on any dataset, including graphs, is a fundamental operator.
Among several distance measures on graphs, edit distance is one of the most powerful and popular mechanisms [30, 51, 49, 20].
Edit distance can be posed in two forms: graph edit distance (G ED ) and subgraph edit distance (S ED ).
Given two graphs G 1 and G 2 , G ED ( G 1 , G 2 ) returns the minimum cost of edits needed to convert G 1 to G 2 , i.e., for G 1 to become isomorphic to G 2 .
An edit can be the addition or deletion of edges and nodes, or the replacement of edge or node labels, with an associated cost.
In S ED ( G 1 , G 2 ) ,
G ED is typically restricted to graph databases containing small graphs to facilitate distance computation with queries of similar sizes.
As an example, given a repository of molecules, and a query molecule, we may want to identify the closest molecule in the repository that is similar to the query [36, 20].
S ED , on the other hand, is useful when the database has large graphs and the query is a comparatively smaller graph.
As examples, subgraph queries are used on knowledge graphs for analogy reasoning [17].
In PPI and chemical compounds, S ED is of central importance to identify functional motifs and binding pockets [42, 20, 16, 38, 35, 37].
Unfortunately, both G ED and S ED are
NP-hard to compute [49, 20].
To mitigate this computational bottleneck, several heuristics [10, 13] and index structures [20, 49, 30, 51] have been proposed.
Recently, graph neural networks have been shown to be effective in learning and predicting G ED [3, 45, 29, 4, 50, 46, 14, 2].
The basic goal in all these algorithms is to learn a neural model from a training set of graph pairs and their distances, such that, at inference time, given an unseen graph pair, we are able to predict its distance accurately.
Other works seek to incorporate non-neural graph matching solvers [40], or generic integer linear programming solvers [33, 34], to learn the graph matching task from natural data such as images in an end-to-end trainable manner.
Here, the NP-hardness of the problem is relegated to the non-neural component, and the neural part is only concerned with representation learning, unlike in our setting where we want the neural network to handle both.
In the domain of subgraphs, N EURO M ATCH [39] and I SO N ET
[41] generate embeddings to detect subgraph isomorphism.
N SC [44] generates subgraph level embeddings that can count the number of subgraph instances of a query graph on a target graph.
While the progress made is impressive, there is scope to do more.
• Preservation of theoretical properties: G ED is a metric distance function.
While S ED is not metric due to being asymmetric, it satisﬁes the triangle inequality , non-negativity , and subgraphidentity (S ED = 0 for subgraphs).
Metrics (and triangle inequality) exhibit signiﬁcant computational advantages over non-metrics.
Speciﬁcally, operations such as clustering [19], nearest neighbor search [15, 21, 43, 12], outlier detection [1] and diameter computation [23] admit efﬁcient algorithms precisely when the objects being studied are embedded in a metric space.
Existing neural approaches do not preserve these properties, which limits their usability for these higher order tasks.
• Indexable embeddings: Given graph pair G 1 and G 2 , neural approaches ﬁrst embed them into a feature space.
Next, they compute a distance on these feature vectors, which is an approximation of the distance in the original graph space.
The literature on indexing range and k -NN queries over feature vectors is rich [18, 25, 12].
Index structures typically allow sub-linear computation costs with respect to the database size.
Unfortunately, none of the existing neural approaches generate indexable feature vectors since they perform pair-dependent computations.
Speciﬁcally, the neural computations on G 1 depend on both G 1 and G 2 (and same for G 2 ).
Consequently, the computations can only be done at query-time and thereby negating the possibility of indexing pre-computed feature space embeddings.
• Modeling S ED : Prior to this work, there have been no neural approaches for S ED .
Further, existing neural methods to learning G ED cannot easily be adapted to learn S ED .
While G ED is symmetric, S ED is not.
Several neural architectures for G ED have the assumption of symmetry at its core and hence modeling S ED is non-trivial [3, 4, 29].
• Exponential Search Space
: Computing S ED ( G 1 , G 2 ) conceptually requires us to compare the query graph G 1 with the exponentially many subgraphs of the target graph G 2 .
Therefore, it is imperative that the model has an efﬁcient and effective mechanism to prune the search space without compromising on the prediction accuracy.
GQ
GT
GNN GNN
Shared Weights
GRAPHS
EMBEDDINGVECTORS
ZQ
ZT
F( Z Q , Z T )PREDICTION (a) Siamese architecture of G REED k GIN layers
Concatenatefrom all layers
PreMLP
PostMLP
NODEFEATUREVECTORS
POOL
EMBEDDINGVECTOR (b
[Figure]
The
[Figure]
In this work, we address the above limitations through the following contributions.
• Novel neural architecture:
We address the
above
mentioned challenges through a novel architecture called G REED :
GRaph Embeddings for Edit Distances.
G REED utilizes a siamese graph isomorphism network [47] to embed graphs in a pair-independent fashion.
A simple, but theoretically well-characterized, function on this embedding space predicts the S ED and G ED .
The carefully crafted prediction function serves as an inductive bias for the model, which, in addition to enabling high generalization accuracy, preserves the metric property of G ED and the triangle inequality of S ED in the embedding space.
• Indexable embeddings: Owing to pair-independent embeddings and preservation of the triangle inequality over the embedding space for both S ED and G ED , G REED can exploit the rich literature on index structures [18, 25, 12] to boost efﬁciency.
• Accurate, Fast and Scalable: Extensive experiments on real graph datasets containing up to a million nodes establish that G REED is more accurate in both G ED and S ED when compared to the state of the art algorithms and is more than 3 orders of magnitude faster in range and k -NN queries.
Furthermore, owing to indexable embeddings, even in a CPU-only environment, G REED is up to 50 times faster than the closest baseline run on a GPU.
Preliminaries and Problem Formulation
We denote a labeled undirected graph as G =
( V , E , L ) where V is the node set, E is the edge set and
L : V ∪ E → Σ is the labeling function over nodes and edges.
Σ is the universe of all labels and contains a special empty label (cid:15) .
L ( v ) and L ( e ) denote the labels of node v and edge e respectively.
G 1 ⊆
G 2 denotes that G 1 is a subgraph of G 2 .
The problem of learning G ED and S ED [20] is deﬁned as follows.
Problem 1 (Learning G ED /S ED )
Given a training set of tuples of the form (cid:104)G 1 , G 2 , G ED ( G 1 , G 2 ) (cid:105) (or G 1 , G 2 , S ED ( G 1 , G 2 ) (cid:105) ), learn a neural model to predict G ED ( Q 1 , Q 2 ) (or S ED ( Q 1 , Q 2 ) ) on unseen graphs Q 1 and Q 2 .
For details on the exact deﬁnition of G ED and S ED , we refer to the appendix (App.
A).
2.1 Properties of G ED and S ED
Theorem 1 Let (cid:98) d : Σ × Σ → R +0 be a distance function over Σ , where (i) (cid:98) d ( (cid:96) 1 , (cid:96) 2 )
=
if (cid:96) 1 = (cid:15) , and (ii) (cid:98) d ( (cid:96) 1 , (cid:96) 2 ) =
d ( (cid:96) 1 , (cid:96) 2 ) otherwise; the following holds: S ED ( G 1 , G 2 ) =
(cid:100)
G ED ( G 1 , G 2 ) , where (cid:100) G ED denotes G ED with (cid:98) d as the label set distance function.
In simple words, the S ED between two graphs is equivalent to G ED with a label set distance function where we ignore insertion costs.
P ROOF .
See App.
B.1.
Observation 1 G ED satisﬁes the triangle inequality if the distance function d over label set Σ satisﬁes the triangle inequality [20].
As deﬁned in the paragraph following Def. 2, d satisﬁes the triangle inequality [20].
Furthermore, it is trivial to see that G ED is symmetric, non-negative and satisﬁes identity as long as d satisﬁes the analogues.
Hence, G ED with distance function d is metric.
Theorem 2 S ED is not metric due to violating the properties of symmetry and identity.
However, it satisﬁes the triangle inequality, i.e., S ED ( G 1 , G 3 ) ≤ S ED ( G 1 , G 2 )
+ S ED ( G 2 , G 3 ) .
P ROOF : See App.
B.2 for details.
Observation 2 Computing G ED and S ED is NP-hard [49].
Hereon, we use G ED as the illustrative distance function being modeled.
The architecture trivially extends to S ED .
The speciﬁc places that need separate treatment will be discussed explicitly.
G REED : The Proposed Architecture
Fig.
2 presents the architecture of G REED .
The input to our learning framework is a pair of graphs G Q (query), G T (target) along with the supervision data G ED ( G Q , G T ) (or S ED ( G Q , G T ) ).
Our objective is to train a model that can predict G ED on unseen query and target graphs.
The design of our model must be cognizant of the fact that computing G ED is NP-hard and high quality training data is scarce.
Thus, we use a Siamese architecture [11], where there are two networks with shared parameters applied to two inputs independently to compute representations.
3.1 Siamese Graph Neural Network
As depicted in Fig.
2a, we use a siamese graph neural network (G NN ) with shared parameters to embed both G Q and G T .
While one could use two different G NN models for the query and the target, this design increases the model parameters and consequently, the training time.
Furthermore, an architecture with higher number of parameters also requires larger amount of training data, which is difﬁcult due to G ED being NP-hard.
Fig.
2b focuses on the G NN component of G REED .
We next discuss each of its individual components.
Pre-M LP : x v in Fig.
2b is a one-hot encoding of the categorical node labels.
The dimension of the one-hot vector increases linearly with the number of labels in a graph database and hence can be very large.
The primary-job of the pre-mlp is to reduce it to a desirable dimension size through the operation µ G v =
M LP ( x v ) .
Indeed, a similar effect may also be obtained by directly feeding the onehot encoding to the ﬁrst layer of G IN .
However, since a G IN constructs embeddings by incorporating both structure and label information, one may desire a different dimensionality in the G IN layers.
Hence, the pre-mlp is motivated more from a conceptual separation of its task to that of G IN rather than a purely performance point of view (See App. G).
We do not explicitly model edge labels in our experiments.
G REED can easily be extended to edge labels by using G INE [22] instead of G IN .
Graph Isomorphism Network (G IN ): G IN [47] consumes the information from the Pre-M LP to learn hidden representations that encode both the graph structure as well as the node feature information.
G IN is as powerful as the Weisfeiler-Lehman (WL) graph isomorphism test [26] in distinguishing graph structures.
Since our goal is to accurately characterize graph topology and learn similarity, G IN emerges as the natural choice.
G IN develops its expressive power by using an injective aggregation function.
Speciﬁcally, in the initial layer, each node v in graph G is characterized by the representation learned by the M LP , i.e., h G v,
= µ G v .
Subsequently, in each hidden layer i , we learn an embedding through the following transformation.
[Equation]
Here, (cid:15) i is a layer-speciﬁc learnable parameter, N G ( v ) is one-hop neighbourhood of the node v , and h G v, 0 = µ G v .
The k -th layer embedding is h G v,k , where k is ﬁnal hidden layer.
Concatenation, Pool and Post-M LP : Intuitively, h G v,i captures a feature-space representation of the i -hop neighborhood of v .
Typically, G NN s operate on node or edge level predictive tasks, such as node classiﬁcation or link prediction, and hence, the node representations are passed through
an
M LP for the ﬁnal prediction task.
In our problem, we need to capture a graph level representation.
Furthermore, the representation should be rich enough to also capture the various subgraphs within the input graph so that S ED can be predicted accurately.
To fulﬁl these requirements, we ﬁrst concatenate the representation of a node across all hidden layers, i.e., the ﬁnal node embedding is z G v = C ONCAT (cid:0) h G v,i , ∀ i ∈ { 1 , 2 , · · · , k } (cid:1) .
This allows us to capture a multi-granular view of the subgraphs centered on v at different radii in the range [1 , k ] .
Next, to construct the graph-level representation, we perform a sum-pool, which adds the node representations to give a single vector.
This information is then fed to the Post-M LP to enable post-processing.
Mathematically:
[Equation]
G ED and S ED Prediction: The ﬁnal task is to predict the G ED (and S ED ) as a function of query graph embedding Z G Q and target graph embedding Z G T .
The natural choice would be to feed these embeddings into another M LP to learn G ED ( Z G Q , Z G T ) .
This M LP can then be trained jointly with the graph embedding model in an end-to-end fashion.
However, an M LP prediction does not have any theoretical guarantees with respect to the preservation of metric properties of G ED and the triangle inequality of S ED .
We, therefore, focus on learning prediction functions F g ( Z G Q , Z G T ) and
F s ( Z G Q , Z G T ) for G ED and S ED respectively, such that they are accurate and respects the desirable properties from the original graph space.
As we will empirically substantiate in § 4.5, the inductive bias injected through the prediction functions also lead to more effective learning over low volumes of training data than an MLP.
3.1. G ED
We require the following four properties to ensure that the prediction is also a metric.
[Equation]
To achieve this, we establish an important connection of metrics on vector spaces to norms.
Every norm (cid:107) .
(cid:107) gives a metric ( x , y )
(cid:55)→ (cid:107) x − y (cid:107) .
Moreover for a metric, there exists a norm (cid:107) .
(cid:107) such that the metric can be expressed as ( x , y )
(cid:55)→ (cid:107) x − y (cid:107) , iff the metric is translation invariant and homogeneous .
Thus, we add these properties to the desiderata for F g :
[Equation]
Armed with these observations, we deﬁne the class of functions that may be used for F g .
Observation 3 F g may be deﬁned as any function ( x, y )
(cid:55)→ (cid:107) x − y (cid:107) for some norm (cid:107) .
(cid:107) over the vector space R d such that F g satisﬁes Eqs.
The L p norm satisﬁes Obs.
Hence, we deﬁne F g as:
[Equation]
In our implementation we use the L 2 norm.
Finally, the parameters of the entire model are learned by minimizing the mean squared error (here T is the training set).
[Equation]
Intuition:
Regardless of the graph representations generated by our model, F g ensures that the predicted distance is a metric.
One the other hand, by training the model to produce embeddings
Z G Q and Z G T such that F g ( Z G Q , Z G T )
≈ G ED ( Z G Q , Z G T ) , we enforce a rich structure on the embedding space such that F g is also accurate.
Thus, F g injects an inductive bias satisfying the dual needs of accuracy and preservation of original space properties.
3.1. S ED
S ED satisﬁes non-negativity and triangle inequality.
Following a similar reasoning as above, we deﬁne F s as follows:
[Equation]
Intuitively, for those co-ordinates where the value of Z G Q is greater than Z G T , a distance penalty is accounted by F s in terms of how much those values differ; otherwise F s considers 0 .
This follows the intuition that the S ED accounts for those features of G Q that are not in G T .
Moreover, consistent with S ED , the additional features in G T that are not in G Q , do not incur any cost.
Lemma
The following properties hold on predicted S ED .
[Equation]
P ROOF .
Properties (1) and (2) follow from the deﬁnition of F itself.
Property (3) follows from the fact that we take the L 2 norm.
Formally, we state it as follows.
Lemma 2 F s ( Z G Q , Z G T ) ≤ F s ( Z G Q , Z G T(cid:48) )
+ F s ( Z G T(cid:48) , Z G T ) , where F s ( x , y ) = (cid:107) ReLU ( x − y )
(cid:107) for any monotonic norm (cid:107) .
(cid:107) .
P ROOF .
See App.
B.4.
3.2 Characterization of G REED
Importance of pair-independence and siamese architecture:
A pair-independent siamese architecture enables theoretical guarantees for G REED by constraining the model to learn a single mapping from graph space to embedding space for both query and target.
Despite these restrictions, G REED outperforms prior works which freely use cross-graph information and don’t provide theoretical guarantees.
This further conﬁrms that the siamese architecture is a useful prior.
Complexity Analysis: The complexity of G ED and S ED inference in G REED is linear in the number of nodes and edges in the query and target graphs (See App.
C for derivation).
This computation cost is drastically lower than the factorial computation cost of optimal G ED and S ED .
With respect to neural methods for graph similarity [3, 4, 29, 45, 50], all have at least quadratic computation cost, i.e., O ( |V| 2 ) .
Indexing Embeddings: Since the generated embeddings for both G ED and S ED satisfy triangle inequality, they are indexable leading to fast querying times.
We develop an index structure to exploit this property.
Due to space limitations, the details are included in App.
D.
In addition, we also design a neighborhood decomposition scheme, which enables fast pruning of the exponential search space § D.4.
In § 4.3, we empirically analyze the impact of index structures on querying time.
Empirical Evaluation
In this section, we establish the following:
• Efﬁcacy: G REED is more accurate than the state of the art approaches for both G ED and S ED .
• Efﬁciency: G REED is orders of magnitude faster than existing approaches and scales well to graphs with millions of nodes.
• Scalability: Pair-independence and indexability further enhances the scalability of G REED and enables it to be run on CPU-only platforms.
Our code base and datasets are available at https://github . com/idea-iitd/greed.
4.1 Experimental Setup
We use a machine with an Intel Xeon Gold 6142 processor and GeForce GTX 1080 Ti GPU for all our experiments.
Datasets: Table A lists the datasets used for benchmarking.
Further details on the dataset semantics are provided in the App.
E. We include a mixture of both graph databases (#graphs >1), as well as single large graphs (#graphs = 1).
Linux and IMDB are unlabeled.
We note that this is the ﬁrst study to evaluate neural graph distance approaches on million-scale graphs.
Baselines: To evaluate performance in G ED , we compare with S IM G NN [3], G ENN -A ∗ [45],
H 2 MN[50] and G OT SIM [14].
These are the most recent neural frameworks and have shown better efﬁcacy than other neural approaches such as S IM G NN [3], G RAPH S IM [4], and GMN [29].
For S ED , no neural approaches exist.
However, H 2 MN and S IM G NN can be trained by replacing
G ED with S ED along with minor modiﬁcations in training.
N SC [31] is a method for counting subgraphs using graph embeddings.
Since this is a related operation, we use N SC as a baseline by changing the loss function to minimize the RMSE between true and predicted S ED .
We also use
N EURO M ATCH [39] as a baseline, which was originally designed to detect subgraph isomorphism.
While N EURO M ATCH cannot predict S ED , it generates a violation score , which can be interpreted as
[Table] [Table]
[Table]
the likelihood of the query being subgraph isomorphic to the target.
The violation score can be used as a proxy for S ED and used in ranking of k -NN ( k -NearestNeighbour) queries.
Thus, N EURO M ATCH comparisons are limited to k -NN queries on S ED .
GMN [29], G RAPH S IM [4], G OT SIM [14] and
G ENN -A ∗ are not included since they cannot be easily adapted for S ED .
See App. F for details.
In the non-neural category, we use mixed integer programming based method M IP -F2 [27] with a time bound of 0 .
1 seconds per pair for both G ED and S ED .
M IP -F2 provides the optimal solution given inﬁnite time.
We also compare with B RANCH [5], which achieves an excellent trade-off between accuracy and time [6].
B RANCH uses linear sum assignment problem with error-correction (L SAPE ) to process the search space.
We use G EDLIB ’s
[7] implementation of these methods.
Training (and Test) Data Generation:
For G ED , we use (cid:104) query, target (cid:105) graph pairs from IMDB ,
AIDS’ , and Linux .
Our setup is identical to S IM G NN [3] and H 2 MN
For S ED , the target graphs are taken from datasets listed in Table A.
For the query graph, in AIDS , we use known functional groups [38].
In the rest of the graph datasets, queries are sampled by performing a random BFS traversal (depth up to 5).
Table A shows the average query sizes ( |V Q | , |E Q | ).We use mixed integer programming method F2 [27] implemented in G EDLIB [7] with a large time limit to generate ground-truth data.
Train-Validation-Test: We use 100 K query-target pairs for training and 10 K pairs each for validation and test.
All models are trained till validation loss is minimized or there is less than 0 .
05% change in validation loss over a number of extended epochs.
For G REED , we set the number of layers in G IN to 8 .
The hidden layer dimension is set to 64 .
For all baselines, we use the default parameters suggested by the authors.
4.2 Prediction Accuracy of S ED and G ED
Tables 1a and 1b present the accuracy of all techniques on G ED and S ED in terms of Root Mean
Square Error (RMSE ).
G REED outperforms all other techniques in 9 out of 10 settings across G ED and S ED .
While H 2 MN and N SC are the second best performers in S ED , G ENN -A ∗ performs well in
G ED .
G ENN -A ∗ , however, is extremely slow and does not scale on graphs of size beyond 10 .
H 2 MN, thus, provides the second best balance between efﬁcacy and efﬁciency after G REED .
The gap in accuracy is the highest in IMDB for G ED , where G REED is more than 10 times better than the neural baselines.
IMDB graphs are signiﬁcantly denser and larger than AIDS’ or Linux .
Thus, computing the optimal G ED is harder.
While all techniques have higher errors in IMDB , the deterioration is more severe in the baselines indicating that G REED scales better with graph sizes.
Impact of Query Size:
We next investigate how the accuracy varies against the query size.
Intuitively, the task gets harder with query size since the combinatorial space of possible maps increases exponentially.
For this analysis, we compare G REED with H 2 MN in IMDB and Dblp for
G ED and S ED respectively.
G ENN -A ∗ fails to scale on both datasets.
In Fig.
3, we plot the heat map of RMSE against query graph size.
In this plot, each dot corresponds to a query graph G Q .
The co-ordinate of a query is ( G ED ( G Q , G T ) , |V Q | ) (analogously deﬁned for
S ED ).
The color of a dot represents the RMSE; the darker the color, the higher is the RMSE.
When we compare the heat maps of G REED with H 2 MN, we observe that H 2 MN is noticeably darker.
Furthermore, the concentration of dark colors is noticeably higher on the upper-right corner indicating deterioration with larger query sizes and higher distance values.
This indicates that G REED scales better with query sizes and distances.
[Figure] [Figure]
Visualization: A case study to visually illustrate the efﬁcacy of G REED is provided in App.
I.
Range and k -NN queries: To quantify performance in range and k -NN queries, we measure F1score (Range query) and Kendalls’s tau ( k -NN)
[24] of the predicted answer set, when compared against the ground truth.
In Figs.
4a-4h, we measure the performance in range queries.
In S ED ,
G REED consistently outperforms all baselines in F1-score.
In G ED , the trend remains similar.
Although, G ENN -A ∗ outperforms G REED for a brief region in Linux , overall, G REED has the highest
F1-score.
We also note the G ENN -A ∗ is not included in Fig.
4h since it fails to scale on IMDB .
In k -NN queries (Tables 2a and 2b), G REED outperforms all algorithms in S ED .
In G ED , similar to the trend in range queries, G REED is the dominant method and G ENN -A ∗ marginally outperforms
G REED in Linux .
4.3 Efﬁciency
Tables 3a-3b present the inference times per 10 K query-target pairs.
In this experiment, we do not index embeddings by G REED so that the comparison unearths the raw difference in computation efﬁciency of solely the neural architectures.
As visible, G REED is up to 1800 times faster than the non-neural baselines and up to 10 to 20 times faster than H 2 MN, the current state of the art in G ED prediction.
Also note that G ENN -A ∗ is exorbitantly slow (Table 3a).
G ENN -A ∗ is slower since it not only predicts the G ED but also the alignment via an A ∗ search.
While the alignment information is indeed useful, computing this information across all graphs in the database may generate redundant information since an user is typically interested only on a small minority of graphs that are in the answer set.
In App.
H, we discuss this issue in detail.
4. Pair-independence and Indexability
Here, we showcase how pair-independent embeddings, and ensuring triangle inequality leads to further boost in scalability.
For this experiment, we use the three largest datasets of PubMed , Amazon and Dblp .
For each dataset, we pre-compute G REED embeddings of all database graphs by exploiting pair-independent embeddings.
Such pre-computation is not possible in the neural or non-neural baselines.
Furthermore, since the predictions of G REED satisfy triangle inequality, we index the pre-computed embeddings of the database graphs as discussed in § D.1.
Consequently, for G REED , we only need to embed the query graph and evaluate F to make predictions at query time.
Table 3c presents the results on range and 10 -NN queries.
When computations are done on a GPU, G REED is more than 1000 times faster than H 2 MN.
In the absence of a GPU, H 2 MN is practically infeasible since expensive pair-dependent computations are done at query time.
In contrast, even on a CPU, through indexing, G REED is ≈ 50 times faster than GPU-based H 2 MN.
Note that indexing enables up to 3 -times speed-up on G REED over linear scan, which demonstrates the gain from ensuring triangle inequality.
These results establish that G REED breaks new ground in scalability of neural
[Figure] [Figure]
[Table]
graph distance computations; not only is it faster, it overcomes the barrier of GPU-dependence and hence better suited for low-resource environments.
4.5 Ablation Study
In this study
[Table]
we explore the impact of our inductive biases in learning from low-volume data.
We create two variants of G REED :
(1) G REED -Dual trains the two parallel G NN models separately without weight-sharing, and (2) G REED -NN uses an MLP instead of F .
Both have strictly better representational capacity than G REED , so are expected to match the performance with inﬁnite data.
Figs.
4i-4k present the results on S ED .
The results of the same experiment on G ED is provided in
Figs.
F in the appendix.
The RMSE of G REED is generally better than G REED -Dual, with the difference being more signiﬁcant at low volumes.
This indicates that siamese structure helps.
Compared to G REED , G REED -NN achieves marginally better performance at larger train sizes in PubMed and
CiteSeer .
However, in Dblp , G REED is consistently better.
The number of subgraphs in a dataset grows exponentially with the node set size.
Hence, an MLP needs growing training data to accurately model the intricacies of this search space.
In Dblp , even 100k pairs is not enough to improve upon F .
Furthermore, since computing G ED and S ED is NP-hard, generating large volumes of training data is not desirable.
Overall, these trends indicate that F enables better generalization and scalability with respect to accuracy.
Furthermore, given that its performance is close to an MLP even on high-volume training data, and it enables indexing, the beneﬁts outweigh the marginal reduction in accuracy.
We also observe that generally, G REED -NN performs better than G REED -dual.
G REED -NN retains the inductive bias imparted by the Siamese architecture, but ablates the inductive bias of the custom prediction function.
The opposite happens in the case of G REED -Dual.
The observed phenomenon of
G REED -NN generally out-performing G REED -Dual can be interpreted as evidence for the Siamese architecture providing a stronger inductive bias than the custom prediction function.
More ablations studies justifying our choice of G IN and the sum-pool layer are provided in App.
G.
4.6 Generalization to Unseen Query Distributions in S ED
We train the model by sampling queries from the graph database through BFS enumerations.
How does G REED generalize to unseen distributions?
Towards that end, we generate queries from the three unseen distributions of (1) Random Walks (R W ), (2) Random Walks with Restarts (R WR ), and (3) SHA D OW [48] (See App.
J for details on the sampling strategies).
We ﬁrst note that in AIDS , we use real queries of functional groups, and thus the good performance in AIDS indicates good generalizability.
In Table 4a, we more exhaustively analyze this aspect.
As visible, the errors remain low.
Even more surprisingly, the errors on R W and R WR are better than the train distribution of BFS itself.
This indicates good generalization to unseen distributions.
4.7 Generalizability to Unseen, Larger Query Sizes:
Generating training data for learning G ED and S ED is expensive since optimal distance computations are NP-hard.
Hence, a desirable property would be to learn from small graphs and then generalize to larger unseen graphs.
We evaluate this ability for G REED and H 2 MN.
Table 4b provides the numbers.
We notice that although there is some deterioration in the quality for query sizes in the range [25 , 50] when compared to the entire set, it is not severe (G REED -50 in Table 4b).
However, if the train set only contains queries till size 25 and we deploy the learned model to infer on queries of larger unseen sizes, the drop in quality is signiﬁcant (G REED -25 in Table 4b).
This drop is even more dramatic in
H 2 MN.
On the positive side, G REED remains superior to the optimal non-neural approach (MIP-F2) when run with a generous time limit of 60 seconds per query.
Overall, this experiment highlights one direction that needs further study and improvement.
Conclusions, Limitation, and Future Directions
The problem of learning graph distances from their embeddings has seen much interest over the last few years.
This thread of research is important since it allows us to overcome the bottleneck of exponential graph alignment space.
Our experiments clearly establish G REED as the state of the art for both G ED and S ED (See App.
K for preliminary results on maximum common subgraph similarity ).
In addition, it is signiﬁcantly faster and provides better theoretical correspondence between properties of the original space and predicted space.
One clear direction of future work that emerges from our experiments is that G REED , and existing methods of graph distance learning, do not generalize well to unseen larger query sizes.
We hope to address this limitation next.
[Table]
[Figure]
[Table]
[Figure]
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section(c)
Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] For additional proofs, check Appendix B. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] For additional details please check Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] Please check Sec.4.1.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] MIT License (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] Link to codebase in §(d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Appendix A G ED and S ED
The computation of G ED relies on a graph mapping .
Deﬁnition 1 (Graph Mapping)
Given two graphs G 1 and G 2 , let ˜ G 1 =
( ˜ V 1 , ˜ E 1 , ˜ L 1 ) and ˜ G
= ( ˜ V 2 , ˜ E 2 , ˜ L 2 ) be obtained by adding dummy nodes and edges (labeled with (cid:15) ) to G 1 and G 2 respectively, such that |V 1 | =
|V 2 | and |E 1 | = |E 2 | .
A node mapping between G 1 and G 2 is a bijection π : ˜ G 1 → ˜ Gwhere (i) ∀ v ∈ ˜ V 1 , π ( v ) ∈
˜ V 2 and at least one of v and π ( v ) is not a dummy; (ii) ∀ e = ( v 1 , v 2 ) ∈ ˜
E 1 , π ( e ) =
( π ( v 1 ) , ( π ( v 2 ))) ∈
˜
E 2 and at least one of e and π ( e ) is not a dummy.
Example 1 Fig.
1 shows a graph mapping.
Edge mappings can be trivially inferred.
Deﬁnition 2 (Graph Edit Distance (G ED ) under mapping π ) G ED between G 1 and G 2 under π is
[Equation]
e ∈ ˜ E
[Equation]
where d : Σ × Σ → R +0 is a distance function over the label set.
d ( (cid:96) 1 , (cid:96) 2 ) models an insertion if (cid:96)
=
(cid:15) , deletion if (cid:96) 2 = (cid:15) and replacement if (cid:96) 1 (cid:54) =
(cid:96) 2 and neither (cid:96) 1 nor (cid:96) 2 is a dummy.
We assume d to be a binary function, where d ( (cid:96) 1 , (cid:96) 2 )
= 1 if (cid:96) 1 (cid:54) =
(cid:96) 2 , otherwise, 0 .
Deﬁnition 3 (Graph Edit Distance (G ED ))
G ED is the minimum distance under all mappings.
[Equation] [Equation]
Φ( G 1 , G 2 ) denotes the set of all possible node maps from G 1 to G 2 .
Deﬁnition 4 (Subgraph Edit Distance (S ED ))
S ED is the minimum G ED over all subgraphs of G 2 .
[Equation]
Observation 4 (i) G ED ( G 1 , G 2 ) ≥ 0 , (ii) S ED ( G 1 , G 2 ) ≥ 0 .
Observation 5 (i) G ED ( G 1 , G 2 )
=
iff G 1 is isomorphic to G 2 , (ii) S ED ( G 1 , G 2 )
=
iff G 1 is subgraph isomorphic to G 2 .
B Additional Proofs B.1 Proof of Theorem.
Our proof relies on two lemmas.
P ROOF of Theorem 1.
It sufﬁces to prove (i) S ED ( G 1 , G 2 ) ≥ (cid:100)
G ED ( G 1 , G 2 ) and (ii) (cid:100)
G ED ( G 1 , G 2 ) ≥
S ED ( G 1 , G 2 ) .
(i) Let S
=
( V S , E S , L S ) ⊆
G 2 be the subgraph minimizing S ED ( G 1 , S ) (Recall Eq. 14).
Consider the mapping π from G 1 to S corresponding to S ED ( G 1 , S ) (and hence G ED ( G 1 , S ) as well).
We extend π to deﬁne a mapping (cid:98) π from G 1 to G 2 by mapping of all nodes in set V 2 \ V S to dummy nodes in G 1 ; the edge mappings are deﬁned analogously.
Under this construction, S ED ( G 1 , S )
=
G ED ( G 1 , G 2 ) =
(cid:100)
G ED (cid:98) π ( G 1 , G 2 ) ≥ (cid:100)
G ED ( G 1 , G 2 ) .
This follows from the property that under (cid:98) d , insertion costs are zero, that is (cid:98) d
( (cid:15), (cid:96) )
= 0 .
Thus, the additional mappings introduced in (cid:98) π do not incur additional costs under (cid:98) d .
(ii) Consider S ⊆
G 2 and a mapping π from G 1 to S such that G ED π
( G 1 , S )
= (cid:100)
G ED ( G 1 , G 2 ) .
The existence of such a subgraph is guaranteed (See Lemma 3).
From the deﬁnition of G ED ,
G ED π ( G 1 , S ) ≥ G ED ( G 1 , S ) .
Furthermore, since S ⊆ G 2 , G ED ( G 1 , S ) ≥ S ED ( G 1 , G 2 ) .
Combining all these results, we have (cid:100)
G ED ( G 1 , G 2 ) ≥ G ED ( G 1 , S ) ≥ S ED ( G 1 , G 2 ) .
Hence, the claim is proved.
(cid:3)
B.
Proof of
Thm
P ROOF .
From Thm.
1, we know S ED ( G 1 , G 2 ) =
(cid:100)
G ED ( G 1 , G 2 ) .
Combining Obs.
1 with Theorem 1, if we show (cid:98) d ( (cid:96) 1 , (cid:96) 3 ) ≤ (cid:98) d
( (cid:96) 1 , (cid:96) 2 ) + (cid:98) d ( (cid:96) 2 , (cid:96) 3 ) , then the triangle inequality of S ED is established.
We divide the proof into four cases:
(i)
None of (cid:96) 1 , (cid:96) 2 , (cid:96) 3 is (cid:15) .
Hence, (cid:98) d ( (cid:96) 1 , (cid:96) 3 ) =
d ( (cid:96) 1 , (cid:96) 3 ) and the triangle inequality is satisﬁed.
(ii) (cid:96) 1 = (cid:15) .
The L HS is 0 and hence the triangle inequality is satisﬁed.
( iii) (cid:96) 1 (cid:54) =
(cid:15) and (cid:96) 2 = (cid:15) .
L HS ≤ 1 and R HS = 1 .
Hence, satisﬁed.
(iv) Only (cid:96) 3 = (cid:15) .
Here, L HS = 1 and R HS ≥ 1 .
These four cases cover all possible situations and hence, the triangle inequality is established.
(cid:3)
B.3 Proof of Lemma
Lemma 3 There exists a subgraph S of G 2 and a node map π from G 1 to S such that G ED π
( G 1 , S )
= (cid:100)
G ED ( G 1 , G 2 ) .
P ROOF .
Let π (cid:48) be a node map from G 1 to G 2 corresponding to (cid:100) G ED ( G 1 , G 2 ) .
Let h 1 , · · · , h l be the nodes of G 2 which are inserted in π (cid:48) .
Construct subgraph S of G 2 by removing nodes h 1 , · · · , h l and their incident edges from G 2 .
Let π be the node map from G 1 to S which is obtained by removing h (cid:48) 1 , · · · , h (cid:48)
l from the domain and h 1 , · · · , h l from the co-domain of π .
Since insertion costs are 0 in (cid:98) d and π contains only non-insert operations, then G ED π ( G 1 , S )
=
(cid:100)
G ED ( G 1 , G 2 ) .
Hence, the claim is proved.
[Equation]
B.4 Proof of Lemma
P ROOF .
Let x , y ∈ R n be vectors of dimension n .
We use the notation x
[ i ] to denote the i th coordinate of x .
We observe that :
[Equation]
Since (cid:107) .
(cid:107) is monotonic, this implies (cid:107) ReLU ( x ) + ReLU ( y ) (cid:107) ≥ (cid:107) ReLU ( x + y )
(cid:107) .
Using the triangle inequality for (cid:107) .
(cid:107) , we get:
[Equation]
Substituting x = Z G Q − Z G T(cid:48) , y =
Z G T(cid:48)
− Z G T , we get,
[Equation]
ReLU ( Z G Q − Z G T ) (cid:107) .
This implies F ( Z G Q , Z G T(cid:48) )
+ F ( Z G T(cid:48) , Z G T ) ≥ F s ( Z G Q , Z G T ) .
(cid:3)
C Complexity Analysis
For this analysis, we make the simplifying assumption that the hidden dimension in the Pre-M LP ,
G IN and Post-M LP are all d .
The average density of the graph is g .
The number of hidden layers in
Pre-M LP , and Post-M LP are L , and k in G IN .
The computation cost per node for each of these components are as follows.
• Pre-M LP : The operations in the M LP involve linear transformation over the input vector x v of dimension | Σ | , followed by non-linearity.
This results in O ( |V| ( | Σ | · d + d 2 L )) cost.
• G IN : G IN aggregates information from each of the neighbors, which consumes O ( d · g ) time.
The linear transformation consumes an additional O ( d 2 ) time.
Applying non-linearity takes O ( d ) time since it is a linear pass over the hidden dimensions.
Finally these operations are repeated over each of the k hidden layers, results in a total O ( k ( d 2 + dg )) computation time per node.
Across, all nodes, the total cost is O ( |V| kd 2 + |E| kd ) time.
The degree g terms gets absorbed since each edge passes message twice across all nodes.
• Concatenation: This step consumes O ( kd ) time per node.
• Pool:
Pool iterates over the G IN representation of each node requiring O ( |V| dk ) time.
• Post-M LP :
The ﬁnal M LP takes dk dimensional vector as input and maps it to a d dimensional vector over L layers.
This consumes O ( kd 2 + d 2 L ) time.
[Table]
Combining all these factors, the total inference complexity for a graph is O ( |V| ( | Σ |· d + d 2 L + kd 2 )+ |E| kd ) .
This operation is repeated on both the query and target graphs to compute their embeddings, on which distance function F is operated.
Thus, the ﬁnal cost is O ( n ( | Σ | · d + d 2 L + kd 2 ) + mkd ) , where n = |V Q | + |V T | and m =
|E Q | + |E T | .
D Querying in the Embedding Space
We assume the standard querying setup where the database graphs are known apriori, while the query graph is unseen and provided at query time.
Since G REED generates pair-independent embeddings, representations of the database graphs can be generated apriori and stored.
Furthermore, due to both the predicted G ED and S ED satisfying the triangle inequality, the embeddings can be indexed.
Thus, at query time, we need to perform only two operations: (1) embed the query graph G Q , and (2) scan the database embeddings against the query embedding to compute the answer set.
We focus the discussion on the two most common database queries of range and k -NN queries.
Deﬁnition 5 (Range Query)
Given a database D =
{ Z G 1 , · · · , Z G n } of graph embeddings, a query graph G Q and a threshold θ , ﬁnd the answer set A
= {G i | F g ( Z G i , Z G Q ) ≤ θ } .
Deﬁnition 6 ( k -NN Query)
Given a database D =
{ Z G 1 , · · · , Z G n } of graph embeddings, a query graph G Q and k , ﬁnd the database graphs with the k smallest distance to G Q as per F g ( Z G i , Z G Q ) .
The above deﬁnitions can be adopted for S ED by using F s .
For the rest of the discussion, we assume
F s since due to asymmetry, S ED requires some additional considerations.
D.1 Indexing
We exploit the triangle inequality of G ED and S ED to index the database embeddings.
Alg.
1 presents the pseudocode.
We choose a random embedding Z P ∈ D as the pivot (line 2), based on which we split the remaining embeddings into four groups (lines 3-8).
This process continues recursively on each group (lines 9-10) till a partition gets empty (line 1).
Note that in G ED the distances are
[Table]
if F s
Z GQ , Z P ) > m 2 + θ then
[Table]
[Table] [Table]
[Table] [Table]
≤ θ
[Table]
then
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
Name Avg. |V| Avg.
|E| | Σ | #Graphs Avg.
|V Q
| Avg.
|E Q
|
Dblp 1 .
M 7 .
2 M   14 Amazon 334 k 925 k
PubMed 19 .
7 k 44 .
3 k   11 CiteSeer 4 .
2 k 5 .
3 k
[Table]
Cora_
[Table]
3 k 8 .
2 k
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
symmetric and hence m 1 and m 2 will converge.
Consequently, we will have two partitions at each node instead of four.
D.2 Range Query
From the triangle inequality, we can infer the lower bounds listed in lines 2 and 4 of Alg. 2.
Hence, if these bounds are larger than θ , the corresponding sub-trees are pruned.
Similarly, if the upper bound is smaller than θ (line 6), the entire sub-tree is added to the answer set (line 7).
Otherwise, we recurse (lines 11-13).
D.3 k -NN query
k -NN utilizes the same bounds from Alg. 2 to prune and prioritize the search space.
However, exploration proceeds in a best-ﬁrst search manner.
Alg.
3 presents the pseudocode.
Alg.
3 maintains two priority-queues; one to keep track of the k -NN till the current stage of the search process ( A in line 1), and the second to store index nodes in ascending order of their lower bound distance ( Cands in line 2).
We pop the best node P from Cands and include it to the answer set if the distance is within top- k
(lines 6-7).
Further, the sub-tree at P is processed if it satisﬁes the lower bound criteria (lines 8-15).
The search ends when either Cands is either empty or the lower bound of the top-most node is larger than the k -th distance in A (line 4).
In case of G ED , LB 1 (line 8) and LB 2 (line 9) will converge to the same value due to symmetry .
D.4 Scaling S ED to million-scale graphs
S ED is typically encountered in situations where the query is a small graph ( < 50 nodes) [9, 32] and the target graph is a single large graph, potentially containing millions of nodes and edges.
To scale to million-sized target graphs, we perform neighborhood decomposition .
Speciﬁcally, we extract the k -hop neighborhood G v around each node v ∈ V T in the target graph G T , embed them into feature space using G REED , and then indexed as outlined § D.1.
The distance between query G Q and G T is computed as:
[Table]
[Figure]
[Table]
| 9:
LB 2 ← |F s (cid:16) Z GQ , Z P (cid:17) − m 2 | 10:
LB = max { LB 1 ,LB 2 } 11: if LB ≤ θ then 12:
Cands.insert ( (cid:104) t 1 ,LB (cid:105) ) 13:
Cands.insert ( (cid:104) t 2 ,LB (cid:105) )
Cands.insert ( (cid:104) t 3 ,LB (cid:105) ) 15:
Cands.insert ( (cid:104) t 4 ,LB (cid:105) )
Return A
[Equation]
We next show that as long as the k -hop neighborhoods are sufﬁciently large , the proposed neighborhood decomposition strategy is optimal.
Theorem 3 A nearest subgraph S ⊆
G T to G Q can be found in an l/ 2 -hop neighborhood G v ⊆
G T , centered at some v ∈ V G T , where l is the length of the longest path in G Q .
P ROOF .
From Thm.
1, S ED ( G Q , G T ) =
(cid:100)
G ED ( G Q , G T ) .
Let π be the node map produced by (cid:100) G ED ( G Q , G T ) .
Let us now consider the subgraph S ⊆
G T induced by all node maps of π that are not inserts.
It follows that S ED ( G Q , G T ) =
(cid:100)
G ED ( G Q , G T ) =
S ED ( G Q , S ) .
Since there are no inserts, the topology of S is a subgraph of G 1 (i.e., subgraph isomorphic if we ignore label information).
Hence, the diameter of S is ≤ the length l of the longest path in G Q .
Since S ⊆ G T , S is contained in some l/ 2 neighborhood G v of G T .
(cid:3)
Finding the length of the longest path is NP-hard.
However, we do not need to ﬁnd the exact length of the longest path: any upper bound sufﬁces.
Moreover we do not even need the length of the longest path for l .
The nearest subgraph having a diameter equal to l is rare.
In practice, the diameter of the nearest subgraph is unlikely to be much higher than the diameter of the query itself.
E Datasets
Dblp : Dblp is a co-authorship network where each node is an author and two authors are connected by an edge if they have co-authored a paper.
The label of a node is the venue where the authors has published most frequently.
The dataset has been obtained from https://www .
aminer .
org/citation.
Amazon [28]: Each node in Amazon represents a product and two nodes are
connected by an edge if they are frequently co-purchased.
The graph is unlabeled and hence equivalent to a graph containing a single label on all nodes.
PubMed : PubMed
dataset is a citation network which consists of scientiﬁc publications from PubMed database pertaining to diabetes classiﬁed
into
one of
three classes.
Protein :
Protein dataset
consists of protein
graphs
Each node is labeled with
a
one
of
three
functional roles of the
protein.
AIDS : AIDS dataset consists of graphs constructed from the AIDS antiviral screen database.
These graphs representing molecular compounds with Hydrogen atoms omitted.
Atoms are represented as nodes and chemical bonds as edges.
AIDS’ :
AIDS’ dataset is another collection of graphs constructed from the AIDS antiviral screen database.
The graphs and their properties differ from those in AIDS .
These graphs also represent chemical compound structures.
CiteSeer :
CiteSeer is a citation network which consists of scientiﬁc publications classiﬁed into one of six classes.
Generally a smaller version is used for this dataset, but we use the larger version from [8].
Cora_ML :
Cora dataset is a citation dataset consisting of many scientiﬁc publications classiﬁed into one of seven classes based on paper topic.
Cora_ML is a smaller datset extracted from Cora [8].
Linux : Linux dataset is a collection of program dependence graphs, where each graph is a function and nodes represent statements while edges represent dependency between statements.
IMDB : IMDB dataset is a collection of ego-networks of actors/actresses that have appeared together in any movie.
F Baselines
GMN explicitly assumes the distance function to be symmetric, which violates S ED .
G RAPH S IM has an assumption that a large difference in size of the query and target graphs leads to a large distance, which is not true in S ED .
In G OT SIM, there is an explicit assumption of modeling a symmetric distance function since they use cosine similarity to compare node neighborhoods The normalization factor in Eq.
6 of [14] is also based on whole graph matching.
Finally, G ENN -A ∗ is not included for
S ED since it does not scale on graphs beyond 10 nodes (See § 4.3).
[Figure]
G Ablation Study
Impact of G IN :
To highlight the importance of G IN , we conduct ablation studies by replacing the
GIN convolution layers in the model with several other convolution layers.
As visible in the Table B,
G IN consistently achieves the best accuracy.
This is not surprising since G IN is provably the most expressive among G NN s in distinguishing graph structures (essential to S ED or G ED computation) and is as powerful as the Weisfeiler-Lehman Graph Isomorphism test [47]
[Table]
[Table]
Impact of sum-pool: To substantiate our choice of the pooling layer, we have performed ablation studies with various pooling functions as replacements for sum-pool.
It is clear from Table C that sum-pool is the best choice among the considered alternatives.
Sum-pool can better distinguish graph sizes better than other aggregation functions such as mean-pool or max-pool.
To elaborate, let us consider a graph G 1 that is signiﬁcantly larger than another graph
G 2 .
In this scenario, the individual coordinates of G 1 ’s embedding can potentially be signiﬁcantly larger than those of G 2 since in G 1 the summation is being done over a larger set of embeddings.
Both mean-pool and max-pool fail to capture the size information as effectively, since the max and the mean operations do not scale with the number of inputs.
Impact of Pre-mlp layer:
Table D presents the results.
As visible, we do not see any signiﬁcant difference in performance on average.
Performance of G REED -NN and G REED -Dual on G ED : Fig.
F presents the results.
The trends are similar to what we observed for S ED in Sec. 4.5.
G REED -NN closes the performance gap with
G REED as more training data is provided.
Furthermore, G REED -dual is consistently worse that
G REED .
H Alignment
In real-world applications of subgraph similarity search, alignments are of interest only for a small number of similar subgraphs.
Our framework is intended to serve as a ﬁlter to retrieve this small set of similar subgraphs from a large number of candidates.
To elaborate, a graph database may contain
[Table]
With Pre-MLP Without Pre
[Table] [Table]
R M
S E
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
thousands or millions of graphs (or alternatively, thousands or millions of neighborhoods of a large graph) which need to be inspected for similar subgraphs.
A user is typically interested in only a handful of these subgraphs that are highly similar to the query.
Since the ﬁltered set is signiﬁcantly smaller, a non-neural exact algorithm sufﬁces to construct the alignments (Lemma 1 allows us to adapt general cost G ED alignment techniques for S ED alignment).
Computing alignments across the entire database is unnecessary and slows down the query response time.
To substantiate our claim, we show the average running time for answering 10 -NN queries.
We break up the running time into two components: (i) 10 -NN retrieval time by G REED , (ii) exact alignment time using MIP-F2 for the 10 -NN neighborhoods retrieved by G REED .
We observe that exact alignment by existing methods on the 10 -NN neighborhoods completes in reasonable time.
In contrast, G ENN -A ∗ does not scale on either PubMed or Amazon since it computes alignments across all (sub)graphs.
I Visualization
Searching for molecular fragment containment is a routine task in drug discovery [20].
Motivated by this, we show the top- 5 matches to an S ED query on the AIDS dataset produced by G REED in Fig.
E.
The query is a functional group (Hydrogen atoms are not represented).
G REED is able to extract chemical compounds that contain this molecular fragment (except for ranks 3 and 5 , which contain this group with 1 edit) from around 2000 chemical compounds with varying sizes and structures.
This validates the efﬁcacy of G REED at a semantic level.
J Subgraph sampling strategies for S ED generalizability
In R WR , we perform ﬁxed length random walks, where the length of a walk is the average diameter size of the queries generated through BFS during training.
Next, we merge the walks to form a graph.
In R W , we perform random walks, till the diameter of the resultant graph is the same as the average diameter of the BFS sampled train graphs.
The details of the SHA D OW sampler is explained in [48].
K Extension to maximum common subgraph similarity (MCSS)
Besides G ED , MCSS is also a popular similarity measure for whole-graph comparison.
Except the inductive bias injected through F , all components of G REED is generic for any graph distance function.
In this section, we replace F with an M LP and model MCSS.
Table F presents the results.
As visible, the trends hold even in this distance function, where G REED outperforms the closest baseline of H 2 MN.
[Table]
L Heat Maps for Prediction Error
In Figures G to P, we show the variation of the errors on S ED and G ED prediction with query sizes and ground truth values for G REED and the baselines on all the corresponding datasets.
This experiment is an extension of the heat-map results in Fig.
3 in the main paper.
These datasets show variations in the distributions of S ED and G ED values.
It is interesting to observe that among the baselines, different methods perform well on different regions (i.e., combinations of high/low S ED and high/low query sizes).
The baselines do not show good performance on all regions.
However, for G REED , we see a much better coverage for all types of regions in the domain.
Furthermore, for every region, our models outperform (or are at least competitive with) the best performing baseline for that region.
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]