GREED: A Neural Framework for Learning Graph Distance Functions
Abstract
Among various distance functions for graphs, graph and subgraph edit distances(G ED and S ED respectively) are two of the most popular and expressive measures.
Unfortunately, exact computations for both are NP-hard.
To overcome thiscomputational bottleneck, neural approaches to learn and predict edit distance inpolynomial time have received much interest.
While considerable progress hasbeen made, there exist limitations that need to be addressed.
First, the efﬁcacy ofan approximate distance function lies not only in its approximation accuracy, butalso in the preservation of its properties.
To elaborate, although G ED is a metric, itsneural approximations do not provide such a guarantee.
This prohibits their usagein higher order tasks that rely on metric distance functions, such as clustering orindexing.
Second, several existing frameworks for G ED do not extend to S ED dueto S ED being asymmetric.
In this work, we design a novel siamese graph neuralnetwork called G REED , which through a carefully crafted inductive bias, learns
G ED and S ED in a property-preserving manner.
Through extensive experimentsacross 10 real graph datasets containing up to 7 million edges, we establish that
G REED is not only more accurate than the state of the art, but also up to 3 orders ofmagnitude faster.
Even more signiﬁcantly, due to preserving the triangle inequality,the generated embeddings are indexable and consequently, even in a CPU-onlyenvironment, G REED is up to 50 times faster than GPU-powered baselines.
Introduction and Related Work
A distance function on any dataset, including graphs, is a fundamental operator.
Among severaldistance measures on graphs, edit distance is one of the most powerful and popular mechanisms [28,43, 41, 19].
Edit distance can be posed in two forms: graph edit distance (G ED ) and subgraph editdistance (S ED ).
Given two graphs G 1 and G 2 , G ED ( G 1 , G 2 ) returns the minimum cost of edits neededto convert G 1 to G 2 , i.e., for G 1 to become isomorphic to G 2 .
An edit can be the addition or deletion ofedges and nodes, or the replacement of edge or node labels, with an associated cost.
In S ED ( G 1 , G 2 ) ,the goal is to identify the minimum cost of edits so that G 1 is a subgraph ( subgraph isomorphic ) of
G 2 .
For examples, see Fig.
G ED is typically restricted to graph databases containing small graphs to facilitate distance computation with queries of similar sizes.
As an example, given a repository of molecules, and a querymolecule, we may want to identify the closest molecule in the repository that is similar to thequery.
S ED , on the other hand, is useful when the database has large graphs and the query is acomparatively smaller graph.
As examples, subgraph queries are used on knowledge graphs foranalogy reasoning [16].
In PPI and chemical compounds, S ED is of central importance to identifyfunctional motifs and binding pockets [34, 19, 15].
Unfortunately, both G ED and S ED are NP-hardto compute [41, 19].
To mitigate this computational bottleneck, several heuristics [9, 12] and indexstructures [19, 41, 28, 43] have been proposed.
Recently, graph neural networks have been shown tobe effective in learning and predicting G ED [2, 37, 27, 3, 42, 38, 13, 1].
The basic goal in all thesealgorithms is to learn a neural model from a training set of graph pairs and their distances, such that,at inference time, given an unseen graph pair, we are able to predict its distance accurately.
In the domain of subgraphs, N EURO M ATCH [32] and I SO N ET [33] generate embeddings to detect subgraphisomorphism.
N SC [36] generates subgraph level embeddings that can count the number of subgraphinstances of a query graph on a target graph.
While the progress made is impressive, there is scope todo more.
• Preservation of theoretical properties: G ED is a metric distance function.
While S ED is notmetric due to being asymmetric, it satisﬁes the triangle inequality , non-negativity , and subgraphidentity .
Several higher-order tasks such as clustering and indexing rely on such properties [18,14, 20, 35, 11].
Existing neural approaches do not preserve these properties, which limits theirusability for these higher order tasks.•
Indexable embeddings: Given graph pair G 1 and G 2 , neural approaches ﬁrst embed them into afeature space.
Next, they compute a distance on these feature vectors, which is an approximationof the distance in the original graph space.
The literature on indexing range and k -NN queriesover feature vectors is rich [17, 23, 11].
Index structures typically allow sub-linear computationcosts with respect to the database size.
Unfortunately, none of the neural approaches generateindexable feature vectors since they perform pair-dependent computations.
Speciﬁcally, the neuralcomputations on G 1 depend on both G 1 and G 2 (and same for G 2 ).
Consequently, the computationscan only be done at query-time and thereby negating the possibility of indexing pre-computedfeature space embeddings.•
Modeling S ED :
Existing neural approaches to learning G ED cannot easily be adapted to learn S ED .While
G ED is symmetric, S ED is not.
Several neural architectures for G ED have the assumption ofsymmetry at its core and hence modeling S ED is non-trivial [2, 3, 27].• Exponential Search Space:
Computing S ED ( G 1 , G 2 ) conceptually requires us to compare thequery graph G 1 with the exponentially many subgraphs of the target graph G 2 .
Therefore, it isimperative that the model has an efﬁcient and effective mechanism to prune the search spacewithout compromising on the prediction accuracy.
In this work we address the above limitations through the following contributions
• Novel neural architecture:
We address the above
mentioned
challenges through a novel architecture called G REED :
GRaph Embeddings for Edit Distances.
G REED utilizes a siamese graphisomorphism network [39] to embed graphs in a pair-independent fashion.
A simple, but theoretically well-characterized, function on this embedding space predicts the S ED and G ED .
Thecarefully crafted prediction function serves as an inductive bias for the model, which, in additionto enabling high generalization accuracy, preserves the metric property of G ED and the triangleinequality of S ED in the embedding space.• Indexable embeddings: Owing to pair-independent embeddings and preservation of the triangleinequality over the embedding space for both S ED and G ED , G REED can exploit the rich literatureon index structures [17, 23, 11] to boost efﬁciency.• Accurate, Fast and Scalable: Extensive experiments on real graph datasets containing up to amillion nodes establish that G REED is more accurate in both G ED and S ED when compared to thestate of the art algorithms and is more than 3 orders of magnitude faster in range and k -NN queries.
Furthermore, owing to indexable embeddings, even in a CPU-only environment, G REED is up to50 times faster than the closest baseline run on a GPU.
Preliminaries and Problem Formulation
We denote a labeled undirected graph as G =
( V , E , L ) where V is the node set, E is the edge set and
L : V ∪ E → Σ is the labeling function over nodes and edges.
Σ is the universe of all labels andcontains a special empty label (cid:15) .
L ( v ) and L ( e ) denote the labels of node v and edge e respectively.
G
The problem of learning G ED and S ED [19] is deﬁned as follows.
GQ
GT
GNN GNN
Shared Weights
GRAPHS
EMBEDDINGVECTORS
ZQ
ZT
F( Z Q , Z T )PREDICTION (a)
Siamese
[Figure]
k GIN layers
Concatenatefrom all layers
PreMLP
PostMLP
NODEFEATUREVECTORS
POOL
EMBEDDINGVECTOR
[Figure]
Problem 1 (Learning G ED /S ED )
Given a training set of tuples of the form (cid:104)G 1 , G 2 , G ED ( G 1 , G 2 ) (cid:105)(or G 1 , G 2 , S ED ( G 1 , G 2 ) (cid:105) ), learn a neural model to predict G ED ( Q 1 , Q 2 ) (or S ED ( Q 1 , Q 2 ) ) onunseen graphs Q 1 and Q 2 .
For details on the exact deﬁnition of G ED and S ED , we refer to the appendix (App.
A).
2.1 Properties of G ED and S ED
Theorem 1 Let (cid:98) d : Σ × Σ → R +0 be a distance function over Σ , where (i) (cid:98) d ( (cid:96) 1 , (cid:96) 2 )
=
if (cid:96) 1 = (cid:15) ,and (ii) (cid:98) d ( (cid:96) 1 , (cid:96) 2 ) =
d ( (cid:96) 1 , (cid:96) 2 ) otherwise; the following holds: S ED ( G 1 , G 2 ) =
(cid:100)
G ED ( G 1 , G 2 ) , where(cid:100)
G ED denotes G ED with (cid:98) d as the label set distance function.
In simple words, the S ED between twographs is equivalent to G ED with a label set distance function where we ignore insertion costs.
P ROOF .
See App.
B.1.
Observation 1 G ED satisﬁes the triangle inequality if the distance function d over label set Σsatisﬁes the triangle inequality [19].
As deﬁned in the paragraph following Def. 2, d satisﬁes thetriangle inequality [19].
Furthermore, it is trivial to see that G ED is symmetric, non-negative andsatisﬁes identity as long as d satisﬁes the analogues.
Hence, G ED with distance function d is metric.
Theorem 2 S ED is not metric due to violating the properties of symmetry and identity.
However, itsatisﬁes the triangle inequality, i.e., S ED ( G 1 , G 3 )
≤ S ED ( G 1 , G 2 ) + S ED ( G 2 , G 3 ) .
P ROOF : See App.
B.2 for details.
Observation 2 Computing G ED and S ED is NP-hard [41].
Hereon, we use G ED as the illustrative distance function being modeled.
The architecture triviallyextends to S ED .
The speciﬁc places that need separate treatment will be discussed explicitly.
G REED : The Proposed Architecture
[Figure]
Fig.
2 presents the architecture of G REED .
The input to our learning framework is a pair of graphs G Q(query), G T (target) along with the supervision data G ED ( G Q , G T ) (or S ED ( G Q , G T ) ).
Our objectiveis to train a model that can predict G ED on unseen query and target graphs.
The design of our modelmust be cognizant of the fact that computing G ED is NP-hard and high quality training data is scarce.
Thus, we use a Siamese architecture [10], where there are two networks with shared parametersapplied to two inputs independently to compute representations.
3.1 Siamese Graph Neural Network
As depicted in Fig.
2a, we use a siamese graph neural network (G NN ) with shared parameters toembed both G Q and G T .
While one could use two different G NN models for the query and the target,this design increases the model parameters and consequently, the training time.
Furthermore, anarchitecture with higher number of parameters also requires larger amount of training data, which isdifﬁcult due to G ED being NP-hard.
Fig.
2b focuses on the G NN component of G REED .
We next discuss each of its individual components.
Pre-M LP : x v in Fig.
2b is a one-hot encoding of the categorical node labels.
The dimension of theone-hot vector increases linearly with the number of labels in a graph database and hence can bevery large.
The primary-job of the pre-mlp is to reduce it to a desirable dimension size through theoperation µ G v =
M LP ( x v ) .
Indeed, a similar effect may also be obtained by directly feeding the onehot encoding to the ﬁrst layer of G IN .
However, since a G IN constructs embeddings by incorporating both structure and label information, one may desire a different dimensionality in the G IN layers.
Hence, the pre-mlp is motivated more from a conceptual separation of its task to that of G IN ratherthan a purely performance point of view (See App. G).
We do not explicitly model edge labels inour experiments.
G REED can easily be extended to edge labels by using G INE [21] instead of G IN .
Graph Isomorphism Network (G IN ): G IN [39] consumes the information from the Pre-M LPto learn hidden representations that encode both the graph structure as well as the node featureinformation.
G IN is as powerful as the Weisfeiler-Lehman (WL) graph isomorphism test [24] indistinguishing graph structures.
Since our goal is to accurately characterize graph topology and learnsimilarity, G IN emerges as the natural choice.
G IN develops its expressive power by using an injectiveaggregation function.
Speciﬁcally, in the initial layer, each node v in graph G is characterized by therepresentation learned by the M LP , i.e., h G v, 0 = µ G v .
Subsequently, in each hidden layer i , we learnan embedding through the following transformation.
[Equation]
139 Here, (cid:15) i is a layer-speciﬁc learnable parameter, N G ( v ) is one-hop neighbourhood of the node v , andh G v,
= µ G v .
The k -th layer embedding is h G v,k , where k is ﬁnal hidden layer.
Concatenation, Pool and Post-M LP : Intuitively, h G v,i captures a feature-space representation of thei -hop neighborhood of v .
Typically, G NN s operate on node or edge level predictive tasks, such asnode classiﬁcation or link prediction, and hence, the node representations are passed through
an
M LP for the ﬁnal prediction task.
In our problem, we need to capture a graph level representation.
Furthermore, the representation should be rich enough to also capture the various subgraphs withinthe input graph so that S ED can be predicted accurately.
To fulﬁl these requirements, we ﬁrstconcatenate the representation of a node across all hidden layers, i.e., the ﬁnal node embeddingis z G v = C ONCAT (cid:0) h G v,i , ∀ i ∈ { 1 , 2 , · · · , k } (cid:1) .
This allows us to capture a multi-granular view ofthe subgraphs centered on v at different radii in the range [1 , k ] .
Next, to construct the graph-levelrepresentation, we perform a sum-pool, which adds the node representations to give a single vector.
This information is then fed to the Post-M LP to enable post-processing.
Mathematically:
[Equation]
G ED and S ED Prediction: The ﬁnal task is to predict the G ED (and S ED ) as a function of querygraph embedding Z G Q and target graph embedding Z G T .
The natural choice would be to feed theseembeddings into another M LP to learn G ED ( Z G Q , Z G T ) .
This M LP can then be trained jointly withthe graph embedding model in an end-to-end fashion.
However, an M LP prediction does not haveany theoretical guarantees with respect to the preservation of metric properties of G ED and thetriangle inequality of S ED .
We, therefore, focus on learning prediction functions F g ( Z G Q , Z G T ) and
F s ( Z G Q , Z G T ) for G ED and S ED respectively, such that they are accurate and respects the desirableproperties from the original graph space.
As we will empirically substantiate in § 4.5, the inductivebias injected through the prediction functions also lead to more effective learning over low volumesof training data than an MLP.
[Equation] [Equation]
G ED
We require the following four properties to ensure that the prediction is also a metric.
[Equation]
To achieve this, we establish an important connection of metrics on vector spaces to norms.
Everynorm (cid:107) .
(cid:107) gives a metric ( x , y )
(cid:55)→ (cid:107) x − y (cid:107) .
Moreover for a metric, there exists a norm (cid:107) .
(cid:107) suchthat the metric can be expressed as ( x , y )
(cid:55)→ (cid:107) x − y (cid:107) , iff the metric is translation invariant andhomogeneous .
Thus, we add these properties to the desiderata for F g :
[Equation]
Armed with these observations, we deﬁne the class of functions that may be used for F g .
Observation 3 F g may be deﬁned as any function ( x, y )
(cid:55)→ (cid:107) x − y (cid:107) for some norm (cid:107) .
(cid:107) over thevector space R d such that F g satisﬁes Eqs.
The L p norm satisﬁes Obs.
Hence, we deﬁne F g as:
[Equation]
In our implementation we use the L 2 norm.
Finally, the parameters of the entire model are learned byminimizing the mean squared error (here T is the training set).
[Equation]
Intuition:
Regardless of the graph representations generated by our model, F g ensures that thepredicted distance is a metric.
One the other hand, by training the model to produce embeddings
Z G Q and Z G T such that F g ( Z G Q , Z G T )
≈ G ED ( Z G Q , Z G T ) , we enforce a rich structure on theembedding space such that F g is also accurate.
Thus, F g injects an inductive bias satisfying the dualneeds of accuracy and preservation of original space properties.
S
ED
S ED satisﬁes non-negativity and triangle inequality.
Following a similar reasoning as above, wedeﬁne F s as follows:
[Equation]
Intuitively, for those co-ordinates where the value of Z G Q is greater than Z G T , a distance penalty isaccounted by F s in terms of how much those values differ; otherwise F s considers 0 .
This followsthe intuition that the S ED accounts for those features of G Q that are not in G T .
Moreover, consistentwith S ED , the additional features in G T that are not in G Q , do not incur any cost.
Lemma
The following properties hold on predicted S ED .
[Equation]
P ROOF .
Properties (1) and (2) follow from the deﬁnition of F itself.
Property (3) follows from thefact that we take the L 2 norm.
Formally, we state it as follows.
Lemma 2 F s ( Z G Q , Z G T ) ≤ F ( Z G Q , Z G T(cid:48) )
+ F ( Z G T(cid:48) , Z G T ) , where F ( x , y ) = (cid:107) ReLU ( x − y )
(cid:107)for any monotonic norm (cid:107) .
(cid:107) .
P ROOF .
See App.
B.4.
3.2 Characterization of G REED
Importance of pair-independence and siamese architecture: If we do not use a siamese architecture, then the embedding model for the query and the target graphs would be different.
Hence, thepredicted distance would violate symmetry, and therefore, would not be a metric.
Furthermore, if thedistance computations are pair-dependent [27, 42, 2], i.e., it jointly learns the embedding of the queryand the target, then a single graph may correspond to multiple representations.
Hence, it would notbe a metric or satisfy the triangle inequality.
Complexity Analysis: The complexity of G ED and S ED inference in G REED is linear to the numberof nodes and edges in the query and target graphs (See App.
C for derivation).
This computation costis drastically lower than the factorial computation cost of optimal G ED and S ED .
With respect toneural methods for graph similarity [2, 3, 27, 37, 42], all have at least quadratic computation cost,i.e., O ( |V| 2 ) .
Indexing Embeddings: Since the generated embeddings for both G ED and S ED satisfy triangleinequality, they are indexable leading to fast querying times.
We develop an index structure to exploitthis property.
Due to space limitations, the details are included in App.
D.
In addition, we alsodesign a neighborhood decomposition scheme, which enables fast pruning of the exponential searchspace § D.4.
In § 4.3, we empirically analyze the impact of index structures on querying time.
Empirical Evaluation
In this section, we establish the following:
• Efﬁcacy: G REED is more accurate than the state of the art approaches for both G ED and S ED .
• Efﬁciency: G REED is orders of magnitude faster than existing approaches and scales well tographs with millions of nodes.
• Scalability: Pair-independence and indexability further enhances the scalability of G REED andenables it to be run on CPU-only platforms.
Our code base and datasets are available at https://anonymous .
4open .
science/r/greed/.
4.1 Experimental Setup
We use a machine with an Intel Xeon Gold 6142 processor and GeForce GTX 1080 Ti GPU for allour experiments.
Datasets: Table A lists the datasets used for benchmarking.
Further details on the dataset semanticsare provided in the App.
E. We include a mixture of both graph databases (#graphs >1), as well assingle large graphs (#graphs = 1).
Linux and IMDB are unlabeled.
We note that this is the ﬁrst studyto evaluate neural graph distance approaches on million-scale graphs.
Baselines: To evaluate performance in G ED , we compare with S IM G NN [2], G ENN -A ∗ [37],
H 2 MN[42] and G OT SIM [13].
These are the most recent neural frameworks and have shown betterefﬁcacy than other neural approaches such as S IM G NN [2], G RAPH S IM [3], and GMN [27].
For S ED , no neural approaches exist.
However, H 2 MN and S IM G NN can be trained by replacing
G ED with S ED along with minor modiﬁcations in training.
N SC [29] is a method for countingsubgraphs using graph embeddings.
Since this is a related operation, we use N SC as a baseline bychanging the loss function to minimize the RMSE between true and predicted S ED .
We also use
N EURO M ATCH [32] as a baseline, which was originally designed to detect subgraph isomorphism.
While N EURO M ATCH cannot predict S ED , it generates a violation score , which can be interpreted asthe likelihood of the query being subgraph isomorphic to the target.
The violation score can be used asa proxy for S ED and used in ranking of k -NN ( k -NearestNeighbour) queries.
Thus, N EURO M ATCHcomparisons are limited to k -NN queries on S ED .
GMN [27], G RAPH S IM [3], G OT SIM [13] and
G ENN -A ∗ are not included since they cannot be easily adapted for S ED .
See App. F for details.
In the non-neural category, we use mixed integer programming based method M IP -F2 [25] with atime bound of 0 .
1 seconds per pair for both G ED and S ED .
M IP -F2 provides the optimal solutiongiven inﬁnite time.
We also compare with B RANCH [4], which achieves an excellent trade-offbetween accuracy and time [5].
B RANCH uses linear sum assignment problem with error-correction(L SAPE ) to process the search space.
We use G EDLIB ’s
[6] implementation of these methods.
Training (and Test) Data Generation:
For G ED , we use (cid:104) query, target (cid:105) graph pairs from IMDB ,
AIDS’ , and Linux .
Our setup is identical to S IM G NN [2] and H 2 MN [42].
For S ED , the target graphs are taken from datasets listed in Table A.
For the query graph, in AIDS ,we use known functional groups [31].
In the rest of the graph datasets, queries are sampled byperforming a random BFS traversal (depth up to 5).
Table A shows the average query sizes ( |V Q | ,|E Q | ).We use mixed integer programming method F2 [25] implemented in G EDLIB [6] with a largetime limit to generate ground-truth data.
Train-Validation-Test: We use 100 K query-target pairs for training and 10 K pairs each for validation and test.
All models are trained till validation loss is minimized or there is less than 0 .
05%change in validation loss over a number of extended epochs.
The codebase of all baselines have beenobtained from the respective authors.
For all baselines, we use the default parameters suggested bythe authors.
We also conducted experiments on the datasets to ensure that the default parametersindeed provide the best performance.
For G REED , we set the number of layers in G IN to 8 .
Thehidden layer dimension is set to 64 .
4.2 Prediction Accuracy of S ED and G ED
Tables 1a and 1b present the accuracy of all techniques on G ED and S ED in terms of Root Mean
Square Error (RMSE ).
G REED outperforms all other techniques in 9 out of 10 settings across G EDand S ED .
While H 2 MN and N SC are the second best performers in S ED , G ENN -A ∗ performs well in
G ED .
G ENN -A ∗ , however, is extremely slow and does not scale on graphs of size beyond 10 .
H 2 MN,thus, provides the second best balance between efﬁcacy and efﬁciency after G REED .
The gap inaccuracy is the highest in IMDB for G ED , where G REED is more than 10 times better than the neuralbaselines. IMDB graphs are signiﬁcantly denser and larger than AIDS’ or Linux . Thus, computing
Methods AIDS’ Linux
IMDB
G REED
[Table] [Table]
H
[Table]
MN 0.
[Table]
G ENN -A ∗
[Table]
NA
[Table]
SIM
[Table] [Table]
[Figure] [Figure]
40 SED
Q u e r y
S i z e
[Figure]
40 SED      (
[Figure]
H 2 MN
[Figure] Dblp the optimal G ED is harder.
While all techniques have higher errors in IMDB , the deterioration is moresevere in the baselines indicating that G REED scales better with graph sizes.
Impact of Query Size: We next investigate how the accuracy varies against the query size.
Intuitively, the task gets harder with query size since the combinatorial space of possible maps increasesexponentially.
For this analysis, we compare G REED with H 2 MN in IMDB and Dblp for G ED and
S ED respectively.
G ENN -A ∗ fails to scale on both datasets.
In Fig.
3, we plot the heat map of RMSE against query graph size.
In this plot, each dot correspondsto a query graph G Q .
The co-ordinate of a query is ( G ED ( G Q , G T ) , |V Q | ) (analogously deﬁned for
S ED ).
The color of a dot represents the RMSE; the darker the color, the higher is the RMSE.
Whenwe compare the heat maps of G REED with H 2 MN, we observe that H 2 MN is noticeably darker.
Furthermore, the concentration of dark colors is noticeably higher on the upper-right corner indicatingdeterioration with larger query sizes and higher distance values.
This indicates that G REED scalesbetter with query sizes and distances.
Visualization: A case study to visually illustrate the efﬁcacy of G REED is provided in App.
I.
Range and k -NN queries: To quantify performance in range and k -NN queries, we measure F1score (Range query) and Kendalls’s tau ( k -NN)
[22] of the predicted answer set, when comparedagainst the ground truth.
In Figs.
4a-4h, we measure the performance in range queries.
In S ED ,
G REED consistently outperforms all baselines in F1-score.
In G ED , the trend remains similar.
Although, G ENN -A ∗ outperforms G REED for a brief region in Linux , overall, G REED has the highest
F1-score.
We also note the G ENN -A ∗ is not included in Fig.
4h since it fails to scale on IMDB .
Ink -NN queries (Tables 2a and 2b), G REED outperforms all algorithms in S ED .
In G ED , similar tothe trend in range queries, G REED is the dominant method and G ENN -A ∗ marginally outperforms
G REED in Linux .
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table] [Figure] [Table] [Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
4.3 Efﬁciency
Tables 3a-3b present the inference times per 10 K query-target pairs.
In this experiment, we do notindex embeddings by G REED so that the comparison unearths the raw difference in computationefﬁciency of solely the neural architectures.
As visible, G REED is up to 1800 times faster than thenon-neural baselines and up to 10 to 20 times faster than H 2 MN, the current state of the art in G EDprediction.
Also note that G ENN -A ∗ is exorbitantly slow (Table 3a).
G ENN -A ∗ is slower since it not [Figure] only predicts the G ED but also the alignment via an A ∗ search.
While the alignment information isindeed useful, computing this information across all graphs in the database may generate redundantinformation since an user is typically interested only on a small minority of graphs that are in theanswer set.
In App.
H, we discuss this issue in detail
[Figure] [Table]
[Table]
REED (times differ based on whether executed on CPU or GPU).
4. Pair-independence and Indexability
Here, we showcase how pair-independent embeddings, and ensuring triangle inequality leads tofurther boost in scalability.
For this experiment, we use the three largest datasets of PubMed , Amazonand Dblp .
For each dataset, we pre-compute G REED embeddings of all database graphs by exploitingpair-independent embeddings.
Such pre-computation is not possible in the neural or non-neuralbaselines.
Furthermore, since the predictions of G REED satisfy triangle inequality, we index thepre-computed embeddings of the database graphs as discussed in § D.1.
Consequently, for G REED ,we only need to embed the query graph and evaluate F to make predictions at query time.
Table 3cpresents the results on range and 10 -NN queries.
When computations are done on a GPU, G REED ismore than 1000 times faster than H 2 MN.
In the absence of a GPU, H 2 MN is practically infeasiblesince expensive pair-dependent computations are done at query time.
In contrast, even on a CPU,through indexing, G REED is ≈ 50 times faster than GPU-based H 2 MN.
Note that indexing enablesup to 3 -times speed-up on G REED over linear scan, which demonstrates the gain from ensuringtriangle inequality.
These results establish that G REED breaks new ground in scalability of neuralgraph distance computations; not only is it faster, it overcomes the barrier of GPU-dependence andhence better suited for low-resource environments.
4.5 Ablation Study
In this study, we explore the impact of our inductive biases in learning from low-volume data.
Wecreate two variants of G REED :
(1) G REED -Dual trains the two parallel G NN models separatelywithout weight-sharing, and (2) G REED -NN uses an MLP instead of F .
Both have strictly betterrepresentational capacity than G REED , so are expected to match the performance with inﬁnite data.
Figs.
4i-4k present the results on S ED .
The results of the same experiment on G ED is provided in
Figs.
F in the appendix.
The RMSE of G REED is generally better than G REED -Dual, with the difference being more signiﬁcant at low volumes.
This indicates that siamese structure helps.
Comparedto G REED , G REED -NN achieves marginally better performance at larger train sizes in PubMed and
CiteSeer .
However, in Dblp , G REED is consistently better.
The number of subgraphs in a datasetgrows exponentially with the node set size.
Hence, an MLP needs growing training data to accuratelymodel the intricacies of this search space.
In Dblp , even 100k pairs is not enough to improve upon F .
Furthermore, since computing G ED and S ED is NP-hard, generating large volumes of training data isnot desirable.
Overall, these trends indicate that F enables better generalization and scalability withrespect to accuracy.
Furthermore, given that its performance is close to an MLP even on high-volumetraining data, and it enables indexing, the beneﬁts outweigh the marginal reduction in accuracy.
We also observe that generally, G REED -NN performs better than G REED -dual.
G REED -NN retainsthe inductive bias imparted by the Siamese architecture, but ablates the inductive bias of the customprediction function.
The opposite happens in case of G REED -Dual.
The observed phenomenon of
G REED -NN generally out-performing G REED -Dual can be interpreted as evidence for the Siamesearchitecture providing a stronger inductive bias than the custom prediction function.
More ablations studies justifying our choice of G IN and the sum-pool layer are provided in App.
G.
[Figure]
4.6 Generalization to Unseen Query Distributions in S ED
We train the model by sampling queries from the graph database through BFS enumerations.
Howdoes G REED generalize to unseen distributions?
Towards that end, we generate queries from thethree unseen distributions of (1) Random Walks (R W ), (2) Random Walks with Restarts (R WR ), and(3) SHA D OW [40] (See App.
J for details on the sampling strategies).
We ﬁrst note that in AIDS ,we use real queries of functional groups, and thus the good performance in AIDS indicates goodgeneralizability.
In Table 4a, we more exhaustively analyze this aspect.
As visible, the errors remainlow.
Even more surprisingly, the errors on R W and R WR are better than the train distribution of BFSitself.
This indicates good generalization to unseen distributions.
4.7 Generalizability to Unseen, Larger Query Sizes:
training data for learning G ED and S ED is expensive since optimal distance computationsare NP-hard.
Hence, a desirable property would be to learn from small graphs and then generalize tolarger unseen graphs.
We evaluate this ability for G REED and H 2 MN.
Table 4b provides the numbers.
We notice that although there is some deterioration in the quality for query sizes in the range [25 , 50]when compared to the entire set, it is not severe (G REED -50 in Table 4b).
However, if the train setonly contains queries till size 25 and we deploy the learned model to infer on queries of larger unseensizes, the drop in quality is signiﬁcant (G REED -25 in Table 4b).
This drop is even more dramatic in
H 2 MN.
On the positive side, G REED remains superior to the optimal non-neural approach (MIP-F2)when run with a generous time limit of 60 seconds per query.
Overall, this experiment highlights onedirection that needs further study and improvement.
Conclusions, Limitation, and Future Directions
The problem of learning graph distances from their embeddings has seen much interest over thelast few years.
This thread of research is important since it allows us to overcome the bottleneck ofexponential graph alignment space.
Our experiments clearly establish G REED as the state of the artfor both G ED and S ED (See App.
K for preliminary results on maximum common subgraph similarity ).
In addition, it is signiﬁcantly faster and provides better theoretical correspondence between propertiesof the original space and predicted space.
One clear direction of future work that emerges from ourexperiments is that G REED , and existing methods of graph distance learning, do not generalize wellto unseen larger query sizes.
We hope to address this limitation next.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section 5(c)
Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] For additional proofs, check
Appendix B.
If you ran experiments...
[Table]
(a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
[Yes]
[Table]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?
[Yes] For additional details please check Appendix.(c)
Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
[No]
[Table]
(d) Did you include the total amount of compute and the type of resources used (e.g., type ofGPUs, internal cluster, or cloud provider)?
[Yes] Please check Sec.4.1.
[Table]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes]
[Table]
(b) Did you mention the license of the assets?
[Yes] MIT License
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] Linkto codebase in § 4(d)
Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁableinformation or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review Board
[Table]
(IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on
[Table]
participant compensation?
[N/A]
[Table]
AppendixA G ED and S ED
The computation of G ED relies on a graph mapping .
Deﬁnition 1 (Graph Mapping)
Given two graphs G 1 and G 2 , let ˜ G 1 =
( ˜ V 1 , ˜ E 1 , ˜ L 1 ) and ˜ G 2 =(
˜
V 2 , ˜ E 2 , ˜ L 2 ) be obtained by adding dummy nodes and edges (labeled with (cid:15) ) to G 1 and G 2 respectively,such that |V 1 | =
|V 2 | and |E 1 | = |E 2 | .
A node mapping between G 1 and G 2 is a bijection π : ˜ G 1 → ˜ G 2where (i) ∀ v ∈
˜ V 1 , π ( v ) ∈
˜ V 2 and at least one of v and π ( v ) is not a dummy; (ii) ∀ e = ( v 1 , v 2 ) ∈˜ E 1 , π ( e ) =
( π ( v 1 ) , ( π ( v 2 ))) ∈
˜
E 2 and at least one of e and π ( e ) is not a dummy.
Example 1 Fig.
1 shows a graph mapping.
Edge mappings can be trivially inferred.
Deﬁnition 2 (Graph Edit Distance (G ED ) under mapping π ) G ED between G 1 and G 2 under π is
[Equation]
V
[Equation]
( L ( v )
[Equation]
L (
[Equation]
( v )))
[Equation]
(cid:88) e ∈ ˜
Ed ( L ( e ) , L ( π ( e ))) (12) where d : Σ × Σ → R +0 is a distance function over the label set.
d ( (cid:96) 1 , (cid:96) 2 ) models an insertion if(cid:96) 1 = (cid:15) , deletion if (cid:96)
=
(cid:15) and replacement if (cid:96) 1 (cid:54) =
(cid:96) 2 and neither (cid:96) 1 nor (cid:96) 2 is a dummy.
We assume d to be a binary function, where d ( (cid:96) 1 , (cid:96) 2 )
= 1 if (cid:96) 1 (cid:54) =
(cid:96) 2 , otherwise, 0 .
Deﬁnition 3 (Graph Edit Distance (G ED ))
G ED is the minimum distance under all mappings.
[Equation]
(13) Φ( G 1 , G 2 ) denotes the set of all possible node maps from G 1 to G 2 .
Deﬁnition 4 (Subgraph Edit Distance (S ED ))
S ED is the minimum G ED over all subgraphs of G 2 .
[Equation]
Observation 4 (i) G ED ( G 1 , G 2 ) ≥ 0 , (ii) S ED ( G 1 , G 2 ) ≥ 0 .
Observation 5 (i) G ED ( G 1 , G 2 )
=
iff G 1 is isomorphic to G 2 , (ii) S ED ( G 1 , G 2 )
=
iff G 1 issubgraph isomorphic to G 2 .
B Additional Proofs
B.1 Proof of Theorem.
Our proof relies on two lemmas.
P ROOF of Theorem 1.
It sufﬁces to prove (i) S ED ( G 1 , G 2 ) ≥ (cid:100)
G ED ( G 1 , G 2 ) and (ii) (cid:100)
G ED ( G 1 , G 2 ) ≥
S ED ( G 1 , G 2 ) .(i)
Let S
=
( V S , E S , L S ) ⊆
G 2 be the subgraph minimizing S ED ( G 1 , S ) (Recall Eq. 14).
Considerthe mapping π from G 1 to S corresponding to S ED ( G 1 , S ) (and hence G ED ( G 1 , S ) as well).
Weextend π to deﬁne a mapping (cid:98) π from G 1 to G 2 by mapping of all nodes in set V 2 \ V S to dummynodes in G 1 ; the edge mappings are deﬁned analogously.
Under this construction, S ED ( G 1 , S )
=
G ED ( G 1 , G 2 ) =(cid:100)
G ED (cid:98) π ( G 1 , G 2 ) ≥ (cid:100)
G ED ( G 1 , G 2 ) .
This follows from the property that under (cid:98) d , insertion costs are zero,that is (cid:98) d
( (cid:15), (cid:96) )
= 0 .
Thus, the additional mappings introduced in (cid:98) π do not incur additional costs under(cid:98)
d .(ii) Consider S ⊆
G 2 and a mapping π from G 1 to S such that G ED π
( G 1 , S )
= (cid:100)
G ED ( G 1 , G 2 ) .
The existence of such a subgraph is guaranteed (See Lemma 3).
From the deﬁnition of G ED ,
G ED π ( G 1 , S ) ≥ G ED ( G 1 , S ) .
Furthermore, since S ⊆ G 2 , G ED ( G 1 , S ) ≥ S ED ( G 1 , G 2 ) .
Combiningall these results, we have (cid:100)
G ED ( G 1 , G 2 ) ≥ G ED ( G 1 , S ) ≥ S ED ( G 1 , G 2 ) .
Hence, the claim is proved.
(cid:3)
B.2 Proof
of
Thm
P ROOF .
From Thm.
1, we know S ED ( G 1 , G 2 ) =
(cid:100)
G ED ( G 1 , G 2 ) .
Combining Obs.
1 with Theorem 1,if we show (cid:98) d ( (cid:96) 1 , (cid:96) 3 ) ≤ (cid:98) d ( (cid:96) 1 , (cid:96) 2 ) + (cid:98) d ( (cid:96) 2 , (cid:96) 3 ) , then the triangle inequality of S ED is established.
Wedivide the proof into four cases:
(i)
None of (cid:96) 1 , (cid:96) 2 , (cid:96) 3 is (cid:15) .
Hence, (cid:98) d ( (cid:96) 1 , (cid:96) 3 ) =
d ( (cid:96) 1 , (cid:96) 3 ) and the triangle inequality is satisﬁed.(ii)
(cid:96) 1 = (cid:15) .
The L HS is 0 and hence the triangle inequality is satisﬁed.
( iii) (cid:96) 1 (cid:54) =
(cid:15) and (cid:96) 2 = (cid:15) .
L HS ≤ 1 and R HS = 1 .
Hence, satisﬁed.
(iv) Only (cid:96) 3 = (cid:15) .
Here, L HS = 1 and R HS ≥ 1 .
These four cases cover all possible situations and hence, the triangle inequality is established.
(cid:3)
B.3 Proof of Lemma
Lemma 3 There exists a subgraph S of G 2 and a node map π from G 1 to S such that G ED π
( G 1 , S ) =(cid:100)
G ED ( G 1 , G 2 ) .
P ROOF .
Let π (cid:48) be a node map from G 1 to G 2 corresponding to (cid:100) G ED ( G 1 , G 2 ) .
Let h 1 , · · · , h l be thenodes of G 2 which are inserted in π (cid:48) .
Construct subgraph S of G 2 by removing nodes h 1 , · · · , h l andtheir incident edges from G 2 .
Let π be the node map from G 1 to S which is obtained by removingh (cid:48) 1 , · · · , h (cid:48)
l from the domain and h 1 , · · · , h l from the co-domain of π .
Since insertion costs are 0 in (cid:98) dand π contains only non-insert operations, then G ED π ( G 1 , S ) =
(cid:100)
G ED ( G 1 , G 2 ) .
Hence, the claim isproved.
(cid:3)
B.4 Proof of Lemma 2.
P ROOF .
Let x , y ∈ R n be vectors of dimension n .
We use the notation x
[ i ] to denote the i thcoordinate of x .
We observe that :
[Equation]
Since (cid:107) .
(cid:107) is monotonic, this implies (cid:107) ReLU ( x ) + ReLU ( y ) (cid:107) ≥ (cid:107) ReLU ( x + y )
(cid:107) .
Using the triangleinequality for (cid:107) . (cid:107) , we get:(cid:107) ReLU ( x ) (cid:107) + (cid:107) ReLU ( y ) (cid:107) ≥ (cid:107) ReLU ( [Equation]
+ [Equation] [Equation] (cid:107)
[Equation]
(cid:107)
[Equation]
( x [Equation] y ) (cid:107) (15) Substituting x = Z G Q − Z G T(cid:48) , y =
Z G T(cid:48)
− Z G T , we get,(cid:107) ReLU ( Z [Equation] Q − Z G
[Equation]
) (cid:107) + (cid:107) ReLU ( Z G T(cid:48)
− Z G T ) (cid:107) ≥ (cid:107) ReLU ( Z G Q − Z G T ) (cid:107) .
This implies F ( Z G Q , Z G T(cid:48) )
+ F ( Z G T(cid:48) , Z G T ) ≥ F s ( Z G Q , Z G T ) .
(cid:3)
C Complexity Analysis
For this analysis, we make the simplifying assumption that the hidden dimension in the Pre-M LP ,
G IN and Post-M LP are all d .
The average density of the graph is g .
The number of hidden layers in
Pre-M LP , and Post-M LP are L , and k in G IN .
The computation cost per node for each of these components are as follows.
• Pre-M LP : The operations in the M LP involve linear transformation over the input vector x v of
dimension | Σ | , followed by non-linearity.
This results in O ( |V| ( | Σ | · d + d 2 L )) cost.
• G IN : G IN aggregates information from each of the neighbors, which consumes O ( d · g ) time.
The
linear transformation consumes an additional O ( d 2 ) time.
Applying non-linearity takes O ( d ) time
since it is a linear pass over the hidden dimensions.
Finally these operations are repeated over each
of the k hidden layers, results in a total O ( k ( d 2 + dg )) computation time per node.
Across, all
nodes, the total cost is O ( |V| kd 2 + |E| kd ) time.
The degree g terms gets absorbed since each edge
passes message twice across all nodes.
• Concatenation: This step consumes O ( kd ) time per node.
• Pool: Pool iterates over the G IN representation of each node requiring O ( |V| dk ) time.
• Post-M LP :
The ﬁnal M LP takes dk dimensional vector as input and maps it to a d dimensional
[Table]
Combining all these factors, the total inference complexity for a graph is O ( |V| ( | Σ |· d + d 2 L + kd 2 )+|E| kd ) .
This operation is repeated on both the query and target graphs to compute their embeddings,on which distance function F is operated.
Thus, the ﬁnal cost is O ( n ( | Σ | · d + d 2 L + kd 2 ) + mkd ) ,where n = |V Q | + |V T | and m =
|E Q | + |E T | .
D Querying in the Embedding Space
We assume the standard querying setup where the database graphs are known apriori, while the querygraph is unseen and provided at query time.
Since G REED generates pair-independent embeddings,representations of the database graphs can be generated apriori and stored.
Furthermore, due to boththe predicted G ED and S ED satisfying the triangle inequality, the embeddings can be indexed.
Thus,at query time, we need to perform only two operations: (1) embed the query graph G Q , and (2) scanthe database embeddings against the query embedding to compute the answer set.
We focus thediscussion on the two most common database queries of range and k -NN queries.
Deﬁnition 5 (Range Query)
Given a database D =
{ Z G 1 , · · · , Z G n } of graph embeddings, a querygraph G Q and a threshold θ , ﬁnd the answer set A
= {G i | F g ( Z G i , Z G Q ) ≤ θ } .
Deﬁnition 6 ( k -NN Query)
Given a database D =
{ Z G 1 , · · · , Z G n } of graph embeddings, a querygraph G Q and k , ﬁnd the database graphs with the k smallest distance to G Q as per F g ( Z G i , Z G Q ) .
The above deﬁnitions can be adopted for S ED by using F s .
For the rest of the discussion, we assume
F s since due to asymmetry, S ED requires some additional considerations.
D.1 Indexing
We exploit the triangle inequality of G ED and S ED to index the database embeddings.
Alg.
1 presentsthe pseudocode.
We choose a random embedding Z P ∈ D as the pivot (line 2), based on whichwe split the remaining embeddings into four groups (lines 3-8).
This process continues recursivelyon each group (lines 9-10) till a partition gets empty (line 1).
Note that in G ED the distances are
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
Name Avg. |V| Avg.
|E| | Σ | #Graphs Avg.
|V Q
| Avg.
|E Q
|
Dblp 1 .
M 7 .
2 M   14 Amazon 334 k 925 k
PubMed 19 .
7 k 44 .
3 k   11 CiteSeer 4 .
2 k 5 .
3 k   12 Cora_ML 3 k 8 .
2 k
Protein   1 , 11 AIDS 151 , 7 AIDS’ 9  9 Linux   1 , [Table] symmetric and hence m 1 and m 2 will converge.
Consequently, we will have two partitions at eachnode instead of four.
D.2 Range Query
From the triangle inequality, we can infer the lower bounds listed in lines 2 and 4 of Alg. 2.
Hence, ifthese bounds are larger than θ , the corresponding sub-trees are pruned.
Similarly, if the upper boundis smaller than θ (line 6), the entire sub-tree is added to the answer set (line 7).
Otherwise, we recurse(lines 11-13).
D.3 k -NN query
k -NN utilizes the same bounds from Alg. 2 to prune and prioritize the search space.
However,exploration proceeds in a best-ﬁrst search manner.
Alg.
3 presents the pseudocode.
Alg.
3 maintainstwo priority-queues; one to keep track of the k -NN till the current stage of the search process ( A inline 1), and the second to store index nodes in ascending order of their lower bound distance ( Candsin line 2).
We pop the best node P from Cands and include it to the answer set if the distance iswithin top- k (lines 6-7).
Further, the sub-tree at P is processed if it satisﬁes the lower bound criteria(lines 8-15).
The search ends when either Cands is either empty or the lower bound of the top-mostnode is larger than the k -th distance in A (line 4).
In case of G ED , LB 1 (line 8) and LB 2 (line 9) willconverge to the same value due to symmetry .
D.4 Scaling S ED to million-scale graphs
S ED is typically encountered in situations where the query is a small graph ( < 50 nodes) [8, 30] andthe target graph is a single large graph, potentially containing millions of nodes and edges.
To scaleto million-sized target graphs, we perform neighborhood decomposition .
Speciﬁcally, we extract thek -hop neighborhood G v around each node v ∈ V T in the target graph G T , embed them into featurespace using G REED , and then indexed as outlined § D.1.
The distance between query G Q and G T iscomputed as:
[Table]
P (cid:17)(cid:69)(cid:17) 8:
LB 1 ← |F s (cid:16) Z P , Z GQ (cid:17) − m 1 | 9:
LB 2 ← |F s (cid:16) Z GQ , Z P (cid:17) − m 2 | 10:
LB = max { LB 1 ,LB 2 } 11: if LB ≤ θ then 12:
Cands.insert ( (cid:104) t 1 ,LB (cid:105) ) 13:
Cands.insert ( (cid:104) t 2 ,LB (cid:105) )
Cands.insert ( (cid:104) t 3 ,LB (cid:105) ) 15:
Cands insert ( (cid:104) t 4 ,LB (cid:105) ) 16:
Return A
[Equation]
We next show that as long as the k -hop neighborhoods are sufﬁciently large , the proposed neighborhood decomposition strategy is optimal.
Theorem 3 A nearest subgraph S ⊆
G T to G Q can be found in an l/ 2 -hop neighborhood G v ⊆
G T ,centered at some v ∈ V G T , where l is the length of the longest path in G Q .
P ROOF .
From Thm.
1, S ED ( G Q , G T ) =
(cid:100)
G ED ( G Q , G T ) .
Let π be the node map produced by(cid:100) G ED ( G Q , G T ) .
Let us now consider the subgraph S ⊆
G T induced by all node maps of π that arenot inserts.
It follows that S ED ( G Q , G T ) =
(cid:100)
G ED ( G Q , G T ) =
S ED ( G Q , S ) .
Since there are no inserts,the topology of S is a subgraph of G 1 (i.e., subgraph isomorphic if we ignore label information).
Hence, the diameter of S is ≤ the length l of the longest path in G Q .
Since S ⊆ G T , S is containedin some l/ 2 neighborhood G v of G T .
(cid:3)
Finding the length of the longest path is NP-hard.
However, we do not need to ﬁnd the exact length ofthe longest path: any upper bound sufﬁces.
Moreover we do not even need the length of the longestpath for l .
The nearest subgraph having a diameter equal to l is rare.
In practice, the diameter of thenearest subgraph is unlikely to be much higher than the diameter of the query itself.
E Datasets
Dblp : Dblp is a co-authorship network where each node is an author and two authors are connectedby an edge if they have co-authored a paper.
The label of a node is the venue where the authors haspublished most frequently.
The dataset has been obtained from https: //www .
aminer
org/citation.
Amazon [26]: Each node in Amazon represents a product and two nodes are connected by an edge ifthey are frequently co-purchased.
The graph is unlabeled and hence equivalent to a graph containinga single label on all nodes.
PubMed : PubMed dataset is a citation network which consists of scientiﬁc publications from PubMeddatabase pertaining to diabetes classiﬁed into one of three classes.
Protein : Protein dataset consists of protein graphs.
Each node is labeled with a one of threefunctional roles of the protein.
AIDS : AIDS dataset consists of graphs constructed from the AIDS antiviral screen database.
Thesegraphs representing molecular compounds with Hydrogen atoms omitted.
Atoms are represented asnodes and chemical bonds as edges.
AIDS’ :
AIDS’ dataset is another collection of graphs constructed from the AIDS antiviral screendatabase.
The graphs and their properties differ from those in AIDS .
These graphs also representchemical compound structures.
CiteSeer
CiteSeer is a
citation network
which
consists of
scientiﬁc publications classiﬁed
into
one of six classes.
Generally
a smaller version is used for
this
dataset,
but we use the
larger version
from [7].
Cora_ML : Cora dataset is a citation dataset consisting of many scientiﬁc publications classiﬁed into
one of seven classes based on paper topic.
Cora_ML is a smaller datset extracted from Cora [7].
Linux : Linux dataset is a collection of program dependence graphs, where each graph is a function
and nodes represent statements while edges represent dependency between statements.
IMDB : IMDB dataset is a collection of ego-networks of actors/actresses that have appeared together inany movie.
F Baselines
GMN explicitly assumes the distance function to be symmetric, which violates S ED .
G RAPH S IM hasan assumption that a large difference in size of the query and target graphs leads to a large distance,which is not true in S ED .
In G OT SIM, there is an explicit assumption of modeling a symmetricdistance function since they use cosine similarity to compare node neighborhoods The normalizationfactor in Eq.
6 of [13] is also based on whole graph matching.
Finally, G ENN -A ∗ is not included for
S ED since it does not scale on graphs beyond 10 nodes (See § 4.3).
[Figure]
G Ablation Study
Impact of G IN :
To highlight the importance of G IN , we conduct ablation studies by replacing the
GIN convolution layers in the model with several other convolution layers.
As visible in the Table B,
G IN consistently achieves the best accuracy.
This is not surprising since G IN is provably the mostexpressive among G NN s in distinguishing graph structures (essential to S ED or G ED computation)and is as powerful as the Weisfeiler-Lehman Graph Isomorphism test [39]
[Table] [Table]
[Table]
Impact of sum-pool: To substantiate our choice of the pooling layer, we have performed ablationstudies with various pooling functions as replacements for sum-pool.
It is clear from Table C thatsum-pool is the best choice among the considered alternatives.
Sum-pool can better distinguish graph sizes better than other aggregation functions such as mean-poolor max-pool.
To elaborate, let us consider a graph G 1 that is signiﬁcantly larger than another graph
G 2 .
In this scenario, the individual coordinates of G 1 ’s embedding can potentially be signiﬁcantlylarger than those of G 2 since in G 1 the summation is being done over a larger set of embeddings.
Both mean-pool and max-pool fail to capture the size information as effectively, since the max andthe mean operations do not scale with the number of inputs.
Impact of Pre-mlp layer:
Table D presents the results.
As visible, we do not see any signiﬁcantdifference in performance on average.
Performance of G REED -NN and G REED -Dual on G ED : Fig.
F presents the results.
The trendsare similar to what we observed for S ED in Sec. 4.5.
G REED -NN closes the performance gap with
G REED as more training data is provided.
Furthermore, G REED -dual is consistently worse that
G REED .
[Table]
With Pre-MLP Without Pre
[Table]
MLP
[Table]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
H Alignment
In real-world applications of subgraph similarity search, alignments are of interest only for a smallnumber of similar subgraphs.
Our framework is intended to serve as a ﬁlter to retrieve this small setof similar subgraphs from a large number of candidates.
To elaborate, a graph database may containthousands or millions of graphs (or alternatively, thousands or millions of neighborhoods of a largegraph) which need to be inspected for similar subgraphs.
A user is typically interested in only ahandful of these subgraphs that are highly similar to the query.
Since the ﬁltered set is signiﬁcantlysmaller, a non-neural exact algorithm sufﬁces to construct the alignments (Lemma 1 allows us toadapt general cost G ED alignment techniques for S ED alignment).
Computing alignments across theentire database is unnecessary and slows down the query response time.
To substantiate our claim, we show the average running time for answering 10 -NN queries.
Webreak up the running time into two components: (i) 10 -NN retrieval time by G REED , (ii) exactalignment time using MIP-F2 for the 10 -NN neighborhoods retrieved by G REED .
We observe thatexact alignment by existing methods on the 10 -NN neighborhoods completes in reasonable time.
Incontrast, G ENN -A ∗ does not scale on either PubMed or Amazon since it computes alignments acrossall (sub)graphs.
I Visualization
Searching for molecular fragment containment is a routine task in drug discovery [19].
Motivated bythis, we show the top- 5 matches to an S ED query on the AIDS dataset produced by G REED in Fig.
E.
The query is a functional group (Hydrogen atoms are not represented).
G REED is able to extractchemical compounds that contain this molecular fragment (except for ranks 3 and 5 , which contain this group with 1 edit) from around 2000 chemical compounds with varying sizes and structures.
This validates the efﬁcacy of G REED at a semantic level.
J Subgraph sampling strategies for S ED generalizability
In R WR , we perform ﬁxed length random walks, where the length of a walk is the average diametersize of the queries generated through BFS during training.
Next, we merge the walks to form a graph.
In R W , we perform random walks, till the diameter of the resultant graph is the same as the averagediameter of the BFS sampled train graphs.
The details of the SHA D OW sampler is explained in [40].
K Extension to maximum common subgraph similarity (MCSS)
Besides G ED , MCSS is also a popular similarity measure for whole-graph comparison.
Exceptthe inductive bias injected through F , all components of G REED is generic for any graph distancefunction.
In this section, we replace F with an M LP and model MCSS.
Table F presents the results.
As visible, the trends hold even in this distance function, where G REED outperforms the closestbaseline of H 2 MN.
[Table]