N EURO SED: L EARNING S UBGRAPH S IMILARITY VIA G RAPH N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Subgraph similarity search is a fundamental operator in graph analysis.
In this framework, given a query graph and a graph database, the goal is to identify subgraphs of the database graphs that are structurally similar to the query.
Subgraph edit distance (S ED ) is one of the most expressive measures of subgraph similarity.
In this work, we study the problem of learning S ED from a training set of graph pairs and their S ED values.
Towards that end, we design a novel siamese graph neural network called N EURO S ED , which learns an embedding space with a rich structure reminiscent of S ED .
With the help of a specially crafted inductive bias,
N EURO S ED not only enables high accuracy but also ensures that the predicted S ED , like true S ED , satisﬁes the triangle inequality .
The design is generic enough to also model graph edit distance (G ED ), while ensuring that the predicted G ED space is metric , like the true G ED space.
Extensive experiments on real graph datasets, for both S ED and G ED , establish that N EURO S ED achieves ≈
2 times lower RMSE than the state of the art and is ≈ 18 times faster than the fastest baseline.
Further, owing to its pair-independent embeddings and theoretical properties, N EURO S ED allows up to 3 orders of magnitude faster graph/subgraph retrieval.
I NTRODUCTION AND R ELATED WORK
Graphs are used to model data in a wide variety of domains.
Examples include chemical compounds (Sankar et al., 2017), protein-protein interaction networks (PPI)
(Alon et al., 2008), knowledge graphs (Ebsch et al., 2020), and social networks (Kempe et al., 2003).
A distance function on any dataset, including graphs, is a fundamental operator.
Among several distance measures on graphs, edit distance is one of the most powerful and popular mechanisms (Liang & Zhao, 2017; Zhao et al., 2013; Bougleux et al., 2017; Daller et al., 2018; Fankhauser et al., 2011; Riesen & Bunke, 2009;
Zeng et al., 2009; He & Singh, 2006).
Edit distance can be posed in two forms: graph edit distance (G ED ) and subgraph edit distance (S ED ).
Given two graphs G 1 and G 2 , G ED ( G 1 , G 2 ) returns the minimum cost of edits needed to convert G 1 to G 2 .
i.e., for G 1 to become isomorphic to G 2 .
An edit can be the addition or deletion of edges and nodes, or replacement of edge or node labels, with an associated cost.
In S ED ( G 1 , G 2 ) , the goal is to identify the minimum cost of edits so that G 1 is a subgraph ( subgraph isomorphic ) of G 2 .
For examples, see Fig.
5a in the appendix.
While the applications of G ED and S ED are beyond doubt, their applicability is constrained by their computation costs.
Speciﬁcally, both G ED and S ED are NP-complete (Zeng et al., 2009; Neuhaus et al., 2006; He & Singh, 2006).
To mitigate this computational bottleneck, several heuristics (Bougleux et al., 2017; Daller et al., 2018; Fankhauser et al., 2011; Riesen & Bunke, 2009; Neuhaus et al., 2006) and index structures (He & Singh, 2006; Zeng et al., 2009; Liang & Zhao, 2017; Zhao et al., 2013) have been proposed.
Recently, graph neural networks have been shown to be effective in learning and predicting G ED (Bai et al., 2019;
Wang et al., 2021; Li et al., 2019; Bai et al., 2020;
Zhang et al., 2021; Xiu et al., 2020) and subgraph isomorphism (Rex et al., 2020).
The basic goal in all these algorithms is to learn a neural model from a training set of graph pairs and their distances, such that, at inference time, given an unseen graph pair, we are able to predict its distance accurately.
Wang et al.
(2020) designed an algorithm to characterize graphs through sampling random walks and then embedding
them in a feature space.
While Wang et al.
(2020) has not been studied in the context of G ED or S ED , the authors have shown that they can detect graph isomorphism.
While this progress in the area of graph querying is undoubtedly impressive, there is scope to do more.
• Modeling S ED : Existing neural approaches to learning G ED cannot easily be adapted to learn S ED .
While G ED is symmetric, S ED is not.
Most neural architectures for G ED have the assumption of symmetry at its core and hence modeling S ED is non-trivial.
In App.
A, we provide a detailed analysis of the limitations of prior architectures in failing to model S ED .
• Exponential Search Space: Computing S ED ( G 1 , G 2 ) conceptually requires us to compare the query graph G 1 with the exponentially many subgraphs of the target graph G 2 .
Therefore, it is imperative that the model has an efﬁcient and effective mechanism to prune the search space without compromising on the prediction accuracy.
We note that while several index structures and heuristics exist for G ED (He & Singh, 2006; Zeng et al., 2009; Bai et al., 2019; Wang et al., 2021), none exist for S ED .
Thus, scalability of S ED on large graphs remains an unsolved problem.
• Preservation of theoretical properties: G ED is a metric distance function.
While S ED is not metric due to being asymmetric, it satisﬁes the triangle inequality , non-negativity , and subgraphidentity .
Several higher-order tasks such as clustering and indexing rely on such properties (Hadi, 1991; Dohnal et al., 2003; Hjaltason & Samet, 2003; Samet, 2005; Uhlmann, 1991; Yianilos, 1993; Zezula et al., 2006; Brin, 1995; Chávez et al., 2001).
Existing neural approaches do not preserve these properties, which limits their usability for these higher order tasks.
• Pair-independent computation: There is little scope for pre-computation in existing approaches (either neural or non-neural), as the major computations are pair-dependent, i.e., both G 1 and Gneed to be known.
In a typical graph querying framework, the graph database is known apriori; only the query graph is provided at runtime.
If we can generate pair-independent embeddings, and make efﬁcient predictions directly in the embedding space, then retrieval can be sped up tremendously by pre-computing and indexing the embeddings of the database graphs beforehand.
• Learning with low-volume data: Generating a large amount of high quality training data is computationally prohibitive for S ED /G ED .
So, an effective model should encode powerful priors for S ED /G
ED prediction, to enable generalization even with scarce training
data.
We address these challenges by a novel architecture called N EURO S ED .
Our key contributions are:
• Novel formulation: We formulate the problem of learning subgraph edit distance (S ED ) for graphs.
Since S ED is more general than subgraph isomorphism and G ED , the proposed theory extends to prediction of G ED , graph isomorphism and subgraph isomorphism.
• Neural architecture: N EURO S ED utilizes a siamese graph isomorphism network (Xu et al., 2018a) to embed graphs in a pair-independent fashion to an embedding space over which a simple function can predict the S ED (or G ED ).
The same embedding model is used for both G 1 and G 2 , which captures the prior that similar topological properties need to be considered for both graphs.
The carefully crafted prediction function serves as another inductive bias for the model, which, in addition to enabling high generalization accuracy, preserves the key properties of S ED /G ED , which is a major contribution of our work over existing neural approaches.
• Indexable embeddings: The prediction function satisﬁes the triangle inequality over the embedding space for both S ED and G ED .
This allows utilization of the rich literature on index structures (Elkan, 2003; Kwedlo & Czochanski, 2019; Ciaccia et al., 1997) to boost efﬁciency.
• Accurate, Fast and Scalable: Extensive experiments on real graph datasets containing up to a million nodes establish that N EURO S ED is more accurate in both G ED and S ED when compared to the state of the art and is more than 3 orders of magnitude faster in range and k -NN queries.
P RELIMINARIES AND P ROBLEM F ORMULATION
We denote a labeled undirected graph as G =
( V , E , L ) where V is the node set, E is the edge set and
L : V ∪ E → Σ is the labeling function over nodes and edges.
Σ is the universe of all labels and contains a special empty label (cid:15) .
L ( v ) and L ( e ) denote the labels of node v and edge e respectively.
G 1 ⊆
G 2 denotes that G 1 is a subgraph of G 2 .
All notations used in our work are summarized in
Table 6 in the appendix.
For the deﬁnitions of subgraph and graph isomorphism , refer to App.
B in the supplementary.
The computation of G ED relies on a graph mapping .
Deﬁnition 1 (Graph Mapping)
Given two graphs G 1 and G 2 , let ˜ G 1 =
( ˜ V 1 , ˜ E 1 , ˜ L 1 ) and ˜ G
= ( ˜ V 2 , ˜ E 2 , ˜ L 2 ) be obtained by adding dummy nodes and edges (labeled with (cid:15) ) to G 1 and G 2 respectively, such that |V 1 | =
|V 2 | and |E 1 | = |E 2 | .
A node mapping between G 1 and G 2 is a bijection π : ˜ G 1 → ˜ Gwhere (i) ∀ v ∈ ˜ V 1 , π ( v ) ∈
˜ V 2 and at least one of v and π ( v ) is not a dummy; (ii) ∀ e = ( v 1 , v 2 ) ∈ ˜
E 1 , π ( e ) =
( π ( v 1 ) , ( π ( v 2 ))) ∈
˜
E 2 and at least one of e and π ( e ) is not a dummy.
Example 1 Fig.
5b shows a graph mapping.
Edge mappings can be trivially inferred, so are omitted.
Deﬁnition 2 (Graph Edit Distance (G ED ) under mapping π ) G ED between G 1 and G 2 under π is
[Equation]
where d : Σ × Σ → R +0 is a distance function over the label set.
d ( (cid:96) 1 , (cid:96) 2 ) models an insertion if (cid:96)
=
(cid:15) , deletion if (cid:96) 2 = (cid:15) and replacement if (cid:96) 1 (cid:54) =
(cid:96) 2 and neither (cid:96)nor (cid:96) 2 is a dummy.
We assume d to be a binary function, where d ( (cid:96) 1 , (cid:96) 2 ) = 1 if (cid:96) 1 (cid:54) =
(cid:96) 2 , otherwise, 0 .
Our framework easily extends to more general distance functions (details in App. D).
Example 2 G ED for the shown mapping in Fig.
5b is 3.
The red mappings incur a cost of 1 .
Deﬁnition 3 (Graph Edit Distance (G ED ))
G ED is the minimum distance under all mappings.
[Equation]
where Φ( G 1 , G 2 ) denotes the set of all possible node maps from G 1 to G 2 .
Deﬁnition 4 (Subgraph Edit Distance (S ED ))
S ED is the minimum G ED over all subgraphs of G 2 .
[Equation]
Example 3 Revisiting Fig.
5b, G ED ( G 1 , G 2 ) = 3 .
The shown mapping incurs the minimum cost.
S ED ( G 1 , G 2 ) = 1 .
Refer to Fig.
5a for more examples.
Problem 1 (Learning S ED )
Given a training set of tuples of the form (cid:104)G 1 , G 2 , S ED ( G 1 , G 2 ) (cid:105) , learn a neural model to predict S ED ( Q 1 , Q 2 ) on unseen graphs Q 1 and Q 2 .
Problem 2 (Learning G ED )
Given a training set of tuples of the form (cid:104)G 1 , G 2 , G ED ( G 1 , G 2 ) (cid:105) , learn a neural model to predict G ED ( Q 1 , Q 2 ) on unseen graphs Q 1 and Q 2 .
2. P ROPERTIES OF S ED AND G ED
Observation 1 (i) G ED ( G 1 , G 2 ) ≥ 0 , (ii) S ED ( G 1 , G 2 ) ≥ 0 .
Observation 2 (i) G ED ( G 1 , G 2 ) =
iff G 1 is isomorphic to G 2 , (ii) S ED ( G 1 , G 2 )
=
iff G 1 is subgraph isomorphic to G 2 .
Observation 3 G ED is a metric if the distance function d over label set Σ is metric (He & Singh, 2006).
d as deﬁned in § 2 is metric (He & Singh, 2006).
Hence, G ED from § 2 is metric.
We next prove that S ED satisﬁes the triangle inequality in Theorem 1.
Theorem 1 S ED ( G 1 , G 3 ) ≤ S ED ( G 1 , G 2 ) + S ED ( G 2 , G 3 ) .
P ROOF : See App.
D.1 for details.
N EURO S ED
Fig.
1 presents the architecture of N EURO S ED .
The input to our learning framework is a pair of graphs G Q (query), G T (target) along with the supervision data S ED ( G Q , G T ) .
Our objective is to train a model that can predict S ED on unseen query and target graphs.
The design of our model must be cognizant of the fact that computing S ED is NP-hard and high quality training data is scarce.
Thus, we use a Siamese architecture in which the weight sharing between the embedding models boosts learnability and generalization from low-volume data by imposing a strong prior that the same topological features must be extracted for both graphs.
Siamese Networks:
In these models, there are two networks with shared parameters applied to two inputs independently to compute representations.
These representations are then passed through another module to compute a similarity score.
3. S IAMESE G RAPH N EURAL N ETWORK
As depicted in Fig.
1a, we use a siamese graph neural network (G NN ) with shared parameters to embed both G Q and G T .
Fig.
1b focuses on the G NN component of N EURO S ED .
We next discuss each of its individual components.
Pre-M LP : The primary task of the Pre-M LP is to learn representations for the node labels (or features).
Towards that end, let x v denote the initial feature set of node v .
The M LP learns a hidden representation µ G v = M LP ( x v ) .
In our implementation, x v is a one-hot encoding of the categorical node labels.
We do not explicitly model edge labels in our experiments.
N EURO S ED can easily be extended to edge labels by strategies such as using G INE (Hu et al., 2019) layers instead of G IN layers (see App. C).
[Figure]
EMBEDDINGVECTORS
ZQ
ZT
F( Z Q , Z T ) PREDICTION
[Figure]
k
[Figure]
Concatenatefrom
[Figure]
PreMLP
PostMLP
NODEFEATUREVECTORS
POOL
EMBEDDINGVECTOR
[Figure]
Graph Isomorphism Network (G IN ): G IN (Xu et al., 2018b) consumes the information from the
Pre-M LP to learn hidden representations that encode both the graph structure as well as the node feature information.
G IN is as powerful as the Weisfeiler-Lehman (WL) graph isomorphism test (Leman & Weisfeiler, 1968) in distinguishing graph structures.
Since our goal is to accurately characterize graph topology and learn similarity, G IN emerges as the natural choice.
G IN develops its expressive power by using an injective aggregation function.
Speciﬁcally, in the initial layer, each node v in graph G is characterized by the representation learned by the M LP , i.e., h G v,
= µ G v .
Subsequently, in each hidden layer i , we learn an embedding through the following transformation.
[Equation]
Here, (cid:15) i is a layer-speciﬁc learnable parameter, N G ( v ) is one-hop neighbourhood of the node v , and h G v, 0 = µ G v .
The k -th layer embedding is h G v,k , where k is ﬁnal hidden layer.
Concatenation, Pool and Post-M LP : Intuitively, h G v,i captures a feature-space representation of the i -hop neighborhood of v .
Typically, G NN s operate on node or edge level predictive tasks, such as node classiﬁcation or link prediction, and hence, the node representations are passed through
an
M LP for the ﬁnal prediction task.
In our problem, we need to capture a graph level representation.
Furthermore, the representation should be rich enough to also capture the various subgraphs within the input graph so that S ED can be predicted accurately.
To fulﬁl these requirements, we ﬁrst concatenate the representation of a node across all hidden layers, i.e., the ﬁnal node embedding is z G v = C ONCAT (cid:0) h G v,i , ∀ i ∈ { 1 , 2 , · · · , k } (cid:1) .
This allows us to capture a multi-granular view of the subgraphs centered on v at different radii in the range [1 , k ] .
Next, to construct the graph-level representation, we perform a sum-pool, which adds the node representations to give a single vector.
This information is then fed to the Post-M LP to enable post-processing.
Mathematically:
[Equation]
S ED Prediction: The ﬁnal task is to predict the S ED as a function of query graph embedding Z G Q and target graph embedding Z G T .
The natural choice would be to feed these embeddings into another
M LP to learn S ED ( Z G Q , Z G T ) .
This M LP can then be trained jointly with the graph embedding model in an end-to-end fashion.
However, an M LP prediction does not have any theoretical guarantees.
To ensure preservation of original space properties, we introduce an inductive bias in the form of a simple ﬁxed computation function F ( Z G Q , Z G T ) to predict the S ED .
By training the model to produce embeddings Z G Q and Z G T such that F ( Z G Q , Z G T ) ≈ S ED ( Z G Q , Z G T ) , we enforce a rich structure on the embedding space.
We show that these embeddings satisfy many key properties of the
S ED (and G ED ) function that existing neural algorithms fail to do (Bai et al., 2019;
Wang et al., 2021;
Bai et al., 2020; Zhang et al., 2021).
F is deﬁned as follows:
[Equation]
Intuitively, for those co-ordinates where the value of Z G Q is greater than Z G T ; a distance penalty is accounted by F in terms of how much those values differ, otherwise F considers 0 .
This follows the intuition that the S ED accounts for those features of G Q that are not in G T .
Moreover, consistent with
S ED , the additional features in G T that are not in G Q , do not incur any cost.
Finally, the parameters of the entire model are learned by minimizing the mean squared error 1 (here T is the training set).
[Equation]
Adaptation for G ED (N EURO S ED G ): The proposed architecture naturally extends to G ED (denoted by N EURO S ED G ) with a simple modiﬁcation of the computation function.
Speciﬁcally, instead of
Eq. 6, we use
[Equation]
3. T HEORETICAL C HARACTERIZATION
Lemma
The following properties hold on predicted S ED F ( Z G Q , Z G T ) :
[Equation]
P ROOF .
Properties (1) and (2) follow from the deﬁnition of F itself.
Property (3) follows from the fact that we take the L 2 norm (holds for any monotonic norm, details are available in App. D).
[Equation]
We see that F satisﬁes the analogue of properties of S ED (§ 2.1).
Conditions 1, 2, and 3 captures non-negativity, the subgraph relation (Rex et al., 2020), and triangle inequality respectively.
Furthermore, we show in App.
E a complete constructive characterization for any F which can model S ED in the embedding space, while being translation-invariant and non-negative homogeneous.
Speciﬁcally, we show that the general form for F involves solving a constrained optimization problem (intractable in general), but under a mild technical assumption a closed form can be obtained which coincides with equation 6.
This asserts that equation 6 is the natural choice for F , modulo the choice of norm.
L 2 norm is a natural choice due to its isotropy (direction/rotation invariance) and good empirical performance.
Lemma
The predicted G ED , i.e., F g is metric.
(i) Non-negativity: F g ( Z G Q , Z G T ) ≥ 0 , (ii)
Identity: F g ( Z G Q , Z G T ) =
⇐⇒ Z G Q
=
Z G T , (iii) Symmetry: F
[Equation] ( Z G
[Equation] Z
[Equation]
) =
F [Equation] Z [Equation]
, Z G Q ) , (iv) Triangle Inequality: F
[Equation]
P ROOF .
Follows trivially from the fact that F g forms a Euclidean space.
(cid:3)
Similar to F , the form for F g also comes naturally from some mild technical assumptions for any embedding space function which models G ED (see App. E).
Theorem 2 The properties of S ED (and G ED ) in § 2.1 are satisﬁed by the prediction functions
N EURO S ED ( G 1 , G 2 ) (and the G ED version N EURO S ED G ( G 1 , G 2 ) ).
P ROOF S KETCH .
(For S ED ) Let E be the embedding function learned by N EURO S ED ’s GNN.
Then
N EURO S ED ( G 1 , G 2 ) =
F ( E ( G 1 ) , E ( G 2 )) .
Now, the relevant properties follow from Lemma 1 using the substitutions Z G Q (cid:55)→ E ( G 1 ) , Z G T (cid:55)→ E ( G 2 ) , and Z G T(cid:48)
(cid:55)→ E ( G 3 ) .
(cid:3)
We note here that if different embedding models are used for G 1 and G 2 , or if the distance computations are pair-dependent (Li et al., 2019), then Theorem 2 would not hold.
It is easy to verify that the proof fails if E Q and E T are different embedding functions corresponding to the query and the target, or if E is a function of both G 1 and G 2 .
Hence, the siamese architecture is crucial.
Complexity analysis: Inference is linear in query and target sizes (see App.
F for details).
3. C OMPARISON WITH N EURO M ATCH (R EX
ET AL ., 2020):
Both N EURO S ED and N EURO M ATCH use siamese GNNs.
However, there are important differences in (1) their objectives, (2) training and (3) inference procedures and (4) theoretical characterization.
Objective: While N EURO S ED is trained to predict G ED and S ED directly, N EURO M ATCH studies the simpler task of anchored subgraph isomorphism.
Thus, N EURO S ED subsumes the objective of N EURO M ATCH .
Training: N EURO M ATCH trains the model to construct order embeddings wherein if G 1 ⊆
G 2 , then ∀
i, Z G 1 [ i ] ≤ Z G 2 [ i ] , where Z G denotes the embedding of graph G .
To inject this bias, the model is trained using max-margin loss.
N EURO S ED does not enforce order embeddings.
Rather it learns embeddings that make the proposed prediction functions (Eq. 6 and Eq.
8) effective through RMSE loss.
In the context of S ED , Eq. 6 is used in the loss function of both N EURO S ED and N EURO M ATCH .
However, the loss function themselves are different, which results in the proposed embedding space being considerably richer in its ability to preserve properties from the original space.
In addition, in App. E, we characterize the class of functions that may act as a substitute for Eq. 6, and thereby perform a thorough theoretical characterization of our work.
Inference: N EURO S ED creates a single embedding of the query graph, based on which the S ED and G ED predictions are made.
In contrast, N EURO M ATCH generates embeddings for each node neighborhood in the query and target graph and makes | V Q || V T | predictions corresponding to each pair of neighborhoods.
The ﬁnal decision of whether the query is a subgraph of the target is performed using a voting mechanism.
E MPIRICAL E VALUATION
In this section, we establish the following:
• Subgraph Search: N EURO S ED outperforms the state of the art approaches for S ED prediction.
• Graph Search: N EURO S ED G , the variation to predict G ED , outperforms the state of the art approaches for G ED prediction.
• Scalability: N EURO S ED and N EURO S ED G are orders of magnitude faster than existing approaches and scale well to graphs with millions of nodes.
The code base and datasets are available at https://
anonymous
4open science/
r/
NeuroSED/.
4. E XPERIMENTAL S ETUP
Details on the hardware and software platform and parameters are provided in App.
I.
Datasets: Table 4.1 lists the datasets used for benchmarking.
Further details on the dataset semantics are provided in the App.
H. We include a mixture of both graph databases (#graphs >1), as well as single large graphs (#graphs = 1).
Linux and IMDB contain unlabeled graphs.
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Figure]
AIDS’     9 Linux   1 ,
[Figure] [Figure]
[Figure]
Training (and Test) Data Generation:
For G ED , we use (cid:104) query, target (cid:105) graph pairs from IMDB ,
AIDS’ , and Linux .
Our setup is identical to S IM G NN (Bai et al., 2019) and H 2 MN- RW (Zhang et al., 2021).
For S ED , the procedure is more intricate.
The details of datasets can be found in App. H.
• Queries:
In single large graphs, queries are sampled by performing a random BFS traversal (depth up to 5) on a randomly chosen target graph.
In addition, for AIDS , we use known functional groups as test queries (See App.
G for details).
This allows us to benchmark N EURO S ED on both natural as well synthetic queries.
The average query sizes ( |V Q | , |E Q | ) are listed in Table 4.1.
• Ground-truth: We use mixed integer programming method F2 (Lerouge et al., 2017) implemented in G EDLIB (Blumenthal et al., 2019) with a large time limit to generate ground-truth data.
Train-Validation-Test Split: We use 100 K query-target pairs for training and 10 K pairs each for validation and test.
Baselines
To
evaluate performance in G ED , we compare with S IM G NN
(Bai et al
, G ENNA
∗ (Wang et al., 2021) and H 2 MN(Zhang et al., 2021)
H
2 MN
has
two
versions
based on random walks (H 2 MN- RW ) and k -hop neighborhood (H 2 MN- NE ).
We include both.
For S ED , no neural approaches exist.
However, H 2 MN and S IM G NN can be trained by replacing
G ED with S ED along with minor modiﬁcations in training.
While N EURO M ATCH (Rex et al., 2020) cannot predict S ED , it generates a violation score which can be interpreted as the likelihood of the query being subgraph isomorphic to the target.
The violation score can be used as a proxy for S ED and used in ranking of k -NN ( k -NearestNeighbour) queries.
Thus, N EURO M ATCH comparisons are limited to k -NN queries on S ED .
The changes required in to adapt to S ED are in App.
I.
[Table] [Table]
[Figure]
In the non-neural category, we use mixed integer programming based method M IP -F2 (Lerouge et al., 2017) with a time bound of 0 .
1 seconds per pair for both G ED and S ED .
M IP -F2 provides the optimal solution given inﬁnite time.
In addition, we also compare with B RANCH (Blumenthal, 2019), which achieves an excellent trade-off between accuracy and runtime (Blumenthal et al., 2020).
B RANCH uses linear sum assignment problem with error-correction (L SAPE ) to process the search space.
We use G EDLIB ’s
(Blumenthal et al., 2019) implementation of these methods.
4. P REDICTION A CCURACY OF S ED AND G ED
Tables 2a and 2b present the accuracy of all techniques on S ED and G ED in terms of Root Mean
Square Error (RMSE ).
In App.
J, we evaluate the same performance using several other metrics such as R 2 and Mean Absolute Error (MAE).
N EURO S ED outperforms all other techniques in 9 out ofsettings.
The gap in accuracy is the highest in IMDB for G ED , where N EURO S ED is more thantimes better than the neural baselines.
A deeper analysis reveals that IMDB graphs are signiﬁcantly denser and larger than AIDS’ or Linux .
Thus, computing the optimal G ED is harder.
While all techniques have higher errors in IMDB , the deterioration is more severe in the baselines indicating that N EURO S ED scales better with graph sizes.
To gain insight on whether the same patterns also hold in S ED , in Fig.
2, we plot the heat map of
RMSE against query graph size in Dblp .
Speciﬁcally in this plot, each dot corresponds to a query graph G Q .
The co-ordinate of a query is ( S ED ( G Q , G T ) , |V Q | ) .
The color of a dot is the RMSE; the darker the color, the higher is the RMSE.
When we compare the heat maps of N EURO S ED with the most recent baseline H 2 MN- RW , we observe that H 2 MN- RW is noticeably darker.
Furthermore, the dark colors are clustered on higher S ED values and large query sizes (upper-right corner).
This indicates that N EURO S ED scales better with query size and higher S ED .
The heatmaps of all techniques are provided in App.
L.
Range and k -NN queries: Range and k -NN queries are two of the most common database queries.
We next evaluate the performance of the various algorithms on these queries.
In a range query, given a distance threshold θ , the goal is to retrieve all database graphs that are within θ distance from the query graph.
In a k -NN query, given the query graph, we identify the k nearest neighbors from the search space in distance-ascending order.
For S ED , the search space includes all target graphs of the database graphs, whereas for G ED , the search space constitutes of the database graphs.
The performance measures are F1-score (Range query) and Kendalls’s tau ( k -NN)
(Kendall, 1938) of the predicted answer set, when compared against the ground truth.
As visible in Figs.
3a-3h, N EURO S ED consistently outperforms all baselines in F1-score.
Similar trend is also visible in k -NN queries.
Speciﬁcally, in Kendall’s tau (Tables 3a and 3b), barring the exception of G ED in Linux , N EURO S ED consistently scores the highest indicating best preservation of the ranking order.
Overall, this shows that accurate prediction of distance also transfers to accurate querying accuracy.
In App.
K, we also report k -NN performance based on Precision @ k .
In this analysis, we do not include G ENN -A ∗ since it is orders of magnitude slower than all neural-baselines and provides a comparable distance accuracy to H 2 MN.
Next, we discuss efﬁciency in detail.
[Figure]
[Table]
[Figure]
[Table] [Table]
[Table]
4. E FFICIENCY
Table 4 presents the inference time per 10 K graph pairs.
As visible, N EURO S ED is up to 1800 times faster than the non-neural baselines and up to 10 to 20 times faster than H 2 MN- RW , the current state of the art in G ED prediction.
Also note that G ENN -A ∗ is exorbitantly slow (Table 4b).
G ENN -A ∗ is slower since it not only predicts the G ED but also the alignment via an A ∗ search.
While the alignment information is indeed useful, computing this information across all graphs in the database may generate redundant information since an user is typically interested only on a small minority of graphs that are in the answer set.
In App.
K.2, we discuss this issue in detail.
Scalability: Here, we showcase how pair-independent embeddings, and ensuring triangle inequality leads to further boost in scalability.
For this experiment, we use the three largest datasets of PubMed ,
Amazon and Dblp .
For each dataset, we pre-compute N EURO S ED embeddings of all database graphs by exploiting pair-independent embeddings.
Such pre-computation is not possible in the neural or non-neural baselines.
Furthermore, since the predictions of N EURO S ED satisfy triangle inequality, we index the pre-computed embeddings of the database graphs using the Metric Tree (Uhlmann, 1991) index structure adapted to asymmetric distance functions.
Consequently, for N EURO S ED , we only need to embed the query graph and evaluate F to make predictions at query time.
Table 5a presents the results on range and 10 -NN queries.
When computations are done on a GPU, N EURO S ED is more than 1000 times faster than both H 2 MN- RW and H 2 MN- NE .
In the absence of a GPU,
H 2 MN is practically infeasible since expensive pair-dependent computations are done at query time.
In contrast, even on a CPU, with tree indexing, N EURO S ED is ≈ 50 times faster than GPU-based
H 2 MN- RW .
Note that indexing enables up to 3 -times speed-up on N EURO S ED over linear scan, which demonstrates the gain from ensuring triangle inequality.
These results show the beneﬁts of using an easily computable and parallelisable prediction function F with theoretical guarantees.
4. A BLATION S TUDY
In this study, we explore the impact of our inductive biases in learning from low-volume data.
We create two variants of N EURO S ED : (1) N EURO S
ED -Dual trains the two parallel G NN models separately without weight-sharing, and (2) N EURO S ED -NN uses an MLP instead of F .
Both have strictly better representational capacity than N EURO S ED , so are expected to match the performance with inﬁnite data.
Figs.
4a-4c present the results.
The RMSE of N EURO S ED is consistently better than N EURO S ED -Dual, with the difference being more signiﬁcant at low volumes.
This indicates that siamese structure helps.
Compared to N EURO S ED , N EURO S ED -NN achieves marginally better performance at larger train sizes in PubMed and CiteSeer .
However, in Dblp , N EURO S ED is consistently better.
The number of subgraphs in a dataset grows exponentially with the node set size.
Hence, an MLP needs growing training data to accurately model the intricacies of this search space.
In Dblp , even 100k pairs is not enough to improve upon F .
Overall, these trends indicate that F enables better generalization and scalability with respect to accuracy.
Furthermore, given that its
[Figure] [Figure]
is close
4. G ENERALIZATION TO U NSEEN Q UERY D ISTRIBUTIONS
We train the model by sampling queries from the graph database through BFS enumerations.
How does N EURO S ED generalize to unseen distributions?
Towards that end, we generate queries from the three unseen distributions of (1) Random Walks (R W ), (2) Random Walks with Restarts (R WR ), and (3) SHA D OW (Zeng et al., 2021).
Table 5b presents the results.
As visible, the errors remain low.
Even more surprisingly, the errors on R W and R WR are better than the train distribution of BFS itself.
This indicates good generalization to unseen distributions.
See App.
O for further details on the sampling strategies.
C ONCLUSIONS , L IMITATIONS AND F UTURE W ORK
Similarity search is a fundamental operator for graph analytics.
Edit distance is one of the most popular similarity measures deﬁned over graphs.
The applicability of S ED and G ED , however, is constrained by its exponential computation complexity.
Our work is motivated by the following question: Is it possible to design a neural framework that generalizes to both S ED and G ED , preserves their key properties and is capable of learning from a low-volume training data?
We show that this is indeed possible.
The efﬁcacy and efﬁciency of N EURO S ED stems from three algorithmic innovations: a siamese architecture to reduce dependence on high-volume training data, a carefully designed function to introduce an inductive bias and mimic important properties of the original space, and pair-independent embeddings to enable massive scalability.
One limitation of our work (and all neural approaches(Rex et al., 2020; Zhang et al., 2021; Wang et al., 2021; Li et al., 2019; Bai et al., 2019; 2020)) is the ability to predict on large query graph sizes.
While the deterioration in prediction quality with larger query sizes is manageable if the same query sizes are seen in the train data, generalizing from small query sizes to large query sizes remain an unsolved problem (See App. M).
The ability to generalize to unseen larger query sizes is important since computing G ED and S ED are NP-complete.
Hence, generating training data on large query graphs is computationally intractable.
To further advance the ﬁeld of neural graph distance learning, this generalization ability .
R EPRODUCIBILITY S TATEMENT
Please ﬁnd all code, experiments and generated data at the following anonymous link: https:// anonymous .
4open .
science/r/NeuroSED/.
In Appendix we have given details that are needed to make our work more reproducible.
In particular, we have (1) additional proofs in App.
D, (2) descriptions of datasets in App.
H, (3) additional deﬁnitions in App.
B, (4) analyses of computation cost for S ED and G ED computation in App.
F, (5) additional details of the pipeline of our architecture in App.
G, (6) additional details of the experimental setup in App.
I, and (7) additional experimental results in App.
J.
A PPENDIX A I NABILITY OF EXISTING NEURAL APPROACHES G ED IN MODELING S ED
A key difference between S ED and G ED is that while G ED is symmetric, S ED is not.
Hence, if the architecture or training procedure relies heavily on the assumption of symmetry, then the modeling capacity is compromised.
[Figure]
• GMN (Li et al., 2019):
The loss function of GMN computes the euclidean distance between the embeddings and then this signal is used to learn the weight parameters (See Eq.
12 in (Li et al., 2019)).
The assumption of euclidean distance prohibits the model from learning an asymmetric distance function.
Certainly, the loss function can be changed to incorporate an asymmetric distance measure and the performance of the architecture following this change may be studied.
However, the selection of an appropriate loss function that works well with the GMN architecture is a research problem in itself.
Apart from the loss function, the proposed architecture along with the attention
mechanism
is also motivated by symmetry (note the quantiﬁcations for i and j in Eqns.
(4-8) in (Li et al., 2019)).
GraphSim (Bai et al., 2020): GraphSim computes a square similarity matrix across node pairs of two graphs G 1 =
( V 1 , E 1 ) , G 2 =
( V 2 , E 2 ) on which CNNs are applied to predict the similarity.
Since G 1 and G 2 may be of different sizes, the smaller graph is padded with ||V 1 | − |V
||
zeros.
When the similarity matrix has a large number of zeros, it confuses the model into thinking that the two graphs are dissimilar.
The authors of GraphSim acknowledge this themselves while justifying a padding of ||V 1 | − |V 2 || zeros instead of a ﬁxed maximum size limit.
We quote an exact line from the paper: “the similarity matrix between two small but isomorphic graphs may be padded with a lot of zeros, potentially misleading the CNNs to predict they are dissimilar.”
In our problem, even if the query graph is signiﬁcantly smaller than the target graph (tens of nodes vs. hundreds),
S ED can be 0.
Thus, the above mentioned design of relying on node-to-node square similarity matrix is not well suited.
• SimGNN (Bai et al., 2019):
S IM G NN relies on a neural tensor network and a pairwise node comparison module to compute similarity between two graphs.
The pairwise node comparison module assumes the comparison matrix to be symmetric.
Hence, in our adaptation of S IM G NN for S ED , we remove this particular module and only use the neural tensor network.
B D EFINITIONS
Deﬁnition 5 (Graph Isomorphism)
Graph G 1 =
( V 1 , E 1 , L 1 ) is isomorphic to G
= ( V 2 , E 2 , L 2 ) if there exists a bijection π : V 1 → V 2 such that (i) ∀ v ∈ V 1 : L ( v ) =
L ( π ( v )) , (ii) ∀
e
=
( v 1 , v 2 ) ∈
E 1 : π ( e ) =
( π ( v 1 ) , π ( v 2 ))
∈ E 2 , and (iii) ∀ e ∈ E 1 : L ( e ) =
L ( π ( e )) .
Subgraph isomorphism is deﬁned analogously by using an injection instead of a bijection.
Deﬁnition 6 (Nearest Subgraph) S is a nearest subgraph to G 1 in G 2 if S ED ( G 1 , G 2 ) =
G ED ( G 1 , S ) .
C H ANDLING E DGE L ABELS
A straight-forward extension of N EURO S ED to handle edge labels is to use G INE (Hu et al., 2019) layers instead of G IN layers.
Equation 4 can be modiﬁed as follows:
[Equation]
where e G v,u denotes the vector representation of the edge label for edge ( v, u ) .
It is also possible to represent edge labels in the current architecture by constructing a new graph
G (cid:48) with a vertex w v,u for every edge ( v, u ) and edges from v to w v,u and from w v,u to u .
The node representation for w v,u is a vector representation for the label on the edge ( v, u ) (suitably padded for uniformity).
Furthermore a new dimension is introduced which separates these new nodes from the old ones and this dimension is set to 1 if the node is w v,u for some
( v, u ) ∈ E G and 0 otherwise.
N EURO S ED (and N EURO S ED G ) can then be trained with such modiﬁed graphs as inputs, but using the ground truth S ED (respectively G ED ) for the original graphs in the loss function.
D A DDITIONAL P ROOFS D. T HE T RIANGLE I NEQUALITY OF S ED : P ROOF OF T HEOREM .
Our proof relies on two lemmas.
Lemma 3 Let (cid:98) d : Σ × Σ → R +0 be a distance function over Σ , where (i) (cid:98) d ( (cid:96) 1 , (cid:96) 2 )
=
if (cid:96) 1 = (cid:15) , and (ii) (cid:98) d ( (cid:96) 1 , (cid:96) 2 ) =
d ( (cid:96) 1 , (cid:96) 2 ) otherwise; the following holds: S ED ( G 1 , G 2 ) =
(cid:100)
G ED ( G 1 , G 2 ) , where (cid:100) G ED denotes G ED with (cid:98) d as the label set distance function.
In simple words, the S ED between two graphs is equivalent to G ED with a label set distance function where we ignore insertion costs.
P ROOF .
It sufﬁces to prove (i) S ED ( G 1 , G 2 ) ≥ (cid:100)
G ED ( G 1 , G 2 ) and (ii) (cid:100)
G ED ( G 1 , G 2 ) ≥ S ED ( G 1 , G 2 ) .
i)
Let S =
( V S , E S , L S ) ⊆
G 2 be the subgraph minimizing S ED ( G 1 , S ) (Recall Eq. 3).
Consider the mapping π from G 1 to S corresponding to S ED ( G 1 , S ) (and hence G ED ( G 1 , S ) as well).
We extend π to deﬁne a mapping (cid:98) π from G 1 to G 2 by mapping of all nodes in set V 2 \ V S to dummy nodes in
G 1 ; the edge mappings are deﬁned analogously.
Under this construction, S ED ( G 1 , S )
=
G ED ( G 1 , G 2 ) =
(cid:100)
G ED (cid:98) π ( G 1 , G 2 ) ≥ (cid:100)
G ED ( G 1 , G 2 ) .
This follows from the property that under (cid:98) d , insertion costs are zero, that is (cid:98) d
( (cid:15), (cid:96) )
= 0 .
Thus, the additional mappings introduced in (cid:98) π do not incur additional costs under (cid:98) d .
(ii) Consider S ⊆
G 2 and a mapping π from G 1 to S such that G ED π
( G 1 , S )
= (cid:100)
G ED ( G 1 , G 2 ) .
The existence of such a subgraph is guaranteed (See Lemma 5 in Supplementary).
From the deﬁnition of G ED , G ED π ( G 1 , S ) ≥ G ED ( G 1 , S ) .
Furthermore, since S ⊆ G 2 , G ED ( G 1 , S ) ≥ S ED ( G 1 , G 2 ) .
Combining all these results, we have (cid:100) G ED ( G 1 , G 2 ) ≥ G ED ( G 1 , S ) ≥ S ED ( G 1 , G 2 ) .
Hence, the claim is proved.
(cid:3)
Lemma 4 (cid:98) d , as deﬁned in Lemma 3, satisﬁes the triangle inequality.
P ROOF .
We need to show (cid:98) d ( (cid:96) 1 , (cid:96) 3 ) ≤ (cid:98) d ( (cid:96) 1 , (cid:96) 2 ) + (cid:98) d
(cid:96) 2 , (cid:96) 3 ) .
We divide the proof into four cases:
(i)
None of (cid:96) 1 , (cid:96) 2 , (cid:96) 3 is (cid:15) .
Hence, (cid:98) d ( (cid:96) 1 , (cid:96) 3 ) =
d ( (cid:96) 1 , (cid:96) 3 ) and the triangle inequality is satisﬁed.
(ii) (cid:96) 1 = (cid:15) .
The L HS is 0 and hence the triangle inequality is satisﬁed.
( iii) (cid:96) 1 (cid:54) =
(cid:15) and (cid:96) 2 = (cid:15) .
L HS ≤ 1 and R HS = 1 .
Hence, satisﬁed.
(iv) Only (cid:96) 3 = (cid:15) .
Here, L HS = 1 and R HS ≥ 1 .
These four cases cover all possible situations and hence, the triangle inequality is established.
(cid:3)
From Lemma 3, we know S ED ( G 1 , G 2 ) =
(cid:100)
G ED ( G 1 , G 2 ) .
Combining Obs.
3 with Lemma 4, (cid:100)
G ED satisﬁes the triangle inequality.
Thus, S ED satisﬁes the triangle inequality.
(cid:3)
D. P ROOF OF L EMMA
Lemma
There exists a subgraph S of G 2 and a node map π from G 1 to S such that G ED π
( G 1 , S )
= (cid:100)
G ED ( G 1 , G 2 ) .
P ROOF .
Let π (cid:48) be a node map from G 1 to G 2 corresponding to (cid:100) G ED ( G 1 , G 2 ) .
Let h 1 , · · · , h l be the nodes of G 2 which are inserted in π (cid:48) .
Construct subgraph S of G 2 by removing nodes h 1 , · · · , h l and their incident edges from G 2 .
Let π be the node map from G 1 to S which is obtained by removing h (cid:48) 1 , · · · , h (cid:48)
l from the domain and h 1 , · · · , h l from the co-domain of π .
Since insertion costs are 0 in (cid:98) d and π contains only non-insert operations, then G ED π ( G 1 , S )
=
(cid:100)
G ED ( G 1 , G 2 ) .
Hence, the claim is proved.
(cid:3)
Corollary 1 There exists a nearest subgraph (Def. 6) S to G 1 in G 2 such that S ⊆
G 1 .
[Table]
P ROOF .
The node map π in the above proof contains only non-insert operations.
So, S in the above proof is a subgraph of G 1 ignoring node and edge labels.
Since equality holds in Lemma 3, this π gives an optimal edit distance.
Thus, S in the above proof is also a nearest subgraph to G 1 in G 2 .
(cid:3)
D.
M ORE G ENERAL D ISTANCE F UNCTIONS
G ED and S ED can be viewed as error-tolerant graph and subgraph matching respectively.
In practice, the edit costs are used to model the acceptable deviations from ideal graph patterns.
The more tolerant we are towards a certain kind of distortion, the smaller is its corresponding edit cost.
Concrete values for the edit costs are application-dependent.
In this paper we have discussed binary distance functions which impose a cost of 1 for unequal labels and 0 for equal labels as these are the most popular distance functions used in literature.
However our framework is also suitable for more general distance functions.
The results for G ED directly extend to any metric distance function over the label set.
The relevant proofs mentioned in the paper are directly applicable.
The results for S ED can be extended to metric distance functions d over Σ for which replacement is no more expensive than deletion, i.e. d ( (cid:96) 1 , (cid:96) 2 ) ≤ d ( (cid:96) 1 , (cid:15) ) ∀ (cid:96) 1 , (cid:96) 2 ∈ Σ .
A replacement can be seen as a compound edit operation consisting of deletion(s) followed by insertion(s).
If a replacement cost is too high, replacement can be better achieved by deletion(s) followed by insertion(s), of which insertion(s) do not contribute to the cost in S ED .
Hence, this assumption is general enough to capture a large class of interesting distance functions which might be used for the S ED prediction task.
We show a generalization of Lemma 4 to such distance functions d .
The other proofs mentioned in the paper are directly applicable.
Lemma 6 Let d : Σ × Σ → R +0 be a metric and d ( (cid:96) 1 , (cid:96) 2 )
≤ d ( (cid:96) 1 , (cid:15) ) ∀ (cid:96) 1 , (cid:96) 2 ∈ Σ .
As in Lemma 3, let (cid:98) d : Σ × Σ → R +0 be a distance function over Σ , where (i) (cid:98) d ( (cid:96) 1 , (cid:96) 2 )
=
if (cid:96) 1 = (cid:15) , and (ii) (cid:98) d ( (cid:96) 1 , (cid:96) 2 ) =
d ( (cid:96) 1 , (cid:96) 2 ) otherwise.
Then, (cid:98) d satisﬁes the triangle inequality.
P ROOF .
We need to show (cid:98) d ( (cid:96) 1 , (cid:96) 3 ) ≤ (cid:98) d ( (cid:96) 1 , (cid:96) 2 ) + (cid:98) d ( (cid:96) 2 , (cid:96) 3 ) .
Consider the three cases: (i) (cid:96)
=
(cid:15) .
L HS = 0 , R HS = (cid:98) d ( (cid:96) 2 , (cid:96) 3 ) ≥ 0 by non-negativity of d .
Hence, L HS ≤ R HS .
( ii) (cid:96) 1 (cid:54) =
(cid:15) and (cid:96) 2 = (cid:15) .
L HS =
d ( (cid:96) 1 , (cid:96) 2 ) , R HS =
d ( (cid:96) 1 , (cid:15) ) .
Hence,
L HS ≤ R HS by assumption.
(iii) (cid:96) 1 (cid:54) =
(cid:15) and (cid:96) 2 (cid:54) = (cid:15) .
L HS =
d ( (cid:96) 1 , (cid:96) 3 ) , R HS =
d ( (cid:96) 1 , (cid:96) 2 ) + d ( (cid:96) 2 , (cid:96) 3 ) .
L HS ≤ R HS by triangle inequality for d .
These cases cover all possible situations.
Hence, triangle inequality is established.
(cid:3)
D
D ATA A UGMENTATION
Observation 4 Let S be a nearest subgraph (Def. 6) of G 1 in G 2 .
Let S (cid:48) be a subgraph of Gwhich is also a supergraph of S .
Then S is also a nearest subgraph of G 1 in S (cid:48) and S ED ( G 1 , S (cid:48) )
=
S ED ( G 1 , G 2 ) .
P ROOF .
The set of subgraphs of S (cid:48) is a subset of the set of subgraphs of G 2 .
S achieves the minimum
G ED ( G 1 , S ) among all subgraphs of G 2 and so also the minimum G ED ( G 1 , S ) among all subgraphs of S (cid:48) .
Thus, S ED ( G 1 , S (cid:48) )
=
S ED ( G 1 , G 2 ) =
G ED ( G 1 , S ) .
(cid:3)
Observation 4 gives a natural strategy for data augmentation .
Once we have S ED ( G 1 , G 2 ) and a nearest subgraph S from the expensive non-neural computation of G ED ( G 1 , G 2 ) , we can use ( G 1 , S (cid:48) , S ED ( G 1 , G 2 )) for all S ⊆ S (cid:48) ⊆
G 2 as training examples.
D. T HE T RIANGLE I NEQUALITY FOR F
This is referred to in the proof for Lemma 1.
Lemma 7 F ( Z G Q , Z G T ) ≤ F ( Z G Q , Z G T(cid:48) )
+ F ( Z G T(cid:48) , Z G T ) , where F ( x , y ) = (cid:107) ReLU ( x − y )
(cid:107) for any monotonic norm (cid:107) .
(cid:107) .
P ROOF .
Let x , y ∈ R n .
We observe that ( ReLU ( x ) + ReLU ( y ))
i
=
ReLU ( x ) i + ReLU ( y ) i =
ReLU ( x i ) + ReLU ( y i ) ≥ ReLU ( x i + y i ) =
( ReLU ( x + y ))
i .
Since (cid:107) .
(cid:107) is monotonic, this implies (cid:107) ReLU ( x ) + ReLU ( y ) (cid:107) ≥ (cid:107) ReLU ( x + y )
(cid:107) .
Using the triangle inequality for (cid:107) .
(cid:107) , we get (cid:107) ReLU ( x ) (cid:107) + (cid:107) ReLU ( y ) (cid:107) ≥ (cid:107) ReLU ( x )+ ReLU
( y ) (cid:107) ≥ (cid:107) ReLU ( x + y )
(cid:107) ).
Substituting x = Z G Q − Z G T(cid:48) , y =
Z G T(cid:48)
− Z G T , we get, (cid:107) ReLU ( Z G Q − Z G T(cid:48) )
(cid:107) + (cid:107) ReLU
( Z G T(cid:48)
− Z
G T ) (cid:107) ≥ (cid:107) ReLU ( Z G Q − Z G T ) (cid:107)
This implies that F ( Z G Q , Z G T(cid:48) )
+ F ( Z G T(cid:48) , Z G T ) ≥ F ( Z G Q , Z G T ) .
Our claim is proved.
(cid:3)
E F URTHER C HARACTERIZATION OF F
Let E : G → R d , where G is the space of graphs, be the learned embedding function and F : R d × R d
→ R be the embedding space prediction function.
We aim to have the graph space prediction function P : G × G → R , deﬁned by P ( G, H )
=
F ( E ( G ) , E ( H )) , to be able to approximate S ED : G × G → R in the best possible way, while preserving the key theoretical properties listed below:
[Equation] [Equation]
[Equation] [Equation]
We allow E to be arbitrary.
Then, P satisﬁes the analogous properties
[Equation]
14) if and only if F satisﬁes
[Equation] [Equation]
For the analogue to property in Eq. 11, we ﬁrst deﬁne the subgraph prediction function P S : G × G → { 0 , 1 } as P S ( G, H ) =
[[ ∀ i : E ( G )
[ i ] ≤ E ( H )
[ i ]]] .
P S is motivated by N EURO M ATCH (Rex et al., 2020) where a subgraph prediction function of this form satisﬁes the key properties of subgraph isomorphism: transitivity, anti-symmetry, intersection set and non-trivial intersection.
Hence, P must satisfy:
[Equation]
Again, E being arbitrary, Eq.
17 holds if and only if, F satisﬁes:
[Equation]
Any F that satisﬁes Eqs.
15, 16 and 18 is acceptable in our framework.
However, it is not trivial to characterize such functions.
To show a complete constructive characterization we add two important properties in our desiderata for F .
To motivate these properties, we ﬁrst discuss about F g in G ED and then S ED .
E. G ED :
For G ED , we require the following three properties:
[Equation]
With these properties in Eqs.
19, 20, and 21; F g is a metric.
However, giving a complete constructive characterization of metrics is not trivial.
To achieve this, we establish an important connection of metrics on vector spaces to norms.
Every norm (cid:107) .
(cid:107) gives a metric ( x, y )
(cid:55)→ (cid:107) x − y (cid:107) .
Moreover for a metric, there exists a norm (cid:107) .
(cid:107) such that the metric can be expressed as ( x, y )
(cid:55)→ (cid:107) x − y (cid:107) , if and only if the metric is translation invariant and homogeneous .
Thus, we add these properties to the desiderata for F g :
[Equation]
Observation 5 The followings give a complete constructive characterization of F g satisfying Eqs.
F g is ( x, y )
(cid:55)→ (cid:107) x − y (cid:107) for some norm (cid:107) .
(cid:107) over the vector space R d . 2.
For any norm (cid:107) .
(cid:107) over the vector space R d , ( x, y )
(cid:55)→ (cid:107) x − y (cid:107) is a valid F g .
Hence, the L 2 norm is a natural choice because it is isotropic (i.e. direction/rotation invariant).
E. S ED :
For S ED , we add translation invariance and non-negative homogeneity to the desiderata, to make a complete constructive characterization feasible:
[Equation]
Note that non-negativity of r in Eq.
25 is necessary to be consistent with Eq.
If r is negative, then with r = − 1 , we will have x ≤ y =
⇒ F ( x, y )
=
= ⇒ F ( − x, − y )
= 
 =
⇒ − x ≤ − y =
⇒ x ≥ y , which is a contradiction for x (cid:54) =
y .
For these two properties, translation invariance and non-negative homogeneity, we provide intuitive interpretations in the context of S ED .
The translation invariance would imply that if graphs G and
H are modiﬁed in the same way then the S ED ( G, H ) will be preserved.
The positive homogeneity would mean that if the costs are scaled by a positive factor, then the S ED is also scaled by the same positive factor.
From Eq.
24, we have F ( g, h ) =
F ( g − h, 0) =
p ( g − h ) , where p : R d → R is deﬁned as p ( x ) =
F ( x, 0) .
Then Eqs.
16 and 25 are satisﬁed if and only if:
[Equation] [Equation]
Here, Eq.
26 reﬂects subadditivity property and Eq.
27 is nonnegative homogeneity.
With these properties, p is an asymmetric seminorm .
Additionally, Eq. 18 is satisﬁed if and only if:
[Equation]
We call this Eq.
28 as rectiﬁcation property and refer to the seminorms satisfying this property as rectiﬁed seminorms .
Our claim is that F satisﬁes all the desired properties presented earlier, if and only if p is a rectiﬁed asymmetric seminorm.
Thus, we have the following complete characterization for F .
Observation 6 The followings give a complete characterization for F :
F is ( g, h ) (cid:55)→ p ( g − h ) for some rectiﬁed asymmetric seminorm p .
For any rectiﬁed asymmetric seminorm p , ( g, h )
(cid:55)→ p
( g − h ) is a valid F .
Now, we describe the following characterization for p .
Recall that in d -dimensional vector spaces, asymmetric seminorms are in one-to-one correspondence with convex sets containing the origin.
Speciﬁcally, every asymmetric seminorm p corresponds to a convex set B , containing the origin, called the dual unit ball of p .
Furthermore, p and B are related as follows:
[Equation]
Additionally, p is symmetric if and only if B is symmetric about the origin and p is positive deﬁnite (i.e. p ( x ) =
=
⇒ x = 0 )
if and only if B contains the origin in its topological interior.
Note that monotonic norms are also asymmetric seminorms.
Lemma 8 (Rectiﬁed dual unit ball) Eq.
28 holds if and only if B satisﬁes the following:
[Equation]
In other words, the necessary and sufﬁcient conditions say that B lies completely in the positive orthant.
P ROOF .
( = ⇒ )
If z [ i ] < 0 for some i and some z ∈ B , then taking x in the negative orthant, with x [ i ] having a sufﬁciently large magnitude, gives positive z · x , which leads to a positive p ( x ) , contradicting the requirement that p ( x ) = 0 when ∀ i : x [ i ] ≤ 0 . 2.
( ⇐ = )
If ∀ z ∈ B : ∀ i : z [ i ] ≥ 0 , then for any x in the negative orthant, z · x ≤ 0 , ∀ z ∈ B . 0 ∈ B , so the upper bound of 0 is achievable, hence p ( x ) = 0 .
[Equation]
Note that, Lemma 8 implies that the origin is at the boundary of B , which negates the positive deﬁniteness of p .
Hence, every valid p can be constructed by ﬁrst choosing a convex set B in the positive orthant of R d and then deﬁning p ( x ) using Eq. 29.
Conversely, any such construction gives a valid p .
However, depending on B , it may not be always be feasible to simplify or solve the optimization problem of Eq. 29.
In the special case where B is restricted to polytopes, Eq. 29 is a
Linear Program.
With recently proposed differentiable Linear Programming layers, B could be learnt in an end-to-end fashion via backpropagation (with non-negativity constraints on the variables to enforce the positive orthant).
However, this adds more parameters to learn and hyperparameters to tune, and makes training and inference signiﬁcantly costlier.
Instead we add a different, and arguably milder, restriction on B , to get a closed form for Eq. 29 in terms of norms.
Speciﬁcally, we assume that B is such that mirroring it on all orthants results in a convex set.
Not surprisingly, this closed form coincides with the form of F used in the paper.
This establishes that our embedding space prediction function is a natural choice for modelling S ED .
First, we show a characterization for the dual unit ball of a monotonic norm.
Recall that a norm (cid:107) .
(cid:107) on the vector space
R
d is said to be monotonic if ( ∀ i : | x (cid:48)
[ i ] | ≥ | x [ i ] | ) =
⇒ ( (cid:107) x (cid:48) (cid:107) ≥ (cid:107) x
(cid:107) ) .
This characterization allows us to construct the dual unit ball of a rectiﬁed asymmetric seminorm p from the dual unit ball of a monotonic seminorm (cid:107) .
(cid:107) , and vice versa, such that p ( x ) =
(cid:107) max (0 , x ) (cid:107) .
Lemma 9 Let (cid:107) .
(cid:107) be a norm over R d .
Let B (cid:48) be its dual unit ball.
(cid:107) .
(cid:107) is monotonic if and only if the following holds:
∀ x : ∀ i : z
[ i ] x
[ i ] ≥ 0 , where, z ∈ arg max z ∈ B (cid:48) z · x .
P ROOF .
( = ⇒ )
Assume that z
[ i ] x
[ i ] < 0 for some i and x .
Wlog, assume x
[ i ] > 0 .
Then, by assumption, z
[ i ] < 0 .
Construct x (cid:48) by replacing x [ i ] in x (cid:48) with x [ i ] + ∆ , with ∆ > 0 .
Let z (cid:48) ∈ arg max z (cid:48)
∈ B (cid:48)
z (cid:48) · x (cid:48) .
By continuity, for sufﬁciently small ∆ , there is a z (cid:48)
which is arbitrarily close to z .
In particular, for such ∆ , z (cid:48)
[ i ] < 0 .
This gives z (cid:48) · x (cid:48) <
z (cid:48) · x .
But monotonicity gives z (cid:48) · x (cid:48) ≥ z · x and maximality of z for x gives z · x ≥ z (cid:48) · x , which on chaining give z (cid:48) · x (cid:48) ≥ z (cid:48) · x .
This contradicts the previous inequality.
( ⇐ = ) Consider x and x (cid:48) with | x [ i ] | ≤ | x (cid:48)
[ i ] | , ∀ i .
Let z ∈ arg max z ∈ B (cid:48)
z · x , such that z [ i ] x [ i ] ≥ 0 , ∀ i .
Then (cid:107) x (cid:48) (cid:107) ≥ z · x (cid:48) ≥
z · x = (cid:107) x (cid:107) .
Hence, (cid:107) .
(cid:107) is monotonic.
[Equation]
Lemma 10 Let (cid:107) .
(cid:107) be a monotonic norm.
Then x (cid:55)→ (cid:107) max(0 , x ) (cid:107) is a rectiﬁed asymmetric seminorm.
P ROOF .
Let B (cid:48) be the dual unit ball of a monotonic norm (cid:107) .
(cid:107) .
Let B be the positive orthant of B (cid:48) .
Clearly B is convex and contains the origin, so let B be the dual unit ball of a rectiﬁed asymmetric seminorm p (see Lemma 8).
We show that (cid:107) max(0 , x ) (cid:107) =
p ( x ) , which completes the proof.
[Equation]
Here, max(0 , x ) ≥ 0 , as by Lemma 9 it sufﬁces to consider z ’s in the positive orthant, where B coincides with B (cid:48) .
Additionally, for x [ i ] < 0 as the supremum is at z [ i ] = 0 ; max(0 , x ) coincides with x at i : x [ i ] ≥ 0 .
[Equation]
Lemma 11 Let p be a rectiﬁed asymmetric seminorm with a dual unit ball B such that mirroring B on all orthants results in a convex set.
Then p is x (cid:55)→ (cid:107) max(0 , x ) (cid:107) for some monotonic norm (cid:107) .
(cid:107) .
P ROOF .
Let B be the dual unit ball of p .
Let B (cid:48)
= { z : | z | ∈ B } .
B (cid:48) is convex (by mirroring assumption), symmetric about the origin, and contains the origin in its topological interior (except in degenerate cases which we ignore), so B (cid:48) is the dual unit ball for a norm (cid:107) .
(cid:107) .
For any x , let z ∈ arg max z ∈ B (cid:48)
z · x , and consider z (cid:48) obtained from z by keeping the magnitudes of z [ i ] ’s but taking the sign of x [ i ] ’s. By construction z (cid:
) ∈ B (cid:48) , and z (cid:48) · x ≥ z · x , which gives z (cid:48) ∈ arg max z ∈ B (cid:48)
z · x .
So by Lemma 9, (cid:107) .
(cid:107) is a monotonic norm.
Now, B is the positive orthant of B (cid:48) , so by the reasoning of Lemma 10, p ( x )
= (cid:107) max(0 , x ) (cid:107) , and it completes the proof.
(cid:3)
Discussion: L p norms are natural choices for the monotonic norms (cid:107) .
(cid:107) .
For L 1 norm, the dual unit ball is a hypercube with side length 2 units and centered at the origin.
For L 2 norm, the dual unit ball is a hypersphere of unit radius centered at the origin.
The dual unit balls of the corresponding rectiﬁed asymmetric seminorms are simply the positive orthants of the hypercube and the hypersphere respectively, which easily satisfy the mirroring assumption.
We choose L 2 norm because of it’s isotropy, although other L p norms are also likely to work as well as the L 2 norm.
In fact, any rectiﬁed asymmetric seminorm whose dual unit ball is obtained by taking the positive orthant of a monotonic norm’s dual unit ball will satisfy the mirroring assumption, and will therefore be expressible in the form (cid:107) max(0 , x ) (cid:107) .
F C OMPUTATION C OST OF S ED AND G ED I NFERENCE
For this analysis, we make the simplifying assumption that the hidden dimension in the Pre-M LP ,
G IN and
Post-
M LP are all d .
The
average density of the graph is g .
The
number of hidden layers in
Pre-M LP , and Post-M LP are L , and k in G IN .
The computation cost per node for each of these components are as follows.
• Pre-M LP : The operations in the M LP involve linear transformation over the input vector x v of dimension | Σ | , followed by non-linearity.
This results in O ( |V| ( | Σ | · d + d 2 L )) cost.
• G IN : G IN aggregates information from each of the neighbors, which consumes O ( d · g ) time.
The linear transformation consumes an additional O ( d 2 ) time.
Applying non-linearity takes O ( d ) time since it is a linear pass over the hidden dimensions.
Finally these operations are repeated over each of the k hidden layers, results in a total O ( k ( d 2 + dg )) computation time per node.
Across, all nodes, the total cost is O ( |V| kd 2 + |E| kd ) time.
The degree g terms gets absorbed since each edge passes message twice across all nodes.
• Concatenation: This step consumes O ( kd ) time per node.
• Pool:
Pool iterates over the G IN representation of each node requiring O ( |V| dk ) time.
• Post-M LP :
The ﬁnal M LP takes dk dimensional vector as input and maps it to a d dimensional vector over L layers.
This consumes O ( kd 2 + d 2 L ) time.
Combining all these factors, the total inference complexity for a graph is O ( |V| ( | Σ |· d + d 2 L + kd 2 )+ |E| kd ) .
This operation is repeated on both the query and target graphs to compute their embeddings, on which distance function F is operated.
Thus, the ﬁnal cost is O ( n ( | Σ | · d + d 2 L + kd 2 ) + mkd ) , where n = |V Q | + |V T | and m =
|E Q | + |E T | .
G P IPELINE D ETAILS
Additional Model Details: We use residual connections across blocks of two G IN convolution layers.
This eases the ﬂow of gradients to the earlier layers by skipping the intermediate non-linearities.
We observed about 5 x speedup in convergence and a marginal improvement in the generalization error with residual connections.
We use 8 layers, 64 hidden dimensions and 64 embedding dimensions.
We use a single layer for pre-M LP and 2 layers with ReLU non-linearity for the other M LP ’s.
Data generation: We generate training data under two different settings (described in § 4).
S ED computation is NP-hard, and to the best of our knowledge, there is no existing method which computes S ED directly.
Therefore, we generate the ground truth S ED using existing methods on G ED by leveraging Lemma 3 which establishes the relationship between S ED and G ED .
More speciﬁcally, we use time limited mixed integer programming method M IP -F
(Lerouge et al., 2017), which provides us with the best so far lower bound and upper bound for S ED .
The time limit is kept atseconds per pair.
Each pair is run with 64 threads on a 64 core machine.
This gives sufﬁciently tight bounds (with exact values for a large majority of pairs).
For evaluation, we use S ED =
( L B + U B ) /as
the ground truth, where L B and U B are the lower and upper bounds respectively.
Fig.
6 presents a visualization of tightness of S ED bounds.
[Figure]
Loss Function: While Z G Q and Z G T be the embeddings for the query graph G Q and target graph
G T respectively, we use the following function (for training):
[Equation]
where l and u are the lower and upper bounds to S ED in our generated data.
With H , the parameters of the entire model are learned by minimizing the mean squared error.
[Equation]
Here, T is the set of all training samples.
Training details: N EURO S ED , N EURO S ED -NN and N EURO S ED -Dual were trained using the
AdamW optimizer with a weight decay of 10 − 3 .
A batch-size of 200 graph pairs per mini-batch was used.
The learning rate schedule was cyclic .
In each cycle, learning rate was increased fromto 10 − 3 for 2000 iterations and then decreased back to 0 for 2000 iterations.
Training was stopped when the best validation loss did not improve for 5 cycles.
We observe that a cyclic learning rate schedule usually gives faster and better convergence than the other attempted training schemes.
A cyclic learning rate schedule allows using higher learning rates and thus speeds up training.
It offers some regularization due to the high learning rates seen in each cycle encouraging the optimizer to ﬁnd ﬂatter regions of minima, giving better generalization.
It is also robust to hyper-parameter settings.
Subgraph search via neighborhood decomposition:
For large target graphs, it is infeasible to compute the S ED .
In this paper, we proposed a neighborhood decomposition scheme to handle subgraph similarity queries in the following setting: compute the
S ED between the query and the k -hop neighborhood centered at each node in the target graph for some suitable k .
The minimum S ED thus computed can be considered as an approximation for the true S ED .
Ideally, such an S ED would be an upper bound to the true S ED .
In our experiments the minimum S ED is usually small.
This is expected in general as the large targets graphs are likely to contain small query graphs (or their small variations).
Thus the neighborhood decomposition is satisfactory for S ED computation between small queries and large targets in practice.
One practical advantage of using neighborhood decomposition is that it facilitates locating the region of the target graph where the nearest subgraph is found.
We rank these neighbourhoods and that allows us to consider regions of the target graph in order of relevance.
This generates much smaller candidate sets and traditional expensive methods can be used to reﬁne these candidate sets further.
In this section, we seek to give further insight into the efﬁcacy of neighborhood decomposition and design guidelines for choosing the right value of k depending on the expected query distribution.
In practice, we are concerned only with connected subgraphs.
Assuming that all the nearest subgraphs (Def. 6) are connected leads to the following result:
Theorem 3 Assuming that the nearest subgraph to the query is a connected graph, a nearest subgraph S to G 1 in G 2 can be found in an l/ 2 -hop neighborhood N v , centered at some v ∈ V G 2 , where l is the length of the longest path in G 1 .
P ROOF .
By Corollary 1, there exists a nearest subgraph S of G 1 in G 2 which is also a subgraph of
G 1 ignoring node and edge labels.
So, the diameter of S is ≤ the length l of the longest path in G 1 .
Since S ⊆ G 2 , S is contained in some l/ 2 neighborhood of G 2 .
[Equation]
Finding the length of the longest path is NP-hard.
However, we do not need to ﬁnd the exact length of the longest path: any upper bound sufﬁces.
Moreover we do not even need the length of the longest path for l .
The nearest subgraph having a diameter equal to l is rare.
In practice, the diameter of the nearest subgraph is unlikely to be much higher than the diameter of the query itself.
In conclusion, subgraph similarity search suffers from the problem of an exponential search space of subgraphs.
We resolve this issue in two ways: the neighborhood decomposition allows us to consider a number of subgraphs linear in the number of nodes in the target, all of which are much smaller than the target graph itself.
Second, our neural model learns to predict the S ED directly with the small neighborhood sizes which make training data generation as well as neural processing using GNNs computationally feasible.
H D ATASETS
Dblp : Dblp is a co-authorship network where each node is an author and two authors are connected by an edge if they have co-authored a paper.
The label of a node is the venue where the authors has published most frequently.
The dataset has been obtained from https://www .
aminer .
org/citation.
Amazon : Each node in Amazon represents a product and two nodes are connected by an edge if they are frequently co-purchased.
The graph is unlabeled and hence equivalent to a graph containing a single label on all nodes.
The dataset has been downloaded from (Leskovec & Sosiˇc, 2016).
PubMed : PubMed dataset is a citation network which consists of scientiﬁc publications from PubMed database pertaining to diabetes classiﬁed into one of three classes.
Protein : Protein dataset consists of protein graphs.
Each node is labeled with a one of three functional roles of the protein.
AIDS : AIDS dataset consists of graphs constructed from the AIDS antiviral screen database.
These graphs representing molecular compounds with Hydrogen atoms omitted.
Atoms are represented as nodes and chemical bonds as edges.
CiteSeer :
CiteSeer is a citation network which consists of scientiﬁc publications classiﬁed into one of six classes.
Generally a smaller version is used for this dataset, but we use the larger version from (Bojchevski & Günnemann, 2017).
Cora_ML : Cora
dataset is
a
citation dataset
consisting
of many scientiﬁc publications classiﬁed
into
one of seven classes based on paper topic
Cora_ML is a smaller datset extracted from Cora (Bo jchevski & Günnemann, 2017)
AIDS’
AIDS’ dataset is another collection of graphs constructed from the AIDS antiviral screen database.
The graphs and their properties differ from those in AIDS .
These graphs also represent chemical compound structures
Linux : Linux dataset is a collection of program dependence graphs, where each graph is a function and nodes represent statements while edges represent dependency between statements.
IMDB : IMDB dataset is a collection of ego-networks of actors/actresses that have appeared together in any movie.
We
use
the
same
versions of AIDS’ , Linux ,
and
IMDB as used in (Bai et al.,
I E XPERIMENTAL S ETUP
Hardware Details: We use a machine with an Intel Xeon Gold 6142 processor and GeForce GTX 1080 Ti GPU for all our experiments.
Baselines: Here we describe the changes made to neural baselines to adapt to S ED problem.
• S IM G NN (Bai et al., 2019):
We train the model with L B + U Bas the ground truth value of S ED (L B and U B are lower and upper bounds for S ED respectively).
Now let G 1 be the query graph, and Gbe the target graph, we use n S ED ( G 1 , G 2 ) =
S ED ( G 1 , G 2 ) |G 1 | instead of n G ED ( G 1 , G 2 ) =
G ED ( G 1 , G 2 ) ( |G 1 | + |G 2 | ) / 2 as the normalized value of similarity in S IM G NN .
This is because the absolute value of S ED scales with the size of the query, as opposed to G ED , which scales with size of the query as well as the target.
• N EURO M ATCH (Rex et al., 2020):
We use the un-anchored version of N EURO M ATCH trained on its original task (subgraph isomorphism decision problem) using its own training framework.
For ranking, we
use the violation scores it produces directly rather than thresholding the violation and predicting subgraph isomorphism as intended.
We change the size limits for sampling query/target graphs in training to better ﬁt our data distribution.
Querying for S ED : We embed the neighbourhoods of the nodes in the given graph (or set of graphs) and store the embeddings (denoted by set Z G T ) accordingly.
For a new query G Q , we ﬁrst obtain its embedding Z G Q .
Based on the stored embeddings Z G T , we compute S ED ( Z G Q , Z G T ) for all Z
G T ∈
Z G T using the function F ( Z G Q , Z G T ) .
We sort the targets based on the obtained S ED values and it can trivially answer top-k or range queries.
From the selected targets one can extract the nearest subgraph by running a traditional algorithm to compute S ED and use the returned node mapping to interpret it.
An alternative is to choose a larger ( k (cid:48) , where k (cid:48) > k ) top set and use traditional methods to further ﬁlter these.
Note that usually the graphs required in the top-k or range queries are much smaller than the total number of targets, so using our model to ﬁlter out the unimportant targets from the target candidate set can save signiﬁcant amount of resources.
Software details: We used Pytorch and Pytorch-Geometric for implementation.
All experiments were done using Jupyter Notebooks.
The full code, notebooks for all experiments and the generated data are available on an anonymous github repository 3 .
License: The provided code and data are licensed under CC-BY-SA .
J A DDITIONAL R ESULTS J. O THER MEASURES FOR PREDICTING S ED AND G ED
In Section 4.2, we present the accuracy of all techniques on S ED and G ED in terms of Root Mean
Square Error (RMSE ).
Here we evaluate the same performance using several other metrics such as
Mean Absolute Error (MAE) and R 2 .
Tables 7a and 7b show the MAE scores for S ED and G ED respectively.
Tables 8a and 8b present the results based on R 2 results.
Consistent with the previous results, N EURO S ED outperforms all techniques both in S ED and G ED in these two measures except
[Table] [Table]
[Table] [Table]
[Figure]
[Table]
K P RECISION @ k ON k -NN QUERIES
Fig.
7 shows Precision @
k on k -NN queries.
As visible, either N EURO S ED comfortably performs better than all baselines or is among the best performing algorithms.
K. A DDITIONAL R ESULTS ON A BLATION S TUDY
Impact of G IN :
To highlight the importance of G IN , we conduct ablation studies by replacing the
GIN convolution layers in the model with several other convolution layers.
As visible in the Table 10,
G IN consistently achieves the best accuracy.
This is not surprising since G IN is provably the most expressive among G NN s in distinguishing graph structures (essential to S ED or G ED computation) and is as powerful as the Weisfeiler-Lehman Graph Isomorphism test (Xu et al., 2018b)
[Table]
Impact of sum-pool: To substantiate our choice of the pooling layer, we have performed ablation studies with various pooling functions as replacements for sum-pool.
It is clear from Table 11 that sum-pool is the best choice among the considered alternatives.
[Table] [Table]
K. A LIGNMENT
In real-world applications of subgraph similarity search, alignments are of interest only for a small number of similar subgraphs.
Our framework is intended to serve as a ﬁlter to retrieve this small set of similar subgraphs from a large number of candidates.
To elaborate, a graph database may contain thousands or millions of graphs (or alternatively, thousands or millions of neighborhoods of a large graph) which need to be inspected for similar subgraphs.
A user is typically interested in only a handful of these subgraphs that are highly similar to the query.
Since the ﬁltered set is signiﬁcantly smaller, a non-neural exact algorithm sufﬁces to construct the alignments (Lemma 1 allows us to adapt general cost G ED alignment techniques for S ED alignment).
Computing alignments across the entire database is unnecessary and slows down the query response time.
To substantiate our claim, we show the average running time for answering 10 -NN queries.
We break up the running time into two components: (i) k -NN retrieval time by N EURO S ED , (ii) exact alignment time using MIP-F2 for the 10 -NN neighborhoods retrieved by N EURO S ED .
We observe that exact alignment by existing methods on the 10 -NN neighborhoods completes in reasonable time.
In contrast, G ENN -A ∗ does not scale on either PubMed or Amazon since it computes alignments across all (sub)graphs.
[Table] [Table]
In Figures 8 to 17, we show the variation of the errors on S ED and G ED prediction with query sizes and ground truth values for N EURO S ED (N EURO S ED G for G ED ) and the baselines on all the corresponding datasets.
This experiment is an extension of the heat-map results in Fig.
2 in the main paper.
These datasets show variations in the distributions of S ED and G ED values.
It is interesting to observe that among the baselines, different methods perform well on different regions (e.g., combinations of high/low S ED and high/low query sizes).
In the ﬁgures, darker means higher error.
The baselines do not show good performance on all regions.
However, for N EURO S ED and
N EURO S ED G , we see a much better coverage for all types of regions in the domain.
Furthermore, for every region, our models outperform (or are at least competitive with) the best performing baseline for that region.
[Figure]
[Figure]
M S CALABILITY WITH QUERY SIZE AND GENERALIZABILITY TO UNSEEN LARGER QUERY SIZES
In this analysis
[Figure]
we aim to analyze the following aspects:
Performance on larger query sizes.
Generalizability to unseen larger query sizes
In Table 13 sheds light on the above objectives.
We notice that although there is some deterioration in the quality for query sizes in the range [25 , 50] when compared to the entire set, it is not severe (N EURO S ED -50 in Table 13).
However, if we the train set only contains queries till size 25 and we deploy the learned model to infer on queries of larger unseen sizes, the drop in quality is signiﬁcant (N EURO S ED -25 in Table 13).
Nonetheless, N EURO S ED remains superior to the optimal non-neural approach (MIP-F2) when run with a generous time limit of 60 seconds per query.
Overall, this experiment highlights one direction that needs further study and improvement
[Table]
[Figure] [Figure]
N V ISUALIZATION AND A PPLICATION OF S ED
Searching for molecular fragment containment is a routine task in drug discovery (Ranu et al., 2011).
Motivated by this
[Figure]
we show the top- 5 matches to an S ED query on the AIDS dataset produced by
N EURO S ED in Fig.
The query is a functional group (Hydrogen atoms are not represented).
N EURO S ED is able to extract chemical compounds that contain this molecular fragment (except for ranks 3 and 5 , which contain this group with 1 edit) from around 2000 chemical compounds with varying sizes and structures.
This validates the efﬁcacy of N EURO S ED at a semantic level
[Figure]
O A LTERNATIVE Q UERY D ISTRIBUTIONS
In R WR
[Figure]
we perform ﬁxed length random walks, where the length of a walk is the average diameter size of the queries generated through BFS during training.
Next, we merge the walks to form a graph.
In R W , we perform random walks, till the diameter of the resultant graph is the same as the average diameter of the BFS sampled train graphs.
The details of the SHA D OW sampler is explained in Zeng et al.
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]