NeuroSED: Learning Subgraph Similarity via Graph Neural Networks

1 INTRODUCTION AND RELATED WORK . Graphs are used to model data in a wide variety of domains . Examples include chemical compounds ( Sankar et al. , 2017 ) , protein-protein interaction networks ( PPI ) ( Alon et al. , 2008 ) , knowledge graphs ( Ebsch et al. , 2020 ) , and social networks ( Kempe et al. , 2003 ) . A distance function on any dataset , including graphs , is a fundamental operator . Among several distance measures on graphs , edit distance is one of the most powerful and popular mechanisms ( Liang & Zhao , 2017 ; Zhao et al. , 2013 ; Bougleux et al. , 2017 ; Daller et al. , 2018 ; Fankhauser et al. , 2011 ; Riesen & Bunke , 2009 ; Zeng et al. , 2009 ; He & Singh , 2006 ) . Edit distance can be posed in two forms : graph edit distance ( GED ) and subgraph edit distance ( SED ) . Given two graphs G1 and G2 , GED ( G1 , G2 ) returns the minimum cost of edits needed to convert G1 to G2 . i.e. , for G1 to become isomorphic to G2 . An edit can be the addition or deletion of edges and nodes , or replacement of edge or node labels , with an associated cost . In SED ( G1 , G2 ) , the goal is to identify the minimum cost of edits so that G1 is a subgraph ( subgraph isomorphic ) of G2 . For examples , see Fig . 5a in the appendix . While the applications of GED and SED are beyond doubt , their applicability is constrained by their computation costs . Specifically , both GED and SED are NP-complete ( Zeng et al. , 2009 ; Neuhaus et al. , 2006 ; He & Singh , 2006 ) . To mitigate this computational bottleneck , several heuristics ( Bougleux et al. , 2017 ; Daller et al. , 2018 ; Fankhauser et al. , 2011 ; Riesen & Bunke , 2009 ; Neuhaus et al. , 2006 ) and index structures ( He & Singh , 2006 ; Zeng et al. , 2009 ; Liang & Zhao , 2017 ; Zhao et al. , 2013 ) have been proposed . Recently , graph neural networks have been shown to be effective in learning and predicting GED ( Bai et al. , 2019 ; Wang et al. , 2021 ; Li et al. , 2019 ; Bai et al. , 2020 ; Zhang et al. , 2021 ; Xiu et al. , 2020 ) and subgraph isomorphism ( Rex et al. , 2020 ) . The basic goal in all these algorithms is to learn a neural model from a training set of graph pairs and their distances , such that , at inference time , given an unseen graph pair , we are able to predict its distance accurately . Wang et al . ( 2020 ) designed an algorithm to characterize graphs through sampling random walks and then embedding them in a feature space . While Wang et al . ( 2020 ) has not been studied in the context of GED or SED , the authors have shown that they can detect graph isomorphism . While this progress in the area of graph querying is undoubtedly impressive , there is scope to do more . • Modeling SED : Existing neural approaches to learning GED can not easily be adapted to learn SED . While GED is symmetric , SED is not . Most neural architectures for GED have the assumption of symmetry at its core and hence modeling SED is non-trivial . In App . A , we provide a detailed analysis of the limitations of prior architectures in failing to model SED . • Exponential Search Space : Computing SED ( G1 , G2 ) conceptually requires us to compare the query graph G1 with the exponentially many subgraphs of the target graph G2 . Therefore , it is imperative that the model has an efficient and effective mechanism to prune the search space without compromising on the prediction accuracy . We note that while several index structures and heuristics exist for GED ( He & Singh , 2006 ; Zeng et al. , 2009 ; Bai et al. , 2019 ; Wang et al. , 2021 ) , none exist for SED . Thus , scalability of SED on large graphs remains an unsolved problem . • Preservation of theoretical properties : GED is a metric distance function . While SED is not metric due to being asymmetric , it satisfies the triangle inequality , non-negativity , and subgraphidentity . Several higher-order tasks such as clustering and indexing rely on such properties ( Hadi , 1991 ; Dohnal et al. , 2003 ; Hjaltason & Samet , 2003 ; Samet , 2005 ; Uhlmann , 1991 ; Yianilos , 1993 ; Zezula et al. , 2006 ; Brin , 1995 ; Chávez et al. , 2001 ) . Existing neural approaches do not preserve these properties , which limits their usability for these higher order tasks . • Pair-independent computation : There is little scope for pre-computation in existing approaches ( either neural or non-neural ) , as the major computations are pair-dependent , i.e. , both G1 and G2 need to be known . In a typical graph querying framework , the graph database is known apriori ; only the query graph is provided at runtime . If we can generate pair-independent embeddings , and make efficient predictions directly in the embedding space , then retrieval can be sped up tremendously by pre-computing and indexing the embeddings of the database graphs beforehand . • Learning with low-volume data : Generating a large amount of high quality training data is computationally prohibitive for SED/GED . So , an effective model should encode powerful priors for SED/GED prediction , to enable generalization even with scarce training data . We address these challenges by a novel architecture called NEUROSED . Our key contributions are : • Novel formulation : We formulate the problem of learning subgraph edit distance ( SED ) for graphs . Since SED is more general than subgraph isomorphism and GED , the proposed theory extends to prediction of GED , graph isomorphism and subgraph isomorphism . • Neural architecture : NEUROSED utilizes a siamese graph isomorphism network ( Xu et al. , 2018a ) to embed graphs in a pair-independent fashion to an embedding space over which a simple function can predict the SED ( or GED ) . The same embedding model is used for both G1 and G2 , which captures the prior that similar topological properties need to be considered for both graphs . The carefully crafted prediction function serves as another inductive bias for the model , which , in addition to enabling high generalization accuracy , preserves the key properties of SED/GED , which is a major contribution of our work over existing neural approaches . • Indexable embeddings : The prediction function satisfies the triangle inequality over the embedding space for both SED and GED . This allows utilization of the rich literature on index structures ( Elkan , 2003 ; Kwedlo & Czochanski , 2019 ; Ciaccia et al. , 1997 ) to boost efficiency . • Accurate , Fast and Scalable : Extensive experiments on real graph datasets containing up to a million nodes establish that NEUROSED is more accurate in both GED and SED when compared to the state of the art and is more than 3 orders of magnitude faster in range and k-NN queries . 2 PRELIMINARIES AND PROBLEM FORMULATION . We denote a labeled undirected graph as G = ( V , E , L ) where V is the node set , E is the edge set and L : V ∪ E → Σ is the labeling function over nodes and edges . Σ is the universe of all labels and contains a special empty label . L ( v ) and L ( e ) denote the labels of node v and edge e respectively . G1 ⊆ G2 denotes that G1 is a subgraph of G2 . All notations used in our work are summarized in Table 6 in the appendix . For the definitions of subgraph and graph isomorphism , refer to App . B in the supplementary . The computation of GED relies on a graph mapping . Definition 1 ( Graph Mapping ) Given two graphs G1 and G2 , let G̃1 = ( Ṽ1 , Ẽ1 , L̃1 ) and G̃2 = ( Ṽ2 , Ẽ2 , L̃2 ) be obtained by adding dummy nodes and edges ( labeled with ) to G1 and G2 respectively , such that |V1| = |V2| and |E1| = |E2| . A node mapping between G1 and G2 is a bijection π : G̃1 → G̃2 where ( i ) ∀v ∈ Ṽ1 , π ( v ) ∈ Ṽ2 and at least one of v and π ( v ) is not a dummy ; ( ii ) ∀e = ( v1 , v2 ) ∈ Ẽ1 , π ( e ) = ( π ( v1 ) , ( π ( v2 ) ) ) ∈ Ẽ2 and at least one of e and π ( e ) is not a dummy . Example 1 Fig . 5b shows a graph mapping . Edge mappings can be trivially inferred , so are omitted . Definition 2 ( Graph Edit Distance ( GED ) under mapping π ) GED between G1 and G2 under π is GEDπ ( G1 , G2 ) = ∑ v∈Ṽ1 d ( L ( v ) , L ( π ( v ) ) ) + ∑ e∈Ẽ1 d ( L ( e ) , L ( π ( e ) ) ) ( 1 ) where d : Σ× Σ→ R+0 is a distance function over the label set . d ( ` 1 , ` 2 ) models an insertion if ` 1 = , deletion if ` 2 = and replacement if ` 1 6= ` 2 and neither ` 1 nor ` 2 is a dummy . We assume d to be a binary function , where d ( ` 1 , ` 2 ) = 1 if ` 1 6= ` 2 , otherwise , 0 . Our framework easily extends to more general distance functions ( details in App . D ) . Example 2 GED for the shown mapping in Fig . 5b is 3 . The red mappings incur a cost of 1 . Definition 3 ( Graph Edit Distance ( GED ) ) GED is the minimum distance under all mappings . GED ( G1 , G2 ) = min ∀π∈Φ ( G1 , G2 ) GEDπ ( G1 , G2 ) ( 2 ) where Φ ( G1 , G2 ) denotes the set of all possible node maps from G1 to G2 . Definition 4 ( Subgraph Edit Distance ( SED ) ) SED is the minimum GED over all subgraphs of G2 . SED ( G1 , G2 ) = min S⊆G2 GED ( G1 , S ) ( 3 ) Example 3 Revisiting Fig . 5b , GED ( G1 , G2 ) = 3 . The shown mapping incurs the minimum cost . SED ( G1 , G2 ) = 1 . Refer to Fig . 5a for more examples . Problem 1 ( Learning SED ) Given a training set of tuples of the form 〈G1 , G2 , SED ( G1 , G2 ) 〉 , learn a neural model to predict SED ( Q1 , Q2 ) on unseen graphs Q1 and Q2 . Problem 2 ( Learning GED ) Given a training set of tuples of the form 〈G1 , G2 , GED ( G1 , G2 ) 〉 , learn a neural model to predict GED ( Q1 , Q2 ) on unseen graphs Q1 and Q2 .

Summary: This paper presents a simple and effective model for subgraph similarity search. Two GNNs with shared weights are applied first to get the graph embeddings of the query and target graphs. Then, instead of using an MLP over the concatenated graph embeddings to regress the subgraph edit distance (SED) score, the paper proposes to use the l2 norm of the positive portion of the difference between two embeddings. This inductive bias is proved to preserve SED's triangle inequality property, and shows better accuracy and generalization power for large graphs. Experiments show the proposed model outperforms state-of-the-art neural approaches in both graph similarity search and subgraph similarity search tasks.