NeuroSED: Learning Subgraph Similarity via Graph Neural Networks

1 INTRODUCTION AND RELATED WORK . Graphs are used to model data in a wide variety of domains . Examples include chemical compounds ( Sankar et al. , 2017 ) , protein-protein interaction networks ( PPI ) ( Alon et al. , 2008 ) , knowledge graphs ( Ebsch et al. , 2020 ) , and social networks ( Kempe et al. , 2003 ) . A distance function on any dataset , including graphs , is a fundamental operator . Among several distance measures on graphs , edit distance is one of the most powerful and popular mechanisms ( Liang & Zhao , 2017 ; Zhao et al. , 2013 ; Bougleux et al. , 2017 ; Daller et al. , 2018 ; Fankhauser et al. , 2011 ; Riesen & Bunke , 2009 ; Zeng et al. , 2009 ; He & Singh , 2006 ) . Edit distance can be posed in two forms : graph edit distance ( GED ) and subgraph edit distance ( SED ) . Given two graphs G1 and G2 , GED ( G1 , G2 ) returns the minimum cost of edits needed to convert G1 to G2 . i.e. , for G1 to become isomorphic to G2 . An edit can be the addition or deletion of edges and nodes , or replacement of edge or node labels , with an associated cost . In SED ( G1 , G2 ) , the goal is to identify the minimum cost of edits so that G1 is a subgraph ( subgraph isomorphic ) of G2 . For examples , see Fig . 5a in the appendix . While the applications of GED and SED are beyond doubt , their applicability is constrained by their computation costs . Specifically , both GED and SED are NP-complete ( Zeng et al. , 2009 ; Neuhaus et al. , 2006 ; He & Singh , 2006 ) . To mitigate this computational bottleneck , several heuristics ( Bougleux et al. , 2017 ; Daller et al. , 2018 ; Fankhauser et al. , 2011 ; Riesen & Bunke , 2009 ; Neuhaus et al. , 2006 ) and index structures ( He & Singh , 2006 ; Zeng et al. , 2009 ; Liang & Zhao , 2017 ; Zhao et al. , 2013 ) have been proposed . Recently , graph neural networks have been shown to be effective in learning and predicting GED ( Bai et al. , 2019 ; Wang et al. , 2021 ; Li et al. , 2019 ; Bai et al. , 2020 ; Zhang et al. , 2021 ; Xiu et al. , 2020 ) and subgraph isomorphism ( Rex et al. , 2020 ) . The basic goal in all these algorithms is to learn a neural model from a training set of graph pairs and their distances , such that , at inference time , given an unseen graph pair , we are able to predict its distance accurately . Wang et al . ( 2020 ) designed an algorithm to characterize graphs through sampling random walks and then embedding them in a feature space . While Wang et al . ( 2020 ) has not been studied in the context of GED or SED , the authors have shown that they can detect graph isomorphism . While this progress in the area of graph querying is undoubtedly impressive , there is scope to do more . • Modeling SED : Existing neural approaches to learning GED can not easily be adapted to learn SED . While GED is symmetric , SED is not . Most neural architectures for GED have the assumption of symmetry at its core and hence modeling SED is non-trivial . In App . A , we provide a detailed analysis of the limitations of prior architectures in failing to model SED . • Exponential Search Space : Computing SED ( G1 , G2 ) conceptually requires us to compare the query graph G1 with the exponentially many subgraphs of the target graph G2 . Therefore , it is imperative that the model has an efficient and effective mechanism to prune the search space without compromising on the prediction accuracy . We note that while several index structures and heuristics exist for GED ( He & Singh , 2006 ; Zeng et al. , 2009 ; Bai et al. , 2019 ; Wang et al. , 2021 ) , none exist for SED . Thus , scalability of SED on large graphs remains an unsolved problem . • Preservation of theoretical properties : GED is a metric distance function . While SED is not metric due to being asymmetric , it satisfies the triangle inequality , non-negativity , and subgraphidentity . Several higher-order tasks such as clustering and indexing rely on such properties ( Hadi , 1991 ; Dohnal et al. , 2003 ; Hjaltason & Samet , 2003 ; Samet , 2005 ; Uhlmann , 1991 ; Yianilos , 1993 ; Zezula et al. , 2006 ; Brin , 1995 ; Chávez et al. , 2001 ) . Existing neural approaches do not preserve these properties , which limits their usability for these higher order tasks . • Pair-independent computation : There is little scope for pre-computation in existing approaches ( either neural or non-neural ) , as the major computations are pair-dependent , i.e. , both G1 and G2 need to be known . In a typical graph querying framework , the graph database is known apriori ; only the query graph is provided at runtime . If we can generate pair-independent embeddings , and make efficient predictions directly in the embedding space , then retrieval can be sped up tremendously by pre-computing and indexing the embeddings of the database graphs beforehand . • Learning with low-volume data : Generating a large amount of high quality training data is computationally prohibitive for SED/GED . So , an effective model should encode powerful priors for SED/GED prediction , to enable generalization even with scarce training data . We address these challenges by a novel architecture called NEUROSED . Our key contributions are : • Novel formulation : We formulate the problem of learning subgraph edit distance ( SED ) for graphs . Since SED is more general than subgraph isomorphism and GED , the proposed theory extends to prediction of GED , graph isomorphism and subgraph isomorphism . • Neural architecture : NEUROSED utilizes a siamese graph isomorphism network ( Xu et al. , 2018a ) to embed graphs in a pair-independent fashion to an embedding space over which a simple function can predict the SED ( or GED ) . The same embedding model is used for both G1 and G2 , which captures the prior that similar topological properties need to be considered for both graphs . The carefully crafted prediction function serves as another inductive bias for the model , which , in addition to enabling high generalization accuracy , preserves the key properties of SED/GED , which is a major contribution of our work over existing neural approaches . • Indexable embeddings : The prediction function satisfies the triangle inequality over the embedding space for both SED and GED . This allows utilization of the rich literature on index structures ( Elkan , 2003 ; Kwedlo & Czochanski , 2019 ; Ciaccia et al. , 1997 ) to boost efficiency . • Accurate , Fast and Scalable : Extensive experiments on real graph datasets containing up to a million nodes establish that NEUROSED is more accurate in both GED and SED when compared to the state of the art and is more than 3 orders of magnitude faster in range and k-NN queries . 2 PRELIMINARIES AND PROBLEM FORMULATION . We denote a labeled undirected graph as G = ( V , E , L ) where V is the node set , E is the edge set and L : V ∪ E → Σ is the labeling function over nodes and edges . Σ is the universe of all labels and contains a special empty label . L ( v ) and L ( e ) denote the labels of node v and edge e respectively . G1 ⊆ G2 denotes that G1 is a subgraph of G2 . All notations used in our work are summarized in Table 6 in the appendix . For the definitions of subgraph and graph isomorphism , refer to App . B in the supplementary . The computation of GED relies on a graph mapping . Definition 1 ( Graph Mapping ) Given two graphs G1 and G2 , let G̃1 = ( Ṽ1 , Ẽ1 , L̃1 ) and G̃2 = ( Ṽ2 , Ẽ2 , L̃2 ) be obtained by adding dummy nodes and edges ( labeled with ) to G1 and G2 respectively , such that |V1| = |V2| and |E1| = |E2| . A node mapping between G1 and G2 is a bijection π : G̃1 → G̃2 where ( i ) ∀v ∈ Ṽ1 , π ( v ) ∈ Ṽ2 and at least one of v and π ( v ) is not a dummy ; ( ii ) ∀e = ( v1 , v2 ) ∈ Ẽ1 , π ( e ) = ( π ( v1 ) , ( π ( v2 ) ) ) ∈ Ẽ2 and at least one of e and π ( e ) is not a dummy . Example 1 Fig . 5b shows a graph mapping . Edge mappings can be trivially inferred , so are omitted . Definition 2 ( Graph Edit Distance ( GED ) under mapping π ) GED between G1 and G2 under π is GEDπ ( G1 , G2 ) = ∑ v∈Ṽ1 d ( L ( v ) , L ( π ( v ) ) ) + ∑ e∈Ẽ1 d ( L ( e ) , L ( π ( e ) ) ) ( 1 ) where d : Σ× Σ→ R+0 is a distance function over the label set . d ( ` 1 , ` 2 ) models an insertion if ` 1 = , deletion if ` 2 = and replacement if ` 1 6= ` 2 and neither ` 1 nor ` 2 is a dummy . We assume d to be a binary function , where d ( ` 1 , ` 2 ) = 1 if ` 1 6= ` 2 , otherwise , 0 . Our framework easily extends to more general distance functions ( details in App . D ) . Example 2 GED for the shown mapping in Fig . 5b is 3 . The red mappings incur a cost of 1 . Definition 3 ( Graph Edit Distance ( GED ) ) GED is the minimum distance under all mappings . GED ( G1 , G2 ) = min ∀π∈Φ ( G1 , G2 ) GEDπ ( G1 , G2 ) ( 2 ) where Φ ( G1 , G2 ) denotes the set of all possible node maps from G1 to G2 . Definition 4 ( Subgraph Edit Distance ( SED ) ) SED is the minimum GED over all subgraphs of G2 . SED ( G1 , G2 ) = min S⊆G2 GED ( G1 , S ) ( 3 ) Example 3 Revisiting Fig . 5b , GED ( G1 , G2 ) = 3 . The shown mapping incurs the minimum cost . SED ( G1 , G2 ) = 1 . Refer to Fig . 5a for more examples . Problem 1 ( Learning SED ) Given a training set of tuples of the form 〈G1 , G2 , SED ( G1 , G2 ) 〉 , learn a neural model to predict SED ( Q1 , Q2 ) on unseen graphs Q1 and Q2 . Problem 2 ( Learning GED ) Given a training set of tuples of the form 〈G1 , G2 , GED ( G1 , G2 ) 〉 , learn a neural model to predict GED ( Q1 , Q2 ) on unseen graphs Q1 and Q2 .

Summary: The paper addresses the problem of graph/subgraph similarity search in terms of the edit distance, termed GED/SED, respectively. Unfortunately, both GED and SED are NP hard, with exponential search space, making the cost prohibitive on even moderately large graphs/queries. Instead, the authors apply a neural network based approach, by designing a siamese graph neural network called NEUROSED. In particular, NEUROSED is able to preserve theoretical properties of  SED/GED, such as triangle inequality, through a purposely designed prediction function. Experiments are conducted on real graph datasets for both SED and GED, which evaluates the effectiveness and efficiency of GED/SED.