Neural Subgraph Isomorphism Counting

1 INTRODUCTION . Graphs are general data structures widely used in many applications , including social network analysis , molecular structure analysis , natural language processing and knowledge graph modeling , etc . Learning with graphs has recently drawn much attention as neural network approaches to representation learning have been proven to be effective for complex data structures ( Niepert et al. , 2016 ; Kipf & Welling , 2017 ; Hamilton et al. , 2017b ; Schlichtkrull et al. , 2018 ; Velickovic et al. , 2018 ; Xu et al. , 2019 ) . Most of existing graph representation learning algorithms focus on problems such as node classification , linking prediction , community detection , etc . ( Hamilton et al. , 2017a ) . These applications are of more local decisions for which a learning algorithm can usually make inferences by inspecting the local structure of a graph . For example , for the node classification problem , after several levels of neighborhood aggregation , the node representation may be able to incorporate sufficient higher-order neighborhood information to discriminate different classes ( Xu et al. , 2019 ) . In this paper , we study a more global learning problem : learning to count subgraph isomorphisms ( counting examples are shown as Figure 1 ) . Although subgraph isomorphism is the key to solve graph representation learning based applications ( Xu et al. , 2019 ) , tasks of identifying or counting subgraph isomorphisms themselves are also significant and may support broad applications , such as bioinformatics ( Milo et al. , 2002 ; Alon et al. , 2008 ) , chemoinformatics ( Huan et al. , 2003 ) , and online social network analysis ( Kuramochi & Karypis , 2004 ) . For example , in a social network , we can solve search queries like “ groups of people who like X and visited Y-city/state. ” In a knowledge graph , we can answer questions like “ how many languages are there in Africa speaking by people living near the banks of the Nile River ? ” Many pattern mining algorithms or graph database indexing based approaches have been proposed to tackle subgraph isomorphism problems ( Ullmann , 1976 ; Cordella et al. , 2004 ; He & Singh , 2008 ; Han et al. , 2013 ; Carletti et al. , 2018 ) . However , these approaches can not be applied to large-scale graphs because of the exponential time complexity . Thanks to the powerful graph representation learning models which can effectively capture local structural information , we can use a learning algorithm to learn how to count subgraph isomorphisms from a lot of examples . Then the algorithm can scan a large graph and memorize all necessary local information based on a query pattern graph . In this case , although learning based approaches can be inexact , we can roughly estimate the range of the number of subgraph isomorphism . This can already help many applications that do not require exact match or need a more efficient pre- processing step . To this end , in addition to trying different representation learning architectures , we develop a dynamic intermedium attention memory network ( DIAMNet ) to iteratively attend the query pattern and the target data graph to memorize different local subgraph isomorphisms for global counting . To evaluate the learning effectiveness and efficiency , we develop a small ( ≤ 1,024 subgraph isomorphisms in each graph ) and a large ( ≤ 4,096 subgraph isomorphisms in each graph ) dataset and evaluate different neural network architectures . Our main contributions are as follows . • To our best knowledge , this is the first work to model the subgraph isomorphism counting problem as a learning problem , for which both the training and prediction time complexities are polynomial . • We exploit the representation power of different deep neural network architectures in an end-toend learning framework . In particular , we provide universal encoding methods for both sequence models and graph models , and upon them we introduce a dynamic intermedium attention memory network to address the more global inference problem for counting . • We conduct extensive experiments on developed datasets which demonstrate that our framework can achieve good results on both relatively large graphs and large patterns compared to existing studies . 2 RELATED WORK . Subgraph Isomophism Problems . Given a pattern graph and a data graph , the subgraph isomorphism search aims to find all occurrences of the pattern in the data graph with bijection mapping functions . Subgraph isomorphism is an NP-complete problem among different types of graph matching problems ( monomorphism , isomorphism , and subgraph isomorphism ) . Most subgraph isomorphism algorithms are based on backtracking . They first obtain a series of candidate vertices and update a mapping table , then recursively revoke their own subgraph searching functions to match one vertex or one edge at a time . Ullmann ’ s algorithm ( Ullmann , 1976 ) , VF2 ( Cordella et al. , 2004 ) , and GraphQL ( He & Singh , 2008 ) belong to this type of algorithms . However , it is still hard to perform search when either the pattern or the data graph grows since the search space grows exponentially as well . Some other algorithms are designed based on graph-index , such as gIndex ( Yan et al. , 2004 ) , which can be used as filters to prune out many unnecessary graphs . However , graph-index based algorithms have a problem that the time and space in indexing also increase exponentially with the growth of the graphs ( Sun et al. , 2012 ) . TurboISO ( Han et al. , 2013 ) and VF3 ( Carletti et al. , 2018 ) add some weak rules to find candidate subregions and then call the recursive match procedure on subregions . These weak rules can significantly reduce the searching space in most cases . Graph Representation Learning . Graph ( or network ) representation learning can be directly learning an embedding vector of each graph node ( Perozzi et al. , 2014 ; Tang et al. , 2015 ; Grover & Leskovec , 2016 ) . This approach is not easy to generalize to unseen nodes . On the other hand , graph neural networks ( GNNs ) ( Battaglia et al. , 2018 ) provide a solution to representation learning for nodes which can be generalized to new graphs and unseen nodes . Many graph neural networks have been proposed since 2005 ( Gori et al. , 2005 ; Scarselli et al. , 2005 ) but rapidly developed in recent years . Most of them focus on generalizing the idea of convolutional neural networks for general graph data structures ( Niepert et al. , 2016 ; Kipf & Welling , 2017 ; Hamilton et al. , 2017b ; Velickovic et al. , 2018 ) or relational graph structures with multiple types of relations ( Schlichtkrull et al. , 2018 ) . More recently , Xu et al . ( 2019 ) propose a graph isomorphism network ( GIN ) and show its discriminative power . Others use the idea of recurrent neural networks ( RNNs ) which are originally proposed to deal with sequence data to work with graph data ( Li et al. , 2016 ; You et al. , 2018 ) . Interestingly , with external memory , sequence models can work well on complicated tasks such as language modeling ( Sukhbaatar et al. , 2015 ; Kumar et al. , 2016 ) and shortest path finding on graphs ( Graves et al. , 2016 ) . There is another branch of research called graph kernels ( Vishwanathan et al. , 2010 ; Shervashidze et al. , 2011 ; Yanardag & Vishwanathan , 2015 ; Togninalli et al. , 2019 ; Chen et al. , 2019 ) which also convert graph isomorphism to a similarity learning problem . However , they usually work on small graphs and do not focus on subgraph isomorphism identification or counting problems . 3 PRELIMINARIES . We begin by introducing the subgraph isomophism problems and then provide the general idea of our work by analyzing the time complexities of the problems . 3.1 PROBLEM DEFINITION . Traditionally , the subgraph isomorphism problem is defined between two simple graphs or two directed simple graphs , which is an NP-complete problem . We generalize the problem to a counting problem over directed heterogeneous multigraphs , whose decision problem is still NP-complete . A graph or a pattern is defined as G = ( V , E , X , Y ) where V is the set of vertices . E ⊆ V × V is the set of edges , X is a label function that maps a vertex to a vertex label , and Y is a label function that maps an edge to a set of edge labels . We use an edge with a set of edge labels to represent multiedges with the same source and the same target for clarity . That is , there are no two edges in a graph such that they have the same source , same target , and the same edge label . To simplify the statement , we assume Y ( ( u , v ) ) = φ if ( u , v ) 6∈ E . In this paper , we discuss isomorphic mappings that preserve graph topology , vertex labels and edge labels , but not vertex ids . More precisely , a pattern GP = ( VP , EP , XP , YP ) is isomorphic to a graph GG = ( VG , EG , XG , YG ) if there is a bijection f : VG → VP such that : • ∀v ∈ VG , XG ( v ) = XP ( f ( v ) ) , • ∀v ∈ VP , XP ( v ) = XG ( f−1 ( v ) ) , • ∀ ( u , v ) ∈ EG , YG ( ( u , v ) ) = YP ( ( f ( u ) , f ( v ) ) ) , • ∀ ( u , v ) ∈ EP , YP ( ( u , v ) ) = YG ( ( f−1 ( u ) , f−1 ( v ) ) ) . Furthermore , GP being isomorphic to a graph GG is denoted as GP ' GG and the function f is named as an isomorphism . A pattern GP = ( VP , EP , XP , YP ) is isomorphic to a subgraph G′G = ( V ′G , E ′G , XG , YG ) of a graph GG = ( VG , EG , XG , YG ) : V ′G ⊆ VG , E ′G ⊆ EG ∩ ( V ′G × V ′G ) if GP ' G′G . The bijection function f : V ′G → VP is named as a subgraph isomorphism . The subgraph isomorphism counting problem is defined as to find the number of all different subgraph isomorphisms between a pattern graph GP and a graph GG . Examples are shown in Figure 1 . 3.2 GENERAL IDEA . Intuitively , we need to compute O ( Perm ( |VG| , |VP | ) · d|VP | ) to solve the subgraph isomorphism counting problem by enumeration , where Perm ( n , k ) = n ! ( n−k ) ! , |VG| is the number of graph nodes , |VP | is the number of pattern nodes , d is the maximum degree . The first subgraph isomorphism algorithm , Ullmann ’ s algorithm ( Ullmann , 1976 ) , reduces the seaching time toO ( |VP ||VG| · |VG|2 ) . If the pattern and the graph are both small , the time is acceptable because both of two factors are not horrendously large . However , since the computational cost grows exponentially , it is impossible to count as either the graph size or the pattern size increases . If we use neural networks to learn distributed representations for VG and VP or EG and EP , we can reduce the complexity to O ( |VP | · |VG| + |VG|2 ) or O ( |EP | · |EG| + |EG|2 ) via source attention and self-attention . Assuming that we can further learn a much higher level abstraction without loss of representation power for GP , then the computational cost can be further reduced toO ( |VG|2 ) orO ( |EG|2 ) . However , the complexity of the latter framework is still not acceptable when querying over large graphs . If we do not consider self-attention , the computational cost will be O ( |VG| ) or O ( |EG| ) , but missing the self-attention will hurt the performance . In this work , we hope to use attention mechanism and additional memory networks to further reduce the complexity compared with O ( |VG|2 ) or O ( |EG|2 ) while keeping the performance acceptable on the counting problem .

Summary: This paper proposes a method called Dynamic Intermedium Attention Memory Network (DIAMNet) to learn the subgraph isomorphism counting for a given pattern graph P and target graph G. This requires global information unlike usual GNN cases such as node classification, link prediction, community detection. First, input graphs P and G are converted embedding vectors through sequence models (CNN, RNN, Transformer-XL) or graph models (RGCN), and fed into their DIAMNet that uses an external memory as an intermedium to attend both the pattern and the graph. The external memory is updated based on multi-head attention as in Transformer. The output of DIAMNet is passed to FC that outputs 'count' directly. The training is based on minimizing MSE loss as a regression problem. Extensive experimental evaluations report that DIAMNet showed superior performance over competing methods and baselines.