Graph Edit Networks

1 INTRODUCTION . Recent advances in graph representation learning have mostly focused on tasks of classification or regression , i.e . tasks with graph-structured input but numeric output ( Battaglia et al. , 2018 ; Kipf & Welling , 2016a ; Veličković et al. , 2018 ) . By contrast , few approaches to date can transform a graph-structured input to a graph-structured output ( Hajiramezanali et al. , 2019 ; Paaßen et al. , 2018 ; Zambon et al. , 2019 ) . This lacuna is crucial because time series prediction on graphs requires graphstructured output , namely the next graph in a time series . Applications of time series prediction on graphs include epidemiological models ( Keeling & Eames , 2005 ) , social ( Liben-Nowell & Kleinberg , 2007 ; Masuda & Holme , 2019 ) , telecommunications ( Nanavati et al. , 2006 ) , traffic ( Cui et al. , 2019 ) , citation ( Shibata et al. , 2012 ) , and financial transaction networks ( Chan & Olmsted , 2017 ) , as well as student solutions in intelligent tutoring systems ( Paaßen et al. , 2018 ) . In each of these settings , predicting the changes in graphs can deepen the understanding of the domain and provide useful knowledge for designing interventions . Currently , methods for time series prediction on graphs are limited to the dynamics of the node attributes ( Yu et al. , 2018 ) , or changes in connectivity ( Goyal et al. , 2020 ; Hajiramezanali et al. , 2019 ) , but do not cover changes in the node set . Fortunately , there exists a rich research tradition of edit distances ( e.g . Levenshtein , 1965 ; Zhang & Shasha , 1989 ; Sanfeliu & Fu , 1983 ) which can describe any change between two graphs . Further , edits are sparse and have a simple semantic ( delete , insert , relabel ) , which makes them easier to interpret for human observers and makes them computationally more efficient ( linear instead of quadratic ) compared to a dense representation . Finally , edits are local , enabling us to make edit decisions at each node instead of coordinating information across the entire graph . In this work , we connect graph neural networks to edit distances by developing a simple , linear output layer that maps node embeddings to graph edits . We call our output layer the graph edit network ( GEN ) . We also develop a general training and inference scheme to transform any graph Gt to its successor Gt+1 using only local binary edit decisions and a regression for node attributes . Theoretically , we prove that a ) a mapping between the nodes of Gt and Gt+1 is sufficient to construct training data for the GEN , b ) this construction yields almost no overhead compared to directly transforming the mapping to graph edits , and c ) provided that the mapping between Gt and Gt+1 is optimal and the GEN can perfectly reproduce the training data , the edit script is almost as short as the graph edit distance ( Sanfeliu & Fu , 1983 ) . In addition to this core theoretical contribution , we provide a proof-of-concept of our model by demonstrating that GENs can learn a variety of dynamical systems on graphs which are more difficult to handle for baseline systems from the literature . We also show that the sparsity of edits enables GENs to scale up to realistic graphs with thousands of nodes . 2 BACKGROUND . Graph Neural Networks : Graph neural networks ( GNNs ) compute representations of nodes in a graph by aggregating information of neighboring nodes ( Bacciu et al. , 2020 ; Defferrard et al. , 2016 ; Kipf & Welling , 2016a ; Micheli , 2009 ; Scarselli et al. , 2009 ) . In particular , the representation φl ( v ) ∈ Rnl of node v in layer l is computed as follows : φl ( v ) = f lmerge ( φl−1 ( v ) , f laggr ( { φl−1 ( u ) |u ∈ N ( v ) } ) ) ( 1 ) where N ( v ) is some neighborhood of v in the graph and f lmerge as well as f laggr are functions that aggregate the information of their arguments , returning a single vector ( Xu et al. , 2019 ) . The representation in the 0th layer is usually defined as the initial node attributes or a constant vector for all nodes ( Kipf & Welling , 2016a ) . Recently , many implementations of f lmerge , f l aggr , and neighborhood N have been suggested , such as a weighted sum via the graph Laplacian ( Kipf & Welling , 2016a ) , recurrent neural networks ( Hamilton et al. , 2017 ) , or attention mechanisms ( Veličković et al. , 2018 ) . Our approach is agnostic to the choice of graph neural network . We merely require some vectorial embedding for each node in the input graph . Graph Generators : Multiple works in recent years have proposed recurrent models to generate graphs ( Bacciu et al. , 2019 ; Li et al. , 2018 ; You et al. , 2018a ; b ; Zhang et al. , 2019 ) . Roughly speaking , these recurrent models first output a node , and then all connections of this node to previous nodes until a special end-of-sentence token is produced . While such a scheme does enable time series prediction , it only works for insertions , i.e . starting at an empty graph and inserting nodes and edges over time . If one wishes to account for general graph changes , one first has to encode a graph into a vector and then decode from this vector the graph in the next time step , similar to molecular design ( Jin et al. , 2019 ; Fu et al. , 2020 ) . However , such models have to generate the next graph from scratch and can not exploit the sparsity and interpretability of edits , as we suggest . Link Prediction : Link prediction is complementary to graph generation . It assumes a constant number of nodes , but changing connectivity between them ( Liben-Nowell & Kleinberg , 2007 ; Richard et al. , 2014 ; Shibata et al. , 2012 ) . Typical link prediction approaches compute node features first , followed by an affinity index between nodes based on their features . Finally , edges with low index are predicted to vanish , while edges with high index are predicted to appear . For example , Goyal et al . ( 2020 ) combine dense and recurrent blocks to build an autoencoder for link prediction , while Hajiramezanali et al . ( 2019 ) combine a GNN and a RNN to obtain a spatio-temporal variational graph autoencoder . In GENs , we predict edge changes with a similar scheme , using a graph neural network to obtain the node features and then mapping these node features to the graph changes . However , in contrast to prior work , we do not predict the next adjacency matrix but only the change in adjacencies , which is a much sparser signal , reducing the time complexity from quadratic to linear . Additionally , GENs can not only handle edge changes , but also node changes . We note that our current paper is limited to a Markovian setting , i.e . we do not consider the past for computing node representations . This limitation could be addressed by combining our output layer with EvolveGCN ( Pareja et al. , 2020 ) which uses a recurrent net to predict the weights of a graph neural net , thus being able to handle changes in the node set . Dynamic Attributes : Recently , graph neural networks have been extended to predict changes to node attributes , while the nodes and edges remain fixed ( Cui et al. , 2019 ; Seo et al. , 2018 ) , which is particularly useful for traffic networks . GENs are complementary to these works , in that we consider the more general case of graph topology changes . Time Series Prediction on Graphs : To our knowledge , only very few works to date have addressed the most general case of time series of graphs , where both nodes and edges are permitted to change . In particular , Paaßen et al . ( 2018 ) suggest several kernel-based time series prediction methods for graphs . However , their scheme is limited to predictions in the kernel space and mapping a prediction back to a graph requires solving an inverse kernel problem , relying on approximations that impact accuracy ( Paaßen et al. , 2018 ) . Zambon et al . ( 2019 ) embed the time series into a vector space using a GNN and use a recurrent neural network to predict the next time step . To obtain the corresponding graph , a multi-layer perceptron is used to compute the adjacency matrix and node features from the predicted embedding . Besides being computationally expensive , this dense decoder also assumes a fixed order of the nodes . Graph Edits : The basis for our approach are graph edits , which are functions that describe changes in graphs ( Sanfeliu & Fu , 1983 ) . Formally , we first define an attributed , directed graph as a triple G = ( V , E , X ) , where V = { 1 , . . . , N } is a finite set of node indices , E ⊆ V × V is a set of edges , and X ∈ RN×n is a matrix of node attributes for some n ∈ N. We define the nodes as indices for notational simplicity , but we do not assume any specific order , i.e . we treat isomorphic graphs as the same . Now , let G be the set of all possible attributed directed graphs . We define a graph edit as some function δ : G → G. In particular , we consider the graph edits of Sanfeliu & Fu ( 1983 ) , namely node deletions deli , which delete the ith node from a graph , node replacements repi , x , which set the attribute of node i to x , node insertions insx , which add a new node with attribute x to a graph , edge deletions edeli , j , which delete the edge ( i , j ) from a graph , and edge insertions einsi , j , which insert the edge ( i , j ) into a graph . We then define an edit script δ̄ as a finite sequence δ̄ = δ1 , . . . , δT of graph edits and we define the application of δ̄ as the composition of all edits , i.e . δ̄ ( G ) : = δT ◦ . . . ◦ δ1 ( G ) . Finally , we define the graph edit distance dGED ( G , G′ ) between two graphs G and G′ as the length of the shortest script δ̄ such that δ̄ ( G ) ∼= G′ , where∼= means isomorphic . The GED is well-defined and a proper metric , i.e . a script connecting any two graphs always exists , the GED between two isomorphic graphs is zero , the GED is symmetric , and it conforms to the triangular inequality ( Abu-Aisheh et al. , 2015 ; Sanfeliu & Fu , 1983 ) . While prior work has already attempted to approximate the graph edit distance with graph neural nets ( Bai et al. , 2019 ; Li et al. , 2019 ) our work is , to our knowledge , the first to produce actual graph edits as network output , and to avoid graph pooling layers .

Summary: This paper proposes a model that, given one graph, predicts a sequence of edits that transforms this graph to the next one in a sequence of evolving graphs. To this end, it proposes the graph edit network (GEN), which is a linear output layer that transforms node embeddings to a set of scores that are then used to deterministically select graph edits. The space of edits predictable by this model covers the full space of graph changes. The paper moreover shows that, given a graph matching and a pair of graphs, we can algorithmically find a near-optimal graph edit sequence for generating training data. Finally, the paper demonstrates the model's capabilities on a set of synthetic benchmarks.