Graph Edit Networks

1 INTRODUCTION . Recent advances in graph representation learning have mostly focused on tasks of classification or regression , i.e . tasks with graph-structured input but numeric output ( Battaglia et al. , 2018 ; Kipf & Welling , 2016a ; Veličković et al. , 2018 ) . By contrast , few approaches to date can transform a graph-structured input to a graph-structured output ( Hajiramezanali et al. , 2019 ; Paaßen et al. , 2018 ; Zambon et al. , 2019 ) . This lacuna is crucial because time series prediction on graphs requires graphstructured output , namely the next graph in a time series . Applications of time series prediction on graphs include epidemiological models ( Keeling & Eames , 2005 ) , social ( Liben-Nowell & Kleinberg , 2007 ; Masuda & Holme , 2019 ) , telecommunications ( Nanavati et al. , 2006 ) , traffic ( Cui et al. , 2019 ) , citation ( Shibata et al. , 2012 ) , and financial transaction networks ( Chan & Olmsted , 2017 ) , as well as student solutions in intelligent tutoring systems ( Paaßen et al. , 2018 ) . In each of these settings , predicting the changes in graphs can deepen the understanding of the domain and provide useful knowledge for designing interventions . Currently , methods for time series prediction on graphs are limited to the dynamics of the node attributes ( Yu et al. , 2018 ) , or changes in connectivity ( Goyal et al. , 2020 ; Hajiramezanali et al. , 2019 ) , but do not cover changes in the node set . Fortunately , there exists a rich research tradition of edit distances ( e.g . Levenshtein , 1965 ; Zhang & Shasha , 1989 ; Sanfeliu & Fu , 1983 ) which can describe any change between two graphs . Further , edits are sparse and have a simple semantic ( delete , insert , relabel ) , which makes them easier to interpret for human observers and makes them computationally more efficient ( linear instead of quadratic ) compared to a dense representation . Finally , edits are local , enabling us to make edit decisions at each node instead of coordinating information across the entire graph . In this work , we connect graph neural networks to edit distances by developing a simple , linear output layer that maps node embeddings to graph edits . We call our output layer the graph edit network ( GEN ) . We also develop a general training and inference scheme to transform any graph Gt to its successor Gt+1 using only local binary edit decisions and a regression for node attributes . Theoretically , we prove that a ) a mapping between the nodes of Gt and Gt+1 is sufficient to construct training data for the GEN , b ) this construction yields almost no overhead compared to directly transforming the mapping to graph edits , and c ) provided that the mapping between Gt and Gt+1 is optimal and the GEN can perfectly reproduce the training data , the edit script is almost as short as the graph edit distance ( Sanfeliu & Fu , 1983 ) . In addition to this core theoretical contribution , we provide a proof-of-concept of our model by demonstrating that GENs can learn a variety of dynamical systems on graphs which are more difficult to handle for baseline systems from the literature . We also show that the sparsity of edits enables GENs to scale up to realistic graphs with thousands of nodes . 2 BACKGROUND . Graph Neural Networks : Graph neural networks ( GNNs ) compute representations of nodes in a graph by aggregating information of neighboring nodes ( Bacciu et al. , 2020 ; Defferrard et al. , 2016 ; Kipf & Welling , 2016a ; Micheli , 2009 ; Scarselli et al. , 2009 ) . In particular , the representation φl ( v ) ∈ Rnl of node v in layer l is computed as follows : φl ( v ) = f lmerge ( φl−1 ( v ) , f laggr ( { φl−1 ( u ) |u ∈ N ( v ) } ) ) ( 1 ) where N ( v ) is some neighborhood of v in the graph and f lmerge as well as f laggr are functions that aggregate the information of their arguments , returning a single vector ( Xu et al. , 2019 ) . The representation in the 0th layer is usually defined as the initial node attributes or a constant vector for all nodes ( Kipf & Welling , 2016a ) . Recently , many implementations of f lmerge , f l aggr , and neighborhood N have been suggested , such as a weighted sum via the graph Laplacian ( Kipf & Welling , 2016a ) , recurrent neural networks ( Hamilton et al. , 2017 ) , or attention mechanisms ( Veličković et al. , 2018 ) . Our approach is agnostic to the choice of graph neural network . We merely require some vectorial embedding for each node in the input graph . Graph Generators : Multiple works in recent years have proposed recurrent models to generate graphs ( Bacciu et al. , 2019 ; Li et al. , 2018 ; You et al. , 2018a ; b ; Zhang et al. , 2019 ) . Roughly speaking , these recurrent models first output a node , and then all connections of this node to previous nodes until a special end-of-sentence token is produced . While such a scheme does enable time series prediction , it only works for insertions , i.e . starting at an empty graph and inserting nodes and edges over time . If one wishes to account for general graph changes , one first has to encode a graph into a vector and then decode from this vector the graph in the next time step , similar to molecular design ( Jin et al. , 2019 ; Fu et al. , 2020 ) . However , such models have to generate the next graph from scratch and can not exploit the sparsity and interpretability of edits , as we suggest . Link Prediction : Link prediction is complementary to graph generation . It assumes a constant number of nodes , but changing connectivity between them ( Liben-Nowell & Kleinberg , 2007 ; Richard et al. , 2014 ; Shibata et al. , 2012 ) . Typical link prediction approaches compute node features first , followed by an affinity index between nodes based on their features . Finally , edges with low index are predicted to vanish , while edges with high index are predicted to appear . For example , Goyal et al . ( 2020 ) combine dense and recurrent blocks to build an autoencoder for link prediction , while Hajiramezanali et al . ( 2019 ) combine a GNN and a RNN to obtain a spatio-temporal variational graph autoencoder . In GENs , we predict edge changes with a similar scheme , using a graph neural network to obtain the node features and then mapping these node features to the graph changes . However , in contrast to prior work , we do not predict the next adjacency matrix but only the change in adjacencies , which is a much sparser signal , reducing the time complexity from quadratic to linear . Additionally , GENs can not only handle edge changes , but also node changes . We note that our current paper is limited to a Markovian setting , i.e . we do not consider the past for computing node representations . This limitation could be addressed by combining our output layer with EvolveGCN ( Pareja et al. , 2020 ) which uses a recurrent net to predict the weights of a graph neural net , thus being able to handle changes in the node set . Dynamic Attributes : Recently , graph neural networks have been extended to predict changes to node attributes , while the nodes and edges remain fixed ( Cui et al. , 2019 ; Seo et al. , 2018 ) , which is particularly useful for traffic networks . GENs are complementary to these works , in that we consider the more general case of graph topology changes . Time Series Prediction on Graphs : To our knowledge , only very few works to date have addressed the most general case of time series of graphs , where both nodes and edges are permitted to change . In particular , Paaßen et al . ( 2018 ) suggest several kernel-based time series prediction methods for graphs . However , their scheme is limited to predictions in the kernel space and mapping a prediction back to a graph requires solving an inverse kernel problem , relying on approximations that impact accuracy ( Paaßen et al. , 2018 ) . Zambon et al . ( 2019 ) embed the time series into a vector space using a GNN and use a recurrent neural network to predict the next time step . To obtain the corresponding graph , a multi-layer perceptron is used to compute the adjacency matrix and node features from the predicted embedding . Besides being computationally expensive , this dense decoder also assumes a fixed order of the nodes . Graph Edits : The basis for our approach are graph edits , which are functions that describe changes in graphs ( Sanfeliu & Fu , 1983 ) . Formally , we first define an attributed , directed graph as a triple G = ( V , E , X ) , where V = { 1 , . . . , N } is a finite set of node indices , E ⊆ V × V is a set of edges , and X ∈ RN×n is a matrix of node attributes for some n ∈ N. We define the nodes as indices for notational simplicity , but we do not assume any specific order , i.e . we treat isomorphic graphs as the same . Now , let G be the set of all possible attributed directed graphs . We define a graph edit as some function δ : G → G. In particular , we consider the graph edits of Sanfeliu & Fu ( 1983 ) , namely node deletions deli , which delete the ith node from a graph , node replacements repi , x , which set the attribute of node i to x , node insertions insx , which add a new node with attribute x to a graph , edge deletions edeli , j , which delete the edge ( i , j ) from a graph , and edge insertions einsi , j , which insert the edge ( i , j ) into a graph . We then define an edit script δ̄ as a finite sequence δ̄ = δ1 , . . . , δT of graph edits and we define the application of δ̄ as the composition of all edits , i.e . δ̄ ( G ) : = δT ◦ . . . ◦ δ1 ( G ) . Finally , we define the graph edit distance dGED ( G , G′ ) between two graphs G and G′ as the length of the shortest script δ̄ such that δ̄ ( G ) ∼= G′ , where∼= means isomorphic . The GED is well-defined and a proper metric , i.e . a script connecting any two graphs always exists , the GED between two isomorphic graphs is zero , the GED is symmetric , and it conforms to the triangular inequality ( Abu-Aisheh et al. , 2015 ; Sanfeliu & Fu , 1983 ) . While prior work has already attempted to approximate the graph edit distance with graph neural nets ( Bai et al. , 2019 ; Li et al. , 2019 ) our work is , to our knowledge , the first to produce actual graph edits as network output , and to avoid graph pooling layers .

Summary: Graph Editing is a prominent research area which overlaps a variety of fields in computer science. As a typical example, a series of graphs - each obtained from its predecessor - can naturally represent the evolution of a system over time. From this viewpoint, it would be natural to obtain a means of predicting how such a series of graphs will evolve over time. The authors introduce a simple output layer (called "GEN") that can be used in graph neural networks to do precisely that.