Graph2Seq: Scalable Learning Dynamics for Graphs

1 INTRODUCTION . Graph algorithms appear in a wide variety of fields and applications , from the study of gene interactions ( Özgür et al. , 2008 ) to social networks ( Ugander et al. , 2011 ) to computer systems ( Grandl et al. , 2016 ) . Today , most graph algorithms are designed by human experts . However , in many applications , designing graph algorithms with strong performance guarantees is very challenging . These algorithms often involve difficult combinatorial optimization problems for which finding optimal solutions is computationally intractable or current algorithmic understanding is limited ( e.g. , approximation gaps in CS theory literature ) . In recent years , deep learning has achieved impressive results on many tasks , from object recognition to language translation to learning complex heuristics directly from data ( Silver et al. , 2016 ; Krizhevsky et al. , 2012 ) . It is thus natural to ask whether we can apply deep learning to automatically learn complex graph algorithms . To apply deep learning to such problems , graph-structured data first needs to be embedded in a high dimensional Euclidean space . Graph representation refers to the problem of embedding graphs or their vertices/edges in Euclidean spaces . Recent works have proposed several graph representation techniques , notably , a family of representations called graph convolutional neural networks ( GCNN ) that use architectures inspired by CNNs for images ( Bruna et al. , 2014 ; Monti et al. , 2017 ; Khalil et al. , 2017 ; Niepert et al. , 2016 ; Defferrard et al. , 2016 ; Hamilton et al. , 2017b ; Bronstein et al. , 2017 ; Bruna & Li , 2017 ) . Some GCNN representations capture signals on a fixed graph while others support varying sized graphs . In this paper , we consider how to learn graph algorithms in a way that generalizes to large graphs and graphs of different topologies . We ask : Can a graph neural network trained at a certain scale perform well on orders-of magnitude larger graphs ( e.g. , 1000× the size of training graphs ) from diverse topologies ? We particularly focus on learning algorithms for combinatorial optimization on graphs . Graph neural networks that generalize to large unseen graphs can be useful for many practical applications . For example , in emerging applications of reinforcement learning ( RL ) in computer systems ( e.g. , task device placement ( Mirhoseini et al. , 2017 ) , database query optimization ( Krishnan et al. , 2018 ) , job scheduling ( Mao et al . ) ) , it is often difficult to train RL agents on large graphs , as executing a policy for a large graph ( e.g. , running a large database query ) can be slow . It is therefore desirable to train on small graphs in a way that generalizes to large graphs . Prior works have typically considered applications where the training and test graphs are similar in scale , shape , and attributes , and consequently have not addressed this generalization problem . For example , Khalil et al . ( 2017 ) train models on graph of size 50-100 and test on graphs of up to size 1200 from the same family ; Hamilton et al . ( 2017a ) propose inductive learning methods where they train on graphs with 15k edges , and test on graphs with 35k edges ( less than 3× generalization in size ) ; Ying et al . ( 2018 ) train and test over the same large Pinterest graph ( 3 billion nodes ) . We propose GRAPH2SEQ , a scalable embedding that represents vertices of a graph as a time-series ( §3 ) . Our key insight is that the fixed-sized vector representation produced by prior GCNN designs limits scalability . Instead , GRAPH2SEQ uses the entire time-series of vectors produced by graph convolution layers as the vertex representation . This approach has two benefits : ( 1 ) it can capture subgraphs of increasing diameter around each vertex as the timeseries evolves ; ( 2 ) it allows us to vary the dimension of the vertex representation based on the input graph ; for example , we can use a small number of graph convolutions during training with small graphs and perform more convolutions at test time for larger graphs . We show both theoretically and empirically that this time-series representation significantly improves the scalability and generalization of the model . Our framework is general and can be applied to various existing GCNN architectures . We prove that GRAPH2SEQ is information-theoretically lossless , i.e. , the graph can be fully recovered from the time-series representations of its vertices ( §3.1 ) . Our proof leverages mathematical connections between GRAPH2SEQ and causal inference theory ( Granger , 1980 ; Rahimzamani & Kannan , 2016 ; Quinn et al. , 2015 ) . Further , we show that GRAPH2SEQ and many previous GCNN variants are all examples of a certain computational model over graphs that we call LOCAL-GATHER , providing for a conceptual and algorithmic unification . Using this computational model , we prove that unlike GRAPH2SEQ , fixed-length representations fundamentally can not compute certain functions over graphs . To apply GRAPH2SEQ , we combine graph convolutions with an appropriate RNN that processes the time-series representations ( §4 ) . We use this neural network model , G2S-RNN , to tackle three classical combinatorial optimization problems of varying difficulty using reinforcement learning : minimum vertex cover , maximum cut and maximum independent set ( §5 ) . Our experiments show that GRAPH2SEQ performs as well or better than the best non-learning heuristic on all three problems and exhibits significantly better scalability and generalization than previous state-of-the-art GCNN ( Khalil et al. , 2017 ; Hamilton et al. , 2017a ; Lei et al. , 2017 ) or graph kernel based ( Shervashidze et al. , 2011 ) representations . Highlights of our experimental findings include : 1 . G2S-RNN models trained on graphs of size 15–20 scale to graphs of size 3,200 and beyond . To conduct experiments in a reasonable time , we used graphs of size up to 3,200 in most experiments . However , stress tests show similar scalability even at 25,000 vertices . 2 . G2S-RNN models trained on one graph type ( e.g. , Erdos-Renyi ) generalize to other graph types ( e.g. , random regular and bipartite graphs ) . 3 . G2S-RNN exhibits strong scalability and generalization in each of minimum vertex cover , maximum cut and maximum independent set problems . 4 . Training over a carefully chosen adversarially set of graph examples further boosts G2SRNN ’ s scalability and generalization capabilities . 2 RELATED WORK . Neural networks on graphs . Early works to apply neural-network-based learning to graphs are Gori et al . ( 2005 ) ; Scarselli et al . ( 2009 ) , which consider an information diffusion mechanism . The notion of convolutional networks for graphs as a generalization of classical convolutional networks for images was introduced by Bruna et al . ( 2014 ) . A key contribution of this work is the definition of graph convolution in the spectral domain using graph Fourier transform theory . Subsequent works have developed local spectral convolution techniques that are easier to compute ( Defferrard et al. , 2016 ; Kipf & Welling , 2016 ) . Spectral approaches do not generalize readily to different graphs due to their reliance on the particular Fourier basis on which they were trained . To address this limitation , recent works have considered spatial convolution methods ( Khalil et al. , 2017 ; Monti et al. , 2017 ; Niepert et al. , 2016 ; Such et al. , 2017 ; Duvenaud et al. , 2015 ; Atwood & Towsley , 2016 ) . Li et al . ( 2015 ) ; Johnson ( 2016 ) propose a variant that uses gated recurrent units to perform the state updates , which has some similarity to our representation dynamics ; however , the sequence length is fixed between training and testing . Veličković et al . ( 2017 ) ; Hamilton et al . ( 2017a ) use additional aggregation methods such as vertex attention or pooling mechanisms to summarize neighborhood states . In Appendix A we show that local spectral GCNNs and spatial GCNNs are mathematically equivalent , providing a unifying view of the variety of GCNN representations in the literature . Another line of work ( Jain et al. , 2016 ; Marcheggiani & Titov , 2017 ; Tai et al. , 2015 ) combines graph neural networks with RNN modules . They are not related to our approach , since in these cases the sequence ( e.g. , time-series of object relationship graphs from a video ) is already given as part of the input . In contrast our approach generates a sequence as the desired embedding from a single input graph . Perozzi et al . ( 2014 ) ; Grover & Leskovec ( 2016 ) use random walks to learn vertex representations in an unsupervised or semi-supervised fashion . However they consider prediction or classification tasks over a fixed graph . Combinatorial optimization . Using neural networks for combinatorial optimization problems dates back to the work of Hopfield & Tank ( 1985 ) and has received considerable attention in the deep learning community in recent years . Vinyals et al . ( 2015 ) ; Bello et al . ( 2016 ) ; Kool & Welling ( 2018 ) consider the traveling salesman problem using reinforcement learning . These papers consider two-dimensional coordinates for vertices ( e.g . cities on a map ) , without any explicit graph structure . Graves et al . ( 2016 ) propose a more general approach : a differential neural computer that is able to perform tasks like finding the shortest path in a graph . The work of Khalil et al . ( 2017 ) is closest to ours . It applies a spatial GCNN representation in a reinforcement learning framework to solve combinatorial optimization problems such as minimum vertex cover . 3 GRAPHS AS DYNAMICAL SYSTEMS . 3.1 THE GRAPH2SEQ REPRESENTATION . The key idea behind GRAPH2SEQ is to represent vertices of a graph by the trajectory of an appropriately chosen dynamical system induced by the graph . Such a representation has the advantage of progressively capturing more and more information about a vertex as the trajectory unfolds . Consider a directed graph G ( V , E ) whose vertices we want to represent ( undirected graphs will be represented by having bi-directional edges between pairs of connected vertices ) . We create a discrete-time dynamical system in which vertex v has a state of xv ( t ) ∈ Rd at time t ∈ N , for all v ∈ V , and d is the dimension of the state space . In GRAPH2SEQ , we consider an evolution rule of the form xv ( t+ 1 ) = relu ( W1 ( ∑ u∈Γ ( v ) xu ( t ) ) + b1 ) + nv ( t+ 1 ) , ( 1 ) ∀v ∈ V , t ∈ N , where W1 ∈ Rd×d , b1 ∈ Rd×1 are trainable parameters and relu ( x ) = max ( x , 0 ) . nv ( · ) is a d-dimensional ( 0 , σ2I ) Gaussian noise , and u ∈ Γ ( v ) if there is an edge from u to v in the graphG . For any v ∈ V , starting with an initial value for xv ( 0 ) ( e.g. , random or all zero ) this equation defines a dynamical system , the ( random ) trajectory of which is the GRAPH2SEQ representation of v. More generally , graphs could have features on vertices or edges ( e.g. , weights on vertices ) , which can be included in the evolution rule as additional terms within the ReLU function ( Appendix B.1 ) ; these generalizations are outside the scope of this paper . We use GRAPH2SEQ ( G ) to mean the set of all GRAPH2SEQ vertex representations of G. GRAPH2SEQ is invertible . Our first key result is that GRAPH2SEQ ’ s representation allows recovery of the adjacency matrix of the graph with arbitrarily high probability . Here the randomness is with respect to the noise term in GRAPH2SEQ ; see equation 1 . In Appendix B.2 , we prove : Theorem 1 . For any directed graph G and associated ( random ) representation GRAPH2SEQ ( G ) with sequence length t , there exists an inference procedure ( with time complexity polynomial in t ) that produces an estimate Ĝt such that limt→∞ P [ G 6= Ĝt ] = 0 . Note that there are many ways to represent a graph that are lossless . For example , we can simply output the adjacency matrix row by row . However such representations depend on assigning labels or identifiers to vertices , which would cause downstream deep learning algorithms to memorize the label structure and not generalize to other graphs . GRAPH2SEQ ’ s key property is that it is does not depend on a labeling of the graph . Theorem 1 is particularly significant ( despite the representation being infinite dimensional ) since it shows lossless label-independent vertex representations are possible . To our understanding this is the first result making such a connection . Next , we show that the noise term in GRAPH2SEQ ’ s evolution rule ( equation 1 ) is crucial for Theorem 1 to hold ( proof in Appendix B.3 ) . Proposition 1 . Under any deterministic evolution rule of the form in equation 1 , there exists a graph G which can not be reconstructed exactly from its GRAPH2SEQ representation with arbitrarily high probability . The astute reader might observe that invertible , label-independent representations of graphs can be used to solve the graph isomorphism problem Schrijver ( 2003 ) . However , Proposition 1 shows that GRAPH2SEQ can not solve the graph isomorphism problem , as that would require a deterministic representation . Noise is necessary to break symmetry in the otherwise deterministic dynamical system . Observe that the time-series for a vertex in equation 1 depends only on the time-series of its neighboring nodes , not any explicit vertex identifiers . As a result , two graphs for which all vertices have exactly the same neighbors will have exactly the same representations , even though they may be structurally different . The proof of Proposition 1 illustrates this phenomenon for regular graphs .

Summary: This paper proposes a new representation learning model for graph optimization, Graph2Seq. The novelty of Graph2Seq lies in utilizing intermediate vector representation of vertices in the final representation. Theoretically, the authors show that an infinite sequence of such intermediate representations is much more powerful than existing models, which do not maintain intermediate representations. Experimentally, Graph2Seq results in greedy heuristics that generalize very well from small training graphs (e.g. 15 nodes) to large testing graphs (e.g. 3200 nodes).