Deep Graph Translation
Authors: Xiaojie Guo
Lingfei Wu
Liang Zhao
Venue:  (2020)
Keywords: Graph translation
graph generation
deep neural network

Abstract: Deep graph generation models have achieved great successes recently, among which, however, are typically unconditioned generative models that have no control over the target graphs are given an input graph. In this paper, we propose a novel Graph-Translation-Generative-Adversarial-Networks (GT-GAN) that transforms the input graphs into their target output graphs. GT-GAN consists of a graph translator equipped with innovative graph convolution and deconvolution layers to learn the translation mapping considering both global and local features, and a new conditional graph discriminator to classify target graphs by conditioning on input graphs. Extensive experiments on multiple synthetic and real-world datasets demonstrate that our proposed GT-GAN significantly outperforms other baseline methods in terms of both effectiveness and scalability. For instance, GT-GAN achieves at least 10X and 15X faster runtimes than GraphRNN and RandomVAE, respectively, when the size of the graph is around 50.

Under review as a conference paper at ICLR 2020
DEEP GRAPH TRANSLATION
Anonymous authors
Paper under double-blind review
ABSTRACT
Deep graph generation models have achieved great successes recently, among which however, typically unconditioned generative models that have control over the target graphs given an input graph. In this paper, we propose novel Graph- Geeneraiie-AAvvessarii
(GT-GAN) that transforms the input graphs into their target output graphs. GT-GAN consists of a graph translator equipped with innovative graph convolution and deconvolution layers learn the translation mapping considering both global and local features. A new conditional graph diseriminator is proposed to classify the target graphs by conditioning on input graphs while training. Extensive experiments on multiple synthetic and real- world datasets demonstrate that our proposed GT-GAN significantly outperforms other baseline methods in terms of both effectiveness and scalability, For instance, GT-GAN performs at least 10 and 15 time faster than GraphRNN and Random VAE, respectively, when the size of the graph is around 50.
INTRODUCTION
In recent years, deep learning on graphs has seen a surge of interests, especially for graph representa- tion and recognition tasks such as node-level classification (Li et al., 2016 Kipf & Welling 2017]]
Veličković et al., 2017, Gilmer et al. 2017 Hamilton et al. 2017) and graph-level classification
(Niepert et al., 2016 Atwood et al. 2016 Wu et al.) (017). Because of the successes in graph neural networks, researchers have recently started to explore the use of deep generative models for graph synthesis on practical applications such as designing new chemical molecular structures (Simonovsky & Komodakis] 2018 You et al. 2018). This has led to many of the recent advances in deep graph generative models where some of these approaches are domain dependent models (Kusner et al., 2017 Dai et al. 2018) for generating graphs with physical constrains, while some others consider the generation of generic graphs (Li ct al. 2018; Samanta et al., 2018: Jin et al.) 2018a).
However, there are two main drawbacks of existing deep graph generative models. First, one significant limitation of the previous approaches is that most of these models are only suitable for small graphs with 40 or fewer nodes, which is mainly due to their one-node-per-eeeeeeo sere-per-step generation manner. More importantly, most of the existing graph generation models are unconditioned and thus ignore rich input graph information for generating a new graph. In many applications, it is crucial to guide the graph generation process by conditioning on an input graph, which can be cast as graph translation learning problem translating the input graph to the output graph.
One straightforward way is to build a translation system by using graph encoder-decoder architecture. However, there are several challenges for this type of approaches: 1) how to learn one-to-more mapping between the input graph and the target graphs. Different from the plain graph generation problem, conditional graph synthesis task learn distribution of target graphs conditioning on the input graph, which aims to capture the underlying implicit properties of the graphs, such as their scale-free characteristic. 2) how to jointly learn both local and global information for translation. One needs to not only learn the translation mapping in the local information (i.e. neighborhood pattern of each node), but also in the global property of the whole graph (e.g..node degree distribution or graph density).
To address the aforemenntooed challenges, we present a novel neural network architecture - Graph- wwwaaaa-eeeeeeeeeveeeeeeeee
(GT-GAN). We first propose conditional graph GAN architecture that consists of an encoder-decoder translator and a conditional graph discriminator to learn the one-to-more mapping (a conditional distribution) for graph translation. To jointly embed

---

Under review as a conference paper at ICLR 2020
the local and global information, present novel graph encoder including both the edge and the node convolution layers. In addition, we further propose a novel graph U-net with graph skips and dedicated graph deconvolution layers including both the edge and the node deconvolution layers. Finally, GT-GAAN is scalable with at most quadratic computation and memory consumption in terms of the number of nodes in graph, making it suitable for at least modest-scale graphs (with hundreds of nodes, compared to the tens of nodes in most of existing graph generative models).
We highlight our main contributions as follows:
We develop generic framework GT-GAN consisting of a novel graph translator and conditional graph discriminator for learning a conditional distribution of target graphs given the input graphs.
We propose a novel graph encoder consisting of "edge convolution" layers that extract various relations among nodes containing both local and global information, and "node convolution" layers that embed the node representations based on the extracted relations.
We propose novel graph decoder consisting of the "edge deconvolution" and "node deconvolution" layers, which can decode the node representations first into the latent relations of the target graph and then generate the final target graph. The graph skip-connection is also utilized map the learned latent relations between the input and target graphs.
Extensive experiments have been conducted on both synthetic and real-world datasets on eight performance metrics to demonstrate the effectiveness and efficiency of the proposed model.
RELATED WORKS
Graph Neural Networks. The recent surge of research into GNN (Graph Neural Networks) can be generally divided into two categories: Graph Recurrent Networks and Graph Convolutional Networks. Graph Recurrent Networks originate from early work by Corr et al. (2005): Scarselli et al. (2009) and based on recursive neural networks that have been extended by modern deep learning techniques such as gated recurrent units (Li et al. 2016). The other category, Graph Convolutional Networks, originate from spectral graph convolutional neural networks (Bruna et al.] 2014), which were then extended by Defferrard et al. (2016) using fast localized convolutions, and further approximated by an efficient architecture for attention mechanism and subgraph-level information are also explored later to further improve the representation power of learned node embeddings (Veličković et al.) 2017 2018 Bai al. 2019).
supervised setting proposed by Kippf & Welling (2017). Self-
Graph generation. Most of the existing GNN based graph generation for general graphs have been proposed in the last two years and are based on VAE (Simonovsky & Komodakis) 2018 Samanta et al. 2018) and generative adversarial nets (GANs) (Bojchevski et al., 2018)) among others (Li et al. 2018 You et al. 2018). Most of these approaches generate nodes and edges sequentially to form a whole graph, leading to the issues of being sensitive to the generation order and very time-consuming for large graphs. Differently, GraphRNN (You et al. 2018) builds an autoregressive generative model on these sequences with LSTM model and has demonstrated its good scalability.
Data Translation involved Graphs. A variety of graph-to-sequence models have been proposed to cope with different tasks including machine translation (Beck et al.) 2018 Bastings al., (017). semantic parsing (Xu et al. 201 Sa b [Sng et al. 2018), and question generation (Chen et al. 2019
and health status prediction (Gao et al. 2019). The sequence-to-gres
algorithms are generally popular with those working on NLP methods, including generating dependency graphs (Gildea et al.,
2018, Wang al.] 2018) and AMR structures (Peng et al. 2018). A few of very recent attempts have also been made to develop graph-to-graph translation models. Jin et al.) (0188) proposed a domain-specifie graph translation model to deal with molecular optimization task by utilizing the domain knowledge- junction tree and molecule graph. [D et al. (2019) dealt with the chemical reaction product prediction problem by predicting the reaction sequences based on the input graph of molecules. Sun & Li (2019) proposed RNN based model for encoding and decoding the directed acyclic graph (converted from regular graphs), which can be viewed as a contemporary work to our work. However, this method is trained following the encoder-decoder architecture but in supervised setting instead of learning a distribution of graphs. More importantly, it is difficult to scale to even modest-scale graph due to its me--ode-per--tte generation manner.
2

---

Under review as a conference paper at ICLR 2020
Fake Terges Graph Y
Graph Translator
Fake? Real?
Graph Real Targes Graph
Conditional Graph Discriminator
Figure I: GT-GANs consisting graph translator and a conditional graph discriminator. novel graph encoder and decoder are designed for the graph translation problem.
THE OVERALL ARCHITECTURE OF GT-GAN
In this section, we first present our problem formulation of graph translation problem. We then propose our new GT-GAN model for graph translation and discuss each component in detail in the subsequent sections.
3.1 PROBLEM FORMULATION FOR DEEP GRAPH TRANSLATION
Our goal is to learn end-to-end translation mapping from an input graph to target graph. Let an input graph Gx = (V, &, A, 5) such that the set of N nodes, C V x V is the set of edges, and A E NNNN is adjacency matrix (binary weighted), where Gx be weighted or unweighted, directed or undirected. Let S E NxF be a node feature matrix with each row representing a node feature vector S,. Denote Gi E & therefore denotes the corresponding weight of the edge ei Similarly, we define a target graph Gy = "V", A', S') that shares the same node sets and node features with Gx but with different
edge from the node U2 E V to Uj V: Ai, A topology and connection weights. Formally, graph translation is to learn a translator from an input graph Gx E 9x with a random noise to generate a target graph Gy gr, where 9x and Gy denote the domains of the input and target graphs, respectively. The translation mapping is denoted as T U, Gx Gy.
Note that since our aim is to learn conditional distribution of the target graphs given an input graph, we can cast the graph translation as a conditional graph generation problem, where an input graph can be mapped into any target graph that may have different topologies yet follow the same distribution. In contrast, the graph generation, that are designed to learn distribution of grgpph and generate a new graph sample based on this distribution, typically uses variational autoencoder framework for graph generation. Therefore, the previous graph generation frameworks such as graph VAE (Simonovsky & Komodakis 2018) and GraphRNN You al. 2018) do not directly fit into "translation" setting.
The Propused GT-GAN Framework. Fig shows our proposed generic GAN framework for graph translation that consists of a graph translator 'T and conditional graph discriminator D. In this figure we assume the node feature has only one dimension for simplicity. Since our task is to train conditional generator with "one-to-many mapping" instead of deterministic one, the noise U is introduced by the dropout function (Setzer et al., (013) in each convolution and deconvolution layer, as shown (in green lines) Fig Our graph translator is trained to produce target graphs that cannot be distinguished from "real" ones by our conditional graph discriminator D. Specifically, the generated target graph Gy = TGxx, (Gx,U), cannot be distinguished from the real one, Gy, based on the current input graph Gx. T and 'D undergo an adversarial training process based on input and target graphs by solving the following the loss function:
L(T, D) = -Ex (log D(Gr [xx)] + Ea a [log(1 D(T U)Gx)))
(1)
where T tries to minimize this objective while an adversarial D tries to maximize it, i.e. T arg miny maxp L(T, D). We also mix the GAN loss with the L1 loss to enforce sparsity similarity, which is also found useful in image translation problem (Isola et al. 2017).
Cn(T) EA, IIIA' U))a),
(2)
3

---

Under review as a conference paper at ICLR 2020
where T(xx U) refers to the adjacent matrix of generated graph. The training process is trade-off between L11 and L(T, D), which jointly enforces T(Gx, U) and Gy to follow a similar, but not necessarily identical topologiall pattern. Specifically, Lu makes T(Gx x U) share the same rough outline of sparsity pattern as Gy, while L(T, D) allows T(Gx. x. U) to vary to some degree. Thus, the optimal objective T of the translator, which generates graphs that are "real" possible, is defined as:
T arg min max ((T,D) (D))),
D
The graph translator' T is an encoder-decoder architecture, where we propose a new graph encoder to obtain the node representations of the input graph and propose the graph deconvolution with skips to generate the target graph, as shown in Fig which we elaborated in the followings sections.
3.2 GRAPH ENCODER
The graph encoder aims to learn the representations of nodes based on the node features and graph topol- ogy of the input graph. One of crucial challenges is learn both local and global information in the graph embedding. For instance, when learning translation between two scale-free graphs, one needs to translate both the local information (i.e. n-hop neighborhood of each node) and the ccale-free property (i.e. node degree distributions of whole graph) from an input graph to target graph.
Extracted Intent relation R
Extracted Inlent relation 1
Network
latent relations used for node embeddings
The Proposed Graph Convolution. To learn the
local information, the proposed encoder learns each node representation based on its n-hop neighbors. To learn the global information, it learns each node representation by looking for more "virtual neighbors" regarding the latent relations from the aspect of the whole graph. As shown in Fig 2, though Node 1 and 2 are separated far away in the original network, they have the similarities in some properties, such neighborhood structure and node degree. These nodes treated virtually connected ("virtual neighbors"). Thus, we first propose the "edge convolution" layers to learn a group of multi- mode relations from the topology of the input graph, which can include both the n-hop connections and the latent relations that are derived from their adjacent edges/relations. And then the "node convolution" layer is used to embed each node representations by aggregating its "virtual neighbors" that related to each latent relations. F illustrates the details of these matrix operations involving graph convolution.
Figure 2: Latent relations graph convolution
In each "edge convolution" layer, each node pair's latent relation is computed by its adjacent edges or the extracted adjacent relations from the last layer. In the directed graph, each node has in- coming edge(s) and out-going edge(s). Thus, there are two learnable parametric vectors and z as convolution filters for two directions to convolute the adjacent edges/relations for each node pairs.
The relation E..i in the mith relation mode of the ith layer is learned by the out-going edges/relations of node i; and the in-coming edges/relations of node Uj.
E.m Σ
Ri 2n= (a
N E
0K + a E N
El-1 (dm))
Ri E
where E.y Ar, and @. Nx1 refers to the filter vector to learned and di.T refers to the element of .0 that is related to node uk;- Ri-1 refers to the number of relation modes extracted for the (1 - 1)th layer of the graph encoder.
After learning the various modes of relations, the "node convolution" layer learns each node's representations by aggregating its "virtual neighbors" in terms each mode of relation. The math feature vector of node representation tensor H." 1xx for node U, is computed as:
Ri-1 (a
d--.n Bik
ukg St, + o E ky=1
E vky (k,)),
(5)
n=1
k
4

---

Under review as a conference paper at ICLR 2020
where H, E R RRxP and R2 refers to the number of feature vectors in the "node convolution" layer. Here ym, vm E N refer to the filter vectors for the two directions to be learned and Meg refers to the element of nM that is related to node Uk,, H; is then flattened and transformed into node representation vector H; RIxC by fully connected layer. C' is the length of the node representation. Note that our graph encoder is designed for directed graph, and it is easily generalized to an undirected graph, where the weight vector shared by both directions.
fllers
Receptive Golds
Multiply
Add
M Lim
MY
X
8 8142ma2
Eij
gl+Lm
Vinus
Vi
8 4ummm
Vimm
By 1/42m.n
(8) edge-to-node convolution
(b) node to edge deconvolution
(c) odge-to-edge convolution
(d) edge-to-edge
Figure 3: Matrix operations for graph convolution and graph deconvolution. convolution operations, we need to utilize row filter to convolute "incoming" edges and column filter for "outgoing" edges. However, in deconvolution operations, we have to utilize the transposed filters, namely column filter to decode for "incoming" edges and row filter to decode for "outgoing" edges."
3.3 GRAPH DECODER
The decoder aims to generate the edges of the target graph by taking the extracted latent information of the input graph. It is straightforward to directly use the embedded node representation of the last layer to generate the target graph. However, the extracted information from each layer in the encoder could also be useful for generating the target graph. Thus, consider all possible information learned in the encoder to be fed into a graph decoder.
Motivated by these observations, propose graph U-Net consisting of graph skips and dedicated graph deconvolution layers. The graph deconvolution decodes the single node (or edge) information to yield its incoming and outgoing adjacent edges mirrored graph convolution process. In addition, several skips are implemented to map the learned information of each layer in the encoder mirror the corresponding layers in the decoder. Similar Graph U-Net was proposed in (Gao & Jil 2019). The key difference is that their U-Net is barely a graph embedding method by using the old graph topology from pooling part to embed nodes during unpooling part. However, our Graph U-Net can not only do node embedding in graph encoder but also generate the new graph's 's topology in the graph decoder, which is necessary for the graph translation problem.
The proposed Graph Deconvolution. The proposed graph deconvolution technique incorporates both "node deconvolution" and "edge deconvolution" layers. First, the "node deconvolution" layer used to generate the latent multi-mode relations of the target graph based the learned latent reversed process of the "node"
node representations. As shown in Fig. Cc... "node deconvolution" convolution. Since each node has influence to its relations connecting to other nodes. Then the relation E between node 2 and node vy in the mth relation mode of the Ith "node" deconvolution layer in the decoder can be computed as follows:
El.m Σ (o(H)1))+
where a (H " means the deconvolution contribution of node to its relation with node fy made by the nthh element of its node representations, and z2 represents the element of the deconvolution filter vector nM E RlxN N that is related to node u..
We can now recursively apply our proposed "edge deconvolution" layer to decode the latent relation between each pair of nodes from the upper layer to those of lower layer. As a reversed way of "edge" convolution, the relation of each pair of nodes in the (( 1)th layer can make contribution generating itself and its adjacent relations in the ith layer, shown in Fig. d). Thus, the relation fm. between node and node vy in the lth layer is computed as follows:
pl. Σ
(o(
Ry
a (
J.m Σn N
n=
k,=1
5

---

Under review as a conference paper at ICLR 2020
El-1." is interpreted as the decoded contribution of node i to its relations with where &l.mm -k:=1 L,
node Uj, and @l-" refers to the element of deconvolution filter vector that is related to node vy. R refers to the number of relation modes extracted by the (1 1)th layer the graph decoder. The output of the last "edge" deconvolution layer denotes the edges of the target graph.
Skips for graph deconvolution. Based on the graph deconvolution above, it is possible to utilize skips to link the extracted latent relation sets of each layers in the graph encoder with those in the graph decoder. Specifically,, the output of the /th "edge deconvolution" layer with RG channels in the decoder is concatenated with the output of the /th "edge convolution" layer with RG channels in encoder to form joint R₂ + R channels, which are then input into the (7 + 1)th deconvolution layer.
3.4 CONDITIONAL GRAPH DISCRIMINATOR
The graph discriminator must distinguish between the "translated" target graph and the "real" ones based on the input graphs, as this helps to train the generator in an adversarial way. Technically, this requires the discriminator to accept two graphs simultancously as inputs (a target graph and an input graph or a generated graph and an input graph), and classify the two graphs as either related or not. Thus, we propose conditional graph discriminator (CGD) which leverages the same graph convolution layers the translator for the graph classification, as shown in Fig T Specifically, the input and target graphs are both ingested by CGD and stacked into a N x N x 2 tensor which can be considered a 2-channel input. After obtaining the node representations, the graph-level embedding is computed by summing these node embeddings. Finally, a softmax layer is implemented to distinguish the input graph-pair from the real graph or generated graph.
3.5 COMPUTATIONAL COMPLEXITY ANALYSIS
The graph encoder and decoder shares the same time complexity. Without loss of generality, we assume all the hidden layers have the same number of feature maps as M. P is the length of the fully connected layer in CGD. The worst-case total complexity of GT-GAN (i.e., the dense graph) is now ( M² + 3 2² 2 PP. where the first, second, and third terms represent "edge convolutions", "node convolutions", and fully connected layers in the graph discriminator, respectively. Similarly, the total memory consumption for GT-GAN is O(99N M + M)) - (3N + 3N + MP + P)). In practice, many graphs are likely be sparse, thus it further reduces the computational and memory cost to O(N) by using sparse matrix
operations (You) et al. 2018)), which paves the way toward modest scale graphs with hundreds or thousands of nodes, compared to most of existing graph generation methods, which often have O(N)) or even O(N)).
4 EXPERIMENT
This section reports the results of extensive experiments and ablation studies carried out to test the per- formance of GT-GAN on two synthetic and two real-world datasets. All experiments were conducted on a 64- machine with Nvidia GPU (GTX 1683 MHz, 8 GB GDDR5). The code and data uti- Graph irraasaaaiion
lized available at htthtts:: /tttt:///gttbb..t/// wtt. http://nnnnnccss125..2
4.1 DATASETS
The experimental settings for each dataset were as follows. The rules for generating synthetic input-target graph pairs and the process of collecting the real-world graphs is provided in Appendix.
Two synthetic datasets: Two groups of synthetic datasets were used to validate the performance of the proposed GT-GAN: scale-free graph dataset and Doisson-random graph dataset. Each group has five subsets with different graph sizes (number of nodes): 10, 20, 50, 100 and 150. Each subset consists of 5000 input-target graph pairs; 2500 pairs were used for training and the remaining 2500 for testing.
User authentication datasets. The goal of this application was to forecast future potential malicious authentication graphs given the user's normal authentication graph. Each user authentication graph is a directed weighted graph, where nodes represent computers and the weights of the edges represent the authentication activities at certain frequencies. There are 78 pairs of graphs (malicious and normal behavior) of graph size 50 and 315 pairs of graphs of graph size 300 from 97 users in two subsets. We performed a 2-fold coss-validations and 3-fold cross-validation, respectively, for the two subsets.

---

Under review as a conference paper at ICLR 2020
Table Evaluation results for the scale-free graphs
Size
Methods
15
HID
BD
WD
En-dist
C-dist
w1-sim
It-im
Random- VAE
0,42
0.98
Inf
7.58
0.3787
0,4528
0.3333
0.2494
GraphRNN
0.47
0.98
Inf
1.64
0.7226
0,5319
0.2470
0.0055
Graph VAE
0.67
1.00
Inf
2.85
0.6849
0.6664
0.3723
0.1576
10
GraphGMG
0.43
0.98
Inf
1.69
0.6849
0.4763
0.3701
0.0120
S-Generator
0.35
0.98
3.45
0.80
0.2097
0.2465
0.4185
0.5431
GT-GAN
0.35
0.98
3.44
0.77
0.2034
0.2379
0.4195
0.5469
Random VAE
0.51
0.97
Ini
1.74
0.4513
0.5400
0.3333
0.3813
GraphRNN
0.50
0.98
Inf
1.44
0.7222
0.6087
0.2652
0.2373
20
S-Generator
0.36
0.96
2.84
0.67
0.1367
0.1900
0,4665
0.7017
GT-GAN
0.35
0.96
2.74
0.66
0.1367
0.1894
0.4681
0.7018
GraphRNN
0,48
0.88
Ini
0.90
0.7147
0.6519
0.2713
0.2138
100
S-Generator
0.14
0.68
0.64
0.30
0.1149
0.1501
0.3522
0.8891
GT-GAN
0.15
0.43
0.24
0.31
0.1153
0.2087
0.4078
0.9217
GraphRNN
0.42
0.74
Inf
0.95
0.7494
0.6266
0.2891
0.1874
150
S-Generator
0.08
0.31
0.11
0.29
0.0949
0.1101
0.3493
0.8493
GT-GAN
0.07
0.30
0.11
0.27
0.0931
0.2105
0.3926
0.8714
Internet of Things (IOT) datasets This application focused IOT network malware confinement prediction (predicting optimal network operation given a compromised one). There are three subsets of graph pairs with different sizes (20, 40 and 60), where the nodes represent devices and the node attributes indicating the compromised status of the nodes. The weights of the edges represent the distance between two devices. There are 334 pairs of input (compromised IOT) and target graphs (optimal IOT) in each subset and each is divided into two parts for the 2-fold cross-validation.
4.2 BASELINE METHODS
We compare our GT-GAN against five state-of-the-art graph generation methods: 1) GraphRNN (You al. 2018) new graph generation method based on sequential generation with the LSTM VAE
model; 2) Graph VAE (Simonovsky & Komodakis 2018) is probability-based graph generation
method for small graphs: 3) GraphGMG (Li et 2018 framework based upon graph neural networks for small single graphs; 4) Random VAE (Samanta et al. 2018) was described earlier; and 5) S-Generator the part of our full model GT-GAN, which essentially a graph translator with loss but no discriminator. We propose this S S-Generator model in order to evaluate the necessity of the proposed GT-GAN framework to learn the one-to-many mappings. All the comparison methods were trained on the malicious graphs without conditioning on the input graphs due to the models' inherent capability limitations. The datasets were assigned to each comparison model for the experiment based on their scalability in terms of graph size.
4.3 EVALUATION RESULTS ON SYNTHETIC DATASETS
Results for the synthetic datasets. To evaluate the similarity between the generated and real target graphs for scale-free dataset, we selected eight performance metrics: 1) two metrics are distances between generated and real graph in terms of Eigenvector centrality (En-dist) (Bonacich) 1987) and Closeness centrality (C-dist) (Freeman 1978). where the lower the distance, the better the performance; 2) two metrics are similarity score based on the graph kernels of Weisfeiler Lehman kernel (wl-sim) (Shervashidze et al.) 2011) and Lovasz Theta Kernel(It-sim) (Johansson & et al (014). where the higher the score, the better the performance; 3) four metrics are used to evaluate the the node degree distribution correlation between the generated and real target graphs by: Jensen-Shannon distances (JS), the Hellinger Distance (HD), the Bhattacharyya Distance (BD) and the Wasserstein Distances (WD), where the lower the score, the better the performance.
As shown table T our GT-GAN consistently outperforms all other baselines by large margin, especially when the graph size becomes large (i.e.having the superiority of 34.6% than other methods when size 150). The "Inf" entries represent distance over 1000. S-Generator is generally the second best methods in terms of these four evaluation metrics, highlighting the effectiveness of our proposed graph encoder and decoder.
To verify whether GT-GAN can indeed discover the underlying ground-truth translation rules between tnput-target pairs, we draw the node degree distribution curve for three pairs of generated and real target graphs by GT-GAN, shown in Fig. 4 The curves of the generated graphs closely follow
7

---

Under review as a conference paper at ICLR 2020
the power-law rule and become even closer to the real graphs the graph size increases, which is consistent with the findings in Table [ This demonstrates that our GT-GAN model successfully learns the inherent properties of scale-free graphs during translation. Similar observations for the evaluation metrics (e.g. average degree, repository and density) of the Poisson random datasets and remaining scale-free subsets can be found in Appendixes B and C.
(a) Graph Size 50
(b) Graph Size 100
(c) Graph Size 150
10²
Counts on log scale
Generated Graph
Counts on log scale
Counts on log scale
Real Graph
10'
10*
10
Generated Graph
Generated Graph
Real Graph
Real Graph
10
10
10"
6
8
10
16
2
6
10
12
14
4
10
Node Degree
Node Degree
Node Degree
Figure 4: Examples of node degree distributions of generated and target graphs for scale-free graphs
4.4 EVALUATION RESULTS ON REAL APPLICATION DATASETS
Results for the user authentication datasets. For the real world dataset, we design an indirect evaluation metric inspired from a real-world classification problem: label imbalance issues. For example, we may want to build a classifier to determine whether an authentication graph of a user is malicious (positive) normal (negative), but this user has few malicious records. For this difficult task, the graphs (i.e., malicious graphs) generated by GT-GAN, which has been trained on other users' records, can be utilized positive samples to train the classifier. Specifically, when evaluating, the test set is further split evenly into two subsets. The first subset is used to train a graph classifier, as proposed by Nikolentzos al. (2017), using only the normal graphs plus the generated malicious graphs. The second subset, which contains both the normal and real malicious graphs can then be used to validate the trained classifier. In addition, a "gold standard" classifier trained on both normal and real malicious graphs acts as the "best ceesst-ossiibl-errrooe different generative models to judge how "real" the graphs they generate are. We refer readers the detailed evaluations in Appendix E.
and is used to evaluate all the
Table 2: User authentication datasets
Table 3: IOT datasets
Size
Method
R2
MSE
ACC
Size
Method
P
R
AUC
F1
GraphRNN
0.16
1775.58
0.23
83.97%
VAL
0.32
0.51
U.26
0.39
20
GraphVAB
0.39
2109.66 64
0.32
81. 11.19%
GraphRNN
0.34
0.36
0.50
0.36
GT
0.67
370.91
0.85
92.00%
S-Generator
0.72
0.74
0.66
GraphRNN
0.44
1950.46
0.29
70.34%
GT-GAN
0.79
0.68
0.78
0.73
40
GraphVAR
0.73
2410.57
0.16
66.60%
Gold Standard
0.97
0.97
0.97
0.97
GT-GAN
408.50
93,94%
S-Geeerator
0.77
0.58
0.62
0.66
GmphRNN
0.52
1831.43
0.04
61.07%
300
GT-GAN
0.84
0.66
0.79
0.74
60
GrapVVAE
0.00
2453.61
0.04
50.64%
Gold Slamdord
0.98
0.96
037
0.97
GT-GAN
566.88
0.80
94.63%
As shown in Table 2 classifiers trained by the graphs generated by GT-GAN can classify normal and hacked behaviors effectively with AUC above 0.78, which is well above the 0.5 obtained using a random model. GT-GAN significantly outperforms other methods by around 25%, 16%, 24.5% and 22.1% %, respectively, on the four metries: precision (P), recall (R), AUC and Fl-score for the trained classifier. GT-GAN performs consistently better than other methods when the graph size rises from 50 to 300. In addition, GT-GAN clearly outperformed the S-Genertor in this evaluation setting. This confirms that using translator alone to learn a deterministic output given an input graph not sufficient to capture the generic distribution of the target graphs. In addition, the four direct evaluation mentioned above are also tested and the results can be found in Appendix C.
Results on 1OT dataset. Table 3 compared the performance of GT-GAN and other comparison methods for the IOT dataset by examining the edges of the generated and real target graphs for four metrics: MSE (mean squared error), R2 (coefficient of determination score), Pearson Correlation (P) of adjacent matrix, and ACC (Accuray) for the correct existence of edges among all the pairs of nodes. The results show that GT-GAN performed almost the best for all the three subsets. GT-GAN got highest Pearson Correlation of around 0.8 for all three subsets compared to the other methods
8

---

Under review as a conference paper at ICLR 2020
which had Pearson Correlations below 0.4. Due to the LI-loss required to maintain the topology pattern similarity, GT-GAN also outperformed the comparison methods with around 8% ,26% and 40% superiority in ACC for the three subsets, respectively, and had the smallest MSE, at just one tenth of those achieved by comparison methods.
4.5 ABLATION STUDY ON THE GRAPH ENCODERS AND DECODER
Table 4: Ablation study on four datasets
Dataset
Method
HD
BD
WD
En-dist
C-dist
wl-sim
GCN#decoder
0.18
0.48
0.27
18.84
0.6903
0.6751
0.4031
DCNN+decoder
0.65
0.96
Inf
0.77
0.6907
0.6745
0,4032
Scale-11l
Grapo-U+decoddr
0.69
0.99
Inf
5.77
0.6931
0.6496
0.4040
Aacoder+VGARe
0.31
0.63
0.51
43.78
0.0922
0.2559
0,4003
GT-GAN
0.15
0.43
0.24
0.31
0.1153
0.2087
0.4078
P
R
AUC
FI
En-dist
C-dist
wi-sim
GCN+decoder
0.31
0.35
0.52
0.33
0.7394
0.7494
0.6632
DCNN+decnder
0.59
0.55
0.55
0.57
0.0186
0.3349
0.6851
Auth-I
Graph- deecodrr
0.41
0.60
0.30
0.49
0.6789
0.6859
0.9239
Encoder+VGAIE
0.49
0.46
0.61
0.47
0.0231
0,3129
0.6111
GT-GAN
0.79
0.68
0.78
0.73
0.0134
0.1924
0.9439
+dNNedecoder
0.58
0.42
0.62
0.51
0.0007
0.1896
0.7033
Auth-II
Graph- Uedcodrr
0.42
0.44
0.23
0.32
0.6931
0.6842
0.9744
GT-GAN
0.84
0.79
0.0054
0.0681
0.9864
R2
MSE
P
ACC
En-dist
C-dist
wl-sim
GCN+decoder
0.46
818.25
0.71
92.69
0.4990
0.4349
0.3304
DdeNede
0.52
721.98
0.74
93.26
0.3596
0.3217
0.3292
I0T-1II
Graph-U Uddccodr
0.45
826.63
0.70
92.46
0.3526
0,2771
0,3310
Encoder+ VGAE
0.12
1337.16
0.44
88.14
0.4811
0.4876
0.3333
GT-GAN
0.62
566.88
0.80
94.63
0.3350
0.3051
0.3899
To further validate the superiority of the proposed graph convolution and deconvolution layers, an ablation experiment was conducted by replacing the encoder and decoder with node embedding and decoder methods normaly used. The graph encoder was replaced by the GCN (Kipf & Welling 2017), DCNN (Atwood & Towsley] 2016) and Graph U-NET (Gao & Ji] 2019), both of which consider edge and node features for graph embedding. The graph decoder was replaced by the decoder in VGAE Kiip & Welling 2016). There were thus three method combinations for comparison.
Table. shows the results of the ablation study the proposed encoder and decoder on part of scale-fie free (Scale), user authentication (Auth) and IOT datasets. There
major findings here. First, the encoder of GT-GAN outperformed both the GCN- and DCNN- based encoders by large margin on these datasets, especially for the real-world datasets, where the edges of the graphs can have a very complex meaning. For example, on Auth-I, GT-GAN performed 43%, 50%, 31 %, and 38% better on average, when compared with the GCN and DCNN encoders in terms of precision, recall, AUC and Fl-scores, respectively. Second, the proposed decoder in GT-GAN was deemed both effective and irreplaceable for graph generation. For example, IOT-III, GT-GAN performed 6.97%, 45.00% and 83.33% better than the decoder in VGAE in terms of ACC, P and R2, respectively, as well as a low MSE below 1000.
4.6 MODEL SCALABILITY ANALYSIS
We compare the scalability of GT-GAN against three graph generation methods as shown in Fig Our GT-GAN model significantly outperforms other state-of-the-ar baselines in terms of both com- putational time and memory consumption. As the graph size increases up to 50, both computational time and the memory consumption of the GT-GAN remains almost constant. In contrast, the runtime and memory consumption of Random VAE and the runtime of Graph VAE increase super-linearly as the graph size increases, making it hard to scale even to graph size of 50. Interestingly, the runtime and memory consumption of GraphRNN also increases only slightly the graph size increases. However, our GT-GAN model achieves around ten times speedups while requiring almost half of memory, compared to GraphRNN, highlighting the strong linear complexity of GT-GAN in practice.

---

Under review as a conference paper at ICLR 2020
5000
An GrepARNN
4000
3ruphVAE GT GAN
6000
grighVAk
GrspaRNN
Memory (MBB
5000
GE-GAN
3000
4000
2000
3000
2000
1000
1000
15
10
15
20
25
30
35
40
45
50
0g
15
20
25
30
35
40
45
50
Graph size
Graph size
(a) Time Cost
(b) Memory Cost.
Figure Scalability plots for memory and time cost of GT-GAN, Random VAE, Graph GraphRNN
CONCLUSIONS
This paper focuses on new problem: deep graph translation. To achieve this, we propose novel GT-GAN which translates an input graph to target graph. To learn both global and local mapping between graphs, new graph encoder-decoder model have been proposed while preserving graph patterns in various scales. Extensive experiments have been conducted on the synthetic and real-world dataset to compare with the state-of-the-ar the-art graph generation models. Experimental results show that our GT-GAN can discover the ground-trute -ttnh translation rules, and significantly outperform other baselines in terms of both effectiveness and scalability. This paper opens thread research for deep graph translation in many practical applications.
REFERENCES
James Atwood and Don Towsley. Diffussion-convllutionaal neural networks. In Advances Neural Information Processing Systems, pp. 1993-2001, 2016.
James Atwood, Siddharth Pal, Don Towsley, and Ananthram Swami. Sparse diffussion-convolutionaal neural networks. In NIPS, 2016.
Yunsheng Bai, Hao Ding, Yang Qiao, Agustin Marinovic, Ken Gu, Ting Chen, Yizhou Sun, and Wei Wang. Unsupervised inductive graph-level representation learning via graph-graph proximity. International Joint Conferences on Artificial Intelligenee 2019.
Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Sima'an an. Graph convolu- tional encoders for syntax-aware neural machine translation. arXiv preprint arXiv: arXiv:/704.0e 0e04./0447.... 2017.
Daniel Beck, Gholamreza Haffari, and Trevor Cohn. Traph-to-sqquence learning using gated graph neural networks. arXiv preprint 1806.08835. 2018.
Aleksandar Bojchevski, Oleksandr Shchur, Daniel Zügner, and Stephan Günnemann. Netgan: Generating graphs via random walks. arXiv preprint 18X33 2018.
Béla Bollobás, Christian Borgs, Jennifer Chayes, and Oliver Riordan. Directed scale-free graphs. In
Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms, pp. 132-139. Society for Industrial and Applied Mathematics, 2003.
Phillip Bonacich. Power and centrality: family of measures. American journal of sociology. 92(5): 1170-1182, 1987.
Joan Bruna, Wojeiech Zaremba, Arthur Szlam, and Yann Lecun. Spectral networks and locally con- nected networks on graphs. In International Conference on Learning Representations (ICLR2014), CBLS, April 2014, 2014.
Yu Chen, Lingfei Wu, and Mohammed Zaki. Reinforcement learning based graph-to tosqueece model for natural question generation. arXiv preprint arXiv:/908.044 9999009422 2019.
10

---

Under review as a conference paper at ICLR 2020
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax- variational autoencoder for structured data. arXiv preprint orXiv:1802.08786. 2018.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pp. 3844-3852, 2016.
Kien Do, Truyen Tran, and Svetha Venkatesh. Graph transformation policy network for chemical reaction prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 750-760. ACM, 2019.
Linton C Freeman. Centrality in social networks conceptual clarification. Social networks, 1(3): 215-239, 1978.
Hongyang Gao and Shuiwang Ji. Graph u-nets. arXiv preprint 1rX.v::905.00178, 2019.
Yuyang Gao, Lingfei Wu, Houman Homayoun, and Liang Zhao. Dyngraph2seq: aymamic-graphee sequence interpretable learning for health stage prediction in online health forums. arXiv preprint OrXiv:/908
1909. 08497, 2019.
Daniel Gildea, Giorgio Satta, and Xiaochang Peng. Cache transition systems for graph parsing. Computational Linguistics, 44(1):85-118, 2018.
Justin Gilmer, Samuel Schoenholz, Patrick Riley, Oriol Vinyals, and George E Dahl. Neural 2017.
message passing for quantum chemistry. arXiv preprint
Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In Neural Networks, 2005. IJCNN'05. Proceedings. 2005 IEEE International Joint Conference on, volume 2, pp. 729-734. IEEE, 2005.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. pp. 1,,5,103,,2111
Phillip Isola, Jun- Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 1 1125-11344 2017.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. arXiv preprint arXiv:/802 /802.04344 2018a.
Wengong Jin, Kevin Yang, Regina Barzilay, and Tommi Jaakkola. Learning multimodal graph-to- graph translation for molecular optimization. arXiv preprint arXiv: 1/2.0:18/, 2018b.
Fredrik Johansson and et al. Global graph kernels using geometric embeddings. In ICML 2014, 2014.
Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint arXiv: 161.v:1611.07308. 2016.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR), 2017.
Paul L. Krapivsky and Sidney Redner. Organization of growing random networks. Physical Review E, 63(6):066123, 2001.
Matt J Kusner, Brooks Paige, and José Miguel Hemánndez-Lobato. Grammar variational autoencoder.
In International Conference on Machine Learning, pp. 9945-1954, 2017.
Yujia Li, Daniel Tarlow, Mare Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. International Conference Learning Representations, 2016.
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. Learning deep generative models of graphs. arXiv preprint 3rXiv:1803.03324, 2018.
Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In ICML., pp. 2014-2023, 2016.

---

Under review as a conference paper at ICLR 2020
Giannis Nikolentzos, Polykarpos Meladianos, Antoine Jean-Piers Tixier, Konstantinos Skia- nis, and Michalis Vazirgiannis. Kernel graph convolutional neural networks. arXiv preprint arXiv:1710. 10689, 2017.
Xiaochang Peng, Daniel Gildea, and Giorgio Satta. Amr parsing with cache transition systems. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Bidisha Samanta, Abir De, Niloy Ganguly, and Manuel Gomez-Rodriguez. Designing random graph models using variational autoencoders with applications chemical design. arXiv preprint arXiv:1802.05283. 2018.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-80, 2009.
Michael L Seltzer. Dong Yu, and Yongqiang Wang. An investigation of deep neural networks for noise robust speech recognition. Acousties, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pp. 7398-7402. IEEE, 2013.
Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(Sep): 2539-2561, 2011.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. arXiv preprint
2018.
Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. A graph-to-sequence model for amr-to-text generation. arXiv preprint arXiv: 02447, 2018.
Mingming Sun and Ping Li. Graph to graph: topology aware approach for graph structures learning and generation. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 2946-2955, 2019.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. arXiv preprint 17X0v:/710.10000 2017.
Petar Veličković, William Fedus, William Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. arXiv preprint 17XXiv1809.100 2018.
Yuxuan Wang, Wanxiang Che, Jiang Guo, and Ting Liu, A neural transition-based approach for semantic dependency graph parsing. Thirty- Second AAAI Conference on Artificial Intelligence, 2018.
Bo Wu, Yang Liu, Bo Lang, and Lei Huang. Dgenn: Disordered graph convolutional neural network based on the gaussian mixture model. arXiv:/712.0000 171::03122.333 2017.
Kun Xu, Lingfei Wu, Zhiguo Wang, and Vadim Sheinin. Graph2seq: Graph to sequence learning with attention- based neural networks. arXiv preprint arXiv: 804400823, 2018a.
Kun Xu, Lingfei Wu, Zhiguo Wang, Mo Yu, Liwei Chen, and Vadim Sheinin. Exploiting rich syntactic information for semantic parsing with seup--t--eqqeenc model. arXiv preprint 1rXiv:1808.0760 2018b.
Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. Graphron: Generating realistic graphs with deep auto-regressive models. In International Conference Machine Learning, pp. 5694-5703, 2018.
12

---

Under review as a conference paper at ICLR 2020
A MORE DESCRIPTIONS AND EXPERIMENTS FOR SCALE FREE DATASET
Generation of graph pairs. Each input graph is generated as directed scale-free network, whose degree distribution follows power-law property [Bollobás et al. (2003). A node will be selected as target node with probability proportional node with probability of 0,41. Similarly, a node will be selected as source node with probability proportional to its out-degree, which will be linked to a new target node with probability of 0.54. Then, in the target graph, each weight between two connected nodes will be added by m, where m could be any value larger than 1. Thus, both input and target graphs are scale-free graphs.
its in-degree, which will be linked
new source
Indirect evaluation and ablation study. Other than directly measuring the node degree distribu- tion similarity between the generated and real tar- get graphs, we also conduct an indirect evaluation done for user authentication dataset. Table 5 shows the average results of graph classifiers: Pre- cision, Recall, AUC, and Fl-measure for different methods. For small graphs (e.g., nodes less then 10), the power-law property of scale-free networks is less obvious compared to the larger size graphs, which may explain why the tasks on smaller scale- free graphs are more difficult. However, when the sizes of graph increase, GT-GAN become more close to the performance of "Gold Standard" with average difference of 10%, 4%, 5% and 9% on Fl accordingly, and significantly outperforms other methods by large margin up to 51%, 35%, 10%, and 19% on F1, respectively.
Table 5: Indirect evaluation for scale- free graphs
Dataset
Method
P
R
AUC
F1
RandomVAF
0.31
0.42
GraphRNN
0.31
0.11
0.49
0.16
Gra
0.75
0.23
0.65
0.35
Scale-1
GrapGGMG
0.49
0.18
S-Generator
0.46
0.83
0.43
0.59
GT-GAN
1.00
0.50
0.52
0.67
Gold Standard
0.82
0.77
VAL
0.50
1,00
0.54
0.66
GrahRRNN
0.67
0.12
0.50
0.21
Scalo-II
S-Generator
0.50
1.00
0.50
0.67
UT-GAN
1.00
0.50
0.50
0.67
Gold Standard
0.76
0.67
0.72
0.71
VAL
0.89
0.67
0.84
0.76
GraphRNN
0.52
0.53
0.70
0.52
Scale-III
S. Generator
0.50
1,00
0.37
0.67
GT-GAN
0.93
0.94
0.87
Gold Standard
0.94
0.90
0.97
0.91
GraphRNN
0.61
0.67
0.60
Scale-1V
S-Generator
GT-GAN
0.72
0.50
0.69
1.00
0.68
0.50
0.70
0.67
Gold Standard
0.99
0.81
0.75
GraphRNN
0.73
0.92
0.92
0.81
Scale-V
S-Generator
1.00
0.50
0.50
0.67
Table 6 shows the ablation study on dataset of Scale-III and Scale-IV in both metric evaluation and indirect distribution evaluation.
GT-GAN
0,96
0,86
Gold Standard
0.99
0.93
0.96
0.95
Table 6: Ablation study on Scale- free datasets
Indirect
Direct
Dataset
Method
WD
GUNadecoder
0.85
0.16
0.89
0.27
0.32
0.65
0.58
2.87
Scale-III
Encoder+ VGGA VGAE
DCNN-decoder
0.66
0.64
0.93
0.59
0.83
0.76
0.37
0.68
0.59
0.99
0.45
Inf
49.15
5.59
GT-GAN
0.93
0.82
0.94
0.87
0.43
0.89
1.66
2.43
GCN+deodder
0.74
U.31
0.30
0.60
0.18
0.48
0.7
18.34
Scale IV
DCNN-deculer
Encodery
0.50
0.62
0.50
0.55
0.31
0.63
0.96
0.51
Inf
43.78
0.77
GT-GAN
0.72
0.69
0.68
0.70
0.15
0.43
0.24
0.31
Fig. 10 shows 18 examples of the node degree distribution curve in generated and real target graphs for scale free dataset from size 50 to 150.
B DESCRIPTIONS AND EXPERIMENTS FOR POISSON RANDOM DATASET
Generation for graph pairs. Each input graph is generated by Krapivsky & Redner (200T), which is directed growing random network. Then for an input graph with E number of edges, we randomly add KE number of edges on it to form the target graph, where follows the Poisson distribution with the mean of 5.
Distribuion on Graph Distribution of Real Graph
0.35
3025-
0o15
Experiment results. For Poisson random graphs, the distributions of k in the real target graphs and those generated graphs are compared. The mean of edge increasing ratio io for gener- ated graphs by our GT-GAN is 3.6, compared to the real value of 5, which implies that the GT-GAN generally is able discover the underlying
0.05
15.0
Number added Figure 6: Distribution of for generated and real
graphs in Poisson random dataset
13

---

Under review as a conference paper at ICLR 2020
increasing ratio between input and target graphs. More evaluation results (e.g. degree and repository) can be found in Appendix B. We draw the probability density curve of the proportion k. Fig. shows the distribution of the k in graphs generated by GT-GAN and the real graphs. The distribution plot is drew based on 3000 samples. Both of the two distribution have main degree values in the range from 2 to 7, while there is difference in the max frequency due the limit of the samples amount. However, it prove that the proposed GT-GAN do learn the distribution type of translation parameter k in this task.
Results for indirect evaluation on Poisson Random graphs are listed in Table Though the Poisson random task is easier than scale-free graphs, the GT-GAN still outperforms others on AUC and FI, and its performance is highly close to the "Gold Standard". Table shows the distance measurement between generated graphs and real graphs in several metrics. For the metric "degree", we use Wasserstein distances to measure the distance of two degree distribution. For other metrics, we calculate the MSE between generated graphs and real graphs.
Table 7: Distribution evaluation for Poisson ran- dom datasets
Table 8: MSE of Graph properties measurements for Poisson random dataset
Dataset
Mothod
R
AUC
FI
Random
0.98
0.75
0.99
0.85
Dataset
Method
Density
Ave
Reciprocity
GraphRNN
0.98
0.99
0.99
0.98
Degres
Gra
0.98
0.92
0.97
0.94
RandomVAE
0,1772
0.3917
Poisson 1
GraphCMMG
0.98
0.98
0.98
0.98
GraphRNN
0.2665
2.2078
0.1344
GT-GAN
SeGnerrttrr
1.00
1.00
0.87
0.94
0.90
Poisson-1
Gra
GrapgGMGG
0.2881
0,3519
3.1986
2.4286
0.3103
0.1338
Gold Standard
0.90
1.00
1,00
0.99
S-Generator
0.2993
1.5751
0.0737
Random VAE
1.00
0.70
0.82
GT-GAN
0,3084
1.7707
0.1327
GraphRNN
1.00
1.00
1.00
1.00
Random VAE
0,2078
7,0860
04482
Poisson
S-Generator
1.00
1.00
1.00
1.00
GraphRNNN
0.2305
4.9256
0.1
GT-GAN
0.99
1.00
0.99
Poisson-I1
S-Genorator
0,2111
3.2307
0.0430
Gold Standard
0.99
1.00
1.00
0.99
GT-GAN
0,2013
3,2047
0.0388
Random VAE
0.93
0..45
1,00
0.63
Random
lnt
23.680
U.3362
Prisson-111 111
GraphRNN
S Generator
1.00
0.49
0.99
0.98
0.35
0.99
0.65
0.99
Poisson
GraphRNN
S-Generator
0.0110
0
3.6000
0.0125
0.0125
GT-GAN
1.00
0.99
1,00
0.99
GT-GAN
0.0155
3,2960
0.0047
Standard
1.00
1.00
0.99
GraphRNN
0.0123
3.5475
0.0034
GraphRNN
1.00
0.99
1.00
0.99
Poisson IV S-Gencrator
0.0029
2.9167
0.0034
Prisson
GT-GAN
S. Generator
0.50
0,90
1.00
1.00
0,51
1.00
0.66
0.94
GT-GAN
GraphRNN
0,0012
0.0142
4.3730
0.0043
optimal
1.00
1.00
1.00
1,00
Poisson V S. Generator
0.0013
2.9467
0.0016
GraphRNN
0.95
0.59
1,00
0.96
GT-GAN
0.0061
5.0410
0.0019
Poisson V
GT GAN
S-Generator
0.97
0.50
1,00
1.00
0.49
1.00
0.66
0.98
Gold Standard
1.00
0.99
1.00
0.99
C DESCRIPTION OF IOT DATASET
There are three sets of loT nodes different amount (20, 40 and 60) encompassing temperature sensors connected with Intel ATLASEDGE Board and Beagle Boards (BeagleBone Blue), commu- nicating via Bluetooth protocol. Benign and malware activities are executed on these devices to generate the initial attacked networks as the input graphs. Benign activities include Mißench and SPEC2006, Linux system programs, and word processor. The real target graphs are generated by the classical malware confinement methods: stochastic controlling with malware detection.
D MORE EXPERIMENTAL RESULTS FOR USER AUTHENTICATION GRAPH SET
About Original Dataset This dataset includes the authentication activities of 97 users on their accessible computers and servers in an enterprise computer network (Kent, 2015). Each user account generates a log file recording the computer accessing history, which could be formulated as a directed weighted graph called authentication graph, where nodes represent computers and the directed edges weights represent the authentication activities with certain frequencies. This data set spans one calendar year of contiguous activity spanning 2012 and 2013. It originated from 33.9 billion raw event logs (1.4 terabytes compressed) collected across the LANL enterprise network of approximately 24,000 computers. Here we consider two sub dataset. First the user log-on activity set. This data represents authentication events collected from individual Windows-based desktop computers, servers, and Active Directory servers. Another dataset presents specific events taken from the authentication data that present known red team compromise events, we call malicious event. The red team data can used as ground truth of bad behavior which is different from normal user. Each graph can represent the log-on activity of one user in time window. The event graphs are defined like this: The
14

---

Under review as a conference paper at ICLR 2020
node refers to the computers that are available to user and the edge represents the log-on activity from one computer to another computer of the user.
Direct evaluation of User authentication Graph Set. For the user authentication graphs, the real target graphs and those generated are compared under well-recognized graph metrics including degree of nodes, reciprocity, and den- sity. We calculate the distance of degree dis- tribution and Mean Sqaured Error (MSE) for reciprocity and density. Table shows the mean square error of the generated graphs and real graphs for all users evaluated for both graph generation methods and ablation models.
Table MSE of Graph properties measurements
for user authentication dataset
Datavet Method
Density
Reciprecity
Ave Degree
Random VAE
0.0005
000000
6.4064
GraphRNN
0.0032
0.0030
2,7751
S-Generator
0.0244
0.0342
24,130
Auth-I
GCN-dscoder
0.0006
0.0000
0.3510
DCNN+decoder
0.0000
0.0030
0.0137
Eacedere VGAR
1,1050
0.0000
0.1052
GT-GAN
0.0003
0.0000
0,0002
S-Generator
0,0113
0.0010
8,6839
Auth-Il +dcoodrc
0,0000
0.0000
0.0039
GT-GAN
0.0004
0.0000
0.0006
(a) Regular activity
(b) Malicious activity
(c) Generated malicious actixity
182
Two connections added
connections added
Figure 7: Regular graphs, malicious graphs and generated graphs of User 049
Two connections added
three connections udded
30
20
20
19
19
8
28
(c) Senerated malriius artisity Figure 8: Regular graphs, malicious graphs and generated graphs for User 006
(4) Regular activity
[b] Malicious artivity
Case Studies on the generated target graphs.
Fig shows the example of User 049 with regular activity graph, real malicious activity graph and malicious activity graph generated by our GT-GAN from left to right. Only those of edges with difference among them are drawn for legibility. can be seen that, the hacker performed attacks on Computer 192, which has been successfully simulated by our GT-GAN. In addition, GT-GAN also correctly identified that the Computer 192 the end node (i.e., with only incoming edges) in this attack. This is because GT-GAN can learn both the global hacking patterns (i.e., graph density, modularity) but also can learn local properties for specific nodes (i.e., computers). GT-GAN even successfully predicted that the hacker connect from Computers 0 and I, with Computers 7 and 14 as false alarms. For User006 in Fig. g the red team attackers make more connections on Node 36 compared user's regular activity, marked in red rectangle. GT-GAN leans how choose the Node 36 and it generated more connections too in the Node 36.
E APPENDIX E: FLOWCHART OF INDIRECT EVALUATION PROCESS
Fig. shows the process of the indirect evaluation process for evaluating whether the generated and the real target graphs follow the same distribution.
OT-GAN puining
Grip
0T-DAN training set
GT-GAN
F ARCHITECTURE
tuining set
PARAMETER FOR GT-GAN MODEL
Chesifer tairieg
Graafie textrg at
Ough
Graph Translator: Given the graph size (number of nodes) N of graph. The output feature map size of
Clecstee
Data set partiion
Volidation process
Figure 9: Flow chart of validation
15

---

Under review as a conference paper at ICLR 2020
each layer through graph generator can be expressed
as:
N X N x 1 N x N 5 + N x N x 10 - N x x 10 + N x N 10 +
NXNXI
Diseriminator: Given the graph size (number of nodes) N of a graph. The output feature map size of each layer through graph discriminator can be expressed as:
N x N 1 + N x N x 5 + N x N x 10 + x 1 x 10
For the odge to edge layers, the size of two kemels in two directions are N x and x N. For the node to edge layer, the kernel size is x N
102
scale
hea Credd alangs sie 93
Counts scaae
Real Drogs of sie
Courts on log scale
Real Craph eigggiii 116
201.
201
10F
3UP
14
1.
12
10
Roda Degres
Nodu Degres
Node Degree
Genersss lanpt proph
Gesernted Gregh of grop 100
10"
cor Siagp wax 150
Counts on log soale
Counto on log sools
207
3m
Node Dogree
Node Dogice
Naxio Dogg
1.0
Counts on soals
usag cagi die 200
Courts log scale
How leape pripei dan 1o0
Bog
30t
201
30
10'
14
10
Mode Degrce
Node Degree
Node Degroe
Generatec Gragh grohh das uaaat grggnnn
Generate Googh graph coople (aach
10a
Genernted Geoph prapp iin
Counts log scal
Courts log scalu
Counte on log scake
Hew lanpl pripes lan 1x0
201-
301
102
10a
34
Degios
Counts or top scale
101
has Grish grach dos 50
Faal Croph of prach los 200
Courts or tog stgie
hee Grape prono dos 150
20
Counts on
10P
307
10a
2
10 Mode Degroe
12
14
26
2
10
12
14
10
Dogree
Node Degroe
Seneratic feadh din dne teaut crggraaiin
Gensrate copiiiie baogle Cagh 200
10a
Gocoartd fiogh Niw bace srrneee sain
Court: on log sate
207
County on tog soaale
101-
20
300
14
16
(a) Graph Size of 5O
(b) Graph Size 100
(ci Graph Size of 150
Figure 10: Examples of node degree distrbution for generated graphs and real graphs
16


Reviews:
[{"review_id": "H1x0ltmTFS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies a new problem, i.e., generating graphs conditioned on an input graph. The authors proposed a new framework GT-GAN, which is composed of (i) a graph encoder to representation of the input graph; (ii) a generator to generate graphs; (iii) a discriminator to fool the discriminator so that the generated graph can be more realistic; and (iv) a l1-norm regularizer to make the generated graph similar to the target graph. Experimental results on synthetic and real-world datasets demonstrate the effectiveness of the proposed method for graph translation. Overall, the problem studied in this paper is interesting and novel, and the proposed method makes sense. There are some concerns about the paper and I would like to increase my rating if the authors can address my concerns:\n\n(i) Could you double check if E_{ij}^{1,1}=A? The dimension seems to be problematic in Eq.(5) if E_{ij}^{1,1} is of dimension N x N. If E_{ij}^{1,1} is N x N, then E_{I,k1}^{l-1,n} is also N x N. In Eq.(5), mu_{k1}^m is 1 x 1 and S_{k1} is 1 x N. Please double check. E_{ij}^{1,1} is of dimension N x N\n\n(ii) The explanation of why the proposed graph convolution can learn global information in the graph embedding is unclear. For example, how the embedding can preserve the scale-free property? Could you provide more explanations?\n\n(iii) Though the studied problem is interesting, the proposed method makes sense but is not very novel. It seems to be adopting GAN with GNN and l1 regularizer.\n\n(iv) The contribution of the l1 regularizer is not analyzed. What the performance will be if we remove the l1 regularizer?\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "H1eMKX61qS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work the authors tackle the problem of generating a given graph to a target output graph.  To achieve this they develop a novel deep graph generative model. The authors place a lot of emphasis on scalability. This is indeed a major computational bottleneck in prior work, allowing the deep generative models to generate graphs with few tenths of nodes.   The authors propose an architecture that consists of a graph translator, and a conditional graph discriminator. The GAN approach is able to give significant insights into the conditional distribution p(G|H) where H is the input graph. For the graph translator, the authors design novel graph encoders and decoders. The proposed encoder-decoder achieve the best possible results, compared to using established encoder/decoders as shown in ablation study.  The authors analyze the computational complexity of their work, and while they do not discuss the computational complexities of the other methods, it is clear from the experiments that their method scales better (e.g., Figure 4). To evaluate the output of the architecture, they use a variety of different graph characteristics. The proposed method outperforms the state-of-the-art. Furthermore, the proposed method is able to detect interesting anomalies as illustrated in the appendix  (hacker detection). \n\nOverall this paper makes several interesting contributions on a challenging problem, it is well written (modulo few typos, e.g.,10X instead of 10$\\times$, generater->generator), and has convincing experiments. For these reasons I suggest its acceptance.   ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BklKS6YRtB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies a problem of graph translation, which aims at learning a graph translator to translate an input graph to a target graph. The authors propose an adversarial training framework to learn the graph translator, where a discriminator is trained to discriminate between the true target graph and the translated graph, and the translator is optimized by fooling the discriminator. The authors conduct experiments on both synthetic and real-world datasets. The results prove the effectiveness and the efficiency of the proposed approach over many baselines.\n\nStrengths:\n\n1. The problem is new and well-motivated.\nData translation is an important problem and has been widely studied in many research domains such as computer vision and natural language processing. Despite the importance, the problem has not been thoroughly explored in the graph domain, and most existing studies only focus on standard graph generation problems. In this sense, this paper studies a very new problem, which is quite novel. Moreover, the problem is important, which can have many potential downstream applications on graph data. Overall, the problem is new and well-motivated.\n\n2. The proposed approach is quite intuitive.\nThe paper proposes an adversarial training approach to the problem, where a graph translator is learned based on a graph discriminator. During training, the graph discriminator aims at discriminating between the true target graph and the translated graph conditioned on an input graph, and the graph translator is trained by fooling the discriminator. The graph translator is built on top of an encoder-decoder framework, where the encoder and decoder are parameterized by graph neural networks. Overall, the proposed method is quite reasonable, which is easy to follow.\n\n3. The results are promising.\nThe authors conduct extensive experiments on both synthetic and real-world datasets, and compare the proposed approach against many strong baseline methods for graph generation. The results are quite promising, which prove both the effectiveness and the efficiency of the approach.\n\nWeaknesses:\n\n1. The novelty of the proposed approach is limited.\nThe proposed approach is mainly built on top of the adversarial training framework, where a graph neural network is used to parameterize the graph translator. For adversarial training, although it is very intuitive, such a framework has been widely explored in the data translation problem in other domains, such as image style transfer in CV and text style transfer in NLP. Compared with these works, although the proposed approach studies a new problem, the major idea is the same as the existing studies. For the graph encoder and graph decoder, they are designed based on the idea of graph neural networks, where some propagation layers are designed to propagate information across different nodes. Although the propagation layers are specifically designed for the graph translation problem, I feel like they are not so different from existing studies (e.g, message passing neural network, graph U-net). Therefore, from the model-wise, this paper combines several existing ideas, but does not provide new insights or techniques, so the contribution is quite limited.\n\n2. The writing can be further improved.\nThe paper is not very well-written. Some parts of the paper are quite hard to follow, and the intuition behind the approach is not well explained. In section 3.2, it is said that \"the approach learns global information by looking for more virtual neighbors regarding the latent relations\". Here, it is unclear to me what is a virtual neighbor, and what is a latent relation. The authors try to illustrate their idea in figure 2, but the figure is also quite hard to understand. It would be better if the authors could explain the idea of the encoder in a more intuitive way, or give more concrete examples for illustration. Besides, equation (4) (5) and (7) are also hard to understand. The notations in these equations are quite messy, where multiple indices are used (e.g., i, j, k, l, m, n), and the intuition underlying the equations is not well explained, making it hard to understand how the encoder and the decoder work. \nAlso, there are many typos in the paper. For example:\nIn the directed graph, each node have incoming edge(s) and out-going edge(s) -> In the directed graph, each node has incoming edge(s) and out-going edge(s)\nin the \"node comvolution\" layer -> in the \"node convolution\" layer\nFirst, the “node deconvolution” layer are used to generates -> First, the “node deconvolution” layer is used to generate\nThe caption of table 1 says that the table shows the node degree distribution distance, but from the main body of texts, only four metrics are about the distribution distance, which is inconsistent to the caption.\n\nOverall, the intuition of the proposed approach is not well explained, and there are many typos to be fixed, so I feel like the writing of the paper should be further improved.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies a problem of graph translation, which aims at learning a graph translator to translate an input graph to a target graph using adversarial training framework. The reviewers think the problem is interesting. However, the paper needs to improve further in term of novelty and writing. 
Decision: Reject