G RAPH GAN: G ENERATING G RAPHS VIA R ANDOM W ALKS
Anonymous authors
Paper under double-blind review
A BSTRACT
We propose GraphGAN – the ﬁrst implicit generative model for graphs that enables to mimic real-world networks.
We pose the problem of graph generation as learning the distribution of biased random walks over a single input graph.
Our model is based on a stochastic neural network that generates discrete output samples, and is trained using the Wasserstein GAN objective.
GraphGAN enables us to generate sibling graphs , which have similar properties yet are not exact replicas of the original graph.
Moreover, GraphGAN learns a semantic mapping from the latent input space to the generated graph’s properties.
We discover, that sampling from certain regions of the latent space leads to varying properties of the output graphs, with smooth transitions between them.
Strong generalization properties of GraphGAN are highlighted by its competitive performance in link prediction and node classiﬁcation, even though not speciﬁcally trained for these tasks.
I NTRODUCTION
Generative models for graphs have a longstanding history, with applications including data augmentation, anomaly detection and recommendation (Chakrabarti & Faloutsos, 2006).
Explicit probabilistic models such as Barab ´ asi-Albert or stochastic blockmodels are the de-facto standard in this ﬁeld (Goldenberg et al., 2010).
However, it has also been shown on multiple occasions that our intuitions about structure and behavior of graphs may be misleading.
For instance, heavy-tailed degree distributions in real graphs were in stark disagreement with the models existing at the time of their discovery (Barab ´ asi & Albert, 1999).
More recent works, like Dong et al.
(2017), keep bringing up other surprising characteristics of real-world networks, not accounted for by the models at hand.
This leads us to the question: ”How do we deﬁne a model that captures all the essential (potentially still unknown) properties of real graphs?”
An increasingly popular way to address this issue in other ﬁelds is by switching from explicit (prescribed) models to implicit ones.
This transition is especially notable in Computer Vision, where
Variational Autoencoder (Kingma & Welling, 2013) and Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) signiﬁcantly advanced the state of the art over the classic prescribed approaches like Mixtures of Gaussians (Blanken et al., 2007).
GANs achieve unparalleled results in scenarios such as image and 3D objects generation (e.g., Radford et al., 2015; Berthelot et al., 2017; Wu et al., 2016).
However, despite their massive success when dealing with real-valued data, adapting GANs to handle discrete objects like graphs or text remains an open research problem (Goodfellow, 2016).
Indeed, the combinatorial structure of the graph is only one of the obstacles when applying GANs to graphs.
Second, large repositories of graphs, which all come from the same distribution, do not exist.
This means that in a typical setting one has to learn from a single graph .
And last, any model operating on a graph necessarily has to be permutation invariant , as the graphs remain isomorphic under node reordering.
In this work we introduce GraphGAN - the ﬁrst implicit generative model for graphs, that tackles all of the above challenges.
We formulate the problem of learning the graph topology as learning the distribution of biased random walks over the graph.
Like in the typical GAN setting, the generator
G – in our case deﬁned as a stochastic neural network with discrete output samples – learns to generate random walks that are plausible in the real graph, while the discriminator D then has to distinguish them from the true ones, that are sampled from the original graph.
The objective function of our model is based on the Wasserstein GAN (Arjovsky et al., 2017), which allows to
[Figure]
learn multimodal distributions and leads to more stable convergence.
Our GraphGAN exhibits strong generalization properties, which we study in detail in the experimental section.
The example in
Figure 1 shows that the graphs generated by GraphGAN possess similar properties as the input graph, as shown by the degree distributions in Figure 1c.
The generated graphs, however, are not simply exact replicas: as the visualized subset of nodes in Figures 1a and 1b shows, the graphs exhibit similar structure while being not identical; in fact, the two graphs have less than 50% of edges in common.
This initial insight is underlined by an extensive comparison of graphs generated by GraphGAN and the respective input networks in the experimental section of this work.
And even more, when generating graphs based on speciﬁc regions of the latent space learned by GraphGAN, we can smoothly interpolate between graphs with varying properties.
Our main contributions are:
• We introduce GraphGAN - the ﬁrst of its kind GAN that generates graphs via random walks.
Our model tackles the associated challenges of staying permutation invariant, learning from a single graph and generating discrete output.
• We show that our model generalizes, and is able to produce sibling graphs to the given input graph.
These graphs posses similar topological characteristics, but are not exact replicas (see Figure 1).
We further demonstrate how latent space interpolation leads to generation of graphs with smoothly changing properties.
• We highlight the generalization properties of GraphGAN by its link prediction performance, which is competitive with the state of the art on real-word datasets, although not trained
explicitly for this task.
Additionally, our model learns meaningful node embeddings as a byproduct of the training process, that can be used for node classiﬁcation.
R ELATED WORK
There are only two attempts of using GANs in the context of graphs (Tavakoli et al., 2017; Liu et al., 2017).
Tavakoli et al.
(2017)’s approach tries to generate the full adjacency matrix of the graph directly (treating it as a binary image).
To circumvent the issue of having a single graph, they apply random permutations to the adjacency matrix to generate additional training data.
Since their model explicitly generates the full adjacency matrix – including zero-elements – the output (input) size of their generator (discriminator) is equal to the number of nodes squared.
Such a quadratic complexity is infeasible in practice, allowing to process only small graphs.
Indeed, this fundamental limitation becomes evident in their reported runtime of over 60 hours for a graph with onlynodes – the largest graph they processed.
In contrast, our model operates on the random walks, thus only considering the non-zero elements of the adjacency matrix and efﬁciently exploiting the sparsity of real-wolrd graphs.
While not focusing on generating full graphs, Liu et al.
(2017) use GANs to learn graph topological features.
They decompose the graph into multiple subgraphs, where each subgraph is then processed by a GAN using standard image operations (e.g. convolution, deconvolution).
Such operations, however, assume spatial correlation, which does not apply in the graph setting: while the pixels in a local image patch are likely related, spatial correlation in a small patch of the adjacency matrix is generally not fulﬁlled due to the property of permutation invariance.
Moreover, since their GAN only generates edges within the subgraphs, but not between them, all inter-subgraph edges from the original graph have to be stored and copied manually to a potential new graph.
In contrast, our model is learned end-to-end, it allows to generate the whole graph, and it handles arbitrary permutations of the given input graph (since it is not based on spatial properties of the adjacency matrix).
Due to the challenging nature of the problem, only few approaches able to generate discrete data using GANs exist.
Most approaches focus on generating discrete sequences such as text, with some of them using reinforcement learning techniques to address the difﬁculty of backpropagation through sampling discrete random variables (Yu et al., 2017; Kusner & Hern´andez-Lobato, 2016; Li et al., 2017; Liang et al., 2017).
Other approaches modify the GAN objective to tackle the same challenge (Che et al., 2017; Hjelm et al., 2017).
Focusing on non-sequential discrete data, Choi et al.
(2017) generate high-dimensional discrete features (e.g. binary indicators, counts) in patient records.
None of these methods has considered graph structured data.
Apart from GANs, prescribed generative models for graphs have a long history and are well-studied.
For a comprehensive survey see Chakrabarti & Faloutsos (2006); Goldenberg et al.
Based on the different modeling assumptions, the generative power of these models varies signiﬁcantly.
A substantial portion cannot even handle power-law degree distributions as found in most real-world networks.
A strong representative able to capture such degree distributions, as well as diverse network topologies and community structure is the well-established degree-corrected stochastic blockmodel (DC-SBM) (Karrer & Newman, 2011).
We compare against the DC-SBM as a baseline.
GAN S FOR GRAPHS
In this section we introduce GraphGAN - a Generative Adversarial Network model for graphs.
Its core idea lies in learning the topology of a graph by learning the distribution over the random walks.
Given is an input graph of N nodes, deﬁned by an unweighted adjacency matrix A ∈ { 0 , 1 } N × N .
First, we sample a set of random walks of length T from A .
This collection of random walks serves as a training set for our model.
We use the biased second-order random walk sampling strategy described in Grover & Leskovec (2016), as it better captures both local and global graph structure.
An important advantage of using random walks is their invariance under node reordering.
Additionally, random walks only include the nonzero entries of A , thus efﬁciently exploiting the sparsity of real-world graphs.
Like any typical GAN architecture, GraphGAN consists of two main components - a generator G and a discriminator D .
The goal of the generator is to generate synthetic random walks that are plausible in the input graph.
At the same time, the discriminator learns to distinguish the synthetic random walks from the real ones, that come from the training set.
Both G and D are trained end-toend using backpropagation.
At any point of the training process it is possible to use G to generate a set of random walks, which can then be used to produce an adjacency matrix of a new generated graph.
In the rest of this section we describe each stage of this process and our design choices in more detail.
An overview of our model’s complete architecture can be seen in Figure 2.
3. A RCHITECTURE
[Equation]
Generator.
The generator G deﬁnes an implicit probabilistic model for generating the random walks: ( v 1 , ..., v T ) ∼ G .
We model G as a sequential process based on a neural network f θ parametrized by θ .
At each step t , f θ produces two values: the probability distribution over the next node to be sampled, denoted as p t , and the current memory state of the model, denoted as m t .
The new node v t (represented as a one-hot vector) is sampled from Cat( p t ) , and together with m t passed into f θ at the next step t + 1 .
Similarly to the classic GAN setting, a latent code z drawn from a multivariate standard normal distribution is passed through a parametric function g θ (cid:48) to initialize m 0 .
The generative process of G is summarized in the box above.
In this work we focus our attention on the Long short-term memory (LSTM) architecture for f θ , introduced by Hochreiter & Schmidhuber (1997).
The memory state m t of an LSTM is represented by the cell state C t , and the hidden state h t .
The latent code z goes through two separate streams, each consisting of two fully connected layers with tanh activation, and then used to initialize ( C 0 , h 0 ) .
A natural question might arise: ”Why use a model with memory and temporal dependencies, when the random walks are Markov processes?”
(2nd order Markov for biased RWs).
Or put differently, what’s the beneﬁt of using random walks of length greater than 2.
In theory, a model with large enough capacity could simply memorize all existing edges in the graph and recreate them.
However, for large graphs achieving this in practice is not feasible.
More importantly, pure memorization is not the goal of GraphGAN, rather we want to have generalization and generate similar sibling graphs, not exact replicas.
Having longer random walks combined with memory helps the model to learn the topology and general patterns in the data (e.g. community structure).
Our experiments in Sec.
4.2 conﬁrm this, showing that longer random walks are indeed beneﬁcial.
After each time step, in order to generate the next node in the random walk, the network f θ needs to output the vector of probabilities p t of length N .
Operating in such high dimensional space within the LSTM cell is infeasible, and leads to unnecessary computational overhead.
For this reason, we do the following: the model outputs o t ∈ R H , with H (cid:28) N , which is then projected up to R N using the context embedding matrix W up ∈ R N × H .
Given the probability distribution over the next node in the random walk, p t ∈ ∆ N − 1 , from which v t is to be drawn, we are faced with another challenge: Sampling from a categorical distribution is a non-differentiable operation – thus, it blocks the ﬂow of gradients and precludes backpropagation.
We circumvent this problem by using the Gumbel-Softmax estimator from Jang et al.
More speciﬁcally, we perform the following transformation: First, we let v ∗ t = softmax (cid:16) log p t + g τ (cid:17) , where τ is a temperature parameter, and g i ’s are i.i.d.
samples from a Gumbel distribution with zero mean and unit scale.
Then, the next sample is computed as v t = onehot(arg max v ∗ t ) .
While the one-hot sample v t is passed as input to the next time step, during the backward pass the gradients will ﬂow through the differentiable v ∗ t .
The choice of τ allows to trade-off between better ﬂow of gradients (large τ , more uniform v ∗ t ) and more exact calculations (small τ , v ∗ t ≈ v t ).
Now, that a new node v t is sampled, it needs to be projected back to a lower-dimensional representation before feeding into the LSTM.
This is done by means of the embedding matrix
W down ∈ R H × N .
Together with the context embedding matrix W up , the matrix W down deserves a special mention, as it happens to learn – as a byproduct – meaningful embeddings for the nodes, that can be used for other tasks.
We study properties of these embeddings in Sec. 4.3.
[Figure]
[Equation]
[Equation]
⋯ ⋯ ⋯
[Equation]
m b pp
[Equation]
p N contextemb
[Figure] GraphGAN architecture Graph G( z )
Random walk
Discrimi-nator
[Figure]
in this work (b) and the generator architecture (a).
emb denotes projection of the one-hot nodes into low-dimensional embedding space, while context emb denotes projection of low-dimensional space into dimension N .
Discriminator.
The discriminator D is based on the standard LSTM architecture.
At every time step t , a one-hot vector v t , denoting the node at the current position, is fed as input.
After processing the entire sequence, the discriminator outputs a single score that represents the probability of the random walk being real.
3. T RAINING
Wasserstein GAN.
We train our model based on the Wasserstein GAN (WGAN) framework (Arjovsky et al., 2017).
To enforce the Lipschitz constraint of the discriminator, we use the gradient penalty as in Gulrajani et al.
We observe, that in our setting using the WGAN objective leads to noticeable improvements over the vanilla GAN: it prevents mode collapse, as well as leads to a more stable learning procedure overall.
The model parameters { θ, θ (cid:48) } are trained using stochastic gradient descent with Adam (Kingma & Ba, 2014).
Weights are regularized with an L 2 penalty.
Early stopping.
Because we are interested in generalizing the input graph, the ”trivial” solution where the generator has memorized all existing edges is of no interest to us.
This means that we need to control overﬁtting of our model.
To achieve this, we employ two early stopping strategies.
The ﬁrst strategy, named V AL -C RITERION is concerned with the generalization properties of
GraphGAN.
During training, we keep a sliding window of the random walks generated in the last 1,000 iterations and use them to construct a matrix of transition counts.
This matrix is then used to evaluate the link prediction performance on a validation set (i.e. ROC and AP scores, for more details see Sec. 4.2).
We stop with training when the validation performance stops improving.
The second strategy, named EO-C RITERION makes GraphGAN very ﬂexible and gives the user control over the graph generation.
We stop with training when we achieve a user speciﬁed edge overlap between the generated graphs (see next section) and the original one at a given iteration.
Based on her end task the user can choose to generate graphs with either small or large edge overlap with the original, while maintaining structural similarity.
This will lead to generated graphs that either generalize more or are closer replicas respectively, yet still capture the properties of the original.
3. A SSEMBLING THE ADJACENCY MATRIX
After ﬁnishing the training, we use the generator G to construct a score matrix S of transition counts, i.e. we count often an edge appeared in the set of generated random walks (typically, using a much larger number of random walks than for early stopping, e.g., 500K).
While the raw counts matrix
S is sufﬁcient for link prediction purposes, we need to convert it to a binary adjacency matrix
˜
A , if we wish to reason about the synthetic graph.
First, S is symmetrized by setting s ij =
s ji = max { s ij , s ji } .
Because we cannot explicitly control for the starting node of the random walks generated by G , some high-degree nodes will likely be overrepresented.
Thus, a simple binarization strategy like thresholding or choosing top- k entries might lead to leaving out the low-degree nodes and producing singletons.
To address this issue, we use the following approach.
(i)
We ensure that every node i has at least one edge by sampling a neighbor j with probability p ij =
s ij (cid:80) v s iv .
If an edge was already sampled before, we repeat the procedure.
(ii) We continue sampling edges without replacement, using for each edge ( i, j ) the probability p ij =
s
ij (cid:80) u,v s uv , until we reach the desired amount of edges (e.g., as many edges as in the original graph).
Note, that this procedure is not guaranteed to produce a fully connected graph.
E XPERIMENTS
Besides using GraphGAN to generate graphs, we also evaluate its output and learned representations on other typical graph mining tasks, most prominently link prediction and node classiﬁcation.
We evaluate GraphGAN on these tasks and several real-world datasets and compare it with state-of-theart methods.
Furthermore, we demonstrate how we can generate graphs with smoothly changing properties via latent space interpolation.
Datasets.
For our evaluation, we use several well-know citation datasets, as well as the Political
Blogs dataset.
Table 1 show the dataset statistics.
Cora-ML is the subset of machine learning papers from the original Cora dataset typically considered in other works.
For all our experiments, we consider only the largest connected component in each network and treat them as undirected.
[Table]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
4. G RAPH GENERATION
In this task
[Figure]
we use GraphGAN to generate sibling graphs to a given input graph
[Figure]
and compare its performance to DC-SBM.
The goal is to generate graphs that are similar in their properties to the input graph – while not trivially copying the input network.
We randomly hide 15% of the edges (which are used for the stopping criterion; see Sec. 3.2) and train GraphGAN and DC-SBM on the remaining graph.
We then sample graphs from the trained models and compare their properties with the input graph.
We report here the results for CORA-ML
[Figure]
and for C ITESEER in the appendix
[Figure]
Evaluation.
Figure 3a shows that GraphGAN and DC-SBM are able to generate graphs whose degree distributions nicely match the input graph’s.
For DC-SBM, this is not surprising, given that it explicitly encodes the degree distribution in the model.
GraphGAN, however, does not have access to the degree distribution of the input graph – yet is still able to model it to a high accuracy.
Going beyond the degree distribution, in Table 2 we show ﬁve other important graph statistics and see that for most of them GraphGAN is closer to the original graph compared to DC-SBM.
We report the results for both early stopping strategies: V AL -C RITERION and EO-C RITERION .
In line with our intuition, we can see that higher EO leads to generated graphs with closer statistics to the original.
For even more statistics, as well as all of their deﬁnitions, see the appendix.
Figures 3b and 3c show how the graph statistics evolve as we train GraphGAN on C ORA -ML.
In
Fig.
3b we see that after 40K training iterations we are able to reach the assortativity value of the original graph.
Fig.
3c shows that the edge overlap smoothly increasing with the number of epochs.
Note: EO is a suitable measure of closeness since we generate graphs with same number of edges as the input.
We provide similar plots for the other graph statistics and for C ITESEER in the appendix.
4. L INK PREDICTION
Link prediction is a classical task in graph mining, where the goal is to predict new links in a given graph.
We use it to evaluate the generalization properties of GraphGAN.
We hold out 10% of edges from the graph for validation, and 5% as the test set, along with the same amount of randomly selected non-edges.
We also ensure that the training network remains connected and does not contain any singletons.
We measure the performance with the commonly used metrics ROC area-under-curve (AUC) score and precision-recall AUC score, known as average precision (AP).
To evaluate GraphGAN’s link prediction performance, we sample a speciﬁc number of random walks from the trained generator.
We use the observed transition counts between any two nodes as a measure of how likely there is an edge between them.
Additionally, we concatenate the learned node and context embeddings and use the dot product as an alternative way to perform link prediction.
We compare with Adamic/Adar (Adamic & Adar, 2003), the degree-corrected stochastic blockmodel (DC-SBM) (Karrer & Newman, 2011), and node2vec (Grover & Leskovec, 2016).
Evaluation.
The results are listed in Table 3.
There is no overall dominating method, with different methods achieving best results on different datasets.
GraphGAN shows competitive performance for all datasets, for both the transition count based and the embedding based link prediction, even achieving state-of-the-art results for some of them, despite not being explicitly trained for this task.
[Table]
Interestingly, for the transition count based link prediction, the GraphGAN performance increases when increasing the number of random walks sampled from the generator.
This is especially true for the larger networks (C ORA , DBLP, P UBMED ), since given their size we need more random walks to cover the entire graph.
This suggests that for an additional computational cost, one can get signiﬁcant gains in performance.
Note that while 100M may seem like an large number, the sampling process is trivially parallelizable.
[Figure]
L i n k p
r e d i c t i o n s c o r e
[Figure]
Sensitivity analysis.
Althogh GraphGAN has many hyperparameters – typical for a GAN model, in practice most of them are not critical for performance.
The two important exceptions are the length of the random walks T , and the discriminator type.
Figure 4 empirically conﬁrms the choice of a neural network that generates random walks of length T as opposed to just edges; the model does not have the capacity to ﬁt the model by just considering edges (i.e. random walks of length 2).
The performance gain for random walk length 20 over 16 is marginal and does not outweigh the additional computational cost; for all experiments, we therefore trained GraphGAN on random walks of length 16.
In the appendix we show that our choice of a recurrent discriminator achieves better link prediction performance than a variant based on convolutions, hence we use it for all experiments in this work.
4. N ODE CLASSIFICATION
We perform node classiﬁcation using the learned embeddings to show the generalization properties of GraphGAN.
We evaluate both the regular embeddings W down , the context embeddings W up , as well as their combination, comparing with node2vec as a strong baseline.
Note that unlike the
[Figure]
link prediction task we cannot use the generated graph to perform node classiﬁcation. Similar to (Grover & Leskovec, 2016; Perozzi et al., 2014) we train a logistic regression model on a small randomly selected subset ( < 20% of nodes) using the ground-truth labels and evaluate the classiﬁcation performance on the remaining samples. Additionally, we visualize the embeddings using t-SNE to demonstrate that a community structure has emerged in the embedding space.
Evaluation.
In Figure 5a, we visualize the weighted macro F1 score for node classiﬁcation.
The results are averaged over ﬁve trials, and the shaded areas indicate the standard deviation of the respective curves.
The embeddings produced by GraphGAN as a byproduct show competitive performance with node2vec – an algorithm speciﬁcally designed to learn useful node embedding.
This indicates that during the process of learning to generate random walks GraphGAN also learned meaningful node embeddings.
Figure 5b further emphasizes this.
In this t-SNE visualization of the context embeddings learned by GraphGAN, we can clearly see a grouping of nodes from the same community, which means that they lie in similar regions of the embedding space.
Interestingly, the context embeddings outperform the regular node embeddings by a surprisingly large margin, which may be a starting point for follow-up research.
Although GraphGAN’s learned embedding clearly show they they capture useful information about the nodes, they are only a byproduct of the training procedure.
The main goal of GraphGAN is graph generation, if node classiﬁcation is the user goal, dedicated node embedding algorithms are better suited.
4. L ATENT VARIABLE INTERPOLATION
Latent space interpolation is a good way to gain insight into what kind of structure the generator was able to capture.
To be able to visualize the properties of the generated graphs we train our model with only two dimensions of unit Gaussian noise as the input.
Then instead of sampling from the entire latent space z , we now sample from subregions and visualize the results.
More speciﬁcally, we use the inverse of the cumulative distribution Φ( p ( z )) to divide the two-dimensional latent space Ω into 20 × 20 bins of equal probability mass and generate 62.5K random walks per bin from our model.
We evaluate properties of both the generated random walks themselves, as well as properties of the resulting graph when sampling a binary adjacency matrix for each bin, visualizing them as heatmaps.
Evaluation.
In Figure 6 we see some statistics of the generated random walks on C ORA -ML within their respective bins.
In Fig.
6a and 6b we see properties of the generated random walks; in Fig.
6c and 6d, we visualize properties of graphs sampled from the random walks in the respective bins.
In all four heatmaps, we see distinct patterns, e.g. higher average degree of starting nodes for the bottom right region of Fig.
6a, or higher degree distribution inequality in the top-right area of
Fig.
6c.
While Figure 6c and 6d, show that certain regions of z correspond to generated graphs with very different degree distributions, recall that sampling from the entire latent space ( Ω ) yields sibling graphs with degree distribution similar to the original graph (see Fig. 1c).
We provide further heatmaps for other metrics (16 in total) as well as visualizations for C ITESEER in the appendix.
This experiment clearly demonstrates that by interpolating in the latent space we can obtain graphs with smoothly changing properties.
The smooth transitions in the heatmaps provide evidence that our model learns to map speciﬁc parts of the latent space to speciﬁc properties of the graph.
[Figure]
[Figure]
is the community distribution when from the
) is the community distribution of the C ORA -
We can also see this mapping from latent space to the generated graph properties in the community distribution histograms on a 10 × 10 grid in Fig.
Marked by (*) and ( Ω ) we see the community distributions for the input graph and the graph obtained by sampling on the complete latent space, respectively.
In Fig.
7b and 7c, we see the evolution of selected community shares when following a trajectory from top to bottom, and left to right, respectively.
The community histograms resulting from sampling random walks from opposing regions of the latent space are very different; again the transitions between these histograms are smooth, as can be seen in the trajectories in Fig.
7b and 7c.
D ISCUSSION AND FUTURE WORK
Being the ﬁrst model of its kind, GraphGAN opens numerous avenues for future work.
Other types of data.
While plain graphs are ubiquitous, many of real-world applications deal with attributed, k-partite or heterogeneous networks.
Adapting the GraphGAN model to handle these other modalities of the data is a promising direction for future research.
Especially important would be an adaptation to the dynamic / inductive setting, when new nodes are added over time.
Scalability.
There are multiple directions in which the GraphGAN model can be improved to permit scaling to massive graphs (100K+ nodes).
We have observed in Sec.
4.2, that it takes a large number of generated random walks to get representative transition counts for large graphs.
While sampling random walks from GraphGAN is trivially parallelizable, a possible extension of our model is to use a conditional generator, i.e. the generator can be provided a desired starting node, thus ensuring an even coverage.
On the other hand, the sampling procedure itself can be sped up by incorporating a hierarchical softmax output layer - a method commonly used in natural language processing.
Evaluation measures.
It is nearly impossible to judge whether a graph is realistic by visually inspecting it (unlike images, for example).
In this work we already used a number of standard graph properties to quantitatively evaluate performance of our model.
However, such metrics might not be optimal for the given problem.
Developing new measures will allow us to more fairly evaluate competing methods, and better understand the behavior of generative graph models.
C ONCLUSION
GraphGAN is the ﬁrst work to successfully bridge the worlds of implicit modeling and graphs.
Our work enables future researchers to gain better insight into the properties of real networks and opens new and exciting lines of research.
We are able to generate realistic graphs by learning to generate (biased) random walks from the same distribution as the random walks from an input graph.
We employ the GAN framework to learn our implicit generative model, overcoming key challenges such as permutation invariance, working in the discrete domain and having a single graph as input.
Our generator is able to generate sibling graphs that maintain structural similarity with the original graph without being exact replicas.
Better yet, using our deﬁned stopping criteria, we can control how close are the generated graphs to the original.
We further show that GraphGAN learns a semantic mapping from the latent space to the properties of the generated graph, which is evidenced by the smooth transitions of the output.
GraphGAN shows strong generalization properties, as demonstrated by the competitive performance on the link prediction and node classiﬁcation tasks, without being explicitly trained with these tasks in mind.
A PPENDIX
We denote as V the set of all nodes in a graph, E as the set of edges, and
as C i ⊆ V , i ∈ { 1 , . . .
, k } communities in a graph.
Every node in the graphs we consider is assigned to exactly one of these communities.
N ( v ) =
{ v (cid:48) | ( v, v (cid:48) )
∈ E } denotes the set of neighbors of a node v [Table]
and d ( v ) = | N ( v ) | is the degree of node v .
Given two graphs G 1 and G 2 with the same number of nodes and edges, i.e. | V 1 | = | V 2 | , | E 1 | = | E 2 | , we deﬁne their edge overlap as
[Equation]
[Table]
Size
[Table]
[Table]
[Figure]
[Figure] [Figure]
[Figure] [Figure]
[Table]
[Figure]