Original: SHADOWCAST: C ONTROLLABLE G RAPH G ENERATION WITH E XPLAINABILITY
Revised: SHADOWCAST: C ONTROLLABLE G RAPH G ENERATION WITH E XPLAINABILITY

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce the problem of explaining graph generation, formulated as controlling the generative process to produce  graphs of explainabledesired structures.
Revised: We introduce the problem of explaining graph generation, formulated as controlling the generative process to produce desired graphs with explainable structures.

Original: By directing this generative process, we canmeasure and explain the observed outcomes.
Revised: By directing this generative process, we can explain the observed outcomes.

Original: We propose S HADOW C AST , a controllable generative model capable of mimicking real-world networks and directing the generation, as a novel approach to this problem.
Revised: We propose S HADOW C AST , a controllable generative model capable of mimicking real-world networks and directing the generation, as a novel approach to this problem.

Original: The proposed model is based on a conditional generative adversarial network for graph data.
Revised: The proposed model is based on a conditional generative adversarial network for graph data.

Original: We design it with the capability to maintain generation control by managing the conditions.
Revised: We design it with the capability to maintain generation control by managing the conditions.

Original: Comprehensive experiments on three real-world network datasets demonstrate our model’s competitive performance in the graph generation task.
Revised: Comprehensive experiments on three realworld network datasets demonstrate our model’s competitive performance in the graph generation task.

Original: Furthermore, we direct S HADOW C AST to generate explainable graphs of different structures to show its effective controllability.
Revised: Furthermore, we direct S HADOW C AST to generate explainable graphs of different structures to show its effective controllability.

Original: As the ﬁrst work to pose the problem of explaining generated graphs by controlling the generation, S HADOW C AST paves the way for future research in this exciting area.
Revised: As the ﬁrst work to pose the problem of explaining generated graphs by controlling the generation, S HADOW C AST paves the way for future research in this exciting area.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In many real-world networks, including but not limited to communication, ﬁnancial, and social networks, graph generative models are applied to model relationships among actors.
Revised: In many real-world networks, including but not limited to communication, ﬁnancial, and social networks, graph generative models are applied to model relationships among actors.

Original: It is crucial that the models  generate graphs of desired properties that are easily understandable. Moreover, as complementary information such as ground-truth labels and attributes become more readily available in networks, the valuable contextual data could further increase our understanding of the modelgenerated results.
Revised: It is crucial that the models not only mimic the structure of observed networks but also generate graphs with desired properties because it allows for an increased understanding of these relationships. Currently, there are no such methods for explaining graph generation.

Original: Meaningful interactions between agents are often investigated under different what-if scenarios, which determines the feasibility of the interactions under abnormal and unforeseen circumstances.
Revised: Meaningful interactions between agents are often investigated under different what-if scenarios, which determines the feasibility of the interactions under abnormal and unforeseen circumstances.

Original: In such investigations, instead of using actual data, we can generate synthetic data to study and test the systems (Barse et al., 2003; Skopik et al., 2014).
Revised: In such investigations, instead of using actual data, we can generate synthetic data to study and test the systems (Barse et al., 2003; Skopik et al., 2014).

Original: However, there are many challenges.
Revised: However, there are many challenges.

Original: (1) Data is not accessible by direct measurement of the system.
Revised: (1) Data is not accessible by direct measurement of the system.

Original: (2) Data is not available.
Revised: (2) Data is not available.

Original: (3) Data produced by generative models cannot be explained.
Revised: (3) Data produced by generative models cannot be explained.

Original: To address these challenges, we have to answer a natural and meaningful question: Can we control the generative process to shape and explain the generated graphs?
Revised: To address these challenges, we have to answer a natural and meaningful question: Can we control the generative process to shape and explain the generated graphs?

Original: In this work, we introduce the novel problem of explaining graph generation.
Revised: In this work, we introduce the novel problem of explaining graph generation.

Original: The goal is to generate graphs of desired shapes by learning to control the associated graph properties and structure to inﬂuence the generative process.
Revised: The goal is to generate graphs of desired shapes by learning to control the associated graph properties and structure to inﬂuence the generative process.

Original: We provide an illustrative case study of email communications in an organization  (Figure 1), where interactions of the employees follow a regular pattern during normal operations.
Revised: We provide an illustrative case study of email communications in an organization with two departments (Figure 1), where interactions of the employees follow a regular pattern during normal operations.

Original: Due to limited data, previously observed network information may be missing scenarios of intra-department email surge within either the Human Resources or Accounting departments.
Revised: Due to limited data, previously observed network information may be missing scenarios of intra-department email surge within either the Human Resources or Accounting departments.

Original: When such situations are required for analyzing the system, an ideal model should generate graphs that reﬂect these scenarios (seethe box in Figure 1) while maintaining the organizational structure.
Revised: When such situations are required for analyzing the system, an ideal model should generate graphs that reﬂect these scenarios (see box in Figure 1) while maintaining the organizational structure.

Original: By effectively controlling the generative process, S HADOW C AST allows users to generate designed graphs that meet conditions resembling a wide range of possibilities.
Revised: By effectively controlling the generative process, S HADOW C AST allows users to generate designed graphs that meet conditions resembling a wide range of possibilities.

Original: Overall, this is a meaningful problem because controlling the generative process to explain generated networks proves to be valuable in many applications such as anomaly detection and data augmentation.
Revised: Overall, this is a meaningful problem because controlling the generative process to explain generated networks proves to be valuable in many applications such as anomaly detection and data augmentation.

Original: Existing graph generative models aim to mimic the structure of given networks, but they cannot easily shape graphs into other desired states.
Revised: Existing graph generative models aim to mimic the structure of given networks, but they cannot easily shape graphs into other desired states.

Original: These works either directly capture the graph structure (Cao & Kipf, 2018;
Revised: These works either directly capture the graph structure (Cao & Kipf, 2018;

Original: Liu et al., 2017; Tavakoli et al., 2017; Zhou et al., 2019; Ma et al., 2018;
Revised: Liu et al., 2017; Tavakoli et al., 2017; Zhou et al., 2019; Ma et al., 2018;

Original: You et al., 2018; Simonovsky & Komodakis, 2018; Bojchevski et al., 2018) or model node feature information (Kipf & Welling, 2016; Wang et al., 2018; Grover et al., 2019; Zou & Lerman, 2019).
Revised: You et al., 2018; Simonovsky & Komodakis, 2018; Bojchevski et al., 2018) or model node feature information (Kipf & Welling, 2016; Wang et al., 2018; Grover et al., 2019; Zou & Lerman, 2019).

Original: Most of them adopt implicit model approaches, such as the popular generative adversarial networks (GANs) (Goodfellow, 2016).
Revised: Most of them adopt implicit model approaches, such as the popular generative adversarial networks (GANs) (Goodfellow, 2016).

Original: Only very recent advances (Li et al., 2018; Yang et al., 2019) in network generation have started injecting auxiliary information into the model by adding graph-level conditions as additional inputs.
Revised: Only very recent advances (Li et al., 2018; Yang et al., 2019) in network generation have started injecting auxiliary information into the model by adding graph-level conditions as additional inputs.

Original: However, none of them allow direct control over the generative process, which addresses the fundamental challenge of generating explainable graphs.
Revised: However, none of them allow direct control over the generative process, which addresses the fundamental challenge of generating explainable graphs.

Original: While there are no existing methods for explaining graph generation, studies of explainability in other AI methods are increasing in popularity.
Revised: While there are no existing methods for explaining graph generation, studies of explainability in other AI methods are increasing in popularity.

Original: One family of work, proxy methods (Huysmans et al., 2011; Augasta & Kathirvalavakumar, 2011; Zilke et al., 2016; Lakkaraju et al., 2017), learns to approximate model predictions with simpler surrogate models.
Revised: One family of work, proxy methods (Huysmans et al., 2011; Augasta & Kathirvalavakumar, 2011; Zilke et al., 2016; Lakkaraju et al., 2017), learns to approximate model predictions with simpler surrogate models.

Original: Another line of work (Adadi &
Revised: Another line of work (Adadi &

Original: Berrada, 2018; Guidotti et al., 2018; Koh & Liang, 2017) treats models as black-boxes and carefully queries them for relevant information to form interpretations of the results.
Revised: Berrada, 2018; Guidotti et al., 2018; Koh & Liang, 2017) treats models as black-boxes and carefully queries them for relevant information to form interpretations of the results.

Original: The works closest to our problem are in interpretable Graph Neural Network (GNN) models, where models predict and assign values to edges via attention mechanisms (Veliˇckovi´c et al., 2018;
Revised: The works closest to our problem are in interpretable Graph Neural Network (GNN) models, where models predict and assign values to edges via attention mechanisms (Veli ˇ ckovi ´ c et al., 2018;

Original: Neil et al., 2018; Xie &
Revised: Neil et al., 2018; Xie &

Original: Grossman, 2018).
Revised: Grossman, 2018).

Original: Notably, even the latest work (Ying et al., 2019), which considers both graph structure and node feature information, still only explains predictions of individual nodes but cannot produce explanations for entire graphs.
Revised: Notably, even the latest work (Ying et al., 2019), which considers both graph structure and node feature information, still only explains predictions of individual nodes but cannot produce explanations for entire graphs.

Original: We propose S HADOW C AST , an approach for explaining graph generation, to address the challenge of controlling the generative process to generate graphs of desired properties.
Revised: We propose S HADOW C AST , an approach for explaining graph generation, to address the challenge of controlling the generative process to generate graphs of desired properties.

Original: The model architecture is essentially based on conditional GANs (Mirza & Osindero, 2014).
Revised: The model architecture is essentially based on conditional GANs (Mirza & Osindero, 2014).

Original: The model introduces control by leveraging the conditions as a control vector to inﬂuence the generative process.
Revised: The model introduces control by leveraging the conditions as a control vector to inﬂuence the generative process.

Original: It allows for user-speciﬁed parameters such as density distributions to generate designed graphs that are explainable.
Revised: It allows for user-speciﬁed parameters such as density distributions to generate designed graphs that are explainable.

Original: Finally, the generator captures essential graph structures while exploring a myriad of other possibilities in multifarious networks.
Revised: Finally, the generator captures essential graph structures while exploring a myriad of other possibilities in multifarious networks.

Original: We ﬁrst evaluate S HADOW C AST on three real-world social and information networks to demonstrate its competitive performance against several state-of-the-art graph generation methods in mimicking given graphs.
Revised: We ﬁrst evaluate S HADOW C AST on three real-world social and information networks to demonstrate its competitive performance against several state-of-the-art graph generation methods in mimicking given graphs.

Original: Our model achieves impressive results that are superior in most datasets.
Revised: Our model achieves impressive results that are superior in most datasets.

Original: In addition, we demonstrate the capability of S HADOW C AST to produce customizable synthetic graphs through tunable parameters, which existing generative models are incapable of performing.
Revised: In addition, we demonstrate the capability of S HADOW C AST to produce customizable synthetic graphs through tunable parameters, which existing generative models are incapable of performing.

Original:  E XPLAINABLE G RAPH G ENERATION
Revised:  E XPLAINABLE G RAPH G ENERATION

Original: In this section, we describe the explainable graph generation problem.
Revised: In this section, we describe the explainable graph generation problem.

Original: The core idea of the problem lies in generating synthetic graphs from an observed graph that holds some elementary properties— such as node attributes—where the ability to generate desired graphs through control of these properties is a form of explainability.
Revised: The core idea of the problem lies in generating synthetic graphs from an observed graph that holds some elementary properties— such as node attributes—where the ability to generate desired graphs through control of these properties is a form of explainability.

Original: We deﬁne the properties and its structure as a shadow and introduce our approach S HADOW C AST .
Revised: We deﬁne the properties and its structure as a shadow and introduce our approach S HADOW C AST .

Original: Since graphs are complex objects due to their interconnected nature, it is a challenge to control their generation directly; we instead model them through shadows, which are easy-to-understand, node-level properties that are intended to capture graph semantics in an interpretable way, which can then be manipulated to control the graph generation.
Revised: Since graphs are complex objects due to their interconnected nature, it is a challenge to control their generation directly; we instead model them through shadows, which are easy-to-understand, node-level properties that are intended to capture graph semantics in an interpretable way, which can then be manipulated to control the graph generation.

Original: Given an input graph and its shadow, we train our model to capture the topology of a graph and achieve controllable generation.
Revised: Given an input graph and its shadow, we train our model to capture the topology of a graph and achieve controllable generation.

Original: We depict the problem and our approach in detail below (Sections 2.1 and 2.2).
Revised: We depict the problem and our approach in detail below (Sections 2.1 and 2.2).

Original: 2. P ROBLEM F ORMULATION
Revised: 2. P ROBLEM F ORMULATION

Original: We focus on the novel problem of explaining graph generation.
Revised: We focus on the novel problem of explaining graph generation.

Original: Let G =
Revised: Let G =

Original: ( V , E ) denote a graph with N nodes v i ∈ V and E edges ( v i , v j ) ∈ E .
Revised: ( V , E ) denote a graph with N nodes v i ∈ V and E edges ( v i , v j ) ∈ E .

Original: Each node is associated with some identity information, e.g., the employee ID.
Revised: Each node is associated with some identity information, e.g., the employee ID.

Original: In addition, we induce another graph with N nodes and the same edge connections as in G ,  which we deﬁne as shadow S .
Revised: In addition, we induce another graph with N nodes and the same edge connections as in G , by the node properties, which we deﬁne as shadow S .

Original: Each node in the shadow is associated with some property label k i ∈ K , e.g., the employee’s department, and it “shadows” the corresponding node in G .
Revised: Each node in the shadow is associated with some property label k i ∈ K , e.g., the employee’s department, and it “shadows” the corresponding node in G .

Original: Every node in G can be uniquely identiﬁed by the identity, whereas the labels of nodes in S are not necessarily unique.
Revised: Every node in G can be uniquely identiﬁed by the identity, whereas the label of each node in S is not necessarily unique.

Original: Shadow nodes provide important explanatory information that is useful in understanding the generated graphs.
Revised: Shadow nodes provide important explanatory information that is useful in understanding the generated graphs.

Original: We note that there could be other attributes of interest, e.g., node degree, or a shadow with different connectivity than G .
Revised: We note that there could be other attributes of interest, e.g., node degree, or a shadow with different connectivity than G .

Original: We leave the inclusion of additional properties as extensions for future work.
Revised: We leave the inclusion of additional properties as extensions for future work.

Original: In this work, we aim to develop an explainable network graph generative model.
Revised: In this work, we aim to develop an explainable network graph generative model.

Original: By training the model Θ on a graph G and its shadow S , the model would then monitor the generative process and subject the generation to direction—aiding in the explainability of the generated graphs.
Revised: By training the model Θ on a graph G and its shadow S , the model would then monitor the generative process and subject the generation to direction—aiding in the explainability of the generated graphs.

Original: We formulate thetwo-step process as (1) given a graph G with some properties, discover key explanatory  properties S of the graph,and (2) train a model Θ to learn a representation ˜ S of these properties S , and control the generative process through ˜ S to generate graphs ˜ G ’s.
Revised: We formulate the process as follows: given a graph G and key explanatory node properties S of the graph, train a model Θ to learn a representation ˜ S of these properties S , and control the generative process through ˜ S to generate graphs ˜ G ’s.

Original: Let us deﬁne the Explainable Graph Generation (X2G) problem as such:
Revised: Let us deﬁne the Explainable Graph Generation (X2G) problem as such:

Original: Given a graph G , uncover key explanatory node properties of G to form another graph, deﬁned as shadow S ; learn a representation ˜ S of the shadow and control ˜ S to generate explainable graphs ˜ G ’s.
Revised: Given a graph G and key explanatory node properties of G , induce another graph by these properties, deﬁned as shadow S ; learn a representation ˜ S of the shadow and control ˜ S to generate explainable graphs ˜ G ’s.

Revised: While ground-truth labels and node attributes are valuable in understanding the model-generated results, these auxiliary data are not always readily available.

Revised: We leave the uncovering of such complementary information as a potential direction for further development.

Original: 2. P ROPOSED M ODEL
Revised: 2. P ROPOSED M ODEL

Original: We propose S HADOW C AST , an explainable generative approach that leverages both conditional modeling and GANs to generate graph-structured data.
Revised: We propose S HADOW C AST , an explainable generative approach that leverages both conditional modeling and GANs to generate graph-structured data.

Original: Our approach is inspired by the recent work (Bojchevski et al., 2018) that poses the graph generation problem as learning a distribution of biased random walks over the input graph, which captures the underlying distribution of a graph where nodes belong in some ground-truth communities.
Revised: Our approach is inspired by the recent work (Bojchevski et al., 2018) that poses the graph generation problem as learning a distribution of biased random walks over the input graph, which captures the underlying distribution of a graph where nodes belong in some ground-truth communities.

Original: Similar to any archetypal generative adversarial nets, S HADOW C AST consists of two ‘adversaries’—a generative model G and a discriminative model D .
Revised: Similar to any archetypal generative adversarial nets, S HADOW C AST consists of two ‘adversaries’—a generative model G and a discriminative model D .

Original: In addition, our approach consists of a shadow caster model S that takes in some sequences sampled from the shadow and produces shadow walks that directly inﬂuence the generator G .
Revised: In addition, our approach consists of a shadow caster model S that takes in some sequences sampled from the shadow and produces shadow walks that directly inﬂuence the generator G .

Original: The goal of G is to capture the distribution over the data x and generate synthetic graph random walks and conditions that are similar to the real walks.
Revised: The goal of G is to capture the distribution over the data x and generate synthetic graph random walks and conditions that are similar to the real walks.

Original: At the same time, D estimates the probability that a graph random walk and its conditions came from the real graph rather than G , to distinguish between the synthetic and real walks.
Revised: At the same time, D estimates the probability that a graph random walk and its conditions came from the real graph rather than G , to distinguish between the synthetic and real walks.

Original: We provide details of our model architecture (Figure 2) and design choices below.
Revised: We provide details of our model architecture (Figure 2) and design choices below.

Original: Using the conditional GAN framework, we train both G and D conditioned on some extra information—sampled shadow walks s from the shadow S .
Revised: Using the conditional GAN framework, we train both G and D conditioned on some extra information—sampled shadow walks s from the shadow S .

Original: By allowing our model to consider any auxiliary information such as ground-truth communities or data from other sources, the model can (1) leverage extra information from different data modalities, and (2) directly control the data generation process.
Revised: By allowing our model to consider any auxiliary information such as ground-truth communities or data from other sources, the model can (1) leverage extra information from different data modalities, and (2) directly control the data generation process.

Original: For example, by using contextual information in the social communications of an organization, we learn semantically meaningful graph representations.
Revised: For example, by using contextual information in the social communications of an organization, we learn semantically meaningful graph representations.

Original: We can then explicitly generate networks of any given context.
Revised: We can then explicitly generate networks of any given context.

Original: Following Mirza & Osindero (2014), we introduce the conditional GAN training for graph shadow random walks and deﬁne the loss as:
Revised: Following Mirza & Osindero (2014), we introduce the conditional GAN training for graph shadow random walks and deﬁne the loss as:

Original: [Equation]
Revised: [Equation]

Original: where z ∼ N ( 0 , I d ) is a latent noise from a multivariate standard normal distribution.
Revised: where z ∼ N ( 0 , I d ) is a latent noise from a multivariate standard normal distribution.

Original: We represent a social transaction network as an input graph of N nodes as a binary adjacency matrix
Revised: We represent a social transaction network as an input graph of N nodes as a binary adjacency matrix

Original: A ∈ { 0 , 1 } N × N .
Revised: A ∈ { 0 , 1 } N × N .

Original: We then sample sets of random walks of length T from A to use as training data x for our model.
Revised: We then sample sets of random walks of length T from A to use as training data x for our model.

Original: Following Bojchevski et al.
Revised: Following Bojchevski et al.

Original: (2018), we use a biased second-order random walk sampling strategy (Grover & Leskovec, 2016)—one of the advantageous properties of random walks is their invariance under node reordering—in order to better capture both global and local graph structures.
Revised: (2018), we use a biased second-order random walk sampling strategy (Grover & Leskovec, 2016)—one of the advantageous properties of random walks is their invariance under node reordering—in order to better capture both global and local graph structures.

Original: Another advantage of random walks is that the walks only include connected nodes, which efﬁciently exploits the sparsity of real-world graphs by including nonzero values of the adjacency matrix A .
Revised: Another advantage of random walks is that the walks only include connected nodes, which efﬁciently exploits the sparsity of real-world graphs by including nonzero values of the adjacency matrix A .

Original: In the rest of this section, we describe in detail each stage of the S HADOW C AST generation process and formally present the procedure (Algorithm 1).[Table]
Revised: In the rest of this section, we describe in detail each stage of the S HADOW C AST generation process and formally present the procedure (Algorithm 1).

Original: [Table]
Revised: [Table]

Original: Shadow Caster
Revised: Shadow Caster

Original: The shadow caster S is a sequence-to-sequence model that learns arrays of contiguous node properties from sampled shadow walks on the shadow.
Revised: The shadow caster S is a sequence-to-sequence model that learns arrays of contiguous node properties from sampled shadow walks on the shadow.

Original: The network predicts a sequence of inputs one at a time when some sequence is observed.
Revised: The network predicts a sequence of inputs one at a time when some sequence is observed.

Original: We model S with a long short-term memory (LSTM) (Hochreiter & Schmidhuber, 1997) neural network.
Revised: We model S with a long short-term memory (LSTM) (Hochreiter & Schmidhuber, 1997) neural network.

Original: Given sampled sequences of shadow walks ( s 1 , . . .
Revised: Given sampled sequences of shadow walks ( s 1 , . . .

Original: , s T ) from the shadow as inputs, the shadow caster S then generates synthetic shadow walks (˜ s 1 , . . .
Revised: , s T ) from the shadow as inputs, the shadow caster S then generates synthetic shadow walks (˜ s 1 , . . .

Original: , ˜ s T ) to mimic the sampled walks.
Revised: , ˜ s T ) to mimic the sampled walks.

Original: Generator
Revised: Generator

Original: The generator G is a probabilistic sequential learning model that generates conditional graph random walks ( v 1 , . .
Revised: The generator G is a probabilistic sequential learning model that generates conditional graph random walks f θ .

Original: . , v T ) ∼ G . We model G using another parameterized LSTM network f θ . At each step t , f θ takes as input the previous memory state m t − 1 of the LSTM model, the current additional information ˜ s t , and the last node v t − 1 .
Revised: At each step t , f θ takes as input the previous memory state m t − 1 of the LSTM model, the current additional information ˜ s t , and the last node v t − 1 .

Original: The model produces two values ( p t , m t ) , where p t denotes the probability distribution over the current node and m t the current memory state.
Revised: The model produces two values ( p t , m t ) , where p t denotes the probability distribution over the current node and m t the current memory state.

Original: Next, the current node v t is sampled from a categorical distribution v t ∼ Cat ( σ ( p t )) using a one-hot vector representation, where σ ( · ) is the softmax function.
Revised: Next, the current node v t is sampled from a categorical distribution v t ∼ Cat ( σ ( p t )) using a one-hot vector representation, where σ ( · ) is the softmax function.

Original: In order to initialize the model, we draw a latent noise from a multivariate standard normal distribution z ∼ N ( 0 , I d ) and pass it through a hyperbolic tangent function g θ (cid:48)
Revised: In order to initialize the model, we draw a latent noise from a multivariate standard normal distribution z ∼ N ( 0 , I d ) and pass it through a hyperbolic tangent function g θ (cid:48)

Original: ( z ) to compute memory state m 0 .
Revised: ( z ) to compute memory state m 0 .

Original: Generator G takes as inputs the noise z and sampled shadow walks s , and outputs graph random walks ( v 1 , . . .
Revised: Generator G takes as inputs the noise z and sampled shadow walks s , and outputs graph random walks ( v 1 , . . .

Original: , v T ) .
Revised: , v T ) .

Original: Through this process, we generate fake random walks.
Revised: Through this process, we generate fake random walks.

Original: At this point, we ﬁnd that our model is closely related to the recently introduced random walk graph generative model (Bojchevski et al., 2018).
Revised: At this point, we ﬁnd that our model is closely related to the recently introduced random walk graph generative model (Bojchevski et al., 2018).

Original: In addition to random noise model initialization, we ﬁnd that the generator greatly beneﬁts from the auxiliary information ˜ s t to model a more accurate representation of the original graph.
Revised: In addition to random noise model initialization, we ﬁnd that the generator greatly beneﬁts from the auxiliary information ˜ s t to model a more accurate representation of the original graph.

Original: We summarize the generative process of G in the box below.
Revised: We summarize the generative process of G in the box below.

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: T ))

Original: Discriminator
Revised: Discriminator

Original: The discriminator D is a binary classiﬁcation LSTM model.
Revised: The discriminator D is a binary classiﬁcation LSTM model.

Original: The goal of D is to discriminate between real walks sampled from walking on the original graph and fake walks generated by G .
Revised: The goal of D is to discriminate between real walks sampled from walking on the original graph and fake walks generated by G .

Original: At each time-step t , the discriminator takes two inputs: the current node v t and the associated shadow s t , both represented as one-hot vectors.
Revised: At each time-step t , the discriminator takes two inputs: the current node v t and the associated shadow s t , both represented as one-hot vectors.

Original: After processing each presented sequence of shadow and graph walks, D outputs a score between 0 and 1 , indicating the probability that a walk is real.
Revised: After processing each presented sequence of shadow and graph walks, D outputs a score between 0 and 1 , indicating the probability of a real walk.

Original: After training the model, we have a shadow caster S and a generator G that can produce synthetic graphs.
Revised: After training the model, we have a shadow caster S and a generator G that can produce synthetic graphs.

Original: The shadow caster ﬁrst constructs shadow walks (˜ s 1 , . . . , ˜ s T ) of some user-deﬁned class distribution (a relatively small number of shadow walks, e.g., 10,000).
Revised: The shadow caster ﬁrst constructs shadow walks (˜ s 1 , . . . , ˜ s T ) of some user-deﬁned class distribution (a relatively small number of shadow walks, e.g., 10,000).

Original: The generator then takes (˜ s 1 , . . .
Revised: The generator then takes (˜ s 1 , . . .

Original: , ˜ s T ) and generates a large set of random graph walks (a much larger number of random walks than for training, e.g., 10M).
Revised: , ˜ s T ) and generates a large set of random graph walks (a much larger number of random walks than for training, e.g., 10M).

Original: We construct a score matrix S by counting how often an edge appears in the set of graph walks.
Revised: We construct a score matrix S by counting how often an edge appears in the set of graph walks.

Original: Next, we convert S into a binary adjacency matrix ˆ A by ﬁrst setting s ij =
Revised: Next, we convert S into a binary adjacency matrix ˆ A by ﬁrst setting s ij =

Original: s ji = max { s ij , s ji } to get a symmetric matrix.
Revised: s ji = max { s ij , s ji } to get a symmetric matrix.

Original: Next, we could use simple binarization strategies such as thresholding or choosing top- k entries.
Revised: Next, we could use simple binarization strategies such as thresholding or choosing top- k entries.

Original: However, we follow a probabilistic strategy, introduced in Bojchevski et al.
Revised: However, we follow a probabilistic strategy, introduced in Bojchevski et al.

Original: (2018), that mitigates the issue of leaving out the low-degree nodes and producing singletons because the starting nodes of every walk is random.
Revised: (2018), that mitigates the issue of leaving out the low-degree nodes and producing singletons because the starting nodes of every walk is random.

Original: 2. E XPLAINING G ENERATED G RAPHS
Revised: 2. E XPLAINING G ENERATED G RAPHS

Original: Different from existing approaches, our model takes shadow walks—a series of random walks on the node properties graph—as inputs to the generator, and create graphs with different densities between classes.
Revised: Different from existing approaches, our model takes shadow walks—a series of random walks on the node properties graph—as inputs to the generator, and create graphs with various densities.

Original: To answer questions like: “Why did the model generate such graphs?
Revised: To answer questions like: “Why did the model generate such graphs?

Original: Could we modify it to our desire?”
Revised: Could we modify it to our desire?”

Original: , we generate graphs that are more explainable by controlling these shadow walk inputs.
Revised: , we generate graphs that are more explainable by controlling these shadow walk inputs.

Original: Our goal is to provide insight into how black-box generative models produce graphs.
Revised: Our goal is to provide insight into how black-box generative models produce graphs.

Original: For any desired graph, we ﬁrst construct sequences of node properties based on some user-speciﬁed transition distribution.
Revised: For any desired graph, we ﬁrst construct sequences of node properties based on some user-speciﬁed transition distribution.

Original: These sequences are then injected into the shadow caster S to generate shadow walks (˜ s 1 , . . .
Revised: These sequences are then injected into the shadow caster S to generate shadow walks (˜ s 1 , . . .

Original: , ˜ s T ) that mimic the original shadow.
Revised: , ˜ s T ) that mimic the original shadow.

Original: Next, given a trained S HADOW C AST model Θ and constructed shadow walks (˜ s 1 , . . . , ˜ s T ) , the generator G produces desired graphs ˜ G ’s. Through this process, one canthen control the shadow distributions and study the generated graphs by comparing the results.
Revised: Next, given a trained S HADOW C AST model Θ and the shadow walks (˜ s 1 , . . . , ˜ s T ) , the generator G produces desired graphs ˜ G ’s. Through this process, one can control the shadow distributions and study the generated graphs by comparing the results.

Original:  C ONNECTIONS TO E XISTING W ORK
Revised:  C ONNECTIONS TO E XISTING W ORK

Original: Although many existing works study the generalizability of graph generation methods, explaining generated graphs remains an open question.
Revised: Although many existing works study the generalizability of graph generation methods, explaining generated graphs remains an open question.

Original: From a broader point of view, we can consider the related problems of (1) constructing generative models for graph-structured data and (2) interpreting machine learning models and understanding their results.
Revised: From a broader point of view, we can consider the related problems of ( constructing generative models structured and ( models and understanding their results.

Original: Graph Generation
Revised: Graph Generation

Original: Most existing graph generation models are designed to generate graphs mimicking the structure of observed graphs.
Revised: Most existing graph generation models are designed to generate graphs mimicking the structure of observed graphs.

Original: So far, no generative method that shapes graphs into new desired states have been proposed.
Revised: So far, no generative method that shapes graphs into new desired states have been proposed.

Original: In general, we can group these graph generative models into two main families—those that directly model the graph structure (Cao & Kipf, 2018; Liu et al., 2017;
Revised: In general, we can group these graph generative models into two main families—those that directly model the graph structure (Cao & Kipf, 2018; Liu et al., 2017;

Original: Tavakoli et al., 2017; Zhou et al., 2019; Ma et al., 2018; Simonovsky & Komodakis, 2018) and others that study the graph in the context of node representations (Kipf & Welling, 2016; Wang et al., 2018; Grover et al., 2019; Zou & Lerman, 2019).
Revised: Tavakoli et al., 2017; Zhou et al., 2019; Ma et al., 2018; Simonovsky & Komodakis, 2018) and others that study the graph in the context of node representations (Kipf & Welling, 2016; Wang et al., 2018; Grover et al., 2019; Zou & Lerman, 2019).

Original: While modeling graph structures approximates the distribution of graphs with minimal assumptions about their structure, modeling node embedding estimates the probabilities of each edge’s existence, which effectively models the relational structure of large graphs.
Revised: While modeling graph structures approximates the distribution of graphs with minimal assumptions about their structure, modeling node embedding estimates the probabilities of each edge’s existence, which effectively models the relational structure of large graphs.

Original: However, the methods are designed to mainly generate graphs that match the structural characteristics of any given graph.
Revised: However, the methods are designed to mainly generate graphs that match the structural characteristics of any given graph.

Original: Recently, some works in graph generation have started exploring network structures of various conditions.
Revised: Recently, some works in graph generation have started exploring network structures of various conditions.

Original: These works employ graph-level condition information.
Revised: These works employ graph-level condition information.

Original: In one work, Li et al.
Revised: In one work, Li et al.

Original: (2018) produce some conditional generation results, where the conditions are graph properties such as the number of nodes and edges.
Revised: (2018) produce some conditional generation results, where the conditions are graph properties such as the number of nodes and edges.

Original: Another work, CondGEN (Yang et al., 2019), injects semantics into the graphs by conditioning the model on supplementary contextual information.
Revised: Another work, CondGEN (Yang et al., 2019), injects semantics into the graphs by conditioning the model on supplementary contextual information.

Original: The model mainly considers multiple small graphs, each with an accompanying semantic condition to learn a distribution over graphs.
Revised: The model mainly considers multiple small graphs, each with an accompanying semantic condition to learn a distribution over graphs.

Original: While GraphRNN (You et al., 2018) is not a direct conditional model, it decomposes the generative process into sequences of nodes and edges, which potentially allows for explicit conditioning.
Revised: While GraphRNN (You et al., 2018) is not a direct conditional model, it decomposes the generative process into sequences of nodes and edges, which potentially allows for explicit conditioning.

Original: However, these methods only generate graphs mimicking the observed graphs.
Revised: However, these methods only generate graphs mimicking the observed graphs.

Original: To allow state manipulation and controllable graph generation, our model borrows the concept from NetGAN (Bojchevski et al., 2018), which adapts the standard LSTM to learn a distribution of random walks and exploit sparsity in real-world graphs.
Revised: To allow state manipulation and controllable graph generation, our model borrows the concept from NetGAN (Bojchevski et al., 2018), which adapts the standard LSTM to learn a distribution of random walks and exploit sparsity in real-world graphs.

Original: In contrast to NetGAN, we integrate a condition-based control mechanism to learn a model that generates explainable graphs.
Revised: In contrast to NetGAN, we integrate a condition-based control mechanism to learn a model that generates explainable graphs.

Original: Due to the challenging nature of the problem, to the best of our knowledge, no work has deﬁnitively considered shaping graphs into new desired states
Revised: Due to the challenging nature of the problem, to the best of our knowledge, no work has deﬁnitively considered shaping graphs into new desired states.

Original: Explainable AI Explainable AI studies the task of improving the interpretability of AI systems.
Revised: Explainable AI Explainable AI studies the task of improving the interpretability of AI systems.

Original: While proxy model methods (Huysmans et al., 2011; Augasta & Kathirvalavakumar, 2011; Zilke et al., 2016; Lakkaraju et al., 2017) often resort to learning local approximations of predictions using sets of rules in applying conditions on the prediction, advances in interpretability methods (Adadi & Berrada, 2018; Guidotti et al., 2018; Koh & Liang, 2017) treat black-box models as such and query them for information.
Revised: While proxy model methods (Huysmans et al., 2011; Augasta & Kathirvalavakumar, 2011; Zilke et al., 2016; Lakkaraju et al., 2017) often resort to learning local approximations of predictions using sets of rules in applying conditions on the prediction, advances in interpretability methods (Adadi & Berrada, 2018; Guidotti et al., 2018; Koh & Liang, 2017) treat black-box models as such and query them for information.

Original: Among the many recently developed interpretable models, Graph Neural Network (GNN) models have been studied to explain predictions on graph-structured data via attention mechanisms (Veli ˇ ckovi ´ c et al., 2018; Neil et al., 2018; Xie & Grossman, 2018).
Revised: Among the many recently developed interpretable models, Graph Neural Network (GNN) models have been studied to explain predictions on graph-structured data via attention mechanisms (Veli ˇ ckovi ´ c et al., 2018; Neil et al., 2018; Xie & Grossman, 2018).

Original: These approaches learn important graph structures by predicting and assigning attention values to the edges.
Revised: These approaches learn important graph structures by predicting and assigning attention values to the edges.

Original: The attention values are the same for all nodes in the same structure, which limits the predictive power.
Revised: The attention values are the same for all nodes in the same structure, limiting the predictive power.

Original: Moreover, these models cannot explain predictions by combining node feature information with the graph structure.
Revised: Moreover, these models cannot explain predictions by combining node feature information with the graph structure.

Original: To circumvent the limitations of attention-based GNN models, GNNExplainer (Ying et al., 2019) is proposed to consider both graph structure and node feature information to explain predictions.
Revised: To circumvent the limitations of attention-based GNN models, GNNExplainer (Ying et al., 2019) considers both graph structure and node features to explain predictions.

Original: However, explainable GNN models identify explanations in graph structures and node features, which are suitable for link prediction, node/graph classiﬁcation tasks but not graph generation.
Revised: However, explainable GNN models identify explanations in graph structures and node features, which are suitable for link prediction, node/graph classiﬁcation tasks but not graph generation.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we ﬁrst compare and evaluate our approach with other baseline graph generation methods on three datasets to establish our model’s ability to generate high-quality graphs of complex networks.
Revised: In this section, we ﬁrst compare and evaluate our approach with other baseline graph generation methods on three datasets to establish our model’s ability to generate high-quality graphs of complex networks.

Original: Next, we demonstrate the explainability of S HADOW C AST by controlling the generative process to create graphs according to speciﬁcation.
Revised: Next, we demonstrate the explainability of S HADOW C AST by controlling the generative process to create graphs according to speciﬁcation.

Original: Note that generating graphs mimicking any given graph as closely as possible is not our goal.
Revised: Note that generating graphs mimicking any given graph as closely as possible is not our goal.

Original: Our objective is to introduce a more explainable graph generative approach.
Revised: Our objective is to introduce a more explainable graph generative approach.

Original: Through our experiments, we not only demonstrate that S HADOW C AST exhibits competitive performance in the task of graph generation, but we also show that our model can generate graphs of different density distributions by controlling the 
Revised: Through our experiments, we not only demonstrate that S HADOW C AST exhibits competitive performance in the task of graph generation, but we also show that our model can generate graphs of different density distributions by controlling the shadows.

Original: Datasets We consider three real-world graphs in social and information networks, where each node belongs to one of the ground-truth communities.
Revised: Datasets We consider three real-world graphs in social and information networks, where each node belongs to one of the ground-truth communities.

Original: Two of the datasets are email communication networks EUcore-top ( N = 348, E = 3342, K = 5) and Enron ( N = 154, E = 1843, K = 3).
Revised: Two of the datasets are email communication networks EUcore-top ( N = 348, E = 3342, K = 5) and Enron ( N = 154, E = 1843, K = 3).

Original: The other dataset Cora-ML ( N = 2810, E = 7981, K = 7) is a commonly used subset of a large author citation dataset.
Revised: The other dataset Cora-ML ( N = 2810, E = 7981, K = 7) is a commonly used subset of a large author citation dataset.

Original: We provide the links to datasets used in our experiments (See Appendix for details).
Revised: We provide the links to datasets used in our experiments (see Appendix for details).

Original: We study communication networks: (1) EUcore-top is a network that consists of the top ﬁve largest departments in the EUcore email dataset that was created using anonymized emails from a large European research institution.
Revised: We study communication networks: (1) EUcore-top is a network that consists of the top ﬁve largest departments in the EUcore email dataset that was created using anonymized emails from a large European research institution.

Original: The list of top ﬁve departments is { 14 , 4 , 7 , 21 , 1 } .
Revised: The list of top ﬁve departments is { 14 , 4 , 7 , 21 , 1 } .

Original: (2) Enron is a dataset of the Enron email corpus where nodes are employees labeled according to their department information.
Revised: (2) Enron is a dataset of the Enron email corpus where nodes are employees labeled according to their department information.

Original: The citation network we consider: (3) Cora-ML is a popular benchmark citation dataset.
Revised: The citation network we consider: (3) Cora-ML is a popular benchmark citation dataset.

Original: Nodes labeled according to their paper topic are authors, and edges between them indicate that an author cited another author’s paper.
Revised: Nodes labeled according to their paper topic are authors, and edges between them indicate that an author cited another author’s paper.

Original: Baselines Since controlling the generative process to provide explainable graph generation is a novel task, and no such method is developed, we compare our approachin graph generation against four state-of-the-art baseline methods—GraphRNN (You et al., 2018), GVAE (Simonovsky & Komodakis, 2018), NetGAN (Bojchevski et al., 2018), and CondGEN (Yang et al., 2019).
Revised: Baselines Since controlling the generative process to provide explainable graph generation is a novel task, and no such method is developed, we compare our approach against four current stateof-the-art graph generation baseline methods—GraphRNN (You et al., 2018), GVAE (Simonovsky & Komodakis, 2018), NetGAN (Bojchevski et al., 2018), and CondGEN (Yang et al., 2019).

Original: We randomly select 85% of the edges ingraphs of each dataset for training and use the remaining 15% for validation and testing.
Revised: We randomly select 85% of the edges in each graph for training and use the remaining 15% for validation and testing.

Original: We refer readers to the Appendixin the supplementary material for more details about the model implementation settings, baseline models, datasets, and explainable generated visualizations.
Revised: We refer readers to the Appendix for more details about the model implementation settings, baseline models, datasets, and explainable generated visualizations.

Original: Performance We evaluate S HADOW C AST against existing benchmark generative models (You et al., 2018; Simonovsky & Komodakis, 2018; Bojchevski et al., 2018; Yang et al., 2019) and present the comparison statistics 1 (Table 1).
Revised: Performance We evaluate S HADOW C AST against existing benchmark generative models (You et al., 2018; Simonovsky & Komodakis, 2018; Bojchevski et al., 2018; Yang et al., 2019) and present the comparison statistics 1 (Table 1).

Original: By comparing the statistics of the real graphs and those generated by each method, closer mean values indicate greater resemblance to the original graphs, thus better performance.
Revised: By comparing the statistics of the real graphs and those generated by each method, closer mean values indicate greater resemblance to the original graphs, thus better performance.

Original: In general, baseline methods succeed at replicating the graphs that are directly modeled.
Revised: In general, baseline methods succeed at replicating the graphs that are directly modeled.

Original: Unsurprisingly, GVAE, designed for generating small graphs, performs well in the smaller
Revised: Unsurprisingly, GVAE, designed for generating small graphs, performs well in the smaller

Original: Enron and EUcore-top datasets.
Revised: Enron and EUcore-top datasets.

Original: However, it does not recover statistics of the larger graph Cora-ML well.
Revised: However, it does not recover statistics of the larger graph Cora-ML well.

Original: On the other hand, our model captures all graph properties of the datasets, especially excelling in preserving properties of larger graphs, as shown in its generation of the Cora-ML dataset
Revised: On the other hand [Table] our model captures all graph properties of the datasets, especially excelling in preserving properties of larger graphs, as shown in its generation of the Cora-ML dataset

Original: [Table]
Revised: [Table]

Original: Explaining Generated Graphs
Revised: Explaining Generated Graphs

Original: In addition to recreating graphs that closely match statistics of the input  graphs 
Revised: In addition to recreating graphs that closely match statistics of the input graphs, we demonstrate our model’s ability to generate desired graphs by controlling

Original: [Table]
Revised: [Table]

Original: we demonstrate our model’s ability to generate desired graphs by controlling

Original: [Table]
Revised: [Table]

Original: parameters of the shadows.
Revised: parameters of the shadows.

Original: The controlled generation is a good way to gain insight into how graphs are generated and provide a form of explainability.
Revised: The controlled generation is a good way to gain insight into how graphs are generated and provide a form of explainability.

Original: We inﬂuence the generative process by constructing shadow walks of preferred distribution using shadow caster S .
Revised: We inﬂuence the generative process by constructing shadow walks of preferred distribution using shadow caster S .

Original: First, we create sequences of node ground-truth labels by specifying the parameters of a simple and transparent
Revised: First, we create sequences of node ground-truth labels by specifying the parameters of a simple and transparent

Original: Markov model: (1) initial probability distribution over K labels π
Revised: Markov model: (1) initial probability distribution over K labels π

Original: =
Revised: =

Original: ( π 1 , π 2 , . . .
Revised: ( π 1 , π 2 , . . .

Original: , π K ) , where π i is the probability that the Markov chain will start from label i , and (2) transition probability matrix
Revised: , π K ) , where π i is the probability that the Markov chain will start from label i , and (2) transition probability matrix

Original: A = ( a 11 a 12 . . .
Revised: A = ( a 11 a 12 . . .

Original: a k 1 . . .
Revised: a k 1 . . .

Original: a kk ) , where each a ij represents the probability of moving from label i to label j .
Revised: a kk ) , where each a ij represents the probability of moving from label i to label j .

Original: Next, we input these constructed sequences into shadow caster S , which returns modelgenerated shadow walks.
Revised: Next, we input these constructed sequences into shadow caster S , which returns modelgenerated shadow walks.

Original: Finally, by injecting these designed shadows into our trained generator G , we generate explainable graphs of different structures.
Revised: Finally, by injecting these designed shadows into our trained generator G , we generate explainable graphs of different structures.

Original: [Figure]
Revised: [Figure]

Original: In Figure 3, we show controlled generation examples of the Enron email network, where each employee represented by a node belongs to one of three departments (e.g., Legal, Trading, and Finance ofﬁces in the organization).
Revised: In Figure 3, we show controlled generation examples of the Enron email network, where each employee represented by a node belongs to one of three departments (e.g., Legal, Trading, and Finance ofﬁces in the organization).

Original: Figure 3a is an observed instance of interactions between the departments during normal operations.
Revised: Figure 3a is an observed instance of interactions between the departments during normal operations.

Original: Due to limited observations, network data of some unprecedented, extraordinary situations may be unavailable.
Revised: Due to limited observations, network data of some unprecedented, extraordinary situations may be unavailable.

Original: To simulate such occurrences, we can set the distribution of the Legal (red), Trading (blue), and Finance (green) departments with parameters ( π , A ) to control the generative process.
Revised: To simulate such occurrences, we can set the distribution of the Legal (red), Trading (blue), and Finance (green) departments with parameters ( π , A ) to control the generative process.

Original: Distribution conﬁgurations π
Revised: Distribution conﬁgurations π

Original: = ( π 1 , π 2 , π 3 ) correspond to how likely a sequence of model-generated shadow walks start from a particular department, while the transition probability matrix A
Revised: = ( π 1 , π 2 , π 3 ) correspond to how likely a sequence of model-generated shadow walks start from a particular department, while the transition probability matrix A

Original: = ( a 11 a 12 . . .
Revised: = ( a 11 a 12 . . .

Original: a 31 . . .
Revised: a 31 . . .

Original: a 33 ) determines the probability of moving from one department to another.
Revised: a 33 ) determines the probability of moving from one department to another.

Original: Various conﬁgurations ( π , A ) correspond to different cases such as (Figure 3b) internal communication surge in the legal team during court pre-trial period, (Figure 3c) internal surge in the ﬁnance department during ﬁnancial accounts reporting period, and (Figure 3d) increased outgoing communication between the trading team and the other two departments when purchasing a subsidiary trading ﬁrm.
Revised: Various conﬁgurations ( π , A ) correspond to different cases such as (Figure 3b) internal communication surge in the legal team during court pre-trial period, (Figure 3c) internal surge in the ﬁnance department during ﬁnancial accounts reporting period, and (Figure 3d) increased outgoing communication between the trading team and the other two departments when purchasing a subsidiary trading ﬁrm.

Original: Thus, by specifying these parameters, we can control and explain the structure of the generated graphs (see Appendix for the speciﬁc parameter settings).
Revised: Thus, by specifying these parameters, we can control and explain the structure of the generated graphs (see Appendix for the speciﬁc parameter settings).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we present S HADOW C AST , a novel controllable graph generative model, which generates graphs that are explainable.
Revised: In this work, we present S HADOW C AST , a novel controllable graph generative model, which generates graphs that are explainable.

Original: To the best of our knowledge, this method is the ﬁrst of its kind to address the unique problem of controlling the generative process to explain the structures of generated graphs.
Revised: To the best of our knowledge, this method is the ﬁrst of its kind to address the unique problem of controlling the generative process to explain the structures of generated graphs.

Original: Our model demonstrates how it can leverage graph properties as controls and allow for adjustable parameters to direct the generative process.
Revised: Our model demonstrates how it can leverage graph properties as controls and allow for adjustable parameters to direct the generative process.

Original: By introducing explainability in graph generation, a meaningful problem for a better understanding of generated graph data, we hope to encourage further investigation in this line of work and expand on its applications in different areas.
Revised: By introducing explainability in graph generation, a meaningful problem for a better understanding of generated graph data, we hope to encourage further investigation in this line of work and expand on its applications in different areas.

Original: AA PPENDIX A. I MPLEMENTATION D ETAILS
Revised: A PPENDIX A. I MPLEMENTATION D ETAILS

Original: The S HADOW C AST model incorporates a sequence-to-sequence (Seq2Seq) learner, a generator, and a discriminator.
Revised: The S HADOW C AST model incorporates a sequence-to-sequence (Seq2Seq) learner, a generator, and a discriminator.

Original: Shadow Caster (Seq2Seq)
Revised: Shadow Caster (Seq2Seq)

Original: In the sequence-to-sequence model, we use an LSTM with 10 cells for all three datasets.
Revised: In the sequence-to-sequence model, we use an LSTM with 10 cells for all three datasets.

Original: The input of this LSTM is a batch of shadow walk sequences length n and dimension d, where the batch size is 128, walk length 16, and dimension is set as the number of classes K in each dataset (128 x 16 x d).
Revised: The input of this LSTM is a batch of shadow walk sequences length n and dimension d, where the batch size is 128, walk length 16, and dimension is set as the number of classes K in each dataset (128 x 16 x d).

Original: We select the walk length as 16 because it should capture structures of most graphs of different sizes.
Revised: We select the walk length as 16 because it should capture structures of most graphs of different sizes.

Original: The LSTM hidden layer with 10 memory units should be more than sufﬁcient to learn this problem.
Revised: The LSTM hidden layer with 10 memory units should be more than sufﬁcient to learn this problem.

Original: A dense layer with Softmax activation is connected to the LSTM layer, and the generated output is a batch of sequences with size (128 x 16 x d).
Revised: A dense layer with Softmax activation is connected to the LSTM layer, and the generated output is a batch of sequences with size (128 x 16 x d).

Original: Generator In the generator, we use a conditional LSTM with 50 layers.
Revised: Generator In the generator, we use a conditional LSTM with 50 layers.

Original: We follow a similar architecture to NetGAN to generate sequences of walks on the graph nodes.
Revised: We follow a similar architecture to NetGAN to generate sequences of walks on the graph nodes.

Original: Different from NetGAN, our generator not only initializes the model with Gaussian noise, but it also takes the shadow walks as conditions at each step of the process.
Revised: Different from NetGAN, our generator not only initializes the model with Gaussian noise, but it also takes the shadow walks as conditions at each step of the process.

Original: Interestingly, we notice that the LSTM generator is more sensitive to the input conditions then hyperparameters for performance.
Revised: Interestingly, we notice that the LSTM generator is more sensitive to the input conditions then hyperparameters for performance.

Original: Hence, the set of hyperparameters for the generator is the same for all the datasets.
Revised: Hence, the set of hyperparameters for the generator is the same for all the datasets.

Original: Discriminator Our discriminator is an LSTM with 40 layers.
Revised: Discriminator Our discriminator is an LSTM with 40 layers.

Original: The inputs are sequences of graph nodes concatenated with the respective conditions.
Revised: The inputs are sequences of graph nodes concatenated with the respective conditions.

Original: The discriminator has similar architecture as the shadow caster, where they are LSTM models that take sequences as inputs, expect that the LSTM layer is connected to a ﬁnal dense layer.
Revised: The discriminator has similar architecture as the shadow caster, where they are LSTM models that take sequences as inputs, expect that the LSTM layer is connected to a ﬁnal dense layer.

Original: The output is a single value between 0 and 1, which distinguishes real sequences from generated ones.
Revised: The output is a single value between 0 and 1, which distinguishes real sequences from generated ones.

Original: In the S HADOW C AST training, we use Adam optimizers for all the models.
Revised: In the S HADOW C AST training, we use Adam optimizers for all the models.

Original: The learning rate of the shadow caster sequence-to-sequence training is 0.01, while both the generator and the discriminator 
Revised: The learning rate of the shadow caster sequence-to-sequence training is 0.01, while both the generator and the discriminator use a learning

Original: use a learning rate of 0.0002.
Revised: rate of 0.0002.

Original: A. B ASELINES
Revised: B. B ASELINES

Original: • NetGAN.

Original: We use the ofﬁcial TensorFlow implementation provided by the authors ( https://github . com/danielzuegner/netgan ), following the recommended hyperparameter settings.

Original: We set random walk length to 16, learning rate to 0.0003, generator L2 penalty to 1e-7, and discriminator L2 penalty to 5e-5.

Original: • CondGEN.
Revised: • CondGEN.

Original: We use the ofﬁcial PyTorch implementation ( https://github .
Revised: We use the ofﬁcial PyTorch implementation ( https://github .

Original: com/ KelestZ/CondGen ).
Revised: com/ KelestZ/CondGen ).

Original: However, CondGEN is designed to learn a distribution over multiple small graphs.
Revised: However, CondGEN is designed to learn a distribution over multiple small graphs.

Original: To ensure a fair comparison, we modify it to train on randomly selected 85% of the edges in a graph, validate on the remaining 15% , and generate graphs. 
Revised: To ensure a fair comparison, we modify it to train on randomly selected 85% of the edges in a graph, validate on the remaining 15% , and generate graphs. • GraphRNN.

Revised: We use the ofﬁcial PyTorch implementation ( https://github .

Revised: com/ JiaxuanYou/graph-generation ) of GraphRNN.

Revised: The default hyperparameter settings were used in all our experiments.

Original: • GVAE.
Revised: • GVAE.

Original: To compare with Graph VAE (no public code available), we adapt the reference implementation provided by Yang et al.
Revised: To compare with Graph VAE (no public code available), we adapt the reference implementation provided by Yang et al.

Original: (2019) in their experiments for a single graph
Revised: (2019) in their experiments for a single graph

Original: and use the suggested hyperparameter settings.
Revised: and use the suggested hyperparameter settings.

Revised: • NetGAN.

Revised: We use the ofﬁcial TensorFlow implementation provided by the authors ( https://github . com/danielzuegner/netgan ), following the recommended hyperparameter settings.

Revised: We set random walk length to 16, learning rate to 0.0003, generator L2 penalty to 1e-7, and discriminator L2 penalty to 5e-5.

Original: A. D ATASETS
Revised: C. D ATASETS

Original:  • EUcore-top : An email communication network we created that consists of the top ﬁve largest departments in the EU-core dataset (Leskovec et al., 2007) ( http:// snap .
Revised: Details of the datasets are listed below (see Table 2). • EUcore-top : An email communication network we created that consists of the top ﬁve largest departments in the EU-core dataset (Leskovec et al., 2007).

Revised: For all nodes in the graph, if a person i sends at least one email to person j , then there exists an edge ( i, j ) between the two nodes.

Revised: Each node belongs to exactly one department.

Revised: We sort

Revised: [Table]

Original:  stanford .
Revised: [Table] the data by the intra-department email counts in descending order. Link here: http: //snap . stanford .

Original: edu/data/email-Eu-core . html).
Revised: edu/data/email-Eu-core . html

Original: • Enron : The Enron Corporation email corpus dataset(Perry & Wolfe, 2013) ( https:// github .
Revised: • Enron : It is the Enron Corporation email corpus dataset ( Perry & Wolfe , 2013), where an edge exists between any two nodes as long as they share at least one email.

Original: com/patperry/interaction-proc/tree/master/data/enron ). • Cora-ML :
Revised: Link here: https://github . com/patperry/interaction-proc/ tree/master/data/enron • Cora-ML :

Original: A scientiﬁc publication citation dataset (Bojchevski & G¨unnemann, 2018) consisting of machine learning papers ( https://github . com/abojchevski/ graph2gauss/tree/master/data ).
Revised: A scientiﬁc publication citation dataset (Bojchevski & G ¨ unnemann, 2018) consisting of machine learning papers. Link here: https://github . com/ abojchevski/graph2gauss/tree/master/data

Revised: D. G RAPH G ENERATION S TATISTICS

Revised: [Table]

Original: A. E XPLAINING G ENERATED G RAPHS
Revised: E. E XPLAINING G ENERATED G RAPHS

Original: We provide the Markov model parameters, initial probability distribution π
Revised: We provide the Markov model parameters, initial probability distribution π

Original: = ( π 1 , π 2 , π 3 ) and transition probability matrix A
Revised: = ( π 1 , π 2 , π 3 ) and transition probability matrix A

Original: = ( a 11 a 12 . . .
Revised: = ( a 11 a 12 . . .

Original: a 31 . . .
Revised: a 31 . . .

Original: a 33 ) , used in our experiments.
Revised: a 33 ) , used in our experiments.

Original: Observed

Original: : Enron normal

Original: operations

Original: Generated: Legal (red) internal surge

Original: Initial probability distribution: π = [0 . 9 , 0 . 05 , 0 . 05]
Revised: Initial probability distribution: π = [0 . 9 , 0 . 05 , 0 . 05]

Original: Transition probability matrix:
Revised: Transition probability matrix:

Original: A =
Revised: A =

Original: [Table]
Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: : A = [[0 .

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]