Original: G RAPH E NERGY - BASED M ODEL FOR M OLECULAR G RAPH G ENERATION
Revised: G RAPH E NERGY - BASED M ODEL FOR M OLECULAR G RAPH G ENERATION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present Graph Energy-based Model ( GEM ), an energy-based model for molecular graph generation.
Revised: We present Graph Energy-based Model ( GEM ), an energy-based model for molecular graph generation.

Original: GEM uses dequantization and gradient symmetrization to incorporate generation by stochastic gradient Langevin dynamics for graph representation that is discrete and includes symmetric constraint.
Revised: GEM uses dequantization and gradient symmetrization to incorporate generation by stochastic gradient Langevin dynamics for graph representation that is discrete and includes symmetric constraint.

Original: Experimental results show that GEM can comparably design compounds as other deep generative approaches.
Revised: Experimental results show that GEM can comparably design compounds as other deep generative approaches

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Discovering novel molecules is important but costly and time-consuming.
Revised: Discovering novel molecules is important but costly and time-consuming.

Original: Machine learning-based novel molecule generation approaches are expected to remedy this problem.
Revised: Machine learning-based novel molecule generation approaches are expected to remedy this problem.

Original: Speciﬁcally, recent approaches use deep generative models, such as GANs (De Cao & Kipf (2018); Maziarka et al.
Revised: Speciﬁcally, recent approaches use deep generative models, such as GANs (De Cao & Kipf (2018); Maziarka et al.

Original: VAEs (Simonovsky & Komodakis (2018); Jin et al.
Revised: VAEs (Simonovsky & Komodakis (2018); Jin et al.

Original: and normalizing ﬂows (Kaushalya et al.
Revised: and normalizing ﬂows (Kaushalya et al.

Original: (2019); Zang & Wang (2020)), to produce graph representation of compounds.
Revised: (2019); Zang & Wang (2020)), to produce graph representation of compounds.

Original: This paper introduces Graph Energy-based Model ( GEM ), which uses another generative model, namely energy-based model (EBM) for molecular graph generation.
Revised: This paper introduces Graph Energy-based Model ( GEM ), which uses another generative model, namely energy-based model (EBM) for molecular graph generation.

Original: We empirically demonstrate that GEM can generate novel molecules as other approaches.
Revised: We empirically demonstrate that GEM can generate novel molecules as other approaches.

Original: Additionally, its generation in the input space enables preserving speciﬁed substructures, which distinguishes GEM from different deep generative approaches.
Revised: Additionally, its generation in the input space enables preserving speciﬁed substructures, which distinguishes GEM from different deep generative approaches.

Original:  [Equation] [Figure] [Equation] [Figure]
Revised:  G RAPH E NERGY - BASED M ODELS

Original: 2. N OTATIONS
Revised: 2. N OTATIONS

Original: A molecular graph G can be represented as an undirected graph depicted by a pair of tensors: a feature tensor X ∈ { 0 , 1 } N × # M and an adjacency tensor A ∈ { 0 , 1 } N × N × # B .
Revised: A molecular graph G can be represented as an undirected graph depicted by a pair of tensors: a feature tensor X ∈ { 0 , 1 } N × # M and an adjacency tensor A ∈ { 0 , 1 } N × N × # B .

Original: The feature tensor X represents atoms in the molecule, and the adjacency tensor represents bonds among them.
Revised: The feature tensor X represents atoms in the molecule, and the adjacency tensor represents bonds among them.

Original: N is the maximum number of atoms in molecules in a dataset, M is a set of considered atoms, e.g ., M = { C , N , O , F , virtual node } , and B is the set of bond types, i.e .,
Revised: N is the maximum number of atoms in molecules in a dataset, M is a set of considered atoms, e.g ., M = { C , N , O , F , virtual node } , and B is the set of bond types, i.e .,

Original: B = { single , double , triple , virtual bond } .
Revised: B = { single , double , triple , virtual bond } .

Original: “virtual node” and “virtual bond” are used for padding in case the number of atoms in a given molecule is smaller than N .
Revised: “virtual node” and “virtual bond” are used for padding in case the number of atoms in a given molecule is smaller than N .

Original: For each triplet of ( N, M , B ) , there is a set of valid molecular graphs G = G ( N, M , B ) .
Revised: For each triplet of ( N, M , B ) , there is a set of valid molecular graphs G = G ( N, M , B ) .

Original: Validity includes the symmetry of adjacency tensor slices: A : , : ,b is a symmetric matrix for b = 1 , 2 , . .
Revised: Validity includes the symmetry of adjacency tensor slices: A : , : ,b is a symmetric matrix for b = 1 , 2 , . .

Original: . , # B .
Revised: . , # B .

Original: Practically, we use datasets D ⊂ G .
Revised: Practically, we use datasets D ⊂ G .

Original: 2. G ENERATING G RAPHS BY EBM S
Revised: 2. G ENERATING G RAPHS BY EBM S

Original: We propose to generate a novel molecule by using an energy function E θ : G → R , parameterized by a real vector θ .
Revised: We propose to generate a novel molecule by using an energy function E θ : G → R , parameterized by a real vector θ .

Original: Speciﬁcally, we use a graph neural network to represent this parameterized function.
Revised: Speciﬁcally, we use a graph neural network to represent this parameterized function.

Original: The energy function is expected to assign smaller values to valid molecules and higher values to invalid ones.
Revised: The energy function is expected to assign smaller values to valid molecules and higher values to invalid ones.

Original: This energy function determines a Boltzmann-Gibbs distribution p θ ( G )= [Equation] exp
Revised: This energy function determines a Boltzmann-Gibbs distribution p θ ( G )

Original: [Equation]
Revised: [Equation]

Original: E
Revised: G

Original: [Equation]
Revised: [Equation]

Original: G )) , from which molecules are expected to be sampled with a high probability.
Revised: ) , from which molecules are expected to be sampled with a high probability.

Original: If graphs are continuous, we can sample graphs from this distribution by using stochastic gradient
Revised: If graphs are continuous, we can sample graphs from this distribution by using stochastic gradient

Original: Langevin dynamics (SGLD, Welling & Teh (2011)):
Revised: Langevin dynamics (SGLD, Welling & Teh (2011)):

Original: [Equation]
Revised: [Equation]

Original: where α t ∈ R + is a step size, g X = ∇ X E θ and g A = ∇ A E θ are score functions, and
Revised: where α t ∈ R + is a step size, g X = ∇ X E θ and g A = ∇ A E θ are score functions, and

Original: (cid:15) X and (cid:15)
Revised: (cid:15) X and (cid:15)

Original: A are standard normals.
Revised: A are standard normals.

Original: This generation (Equation (1)) can also be achieved by directly estimating g X and g A as (Niu et al.
Revised: This generation (Equation (1)) can also be achieved by directly estimating g X and g A as (Niu et al.

Original: X (0) and A (0) are sampled from a uniform distribution on [0 , 1] .
Revised: X (0) and A (0) are sampled from a uniform distribution on [0 , 1] .

Original: The distribution of G ( ∞ ) =
Revised: The distribution of G ( ∞ ) =

Original: ( X ( ∞ ) , A ( ∞ ) ) is asymptotically equal to p θ ( G ) , and we assume that this property can be approximated with ﬁnite steps with a small constant state size, i.e ., α t
Revised: ( X ( ∞ ) , A ( ∞ ) ) is asymptotically equal to p θ ( G ) , and we assume that this property can be approximated with ﬁnite steps with a small constant state size, i.e ., α t

Original: = α , following the literature.
Revised: = α , following the literature.

Original: Actually, simply applying Equation (1) does not work in our case, because they do not consider the following requirements: 1.
Revised: Actually, simply applying Equation (1) does not work in our case, because they do not consider the following requirements: 1.

Original: X and A are discrete, and 2. slices of A is symmetric.
Revised: X and A are discrete, and 2. slices of A is symmetric.

Original: To ﬁx the ﬁrst issue, we relax the domains of X and A to be (0 , 1) N × # M
Revised: To ﬁx the ﬁrst issue, we relax the domains of X and A to be (0 , 1) N × # M

Original: and (0 , 1) N × N × # B .
Revised: and (0 , 1) N × N × # B .

Original: For discrete tensors from datasets, we modify them by using dequantization and applying softmax function along the last axes.
Revised: For discrete tensors from datasets, we modify them by using dequantization and applying softmax function along the last axes.

Original: Dequantization is a technique used in Kaushalya et al.
Revised: Dequantization is a technique used in Kaushalya et al.

Original: (2019), which adds random values to the tensor elements
Revised: (2019), which adds random values to the tensor elements

Original: X ← X
Revised: X ← X

Original: + c U X , A ← A + c U A , where c ∈ (0 , 1) is a scaling parameter, and
Revised: + c U X , A ← A + c U A , where c ∈ (0 , 1) is a scaling parameter, and

Original: U X , U A are uniform noise on (0 , 1) .
Revised: U X , U A are uniform noise on (0 , 1) .

Original: We set c = 0 .
Revised: We set c = 0 .

Original: 9 in the experiments.
Revised: 9 in the experiments.

Original: To avoid sampled adjacency tensors being asymmetric, we sample A (0) and (cid:15)
Revised: To avoid sampled adjacency tensors being asymmetric, we sample A (0) and (cid:15)

Original: A from symmetric distributions, where ( A (0) )
Revised: A from symmetric distributions, where ( A (0) )

Original: i,j,b =
Revised: i,j,b =

Original: ( A (0) )
Revised: ( A (0) )

Original: j,i,b , ( (cid:15) A ) i,j,b
Revised: j,i,b , ( (cid:15) A ) i,j,b

Original: = ( (cid:15) A )
Revised: = ( (cid:15) A )

Original: j,i,b , for i, j ∈ { 1 , 2 , . .
Revised: j,i,b , for i, j ∈ { 1 , 2 , . .

Original: . , N } and b ∈ { 1 , 2 . . .
Revised: . , N } and b ∈ { 1 , 2 . . .

Original: , # B} .
Revised: , # B} .

Original: Additionally, the score function g A needs to be symmetric, which we will describe in the next section.
Revised: Additionally, the score function g A needs to be symmetric, which we will describe in the next section.

Original: 2. S YMMETRIZE G RADIENT OF A DJACENCY T ENSOR
Revised: 2. S YMMETRIZE G RADIENT OF A DJACENCY T ENSOR

Original: We use a neural network based on Relational GCN (RGCN, (Schlichtkrull et al.
Revised: We use a neural network based on Relational GCN (RGCN, (Schlichtkrull et al.

Original: as an energy function.
Revised: as an energy function.

Original: RGCN is a graph convolutional neural network for graphs with multiple edge types.
Revised: RGCN is a graph convolutional neural network for graphs with multiple edge types.

Original: For each graph G =
Revised: For each graph G =

Original: ( X , A ) , the l th RGCN layer processes node representation H l ∈ R N × C as
Revised: ( X , A ) , the l th RGCN layer processes node representation H l ∈ R N × C as

Original: [Equation]
Revised: [Equation]

Original: where H 0 = X , W (0) l , W ( b ) l ∈ R C
Revised: where H 0 = X , W (0) l , W ( b ) l ∈ R C

Original: × D are learnable parameters, σ is a nonlinear activation function, and C, D are input and output feature dimensions.
Revised: × D are learnable parameters, σ is a nonlinear activation function, and C, D are input and output feature dimensions.

Original: After several RGCN layers, a graph-level representation is obtained by the aggregation of (Li et al.
Revised: After several RGCN layers, a graph-level representation is obtained by the aggregation of (Li et al.

Original: This representation is transformed into a scalar value E θ ( G ) by a multi layer perceptron.
Revised: This representation is transformed into a scalar value E θ ( G ) by a multi layer perceptron.

Original: Crucially, with this energy function, the score function g A is asymmetric.
Revised: Crucially, with this energy function, the score function g A is asymmetric.

Original: Indeed, by focusing on the ﬁrst layer of RGCN layers and ignoring the nonlinear activation for simplicity, we obtain a Jacobian tensor of
Revised: Indeed, by focusing on the ﬁrst layer of RGCN layers and ignoring the nonlinear activation for simplicity, we obtain a Jacobian tensor of

Original: [Equation]
Revised: [Equation]

Original: 1) b , where J ( i,j ) denotes a single entry matrix of 1 at ( i, j ) and 0 elsewhere (Petersen & Pedersen (2006)).
Revised: [Equation] , where J ( i,j ) denotes a single entry matrix of 1 at ( i, j ) and 0 elsewhere (Petersen & Pedersen (2006)).

Original: This gradient is not symmetric for each b .
Revised: This gradient is not symmetric for each b .

Original: To remedy this, we modify Equation (2) as
Revised: To remedy this, we modify Equation (2) as

Original: [Equation]
Revised: [Equation]

Original: Though this modiﬁcation does not change the output because each A : , : ,b is symmetric by deﬁnition, now the Jacobian tensor is also symmetrized as
Revised: Though this modiﬁcation does not change the output because each A : , : ,b is symmetric by deﬁnition, now the Jacobian tensor is also symmetrized as

Original: [Equation]
Revised: [Equation]

Original: , from which we can deduce
Revised: , from which we can deduce

Revised: [Equation]

Revised: ∂

Original: [Equation]
Revised: [Equation]

Original: A ) j,i,b , the symmetry of the score function g A .
Revised: A ) j,i,b , the symmetry of the score function g A .

Original: Practically, the modiﬁcation of Equation (3) can be separately done before the forward pass of the model, which means the actual modiﬁcation to the off-the-shelf models is minimum.
Revised: Practically, the modiﬁcation of Equation (3) can be separately done before the forward pass of the model, which means the actual modiﬁcation to the off-the-shelf models is minimum.

Original: In the experiments, we use the abovementioned RGCN variant, which is also used in other graph-based molecular generation methods (De Cao & Kipf (2018); Kaushalya et al.
Revised: In the experiments, we use the abovementioned RGCN variant, which is also used in other graph-based molecular generation methods (De Cao & Kipf (2018); Kaushalya et al.

Original: 2. T RAINING OF GEM
Revised: 2. T RAINING OF GEM

Original: To optimize the energy function E θ , we can use stochastic gradient of ∇ θ E D [log p θ ( G )]
Revised: To optimize the energy function E θ , we can use stochastic gradient of ∇ θ E D [log p θ ( G )]

Original: =
Revised: =

Original: E
Revised: E

Original: D [ ∇ θ E θ ( G )] − E p θ ( G (cid:48) )
Revised: D [ ∇ θ E θ ( G )] − E p θ ( G (cid:48) )

Original: [ ∇ θ E θ ( G (cid:48) )] .
Revised: [ ∇ θ E θ ( G (cid:48) )] .

Original: At the LHS’s second term, samples from the model
Revised: At the LHS’s second term, samples from the model

Original: G (cid:48) ∼ p θ ( G (cid:48) ) are used.
Revised: G (cid:48) ∼ p θ ( G (cid:48) ) are used.

Original: As discussed in Section 2.2, we use a ﬁnite step of SGLD to approximate this sampling, resulting in diverged samples from the model distribution.
Revised: As discussed in Section 2.2, we use a ﬁnite step of SGLD to approximate this sampling, resulting in diverged samples from the model distribution.

Original: To remedy this problem, we use the persistent contrastive divergence (PCD, Tieleman (2008)), which reuses the past generated samples.
Revised: To remedy this problem, we use the persistent contrastive divergence (PCD, Tieleman (2008)), which reuses the past generated samples.

Original: Additionally, we found that penalizing { E θ ( G ) } 2 improves empirical performance.
Revised: Additionally, we penalized { E θ ( G ) } 2 (Du & Mordatch, 2019).

Original: 2. G ENERATION BY GEM
Revised: 2. G ENERATION BY GEM

Original: GEM generates molecular graphs using SGLD (Equation (1)), which adds noise to graph representation, and thus, sometimes collapses its validity.
Revised: GEM generates molecular graphs using SGLD (Equation (1)), which adds noise to graph representation, and thus, sometimes collapses its validity.

Original: To remedy this issue, we apply validity correction (Zang & Wang (2020)) to feature and adjacency tensors after generation steps.
Revised: To remedy this issue, we apply validity correction (Zang & Wang (2020)) to feature and adjacency tensors after generation steps.

Original: One of the most appealing ability of GEM is substructure preserving generation.
Revised: One of the most appealing ability of GEM is substructure preserving generation.

Original: Because GEM samples molecular graphs in the input space by SGLD, this ability is achieved by updating parts of graph representation (see also Figure 2 and Appendix A).
Revised: Because GEM samples molecular graphs in the input space by SGLD, this ability is achieved by updating parts of graph representation (see also Figure 2 and Appendix A).

Original:  E XPERIMENTS 3. E XPERIMENTAL S ETTINGS
Revised:  E XPERIMENTS 3. E XPERIMENTAL S ETTINGS

Original: We used QM9 (Wu et al.
Revised: We used QM9 (Wu et al.

Original: and ZINC-250k (Irwin & Shoichet (2015)) as datasets D .
Revised: and ZINC-250k (Irwin & Shoichet (2015)) as datasets D .

Original: QM and ZINC-250k contain 1 .
Revised: QM and ZINC-250k contain 1 .

Original: 3 × 10 5 and 2 .
Revised: 3 × 10 5 and 2 .

Original: 5 × 10 5 molecules, respectively.
Revised: 5 × 10 5 molecules, respectively.

Original: Following the preprocessing protocols in Kaushalya et al.
Revised: Following the preprocessing protocols in Kaushalya et al.

Original: (2019), we kekulize each molecule in each dataset and ignore hydrogens as the SMILES format.
Revised: (2019), we kekulize each molecule in each dataset and ignore hydrogens as the SMILES format.

Original: As a result, the maximum number of atoms in a molecule
Revised: As a result, the maximum number of atoms in a molecule

Original: N is 9 for QM9 9 and 38 for ZINC-250k .
Revised: N is 9 for QM9 9 and 38 for ZINC-250k .

Original: The number of atom types # M including the virtual node is 5 for QM9 and 10 for ZINC-250k .
Revised: The number of atom types # M including the virtual node is 5 for QM9 and 10 for ZINC-250k .

Original: The number of bond types # B is 4, namely
Revised: The number of bond types # B is 4, namely

Original: B = { single , double , triple , virtual bond } , for both datasets.
Revised: B = { single , double , triple , virtual bond } , for both datasets.

Original: We also followed the data split of
Revised: We also followed the data split of

Original: Kaushalya et al.
Revised: Kaushalya et al.

Original: Each input feature tensor is embedded in 16-dimensional space and processed by a two-layer RGCN of 128 hidden dimensions.
Revised: Each input feature tensor is embedded in 16-dimensional space and processed by a two-layer RGCN of 128 hidden dimensions.

Original: Its output is aggregated in a 256-dimensional space and converted toscalar energy byan MLP of (1024 , 512) hidden units.
Revised: Its output is aggregated in a 256-dimensional space and converted to energy by units.

Original: The hyperbolic tangent function isused as an activation function, and the sigmoid function ς( x ) =
Revised: The hyperbolic tangent function is as an activation function, and the sigmoid function ς x ) =

Original: { 1+exp( − x ) } − 1 is applied to the ﬁnal output that restricts the range to [0 , 1] .
Revised: { 1+exp( − x ) } − 1 is applied to the ﬁnal output that restricts the range to [0 , 1] .

Original: Please refer to Appendix B for details of training and generation.
Revised: Please refer to Appendix B for details of training and generation.

Original: 3. R ESULTS
Revised: 3. R ESULTS

Original: We present validity, novelty, and uniqueness of molecular graphs generated from 1,000 random initial states in Table 1 compared with baselines of MoFlow (Zang & Wang (2020)), GraphNVP (Kaushalya et al.
Revised: We present validity, novelty, and uniqueness of molecular graphs generated from 1,000 random initial states in Table 1 compared with baselines of MoFlow (Zang & Wang (2020)), GraphNVP (Kaushalya et al.

Original: , MolGAN (De Cao & Kipf (2018)), and RVAE (Ma et al.
Revised: , MolGAN (De Cao & Kipf (2018)), and RVAE (Ma et al.

Original: GEM shows comparable performance with other methods using other deep generative methods on both datasets.
Revised: GEM shows comparable performance with other methods using other deep generative methods on both datasets.

Original: Additionally, samples of substructure preserving generation are presented in Figure 2.
Revised: Additionally, samples of substructure preserving generation are presented in Figure 2.

Original: GEM can exactly
Revised: GEM can exactly

Original: ﬁx speciﬁed substructures, which distinguishes our approach from different generative methods.
Revised: ﬁx speciﬁed substructures, which distinguishes our approach from different generative methods.

Original: Other samples are presented in Figure 3 of Appendix A.
Revised: Other samples are presented in Figure 3 of Appendix A.

Original: [Table]
Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper
Revised: In this paper

Original: [Table]
Revised: [Table]

Original: we have proposed GEM [Table] an EBM for molecular graphs.
Revised: we have proposed GEM , an EBM for molecular graphs.

Original: Dequantization and gradient symmetrization have been introduced to generate discrete and symmetric representations of graphs in continuous space.
Revised: Dequantization and gradient symmetrization have been introduced to generate discrete and symmetric representations of graphs in continuous space.

Original: We empirically demonstrate the effectiveness of GEM and its unique ability, substructure preserving generation.
Revised: We empirically demonstrate the effectiveness of GEM and its unique ability, substructure preserving generation.

Original: We hope energy-based molecular generation, including GEM , opens a new direction of de novo design.
Revised: We hope energy-based molecular generation, including GEM , opens a new direction of de novo design.

Original: A S UBSTRUCTURE P RESERVING G ENERATION
Revised: A S UBSTRUCTURE P RESERVING G ENERATION

Original: To ﬁx substructures, we apply masks to both feature and adjacency tensors.
Revised: To ﬁx substructures, we apply masks to both feature and adjacency tensors.

Original: Suppose the number of atoms in a given substructure is S < N , where N is the maximum number of atoms in molecules of a dataset.
Revised: Suppose the number of atoms in a given substructure is S < N , where N is the maximum number of atoms in molecules of a dataset.

Original: Because GEM is permutation invariant to an input representation, we can re-index atoms in the substructure to 1 , 2 , . . .
Revised: Because GEM is permutation invariant to an input representation, we can re-index atoms in the substructure to 1 , 2 , . . .

Original: , S such that the S th atom to be connected with the rest part, without loss of generality.
Revised: , S such that the S th atom to be connected with the rest part, without loss of generality.

Original: Then, we use a mask to update only a part of the feature tensor corresponding to S + 1 , S + 2 , . . .
Revised: Then, we use a mask to update only a part of the feature tensor corresponding to S + 1 , S + 2 , . . .

Original: , N th atoms and ﬁx the atoms in the substructure.
Revised: , N th atoms and ﬁx the atoms in the substructure.

Original: Similarly, we only update connections among S, S + 1 , . . .
Revised: Similarly, we only update connections among S, S + 1 , . . .

Original: , N th atoms and ﬁx the connections among the rests by masking the adjacency tensor.
Revised: , N th atoms and ﬁx the connections among the rests by masking the adjacency tensor.

Original: This masking can be extended to appending the rest parts to multiple atoms.
Revised: This masking can be extended to appending the rest parts to multiple atoms.

Original: B E XPERIMENTS B. D ETAILS OF TRAINING AND GENERATION
Revised: B E XPERIMENTS B. D ETAILS OF TRAINING AND GENERATION

Original: We used PyTorch v1.7 (Paszke et al.
Revised: We used PyTorch v1.7 (Paszke et al.

Original: for model implementation, chainer-chemistry v0.7for data preprocessing, and RDKit v2020.09 2 for handling molecule information.
Revised: for model implementation, chainer-chemistry v0.7for data preprocessing, and RDKit v2020.09 2 for handling molecule information.

Original: We trained GEM using Adam (Kingma & Ba (2015)) with a learning rate of 1 . 

Revised: We trained GEM using Adam (Kingma & Ba (2015)) with a learning rate of 1 .

Original: × 10 − 4 for 30 epochs.
Revised: × 10 − 4 for 30 epochs.

Original: For SGLD, we set a step size α to 1 . 

Revised: For SGLD, we set a step size α to 1 .

Original: × 10 − 4 and the number of steps to 40 .
Revised: × 10 − 4 and the number of steps to 40 .

Original: Following Grathwohl et al.
Revised: Following Grathwohl et al.

Original: Du et al.
Revised: Du et al.

Original: (2020), we set the buffer size of PCD to 10 4 and the reinitialization probability ρ to 5 . 

Revised: (2020), we set the buffer size of PCD to 10 4 and the reinitialization probability ρ to 5 .

Original: × 10 − 2 , and reduced the effect of additive noise by multiplying 0 .
Revised: × 10 − 2 , and reduced the effect of additive noise by multiplying 0 .

Original: 1 to the standard deviation as common practice.
Revised: 1 to the standard deviation as common practice.

Original: For SGLD, we used an exponential moving average of the model with a decay rate of 1 . 

Revised: For SGLD, we used an exponential moving average of the model with a decay rate of 1 .

Original: × 10 − 3 for the stability.
Revised: × 10 − 3 for the stability.

Original: To generate molecular graphs, we used SGLD of step size of 1 . 

Revised: To generate molecular graphs, we used SGLD of step size of 1 .

Original: × 10 − 1 for QM9 and 1 . 

Revised: × 10 − 1 for QM9 and 1 .

Original: × 10 − 2 for
Revised: × 10 − 2 for

Original: ZINC-250k , and the number of steps of 10 3 .
Revised: ZINC-250k , and the number of steps of 10 3 .

Original: Adding noise in Equation (1) sometimes turns once generated valid molecular graphs into invalid ones.
Revised: Adding noise in Equation (1) sometimes turns once generated valid molecular graphs into invalid ones.

Original: Therefore, we record all valid graphs generated at each step.
Revised: Therefore, we record all valid graphs generated at each step.

Original: We discarded the graphs generated during the ﬁrst 100 steps to reduce the effects of initial states.
Revised: We discarded the graphs generated during the ﬁrst 100 steps to reduce the effects of initial states.