GraphEBM: Towards Permutation Invariant and Multi-Objective Molecular Graph Generation
Authors: Meng Liu
Keqiang Yan
Bora Oztekin
Shuiwang Ji
Venue: ICLR 2022 Submitted (2022)
Keywords: molecular graph generation
energy-based models
permutation invariance
multi-objective

Abstract: Although significant progress has been made in molecular graph generation recently, permutation invariance and multi-objective generation remain to be important but challenging goals to achieve. In this work, we propose GraphEBM, a molecular graph generation method via energy-based models (EBMs), as an exploratory work to perform permutation invariant and multi-objective molecule generation. Particularly, thanks to the flexibility of EBMs and our parameterized permutation-invariant energy function, our GraphEBM can define a permutation invariant distribution over molecular graphs. We learn the energy function by contrastive divergence and generate samples by Langevin dynamics. In addition, to generate molecules with a specific desirable property, we propose a simple yet effective learning strategy, which pushes down energies with flexible degrees according to the properties of corresponding molecules. Further, we explore to use our GraphEBM for generating molecules towards multiple objectives via compositional generation, which is practically desired in drug discovery. We conduct comprehensive experiments on random, single-objective, and multi-objective molecule generation tasks. The results demonstrate our method is effective.

Under review as a conference paper at ICLR 2022
GRAPHEBM: TOWARDS PERMUTATION INVARIANT AND MULTI-OBIECTIVE MOLECULAR GRAPH GENER- ATION
Anonymous authors Paper under double-blind review
ABSTRACT
Although significant progress has been made in molecular graph generation cently, permutation invariance and multi-objective generation remain to be im- portant but challenging goals achieve. In this work, we propose GraphEBM, a molecular graph generation method via energy-based models (EBMs), as an exploratory work perform permutation invariant and multi-objective molecule generation. Particularly, thanks to the flexibility of EBMs and our parameterized permutation-invann energy function, our GraphEBM can define permutation invariant distribution over molecular graphs. We learn the energy function by con- trastive divergence and generate samples by Langevin dynamics. addition, generate molecules with a specific desirable property, we propose simple yet effective learning strategy, which pushes down energies with flexible degrees ac- cording to the properties of corresponding molecules. Further, we explore to use our GraphEBM for generating molecules towards multiple objectives via com- positional generation, which is practically desired in drug discovery. We con- duct comprehensive experiments on random, single-objective, and multi-objective molecule generation tasks. The results demonstrate our method is effective.
INTRODUCTION
A fundamental problem in drug discovery and material science is to find novel molecules with desir- able properties. One popular method to search in the chemical space based on molecular property prediction (Gilmer et al. [017] [Wu et al. 2018 Yang et al.) 2019 Stokes et al. 22200] Wang al. 2020). addition, molecular graph generation provides an alternative and promising way for this problem by directly generating desirable molecules, thus circumventing the expensive search of the chemical space. As reviewed in Section 2.1 existing approaches have achieved promising success by generating molecular graphs based on various generative methods, including variational autoen- coders (VAEs) (Kingma & Welling, (013)) generative adversarial networks (GANs) (Goodfellow al., 2014), flow models (Dinh et al. 2014 Rezende Mohamed] 2013) and recurrent neural networks (RNNs). Despite these intensive efforts, we still face the following significant challenges.
(i) Data distribution is permutation invariant. Permutation invariance is known as an intrinsic and desirable inductive bias for graph data. we permute the node order of given molecular graph, the resulting graph still corresponds to the same molecule. In other words, the underlying data distribution of molecular graphs is invariant to permutations. Thus, an ideal generative model, which aims to capture the underlying data distribution, should associate the same probability to various permutations of the same molecular graph (Niu et al., (220)). However, as analyzed in Section 2.1 most existing methods fail to preserve the intrinsic property of permutation invariance during density modeling.
(ii) Multi-objective generation in demand but challenging. An ultimate goal of drug discovery is to obtain molecules that have multiple properties simultaneously (Jin et al. 2020b). Unfortunately, we usually do not have a training set of enough molecules that satisfy multiple desired properties simultaneously. In other words, if we have such dataset, there is no need to generate more molecules since we already have enough desired molecules in the dataset. Hence, it greatly desired but challenging generate molecules with multiple objectives.

---

Under review as a conference paper at ICLR 2022
We are the first to observe that developing molecular graph generative model based energy- based models (EBMs) (LeCun et al., 2006) has the potential to perform permutation invariant and multi-objective molecular graph generation. In this study, we propose GraphEBM to explore per- mutation invariant and multi-objective molecular graph generation. Since we use the flexible EBM framework and permutation invariant energy function, our GraphEBM can provide permutation invariant distribution over molecular graphs, thus preserving the intrinsic permutation invariance property during density modeling. We apply contrastive divergence (Hinton (202) to learn the en- ergy function and generate samples from it via Langevin dynamics (Welling & Teh 2011). To our knowledge, our GraphEBM is the first energy-based model that can generate attributed molecular graphs. To achieve multi-objective generation, we first need to generate molecules with a single objective property. Particularly, we propose novel, simple, and effective strategy to train our GraphEBM for single-objective generation by pushing down energies with flexible degrees accord- ing to the property values of corresponding molecules. Further, we propose that GraphEBM can generate molecules with multiple objectives in a compositional manner. To be specific, we first learn multiple single-objective EBMs since the training set for a single objective is usually avail- able. Afterwards, we can perform multi-objective generation via the compositionality endowed by EBMs. This provides a new and promising way for multi-objective molecule generation, which is significantly helpful for drug discovery.
Contributions. (i) We propose the first EBM that is capable of generating molecular graphs with preserving permutation invariance property. (ii) To achieve single-objective generation with GraphEBM, we propose push down energies with flexible degrees according to the correspond- ing property values, which is simple and effective. (ii)) Based on singlc-objective generation, we demonstrate that our GraphEBM can perform multi-objective molecular graph generation compo- sitionally, which provides a novel and promising way for multi-objective molecule generation. (iv) We empirically demonstrate that our approach effective via random, single-objective, and multi- objective molecule generation.
2 PRELIMINARY AND RELATED WORK
2.1 MOLECULAR GRAPH GENERATION
Since molecules can be represented as SMILES strings (Weininger 1988). early studies generate molecules based on SMILES strings, such as CVAE
Bommaz-Bombarelli et al. 2018), GVAE (Kus-) ner et al. 2017), and SD-VAE (Dai al. 2018). Recent studies mostly represent and generate molecules as graphs (Simonovsky & Komodakis 2018 De Cao & Kipi 2018, Madhawa et al.) 2019). We can categorize existing molecular graph generation methods based on the underlying generative methods or the generation processes. Current molecular graph generation approaches can be grouped into four categories according to their underlying generative models, i.e., VAEs, GANs, flow models, and RNNs. They also be classified into two primary types based on their generation processes; those are, sequential generation and one-shot generation. The sequential pro- cess generates nodes and edges in sequential order by adding nodes and edges one by one. The one-shot process generates all nodes and edges at one time.
To facilitate comparison, we summarize existing methods in Table We can observe that most of them fail to satisfy intrinsic property of graphs; that is, permutation invariance. Specifically, generative model should ideally yield the same likelihood for different permutations of the same graph, since the underlying data distribution is permutation invariant. However, the sequential gen- eration approaches model graphs by choosing a specific order of nodes, thus failing to preserve per- mutation invariance. Among the one-shot methods, Bresson & Laurent (2019) also use the specific node order given by the SMILES representation. Graph VAE and RVAE perform an approximate and expensive graph matching to train the VAE model, and they cannot achieve exact permutation invariance. MolGAN circumvents this issue by using a likelihood-free method. The recent one-shot flow methods have the potential to satisfy this property. However, GraphNVP, GRF, and MoFlow cannot preserve this property since the masking strategies in the coupling layers sensitive to node order. An exception is GraphCNF, which achieves permutation invariance by assigning likelihood independent of node ordering via categorical normalizing flows. Furthermore, the existing methods, shown in Table cannot easily achieve multi-objective generation for practical use. Compared with these existing works, we propose to develop EBMs for molecular graph generation. We note that
2

---

Under review as a conference paper at ICLR 2022
Table Summary and comparison of existing molecular graph generation methods.
Method
Generative method
Generation pricess Sequentia!
Permutation
Multt-ohjectiv generation
GAN Flow
GraphVAE Samoporsky & Komodakis. 2018
DespGiMii Le 2013
CGVAE
23155
K.p
RVAE Mi Ri 2013
X
GCPN Tre
2018
7
X
MolecalaeNNN Popora et
2019
GraphNVP Madniii 2019
Bresson and
Laurren 2019
Honda
GraphFF Mil ai 20r0
HierVAF eirera A120
MoHlow
GraphCNF
Liss & Uaee 2020
GraphiF [Lar all 2021
GraphEBM
EBMs have unique advantages for molecular graph generation, including preserving permutation invariance and enabling multi-objective generation via compositionality.
2.2 ENERGY-BASER MODELS
Modeling variables by defining an unnormalized probability density has been explored for decades (Hopfield 1982, Ackley et al. 1985 Cipra 1987; Dayan al. 1995 Zhu al. [998
Hinton 2012). Such methods are known as energy-based models (EBMs) (LeCun et al.) 2006) in
machine learning. Given data point æ, let (x) be the corresponding energy, where 9 denotes the learnable parameters of the energy function. Then, the energy function defines distribution as
(x)(x
Z(6) x e Ex(x)
(1)
where Z(0) f e- (s)) is the normalization constant and usually intractable. Currently, EBMs
have been used as generative models in multiple domains, including images (Xie et al. 2015 2016
Du & Mordatch 2019 Du al. 2020a b). videos (Xie al. 2017). texts (Deng al. 2020). 3D
objects (Xie et al. 2018). 8), point sets (Xie et al. 2020). and proteins (Du et al. 2020c).
To date, EBMs have rarely been studied in the graph domain. Liu et al. 2020 attempts to generate graph structures by building EBMs based graph neural networks. However, it can only generate graph structures, i.e., binary adjacency matrices. It is not straightforward to extend for attributed graphs since the number of data dimensions of attributed graphs much larger than graphs without attributes, thereby leading to difficulty for discrete sampling method used in Liu et al. (2020) to draw samples from model distribution. In addition, they do not consider objective-directed graph gener- ation. We note that there is concurrent work (Hataya et al.) (02T) with ours, which also generate molecule generation, while we em-
molecules with EBMs. focuses on dubstrucctree-peserviin phasize the advantage of permutation invariance endowed by EBMs for generating molecules from scratch. addition, we further investigate multi-objective molecule generation, which is practically desired but unexplored in Hataya et al. (222
THE PROPOSED GRAPHEBM
In this section, we present GraphEBM by describing our parameterized energy function (Sec- tion 3.2). showing that GraphEBM achieves permutation invariant generation (Section 3.3) and deseribing the learning (Section 3.4) and generation (Section (.35) process of GraphEBM. Then, we introduce proposed strategy for single-objective generation based GraphEBM (Section 3.6). Finally, we explain how to use GraphEBM for multi-objective molecule generation via composi- tional generation (Section 3.7
3.1 PROBLEM FORMULATION
Molecules can be naturally represented as graphs by considering atoms and bonds as nodes and edges, respectively. We formally represent molecular graph as (X, A), where X is the node
3

---

Under review as a conference paper at ICLR 2022
feature matrix and A is the adjacency tensor. Let ic be the number of nodes in the graph. b and denote the number of possible types of nodes and edges. respectively. Then we have X E {0, (0,11 1 and (i.))) = 1 if node belongs to type p. A E {0, 1] and A(jj..) denotes that an edge with type @ exists between node and node j. Following prior works (Madhawa et al. 2019 Zang & Wang] 2020). we let n denote the maximum number of atoms that molecule has in given dataset. We insert virtual nodes into molecular graphs that have less than n nodes such that the dimensions of X and A keep the same for all molecules. Also, for any two nodes that are not connected in the molecule, we add virtual edge between them. We can consider the virtual node and the virtual edge as an additional node type and edge type, respectively. Hence, for all molecules in a certain dataset, X E {0, 1] 1xx((++)) and A E {0, 1) 1)xxmx(c+1))
3.2 PARAMETERIZED ENERGY FUNCTION
Following the above notations, the energy function for molecular graphs can be denoted as Fo(X, A). In this work, we model (x, A) by graph neural network, where 0 denotes pa- rameters in the network. Particularly, we use a variant of relational graph convolutional networks (R-GCN) (Schlichtkrull et al. 2018) to learn the node representations since molecular graphs have categorical edge types. Formally, the layer-wise prwaard-roppggtion is defined as
H =0
(A (AG.,))H'W))
AGs.A) is the k-th channel of the adjacency tensor. Hº E Rnxdd is the node representation matrix at layer E, where de denotes the hidden dimension at layer f. W. E Rdexdr+1 represents the trainable weight matrix for edge type k at layer L. o()) denotes non-linear activation function. The initial node representation matrix H = X. In each layer. message passing is conducted among the nodes independently for each type of edge. Then, the information is integrated together by We stack L. such layers. Hence, the final node representation matrix is H E Raxd, where is the hidden dimension. Then, the representation of the whole graph can be derived by a readout function. In this work, we use the sum operator to compute the graph-level representation he as hG = E ((.)) E Rd. Finally, the scalar energy associated with the molecular graph can be obtained as E = h&W W E R. where W E Rd is the trainable transformation.
operator.
3.3 PERMUTATION INVARIANT GENERATION
Naturally, a distribution of molecular graphs can be denoted as p(X, Since the underlying data distribution of molecular graphs is invariant to permutations, as analyzed in Section [] an ideal gen- erative model should associate the same probability to various permutations of the same molecular graph. We show that our proposed GraphEBM satisfies this fundamental property. Specifically, each layer of our graph neural network in Eq. 2 is permutation equivariant. In addition, the readout operation is permutation invariant. Therefore, our parameterized energy function is permutation in- variant, thus satisfying Eg (X, = Eg (X* AT), where π denotes any permutation of node order. For simplicity, we the superscript according to the node order given by л. Further, the energy function defines a distribution over data space according Eq ( Specifically, the likelihood is proportional to the exponential of the negative energy. Hence, we can obtain po (X, A) = po (X", A*). Thus, our GraphEBM can define a permutatioo invariant distribution over molecular graphs
denote that the corresponding matrix tensor is arranged
Although it seems easy for our GraphEBM achieve permutation invariant generation, the under- lying rationale is non-trivial. It is brought by the unique advantage of EBMs, and this is why most existing methods based on other generative methods, as shown in Table [ cannot achieve permu- tation invariant generation. To be specific, EBMs have great flexibility since the energy function defines the distribution by unnormalized values. Thus, the distribution given by an EBM can be simply represented as Eq (1). which is crucial for showing our GraphEBM can achieve permutation invariant generation as above. In contrast, generative processes with a tractable likelihood, such as autoregressive models (Graves 2013) which are commonly used for sequential molecule generation, have less flexibility, thus losing the permutation invariance when modeling graphs. For example, in autoregressive models, model distribution has to be factorized as series of conditional distributions. Hence, a specific order of nodes has to be predetermined, thereby failing to providing a permutation invariant distribution for graphs.
4

---

Under review as a conference paper at ICLR 2022
Eal"" ,4 1 340)
Pust up
K steps
Ea(X,
Langevin dynamiis
Shop grachent
Es (
Nom
A
Figure 1: The learning process of our GraphEBM. The detailed annotation is in Appendix A
3.4 LEARNING
Intuitively, good energy function should assign lower energies to data points that correspond to real molecular graphs and higher energies to other data points. In this study, we apply contrastive divergence (Hinton 2002) to learn the energy function.
Let PD be the unknown distribution of the real data. LML = (xx,A))<p [- log pe (X, A)], where log pa (X = (x A) + log Z(0) according to
To achieve maximum likelihood, we have
Eq. TT. It has been shown (Hinton 2002
Turner 2005 Song & Kingma 2021 that this objective
has the below gradient:
VelMi = (Ex VeEx(X**
A
Exx ((xo,A9))pp ~ V Ex (A))
(3)
As defined in Eq. (T). is the distribution given by the energy function. Following et al. (2020b),
we refer (X ) as hallucinated samples. Intuitively, this gradient pushes down the energies of positive samples (x A²) and pushes up the energies of hallucinated samples (X", A²0 that are drawn from pa. In practice, however, sampling (X Ao) from pe challenging, since (()) in Eq. 4 is intractable.
To overcome this issue, we follow Du & Mordatch (2019) to sample (X", A0) from approx-
imated pe using Langevin dynamics (Welling & Tch 2011). Particularly, sample (x is initialized randomly and refined iteratively by
x x Vx x Eo .A--)) + w
(4)
> (xk-1 1AA,,AAA)) + +
A A k-
A
2
where w ry*h ~ NN( x2) are added noise sampled from a Gaussian distribution, i denotes the iteration step, and a is the step size. As demonstrated by Welling & Teh (201)). the obtained samples (x* A,At) approach samples from Po + and 0. In practice, we let K denote the number of iteration steps of Langevin dynamics and use the resulting sample (X* AK) as (x,,49) in Eq. (3)
We illustrate the training process of our GraphEBM in Figure 1 Since Langevin dynamics for continuous data, model the hallucinated samples by continuous format. For consistency, can also use dequantization techniques (Dinh et al. 2016 Kingma & Dhariwal 2018) to convert the discrete positive samples to continuous data by adding uniform noise") as shown the left part of Figure I The dequantization can be formally expressed as
-'= + tu, u ~ [0, 1)" (xx(b+)); A' A+ lu, u ~ [0, 1)*xxx(o+1),
(5)
t E [0. 1) is scaling hyperparameter. After dequantization, we apply a normalization to the adja- cency tensor. Formally,
A (c..k))
D- Aoaneyy k= ==
(6)
More discussion about dequantization available in Appendix
5

---

Under review as a conference paper at ICLR 2022
where D is the diagonal degree matrix in which ((..)) Ejik Alo. We treat the above A and
x X' as the input for the energy function. In our case, each element of x is in [0, 1 + t) and each element of A is in [0, 1).
To make the hallucinated sample xx",A9) have the same value range as (X A we initialize
it as X [0, + ()mx((b+1) and A [0, (111)x((==)) Then we apply K steps of Langevin dynamics as Eq. 4 to refine the sample, as illustrated in the right part of Figure [ After each step of refinement, we clamp the data to guarantee that the values are still in the desirable ranges.
As demonstrated in Eq. (3), the energies of positive samples are expected to be pushed down and the energies of hallucinated samples should be pushed up. Hence, to shape the energy function as expected, our loss function is defined
Lenergy Ex AAa) Ex 10).
As shown in the middle part of Figure D the gradient backpropagated from Lenergy can update the parameters thus pushing the energy function ( A) to approach our expected shape. Notably, the gradient from Lenergy will not be propagated to the energy function used in Langevin dynamics. We apply parameter sharing to keep the energy function used Langevin dynamics up-to-date. The overall learning process of our GraphEBM is summarized in Algorithm Appendix C
3.5 GENERATION
Let Egg (X, denote the learned energy function, where H* represents the obtained parameters. Intuitively, if energy function is well-shaped, the configurations with low energies should corre- spond to desirable molecular graphs. Hence, the generation process is to generate molecules based on the configurations (X,A) that yield low energies.
An overview of the generation process is shown in Figure Appendix D The steps are as follows. First, we initialize a data point as (8) and then apply steps of Langevin dynamics as in Eq. (4) to obtain data points that have low energies. We denote the obtained configuration as (X*, A.). Second, since molecular graphs are undirected, we make the adjacency tensor to be symmetric by using A A as the new adjacency tensor. Third, we convert the continuous data discrete ones by applying the argmax operation in the dimensions of atom types and bond types. Finally, we use validity correction introduced by Zang & Waag (222)) to refine the corresponding molecule so that the valency constraint is satisfied.
3.6
GENERATION
For drug discovery and material design, we further need to generate molecules with a desirable chemical property. This task is termed as single-objective generation (a.k.a. goal-directed genera- tion). As noted in Appendix [. it is not straightforward to apply existing strategies our GraphFBM for single-ohjective generation. To generate molecules with desirable chemical properties, we pro- pose novel, simple, and effective strategy for learning our GraphEBM. Our basic idea is to push down energies with flexible degrees according to the property values of corresponding molecules. molecule has a higher value of desirable property, we push down the corresponding energy harder. Formally, for single-objective generation, the loss function defined in Eq. 77) becomes
Lenergy
,A (
where y E [0, 1] the normalized property value and f(y) E determines the degree of the push down. We use f(y) = 1 + e in this work. Thus, energies of molecules with higher property values are pushed down harder. After training, the generation process is the same as described in Section
3.7 MULTI-OBIECTIVE GENERATION
In addition to the single property constraint, is more significant and desired to generate molecules with multiple property constraints in drug discovery. As analyzed in Section I multi-objective gen- eration still remains challenging for current methods since we usually lack such datasets, in which molecules satisfy multiple constraints simultaneously. We observe that compositionality brought by EBMs (Hinton (002), which has been shown to be effective the image domain (Du et al., 2020a), can be naturally applied generate molecules with multiple constraints based our GraphEBM.

---

Under review as a conference paper at ICLR 2022
Table 2: Generation performance on QM9.
Table 3: Generation performance on ZINC250K.
Method
Validity($)
Uniqueness( (6)
Novelty(%8
CVAE
10.30
67.50
90.00
Method
Validity(%s)
Uniqueness( [8]
Novelty(8)
GVAE
60.20
9.30
80.90
GCPN
100.00
90.97
100.00
GraphVAE
55.70
76,00
61.60
VAE
100.00
100.00 00
100,00
RVAE
96.60
97.50
MolecularRNN
100.00
90.89
100.00
MoIGAN
98.10
10.40
94.20
GraphNVP
42.60:1.0
94.802
100.00):0.0
GraphNVP
83.10=0.50
99 20
58.20+1.90
GRF
73.40_m082
53.7.12 13
100000...00
GRF
84.50=0.70
66 1..00
58.60+...6
GraphAF
100.00
99.10
100.00
GrahhAF
100.00
94.51
88.83
MoFlow
1000000+0.00
100.000=0.00
MoFlow
100.00+a.00
98.53+0.14
96.04.so.10
GraphEBM
99.96_in.02
98.791.935
100.00....00
GraphEBM
100.00=n.00
97. 97.90
97.01+0.17
Since the dataset for a single property is often available, suppose we have two energy functions Eoz (X, A) and Eag A). which are trained towards two property objectives O, and O2 respec- tively, as described in Section 3.6 To generate molecules that have properties O1 and O2 simul- taneously, we can obtain a new energy function by summing the above two energy functions since the product of probabilities is equivalent to the sum of corresponding energies, according to Eq. (I. This is known as the product of experts (Hinton 2002). Formally,
E FFgg (X.4)
-Fag (X.A)
x -Fog Eo
Pe; A) Pag (X, A)
Z(0
2(05)
Nota that 2(0)) and Z are constant w.r.f. various (X, A). Then we can apply the generation
process described in Section 3.5 to Eg- (X, A) Fl; (X, A) Fag (X, A) to generate molecules
towards multiple objectives.
EXPERIMENTS
4.1
SETUP
Datasets. We evaluate our proposed method on two widely used molecule datasets, QM9 (Ramakr- ishnan et al., 2014) and ZINC250K (Irwin al., 2012). QM9 consists of 134k organic molecules and the maximum number of atoms is 9. It contains 4 atom types and 3 bond types. ZINC250k has 250k drug-like molecules and the maximum number of atoms 38. includes atom types and edge types.
Implementation details. We kekulize molecules and remove their hydrogen atoms using RD- Kit (Landrum et al. 2006). In our parameterized energy function, we adopt network of layers with hidden dimension d = 64. We use Swish (Ramachandran et al. 2017) the activation function. We set the standard variance = 0.005 in the gaussian noise. For training, we tune the following hyperparameters: the scale t of uniform noise E [0, 1), the sample step K of Langevin dynamics E [30, 300], and the step size E [10, 50]. All models are trained for up to 20 epochs with learning rate of 0.0001 and batch size of 128. We follow the techniques adopted by Du & Mordatch| 2019) to stabilize the training process. Specifically, we add spectral normalization (Miy- ato al. 2018) to all layers of the network. In addition, we clip the gradient used in Langevin dynamics so that its value magnitude can be less than 0.01. All experiments are run with a single GeForce RTX 2080 Ti GPU.
Evaluation standards. We evaluate our proposed method for molecule generation under three settings: random generation, single-objective generation, and multi-objective generation. For ran- dom generation, we compare the widely used quantitative metrics with baselines and visualize the generated molecules. For single-objective and multi-objective generation, we show the distribu- tion comparison with random generation in terms of property values. addition, we perform molecule optimization, including property optimization and constrained property optimization, to further demonstrate the effectiveness our proposed single-objective generation method. The de- tails of the experimental setup for each setting included in Appendix F
4.2 RANDOM GENERATION
The quantitative results on QM9 and ZINC250k are shown in Table 3 We can observe that GraphEBM performs competitively with existing methods, which is significant considering that the
7

---

Under review as a conference paper at ICLR 2022
0.948
0.948
0.948
0.948
0.94K
0.948
0.948
0.948
0.948
0 948
0.948
Figure 3: Discovered examples with highest QED scores.
Figure 2: Visualization of generated molecules.
study of EBMs is still in its early stage, compared with other generative models such GANs and Flows. Generated samples are visualized in Figure 2 which further demonstrates that GraphEBM generate non-trivial molecules.
To better understand the implicit generation via Langevin dynamics, visualize this process for an example Figure 7 Appendix [ We can observe that Langevin dynamics effectively refines the random initialized sample to approach data point that corresponds realistic molecule.
4.3
SINGLE-OBJECTIN GENERATION
Figure 4 (b) compare the property value distribution between molecules obtained by single-objective generation and random gener- ation. can be observed that sbbele-objective generation generate
Probability density
Random generation Single Objective generation
Tenker censabie
molecules with high property values, indicating that our pro- posed strategy for single-objective genera- tion, which aims to assign lower energies to molecules with higher property values, is effec- tive.
(b)
QED
Pbogg
Probabliity density
km
Random generation Single objective generation
The property optimization results are shown in Table Property optimization aims at generat- ing novel molecules with high QED scores. We
observe that GraphEBM can find more novel molecules with the best QED score (0.948) than baselines. This strongly demonstrates the effec- tiveness of our proposed single-objective gen- cration method. Examples of discovered novel molecules with highest QED scores are illus- trated in FigureB3
plogp
QED
Figure 4: (b)&(b) Comparison of QED and plogp distributions between single-objective generation and random generation, respectively. (c) Com- parison distributions on QED and plogp be- tween multi-objective generation and random gen- eration.
For constrained property optimization, given molecule, task is to obtain has better desired chemical property while preserv- ing similarity. As demonstrated in Table GraphEBM can obtain higher property improvements over JT- VAE, GCPN, and MoFlow by significant margins, and performs competitively with GraphAF. In addition, it can be observed from Shi et al, (2019) that GraphAF learns to improve plogp by simply adding long carbon chains, while our GraphEBM learns more advanced chemical knowledge. Several examples of constrained property optimization are shown in Figure It is interesting that the modifications are interpretable to some degree. Specifically, in the first example, our model optimizes the plogp score with markable margin of 18.03 by replacing several carbon atoms with sulfur atoms, which could make the molecule more soluble in water, thus leading to larger logP value. Additionally, plogp is highly
Table Property optimization results in terms of the best QED scores. Our GraphEBM finds more novel molecules with the best QED scores.
molecule that
Method
2nd
3rd
4th
JT-VAE GCPN
0.925
0.911
0.910
0.948
0.947
0.946
GraphAF
0.948
0.948
0.947
0.946
MoHlow
0.948
0.948
0.948
0.948
GraphEBM
0.948
0.948
0.948
0.948
8

---

Under review as a conference paper at ICLR 2022
Table 5: Constrained property optimization results. Note that there two different settings in baselines. JT-VAE and GCPN choose 800 molecules with the lowest plogp scores in the test set and use them as initialization, while GraphAF and MoFlow choose from the training set. We report our results on both of these two settings for extensive comparisons. is the similarity constraint. The top two highest property improvements on each constraint are highlighted as 1st and 2nd.
JT-VAE Similarity
GruphEBM
6
Improvement
Success
Improvement
Similarity
Success
Improvement
Similarity
Success
0.0
1.91
0.28 15
98%
4.20
0.32
100%
5.762-1.6
0.08
98%
0.2
1.68
0.33=0.13
97%
4.12
0.34
100%
3.97
0.35+a.14
92%
0.4
1.844
0.
84%
2.49-130
1.47
100%
2.84
0.531..10
88%
0.6
0.21
0.69=0000
46%
0.79-0.68
0.08
100%
1.52=266
0.68
61%
GraphAF Similarity
MoFlow Similarity
GraphEBM Similarity
6
Improvement
Success
Improvement
Success
Improvement
Success
0.0
13.13 so
0.20 15
100%
8.61 44
0.30
99%
15.75 so
0.01
90%
0.2
11.90
0.33=0.1
100%
7.06=6.04
0.43+0.20
97%
8.40
0.35+a.15
91%
0.4
8.21
0.49=0.00
100%
4.71=4.55
0.61
80%
4.95+ 55.00
0.54
79%
0.6
4.98
0.06=0.06
97%
2.10-2.80
0.79
58%
15
0.67
45%
0.57
+18.00
+7.29
+14,50
5.90
plogp
Figure 5: Examples of constrained property optimization. The values above and below arrows denote the similarity scores and improvements, respectively. The modifications highlighted with red rectangles.
related to the number of long cycles and synthetic accessibility. As shown in the second and third examples, our model improves the synthetic accessibility and reduces the number of long cycles by removing or breaking long cycles. These facts indicate that our single-objective generation method can explore the underlying chemical knowledge related to the corresponding property objective.
4.4 MULTI-OBIECTIVE GENERATION
The comparison of the distributions on QED and plogp between molecules obtained by multi- objective generation and random generation is illustrated in (c) We can observe that multi- objective generation tends to generate more molecules with both high QED and plogp scores. Ad- ditionally, the distribution of QED or plogp is similar to the corresponding distribution obtained by single-objective generation towards a single objective (Figure ! (a)& These facts demonstrate that GraphEBM is able to generate molecules towards multiple objectives in compositional manner, which provides a novel and promising way for multi-objective molecule generation.
CONCLUSION
In this paper, we propose GraphEBM as an exploratory work towards permutation invariant and multi-objective molecule generation. GraphEBM is the first EBM for generating molecular graphs with defining a permutation invariant distribution over molecular graphs. For objective-dires generation, propose to flexibly push down energies according given property values and further explore to generate molecules towards multiple objectives in a compositional manner, leading to a promising method for multi-objective molecule generation. Experimental results demonstrate that our GraphEBM can generate realistic molecules and the proposals of single-objective and multi- objective generation are effective and promising.
Since EBMs have unique advantages for molecular graph generation, including preserving permu- tation invariance and enabling multi-objective generation via compositionality, and have rarely been explored for graph generation, hope our exploratory work would open the door for future search in this area.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
A lot of efforts have been made ensure the reproducibility of our study. To make methodology as clear as possible, in addition to the description in Section we include more explanations, discus- sions, and illustrations in Appendix [A B & E In terms of experiments, we provide the details of our experimental settings, including random, singlc-objective, and multi-objective generation, in Appendix F In addition, we describe how our hyperparameters are determined in Section 4.1 We will open-source our implementations once the paper is published.
REFERENCES
David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. A learning algorithm for boltzmann machines. Cognitive science, 91)::477-169 1985.
G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.
Xavier Bresson and Thomas Laurent. A two-step graph convolutional decoder for molecule gener- ation. arXiv preprint arXiv:1906.03412. 2019.
Barry A Cipra. An introduction to the ising model. The American Mathematical Monthly, 94(10): 937-959, 1987.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Song. Syntax-directed variational autoen- coder for structured data. In International Conference on Learning Representations, 2018.
Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel. The helmholtz machine. Neural computation, 7(5):889-904, 1995.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molccular graphs. arXiv preprint arXiv: /805 1973, 2018.
Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc' Aurelio Ranzato. Residual energy-based models for text generation. In International Conference on Learning Represen- tations, 2020.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components esti- mation. arXiv preprint arXiv:1410.8516, 2014.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605.08880 2016.
Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. In Advances in Neural Information Processing Systems, volume 32, pp. 3608-3618, 2019.
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based mod- els. Advances in Neural Information Processing Systems, 33, 2020a.
Yilun Du, Shuang Li. Joshua Tenenbaum, and Igor Mordatch. Improved contrastive divergence training of energy based models. arXiv preprint arXiv:2012.013/6 2020b.
Yilun Du, Joshua Meier, Jerry Ma, Rob Fergus, and Alexander Rives. Energy Energy-based models for atomic-resolution protein conformations. In International Conference on Learning Representa- fions, 2020c.
Justin Gilmer, Samuel Schoenholz, Patrick Riley, Oriol Vinyals, and George Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th international conference on machine learning, pp. 1263-1272, 2017.
Rafael Gómez-Bombaree
Jennifer Wei, David Duvenaud, José Miguel Hernández-Lobato, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzin Automatic chemical design using a data-driven contin- representation of molecules. ACS central science, 4(2):268-21 276.26666211 2018.
Benjamín Sánchez-Lengelingg Dennis Sheberla, Jorge aguilera-Iparragee
10

---

Under review as a conference paper at ICLR 2022
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27:2672-2680 2014.
Alex Graves. Generating sequences with recurrent neural networks.
arXiv preprint
arXiv:1308.08500 2013.
Ryuichiro Hataya, Hideki Nakayama, and Kazuki Yoshizoe. Graph energy-based model for sub- structure preserving molecular design. arXiv preprint arXiv:2102.04600, 2021.
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8)1171-1800,, 2002.
Geoffrey Hinton. A practical guide to training restricted boltzmann machines. In Neural networks: Tricks of the trade, pp. 599 599-619. Springer, 2012.
Shion Honda, Hirotaka Akita, Katsuhiko Ishiguro, Toshiki Nakanishi, and Kenta Oono. Graph residual flow for molecular graph generation. arXiv preprint arXiv:1909.13521, 2019.
John J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 7954):2554-2558, 1982.
John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman. Zine: free tool to discover chemistry for biology. Journal of chemical information and modeling, 52 (7):1757-1768, 2012.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning, pp. 2323-2332, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In International Conference on Machine Learming, 2020a.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. In International Conference on Machine Learning, pp. 4849-4859. PMLR, 2020b.
Diederik Kingma and Max Welling. arXiv:1312.6114 14,,,611121111 2013.
Auto-encoding variational bayes.
arXiv preprint
Durk P Kingma and Prafulla Dhariwal. Glow:
Generative flow with invertible 1x1 convolutions. In Advances in neural information processing systems, pp. 10215-10224, 2018.
MJ Kusner, B Paige, and JM Hernández-Lobato. Grammar variational autoencoder. In Proceedings of the 34 th International Conference Machine Learning, Sydney, Australia, PMLR 70, 2017, volume 70, pp. 1955-1954. ACM, 2017.
Greg Landrum et al. RDKit: Open-source cheminformatics. 2006.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang. tutorial energy-based learning. Predicting structured data, 100), 2006.
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. Learning deep generative models of graphs. In International Conference on Machine Learning. 2018.
Phillip Lippe and Efstratios Gavves. Categorical normalizing flows via continuous transformations. arXiv preprint axXiv:2006.09790. 2020.
Jenny Liu, Will Grathwohl, Jimmy Ba, and Kevin Swersky. Graph generation with energy-based models. ICML. Workshop Graph Representation Learning and Beyond (GRL++,, 2020.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt. Constrained graph varia- tional autoencoders for molecule design. Advances in neural information processing systems, 31: 7795-7804, 2018.
11

---

Under review as a conference paper at ICLR 2022
Youzhi Luo, Keqiang Yan, and Shuiwang Ji. GraphDF: A discrete flow model for molecular graph generation. International Conference on Machine Learning, pp. 7192-7203, 2021.
Tengfei Ma, Jie Chen, and Cao Xiao. Constrained generation of semantically valid graphs via regularizing variational autoencoders. In Advances in Neural Information Processing Systems, pp. 7113-7124, 2018.
Kaushalya Madhawa, Katushiko Ishiguro, Kosuke Nakago, and Motoki Abe. Graphnvp: An invert- ible flow model for generating molecular graphs. arXiv preprint arXiv: /600, 2019.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In International Conference on Learning Representations, 2018.
Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Per- mutation invariant graph generation via score-based generative modeling. Artificial Intelligence and Stafistics, 2020.
Mariya Popova, Mykhailo Shvets, Junier Oliva, and Olexandr Isayev. Molecularmn: Generating realistic molecular graphs with optimized properties. arXiv preprint 1rX.v: 2019.
Prajit Ramachandran, Barret Zoph, and Quoe Le. Swish: a self-gated activation function. arXiv preprint arXiv: (rXvv:/710005941 2017.
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.
Danilo Rezende and Shakir Mohamed, Variational inference with normalizing flows. Interna- tional Conference on Machine Learning, pp. 1530- 1530-1538, 2015.
David Rogers and Mathew Hahn. Extended-coomeccivv fingerprints. Journal of chemical informa- tion and modeling, 50(5)7442-75,, 2010.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling, Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pp. 593-607. Springer, 2018.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. In International Conference Learning Representations, 2019.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation small graphs using variational autoencoders. In International Conference Artificial Neural Networks, pp. 412- 422. Springer, 2018
Yang Song and Diederik P Kingma. How to train your energy-based models. arXiv preprint arXiv:2101.03288, 2021.
Jonathan M Stokes, Kevin Yang, Kyle Swanson, Wengong Jin, Andres Cubillos-Ruiz. Nina M Donghia, Craig R MacNair, Shawn French, Lindsey A Carfrae, Zohar -oom--ckerman, et al. A deep learning approach antibiotic discovery. Cell, 180(4)6688-702 2020.
Richard Turner CD notes. 2005.
Zhengyang Wang, Meng Liu, Youzhi Luo, Zhao Xu, Yaochen Xie, Limei Wang, Lei Cai, and Shui- wang Ji. Advanced graph and sequence neural networks for molecular property prediction and drug discovery. arXiv preprint arXiv:20/2.0/99/1 2020.
David Weininger. Smiles, chemical language and information system. 1. introduction method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-38 1988.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-I1), pp. 681-688, 2011.
12

---

Under review as a conference paper at ICLR 2022
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: benchmark for molecular machine learn- ing. Chemical science, 9(2):513-5300 2018.
Jianwen Xie, Wenze Hu, Song-Chun Zhu, and Ying Nian Wu. Learning sparse frame models for natural image patterns. International Journal of Computer Vision, 114(233:91-112, 2015.
Jianwen Xie, Yang Lu, Song-Chur Zhu, and Yingnian Wu. A theory of generative convnet. In International Conference Machine Learning, pp. 2635 2635-2644, 2016.
Jianwen Xie, Song-Chun Zhu, and Ying Nian Wu. Synthesizing dynamic patterns by spatial- temporal generative convnet. In Proceedings of the ieee conference computer vision and pattern recognition, pp. 7093-7101, 2017.
Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Joug Zhu, and Ying Nian Wu. Learning descriptor networks for 3d shape synthesis and analysis. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8629-8638, 2018.
Jianwen Xie, Yifei Xu, Zilong Zheng, Song-Chun Zhu, and Ying Nian Wu. Generative pointnet:
Energy-based learning on unordered point sets for 3d generation, reconstruction and classification. arXiv, pp. arXiv-2004, 2020.
Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman- Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, et al. Analyzing learned molecular repre- sentations for property prediction. Journal of chemical information and modeling, 59(8):3370-
3388, 2019.
Jiaxuan You, Bowen Liu, Zhitao Ying. Vijay Pande, and Jure Leskovec. Graph convolutional pol- icy network for goal-directed molecular graph generation. In Advances neural information processing systems, pp. 6410-6421, 2018.
Chengxi Zang and Fei Wang. Moflow: invertible flow model for generating molecular graphs. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 617-626, 2020.
Song Chun Zhu, Yingnian Wu, and David Mumford. Filters, random fields and maximum entropy (frame): Towards unified theory for texture modeling, International Journal of Computer Vision, 27(2):107-126, 1998.
13

---

Under review as a conference paper at ICLR 2022
A
THE DETAILED ANNOTATION OF FIGURE 1
The left part and right part illustrate the processes of obtaining the positive sample and the hallu- cinated sample, respectively. The middle part shows the forward and backward propagation of the training process. Green and purple arrows represent the forward computation of energy value for the positive sample and the hallucinated sample, respectively. The black dashed arrow denotes the gradient backpropagation. In this example, = 9, k 8, b 4, and 3. The annotations of node representation matrices denote the atom types, including carbon (C), nitrogen (N), oxygen (O), fluorine (F), and virtual atom (+). Note that we remove hydrogen atoms, which is common technique in the community. The annotations of adjacency tensor indicate the bond types, including single (S), double (D), triple (T), and virtual bond (+).
B MORE DISCUSSION ABOUT DEQUANTIZATION
Note that the dequantization for positive samples is optional. This indicates that we can set and keep the positive data discrete since Langevin dynamics is only required for obtaining halluci- nated samples. Applying dequantization to positive samples can be viewed as a data augmentation easily convert the dequantized continuous data back to the original one-hot
technique and discrete data by simply applying the argmax operation.
C LEARNING ALGORITHM
Algorithm Learning algorithm of GraphEBM
Input: Observed molecular graph dataset D = {(X, (m)
m=1 with n, b, and c, parameterized
[P]
energy function (-).). step size å variance of noise o², number of steps K, scaling hyperpa- rameter t, weight a E R for regularization term
1: for (X,A) A) D do
Batch training is applied in practice [0, (b+x()));; = tu, 10 00,))xxxx(++))
2:
x = X + tu,
Eq. Eq.
3:
A (..ik) = D-1 (::.))) k= 1,,,.cc++ 1, where D(r.)) = E,, A Ale.5.))
(6)
4:
Compute Ex A
5:
X [0, + (b+111+
Ao [0. mx)xxx((e())
6:
for sample step
to K do
Eq.
7:
X X
w ~((,002 4 = - A V Eo (Xx=1, ( + n²² r/ ~ o²(,00
2 V Eg | + w
8:
9:
Clamp x and A such that the values in the desirable ranges end for
10:
11:
= X 1 = AK Compute ( A (
X
12:
Compute Lenergy
1x A 6 AA0
Eq.
14:
Compute Lrey Ex An)² (x .002 Regularizer is used for stable training & Total loss function
15:
Compute -= Cenergy + afreg
16:
Update / based on VoL
17:
end for
14

---

Under review as a conference paper at ICLR 2022
D AN ILLUSTRATION OF GENERATION PROCESS
steps
Egg (x,
Langevin dynamics
Validity correction
Symmetrization
Discretization
HO
Figure 6: The generation process of our GraphEBM.
E EXISTING SINGLE-OBIECTIV GENERATION STRATEGIES
There are mainly three approaches in the literature for single-objective generation. First, this task can be modeled as a conditional generation problem, where the property value can be utilized as the condition (Simonovsky & Komodakis 2018). Second, for methods using the latent space, pre- dictor can be applied to learn the property value from the latent representation (Gómez-Bombarelli al. 2018). Third, reinforcement learning can be used to optimize the properties of generated molecules You et al. 2018). However, it is not straightforward to apply these methods to our GraphEBM for single-objective generation since GraphEBM generates molecules implicitly using Langevin dynamics and no latent space exists. Also, using EBMs for generation that is conditional on continuous conditions rarely studied by the community. Hence, it remains challenging to apply EBMs for oingle-objective generation.
F EXPERIMENTAL SETUP FOR EACH SETTING
Random generation. evaluate the ability of our proposed GraphEBM model and generate molecules. We consider most methods reviewed in Section 2.1 baselines. The following com- monly used metrics are adopted. Validity is the percentage chemically valid molecules among all generated molecules. Uniqueness denotes the percentage of unique molecules among all valid molecules. Novelty corresponds to the percentage of generated valid molecules that are not present in the training set. The metries are computed on 10, 000 randomly generated molecules. Results averaged over 5 runs are reported in Table 2 For QM9, the results of CVAE and GVAE are obtained from Simonovsky & Komodakis (2018) The result of MoFlow is obtained by evaluating its public trained model. All other results are from their original papers. For ZINC250K, the results of GCPN and JT-VAE are obtained from Shi et al. (2019). The result of MoFlow is obtained by evaluating its public trained model. All other results are from their original papers.
Single-Objective generation. To empirically show the effectiveness of our single-objective gen- eration method proposed in Section 3.6 we train models on ZINC250K accordingly and compare the distribution of the property score between molecules obtained by single-objective generation and random generation. We consider two chemical properties, including Quantitative Estimate of Druglikeness (QED) [Bickerton et al. 2012) and penalized logP (plogp), which is the water-octanol partition coefficient penalized by the number of long cycles and synthetic accessibility.
We further verify the effectiveness of our proposed single-objective generation method by perform- ing molecule optimization, including property optimization and constrained property optimization. Property optimization aims at generating novel molecules with high QED scores. We directly use the model trained for single-objective generation and leverage the molecules in the training set as initialization for Langevin dynamics, following prior works (Jin et al. 2018 Zang & Wang 2020). We report the highest QED scores and the corresponding novel molecules discovered by our method.
15

---

Under review as a conference paper at ICLR 2022
For constrained property optimization, given molecule our task is to obtain molecule m' that has a better desired chemical property with the molecular similarity sim(m.. (m. m 2 for some threshold 8. We adopt Tanimoto similarity of Morgan fingerprint (Rogers & Hahn 2010 to measure the similarity between molecules. We find that there are two different settings baselines. J'T-VAE and GCPN choose 800 molecules with the lowest plogp scores in the test set and use them initial- ization, while GraphAF and MoFlow choose from the training set. We report our results on both of these settings for extensive comparisons.
Multi-Objectivy generation. As investigated in Section 3.7 our GraphEBM has the potential to conduct compositional generation towards multiple objectives. To verify this, we combine the two energy functions obtained in single-objective generation experiments, as formulated in Eq. ( Then we apply the generation process described in Section to the resulting energy function to generate molecules. To show the effectiveness of our proposal, we compare the distribution of the property scores between molecules generated by multi-objective generation and random generation.
G VISUALIZATION OF THE IMPLICIT GENERATION PROCESS
Argmax
Validity correction
4
Argmax
Initialization
Langevin dynamics
Figure 7: Visualization of the implicit generation process of our GraphEBM. The first row denotes atom matrices and the remaining rows represent fours channels of adjacency tensors, corresponding single, double, triple, and virtual bonds. For better visual results, each atom matrix and adjacency tensor is normalized by dividing by its maximum value.
16


Reviews:
[{"review_id": "j9NsDIkCRNJ", "reviewer": "Reviewer_Fo2L", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the paper is well-written. I like the simplicity of the proposal and the pursuit of permutation invariant generation. Notably, this seems to be the first proposal of EBM-based generative models for molecules and attributed graphs. I also agree with the authors that achieving multi-objective property optimization via the compositionality of energy functions is an interesting aspect of EBMs.\n\nMy concerns/questions mainly refer to the degree of novelty (C1), simplified evaluation setup (C2), lack of performance gains (C3), ability to generate valid molecules without posthoc correction (C4), and better motivation for some modeling choices (C5). \n\n- C1: Some of the contributions listed by the authors (in Section 1) derive naturally from the use of EBMs. For instance, the compositional generative aspect of EBMs has been explored for image data in [1]. Therefore, it seems like the paper's main contribution is to use GNNs to parameterize energy functions.  \n\n- C2: The multi-objective setup only includes the QED and LogP properties, which are highly correlated. It would be important to consider more realistic scenarios with different targets, such as those used in [2]. \n\n- C3: The method only achieves competitive results and doesn't provide significant gains over the baselines. For instance, GraphEBM is not the best performing method in any of the metrics in Tables 2 and 3. Also, in Table 5, the proposed GraphEBM reports higher variances than GCPN.\n\n- C4: The authors should report the percentage of generated valid molecules before applying the posthoc correction procedure. This would allow us to assess the quality of the generative process via SGLD.\n\n- C5: What is the rationale of choosing $f(y)=1+e^{y}$, where $y \\in [0, 1]$? This seems to significantly limit how much the model pushes down the energy of favorable molecules.\n\n**Minor issue**: I would not say that \"...the study of EBMs is still in its early stage compared with other generative models such as GANs and Flows\" as the appearance of EBMs dates to the '80s.\n\nBased on the issues above, my recommendation is marginally below the acceptance threshold. As usual, I am open to increasing my score depending on the authors' rebuttal.\n\n[1] Du & Mordatch. Implicit Generation and Modeling with Energy-Based Models. NeurIPS, 2019. \n\n[2] Jin et al., Multi-Objective Molecule Generation using Interpretable Substructures. ICML, 2020.", "summary_of_the_paper": "The paper proposes a new energy-based generative model for molecules. The idea is straightforward and relies on using R-GNNs (relational graph neural networks) to parameterize the energy function. The authors show how to achieve single- and multi-objective generation via scaling, and composition, respectively. The experiments mostly follow the setup introduced in JT-VAE (Junction-Tree VAEs), and the results show the proposal is competitive against existing models.", "summary_of_the_review": "The main positive aspects of this paper are: $i$) first EBM-based model for molecules; $ii$) a model that leverages the node permutation-invariant nature of graphs/molecules. On the other hand, the main negative aspects include $i$) limited evaluation setup; $ii$) no significant improvements over baselines (the empirical gains are questionable). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MM9pxhMqFJ", "reviewer": "Reviewer_tnpH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros：\n\n- This paper proposes to use EBMs for task-based molecule generation, which is exploring a different direction from the most popular autoregressive-based generation.\n\n- The paper is well-written and easy to follow. \n\n- Experiments are comprehensive in the sense that the authors conduct both single-objective optimization, constrained optimization, and multi-objective optimization to verify the proposed method.\n\nCons:\n\n- The idea of using EBMs in this paper is somewhat straightforward and incremental. From my understanding, one of the most technically difficult parts of using EBMs on graphs lies in the generation, from continuous domain to discrete domain. However, this paper applies simple post-processing and does not address any of the issues here. It would be more helpful to justify such post-processing using theoretical or empirical evidence.\n\n- In the constrained property optimization experiment, the authors only show the results without mentioning how to use their method for tasks with structure constraints.\n\n- The paper shows many experiments, but it is not clear which one is the main experiment. In previous work, the harder task is to generate molecules with multiple (4) properties[1,2,3]. In the current paper, the single-objective optimization seems to be a sanity check. It is unclear how to do constrained property optimization. And it is hard to quantify the multi-objective results.\n\n- Writing issue: the paper puts too much emphasis on the permutation invariance property of the proposed method. Admittedly, the proposed method has the advantage of permutation invariance compared with autoregressive generative models because it does not rely on conditional generation and the energy function itself is a permutation invariant GCN. However, there is no necessity to spend the entire subsection 3.3 on this. The paper would be more persuasive if the authors can demonstrate this benefit directly using empirical results.\n\n\nQuestion:\n\n- The authors propose to use sum pooling for aggregating the graph level information. Would this result in different scaling when computing the energy? i.e., large molecule will be more likely to result in larger energy values?\n\n- How does summing two energy functions compared with using the geometric mean of two objective scores and train only one energy function? Does training one extra energy function helps generate better molecules?\n\n\n[1] Li, Yibo, Liangren Zhang, and Zhenming Liu. \"Multi-objective de novo drug design with conditional graph generative model.\" Journal of cheminformatics 10.1 (2018): 1-24.\n[2] Jin, Wengong, Regina Barzilay, and Tommi Jaakkola. \"Multi-objective molecule generation using interpretable substructures.\" International Conference on Machine Learning. PMLR, 2020.\n[3] Chen, Binghong, et al. \"Molecule Optimization by Explainable Evolution.\" International Conference on Learning Representations. 2021.", "summary_of_the_paper": "The paper proposes to use energy-based models (EBMs) for task-based molecule generation. The energy function is parameterized using relational graph convolutional networks (R-GCN) which have the permutation invariance property. In order to learn an energy landscape that can generate high-scoring molecules, the authors propose to weigh the positive samples by their objective scores during the contrastive divergence training and to combine several energy functions for multi-objective molecule generation. Experiments on single- and multi-objective optimization tasks demonstrate its ability to generate molecules with high scores.", "summary_of_the_review": "Overall, due to the weakness in novelty and experiments of this paper, I recommend rejecting it in its current form. But I encourage the authors to improve the paper according to the comments and resubmit it in the future.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "EMVGNwoBWTu", "reviewer": "Reviewer_EHKx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: as the auther claim, this is the very first paper that proposes to use energy-based models to generate molecule graphs. It is well written and experimental results look promising.\n\nweakness: As the main goal of the paper was to overcome the challenge of permutation invariance property when we generate graphs, it would be good to have a discussion in the paper the computational cost of the proposed method when compared to other baselines that use different tricks to handle the permutation invariance.\n\nI also wonder about the sampling cost of this method, as it applies Langevin dynamics to generate samples, what is the time cost and how many iteration it normally requires?", "summary_of_the_paper": "The paper proposes an energy-based molecular graph generative model. By using a permutation invariant energy function, the model is able to model a permutation invariant distribution over molecular graphs, thus preserving the intrinsic permutation invariance\nproperty during density modeling. They propose to use contrastive divergence to learn the energy function and generate samples from it via Langevin dynamics. They also proposed goal-directed generation where they push down the energy with a flexible degree according to the property value of desirable properties.", "summary_of_the_review": "The paper is well written and the presenting idea seems interesting and novel.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "7jfdZ1m__x3", "reviewer": "Reviewer_JCco", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength\n1. This paper propose an energy-based generative model for molecule generation, which is a novel contribution of molecule generation literature.\n\n2. This paper is well-written, and easy to read.\n\nWeakness\n1. The discussion on the main contribution of single objective optimization (Eq. 8) lacks. The proposed modification scale up the positive energy more than 1, then it is not MLE solution of Boltzman distribution modeled by the energy function. I know this modification practically works well as in the experiments, but this should be discussed in the paper.\n\n2. I am not expert of molecule generation, but the improvement of proposed method is too marginal. For example, the Table 4 does not show any meaningful results; and in the Table 5 the proposed method only shows better performance on the most relaxed constraints setting that allows any modification (delta=0.0).", "summary_of_the_paper": "This paper proposed a graph energy-based model based for molecular graph generation. The energy function is implemented by graph neural networks, and a molecule is generated by the Langevin dynamics that maximizes the trained energy function. Further, the author proposed to re-weight energy by a function of property value for the single objective optimization, and to sum up energy functions trained on different property datasets for the multiple objective optimization. The experimental results show the effectiveness of the proposed method on benchmark molecular graph generation tasks.", "summary_of_the_review": "This paper propose a novel method of energy-based model for molecule generation implemented by GNN. However, the modification of loss function is not justified properly and the improvement is too marginal. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to use an energy-based model for a multi-objective molecular generation. The energy function is parameterized by relational graph convolutional network (R-GCN) so that it has a permutation invariance property. The model is trained by contrastive divergence and the generation is performed by Langevin dynamics. Experiments on single and multi-objective molecule generation are conducted to verify the effectiveness of the proposed framework. The paper is well-written, and the experiments are comprehensive. The major shortcoming of the paper is its limited novelty, since using EBM for graph generation is a straightforward application of the existing deep EBM framework. The contribution is marginal.  

During the discussion, two of the reviewers pointed out that the contribution is limited and marginal. Two reviewers pointed out that the performance gain obtained by the proposed model is marginal and not significant. One reviewer has a concern about the computational cost of MCMC. However, the authors didn’t provide a rebuttal to address the concerns raised by the reviewers. Given the fact that all the concerns from the reviewers remain, and the contribution and performance gain of the work are marginal, the AC recommends rejecting the paper.
Decision: Reject