Molecule Optimization by Explainable Evolution
Authors: Binghong Chen
Tianzhe Wang
Chengtao Li
Hanjun Dai
Le Song
Venue: ICLR 2021 Poster (2021)
Keywords: Molecule Design
Explainable Model
Evolutionary Algorithm
Reinforcement Learning
Graph Generative Model

Abstract: Optimizing molecules for desired properties is a fundamental yet challenging task in chemistry, material science, and drug discovery. This paper develops a novel algorithm for optimizing molecular properties via an Expectation-Maximization (EM) like explainable evolutionary process. The algorithm is designed to mimic human experts in the process of searching for desirable molecules and alternate between two stages: the first stage on explainable local search which identifies rationales, i.e., critical subgraph patterns accounting for desired molecular properties, and the second stage on molecule completion which explores the larger space of molecules containing good rationales. We test our approach against various baselines on a real-world multi-property optimization task where each method is given the same number of queries to the property oracle. We show that our evolution-by-explanation algorithm is 79% better than the best baseline in terms of a generic metric combining aspects such as success rate, novelty, and diversity. Human expert evaluation on optimized molecules shows that 60% of top molecules obtained from our methods are deemed successful. 

Published as a conference paper at ICLR 2021
MOLECULE OPTIMIZATION BY EXPLAINABLE EVOLUTION
Binghong Chen', "Tianzhe Wang'., Chengtao Lie, Hanjun Dai", Le Song* a
1 Georgia Institute of Technology Galiixir *Google Research, Brain Team Moohaeed bin Zayed University of Al "Shangaai Qi Zhi Institute
ABSTRACT
Optimizing molecules for desired properties is a fundamental yet challenging task in chemistry, material science, and drug discovery. This paper devel- ops a novel algorithm for optimizing molecular properties via an Expectation- Maximization (EM) like explainable evolutionary process. The algorithm is de- signed to mimic human experts in the process of searching for desirable molecules and alternate between two stages: the first stage on explainable local search which identifies rationales, L.e, critical subgraph patterns accounting for desired molec- ular properties, and the second stage on molecule completion which explores the larger space of molecules containing good rationales. We test our approach against various baselines on a real wealdworld method is given the same number of queries to the property oracle. We show that our voolution-by-explannn
optimization task where each
algorithm is 79% better than the best baseline in terms of a generic metric combining aspects such as success rate, novelty, and diversity. Human expert evaluation on optimized molecules shows that 60% of top molecules obtained from our methods are deemed successful.
INTRODUCTION
The space of organic molecules is vast, the size of which is exceeding 1060 (Reymond et al. 2010) Searching over this vast space for molecules of interest is challenging task in chemistry, material science, and drug discovery, especially given that molecules are desired to meet multiple criteria, e.g., high potency and low toxicity in drug discovery. When human experts optimize molecules for better molecular properties, they will first come up with rationales within desirable molecules. Typically, the rationales subgraphs in molecule deemed to contribute primarily certain de- sired molecular properties. Once rationales are identified, chemists will design new molecules on top of rationales hoping that, the desired properties of new molecules will be further enhanced due to the existence of rationale and changes of non-rationale parts. The cycle of identifying molecular rationales and redesigning new hypothetical molecules will be carried on until molecules that meet certain property criteria are discovered.
In this paper, we develop novel algorithm that mimics the process of molecule optimization by human experts. Our algorithm finds new molecules with better properties via an FM-like explainable evolutionary process (T.. The algorithm alternates between two stages. During the first stage, explainable local search method to identify rationales within high-quality molecules that account for their high property scores. During the second stage, we use a conditional generative model explore the larger space of molecules containing useful rationales.
Our method is novel in that we are using explainable models to help us exploit useful patterns in the molecules. yet leveraging generative models to help explore the molecule landscape. Comparing to existing methods that directly learn a generative model using Reinforcement Learning or perform continuous optimization in the latent space of molecules (Olivecrona et al., 2017]] You et al. 201 8a] Dai et al. 2018b), our method is more sample-efficient and can generate more novel and unique molecules that meet the criteria.
We evaluate our algorithm against several state-of-the-art methods on a molecule optimization task involving multiple properties. Compared with baselines, our algorithm is able to increase the success
"Corespondence to: Binghong Chen <hinghongg@aacchhhcdu. indicates equal contribution. Source code at atttpss: //g1 gightttn thub. shut.ccom//inghooggmm111 N

---

Published as a conference paper at ICLR 2021
Explainable local search
E-step
Explainis)
J(6.p(o))
ratiorale
Intlccul
Rationade socuples p'(s((s)
Seed molecntes
po(oks)
P(s)
reward f(g)
M-step
gi Mo(
rationale
melecntle
Update
Molocule completion
Overview of EM-like evolution-by-explaaadee algorithm. Left: climbing up the energy landscape J(0, p())) by alternatively taking an E-step and M-step. Right: illustrations for the E-step and M-step. In the E-step of time t, draw samples from ((p())16'=1) to approximate Q()lp'(s)) using rationales extracted from the seed molecules via an explainable model. Then the M-step, optimize Q(Olp' w.r.t 0, i.e. pushing the graph completion model pa(-|s) towards generating higher scoring molecules conditioned on the rationale samples.
Figure 1:
rate by 50%, novelty by 14%, while having competitive diversity. We further propose metric, QNU score, to jointly consider all three aspects, and show that we achieve score of 52.7% compared with 29.5% by the best bascline. We also ask experienced chemists to evaluate top-50 generated molecules and find that 30 of them are as good as existing ones.
The main contributions of this paper are summarized below:
We propose novel EM-like vooltiin--y-eepplnnal algorithm for molecule optimization; We present a novel, principled. explainable graph model based on an cetemaaiion-thhoreee ap- proach to extract subgraphs essential for maintaining certain desired properties;
Our approach outperforms existing state-of-the-art by large margin in terms of success rate (50% better), novelty (14% better), and an overall metric (79% better) on a real-world multi- property optimization task.
RELATED WORK
There has been a surge of interest using machine learning to discover novel molecules with cer- tain properties in recent years. Most of the existing work defines generative model for either the
SMILES strings (Weininger 1988) or molecular graphs, and uses Reinforcement Learning algo-
rithms to optimize the properties of the generated molecules (Segler et al.) 2018 Olivecrona al.
2017 Guimaraes et al. 2017: You et al. 2018a [Popova et al. 2018 2019; [Samanta et al.) 2019
Zhou al.) 2019 De Cao & Kipt 2018 Kearnes al. 2019: Shi et al. 2020 Jin al. 2020)
Others optimize the continuous representation of molecules in latent space learned by variants of
variational autoencoders (Kusner et al., 2017, Dai al. 2018b Jin al. 2018 Gomez-Bombarelll
et al. 2018 Kang & |018
2018
Liu et al. 2018 Kajino 2011).. More recent work attempts Evolu- 2020; Leguy et al. (2020) Winter
(219), or focuses on finding high-quality molecules with synthesis paths (Bradshaw et al. 2019 Korovina et al. 2020 Gottipati jet al. 2020). Most similar to our approach RationaleRI. (Jin et al.] (020), which extracts sub- graphs from seed molecules using Monte Carlo Tree Search (MCTS) and generates full molecules by completing the subgraphs. Compared with previous work, our approach is the first to incorporate explainable model in the iterative search process.
tionary algorithms (Nigam al. 2020
Existing work on explainable models approaches the problems from three directions. The first line of work gradients of the outputs with respect to inputs identify the salient features in the inputs (Simonyan et al.) 2013) Springenberg et al., 2014, Bachrens et al.) 2010); the second line of work approximates the model with simple interpretable models, such locally additive mod-
2

---

Published as a conference paper at ICLR 2021
els (Bach al. 2015 Kindermans et al. 2016 Ribeiro et al. 2016 Lundberg & Lee 2017; Shriku- mar et al.] 2017); the third line of work defines input pattern selection operators, such that the outputs of the model based on the selected imput patterns have high mutual information with the original model outputs (Chen et al. 2018: Ying et al. (019). Our explainable model is different from GNNExplainer (Ying et 2019) that optimize the discrete subgraph structure with learned variational predictor, instead of directly feeding continuous edge masking into the target model.
PROBLEM SETTING
In this paper, study the problem of discovering molecules 9 from the molecular space 9 with high property score, measured by a scoring function f. And usually, there is a set of seed molecules Go C C from experts with high scores start with. More formally, the problem can be stated
Molecule Optimization. Given a scoring function f g - [0, 1], and set of seed molecules Go C 9. the goal to learn a molecule generative model p(g) such that the expected score of the generated molecules is maximized, i.e.,
- JS geg
max p()
(g)dg
(1)
To prevent the model p(g) from generating a small set of fixed molecules with high scores, we additionally require the learned distribution to be both novel and diverse, i.e., generating molecules that are dissimilar to the set of reference molecules (a subset of Go) and each other.
The molecule optimization problem in Eq 1 is combinatorial in nature, which poses significant challenge. To mimic the scientific discovery process, we allow the algorithm to query / on new molecules under querying budget. Examples of some well-known scoring functions include the QED score measuring the drug-likeness (Bickerton et al.] 2012), the SA score measuring the syn- thetic accessibility (Ertl & Schuftenhauer 2009)), the TPSA score measuring the ability permeate cells (Prasanna & Doerksen|[2009) etc. The scoring function is general and could also encode multi- property objectives (Olivecrona et al.) 2017 Brown al.) 2019). Optimizing multiple properties together suffers from the sparsity of high scores, a scenario which is shown to be more challenging than single property optimization (Jin et al. 2020).
When experts are optimizing the molecular property, they will first look for substructures that result in the formation of that property, and use them as the foundation for building novel molecules. These The set of rationales is formally defined as, s.t. s a subgraph of g}.
subgraphs are called rationales (examples in Figure [
S (s 3g 9,
OUR FRAMEWORK
Our novel framework for optimizing molecular property with generative models consists of a mod- eling component and an algorithm component. In our modeling component, we propose a rationale- based hierarchical generative model for p(g), which first generates rationales and then completes molecules. In our algorithm component, we design an alternating optimization procedure that in- terleaves between rationale distribution optimization and molecule generative model optimization. Furthermore, we develop a novel explainable graph model to effectively carry out the rationale model optimization. Next, we will first start describing our hierarchical generative model.
4.1 RATIONALE BASED HIERARCHICAL GENERATIVE MODEL
To tackle the challenging search problem, develop hierarchical generative model that mimics the process of molecule optimization by human experts. In our model, we first sample rationales s from a distribution p(s), and then molecules g will be generated according conditional distribution pg])).). More specifically, our overall molecular generative model po(g) can be defined as
/
pa(g) (g)
p(s) pe(g]s) ds,
(3)
J.CS
3

---

Published as a conference paper at ICLR 2021
where 0 is the parameter of the conditional generative model, p(s) is the latent rationales distribution.
Here po(g)) is a graph completion model from rationale s. The architecture of po(gls) can be arbitrary. In this work, we use latent variable model with Gaussian prior p(z),
Po (o(o)s) = - p(=)po(ols,
(4)
where pe (g) () is a variant of the GraphRNN You al. 2018b Liu et al. 2018) by condition- ing the graph generation on subgraphs. As part the initialization, pe (g $) Is first pretrained on
ChEMBL (Gaulton et al.] [2017), like molecule dataset, in the same fashion the variational autoencoder (Kingma & Welling 2013), where the encoder is a standard GCN with atoms as vertices and bonds edges.
Note that different from p(=). which is a fixed prior, p(s) will be updated in each round. And since representing distribution on S is difficult, will use particles to represent p(s) in the algorithm.
In order to improve the diversity of the generated molecules, we will also regularize the entropy of the rationale distribution p(s), leading the following
objective function
[(0, p(s)) E (f) + X Hip(s)),
(5)
with hyperparameter À 0 controlling the strength of the regularization.
4.2 ALTERNATING OPTIMIZATION ALGORITHM
As the rationales and the molecules are coupled in the molecular graph space, directly optimizing the diversity prrersity-roomoting objective in Eq (5) would be challenging. Therefore we seek to optimize pa (g())) and p(s) in an alternating fashion, akin to the Expectaion-MMxximizttin (EM) algorithm. That is, the algorithm alternates between two stages:
Expectation step (E-step) for obtaining an updated distribution p(s), and Maximization step (M-step) for improving the molecule completion model po(g]s).
We name this algorithm Mo Evoo (Algorithm I by making an analogy to evolving a group of molecules over time (Figure D. Assume that, iteration t 1, we already have (g]s) and p² (s), and the set of seed samples G' drawn from p (gls). Then, at iteration t. we have,
E-step. We want maximize the objective J with respect the latent distribution p(s) given Pgr-1 1p--i(gls) s). That is
/ p(s) ( Po- (oll) xf p(s) log ds. (6) JEs
max A(p(s)10*) p(a)
gEG
which is a maximum entropy estimation problem. Interestingly, the solution of the above optimiza- tion problem can be obtained in close form.
1 p (s) argmax Q(p(s)10=-)= - Zo p(s)
exp (
where Zo is a normalizing constant. This updated distribution for the latent rationales will be needed for the later M-step. However, since directly integrating with respect to p²(s) difficult, we will leverage sampling strategies and obtain particles {s; Jm from this distribution for later use in M-step. However, computing the normalizing constant Ze is difficult, making direct sampling from (s(s) not straightforward. Standard sampling algorithms like Markov Chain Monte Carlo (Andrieu et al. 2003) could be extremely slow due to the lack of a good proposal distribution and the absence of gradients in the discrete graph space.
To address this challenge, we will maintain a finite support set St as the proposal, which is obtained from explainable graph model (more details in the next section). More specifically, suppose the explainable graph model, Explain(.) G + S, can take a graph input & and output the corresponding rationale which explains why the graph g can obtain a high property score according to [g). Then support set St can be maintained as follows
St Ü :x E=
: g
(8)
4

---

Published as a conference paper at ICLR 2021
where Go is provided to the algorithm initially by experts. The rationales S* will be treated as the set of particle locations for representing p (s). Furthermore, for each of these particle locations, will compute its unnormalized probability according to p'( in Eq (7). and then re-sample set of m particles, {& >>>=>> as the final representation for p (s) (Andrieu et al. 2003).
M-step. With {s]] from p'(s), the Monte Carlo estimate of the objective function in Eq 5 becomes
Q(0Pp((s))
Σ pa pe + constant.
We can then maximize with respect to the parameters A using REINFORCE,
m Σ m
0r - 0-1
log (9.1),,, where a > 0, 91 ~ pa-- (-18)).
(10)
After the parameter is updated to 0", we will sample seed set of molecules g from pa (gls) by completing the rationale samples { using the updated model. That is
1h G' = (9: 1 where 9s ~ g (-Is), ~ Uniform((ss , 8m
(11)
The overall algorithm is summarized in Algorithm ! p(s) and po(g|s) are updated in the E-step (line 3-4) and M-step (line 5-8), respectively. A discussion on its convergence can be found in Appendix A.2
Algorithm 1: Molecule Optimization by Explainable Evolution (MolEvol)
Input: Seed molecules go, pretrained graph completion model po(g))) on ChEMBL Initialize 5 = U)
for t 6 to N Nrounds do S 5 (Explaingg) : E G'-1).
Sample 81, $2. 8m from St using Eq for < to Nepochs do
with saff-normalization.
Sample 91. Im from pe (g)
po (olg (m.) respectively.
Update 0 with REINFORCE (Eq 100).
Sample seed molecules G. with Eq
return (g(g)
4.3 EXPLAINABLE GRAPH MODEL FOR RATIONALES
In the E-step, it is crucial to update the support set for the rationales, such that the particles can be placed in spaces where p'(s) is large. As optimize p& (g]s). this model can generate molecules with improved property scores. Intuitively, we would also like to have an increasingly "good" sup- port set for the rationales. To do this, we will identify substructures in the current seed set of molecules G which can best explain their high property scores, and add these discovered substruc- tures as new rationales. Furthermore, we can measure the goodness of these substructure using their mutual information with the property value, and optimize the selector for these substructures using a variational formulation (Chen et al. (2018). This entire procedure is illustrated Figure Jea also be seen as seeking explanations why molecules have high property scores.
Explainer. molecular graph g represented by g = (Vg. Eg) with atoms Vg as vertices and bonds Ey as edges. For any subset 2/ S Vg. the induced subgraph by the vertices U and the edges gut = {e E E Estart; Cand E U}} connecting pairs of vertices in the subset. An explainer Expl explaing ain)))))))))) G + takes a graph g as an input, and outputs an induced subgraph of k vertices.
(U,)))) is a subgraph g formed
Variational Objective. We want learn an explainer for the conditional distribution P(Y 1lg) A fg) (treating f (g) as a probability), with random variables Y e (0, 1} where Y indicates that the molecule has the property, and otherwise. We will learn graph vertex sampler hg(g) jointly
5

---

Published as a conference paper at ICLR 2021
Vestox sompling polley Ky(s)
During training
Figure 2: Steps of the explaining pro- cess (Alg. 2 The explainer a sub- graph selector containing two steps. First, a vertex sampling policy ho(g) is computed (top-left). Then & vertices are selected using the Gumbel-softmax trick (bottom-left). During training, the embeddings on the selected ver- tices are pooled together and fed into MLP 9u which predicts the prop- erty score (top-right). During explain- ing, The induced subgraph of the se- lected vertices and their neighbors extracted as the predicted rationale.
Perform SGD loss ylog 8 log(1 9)
Gembel-ooltuas
During explaining
000
800
Extracted rationale
Include neighteor
with a variational approximation Qg of (9) such that the mutual information between Y and is maximized
max Ey-P(-10) log Q( })] such that & = (4, 8 and U ~ ha (g). h2(-).2
(12)
Details on sampling from hg(g)) are presented in the next paragraph. After sampling U, we can construct induced subgraph (u, E During the explanation, we then perform an additional expanding step,
8' = (LL', ( where ut = u to E EU, s.t. e(u, v) E &g or u share a Benzene.), (13)
to obtain s', which defines the mapping s' Explain(g) (Algorithm 2).
Parameterization of (g). Sampling subgraph from is equivalent sampling size-k subset from the vertices Vg. We use GNN he to define a vertex sampling policy ho(g) E A over the space g's vertices. Specifically, (g) consists of two parts:
A message passing network (MPN) which outputs a matrix X, = MPN
R V
representing the d-dimensional embeddings for each vertex;
2. A fully-com calnecco layer (FC) followed by a softmax layer to implement the vertex sampling policy (g) Softmax(FC, (X,)) Apyl-
Then we follow the procedure L2X (Chen et al. 2018) for sampling & vertices one-by-one from the distribution (g) using the Gumbel-softmax trick. The sampled feature matrix can be written X, Vád () O X, I where is a collection of auxiliary random variables sampled independently from the Gumbel distribution, ((b. e {0, 13 VV,l is a mask on the rows of Xg. and @ is the element- wise product.
Parameterization of Q. Since directly using generic choices of Q to perform the variational ap- proximation is hard, we approximate it with a MLP qu that takes the aggregated masked embedding vector x, X.[r.] E Id as input, and predicts the target via Q(Y = 1|s) q (X,) = Sigmoid (x..))).))).
Final Objective for Training. After applying the Gumbel-softmax trick, we transform the varia- tional objective in Eq [2)
into:
Egg ((0) log q. (V() X,) + f(g)) log(1 (V x,))]_
max 0,0
(14)
We can then apply stochastic gradient ascent jointly optimize & and 1 by sampling molecule from the dataset and Ç from the Gumbel distribution. Please refer to Appendix A. for more details of the training procedures as well the implementation of the explainer.
We note that our design of the explainer model and the learning method is very different from those in GNNExplainer (Ying al., 2019). which may be of independent interest in terms of explainable models for GNNs. For instance, our explainable model h by itself is a GNN model, and we introduce a variational distribution qo which optimized jointly with hgg

---

Published as a conference paper at ICLR 2021
Table I: Results on multi-property molecule optimization. Mo Evol is compared with three vari-
ants and four baselines in terms of success rate, novelty, diversity and an overall metric (QNU). The diversity of MSO and GA-D(t) is not reported here due their extremely low novelty scores.
Algorithm
MolEvol
[MCTS]
[FixM]
[FixR]
RationaleRL REINVENT MSO GA-D(t)
Success rate
93.0%
77.7%
67.3%
66.3%
61.1%
46.6%
57.7%
62.0%
Novelty 75.7%
72.5%
67.4%
54.6%
57.4%
66.4%
28.6%
19.4%
Diversity 0.681
0.707
0.723
0.727
0.749
0.666
QNU
52.7%
47.4%
39.3%
28.3%
29.5%
7.4%
16.4%
12.0%
Algorithm 2: Explaing(9)
Rationale Extraction as Explaining. During the E-step our Algorithm [ we utilize the trained explainer Explay a an)) to extract ratio- nales candidates s from the seed molecules. Then the candidates with the top Q-scores are added to the rationale support set to update St.
Input: Molecule g. vertex sampling policy ø. (g)
Sample U hg(g) with Gumbel-softme trick. (xp E)) )) as defined Eq 13
return
Remark on Explanation. In this paper, we use the word "explanation" refer critical com- ponent of the imput that is of most importance for the final prediction, following the convention of L2X (Chnn al. 2018) and GNNExplainer (Ying al., 2019). However, more rigorous expla- nation using scientific language is rather important and helpful for scientific research. Generating such an explanation using machine learning model could be highly relevant general, but that is beyond the scope of this paper.
EXPERIMENTS
We evaluate MolEvol multi colii-property molecule optimization task (Li et al. 2020) involving four properties:
2018 Jin et al.
GSK-333 inhibition levels against glycogen synthase kinase-3 beta (L et al.
2018
JNK3: inhibition levels against c-Jun N-terminal kinase 3 (Li et al.
2018):
QED: quantitative estimate of drug-likenes, (Bickerton al. 2012);
SA: synthetic accessibility (Ertl & Schuffenhauer/ 2009).
GSK-3/ and JNK3 potential targets in the treatment of Alzheimer's disease. Their corresponding property predictors are random forests trained on real-world experimental data using Morgan finger- print features (Rogers & Hahn 2010). our experiment, we consider all properties by combining their scores into unilied scoring function!
f(g) GSK-33(g)) JNK3(g) QED(g) SA(g))
(15)
Note that in the eMolecules dataset (eMolecules 2020) of commercially available molecules, only 0.007% out of over 27M molecules meet the criteria with f(g) > 0.5.
Experiment Setting. We provide a set of 3. 3.4K seed molecules for the algorithms to start with. Each seed molecule has a high value in GSK-33 or JNK3 or both. There is a budget on both the time and the number of queries. Each algorithm is allowed to query f-scores no more than 5M times and to run no more than day on a Ubuntu 16.04.6 LTS server with Nvidia RTX 2080 Ti GPU, and 20 Intel(R) Xcon(R) E5-2678 2.50GHz CPUs. We evaluate the algorithms 20K generated molccules using the following metrics. We call a molecule qualified if f(g) > 0.5, novel the distance between g and the reference molecule contains 315 qualified molecules, which is a subset of the provided seed molecules.
larger than threshold The reference set
Success rate: the percentage of qualified molecules out of 20K molecules.
'The range of GSK-38, JNK3, QED are [o, 1). We re-normalize SA to [0. 1] using SA(g) <- ( 1). 'Novel(g) = I(maxyreg.., Sim(g, g') < 0.4), Diversity = w(-- Egsgr Sim(g.g') g'), Sim(-,) the Tanimotoosimilarriy on Morgan fingerprints.

---

Published as a conference paper at ICLR 2021
Novelty: the percentage of novel molecules out of all qualified molecules.
Diversity: the average pairwise distance between all qualified and novel molecules'.
QNU score: the percentage of qualified, novel and unique molecules out of 20K molecules.
Success rate, novelty and diversity have been adopted as evaluation metrics in previous work (Olve- crona et al. 2017 Li et al., 2018 Jin et al. 2020). However, the trade-off among the three targets complicates the comparisons between algorithms. Therefore we propose a new metric, QNU score, to jointly consider the three aspects. QNU will serve as the major factor for comparison.
Implementing MolEvol. We first pretrain the graph completion model po(g]s) on a dataset con- structed from ChEMBL (Gaulton al. 2017), which contains over 1.4M drug-like molecules. The pretraining dataset consists of 4.2M (s.g) tuples, where is random molecule from ChEMBL and s random subgraph of g. In our experiment, MolEvol run for 10 rounds. Within each round, 200 rationales are added to the support set during the explainable local search stage. During the local search stage, to atoms will be sampled according to the vertex sampling policy (g) and we include the neighbors of the sampled atoms, i.e., the atoms which share a common bond to the sampled atoms, to form the rationale (Eq (3)). In the molecule completion stage, the parameter 0 is updated with gradient descent for epoch using a total number of 20000 (s, g) pairs with minibatch size of 10 and a learning rate of le-3.
Baselines. We compare Mo 1Evol against state-of-the molecule optimization algorithms below:
RationaleRL (Jin et al. 2020) learns a graph completion model, but relies on a fixed set multi- property rationales composed by single-property rationales extracted by MCTS. Concretely, each state in MCTS represents a subgraph of the molecule and the reward function is defined as the property score of the subgraph.
REINVENT (Olivecrona et al., 2017) learns a RNN model with Reinforcement Learning for generating molecules in the form of SMILES strings;
MSO (Winter et al.) 2019) optimizes the property using Particle Swarm Optimization (PSO) (Kennedy & Eberhart 1995) in continuous latent space of molecules.
GA-D(() (Nigam et al.) 2020) employs genetic algorithm enhanced with neural network based discriminator component promote diversity. The discriminator used here tries to distinguish between molecules generated by the GA and the reference molecule set. The time-dependent adaptive penalty also used for further promoting exploration.
Since MSO and GA-D(t) do not explicitly learn generative model, we use the best 20K out of 5M molecules encountered in the search process for comparison.
Results. The results are reported in Table [ Comparing to the baselines, MolEvol achieves higher success rate in generating qualified molecules (30% higher than RationaleRL, MSO and GA-D(t), 45% higher than REINVENT), Meanwhile, MolEvo maintains high novelty (75.7%) which may benefit from the alternating process in the framework. Although the diversity is slightly lower than RationaleRL due to the distribution shift during optimization, the QNU score, which takes all the above metrics into consideration, is significantly higher than RationaleRI. (52.7% versus 29.5%). Please refer Appendix A..3 for more discussions.
Ablation Studies. We introduce baselines below to understand the importance of each component:
[MCTS] replaces the explainable local search with MCTS in Jn et al. 2200);
[FixR] uses a fixed set of rationales, i.e. only having one round of explainable local search;
[FixM] uses fixed (pretrained) model, i.e. having no molecule completion stage.
As illustrated in Table D Mo 1 Evol achieves the highest QNU score among all variants. The large performance gap (success rate: 93.0% vs. 67.3% QNU score: 52.7% vs. 39.3%/28.3%% between MolEvol and [FixM]/[FixR] justifies the necessity of taking both E-step and M-step into consideration. Compared with [MCTS], the 5% QNU increase may result from the larger space when doing the local search, while MCTS only proposes connected subgraphs of molecules as rationales.
Distribution of the Generated Molecules. In Figure Ieff we plot the evolution of the generative model performance over time. As we can see, the distribution gradually shifts to regions with higher property scores, which demonstrates that Mo MolEvol does improve the molecule generative model via EM iteration. As shown in Figure right, MolEvol can propose molecules with improved
8

---

Published as a conference paper at ICLR 2021
1,9
10
12
Reference
0.6
0.4
0.0
0
0.35 0.45
0.50
JN33 Property
Figure 3: Property score distribution of the generated molecules. Left: the evolution of the f-score distribution of Mo Evol over the number of iterations. Right: the distribution of four property scores of our generated molecules, the ground truth molecules in the reference set, and the molecules in ChEMBL. The higher the better for JNK3/GSK-3///ED,, the lower the better for SA.
QED and SA compared to molecules in ChEMBL and the reference set. The distribution for the property scores of molecules generated by MolEvol is more compact than others, which suggests that Mo Evol propose molecules with high property score and low score variance.
Example of Rationale/Generated Molecule. Figure 4 gives an example of molecules generated by some rationale discovered using MolEvol. The molecules of high diversity and pertain consistently high level of scores, which proves MolEvol 's superiority.
Expert Evaluation. We asked experienced chemist to evaluate generated molecules. The top-scoring 50 molecules from MolEvol and ChEMBL are selected, shuffled, and grouped with one another to construct 50 pairs. Given a pair of molecules, the chemist is asked to provide a comparative score in each of the four criteria. For the sum four scores, 30/50 molecules by Mo Mo1Evo higher or identical compared to their counterparts from ChEMBL. For individual scores, 7/50 molecules by MclEvol are all higher identical compared to their counterparts. This result shows that our algorithm can propose high-quality realistic molecules that compet- itive with existing ones. Please refer to Appendix A.4 for more details.
Missi
Mileou
Figure 4: An example of rationale and corresponding generated molecules with f-scores.
6 DISCUSSION
In this paper, we proposed an EM-like algorithm for optimizing molecules by an explainable evolu- tionary process. Although we focus paper and evaluation on molecule design, the method can be generically applied for optimizing discrete structures in other structured prediction domains like program synthesis (Ellis al. 2020 and graph adversarial learning (Dai al. 2018a Zügner al.) 2018). Our method mimics humans' general design process for discrete structures by first identify- ing useful structural elements and then improving the design based these elements. The process of discovering more useful substructures and then reiterating the design is carried on to gradually improve the final product. Furthermore, the explainable graph model developed in the paper can be applied to other general graph problems as well. We believe multiple aspects of our method have broader applications beyond the current molecule optimization problem.
ACKNOWLEDGMENTS
This work is supported in part by NSF grants CDS&E-1900017 D3SC, CCF-1836936 FMitF, IIS- 1841351, CAREER 115-1350983, CNS-1704701, ONR MURI grant L.S.

---

Published as a conference paper at ICLR 2021
REFERENCES
Christophe Andrieu, Nando De Freitas, Arnaud Doucet, and Michael Jordan. An introduction to meme for machine learning. Machine learning, 50(1-2):5 43, 2003.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10())e01331400 2015.
David Bachrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus- Robert MÅžller. How to explain individual classification decisions. Journal of Machine Learning Research, (IJum)):1003-
2010.
G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90 2012.
John Bradshaw, Brooks Paige, Matt J Kusner, Marwin Segler, and José Miguel Hemánneez-Lobato. A model to search for synthesizable molecules. In Advances in Neural Information Processing Systems, pp. 7937-7949, 2019.
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3): 1096-1108, 2019.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning explain: An iinommation-thhorttti perspective on model interpretation. In International Conference on Ma- chine Learning, pp. 883-892, 2018.
Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang. Jun Zhu, and Le Song. Adversarial attack on graph structured data. arXiv preprint arXiv: (eo::0207677777 2018a.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoen- coder for structured data. arXiv preprint 1802.0888 2018b.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs. arXiv preprint 1105: 2018
Kevin Ellis, Catherine Wong. Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lucas Morales, Luke Hewitt, Armando Solar-Lezama, and Joshua B Tenenbaum. Dreamcoder: Growing gen- eralizable, interpretable knowledge with wake-sleep bayesian program learning. arXiv preprint arXiv:2006.0838) 2020.
eMolecules. eMolecules Dataset.
httpp::/doww www.dowwwwoaa...oooooooooo emol eeueelllle malles.co www. frem/free/ 2020 09-0009-000 1000--01veeiioio smi. gz. 2020. [Online; accessed 01-Scp-2020J.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminfor- matics, 1(1):8, 2009.
Anna Gaulton, Anne Hersey, Michal Nowotka, A Patricia Bento, Jon Chambers, David Mendez, Prudence Mutowo, Francis Atkinson, Louisa J Bellis, Elena Choritn-Uhaate,, et The chembl database in 2017. Nucleic acids research, 45(D1):D945-DD)) 2017.
Rafael G66mez-Bombarele Jennifer Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling Dennis Sheberla, Jorge Aguilera-lparraguiree Timothy D Hirzel, Ryan Adams, and Alán Aspuru-Guzik. Automatic chemical design using data-driven contin- representation of molecules. ACS central science, 4(2):268 276, 2018.
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Karam MJ Thomas, Simon Blackburn, Connor W Coley, Jian Tang, al. Learning navi- gate the synthetically accessible chemical space using reinforcement learning. arXiv preprint arXiv:2004. /2485, 2020.
10

---

Published as a conference paper at ICLR 2021
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Sspuru-Guzik Objective- reinforced generative adversarial networks (organ) for se- quence generation models. arXiv preprint 1rXiv:1705.10843 2017.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junetion tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning, pp. 2323-2332, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Composing molecules with multiple property constraints. arXiv preprint arXiv:2002.03244, 2020.
Hiroshi Kajino. Molecular hypergraph grammar with its application to molecular optimization. In International Conference Machine Learning, pp. 3183-3191. PMLR, 2019.
Seokho Kang and Kyunghyun Cho. Conditional molecular design with deep generative models. Journal of chemical information and modeling, 59(1):43-52, 2018.
Steven Kearnes, Li Li, and Patrick Riley. Decoding molecular graph embeddings with reinforcement learning. arXiv preprint arXiv:1904.000 1904.0995,,,, 2019.
James Kennedy and Russell Eberhart. Particle swarm optimization. In Proceedings of ICNN'95- International Conference on Neural Nenworks, volume 4, pp. 1942-1948, IEEE, 1995.
Pieter-Jan Kindermans, Kristof Schüit,, Klaus-Robert Müller, and Sven Dähne. Investigating the influence of noise and distractors the interpretation of neural networks. arXiv preprint arXiv:1611.07270, 2016.
Diederik Kingma and Max Welling, Auto-encoding variational bayes. arXiv:1312.6114, 2013.
arXiv preprint
Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger. Barnabas Poczos, Jeff Schneider, and Eric Xing. Chembo: Bayesian optimization of small organic molccules with syn- thesizable recommendations. In International Conference on Artificial Intelligence and Statistics, pp. 3393-3403, 2020.
Matt J Kusner, Brooks Paige, and José Miguel Hernández-Lobato. Grammar variational autoen- coder. arXiv preprint arXiv: 1703.070 2017
Jules Leguy, Thomas Cauchy, Marta Glavatskikh, Béatrice Duval, and Benoit Da Mota. Evomol: a flexible and interpretable evolutionary algorithm for unbiased de novo molecular generation. Journal of Cheminformatics, 12(1):1-19) 2020.
Yibo Li, Liangren Zhang, and Zhenming Liu. Multi-objective de drug design with conditional graph generative model. Journal of cheminformatics, 10(1):33, 2018.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt. Constrained graph varia- tional autoencoders for molecule design. In Advances in neural information processing systems, pp. 7795-7804, 2018
Scott M Lundberg and Su-In Lee. unified approach to interpreting model predictions. pp. 4768- 4777, 2017.
Laurens der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-2)) 2605, 2008.
AkshatKumar Nigam, Pascal Friederich, Mario Krenn, and Alan Aspuru-Guzik. Augmenting ge- netic algorithms with deep neural networks for exploring the chemical space. In International Conference on Learning Representations, 2020. UR//tttps://opereviiee
id=H1 1IY
Mareus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):48, 2017.
Mariya Popova, Olexandr Isayev, and Alexander Tropsha. Deep reinforcement learning for de novo drug design. Science advances, 4(7)ceaap7885, 2018.

---

Published as a conference paper at ICLR 2021
Mariya Popova, Mykhailo Shvets, Junier Oliva, and Olexandr Isayev. Molecularmn: Generating realistic molecular graphs with optimized properties. arXiv preprint arXiv:190S.13372. 2019.
S Prasanna and RJ Doerksen. Topological polar surface area: a useful descriptor in 2d-qsar. Current medicinal chemistry, 16(1):21-41, 2009.
Jean-Louis Reymond, Ruud Van Deursen, Lorenz C Blum, and Lars Ruddigkeit. Chemical space as source for new drugs. MedChemConm, 1(1):00-38, 2010.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 1135- 1154. ACM, 2016.
David Rogers and Mathew Hahn. Extended-comnecttvig fingerprints. Journal of chemical informa-
tion and modeling, 50(5):742-75, 2010.
Bidisha Samanta, DE Abir, Gourhari Jana, Pratim Kumar Chattaraj, Niloy Ganguly, and Manuel Gomez Rodriguez. Nevae: A deep generative model for molecular graphs. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1110-1117, 2019.
Marwin HS Segler, Thierry Kogej. Christian Tyrchan, and Mark P Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1): 120-131, 2018.
Chence Shi, Minkai Xu, Zhaocheng Zhu Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09382. 2020.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In ICML, volume 70 of Proceedings of Machine Learning Research, pp. 3145-3153, PMLR, 06-11 Aug 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint arXiv:/312.6033, 2013.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint crXiv:1412.6806 2014.
David Weininger. Smiles, chemical language and information system. 1. introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-33) 1988.
Robin Winter, Floriane Montanari, Andreas Steffen, Hans Briem, Frank Noé, and Djork-Amé Clev- ert. Efficient multi-objective molecular optimization in a continuous latent space. Chemical science, 10(3))8816-80024 2019.
Zhitao Ying. Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In Advances in neural information processing systems, pp. 9244-9255, 2019.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional pol- icy network for goal-directed molecular graph generation. In Advances in neural information processing systems, pp. 14101 2018a.
Jiaxuan You, Rex Ying, Xiang Ren, William L Hamilton, and Jure Leskovec. Graphran Generating realistic graphs with deep auto-regressive models. arXiv preprint arXiv:1802.08777. 2018b.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1):1-10, 2019.
Daniel Zügner, Amir Akbarejad, and Stephan Günnemann. Adversarial attacks neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2847-2856, 2018.
12

---

Published as a conference paper at ICLR 2021
A APPENDIX
A.1 EXPLAINER IMPLEMENTATION AND TRAINING
Here provide the details for implementing the graph explainer described in Section 4.3
For the MPN hi in the explainer, we use a two-layer GCN with the embedding size of each layer equaling 32. The GCN input is node embedding derived by Embedding Layer that embeds each node (atom) within the graph (molecule) into a 228-dimensional vector according to its type. The FC layer following MPN outputs I-dimensional vector which is used for Gumbel Softmax.
For the MLP of de in the explainer, we use a two-layer FC network to emhed the information, each of whose hidden dimension equals 200. We add batchnorm layer after each FC layer make the training phase more stable. After that, a sigmoid layer is used to get the final prediction.
Training procedures are described below.
Algorithm 3: Training Procedures for the Explainer.
Input: Molecules dataset D with each pair (g, y) denoting molecule and label, initial vertex sampling policy network Φ, MLP network & for approximating Q
for t - to eepochs do
Sample 91.".. 9m from D.
for f- 1 to do
X MPN (g:) IR Vu,
X () X E R 1,,xd, where Ç Gumbel 1).
f = ge (X
Update & and vb using gradient ascent by maximizing f(:) log di + (1 (((()) log(1 -e)).
return
A.2 CONVERGENCE ANALYSIS OF MOLEVOL
From a theoretical standpoint, here we assume 1) we use the true support set instead of the finite support set S, in (Eq. [8. and 2) and m in (Eq. I0 carefully selected such that the gradient update has small enough variance.
Proof
As ((0,((s)) has an upper bound, we only need to show that is non-decreasing over step and M step.
step: we need to show that J(8", (s)) J(O', p (s)). obvious with assumption
1) as (Eq.6) has the closed form solution (Eq. (). the updated value of J is the maximum after the argmax operation.
M-step: need to show that J(0', ps)))) 2 J p())). First, it is worth noticing we used the same trick as in REINFORCE to get Eq. TO from Eq. g iee. we can do SGD with the gradient get in Eq. TO Then, with assumption 2), by doing SGD, the unbiased gradient estimator with small variance will always converge to non-decreasing result in the objective value.
By the above analysis, we can justify that this EM-like method can converge to local optimum.
Note that both assumptions rather mild, since for assumption 1), S, grows with time t and gradually converges to S, and for assumption 2), large enough m and small enough a should suffice.
As will be discussed later, the plot (Figure 5) of the final objective's convergence curve justifies that algorithm can converge empirically.
13

---

Published as a conference paper at ICLR 2021
MolEval Reference
MolEvol
4.0
2
8
Round
Figure 5: t-SNE plot and the objective value over time. Left: the t-SNE plot of the generated molecules distribution from MolEvol and the reference molecules. Right: The diversity-promoting objective (Eq (5) over time.
A.3 MORE EXPERIMENT RESULTS AND DISCUSSIONS
Molecule Distribution. We projected the generated molecules onto a two-dimensional space by SNE (Matten & Hinton 2008) together with the reference molecules in Fig The molecules generated by MolEvol occupy the chemical space expanded by the reference molecules and their neighboring regions.
Optimization Objective. We plot the value of J(0) Eq (5) during training. As can seen in Figure right, the value of objective J(Ø) is consistently improved, which shows that MolEvol does help to optimize the diversity-promoting objective in an alternating fashion.
Analysis of Baselines. The main reason for MSO's low performance is that it produced molecules with relatively low diversity, so most queries were wasted for evaluating highly similar molecules. Therefore MSO is not well suitable for producing high-scoring molecules with high diversity since there is no regularization for the diversity of molecules it generates. GA-D(t) incorporates the dis- crimination score to promote unseen molecule generation. However, there is no guarantee that the generated molecules are dissimilar enough to be deemed novel, thus leading to the degradation of overall performance. In comparison with them, REINVENT and RationaleRL resort REIN- FORCE for optimization, and achieve more competitive performance. Nevertheless, RationaleRL. generates molecules from rationales in one-shot, which does not take the insight that the generated molecules might contains some subgraphs (i.e. rationales) that are more qualified into consideration.
A.4 EXPERT EVALUATION EXPERIMENT
We provide more details on the setting the expert evaluation experiment. We first construct the evaluation molecule set by choosing 50 top-scoring molecules of the same size from our generative model and ChEMBL dataset. The molecules are then grouped into pairs such that each pair contains from the model and one from the dataset. The order of the two molecules in each pair is randomly shuffled. We then ask experts to evaluate these 50 pairs of molecule with respect to the four molecular properties, i.e., GSK-33, JNK-3, QED, SA. For each property, the experts will provide their opinions using one of the following choices:
1. The first molecule is clearly better;
2. The second one is clearly better;
3. The difference is minor and hard tell.
We the following two metrics to interpret the result.
[M-Single]: We score each molecule by summing over the results of all four criteria. molecule scores points each criterion if it is clearly better, point if the difference is hard to tell, and points is clearly worse. We found that 30 out of 50 generated molecules have better or equivalent scores than its counterpart.
14

---

Published as a conference paper at ICLR 2021
[M-Overall]] We count the number of pairs where all four properties of the generated molecule are better than or equivalent to the ChEMBL counterpart. Within these pairs, we discard the ones if there is confident evaluation, i.e., the differences between the pair of molecules all four criteria are hard to tell. We found that out of 50 remains, meaning that 14% of all the generated molecules strictly better than their counterpart.
15


Reviews:
[{"review_id": "vKhgj_BtX2", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "SUMMARY:\n\nThe authors propose a method that \"explains\" molecular properties based on molecular fragments and call it Molecular Evolution. The subgraph \"explanations\" then are used to explore larger swaths of chemical space.\n\nPROS:\n\n- As far as the reviewer notes, this approach is novel in the (now increasingly crowded) set of alternatives for molecular generative models.\n- The authors have a model that compares favorably to the baselines\n- The authors use a very relevant set of optimization parameters for the multiobjective task.\n- The paper is well explained.\n\n\nCONS:\n- The reviewer believes that there is much more to explain why a molecule is better for a task than identifying a subgraph. This should be made clear in the manuscript as materials scientists want to know for example quantum properties of the fragment(s) and how they influence the given property to provide a valuable explanation.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "tsaoPfOyhG1", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper tackles the problem of molecule property optimisation. To this end, the authors proposes an alternating approach consisting of an explainer model and a molecule completion model. The explainer model takes a complete molecule as input and outputs a subgraph that represents the part that contributes most to property prediction. Then, the molecule completion model uses the subgraphs to sample a complete graph that can maximise the property scores. The loss function of molecule completion model directly maximises the properties, which is non-differentiable so that the authors use a REINFORCE algorithm for optimisation. \n\nPros:\n\n1. The paper proposes use subgraphs that contributes most to the property prediction for searching better molecules. The subgraphs are learned supervisedly through the signals from the property to be optimised. Compared to unconditional VAE models, this approach might be easier to optimise, since the subgraphs can serve as templates. \n\n2. The method is novel and the experiments demonstrate the effectiveness of the methods compared to previous methods.\n\n3. The paper is well-written and the idea is articulated in a formal description.\n\nCons (or questions):\n\n1. Some convergence analysis shall be needed, i.e. why this method will converge to the optimal values of the objective. The authors claim the method is an EM algorithm, and some proofs about convergence might be needed. Otherwise, some learning curves might be helpful, since the REINFORCE algorithm is known to suffer from high variances. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "bre1m2NHqjq", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a two step procedure for generating molecular graphs that optimize some desirable properties. The method consists of a rationale extraction phase, where the subgraph \"important\" for the desired property is identified and a graph completion step. \n\nWhen reading the paper for the first time, I found it a bit hard to follow the approach. The paper might be easier to read when the individual model components are introduced directly with the E- and M-step. Currently, the graph completion model is introduced in Sec 3.1, then the E- and M-steps are described in Sec. 3.2, while the \"explainer\" is only mentioned and referred to Sec. 3.3. An easier to follow structure might be: 1) Rationale extraction 2) Graph completion 3) E/M iteration.\nBeyond that, the terms \"evolution\" and \"explanation\" might be misleading here: The method is not an evolutionary algorithm, as the title might suggest. Even if subgraphs are extracted that lead to the generation of promising molecules in the graph completion step, this does not show that these subgraphs responsible. To warrant the term explaination, a  thorough analysis of the extracted rationales would be necessary, in particular since they might not even be connected graphs.\n\nThe evaluation of the method in Table 1 and Fig. 3 show that the method outperforms previous appraoches and is capable of shifting the generated distribution shifts to higher scores. The human evaluation is weak, since only a single expert was asked. A panel of experts with reported agreement among the panel would improve the paper. However, since the scores of the molecules are available, it does not become quite clear to me, how human experts benefit the performance evaluation under those same criteria.\n\nPros\n-----\n- Interesting approach to update the pool of rationales\n- Outperforms previous approaches, Fig 3. show that desired properties improve over rounds\n- Ablation study demonstrates improvement by proposed procedure\n\nCons\n----\n- Structure of the paper could be improved\n- The paper states that in the explainer a \"subgraph s of k vertices\" is extracted, therefore I assume the size of the rationale is fixed. This would severely restrict the space of rationales.\n- The notion of explainability is not sufficiently discussed in the paper and the claim(?) that the rationales are somehow meaningful is not examined.\n\nUpdate: I read the reply and thank the authors for the clarifications.", "extra_scores": {}, "extra_text": {}}, {"review_id": "HtEQzXdUw4", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Initial Review\n\nThe paper proposes a new algorithm for de-novo molecular design, which uses a model to extract explanatory subgraphs from a set of support molecules which \"explain\" high scores wrt a scoring function, and a generative model which is conditioned on these subgraphs to produce full molecules. \n \nAll in all, I think this is an interesting combination of several existing approaches in generative models for molecules (all referenced in the paper). The aspect of explainability is novel. The presentation of the paper is mostly clear. The approach is quite geared to molecule generation, but can potentially also inspire applications in other domains, which makes it interesting from a general ML perspective as well. \n \nI like the paper from the theoretical side, which alone warrants acceptance of the paper at ICLR in my opinion. \n \nHowever, I have a few minor concerns / comments: \n \nI am a bit on the fence with the validation method. Since almost every new paper in the field proposes a new validation approach, it has become pretty much impossible to assess what the state of the art of the field is (or if the concept of SOTA is even something meaningful), and this paper is no exception in this regard. But I assume the authors will disagree here. \n \nIn practice, generating 20k molecules is a lot. Looking at the statistics of the top100 molecules would probably be sufficient. \n \nAlso, I find it somewhat surprising that some of the baseline algorithms (in particular the Winter et al MSO model), which are less constrained than algorithm presented here, are not achieving higher scores, in particular when the algorithms can query the scoring function 5 M times. Maybe this is something the authors could comment on in the rebuttal. \n \nAs an additional baseline, I would suggest to report the \"best in dataset\", I.e. run the scoring function on the seeds and all molecules used to train the generator, and pick the top molecules. \n \n \nRelated work: \n \nI would suggest to additionally cite https://arxiv.org/abs/1701.01329 which was the first paper to apply neural models to molecule generation in drug discovery, and the first of such papers which has been prospectively validated in laboratory experiments by scientists not affiliated with the authors.\n\n### Update 1 after discussion:\nScore raised.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors appreciated this submission because (a) the aspect of explainability is novel, (b) its strong performance, (c) the clarity of the paper. I urge the authors to double check all of the reviewer comments to make sure they are all addressed in the updated version. I vote to accept.
Decision: Accept (Poster)