MARS: Markov Molecular Sampling for Multi-objective Drug Discovery
Authors: Yutong Xie
Chence Shi
Hao Zhou
Yuwei Yang
Weinan Zhang
Yong Yu
Lei Li
Venue: ICLR 2021 Spotlight (2021)
Keywords: drug discovery
molecular graph generation
MCMC sampling

Abstract: Searching for novel molecules with desired chemical properties is crucial in drug discovery. Existing work focuses on developing neural models to generate either molecular sequences or chemical graphs. However, it remains a big challenge to find novel and diverse compounds satisfying several properties. In this paper, we propose MARS, a method for multi-objective drug molecule discovery. MARS is based on the idea of generating the chemical candidates by iteratively editing fragments of molecular graphs. To search for high-quality candidates, it employs Markov chain Monte Carlo sampling (MCMC) on molecules with an annealing scheme and an adaptive proposal. To further improve sample efficiency, MARS uses a graph neural network (GNN) to represent and select candidate edits, where the GNN is trained on-the-fly with samples from MCMC. Experiments show that MARS achieves state-of-the-art performance in various multi-objective settings where molecular bio-activity, drug-likeness, and synthesizability are considered. Remarkably, in the most challenging setting where all four objectives are simultaneously optimized, our approach outperforms previous methods significantly in comprehensive evaluations. The code is available at https://github.com/yutxie/mars.

Published as a conference paper at ICLR 2021
MARS: MARKOV MOLECULAR SAMPLING FOR MULTI-OBJECTIVE DRUG DISCOVERY
Yutong Xie Chence ShitA, Hao Zhoule, Yuwei Yang', Weinan Zhangl, Yong Yu!, Lei Lif "ByteDance Al Lab, Shanghai, China University of Michigan, Ann Arbor, MI, USA
Montréal Institute of Learning Algorithms, Montreal, Canada
DDpaarment of Computer Science and Engineering, Shanghai Jiao Tong University, China
ABSTRACT
Searching for novel molecules with desired chemical properties is erucial in drug discovery. Existing work focuses on developing neural models to generate ei- ther molecular sequences or chemical graphs. However, it remains a big chal- lenge to find novel and diverse compounds satisfying several properties. In this paper, we propose MARS, a method for multi-ohjective drug molecule discov- ery. MARS is based on the idea of generating the chemical candidates by iter- atively editing fragments of molecular graphs. To search for high-quality candi- dates, it employs Markov chain Monte Carlo sampling (MCMC) molecules with an annealing scheme and an adaptive proposal. To further improve sam- ple efficiency, MARS uses a graph neural network (GNN) to represent and select candidate edits, where the GNN is trained on-the-fly with samples from MCMC. Experiments show that MARS achieves state-of-the-art performance in various multi-objective settings where molecular bio-activity, drug mikg-likeness, and synthe- sizability are considered. Remarkably, the most challenging setting where all four objectives are simultaneously optimized, our approach outperforms previ- ous methods significantly in comprehensive evaluations. The code is available https: /tttps:/ggithin com/ httmm/ytxxie//ee aettxieemaars.
INTRODUCTION
Drug discovery aims find chemical compounds with desired target properties, such high drug- likeness (Bickerton et al. 2012] QED). The problem is also referred to as molecular design, molec- ular generation, or molecular search. The space of drug-like chemicals is enormous, approximate 1033 for realistic drugs that could ever be synthesized (Polishchuk et al. 2013). Therefore it is very challenging to search for high-quality molecules from such vast space enumeration would take almost forever. For a particular disease, finding the right candidates targeting specific proteins further complicates the problem.
Instead of enumerating or searching from the immense chemical space. recent work utilizes deep generative models to generate candidate molecules directly (Schwalbe-Koda & Któmez-Bomannd 2020). However, most prior work focuses on generating molecules concerning single property such as drug-likeness (EE)) or octanol-water partition coefficient (logP) (Jin et al. 20188 You al., 2018 Popova et al.) 2019 Shi et al. 2020 Zang & Wang 2020). While in practical settings, typical drug discovery requires consideration of multiple properties jointly (Nicolaou et al. 2012). For example, to find drug-like molecules that easy to synthesize and exhibit high biological activity against the target protein. Naturally, multi-oojective molecule design is much more challenging than the single-objective scenario (Jin al. 2020).
This paper studies the problem of multi-ohjective molecule design for drug discovery. An ideal solu- tion should be efficient and meet the following criteria. Cl: It should satisfy multiple properties with high scores; C2: It should produce novel and diverse molecules; C3: Its generation process does not
WWrk was done while Yutong Xie and Chence Shi were research interns at ByteDance. Corresponding to: zhouhao. Secihhahnnnn@@@hhheheho wwwplpdreetttdddnnnneneomm and lileilabe bytedance.com

---

Published as a conference paper at ICLR 2021
rely on either expert annotated wet experimental data collected from a biochemistry lab (since it requires tremendous effort and hard to obtain). Existing molecule generation approaches are mainly designed for the single objective setting, and they could not meet all criteria in the setting of mul- tiple objectives. These methods belong to four categories: a) generating candidates from a learned continuous latent space Kommez-Bombarelli al., 2018: Jin et al.. 2018), b) through reinforcement learning (You et al. 2018). c) using an encoder-decoder translation approach (Jin et al., 2019), or d) optimizing molecular properties through genetic algorithms (Nigam et al., 2020). Current state- of-the-an art nulti-objecttve molecular generation is a rationale- taseenale-basd method (Jin et al. 2020). In this approach, the authors propose build molecules by composing multiple extracted rationales, and the model can generate compounds that are simultaneously active to multiple biological targets. However, such an approach will result in quite complex molecules when we have many objectives. This is because different objectives correspond to different rationales, and including all these ra- tionales could lead to large molecules, which may be less drug-like and hard to be synthesized practically.
In this paper, we propose MArkov moleculaR Sampling (MARS), a simple yet flexible method for drug discovery. The basic idea is to start from seed molecule and keep generating candidate molecules by modifying fragments of molecular graphs from previous steps. meets all the criteria
T In MARS, the molecular design is formulated as iterative editing procedure with its total objective consisting of multiple property scores (QT). MARS employs the annealed Markov chain Monte Carlo sampling method search for optimal chemical compounds, which allows for the exploration of chemicals with novel and different fragments (C2)). The proposal to modify molecular fragments is represented using graph neural networks (GNNs), whose parameters are adaptively learned. We used message passing neural networks (MPNNs) in practice (Gilmer et al., 2017), but other GNNs can the framework as well. Furthermore, MARS utilizes the sample paths generated on-the-fly fly to train the proposal network adaptively. Therefore, it does not rely external annotated data (C3). With such an adaptive learnable proposal, it keeps improving the generation quality throughout the process.
We evaluate MARS and four other baselines, one latest method for each of the four method cate- gories. The benchmark includes variety of multi-objective generation settings. Experiments show that our proposed MARS achieves state-of-the- performance on five out of six tasks in terms of comprehensive evaluation consisting of the success rate, novelty, and diversity of the generated molecules. Notably, in the most challenging setting where four objectives bio-activities to two different targets. drug-likeness, and synthesizabilityy are simultaneously considered, our method achieves the state-of-the-arr result and outperforms existing methods by 77% in the comprehensive evaluation.
Our contributions are as follows:
We present MARS, a generic formulation of molecular design using Markov sampling, which can easily accommodate multiple objectives.
We develop an adaptive fragment-editing proposal based on GNN that learnable on the fly with only samples self-generate and efficient in exploring the chemical space. Experiments verifies our proposed MARS framework can find novel and diverse bioactive molecules that are both drug-like and highly synthesizable.
2 RELATED WORK
Recent years have witnessed the success of applying deep generative models and molecular graph representation learning in drug discovery (Schwalbe-Kode &
20200] Guoo & Zhao 2020 Existing approaches for molecular property optimization can be grouped into four categories, including generation with a) Bayesian inference, b) reinforcement learning, c) encoder- decoder translation models, and d) evolutionary and genetic algorithms. The first category is learning continuous latent spaces for molecular sequences or graphs and generating from such spaces using Bayesian optimization (BO) These methods rely heavily on the quality of latent representations, which imposes huge challenges to the encoders when there are multiple properties to consider.
al. 2018 Jin et al.) 2018 Winter et al.) 2019).
Unlike the first class, other work uses reinforcement learning (RL) to optimize desired objectives directly in the explicit chemical space (De Cao & Kipf 2018. Popova et al. 2018 You et al. 2018
2

---

Published as a conference paper at ICLR 2021
Popova et al. 2019 Shi et al. 2020 However, the models usually hard to train due to the high variance of RL.
The third category directly trains translation model that maps from input molecule to high- quality output molecule (Jin et al., 2019; 2020). Although simple, such methods require many high-quality labeled data, making them impractical in scenarios where the data is limited.
The last category of methods are evolutionary algorithms (EAs) and genetic algorithms (GAs) to explore large chemical space with certain property (Nicolaou et al. 2012 Devi et al.] 2015 Jensen 2019 Ahh et al., 2020). In Nigam et al. 2020), the authors propose to augment GA by adding an adversarial loss into the fitness evaluation increase the diversity, and the augmented GA outper- forms all other generative models in optimizing logP. Though flexible and straightforward, to make the search process efficient enough, most GA and EA methods require domain experts design molecular mutation and crossover rules, which could be non-trivial to obtain.
Besides single property optimization, there recent work address the multi-objective molecule generation problem. For example, Li et al. (2018) proposes to use conditional generative model to incorporate several objectives flexibly, while Lim al. (2020) leverages molecular scaffolds to con- trol the properties of generated molecules better. Among them, the current state-of-the-art approach rationale-based method proposed by Jin al. (2020). this method, the authors propose to build molecules by assembling extracted rationales. Despite its great success in generating com- pounds simultaneously active to multiple biological targets, the combination of rationales might hinder the synthesizability and drug-likeness of produced molecules, they tend to be large the number objectives grows. In contrast, our MARS framework turns the generation problem into a sampling procedure, which serves as alternative way compared with deep generative models, and can efficiently discover bio-active molecules that are both drug-like and highly synthesizable.
Remotely related is recent work to generate molecules through sampling. [eef et al. (2019) defines a Gibbs sampling procedure, in which the Markov chain alternates between randomly corrupting the molecules and recovering the corrupted ones with learned reconstruction model. However, this method mainly focuses on generating molecules that follow the observed data distribution and cannot be directly tailored for property optimization. Different from this work, MARS is built upon the general MCMC sampling framework, which allows further enhancement with adaptive proposal learning to edit molecular graphs efficiently. Actually, generating instances from a discrete space with MCMC sampling methods is previously employed in various other applications, e.g.. generating natural language sentences under various constraints (Miao et al. 2019: Zhang et al. 2019 Liu et al.) 2020; Zhang et al., 2020).
PROPOSED MARS APPROACH
In this section, we present the MArkov moleculaR Sampling method (MARS) for multi-objective molecular design. We define Markov chain over the explicit molecular graph space and design kernel to navigate high probable candidates with acceptance and rejection.
3.1 SAMPLING FROM THE MOLECULAR SPACE
Our proposed MARS framework aims at sampling molecules with desired properties from the chem- ical space. Specifically, given K properties of interest, the desired molecular distribution can be formulated as a combination of all objectives:
x (x) (1) 52(x) 83(0))(())))))))) ox
(1)
desired properties
where x is a molecule in the molecular space X. x (xx) is an unnormalized distribution over molecules integrating the desired properties. su (z) ) is a scoring function for the k-th property and the "o" operator stands for combination of scores (e.g., summation multiplication). In practical drug discovery, these terms could be related to the biological activity, drug- likeness,,, and synthesizability of molecules (Nicolaou et al. 2012). This framework allows flexible configuration according to various concrete applications. However, as the number objectives grows, the joint distribution T (æ: will become more complex and intractable, making the sampling non-trivial.
3

---

Published as a conference paper at ICLR 2021
In MARS, propose sample molecules from the desired distribution Eq. uuing Markov chain Monte Carlo (MCMC) methods (Andrieu et al. 2003). Given a desired molecular distribution T(x) as the unnormalized target distribution, define Markov chain the explicit chemical space X' (i.e., each state of the Markov chain is a particular molecule) and introduce a proposal distribution q(r' (x) to perform state transitions.
x' e (x' (x Propose
x x'~q (x(1)
Initialize
Propose
H₂C OH2y
Accept
Reject
x(0)
x
A(x(t-1),x)
x(1
A(x()
(a)
(d)
Figure I: The framework of MARS. During the sampling process: (a) starting from an arbitrary initial molecule (0)) in the molecular space X, (b) sampling candidate molecule z' X' from the proposal distribution a(x' | (=(+)1)) at each step, and (c/d) the candidate x' is either accepted or rejected according to the acceptance
rate A(z(tt11),=') [0, 1). By repeating this process, we can generate sequence molecules (xx(t))²2
Specifically, as shown in Figure I the sampling procedure of MARS starts from an initial molecule (())) E X. At each time step L, a molecule candidate x' E X will be sampled from the proposal distribution q(x') 1.(t-1)), where (f-1)) denotes the molecule at time step 1- Then the proposed candidate x' could be either accepted ((()) = x' or rejected (x(+) = (tt-1) according to the acceptance rate ((x((-1), x²) E [0 controlled by the target distribution =(r). By repeating this process, sequence of molecules 12 can be generated. Such sequence of molecules will converge to the target distribution क (2) if the proposal distribution and the acceptance mechanism are configured properly.
The acceptance rate is calculated as follow:
A(x, x') = min { n (x')g(z)2))
(2)
T (x)
where a is coefficient that varies in different instantiations of MCMC algorithms. Here to find molecules that globally maximize the target distribution, we employ an annealing scheme (Laarhoven & Aarts 1987) where a 1/7 and T is temperature controlled by cooling schedule. In addition to this, other instantiations such as Metropolis-Hatting (MH) al- are also feasible under our general framework.
gorithm (Metropolis et al. 1953) where
As for the proposal distribution q(x' x), it largely affects the sampling performance and should be designed elaborately. In general, it is crucial that the proposal distribution q(x' x) and the target distribution π (x') are as close as possible to ensure high sampling efficiency. So we propose using a proposal distribution go(x' æ) with learnable parameters to capture the desired molecular properties and develop strategy to train the proposal throughout the sampling process adaptively. The details will be described in the next section.
3.2 ADAPTIVE MOLECULAR GRAPH EDITING PROPOSAL
In this section we will examine in detail our proposed adaptive proposal distribution ço(x' 2).. A molecule represented as a graph whose nodes are heavy atoms and edges are chemical bonds. The proposal distribution is defined over molecular graph editing actions. We use the message passing neural network (MPNN) to represent the proposal. Alternative parameterization schemes such other graph neural networks also possible. To sample molecules with desired properties effectively and efficiently, we also design self-training strategy to learn the proposal MPNN during sampling in an adaptive manner.
Molecular graph editing actions. To transform a molecule x into another molecule x', we consider two sets of graph editing actions, i.e., fragment adding and deleting. These actions are inspired by fragment-basee based drug design (FBDD) methodology. whose success in drug discovery has been proved in past decades (Kumar al. 2012) In MARS, define frugments as connected components in
4

---

Published as a conference paper at ICLR 2021
molecules separated by single bonds. To reduce the complexity of editing actions, only consider fragments with single attachment position. Moreover, we also define a frragment vocabulary that contains finitely many fragments, and only fragments in the vocabulary are allowed to be added onto a molecule. Examples for fragment adding and deleting actions are shown in Figure 2
Fragments in molecules
(a) Molecular graph adding action:
O
(b) Molecular graph deleting action:
Fragment vocabulary
Examples of molecular fragments a fragment vocabulary. Red dashed lines represents cuttable bonds to extract fragments. Right: Examples of molecular graph editing actions.
Figure 2: Left:
Specifically, given a molecule x with n atoms and bonds, we choose to add or delete a fragment onto or from this molecule randomly with probability I for cach set of actions. For the adding action, suppose have a probability distribution over atoms Paua u) and probability distribution over fragments in the vocabulary Pirg (x, u, k). Here u E [n] is an indicator of the atom in to which the fragment is adding to and i [V] an indicator of fragments in the vocabulary of size V. We can compute the proposal distribution as follows:
q('' xx)
Puda Pusa((r, u) Pirug (2, u, k))
where is the molecule obtained by adding the k-th fragment onto the atom u in molecule x.
As for the deleting action, suppose we have probability distribution over bond (ar b) where b E [2m] is an indicator of bonds in æ. We can compute the proposal distribution as follow:
1
g( (x)
(a,
(2,
(4)
where z' is the molecule obtained by removing bond b and the attached fragment from molecule
Parameterizing with MPNNs. To better model the molecular graph editing actions, we propose MPNNs to suggest the probability distributions (Paad: Ping: Pdel) Ma(a)) where Mo is MPNN model specified by parameters 0, which has been proven powerful to predict chemical prop- erties with molecular graphs (Gilmer et al. 2017). Given molecule æ, compute the probability distributions as follow:
h pode MPNN(x),, E IRd
(5)
h edge
Concat mide
node mandee E 222d
(6)
Pada (a:)
Softmax ((MLP ponte mode
}) E [0
Pirag (x, u) Softmax (MLP' mode (hoodd)
[0, 1] VI
(8)
Paal (x: Softmax (MLP Pdge (h, edge )) E [0, 2m
where u atom indicators, {h mode I with bond b, (h odge 12m are edge hidden representations, and ML.Ppode: MLP pode MLP are multi- layer peceptrons (MLPs), similar to Hu et al. (2020).
Ju=1 node hidden representations, v, w atoms connected
Adaptive self-training. To capture the desired properties and improve the sampling effective- ness, can train the editing model to increase the probability of suggesting high-quality candidate
Molecular bonds treated as directional specify the fragments drop from molecules.
5

---

Published as a conference paper at ICLR 2021
Algorithm 1: MARS
Set N initial molecules 1 and initialize the molecular graph editing model Mr Create an empty editing model training dataset {
for = 1, .......
for i 1,,,..... N
Compute probability distributions (Pass, Piug: pae)) = Ma (2r("11) as Equations 7
Sample candidate molecule z' from the proposal distribution g(a' (x(-)) defined with probability distributions Pada, Pling, Pdel Equations
A(x(*~1), x²) where u ~ 1j..1 then Accept the candidate molecule a;; x
Refuse the candidate molccule 2!(1) oi (t-11)
The candidate improves the objectives, i.e. (<x') x<xx then Adding editing record (x: (x((=1),, xx') into dataset D
11
13
arg max log MD(
molecules. Here we propose to train the model on-the-fly during the sampling process in an adaptive manner where the training data is collected from the sampling paths. By doing we can bypass the difficulty of lacking training instances that satisfy all property constraints. Mainly, we collect molecule candidates that improve our desired objectives and train the model MM in maximum likelihood estimation (MLE) manner (i.e., to maximize the probability of producing the collected candidates). The overall MARS is described in Algorithm
Discussion on convergence. Compared with standard MCMC algorithms, MARS still falls in the Metropolis- Hastiigss algorithm but with annealing scheme and adaptive proposal, which results in inhomogeneous transition kernels. The convergence of adaptive MCMC is discussed in Rooennhaal (201T). According to the diminishing adaptation condition, gence by making the difference proposals in consecutive iterations diminish to zero. MARS can satisfy this condition by using optimizer whose learning rate will shrink zero eventually (e.g.. Adam). Annealed MCMC is to find samples maximizing the target probability. The convergence of annealed MCMC discussed in Andrieu al. (2003).
ensure conver-
EXPERIMENTS
4.1 EXPERIMENT SETUP
Biological objectives. Following Jin et al. (220)) we consider the following inhibition scores against two Alzheimer-rclated target proteins as the biological activity objectives. The score given by a random forest model that predicts based on Morgan fingerprint features of a molecule (Rogers & Hahn 2010).
GSK333 Inhibition against glycogen synthase kinase-3,3. JNK3: Inhibition against c-Jun N-terminal kinase-3.
Non-biological objectives. Following Jin et al. (2020), adopt QED (Bickerton al.) 2012 and synthetic accessibility (SA) (Ertl & Schuftenhauer 2009) to quantify the drug-likeness and synthesizability. We rescale the SA score (initially between 10 and 1) into (0, 1] such that molecules with higher scores are more synthesizable.
Multi-objective generation setting. To evaluate the effectiveness of the proposed method for multi- objective drug design, we also consider the following more challenging objective combinations:

---

Published as a conference paper at ICLR 2021
GSK33+JNK3: Jointly inhibiting GSK38 and JNK3. The combination may provide poten-
tial benefits for the treatment of Alzheimer's 's disease reported by Hu et al. (2009; Martin
ct al. (2013).
QEAAAA Inhibiting 3SK3,3 or JNK3 while being drug-like and syntheti- cally accessible, which are quantified by QED and SA, respectively.
GSK3J+JNK3++ED-S3 Jointly inhibiting GSK33 and JNK3 while being drug-like and synthetically accessible, which are quantified by QED and SA, respectively.
Baselines. We compare MARS with the following methods the latest ones from four categories mentioned in the related work (Sec. 2 GCPN You et al. 2018) leverages RL to generate molecules atom by atom, and the adversarial Joss is incorporated molecules. JT-VAE (Jin et al. 2018) is a VAE-based approach that firstly generates junction trees and then assembles them into molecules. It performs Bayesian optimization (BO) to guide molecules towards desired properties. RationaleRL (Jin et al. 2020 is state-of sate--of-thearn the-thertt art approach for multi- property optimization, which generates molecules from combined rationales. GA+D (Nigam al.) 2020) is a heuristic search method that applies the genetic algorithm (GA) to find molecules with high property scores. An adversarial loss is incorporated in the fitmess evaluation increase the diversity of generated molecules.
the objective to generate more realistic
Evaluation metries. Following Jin et al. (2020) we generate N 5000 molecules for each approach and compare the proposed method with the baselines on the following evaluation met- rics: Success rate (SR) is the percentage of generated molecules that are evaluated as positive on all given objectives (QED 2 0.6, SA 2 0.67, the inhibition scores of GSK333 and JNK3 2 0.5); Novelty (Nov) is the percentage of generated molecules with similarity less than 0.4 compared to the nearest neighbor 2SNN in the training set (Olivecrona et al. 2017): Nov I Exes [[sim(x, ISNN) < 0.4]; Diversity (Div) measures the diversity of generated molecules, which can be calculated based on pairwise Tanimoto similarity over Morgan fingerprints sim(z, x²) I[n-1] Exprrre sim(ar, x²); PM is the product of the above three metries, which is more comprehensive evaluation of the proposed method. Intuitively, PM presents the percentage of generated molecules that are simultaneously bio-active, novel and diverse, which are essential criteria for molecules to considered in building a suitable drug candidate library in early-stage drug discovery (Huggins et al. 2011).
as Div
Implementation details. For the fragment vocabulary, we extract the top 1000 frequently appear- ing fragments that contain no more than 10 heavy atoms from the ChEMBL database (Gaulton et al.) 2017) by enumerating single bonds to break. As for the sampling process, the unnormalized target distribution is set as (xx) = Ex (z) where sx(x)) is a scoring function for the above-mentioned properties of interests, the temperature is set T 0.95 (t/5] and sample N = 5000 molecules at one time. During sampling, the computation of q(<r x²) is ignored and we approximate A(x, x') with min(1, n. (x'), / (x))) } increase the computation efficiency. This is acceptable because in practice q(x | x²) and g(z' x) is of order ((1) and A(I,z') will be gradually bounded by r (x')/x² (z) as the temperature T decrease to zero. The sampling paths are all starting with an identical molecule "C-C", which is also adopted by previous graph generation methods for organic molecules (You et al., 20188. The MPNN model has six layers, and the node embedding size is d = 64. Moreover, for the model training, we use an Adam optimizer (Kingma & Ba. 2015) to update the model parameters with an initial learning rate set as x 10-4, the maximum dataset size is limited as D < 75, 000, and at each step, update the model for more than 25 times.
4.2 MAIN RESULTS AND ANALYSIS
We perform ten independent runs for MARS. The quantitative results are summarized in Table and Table 2 From these tables, we observe that MARS outperforms all the baselines on five out of six tasks in terms of PM. Furthermore, CSK33++NK3+QED+SA, it significantly surpasses the best bascline with a 77% improvement for the product of metrics PM. Additional results are shown in Appendix
the most challenging multi-objective optimization task, i.c.,
In comparing all these methods, the GA+D baseline is most similar to our MARS in terms of the high novelty and PM score, as both methods focus molecular space exploration. However, the diversity score of GA+D drops a lot when optimizing multiple properties simultancously, as GAs likely to get trapped in regions of local optima (Paszkowicz) 2009). RationaleRL. is very strong
7

---

Published as a conference paper at ICLR 2021
Table I: Comparison of different methods molecular generation with only bio-activity objectives. Results of GA+D are obtained by running its open-soures code. Results of other baselines are taken from Jin et 20200) For MARS, we report the mean and standard deviation of 10 independent experiments.
GSK33
JNK3
GSK33 INK3
Method
SR
Nov
Div
PM
SR
Nov
Div
PM
SR
Nov
Div
PM
GCPN
42.4%
11.6%
0.904
0.04
32,3%
4.4%
0.884
0.01
3.5%
8.0%
0.874
0.00
JT-VAE
32.2%
11.8%
0.901
0.03
23.5%
2.9%
0.882
0.01
3.3%
7.9%
0.883
0.00
RationaleRL
100.0%
53.4%
0,888
0.47
100.0%
46,
0,862
0.40
100.0%
97.3%
0.824
0.80
GA+D
84.6%
100.0%
0.714
0.60
52.8
98.3%
0.726
0.38
84.7%
100.0%
0.424
0.36
MARS
100.0%
84.0%
0.718
98.8%
88.9%
0.748
0.66
99.5%
75.3%
0.691
0.52
0.04
0.04
0.08
Table 2: Comparison of different methods molecular generation with bio-activity, QED, and SA objectives. Results of baselines are obtained by running their open- codes. For the results MARS, we report the mean and standard deviation of 10 independent experiments.
GSK33
QED SA
JNK3 QED
GSK33
JNK3 QED
SA
Method
SR
Nov
Div
PM
SR
Nov
Div
PM
SR
Nov
Div
PM
GCPN
0.0%
0.0%
0.000
0.00
0.0%
0.0%
0.000
0.00
0.0%
0.0%
0.000
0.00
JT-VAE
9.6%
95.8%
0.680
0.06
21.8%
100.0%
0.600
0.13
5.4%
100.0%
0.277
0.02
RationaleRL
69.9%
40.2%
0.893
0.25
62.3%
37.6%
0,865
0.20
75.0%
55.5%
0.706
0.29
100,0%
0.682
85.7%
0.504
0.43
100.0%
MARS
99.5%
95.0%
0.719
0.68
91.3%
94.8%
0.779
0.67
92.3%
82.4%
0.719
0.55
+ 0.03
0.02
0.05
baseline that performs better than MARS in the GSK3Ø+JNK3 setting. Nevertheless, when taking the drug-likenese and synthetic accessibility into consideration, their performance falls short of ours and fails to generate novel molecules. The performance of GCPN and JT-VAE remains relatively low in most settings, as they are not tailored for multi-objective property optimization.
Visualization. We use t-SNE (van der Maaten & Hinton 2008) to visualize the distri- bution of generated positive molecules with the positive ones the training set under the CSK33++NK++EE+SA setting. In the visualization, we use the ECFP6 fingerprints as suggested in Li (2018). As shown by Figure most molecules generated by GA+D fall into two mas- sive clusters, which aligns their low diversity. Molecules generated by RationaleRL also tend to be clustered, with each cluster standing for a specific combination of rationales. By contrast, the molecules generated by MARS are evenly distributed in the space with a range of novel regions covered, which justifies our high novelty and diversity scores. We further visualize some molecules generated by MARS with high property scores in Figure 4 indicating its ability to generate highly synthesizable drug-like molecules that jointly inhibit GSK33 and JNK3. Additional examples of sampled molecules are shown in Appendix
Generated by RationaleRL
Generated by GA+D
Generated by MARS Positive Samples
Positive Samples
Positive Samples
(a) RationaleRI.
(b) GA+D
(c) MARS
Figure 3: t-SNE visualization of generated molecules (gray) and positive molecules in the training (blue).
Running time. The computing server has two CPUs with 64 virtual cores 10GG), 231G mem- ory (about 50G used), and one Tesla V100 GPU with 32G memory. In the GSK3 (TT33+NNK33+EDDSS setting, MARS takes roughly T 550 sampling steps and 12 hours in total to converge (including the time used in proposing and evaluating molecules as well as MPNN model training). For other baselines, RationaleRL. takes 5.7 hours to fine-tune the model, and GA+D takes 278 steps and 2.2h to
8

---

Published as a conference paper at ICLR 2021
(0.91, 0.85, 0.78, 0.92) (0.95, 0.76, 0.75, 0.88) (0.85, 0.87, 0.74, 0.87) (0.91, 0.71, 0.78, 0.90)
Figure 4: Sample molecules generated by MARS (SK33++NK3--EE-SS setting. The numbers in brackets are GSK38, JNK3, QED, and SA scores of each molecule respectively.
achieve its best performance. Compared to the conventional drug discovery process, which usually takes months to years, the time we spent on molecular generation models is almost ignorable.
4.3 EFFECTS OF PROPOSAL AND ACCEPTANCE STRATEGY
To justify the contributions of the designed proposal and acceptance strategy. we compare them with some naive ones and summarize the results of different combinations in Table 3 For acceptance strategies, Annealed stands for annealed MCMC where the acceptance rate is computed as Equa- tion 2 given a = 1/T, AlwaysAC stands for always accepting the candidate, i.e., A(x,, x') 1, and HillClimb stands for accepting the candidate only when the overall score is improved, i.e., A(I,x') = sign(s(x') > s(x)]. For proposal strategies, Random stands for random proposal where we randomly select atoms, bonds, and fragments to edit, and Adaptive stands for the adaptive fragment-based graph editing model trained during the sampling process as described in Section 3.2
Table 3: Results of different acceptance strategies and proposal strategies for molecular sampling.
GSK333 + JNK3
GSK33 + JNK3 + QED + SA
AC Strategy
Proposal
SR
Nov
Div
PM
SR
Nov
Div
PM
Annealed AlwaysAC
Random
40.9%
94.9%
0.828
0.32
25.5%
80.4%
0.793
0.16
Adaptive
49.1%
88.4%
0.742
0.32
10.1%
94.6%
0.716
0.07
HillClimb
Adaptive
53.7%
96.1%
0.814
0.42
51.4%
86.6%
0.777
0.35
Annealed
Adaptive
99.5%
75.2%
0.688
0.52
92.3%
82.4%
0.719
0.55
The results in Table Bindicate that proposals will influence the performance of MARS dramatically (the first and the last row), especially when the number objectives increases. The proposed adap- tive proposal outperforms the random proposal and converges 4.6x faster in practice. By comparing the last three rows, we find the Annealed strategy outperforms the other two strategies by a large margin both settings, samples from such strategy more likely to jump out of local opti- mums. Another interesting observation that even with the naive AlwaysAC or heuristic HillClimb strategy, the MARS achieves comparable even better performance than GA+D and RationaleRI. in some settings, e.g., HillClimb on ANK3, optimization, which again proves the effectiveness of the proposed proposal.
CONCLUSION AND FUTURE WORK
This paper proposes a simple yet flexible MArkov moleculaR Sampling framework (MARS) for multi-objective drug discovery. MARS includes a trainable proposal to modify chemical graph fragments, which is parameterized by an MPNN. Our experiments verify that MARS outperforms prior approaches five out of six molecule generation tasks, and it is capable of finding novel and diverse bioactive molecules that are both drug-like and highly synthesizable. Future work can include further study parameterization and training strategy of the molecular-editing proposal.
ACKNOWLEDGEMENT
We would like to thank Meihua Dang for refactoring much of the MARS code. Meihua also per- formed multiple experiments, which generates the results for the tables. We also thank Jiangjie Chen. Yuxuan Song, Jingjing Xu, Weiying Ma, Hang Li, and anonymous reviewers for their con- structive comments and suggestions.

---

Published as a conference paper at ICLR 2021
REFERENCES
Sungsoo Ahn, Junsu Kim, Hankook Lee, and Jinwoo Shin. Guiding deep molecular optimization with genetic exploration. In Advances in Neural Information Processing Systems 33: Annual Conference on Newral Information Processing Systems, 2020.
Christophe Andrieu, Nando De Freitas, Arnaud Doucet, and Michael Jordan. An introduction to meme for machine learning. Machine Learning, 50(1):5--4)) 2003.
G. Richard Bickerton, Gaia V. Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L. Hopkins. Quantifying the chemical beauty of drugs. Nature Chemistry, 4(2):90-98, 2012.
Nicola De Cao and Thomas Kipf. MolGAN: An implicit generative model for small molecular graphs. ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models, 2018.
R. Vasundhara Devi, S. Siva Sathya, and Mohane Selvaraj Coumar. Evolutionary algorithms for de novo drug design a survey. Applied Soft Computing, 27:543-552, 2015.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminfor- matics, 1(1):8 8, 2009.
Anna Gaulton, Anne Hersey, Michat Nowotka, A. Patrícia Bento, Jon Chambers, David Mendez, Prudence Mutowo, Francis Atkinson, Louisa J. Bellis, Hlena Cibrián-Uhalte, Mark Davies, Nathan Dedman, Anneli Karlsson, María Paula Magariños, John P. Overington, George Pa- padatos, Ines Smit, and Andrew R. Leach. The chembl database in 2017. Nucleic Acids Research, 45, 2017.
Justin Gilmer, Samuel Schoenholz, Patrick Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, 2017.
Xiaojie Guo and Liang Zhao. A systematic survey on deep generative models for graph generation. arXiv: Learning, 2020.
Rafael Gómez-Bombarelli, Jennifer Nansean Wei, David Duvenaud, José Miguel Hernández- Lobato, Benjamín Sánchez-Lengeling. Dennis Sheberla, Jorge Aguilera-Iparrauuire, Timothy D. Hirzel, Ryan Prescott Adams, and Alan Aspura-Guzik. Automatic chemical design using data- driven continuous representation of molecules. ACS central science, 4(2):268-276, 2018.
Shuxin Hu, Aynun N. Begum, Mychica R. Jones, Mike S. Oh, Walter K. Beech, Beverly Hudspeth Beech, Fusheng Yang. Pingping Chen, Oliver J. Ubeda, Peter C. Kim, Peter Davies, Qiulan Ma, Greg M. Cole, and Sally A. Frautschy. Gsk3 inhibitors show benefits in alzheimer's disease (ad) model of neurodegeneration but adverse effects in control animals. Neurobiology of Disease, 33(2):193-206, 2009.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik. Percy Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In Proceedings of the 8th Interna- fional Conference on Learning Representations, 2020.
David John Huggins, Ashok Ramakrishnan Venkitaraman, and David Robert Spring. Rational meth- ods for the selection of diverse screening compounds. ACS Chemical Biology, 6(3):208-217, 2011.
Jan H. Jensen. A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical Science, 10(12):3567-335 2019.
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Junction tree variational autoencoder for molecular graph generation. In Proceedings of the 35th International Conference on Machine Learning. 2018
Wengong Jin, Kevin Yang. Regina Barzilay, and Tommi S. Jaakkola. Learning multimodal graph-to- graph translation for molecule optimization. In Proceedings of the 7th International Conference Learning Representations, 2019.
10

---

Published as a conference paper at ICLR 2021
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Composing molecules with multiple property constraints. In Proceedings of the 37th Interaational Conference on Machine Learning. 2020.
Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In Proceedings of the 37th International Conference on Machine Learn- ing. 2020.
Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, 2015.
Ashutosh Kumar, Voet, and K. Y. J Zhang. Fragment based drug design: from experimental to computational approaches. Current Medicinal Chemistry, 19(30):512-8544 2012.
J. M. Laarhoven and E. H. L. Aarts. Simulated Annealing: Theory and Applications. 1987.
Yibo Li, Liang Ren Zhang, and Zhenming Liu. Multi-objective de novo drug design with conditional graph generative model. Journal of Cheminformatics, 10(1):33, 2018.
Jaechang Lim, Sang-Yeon Hwang. Seungsu Kim, Seokhyun Moon, and Woo Youn Kim. Scaffold- based molecular design using graph generative model. Chemical Science, 11(4):1153-116) 2020.
Xianggen Liu, Lili Mou, Fandong Meng, Hao Zhou, Jie Zhou, and Sen Song. Unsupervised para- phrasing by simulated annealing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
Ludovic Martin, Xenia Latypova, Cornelia M. Wilson, Amandine Magnaudeix, Marie-Laure Perrin, Catherine Yardin, and Faraj Terro Tau protein kinases: Involvement in alzheimer's disease. Ageing Research Reviews, 12(1):299-309 2013.
Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Ed- ward Teller. Equation of state caleulations by fast computing machines. Journal of Chemical Physics, 21(6):1087-1092 1953.
Ning Miao, Hao Zhou, Lili Mou, Rui Yan, and Lei Li. CGMH: constrained sentence generation by metropolis-hastie hastingss sampling. Proceedings of the 33rd Conference on Artificial Intelligence, 2019.
C. A. Nicolaou, C. Kannas, and Erika Loizidou. Multi-objective optimization methods in de novo drug design. Mini-reviews in Medicinal Chemistry, 12(10):979 2012
AkshatKumar Nigam, Pascal Friederich, Mario Krenn, and Alán Aspuru-Gust Guzik. Augmenting ge- netic algorithms with deep neural networks for exploring the chemical space. In Proceedings of the 8th International Conference on Learning Representations, 2020.
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatics, 9(1::48-48, 2017.
W. Paszkowicz. Properties of a genetic algorithm equipped with a dynamic penalty function. Com- putational Materials Science, 45(1):77-83, 2009.
Pavel G. Polishchuk, Timur Madzhidov, and Alexandre Varnek. Estimation of the size of drug- like chemical space based gdb-17 data. Journal of Computer-aided Molecular Design, 27(8): 675-679, 2013.
Mariya Popova, Olexandr Isayev, and Alexander E Tropsha. Deep reinforcement learning for de novo drug design. Science Advances, 4(7), 2018.
Mariya Popova, Mykhailo Shvets, Junier Oliva, and Olexandr Isayev. Molecularmn: Generating realistic molecular graphs with optimized properties. arXiv: Learning, 2019.
David Rogers and Mathew Hahn. Extended- conneeed-comnectivity fingerprints. Journal of Chemical Infor- mation and Modeling, 5045):742-75,, 2010. 2010.

---

Published as a conference paper at ICLR 2021
Jeffrey S. Rosenthal. Optimal proposal distributions and adaptive meme. In Handbook of Markov Chain Monte Carlo, chapter 4. CRC Press, 2011.
Daniel Sehwalbe-Koda and Rafael Gómez-Bombarelli, Generative models for automatic chemical design. In Machine Learning Meets Quantum Physics. 2020.
Ari Seff, Wenda Zhou, Farhan Damani, Abigail Doyle, and Ryan P. Adams. Discrete object gen- eration with reversible inductive construction. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems, 2019.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: flow-based autoregressive model for molecular graph generation. In Proceedings of the 8th International Conference on Learning Representations, 2020.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9:2579-2605, 2008
Robin Winter, Floriane Montanari, Andreas Steffen, Hans Briem, Frank Noé, and Djork-Aréé Clev- ert. Efficient multi-objective molecular optimization in continuous latent space. Chemical Science, 10134)8811--0024 2019.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay S. Pande, and Jure Leskovec. Graph convolutional policy network for goal goal-directed molecular graph generation. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems, 2018.
Chengxi Zang and Fei Wang. Moflow: An invertible flow model for generating molecular graphs. In The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2020.
Huangzhao Zhang. Hao Zhou, Ning Miao, and Lei Li. Generating fluent adversarial examples for natural languages. In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, Florence, Italy, 2019.
Maosen Zhang, Nan Jiang, Lei Li, and Yexiang Xue. Language generation via combinatorial con- straint satisfaction: A tree search enhanced Monte-Cari approach. In Proceedings of the Confer- ence on Empirical Methods in Natural Language Processing, 2020.
12

---

Published as a conference paper at ICLR 2021
Appendix
A PROPERTY SCORES OF SAMPLED MOLECULES
The property score distributions of sampled N = 5000 molecules of the GSK3/+JNK3+QED+S
setting are shown in Figure The average of the metrics over the sampling path is shown in Figure
4-
2-
(a) GSK38 inhibition score distribution.
(b) JNK3 inhibition score distribution.
Denety
3.4
2.6
0.8
3.0
0.0
0.2
(c) QED score distribution.
(d) SA score distribution.
Figure Property score distributions sampled 5000 molecules. The lines are success thresholds.
1.0
0.8
0.6
0.4
0.2
PM
0.0
Figure 6: MARS sampling curves (average of 10 runs) for the CSK38+INK++QED4SS setting. SR: success rate. Nov: novelty. Div: diversity. PM: product of the three metrics. Shaded area shows the standard deviations 10 independent runs.
13

---

Published as a conference paper at ICLR 2021
B SINGLE OBJECTIVE GENERATION
To study whether our proposed method capable of single-objective molecular generation, we also investigate how MARS performs on the drug-likeness (QED) and the penalized octanol-water partition coefficient (penalized logP) optimization. The experiment results are shown in Table 4 In the experiments, our approach can obtain the best performance on both QED and logP optimization. And especially, MARS outperforms previous methods significantly in the logP generation task.
Table 4: Comparison of different methods on single-objective molecular generation. Results of other baselines are taken from Shi et al. (2020) and Nigam et al. 2020).
QED
Penalized logP
Method
1st
2nd
3rd
1st
2nd
3rd
GCPN You et al.
2018)
0.948
0.947
0.946
7.98
7.85
7.80
JT-VAE (Jin et 2018)
0.925
0.911
0.91
5.30
4.93
4.49
GraphAF (Shi et al.
2020
0.948
0.948
0.947
12.23
11.29
11.05
GB-GA (Jensen
2019)
15.76 + 5.71
/
/
GA+D NNgam et al. 2020)
20.72
3.14
MARS
0.948
0.948
0.948
44.99
44.32
43.81
Moreover, from the results, we also see how these two previously widely used metrics (Jin al. 2018 You et al. 2018 Popova et al., 2019; Shi et al. 2020 Nigam et al. 2020) are questionable for both scientific study and practical use. Most of the generative methods (i.e., GCPN, JAE, and GraphAF) can produce molecules with the highest possible QED score of 0.948, making the top QED score metric hard to distinguish different methods. As for logP optimization, heuristic search-based (i.e., GB-GA and GA+D) and sampling-ban
methods (i.e., MARS) can all easily beat generative models. This is because penalized logP score will prefer larger molecules that gen- erative models can hardly produce. However, such large molecules are unrealistic for practical drug discovery, making the top penalized logP score metric problematic.
C EXAMPLES OF SAMPLED MOLECULES
We also provide some examples sampled molecules from the GSK33++NK3+QE++SA setting. The numbers under molecule graphs are GSK33, JNK3, QED, and SA scores, respectively.
14

---

Published as a conference paper at ICLR 2021
081, 081,0,860074, 0.7.0000.08
0,90,080 3,8000830.788088 0.78.088
088, 085.0.75.0778088
0.8.0081..78.000
082,0.78.078.
027, 077.07740887
0.7
0.91 0.1.0.72.070.008
0.8, 0.5.0.77.0799000
0.910.71..700000
0.88, 2.8.0.74.0.80008
0.22. 0.800078,008
089,071,00.8. 0.88
0.88, 0.8,0..1..033 0.8.0.71.777..88
0,88,0.7,00744 0.78.088
0.88, 088.0.89.000 0.79 0.8
087. 0,86,079,008 0.8
3.8,066, 0.79.086
087, 087,084, 0.80, 0880088,0080 0.86
0,87,
0.85 0.2 088,088.0.80,000 08
66,00,80,000
2.66,081.0
0.8. 00,0080.088
0.8. 0.59.
D.8
0.8, 0.58 081
088, 0.57.0 081,088
0.8
088, 0.55
Figure 40 sampled molecules with highest average property scores.
15

---

Published as a conference paper at ICLR 2021
088, 0.880076.088
0,80,074 3.8,0079.0778 077.0.760088
082, 082.0.73.078.0882
088.0.690.78.04
081,085,078.088
0,800.74,087
0.7,
091,0.1,0.78.00
0.910.50,000 0.74.000888
0.00,
880.080,074.08
0,90,0811 1.80,081.0.71
0.8. 0.9.0.800077.0088
0.88,0 0,85
0.8, 0.0.000.0000 0.74,
088. 188.0.71.078.088
0.88,071, 038.0.710079..0 0.88
0.88, 0.880.70.0033 0.8.0.00.0778.0888
0,88,0.8, 0.78,0.8 0.88
088.081.0.78.088
OBB. 0.700000,.08
0.88,0 0.000.22, 0.8.0.85
O88, 089.0079.087
0.88, 0,880068,0777 088.0.78.88
088, 088.087.0778.08
037.0.7.079.087
0.87.
087, 0.2.
087. 0.8.0.88.0000 0.8
0.88
087.0 107.0.8.000 087.039.073.087
057.0.540766.00
087.081.0755008
Figure 40 sampled molecules highest GSK3,9 scores.
16

---

Published as a conference paper at ICLR 2021
080. 0.88,077,088
0.8,091,0.76. 01700.7..088
081,038,0.75.08
0,80,080,0078 0.73,088
281,088,0.88, 888
0,800.75,08 0.1
0.50,
082
0.8.0.6..770.00
0.50, 0.05, 0.78
058, 0.8.0880077.0844
0.73 0,80
0.53 0.8,
091. 091.0.66.078.088
0.71, 0,85 0.58..8... 0.84
0.89, 0.8.0..566 0.56, 0.89
0,86,0 0.58..82
0.82, 082.0.8.0778088
082 0,840777,88
0.1.0.2
054, 084,077,,080
3.440084.0771 0.1.0071..81
0.53 08400.8,,0888 088
0,890074.089
0.77.
0.83
057.
0.8.0063.7000
0.88 0.2,
064. 084.082.073.088 0,83,88, 0.7.088..777088
060, 082 0.822077..00
Figure sampled molecules with highest JNK3 scores.
17

---

Published as a conference paper at ICLR 2021
BBB. 0,80,0.4,088 0,8
0,60, 0,00 094,0.00
0.72 088,084,080
0,84,0.60008 088,088
0.85, 0,8,0040 0.8
070.0.81.0300087
0.59,
088, 088.0.88,887
0.74,.50, 0.9,
087.0.80..82.0076
071,0.55.88
0.74, 024.0.80.082.086
0,60.063,000
087,0588..92.001
088. 0,84,082,.088 0.82
0.64,056, 0.56, 0.89
0.89, 088.0.81.082.080
0,480.71.091.088
0.58, 3.880.59...91.00 088
088. 0.36,091,077
0,83,065, 081.0.44
0.7,0 081.081,888
0.8. 088.088.0.91088
0.64.091..77
0.91.0.9
081. 098.0498
066.0664.0311.00 0..1......0
0.43 0.00
000.0.2.09900011
0.2 0.57. 0.5 0.74,062,
0.36 1.8.04888.0000
Figure 10: sampled molecules with highest QED scores.
18

---

Published as a conference paper at ICLR 2021
088 0.760.8,
0.90, 3.900.83.0.88.088 0.78.0.89
0.22 082.0.79..78.088
0,81,086, 0.86, 0.78..88
044 384,087,077,00
031
055,0.85, 0.1.0.91
0.78..88081
0,74,.50.00 0.70..000
374.0.59.075
0.89,087,088
0,80, 0.75 7.7.0.78..99
0.4, 038.0.59.080008
0.72 372.0.52..778000
0.99. 059.0.99.773.00
0.6,0.59, 0.740.74,000 0.90
0.73, 375,0.88.0.76 0.760000
0,88,080,088.000
071,0.59..79..00
031.0.2.0.78.00
088. 0.77,
0.7.0.76..800
0.88, 088,081.0778.080
0.72,0.57 372.0.57.073.088 0.7.0.00
0.89, 088.081.,088 0.90
039
088,0.4.4
0.9. 0.00
0.64.04 0.44,045,0003 0.780.00..030
0.0. 0.54
081.0.71077.0.0
2.666079..08
061,
0664034.00 03..034..1.08
0.67 387.05440.78.08
Figure sampled molecules with highest SA scores.
19


Reviews:
[{"review_id": "WKsWGhJ06Lk", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "SUMMARY: The authors present an elegant Markov-Chain Monte Carlo (MCMC) method to carry out the task of generating molecular structures that satisfy several objectives.\n\nPROS: \n- The work is well written, concise and easy to follow\n- The methodology is competitive with other approaches that are state-of-the-art in the optimization of single properties (such as GA-D) and show that they outperform them in most cases.\n- The references that it cites are balanced.\n- The multiobjective optimization is based on biological objectives\n\nCONS:\n- I see no major cons with this work.", "extra_scores": {}, "extra_text": {}}, {"review_id": "NNf_T7ylDnY", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe authors propose a novel way to generate molecules with specified objectives, named MArkov moleculaR Sampling (MARS). The idea of MARS is based on generating the chemical candidates by iterative editing fragments of molecular graphs. To transform a molecule x into another molecule x′, the authors considers two sets of graph editing actions fragment adding and fragment deleting, where fragments are connected components in molecules separated by single bonds. To generate the molecules with desired objectives, MARS is using Markov chain Monte Carlo sampling with specified annealing scheme, together with graph convolutional neural network. The results reported in the following paper are very promising and show that this could be a good direction in the area of multi-objective molecules optimization.\n \n\n=============================================================================\n\nPros:\n\n1. The idea proposed by authors seems novel. It is the combination of MCMC based on molecules, together with message passing neural networks.\n\n2. The proposed model is a new state of the art in the area of multi-objective molecules optimization. In the experimental section the authors shows that molecules generated by MARS have the highest desired objectives in 5 out of 6 proposed tasks. Simultaneously the generated set of molecules seems novel and diverse.\n\n3. The molecules generated by MARS are evenly distributed in the space with a range of novel regions covered, as showed in Figure 3. This is a desirable behavior, better than in the other generative models, where we can see clusters of generated molecules.\n\n=============================================================================\n\nCons:\n\n1. The proposed strategy seems constrained by the selection of the fragment dataset.\n\n2. The instruction of calculating probability densities over fragments in the vocabulary is not clear. More precisely, the authors states that hidden state for fragment graphs is given by h^{graph} = MaxPooling({h^{node}_{u}}), however they do not state whether the nodes taken for aggregation are from the fragment itself or from combined fragment and molecule x or whether from the molecule x only.  \n\n\n=============================================================================\n\nQuestions during rebuttal period:\n\n1. It is not stated in the text what is the initial molecule. Do you have some starting set of molecules or always start from the same?\n\n2. How fast is the convergence of the proposed method? It would be nice to see some plot with time steps on X axis and score on Y axis.\n\n3. The authors did not state what was the number of train steps used to converge MARS in their experiments. What number of time steps did you use? How long does the MARS training takes, compared to other methods?\n\n=============================================================================\n\n=============================================================================\n\nReasons for score: \n \nOverall, I vote for accepting this paper. The problem of multi-objective molecules optimization is hard and really important in cheminformatics. The idea proposed by the authors is novel and confirmed experimentally. Hopefully the authors can address my concern in the rebuttal period. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "0nBFEm3AhNS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: The paper proposes a sampling-based approach for multiple property optimization in molecule generation.\n\nStrength: the sampling approach is an interesting new direction for molecule generation. Also multi-objective molecule generation is an important task.\n\nWeakness: I have several concerns regarding this paper.\n\n(1) Section 3.2 is unclear, with lots of details missing. For example,\n--- what is the input for MPNN? i.e., x in M_\\theta(x).\n--- How to determine the supervision signal p_add, p_delete for M_\\theta(x)?\n--- When  add/deleted fragments, how many fragments are added/deleted in a single transition kernel?\n--- If multiple fragments are added/deleted, are they added/deleted sequentially or following some particular rule?\n--- The MPNN description (Eq.5--9) is borrowed from [1], please cite the paper.  \n--- The paper trains the MPNN during sampling procedure, which seems very challenging, in the first several steps the model is from scratch, how to guarantee the sampled molecule is reasonable? If initially, the sampled molecules can not provide informative supervision, it will be hard to train a strong MPNN model. Also, the number of training samples also looks very limited. In [1], it requires a large amount of existing drug molecules to pretrain a good MPNN model. \n\n(2) Unfair comparison between the proposed sampling model and non-sampling baselines. For example, when sample size N=5000, the proposed model will query oracle to select a subset of generated molecules. \nHowever, baselines such as JTVAE directly output the generated molecules, all of which are kept. Therefore, in performance comparison if you compare the best molecules in 5000 molecules with a single molecule generated by JTVAE, this is unfair. Please clarify.\n\n(3) complexity issue: How many times do the authors need to query the oracles when performing optimization using the proposed sampling algorithm? How does this compare to the baselines? Each time you sample a new molecule, you need to evaluate the acceptance rate in Eq.2, while such an evaluation requires evaluation of target distribution in Eq.1 as well as need to call the oracle (e.g., evaluating the property), which is very expensive.\n\n(4) Lack of theoretical guarantee. The authors need to show the sampled molecules follow target distribution. The authors also argue they adaptively train the MPNN, so the transition kernel will change during sampling. It is non-trivial to guarantee the convergence to target distribution. \n\n(5) There are also issues with experimental setting and results. For example\n--- Important baselines are missing. For example, Graph2Graph [2] and hierarchical generation [3].\n--- One baseline RationaleRL outperform the proposed MARS in terms of success rate in many settings.\n--- The fairness in model comparison issue I mentioned in (2)\n--- How to use t-SNE to visualize molecule distribution? It needs more details.\n\n[1] Hu et al, ICLR 2020, Strategies For Pre-training Graph Neural Networks\n[2] Jin et al, ICLR 2019, Learning Multimodal Graph-to-Graph Translation for Molecule Optimization.\n[3] Jin et al, ICML 2020, Hierarchical Generation of Molecular Graphs using Structural Motifs.", "extra_scores": {}, "extra_text": {}}, {"review_id": "02NDzIaiWQF", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\n\nThis paper proposes a method to generate molecular graphs with multiple optimized properties. Molecular graphs are constructed/edited by the iterative addition and removal of molecular fragments. A MCMC search procedure, guided by a learned graph neural network that proposes good graph edit actions, is used to sample molecules with optimized properties. The proposed model is compared with some baselines on a few multi-objective optimization tasks and shows good performance.\n\n##########################################################################\n\nReasons for score: \n\nOverall, I vote for acceptance. The paper presents an interesting method for multi-objective molecule optimization that shows good performance in the evaluation. However, I have some concerns about the benchmark tasks and baselines that hopefully could be addressed\n\n##########################################################################\n\nStrengths:\n\n*Paper is written in a clear way, and is well structured\n\n*Proposed model shows very good performance in the evaluation, compared to other baselines\n\n*Ablation studies that provide useful insight about the model\n\nWeaknesses:\n\n*Some concerns about the benchmark tasks and baselines (see below)\n\n##########################################################################\n\nQuestions and other comments:\n\n*I think that the summary product score (success rate x novelty x diversity) could be a bit misleading for the casual reader. Looking at Table 2, it is clear to me that different models have different performance characteristics in relation to the success rate, novelty and diversity model performance metrics. In a practical use case, one may not equally weight the importance of these performance metrics, and instead decide on a trade off depending on the specific problem. Eg based on Table 2, we would pick the MARS model for high success rate, GA+D for high novelty, RationaleRL for high diversity. However, the simple summary product score (especially with the highlighted column in Table 1 and 2) make this nuance harder to see.\n\n*The abstract claim: \"[The method] outperforms the best prior methods by 100% in terms of success rate, novelty, and diversity of generated molecules.\" is a bit ambiguous, since it seems to only happen with the product score in 1 of the 6 tasks\n\n*Do you have any data on the ability of the proposed model to optimize raw property scores? Although it is useful to see the proportion of molecules generated containing properties above a predefined threshold, sometimes we may be interested getting the handful of best molecules. Eg the most potent inhibitor, or to a lesser extent the molecules with the highest QED and SA scores.\n\n*For the success rate metric, what was the reasoning for the thresholds? Eg GSK3B/JNK3 > 0.5, QED > 0.6, SA > 0.67\n\n*How did you implement the weighting for the multiple objectives (GSK3B/JNK3, QED, SA) in GCPN and JT-VAE? It seems that the weighting of the different objectives is another set of model hyperparameters that requires appropriate tuning.\n\n*Any thoughts about the applicability of some public molecule generation/optimization benchmarks for this work, eg https://github.com/BenevolentAI/guacamol?  \n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work proposes a method for generating candidate molecules using a novel fragment-based MCMC proposal mechanism.

Pros:
* Well-written paper
* Novel idea for an important application
* Very good empirical performance compared to the state-of-the-art in multi-objective molecule generation
* Careful ablation studies

Cons:
* Some details were missing (runtime, experimental details) and have been added to the revised version.

The authors engaged in an extensive discussion with the reviewers and modified their paper to address the reviewer concerns.

After discussions three reviewers recommend accepting the work and consider it a novel and useful contribution to the field.

One reviewer (Reviewer 3) is not satisfied by the authors comments and has concerns about the work regarding: asymptotic correctness of the sampling; fairness of the experimental comparison; and computational complexity.  The authors provide detailed justifications for their choices.  After looking at the discussion there are two factors:
1. technical arguments regarding the correctness of the sampling method; the authors justify the correctness by known results for adaptive MCMC methods, and the argument is sound, and the area chair fully accepts the authors' arguments as correct and applicable.
2. extend of the experimental evaluation and suitable baseline methods; this is partially subjective.  The authors provide extensive experiments in their work and justify exclusion of certain methods in that they do not easily apply to the multi-objective setting.  In addition, Reviewer 3 demands a comparison of generated molecules per time, which is plausibly useful, however, none of the prior works have used such a metric in a consistent manner and it is clearly challenging to do so fairly as such metric would depend on specifics of the implementation and computer.  The authors have updated their paper and added runtime information for their method.  The area chair fully accepts the authors' arguments and justification for the current experimental scope.

In summary the area chair considers the remaining concerns by Reviewer 3 as invalid; in particular, the authors have made extensive efforts to engage and educate the reviewer.
Decision: Accept (Spotlight)