Deep Evolutionary Learning for Molecular Design
Authors: Yifeng Li
Hsu Kiang Ooi
Alain Tchagang
Venue:  (2021)
Keywords: Deep Evolutionary Learning
Fragment-Based Drug Design
Deep Generative Model
Drug Design
Multi-objective Optimization

Abstract: In this paper, we propose a deep evolutionary learning (DEL) process that integrates  fragment-based deep generative model and multi-objective evolutionary computation for molecular design. Our approach enables (1) evolutionary operations in the latent space of the generative model, rather than the structural space, to generate novel promising molecular structures for the next evolutionary generation, and (2) generative model fine-tuning using newly generated high-quality samples. Thus, DEL implements a data-model co-evolution concept which improves both sample population and generative model learning. Experiments on two public datasets indicate that sample population obtained by DEL exhibits improved property distributions, and dominates samples generated by multi-objective Bayesian optimization algorithms.

Under review as a conference paper at ICLR 2021
DEEP EVOLUTIONARY LEARNING FOR MOLECULAR DESIGN
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we propose a deep evolutionary learning (DEL) process that in- tegrates fragment-based deep generative model and multi-objective evolutionary computation for molecular design. Our approach enables (1) evolutionary op- erations in the latent space of the generative model, rather than the structural space, to generate novel promising molecular structures for the next evolution- ary generation, and (2) generative model fine-tuning using newly generated high- quality samples. Thus, DEL implements a data-model co-evolution concept which improves both sample population and generative model learning. Experiments on two public datasets indicate that sample population obtained by DEL ex- hibits improved property distributions, and dominates samples generated by multi- objective Bayesian optimization algorithms.
INTRODUCTION
A drug is a molecule that binds to a target (e.g. protein) to inhibit or activate specific pathways in pathogens host cells that cause abnormal phenotype. Drug discovery and development is costly and time-consuming process, which is compounded by personalized medicines development for cancer other complex and rare diseases. Computational drug discovery has been shown to accelerate the whole discovery process using simulations and machine intelligence. However, chal- lenge remains in this field by demands for robust and unbiased feature representation theories for molecules and their corresponding receptors, and efficient search algorithms. The rise of Al and data science provides us with unique opportunity reevaluate the problem and develop fast intelligent search or design approaches (Gromski et al. 2019: Chen et al. (2018). These new technologies claim differences from the traditional ones in two aspects: (1) features can automatically learned using embedding techniques on large number of training samples, and (2) high-level relationships (in su- pervised case) and complex distributions (in unsupervised case) can be captured using appropriate deep architectures. Recently, new representation theories and architectures have been proposed in the domain of molecular generation. There exist two new major methods to present a molecule for machine learning algorithm. The first method converts a molecule structure to a string, such the simplified molecular-inpe input line-entry system (SMILES) string (Weininger, 1988), and adopt natural language processing (NLP) methods for supervised unsupervised learning. The second undirected graph to present a molecular structure and applies graph convolutional neural networks (Duvenaud al. 2015). Three major families of AI algorithms have been developed for novel drug discovery: deep generative models (DGMs), reinforcement learning, and the combination of both.
an
As one family of major neural probabilistic models for data modelling, generative autoencoders (e.g. variational autoencoder (VAE) (Kingma & Welling 2014)) have been adopted to learn on either SMILES strings KRomez-Bombuut et al. (2018) or molecular graphs (Simonovsky & Ko- modakis 2018) with corresponding physical and biochemical properties for molecular generation. By integrating the generative adversarial nets and autoencoders, adversarial autoencoders (AAEs) have also been well applied to molecular design (Kadurin et al.) 2017). The advantage of using generative autoencoders that, molecules, as discrete objects in our world, are mapped to the con- tinuous latent space, whose landscape can be organized by their properties, which helps generate new structures with preferred property values. However, critical problems remain due to imperfect representation methods. When SMILES strings used in VAE, the model suffers from imbalance of tokens in embedding, generation of invalid structures, and the problem where two almost identi- cal molecules have markedly different canonical SMILES strings. When using graph molecular

---

Under review as a conference paper at ICLR 2021
representation in VAE, technical difficulty is to design an effective graph decoder. In addition to other heuristic methods, a SMILES decoder can be used to pair with graph encoder. While DGMs offer convenience of searching in latent space, reinforcement learning algorithms can directly search in the molecules' structural space by adding or deleting bounds and atoms (Zhou et al. 2019). ) In the Markov decision process (MDP) for drug design, the agent is molecular generator, the molecular structure indicates the state, the actions are modifications to the current structure, and simulator (e.g. surrogate model) is often used as the environment to provide reward. Furthermore, generative and predictive models can be integrated in MDP to form deep reinforcement learning (DRL) meth- ods, where the generative model trained as a policy approximation and the predictive model can be used as a value function approximation (You et al. 2018: Popova et al. (018). Search in the discrete input space and inefficient learning are arguable concerns to be addressed when applying monfoorement-leaninng--e
solutions for compound design.
Interestingly, as an old peer of reinforcement learning for black-box optimizations, evolutionary computation (EC) methods (Eiben & Smith 2015) have been catching up with promising perfor- mances in modern optimization, design and modelling problems. Besnard et al. (2012) present a strategy for evolution of ligands along multiple properties in the structural space, where library of knowledge-basee based chemical structural transformation is used as the mutation operator. Interactions between EC and neural networks have mainly focused network evolution and neural surrogate models for fitness functions. For examples, EC has been used at a large scale for neuroevolution that leads to evolution of neural network architectures (Stanley et al. 2019); feedforward neural network is commonly used as fitness function in EC (Mandal et al. 2019). Furthermore, it has been recently discovered that evolutionary strategy (ES) can periorm competitively with reinforcement learning in game AI (Salimans et al., 2017). ES (Wierstra et al.) 2014) and estimation of distribution algorithms (EDA) (Hauschild & Pelikan| (001]) build parameterized search distributions over promising points and either employ gradient information or sample from such a probabilistic model to find better points. Both probabilistie strategies from EC can potentially be used as alternatives to Bayesian op- timization (BO) in (continuous) black-box optimizations. A single-objective BO has been recently applied to molecule optimization in the latent space of VAE (Romez-Bombaree Bombarelli et al. (0188).
Since model learning is essentially parameter estimation from the statistical modelling perspective, the quality of data in deep learning is crucial for model performance. Data augmentation is becoming new strategy in deep learning to improve the training of model. For example, in computer vision, transformations (such as rotation and flipping) of images are used to increase the sample size when the original data set is insufficient (Perez & Wang 2017: Cubuk et al. 2019 Shorten & Khoshgoftaar 2019). In NLP, a text dataset can be augmented using tricks such as replacing words or phrases with their synonyms (Wei & Zou 2019). and resorting aids from other language models (e.g. word embedding and neural machine translations) (Sennrich et al., 2016 Wei & Zou] (211)). Basically, these methods either increase the data by transforming existing information which can only alleviate the limit of certain techniques (e.g. convolution), or indirectly borrow new information from other sources (e.g. methods in NLP).
In summary, even though modern machine learning, evolutionary computation, and data science methods have been applied to molecular design and achieved promising results, we are still chal- lenged by three chief issues. (1) An effective representation method for compound structures, that is friendly and invariant to multimorphic forms, is still missing. (2) New idcas are expected for effective representation and coding of discrete structures in EC. And, (3) quality of data can be further improved as current data augmentation tricks only increase the number of samples but does not specifically address data quality. In this paper, we propose novel deep evolution- ary learning (DEL) process that combines the merits of deep generative model and multi-objective evolutionary computation together for molecular design. Specifically, our work has three major con- tributions. (1) In our approach, latent representations of phenotypic samples in population serve as genotypic codes for evolutionary operations. This approach differs from traditional evolutionary al- gorithms that search in the original space of a problem. Specifically, our framework's DGM encoder projects molecular structures in a population from discrete space to continuous latent space where evolutionary operations are applied to help explore the latent representation space. Subsequently, the DGM decoder maps the genotypic representation to the phenotypic space for generating new molecules with desired property values. (2) In each evolutionary generation, the newly formed population containing novel competitive molecules can be used to further fine-tune the DGM. This approach is an innovative data augmentation strategy that enriches training data with novel high-
2

---

Under review as a conference paper at ICLR 2021
quality samples. The whole DEL process implements a new learning paradigm that co-evolves data and model alternatingly through multiple evolutionary generations. (3) Our comprehensive experi- ments demonstrate that DEL is able to produce populations of novel samples with improved values of properties and outperforms state-of-the-art multi-objective BO algorithms (MOBO).
2
METHOD
The proposed deep evolutionary learning (DEL) process combines deep learning and multi-objective EC through the latent representation space of molecules. One of the theoretical innovations of our approach is that demonstrates that EC methods are extendable to corresponding deep versions. The main idea is illustrated in T formally presented in Algorithm [ (see Appendix (..en This algorithm consists of the following steps. (a) A VAE (as molecule modeller) and a multi- layer perceptron neural network (MLP, property predictor as regularizer) are pretrained using all the original training data to start the first evolutionary generation, or, if not in the first generation, using samples from the previous population. (b) Training samples (if first generation) or popula- tion samples (otherwise) are projected to the latent space using the encoder of the VAE. (c) Based on non-dominated ranking and crowding distances of samples with respect to multiple properties, evolutionary operations (selection, recombination/erossoeee and mutation) are conducted on the la- tent representations of the samples. (d) Given these new latent codes after evolutionary operations, new molecule samples are generated by the decoder of the VAE, (e) Properties of these generated samples are obtained using a simulator (e.g. RDKit (Landrum] 2006) in our experiment). (f) New samples with good desired properties and good samples from the previous generation form the new population. (g) Steps (b-f) are iterated for multiple generations. (h) The final population is returned.
Lewerr Sottma
GRU
GRU
GRU
GRU
Poiro memu popooittion
Porto uew
Poputatren 113
Wopelsss
(a) DEL.
(b) VAa.
Figure 1: Deep evolutionary learning process and deep generative model integrated in DEL.
The major advantages of our DEL algorithm over existing interactions between EC and neural net- works can be explained as follows. (a) We directly evolve a collection of data rather than many neural network structures and parameters. Data evolution tends to be more efficient than direct evo- lution of model structures and parameters. (b) The single neural network model (i.e. DGM in DEL) can be indirectly improved through learning the evolved data with modern gradient-based varia- tional learning and inference algorithms. Thus, the improvement of populations along evolutionary generations be viewed as effective data augmentation strategy that includes novel and high- quality samples for further training of the neural network. (c) The continuous latent representation space established by the encoder of DGM can be naturally used encoding (genotypic) space for evolutionary computation. Thus, evolutionary operations are carried out in the latent space instead of the discrete structural input space, allowing more efficient and smooth exploration, because the latent space is often multimodal and can be organized by properties (regularized by the property predictor) and evolutionary operations in this space can help the search escape from local regions and explore new regions of interest. (d) The multi-objeccive operations crowding distance, can help identify competitive and diverse parent samples breed offspring. In summary, DEL takes advantages of both multi-objective EC and probabilistic neural model learning- The DGM, multi-objective components (non-dominated sorting and crowding distance), evolution- ary operations, formation of new populations are discussed in details as below.
con-dominated sorting and
2.1 FRAGVAE FOR FRAGMENT-BASER MOLECULAR MODELLING
In our DEL process, we adopted a VAE model originally for fragment-based molecular genera- tion (Podda et al. 2020). The concept of fragment-based drug design (FBDD) introduced in
3

---

Under review as a conference paper at ICLR 2021
Shuker et al. 1996). In FBDD-based approaches, small organic molecules that bind proximal subsites of a protein are identified, optimized, and linked together to produce high-affinity ligands. Wet-lab approaches for FBDD include X-ray crystallography and NMR spectroscopy. Compared to atom-based drug design, FBDD has the following advantages (Frlanson 2011). (1) The search space in FBDD is much smaller (10" versus 10 (2) Identifying fragment with certain affinity to the target may mean finding a pharmacophore. (3) Fragment-based synthesis could be more ef- ficient than high-throughput screening. Majority fragmentation methods, which break a molecule into parts, are based on synthetic accessibility. For examples, RECAP (retrosynthetic combinato- rial analysis procedure) is method that breaks bonds formed by chemical reactions (Lewell al.] 1998]; BRICS (breaking of retrosynthetically interesting chemical substructures) generates a more elaborated set of fragmentation rules along synthetically accessible bonds and generates more frag- ments than RECAP (Degen et al.] 2008). BRICS is used in (Podda et al., (020) to chop SMILES string into several fragments. Then, fragment embeddings are produced using Word2Vee (Mikolov) et al. 2013). Next, the sequences of fragments are modelled by GRU-based VAE. In our work, a multi-head feedforward neural network component for predicting values of properties is added to the original model such that the latent representations can be regularized by properties of interest. Additionally, we normalize the three loss terms using batch size, and allow tuning of the weights among the loss terms. A crucial implementation bug in the original VAE model was also corrected (see Appendix (A.2). Hereafter, we name this modified VAE model for fragments as FragVAE whose architecture displayed in Figure lb
We denote the encoder parameter by Φ. the decoder parameter by 0, and the property predictor network by (z) parameterized by sb. The objective (to be minimized) of this DGM employed in DEL is a weighted combination of three terms:
10.0. (=(p)) [log po (ax [h]] + (XL (z)()) + (=(m)) [MS((f v)] (1)
where the first term is to reduce the reconstruction error, the second term is to regularize the posterior latent distribution with a simple prior, and the third term uses mean squared error of property predic- tion to further regularize the posterior distribution of latent codes. Previous studies unveil that VAE can easily fail on modelling text data because of the training imbalance between the reconstruction error (difficult to reduce once the KI. divergence becomes very small) and the KI. divergence (easy to diminish to zero). Thus, proper trade-off between the reconstruction error and KL divergence through 3-VAE (Higgins et al. (017) is vital in text generation and molecular generation (Yan et al.) 2020 Bowman al.) (016). In practice, the value of / should be smaller than 1. To look for suitable value of [3, we design a versatile function, called (-function, as formulated below,
B() min { max (1- u.
where T represents the total number of epochs, t E {1. ... T) indicates the current index of epoch, i controls the incremental speed, a defines the amplitude, and serve as lower and upper bounds respectively for the value of 3. With different settings, a variety of curves of this function are shown Figure 6 (see Appendix A.7 The value of can be set similarly in Frag VAE.
2.2 NON-DOMINATION RANK AND CROWDING DISTANCE
To get non-domination rank and crowding distance of feasible solution for guiding the sample selection (Section (.3) and the population merging (Section (2.4), the fast non-dominated sort and crowding comparison methods are adopted from the classic NSGA-II algorithm for oujti-obeective optimization (Deb et al. 2002). The properties in molecular design are treated as objectives. In an optimization problem with K objectives f(z) = (fi(z). ..................... .... [zx])) feasible solution 21 is said to dominate 22 (denoted by 2 < x2), if vk E {1,..., "1....,K.. fx(2)) < (z2) and 3k E {1,---- , K} }: fu(z1) < fi<(z)). Using this concept of domination, all feasible solutions in collection can be sorted to form Pareto frontiers (or fronts, ranks) F == =[F, F2, ... }. Samples in the same frontier do not dominate each other. Frontier F, dominates F, for j > i. Thus, we define function F(z) to retrieve the rank (i.e. frontier index) of any feasible solution in the population.
The crowding distance of a feasible solution computed as the normalized perimeter of the cuboid formed by its immediate neighbours along all objective axes. To compute the crowding distance of 2,, the normalized distance between its nearest neighbours above (denoted by za) and below (denoted by 2b) it w.r.t. the k-th objective axis is calculated using du = fe(x,) fe6l where F mmlm
4

---

Under review as a conference paper at ICLR 2021
Jk Ja and Ji are respectively the maximal and minimal values of the k-thh objective. Then, these individual results are summed up form the crowding distance 2;: d(z,) = Σ de(zi). The concept of crowding distance measures the density of the area around feasible solution.
Using the two concepts, partial order can defined. We say 21 Xn 22 either (1) 21 22 (that F(zi) < F22))), or (2) F(z1)= (22) and ((=1)= > d(z2). When two solutions have same rank, the one with larger crowding distance is preferred because it helps maintain a diverse population.
2.3 EVOLUTIONAAY OPERATIONS
The evolutionary operations include parent selection, recombination and mutation to produce new offspring in the evolutionary process. Binary tournament selection is applied to select one of two randomly drawn samples from the current population. In such a selection process, let us suppose 21 and 22 are randomly taken from the population and 21 Xn Z2. Sample 21 will be selected with selection probability Ps which is close to one, and will be selected with a small chance 1-ps This selection process is repeated M times to thus find M parents where M is the fixed population size. A pair of such parents will produce two children through recombination and mutation operations. Given two parents' latent representation zp1 and xp2. there are two recombination options linear and discrete methods, produce their new children & and 22. For linear recombination, & pp +ri (xp2 (xp2-$p1 and & = 2p1 2 (2p2 -xp1 xp1) where 71 = - =d+(d++)) r2 = -d+ (1+(2dd))a) Uniform(0,1), For discrete method, supposing latent representation vector is of length L, an integer 7 is randomly drawn from {1,
d = 0.25, and 01 01,02
..... ........ 1] such that & = [=pi[1 1]. xp2 [1 + 1 L]] and & = [2p2[1 1. zp1ll + L]]. After crossover, new sample 2m (say 0.01) of getting mutation. For 2m. a random value is drawn from Uniform(O, 1). If Pm, then random integer / is randomly selected from {I, L} such that the 1-th position of 2, is replaced with a value drawn from standard Gaussian distribution: 2n ~ NN
(m E {1, {1, M) }) will have small mutation probability
2.4 FORMING NEW POPULATION
Ideally we need to maintain excellent and diverse populations. After possible mutation operations, all M genotypic coding vectors will pass through the decoder of the DGM produce phenotypic samples. All valid samples (supposed in set P.+1) will be kept merge with the previous population (denoted by Pt) to produce a new generation (denoted by 1)+ ). To implement it, all samples in
P++1+ + Pe are sorted based on their non con-ommination ranks first and then on their crowding distances.
Finally, only the top M samples are taken from them form the new population.
3 EXPERIMENTS
The performance of DEL was investigated on the ZINC (Irwin & Shoichet 2005) and PCBA (Wang jet al. 2016) datasets. These data were processed in the work Podda et al. (220)). ZINC and PCBA are respectively composed of 227,94; and 383,790 molecules with two or more fragments. More statistics of both data can be found (Podda et al. 2020). We comprehensively investigated the empirical performance of FragVAE and DEL. Three properties (QED: quantitative estimation of drug-likenen SAS: synthetic accessibility score, and logP: water-octanol partition coefficient) selected as objectives in DEL. Molecules with large QED, low SAS, and small logP values prioritized. Incorporation of other properties (e.g. binding affinity, uruuctue-proppery relationship. and ADME) will be considered future work. QED, a scalarization of eight molecular properties (including logP) (Bickerton et al. 2012), is an adequate initial screening step for drug candidates. As we dive into more specific applications, selective properties can be tailored for subsequent screening. Thus, explicit usage of logP as one objective in DEL can help better assess lipophilicity, a key factor drug design for some diseases, e.g. kidney and heart problems.
3.1 EVALUATION OF FRAGVAE
As Vra a significant modification of the original model used in (Podda et al. 2020), we investigated the impact of 8 value to the performance of FragVAE VAE in terms of loss function values
5

---

Under review as a conference paper at ICLR 2021
through Figure 7 (see Appendix A.7 Other hyperparameter values can be found in Appendix A.3] One can see that a large 3 value can quickly reduce the KL loss to near zero which leads to stagnant reductions of reconstruction error and property regression error the notorious posterior collapse problem (Goyal et al. 2017), because it is much easier to reduce the KI. divergeee than the reconstruction error in complex sequence modelling. Using suitable small value of [3 would allow the continuous decrease of the reconstruction error and property regression error. This observation consistent with discoveries language generative models (Yan et al. 2020 Bowman et al. 20116).
Table 2 (see Appendix A.6) shows the validity, novelty and diversity of 20,000 samples generated from trained Frag using standard Normal prior to sample followed by the decoder. Results of previous language-man woaggge--o----eee model-based and graph- methods are also given for comparison. In general, the validity is defined as the ratio of number of valid generated samples to total number of generated samples. To clarify, the perfect validity reported in Podda et al. (2020) is actually calculated as the ratio of valid generated SMILES strings after discarding invalid fragment sequences versus total number of valid fragment sequences, i.c. Validity (SMILES) in Table 2 We found that this ratio is always I in fragment- puted the validity of fragment sequences the percentage of number of valid fragment sequences to total number of generated fragment sequences, i.c. Validity (Fragments) in Table [2] The novelty is defined as the ratio of number of generated novel valid molecules that do not exist the training data versus total number of generated valid samples. The diversity is calculated as the percentage of generated unique valid samples among total number of generated valid samples. When the value of B is very small (0.01), the posterior pl x (() is highly different from the simple standard Normal prior p(z). Thus, it is reasonable to see relatively low diversity in samples derived using standard Normal distribution. However, it does not imply that Frag VAE with very small value of / is poor at learning latent representation. In fact, previous work in 3-VAE shows that small values of B tend to encourage disentangled representations and form latent clusters (Li et al.) 2020).
models. To have a better understanding about the model, we hence com-
The property distributions of generated samples can be important indicators to the proximity of gen- erated samples with actual samples. Figure 2 shows the distributions of QED, SAS and logP in samples generated using standard Normal prior. Large 3 value can lead to the sample SAS distri- bution appearing at the right side of the actual SAS distribution. Interestingly, the property distribu- tions when using 3 = 0.01 do not resemble actual data, implying that using a very small value of B would lead to latent representations that deviate from standard normal distributions. To summa- rize, 3 S 0.1 can prevent the model training from posterior collapse and can form structured latent representation space which is useful for latent space exploration using optimization techniques.
ATOMS WN WO wOher
DON SIOGE WOUULL WTILLE
RINGS DDud wfeet
2BC
2BC
62
340.3
QED
LOGP
p-13
(a) Property distrihutions over ZINC
(b) Structural feature distributions over ZINC.
ATOMS WT WN WO water
BONDS SAOCE MDDUBLE TTRFEE
RINGS WTHH #D oPrel wHox
A1
62
20
$-0.11
1.0
10
QED
SAS
LOGP
PCAA P+i
$163 8-031
FQAA
p-11
ECBA
p-11p:400
(c) Property distributions over PCBA.
(d) Structural feature distributions over PCBA.
Figure 2: Property and structural feature distributions over 20K randomly sampled molecules.
3.2 DEL PERFORMANCE
We executed DEL processes using fixed or annealed loss trade-off weights: (1) 3 = 0.1 and a = 1, 1, (3) 3 = 0.1 and = 1 in the initial training of Frag then annealing to 0.4 and a to 4 in the second generation of DEL (denoted by 3 = 0.1 > 0.4, a = 1 -> 4), and similarly (4) 3 0.01 + 0.4, a 1 + 4. Other hyperparameter values can be found in Appendix A.4 The change of losses is shown in Figure g We observe that (1) all settings lead to similar reconstruction error convergence, (2) smaller values of tend to obtain smaller property
(2) 3 0.01 and

---

Under review as a conference paper at ICLR 2021
prediction error, (3) annealing can help reduce the KL divergence compared to the corresponding
fixed settings.
Figure shows population validity, novelty and diver- sity in different DEL processes. In this chart, validity (SMILES) is the validity of SMILES strings in the pop- ulation; validity (fragments) is the validity of fragment sequences sampled using FragVAR VAE after evolutionary operations; novelty is the ratio of population samples that are not in the training data against the population size; and diversity is the ratio of unique population samples against the population size. We observe that almost all samples in the populations are novel; and the population samples in the last generation are quite diverse, ranging from 0.798 to 0.988. Table ( lists the increasing numbers of high-quality novel molecules discovered along the DEL processes. In contrast to
Gerenaice
(SBLES))
(Foppracts)
Divrnay
01385
0.10
10
1.000
0.224
1.00
0.805
112-4
0.227
0.935
ZMCC
0.00
1.000
1.500
0.333
0.533
0.566
0.56
0339
1390
0.81
1,000
0.233
3.060
6.933
0.01 in B-l
1
1.000
0.31
3.800
D.81
1
0.343
0314
0:10
1.000
0.233
2.004
0.805
0.717
0.391
0337
U135
03004 4
1.00
03384
0166
0.81
1.000
0.351
8/677
0.51
1.000
0.23
1.000
0.225
0782
0.38
1184
1.000
0.27
1000
0.353
0.01
0.992
Figure 3: Population validity, novelty and diversity during DEL processes.
the training molecules visualized in Figures 99 and I0 (Appendix), DEL is able to discover novel and diverse high-quality molecules (see Figures [1]88] in Appendix). Furthermore, the distributions of proper- ties and structural features of samples along the evolutionary process are compared in Figure 4 (and Figures 2 in Appendix). It can be seen that the process can be quite different from the actual data distribution and able to gradually improve the distributions of QED, SAS and logP in popula- tion samples towards the preferred goals. Interestingly, samples randomly generated using standard Normal prior do not clearly show this trend (Figures 22 in Appendix).
ATOMS
BONDS
RINGS
JFC
JBC
00L.5
30
0.0
23
0.25
3.5 QED
6:15
1.00
4
-36
10
SAS
LOGP
TNC 08.1
MN DS
BVC (010) DRUN DRIP
ZBE DS 11 reto DS
(a) Property distributions over ZINC.
(b) Structural feature distributions over ZINC.
ATOMS WCC WT WN WD COMY
BONDS
RINGS WITH CAN mret mile
WSNGLC MODUDLE TRIPELE
LO
DELIE DOUS
C2
28
39
QED
LOGP
DH.P1
(c) Property distributions over PCBA.
(d) Structural feature distributions over PCBA.
Figure 4: Property & structural feature distributions of DEL populations (B=0.01 0.4, a=1 4).
3.3 COMPARISON WITH MULTI-OBJECTIVE BAYESIAN OPTIMIZATION
DEL was compared with two MOBO methods: q-Pareto Efficient Global Optimization (qParEGO) and q-Expected Hypervolume Improvement (qEHVI) (Daulton et al. 2020). These MOBO methods were run in the latent space of Vra trained in the first generation of DEL using all training samples. Hyperparameter settings of these algorithms are listed in Appendix A.5] Figure 26 shows the hypervolumes of both algorithms and the quasi-random baseline which selects candidates from a scrambled Sobol sequence along batches. Hypervolumes obtained using these models are plotted in Figure 26
It shows that qParEGO and qEHVI work better with B = 0.01 than that with 3 0.1.
To qualitatively compare DEL with the MOBO algorithms, the first five Pareto fronts obtained using DEL and last five batches obtained using qParEGO and qEHVI are visualized in Figure Figure 27 in Appendix. We can see that the solutions from qPaEGGO and qEHVI are behind the Pareto fronts from DEL. To quantitatively compare DEL with qParEGO and qEHVI, we conducted non- dominated sorting on the combination of the first Pareto front of DEL and the last six batches of qParEGO and qEHVI and report the results in Table [ It shows that all solutions from the DEL Pareto front stay in the new integrated Pareto front, while almost all solutions from gPrEEGO and qEHVI are behind the integrated Pareto front. Furthermore, as indicated in Table in Appendix,
7

---

Under review as a conference paper at ICLR 2021
DEL runs more efficiently than these MOBO algorithms, though the population size (20,000)
of DEL is much larger than the batch sizes (8) of MOBO algorithms. It has been a well-known challenge to scale up BO algorithms.
REL hert1
Dt. FronU1
Frentz
GEL Frent
FrentA
661 Frent5
DE, FrentS
Frent
gbest Front
opost Front
gdoat Froe
Froees
menks
ghutco
ghatco menk3
shutco Freek3
ghutco freks
shutco Froets
dendion frees
QED
oportio Froees
QED
63
10
logp
10
logP
(a) ZINC.
(b) PCBA.
Figure 5: Pareto fronts of DEL and MOBO algorithms (B 0.01 - 0.4). In the legend, the last
batch of qParEGO or qEHVI written as Front 1.
Table 1: Non-dominated sorting of combination of DEL, qParEGO and qEHVI Pareto fronts.
Pareto Front Size for Comparison
In New Pareto Front
Data
DEL Hyperarrrret
DEL
PaaEGGO
qEHVI
DEL
qParEGO
qEHVI
ZINC
0.1 -
243
46
45
243 (100%)
0 (0%)
(0%)
ZINC
0.01 - 0.4
200
46
40
200 (100%)
0 (0%)
(2.5%)
PCBA
3= 0.4
228
46
36
228 (100%)
0 (0%)
(0%)
PCBA
0.00 0.4
183
46
43
183 (100%)
(2.17%)
(4.65%)
3.4
ABLATION STUDIES
By default, DEL uses the property predictor for latent representation regularization, fine-tunes Frag- VAE using new population data each generation, forms new child latent codes using the linear crossover method, and fixes the population size 20K. Variants were created by (1) disabling the property predictor. (2) disabling finetuning, (3) using the discrete crossover method, and (4) allow- ing much larger population size (100K). While it is difficult to compare these variants in terms of validity, novelty and diversity of population samples (see Figure 28 in Appendix). it turns out that non-dominated sorting is informative method for comparison. Table 4 in Appendix A.6 indi- cates that DEL with the property predictor outperforms the variant without it. Table 5 shows that VAE finetuning in DEL can help obtain better Pareto fronts. Table(Jimplies that both linear and discrete crossover operations behave well in DEL. Also, DEL with larger population size can form better Pareto fronts (see Table 7 Additionally, we applied non-dominated sorting to compare the quality of Pareto fronts obtained using different values of B DEL, and found that the integrated Pareto front consists of samples relatively evenly from all settings (Table g
CONCLUSION
In this paper, we presented our DEL framework where a fragment-based VAE is integrated such that evolutionary exploration is conducted in the continuous latent representation space rather than the discrete structural space. Our intensive experiments show that DEL is able to generate novel populations of molecules with improved properties, and outperforms state-of-the -frt multi-objective Bayesian optimization algorithms. Applications small molecules. As future work. DEL will be tested on different datasets, other design problems, and more specific applications. Other types of DGMs and search strategies will be explored to further enhance DEL. New MOBO algorithms for latent-space based optimization need be studied further to address issues such as scalability, unknown invalid domains latent space, and the curse of dimensionality. Github link of our PyTorch and BoTorch implementation will be available.
DEL are certainly not restricted to design of
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
TJ. Besnard, G.F. Ruda, V. Setola, K. Abecassis, R.M. Rodriguiz, X. Huang, S. Norval, M.F. Sas- sano, A.I. Shin, L.A. Webster, F.R.C. Simeons, L. Stojanovski, A. Prat, N.G. Seidah, D.B, Con- stam, G.R. Bickerton, K.D. Read, W.C. Wetsel, I.H. Gilbert, B.L. Roth, and A.I. Hopkins. Auto- mated design of ligands to polypharmacological profiles. Nature, 412:215-220, 2012.
G.R. Bickerton, G.V. Paolini, J. Besnard, S. Muresan, and A.L. Hopkins. Quantifying the chemical beauty drugs. Nature Chemistry, 4:90-98. 2012.
S.R. Bowman, L. Vilnis, O. Vinyals, A.M. Dai, R. Jozefowicz, and S. Bengio. Generating sentences from continuous space. SIGNLL Conference on Computational Natural Language Learning (CoNLL), pp. 10-21, 2016.
H. Chen, O. Engkvist, Y. Wang, M. Olivecrona, and T. Blaschke, The rise of deep learning in drug discovery. Drug Discovery Today, 23(6):124111259 2018.
E.D. Cubuk, Zoph, D. Mane, V. Vasudevan, and Q.V. Le. AutoAugment: Learning augmentation strategies from data. In CVPR, 2019.
S. Daulton, M. Balandat, and E. Bakshy. Differentiable expected hypervolume improvement for parallel multi-objectivv Bayesian optimization. arXiv Preprint, pp. arXiv:2006.05078, 2020.
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjecctive genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2):182-197, 2002.
J. Degen, C. Wegscheid-Gerlach, A. Zaliani, and M. Rarey. On the of compiling and using 'duuglikee' chemical fragment spaces. ChemMedChem, 3030):1503 2008
D. Duvenaud, D. Maclaurin, J. Aguilena-Iparraguiree, R. Bombarell, T. Hirzel, A. Aspuru- Guziu, and R.P. Adams. Convolutional networks on graphs for learning molecular fingerprints. In NIPS, 2015.
A.E. Eiben and Smith. From evolutionary computation to the evolution of things. Nature, 521 (2014):476 482, 2015.
D.A. Erlanson. Introduction to fragment- drug discovery. Topics Current Chemistry, 317: 1-32, 2011.
A. Goyal, Sordoni, M. Cote, N.R. Ke, and Y. Bengio. Z-Forcing: Training stochastic recurrent networks. In NIPS, 2017.
P.S. Gromski, A.B. Henson, J.M. Granda, and L.. Cronin. How to explore chemical space using algorithms and automation. Nature Review Chemistry, 3:1 1:189,1 2019.
M. Hauschild and M. Pelikan. An introduction and survey of estimation of distribution algorithms. Swarm and Evolutionary Computation, (33:111-121) 2011.
I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner. beta-VAE: Learning basic visual concepts with constrained variational framework. In ICLR, 2017.
JJJ. Irwin and B.K. Shoichet. ZINC a free database of commercially available compounds for virtual screening. Journal of Chemical Information and Modeling. 45(1):177-182, 2005.
A. Kadurin, S. Nikolenko, K. Khrabrov, A. Aliper, and A. Zhavoronkov. druGAN: An advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico. Molecular Pharmaceutics, 14:3098-3104, 2017.
D.P. Kingma and M. Welling. Auto-encoding variational Bayes. In International Conference Learning Representations, 2014.
G. Landrum. RDKit: Open-source cheminformaties, 2006. URL http:
httt:://www.wte wwwp rok

---

Under review as a conference paper at ICLR 2021
X.Q. Lewell, D.B. Judd, S.P. Watson, and M.M. Hann. RECAP retrosynthetic combinatorial analysis procedure: a powerful new technique for identifying privileged molecular fragments with useful applications in combinatorial chemistry. Journal of Chemical Information and Computer Sciences, (1(3):511-522 1998.
X. Li, Kiringa, T. Yeap, X. Zhu, and Y. Li. Anomaly detection based unsupervised disentangled representation learning in combination with manifold learning. In International Joint Conference on Neural Networks, 2020.
S. Mandal, T.A. Anderson, J. Gottschlich, S. Zhou, and A. Muzahid. Learning fitness functions for genetic algorithms. arXiv Preprint, pp. arXiv: 199800888,, 2019.
T. Mikolov, K. Chen, G.S. Corrado, and J. Dean. Efficient estimation of word representations in vector space. In International Conference on Learning Representations, 2013.
L. Perez and J. Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv Preprint, pp. arXiv:1712.0462. 2017.
M. Podda, D. Bacciu, and A. Micheli. A deep generative model for fragment- based molecule gener- ation. In International Conference on Artificial Intelligence and Statistics, pp. 2240-2250, 2020.
M. Popova, O. Isayev, and A. Tropsha Deep reinforcement learning for de novo drug design. Science Advances, 4:85 2018.
R. Nomez-Bombarclee J.N. Wei, D. Duvenaud, J.M. Hernamdez-Lobat,, B. Sanchez-Lengeling T.D. Hirzel, R.P. Adams, and A. Aspuru-Guzik, Auto- matic chemical design using a data-driven continuous representation of molecules. ACS Central Science, 4:268- 276, 2018.
D. Sheberla,
T. Salimans, J. Ho, X. Chen, S. Sidor, and I. Sutskever. Evolution strategies scalable alternative to reinforcement learning. arXiv Reprint, pp. Sefi:::170.038888 2017.
R. Sennrich, B. Haddow, and A. Birch. Improving neural machine translation models with mono- lingual data. In ACL, 2016.
C. Shorten and T.M. Khoshgoftaar. A survey on image data augmentation for deep learning, Journal of Big Data, 6:60, 2019.
S.B. Shuker, P.J. Hajduk, R.P. Meadows, and S.W. Fesik. Discovering high-affinity ligands for proteins: SAR by NMR. Science, 274(5292): 1996.
M. Simonovsky and N. Komodakis. GraphVAE: Towards generation of small graphs using varia- tional autoencoders. In ICANN, pp. 412-422, 2018.
K.O. Stanley, J. Clune, J. Lehman, and R. Miikkulainen. Designing neural networks through neu- roevolution. Nature Machine Intelligence, 1:24-30, 2019.
Y. Wang, S.H. Bryant, T. Cheng. J. Wang, A. Gindulyte, B.A. Shoemaker, P.A. Thiessen, S. He, and
J. Zhang. PubChem BioAssay: 2017 update. Nucleic Acids Research, 45(D1): 2016.
J. Wei and K. Zou. EDA: Easy data augmentation techniques for boosting performance text classification tasks. In EMNLP-IJCNLP, 2019.
D. Weininger. SMILES, chemical language and information system. 1. introduction to methodol- ogy and encoding rules. Journal of Chemical Information and Computer Sciences, 28(1):31-36 1988.
D. Wierstra, T. Schaul, T. Glasmachers, Y. Sun, J. Peters, and J. Schmidhuber. Natural evolution strategies. Journal of Machine Learning Research, 15(2014):49-9880 2014.
C. Yan, S. Wang, J. Yang, T. Xu, and J. Huang. Re-balancing variational autoencoder loss for molecule sequence generation. arXiv Preprint, pp. arXiv:1910.00699,, 2020.
J. You, B. Liu, R. Ying, V. Pande, and J. Leskovec. Graph convolutional policy network for goal- directed molecular graph generation. In NeurIPS, 2018.
Z. Zhou, S. Kearnes, L. Li, R.N. Zare, and P. Riley. Optimization of molecules vis deep reinforce- ment learning. Scientific Reports, 9:10752, 2019.
10

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 DEL ALGORITHM
Algorithm 1: DEL Algorithm
Inputs: Xo. Yo training samples and corresponding properties;
Result: population of designed molecules; learned DGM: for each evolutionary generation do if first generation then
Yn: Train DGM using (X, Y):
X Xo:
// use all training data to learn DGM // a subset of M examples from the
X, Y = Subset(X,,') XM
original training data
else end
Train DGM using
// use population to further train DGM
end
Z Encoder (X)) // obtain latent representations of samples in X
F. nondominated sort(Y); // r: con-domination ranks, F: Pareto 10
fronti httpteecooooiindddhhiee Y F)) // ovolutionary operations: selection,
d
13
Z' = EvÖp(Z, Z, r)
recombination and mutation
14
X' = Decoder Z'):
// generate novel samples // obtain properties of the generated samples US ug simulaton e e.g. ROKit
15
Y get property(X").
",Y, X. Y crrm_new_populatinn (X, X', Y', M); 1/ keep the top M good samples from the combination of [X,Y] and (X',Y')
17 end
Return X, Y. DGM
A.2 BUG CORRECTION
In the VAE implementation by Podda et al. (2020), the incorrect use view function makes the forward flow of information messed up among different samples in batch.
Listing 1: Wrong of view function in the original VAE model in (Podda al. 2020
state = self.. rnn ( ppacedd, state)
batch by hidden_layer hidden_size state = state view batchhsize, self hidden-size * self hidden mean self rnn2mean state) mean: batch by latent_size
num almelsyers by batch by hidden_size
logvar = self rnn2logy (state) # logr: batch by latent-size
Listing 2: Our correction.
state self. rnn (packed, state)
num layers by batch by hidden -> batch by num layers by hidden size state state transpose .0)
state = state.. flatten simaat-dim=]) # batch by hidden-layer hidden mean self rnn2mean state) mean: batch by latent-size logvar = self. (state) # logv: batch by sitent-siee
A.3 HYPERPARAMETER SETTING FOR FRAGVAE
When not specified in the main text, the following values of hyperparameters used in experi- ments.
size of the embedding layer: 128

---

Under review as a conference paper at ICLR 2021
mask low-frequency fragments: Yes
masking frequency: 2
number of low frequency clusters: 5
window for word2vec embedding: 3
number of recurrent neurons per layer: 128
number of recurrent layers: 2
size of the VAE latent space: 64
maximum length of the sampled sequence: 10
batch size: 128
shuffle batches: Yes
number of epochs to train: 50
learning rate: 0.0005
dropout for the recurrent layers: 0.3
annealing step size for the scheduler:
annealing rate for the scheduler: 0.8
threshold to clip the gradient norm:
number of layers in MI. for properties:
number of hidden units in each hidden layer of MLP: 64
weight on property regression loss (a)): 1
weight on KL divergence (3): {0.01, 0.1, 1}
A.4 HYPERPARAMETER SETTING FOR DEL.
When not specified in the main text, the following values of hyperparameters are used in experi- ments.
number of evolutionary generations:
popularization size: {20000 (default), 100000}
number of epochs in the initial training of DGM: 50
annealing step size for the scheduler in initial training of DGM:
annealing rate for the scheduler: 0.8
number of epochs in a subsequent training of DGM: 30
learning rate in the initial training of DGM: 0.0005
annealing step size for the scheduler in initial training: 2
probability in tournament selection: 0.95
crossover method: (linear (default), discrete)
mutation rate: 0.01
A.5 HYPERPARAMETER SETTING FOR MOBO
When specified in the main text, the following values of hyperparameters are used in experi- ments.
number of initial data points: 1000
number of batches: 30
batch size:
number of MC samples: 128
A.6 ADDITIONAL TABLES
12

---

Under review as a conference paper at ICLR 2021
Table 2: Performance of Frag VAE in comparison with existing methods.
Model
Data
Validity (SMILES)
Validity (Fragments)
Novelty
Diversity
Chem VAE
ZINC
0.170
0.980
0.310
Grammar VAE
ZINC
0.310
1.000
0.108
SDVAE
ZINC
0,435
GraphVAR
ZINC
0.140
1.000
0.316
CGVAE
ZINC
1.000
1.000
0.998
NeVAE
ZINC
1.000
0.999
1.000
Frag VAB (,==1.00)
ZINC
1,000
0,922
1.000
0.961
Vra (f3=0.1)
ZINC
1,000
0.953
0.999
0.985
Frag VAB (8=0.01)
ZINC
1.000
0.655
0.997
0.809
Frag VAE ((3=1000)
PCBA
1.000
0,443
1.000
0.925
Frag VAB (6=0.10)
PCBA
1,000
0.481
0.983
0.886
Frag VAB (3=0.01)
PCBA
1,000
0.777
0.997
0.632
Table 3: Running time of DEL and MOBO. Since both methods involve training FragVAF, the FragVAF training time was not counted in this table. Format: Hours-iinuts:Secondds Note: when annealed to 0.4 in DEL, a is annealed from to All experiments were carried out on a Dell Precision 5820 Workstation equipped with Intel Xeon W-2255 CPU (10C), RAM of 128GB, and a Nvidia Quadro RTX 6000 (24GB) GPU.
Data DEL Hyperparameter
DEL
MOBO
ZINC
3= + 0.4
3:03:49
25:23:01
ZINC
==01
0.01 + 0.4
3:08:09
141:49:37
PCBA
3= 0.4
3:01:32
31:16:18
PCBA
B 0.01 + 0.4
1:50:20
45:31:38
Table Non-dominated sorting of combination of Pareto fronts from DEL with and without prop- erty prediction (PP) component as latent space regularization.
Pareto Front Size for Comparison
In New Pareto Front
Data ZINC
DEL Parameter
With PP
Without PP
With PP
Without PP
0.1
235
234
206 (87.66%)
0 (0%)
PCBA
= 0.1
224
108
216 (96.43%)
0 (0%)
Table Non-dominated sorting of combination Pareto fronts from DEL with and without DGM finetuning (FT) phrase.
Pareto Front Size for Comparison
In New Pareto Front
Data
DEL Parameter
With FT
Without FT
With FT
Without FT
ZINC
B = 0.1
235
193
235 (100%)
0 (0%)
PCBA
8= 0.1
224
214
199 (88.84%)
0 (0%)
Table 6: Non-dominated sorting of combination of Pareto fronts from DEL with linear or discrete crossover operation. Note: when 33 is annealed to 0.4, a is annealed from to 4.
Pareto Front Size for Comparison
New Pareto Front
Data ZINC
DEL Parameter
Linear
Discrete
Linear
Discrete
5 = 0.01 + 0.4
200
181
196 998.00%)
173 (95.58%)
PCBA
0.01 + 0.4
183
195
146 (79.78%)
184 (94.36%)
A.7 ADDITIONAL FIGURES
13

---

Under review as a conference paper at ICLR 2021
Table 7: Non-dominated sorting of combination of Pareto fronts from DEL with population sizes of 20K and 100K. Note: when 3 is annealed to 0.4, a annealed from 4.
Pareto Front Size for Comparison
In New Pareto Front
Data
DEL Parameter
20K
100K
20K
100K
ZINC PCBA
8 0.1 - 0.4
243
316
78 (32. 10%)
304 (96.20%)
3 = 0.1 - 0.4
228
354
88 (38.60%)
334 (94.35%)
Table Non-dominated sorting of combination Pareto fronts from DEL with different values g.
Note: when 3 is annealed to 0.4, is annealed from to 4. Population size: 20K.
Pareto Front Size for Comparison
In New Pareto Front
Data B = 0.13
0.1
0.43
0.01
0.01
0.4
8 =
0.1
0.1 + 0.4
3= 0.01
0.01 0.4
ZINC
235
243
228
200
195(82.98%
186(76.54%)
170(74.56%)
143(71.50%)
PCBA
224
228
189
183
169(75.45%)
183(80.26%)
122(64.55%)
111(60.66%)
Table 9: Numbers of novel molecules that satisfy properties QED 0.88, SAS < 3, and logP the 1st, 5th and final (10th) generations of DEL. Numbers of training molccules satisfying same conditions also given. Note: when 3 is annealed 0.4, is annealed from 1 to 4.
Data
Hyperparameter
Train
DEL(I)
DEL(5)
DEL(F)
ZINC
0.1
406
14
40
ZINC
0.1 0.4
22
32
ZINC
8 = 0.01
54
115
ZINC
0.01 + 0.4
20
PCBA
3 = 0.1
196
13
27
PCBA
0.1 0.4
10
52
PCBA
8 = 0.01
0
8
16
PCBA
0.01 0.4
10
== 11 0.01 0.4
20
40
60
80
100
Epoch
Figure 6: Different of forms of the 3-function. The total number epochs is 100.
14

---

Under review as a conference paper at ICLR 2021
30
1.5
0.0
CE Loss
0.01
KL Loss
20
1.0
15
10
0.5
0.0
10
20
30
10
30
40
14
12
MSE Loss
4.5
10
Loss
3.5
20
30
10
40
Epoch
Epach
(a) ZINC.
1.5
= 0.1
CE Loss
Loss
20
1.0
10
0.0
10
20
30
40
10
30
40
MSE Loss
3.5
3.0
Loss
2.0
10
20
30
40
10
30
40
Epoch
Epoch
(b) PCBA.
Figure 7: Loss of FragVAE VAE in initial training on ZINC and PCBA respectively.
15

---

Under review as a conference paper at ICLR 2021
foULowl
1.53
2-01-84.00000 3-m
Loss
15
0.53
130
130
220
220
120
130
200
250
3.6
Loss
MSE
2.4
3.2
2.4
50
100
150
200
250
0
50
120
150
200
250
0.43
4.5
255
2.
0.13
0.00
1.6
50
120
150
220
250
120
150
200
250
Epech
Epoch
(a) ZINC.
1.5
1.4
5-0.1-22.0----
2.0.01 a-
0.8
15
CE
0.5
10
0.4
0.3
135
150
220
350
50
120
150
200
250
2.25
2.00
3.5
3.5
0.25
2,4
13
0
30
100
1,30
250
229
0
50
120
120
200
250
Epech
Epoch
(b) PCBA.
Figure Loss of of Frag VAE in DEL ZINC and PCBA respectively.
16

---

Under review as a conference paper at ICLR 2021
Ooe Hod
onpos out $mo
omolo
BAO
Arrt
aoto
bo S
Aodo Scoto
t0y5
8
GRAINE MELEAA
GRARCIMA STUTO
xxx
Figure 9: ZINC training molecules that satisfy properties QED2 0.88, SAS 3, and logP< 1. Note: 196 molecules satisfy these conditions, but 64 visualized.
17

---

Under review as a conference paper at ICLR 2021
HoLa 3oO autp
dloe oomo og tard 3,000
1010
Oatt
toto
grot
tono OHO
out
xato
oniis
Figure 10: PCBA training molecules that satisfy properties QED 2 0.88, SAS < 3, and logP < 1. Note: 406 molecules satisfy these conditions, but 64 are visualized.
18

---

Under review as a conference paper at ICLR 2021
-ooo
plok mnto Plo
youoo
4OK VOOO Xrofo
XHHO
gook
mora mota
toto
Figure 11: Novel molecules that satisfy properties QED 2 0.88, SAS 3, and logP S 1 in the final generation of DEL trained on ZINC with hyperparameter. B 0.1, &
rtos
Foo
5ropp
OHOK
woo toodo
Figure 12: Novel molecules that satisfy properties QED > 0.88. SAS < 3. and logP 1 in the final generation of DEL trained on ZINC with hyperparameter: B = 0.1 + a1
19

---

Under review as a conference paper at ICLR 2021
plor
mod
coto toxe
g 1o1o Xooo
you coto
ot04 tooo you
moro
O-OOO vooa
80lo hard aco Otox toro
too Xopo for
5ooa ths
ooox
Fopo
Eoto Xaoo
tog
Dtts Aono
No
Figure 13: Novel molecules that satisfy properties QED > 0.88, SAS < 3, and log? < 1 in the final generation of DEL trained on ZINC with hyperparameter: B 0.01, 1.
20

---

Under review as a conference paper at ICLR 2021
plor soto
OOCIIN SAAAI
OBAIHI STOVOAA
NOK
Figure 14: Novel molecules that satisfy properties QED 0.88, SAS < 3, and logP S 1 in the final generation of DEL trained on ZINC with hyperparameter: / 0.01 0.4, a -
Loto Dod ooo uho gorg
foxx auts t0g
otgo
pooe
End
Xapoo
foro
Plor
Hapoo
Figure 15: Novel molecules that satisfy properties QED 2 0.88, SAS < 3, and logP < 1 in the final generation of DEL trained on PCBA with hyperparameter: 3 = 0.1, =
21

---

Under review as a conference paper at ICLR 2021
goox
Oo plo
COURD MASTINAA
200ING
OECLANT opter
GRAAI VENYAAA
Loto
fooaa 100x
Orly 1010
mayr so1a
ppo notd
X101
tooo R
gHos
OOK axtor
Ints OHOOK
Figure 16: Novel molecules that satisfy properties QED > 0.88, SAS 3, and logP < 1 in the final generation of DEL trained on PCBA with hyperparameter: 3 0.1 - 0.4, a 4
Jotery PrH
wel Doo
Figure 17: Novel molecules that satisfy properties QED > 0.88. SAS 3. and logP < 1 in the final generation of DEL trained on PCBA with hyperparameter: 3 = 0.01, 1.
22

---

Under review as a conference paper at ICLR 2021
Figure 18: Novel molecules that satisfy properties QED 2 0.88, SAS < 3, and logP S 1 in the final generation of DEL trained on PCBA with hyperparameter: , = 01.11 - 0.4, C 4.
ATOMS
BONDS
RINGS
JBC
DCL2
100
101
2.25
0:00
QED
5A5
LOGP
ZHC 06.10 00.50 03.7
BKC DEUDI DEUSH DEUT
ZBC
00.10 DELSS
(a) Property distributions ZINC.
(b) Structural feature distributions ZINC.
ATOMS
BONDS MODURA
RINGS WOWK whot WHA
WM
WOCK
WAOSAS
TUPA
X5
08J1 30.30 0231
KAA 06U21 DEUSI DEATT
PCLA 0001 00U31 023)
(c) Property distributions over PCBA.
(d) Structural feature distributions over PCBA.
Figure 19: Property and structural feature distributions of population samples during DEL (5 0.1).
ATOMS WOMY
BONDS MOOUBLE WTIEEE
RINGS
WSAGLC
2HC
21C
A4
10
2NC DRLL 34.30 D6
0RUZI 3EU5 DEP
JBC DELL DRL20 D&.)
(a) Property distrihutions over ZINC.
(b) Structural feature distributions over ZINC.
ATOMS
BONDS
RINGS
MODUBLE
FCB
-FOA
XEA
DCLF)
6.0
3.59 0.5 QED
SAS
LOGP
KOLA OEJ1 08.50
KBA 0EU21 DEUSI
KGR 02L13 05435
Property distributions over PCBA.
(d) Structural feature distributions PCBA.
Figure 20: Property and structural feature distributions of population samples during DEL (8
0.1 + 0.4).
23

---

Under review as a conference paper at ICLR 2021
ATOMS WM WO WAMY
BONDS
RINGS WFed
31C
ZNC 00J1 00.31 02-11
2NC (6UZ) SEUSI
SEUT
2NC DOLD DOU!! DAT)
(a) Property distributions over ZINC.
(b) Structural feature distributions over ZINC.
ATOMS
BONDS
RINGS
FCR
FCBA
D0.07
D0L71
22.F7
1.3 87% QED
tha
SA5
LOGP
PCEA 00J1 50.55 02.7
IODA (CU2) CCUSI ECUTT
PCDA D2.1D 0OUS!
(c) Property distributions over PCBA.
(d) Structural feature distributions PCBA.
Figure 21: Property and structural feature distributions of population samples during DEL (B 0.01).
ATOMS
BONDS MODUDLE WHPPL
RINGS mo WEWE wilec
WN WD WOMM
WENGLC
DOLID
DOLAS
935 355
QED
SAS
LOGP
(a) Property distributions over ZINC.
(b) Structural feature distributions over ZINC.
ATOMS
RINGS
WENGLC MOOUBLE TRIPEE
witex
FENA
FERA DELSI
TM
A5
C2
20
30
3.04
QED
ICEA DEJD 20.25 05.F1
KIA DEUJI DEUSI DEUT
PCA 00.10 00U.5 03.7)
(c) Property distributions over PCBA.
(d) Structural feature distributions over PCBA.
Figure 22: Property and structural feature distributions of randomly sampled molecules using Frag- VAE during DEL (3- = 0.1).
ATOMS
BONDS
2BC
21C
2
0.00
0.2h
2.5 QED
3:10
SAS
LOGP
EHC OBLLO 2M-36 DALF
05C ERUZI 2445 DRUH
JIC DiLL BAAS DAJ)
(a) Property distributions over ZINC.
(b) Structural feature distributions over ZINC.
ATOMS WCC HF WH WD CONY
BONDS MODUBLE WTIPLE
RINGS WITH OOM HFre wroo
WSNGLE
LG
D0L.1
DOLT1
QED
LOGP
(c) Property distributions over PCBA.
(d) Structural feature distributions over PCBA.
Figure 23: Property and structural feature distributions of randomly sampled molecules using Frag- VAE during DEL (3 = 0.1 + 0.4).
24

---

Under review as a conference paper at ICLR 2021
ATOMS
BONDS
RINGS
WORK
WONGLC
MODUBLE
mO
DOLID
825 C.5
QED
SA5
LOGP
(a) Property distributions ZINC.
(b) Structural feature distributions over ZINC.
ATOMS
BONDS
RINGS
RCA
RCA
0.00 8.3
1.0
38
KMA 08.1 28.25 05.1
KRA (EU3 DEUS DEUP
KCM 05.10 Detot 05.)
(c) Property distributions over PCBA.
(d) Structural feature distributions over PCBA.
Figure 24: Property and structural feature distributions of randomly sampled molecules using Frag- VAE during DEL (3 = 0.01).
ATOMS
BONDS
RINGS
19039
3.9 0.25
QED
SAS
LOGP
(a) Property distributions ZINC.
(b) Structural feature distributions ZINC.
WENGAC MOOUDLE THILLE
PLRA
DOLIS
D0L.5
20.01
QED
KM6 DR.0 34.20 DK
K3G (8131 2NUN DRIM
KCA DH.10 DHO DK
(c) Property distributions over PCBA.
(d) Structural feature distributions over PCBA.
Figure 25: Property and structural feature distributions of randomly sampled molecules using Frag- VAE during DEL (3 = 0.01 - 0.4).
25

---

Under review as a conference paper at ICLR 2021
76.55
Sabel
79.4
Sobol QENVI
-ParEG0
78.75
79.2
Hypervolume
Hypervolumt
78.70
79.0
78.65
78.60
78.8
78.55
78.00
Batch
Batch
(a) ZINC (3 = 0.1).
(h) ZINC (8 = 0.01).
193.5
Subel
Subel
-ParEGO
193.0
-ParEGO
189.9
Hypervolume
Hypervolume
189.5
191.5
189.7
191.0
189.0
189.5
189.5
Batchh
Batch
(c) PCBA (3 0.1).
(d) PCBA (3 0.01).
Figure 26: Hypervolume along batches when running Sobol random search, qParEGO, and qEHVI.
Fort2
Free2
CEL hert3
frents
Etl horts
Dt. Fronts
hrrrt
Freet
dial Froet2
gfbst Froet3
gfall Front
shutcofrett
oPutoo meet2
oPuted tree2
ghurico meet3
shutico Freet3
merts
ghuriod Frenes
QED
QED
0.
SAS
SAS
logp
logP
10
20
ZINC.
(b) PCBA.
Figure 27: Pareto fronts of DEL and MOBO algorithms (3 = -.11). In the legend, the last batch of qParEGO or qEHVI written Front
26

---

Under review as a conference paper at ICLR 2021
Data
Property Predictor
DGM
Crossover
Population Size
beta
alpha
Evolutionary Generation
Validity (SMILES)
Validity (Fragments)
Novelty
Diversity
linear
0.10
1.000
linear
20K
0.10
1.000
0.961
0.999
0.921
linear
0.10
1.000
0.554
1.000
0.805
N
linear
20K
0.10
1.000
0,986
1.000
0.960
N
linear
20K
0.10
5
1.000
0.539
1.000
0.989
N
linear
20K
0.10
10
1.000
0.971
1.000
0.920
Y
N
linear
20K
0.10
1
1
1.000
0.936
0.999
0.977
Y
linear
20K
0,10
5
1.000
0.947
0.999
0.978
Y
N
linear
20K
0.10
10
1.000
0,948
0.999
0.975
linear
20K
0.1 0.4
to 4
1
1.000
0.957
0.999
0.986
Y
linear
20K
to 0.4
to 4
5
1.000
0.259
0.999
0.986
Y
linear
20K
t. 0.4
to 4
10
1.000
0.396
0.996
0.986
ZINC
linear
100K
to 0.4
to 4
1
1.000
0.706
0.996
0.880
linear
100K
t. 0.4
to 4
5
1.000
0.432
1.000
0.334
linear
0.4
to
1.000
0.591
0.999
linear
20K
0.01
1.000
0.819
0.995
0.956
linear
0.01
1.000
0.998
linear
0.01
1.000
0.973
0.999
0.963
linear
linear
0.01 0.4
1.000
0,940
0.999
0.978
linear
20K
0.01 0.4
10 4
1.000
0,919
1.000
0.976
discrete
20K
0.01 0.4
10
1.000
0.888
0.997
0.887
discrete
20K
0.01 0.4
1o 4
1.000
0.974
0.995
0.934
discrete
20K
0.01 0.4
to 4
10
1.000
0.966
0.997
0.904
linear
20K
0.10
1
1.000
0.513
0.994
0.976
linear
20K
0.10
5
1.000
0.787
0.999
0.981
linear
20K
0.10
10
1.000
0.839
0.993
0.976
N
linear
20K
0.10
1
1.000
0.467
1.000
0.987
N
linear
20K
0.10
5
1.000
0.983
0.999
0.912
N
linear
20K
0.10
10
1.000
0.907
1.000
0.818
N
linear
20K
0.10
1
1
1.000
0.453
1.000
0.990
linear
20K
0.10
5
1.000
0.444
1.000
0.989
linear
20K
0.10
10
1.000
0.433
1.000
0.990
linear
20K
0.1 0.4
to
1.000
0.980
0.935
linear
0.
to 4
1.000
0.984
0.994
linear
PCBA
to
linear
t.
1.000
0.808
1.000
0.897
linear
100K
to
1.000
0.526
1.000
0.896
linear
20K
0.01
1.000
0.301
0.997
0.417
linear
20K
0.01
1.000
0.912
1.000
0.655
linear
20K
0.01
10
1,000
0,762
1,000
0.798
linear
20K
0.01 0.4
to 4
1
1.000
0.849
0.998
0.633
Y
linear
20K
0.01 0.4
to 4
5
1.000
0.827
1.000
0.863
linear
20K
0.01 to 0.4
to 4
10
1.000
0.755
1.000
0.895
discrete
20K
0.01 to 0.4
1o 4
1
1.000
0.670
0.996
0.318
discrete
20K
0.01 to 0.4
1 to 4
5
1.000
0.838
0.998
0.566
discrete
20K
0.01 to 0.4 to
10
1.000
0.869
0.996
0.568
Figure 28: Validity, novelty and diversity of population samples (after evolutionary operations and before merging with previous population) in different variants of DEL.
27


Reviews:
[{"review_id": "XoQhLfl-nX", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n= quality/ clarity = \nneeds work. see below.  too much focus on minor technical details (e.g. KL annealing for VAEs). not enough discussion of the more technically novel content, like the details of the multi-objective optimization method.\n\n= originality / significance = \nDraws on some interesting ideas from the evolutionary computation community on multi-objective optimization. However, these are not the focus of exposition. The VAE details overlap considerably with prior work.\n\nExperiments are for standard tasks from recent literature, but there is limited comparison to recently published methods.\n\n=Major Comments=\n\nThe Methods section is very dense. You should start with a definition of the problem. It was unclear, for example, that your optimization would be collecting multiple rounds of data and that you have a starting set of labeled and unlabeled data. You should definitely put an algorithm box in the main paper body and put many of the details that have been introduced in prior work (e.g. KL annealing) in an appendix. \n\nThe Methods section does not clearly delineate what is prior work and what is a novel contribution. Is there anything in Sec 2.1 novel, for example? Overall, is my assessment correct that your work combines an existing encoder-decoder model (FragVAE) with some methods from the evolution literature for multi-objective optimization? Is the primary novelty to adapt these evolution methods to work in latent space?\n\nThe difference between FragVAE and Podda et al., 2020 is minor, mostly consisting of the addition of a supervised regression network and some re-balancing of the terms in the loss, KL annealing, etc. Sec 3.1 of the results analyzes the impact of tuning these details. While this is important for a practitioner's model development, they will not be of sufficient general interest to readers. Sec 3.2 is also mostly focused again on understanding the impact of tuning parameter choice and does not compare to alternative methods from the literature.\n\nSec 3.3: The choice of baselines is very focused on comparing against other methods for multi-objective optimization. Why not also compare against other optimization methods for recent ML/chemistry papers that perform single-objective optimization? Many of these are referred to in the introduction. \n\nOverall, the methods in section 2.2 for multi-objective optimization are interesting and introducing them to the modern ML community is a valid contribution. However, the paper's exposition and experiments are too focused on details of the FragVAE model and do not analyze the details of the multi-objective optimization approach enough.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CQPrV1nQ7SW", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose to optimize the continuous representation of molecules in a latent space learned by a fragment-based variational autoencoder using an evolutionary algorithm. To improve the quality of the generated molecules over time, they use new generated samples as augmented data to fine-tune the generative model in each iteration. Experiments are conducted to demonstrate the effectiveness of the proposed algorithm compared with Bayesian optimization-based methods.\n\nThe paper paves a way for combining multi-objective evolutionary computation and deep generative modelling, which could be a potential benefit to other areas in machine learning. However, the paper is not ready for publication at ICLR, since it fails to discuss/empirically compare with highly related work, makes bold but incorrect statements about existing method, and lacks of clarity in presenting its own method.\n\n**Related work**\nRegarding multi-objective property optimization, \\cite{li2018multi, jin2020composing} investigate the same problem and the latter one is one of the state-of-the-arts. Regarding evolutionary algorithm/genetic algorithm in the drug design space, \\cite{Nigam2020Augmenting, leguy2020evomol} are highly relevant, and the authors should discuss how to position their work among these ones.\n\nThe second last sentence in the first page says, \"When SMILES strings are used in VAE, the model suffers from imbalance of tokens in embedding, generation of invalid structures,\" which is incorrect. There are many work in the space to address this problem, e.g. grammar/syntax guided variational autoencoders~\\cite{{kusner2017grammar, dai2018syntax, jin2018junction}.\n\n\n**Presentation**\nThe lack of details and clarity in the method section makes the paper hard to understand. For example, symbol \\bm{y} in equation (1) is never introduced in the context. Since the descriptions below mentioned \"mean square error\", I assume that \\bm{y} represents the ground true property produced by the simulator here. But the authors never explain the intuition behind this. Why is using the mean square error produced by the property predictor is helpful for training the generative model? This is very confusing and it would be better to jusify this in the text.\n\nThe role of the property predictor also confuses me, because we have this simulator which can produce ground truth properties. Is regularizing the generative model the sole purpose of the property predictor? It seems that in section 2.2 the predictor is involved in the evolution step. But this contradicts with section 3.4 where there is a variant in which the property predictor is disabled.\n\n**Evaluation**\nAlthough the paper compared with Bayesian optimization in the experiment section, it's still insufficient to convince the audience. Comparison with Bayesian optimization seems like an ablation study, and it would be hard to understand its effectiveness without comparing to other, especially state-of-the-art approaches. I suggest the author at least compare with the following baselines:\n\n1. a gradient-based baseline: continuous optimization in latent space using gradient of the learned property predictor.\n2. \\citep{jin2020composing}: one of the state-of-the-art in drug design, or other good-performing baselines using RL.\n\n@article{kusner2017grammar,\n  title={Grammar variational autoencoder},\n  author={Kusner, Matt J and Paige, Brooks and Hern{\\'a}ndez-Lobato, Jos{\\'e} Miguel},\n  journal={arXiv preprint arXiv:1703.01925},\n  year={2017}\n}\n\n@article{dai2018syntax,\n  title={Syntax-directed variational autoencoder for structured data},\n  author={Dai, Hanjun and Tian, Yingtao and Dai, Bo and Skiena, Steven and Song, Le},\n  journal={arXiv preprint arXiv:1802.08786},\n  year={2018}\n}\n\n@inproceedings{jin2018junction,\n  title={Junction Tree Variational Autoencoder for Molecular Graph Generation},\n  author={Jin, Wengong and Barzilay, Regina and Jaakkola, Tommi},\n  booktitle={International Conference on Machine Learning},\n  pages={2323--2332},\n  year={2018}\n}\n\n\n@article{li2018multi,\n  title={Multi-objective de novo drug design with conditional graph generative model},\n  author={Li, Yibo and Zhang, Liangren and Liu, Zhenming},\n  journal={Journal of cheminformatics},\n  volume={10},\n  number={1},\n  pages={33},\n  year={2018},\n  publisher={Springer}\n}\n\n@article{jin2020composing,\n  title={Composing Molecules with Multiple Property Constraints},\n  author={Jin, Wengong and Barzilay, Regina and Jaakkola, Tommi},\n  journal={arXiv preprint arXiv:2002.03244},\n  year={2020}\n}\n\n@inproceedings{\nNigam2020Augmenting,\ntitle={Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space},\nauthor={AkshatKumar Nigam and Pascal Friederich and Mario Krenn and Alan Aspuru-Guzik},\nbooktitle={International Conference on Learning Representations},\nyear={2020},\nurl={https://openreview.net/forum?id=H1lmyRNFvr}\n}\n\n@article{leguy2020evomol,\n  title={EvoMol: a flexible and interpretable evolutionary algorithm for unbiased de novo molecular generation},\n  author={Leguy, Jules and Cauchy, Thomas and Glavatskikh, Marta and Duval, B{\\'e}atrice and Da Mota, Benoit},\n  journal={Journal of Cheminformatics},\n  volume={12},\n  number={1},\n  pages={1--19},\n  year={2020},\n  publisher={BioMed Central}\n}\n\nUpdate after rebuttal:\nThanks for the response! Combining deep generative modeling with evolutionary algorithms is a very interesting idea in general. I hope the authors can continue to improve the paper (especially on the evaluation part) and resubmit in the future. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "3y3gVpvDs1", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors combine deep generative models and multi-objective evolutionary computation for computer-aided molecular design. In particular, they employ a variational autoencoder (FragVAE, as molecule modeler) and a multilayer perceptron (as property predictor). Evolutionary operations explore the latent space of the generative model to produce novel competitive molecules. The paper is interesting and tackle a very relevant problem. However, I detect the following severe limitation: \n\n- My main concern is that I have the impression that this research is quite incremental and may not be suitable for publication in a top-tier machine learning conference. We should not forget that Evolutionary Computation techniques have been extensively applied to computer-aided molecular design and bioinformatics. In fact, there are works on the subject already published in the 90's (see, for instance, [1, 2, 3]). In the same way, there are already successful examples of automated drug-design applications using multi-objective evolutionary algorithms [4]. In this regard, I am not sure that the fact that the contributions presented in the paper (last paragraph of page 2) are sufficient. Furthermore, if I'm not mistaken, the authors did not design any specific evolutionary operator (selection, crossover and mutation) for this task and relied on conventional ones. In the same way, the authors adopted an already existing VAE model for molecular generation (Podda et al., 2020).\n\n[1] Clark, David E., and David R. Westhead. \"Evolutionary algorithms in computer-aided molecular design.\" Journal of Computer-Aided Molecular Design 10.4 (1996): 337-358.\n\n[2] Parrill, Abby L. \"Evolutionary and genetic methods in drug design.\" Drug Discovery Today 1.12 (1996): 514-521.\n\n[3] Willett, Peter. \"Genetic algorithms in molecular recognition and design.\" Trends in biotechnology 13.12 (1995): 516-521.\n\n[4] Besnard, Jérémy, et al. \"Automated design of ligands to polypharmacological profiles.\" Nature 492.7428 (2012): 215-220.\n\nThere are other aspects that, in my humble opinion, should be clarified or improved in the paper:\n\n- Not clear if the properties of the generated samples are obtained using the MLP (as indicated in page 3 and Figure 1) or by the RDKit simulator (Landrum, 2006) (as also mentioned in page 3). Could the authors clarify this point?\n\n- How all hyperparameters (FragVAE, MLP, evolutionary algorithm) were selected and how dependent are the results obtained to the values selected? \n\n- Why only multi-objetive Bayesian optimization methods are included in the final comparison? Also, in relation to the experimental section, I think that in this final comparison the authors should employ statistical hypothesis testing to check the existence of statistically significant differences between the methods under comparison. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "1tT-49L0H4d", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: The paper proposes to tackle multi-objective optimization of molecular properties by combining a genetic algorithm with a fragment-based generative model of SMILES strings. Using a generative model allows to perform evolution in the latent space, as opposed to molecule space. Experiments show the model can produce a rich Pareto front of samples, outperforming bayesian optimization ran in the latent space of the same generative model.\n\nRecommendation: Overall, I am voting to reject. While combining evolutionary algorithms with deep molecular generative models is a very interesting research direction, the current draft uses a very limited set of metrics and baselines, making it unclear if the proposed method is comparable to the state-of-the-art. Technical novelty of the paper is rather limited (combining several previously published ideas), although the particular combination does seem novel. The approach looks promising, but with the provided set of experiments it's hard to compare it to existing methods.\n\nMain pros:\n1. The paper is easy to understand and explores a timely and important topic.\n2. Using concepts from multi-objective optimization (non-domination ranks and crowding distance) sounds like a good idea, given that some established molecular optimization works simply convert the multi-objective problem into a single-objective one by weighing the different terms, and using Pareto frontiers as done in this work could uncover a much richer structure in solution space.\n3. Evolution in the latent space is an interesting approach, which bears some similarities to using particle swarm optimization as in [1]. It would be interesting to see a comparison (quantitative and qualitative) of these two methods.\n\nMain cons:\n1. Most of my concerns are about weak metrics and baselines:\n- For benchmarking the generative model in isolation, the authors use validity, novelty, diversity, and simple molecular distribution statistics (atom, ring, and bond types, together with distributions of simple properties). This is a good sanity check, but a rather weak metric to compare to established models. Perfect validity, novelty and diversity can be obtained with an untrained graph-based model (e.g. [3]), while the other metrics do not capture more fine grained distribution statistics. It would be useful to also compute more complex metrics, such as those in [4] (KL divergence on a larger set of chemical properties, and Frechet ChemNet Distance), and additionally benchmark reconstruction success. Of course, a weak result on any distribution matching metric does not mean the model will not perform well for optimization; nevertheless, having the metrics for FragVAE would provide more insight into the model.\n- For benchmarking optimization, the authors jointly optimize three simple molecular properties. It is a bit unclear if that single task is representative of the broader class of problems encountered in generative chemistry. One could additionally optimize for the 20 tasks proposed in [4], and run quality filters on optimized samples to see if the optimizer exploited the scoring function (see [4] for the filters, and [5] for more discussion on the topic).\n2. The generative model itself is the one of [2], with the addition of a property predictor (to regularize the latent space), and a warm-up scheme for the KL-divergence part of the loss. Both modifications are rather standard, which raises expectations with respect to the experimental evaluation of the method.\n3. Some details of the comparison with BO are unclear:\n- Depending on the comparison, molecules from either 5 or 6 BO batches are selected; how was the number of batches chosen? Given that the evolutionary process by design is likely to retain all Pareto-optimal samples seen during optimization, wouldn't it be more fair to take all samples produced by BO, instead of a fixed number of batches? The paper mentions that the BO batch size is 8 - is it the case that evolution produces 20k samples, which are then compared to 40-48 samples from BO?\n- How many steps does BO do, and what is the dimension of the latent space? I am wondering what is contributing to BO being so slow (as seen in Table 3).\n\nOther comments:\n- In the loss for the property predictor, are the losses for the different kinds of properties normalized in some way?\n- The paper says \"One of the theoretical innovations of our approach is that it demonstrates that EC methods are extendable to corresponding deep versions.\" - this contribution is not theoretical; moreover, I wouldn't call the evolutionary algorithm deep (the generative model is)\n- \"Data evolution tends to be more efficient than direct evolution of model structures and parameters.\" - can the authors expand on what that means? Is that a result shown in the literature?\n- In Section 2.3, the sampling of r_i could be just from Uniform(-d, 1+d), instead of linearly transforming a sample from Uniform(0, 1). Moreover, I am curious about the fact that r_i can be outside of (0, 1), which means the child may land outside of the segment connecting the parents. Did the authors observe this to work better in practice that sampling r_i from (0, 1)? One danger there is that the samples may get drawn towards the origin, was this observed empirically?\n- \"To clarify, the perfect validity reported in Podda et al. (2020) is actually calculated as the ratio of valid generated SMILES strings after discarding invalid fragment sequences\" - can the authors expand on this? I assumed predicting the next fragment is a classification problem (with a very large number of classes), if so, how can invalid fragments be produced?\n- Since using a larger population size performs better (Table 7), is there any reason this is not the default setting for the method described in the paper? Would the Pareto front keep improving with even larger population sizes?\n- I cannot parse lines 2-9 of the pseudocode in Appendix A.1. Moreover, why does compute_crowding_distance take the Pareto fronts as one of the arguments?\n- The bug shown in Appendix A.2 looks very serious; it seems latent codes of different samples in a batch get arbitrarily mixed up. Is it fair to say that the original implementation of [2] was mostly ignoring the latent code?\n- I am confused by Table 4: as far as I understand, two groups of samples are being lumped together, and the Pareto front is recalculated. How is it possible that there are no samples from the \"Without PP\" setting in the combined Pareto front, yet the combined front doesn't include some samples that were Pareto-optimal in the \"With PP\" setting? If all samples from \"Without PP\" were dominated by Pareto-optimal samples from \"With PP\", then I would expect the combined front to be the same as for the \"With PP\" samples.\n- What is causing the sudden jumps in the loss plots in Figure 8?\n\nSmall remarks, typos, and grammar issues (did not influence my rating recommendation):\n- Page 1: \"adopt\" -> \"adopts\"\n- Page 1: the paper distinguishes two classes of approaches: one based on SMILES, and one based on Graph Convolutional Neural Networks. It would be better to drop the word \"Convolutional\", as it does not really apply to some methods based on graph message passing (e.g. [3, 6])\n- Page 1: \"that, molecules,\" -> \"that molecules,\"\n- Page 2: \"bounds\"\n- Page 2: \"optimizations\" -> \"optimization\" (twice)\n- Page 2: \"for neuroevolution that leads to evolution\" -> \"for evolution\"\n- The EDA acronym is introduced, and never used - I would just skip introducing the acronym then\n- The paper uses terms \"evolutionary computation (EC)\" and \"evolutionary strategy (ES)\", how do those differ? If they mean the same thing, I'd suggest sticking to one for clarity\n- Page 2: \"structural transformation\" -> \"structural transformations\"\n- Page 2: missing space in \"operations.This\"\n- Pages 2 & 4: \"for examples\" -> \"for example\"\n- Page 6: \"The novelty is defined as the ratio of number of generated novel valid molecules that do not exist in the training data (...)\" - I would drop the word \"novel\", since novelty is being defined by this very sentence\n- Page 6: \"proximity (...) with\" -> \"proximity (...) to\"\n- Page 13: \"phrase\"\n\nReferences:\n- [1] Efficient multi-objective molecular optimization in a continuous latent space\n- [2] A Deep Generative Model for Fragment-Based Molecule Generation\n- [3] Constrained Graph Variational Autoencoders for Molecule Design\n- [4] GuacaMol: Benchmarking Models for de Novo Molecular Design\n- [5] On Failure Modes of Molecule Generators and Optimizers\n- [6] Junction Tree Variational Autoencoder for Molecular Graph Generation\n\n----------------------------------------------------------------------------------------------------\n\nComments after rebuttal:\n\nI would like to thank the authors for their response. I am keeping my score, but I encourage the authors to resubmit after improving the things that were discussed (most importantly, using better metrics and comparing to more established baselines); I think this will make the paper much stronger.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work combines deep generative models (variational autoencoders, FragVAE) and multi-objective evolutionary computation for molecular design. They use a multilayer perceptron as a predictor for properties. Evolutionary operations are used to explore the latent space of the generative model to produce novel competitive molecules. Experiments are executed to show the effectiveness of the proposed method with respect to Bayesian optimization-based methods.

Strengths:

1 - Combines multi-objective evolutionary computation and deep generative modeling, which is a promising approach to tackle multi-objective optimization in structured spaces.

Weaknesses:

All the reviewers agree that the paper is not yet ready for publication. They point out the following areas to improve:

1 - The lack of details and clarity in the method.

2 - The experimental section needs to be improved. The evaluation metrics and baselines are weak.

3 - Describe better and more clearly the novelty of the proposed approach with respect to previous work in the area.
Decision: Reject