Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space
Authors: AkshatKumar Nigam
Pascal Friederich
Mario Krenn
Alan Aspuru-Guzik
Venue:  (2020)
Keywords: Generative model
Chemical Space
Inverse Molecular Design

Abstract: Challenges in natural sciences can often be phrased as optimization problems. Machine learning techniques have recently been applied to solve such problems. One example in chemistry is the design of tailor-made organic materials and molecules, which requires efficient methods to explore the chemical space. We present a genetic algorithm (GA) that is enhanced with a neural network (DNN) based discriminator model to improve the diversity of generated molecules and at the same time steer the GA. We show that our algorithm outperforms other generative models in optimization tasks. We furthermore present a way to increase interpretability of genetic algorithms, which helped us to derive design principles

Published as a conference paper at ICLR 2020
AUGMENTING GENETIC ALGORITHMS WITH DEEP NEURAL NETWORKS FOR EXPLORING THE CHEMICAL SPACE
AkshatKumar Nigam 1i,, Pascal Friederich 1.. Mario Krenn, 1,3,4 Alán Aspuru-Guzik.-5 1.345. 13,,4,5,*
'Department of Computer Science, University of Toronto, Canada.
2 Inntitttee Nanotechnology. Karlsruhe Institute of Technology. Germany.
Department of Chemistry, University of Toronto, Canada.
Vector Institute for Artificial Intelligence, Toronto, Canada.
" CCnaddian Institute for Advanced Research (CIFAR) Senior Fellow, Toronto, Canada
ABSTRACT
Challenges in natural sciences can often be phrased as optimization problems. Ma- chine learning techniques have recently been applied solve such problems. One example in chemistry is the design of tailor-made organic materials and molecules, which requires efficient methods explore the chemical space. We present a ge- netic algorithm (GA) that is enhanced with a neural network (DNN) based dis- criminator model improve the diversity of generated molecules and the time steer the GA. We show that our algorithm outperforms other generative mod- els optimization tasks. We furthermore present a way increase interpretability of genetic algorithms, which helped us to derive design principles. Our open source implementation: https://gttub..co//spuuu guzik guzig-grouppeioo group/GA
INTRODUCTION
The design of optimal structures under constraints is an important problem spanning multiple do- mains in the physical sciences. Specifically, in chemistry, the design of tailor-made organic mate- rials and molecules requires efficient methods to explore the chemical space. Purely experimental approaches are often time consuming and expensive. Reliable computational tools can accelerate and guide experimental efforts to find new materials faster.
We present genetic algorithm (GA) (Davis 1991 Devillers 1996 Sheridan & Kearsley 1995
Pamil 1996) for molecular design that is enhanced with two features:
1. A neural network based adaptive penalty. This promotes exploratory behaviour of the GA and thus improve the diversity of generated molecules.
2. Exploiting the robustness of SELFIES (Krenn et al. 2019), we not need incorporate any expert based mutation or cross-over rules.
Starting from only simple methane molecules, our algorithm outperforms other generative models in optimization tasks for molecular design. By introducing machine learning (ML) techniques, the long term behaviour is not subject to stagnation, thus solving a significant problem in genetic algorithms (Paszkowicz] 2009).
No domain knowledge required; thus, our approach is not limited -hemistry-speciffc questions but can be applied to wide range of optimization problems.
1 These authors contributed equally
" "oorrsspnndence alan aspuni.com

---

Published as a conference paper at ICLR 2020
2 RELATED WORKS
Inverse design is the systematic development of structures with desired properties. In chemistry (Sanchez-Lengllngg & Aspuru-Guzik] 2018), the challenge of inverse design has been tackled as an optimization problem, among others in the form of variational autoencoders (VAEs), generative adversarial networks (GANs) and genetic algorithms.
Variational autoencoders and generative adversarial networks VAEs (Kingma & Welling 2013) are a widely used method for direct generation of molecular string or graph representations (Gómez- Bombarelli et al. 2018). They encode discrete representations into continuous (latent) space. Molecules resembling a known structure can be found by searching around the region of the en- coded point. Making using of the continuous latent representation, it is possible to search via gradi- ents or Bayesian Optimization (BO). However, the generation of semantically and syntactically valid molecules is a challenging task. Thus, several follow up works to the VAE have been proposed for inverse design in chemistry. Among them, CVAE (Gômez- Bombarelli et al.) 2018), GVAE (Kusner et al. 2017) and SD- VAE (Dai al. 2018) work directly string molecular representations. Alter- natively, JT VAE (Jin et al. (0188) as well as CGVAE Liu et al. (2018) work on molecular graphs. Unlike latent space property optimization, the policy network (PN) based GCPN model (You al.) 2018) proposes a reinforcement learning (RL)-based method for direct optimization on molecular graphs. ORGAN (Guimaraes al., (017) demonstrate training string-basee generative adversarial networks (GANs) Goodfellow et al. (2014) via RL. Another method based on adversarial training is VJTNN (Tin et al. 2018b). Segler al. (2017) first introduced molecule generating models based on language models and reinforcement learning, where actions in an environment are taken to construct molecule, receiving reward from an external scoring function. This model has also shown strong performance in the GuacaMol benchmark (Brown et al. (2199).
In all the approaches mentioned above, generative models are trained to mimic the reference data set distributions, thus limiting the exploration ability of VAEs and GANs.
Genetic algorithms & related methods There exist several examples of GA based molecule opti- mization algorithms in literature (O BoBle et al. 2011 Virshup et al.) 2013 Rupakheti et al. 20015] Jensen 2019; Sheridan & Kearsley] 1995; Parrill 1996)). While some of these examples pre-define fragment-
mutations a SMILES level ensure validity of the molecules, other approaches based assembly of molecules. GAs are likely to get trapped in regions of local optima (Paszkowicz) 2009). Thus, for selection of the best molecules, (Rupakheti et al. 2015 Jensen 2019) report mul- tiple restarts upon stagnation.
We include the aforementioned models as baselines for numerical comparison.
3
GA-D ARCHITECTURE
3.1 OVERVIEW
Our approach is illustrated in Figure I Our generator is a genetic algorithm with a population of molecules m. In each generation, the fitness of all molecules is evaluated linear combination of molecular properties (m) and the discriminator score D(m):
F(m) J (mm + B D(m)-
Random mutations of high fitness (best performing) molecules replace inferior members, while best- performing molecules continue is evaluated using a smooth logistic function based on ranking of fitness among the molecules of a generation. At the end of each generation, a neural network based discriminator is trained jointly on molecules generated by the GA and reference data set. The fitness evaluation accounts for the discriminator predictions for each molecule. Therefore, the discriminator plays a role in the selection of the subsequent population.
subsequent generation. The probability of replacing a molecule
3.2 MUTATION & CROSS-OVER RULES
Mutation of molecules to populate subsequent generations is an important element of the GA. A low degree of mutation can lead to slow exploration of the chemical space, causing stagnation
2

---

Published as a conference paper at ICLR 2020
Generator Genetic Algorithm
Molecules m
Discriminator Neural Network
Properties
F(m)
J(m) Evaluation
Fitmess eval:
Discriminator (m)
F(m) (0m))).0(m)
Dataset
Molecules
Figure Overview our hybrid structure, which augments genetic algorithms with ML based neural networks.
of the fitness function. Robustness of SELFIES allows us to do random mutations to molecular strings while preserving their validity. Thus, our mutations only include 50% insertions or 50% replacements of single SELFIES characters. To accelerate the exploration of the GA, we add one domain-specific mutation rule the direct addition of phenyl groups approximately 4% of cases. Character deletion is implicitly taken into account in SELFIES mutations, and we do not use cross- over nules.
Kil
10
14
Generation
Figure 2: Reducing Fimess of o-terphenyl, initially possessing the largest fitness generation
(Rank = 0). Due to decreasing discriminator predictions D(m) fitness decreases with an increased probability of getting killed (Kill Prob.). The molecule is completely replaced in generation 16. Rank denotes the position of the molecule in an increasing fitness list of generation molecules.
3.3 ROLE THE DISCRIMINATOR
The fundamental role of the discriminator is increase molecular diversity by removing long- surviving molecules. Consider the realistic scenario in which a GA has found molecule close to a local optimum, where all mutations are lowering the fitness. As result, this molecule survives for multiple generations while occupying a large fraction of the population. During such periods, the GA has limited view of the chemical space as it repeatedly explores mutations of the same high-fitness molecule.
A straightforward solution would be the addition a linear penalty (Nanakom & Meesomklin 2001) in the fitness function that accounts for the number of successive generations molecule survives. However, this method assigns independent scores to similar-looking molecules, which again results in less variety.
3

---

Published as a conference paper at ICLR 2020
Our solution is the addition of adaptive penalty (in our case neural network based discriminator), thus resolving the problem of stagnation. Molecules with similar representation receive similar trained longer and receive weaker
classification scores. Furthermore, Long-surviving molecules scores, resulting in decreasing fitness reducing the chance of long periods of stagnation (illustrated in Figure 2). The task of the discriminator thus is memorize families of high performing molecules and penalize their fitness to force the GA to explore different regions in chemical space.
4 EXPERIMENTS
Comparing the literature standard, we aim at maximizing the penalized logP objective ((m) proposed by (Gómez- (Gómez-Bombane Bombarelli et al. 2018). For molecule m, the penalized logP function is defined as
(m) logP(m) SA(m) - RingPenalty(m).
where logP indicates the water-octanol partition coefficient, SA (Ertl & Schuffenhauer 2009) rep- resents the synthetic accessibility and prevents the formation of chemically unfeasible molecules, and RingPenalty linearly penalizes the presence of rings of size larger than 6. Our reference data set consists of 250,000 commercially available molecules extracted from the ZINC database (Irwin)
al. 2012 All three quantities in the above equation normalized based on this data
Table 1: Comparison our model with maximum penalized logP scores reported in literature. Models in the upper part are forced to optimize within the distributions of given reference data sets while the GA based approaches in the lower part can freely explore chemical space. Direct comparisons need take into account these different objectives and scopes. The parameter / in our GA can be used to balance exploration and exploitation (see Section 4.6).
Model
Max. Penalized logP
Model
GVAE + BO (usner et al.
2017
2.87 + 0.06
VAE
SD-VAE (Dai et
2018)
3.50 + 0.44
VAE
Exploitation
CVAE BO (Gómez Bombarelli
2018
4.85 + 0.17
VAE
ORGAN Guimaraes et al.
3.52 0.08
GAN
JT-VAE (Jin et al. 201 8a
4.90 + 0.33
VAE
Chem'TS Yang et al.
2017
5.6 + 0.5
RNN
GCPN You et al.
2018)
7.87 + 0.07
PN + GAN
Random SELFIES
6.19 + 0.63
Random Search
Exploration
GB-GA Jensen
2019
7.4 0.9
GA
GB-GA Jensen
2019
15.76 + 5.71
GA
GA (here)
12.61 + 0.81
GA
GA + D (here)
13.31 + 0.63
GA + DNN
(GA + D(t) (here) 5
20.72 + 3.14)
GA DNN
average of three best molecules after performing Bayesian optimization on the latent representation trained VAE
2 two best molecules of single run averaged 10 runs with 20 molecules per generation with molecular weight (excl. hydrogen) smaller than 39.15 + 3.50 g/mol run averaged over 10 runs with 500 molecules up to 81 characters per generation and 100 generations averaged over 5 runs with 1000 generations each
generations
4.1 UNCONSTRAINED OPTIMIZATION AND COMPARISON WITH OTHER GENERATIVE MODELS
We define our fitness function according to Eq. and 2 with , run for 100 generations with population size of 500. All generated molecules are constrained to a canonical smile length of 81 characters (as in (Yang et al.) 2017 Jensen 2019 We train the discriminator (fully connected neural network with ReLU activation and sigmoid output layer, the input is a vector of chemical and geometrical properties characterizing the molecules) at the end of each generation for 10 epochs 500 molecules proposed by the GA and 500 randomly drawn molecules from the ZINC data set. We report maximum J(m) scores, averaged over 10 independent runs. The highest (m) achieved by our approach are 13.31 + 0.63 (8 10) and
and 10. The algorithm is
4

---

Published as a conference paper at ICLR 2020
12.61 = 0.81 (3 = 0), respectively, which is almost twice as high the highest literature value of 0.07 (See Table I. Furthermore, we compare to 50,000 random valid SELFIES strings, which surprisingly outperforms existing generative models. The GA-D(t) results are explained in the next section.
4.2
LONG TERM EXPERIMENT WITH TIME-DEPENDENT ADAPTIVE PENALTY
20
Max Max,fmm fimi
8- 0 (methane)
O tzinc)
0-
adaptive B (methane)
20D
400
500
ado
1000
Figure 3: Maximum J(m) values found for 10 independent runs with no discriminator (3 = 0) and
5 independent with the introduction of time-dependent adaptive penalty. The full line is the average of all runs, the shaded areas are the boundaries of all runs and the dashed lines denote the average of all maximal J (m) values found any generation.
In Figure 3. we show the results of runs where we use a lime-dependent adaptive penalty. During periods of saturation, the weight of the discriminator predictions is switched from 0 to 1000 until stagnation overcome. The genetic algorithm is hence forced to propose new families of molecules increase J(m). As can be observed in Figure 3. even after steep decreases in max (m), the scores recover and potentially reach values higher than in previous plateaus. We observe that this approach significantly outperforms all previous methods in maximizing the objective J (m).
Visual inspection of the highest performing molecules shows that the GA is exploiting deficiencies of the (penalized) logP metrie by generating chemically irrelevant motifs such as sulfur chains. While being of limited relevance for application, this nonetheless shows the that the GA is very efficient in exploring the chemical space and finding solutions for given task. Analysis of these solutions will help us to better understand and eventually improve objective functions. At the same time, sur- prising solutions found by an unbiased algorithm can lead unexpected discovery or boost human creativity (Lehman et al. (2018)). While exploitative tasks that follow some reference database are significant for questions involving drug design, more explorative behaviour could be beneficial in domains such solar cell design (Yan et al. (2018)), flow battery design (Yang et al. (2018)), and in particular for questions where datas sets are not available, such as design targets in molccular beam interferometry (Fein al. (2019)).
4.3 ANALYSIS OF MOLECULE CLASSES EXPLORED BY THE GA
Figure shows classes of molecules explored by the GA in a trajectory with 1000 generations and generation size of 500 (see trajectories in Figure 3). A K-means clustering analysis based in the RDKit fingerprint bit-vectors with 20 clusters was used to automatically generate labels for the 50 best performing molecules (in terms of their properties J J(m)) in each generation. We find that the algorithm starts with class of relatively small molecules ( 40 generations, see class 10), after which the algorithm explores several different classes of large molecules with egg. aromatic rings and conjugates carbon chains (see class 15) and long linear sulfur chains (see class 1), Class includes the molecules with the highest J (m) scores found in this work, exceeding values 20.
The clustering analysis furthermore helps to analyze the large number of generated molecules. This allows us learn from the data and derive design rules to achieve high target properties. In case of the penalized logP score, we find that representative examples (short distance to the cluster center) of high performing classes contain aromatic rings, conjugated carbon chains and linear sulfur chains, which can act as design rules for molecules with high penalized logP scores. Using these design rules to construct molecules the maximally allowed length consisting of only sulfur chains, conjugated carbon chains and chains of benzene rings with sulfur bridges yields (mm) scores of 31.79, 8.58 and 10.02, respectively. The J(m) score reached by the linear sulfur chain outperforms the best scores found by the GA and other generative models.
5

---

Published as a conference paper at ICLR 2020
50
40
Molecula
30
20
200
400
Generation
600
800
1000
10
15
10
11
13
14
15
16
17
18
19
Figure 4: Classes molecules explored by the GA.
To visualize the explorative behaviour of the GA, we did a two-dimensional principal component analysis (PCA) of the molecular fingerprint vectors of all molecules generated by the GA-D((t) in the trajectory shown Figure 4 Five snapshots of the trajectory are shown in Figure [ colored according their chemical family. The property score ((m) of the trajectory is visualized in the images. We find that the GA sequentially explores different parts of the 2D projection of the chemi- cal space, finding different families of high performing molecules, while the discriminator prevents the algorithm from searching in the same space repeatedly.
4.4 CONSTRAINED OPTIMIZATION
In the previous sections, aimed maximize the penalized logP objective. Here, consider two different tasks: firstly, generating molecules of specific chemical interest and secondly, modifying existing molecules increase penalized logP scores. We used throughout Section 4.4
4.4.1 GENERATING MOLECULES WITH SPECIFIC PROPERTIES
We run 250 instances randomly selected sets of target properties (logP in the interval from -5 to 10, SA scores in the range from to 5 and ring penalty for 0 to 3). The fitness function is modified to minimize the summed squared difference between the actual and desired properties. We compute the number of experiments that successfully proposed molecules with squared difference less than 1.0. Each run is constrained to run for 100 generations with maximum canonical SMILES length of 81 characters. In 90.0% of the cases, our approach proposes the desired molecules.
Figure 5: Two-dimensional PCA of five time snapshots the molecular space explored by the GA-D(t).

---

Published as a conference paper at ICLR 2020
Table 2: Comparison on constrained improvement of penalized logP of specific molecules.
6=0.4
6=0.6
Improvement
Success
Improvement
Success
JT-VAE (Jin et al. GCPN
2018a
0.84 1.45
83.6%
0.21 0.71
46.4%
You et
2018
2.49 1.30
100.0%
0.79 0.63
100.0%
MMPA (Jin cl al.
2018b
3.29 = 1.12
1.63 1.44
DEFactor (Assouel
2018
3.41 1.67
85.9%
1
72.6%
VJTNN (Jin et al.
20186
3.55 = 1.67
2.33 - 1.17
GA (here)
5.93 + 1.41
100.0%
3.44 + 1.09
99.8%
66 Turget
Target
-2
-2
Distributions of logP and QED for a) ZINC and b) GuacaMol data compared to molecules generated using the GA. c) Examples of molecules generated by the GA with high logP and QED scores.
Figure 6:
4.4.2 IMPROVING PENALIZED LOGP SCORES OF SPECIFIC MOLECULES
In this experiment, we follow the experimental setup proposed by You et al. 2018). We optimize the penalized logP score of 800 low-scoring molecules from the ZINC data set. Our genetic algorithm initiated with molecule from the data set, and a population size of 500 without the discriminator. For each run, we report the molecule m that increases the penalized logP the greatest, while possessing a similarity sim(m, m') 8 with the respective reference molecules m'. We calculate molecular similarity based on Morgan Fingerprints of radius 2. ensure generation of molecules possessing a certain similarity, for molecule m we modify the fitness to:
each experiment for 20 generations and
F(m) = (m) + Similarity
(3)
is 0 if sim(m, m') 8 and -10 otherwise. Table 2 we report the average improvement for each molecule. Success is determined when the GA successfully im- prove upon the penalized logP score, while not violating the similarity constraint. Figure S1 shows examples of improved molecular structures.
Here,
4.5
SIMULTANBOUS LOGP AND QED OPTIMIZATION
To show the performance of the GA on a drug-discovery task (Brown et al. 2019 Polykovskiy et al. modified the objective function include the drug-likeness score QED (Bickerton al. 2012 Solubility metric logP and drug-likeness cannot be maximized at the same time, which is shown in Figure 6 at the example the ZINC and the GuacaMol data set (Brown al. 2019). The experiment shows that the GA is able to efficiently and densely sample the edge of the property distributions of both data sets. Example molecules that simultancously maximize logP and QED shown in Figure GC.
2018
4.6 MODIFICATION OF THE HYPERPARAMETER 3
The definition of the fitness function used in this work (see Equation D has a free parameter (5), that balances the relative importance of molecular target properties J(m and discriminator score

---

Published as a conference paper at ICLR 2020
a)
b)
pm0
p-0
13
B-10
0.5
8-10
8-14
4-16
p-1b
2-16
10-
2-10
0-10
R-20
0.20
P-22
Avenge dont
8-14
02-
d)
eccurneces
P-28
P-20
B+22
P-24
P-330
9-50
Figure 7: Variation of the model parameter 3 that weights the discriminator score in the overall fitness function. a) Average ((m) as a function of the generation. b) Average discriminator score as a function of the generation. c) Distributions of (mm and discriminator score D for multiple values of B. d) Examples of molecules generated with 8 = 0 and B 50.
D(m). Large values of 3 promote the generation of molecules that resemble the molecules from the reference data set, while small values of / let the GA explore molecules outside of the distribution of the reference data set. To systematically explore this behaviour, we varied the 3 parameter and analyzed the resulting values of [m), (mm) as well as their distributions. Figure 7 illustrates the results of this test, where each curve obtained from averaging five independent runs with equal settings.
In Figure 7, we show the average property score (mm for different values 3 and observe, as expected, that low values of 33 on average lead to higher J(m) score while very high values of B limit the algorithm to stay inside the reference data set distribution, which, per definition, has a mean J (m) score of O. Figure 7p illustrates the low average discriminator scores encountered after a few generations in case of low values of B. The curves shown in Figure 77 and suggest that 33 allows us interpolate between high and low ((m) and high and low resemblance with the reference data set. However, as shown in Figure 7p. this is not the case. At values of B = 20, there is a rather fast transition from molecules with high values of ((m) of approximately 12-13 to (m) distribution centered around which is similar to the (mm) distribution of the reference data set. The same abrupt change is visible in the distributions of the discriminator scores. Only 8 v 18, find slightly wider distribution of intermediate J (m) values, which we assume is partially related to the fact that the 18 curve shown in Figure is not converged yet.
Figure 7 shows examples of molecules generated with 3 = O (upper panel) and 3 = 50 (lower panel). While in the first case, the GA finds large molecules with many aromatic rings and long linear chains, that do not resemble the molecules in the reference data set, the latter case shows molecules with structures and properties resembling the reference data set. In this case, the distribution of (mm is comparable to that of the data set (with 0 mean and unit standard deviation), while the discriminator encounters difficulty in correctly classification of the GA molecules, indicated by a mean discriminator score varying around 0.5 across many generations.
CONCLUSIONS AND FUTURE WORK
We presented hybrid GA and ML-based generative model and demonstrated its application in molecular design. The model outperforms literature approaches in generating molecules with de- sired properties. A detailed analysis of the data generated by the genetic algorithm allowed to
8

---

Published as a conference paper at ICLR 2020
interpret the model and learn rules for the design of high performing molecules. This human expert design inspired from GA molecules outperformed all molecules created by generative models.
For computationally more expensive property evaluations, will extend approach by the in- troduction of an on-the-fly trained ML property evaluation method, which will open new ways of solving the inverse design challenge in chemistry and materials sciences.
Our approach independent of domain knowledge, thus applicable to design questions in other scientific disciplines beyond chemistry. We therefore plan to generalize the GA-D approach to make it a more general concept of generative modelling.
ACKNOWLEDGMENTS
The authors thank Gabriel dos Passos Gomes & Andrés Aguilar Granda for useful discussions. A. A.-G. acknowledges generous support from the Canada 150 Research Chair Program, Tata Steel, Anders G. Froseth, and the Office of Naval Research. M.K. acknowledges support from the Austrian Science Fund (FWF) through the Erwin Schrödinger fellowship No. J4309. P.F. has received funding from the European Union's 's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curit grant agreement No 795206.
REFERENCES
Rim Assouel, Mohamed Ahmed, Marwin H Segler, Amir Saffari, and Yoshua Bengio. De- factor: Differentiable edge hctorization-basse probabilistic graph generation. arXiv preprint 11119:181, 2018
G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90, 2012.
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3): 1096-110S 2019.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational au- toencoder for molecule generation. In Proceedings of the International Conference on Learning Representations, 2018.
Lawrence Davis. Handbook of genetic algorithms. 1991.
James Devillers. Genetic algorithms in molecular modeling. Academic Press, 1996.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based molecular complexity and fragment contributions. Journal of cheminfor- matics, 1(1):8, 2009.
Yaakov Fein, Philipp Geyer, Patrick Zwick, Filip Kialka, Sebastian Pedalino, Marcel Mayor, Stefan Gerlich, and Markus Arndt. Quantum superposition of molecules beyond 25 kda. Nature Physics, pp. 1-4, 2019.
Rafael GomezzBom Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez- Lengelin. Dennis Sheberla, Jorge aguilera-Iparragun Timothy Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. Automatic chemical design using data-driven contin- uous representation of molecules. ACS central science, 4(2):268-276. 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Sapuru-Guziae Objeetive-reinforee generative adversarial networks (organ) for se- quence generation models. arXiv preprint arXiv: 1705. 10843, 2017.

---

Published as a conference paper at ICLR 2020
John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman. Zinc: a free tool to discover chemistry for biology. Journal of chemical information and modeling, 52 (7):1757-1768, 2012.
Jan H Jensen. graph- based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):357-3557,222 2019, 2019.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. arXiv preprint arXiv: /ttl. 802.00364, 2018a.
Wengong Jin, Kevin Yang, Regina Barzilay, and Tommi Jaakkola. Learning multimodal graph-to- graph translation for molecular optimization. arXiv preprint arXiv:/8/2.01077 2018b.
Diederik P Kingma and Max Welling. arXiv:1312.6114, 2013.
Auto-encoding variational
bayes.
arXiv preprint
Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alán Aspuru-Gusi Self- ies: robust representation of semantically constrained graphs with an example application in chemistry. arXiv preprint arXiv: 1905. 13741, 2019.
Matt Kusner, Brooks Paige, and José Miguel Hemúndezz-Lobato. Grammar variational autoen- coder. In Proceedings of the 34th International Conference on Machine Learning Voluue 70, pp- 1945-1954. JMLR. org, 2017.
Joel Lehman, Jeff Clune, and Dusan Misevic. The surprising creativity of digital evolution. In Artificial Life Conference Proceedings, pp. 55-56. MIT Press, 2018.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt. Constrained graph varia- tional autoencoders for molecule design. In Advances in Neural Information Processing Systems, pp. 7795-7804, 2018.
Pructtha Nanakorn and Konlakarn Meesomklin. An adaptive penalty function genetic algorithms for structural design optimization. Computers & Structures, 79(29-30)2227-2555, 2001.
Noel M O' O'Boyle, Casey M Campbell, and Geoffrey R Hutchison. Computational design and selec- tion of optimal organic photovoltaic materials. The Journal of Physical Chemistry C, 115(32): 16200-16210, 2011.
Abby Parrill. Evolutionary and genetic methods in drug design. Drug Discovery Today, 1(12): 514-521, 1996.
W Paszkowicz. Properties of genetic algorithm equipped with dynamic penalty function. Com- putational Materials Science, 45(1):77-83, 2009.
Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez-Lengeling, Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Aleksey Artamonov, Vladimir Aladinskiy, Mark Veselov, et Molecular sets (moses): a benchmarking platform for molecular generation models.
arXiv preprint 1811.::811..222.. 2018
Chetan Rupakheti, Aaron Virshup, Weitao Yang, and David N Beratan. Strategy to discover diverse optimal molecules in the small molecule universe. Journal of chemical information and modeling, 5293):52--337, 2015.
Benjamin Sanchez-Lengeling and Alán Aspuru-Guzik. Inverse molecular design using machine learning: Generative models for matter engineering. Science, 361(6400) 2018.
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1): 120-131, 2017.
Robert Sheridan and Simon K Kearsley. Using a genetic algorithm to suggest combinatorial libraries. Journal of Chemical Information and Computer Sciences, 35(2)3100-32,, 1995.
10

---

Published as a conference paper at ICLR 2020
Aaron M Virshup, Julia Contreras- Gareía, Peter Wipf. Weitao Yang, and David N Beratan. Stochas- tic voyages into uncharted chemical space produce a representative library of all possible drug-like like compounds. Journal of the American Chemical Sociery, 135(19):7296-703, 2013.
Cenqi Yan, Stephen Barlow, Zhaohui Wang, He Yan, Alex K-Y Jen, Seth R Marder, and Xiaowei Zhan. Non-fullerene acceptors for organic solar cells. Nature Reviews Materials, 380)3 2018.
Xiufeng Yang, Jinzhe Zhang, Kazuki Yoshizoe, Kei Terayama, and Koji Tsuda. Chemts: an efficient python library for de novo molecular generation. Science and technology of advanced materials, 18(1):972-976, 2017.
Zhengjin Yang. Liuchuan Tong, Daniel P Tabor, Eugene S Beh, Marc-Antoni Goulet, Diana De Por- cellinis, Alán Aspuru-Guzik, Roy G Gordon, and Michael Aziz. Alkaline benzoquinone aque- ous flow battery for large-scale storage of electrical energy. Advanced Energy Materials, 8(8); 1702056, 2018.
Jiaxuan You, Bowen Liu, Zhitao Ying. Vijay Pande, and Jure Leskovec. Graph convolutional pol- icy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems, pp. 6410-6421, 2018.

---

Published as a conference paper at ICLR 2020
SUPPLEMENTARY INFORMATION
Figure S1 shows examples of the molecules optimized Section 4.4
10.39
11.08
6.29
7.2
Figure S1: Molecular modifications resulting in increased penalized logP scores under similarity constraint sim m') 0.4, 0.6. We show the molecules that resulted in largest score improve- ment.
Figures $2-S4 show comparisons between the property distributions observed in molecule data sets such the ZINC and the GuacaMol data set and property distributions of molecules generated using random SELFIES (Figure S2), GA generated molecules with the penalized logP objective (Figure S3) and GA generated molecules with objective function which includes logP and QED (Figure S4). While the average logP scores of average SELFIES are low, the tail of the distribution reaches to high values, explaining the surprisingly high penalized logP scores shown Table The QED and weight distributions of molecules optimized using the penalized logP objective significantly differ from the distributions of the ZINC and the GuacaMol data set (see Figure S3). As soon as the QED score is simultaneously optimized, the distributions of GA generated molecules and molecules from reference data sets become more similar (see Figure S4). Figure shows that the GA can simultaneously optimize logP and QED.
12

---

Published as a conference paper at ICLR 2020
sandom SEEPIIS
bJ
rancom SEU-EE
Number
OuscaMo
Quacaloel
3.8
-25
0.0
2.5
5.0 key?
6.5
10.0
12.5
15.0
C1
6)
random
ANC
Number
400
630 woight Iglnol
1000
Figure S2: Distributions of a) logP, b) SA, c) QED and d) molecular weight for randomly generated SELFIES, molecules from the ZINC data set and molecules from the GuacaMol data set.
SCLNES
GA SELFIC
Number of ocourences
ONC
Number
GuacaVall
GueraNtol
d)
C4 SCLLELE
GA SEURES
Number of accurences
Guscalool
Guecaltol
0.8
402
500
1000
000
Molecuter warght (ginott
Figure S3: Distributions of a) logP, b) SA, c) QED and d) molecular weight for GA generated SELFIES (penalized logP objective function), molecules from the ZINC data and molecules from the GuacaMol data set.
13

---

Published as a conference paper at ICLR 2020
GAOED STUTES
GAGED SELRES
Number of occurancas
38.0
Numbor DOCUPIIICNS
P:N:
Numbo
0.
9.4
0.5
410)
1000
803
1000
Velecular weight [p/mml
Figure $4: Distributions of a) logP, b) SA, c) QED and d) molecular weight for GA generated SELFIES (logP and QED objective function), molecules from the ZINC data set and molecules from the GuacaMol data set.
14


Reviews:
[{"review_id": "HygnUBpOqr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper describes a model for the generation and optimization of molecules (de novo molecular design), which combines a genetic algorithm based on the SELFIES string representation and a trained discriminator. \nThe performance of the method is then shown on a small number of optimization tasks, and analyzed in detail.\n\nOverall, this reviewer thinks this is a very interesting approach with a lot of potential, however, the experimental validation could be stronger.\n\n\nEmbedding in previous work:\nSeveral key references should be added. In particular, this reviewer would suggest that the paper from the AZ group (https://arxiv.org/abs/1701.01329) is added, \n which first introduced the language model (LM)-based class of molecule generation models, and the RL paradigm of a molecule-generating agent, which takes actions in an environment to construct a molecule, receiving reward from an external scoring function.\nAlso, the previous paper by Liu et al (“Constrained Graph Variational Autoencoders for\nMolecule Design”, NeurIPS 2018,  https://arxiv.org/pdf/1805.09076.pdf ) should be cited, in particular, because it highlights the competitiveness of the LM based baselines.\n\nIn Table 2, it would be great to include a reference to the DEFactor paper from Yoshua Bengio’s lab (https://arxiv.org/abs/1811.09766), which achieves comparable performance to the strongest baseline.\n\nIn addition, the authors should cite earlier work on using GAs for de novo design, which have a rich tradition in chemoinformatics since the 1990ies.\n\nExperimental:\n\nRe 4.1. In the unconstrained optimization task, several baselines are missing. In particular, a language model + RL baseline should be considered. This model has also shown strong performance (in terms of optimization and quality) in the state of the art benchmark set (Guacamol https://pubs.acs.org/doi/10.1021/acs.jcim.8b00839 ) -  the code is available on github\n\nCan the authors explain why they believe the physchem only function ”log(P) - SA(m) - RingPenalty(m)” is a relevant scoring function for the tasks that are faced in real life drug and materials discovery? It would be great if the authors would consider using the Guacamol benchmarks which have been designed by domain experts to allow a more fine grained assessment on tasks relevant to practical drug and materials design, while still being tractable computationally. Also, the guacamol paper provides a benchmark to assess the quality of molecules, which is currently missing in the present work.\n\nRe 4.2. And 4.3.\nThe long term experiment and the molecule class exploration analysis are great, and well presented. I like the analysis of how the different classes emerge. However, are you sure that the GA is not just exploited the deficiencies of the rdkit logP calculator (which itself is just a model) by creating the molecules with the long sulfur chains? Also, I am not sure that it is likely that one would be able to synthesize these molecules in the lab.\n\nRe 4.4\nThe results on the constrained optimization are impressive, however, it is not really clear if the GA is not simply exploiting deficiencies in the scoring function again.\n\n\nOverall, this reviewer believes the proposed model is very promising (in particular also because GA’s are cool), however, the experimental validation could be more rigorous, which can be addressed by considering the guacamol benchmarks. If this results can provided, this reviewer will raise their recommendation to 'accept' (regardless of outcome of the benchmark)\n\n#####\nAfter response:\nConsidering the authors' response, I adapt my recommendation to accept. I acknowledge the short time the authors have  to run additional experiments. nevertheless, I would suggest that the authors at least run more challenging benchmarks later and report the results e.g. as a blogpost or in a leaderboard.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "BkeHDjoRtr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Generating novel drug molecules is an important problem. Prior approaches to solve this problem has explored approaches like VAEs, GANs and Genetic Algorithms. The authors propose a genetic algorithm that is augmented with Neural Networks for this purpose. The authors claim impressive performance with their algorithms. While maximizing for Penalized logP, the molecules that they generate have almost twice the maximum Penalized logP reported in the literature. \n\nFrom a purely molecular optimization point of view, their results are impressive. However, I suspect the paper does not clearly illuminate what is going on. For example, the Random SELFIES performs twice as good as state of the art generative models from couple of years back like JT-VAE, SD-VAE and Grammar VAE.  I suspect what is happening is that the random model and the models proposed by the authors are veering too much from the training data. The authors indeed suggest that this is the case in the discussion about the parameter beta. For small values of beta, the generated molecules move away from the reference distribution. While standard VAE models tries to ensure that the training and generated models are from similar distributions, the models proposed by the authors face no such constraint, for small values of beta. This makes the comparison with other models unfair - because other models try their best to generate molecules from the reference distribution, while at the same time trying to optimize for molecular properties like higher penalized logP. If those models had the freedom not to restrict its exploration to the training distribution, one can obviously generate molecules with higher penalized logP.\n\nThe authors seem to suggest that moving away from the reference distribution might be desirable, if we can get molecules with higher penalized logP. Indeed, this is not the case. Zinc is a database of small drug-like molecules. When they move away from this distribution, they will start generating molecules which are not drug-like and which perform poorly in other metrics like QED and Synthetic Accessibility. In this sense, the results claimed by the authors are misleading.\n\nCan the authors plot a distribution of molecular properties (molecular weight, Synthetic Accessibility, QED, logP and number of Rings) for zinc, and the generated molecules with different values of beta, so that we can analyze this phenomenon better? Code for this (along with examples) can be found in the MOSES toolkit: https://github.com/molecularsets/moses", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "B1eeq2hD9H", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present an algorithm to explore the generation of new organic molecules using a genetic algorithm.\nThe genetic algorithm will select molecules generated via mutation and cross over, according to a fitness function that includes a discriminator trained from generated molecules.\nThe mutation rule seems to be 50% insertions and 50% replacements of single SELFIES characters, plus a rule for the direct addition of phenyl groups with 4% chance.\nNo crossover rule is used.\nThe metric used is a maximum penalized logP score proposed by Gomez-Bombarelli, et al. \nThe work shows a huge improvement on this score using their proposed algorithm.\nAdditionally, the authors show some of the promise of the work in constrained optimization:\nThe algorithm can, with minor modifications, generate molecules with specific properties, and improve low scoring molecule.\nThere is also a study on the hyperparameter \\beta - low beta tends generate high scoring molecules, and high beta generates molecules similar to the dataset.\n\nBecause of the novel genetic algorithm based search method, as well as the large improvement on prior literature, I am leaning towards an accept for this paper.\n- Paper uses a metric well motivated by prior literature, J(m)\n- There are constrained optimization studies showing different ways to use the algorithm\n- The authors show the effect of the hyperparameter \\beta on the algorithm - it seems to be difficult to interpolate between molecules in the dataset and molecules with high scores.\n- All previous work have been with SMILES strings, where this work is the only one that builds up on SELFIES strings.\n  It is not a sincere comparison - it would be better if the authors indicated which works use SMILES and which one uses SELFIES.\n  A more sincere baseline would be to take a baseline model and apply it to SELFIES, while showing that the genetic algorithm based approach shows more viability.\n  At least ORGAN seems to operate on SMILES strings, so it would be not too hard to change the underlying sequences to SELFIES strings.\n- Some quantifiable metric of diversity of generated molecules might be a good analysis, it's unclear whether the model is simply memorizing a few high scoring molecules.\n\n\nTwo questions I have might improve the paper if they are answered in the text:\n1) It seems to be the discriminator will decide whether molecules are in the dataset or not.\n   If it is well trained and with a high penalty (e.g. time adaptive case), the GA will pick high J(m) molecules that are in the dataset.\n   An analysis of the topline of the dataset might be useful - what is the J(m) of the best molecule in the dataset?\n   If you simply seed a non-time adaptive GA with the best J(m) molecule, would it be able to reach the same levels?\n\n2) Seeing the molecules generated in Figure 7d, it seems to imply to me that that algorithm is finding \"bugs\" in the simulation metric.\n   Since the ICLR community does not have many chemists, it would be useful to make some claims about the chemical viability of these compounds outside of simulations.\n   Are there molecules that are similar looking to the generated ones?\n   Perhaps some analysis on whether the GA is overfitting to the synthetic metric of J(m) would be helpful.\n\nNits:\n- Use uppercase ZINC in all cases to refer to the dataset\n- Last sentence \"An important future generalization...\" awkward phrasing.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Paper received reviews of A, WA, WR. AC has carefully read all reviews/responses. R1 is less experienced in this area. AC sides with R2,R3 and feels paper should be accepted. Interesting topic and interesting problem. Authors are encouraged to strengthen experiments in final version. 
Decision: Accept (Poster)