Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space

Challenges in natural sciences can often be phrased as optimization problems . Machine learning techniques have recently been applied to solve such problems . One example in chemistry is the design of tailor-made organic materials and molecules , which requires efficient methods to explore the chemical space . We present a genetic algorithm ( GA ) that is enhanced with a neural network ( DNN ) based discriminator model to improve the diversity of generated molecules and at the same time steer the GA. We show that our algorithm outperforms other generative models in optimization tasks . We furthermore present a way to increase interpretability of genetic algorithms , which helped us to derive design principles . Our open source implementation : https : //github.com/aspuru-guzik-group/GA 1 INTRODUCTION . The design of optimal structures under constraints is an important problem spanning multiple domains in the physical sciences . Specifically , in chemistry , the design of tailor-made organic materials and molecules requires efficient methods to explore the chemical space . Purely experimental approaches are often time consuming and expensive . Reliable computational tools can accelerate and guide experimental efforts to find new materials faster . We present a genetic algorithm ( GA ) ( Davis , 1991 ; Devillers , 1996 ; Sheridan & Kearsley , 1995 ; Parrill , 1996 ) for molecular design that is enhanced with two features : 1 . A neural network based adaptive penalty . This promotes exploratory behaviour of the GA and thus improve the diversity of generated molecules . 2 . Exploiting the robustness of SELFIES ( Krenn et al. , 2019 ) , we do not need to incorporate any expert based mutation or cross-over rules . Starting from only simple methane molecules , our algorithm outperforms other generative models in optimization tasks for molecular design . By introducing machine learning ( ML ) techniques , the long term behaviour is not subject to stagnation , thus solving a significant problem in genetic algorithms ( Paszkowicz , 2009 ) . No domain knowledge is required ; thus , our approach is not limited to chemistry-specific questions but can be applied to a wide range of optimization problems . †These authors contributed equally ∗Correspondence to : alan @ aspuru.com 2 RELATED WORKS . Inverse design is the systematic development of structures with desired properties . In chemistry ( Sanchez-Lengeling & Aspuru-Guzik , 2018 ) , the challenge of inverse design has been tackled as an optimization problem , among others in the form of variational autoencoders ( VAEs ) , generative adversarial networks ( GANs ) and genetic algorithms . Variational autoencoders and generative adversarial networks VAEs ( Kingma & Welling , 2013 ) are a widely used method for direct generation of molecular string or graph representations ( GómezBombarelli et al. , 2018 ) . They encode discrete representations into a continuous ( latent ) space . Molecules resembling a known structure can be found by searching around the region of the encoded point . Making using of the continuous latent representation , it is possible to search via gradients or Bayesian Optimization ( BO ) . However , the generation of semantically and syntactically valid molecules is a challenging task . Thus , several follow up works to the VAE have been proposed for inverse design in chemistry . Among them , CVAE ( Gómez-Bombarelli et al. , 2018 ) , GVAE ( Kusner et al. , 2017 ) and SD-VAE ( Dai et al. , 2018 ) work directly on string molecular representations . Alternatively , JT-VAE ( Jin et al. , 2018a ) as well as CGVAE Liu et al . ( 2018 ) work on molecular graphs . Unlike latent space property optimization , the policy network ( PN ) based GCPN model ( You et al. , 2018 ) proposes a reinforcement learning ( RL ) -based method for direct optimization on molecular graphs . ORGAN ( Guimaraes et al. , 2017 ) demonstrate training string-based generative adversarial networks ( GANs ) ( Goodfellow et al. , 2014 ) via RL . Another method based on adversarial training is VJTNN ( Jin et al. , 2018b ) . Segler et al . ( 2017 ) first introduced molecule generating models based on language models and reinforcement learning , where actions in an environment are taken to construct a molecule , receiving reward from an external scoring function . This model has also shown strong performance in the GuacaMol benchmark ( Brown et al . ( 2019 ) ) . In all the approaches mentioned above , generative models are trained to mimic the reference data set distributions , thus limiting the exploration ability of VAEs and GANs . Genetic algorithms & related methods There exist several examples of GA based molecule optimization algorithms in literature ( O ’ Boyle et al. , 2011 ; Virshup et al. , 2013 ; Rupakheti et al. , 2015 ; Jensen , 2019 ; Sheridan & Kearsley , 1995 ; Parrill , 1996 ) ) . While some of these examples pre-define mutations on a SMILES level to ensure validity of the molecules , other approaches use fragmentbased assembly of molecules . GAs are likely to get trapped in regions of local optima ( Paszkowicz , 2009 ) . Thus , for selection of the best molecules , ( Rupakheti et al. , 2015 ; Jensen , 2019 ) report multiple restarts upon stagnation . We include the aforementioned models as baselines for numerical comparison . 3 GA-D ARCHITECTURE . 3.1 OVERVIEW . Our approach is illustrated in Figure 1 . Our generator is a genetic algorithm with a population of molecules m. In each generation , the fitness of all molecules is evaluated as a linear combination of molecular properties J ( m ) and the discriminator score D ( m ) : F ( m ) = J ( m ) + β ·D ( m ) . ( 1 ) Random mutations of high fitness ( best performing ) molecules replace inferior members , while bestperforming molecules continue to a subsequent generation . The probability of replacing a molecule is evaluated using a smooth logistic function based on a ranking of fitness among the molecules of a generation . At the end of each generation , a neural network based discriminator is trained jointly on molecules generated by the GA and a reference data set . The fitness evaluation accounts for the discriminator predictions for each molecule . Therefore , the discriminator plays a role in the selection of the subsequent population . 3.2 MUTATION & CROSS-OVER RULES . Mutation of molecules to populate subsequent generations is an important element of the GA. A low degree of mutation can lead to a slow exploration of the chemical space , causing stagnation of the fitness function . Robustness of SELFIES allows us to do random mutations to molecular strings while preserving their validity . Thus , our mutations only include 50 % insertions or 50 % replacements of single SELFIES characters . To accelerate the exploration of the GA , we add one domain-specific mutation rule – the direct addition of phenyl groups in approximately 4 % of cases . Character deletion is implicitly taken into account in SELFIES mutations , and we do not use crossover rules . 3.3 ROLE OF THE DISCRIMINATOR . The fundamental role of the discriminator is to increase molecular diversity by removing longsurviving molecules . Consider the realistic scenario in which a GA has found a molecule close to a local optimum , where all mutations are lowering the fitness . As a result , this molecule survives for multiple generations while occupying a large fraction of the population . During such periods , the GA has limited view of the chemical space as it repeatedly explores mutations of the same high-fitness molecule . A straightforward solution would be the addition of a linear penalty ( Nanakorn & Meesomklin , 2001 ) in the fitness function that accounts for the number of successive generations a molecule survives . However , this method assigns independent scores to similar-looking molecules , which again results in less variety . Our solution is the addition of an adaptive penalty ( in our case a neural network based discriminator ) , thus resolving the problem of stagnation . Molecules with similar representation receive similar classification scores . Furthermore , long-surviving molecules are trained longer and receive weaker scores , resulting in decreasing fitness - reducing the chance of long periods of stagnation ( illustrated in Figure 2 ) . The task of the discriminator thus is to memorize families of high performing molecules and penalize their fitness to force the GA to explore different regions in chemical space . 4 EXPERIMENTS . Comparing to the literature standard , we aim at maximizing the penalized logP objective J ( m ) proposed by ( Gómez-Bombarelli et al. , 2018 ) . For molecule m , the penalized logP function is defined as J ( m ) = logP ( m ) − SA ( m ) − RingPenalty ( m ) , ( 2 ) where logP indicates the water-octanol partition coefficient , SA ( Ertl & Schuffenhauer , 2009 ) represents the synthetic accessibility and prevents the formation of chemically unfeasible molecules , and RingPenalty linearly penalizes the presence of rings of size larger than 6 . Our reference data set consists of 250,000 commercially available molecules extracted from the ZINC database ( Irwin et al. , 2012 ) . All three quantities in the above equation are normalized based on this data set . 4.1 UNCONSTRAINED OPTIMIZATION AND COMPARISON WITH OTHER GENERATIVE MODELS . We define our fitness function according to Eq . 1 and 2 with β = 0 and 10 . The algorithm is run for 100 generations with a population size of 500 . All generated molecules are constrained to a canonical smile length of 81 characters ( as in ( Yang et al. , 2017 ; Jensen , 2019 ) ) . We train the discriminator ( fully connected neural network with ReLU activation and sigmoid output layer , the input is a vector of chemical and geometrical properties characterizing the molecules ) at the end of each generation for 10 epochs on 500 molecules proposed by the GA and 500 randomly drawn molecules from the ZINC data set . We report maximum J ( m ) scores , averaged over 10 independent runs . The highest J ( m ) achieved by our approach are 13.31 ± 0.63 ( β = 10 ) and 12.61 ± 0.81 ( β = 0 ) , respectively , which is almost twice as high es the highest literature value of 7.87±0.07 ( See Table 1 ) . Furthermore , we compare to 50,000 random valid SELFIES strings , which surprisingly outperforms some existing generative models . The GA-D ( t ) results are explained in the next section . 4.2 LONG TERM EXPERIMENT WITH A TIME-DEPENDENT ADAPTIVE PENALTY . In Figure 3 , we show the results of runs where we use a time-dependent adaptive penalty . During periods of saturation , the weight of the discriminator predictions is switched from 0 to 1000 until stagnation is overcome . The genetic algorithm is hence forced to propose new families of molecules to increase J ( m ) . As it can be observed in Figure 3 , even after steep decreases in max J ( m ) , the scores recover and potentially reach values higher than in previous plateaus . We observe that this approach significantly outperforms all previous methods in maximizing the objective J ( m ) . Visual inspection of the highest performing molecules shows that the GA is exploiting deficiencies of the ( penalized ) logP metric by generating chemically irrelevant motifs such as sulfur chains . While being of limited relevance for application , this nonetheless shows the that the GA is very efficient in exploring the chemical space and finding solutions for a given task . Analysis of these solutions will help us to better understand and eventually improve objective functions . At the same time , surprising solutions found by an unbiased algorithm can lead to unexpected discovery or boost human creativity ( Lehman et al . ( 2018 ) ) . While exploitative tasks that follow some reference database are significant for questions involving drug design , more explorative behaviour could be beneficial in domains such as solar cell design ( Yan et al . ( 2018 ) ) , flow battery design ( Yang et al . ( 2018 ) ) , and in particular for questions where datas sets are not available , such as design of targets in molecular beam interferometry ( Fein et al . ( 2019 ) ) .

Summary: Generating novel drug molecules is an important problem. Prior approaches to solve this problem has explored approaches like VAEs, GANs and Genetic Algorithms. The authors propose a genetic algorithm that is augmented with Neural Networks for this purpose. The authors claim impressive performance with their algorithms. While maximizing for Penalized logP, the molecules that they generate have almost twice the maximum Penalized logP reported in the literature.