Molecule Optimization by Explainable Evolution

1 INTRODUCTION . The space of organic molecules is vast , the size of which is exceeding 1060 ( Reymond et al. , 2010 ) . Searching over this vast space for molecules of interest is a challenging task in chemistry , material science , and drug discovery , especially given that molecules are desired to meet multiple criteria , e.g. , high potency and low toxicity in drug discovery . When human experts optimize molecules for better molecular properties , they will first come up with rationales within desirable molecules . Typically , the rationales are subgraphs in a molecule deemed to contribute primarily to certain desired molecular properties . Once rationales are identified , chemists will design new molecules on top of rationales hoping that , the desired properties of new molecules will be further enhanced due to the existence of rationale and changes of non-rationale parts . The cycle of identifying molecular rationales and redesigning new hypothetical molecules will be carried on until molecules that meet certain property criteria are discovered . In this paper , we develop a novel algorithm that mimics the process of molecule optimization by human experts . Our algorithm finds new molecules with better properties via an EM-like explainable evolutionary process ( Figure 1 ) . The algorithm alternates between two stages . During the first stage , we use an explainable local search method to identify rationales within high-quality molecules that account for their high property scores . During the second stage , we use a conditional generative model to explore the larger space of molecules containing useful rationales . Our method is novel in that we are using explainable models to help us exploit useful patterns in the molecules , yet leveraging generative models to help us explore the molecule landscape . Comparing to existing methods that directly learn a generative model using Reinforcement Learning or perform continuous optimization in the latent space of molecules ( Olivecrona et al. , 2017 ; You et al. , 2018a ; Dai et al. , 2018b ) , our method is more sample-efficient and can generate more novel and unique molecules that meet the criteria . We evaluate our algorithm against several state-of-the-art methods on a molecule optimization task involving multiple properties . Compared with baselines , our algorithm is able to increase the success ∗Correspondence to : Binghong Chen < binghong @ gatech.edu > . ∗ indicates equal contribution . Source code at https : //github.com/binghong-ml/MolEvol . rate by 50 % , novelty by 14 % , while having a competitive diversity . We further propose a new metric , QNU score , to jointly consider all three aspects , and show that we achieve a score of 52.7 % compared with 29.5 % by the best baseline . We also ask experienced chemists to evaluate top-50 generated molecules and find that 30 of them are as good as existing ones . The main contributions of this paper are summarized below : • We propose a novel EM-like evolution-by-explanation algorithm for molecule optimization ; • We present a novel , principled , explainable graph model based on an information-theoretic ap- proach to extract subgraphs essential for maintaining certain desired properties ; • Our approach outperforms existing state-of-the-arts by a large margin in terms of success rate ( 50 % better ) , novelty ( 14 % better ) , and an overall metric ( 79 % better ) on a real-world multiproperty optimization task . 2 RELATED WORK . There has been a surge of interest in using machine learning to discover novel molecules with certain properties in recent years . Most of the existing work defines a generative model for either the SMILES strings ( Weininger , 1988 ) or molecular graphs , and uses Reinforcement Learning algorithms to optimize the properties of the generated molecules ( Segler et al. , 2018 ; Olivecrona et al. , 2017 ; Guimaraes et al. , 2017 ; You et al. , 2018a ; Popova et al. , 2018 ; 2019 ; Samanta et al. , 2019 ; Zhou et al. , 2019 ; De Cao & Kipf , 2018 ; Kearnes et al. , 2019 ; Shi et al. , 2020 ; Jin et al. , 2020 ) . Others optimize the continuous representation of molecules in a latent space learned by variants of variational autoencoders ( Kusner et al. , 2017 ; Dai et al. , 2018b ; Jin et al. , 2018 ; Gómez-Bombarelli et al. , 2018 ; Kang & Cho , 2018 ; Liu et al. , 2018 ; Kajino , 2019 ) . More recent work attempts Evolutionary algorithms ( Nigam et al. , 2020 ; Leguy et al. , 2020 ; Winter et al. , 2019 ) , or focuses on finding high-quality molecules with synthesis paths ( Bradshaw et al. , 2019 ; Korovina et al. , 2020 ; Gottipati et al. , 2020 ) . Most similar to our approach is RationaleRL ( Jin et al. , 2020 ) , which extracts subgraphs from seed molecules using Monte Carlo Tree Search ( MCTS ) and generates full molecules by completing the subgraphs . Compared with previous work , our approach is the first to incorporate an explainable model in the iterative search process . Existing work on explainable models approaches the problems from three directions . The first line of work uses gradients of the outputs with respect to inputs to identify the salient features in the inputs ( Simonyan et al. , 2013 ; Springenberg et al. , 2014 ; Baehrens et al. , 2010 ) ; the second line of work approximates the model with simple interpretable models , such as locally additive mod- els ( Bach et al. , 2015 ; Kindermans et al. , 2016 ; Ribeiro et al. , 2016 ; Lundberg & Lee , 2017 ; Shrikumar et al. , 2017 ) ; the third line of work defines input pattern selection operators , such that the outputs of the model based on the selected input patterns have high mutual information with the original model outputs ( Chen et al. , 2018 ; Ying et al. , 2019 ) . Our explainable model is different from GNNExplainer ( Ying et al. , 2019 ) in that we optimize the discrete subgraph structure with learned variational predictor , instead of directly feeding continuous edge masking into the target model . 3 PROBLEM SETTING . In this paper , we study the problem of discovering molecules g from the molecular space G with a high property score , measured by a scoring function f . And usually , there is a set of seed molecules G0 ⊂ G from experts with high scores to start with . More formally , the problem can be stated as Molecule Optimization . Given a scoring function f : G 7→ [ 0 , 1 ] , and a set of seed molecules G0 ⊂ G , the goal is to learn a molecule generative model p ( g ) such that the expected score of the generated molecules is maximized , i.e. , max p ( · ) Eg∼p ( · ) [ f ( g ) ] = ∫ g∈G p ( g ) f ( g ) dg ( 1 ) To prevent the model p ( g ) from generating a small set of fixed molecules with high scores , we additionally require the learned distribution to be both novel and diverse , i.e. , generating molecules that are dissimilar to the set of reference molecules ( a subset of G0 ) and each other . The molecule optimization problem in Eq ( 1 ) is combinatorial in nature , which poses a significant challenge . To mimic the scientific discovery process , we allow the algorithm to query f on new molecules under a querying budget . Examples of some well-known scoring functions include the QED score measuring the drug-likeness ( Bickerton et al. , 2012 ) , the SA score measuring the synthetic accessibility ( Ertl & Schuffenhauer , 2009 ) , the TPSA score measuring the ability to permeate cells ( Prasanna & Doerksen , 2009 ) , etc . The scoring function is general and could also encode multiproperty objectives ( Olivecrona et al. , 2017 ; Brown et al. , 2019 ) . Optimizing multiple properties together suffers from the sparsity of high scores , a scenario which is shown to be more challenging than single property optimization ( Jin et al. , 2020 ) . When experts are optimizing the molecular property , they will first look for substructures that result in the formation of that property , and use them as the foundation for building novel molecules . These subgraphs are called rationales ( examples in Figure 1 ) . The set of rationales is formally defined as , S = { s | ∃g ∈ G , s.t . s is a subgraph of g } . ( 2 ) 4 OUR FRAMEWORK . Our novel framework for optimizing molecular property with generative models consists of a modeling component and an algorithm component . In our modeling component , we propose a rationalebased hierarchical generative model for p ( g ) , which first generates rationales and then completes molecules . In our algorithm component , we design an alternating optimization procedure that interleaves between rationale distribution optimization and molecule generative model optimization . Furthermore , we develop a novel explainable graph model to effectively carry out the rationale model optimization . Next , we will first start describing our hierarchical generative model . 4.1 RATIONALE-BASED HIERARCHICAL GENERATIVE MODEL . To tackle the challenging search problem , we develop a hierarchical generative model that mimics the process of molecule optimization by human experts . In our model , we first sample rationales s from a distribution p ( s ) , and then molecules g will be generated according to conditional distribution pθ ( g|s ) . More specifically , our overall molecular generative model pθ ( g ) can be defined as pθ ( g ) = ∫ s∈S p ( s ) pθ ( g|s ) ds , ( 3 ) where θ is the parameter of the conditional generative model , p ( s ) is the latent rationales distribution . Here pθ ( g|s ) is a graph completion model from rationale s. The architecture of pθ ( g|s ) can be arbitrary . In this work , we use a latent variable model with a Gaussian prior p ( z ) , pθ ( g|s ) = ∫ z p ( z ) pθ ( g|s , z ) dz , ( 4 ) where pθ ( g|s , z ) is a variant of the GraphRNN ( You et al. , 2018b ; Liu et al. , 2018 ) by conditioning the graph generation on subgraphs . As part of the initialization , pθ ( g|s ) is first pretrained on ChEMBL ( Gaulton et al. , 2017 ) , a drug-like molecule dataset , in the same fashion as the variational autoencoder ( Kingma & Welling , 2013 ) , where the encoder is a standard GCN with atoms as vertices and bonds as edges . Note that different from p ( z ) , which is a fixed prior , p ( s ) will be updated in each round . And since representing a distribution on S is difficult , we will use particles to represent p ( s ) in the algorithm . In order to improve the diversity of the generated molecules , we will also regularize the entropy of the rationale distribution p ( s ) , leading to the following diversity-promoting objective function J ( θ , p ( s ) ) = Eg∼pθ ( · ) [ f ( g ) ] + λ ·H [ p ( s ) ] , ( 5 ) with a hyperparameter λ > 0 controlling the strength of the regularization .

Summary: The paper tackles the problem of molecule property optimisation. To this end, the authors proposes an alternating approach consisting of an explainer model and a molecule completion model. The explainer model takes a complete molecule as input and outputs a subgraph that represents the part that contributes most to property prediction. Then, the molecule completion model uses the subgraphs to sample a complete graph that can maximise the property scores. The loss function of molecule completion model directly maximises the properties, which is non-differentiable so that the authors use a REINFORCE algorithm for optimisation.