Constrained Bayesian Optimization for Automatic Chemical Design

1 INTRODUCTION . There are two fundamental ways in which Machine Learning can be leveraged in chemical design : 1 . To evaluate a molecule for a given application . 2 . To find a promising molecule for a given application . There has been much progress in the first use-case through the development of quantitative structure activity relationship ( QSAR ) models using deep learning ( Ma et al. , 2015 ) . These models have achieved state-of-the-art results in predicting properties of known molecules . The second use-case , finding new molecules that are useful for a given application , is arguably more important however . One existing approach for finding molecules that maximise an applicationspecific metric involves searching a large library of compounds , either physically or virtually PyzerKnapp et al . ( 2015 ) . This has the disadvantage that the search is not open-ended ; if the molecule is not in the library you specify , the search won ’ t find it . A second method involves the use of genetic algorithms . In this approach , a known molecule acts as a seed and a local search is performed over a discrete space of molecules . Although these methods have enjoyed success in producing biologically active compounds , an approach featuring a search over an open-ended , continuous space would be beneficial . The use of geometrical cues such as gradients to guide the search in continuous space could accelerate both drug ( Pyzer-Knapp et al. , 2015 ; Gómez-Bombarelli et al. , 2016a ) and material ( Hachmann et al. , 2011 ; 2014 ) discovery by functioning as a high-throughput virtual screen of unpromising candidates . Recently , Gómez-Bombarelli et al . ( Gómez-Bombarelli et al. , 2016b ) presented Automatic Chemical Design , a VAE architecture capable of encoding continuous representations of molecules . In continuous latent space , gradient-based optimization is leveraged to find molecules that maximize a design metric . Although a strong proof of concept , Automatic Chemical Design possesses a deficiency in so far as it fails to generate a high proportion of valid molecular structures . Gómez-Bombarelli et al . ( 2016b ) hypothesise that molecules selected by Bayesian Optimization lie in “ dead regions ” of the latent space far away from any data that the VAE has seen in training , yielding invalid structures when decoded . Although there have been many attempts to address the issue of generating valid molecules , they all rely on model assumptions that can negatively impact either the complexity ( Jaques et al. , 2017 ; Janz et al. , 2017 ; Kusner et al. , 2017 ) or the diversity ( Jin et al. , 2018 ) of the generated molecules . The principle contribution of this paper will be to present an approach based on constrained Bayesian optimization that generates a high proportion of valid sequences without relying on model assumptions that affect the complexity or the diversity of the generated molecules . 2 METHODS . 2.1 SMILES REPRESENTATION . SMILES strings Weininger ( 1988 ) are a means of representing molecules as a character sequence . This text-based format facilitates the use of tools from natural language processing for applications such as organic chemistry reaction prediction Schwaller et al . ( 2017 ) ; Jin et al . ( 2017 ) . For use with the variational autoencoder , the SMILES strings in turn are converted into one-hot vectors indicating the presence or absence of a particular character within a sequence as illustrated in Figure 1 . 2.2 VARIATIONAL AUTOENCODER . Variational autoencoders Kingma & Welling ( 2013 ) ; Kingma et al . ( 2014 ) allow us to map molecules x to and from continuous values z in a latent space . The encoding z is interpreted as a latent variable in a probabilistic generative model over which there is a prior distribution p ( z ) . The probabilistic decoder is defined by the likelihood function pθ ( x|z ) . The posterior distribution pθ ( z|x ) is interpreted as the probabilistic encoder . The parameters of the likelihood pθ ( x|z ) as well as the parameters of the approximate posterior distribution qφ ( z|x ) are learned by maximizing the evidence lower bound ( ELBO ) L ( φ , θ ; x ) = Eqφ ( z|x ) [ log pθ ( x , z ) − log qφ ( z|x ) ] . Variational autoencoders have been coupled with recurrent neural networks by Bowman et al . ( 2015 ) to encode sentences into a continuous latent space . This approach is followed for the SMILES format both by Gómez-Bombarelli et al . ( 2016b ) and here . The SMILES variational autoencoder , together with our constraint function , is shown in Figure 2 . 2.3 OBJECTIVE FUNCTIONS FOR BAYESIAN OPTIMIZATION OF MOLECULES . Bayesian Optimization is performed in the latent space of the variational autoencoder in order to find molecules that score highly under a specified objective function . We assess molecular quality on the following objectives : J logPcomp ( m ) = logP ( m ) − SA ( m ) − ring-penalty ( m ) , JQEDcomp ( m ) = QED ( m ) − SA ( m ) − ring-penalty ( m ) , JQED ( m ) = QED ( m ) . m denotes a molecule , logP ( m ) is the water-octanol partition coefficient , QED ( m ) is the quantitative estimate of drug-likeness ( Bickerton et al. , 2012 ) and SA ( m ) is the synthetic accessibility ( Ertl & Schuffenhauer , 2009 ) . The ring penalty term is as featured in ( Gómez-Bombarelli et al. , 2016b ) . The “ comp ” subscript is designed to indicate that the objective function is a composite of standalone metrics . It is important to note , that the first objective , a common metric of comparison in this area GómezBombarelli et al . ( 2016b ) ; Kusner et al . ( 2017 ) ; Jin et al . ( 2018 ) , is mis-specified . From a chemical standpoint it is undesirable to maximize the logP score as is being done here . Rather it is preferable to optimize logP to be in a range that is in accordance with the Lipinski Rule of Five Lipinski et al . ( 1997 ) . We use the penalized logP objective here because regardless of its relevance for chemistry , it serves as a point of comparison against other methods . 2.4 CONSTRAINED BAYESIAN OPTIMIZATION OF MOLECULES . We now describe our extension to the Bayesian Optimization procedure followed by GómezBombarelli et al . ( 2016b ) . Expressed formally , the constrained optimization problem is max m f ( m ) s.t . Pr ( C ( m ) ) ≥ 1− δ where f ( m ) is a black-box objective function , Pr ( C ( m ) ) denotes the probability that a boolean constraint C ( m ) is satisfied and 1− δ is some user-specified minimum confidence that the constraint is satisfied ( Gelbart et al. , 2014 ) . The constraint is that a latent point must decode successfully a large fraction of the times decoding is attempted . The black-box objective function is noisy because a single latent point may decode to multiple molecules when the model makes a mistake , obtaining different values under the objective . In practice , f ( m ) is one of the objectives described in section 2.3 2.5 EXPECTED IMPROVEMENT WITH CONSTRAINTS ( EIC ) . EIC may be thought of as expected improvement ( EI ) , EI ( m ) = Ef ( m ) [ max ( 0 , η − f ( m ) ) ] , that offers improvement only when a set of constraints are satisfied ( Schonlau et al. , 1998 ) : EIC ( m ) = EI ( m ) Pr ( C ( m ) ) . The incumbent solution η in EI ( m ) , may be set in an analogous way to vanilla expected improvement ( Gelbart , 2015 ) as either : 1 . The best observation in which all constraints are observed to be satisfied . 2 . The minimum of the posterior mean such that all constraints are satisfied . The latter approach is adopted for the experiments performed in this paper . If at the stage in the Bayesian optimization procedure where a feasible point has yet to be located , the form of acquisition function used is that defined by ( Gelbart , 2015 ) EIC ( m ) = { Pr ( C ( m ) EI ( m ) , if ∃m , Pr ( C ( m ) ) ≥ 1− δ Pr ( C ( m ) ) , otherwise with the intuition being that if the probabilistic constraint is violated everywhere , the acquisition function selects the point having the highest probability of lying within the feasible region . The algorithm ignores the objective until it has located the feasible region . It would also have been possible to adopt the methodologies of Rainforth et al . ( 2016 ) ; Mueller et al . ( 2017 ) under the assumption that the constraint is cheap to evaluate . 3 RELATED WORK . Gómez-Bombarelli et al . ( 2016b ) ; Segler et al . ( 2017 ) constructed character-level generative models of SMILES strings using recurrent decoders . These models both suffered from the problem that they produced a high proportion of invalid SMILES strings . Blaschke et al . ( 2017 ) compared a range of autoencoder architectures in terms of their ability to produce valid molecules . Kusner et al . ( 2017 ) ; Dai et al . ( 2018 ) addressed the issue of generating valid molecules explicitly by imposing syntactic and semantic constraints using context free and attribute grammars while Janz et al . ( 2017 ) ; Guimaraes et al . ( 2017 ) leveraged additional training signal to enforce molecular validity . A potential drawback of these methods is that the constraints may favour the generation of simple molecules . Simonovsky & Komodakis ( 2018 ) ; Li et al . ( 2018 ) build generative models of molecular graphs but do not impose any constraints for molecular validity . Jin et al . ( 2018 ) build a generative model of molecular graphs with constraints in order to achieve 100 % validity . One possible disadvantage of this approach is that each molecule is assumed to be constructed from a fixed vocabulary of subgraphs and so it will be impossible to generate a molecule comprised of subgraphs outside this vocabulary . De Cao & Kipf ( 2018 ) use a graph-based generative model which enforces a quality constraint in the latent space . Direct comparison with the aforementioned methods is complicated due to the orthogonality of the assumptions made for each model . As such , we compare out method against Gómez-Bombarelli et al . ( 2016b ) only . Our principle contribution is in showing that valid molecules can be generated without encoding assumptions into the design model that limit the complexity and diversity of the generated molecules . 4 EXPERIMENT I : DRUG DESIGN . In this section we conduct an empirical test of the hypothesis from ( Gómez-Bombarelli et al. , 2016b ) that the decoder ’ s lack of efficiency is due to data point collection in “ dead regions ” of the latent space far from the data on which the VAE was trained . We use this information to construct a binary classification Bayesian Neural Network ( BNN ) to serve as a constraint function that outputs the probability of a latent point being valid . Secondly , we compare the performance of our constrained Bayesian optimization implementation against the original model ( baseline ) in terms of the numbers of valid and drug-like molecules generated . Thirdly , we compare the quality of the molecules produced by constrained Bayesian optimization with those of the baseline model .

Summary: This paper proposes a novel method for generating novel molecules with some targeted properties. Many studies on how to generate chemically valid molecular graphs have been done, but it is still an open problem due to the essential difficulty of generating discrete structures from any continuous latent space. From this motivation, the 'constrained' Bayesian optimization (BO) is applied and analyzed. Posing 'constraints' on the validity is realized by probability-weighting onto the expected improvement scores in BO. The 'validity' probability is learned beforehand by Bayesian neural nets in a supervised way. As empirical evaluations, two case studies are presented, and quality improvements of generated molecules are observed.