Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Design

1 INTRODUCTION . Designing new functional materials , such as energy storage materials ( Hachmann et al. , 2011 ; Janet et al. , 2020 ) , therapeutic molecules ( Zhavoronkov et al. , 2019 ; Lyu et al. , 2019 ) , and environmentally friendly materials ( Zimmerman et al. , 2020 ; Yao et al. , 2021 ) , is key to many societal and technological challenges and is a central task of chemical science and engineering . However , traditional molecular design processes are not only expensive and time-consuming , but also rely heavily on chance and brute-force trial and error ( Sanchez-Lengeling & Aspuru-Guzik , 2018 ) . Thus , a systematic approach to molecular design that can leverage data and minimize the number of costly experiments is of great interest to the field and is a prerequisite for autonomous molecular discovery ( Coley et al. , 2020a ; b ) . The core of computer-aided molecular discovery is molecular design . The objective of the task is to identify novel molecules with desirable properties through de novo generation or to identify known molecules through virtual screening . There has been a growing interest in applying machine learning methods to tackle this task in recent years ( Gómez-Bombarelli et al. , 2018 ; Jin et al. , 2018 ; You et al. , 2018 ; Bradshaw et al. , 2019 ; 2020 ; Jin et al. , 2020 ; Fu et al. , 2021 ) , which has been the subject of many reviews ( Elton et al. , 2019 ; Schwalbe-Koda & Gómez-Bombarelli , 2020 ; Vanhaelen et al. , 2020 ) . Despite the large number of models developed , there are few examples that have proceeded to experimental validation or been used in a realistic discovery scenario ( Zhavoronkov et al. , 2019 ; Schneider & Clark , 2019 ) . One major barrier to the deployment of these algorithms is that they lack considerations of synthesizability ( Gao & Coley , 2020 ; Huang et al. , 2021 ) ; Gao & Coley ( 2020 ) have demonstrated that when applied to goal-directed optimization tasks , de novo molecular design algorithms can propose a high proportion of molecules for which no synthetic plan can be found algorithmically . Planning and executing a practical synthetic route for a hypothetical molecular structure is a bottleneck that hinders the experimental validation of molecular design algorithms . The goal of computerassisted synthesis planning ( CASP ) is to identify a series of chemically plausible reaction steps beginning from available starting materials to synthesize a target chemical compound . Machine learning methods have been applied to improve CASP model performance ( Segler et al. , 2018 ; Coley et al. , 2018 ; 2019b ; Schwaller et al. , 2020 ; Genheden et al. , 2020 ) , and experimental execution has validated significant advances in recent years ( Klucznik et al. , 2018 ; Coley et al. , 2019c ) . However , most current algorithms require tens of seconds or minutes to plan a synthetic route for one target compound due to the combinatorial complexity of the tree search . This cost makes a post hoc filtering strategy impractical in molecular design workflows that decouple de novo design and synthesis planning ( Gao & Coley , 2020 ) . However , synthesizability-constrained generation has emerged as a promising alternative to this two-step pipeline ( Section 2.1 ) . In this paper , we report a strategy to generate synthetic pathways as trees conditioned on a target molecular embedding as a means of simultaneously addressing the problems of design and synthesis . Proposed pathways are guaranteed to make use of purchasable starting materials and are required to follow the “ rules of chemistry ” as codified by expert-curated reaction templates , which can be made more or less conservative depending on the application . When applied to synthesis planning , we ask the model to generate synthetic trees conditioned on the target molecule . When applied to synthesizable molecular design , we optimize the fixed-length embedding vector using a numerical optimization algorithm ; then , we decode the optimized embedding to obtain the corresponding synthetic tree whose root molecule is the output . The idea builds on the work of Bradshaw et al . ( 2019 ) and Bradshaw et al . ( 2020 ) ; however , these methods failed to recover multi-step synthetic paths for any target molecules and were thus only applied to the task of synthesizable analog recommendation . In contrast , the method presented here can successfully recover multi-step retrosynthetic pathways in an amortized manner , in addition to being used for synthesizable analog recommendation . The main contributions of this paper can be summarized as : • We formulate a Markov decision process to model the generation of synthetic trees , allowing the generation of multi-step and convergent ( i.e. , nonlinear ) synthetic pathways . • We propose a model that is capable of ( 1 ) rapid bottom-up synthesis planning and ( 2 ) constrained molecular optimization that can explore a chemical space defined by a discrete action space of reaction templates and purchasable starting materials . • We show the first successful attempt to amortized multi-step synthesis planning of complex organic molecules , achieving relatively high reconstruction accuracy on test molecules . • We demonstrate encouraging results on de novo molecular optimization with multiple objective functions relevant to bioactive molecule design and drug discovery . 2 RELATED WORK . 2.1 SYNTHESIZABLE MOLECULAR DESIGN . While most molecular generative models focus on the generation of valid molecules with desired properties , there is growing interest in the generation of synthesizable molecules , as not all chemically valid molecules are synthetically accessible . MoleculeChef ( Bradshaw et al. , 2019 ) was one of the first neural models to cast the problem of molecular generation as the generation of one-step synthetic pathways , thus ensuring synthesizability , by selecting a bag of purchasable reactants and using a data-driven reaction predictor to enumerate possible product molecules . ChemBO ( Korovina et al. , 2020 ) extends constrained generation to the multi-step case , but is a stochastic algorithm that generates synthetic pathways iteratively using random selections of reactants as input to another data-driven reaction predictor . While MoleculeChef and ChemBO use neural models for reaction outcome prediction as the ground truth for chemical reactivity ( Coley et al. , 2019b ; Schwaller et al. , 2019 ) , reaction templates provide an alternate means of defining allowable chemical steps , algorithmically ( Coley et al. , 2019a ) or by hand-encoding domain expertise ( Molga et al. , 2019 ) . PGFS ( Gottipati et al. , 2020 ) and REACTOR ( Horwood & Noutahi , 2020 ) both use discrete reaction templates and formulate the generation of multi-step synthetic pathways as a Markov decision process and optimize molecules with reinforcement learning . Both are limited to linear synthetic pathways , where intermediates can only react with purchasable compounds and no reaction can occur between two intermediates . Their inability to design convergent syntheses limits the chemical space accessible to the model . It is worth noting that there also exist previously reported methods for non-neural synthesizability-constrained molecular design , such as SYNOPSIS ( Vinkers et al. , 2003 ) and DOGS ( Hartenfeller et al. , 2012 ) , which pre-date deep molecular generation . Most recently , Bradshaw et al . ( 2020 ) introduced the DoG-AE/DoG-Gen model , which treats synthetic pathways as directed acyclic graphs ( DAGs ) . DoG-Gen serializes the construction of the DAGs and uses a recurrent neural network for autoregressive generation . Dai Nguyen & Tsuda ( 2021 ) also employ an autoencoder ( AE ) framework , jointly trained with a junction tree variational autoencoder ( JT-VAE ) ( Jin et al. , 2018 ) . However , none of the previous methods for synthesizable molecular generation have succeeded in achieving high reconstruction accuracy . 2.2 SYNTHESIS PLANNING . Algorithms and models for synthesis planning have been in development since the 1960s when retrosynthesis was first formalized ( Corey & Wipke , 1969 ) . Various data-driven approaches have been introduced in recent years ( Segler et al. , 2018 ; Coley et al. , 2018 ; 2019b ; Schwaller et al. , 2020 ; Genheden et al. , 2020 ) , although expert methods with human-encoded “ rules of chemistry ” have arguably achieved greater success in practice ( Klucznik et al. , 2018 ; Mikulak-Klucznik et al. , 2020 ) . The primary distinction between these methods is how allowable single-step chemical transformations are defined to mimic physical reality as closely as possible ; they can all make use of similar tree search algorithms . While these tools can be used to plan routes to target molecules and filter compounds from de novo generation for which no pathway is found , none of them can be directly used for molecular generation . Moreover , they all approach synthesis planning retrosynthetically , working recursively from the target molecule towards purchasable starting materials ( i.e . in a top-down manner ) , whereas we propose a bottom-up approach that has the potential to be more computationally efficient by mitigating the need for a tree search . 2.3 COMBINING SYNTHESIZABLE DESIGN AND SYNTHESIS PLANNING . Our method can be used for synthesizable molecular design and synthesis planning . While our approach is most similar to Bradshaw et al . ( 2020 ) ’ s RetroDoG model , their model was only applied to structural analog generation , and could not demonstrate the successful recovery of target molecules . Bradshaw et al . ( 2019 ) showed some examples of successful recovery , but their formulation restricts the search to single-step synthetic pathways , which severely limits its practical utility for both tasks . In contrast , our model can successfully handle multi-step reactions . 3 METHOD . 3.1 PROBLEM DEFINITION . We model synthetic pathways as tree structures called synthetic trees ( Figure 6A in Appendix A ) . A valid synthetic tree has one root node ( the final product molecule ) linked to purchasable building blocks via feasible reactions according to a list of discrete reaction templates . A reaction template is a pattern defining a structural transformation on molecules that is intended to represent a valid chemical reaction , usually encoded as a SMARTS string ( Figure 6B & C ) . We use a list of reaction templates to define feasible chemical reactions instead of a data-driven reaction predictor so that the practical utility of the model can be improved by refining or expanding this set without changing its architecture . Given a list of reaction templates , R , and a list of purchasable compounds , C , our goal is to generate a valid synthetic tree , T , that produces a root molecule with a desired structure or function . The product molecule and intermediate molecules in the tree are not themselves generated by the model , but are implicitly defined by the application of reaction templates to reactant molecules . Compared to the generation of molecular graphs , the generation of synthetic trees is more difficult because of the additional constraints of enforcing chemical reaction rules and the commercial availability of starting materials . Synthesis Planning This task is to infer the synthetic pathway to a given target molecule . We formulate this problem as generating a synthetic tree , T , such that the product molecule it produces ( molecule at the root node ) , Mproduct , matches the desired target molecule , Mtarget . Synthesizable Molecular Design This task is to optimize a molecular structure with respect to an oracle function , while ensuring the synthetic accessibility of the molecules . We formulate this problem as optimizing the structure of a synthetic tree , T , with respect to the desired properties of the product molecule it produces , Mproduct .

Summary: The present paper is concerned about a forward-synthesis approach to molecular optimization. The proposed method defines one action to halt the procedure and three actions to develop a synthetic pathway, each of which corresponds to using zero/one/two of the intermediate molecules for chemical reaction to yield the next intermediate molecule. This generation model is controlled by an agent that chooses a series of actions to transit to the next state, given an embedding vector. If applied to synthesis planning, the embedding of the target molecule is used as the embedding vector, while if applied to molecular optimization, the embedding vector is optimized by a genetic algorithm.  The effectiveness of the proposed method is demonstrated by two experiments. The first one examines the ability of synthesis planning using reachable and unreachable data sets. The second one examines the ability of molecular optimization. For both of the experiments, both quantitative and qualitative analyses are provided.