Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Design

1 INTRODUCTION . Designing new functional materials , such as energy storage materials ( Hachmann et al. , 2011 ; Janet et al. , 2020 ) , therapeutic molecules ( Zhavoronkov et al. , 2019 ; Lyu et al. , 2019 ) , and environmentally friendly materials ( Zimmerman et al. , 2020 ; Yao et al. , 2021 ) , is key to many societal and technological challenges and is a central task of chemical science and engineering . However , traditional molecular design processes are not only expensive and time-consuming , but also rely heavily on chance and brute-force trial and error ( Sanchez-Lengeling & Aspuru-Guzik , 2018 ) . Thus , a systematic approach to molecular design that can leverage data and minimize the number of costly experiments is of great interest to the field and is a prerequisite for autonomous molecular discovery ( Coley et al. , 2020a ; b ) . The core of computer-aided molecular discovery is molecular design . The objective of the task is to identify novel molecules with desirable properties through de novo generation or to identify known molecules through virtual screening . There has been a growing interest in applying machine learning methods to tackle this task in recent years ( Gómez-Bombarelli et al. , 2018 ; Jin et al. , 2018 ; You et al. , 2018 ; Bradshaw et al. , 2019 ; 2020 ; Jin et al. , 2020 ; Fu et al. , 2021 ) , which has been the subject of many reviews ( Elton et al. , 2019 ; Schwalbe-Koda & Gómez-Bombarelli , 2020 ; Vanhaelen et al. , 2020 ) . Despite the large number of models developed , there are few examples that have proceeded to experimental validation or been used in a realistic discovery scenario ( Zhavoronkov et al. , 2019 ; Schneider & Clark , 2019 ) . One major barrier to the deployment of these algorithms is that they lack considerations of synthesizability ( Gao & Coley , 2020 ; Huang et al. , 2021 ) ; Gao & Coley ( 2020 ) have demonstrated that when applied to goal-directed optimization tasks , de novo molecular design algorithms can propose a high proportion of molecules for which no synthetic plan can be found algorithmically . Planning and executing a practical synthetic route for a hypothetical molecular structure is a bottleneck that hinders the experimental validation of molecular design algorithms . The goal of computerassisted synthesis planning ( CASP ) is to identify a series of chemically plausible reaction steps beginning from available starting materials to synthesize a target chemical compound . Machine learning methods have been applied to improve CASP model performance ( Segler et al. , 2018 ; Coley et al. , 2018 ; 2019b ; Schwaller et al. , 2020 ; Genheden et al. , 2020 ) , and experimental execution has validated significant advances in recent years ( Klucznik et al. , 2018 ; Coley et al. , 2019c ) . However , most current algorithms require tens of seconds or minutes to plan a synthetic route for one target compound due to the combinatorial complexity of the tree search . This cost makes a post hoc filtering strategy impractical in molecular design workflows that decouple de novo design and synthesis planning ( Gao & Coley , 2020 ) . However , synthesizability-constrained generation has emerged as a promising alternative to this two-step pipeline ( Section 2.1 ) . In this paper , we report a strategy to generate synthetic pathways as trees conditioned on a target molecular embedding as a means of simultaneously addressing the problems of design and synthesis . Proposed pathways are guaranteed to make use of purchasable starting materials and are required to follow the “ rules of chemistry ” as codified by expert-curated reaction templates , which can be made more or less conservative depending on the application . When applied to synthesis planning , we ask the model to generate synthetic trees conditioned on the target molecule . When applied to synthesizable molecular design , we optimize the fixed-length embedding vector using a numerical optimization algorithm ; then , we decode the optimized embedding to obtain the corresponding synthetic tree whose root molecule is the output . The idea builds on the work of Bradshaw et al . ( 2019 ) and Bradshaw et al . ( 2020 ) ; however , these methods failed to recover multi-step synthetic paths for any target molecules and were thus only applied to the task of synthesizable analog recommendation . In contrast , the method presented here can successfully recover multi-step retrosynthetic pathways in an amortized manner , in addition to being used for synthesizable analog recommendation . The main contributions of this paper can be summarized as : • We formulate a Markov decision process to model the generation of synthetic trees , allowing the generation of multi-step and convergent ( i.e. , nonlinear ) synthetic pathways . • We propose a model that is capable of ( 1 ) rapid bottom-up synthesis planning and ( 2 ) constrained molecular optimization that can explore a chemical space defined by a discrete action space of reaction templates and purchasable starting materials . • We show the first successful attempt to amortized multi-step synthesis planning of complex organic molecules , achieving relatively high reconstruction accuracy on test molecules . • We demonstrate encouraging results on de novo molecular optimization with multiple objective functions relevant to bioactive molecule design and drug discovery . 2 RELATED WORK . 2.1 SYNTHESIZABLE MOLECULAR DESIGN . While most molecular generative models focus on the generation of valid molecules with desired properties , there is growing interest in the generation of synthesizable molecules , as not all chemically valid molecules are synthetically accessible . MoleculeChef ( Bradshaw et al. , 2019 ) was one of the first neural models to cast the problem of molecular generation as the generation of one-step synthetic pathways , thus ensuring synthesizability , by selecting a bag of purchasable reactants and using a data-driven reaction predictor to enumerate possible product molecules . ChemBO ( Korovina et al. , 2020 ) extends constrained generation to the multi-step case , but is a stochastic algorithm that generates synthetic pathways iteratively using random selections of reactants as input to another data-driven reaction predictor . While MoleculeChef and ChemBO use neural models for reaction outcome prediction as the ground truth for chemical reactivity ( Coley et al. , 2019b ; Schwaller et al. , 2019 ) , reaction templates provide an alternate means of defining allowable chemical steps , algorithmically ( Coley et al. , 2019a ) or by hand-encoding domain expertise ( Molga et al. , 2019 ) . PGFS ( Gottipati et al. , 2020 ) and REACTOR ( Horwood & Noutahi , 2020 ) both use discrete reaction templates and formulate the generation of multi-step synthetic pathways as a Markov decision process and optimize molecules with reinforcement learning . Both are limited to linear synthetic pathways , where intermediates can only react with purchasable compounds and no reaction can occur between two intermediates . Their inability to design convergent syntheses limits the chemical space accessible to the model . It is worth noting that there also exist previously reported methods for non-neural synthesizability-constrained molecular design , such as SYNOPSIS ( Vinkers et al. , 2003 ) and DOGS ( Hartenfeller et al. , 2012 ) , which pre-date deep molecular generation . Most recently , Bradshaw et al . ( 2020 ) introduced the DoG-AE/DoG-Gen model , which treats synthetic pathways as directed acyclic graphs ( DAGs ) . DoG-Gen serializes the construction of the DAGs and uses a recurrent neural network for autoregressive generation . Dai Nguyen & Tsuda ( 2021 ) also employ an autoencoder ( AE ) framework , jointly trained with a junction tree variational autoencoder ( JT-VAE ) ( Jin et al. , 2018 ) . However , none of the previous methods for synthesizable molecular generation have succeeded in achieving high reconstruction accuracy . 2.2 SYNTHESIS PLANNING . Algorithms and models for synthesis planning have been in development since the 1960s when retrosynthesis was first formalized ( Corey & Wipke , 1969 ) . Various data-driven approaches have been introduced in recent years ( Segler et al. , 2018 ; Coley et al. , 2018 ; 2019b ; Schwaller et al. , 2020 ; Genheden et al. , 2020 ) , although expert methods with human-encoded “ rules of chemistry ” have arguably achieved greater success in practice ( Klucznik et al. , 2018 ; Mikulak-Klucznik et al. , 2020 ) . The primary distinction between these methods is how allowable single-step chemical transformations are defined to mimic physical reality as closely as possible ; they can all make use of similar tree search algorithms . While these tools can be used to plan routes to target molecules and filter compounds from de novo generation for which no pathway is found , none of them can be directly used for molecular generation . Moreover , they all approach synthesis planning retrosynthetically , working recursively from the target molecule towards purchasable starting materials ( i.e . in a top-down manner ) , whereas we propose a bottom-up approach that has the potential to be more computationally efficient by mitigating the need for a tree search . 2.3 COMBINING SYNTHESIZABLE DESIGN AND SYNTHESIS PLANNING . Our method can be used for synthesizable molecular design and synthesis planning . While our approach is most similar to Bradshaw et al . ( 2020 ) ’ s RetroDoG model , their model was only applied to structural analog generation , and could not demonstrate the successful recovery of target molecules . Bradshaw et al . ( 2019 ) showed some examples of successful recovery , but their formulation restricts the search to single-step synthetic pathways , which severely limits its practical utility for both tasks . In contrast , our model can successfully handle multi-step reactions . 3 METHOD . 3.1 PROBLEM DEFINITION . We model synthetic pathways as tree structures called synthetic trees ( Figure 6A in Appendix A ) . A valid synthetic tree has one root node ( the final product molecule ) linked to purchasable building blocks via feasible reactions according to a list of discrete reaction templates . A reaction template is a pattern defining a structural transformation on molecules that is intended to represent a valid chemical reaction , usually encoded as a SMARTS string ( Figure 6B & C ) . We use a list of reaction templates to define feasible chemical reactions instead of a data-driven reaction predictor so that the practical utility of the model can be improved by refining or expanding this set without changing its architecture . Given a list of reaction templates , R , and a list of purchasable compounds , C , our goal is to generate a valid synthetic tree , T , that produces a root molecule with a desired structure or function . The product molecule and intermediate molecules in the tree are not themselves generated by the model , but are implicitly defined by the application of reaction templates to reactant molecules . Compared to the generation of molecular graphs , the generation of synthetic trees is more difficult because of the additional constraints of enforcing chemical reaction rules and the commercial availability of starting materials . Synthesis Planning This task is to infer the synthetic pathway to a given target molecule . We formulate this problem as generating a synthetic tree , T , such that the product molecule it produces ( molecule at the root node ) , Mproduct , matches the desired target molecule , Mtarget . Synthesizable Molecular Design This task is to optimize a molecular structure with respect to an oracle function , while ensuring the synthetic accessibility of the molecules . We formulate this problem as optimizing the structure of a synthetic tree , T , with respect to the desired properties of the product molecule it produces , Mproduct .

Summary:  This paper proposes a bottom-up approach to molecular synthesis planning and synthesizable molecular design. The authors cast the synthesis planning problem as a Markov Decision Process with trainable networks for selecting next actions.  The output of the approach is a binary tree that documents reaction and the combination of building blocks or previously synthesized reagents that would mix to create a new intermediate or a final molecule.  When the approach fails to produce the correct target molecules it typically generates a similar molecule---the authors take advantage of this feature of their approach and optimize their algorithm to design synthesizable molecules with desirable properties.