Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Design
Authors: Wenhao Gao
Rocío Mercado
Connor W. Coley
Venue: ICLR 2022 Spotlight (2022)
Keywords: molecular design
synthesis planning
tree generation
graph generation

Abstract: Molecular design and synthesis planning are two critical steps in the process of molecular discovery that we propose to formulate as a single shared task of conditional synthetic pathway generation. We report an amortized approach to generate synthetic pathways as a Markov decision process conditioned on a target molecular embedding. This approach allows us to conduct synthesis planning in a bottom-up manner and design synthesizable molecules by decoding from optimized conditional codes, demonstrating the potential to solve both problems of design and synthesis simultaneously. The approach leverages neural networks to probabilistically model the synthetic trees, one reaction step at a time, according to reactivity rules encoded in a discrete action space of reaction templates. We train these networks on hundreds of thousands of artificial pathways generated from a pool of purchasable compounds and a list of expert-curated templates. We validate our method with (a) the recovery of molecules using conditional generation, (b) the identification of synthesizable structural analogs, and (c) the optimization of molecular structures given oracle functions relevant to bioactivity and drug discovery.

Published as a conference paper at ICLR 2022
AMORTIZED TREE GENERATION FOR BOTTOM-UP SYNTHESIS PLANNING AND SYNTHESIZABLE MOLECULAR DESIGN
Wenhao Gao', Rocío Mercado' & Connor W. Coley2.2
Department of Chemical Engineering DDepartment of Electrical Engineering and Computer Science
Massachusetts Institute of Technology Cambridge, MA 02142, USA [whgao, rociomer, ccoley @emmm.eei
ABSTRACT
Molecular design and synthesis planning are two critical steps in the process of molecular discovery that we propose to formulate conditional synthetic pathway generation. We report an amortized approach to generate synthetic pathways as a Markov decision process conditioned on a target molecular embedding. This approach allows us to conduct synthesis planning in bottom-up manner and design synthesizable molecules by decoding from optimized conditional codes, demonstrating the potential to solve both problems of design and synthesis simultaneously. The approach leverages neural networks probabilistically model the synthetic trees, one reaction step at time, according to reactivity rules encoded in discrete action space of reaction templates. We train these networks on hundreds of thousands of artificial pathways generated from a pool of purchasable compounds and list of expert-curated templates. We validate our method with (a) the recovery of molecules using conditional generation, (b) the identification of synthesizable structural analogs, and (c) the optimization of molecular structures given oracle functions relevant to drug discovery.
single shared task of
INTRODUCTION
Designing new functional materials, such as energy storage materials (Hachmann et al., 2011 Janet al. 2020). therapeutic molecules (Zhavoronkov al. 2019: Lyu et al. 2019), and environmentally friendly materials (Zimmerman et al. [200 Yao et al. 2021). is key to many societal and technological challenges and is central task of chemical science and engineering. However, traditional molecular design processes are not only expensive and consuinng., but also rely heavily on chance and brute-force trial and error (Sanchez-Leenelling & Aspuru-Guziie 2018). Thus, a systematic approach to molecular design that can leverage data and minimize the number of costly experiments is of great interest to the field and is prerequisite for autonomous molecular discovery (Coley et al. 2020a b).
The core of computer-aided molecular discovery is molecular design. The objective of the task is to identify novel molecules with desirable properties through de novo generation
identify known
molecules through virtual screening. There has been growing interest in applying machine learning methods to tackle this task in recent years et al., 2018 Bradshaw et al.) 2019 2020 Jin et al.] 20.0 [Fu et al. [2021] which has been the subject Hlomez-Bombarele
Hoomez-Bombarelli et 2018 Jin al.) 2018 You
2020 Vanhaelen et 2020). Despite the large number of models developed, there are few examples that have proceeded to experimental validation or been used in realistic discovery scenario (Zhavoronkov et al. 2019 9
of many reviews (Elton et al. 2019 Schwalbe- Koda
Schneider & Clark 2019). One major barrier to the deployment of these algorithms is that they lack considerations of synthesizability (Gao & Coley, 2020; Huang et al. 202T); Gao & Coley (022)) have demonstrated that when applied to goal-directed optimization tasks, de novo molecular design algorithms can propose a high proportion molecules for which no synthetic plan can be found algorithmically.

---

generation has emerged as a Published as a conference paper at ICLR 2022
Planning and executing a practical synthetic route for a hypothettcal molecular structure is a bottleneck that hinders the experimental validation of molecular design algorithms. The goal of computer- assisted synthesis planning (CASP) is to identify series of chemically plausible reaction steps beginning from available starting materials to synthesize a target chemical compound. Machine learning methods have been applied to improve CASP model performance (Segler al.] 2018 Coley et al. 2018 2019b Schwaller et al., 2020] Genheden et al.) 2020), and experimental execution has validated significant advances in recent years (Klucznik al. 2018; Coley et al., 2019c). However, most current algorithms require tens of seconds or minutes to plan a synthetic route for one target compound due to the combinatorial complexity of the tree search. This cost makes a post hoe filtering strategy impractical in molecular design workflows that decouple de novo design and synthesis planning (Gao & Coley 2020). However, synthesizability-conntaaa promising alternative to this two-step pipeline (Section 1)..
In this paper, we report a strategy to generate synthetic pathways as trees conditioned on a target molecular embedding as a means of simultaneously addressing the problems of design and synthesis. Proposed pathways are guaranteed to make use of purchasable starting materials and are required to follow the "rules of chemistry" as codified by expert-curated reaction templates, which can be made more or less conservative depending on the application. When applied to synthesis planning, ask the model generate synthetic trees conditioned on the target molecule. When applied to synthesizable molecular design, we optimize the fixed-length embedding vector using a numerical optimization algorithm; then, we decode the optimized embedding to obtain the corresponding synthetic tree whose root molecule is the output. The idea builds on the work of Bradshaw et al. (2019) and Bradshaw et al. 2020); however, these methods failed to recover multi-step synthetic paths for any target molecules and were thus only applied to the task of synthesizable analog recommendation. In contrast, the method presented here can successfully recover multi-step retrosynthetic pathways in an amortized manner, in addition to being used for synthesizable analog recommendation.
The main contributions of this paper can be summarized as:
We formulate Markov decision process model the generation of synthetic trees, allowing the generation of multi-step and convergent (i.e., nonlinear) synthetic pathways.
We propose a model that is capable of (1) rapid bottom-up synthesis planning and (2) constrained molecular optimization that can explore a chemical space defined by a discrete action space of reaction templates and purchasable starting materials.
We show the first successful attempt amortized multi-step synthesis planning of complex organic molecules, achieving relatively high reconstruction accuracy on test molecules.
We demonstrate encouraging results on de novo molecular optimization with multiple objective functions relevant bioactive molecule design and drug discovery.
RELATED WORK
2.1 SYNTHESIZABLE MOLECULAR DESIGN
While most molecular generative models focus on the generation of valid molecules with desired properties, there is growing interest in the generation of synthesizable molecules, as not all chemically valid molecules synthetically accessible. MoleculeChef (Bradshaw et al.) 2019) was one of the first neural models to cast the problem of molecular generation as the generation of one-step synthetic pathways. thus ensuring synthesizability, by selecting bag of purchasable reactants and using a data-driven reaction predictor to enumerate possible product molecules. ChemBO (Korovinal al. 2020) extends constrained generation to the multi-step case, but a stochastic algorithm that generates synthetic pathways iteratively using random selections of reactants as input to another data-driven reaction predictor. While MoleculeChef and ChemBO use neural models for reaction outcome prediction as the ground truth for chemical reactivity (Coley et al.) 2019b] Schwaller cet al. 2019), reaction templates provide an alternate means of defining allowable chemical steps, algorithmically (Coley et al.) 2019a) or by hand-encoding domain expertise (Molga et al. 2019). PGFS (Gottipati et al. (020) and REACTOR (Horwood & Noutahi 2020) both use discrete reaction templates and formulate the generation of multi-step synthetic pathways as Markov decision process and optimize molecules with reinforcement learning. Both are limited to linear synthetic pathways, where intermediates can only react with purchasable compounds and no reaction can occur between
2

---

Published as a conference paper at ICLR 2022
two intermediates. Their inability to design convergent syntheses limits the chemical space accessible to the model. It is worth noting that there also exist previously reported methods for non-neure brathessizabilit--oonstraiii
molecular design, such SYNOPSIS (Vinkers et al. 2003 and DOGS
(Hartenfeller et al. 2012), which pre-date deep molecular generation.
Most recently, BBadshaw et. (2020) introduced the DoG-AE/DoG- model, which treats synthetic pathways as directed acyclic graphs (DAGs). DoG-Gen serializes the construction of the DAGs and uses a recurrent neural network for autoregressive generation. Dai Nguyen & Tsuda (202I) also employ an autoencoder (AE) framework, jointly trained with a junction tree variational autoencoder VAE) (Jin et al., 2018). However, none of the previous methods for synthesizable molecular generation have succeeded in achieving high reconstruction accuracy.
2.2 SYNTHESIS PLANNING
Algorithms and models for synthesis planning have been in development since the 1960s when
retrosynthesis was first formalized (Corey & Wipke T969). Various data-driven approaches have
been introduced in recent years (Segler et al. [0188] Coley et al. 2018 2019b Schwaller et al. 2020
Genheden et al. 2020), although expert methods with human- encoded "rules of chemistry' have
arguably achieved greater success in practice (Klucznik et al. 2018 Mikkulak-Kluzznn et al. 2020).
The primary distinction between these methods how allowable single-step chemical transformations are defined to mimic physical reality as closely as possible; they can all make use of similar tree search algorithms. While these tools can be used to plan routes to target molecules and filter compounds from de novo generation for which no pathway is found, none of them can be directly used for molecular generation. Moreover, they all approach synthesis planning retrosyntherically, working recursively from the target molecule towards purchasable starting materials (i.e. in a top-down manner), whereas we propose bottom-up approach that has the potential to be more computationally efficient by mitigating the need for tree search.
2.3 COMBINING SYNTHESIZABLE DESIGN AND SYNTHESIS PLANNING
Our method can be used for synthesizable molecular design and synthesis planning. While our approach is most similar to Bradshaw et al. (2020)s 's RetroDoG model, their model was only applied to structural analog generation, and could not demonstrate the successful recovery of target molecules. Bradshaw et al. (201)) showed some examples of successful recovery, but their formulation restricts the search single-step synthetic pathways, which severely limits its practical utility for both tasks. In contrast, our model can successfully handle multi-step reactions.
3 METHOD
3.1 PROBLEM DEFINITION
We model synthetic pathways as tree structures called synthetic trees (Figure 6 in Appendix AA. valid synthetic tree has one root node (the final product molecule) linked purchasable building blocks via feasible reactions according to a list of discrete reaction templates. A reaction template is a pattern defining a structural transformation on molecules that is intended represent valid chemical reaction, usually encoded as a SMARTS string (Figure &BCC). We use a list of reaction templates to define feasible chemical reactions instead of data-driven reaction predictor so that the practical utility of the model can be improved by refining or expanding this set without changing its architecture. Given a list of reaction templates, R, and list of purchasable compounds, C, our goal a valid synthetic tree, T, that produces root molecule with a desired structure or function. The product molecule and intermediate molecules in the tree are not themselves generated by the model, but are implicitly defined by the application of reaction templates to reactant molecules. Compared to the generation of molecular graphs, the generation of synthetic trees is more difficult because of the additional constraints of enforcing chemical reaction rules and the commercial availability of starting materials.
generate
Synthesis Planning This task is to infer the synthetic pathway given target molecule. We formulate this problem as generating a synthetic tree, T, such that the product molecule it produces (molecule at the root node), M Mproduet, matches the desired target molecule, Margn-
3

---

Published as a conference paper at ICLR 2022
Add
Expand
Add
Expand
Merge
Conditioned
Synthetic tree
Bu Building Blocks
Bi-molecular reaction
Products
Most Molccule
Target Molecule
Uni-molecular reaction
Target
Target
Product
molecule
molecule
molecula
embedding
Figure 1: An illustration of the iterative generation procedure. Our model constructs the synthetic tree in bottom-up manner, starting from the available building blocks and building up to progressively more complex molecules. Generation is conditioned on an embedding for a target molecule. the target molecule is in the chemical space reachable by our template and building blocks, the final root molecule should match or at least be similar to the input target molecule.
Synthesizable Molecular Design This task is to optimize a molecular structure with respect to an oracle function, while ensuring the synthetic accessibility the molecules. We formulate this problem optimizing the structure of a synthetic tree, T, with respect to the desired properties of the product molecule it produces, Mproduct.
3.2 SYNTHETIC TREE GENERATION AS A MARKOV DECISION PROCESS
We propose an amortized approach to tackle the probabilistic modeling of synthetic trees. In this approach, we model the construction of synthetic tree as a Markov decision process (MDP), which requires that the state transition satisfies the Markov property: p(S((+1)) (())
(((()) This property is naturally satisfied by synthetic trees: upon obtaining specific compound (an intermediate in a synthetic route), subsequent reaction steps can be inferred entirely from the intermediate compound, and do not depend on the pathway used to get to said compound when conditioned on the target molecule. Below, we first introduce an MDP framework for synthetic tree generation, and then introduce the model to solve it. In our framework, we only allow uni- and bi-molecular reactions.
At a high level, we construct synthetic tree one reaction step at a time in a bottom-up manner. Figure [ illlustrates a generation process for synthesis planning purposes. We enforce that the generation process happens in reverse depth-fire first order, and that more than two disconnected sub-trees generated simultancously.
State Space We define the state, S(t), as the root molecule(s) of an intermediate synthetic tree, (1) at step I. Because we enforce that at most two sub-trees can occur simultaneously, there can be at most two root molecules. All root nodes are generated in a reverse depth-first fashion; additionally, we enforce that the synthetic tree always expands from the most recently added node (M moosssroo roccmt). and that any merging always happens between two root nodes. The state embedding of a synthetic tree is thus computed by concatenating the embeddings for the two root molecules.
Action Space We decompose the action taken at each iteration into four components: (1) the action type, Oaci, which samples from possible actions "Add", "Expand". "Merge", and "End"; (2) the first reactant, agt. which samples from either or Mmost_recmt: (3) the reaction template, aan, which samples from R; and (4) the second reactant, On2, which samples from
(a) If dact "Add", one or two new reactant nodes will be added to T't), well as a new node corresponding to their product given specific reaction template. This is always the first action used in building a synthetic tree and leads an additional sub-tree.
(b) If dast = "Expand", the most recent molecule, Mmost recents is used as the first reactant, and second reactant is selected if arz is bi-molecular reaction template. If aran is uni-molecular reaction template, only new product node is added to (t))) If arxn is a bi-molecular reaction template, both a new product node and a new reactant node are added.
(c) If Gast = "Merge", the two root nodes are used as the reactants in a bi-molecular reaction. In this case, a new product node is added to ((t) and the two tab-trees are joined to form one sub-tree.
(d) If duct "End", T (T)))) and the synthetic tree is complete. The last product node is M Mproduct-
4

---

Published as a conference paper at ICLR 2022
State Transition Dynamics Each reaction represents one transition step. To ensure that each reaction step is chemically plausible and has a high likelihood of experimental success, we incorporate domain-specific reaction rules encoded reaction templates in R. Once valid action is selected, the transition is deterministic; infeasible actions that do not follow a known template are rejected. Importantly, the structure generated by template application is explicitly incorporated into the new state, whereas the RNN model in Bradshaw et al. (2020) had to implicitly learn the dynamics of the environment (i.e., the outcome of the reaction predictor).
Reward For synthesis planning, the reward is the similarity of the product to the target molecule, with similarity of 1.0 being the highest reward and indicating a perfect match. For molecular design. the reward is determined by how well the product properties match the desired criteria.
3.3 CONDITIONAL GENERATION FOR SYNTIESIS PLANNING
We model synthesis planning as a conditional synthetic tree generation problem. To solve this MDP,
we train a model to predict the action, at*), based on the state embedding, "state- at step t, condi-
_(t)
tioned on M Murget- Concretely, at each step, our model, f, estimates at)) (aact Ond Grxns 022
(4) (t) (2) (t)
p(a!")/s(t (((), Marga).
As summarized in Figure 2 our model consists of four modules: (1) an Action Type selection function, fact. that classifies action types among the four possible actions ("Add", "Expand", "Merge", and "End");; (2) a First Reactant selection function, fui, that predicts an embedding for the first reactant. candidate molecule is identified for the first reactant through k-nearest neighbors (k-NN) search from the potential building blocks, C (Cover & Hart 1967). We use the predicted embedding as query to pick the nearest neighbor among the building blocks; (3) a Reaction selection function, fron- whose output is a probability distribution over available reaction templates, from which inapplicable reactions are masked (based on reactant 1) and a suitable template is then sampled using a greedy search; (4) Second Reactant selection function, faz. that identifies the second reactant if the sampled template is bi-molecular. The model predicts an embedding for the second reactant, and a candidate is then sampled via k-NN search from the masked building blocks, C'.
Formally, these four modules predict the probability distributions of actions within one reaction step:
anei ~ fua(S()) Murger) = o (MLP Pact (Zstate
Zearget) )
(1) fui (S(0) MMarget) k-NNe (MLP In (sttat +
Zearget)
(1)
(t) fim(((()) Orn .Marget) a (MLP rxo Sstare
(1)
(t)
Zearget
4n2 (n(((() O.d : 1xx Murges) = k-NN cr (MLP Staat
@ Ziarget ① =rll + -(0)
where @ denotes concatenation, MLP, denotes a multilayer perceptron (MLP), 2, denotes the embedding of the corresponding entity, C' denotes subset of C that masks out reactants that do not match the selected template arzn and k-NNx is a k-NN from set X. The k-NN search is based on the cosine similarity between the query vector and the embeddings of all molecules in X. Whereas all molecular embeddings (Zrurget, 2n11 and 2nz2 are molecular fingerprints (see Representations in
Section 4.1). Z is a one-hot encoding of (() and z is a concatenation of molecular fingerprints for the root molecules in T(t), The a a softmax masking known invalid actions at each step, such as inapplicable action types (based on the topology of the intermediate tree T(0) and reaction templates (based on the requirement for a subgraph match). Each MLP is trained as a separate supervised learning problem using a subset of information from the known synthetic routes. For further details on the model and algorithm, see Appendices D
3.4 GENETIC ALGORITHM FOR MOLECULAR OPTIMIZATION
We approach the problem of synthesizable molecular design by optimizing the molecular embedding. Auargt, which tree generation is conditioned, with respect to the desired properties in M Mproduct. We adopt a genetic algorithm (GA) to perform the numerical optimization on Zugat- This approach is procedurally simpler than reinforcement learning solve the MDP and biases generation toward high-performin molecules, enabled by our conditional generation model. The mating pool is defined as list of molecular embedding vectors and the fitness function is the desired properties of produced
5

---

Published as a conference paper at ICLR 2022
Masked Classificatio
Most recent embedding
Action Network
Action
Nearest Neighbor Search
Reactanti Network
Reactant1
Another root molecule embedding
Update synthetic tree
Masked Classification
Reaction
Target molecule le embedding
embedding
Nearest Neighbor Search
Reaciant2 Network
Reaciant2
Figure 2: Overview of our model. Within each step, at most two root molecules (most recent and another) and the target molecule as a conditional code fully describe the state. The networks take the embedding of the state and predict the action type, first reactant, reaction template, and second reactant, successively. Those results are used to update the synthetic tree for one reaction step.
product molecules. Within each generation, offspring pool is constructed from crossover of two vectors randomly sampled from the mating pool as parents. crossover is defined as inheriting roughly half of the bits from parent and the remaining bits from another. Mutation can happen to each offspring vector with a small probability, and we decode them to synthetic trees to evaluate the fitness function. The top-performing vectors selected form the mating pool for the next generation, and we repeat this process until the stop criteria are met. See Figure for an illustration.
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
Reaction Templates We use set of reaction templates based on two publicly available template sets from Hartenfeller al. (011) and Button et al. (2019). We combine the two sets, removing duplicate and rare reactions, and obtain final set of 91 reaction templates. This set contains 13 uni-molecular and 78 bi-molecular reactions. 63 are mainly used for skeleton formation, 23 are used for peripheral modifications, and the remaining 5 can be used for either. Within the skeleton formation reactions, we include 45 ring formation reactions, comprising 37 heterocycle formations and 8 carbocycle formations.
Purchasable Building Blocks The set of purchasable compounds comprises 147,505 molecules from Enamine Building Blocks (US stock; accessed on May 12, 2021) that match at least one reaction template our set.
Dataset Preparation To prepare dataset of synthetic pathways obeying these constraints. applied a random policy to the MDP described in Section 3.2 After filtering by the QED of the product molecules, obtain 208,66 644 synthetic trees for training, 69,548 trees each for validation and testing after a random split. We refer to Appendix E for further detail.
Representations We use Morgan circular molecular fingerprints of length 4096 and radius 2 to represent molecules as inputs to the MLPs, and Morgan fingerprints of length 256 and radius 2 as inputs the k-NN module. Additional experiments with other molecular representations showed worse empirical results; further analysis is provided in Appendix [
Genetic Algorithm The GA operates on Morgan fingerprints of 4096 bits and radius 2. The number of bits to inherit is sampled from A (2048, 410). Mutation is defined as flipping 24 bits in the fingerprint and occurs with probability 0.5. The population size is 128 and the offspring size is 512. The stop criteria is met when either (a) the model reaches 200 generations, or (b) the increase in the population mean value 0.01 across 10 generations, indicating some degree of convergence.
Optimization Oracle To validate our model, we select several common oracle functions relevant to bioactivity and drug discovery, including a docking oracle. The heuristic oracle functions include: quantitative estimate of drug-likeness (QED); octanol-water partition coefficient (LogP); and JNK3, GSK33, and DRD2 surrogate models which estimate the response against c-Jun N-terminal kinases-3, glycogen synthase kinase 3,3 and dopamine receptor type 2, respectively. For the docking simulations, we use AutoDock Vina (Tott & Olson 2010) to dock against the human dopamine receptor D3 (DRD3, PDB ID: 3PBL), and the main protease, M of the SARS-Cov-2 virus (PDB ID: 7L1 1). We access all oracle functions through the Therapeutic Data Common (TDC) interface (Huang et al. 2021

---

Published as a conference paper at ICLR 2022
A. Synthesis Planning
Molecule
Most Similar Molecule Training Data
Similrity 0.300
Proposed Synthetic Pathweyy
My
B. Synthesizable Analog Recommendation
Target Molocule
Product
Holo
Similarit
Similarity 0.54
Propased Synthetic Pathway
Inspiration Target Bynthesis
Target Molecule
Product Molecule
Most Similar Molecule Training
Similartty = 0.830
Similanty - 0.453
Proposed Synthedic Pathway
Figure 3: Examples from ChEMBL used as the target molecule for conditioned generation. (A) successfully recovered molecule where the low similarity to any training examples indicates the generalizability of the model. (B) A molecule that is not recovered analog recommendation (i.e., a similar product). (C) A molecule that is not recovered but may inspire route development to the true target. Matched substructures are highlighted.
example of synthesizable
4.2 SYNTHESIS PLANNING RESULTS
We evaluate the model's ability to reconstruct target molecules that are reachable and unreachable under specific choice of reaction templates and available building blocks. We use the testing data as "reachable" targets (69,548), and a random sample from ChEMBL as predominantly "unreachable" molecules (20,000). None of the products in the test set seen in the training and validation stages. We use = in the first reactant k-NN search and expand the trees a greedy manner (k = 1) for each choice. From the obtained product molecules, choose the one that is the most similar the target molecule as the output as reflected by the Tanimoto similarity using Morgan fingerprints.
Table 1: Results of synthetic tree construction for "reachable" and "unreachable" target molecules.
Dataset
N
Recovery Ratet
Average Similarity?
KL Divergencet
FC Distance
Reachable (test set) 69.548
51.0%
0.759
0.995
0.067
Unreschable (ChEMB1.)
20,000
4.5%
0.423
0.966
1.944
Results are summarized in Table 1 The recovery rate measures the fraction of molecules that our model succeeds in reconstructing. Our model can reconstruct 51% % of reachable molecules from the held-out test set. As opposed to the typical top-down approach to synthesis planning (i.e., retrosynthesis) that requires tens of seconds or even minutes, our bottom-up approach takes only ~1 second to greedily construct tree with & 1. Our model only recovers 4.5% of ChEMBL molecules. but we note that different choice templates and starting materials can lead to a much higher ChEMBL. recovery rate (Gao & Coley 2020) without changing the model architecture. Figure A
7

---

Published as a conference paper at ICLR 2022
0.700
V5-
yd
0.669
p² 0
0.608
Reachable Maleculls Reurnoll Viln
60-
800
08
25-
000
0.5-
4-
60
0.1
255
200
02
00
250
500
190
011
1.0
1000
Unreachatle Molccules
1 = 0.283
= 0.041
12
0.125
10-
/ = 0.036
800
6-
0.8
0,6
0-
400
500
10
0,0
0-
Original SA Score
-10
Original LogP
0
10
Original Mabcular Weight
250
500
730
1000
0.0
Origial QED
0.6
1.0
Figure 4: Correlation between properties of Marger and Mproduet molecules.
shows an example where our model successfully reconstructs a molecule dissimilar to all molecules in the training set. We also assessed the average similarity between target and product molecule pairs, KL divergence (Brown et al. 2019), and Fréchet ChemNet (FC) distance (Preuer et al.) 2018) between the targets and recovered products. See Appendix F for additional synthesis planning results.
Among the four networks, the most significant error comes from the first reactant network, fui. with validation accuracy of only 30.8% after k-NN retrieval. To compare, frz reaches a validation accuracy of 70.0% without masking out invalid actions. During sampling, we mask out candidate second reactants that are incompatible with the selected reaction template achieve much higher accuracy. The action and reaction networks reach > 99% and 85.8% validation accuracy, respectively.
4.3 SYNTHESIZABLE ANALOG RECOMMENDATION RESULTS
We observed that in the cases of unrecoverable molecules, the final products could serve synthe- sizable structural analogs under the constraints of R. and (see Figure 3 for an example). The metrics in Table [ show how the product molecules in the unrecovered cases are still structurally similar to the input molecules. We illustrate the correlation between the properties of target and product molecules in Figure 4 We investigated the SA Score (Ertl & Schuffenhauer 2009b), QED, CLogP, and molecular weight, and observed that most product properties have a positive correlation with the corresponding input properties. The least successful case is QED due to its high sensitivity to structural changes as quantified by the structure- acttivurr-attvvtyy landscape index (SALI) (Guha (012) (Table in Appendix Gi. Overall, our model can suggest reasonable synthesizable analogs for target molecules, especially when the desired property is highly correlated with molecular structure.
In other cases where the target product not recovered by the generated pathway, the output synthetic tree may still provide inspiration for the synthesis of target molecules. Figure 3C highlights an example where the failure of reconstruction is due to the binary Morgan fingerprint's inability to distinguish repeating units. Our model successfully constructed one side of this symmetric molecule. ceaprroylpiperazziin (the
In this case, a synthetic chemist would likely recognize that replacing reactant added in the second step) with piperazine may lead to synthesis of the target molecule.
4.4 SYNTHESIZABLE MOLECULAR OPTIMIZATION RESULTS
To assess the optimization ability of our algorithm, we first consider common heuristic oracle functions relevant to bioactivity and drug discovery (Tables (aalle=]and]] [nd Note that the baseline methods we compare to do not constrain synthesizability, which means they explore a larger chemical space and are able to obtain molecules that score higher but are not synthesizable. The results show that our model consistently outperforms GCPN You et al.] 2018) and MolDQN (Zhou et al. 2019). and is comparable to GA+D (Nigam et al., 2019) and MARS (Xie et al., 2021) across different tasks. We highlight the case of GSK33 inhibitor optimization in Figure 5 In this task, our model
proposes molecule scored marginally worse than DST (Fu et al. 2021) and MARS, but much
8

---

Published as a conference paper at ICLR 2022
A. Result of optimizing GSK3$
H5
model GSK3B
Top-1 GSK3B 0.95
Top-1 GSK3$
GSK3B 0.97
optimizing docking score
Known Inhibitor Vina score -8,96 kcal/mol
Top-1 from model Vina score -10.50 kcallmol
Second from our model Vina score -9.31 koaklmol
Third from our model Vina -8.25 kcalmmol
Figure 5: Results of synthesizable molecular design. (A) A comparison between results of our model, DST, MARS, and GA+D on GSK3,3 bioactivity optimization. Our model proposes a highly scored molecule with much simpler structure than the other baselines. (B) The results of docking score optimization for M of SARS-Cov-2. Our model successfully proposes multiple molecules with stronger predicted binding affinity than known inhibitor.
simpler in structure. Indeed, this molecule can be accessed within one reaction step from purchasable compounds in C through simple Suzuki reaction (see Figure [2 for the synthetic pathway). This makes model's recommendation more immediately actionable, i.c., ready for experimental validation. As an additional quasi-realistic application to structure-based drug design, we optimized the binding affinity to DRD3 and M of SARS- 2ov--2
example target protein and successfully generated multiple molecules with improved docking scores relative to a known inhibitor (see Figure 5). We used the Guacamol filter (Brown et al. 2019) and SA_Scon (Ertl & Schuffenhaueri (2009a) to quantitatively evaluate the top- 10p-100 generated molecules against DRD3 and compared them to the TDC generative benchmark (Huang et 2021). Our method is the only one which achieved a high passing rate and low -c_or indicating a better quality of the structures (see Table 8). Additional molccular optimization results available Appendix
Table 2: Highest scores of generated molecules for various de novo molecular design tasks. Note that all baselines other than our method don't place constraints on synthetic accessibility.
JNK3
GSK3,3 2nd
QED
Method
1st
2nd
3rd
1st
3rd
1st
2nd
3rd
GCPN
0.57
0.56
0.54
0.57
0.56
0.56
0.948
0.947
0,946
MolDQN
0.64
0.63
0.63
0.54
0.53
0.53
0.948
0.948
0.948
GA+D
0.81
0.79
0.79
0.78
MARS
0.92
0.91
0.90
0.95
0.93
0.92
0.948
0.948
0,948
DST
0.97
0.97
0.97
0.95
0.95
0.95
0.947
0.946
0,946
Our Method
0.80
0.78
0.77
0.94
0.93
0.92
0.948
0.948
0.948
5 CONCLUSION
In this work, we have introduced amortized approach to conditional synthetic tree generation which can used for synthesis planning, synthesizable analog recommendation, and molecular design and optimization. Our model bridges molecular generation and synthesis planning by coupling them together into one rapid step, eliminating the need for two-stage pipelines of generation and filtering. We have demonstrated promising results for variety of molecular optimization tasks with relevance to drug discovery, illustrating how this approach can be used to effectively explore synthesizable chemical space in pursuit of new functional molecules. Additional outlook in Appendix C

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
The code repository is given in the supplementary material, including instructions in README file, all code used for data preprocessing, and all code used to train and evaluate the model. All the data we use are either publicly available or can be calculated by open-sourced software. Section 4.1 and Appendix I describe the experimental setup, implementation details, datasets used, and hardware configuration.
ACKNOWLEDGMENTS
This research was supported by the Office Naval Research under grant number N00014-21-1-2195. RM received additional funding support from the Machine Learning for Pharmaceutical Discovery and Synthesis consortium. We thank John Bradshaw for helpful discussions. We also thank Tianfan Fu, Samuel Goldman and Itai Levin for commenting on the manuscript.
CODE AND DATA AVAILABILITY
All code and relcasable data can be found at witttess//gttiu..cooc////eaeaaas Additional results can be found in the supporting information.
REFERENCES
John Bradshaw, Brooks Paige, Matt Kusner, Marwin HS Segler, and José Miguel Hemannez--Lbato. A model to search for synthesizable molecules. arXiv preprint 1eXiv:1906.0522: 2019.
John Bradshaw, Brooks Paige, Matt Kusner, Marwin HS Segler, and José Miguel Hernández-Lobato Barking up the right tree: an approach to search over molecule synthesis dags. arXiv preprint vrXiv:2012.11522 2020.
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for novo molecular design. Journal of Chemical Information and Modeling, 59(3): 1096-1108, 2019.
Alexander Button, Daniel Merk, Jan Hiss, and Gisbert Schneider. Automated de novo molecular design by hybrid machine intelligence and rule-driven chemical synthesis. Nature Machine Intelligence, 1(7):307-315, 2019.
Connor W Coley, William H Green, and Klavs F Jensen. Machine learning in computer-aided synthesis planning. Accounts of Chemical Research, 51(5):221--1289, 2018.
Connor W Coley, William H Green, and Klavs Jensen. RDChiral: An rdkit wrapper for han- dling stereochemistry in retrosynthetic template extraction and application. Journal of Chemical Information and Modeling, 59(6):2529 22327729992222, 2019a.
Connor W Coley, Wengong Jin, Luke Rogers, Timothy F Jamison, Tommi S Jaakkola, William H Green, Regina Barzilay, and Klavs Jensen. A graph-convolutional neural network model for the prediction of chemical reactivity. Chemical Science, 10(2):370 3777, 2019b.
Connor W Coley, Dale A Thomas, Justin AM Lummiss, Jonathan N Jaworski, Christopher P Breen, Victor Schultz, Travis Hart, Joshua Fishman, Luke Rogers, Hanyu Gao, al. A robotic platform for flow synthesis of organic compounds informed by ai planning. Science, 365(6453), 2019c.
Connor W Coley, Natalie Eyke, and Klavs Jensen. Autonomous discovery in the chemical sciences part i: Progress. Angewandte Chemie International Edition, 3:22858-22899 2020a.
Connor W Coley, Natalie Eyke, and Klavs Jensen. Autonomous discovery in the chemical sciences part i:: Outlook. Angewandte Chemie International Edition, 59(52):2344423448 2020b.
Elias James Corey and W Todd Wipke. Computer-assisted design of complex organic syntheses. Science, 166(3902):78--19, 1969.
10

---

Published as a conference paper at ICLR 2022
T. Cover and P. Hart. Nearest neighbor pattern classification. IEEE Transactions Information Theory, 13(1):21-27, 1967. doi: 10.1109/TT..1967.133966.
Hai Dai Nguyen and Koji Tsuda. A generative model for molecule generation based on chemical reaction trees. arXiv preprint arXiv:2106.03394, 2021.
Daniel C Elton, Zois Boukouvalas, Mark D Fuge, and Peter W Chung. Deep learning for molecular design-a review of the state of the art. Molecular Systems Design & Engineering, 4(4):828-84) 849, 2019.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based molecular complexity and fragment contributions. Journal of cheminfor- matics, 1(1):1-11, 2009a.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics, 1(1):1-11, 2009b.
Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W Coley, and Jimeng Sun. Differentiable scaffolding tree for molecular optimization. arXiv preprint arXhv:2109. 2021.
Wenhao Gao and Connor Coley. The synthesizability molecules proposed by generative models.
Journal of Chemical Information and Modeling, 60( 60(12):5714-572) 2020.
Samuel Genheden, Amol Thakkar, Veronika Chadimová, Jean-Louis Reymond, Ola Engkvist, and Eshen Bjerrum. Aizynthfinder: fast, robust and flexible open-source software for retrosynthetic planning. Journal of Cheminformatics, 12(1):1-9, 2020.
Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Looard Timothy D Hirzel,
Benjamín Sánchez-I Lengeling, Dennis Sheberla, Jorge Ryan P Adams, and Alán Espuru-Guzik, Automatic chemical design using data-driven continuous representation of molecules. ACS Central Science, 4(2):268 2766, 2018.
Sai Krishna Gottipati, Boris Sattarov, Sufeng Niu, Yashaswi Pathak, Haoran Wei, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, et al. Learning to navigate the synthetically accessible chemical space using reinforcement learning. In International Conference on Machine Learning, pp. 3668-3679 3679. PMLR, 2020.
Rajarshi Guha. Exploring atructure-actviit data using the landscape paradigm. Wiley Interdisci- plinary Reviews: Computational Molecular Science, 2(6))829-841, 2012.
Johannes Hachmann, Roberto Olivares-Amane Sule Atahan-Evrend Carlos Amador-Bedoll Roel Sánchez- Carreer, Arych Gold- Mar-Parker, Leslie Vogt, Anna M Brockway, and Alán Aspuru-Guzik. The harvard clean energy project: large-scale computational screening and design of organic photovoltaics on the world community grid. The Journal of Physical Chemistry Letters, 2(17): 2241-2251 2011.
Markus Hartenfeller, Martin Eberle, Peter Meier, Cristina Nieto- Pieto-Oberhuber, Karl-Heinz Altmann, Gisbert Schneider, Edgar Jacoby, and Steffen Renner. A collection of robust organic synthesis reactions for in silico molecule design. Journal of Chemical Information and Modeling, 512)) 3093-3098, 2011.
Markus Hartenfeller, Heiko Zettl, Miriam Walter, Matthias Rupp. Felix Reisen, Ewgenij Proschak, Sascha Weggen, Holger Stark, and Gisbert Schneider. Dogs: reaction-driven de novo design of bioactive compounds. PLoS computational biology, 8(2): 1 0023800, 2012.
Julien Horwood and Emmanuel Noutahi. Molecular design in synthetically accessible chemical space via deep reinforcement learning. ACS Omega, 5(5) ( 32984432944 2020.
Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovee, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitikk. Therapeutics data commons: machine learning datasets and tasks for therapeutics. arXiv preprint
2021.

---

Published as a conference paper at ICLR 2022
Jon Paul Janet, Sahasrajit Ramesh, Chenru Duan, and Heather J Kulik. Accurate multiobjective design in a space of millions of transition metal complexes with neural- efficient global optimization. ACS Central Science, 6(4):513-524, 2020.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning, pp. 2323-2332. PMLR, 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In International Conference on Machine Learning, pp. 4839-4848. PMLR, 2020.
Tomasz Klucznik, Barbara Mikulak- Mikulaa-KKucznik,, Michael P McCormack, Heather Lima, Sara Szymkuć, Manishabrata Bhowmick, Karol Molga, Yubai Zhou, Lindsey Rickershauser, Ewa P Gajewska, et al. Efficient syntheses of diverse, medicinally relevant targets planned by computer and executed in the laboratory. Chem, 4(3):522-532, 2018.
Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, and Eric Xing. ChemBO: Bayesian optimization of small organic molecules with syn- thesizable recommendations. In International Conference Artificial Intelligence and Statistics, pp. 2393-3403. PMLR, 2020.
Greg Landrum. RDKit: Open-source cheminformatics. URL htttp://wwe www.t.ord.i.orgggg.
Jiankun Lyu, Sheng Wang, Trent E Balius, Isha Singh, Anat Levit, Yurii S Moroz, Matthew O'Meara, Tao Che, Enkhjargal Algaa, Kateryna Tolmachova, al. Ultra-large library docking for discovering new chemotypes. Nature, 566(7743) 2019.
Barbara Mikulak-Klueznik, Patrycja Golebiowska, Alison A Bayly, Oskar Popik, Tomasz Klucznik, Sara Szymkuć, Ewa P Gajewska, Piotr Dittwald, Olga Staszewska- Staszewska-Krjoowskka Wiktor Beker, et al. Computational planning of the synthesis of complex natural products. Nature, 588(7836):83-88) 2020.
Karol Molga, Ewa P Gajewska, Sara Szymkuć, and Bartosz A Grzybowski. The logic of translating chemical knowledge into machine-processalee forms: modern playground for physical-organic chemistry. Reaction Chemistry Engineering, 1005-150-,, 2019.
AkshatKumar Nigam, Pascal Friederich, Mario Krenn, and Alán Aspuru-Guzik. Augmenting genetic algorithms with deep neural networks for exploring the chemical space. arXiv preprint arXiv: 1655, 2019.
Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, and Günter Klambauer. Fréchet chemnet distance: metric for generative models for molecules in drug discovery. Journal of Chemical Information and Modeling, 1839)1173-1174,, 2018.
Sereina Riniker and Gregory Landrum. Better informed distance geometry: using what we know to improve conformation generation. Journal of Chemical Information and Modeling, 55(12): 2562-2574, 2015.
Benjamin Sanchez- Lengelinnnnggiig and Alán Aspuru-Guzik. Inverse molecular design using machine learning: Generative models for matter engineering. Science, 361(64000:360-666,, 2018.
Roger Sayle and Daniel Lowe. Nextmove software. URL /Rtp.//pp////wo..atatttttteeeee. com/ nwweemamexx...mm html
Gisbert Schneider and David E Clark. Automated de novo drug design: are nearly there yet? Angewandte Chemie International Edition, 58732):10792--1000, 2019.
Daniel Schwalbe-Kode and Rafael Bombez-Bombarlli.. Generative models for automatic chemical design. In Machine Learning Meets Quantum Physics, pp. 445-467. Springer, 2020.
Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A Hunter, Costas Bekas, and Alpha A Lee. Molecular transformer: a model for uncertainty-calle callbratte chemical reaction prediction. ACS Central Science, 5()):152-1588,, 2019. 2019.
12

---

Published as a conference paper at ICLR 2022
Philippe Schwaller, Riccardo Petraglia, Valerio Zullo, Vishnu H Nair, Rico Andreas Haeuselmann, Riccardo Pisoni, Costas Bekas, Anna Iuliano, and Teodoro Laino. Predicting retrosynthetic pathways using transformer-basee models and hyper-graph exploration strategy. Chemical Science, 1 11(12)3311-3325, 2020.
Marwin HS Segler, Mike Preuss, and Mark P Waller. Planning chemical syntheses with deep neural networks and symbolic ai. Nature, 555(7698):604 610, 2018.
Teague Sterling and John J Irwin. Zinc 15 ligand discovery for everyone. Journal of Chemical Information and Modeling, 55(11):2324-2337 2015.
Sara Szymkuć, Ewa P Gajewska, Tomasz Klueznik, Karol Molga, Piotr Dittwald, Michat Startek, Michat Bajczyk, and Bartosz A Grzybowski. Computer-assisted synthetic planning: the end the beginning. Angewandte Chemie International Edition, 55(20):5904 5937, 2016.
Oleg Trott and Arthur Olson. Autodock vina: improving the speed and accuracy of docking with new scoring function, efficient optimization, and multithreading. Journal of Computational Chemistry, (1(2):555-5 461, 2010.
Quentin Vanhaelen, Yen-Chu Lin, and Alex Zhavoronkov. The advent of generative chemistry. ACS Medicinal Chemistry Letters, 1168):149611555, 2020.
H Maarten Vinkers, Mare R de Jonge, Frederik FD Daeyaert, Jan Heeres. Lucien MH Koymans, Joop H van Lenthe, Paul Lewi, Henk Timmerman, Koen Van Aken, and Paul AJ Janssen. Synopsis: synthesize and optimize system in silico. Journal of medicinal chemistry, 46(13): 2765-2773, 2003.
Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint arXiv:2103 10432, 2021.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826. 2018.
Zhenpeng Yao, Benjamín Sanchez-Lengeling, N Scott Bobbitt, Benjamin J Bucior, Sai Govind Hari Kumar, Sean P Collins, Thomas Burns, Tom K Woo, Omar K Farha, Randall Q Snurr, et al. Inverse design of nanoporous crystalline reticular materials with deep generative models. Nature Machine Intelligence, 3(1):76-88) 86766686622 2021.
Jiaxuan You, Bowen Liu, Rex Ying. Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. arXiv preprint arXiv: 02473, 2018.
Chun-Hui Zhang, Elizabeth Stone, Maya Deshmukh, Joseph A Ippolito, Mohammad M Ghahre- manpour, Julian Tirado-Rive Krasimir A Spasov, Shuo Zhang, Yuka Takeo, Shalley N Kudalkar, et al. Potent noncovalent inhibitors of the main protease of sars-cov-2 from molecular sculpting of the drug perampanel guided by free energy perturbation calculations. ACS Central Science, 7(3): 467-475, 2021.
Alex Zhavoronkov, Yan A Ivanenkov, Alex Aliper, Mark S Veselov, Vladimir A Aladinskiy, Anas- tasiya V Aladinskaya, Victor Terentiev, Daniil Polykovskiy, Maksim Kuznetsov, Arip Asadulaev, et al. Deep learning enables rapid identification of potent DDRI kinase inhibitors. Nature Biotechnology, 1038-1004 2019.
Zhenpeng Zhou, Steven Kearnes, Li Li, Richard Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific Reports, 9()):1-10, 2019.
Julie B Zimmerman, Paul T Anastas, Hanno Erythropel, chemistry future. Science, 367 2020.
C and Walter Leitner. Designing for a green
13

---

Published as a conference paper at ICLR 2022
APPENDIX
A SYNTHETIC TREE AND REACTION TEMPLATE
A. Byyythhlic Thee
Reaction SMARTS
scample)
Bi-molecatar reaction tamplate
Resction SMARTS
CO Mew.CC/ Ooootee CARO-N-CVEORA led the Luuaiii MacEOOI DOSECCCCII
CASCCCCC -CIjEM dage
tMo MIS CMO NADE
NICD
1036 dat NO00t OSSCH XuC qut
C.44 Naagu, S/CH
Product Molecule
Bulding
Reaction BMIL (one moothed exampl))
C-0X00-G 0-00w007+00000
Figure 6: Illustration of synthetic pathway synthetic tree (A) and reaction templates (B & C). (A) is the synthetic tree of remdesivir, a drug authorized for emergency use to treat COVID-19. Different color box labels indicate different types of chemical nodes. (B) and (C) are examples of reaction templates for uni- and bi-molecular reactions, where SMARTS is specific syntax for encoding reaction transforms.
14

---

Published as a conference paper at ICLR 2022
B ILLUSTRATION OF GENETIC ALGORITHM
Decoding
Crossover and Mutation
Selection
Figure 7: Illustration of the genetic algorithm used optimize the molecules. We the conditional synthetic tree generator as decoder to obtain molecules corresponding to input fingerprints. We crossover and mutate on the pool of fingerprints optimize the molecule implicitly.
15

---

Published as a conference paper at ICLR 2022
C
OUTLOOK
While our model shows promising results on the recovery of molecules from generated trees and de novo molecular optimization performance, there remain multiple challenges be addressed:
Initial Reactant Selection Our results show that the first reactant selection is the primary bottleneck target molecule recovery. We consider two major reasons for this: (1) the input to fri is usually the target molecule only, with no action mask applied. The large action space (i.e., all purchasable molecules) and the limited imput information thus make the problem the most difficult of the four tasks; (2) during the generation of the synthetic trees, we adopt depth-first approach that implicitly introduces a canonical ordering during reactant selection. The implicit ordering arbitrary and might harm the training of the network. Allowing the model to predict any valid order or including multiple orders as form of data augmentation may improve performance. Further, it may possible to first select the action (template) to constrain the space of compatible first reactants, which may improve performance given that this a smaller search space.
Reaction Templates and Purchasable Compound Selection The definition of synthesizable used in this work is that the molecule can be reached with a synthetic pathway only containing starting materials belonging
list of purchasable compounds and reactions that follow list of reaction templates. Here, we use only 91 templates, which pales in comparison to the 1,000 reaction families defined by NameRxn (Sayle & Lowe) the tens of thousands of templates in the expert CASP program SYNTHIA (Szymkué et al. 2016). and the hundreds of thousands in the data-driven CASP program ASKCOS (Coley et al. 2019c). Using more comprehensive the chemical space our model can explore. Additional constraints on template applicability could improve the feasibility of pathways, e.g., to mitigate selectivity concerns (cf. Figure 12
of reaction templates would enlarge
Molecular Representation and Molecular Similarity Certain results (e.g.. Figure 3C) reveal limitation of using boolean Morgan fingerprints. this type of fingerprint only accounts for the presence or absence of specific substructures, it cannot distinguish molecules with different numbers of repeated units or other symmetries. Applying a count or summation-based representation, coupled with development of similarity measurement based on that representation, would solve this problem.
Beam Search for Decoding Besides tackling the aforementioned challenges, in future work we also plan to implement beam search with our networks to enhance the model performance for bottom-up synthesis planning. Improving the sample efficiency of the synthesizable molccular design algorithm (the GA module), and applying it to more high-fidelity computational oracles would also be of interest. Those advances could enable faster and more accurate synthesis planning, as well as an even better tool for de novo molecular design.
16

---

Published as a conference paper at ICLR 2022
D ALGORITHM
Algorithm Synthetic tree generation
I: Input: List of reaction templates R, List of building blocks C, input molecule Murget. molecular encoder 2M = E an action network MLPact a reactantl selection network MLPRI, reaction network MLP rx. and reactant2 selection network MLP
2: Output: A rooted binary tree T, representing a synthesis pathway.
3: Encode target molecule, Purget <- E (MMugrr)
4: Initialize state S &- 2, tree T +- 2, Mmestrevent < None
5: for = 1, 2, ... .mar do
6: Predict and sample action type:
Oget ~ Pact (MI.Pacr @ Frarget))
7: Predict first reactant, Ard MLP (Estate Stargot)
8: if Oaa end then
9:
break
10:
else if dwa = add then
MAI k: (cmmm
12:
Znl <- E(Mri)
else
14:
Mai +- -Mosst rooent
2m1 t EEMamoo rrecent)
16:
end if
Predict and sample reaction template:
Grxn Prxn f- oMLPP (Zstate + Zurget O Zrtl) )) if Orxn bi-molecular then
18:
19:
if Oast merge then
20:
Ma2 + S/Ma 2n2 F(Ma))
21:
22:
else
23:
Predict second reactant, ax2 MLP m2 Ztarget 2n1 (pxn)
24:
Ma + k NN (amz)
25:
2n1 E(Mx)
26:
end if
end if
28:
rxn rxn_tem= 4- R[arxn]
29:
Run reaction Mproduet +- rxn_tem( (Mail (s Maa))
30:
Update T,, S and M +- Mproduct
31:
end for
17

---

Published as a conference paper at ICLR 2022
E ADDITIONAL EXPERIMENTAL DETAILS
Network Setup For all experiments reported in this paper, the four networks, fact, Sal, S and fnz, use fully connected layers with 1000, 1200, 3000, and 3000 neurons in the hidden layers, respectively, Batch normalization is applied to all hidden layers before ReLU activation. In faa and fra softmax applied after the last layer and cross entropy loss used, while fai and fnz use linear activation in the last layer and mean squared error (MSE) loss. We use the Adam optimizer to train all networks with a learning rate of le-4 and mini-batch size of 64.
Training Each MLP is trained as separate supervised learning problem using subset of information from the known synthetic routes. For instance, frzn a classification network which learns to select a discrete action given information on the current state of the tree, the target molecule, and the first reactant. Similarly, fact a classification network which learns to select the correct action type given the state of the tree. On the other hand, frei and frr learn embeddings (regression) for the first and second reactant candidates, respectively, followed by a nearest- neighbors search from and C².
applied random policy to generate the synthetic trees. We randomly sampled purchasable reactants, and randomly applied matching reaction templates them. Doing so, obtained 550k synthetic trees and filtered by the QED the product molecules (QED > 0.5) as well as randomly with a probability QED/0.5 increase their drug likeness, crude sense. Ultimately, we obtained 208,644 synthetic trees for training, 69,548 trees for validation, and 69,548 trees for testing after a random split.
Dataset Preparation Following the procedure described in Section 3.2
Docking Procedure We downloaded the crystal structure of the M of SARS-Cov-2 virus with PDB ID: 71.1 1. We removed the water molecules and ions from the file, and estimated the docking box based on the docked pose with a reported inhibitor, compound 5 in Zhang al. (202I). For each ligand generated by our method, we use RDKit to generate molecular conformations (Landrum Riniker & Landrum 2015) and perform docking simulations using AutoDock Vina (Trott & Olson 2010). We set exhaustiveness as 8 during the generation and recorded the high-scored conformations for a rescoring with exhaustiveness as 32. The reported values and rank are based on the rescoring.
Hardware Models were trained on node with double Intel Xeon Gold 6230 20-core 2. 2Z processors, 512 GB DDR4 RAM, and eight RTX 2080Ti graphics cards with 11GB VRAM. Data preprocessing and predictions were made CPU node with 512 GB RAM and two AMD EPYC 7702 64-core 2GHz processors.
18

---

Published as a conference paper at ICLR 2022
F ADDITIONAL RESULTS ON SYNTHESIS PLANNING
Table 3: Results of synthesis planning for molecules from training, validation, and test datasets. Recovery indicates the fraction of final product molecules which match the input target molecules. Average similarity measures the Tanimoto similarity between the fingerprints of the product molecules and target molecules.
Dataset
Recovery Rate
Average Similarity
Average Similarity (Unrecovered))
Train.
92.1%
0.975
0.688
Valid
51.4%
0.761
0.508
Test
51.0%
0.759
0,508
ChEMBL 4.50%
0,423
0.396
Table 4: Analysis of the generated molecules in unrecovered cases. We compare with baseline generative methods to evaluate the similarity of input and output molecules. Baseline data from Guacamol (Brown et al. 2019)
Validity
Uniqueness?
Novelty
KL Divergencef
FCD1
Random Sample
1.000
0.997
0.000
0.998
0,929
SMILES LSTM AAE VAE
0.959
1.000
0.912
0.991
0.913
0.882
1.000
0.998
0.886
0.526
0.870
0.999
0.974
0.982
0.863
Our Model (Reachable)
0.999
1.000
0.920
Our Model (Unreachable)
1.000
1.000
1.000
0.684
19

---

Published as a conference paper at ICLR 2022
G
ADDITIONAL RESULTS ON ANALOGS RECOMMENDATION
,
0.283
2 = 0.041
Recovered Value
Recovered Value
5.0
2,5
4-
3-
2.5
2-
-5.0
55
Original SA Score
Original LogP
1000
22
0.125
22
0.036
Recovered Value
800
Recovered Value
0.8
600-
0.6
400
0.4
0.2
200
0.0
0.0
0.2
0.4
0.6
0.8
200
400
600
800
1000
Original QED
Original Molecular Weight
Figure 8: Correlation between input and output values on ChEMBL molecules.
0.700
7.5
0.669
Recovered Value
Recovered Value
5-
25-
4-
0.0
-5.0
1
Original SA Score
Original LogP
1000
2
0.647
1.0
0.608
Recovered Value
800-
Recovered Value
0.8
600
0.6
400
0.4
0.2
200
0-
400
1000
0.0
0.2
Original QED
0.4
0.6
0.8
1.0
Original Molecular Weight
Figure 9: Correlation between input and output values test set molecules.
20

---

Published as a conference paper at ICLR 2022
0.703
7.5
0.669
Recovered Value
Recovered Value
4-
0.0
3-
2.5
1
-5
Original SA Score
Original LogP
1000
0.652
1.0
0.612
Recovered Value
800
Recovered Value
0.8
600
0.6
400
0.4
0.2
200
0.0
0-
0.2
0.4
0.6
0.8
1.0
200
800
1000
Original QED
Original Molecular Weight
Figure 10: Correlation between input and output values on validation set molecules.
2
0.976
0.947
Recovered Value
Recovered Value
5.0
2,-
255
2-
-5.0
-5
Original SA Score
Original LogP
1000
2
0.973
0.975
Recovered Value
800
Recovered Value
0.8
600
0.6
400
0.4
200
0.0
200
400
600
800
1000
0.0
0.2
Original QED
0.4
0.6
0.8
1.0
Original Molecular Weight
Figure 11: Correlation between input and output values on training set molecules.
1d. d/l/range
SALI
(2)
N
sim(i.j)
(b,1)
21

---

Published as a conference paper at ICLR 2022
Table 5: Results of synthesis planning for unrecovered cases in "reachable" and "unreachable" molecules. Metrics other than recovery rate are measured for unrecovered molecules only.
Dutaset
N
Averuge Similarity
KL Divergencet
FC Distance
Reachable (test set)
69,548
0.508
0.315 1.000
Unreachable (ChEMBLI)
20,000
0.396
1.000
2.140
Table 6: SALI of oracle functions we investigated. higher SALI value means that a small structure change can lead to larger property change, such that the function is more sensitive to detailed structural changes.
SA Score
QED
LogP
Mol. Weight
Reachable
0.139
0.368
0.053
0.127
Unreachable 0.115
0.383
0.013
0.043
H ADDITIONAL RESULTS ON MOLECULAR DESIGN
Table 7: Results of synthesizable molecular optimization on common oracle functions. Seeds are randomly sampled from the ZINC database (Sterling & Irwin 2015). Top-n is the average value for top n molecules. "Seeds" refers the mean scores of the initial mating pool we sampled and "Outputs" refers to the mean scores of the 128 generated molecules.
Best from seeds
Top-1
Top-10
Top-100
Seeds
Outputs
QED LogP
0.947
0.948
0.948
0.947
0.673:1.2899
0.946110:001
23.96
1.09-:35.18
23.72-10.69
JNK3
0.120
0.800
0.758
0.719
0.082::0.025
0.715
GSK3,
0.310
0.940
0.907
0.815
0.050-10.051
0.803
DRD2
0.181
1.000
1,000
0.998
0.007-1:0018
0.906-10.003
22

---

Published as a conference paper at ICLR 2022
Table 8: Leaderboard on TDC DRD3 docking benchmark using ZINC and Docking. Mean and standard deviation across three runs are reported. Arrows (t. 4) indicate the direction better performance. The best method is bolded and the second best is underlined. Note in particular the low SA_Score and high % Pass, which are heuristics for synthetic complexity and drug likeness/quality.
Method Category
Domain Domain--pecific Methoos
State-of-the-AM Methods in ML
Ours
Rest-in-data
Screening
Graph-GA
MoliXQN
SynNer
Top (U)
-12,800
-10. .0
-14811 :441
13.017
710.045..074
-8. -8.236.nar
195009
-11.135
Top10
-11.483-0000
=15.930=9.16
-11.483=0.11
-12,020
Topl (1)
-12.800
-12. 112
16.533:
14.533
-12. 12.300=au993
9.9900
-1.4433a4400
12.300
Novelty
Divensity (1)
0.864
5000
0.626_joo
1.00000000
0.740
1.000s0
0.922 exvc
0.893 owes
0.873_com
0.821
&&Pass (1)
0.780
0.683-turt
0.393
0.257aamee
0.167=ama
0.023=8m12
0.527=00er
0.800
Top Pass (1)
-11.00
10. La0
-14.267 ma.cm
12.533
-9.367 100300
-7 19001
-9.000
-12,300
SA_So
3.096=0014
6.843
6868
2.801
Topel forn rede, JN13 0.06
Top fore Dett 2M63 888
farm MARS
Tope torn DA40 J813-631
Propoaed Fntwwyy
forn model 0.23
Top-l 66423
Tepl form MARS, G503 055
Tep- farm 6DX33
Proposed Syrthelic Patrway
O-C+
Knewn bestster, Vina scom- -000 bestinet
Top-1 thon model Visa scor 10.0 bealinet
Baoeee Vine
4-21 baskret
Wia
Figure 12: The optimized structures and their corresponding synthetic pathways for various inhibitor design tasks. The third row optimizing docking score against the M of SARS-Cov-s
23

---

Published as a conference paper at ICLR 2022
I
HYPERPARAMETER TUNING
Action Network
Reactant1 Network
Fp_ Fp_2_1024
_p_3_2048
Fp_2_1024
TP_3_2048
Fp_2_2048
Fp_3_4096
Fp_2_2048
Fp_3_4096
Fp Fp_2_4096
_p_3_8192
Fp_2_
[p_3_8192
5e33
0.98
4e:3
0.96
3e-3
0.94
2e-3
1e3
0.92
Step
0.9
Step
0
1M
2M
3M
4M
5M
6M
7M
8M
400k
800k
1.2M
1.6M
2M
Reaction Network
Reactant2 Network
jp_2_1024
Fp_3_2048
Fp_2_1024 = Fp_3_2048
Fp_2_2048
Fp_3_4096
Fp_2_2048
Fp_3_4096
Fp_2_4096
Fp_3_ 3_99
Fp_2_4096
- 8192
0.22
0.7
0.2
0.66
0.18
0.62
0.58
0.16
0.54
0.14
Step
0.5
Step
500k 1.5M 2.5M 3M 3.5M
0
400k
800k
1.2M
1.6M
2M
The validation loss during training with different radius and number of bits network input. The validation loss is 1 -cccuracy where the accuracies for the reactant networks are the accuracies of the k-NN searches (k
Figure 13:
Besides Morgan fingerprints, other molecular representations explored during hyperparameter tuning included Graph Isomorphism Network (GIN) (Xu et al. 2018) embeddings and RDKit 2D descriptors (Figure 14). Morgan fingerprints of length 256 and radius 2 were found to work best.
24

---

Published as a conference paper at ICLR 2022
Reactant1 Network
Reactant2 Network
GIN
Fp_4096
GIN
Fp_4096
Fp_256
RDKit2D
Fp_256
RDKit2D
0.95
0.6
0.9
0.85
0.5
0.4
0.75
0.7
0.3
Step
Step
500k
1M
1.5M
2.5M
3M
500k
1.5M
2.5M
3M
Figure 14: The validation loss during training using different action embeddings conduct the k-NN search. The validation loss is 1-accuracy, where the accuracies for the reactant networks are the accuracies of the k-NN searches (k- 1).
25


Reviews:
[{"review_id": "pi8AV175_Zh", "reviewer": "Reviewer_GkiB", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nThis is a well written paper that presents a sound, original idea.\nI'm amazed that nobody had tested this idea before and I'm a little\nsurprised that it works as well as it did on a first try.\n[EDIT: as reviewer \"vmGs\" pointed out, this idea is very closely \nrelated to the NeurIPS2020 paper by Bradshaw et al, which I didn't \nrealize at the time of writing the rest of this review.  I will reassess\nthis work after any revisions, and have temporarily reduced\nmy previously enthusiastic score. EDIT2: the revisions addressed my concerns.]\n\nThe methodology exposition is clear. Perhaps an appendix with a\nsummary of the input/output dimensions of each trainable module would\nhelp organize the scattered information about the sizes and types of\nthe various inputs and outputs and give a quick overview of the full\nmethod (this could be as simple as an annotated version of Figure 2\nwith example dimensions from the best model.) Did the authors try to\ninput to the k-NN modules a (possibly pretrained) embedding of the\nreagent graphs instead of the 256 Morgan Fingerprints of radius 2? And\ndid those Morgan fingerprints they used include the reagent chirality\ninformation or not?\n\nThe results of the baseline models in Figure 5 and Figure 11 are at\nbest embarrassing for those other codes, and at worst some\nmisconfiguration of those codes. I imagine that the codes try hard to\ndesign against a target fingerprint-based input function and just\nthrow the kitchen sink until most of the bits of the produced\nmolecules match something that improves the score in the target\nfunction. Out of curiosity, did the authors of this paper try to see\nif there were any \"reasonable\" molecules produced by the other codes\nwith score better or equal to that of their own model. In any case,\nthe message that the authors want to convey with the current examples\nis clear, but it may be helpful to contact the authors of those other\npapers as a fair warning.\n\nThe paper demonstrates the basic concept behind this work. I was left\nwondering why the authors limited the selection of possible reaction\ntemplates to such a small set---is there a practical reason related to\nthe performance of the model in terms of training or inference time?\nFurthermore, why did the authors only limit the present work to k=1?\nI appreciate the simplicity of documenting the greedy (k=1) selections\nfor the results in table 1, but I'm left wondering how much better\ncould these results become if one included a larger number of\ncandidates at each step and searched all relevant trees.  \n\nAlthough I commend the authors on including source code with the\nsubmission, the reproducibility of this work is currently\nunclear. Unfortunately, the supplemental code was not usable, so I\ncouldn't check any of the claims of performance. The README file\nappears to be incomplete and has a couple of TODO notes in it; the\ntest subdirectory seems to be missing and thus the instruction to run\nthe unittests doesn't run any unittest at all; the code has references\nto undeclared paths (DATAPATH, HOMEPATH, st_pis); references to data\nfiles that don't exist (enamine_us.csv.gz).  I imagine that some of\nthese problems are easy to fix with a couple additional instructions\nand it is possible that the rest could be resolved with a couple extra\nscripts and some minor modifications.  Nevertheless, I felt a bit\ndisappointed that the authors missed an opportunity to submit a\nremarkable work with a reproducible demonstration. I'd suggest that\nthe authors contact Enamine before the conference and request\npermission to publish a snapshot of the current set of building blocks\nor any other data that the authors used from Enamine, so that future\npapers can compare against a benchmark using the same training and\ntesting data to document any algorithmic improvements in a bottom-up\napproach or compare such approaches to retrosynthetic pathway\ngeneration that target the same building blocks.\n\nDespite these concerns on reproducibility, I think that this is a\nrefreshing paper of the type that I expect to see highlighted in the\nICLR conferences and I think that it opens up new directions for\nmethodological improvements to molecule optimization.\n", "summary_of_the_paper": "\nThis paper proposes a bottom-up approach to molecular synthesis\nplanning and synthesizable molecular design. The authors cast the\nsynthesis planning problem as a Markov Decision Process with trainable\nnetworks for selecting next actions.  The output of the approach is a\nbinary tree that documents reaction and the combination of building\nblocks or previously synthesized reagents that would mix to create a\nnew intermediate or a final molecule.  When the approach fails to\nproduce the correct target molecules it typically generates a similar\nmolecule---the authors take advantage of this feature of their\napproach and optimize their algorithm to design synthesizable\nmolecules with desirable properties.\n", "summary_of_the_review": "\n~This is a refreshingly original paper on molecular synthesis planning\nand synthesizable molecular design.  It is a strong, foundational\npaper that is well written. The authors could do some a little\nwork to improve the reproducibility and enable future benchmarks\nagainst the same data.~\n\n[Edit: this work closely relates to Bradshaw et al, but the relationship\nwas not made clear in the exposition.  Until the authors \naddress this concern and clarify their additional important contributions,\nI do not recommend publication of this paper.]\n\nEdit2: This is a strong paper that is well written and deserves publication at ICLR22.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7RxVX6nJSo2", "reviewer": "Reviewer_JJJC", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n### + Reasonable model of synthetic pathway\nThe proposed generation model enables us to use not only the purchasable molecules but also intermediate molecules to chemical synthesis whilst the model being simple enough.\n\n## Weaknesses\n### - The relationship between the proposed method and existing forward-synthesis-based methods is not clear\nI am not very satisfied with the discussion on the relationship to the existing methods. The introduction section does not mention the existing work using the same forward-synthesis idea to construct molecules, and the relationship is discussed only in Section 2.1. In addition, the existing synthesizable molecular design methods are not compared in the experiments. These are just examples of a lack of connection to the existing literature. I would suggest the authors to strongly relate the proposed method to the existing work and discuss what are the contributions of the present work.\n\n## Other suggestions\n- In Section 3.3, the agent uses $k$-NN to choose reactants, but it is not clear how to use $k$ neighbors to select one reactant. It is in fact discussed in Section 4.2, but I suggest to explain it in the method section.\n- Is it possible to select a reaction template first, instead of a reactant? The number of reaction templates is generally smaller than that of reactants, and it would be helpful to alleviate the initial reactant selection problem discussed in Section 5.", "summary_of_the_paper": "The present paper is concerned about a forward-synthesis approach to molecular optimization. The proposed method defines one action to halt the procedure and three actions to develop a synthetic pathway, each of which corresponds to using zero/one/two of the intermediate molecules for chemical reaction to yield the next intermediate molecule. This generation model is controlled by an agent that chooses a series of actions to transit to the next state, given an embedding vector. If applied to synthesis planning, the embedding of the target molecule is used as the embedding vector, while if applied to molecular optimization, the embedding vector is optimized by a genetic algorithm.\n\nThe effectiveness of the proposed method is demonstrated by two experiments. The first one examines the ability of synthesis planning using reachable and unreachable data sets. The second one examines the ability of molecular optimization. For both of the experiments, both quantitative and qualitative analyses are provided.", "summary_of_the_review": "I suggest to reject this paper and encourage the authors to revise it for next opportunities.\n\nWhile I consider the method proposed in this paper is reasonable, the relationship to the existing work is not discussed enough, and therefore, I consider the paper is not very mature to be published as an academic paper. I would suggest the authors to revise the paper so as to clarify the difference from the existing work and how much the difference contributes to improve the performance. To do so, additional experimental results would be necessary.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "63-TgIwSj7V", "reviewer": "Reviewer_1rvZ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The advantages of this paper are as followings:\n1. The idea of constructing a synthetic tree in a bottom-up manner to solve both problems of molecular design and synthesis planning simultaneously is interesting and eye-catching. The tree generation algorithm is succinct and efficient.\n2. Case studies(figure 5, figure 11) demonstrate the results in details.\n3. It’s nice to see hyperparameter tuning results.\n4. Codes are available.\n\nConcerns are as followings:\n1. It seems not very clear to me that how is genetic algorithm integrated into the process of molecular optimization and synthesis planning.\n2. It might be better to compare the recovery rate(shown in table 1) of the proposed method with other synthesis planning methods(i.e., retrosynthe- sis).\n3. It might be valuable to analyse more about why the proposed method is more computationally efficient.\n4. In table 2, it would be better to demonstrate quantitatively the synthetic accessibility of molecules generated by different methods.\n\nSome typos:\n1. The font size of table 6 is too large.\n", "summary_of_the_paper": "This paper aims to construct a synthetic tree in a bottom-up manner to solve both problems of molecular design and synthesis planning simultaneously. The root node represents the final generated molecule, and the leaf nodes represent the reactants that can be purchased. Starting from the bottom leaf nodes, the authors utilize reaction templates to generate the parent node until the root node is reached. To design molecules with desired properties, the authors utilize the genetic algorithm to guide the tree generation.", "summary_of_the_review": "Overall, the paper is well-written, and the method appears to be promising.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "5zEusLc58F", "reviewer": "Reviewer_vmGs", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths:\n- interesting extension of the work by Bradshaw, Gottipatti and Horwood, which addresses some of the limitations of these models.\n- first in-depth investigation of conditional synthesis dag generation with compelling results (this was studied on a very shallow level by Bradshaw et al)\n- interesting observations which contrast prior work: Using fingerprints instead of GNNs seems to be beneficial, and the use of a genetic algorithm on binary fingerprints seems to be at least competitive with optimisation in a continuous latent space.\n- reasonable experiments in line with prior work\n- good results in comparison with other, less restricted molecule search algorithms\n\n## Weaknesses:\n### Relation to prior work\nUpon reading the related work in more detail, it is clear that this work is an extension of the work by Bradshaw et al. 2020 NeurIPS, who reported 1) generation of synthesis trees (synthesis trees are a special case of DAGs) using a generative model for DAGs, and 2) conditional tree generation $p(T,x)$. Conditioned on an embedding of the desired product structure $x$, the generative model generate a synthesis tree $T$. This model is called \"RetroDOG\". Furthermore, the MDP setting has been explored by Horwood and Gottipatti.\n\nTo be more concrete: In section D.3 of the appendix https://papers.nips.cc/paper/2020/file/4cc05b35c2f937c5bd9e7d41d3686fff-Supplemental.pdf  of their Neurips paper, Bradshaw et al. write:\n`Our generative model of synthesis DAGs can be seen as a parametrizable mapping from a vector of real numbers to a synthesis DAG. As a module it can be mixed and matched in different ML frameworks as we have already seen in the main paper with DoG-AE and DoG-Gen. In this section we describe some preliminary results with a third model architecture, called RetroDoG. This model consists of the composition of a GNN followed by our generative synthesis DAG model to produce a learnable mapping from a molecular graph to a synthesis DAG. By training this model on pairs of product molecules and their associated synthesis DAGs we can use this model to perform retrosynthesis (i.e. predict how a particular product can be made). [...] the model described in this section would additionally allow one to feed in a potentially hard or impossible to synthesize molecule, and obtain a similar molecule which is easy to synthesize, which is impossible with current planners.\" [...] \"RetroDoG in contrast tries to construct the DAG in a **bottom-up** manner [...].  we believe such an approach as RetroDoG may be worthy of future research interest, and may for instance be useful in combination with more complex tools to **amortize** and reduce the cost of **searching for synthetic routes**.\"\n\n\nBuilding on other people's work is an essential part of science, but it needs to be acknowledged accordingly, and there is no harm in doing so. In fact, the bigger harm to the scientific community comes from not doing so.\n\nUnfortunately, Bradshaw et al. 2020, Gottipatti et al and Horwood et al are only cited in passing in the related work section, and not in the introduction, and there is not even a mention of the RetroDog model by Bradshaw. In the way the contributions of this paper here are presented, it leads to the impression that they were actually invented by the authors, which is not the case.\n\n`We formulate the tasks of multi-step synthesis planning and synthesizable molecular design as a single shared task of conditional synthetic tree generation. ` => this has already been done by Bradshaw et al as the RetroDog model\n\n`We formulate a Markov decision process to model the generation of synthetic trees, allowing the generation of multi-step and convergent synthetic pathways.` => Gottipatti & Horwood used the MDP framing already\n\n`We propose a model that is capable of (1) rapid bottom-up synthesis planning and (2) constrained molecular optimization that can explore a chemical space` => this is the RetroDog model by Bradshaw et al (who also explicitly did not restrict their model to the molecular transformer).\n\n\nThe paper here should thus be clearly presented as an investigation and extension of the work by Bradshaw, Gottipatti and Horwood, rather than novel modelling. This would not detract from its quality or value, which in my opinion is considerable.\n\nThis reviewer would also suggest to cite non-neural reaction-driven molecular design work, for example https://doi.org/10.1371/journal.pcbi.1002380 or https://doi.org/10.1021/jm030809x which predate generative models for molecules, and already point out the requirement of synthesizability. In fact, these algorithms have also been used in conjunction with fingerprint similarity to a target molecule as the objective, and would thus be a good baseline to explore in this work.\n\n\n### Modelling\nThe model the authors describe should be more correctly referred to as a generative model for synthesis DAGs. There is no MDP here, since there is no notion of reward.\n\nAlso, it can be argued that a synthesis tree does not really satisfy the Markov assumption. For example, one would never introduce a protecting group A, and then immediately deprotect A.\n\n### Experiments and Baselines\nMost if not all molecules that come from the baselines do not appear to be very drug-like. This raises the question whether the used benchmark functions are really realistic enough for drug discovery, or whether other objectives should be considered.\nFurthermore, are cLogP or QED really a quantity that is desirable as an objective for drug discovery on its own? Also, are docking scores really reliable enough to be used a quantitative oracle?\n\n\n", "summary_of_the_paper": "The authors explore an interesting extension of Bradshaw's generative model for synthesis trees, which leads to results for molecule optimisation and generation, which are competitive to less restricted models.", "summary_of_the_review": "Updated after Author response:\n\nI think the paper should now be accepted at ICLR. It is a significant step forward for the field, with a modeling approach interesting from the ML side as well as non-trivial, domain-relevant empirical validation (which many other papers on the topic at ML conferences do not have).\n\n\n\nnit: with regards to the baseline experiments side: I would suggest to move the QED task to the appendix.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: After much back and forth about prior work, 3 reviewers score this paper as an 8 and one scores it as a 3.
Other reviewers have written to the 3 and told them they believe that their review is now too harsh, in light of clarifications w.r.t. related work. I tend to agree, though I must admit that I am not an expert on this topic. 
Given that there is almost unanimous support for accepting and it's possible that the one hold-out has not seen some of the extra information, I recommend acceptance. 
Given the praise from the other three reviewers, I moreover recommend a spotlight.
Decision: Accept (Spotlight)