Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors explore an interesting extension of Bradshaw's generative model for synthesis trees, which leads to results for molecule optimisation and generation, which are competitive to less restricted models.
main_review: ## Strengths:
- interesting extension of the work by Bradshaw, Gottipatti and Horwood, which addresses some of the limitations of these models.
- first in-depth investigation of conditional synthesis dag generation with compelling results (this was studied on a very shallow level by Bradshaw et al)
- interesting observations which contrast prior work: Using fingerprints instead of GNNs seems to be beneficial, and the use of a genetic algorithm on binary fingerprints seems to be at least competitive with optimisation in a continuous latent space.
- reasonable experiments in line with prior work
- good results in comparison with other, less restricted molecule search algorithms

## Weaknesses:
### Relation to prior work
Upon reading the related work in more detail, it is clear that this work is an extension of the work by Bradshaw et al. 2020 NeurIPS, who reported 1) generation of synthesis trees (synthesis trees are a special case of DAGs) using a generative model for DAGs, and 2) conditional tree generation $p(T,x)$. Conditioned on an embedding of the desired product structure $x$, the generative model generate a synthesis tree $T$. This model is called "RetroDOG". Furthermore, the MDP setting has been explored by Horwood and Gottipatti.

To be more concrete: In section D.3 of the appendix https://papers.nips.cc/paper/2020/file/4cc05b35c2f937c5bd9e7d41d3686fff-Supplemental.pdf  of their Neurips paper, Bradshaw et al. write:
`Our generative model of synthesis DAGs can be seen as a parametrizable mapping from a vector of real numbers to a synthesis DAG. As a module it can be mixed and matched in different ML frameworks as we have already seen in the main paper with DoG-AE and DoG-Gen. In this section we describe some preliminary results with a third model architecture, called RetroDoG. This model consists of the composition of a GNN followed by our generative synthesis DAG model to produce a learnable mapping from a molecular graph to a synthesis DAG. By training this model on pairs of product molecules and their associated synthesis DAGs we can use this model to perform retrosynthesis (i.e. predict how a particular product can be made). [...] the model described in this section would additionally allow one to feed in a potentially hard or impossible to synthesize molecule, and obtain a similar molecule which is easy to synthesize, which is impossible with current planners." [...] "RetroDoG in contrast tries to construct the DAG in a **bottom-up** manner [...].  we believe such an approach as RetroDoG may be worthy of future research interest, and may for instance be useful in combination with more complex tools to **amortize** and reduce the cost of **searching for synthetic routes**."


Building on other people's work is an essential part of science, but it needs to be acknowledged accordingly, and there is no harm in doing so. In fact, the bigger harm to the scientific community comes from not doing so.

Unfortunately, Bradshaw et al. 2020, Gottipatti et al and Horwood et al are only cited in passing in the related work section, and not in the introduction, and there is not even a mention of the RetroDog model by Bradshaw. In the way the contributions of this paper here are presented, it leads to the impression that they were actually invented by the authors, which is not the case.

`We formulate the tasks of multi-step synthesis planning and synthesizable molecular design as a single shared task of conditional synthetic tree generation. ` => this has already been done by Bradshaw et al as the RetroDog model

`We formulate a Markov decision process to model the generation of synthetic trees, allowing the generation of multi-step and convergent synthetic pathways.` => Gottipatti & Horwood used the MDP framing already

`We propose a model that is capable of (1) rapid bottom-up synthesis planning and (2) constrained molecular optimization that can explore a chemical space` => this is the RetroDog model by Bradshaw et al (who also explicitly did not restrict their model to the molecular transformer).


The paper here should thus be clearly presented as an investigation and extension of the work by Bradshaw, Gottipatti and Horwood, rather than novel modelling. This would not detract from its quality or value, which in my opinion is considerable.

This reviewer would also suggest to cite non-neural reaction-driven molecular design work, for example https://doi.org/10.1371/journal.pcbi.1002380 or https://doi.org/10.1021/jm030809x which predate generative models for molecules, and already point out the requirement of synthesizability. In fact, these algorithms have also been used in conjunction with fingerprint similarity to a target molecule as the objective, and would thus be a good baseline to explore in this work.


### Modelling
The model the authors describe should be more correctly referred to as a generative model for synthesis DAGs. There is no MDP here, since there is no notion of reward.

Also, it can be argued that a synthesis tree does not really satisfy the Markov assumption. For example, one would never introduce a protecting group A, and then immediately deprotect A.

### Experiments and Baselines
Most if not all molecules that come from the baselines do not appear to be very drug-like. This raises the question whether the used benchmark functions are really realistic enough for drug discovery, or whether other objectives should be considered.
Furthermore, are cLogP or QED really a quantity that is desirable as an objective for drug discovery on its own? Also, are docking scores really reliable enough to be used a quantitative oracle?



summary_of_the_review: Updated after Author response:

I think the paper should now be accepted at ICLR. It is a significant step forward for the field, with a modeling approach interesting from the ML side as well as non-trivial, domain-relevant empirical validation (which many other papers on the topic at ML conferences do not have).



nit: with regards to the baseline experiments side: I would suggest to move the QED task to the appendix.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: 
This paper proposes a bottom-up approach to molecular synthesis
planning and synthesizable molecular design. The authors cast the
synthesis planning problem as a Markov Decision Process with trainable
networks for selecting next actions.  The output of the approach is a
binary tree that documents reaction and the combination of building
blocks or previously synthesized reagents that would mix to create a
new intermediate or a final molecule.  When the approach fails to
produce the correct target molecules it typically generates a similar
molecule---the authors take advantage of this feature of their
approach and optimize their algorithm to design synthesizable
molecules with desirable properties.

main_review: 
This is a well written paper that presents a sound, original idea.
I'm amazed that nobody had tested this idea before and I'm a little
surprised that it works as well as it did on a first try.
[EDIT: as reviewer "vmGs" pointed out, this idea is very closely 
related to the NeurIPS2020 paper by Bradshaw et al, which I didn't 
realize at the time of writing the rest of this review.  I will reassess
this work after any revisions, and have temporarily reduced
my previously enthusiastic score. EDIT2: the revisions addressed my concerns.]

The methodology exposition is clear. Perhaps an appendix with a
summary of the input/output dimensions of each trainable module would
help organize the scattered information about the sizes and types of
the various inputs and outputs and give a quick overview of the full
method (this could be as simple as an annotated version of Figure 2
with example dimensions from the best model.) Did the authors try to
input to the k-NN modules a (possibly pretrained) embedding of the
reagent graphs instead of the 256 Morgan Fingerprints of radius 2? And
did those Morgan fingerprints they used include the reagent chirality
information or not?

The results of the baseline models in Figure 5 and Figure 11 are at
best embarrassing for those other codes, and at worst some
misconfiguration of those codes. I imagine that the codes try hard to
design against a target fingerprint-based input function and just
throw the kitchen sink until most of the bits of the produced
molecules match something that improves the score in the target
function. Out of curiosity, did the authors of this paper try to see
if there were any "reasonable" molecules produced by the other codes
with score better or equal to that of their own model. In any case,
the message that the authors want to convey with the current examples
is clear, but it may be helpful to contact the authors of those other
papers as a fair warning.

The paper demonstrates the basic concept behind this work. I was left
wondering why the authors limited the selection of possible reaction
templates to such a small set---is there a practical reason related to
the performance of the model in terms of training or inference time?
Furthermore, why did the authors only limit the present work to k=1?
I appreciate the simplicity of documenting the greedy (k=1) selections
for the results in table 1, but I'm left wondering how much better
could these results become if one included a larger number of
candidates at each step and searched all relevant trees.  

Although I commend the authors on including source code with the
submission, the reproducibility of this work is currently
unclear. Unfortunately, the supplemental code was not usable, so I
couldn't check any of the claims of performance. The README file
appears to be incomplete and has a couple of TODO notes in it; the
test subdirectory seems to be missing and thus the instruction to run
the unittests doesn't run any unittest at all; the code has references
to undeclared paths (DATAPATH, HOMEPATH, st_pis); references to data
files that don't exist (enamine_us.csv.gz).  I imagine that some of
these problems are easy to fix with a couple additional instructions
and it is possible that the rest could be resolved with a couple extra
scripts and some minor modifications.  Nevertheless, I felt a bit
disappointed that the authors missed an opportunity to submit a
remarkable work with a reproducible demonstration. I'd suggest that
the authors contact Enamine before the conference and request
permission to publish a snapshot of the current set of building blocks
or any other data that the authors used from Enamine, so that future
papers can compare against a benchmark using the same training and
testing data to document any algorithmic improvements in a bottom-up
approach or compare such approaches to retrosynthetic pathway
generation that target the same building blocks.

Despite these concerns on reproducibility, I think that this is a
refreshing paper of the type that I expect to see highlighted in the
ICLR conferences and I think that it opens up new directions for
methodological improvements to molecule optimization.

summary_of_the_review: 
~This is a refreshingly original paper on molecular synthesis planning
and synthesizable molecular design.  It is a strong, foundational
paper that is well written. The authors could do some a little
work to improve the reproducibility and enable future benchmarks
against the same data.~

[Edit: this work closely relates to Bradshaw et al, but the relationship
was not made clear in the exposition.  Until the authors 
address this concern and clarify their additional important contributions,
I do not recommend publication of this paper.]

Edit2: This is a strong paper that is well written and deserves publication at ICLR22.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper aims to construct a synthetic tree in a bottom-up manner to solve both problems of molecular design and synthesis planning simultaneously. The root node represents the final generated molecule, and the leaf nodes represent the reactants that can be purchased. Starting from the bottom leaf nodes, the authors utilize reaction templates to generate the parent node until the root node is reached. To design molecules with desired properties, the authors utilize the genetic algorithm to guide the tree generation.
main_review: The advantages of this paper are as followings:
1. The idea of constructing a synthetic tree in a bottom-up manner to solve both problems of molecular design and synthesis planning simultaneously is interesting and eye-catching. The tree generation algorithm is succinct and efficient.
2. Case studies(figure 5, figure 11) demonstrate the results in details.
3. It’s nice to see hyperparameter tuning results.
4. Codes are available.

Concerns are as followings:
1. It seems not very clear to me that how is genetic algorithm integrated into the process of molecular optimization and synthesis planning.
2. It might be better to compare the recovery rate(shown in table 1) of the proposed method with other synthesis planning methods(i.e., retrosynthe- sis).
3. It might be valuable to analyse more about why the proposed method is more computationally efficient.
4. In table 2, it would be better to demonstrate quantitatively the synthetic accessibility of molecules generated by different methods.

Some typos:
1. The font size of table 6 is too large.

summary_of_the_review: Overall, the paper is well-written, and the method appears to be promising.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The present paper is concerned about a forward-synthesis approach to molecular optimization. The proposed method defines one action to halt the procedure and three actions to develop a synthetic pathway, each of which corresponds to using zero/one/two of the intermediate molecules for chemical reaction to yield the next intermediate molecule. This generation model is controlled by an agent that chooses a series of actions to transit to the next state, given an embedding vector. If applied to synthesis planning, the embedding of the target molecule is used as the embedding vector, while if applied to molecular optimization, the embedding vector is optimized by a genetic algorithm.

The effectiveness of the proposed method is demonstrated by two experiments. The first one examines the ability of synthesis planning using reachable and unreachable data sets. The second one examines the ability of molecular optimization. For both of the experiments, both quantitative and qualitative analyses are provided.
main_review: ## Strengths
### + Reasonable model of synthetic pathway
The proposed generation model enables us to use not only the purchasable molecules but also intermediate molecules to chemical synthesis whilst the model being simple enough.

## Weaknesses
### - The relationship between the proposed method and existing forward-synthesis-based methods is not clear
I am not very satisfied with the discussion on the relationship to the existing methods. The introduction section does not mention the existing work using the same forward-synthesis idea to construct molecules, and the relationship is discussed only in Section 2.1. In addition, the existing synthesizable molecular design methods are not compared in the experiments. These are just examples of a lack of connection to the existing literature. I would suggest the authors to strongly relate the proposed method to the existing work and discuss what are the contributions of the present work.

## Other suggestions
- In Section 3.3, the agent uses $k$-NN to choose reactants, but it is not clear how to use $k$ neighbors to select one reactant. It is in fact discussed in Section 4.2, but I suggest to explain it in the method section.
- Is it possible to select a reaction template first, instead of a reactant? The number of reaction templates is generally smaller than that of reactants, and it would be helpful to alleviate the initial reactant selection problem discussed in Section 5.
summary_of_the_review: I suggest to reject this paper and encourage the authors to revise it for next opportunities.

While I consider the method proposed in this paper is reasonable, the relationship to the existing work is not discussed enough, and therefore, I consider the paper is not very mature to be published as an academic paper. I would suggest the authors to revise the paper so as to clarify the difference from the existing work and how much the difference contributes to improve the performance. To do so, additional experimental results would be necessary.
review_title: null

Meta-review: After much back and forth about prior work, 3 reviewers score this paper as an 8 and one scores it as a 3.
Other reviewers have written to the 3 and told them they believe that their review is now too harsh, in light of clarifications w.r.t. related work. I tend to agree, though I must admit that I am not an expert on this topic. 
Given that there is almost unanimous support for accepting and it's possible that the one hold-out has not seen some of the extra information, I recommend acceptance. 
Given the praise from the other three reviewers, I moreover recommend a spotlight.
Decision: Accept (Spotlight)