Venue: ICLR 2022 Conference

Reviews:

final_score: rating: null
review_content: This change makes it very easy to see what has been changed in the paper, thank you for doing this in your response :)
review_title: Thank you to authors for highlighting changes in red

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Although the term "genetic algorithm" is not used in the paper, this paper effectively proposes a fancy genetic algorithm (GA) for molecule optimization. The contribution is chiefly in the mutation step of the GA, which roughly uses a pre-trained graph neural network to take the derivative with respect to the adjacency and node identity matrices of a graph to produce a probability distribution over mutated graphs. They show results on several different molecular optimization tasks which suggests that their algorithm outperforms several baselines.
main_review: I thought that this paper was well-written and had a lot of genuinely interesting ideas. It was interesting to read. My main concern is that I think it is possible that the proposed algorithm is overly complicated and doesn't work as well as the authors claim. I will provide more details below.

== Pros ==

- To my knowledge, the differential scaffolding tree (DST) idea is novel. I think it is a very interesting contribution
- the details of the DST method are very well-described in section 3. I think it would be possible to re-implement their method based on the information given, which is a good sign
- Choice of experiments seemed good (I'm glad this is not yet another paper maximizing logP). It was nice to see that you included many baselines
- Good mentioning of relevant related work.
- The interpretability of the method is really interesting!

== Cons ==

- Looking at algorithm 1 (the main contribution), it is clearly a genetic algorithm. There is a population, and a loop over "generations" which includes a "mutation" step and a "new population selection" step. I found it slightly odd that the authors did not point out this connection, or use any terminology related to genetic algorithms. Given the similarity, it would be nice to see the authors discuss how their GA compares to previously proposed GAs for molecule optimization (such as the GAs from Guacamol, or the various SELFIES GAs by Alan Aspuru-Guzik's group)
- Thinking of the contribution as a genetic algorithm, it actually isn't clear to me that the whole DST thing is necessary. Fundamentally, the proposed mutation step takes a molecule X (in the form of a scaffold tree), and returns a set of candidate molecules whose scaffold tree differs from the current molecule's scaffold tree by 1 local editing operation (called $\mathcal{N}(X)$ in the paper). The DST effectively provides a way of sampling from $\mathcal{N}(X)$, where samples are (probably) predicted to have a high score by the GNN. While this is sensible, I can think of many similar alternatives which are much less complicated (see below). Overall I worry that the proposed method is just an overly complex way of doing something quite simple.
  1. Just enumerate all of $\mathcal{N}(X)$. My guess is that for small molecules this set is actually not that large ($\le 10,000$) although I could be wrong about this.
  2. Since the above approach may propose many poor molecules (and therefore not be sample efficient), one could enumerate $\mathcal{N}(X)$ and return a subset which the GNN predicts will have high scores. In fact, any oracle could be used here (e.g. random forest, GP) which might be less prone to overfitting and work with less training data.
  3. For computational efficiency, the approaches above could be modified to just randomly sample from $\mathcal{N}(X)$ rather than enumerating it. This is the approach taken by most GAs.
- Many details of the experimental section are not clear in the main text. As a reader I felt that the experiments offered little value without reading a lot of the appendix. Aware that there is a page limit, I nonetheless think that the following things needed to be in the main text:
  1. Brief explanation of the objectives. Why should I care about them? Are they novel objectives, or were they proposed in previous work? (seems like the latter)
  2. Why were your baselines chose, beyond just being other methods which work for molecular optimization?
  3. What is success rate? (turns out this is the only metric related to actually succeeding at optimization)
  4. Why should I care about novelty and diversity? Some readers might just care about finding $n$ molecules with high scores. I think there is a strong case for caring about these but it should at least be mentioned.
  5. How was the number of oracle calls chosen?
- Generally weak baselines w.r.t. sample efficiency: the authors claim that needing a small number of oracle calls is a key advantage of their method, and indeed, their method outperforms other methods with fewer oracle calls. Yet, as far as I can tell, none of the methods they compared with would actually be expected to succeed with a small number of oracle calls. GCPN, MolDQN, RationaleRL are all RL methods which only produce good molecules after a long training period. GA+D and MARS are exploratory methods which, as far as I understand, call the oracle indiscriminately on a lot of random points to explore molecular space. I am unfamiliar with LigGPT. To my knowledge, the most sensible baselines to compare with for sample efficiency would be Bayesian optimization. I suggest perhaps references [1-4] (in that order). If object that GPs don't scale and therefore are not applicable here, I would counter by saying that they will easily scale up to 10k points on a reasonable computer, and approximate GPs with 10k inducing points should be manageable with the 50k oracle calls shown in Figure 3.
- In table 1-2, the number of oracle calls is listed as if it is a fundamental property of each algorithm, which seems strange because it isn't: the number of oracle calls is directly decided by the number of iterations, the number of calls per iteration, etc, which can be chosen by the user. Simply put, all of the methods listed can be run with any number of oracle calls (although with fewer oracle calls performance will almost certainly be worse). It isn't clear whether the number of oracle calls was simply chosen so that DST had the lowest number, whether the number of oracle calls is really just the number of oracle calls necessary to meet some sort of external criterion (e.g. a certain number of successful molecules produced), or something else. In general, I think it is more fair to show comparisons like in Figure 3, which plots performance as a function of the number of oracle calls.
- Strange choice of metrics in Tables 1-2. I see many disadvantages to the metrics proposed in Tables 1-2. Generally, they are based around evaluating the entire set of molecules produced, rather than just evaluating the top points, which is more normal in optimization (since one generally cares about the optimum reached, not how one gets there). I particularly disagree with the use of "success rate" to measure optimization performance. Firstly, the cutoff for "success" seems fairly arbitrary. It is not clear how performance changes when this cutoff is changed. For this reason I think people usually report the top N scores or the average of the top N scores, rather than a measure of how many molecules exceed a threshold. Secondly, by measuring success as a "rate" the scores of sample inefficient methods seem to be unfairly penalized: even if they produce more molecules of better quality, the presence of many poor molecules (e.g. from the beginning of training in RL) will lower the score. Overall, I think that Figure 3 provides a much more valuable comparison of methods than Tables 1-2

== Suggestions for Improvement ==

Overall I think that the paper would benefit greatly from the following improvements:

1. Some sort of ablation study or comparison with modified GAs to determine whether the added complexity of the DST is actually necessary to achieve their optimization performance.
2. The addition of stronger baselines (I suggest specifically Bayesian optimization). This will help evaluate the performance of the proposed method.
3. Re-organization of the experimental section, including more explanations and reporting better metrics.

## References

[1] Korovina, Ksenia, et al. "Chembo: Bayesian optimization of small organic molecules with synthesizable recommendations." International Conference on Artificial Intelligence and Statistics. PMLR, 2020.

[2] Moss, Henry, et al. "BOSS: Bayesian Optimization over String Spaces." Advances in Neural Information Processing Systems 33 (2020).

[3] Moss, Henry B., and Ryan-Rhys Griffiths. "Gaussian process molecule property prediction with flowmo." arXiv preprint arXiv:2010.01118 (2020).

[4] Gómez-Bombarelli, Rafael, et al. "Automatic chemical design using a data-driven continuous representation of molecules." ACS central science 4.2 (2018): 268-276.
summary_of_the_review: I feel quite conflicted on this paper: on the one hand, the DST is to my knowledge novel and fairly interesting. On the other hand, the lack of "good" baselines makes it hard for me to judge the performance of this method, and I am concerned that a lot of the complexity of the DST is not actually necessary. Tentatively I think I will recommend rejection due to these concerns, but I would be happy to raise my score if they are addressed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposed differentiable scaffolding tree (DST), a novel method for molecular generation, that allows for a continuous gradient-based optimization of selected molecular properties. The results of de novo molecular optimization are really good. DST outperformed other methods in terms of optimization success rate, at the same time the proposed molecules were diverse and novel. The method also required less oracle calls (and majority of them before starting the optimization), compared to the other methods.
main_review: Pros:
- The idea proposed by authors seems novel and interesting.
- The results reported by authors are really promising.
- The proposed method allows to use a continuous gradient-based optimization of oracle function. Thus there is no need to train a latent space model before optimization.

Cons:
- In my opinion the section that describes the experimental settings  is not sufficiently developed. E.g. I cannot see how many molecules were generated and how many optimization iterations were performed for every molecule (the authors wrote that DST reached the highest scores within T = 50 iterations, but they do not specify whether they use this amount of steps in every experiments). 

Questions during rebuttal period:
- How many adam steps were conducted in every iteration? 
summary_of_the_review: The method seems novel and interesting and results are promising. The only drawback is the weak section describing the experimental settings, which, however, can be easily corrected.
review_title: null

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: The molecular optimization is an essential chemical science and engineering task that has important applications such as drug discovery. Deep generative models and combinatorial optimization methods achieved initial success but still struggle with directly modeling discrete chemical structures and often heavily rely on brute-force enumeration. The challenge comes from the discrete and non-differentiable nature of molecule structures. To tackle this, the authors propose a differentiable scaffolding tree (DST) that utilizes a learned knowledge network to encode prior information. 

Important contributions include:

(1) Definition of a local derivative of a chemical graph, we propose the differentiable scaffolding tree. The theory allows for gradient-based optimization of a discrete graph structure.

(2) The authors generalize a chemistry-aware graph network to encode prior knowledge. The network is capable of learning the hierarchy between functional groups and atoms, as well as the topological relations among chemical structure fragments.

(3) A unified graph-based optimization framework is proposed that can tackle both single-task and multi-task objective functions.
main_review: Difference to related works: Most of the existing deep generative models do not scale well on large datasets due to non-differentiable nature of molecule structures and they often heavily rely on brute-force enumeration. The proposed DST significantly increases scalability by employing implicit differentiation for gradient updates, i.e., fully differentiable generation of new molecules that preserves computation time and storage space without introducing any additional latency overhead or stochastic variation being induced by random initialization of weights in DST architecture. 

Limitations: More quantitative performance comparisons with related works are needed. 
The authors should expand on the approximation for formulating the problem as an iterative local discrete search specifically as it relates to equation 10. 
Why did the authors allow for single atoms to be utilized as "substructures"? This aligns against the intuition of chemists who usually consider atoms as sidechains for modification as opposed to substructures for clustering or analyzing data by series.
summary_of_the_review: This paper created a molecular graph locally differentiable by using a differential scaffolding tree (DST). To the best of our knowledge, it does appear as a novel approach to make the molecular optimization problem differiable at the structure level rather than utilizing latent spaces or employing RL/evolutionary algorithms. The authors developed a generic molecular optimization approach based on DST, which was validated through comparison when previous methods and benchmark datasets. The paper appears well written and the results are promising.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper designs a novel model to optimize molecular leads with a differentiable scaffolding tree. The idea of learnable parameters in indicator matrix, adjacency matrix, and node weight vector is novel and interesting although the method still contains some weaknesses. The paper still provides a good direction of molecular generation.
main_review: The paper provides a few interesting ideas although it also contains some weaknesses. 

Strengths:
1. **Differentiable scaffolding tree**: The DST is designed as a learnable graph input of molecules. By back propagating the parameters in A, N and w, it samples out new molecular structure along the direction of optimizing purposed properties. DST implies a novel direction of resolving the problem of lead optimization.
2. **Scaffold tree instead of molecular graph**: The proposed method smartly uses scaffold tree instead of molecular graph to alleviate invalid molecules.
3. **Determinantal point process**: The paper leverage DDP to keep expected properties as well as the diversity of generated molecules.

Weaknesses:
1. **restricted backbone**: The backbone of the oracle model (as GNN used in the paper) must be a differentiable model. This limits the usage of the proposed method. For example, if the oracle response comes from the real wet experiments, the proposed method is unable to get gradients from the wet experiments. Moreover, it may also suffer from the activity cliff of molecular properties.
2. **Local optimum**: SInce the proposed method generates new molecules step by step, it is possible that the selected pathway of generating new molecule is not an optimal solution. When it falls into a local optimum, the properties of the generated molecules will not be improved more.
3. **Costly assembling process from scaffold tree**: it is also a problem of JTVAE by practice. Assembling molecular graphs from scaffold tree is very time consuming.

Suggestions:
1. **Better assembling way of scaffold tree**: the authors could leverage some ideas from [1] to avoid costly assembling process.
2. **Add constraints to learnable N, A, and w**: Some lead optimization tasks may require un-changeable portions of the molecules. This could be easily achieved by adding constraints to learnable N, A, and w. 

[1] Jin, Wengong, Regina Barzilay, and Tommi Jaakkola. "Hierarchical generation of molecular graphs using structural motifs." International Conference on Machine Learning. PMLR, 2020.
summary_of_the_review: Overall, it is a good paper with some interesting designs like DST. The proposed method shows a new direction to do lead optimization. I prefer to accept this paper although it contains some cons that hard to avoid.
review_title: null

Meta-review: This paper was a tough call. The key contribution of the paper is a genuinely useful technique for generating chemical compounds satisfying desired properties. However, there are some key issues with paper.

Reviewer *BjiD* found out that baselines are weak. Most importantly, he run thorough experiments with GraphGA, outperforming by a significant margin the baselines. With minor tweaks (e.g. enabling generating larger molecules). GraphGA achieves comparable though slightly weaker results to DST. Importantly, as pointed out by Reviewer BjiD, there is an important flaw in the experiments that some methods have a cap on the number of atoms they can add. For example, on the logP optimization task, it is possible to optimize the score by just adding carbon atoms. I would like to thank very much Reviewer for going beyond and running these experiments.

All reviewers emphasized the novelty as a key contribution. In internal discussion, I raised concern about novelty and framing of the work. One could argue that any autoregressive model (i.e. adding atoms and bonds at each step) forms a DST. One could also argue that training LSTM to produce the distribution of interest, like in [1], is also a DST because the fine-tuned LSTM encodes the distribution of many molecules and is differentiable with respect to the distribution it encodes.

Despite these flaws, it is a solid contribution, which is likely to be useful for the community. Thank you for your submission, and it is my pleasure to recommend acceptance.

For the camera-ready please: (a) include a well-tuned GraphGA (implementing different tradeoffs of diversity and fitness), (b) include LSTM as implemented in Guacamole as baseline, (c) discuss much more clearly novelty of the work. Additionally please ensure that other baselines are not hampered by limit on number of atoms they can add.

References:

[1] Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks, Segler et al, [https://arxiv.org/abs/1701.01329](https://arxiv.org/abs/1701.01329)
Decision: Accept (Poster)