Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper presents a novel approach to generating molecules using Black Box Recurrent Translation.
The authors uses existing machine-translation inspired schemes to generate new, similar, molecules with better properties according to some measure.
Then, the recursion takes the top K best molecules and runs another iteration to generate even better molecules, ad infinitum.
The authors use the newly introduced SELFIES strings as vocabulary for generation.
The authors also analyze the decoding strategy, and how the process generates interpretable molecular traces.
Relating to wetlab work, having a molecular trace available from the recursive translation scheme is valuable for drug-sythesis.
The authors also show that this technique can optimize multiple properties at once.

I am leaning towards an accept for this paper, since not only does the technique presented seem general, the authors does in depth analysis into the model and how it affects drug discovery.
- Recursive black box translation seems to be widely applicable to new models.
- The model seems to reach a significantly better state of the art on the metrics proposed.
- None of the baselines seem to use SELFIES as the string of choice.
  This means it's difficult to tell how much the "Blackbox recursive" part of the algorithm adds to the model.
  An ablation experiment without BBRT might inform us of how much of the benefit is due to the molecule representation (Fig 4A reports the mean, but it would be good to have the same metric as Table 1).
- The authors provide an in depth discussion about how having molecular traces would hhelp in drug design.
  This makes the tool seem more widely appealing and useful.

A few questions would clear up the strengths of the paper:
- Is there a connection to the backtranslation work in Lample 2018? (Phrase-Based & Neural Unsupervised Machine Translation)
  It seems like a similar idea - except in this domain, the target language and source language are the same.
- How can there be multiple scoring functions? 
  Were they combined in one run, or were these separately optimized runs? Are these only used in Figure 4?
- Why would beam search do less well than stochastic? 
  Is it because during recursive translation, the beam search variants have low diversity?
  Then, training with stochastic decoding and generation with a beam search should do even better, right?
  This would highlight that the advantage of stochastic decoding is really online in the context of recursive translation, not generally.
- What is the point of Fig 4A right? Why do we expect that maximizing non-logP properties will increase mean logP?

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper presents a translation-based method for molecular property optimization. It uses a sequence- or graph-based encoder/decoder framework to produce molecules with (hopefully) improved properties, then feeds a subset of these molecules back into the encoder/decoder to generate a new set of molecules; this process is repeated for a fixed number of iterations to arrive at a final set of "optimized" molecules. The method is agnostic to the form of the encoder/decoder; the emphasis is on the iterative approach. Additionally, this approach enables visualization of "molecular" traces that can reveal pathways between molecules that follow relationships similar to matched molecular pairs. The work extends related work in translation-based property optimization [6]. The paper is well-written and generally easy to read.

The method is evaluated on two tasks, logP and QED. Both of these are computed properties that have known issues (see the discussion in [3]), but I understand that these properties are used in many publications and are thus easy to compare. The method presented here performs similar to others on a QED task. They claim superior results on the logP task, but I have concerns about the fairness of the comparison since logP can be exploited by very simple models if there are no limits on the size of the generated molecules (or, similarly, the number of tokens/generative steps allowed for each molecule). Additionally, the authors claim to perform multi-objective optimization but do not actually do this.

The iterative nature of this method is very interesting. However, my concerns about the types of experiments and comparisons that were done (see below for more details) are big enough that I cannot approve this paper in its current form. Weak reject.

Specific notes (starting with page number):

- 1: "potential druggable candidates" does not make sense; compounds are not "druggable" (their targets are), although they may be "drug-like".
- 2: Consider citing Kramer et al.'s seminal work on matched molecular pairs [1].
- 3: Please explain what it means for y to "paraphrase" x?
- 5: For your logP experiments, you need to be more clear about how you are comparing to other models. You are guaranteed to get to higher logP values if you can generate larger molecules (more tokens) than the baselines, since logP is essentially linear in the number of carbons. Are you doing something to limit the number of tokens you can generate in each iteration? Or why should I believe these comparisons are fair?
- 5: In Table 1, note that some literature uses a "normalized" penalized logP, while others use the formula directly without a dataset-specific normalization (which can appear to give better results). Can you confirm which you are using here and whether the baseline models are the same?
- 5: The results in Table 1 would be more compelling if they were not divorced from their starting points. Please include information about the similarity of these molecules to the starting molecule as well as the property delta. Consider an approach like Jin et. al [2], where results were specifically categorized by similarity constraints.
- 5: "All models were trained on the open-source ZINC dataset." What subset of ZINC are you using?
- 5: The supplementary figure showing that logP is broken is missing?
- 5: "Consistent with the literature we report diversity as...". Please cite some literature that you are consistent with.
- 6: "we sample 100 times from a top-2..."; does this mean you are doing 100 iterations? Sampling 100 times from the same top-2 sampler doesn't really make sense, but I'm not entirely sure what you are describing here.
- 7: Figure 4 says these are "ablation" experiments. What exactly are you ablating?
- 8: You state that better performance on logP and similar performance on QED is not known in the literature. In fact, the MolDQN paper [3] calls this out explicitly (and also contains a discussion of bounded vs. unbounded logP).
- 8: "Recent RL methods focus on molecular construction and are therefore not well-suited for the generation of molecular traces"; I disagree with this. RL methods that can start from a predefined graph have the ability to move between compounds, possibly in a way that is orthogonal to traditional similarity-based exploration (see the discussion of "MDP edit distance" in [4]). Also note that one of the key features of graph-based generators like [2] and [5] is that all of the intermediate states are valid, so you could do similar molecular traces for interpretability (although your differences are more like MMPs with functional group-level deltas).
- 9: "synthetic chemists can carry out the individual steps of a molecular trace..."; in general this is not true. The known medicinal chemistry transformations are a relatively small set of operations, and your molecular traces are unlikely to capture them in any systematic way. Please avoid making claims like this unless you can back them up with experimental evidence or comparisons to models explicitly trained for synthetic route planning.
- 9: These experiments are not multi-property optimization. Measuring the value of a secondary property while optimizing a primary property is not the same as optimizing them both simultaneously. The latter requires some strategy for incorporating both property values into the decision function, such as scalarizing (see "multi-objective optimization" on Wikipedia).

References:

[1] Kramer, C. et al. Learning Medicinal Chemistry Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) Rules from Cross-Company Matched Molecular Pairs Analysis (MMPA). J. Med. Chem. 61, 3277–3292 (2018).
[2] Jin, W., Barzilay, R. & Jaakkola, T. Junction Tree Variational Autoencoder for Molecular Graph Generation. arXiv [cs.LG] (2018).
[3] Zhou, Z., Kearnes, S., Li, L., Zare, R. N. & Riley, P. Optimization of Molecules via Deep Reinforcement Learning. Sci. Rep. 9, 10752 (2019).
[4] Kearnes, S., Li, L. & Riley, P. Decoding Molecular Graph Embeddings with Reinforcement Learning. arXiv [cs.LG] (2019).
[5] You, J., Liu, B., Ying, R., Pande, V. & Leskovec, J. Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. arXiv [cs.LG] (2018).
[6] Jin, W., Yang, K., Barzilay, R. & Jaakkola, T. Learning Multimodal Graph-to-Graph Translation for Molecular Optimization. arXiv [cs.LG] (2018).
review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: 
The paper builds on existing translation models developed for molecular optimization, making an iterative use of sequence to sequence or graph to graph translation models by wrapping them in a meta-procedure. The primary contribution is really just to apply the translation models iteratively, i.e., feeding translation outputs from the models back in as inputs for retranslation. A few strategies are introduced to score / rank candidates before they are chosen for retranslation. The overall idea is very simple, and is likely to work in some basic cases where the property has a natural "additive" nature, e.g., logP that you can improve by adding functional groups. This is recognized but not really controlled in the paper except for selecting for input similarity before retranslating. Moreover, I don't think that you really ever want to just maximize logP for any drug so this particular task is a bit artificial in the first place. Other properties are not additive in the same sense, e.g., drug likeness or QED, and the method doesn't appear to improve it (though, to be fair, there may be a ceiling effect for QED in particular). 

One of the main ways that one can control the final output in the iterated translation process is by judiciously selecting or ranking candidates for retranslation. The authors use essentially the score from the model itself, similarity to input, and some basic chemistry metrics to do that. Wouldn't it be much better to train a separate ranking method to guide the iterative steps? 

The empirical results are clean though not convincing (see the logP discussion above). Additional properties should be included to demonstrate that the method might actually have some practical value, i.e., generalize beyond additive logP. Multi-property optimization would be one possible setting since de novo models have a hard time to reach intersections of different property constraints. Abstractly, one could imagine that an iterative, successively guided approach could work well. The proposed approach in the paper is somewhat undeveloped. It merely uses a translation model for the primary property, and ranks candidates by the other. This is unlikely to get you to any challenging intersections. Also, since logP was always one of the properties effectiveness in this regard is not really demonstrated either. A slightly more sophisticated approach might use relaxed, separately trained ranking models in intermediate steps, successively tightened towards the intersection as the iteration progresses. E.g.,

Brookes et al., Design by adaptive sampling, arXiv:1810.03714

The paper is clearly written but for such a simple method one would need really convincing results and experiments. Maybe better as a workshop submission?
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The authors frame molecule optimization as a sequence-to-sequence problem where a source molecule is translated to a target molecule with improved properties. The authors extend existing methods for improving molecules by applying them recursively over multiple rounds, and show that it is beneficial for optimizing logP but not QED. An advantage over existing methods is that the trajectory of optimized molecules is interpretable. Altogether, I find the paper borderline: it is clearly written but the methodological contribution is incremental, some citations to related work missing, and some parts of the results section are weak. Detailed comments below.

Major comment
============
1. Framing optimizing as a sequence to sequence problem is not new. As described in the related work section, the BBRT is closely related to Jin et al. However, it is not clearly described what the major improvement over Jin et al is. Please clarify ‘their inference method restricts the framework’s application to more general problems.’.  The method is also closely related to Zou et al (https://www.nature.com/articles/s41598-019-47148-x) and Mueller et al (http://proceedings.mlr.press/v70/mueller17a.html), which are not cited in the text. Zou et al used RL to learn to optimize molecules by mutating existing molecules. Mueller et al used Seq2Seq to optimize the sentiment of sentences. Please cite these papers and discuss why BBRT is better.

2. Please compare to ChemBO (http://arxiv.org/abs/1908.01425). The current baselines are one-shot in that they are proposing a batch of molecules once without using the acquired target function label to propose subsequent batches. ChemBO optimizes a target function such as logP over multiple rounds similar to recursive BBRT approach, and should therefore be included as a baseline. Another suitable baseline would be performing BO in the latent space by applying Gomez et Bombarelli recursively (embed molecule; optimize GP in embedding space; decode molecule; iterate).

3. The method names (Graph2Graph, Seq2Seq, R-Graph2Graph, R-Seq2Seq, BBRT-JTNN, …) are not defined in section 5.1-baselines, and used inconsistently. Is JTNN the same as Graph2Graph and does BBRT mean recursive (R-)? This makes is hard to follow the results section.

4. Section 5.1: How does the performance depends on the initial seed of sequences? How sensitive is it i) to the choice of the diversity cutoff, and ii) to the target value of the initial molecules? 

5. Fig 4a, right: Is is expected that logP increases fastest when using it as a scoring function. Please show instead QED vs. the number of iterations. QED combines several molecular properties, including logP, and is therefore more suited for quantifying drug likeness.

6. ‘Differences between logP and QED.’ I do not understand this section. Please clarify the goal of an explorative vs interpolative task? Are molecules with the highest QED in the training dataset? Motivate why BBRT does not achieve a higher QED in table 1?

7. ‘The distinction in the vocabulary highlights the usefulness …’. Is the conclusion that representing molecules as sequences is better than representing them as graphs? This would contradict several recent papers on graph-based representations. You are only comparing the top molecules. Is there a significant difference in the complexity between the top 100 (for example) molecules?

8. Section 5.3 is verbose and can shortened to a few sentences saying that applying edits to molecules recursively makes the model interpretable. How do traces look like when logP is used a  selection criteria? How does the trace of the best molecule shown in figure 3 look like? What is the average edit distance between molecules are and intermediate molecules valid? Are transitions plausible?

9. Section 5.4: You are optimizing a single objective (e.g. logP) while reporting in parallel a second objective (e.g. QED). This is not multi-objective optimization, where multiple objectives are optimized in parallel. Optimizing a single objective while reporting a second objective can be also done with methods other than BBRT. Please clarify the take-away message of this paragraph or remove it.


Minor comments
=============
10. Introduction: ‘discrete and unstructured’. Why unstructured? I would say that molecules are structured--they must follow a certain grammar to be valid.

11. Introduction: ‘treating inference as a first class citizen’ is unclear since ‘inference’ is undefined. Either remove this sentence or clarify.

12. Please discuss that BBRT is limited by the need of a labeled dataset for constructing training pairs.

13. Section 5.1, ‘Similar computational budget’. How did you quantify the computation budget?

14. Section 5.1, ‘In Fig 2, we report’. Do you mean Fig 3? Same as with ‘Fig 3’ in the following paragraph.
review_title: Official Blind Review #1

Meta-review: This paper presents a simple method for improving molecular optimization with a learned model. The method operates by repeatedly feeding generated molecules back through an encoder decoder pair trained to maximize a desired property. Reviewers liked the simplicity of the method, and found it interesting but ultimately there were concerns about the metrics used to evaluate the method. Reviewers 3 and 4 both noted issues with the log P (and penalized log P) metric, noting that it is possible to artificially increase both metrics in a way that isn't useful in practice. During the discussion phase, Reviewer 4 constructed a specific example where simply adding long carbon chains to a molecule would yield a linear increase the penalized log P metric, and noted that the "best molecules" found by the method in Figure 3 also have extremely long carbon chains (long carbon chains are not generally desirable for drug discovery). 
I recommend the authors resubmit after finding a better way to evaluate that their method generates molecules with more useful properties for drug discovery.
Decision: Reject