Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper proposed Graph Enhanced Transformer(GET) to combine the graphical and sequential representations of the molecule to improve the retrosynthesis prediction performance. Experiments indicated that the proposed model outperforms state-of-the-art Seq2Seq-based methods on USPTO-50K dataset, and showed ability in reducing invalid SMILES rate.

Two main comments: 

1. This paper provide no novelty with respect to deep learning method. It is just a combination of sequence transformer and graph neural network (using RDKit(Landrum, 2016) to transform a SMILES into the molecular graph). The decoder is the same as vanilla Transformer to generate SMILE string output. 

2. The writing can be improved. For instance, in the caption of Figure 1 - "somehow to be transformed" ...  plus a few other places have wording issues like this.  

 



review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: ~The authors propose an enhancement to the transformer architecture that takes molecule graph structure into account.~

I applaud the authors work on making more physically plausible machine learning constraints. However, I feel like this work is incremental and does not vastly improve SoA.

For all tables, what are the error bars on the accuracy?

There are multiple possible SMILES strings for any unique molecule (which includes the canonical SMILES string.) Does bootstrapping your Transformer model with multiple non-canonical SMILES for the same input molecules improve performance?

Many seq2seq molecules allow an attention mechanism on the input sequence while decoding, and that seems like this would be useful for this data. What would the impact of this be?

Do you have any explanation why GET-LT1 is your top performing model?

Does your model generalize to unseen molecules or reactions better than previous methods?

In Table 5, why does the % invalid SMILES go up with more beams? I would figure that larger beam sizes would result in more valid generated molecules?

In Table 4, please report the number of parameters for each model.

In Table 2, what are the astrices (*) in the GET-LT1 row?

In Table 3, you should bold the Similarity model for the top-5 and top-10 accuracy.

In equation 9, do you mean “sigmoid”?

In equation 8, should the k superscript be a subscript?

How would this model perform with SELFIES representation of small molecules, which are more robust representation [Krenn et al., 2019]?

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: 
This paper focuses on the retrosynthesis prediction problem which to my understanding is the factorization of a target molecule into simpler structures. Previously, retrosynthesis prediction has been tackled as a language translation problem by using a Seq2Seq algorithm called "Transformer". This sequential approach was possible because molecules can be expressed as strings using the following format: Simplified Molecular-Input Line-Entry System (SMILES).

Despite its good performance, language translation methods ignore the graphical structure of molecules. This paper proposes to add a graph neural network in front of the Seq2Seq/Transformer network to exploit the graphical structure, hence the name “Graph Enhanced Transformer (GET)”. The Graph Neural Network considered in this work also uses an attention mechanism similarly to Graph Attention Networks.

The main contribution is the addition of a Graph Neural Network to a Seq2Seq model for retrosynthesis prediction which provides state of the art results.

From a machine learning / representation learning perspective, I do not consider the paper is innovative enough. Even so, the paper shows strong results for retrosynthesis prediction, I guess it would be a good fit in a chemistry conference.


Things to improve:

There are plenty of works related to graph Autoencoders and Graph generative models that are not mentioned in the related work. It would be great to dedicate a section for them. Here some examples: 

- Li, Y., Vinyals, O., Dyer, C., Pascanu, R., & Battaglia, P. (2018). Learning deep generative models of graphs.

-Simonovsky, M., & Komodakis, N. (2018, October). Graphvae: Towards generation of small graphs using variational autoencoders.

-De Cao, N., & Kipf, T. (2018). MolGAN: An implicit generative model for small molecular graphs.

-You, J., Ying, R., Ren, X., Hamilton, W. L., & Leskovec, J. (2018). Graphrnn: Generating realistic graphs with deep auto-regressive models.

All experiments are compared to author’s re-implementations of other works. It would be interesting to directly report other work accuracies.

review_title: Official Blind Review #4

Meta-review: Several approaches can be used to feed structured data to a neural network, such as convolutions or recurrent network. This paper proposes to combine both roads, by presenting molecular structures to the network using both their graph structured and a serialized representation (SMILES), that are processed by a framework combining the strenth of Graph Neural Network and the sequential transformer architecture.

The technical quality of the paper seems good, with R1 commenting on the performance relative to SOTA seq2seq based methods and R3 commenting on the benefits of using more plausible constraints. The problem of using data with complex structure is highly relevant for ICLR.

However, the novelty was deemed on the low side. As a very competitive conference, this is one of the key aspects necessary for successful ICLR papers. All reviewers agree that the novelty is too low for the current (high) bar of ICLR. 
Decision: Reject