Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The authors present an approach to improve performance for retro-synthesis of chemical targets in a seq2seq setting using transformers. The authors have encouraging results and the paper was fairly easy to read and follow. However there are a variety of concerns that the authors need to address:

The technical contributions in this paper are somewhat thin. The main contributions are data augmentation techniques, pre-training and a mixture model that seems to improve performance on the USPTO-50K dataset. The novelty is quite low and it’s not clear if this will transfer to another domain. The impact is also low as the pre-training techniques using bond breaking and template-based are specific to this problem task. Additionally, mixture models for encouraging diversity is a simple instance of ensembling.

Using deep learning to this application area is also not novel. This paper largely builds upon previous work with Transformers from Schwaller et. al and Karpov et. al. 

Other clarifications/issues:
 - The experimental results are based only on the USPTO dataset. It’s unclear how significant the results are. The authors can consider using diverse datasets or applying their techniques to another application domain to bolster their claims.
 - Table 3, lists the average number of unique reactions classes. The authors say “ … we predict the reaction class for every output of our models… “ . It’s not clear how it makes sense to calculate diversity when there’s no ground truth available for determining if the predicted output is a valid synthesis for the target. To say this another way, what good is diversity if the prediction is incorrect?
 - Table 3, lists human eval results. The details here seem quite vague. How does a human determine something to be more diverse? What is the rubric they use? How qualified is the human in being able to judge this task?
 - Figure 6 does not have a color scale.

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: Given a target compound, the authors suggest a method to predict likely chemical reactants to produce the target. The authors provide a transformer based model to predict the reactants. Existing methods do not generalize well for rare reactions and the training data has only one reactant set for each target even though that may not be the only way to synthesize the compound. To solve this problem, the authors use a pretraining method similar to BERT. Instead of just using token masking, they provide alternate proxy decompositions for a target molecule by randomly removing bond types that are likely to break and by transforming the target based on known templates.

This is a well written paper with good baselines.They use multiple techniques that are both domain specific (data augmentation) as well as methods from NLP adapted for this task. The experiments are carefully designed and show that both pretraining and data augmentation helps. Overall, I think the community will benefit from this work.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper is very well-written and combines the state-of-the-art NLP model and the domain knowledge in retrosynthetic reaction predictions.  The authors propose pre-training models to help improve the model's generation to rare reactions. In addition, a discrete latent variable model is used in the model to encourage the model to produce a diverse set of alternative predictions.  The experiments in the paper also show the effectiveness of the two main contributions.

The main contribution of this paper is to apply the state-of-the-art Transformer model and other techniques in NLP to address the specific issues in retrosynthesis. Both the pre-training model and the mixture model are combining the specific domain knowledge to improve the generalization and diversity of retrosynthesis. There is not a huge algorithm novelty for the methods proposed in this paper, but they can well address the domain issues and improve the performance.

My only concern is that the baseline model compared in the paper is Schwaller's work. I am not pretty sure if this baseline achieves state-of-the-art performance. It would be very interesting to see more comparisons with other state-of-the-art 
 work.



review_title: Official Blind Review #3

Meta-review: The authors present a new approach to improve performance for retro-synthesis using a seq2seq model, achieving significant improvement over the baseline. There are a number of lingering questions regarding the significance and impact of this work. Hence, my recommendation is to reject. 
Decision: Reject