Paper:
We propose a new model for making generalizable and diverse retrosynthetic reaction predictions. Given a target compound, the task is to predict the likely chemical reactants to produce the target. This generative task can be framed as a sequence-to-sequence problem by using the SMILES representations of the molecules. Building on top of the popular Transformer architecture, we propose two novel pre-training methods that construct relevant auxiliary tasks (plausible reactions) for our problem. Furthermore, we incorporate a discrete latent variable model into the architecture to encourage the model to produce a diverse set of alternative predictions. On the 50k subset of reaction examples from the United States patent literature (USPTO-50k) benchmark dataset, our model greatly improves performance over the baseline, while also generating predictions that are more diverse.

Review:
The authors present an approach to improve performance for retro-synthesis of chemical targets in a seq2seq setting using transformers. The authors have encouraging results and the paper was fairly easy to read and follow. However there are a variety of concerns that the authors need to address:
The technical contributions in this paper are somewhat thin. The main contributions are data augmentation techniques, pre-training and a mixture model that seems to improve performance on the USPTO-50K dataset. The novelty is quite low and it’s not clear if this will transfer to another domain. The impact is also low as the pre-training techniques using bond breaking and template-based are specific to this problem task. Additionally, mixture models for encouraging diversity is a simple instance of ensembling.
Using deep learning to this application area is also not novel. This paper largely builds upon previous work with Transformers from Schwaller et. al and Karpov et. al. 
Other clarifications/issues:
 - The experimental results are based only on the USPTO dataset. It’s unclear how significant the results are. The authors can consider using diverse datasets or applying their techniques to another application domain to bolster their claims.
 - Table 3, lists the average number of unique reactions classes. The authors say “ … we predict the reaction class for every output of our models… “ . It’s not clear how it makes sense to calculate diversity when there’s no ground truth available for determining if the predicted output is a valid synthesis for the target. To say this another way, what good is diversity if the prediction is incorrect?
 - Table 3, lists human eval results. The details here seem quite vague. How does a human determine something to be more diverse? What is the rubric they use? How qualified is the human in being able to judge this task?
 - Figure 6 does not have a color scale.