Paper:
We address a fundamental problem in chemistry known as chemical reaction product prediction. Our main insight is that the input reactant and reagent molecules can be jointly represented as a graph, and the process of generating product molecules from reactant molecules can be formulated as a sequence of graph transformations. To this end, we propose Graph Transformation Policy Network (GTPN) − a novel generic method that combines the strengths of graph neural networks and reinforcement learning to learn the reactions directly from data with minimal chemical knowledge. Compared to previous methods, GTPN has some appealing properties such as: end-to-end learning, and making no assumption about the length or the order of graph transformations. In order to guide model search through the complex discrete space of sets of bond changes effectively, we extend the standard policy gradient loss by adding useful constraints. Evaluation results show that GTPN improves the top-1 accuracy over the current state-of-the-art method by about 3% on the large USPTO dataset. Our model's performances and prediction errors are also analyzed carefully in the paper.

Review:
Update:
Score increased.
___________________________________
Original review:
The paper presents an approach to predict the products of chemical reactions, given the reactants and reagents. It works by stepwise predicting the atom pairs that change their bonds in course of reaction, and then adjusting the bonds between them. This can be interpreted as a stepwise graph transformation.
I think this is an interesting applied ML paper with fair results. The presentation is clear and understandable. The experimental setup is reasonable. However, the paper is not ready yet to be accepted in my opinion.
I think a higher score is justified if the authors address the following points:
- Relation to previous work, originality
In contrast to what the authors claim, what is predicted here is not exactly the reaction mechanism, but an implementation of the principle of minimal chemical distance, which was already described by Ugi and coworkers in 1980 [see Jochum, Gasteiger, Ugi, The Principle of Minimum Chemical Distance Angew. Chem. Int. Ed. Engl. 1980, 19, p 495-505]. 
The “insight” the authors have about treating reagents and reactants jointly is Organic Chemistry 101, and that reactions are stepwise graph transformations was also reported by Ugi et al, however, already in 1979! [Ugi, et al. "New applications of computers in chemistry." Angewandte Chemie International Edition in English 18.2 (1979): 111-123. ] I assume the authors were not aware of these papers, but now they are, so this needs to be modified accordingly, and these papers need to be referred to in the introduction. 
- Questions:
The authors suggest that graph neural networks are more generic that so-called heuristic features (fingerprints) – which, as Duvenaud et al have elaborated, can be interpreted just as graph neural networks themselves – with fixed weights. Also, there are results by the Hochreiter group which show that graph neural networks perform worse that classical chemical features under rigorous testing { DOI: 10.1039/C8SC00148K } Do the authors think their models could also improve if they used the classical fingerprints?
Is the GRU really needed to encode the past bond changes? What happens if you remove it?
The statement that the method has the advantage of not relying on handcrafted reaction templates is somewhat overselling, because instead it uses a handcrafted complex neural network architecture. How complicated is it to train the network? If you remove some of the “tricks” of shaping the loss function, does it still train well?
To what degree is the ranking of the different models just a matter of hyperparameter tuning or different architectures? If you used a different graph neural net instead of an MPNN on top of your GTPN method, what would you expect? Are the differences between the models significant?
During prediction, you apply a flag to the starting molecules if they are a reagent or reactant. How do you know upfront what a reagent or reactant is during inference? 
On page 5 and 7, you speak of the correct sequence of reaction triples (which implies an ordering), even though earlier you claim the algorithm is order-invariant? Where do you get the ground truth labels from? I assume these are already annotated in the data.
In the appendix, please replace the pie chart with a bar chart.
- Language:
I would suggest the authors to adapt the language of their paper towards a more academic tone. Science is not a sports competition of getting slightly higher numbers in benchmarks, but rather about providing insights and explanations. Words like “beating” or “record” are locker room talk, and to be avoided.