Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper gives an alternative to best-first search ("BFS"), plus heuristics, planning of molecules synthesis. It does so by training a policy network on ECFP4 (string encoding) representations of molecules, to predict which (sub)molecule to apply, with rewards +1 when the molecule to synthetize is complete, -1 if this branch of applications is complete and the synthesis failed, 0 otherwise. As far as I understood, it is trained in _retro_synthesis (decompose the molecule). This policy network is coupled to a search method (BFS of MCTS) both to reduce the branching factor, and to aggressively prune the validation of chemical rules (graph isomorphism).

The details about the policy model are almost inexistant. The run-time of MCTS seems more than twice longer than that of BFS, so, baring an explanation, it feels like those should be compared in wall clock time. The "random" test set selection may be problematic to ensure that the whole model is working properly: you may want to test in generalization, on end-result (==start in retrosynthesis) molecules that are unknown to the policy network.

"essentially the complete published knowledge of chemistry":
1) Paywall :(
2) maybe you are talking about a specific subset of organic chemistry? Even then, I doubt this includes all metabolic pathways and/or protein foldings.

The rest of the paper seems reasonable.
 
This paper seems good enough for a workshop at ICLR, and it could spawn interesting discussions.
review_title: Pruning search complexity with a policy network in retrosynthesis search

final_score: rating: 6: Marginally above acceptance threshold
review_content: Chemical synthesis planning is a long-standing and important problem. While not the first to propose machine learning to model an 'expert chemist' for chemical synthesis, the particular reinforcement learning approach with MCTS in this work is novel as far as I know. However, I have a couple questions and concerns.

Primarily, not enough information is given about the training data or the policy network trained from it. In general, good chemistry data is hard to obtain; information tends to be scattered, in different formats, and of varying quality. So I'm skeptical about the claim that the data set contains "essentially the complete published knowledge of chemistry." What are these reactions, how are they represented, and how do you determine the _best_ move for a particular molecule when training? For the policy network, what is the network architecture? These may have been published elsewhere, but more details should be provided here, especially for a machine learning audience.

I don't completely understand the performance improvement due to the MCTS. It seems less useful in this application than in Go, because here whenever you reach a reward of Q(v)=1, the problem is solved and you stop. In the experiments, MCTS shows an improvement over BFS+NN, which is definitely interesting, but this doesn't account for the fact that BFS is more than twice as fast (due to searching the tree in order?). If the constraint was wall-clock time, would MCTS still outperform BFS? I also wondered whether the MCTS policy described in Equation 1 performs better than simply sampling from the policy network output. 



review_title: Interesting application of reinforcement learning

Decision: Accept