Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper formulates retrosynthesis planning as a reinforcement learning problem and proposes a Goal-dRiven Actor-critic retroSynthetic Planning (GRASP) framework. They propose first to learn a policy network that takes continuous actions encoding the structure-level molecular information to allow navigation in the huge discrete action space of single-step reaction candidates. Then GRASP learns a goal-driven Q-value estimation network to update the policy, by sampling both successful and failed experiences and relabeling the goals of sampled experiences. The learned Q-value estimation and policy networks join to guide the Monte-Carlo Tree Search, after which GRASP returns diverse pathways due to a good exploration-exploitation tradeoff. They evaluated the performance of GRASP on both an academic and an industrial benchmark dataset. The results and user studies demonstrate that GRASP outperforms all baselines in general retrosynthetic planning metrics by a significant margin and is the first to achieve high-quality goal-driven retrosynthetic planning.
strengths_and_weaknesses: The authors successfully adjusted RL to the retrosynthesis plan using the actor-critic RL method to learn from positive and negative experiences and navigate through the single-step reaction space.

They also applied goal-driven relabeling in retrosynthesis by exploiting the data generated from the retrosynthesis policy. This help to overcome the challenge in the quality evaluation of pathways.

The experiments are conducted on multiple datasets, including the Pistachio reaction dataset and a real-world expert dataset ‘WuxiTest’.
Ablation studies are included to investigate the effectiveness of different components.

For experimental comparison, the authors did not include recent related methods [1] as baselines, and still regard Retro* as the state of the art, which makes the significance of the proposed method less.

[1] Han, Peng, et al. "GNN-Retro: Retrosynthetic Planning with Graph Neural Networks." AAAI 2022.
[2] Xie, Shufang, et al. "RetroGraph: Retrosynthetic Planning with Graph Search." KDD 2022.
questions: Could the authors include more recent related methods and properly compare with them?
limitations: No
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a novel multi-step retrosynthetic planning framework called GRASP. The superiority of this reinforcement learning-based framework mainly relies on the capability of capturing synthetic knowledge through self-generated experiences. The paper is a pioneer work performing goal-driven retrosynthetic planning. In practice, the experiments demonstrate the effectiveness and the superiority of GRASP on both pathway datasets and expert datasets. The numerical results show GRASP achieves state-of-the-art performances on multi-step retrosynthetic planning tasks.
strengths_and_weaknesses: Strengths

+ The paper is well-written and well-organized. The notations and figures are easy to follow.
+ The idea is novel and works very well. It is a pioneer work performing goal-driven retrosynthetic planning.
+ The practical results demonstrate that the method could be helpful for chemists.
+ The analysis on the enhancement of GDR is convincing.
+ Codes are provided.

Weaknesses

- The performance depends on an upstream single-step retrosynthesis predictor. It would be better if this predictor could be optimized jointly with the RL-based framework.
- It seems like the expert datasets WuxiTest are not open-source resources.
questions: Lack of ablation studies on different choices of single-step predictor. If it is possible to improve the performance by using a superior single-step predictor like Graph-to-Graph[1], or RetroXpert[2]?

[1] A Graph to Graphs Framework for Retrosynthesis Prediction, ICML 2020
[2] Decompose Retrosynthesis Prediction Like A Chemist, NuerIPS 2020
limitations: The broader impacts of this paper should be discussed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper aims to solve retrosynthetic planning with RL. The authors proposed an actor-critic framework called GRASP, short for Goal-dRiven Actor-critic retroSynthetic Planning. This is claimed to be the first approach that enables goal-conditioned retrosynthetic planning, which mitigates the challenge in quality evaluation of pathways by directly fulfilling the requirements prescribed by chemists. The proposed method also leverage hindsight goal relabeling to speed up learning. They evaluate the performance of GRASP on both an academic and an industrial benchmark dataset (Pistachio and WuxiTest), and show superior results over all baselines. 
strengths_and_weaknesses: Disclaimer: the reviewer has zero background knowledge in retrosynthetic planning and thus is not able check whether the problem setting and evaluation are sensible or not, and whether the claim that "MCTS for retrosynthetic planning induces high-variance" is really a problem that cannot be easily mitigated. To my knowledge, modern approaches that use MCTS with limited rollout length and bootstrap with value network in general don't suffer much from this problem, but I don't know if there is anything specific to retrosynthesis as discussed in Sec. 3.3. I'm inclined to think this claim needs evidence but not sure whether that's obvious enough in this specific setting.

Otherwise, the proposed approach seems sound and mostly outperforms all baselines considered in this work (MCTS, DFPN-E, Retro*, Retro*-0, Hyper-Graph Search) on Pistachio and WuxiTest. The empirical evaluation seems comprehensive and appears to verify most of the main claims:
* Goal-driven planning mitigates the challenge in quality evaluation of pathways by directly fulfilling the requirements prescribed by chemists
* GRASP outperforms all baselines in general retrosynthetic planning metrics


questions: Nitpick:
* Line 189: "Segler et al." is missing in front of [24].
limitations: No discussion on limitations and potential negative societal impact at all.  Please add relevant pieces to the conclusion section.
review_title: null

Meta-review: metareview: This work proposes a goal driven actor-critic method for finding pathways with a specific prescribed goal such as building block materials. Various aspects like self-generation of data, hindsight goal relabeling while not novel by themselves, have found meaningful application on the chemical reaction tasks and the improved results has been appreciated by all reviewers. The authors are encouraged to incorporate reviewer feedback into account and also discuss the concurrent submissions as mentioned in the reviews for the sake of completeness in the camera-ready version.    
confidence: Less certain
Decision: Accept