Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This article introduces a reinforced genetic algorithm to tackle the problem of structure-based drug design, one of the key approaches to achieve fast and efficient drug discovery. The authors overcome limitations of current genetic algorithms, which is one of the most efficient screening procedures for binding affinity, by reformulating it as a Markov decision process, using neural networks to make informed decisions and suppress the random-walk behavior. 
strengths_and_weaknesses: This article is well written and provides meaningful theoretical advances to a key pharmaceutical science problem. It provides thorough comparisons with current existing methods.
questions: In the diversity metric, the RGA algorithm introduced in this paper provides somewhat worse results than competing methods and even brute force screening. Could the authors comment on the importance of this metric and whether some improvements could be made in this respect?

limitations: While this work provides a useful approach to prioritizing promising drug design steps, the pathway to an implementation in real life drug design cycle remains unclear.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a genetic algorithm (mostly based on Autogrow) guided by reinforcement learning. The RL agent in this method can directly perceive the 3D structure of protein targets. In contrast, Autogrow searches molecules in a ``blind’’ way, unable to directly see the 3D structure during sampling.

In this work, RL is used to guide two processes in the genetic algorithm: crossover and mutation. The crossover agent selects parents and mutation for producing ``children’’ molecules. The mutation agent selects chemical reaction to modify the molecules.

This paper presented a very comprehensive evaluation of the proposed method and existing SBDD methods. The improvement over of the method over previous ones is evidenced by the comprehensive experiment, though it is not very significant compared to traditional genetic algorithm.
strengths_and_weaknesses: Strengths

(1) Using RL to empower GA-based SBDD is a good idea. It can make the GA process more directed and uses more information such as protein 3D structures.

(2) Extremely comprehensive survey and evaluation of previous SBDD methods. The survey and evaluation cover a wide range of methods including graph-based, 3D-based, and GA-based.

(3) Acceptable performance improvement. Though the improvement in performance is over GA-based algorithms is not that significant, it surpasses GNN-based and 3D-based methods by a very significant margin.

Weakness

(1) Bias in evaluation metric. The algorithm optimizes Vina score, it is not surprising that the Vina score is good in evaluation. Have the author considered using different scoring functions for GA and testing?

(2) Few examples of generation. I did not find any presentation of generated molecules in the main text or in the supplementary material. Examples are important and their comparison to a ``ground truth’’ molecules are important to understand and evaluate what the model generates.
questions: See strengths and weakness.
limitations: See strengths and weakness.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper proposes a Reinforced Genetic Algorithm that combines different models for structure-based drug design. The algorithm takes the 3D structure of the targets and ligands as inputs and is pre-trained using native complex structures to utilize the knowledge of the shared binding physics from different targets and then fine-tuned during optimization. Empirical studies show that the proposed method outperforms the baselines and the ablation study suggests the contribution of two training strategies. 
strengths_and_weaknesses: Strength: The paper proposes a new method pipeline to solve an existing drug design problem.

Weakness: There is very little technical innovation, more like a direct application of existing methods to a specific problem. Regarding the model design, there is no explanation of the rationalities. For example, why did the authors combine the MDP with a EE neural network? There is also no explanation of the theoretical support for convergence or a better performance guarantee. In addition, the method is heavily adapted to one specific problem and can’t be generalized to a wide range of problems. The experiments are not representative either. 

questions: I would suggest the author focus on one aspect of the method instead of considering multiple model combinations. This creates confusion on which part is more important than others. 
limitations: See above
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This authors of  this submission propose to apply structure based drug design information to the selection, crossover, and mutation operations of evolutionary computation for drug discovery. An equivariance neural network is used to represent target-ligand binding complex and used as a base network for crossover and mutation action selections. In these evolutionary operations, substructure similarity was considered in crossover; and mutation was inspired by chemical reactions. This proposed method is a hybrid model combining deep reinforcement learning and evolutionary optimization for drug design. Experimental results show that the proposed method outperform existing benchmarks. Ablation studies show that the incorporation of structural information in crossover and mutation can improve performance in terms of several metrics.  
strengths_and_weaknesses: Strengths:
1. The consideration of structural information through policy networks for selection, crossover, and mutation are the major contribution of this work from my viewpoint. The combination of reinforcement learning (RL) and evolutionary computation (EC) may be novel for this application. However, the authors should discuss more about existing work that combines RL and EC in general. 
2. Experimental benchmarkings are thorough. Various well-known existing models (modified to integrate docking) were compared. 

Weaknesses: 
1. The regression performance of the ENNs was not discussed. The performance of all other operations depends on the reliability of such ENNs networks. Thus, the authors must show their regression results in pretraining ENNs.
2. Evolutionary operations were largely influenced by Autogrow 4.0. The major difference is that policy networks were employed. The base neural networks used in this work are existing architectures, which weakens the novelty of this work. The signifiance of the result obtained by the proposed work (which is a complicated process) in comparison with other simpler GAs should be further studies, in terms of p-value. 
3. The presentation of this paper could be improved. Line 230: is X a mutation child? a MLP -> an MLP. structured based -> structure based. References should be reformatted. Please pay attention of capitalizations. 
questions: 1. I would like to see the performance of binding affinity regression in ENNs. 
2. I would like to see more discussions of hybrid models, such as RL+EC, DL+EC, etc. 
3. Significance studies, in terms of p-values, should be reported to compare the proposed work with existing (naive) GA algorithms for drug design. 
4. Scalability of this proposed model should be explicitly discussed as well. 
limitations: Limitations and potential improvements of the work were slightly discussed in the conclusion section.
review_title: null

Meta-review: metareview: This paper proposes a genetic algorithm guided by reinforcement learning to design molecules. The RL agent in this method can directly perceive the 3D structure of protein targets. This paper presented a very comprehensive evaluation of the proposed method and existing SBDD methods. The improvement over of the method over previous ones is evidenced by the comprehensive experiment, though it is not very significant compared to traditional genetic algorithm.
Authors may want analyze the biases and show a few examples of generated molecules to better illustrate the effectiveness. 
confidence: Certain
Decision: Accept