Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a new 3D molecule-generating model conditioned on a receptor. It generates a ligand molecule fragment-by-fragment, unlike other similar models that generate a molecule atom-by-atom. They first extract a set of motifs serving as building blocks of generation. Then context encoding, focal motif selection, next motif prediction and attachment, angle prediction, and refinement are followed. The experiment results show that it outperforms all the baselines in almost all metrics. Substructure analysis shows that it better captures the distributions of the dataset compared to other baselines.
strength_and_weaknesses: [Strength]
* The overall framework is not only innovative but also effectively driven by the functional requirements of each step. For example, rotation angle prediction and structure refinement through force field are plausible and shown to be effective by the ablation study.
* The paper is easy to read and well organized.


[Weaknesses]
* 3hwf case in Figure 3: All generated molecules have lower SA scores than the reference molecule. Is there any chance that the proposed model will produce molecules that cannot be synthesized?
* Minor typo - Page 6. "we fix the the coordinates" -> "we fix the coordinates"

clarity,_quality,_novelty_and_reproducibility: * The paper is clear to understand and the novel parts are not only significant but also effective.
summary_of_the_review: Since the model is novel and effective, this paper will be a good addition to ICLR.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper tackles the problem of structure-based drug design. The authors propose a framework, called FLAG, where the molecules are generated fragment-by-fragment. The authors first reconstructed their motif vocabulary and then generate molecules while considering interactions with target atoms. The authors conduct various experiments on the benchmarks of structure-based molecular generation and achieved performance improvements on some metrics.
strength_and_weaknesses: Positives:  The author provides a clear and complete formulation of the problem and conducts various experiments to demonstrate the effectiveness of his design. As the experimental results of the paper suggest, fragment-based molecular generation does yield more rational structures.

Negatives: The technical contributions of this work are somewhat trivial. Motif vocabulary construction is a common strategy and has been well-explored in the field of drug design, and both predicting the torsion angle and selecting the focal motif are not proposed for the first time.  The interaction with the target was designed similarly to the previous method.

Questions:
1. Could the authors explain why the results in Tables 1, 2 in this paper are not consistent with those reported in other benchmarks like Pocket2Mol?
2. Could the authors provide more ablation experiments to demonstrate the effect of different parts of the FLAG on Table 2 results?
3. If limiting the size of molecule generation is dependent on the interaction with the target, would expanding the size of the pocket affect the quality of molecule generation?

clarity,_quality,_novelty_and_reproducibility: Clarity: Good. The paper is overall well-written and organized. 

Quality: Okay.

Novelty: Medium. The proposed method is based on existing methods. The main contribution is to combine motif-based generation with target-based generation. The synergy of torsion angle prediction is interesting but not novel.

Reproducibility: N/A, the code has not been released.

summary_of_the_review: Overall, this work is borderline. The authors did a good job of combining motif-based and target-based approaches, and there are some performance improvements. However, the technical and theoretical contributions are insufficient. The ablation studies are not solid enough. Thus, I vote for a borderline rejection but may change the score during rebuttal.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In order to design ligand molecules that bind to specific protein binding sites, the authors propose a Fragment-based LigAnd Generation framework (FLAG), to generate 3D molecules with valid and realistic substructures fragment-by-fragment. A motif vocabulary is constructed by extracting common molecular fragments (i.e., motif) in the dataset. At each generation step, a 3D graph neural network is first employed to encode the intermediate context information. Then, the model selects the focal motif, predicts the next motif type, and attaches the new motif. The bond lengths/angles can be quickly and accurately determined by cheminformatics tools. Finally, the molecular geometry is further adjusted according to the predicted rotation angle and the structure refinement. In their conducted experiments, results shows that their method is able to generate diverse drug-like molecules with good binding affinity to target proteins, is faster than most of the baseline methods at sampling new molecules and outperforms baselines by a large margin on generating valid molecules with realistic substructures. 
strength_and_weaknesses: Strength:
1.	It is an encouraging attempt since it is a quite difficult task to generate target-based molecules in 3D space.
2.	This method is of fragment-based molecular generation, which is in accord with functional groups in organic chemistry.
3.	Since the assembling process of the first motif and rotation angle prediction are subtly hard to deal with, this paper provides a tentative solution.


Weakness:
1.	The paper does not conduct the experiment or provide sufficient evidence about target specificity of the FLAG model, i.e. the comparison between the model with target information and the model without target information. It is possible that FLAG model just fits the structural distribution of the whole ligand library rather than the distribution conditioned on the specific target. 
2.	The authors do not illustrate key differences between this FLAG model and other fragment-based methods, such as “Fragment-Based Ligand Generation Guided By Geometric Deep Learning On Protein-Ligand Structure” and “Scalable Fragment-Based 3D Molecular Design with Reinforcement Learning”. Besides, the authors say, “However, in these works, starting fragments are required and the motif vocabulary needs to be constructed by experts, limiting their practical applications.” However, introducing experts’ knowledge to generate drug-like molecules in order to avoid uncommon or toxic fragments is quite reasonable in drug design. This statement is invalidated.
3.	As for the motif extraction step, there are some doubts when it comes to find all the rotatable bonds. Rotatable bonds are usually just single bonds in organic chemistry. If rotatable bonds are under the condition that “where either u or v is part of a ring”, how does this model to deal with the situation that several single bonds are connected end-to-end?
4.	The “Motif Attachments Enumeration” way is quite similar to JT-VAE.
5.	It is not clearly demonstrated when the authors introduced the motivation of RBF in the “Encoder” part.
6.	In the “Structure Refinement”, it is not well illustrated that “the force between atoms” proposed by authors has true physical meaning or not.

clarity,_quality,_novelty_and_reproducibility: The paper is well written and organized clearly. Therefore, the clarity and the quality of this paper is good. However, the authors do not illustrate key differences between this FLAG model and other fragment-based methods. Therefore, the novelty may be not strong enough. And the reproducibility is not very reliable and the contribution to the drug design community is limited since the authors do not provide code implementations of this model. 
summary_of_the_review: In order to design ligand molecules that bind to specific protein binding sites, the authors propose a Fragment-based LigAnd Generation framework to generate 3D molecules with valid and realistic substructures fragment-by-fragment. Some design ways of 3D molecule generation are tentative and fairly inspiring. The paper is well written and organized clearly but the novelty may be not strong enough since the authors do not illustrate key differences between this FLAG model and other fragment-based methods. And the contribution to the drug design community is limited since the authors do not provide code implementations of this model.

After reading the responses, I raise my score a bit to 6.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a model for generating molecules (represented as a 3D structure) that bind to a specific protein site (also represented in 3D). The authors propose a fragment-by-fragment sequential approach, which is summarized in the following steps:

- substructure (or motif) vocabulary creation 
- context encoding (the context is initially just the binding site, then it's binding site + existing molecule)
- focal motif selection (i.e. selecting the initial motif / the motif used to expand the existing molecule)
- next motif prediction (i.e. selecting the motif to attach from the vocabulary
- motif attachment and orientation
- substructure refinement

In the experiments, the authors show that the model generates molecules with high binding affinity, sometimes even better than the reference, while also possessing high drug-likeness. 
strength_and_weaknesses: **Strengths**

- Impressive experimental results
- Substructure refinement appears innovative and its importance is confirmed through ablation studies

**Weaknesses**
- not reproducible

clarity,_quality,_novelty_and_reproducibility: **Clarity**

The work is well-written and easy to follow. However, there are sections where I couldn't properly understand what has been done from the text alone. As for the writing style, some symbols are undefined, some occasional typos, and improper use of citations, but nothing of real concern. See below for details.

**Quality**

Overall, the work is medium-to-high quality. The modeling is fairly complicated, but I'd expect that since we are dealing with a very difficult task. The evaluation in particular is done with proper care (i.e. comparing to the baselines trained with the same data splits, which is not to be taken for granted in this field) and the results are convincing.

**Novelty**

The work reuses known concepts and modeling tools and puts them together in an innovative manner (at least, to my knowledge).

**Reproducibility**

The work is **not** reproducible. The code is not provided and the authors state that it will be released upon acceptance.
summary_of_the_review: In general, I find this paper a solid contribution, probably worthy of ICLR. The most disappointing part is that I cannot look at the code, which to me makes it irreproducible. 

Besides this, I have several questions I'd like the authors to answer, and some comments/suggestions/corrections.
Please acknowledge that I am not an expert in structure-based drug design (though I know the general problem and main approaches).

In no particular order:
- What is the vocabulary size? Do 2D-isomorphic motifs with different geometry get encoded as two different "words" in the vocabulary?
- You say that after pruning chemically invalid molecules and _merging isomorphic graphs_ you reduce the set of candidates attachments. How do you handle merging isomorphic graphs, considering that it can be computationally expensive in certain cases?
- What are the valid, novel, and unique fractions of the molecules generated for the test set?
- Can you show examples where the model fails to generate molecules that bind to a given target (as in Fig. 3)?
- Why did you use QVina to evaluate the Vina scores, instead of more recent alternatives (GNina)? 
- You state that all the generated molecules were further refined with universal force fields before being fed to Vina. Did the molecules generated with the baselines get the same treatment?
- Why did you use GIN as a scoring function for candidate attachments?
- At times, the notation is confusing. For example, in Eq. 4), the $\mathrm{MLP}$ function has two arguments, while in Eq. 9 it has three. I thought that would mean concatenation, but then when in the equation where you define the distance matrix (which by the way is not numbered) the $\mathrm{MLP}^d$ uses $concat$ to describe a concatenated input. Please correct or explain.
- you say that $\boldsymbol{e}_{\mathrm{RBF}}$ can be either a Gaussian function or a spherical Bessel function. Which is the actual used in this work?
- you should use \citep instead of \cite for the citations.
- there is a missing closing parenthesis in Eq. 10

Finally, I think there are two citations that should be considered in Section 2 when you talk about motif-based generation.
- Podda et al. A Deep Generative Model for Fragment-Based Molecule Generation. AISTATS 2020
- Seo et al., Fragment-based molecular generative model with high generalization ability and synthetic accessibility. https://arxiv.org/abs/2111.12907

**Overall judgment**

I am giving a score of 6 for the moment being, will wait for the authors' replies to see whether there's space to increase my score.

**EDIT**

Since the authors kindly responded to all my questions and most of the questions raised by the other reviewers, I am changing my vote to an 8.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: 
The goal is to create a 3D molecular binder to a given protein (pocket). The binder is created iteratively, one motif at a time. The key steps include protein and intermediate molecule encoding, focal motif selection, prediction of the new motif to add and how it is attached, torsion angle update, as well as an interleaving predicted-force guided 3D coordinate refinement step. The architecture is trained with masking and based on noise perturbed coordinates. The experiments are carried out relative to a subset of cross-docked dataset. Technical novelty is low since many of the steps and ideas are taken fairly directly with minor variations from previous papers though put together cleanly for the designed purpose here. Other related motif-fragment methods were available prior to this work but comparisons could be hampered by lack of code release. Overall, this is a well-engineered and executed solution that works well in comparison to (limited) baselines. 

justification_for_why_not_higher_score: 
Technical novelty is somewhat low since many of the steps and ideas are taken fairly directly with minor variations from previous papers. Similar approaches have been proposed before though comparison would be challenging due to lack of code availability.
justification_for_why_not_lower_score: 
This is a well-engineered and executed solution that works well in comparison to (limited) baselines.
Decision: Accept: poster