Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: A controllable 3D molecule generation framework named GraohVF is proposed, in particular, the proposed GraohVF is based on variational flow and integrates geometrical and skeletal restraints for protein target generation. 

strength_and_weaknesses: Strength：

(1) Incorporating geometrical and skeletal restraints for controllable generation is novel and interesting.

(2) The experiment results demonstrate the proposed method achieves superior performance than the state-of-the-art methods. 

Weaknesses:

(1) The writing of the paper is obscure. There are many notations used without explanation. The technique part of this paper is very confusing. Some equations about network structure can be put into the appendix. 

(2) The technique contribution of this method is not clear. It seems the core component of the method is same as existing methods, but the difference is not highlighted.

(3) It seems that contributions 1 & 2 are the same to me. More elaborations about this are needed if I am wrong. 

(4) What do graph clustering and tree encoding refer to? The construction of the mean and variance $\mu_{\mathcal{R}}, \Sigma_{\mathcal{R}}$ for target proteins is very important for the whole method but is less discussed. 

(5) From Eq.14, Eq.15, and Eq.16, it seems the mean and variance $\mu_{\mathcal{R}}, \Sigma_{\mathcal{R}}$ about target proteins are fixed while optimizing the variational flow. If this is the case, the novelty of this paper would be straightforward. 

(6) It seems the whole method is highly correlated to GraphBP, thus the technique difference of these two methods should be discussed. 
clarity,_quality,_novelty_and_reproducibility: Clarity & Quality: The paper is hard to follow. Maybe the figure description of the whole structure will be helpful. The notations and equations are obscure. 

Novelty: Marginal. 

Reproducibility: Difficult, the method involves too many components and most of them are less discussed. 
summary_of_the_review: Overall, this paper is hard to follow due to poor organization. Although the motivation for incorporating geometrical and skeletal restraints for controllable generation is novel and interesting, the proposed method is not technically sound. This paper should be highly polished to highlight the technique contributions and the connection of different components.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper proposes to use generation flow to generate molecules with the control of 3D structure and 2D constraints and claims to outperform other methods.
strength_and_weaknesses: ## Major comments

1. In general, the idea is simple and easy to implement. It is a combination of JT-VAE, SchNet, Chemprop, and variational flow without much novelty.
2. There are many evaluation metrics to show, e.g. QED, SA, LogP, but only high affinity is shown in Table 2. The claim is not convincing.
3. In evaluation, the number of potential similar or duplicate molecules is not investigated. Also, the reported Pocket2Mol result is much lower than that in the original paper, which should be explained.
clarity,_quality,_novelty_and_reproducibility: It's clearly unfinished work. 
summary_of_the_review: Overall, the idae is interesting for molecule generation. However, it's clearly unfinished work and should have more comprehensive experiments to support the claim. With just one evaluation metric, the results can be questioned by the readers as cherry-picked.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes GraphVF, a novel method for generating 3D ligand molecules that can bind to specific target proteins. GraphVF adopts junction tree based scaffold encoding and SE(3)-invariant geometry encoding during training, and a variational flow framework to sequentially generate ligands atom-by-atom. GraphVF achieves good performance in ligand generation and drug property targeted generation tasks.
strength_and_weaknesses: Strengths:

(+) This is the work that first considers generating molecules with good drug properties in ligand generation, which is very meaningful and useful for practical applications.

(+) The effectiveness of the proposed GraphVF method is demonstrated by good performance in various experiments.

Weaknesses:

(-) (major) The paper does not clearly motivate and explain the novelty contribution of GraphVF. The design of the generation process in GraphVF (use flow models to atom-by-atom generate ligands, generate 3D positions by distances, angles, and torsion angles) largely follows the existing method GraphBP [1]. To me, the only novelty contribution is to additionally encode ligand scaffolds with junction-tree based encoders. However, it is not clear why this scaffold encoding is used and important, since the ligand generation process does not include any junction-tree generation. It is strongly recommended to add detailed discussion to clarify the difference between GraphVF and GraphBP, and the importance of using scaffold encoding.

(-) (major) It is not clear why a variational-inference-style flow model is used. As described in section 4.3 and figure 3, during training, the flow models output latent variables subjected to the Gaussian distribution $\mathcal{N}(\mu_R,\Sigma_R)$, but in loss function (equation 14-16) the latent distribution is optimized to be close to the standard Gaussian distribution, and the generation process with flow models also sample from the standard Gaussian distribution. Then what is the advantage of this method over directly modeling the outputted latent variables as the standard Gaussian distribution during training? Is it specially designed for controlled generation targeted at good drug properties? It is better to provide more detailed explanation or clarification.

(-) (major) At each generation step of GraphVF, the bonds between the new atom and previously generated atoms are generated. However, prior 3D molecule generation methods (e.g., EDM [2] and GraphBP) do not explicitly generate bonds, but infer bonds from the generated 3D conformations, and they evaluate the bond valency validity. Hence, why GraphVF does not follow this setting? Do the generated bonds match those inferred from 3D conformations? And if possible, could authors report the bond valency validity in experiments?

(-) (minor) Tables and figures in the paper do not follow some frequently used style habits in academic papers. It is better to make Figure 1, 2, 3 at the top of the page like Figure 4, make caption of Table 2 before the table, and use three-line (with \toprule and \bottomrule at the beginning and end of the table) for Table 2, 3, 4.

(-) (minor) The description about EDM in the first paragraph of section 2 is incorrect. EDM generates 3D molecular conformations from scratch, not from 2D structures.

(-) (minor) In section 3.2, use $b_i^{lig}\in\mathbb{Z}^{i-1}$ instead of $|b_i^{lig}|=i-1$ to represent an integer vector.

[1] Liu, Meng, et al. "Generating 3D Molecules for Target Protein Binding." International Conference on Machine Learning. PMLR, 2022.

[2] Hoogeboom, Emiel, et al. "Equivariant diffusion for molecule generation in 3d." International Conference on Machine Learning. PMLR, 2022.
clarity,_quality,_novelty_and_reproducibility: The details about scaffold encoding needs to be better clarified. The novelty and quality of this paper is a little bit limited because the significant part of GraphVF, i.e., ligand generation process, largely follows the existing method GraphBP. Additional details such as hyperparameters are needed to increase the reproducibility.
summary_of_the_review: Though this paper presents an interesting method for controllable ligand generation and achieves good experimental performance, it has major issues concerning the motivation of using scaffold encoding, flow modeling, and bond generation. Hence, I vote for rejection and encourage authors to solve the critical issues in the revision.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: A generative model generates molecules conditioned on a protein pocket, and optionally also conditioned on a reference molecule or some desired substructures. Atom types, bonds, and atom positions are generated one at a time using an autoregressive flow model.
strength_and_weaknesses: The key novelty here seems to be the idea of conditioning the generation of new ligands on the coarse-grained structure of a reference ligand, or on presence of some desired substructures, by sampling the latent z from a distribution that depends on the reference ligand or on the desired substructures. This is a good idea, but the implementation is not clearly explained, and the evaluation is somewhat lacking.


clarity,_quality,_novelty_and_reproducibility: I found it hard to follow the description of how the model is trained. How is the loss (14) used to update parameters of the MLPs that generate atom types and bond types? (Hoogeboom et al. https://proceedings.neurips.cc/paper/2021/file/67d96d458abdef21792e6d8e590244e7-Paper.pdf seems relevant) How are reference ligands associated with training examples? Section 4 may be easier to read if the generation process is described first, and then the training procedure.

The evaluation metrics are not clearly explained (e.g. table 2). They do not assess whether generated molecules have desirable properties other than binding affinity, e.g. validity, stability, synthesisability, novelty.

I do not understand the motivation for equation (32). Would it not make more sense to sample z from a mixture of the gaussians on the right-hand side of the equation?

Code appears not to be provided.
summary_of_the_review: There is a good idea here, but the description of the implementation is hard to understand, and the evaluation is superficial.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies 3D molecule generation conditional binding proteins. All reviewers are concerned with many aspects of this paper, including methods and innovations, writing, and experiments. Thus a reject is recommended.
justification_for_why_not_higher_score: All reviewers are concerned with many aspects of this paper, including methods and innovations, writing, and experiments. Thus a reject is recommended.
justification_for_why_not_lower_score: NA
Decision: Reject