Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors present a method for iterative small molecule generation based on an autoencoder framework with graph neural networks. The method specifically focuses on the ability to extend molecular scaffolds (predefined subparts of a molecule) with structural motifs and individual atoms. The authors show results on unconstrained molecular optimization tasks as well as tasks in which a scaffold is given. 
main_review: The authors do a great job at presenting the general challenge they target, their method, and the design choices. The paper is throughout very well-written and clear. 

The presented method is very similar to work from Jin et al. (2018, 2020). The authors explicitly acknowledge this on multiple occasions and point out why the differences matter. To strengthen this point, I would welcome if the authors could elaborate a bit more on the quantitative comparison in in Figure 2. The performance of their presented method increases with the size of the vocabulary. Is this not an expected observation? There are single dashed lines for HierVAE and JT-VAE. What equivalent vocabulary sizes do these correspond to?
The authors also state that "Due to the high cost of training JT-VAE and HierVAE, we did not tune their hyperparameters and instead used the default values." This also seems to handicap these methods. 

As a minor point, it would be nice if the authors could clarify whether the numbers shown in Table 1 are for the original baseline implementations or whether these already account for the code modifications that the authors made . 

I was also curious about the phrase "After each choice, if the currently selected atom is part of a motif, we add the entire motif into the partial graph at once." in relation to Algorithm 2. As we move to larger vocabulary sizes, are there any individual atoms added? It would be great if these authors could elaborate on this in relation to the vocabulary sizes shown in Figure 2. 
summary_of_the_review: The authors explicitly address the similarity of their work to prior contributions from Jing et al. How significant the differences are hinges in part on the quantitative comparisons. I would welcome if the authors could address my questions with respect to Figure 2 and potentially include additional comparisons.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper study the problem of fragment-based molecule generation.  They propose a model MoLeR which consists of an encoder of molecular graph using Graph convolutional neural network (two GNN, one for complete molecules and one for partial molecules), a MLP decoder layer to predict the fragment, the attaching atom on the fragment, and the bonding atoms on the partial molecule. To train the model, it includes three losses for multi-task learning: a KL term between variational posterior and prior of latent representation, a self-reconstruction loss, and a property prediction MSE loss. 
The paper conduct experiments on GulcaMol and show that it is able to generate molecules similar to the training molecule distribution, from scratch or from a given scaffold. It shows better results than LSTM, JTVAE, CDDD-MCTS, etc. 

main_review: Strengths of the paper: 
1.  The construction of the training objective for the model is reasonable. 
2. The proposed method could directly start with a scaffold, which is usually used in practical drug molecule discovery. 
3. The experiments show the MoLeR could generate high quality molecules comparison to a few prior methods. But there are still concerns about results. (see below)
4. The revised fragment vocabulary construction is novel and seems working well. 
5. The experiment also show that generation order and vocab size is important to the quality.

Weakness of the paper:
1. The organization of the paper could be better. Specification of the model should be placed in the main paper, instead in the appendix. Now much is missing in the main method, making it difficult to understand.  
2. The model using GNN+MLP is not novel. Using GNN+MLP to predict fragment, attaching atom, bond has appeared before.
3. Metrics used in the experiment is unclear (what is score and quality in Table 2 and how they are evaluated? Is the score logP or score produced by RDKit? A bit confusing). 
4. The unconstrained generation and comparison to the original data is not solid to evaluate the generation capability of the model. It just evaluates the reconstruction capability. It could only serve as a validation purpose for whether the model can fit data. Figure 2 left shows the distance between generated molecules and the training set. But being able to generate similar molecules is not the goal, it would be more important to evaluate the model’s capability on generating novel molecules that are different from those in the training while satisfying desired properties. 
5. Certain important baseline could be compared in the experiments, including Transformer with SMILES ([1], since LSTM is a weak baseline for generation), MARS ([2] also fragment-based approach using GNN, and can start with a scaffold and multi-property optimization). 

[1] Transformer neural network for protein-specific de novo drug generation as a machine translation problem. 2018.
[2] MARS: Markov Molecular Sampling for Multi-objective Drug Discovery. 2021.
summary_of_the_review: Reasons to accept: reasonable designed training objective for the application of drug design, good generation quality, new method of constructing molecule fragment vocabulary.

Reason to reject: unclear description of method, limited novelty of model, questionable metric used in the evaluation, missing baselines.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work proposes a generative model for molecules. The approach uses a library of motifs, extracted from the training data by breaking down molecules through acyclic bonds adjacent to a cycle. The model can also sample individual atoms. The method is set up in an autoencoder fashion with a GCN encoder that learns node and edge embeddings that are projected into a vectorial latent space. The nodes are initialized with features representing which motif they belong to, so the latent space is motif-aware. The generative approach from the latent space is reminiscent of autoregressive models, but it does not marginalize over the full sequential set of steps, and only through single-step transformations. Seeded with a starting atom or motif, the model sequentially samples the motif (or atom) to be connected and which atoms will be connected. Because more than one path can connect an initial and a final product, it is important to train over multiple paths
main_review: Among the strengths , i identify
- that the model is practical and pragmatic. It strikes interesting balance of flexibility vs. first-principles, where some technical innovations are put at the service of the task/ 
The "multi-hot objective" (pg 3) and  "GMM posterior" (pg 6) are both interesting and have not been applied frequently in molecule generative models. I find both compelling and helpful. I suspect they can also be applicable to other related works in the future. 
- scaffold-based generation is a meaningful problem in molecular design and the existing tools are not great. The authors text in page 9 about SMILES, graph and junction-tree models is compelling, and their approach does side step all these issues while substracting computational complexity 

On the weakness, 
- Still not order invariant. These canonical ordering procedures are a little arbitrary, and "frontier" is degenerate. This is a fundamental issue that cannot be easily addressed, and the authors assess its impacts quite thoroughly. It is interesting, and further proof that permutation invariance is desirable, that the rank in performance in unconstrained generation is opposite to scaffold-constrained generation.
- Pg 5 "we did not tune their hyperparameters and instead used the default values" This is likely an important reason why the baseline models underperform, since it's not an apples to apples comparison
- except for the benchmarks (see above) I don't see any obvious incremental ways to build upon this class of models. I wonder if the authors could provide a roadmap for high-risk high-reward changes they could propose? Or perhaps this is a terminal approach without much obvious room for incremental improvement.


Some questions that come to mind is

Pg 3 "In each step, it first selects a new atom or entire motif to add to the current partial molecule, or to stop the generation." It is possible for the model to select an atom that is incompatible because there are no more active positions to connect to? That is, if the current M does not have any -H (which are the implicit reactive sites) will it know to stop? 

Is there a path (and how likely is it to happen) for a motif to be reconstructed atom by atom even if it is in the vocabulary? Are there any such cases in the training data?

summary_of_the_review: I find the work overall convincing. It takes a pragmatic approach that uses relatively sophisticated (but not wildly innovative tools) to improve one obvious flaw in motif-based models which is their inability to invent new motifs. I find the addition of per-atom sampling a convincing escape route for that failure mode.

The results are relatively convincing in the innovation is practically useful, although it is hard to quantify the size of the gains, because hyperparameter optimization strategies were different for the new method and the baselines. Overall, I think it is acceptable and I would be interested in using a model like this in the practice.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a graph-based generative model for molecule generation. The proposed framework MoLeR can use scaffolds as the initial seed to incrementlly generate molecules motif by motif or atom by atom so that the generated molecules consist of the specified scaffold. Experiments show the proposed method performs comparably to existing methods on unconstrained molecular optimization tasks and outperforms these methods on scaffold-based tasks. At the same time, the proposed method is more efficient due to that it is not conditioned on the generation history.
main_review: Strengths:

This paper proposes a flexible framework that can use motifs to generate molecules, as well as support atom-by-atom generation. Using a scaffold as the initial seed can make sure the generated molecules contain the specified scaffold. The decoder is not conditioned on the generation history and it conditions only on the hidden vector and the current partial molecule. This design strategy makes the proposed method can be trained using all generation steps in the same batch.

Experimental evaluation is comprehensive. The authors conduct extensive experiments in the unconstrained generation, scaffold-constrained generation, unconstrained optimization, and scaffold-constrained optimization to demonstrate the effectiveness of the proposed method. They also have ablation studies to verify the effectiveness of different components. The proposed method performs comparably to baselines on unconstrained molecular optimization tasks and outperforms them on scaffold-based tasks.

Weaknesses:

Baseline methods are not comprehensive. Some important graph-based [1] or fragment-based [2] molecule generations are not included in the experimental comparison. Especially for the training and sampling speed evaluation, the proposed method needs to embed the partial molecule graph at every generation step, which seems inefficient and slow. Are the authors aware of other methods that do not need to embed the partial graph at every step?

Important related works are not included. [3] works on a different problem but also generates molecules using a similar atom-by-atom method. Particularly, [3] has also investigated different strategies to determine the generation order, which is closely related to this work. The authors mention that it is necessary to use a small weight for the KL loss and increase its weight from 0 gradually to make the VAE model training stable. This strategy has been extensively studied in [4] which investigates the posterior collapse problem of VAE for molecule sequence generation. Could the authors include the closely related work [3, 4] for references?



[1] Shi, Chence, et al. "Graphaf: a flow-based autoregressive model for molecular graph generation." arXiv preprint arXiv:2001.09382 (2020).

[2] Podda, Marco, Davide Bacciu, and Alessio Micheli. "A deep generative model for fragment-based molecule generation." International Conference on Artificial Intelligence and Statistics. PMLR, 2020.

[3] Sacha, Mikołaj, et al. "Molecule edit graph attention network: modeling chemical reactions as sequences of graph edits." Journal of Chemical Information and Modeling 61.7 (2021): 3273-3284.

[4] Yan, Chaochao, et al. "Re-balancing variational autoencoder loss for molecule sequence generation." Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. 2020.

summary_of_the_review: The proposed molecule graph generation method is flexible and achieves good experimentals results.
review_title: null

Meta-review: Most reviewers were positive about the paper, seeing that the proposed method is practical and has convincing experimental performances. One reviewer was a bit negative and raised questions about clarity. After the authors responded, the negative reviewer didn't respond further. After reviewing all the comments, the AC feels that there is enough support from reviewers to accept this paper.
Decision: Accept (Poster)