Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This is a paper working for molecule generation, especially considering the high-frequency and important subgraphs called graph pieces. 
main_review: The graph piece extraction algorithm is not clear. The iteration 1 highlights the most frequent piece CC in Fig4(b). Then how C=CC in Fig4(c) is searched by merging CC and C? Line 12 of the pseudo code in appendix is not clear about how to do  “mol.merge(piece)”. Line 10 finds only the most frequent piece? Or the two most frequent pieces?  What if these two most ferquent pieces are not connectable in molecules? 
And how expensive the process will be?  
For molecules with rare atoms, their piece-level decomposition will be a set of single atoms?  A pseudo code algorithm will be helpful for understanding.  

In Fig 5, the GNN was applied on decomposed molecule. However, section 3.3. introduces that GNN is applied on the whole graph G, like what has been done in previous GNN-based molecular graph representation. 

Another concern is about the generated results. The proposed model has lower uniqueness than JT-VAE and GCPN, although very close in uniqueness metric values. 

QM9 is another very popularly used dataset for molecule generation evaluation. Evaluation and comparison on this QM9 dataset can strengthen the evaluation results. 

An error to correct:  We try to add bonds which has 

summary_of_the_review: The details of the proposed model were not clearly introduced. The evaluation should be strengthened.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to use substructures instead of atoms (nodes) for generating molecular graphs.
main_review: - strengths: 


1. The idea of using common substructures to generate molecular graphs is novel and original.


2. The source code and dataset provided in supplementary material facilitate good reproducibility of this work. 


3. The authors claim that the proposed method has two benefits: (1) captures correlation; (2) accelerates computation. In my opinion, it seems that the essential benefit is that, using substructure instead of atoms could reduce the search space for this combinatorial problem. Maybe the authors could theoretically analyze this.


- weaknesses:


1. I would like to know the connection and difference between graph pieces and frequent subgraph discovery. Could the authors discuss this? 


2. How to obtain ground truth graph piece sequence, and why?


3. The authors state that: larger N indicates more coarse-grained decomposition. Then, how to determine proper N for different down-stream tasks?


4. The experiment is less convincing. The proposed method is only evaluated on one dataset ZINC250K. In Table 1, JT-VAE achieves perfect results, while the proposed method seems to show limited competitiveness.


5. This work misses one very related baseline [ICML 2018] [GraphRNN] Generating Realistic Graphs with Deep Auto-regressive Models, which also generates graphs in an autoregressive fashion. The authors should discuss the differences between them, and compare the performance in the experiment.


6. Figure 3 and Figure 6 are not mentioned and explained in the main text.


7. The notations are confusing. In Section 3.1, the authors state that: "$\tilde \varepsilon$ contains all the connections between the atoms in different graph pieces". In Section 3.2, the authors state that: "$\tilde \varepsilon$ contains all bonds connecting two atoms in A and B".


8. Algorithm 1 is not clear. What is the process of MolToSmiles? The authors are expected to clearly explain each line of the proposed algorithm.
summary_of_the_review: The idea of this work is reasonable, but some technical details have not been clearly presented.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new molecular graph generation method and empirically shows its effectiveness.
The proposed method first decomposes molecular graphs into smaller parts (called graph pieces), followed by training a variational autoencoder so that it can generate the collected graph pieces. The trained model is expected to generate a variety of graphs with desirable properties.

main_review: ## Strength

Although each step of the proposed strategy is a straightforward application of existing methods, the overall proposed strategy is novel and interesting. In particular, it is interesting that the proposed method uses graph pieces in its training instead of directly using a given collection of graphs, which is the key idea of this paper.

Presentation of this paper is good. The paper is clearly written overall.

## Weakness 

- Empirical evaluation is not thorough. Recently it is widely known that benchmarks such as the penalized logP and QED are not appropriate for evaluation and using Guacamol is highly recommended (see https://pubs.acs.org/doi/abs/10.1021/acs.jcim.8b00839).  So please evaluate the proposal on Guacamol.
- Ablation study is not convincing. An important evaluation is to compare the proposed method and that without the graph piece generation step, that is, directly using a given graph datasets in the training of GNN without decomposing them into graph pieces. This experiment tells us the effectiveness of the graph piece step.
- The proposed graph piece extraction is fully based on sequence representations of graphs, hence it does not treat all the possible subgraphs of molecular graphs. Therefore, there should be various subgraphs that cannot be extracted by the proposed method. I wonder how crucial this loss of information is. Some discussion would be desirable.

I am happy to increase my score if my concerns are addressed in the revision.

summary_of_the_review: The idea of the proposed method is technically interesting, while its empirical evaluation is not sufficient.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The author propose a variational auto encoder for graphs based on an autoregressive model for larger fragments and a GNN for edge prediction to connect the fragments. 
main_review: The manuscript is unclear: how exactly is the piece level sequence generation autoregressive model connected to the bond completion? how is the fine structure (ie the nodes and edges) of each piece generated so that the bond-completion model can work only on nodes across different pieces? The summary offered is too high level and crucial information is missing (also Fig 5 is too generic). 

The properties of the proposed way to generate pieces (fragments) are unclear: what is the size distribution of the fragments generated? It seems that very large fragments would be generated, but then those would be very infrequent (?), how is the tradeoff between frequency and size controlled? Why aren't there any experiments to showcase the sensitivity of the approach to the number of fragments?  What is the computational complexity of the fragment generation phase?
How often are novel fragments appearing as a function of the training set size? What are the consequences of this, with respect to the generalisation/representational capacity of the approach? i.e. the smaller the training sample the fewer the fragments (and vice versa).
Efficiency in this case seems to be in trade off with flexibility and capacity to generate novel substructures: can experiments be devised to evaluate this trade-off in a quantitative way?
Why is the vast literature of graph mining completely absent in the review section? 

summary_of_the_review: The justification for this work is that autoregressive models tend to ignore the existence of common substructures and hence a specific mechanism that is aware of larger fragments is needed. This fact is however never demonstrated and possibly incorrect (that is autoregressive models do not encode explicitly substructures but do this in the latent space given the compression constraints they are subject to). It would be of interest if the paper would analyse this statement empirically or theoretically, but the only results available are the usual weakly informative experiments on validity, uniqueness and novelty, and property optimization experiments (without any notion of statistical significance for the comparative results). 
The presentation is overall unclear and the literature review incomplete.
review_title: null

Meta-review: While the reviewers appreciated the new methodology and presentation of the paper the reviewers were concerned about the experimental section. Specifically they wanted to see optimization outside of penalized logP and QED, which are now viewed by the community as toy molecule optimization tasks (e.g., Penalized logP can always be improved by just adding a longer chain of carbon atoms). The authors responded that this would have taken too long to run Guacamol tasks in the rebuttal phase as all methods would need to be rerun for all tasks, but this is not true: many methods e.g., Ahn et al., 2020, already have reported these results and could be directly compared against (as this paper is near state of the art this would have been a convincing comparison). Another odd thing about the experimental setup is that the authors compare with Ahn et al., 2020 only for constrained property prediction. However Ahn et al., 2020 achieves a Penalized logP of 31.40 whereas the proposed method only achieves 13.95. It's suspicious that this result is missing in Table 2 of the current paper. If the authors are able to improve their work beyond Ahn et al., 2020 and related recent work on Guacamol and othe real-world tasks, the paper will make a much stronger submission.
Decision: Reject