Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a new method for molecular consisting of a principle subgraph extraction step, a fragment prediction step, and an assembling step to combine the fragments and obtain the final generated graph. The major novelty is the extraction of the so-called “principle subgraph” from the dataset. The two-step molecule generation process is based on VAE and relies on the vocabulary of fragments (principle subgraphs) that is built in the first step. Experiments on distribution-learning, (constrained) property optimization, and GuacaMol goal-directed benchmarks show the superiority of the proposed PS-VAE method.
strengths_and_weaknesses: Strengths:
1. The proposed method is relatively novel, considering that it is the first work to use frequent subgraphs in the molecule generation domain. Such use of subgraphs is a promising direction and likely to spark future research in this task and other related tasks. 
2. In the appendix, careful hyperparameter selection for N, the number of fragments in the vocabulary, is performed using a curve of entropy-sparsity trade-off. Complexity analysis, runtime cost, and additional details on the method, the experiments, and the experimental results are shown too in the appendix.
3. The paper is well-written, with ample examples and illustrations to facilitate the understanding of the various proposed components.

Weaknesses:
1. Since the proposed method consistently outperforms baseline methods, it is important to know which component provides the most useful information. The authors have done a reasonably good job of ablation studies, e.g. by replacing the built vocabulary with other models’ vocabulary. In Section B, the authors talk about the trade-off with the tuning of N, the number of fragments, but it is still unclear what final performance PS-VAE would achieve if some other choices of N are adopted, e.g. N = 100, N = 800, etc. This requires running the whole model multiple times with different choices of N, and would clearly show the performance trend and experimentally validate the claimed trade-off.
2. Although the authors answer “Yes” for whether error bars are shown, it seems only Table 6 contains error bars without further explanation why this is the case.
3. In Section F, the authors compare the running time of proposed method with two baseline models, but it would be better to compare with more baselines or at least show the complexity associated with all the methods that are compared across the three tasks in the main paper. So far the authors claim the proposed method “exhibits higher computational efficiency than certain widely-used baselines”, so this is a technically correct statement, albeit leaving readers wonder about the comparison to other methods adopted in this domain.

questions: Please see above.
limitations: The authors put N/A as the response to the potential societal impact in the last page of the main paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a molecular generation model based on the vocabulary which consists of frequently observed subgraphs. Unlike existing subgraph-based molecular generation methods using chemical domain knowledge, the vocabulary is constructed in an autoregressive manner based on the dataset. It produces molecular graphs by generating subgraphs that act as building blocks and predicting the edges between them.
strengths_and_weaknesses: Strengths

- Unlike using predefined substructures such as rings or functional groups, the proposed approach is possible to adaptively extract subgraphs that frequently appear in datasets. Hence, the proposed model uses a vocabulary that reflects the characteristics of the dataset.

- The properties of extracted subgraphs are well analyzed through the theorem.

- The proposed model has high performance compared to baseline methods.

- Some analysis of extracted vocabulary is shown. The fact that high-correlated fragments to PlogP are generated frequently in the PlogP-optimization task strengthens the proposed method. It would have been better to add experimental results for QED or other properties.

Weaknesses and Questions:

- In the experiment part, HierVAE and H-VAE are used as the baseline method. Is there a reason why these methods are chosen as the baseline? Since HierVAE is a model focused on polymer compound production, it would not be the proper baseline for the ZINC250k or QM9 dataset.

- It is unclear why Theorem 3.2 is useful to model molecules. The connection between the theorem and the experimental results are vague and not clearly explained.

- There is an ambiguity in Algorithm 1. The rightmost molecule in Figure 2 (b) has two C-C subgraphs. If the right C-C bond is merged in iteration 1, all subsequent iterations will be changed. In this case, is there a way to determine which fragments to merge?

- The baseline method used in the experiment is insufficient. MARS or FREED is a recent model, but not used in all experiments. Additional experiments compared with GEGL [1], GraphDF [2], and GraphCNF [3] will help to understand the performance of the presented model.

[1] Ahn, Sungsoo, et al. "Guiding deep molecular optimization with genetic exploration." *Advances in neural information processing systems* 33 (2020): 12008-12021.

[2] Luo, Youzhi, Keqiang Yan, and Shuiwang Ji. "GraphDF: A discrete flow model for molecular graph generation." *International Conference on Machine Learning*. PMLR, 2021.

[3] Lippe, Phillip, and Efstratios Gavves. "Categorical normalizing flows via continuous transformations." *arXiv preprint arXiv:2006.09790* (2020).

Minor Errors

- Typo in line 265 (effacy)
- In 164 lines, it is written that <bos> and <eos> token are used, but in Figure 3, <start> and <end> token is shown.
questions: - Although it is forced not to have a ring structure except for a ring with nodes 5 or 6 by the cycle check of Algorithm 3, actual molecular data may have a ring of a different size. Is there a reason to limit the size of cycles?
- What is the advantage of HierVae and FREED in line 298?
- I couldn't find any experimental results supporting the sentence "Otherwise the majority of the benchmarks are easy to be optimized to the upper bound"(line 270) in [42]. 
- In the proposed method, the decomposed subgraphs are not overlapped. This is different from JT-VAE and HierVAE, which generate subgraphs that can overlap with others and find overlapped parts to connect edges. Is there any advantage to generating non-overlap subgraphs?
- Is there any correlation between the extracted substructures and other properties except PlogP?
limitations: If there is no significant correlation between the frequently observed subgraph and the desired property, the influence of vocabulary could be diminished.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a novel approach for molecule generation, introducing principal subgraphs. The method can be divided into two steps: The extraction of principal subgraphs from a training dataset and the molecule generation itself. In the first step, principal subgraphs are extracted from a training set by iteratively merging subgraphs, starting with single vertices. In each iteration, two neighboring subgraphs are merged, if the subgraph corresponding to their union is most frequent in the dataset among the possible merges.  For the molecule generation, first subgraphs and then edges between those subgraphs are predicted.
strengths_and_weaknesses: ## Strengths
  * The paper proposes a new approach to molecule generation, the concept of principal subgraphs is nice. Related approaches are adequately cited.
  * The paper is well structured and written.

## Weaknesses
  * The concept of principal subgraphs is not explored in full detail. It would be nice to formalize the relation to frequent subgraphs. The authors state that principal subgraphs can be computed more efficiently than frequent subgraphs. Does this come with any costs/limitations, e.g., in terms of the completeness of the vocabulary? Moreover, a detailed complexity analysis would be very helpful to support this argument. The complexity analysis in the appendix is not sufficient. It appears to assume that functions like GraphToSMILES have constant running time, which needs justification. This function solves the graph canonization problem, for which no polynomial time algorithm is known in general.
  * The experimental evaluation is not fully convincing, e.g., the used implementation and parameter selection of the competitors are not explained.
 
## Minor comments
  * Typos Line 151: neighbos, line 474: exits
  * There is some mix-up of references, especially when referring to the Appendix, which should be corrected: Figure 6 is in the Appendix and not in the submission, so it should be referenced this way. There is no Figure 5 referenced in the submission (maybe 5 and 6 could be swapped). In Question 3 d) you state that you include the total amount of compute and the type of resources used in Appendix H. They are in Appendix F (maybe they should be moved closer to Appendix H).
  * The presentation of the results and examples could be improved to make them readable in grayscale.
  * PS-VAE is called GP-VAE in Table 1
questions: * Q1) Are the generated molecules chemically valid and how is this guaranteed?
* Q2) Which implementation and parameter selection has been used for the different methods?
* Q3) Couly you provide additional results of the different reference methods?
    - FREED and MRNN in Tables 1, 3 and 4
    - MARS in Tables 1-3
    - JT-VAE, GCPN, GraphAF, and GA in Table 4
limitations: Overall these are addressed adequately. However, the limitations of principle subgraphs (e.g., compared to frequent subgraphs) could be explored in more detail.
review_title: null

Meta-review: metareview: This paper proposes a molecule generation method using frequent subgraphs. There was a positive consensus among reviewers that paper is novel (B6na, EktL) and well-analyzed (B6na, ESyH), and minor suggestions for improved evaluations and presentation (EktL, ESyH) were well-handled during rebuttals to alleviate reviewer concerns.
confidence: Certain
Decision: Accept