Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presented a novel fragment (motif) based generative model for molecules. The motif vocabulary is mined from the molecule dataset by iterative merging small motifs into larger ones and keeping the most frequent ones from each iteration. Then a generative model is learned to construct the motifs into molecules. In each generation step, the model embeds the motif candidates, the connection site, and the partial molecule into the embedding space and uses a pointer mechanism to select the next motif to connect to the site. The proposed method achieved competitive distributional learning performance on several molecule datasets. It can also be used in goal-directed generation tasks.
strength_and_weaknesses: Strength:
- The paper presented a Byte-Pair-Encoding-like motif mining algorithm for molecules and demonstrated its usage together with a VAE-based generative model using the mined motifs as building blocks. To my knowledge, such a subgraph mining method is novel and could be used as a generic graph generative model.
- The pointer mechanism in Equation (1) allows the model to select from a set of candidates with varying sizes. Compared with using a trainable embedding vector for each motif, using a GNN to embed the motifs allows the model to generalize to less seen motifs. Similarly, such a method allows the sharing of information between different connection sites.
- Negative sampling (or contrastive learning) is employed to reduce computational costs during training. 
- The distributional learning experiment demonstrated the competitive performance of the proposed method.

Weaknesses:
- Lack of baseline comparisons in the goal-directed generation experiments.
- Several motif-based goal-directed molecule generation methods are not included and discussed in the paper. Jin, Wengong, Regina Barzilay, and Tommi Jaakkola. "Multi-objective molecule generation using interpretable substructures." International conference on machine learning. PMLR, 2020. Chen, Binghong, et al. "Molecule optimization by explainable evolution." International Conference on Learning Representation (ICLR). 2021.

clarity,_quality,_novelty_and_reproducibility: This paper is written clearly. The paper provided implementation details and code for reproducibility. Refer to the previous question for Quality and novelty.
summary_of_the_review: This paper presented a novel fragment-based generative model for molecules. Empirically this method achieved competitive distributional learning performance. Overall the contribution of this paper is solid.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper tackles the problem of fragment-based molecular generation. The authors propose MiCaM that includes a data-driven algorithm to mine a connection-aware motif vocabulary from a molecule library, as well as a connection-aware generator for de novo molecular generation. For motif vocabulary, the authors evaluated MiCaM on three different datasets and achieved improvement in some metrics. For fragment-based molecular generation,  MiCaM also obtains improvement on goal directed benchmarks.


strength_and_weaknesses: Strength:
  1) Building fragment vocabulary by data mining rather than pre-defining, which can avoid artificial bias.
  2) Selecting new connection sites from both motif vocabulary and the intermediate molecule itself, which allows generating larger rings.
  3) The authors conduct different experiments on various datasets and the problem formulation is clear. 

Weaknesses:
  1) The validity rate of the model without chemical validity check was not provided. Some fragments must combine with each other to form chemical groups (e. g. aromatic systems), therefore only depending on data mining to build motif may neglect this entirety. For example, motif ‘Cc1ccnn1’ in Figure 11 is chemically invalid, since it violate the Hückel's rule.
 2) Results in Table 1 indicate that the proposed model did not outperform other baseline models significantly in terms of uniqueness and novelty. The improvements in some metrics for the primary results are very weak, which makes me wonder if the authors' many choices are well justified. The authors could conduct more downstream tasks on the motif-vocabulary to demonstrate the benefits.
 3) How does the model control the size of the molecules generated in the goal directed generation task?
 4) How does the molecule scoring in the Table 2  define?   Can you demonstrate that the improvement comes from the selection of the motif vocabulary? 
 5)  Careless writing, e. g. the “ZINC” dataset sometimes is written as “ZINK”.



clarity,_quality,_novelty_and_reproducibility: Expression of this paper is not good enough, which makes it hard to read. 
The method is relatively novel, but the improvement is not significant.

summary_of_the_review: Expression of this paper is not good enough, which makes it hard to read. 
The insight of the proposed model is reasonable but the implementation may neglect chemical rules.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper develops a novel molecular generation method called MiCaM that simultaneously selects motifs from a motif library and determines how they are connected (or terminate the generation). The key feature of MiCaM is that the motif library (motif vocabulary), the collection of frequent substructural fragments, is built directly from the given set of molecules by iteratively merging subgraphs based on their frequency, also with storing the connection information (this was motivated by byte-pair encoding in MLP). Using this pre-acquired motif library with connection information, MiCaM was trained in a VAE manner to generate novel molecules by selecting motifs and combining them in learned ways. The empirical comparisons to several existing baselines such as JT-VAE, GCPN, and GP-VAE, as well as MoLeR (Maziarz et al., 2021) that generates molecules by extending scaffolds with substructural motifs, demonstrate that it can well resemble the distributions of training sets, keeping high uniqueness and novelty, and it can also perform well on goal-directed benchmarks of the GuacaMol benchmarks.
strength_and_weaknesses: [Strength]

- The paper presented an interesting method that generates molecules based on "mined" and "connection-aware" motifs. Due to these considerations, the generator part was effectively designed as a VAE model to just select motifs and determine how they are combined with each other.

- The experiments include two demonstrative tasks of distributional learning results (to produce a set of new molecules that are structurally similar to the given training set) as well as the goal-directed generation tasks (to produce new molecules better than the given training set with respect to the target property) using the GuacaMol benchmarks, which are one of the well-designed benchmarking datasets.

[Weaknesses]

- One significant feature of the proposed method is the way to construct motif vocabulary that is not predefined but is directly mined from the given data, keeping the connection information for further generation uses. But this main claim would sound very incremental when considering a previous work [1]. (Disclaimer: I have no relationship at all with the authors of this paper [1][2][3] below)

The paper [1] uses a traditional chemoinformatics algorithm BRICS [2] with further refinements to construct a motif tree (motifs at nodes and connection information forms a tree hierarchy on motif vocabulary). The authors could check Figure 2 of [1] to describe the motif vocabulary procedure and would find that this traditional procedure can also be pure data-driven ("directly mined from the given dataset") and connection-aware (due to the constructed motif tree), i.e., both "mined" and "connection-aware." 

[1] Zhang et al. Motif-based Graph Self-Supervised Learning for Molecular Property Prediction. (NeurIPS'21)
    https://arxiv.org/abs/2110.00987

[2] Degen et al. On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces. (2008)
    http://doi.org/10.1002/cmdc.200800178

BRICS is a well-known way for molecule fragmentation (and further use of it for molecule library construction), and the original paper [2] was highly cited (it was also cited in the GuacaMol paper, for example). Furthermore, the examples of MiCaM motif vocabulary shown in Appendix C.1 would look like those from typical BRICS (or RECAP) fragmentation.

So to claim the novelty of the "mined" and "connection-aware" motifs, it would be highly appreciated to compare the difference and investigate the performance and properties of MiCaM over BRICS motifs (or other motif construction methods). It is already well-established to generate new molecules using BRICS motifs, and it would be informative to check the RDKit document to get a BRICS-based generator.

http://www.rdkit.org/docs/GettingStartedInPython.html#brics-implementation

Getting these building blocks is practically quite important and well-investigated (outside the ML community, though) because they can subsequently be used to construct virtual screening libraries for targeted drug discovery. For example, a tool called eMolFrag [3] was published in a chemoinformatics journal, and in this paper, the authors can check the related literature on motif vocabulary construction and their use for generating new molecules.

[3] Liu et al., Break Down in Order To Build Up: Decomposing Small Molecules for Fragment-Based Drug Design with eMolFrag. (2017)
    https://doi.org/10.1021/acs.jcim.6b00596

- Related to the above point, some ablation study would be needed to confirm that this paper's motif-vocabulary algorithm contributes to the reported good performance by defining the MiCaM generation procedure with existing motif vocabularies other than the proposed one. For example, reporting any concrete examples that can be generated from MiCaM but not likely from MoLeR would be very informative to readers.

- From the reported results of Table 1 (GuacaMol), KL Divergence and FCD were better than MoLeR, but Uniqueness and Novelty were worse than MoLeR. This could also suggest that generated molecules from MiCaM are more like the training set and less valuable in the sense of molecular structure exploration. So it would require a more detailed investigation to systematically understand how this difference is in actual examples. 

clarity,_quality,_novelty_and_reproducibility: This paper is well-written and provides the codebase as supplementary materials for ensuring reproducibility. 
As for the novelty, it would be a bit incremental, and would be better to relate this vocabulary construction to the standard fragmentation-based library construction such as BRICS and RECAP in chemoinformatics and that in the previous work [1].
summary_of_the_review: The paper presented an interesting method that generates molecules by selecting and combining "mined" and "connection-aware" motifs. But possible alternative methods for this vocabulary building, in particular widely ones such as BRICS (also used in the NeurIPS'21 GNN paper [1]), are not evaluated, and the claim of novelty and significance would be unconvincing, at least in its present form.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper is concerned about generation of molecules by connecting motifs extracted from the dataset. The main idea of the proposed method is to mine motifs from the dataset. The algorithm, illustrated in Figure 2, first contracts molecular graphs by contracting the most frequent edge, iteratively. Then, motifs are extracted from the contracted graphs as shown in Figure 2 (b), resulting a vocabulary of motifs. 

The authors propose a generative model using the vocabulary of motifs. At each iteration of the generation process, a connection site $v_t$ from partial graph $\mathcal{G}_t$ is obtained from the queue, and the other site to be connected is queried by Eq. 1, where the query vector is computed from the partial graph and the input $z$, and the key vector is computed from the other connection site, which may be in the vocabulary or in the partial graph itself. The model is trained in a similar way to VAE, minimizing the loss function in Eq. 3.

The proposed method is evaluated on GuacaMol benchmarks (distributional and goal-directed benchmarks).
For  the distributional learning task, the proposed method achieves relatively higher scores than the baselines. In addition, the authors provide how the scores change as we change the number of merging operations, $K$. The results suggest that as we increase $K$, we get larger motifs, which improves the similarity between the generated molecules and those in the dataset, at the cost of the decreased novelty score, as expected.

For the goal-directed generation result, the proposed model is combined with off-the-shelf optimization modules to generate optimized molecules. The results show that the proposed method achieves better scores than the baseline methods in GuacaMol benchmark. The authors also analyze the generation trajectory to highlight the benefit of utilizing the motif-based approach.
strength_and_weaknesses: # Strengths
- The proposed approach is reasonable and the algorithm is mostly easy to understand (except for the details elaborated in the clarity section of my review).
- In the empirical studies, the authors not only evaluate and compare the performance, but also highlight the properties of the proposed method (Figs. 4 and 5), which are insightful.

# Weaknesses
Relationship to the work by Guo et al. (ICLR 2022), which is cited in the paper, is not clear. As far as I understand, the method proposed by them generates a molecule using a grammar, which can be interpreted as connecting (connection-aware) motifs, and they also optimize the grammar (equivalently, the set of motifs) so as to bias the generative model towards the direction users specify, which implies that the resultant graph grammar is not heuristically designed but is optimal in some sense. Thus, I consider the method by Guo et al. satisfies the two desirable properties suggested in the introduction. I would like to see in-depth discussion on the relationship between their method and the method proposed in this paper, and if they have similar capabilities, they should be compared empirically.
clarity,_quality,_novelty_and_reproducibility: # Clarity, Quality
Although the illustration in Figure 2 is intuitive and makes it easy to understand the algorithm, I am confused about the contraction operation (the second iteration in the Merging-operation Learning Phase, i.e., lines 15-17 in Algorithm 1). The algorithm says that for each pair $(\mathcal{F}_i, \mathcal{F}_j) \in \mathcal{E}_M^{(k)}$ (edges in the $k$-th iteration), if $\mathcal{F}_i \oplus \mathcal{F}_j = \mathcal{M}^{(k)}$ then merge them in $\mathcal{G}_M^{(k+1)}$ (the graph at the $(k+1)$-th iteration). What I get confused is that when a pair $(\mathcal{F}_i, \mathcal{F}_j)$ is contracted, then another adjacent pair, say $(\mathcal{F}_j, \mathcal{F}_k) = \mathcal{M}^{(k)}$, may not appear in $\mathcal{G}_M^{(k+1)}$ after the contraction, though it exists still in $\mathcal{E}_M^{(k)}$. This could happen in Figure 2. The original graph has 12 `c:c`s, which can be contracted as in the second left figure in Fig 2 (a), where `c:c` nodes are connected in a triangular shape, but there could be other contraction where we have a square-shape like `(c:c)1:c:(c:c):c:1`. So, I suspect that the merging-operation learning phase is order-dependent, which seems not to be highlighted in the paper. If this is correct, I would appreciate if the authors could discuss about it and how they resolve it; otherwise, I would appreciate if the authors correct my misunderstandings.

# Novelty
As pointed out above, grammar-based methods could be closely related to the proposed method. In order to evaluate the novelty of this work, I would like to hear from the authors the relationship to the grammar-based approaches.

# Reproducibility
This work will be reproducible because the authors provide the source code.
summary_of_the_review: I consider this paper is on borderline, leaning to rejection. While I like the proposed method itself, the relationship between the proposed method and the method by Guo et al. is not clear, which is essential to differentiate the proposed method from the methods in the literature. I would like to re-evaluate the novelty and significance after the authors clarify the relationship.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a novel motif-based molecular generation algorithm. The proposal first mines motifs of molecules from a database by iteratively merging smaller motifs based on their frequency. It then trains a VAE-based generative model on the mined motive vocabulary. The performance of the proposed method, called MiCaM, is empirically evaluated on real-world datasets.

### Strength

- The idea of directly building the motif vocabulary from a molecular database by mining frequently appearing motifs and effectively generating molecules using such motifs is interesting. In addition to this interesting main idea, the proposed algorithm is carefully designed with a number of well-prepared techniques.

- Empirical evaluation is thorough and the performance of the proposed algorithm is convincing. Although reviewers pointed out several issues, the authors performed additional experiments to address them in their responses and the revised paper.

- Presentation is good. This paper is well written and easy to follow.

### Weakness

- The novelty is not very high, as the proposal is a combination of existing techniques. 

- In the goal directed benchmarks, currently five molecules are tested in the paper, while there are many more goal directed benchmark molecules in GuacaMol. Thus evaluation on other molecules would be valuable. 

Overall I think this paper is very well written and most of the concerns raised by reviewers have been successfully addressed in the authors' responses and the revised paper. Therefore it deserves to be accepted in my opinion.

By the way, the paper format in P.3 looks strange (main text is displayed at half size) if the file is opened by Preview on MacOS (there is no problem if opened by Adobe Acrobat). It would be better to check this issue as many people will read your paper after it gets accepted and some of them will use Preview.

justification_for_why_not_higher_score: Although the proposed method is well-designed, as I have stated in the meta-review, the novelty is not very high. This means that the core idea is not so inspiring for further development of studies in this field. Based on my opinions and reviewers' scores, Accept with poster is appropriate to this paper. 

justification_for_why_not_lower_score: This paper is well written and the proposed method is well-designed and its performance is superior to other state-of-the-art methods under the standard benchmarks (GuacaMol). There are no crucial concerns and all the reviewers agree with accepting the paper. Therefore I accept the paper.
Decision: Accept: poster