De Novo Molecular Generation via Connection-aware Motif Mining
Authors: Zijie Geng
Shufang Xie
Yingce Xia
Lijun Wu
Tao Qin
Jie Wang
Yongdong Zhang
Feng Wu
Tie-Yan Liu
Venue: ICLR 2023 poster (2023)
Keywords: Molecular generation
Graph generation
Motifs mining

Abstract: De novo molecular generation is an essential task for science discovery. Recently, fragment-based deep generative models have attracted much research attention due to their flexibility in generating novel molecules based on existing molecule fragments. However, the motif vocabulary, i.e., the collection of frequent fragments, is usually built upon heuristic rules, which brings difficulties to capturing common substructures from large amounts of molecules. In this work, we propose MiCaM to generate molecules based on mined connection-aware motifs. Specifically, it leverages a data-driven algorithm to automatically discover motifs from a molecule library by iteratively merging subgraphs based on their frequency. The obtained motif vocabulary consists of not only molecular motifs (i.e., the frequent fragments), but also their connection information, indicating how the motifs are connected with each other. Based on the mined connection-aware motifs, MiCaM builds a connection-aware generator, which simultaneously picks up motifs and determines how they are connected. We test our method on distribution-learning benchmarks (i.e., generating novel molecules to resemble the distribution of a given training set) and goal-directed benchmarks (i.e., generating molecules with target properties), and achieve significant improvements over previous fragment-based baselines. Furthermore, we demonstrate that our method can effectively mine domain-specific motifs for different tasks.

Published as a conference paper at ICLR 2023
DE Novo MOLECULAR GENERATION VIA CONNECTION- AWARE MOTIF MINING
Zijie Geng",, Shufang Xie²t, Yingee Xia Lijun Wu', Tao Qin', Jie Wang!* Yongdong Zhang', Feng Wu', Tie-Yan Liu University of Science and Technology of China
ustcgzjfmail.. ustc. eew.e.edu..... en, (jiewangx, zhyd73, fengwu (estreeeuuttcc.o c. edu
2 Gaoling School of Artificial Intelligence, Renmin University of China shufangx weennggeeeur..ooom ruc. edu. on
Microsoft Research AI4Science {yingee xia, Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
taogin, tyliu//microsoffe con
ABSTRACT
De novo molecular generation an essential task for science discovery. Recently, fragment-based deep generative models have attracted much research attention due to their flexibility generating novel molecules based on existing molecule fragments. However, the motif vocabulary, i.e., the collection of frequent frag- ments, usually built upon heuristic rules, which brings difficulties to capturing common substructures from large amounts of molecules. In this work, we propose a new method, MiCaM, to generate molecules based on mined connection-awe aware motifs. Specifically, leverages data-driven algorithm automatically discover motifs from a molecule library by iteratively merging subgraphs based on their frequency. The obtained motif vocabulary consists of not only molecular motifs (i.e., the frequent fragments), but also their connection information, indicating how the motifs are connected with each other. Based on the mined connection- aware motifs, MiCaM builds connection -aware generator, which simultaneously picks up motifs and determines how they are connected. We test our method on distribution-leariine
benchmarks (i.e., generating novel molecules to resemble the distribution of a given training set) and goal-directed benchmarks (i.e., generating molecules with target properties), and achieve significant improvements over pre- vious fragment-bad based basclines. Furthermore, we demonstrate that our method can effectively mine domain-specifie motifs for different tasks.
INTRODUCTION
Drug discovery, from designing hit compounds developing an approved product, often takes more than ten years and billions dollars (Hughes et al., 2011). De novo molecular generation is a fundamental task in drug discovery, as provides novel drug candidates and determines the un- derlying quality of final products. Recently, with the development of artificial intelligence, deep neural networks, especially graph neural networks (GNNs), have been widely used to accelerate novel molecular generation (Stokes et al., 2020; Bilodeau et al., 2022). Specifically, we can employ a GNN to generate a molecule iteratively: in each step. given an unfinished molecule Go. we first determine a new generation unit Gi to be added; next, determine the connecting sites on Go and G1; and finally determine the attachments between the connecting sites, egg., creating new bonds (Liu et al., 2018) merging shared atoms (Jin al., 2018). In different methods, the generation units could be either atoms (Li et al., 2018: Mercado et al., 2021) or frequent fragments (referred to as motifs) (Jin et al., 2020a; Kong al., 2021; Maziarz al., 2021).
For fragment- based models, building an effective motif vocabulary is a key factor to the success of molecular generation (Maziarz al., 2021). Previous works usually rely on heuristic rules tem-
"This work was done when Zijie Geng was an intern at Microsoft Research A14Science. Corresponding author.

---

Published as a conference paper at ICLR 2023
Benaaddeede 10-Celooced
(b)
Figure 1: (a) Two substructures that occur frequently in ChEMBL. (b) Different connection modes of sulfur atoms. The " *+-S(=*)<**)=** commonly seen Wh**""####* does not appear in ChEMBL.
plates to obtain a motif vocabulary. For example, JT-VAE (Jin al., 2018) decomposes molecules into pre-defined structures like rings, chemical bonds, and individual atoms, while MoLeR (Maziarz al., 2021) separates molecules into ring systems and acyelic linkers or functional groups. Sev- eral other works build motif vocabularies in a similar manner (Jin et al., 2020a; Yang et al., 2021). However, heuristic rules cannot cover some chemical structures that commonly occur yet are a bit more complex than pre-defined structures. For example, the subgraph patterns benzaldehyde ("O=CelececeI1 and triftooromethylbenzeee ("FC(F)(()c cur 398,760 times and 57,545 times respectively in the 1.8 million molecules in ChEMBL (Mendez et al., 2019), and both of them are industrially useful. Despite their high frequency in molecules, the aforementioned methods cannot cover such common motifs. Moreover, in different concrete generation tasks, different motifs with some domain-specific structures or patterns are favorable, which can hardly be enumerated by existing rules. Another important factor that affects the gener- ation quality is the connection information of motifs. This is because although many connections are valid under a valence check, the "reasonable" connections are predetermined and reflected by the data distribution, which contribute to the chemical properties of molecules (Yang et al., 2021). For example, a sulfur atom with two triple-honds, i.e., "*#$#*" (see Figure (b)), is valid under valence check but is "unreasonable" from chemical point of view and does not occur in ChEMBL.
(as shown Figure I(a)) oc-
In this work, we propose MiCaM, a generative model based on Mined Connection aware Motifs. It includes a data-driven algorithm to mine connection-aware motif vocabulary from molecule library, as well as a connection-aware generator for de novo molecular generation. The algorithm mines the most common substructures based on their frequency of appearance in the molecule li- brary. Briefly, across all molecules in the library, we find the most frequent fragment pairs that are adjacent in graphs, and merge them into entire fragment. We repeat this process for pre-defined number of steps and collect the fragments to build a motif vocabulary. We preserve the connection information of the obtained motifs, and thus we call them connection-aware motifs.
Based on the mined vocabulary, we design the generator to simultaneously pick up motifs to be added and determine the connection mode of the motifs. In each generation step, we focus on non- terminal connection site in the current generated molecule, and use it to query another connection either (1) from the motif vocabulary, which implies connecting a new motif, or (2) from the current molecule, which implies cyclizing the current molecule.
We evaluate MiCaM on distribution learning benchmarks from GuacaMol (Brown al., 2019), which aim to resemble the distributions of given molecular sets. We conduct experiments on three different datasets and MiCaM achieves the best overall performance compared with several strong baselines. After that, we also work on goal directed benchmarks, which aim to generate molecules with specific target properties. We combine MiCaM with iterative target augmentation (Yang al., 2020) by jointly adapting the motif vocabulary and network parameters. In this way, achieve state-of-the-an results on four different types of goal-directed tasks, and find motif patterns that are relevant to the target properties.
2 OUR APPROACH
2.1
CONNECTION-AWAER MOLECULAR MOTIF MINING
Our motif mining algorithm aims to find the common molecule motifs from a given training data set D, and build connection-aware motif vocabulary for the follow-up molecular generation.
2

---

Published as a conference paper at ICLR 2023
(a) Merging-operatee Learning Phase
Molecule Graphs:
Merging
12
Graphs:
C:0 12
CC000 6
claccoel
Breeiii
Statistics:
C-C
C-c.c
C-cccs
O-c:c
O-c:c
(b) Motif-vocabulan Construction Phase
Motif Vocabulary:
Merging
C:C
8
Operations:
CCC::
c-+
cl cocccl
Figure 2: An example of connection-aware molecular motif mining. Given a training set D ("BreIecceel", "Celccce(O)e1"), it consists of two phases. (a) Merging-operation Learning Phase. The merging graphs are initialized the same as the molecular graphs. In the first iteration, "exe" (marked blue) is the most frequent pattern. to update the merging graphs. In the second iteration, "c:e:c:e" (marked in yellow) is the most frequent pattern, so we merge such patterns and update merging graphs. We repeat this process for iterations and record the merging operations in order. (b) Motif- vocabulary Construction Phase. We apply the recorded merging operations sequentially on all molecules. Then the two molccules are fragmentized motifs. We break the bonds between different motifs while preserving the broken bonds. In this way we construct a connection-awarn motif vocabulary as shown (c).
merge the patterns "c:c" in the molecules
processes D with two phases: the merging operation learning phase and the motif- vocabulary con- struction phase. Figure 2 presents an example and more implementation details are in Algorithm in Appendix A.I.
Merging-operation Learning Phase In this phase, we aim to learn the top K most common patterns (which correspond to rules indicating how to merge subgraphs) from the training data D, where K is a hyperparameter. Each molecule in represented as graph g(V, E) (the first row in Figure 2(a)), where the nodes V and edges & denote atoms and bonds respectively. For each C(V,E) E D, we use a merging graph (MrVar, Ews) (the second row in Figure 2(a)) to track the merging status, i.e., represent the fragments and their connections. In Gas(VM, Exr), each node FF E VM represents a fragment (either an atom or a subgraph) of the molecule, and the edges in EM indicate whether graph from the molecule graph by treating each atom as single fragment and inheriting the bond connections from G, i.e., 9000) (y:1) e(l)) G(V.E).
fragments are connected with each other. We initialize each merging
We define an operation "o" to create a new fragment F,y = B Fi and F, together. The newly obtained F,j contains all nodes and edges from F,, F,, as well as all edges between them. We iteratively update the merging graphs to learn merging operations. In the merging graph ((h) (v)(k) &&M c(k)) the 5th iteration (k = O,... K 1), each edge represents a pair of fragments, (F,,5)). F5,. that are adjacent in the molecule. It also gives out a new fragment
by merging two fragments
Fy F. Fy. We traverse all edges (F,, FF) EM all merging graphs ((A) to count the frequency Fij = F, @ F,, and denote the most frequent Fsg as (() Consequently, the k-th merging operation is defined as: if F, @ F, = M then merge F, and F, together.? We apply the merging operation all merging graphs to update them into g MM such a process for K iterations obtain a merging operation sequence (M (1))
.CM (k++)) (k+1) c(k+1)). We repeat
Tk=0
Different FF, can make same Fiy. For example, both (CCC, C) and (CC, CC) can make "CCCC". WWee applying merging operations, traverse edges in orders given RDKI (I (Landrum al., 2006).
3

---

Published as a conference paper at ICLR 2023
(a) Motaf Vocatralary
[b) Generttin Sugee Slep
Sun2
GNN motif
GNN pmas
hg
[2, Nge A
NN
NN
NN
Figure 3: The Generation Steps. (a) The generation procedure is based on the connection aware motif vocabulary. We obtain the graph representations h (yellow) and the node representations h. (dark blue) via GNN (b) In the gth generation step, we obtain the graph representation hg, (dark green) and node representations h, (light blue) via Gmol We focus on connection site et and use [z, hg. ft..] to query another connection site either from the motif vocabulary (which implies adding a motif) or from the partial molecule (which implies cyclizing). The right answers of every steps marked by red boxes.
Motif-vocabulary Construction Phase For each molecule G(V,E) E) E D, we apply the merging operations sequentially to obtain the ultimate merging graph GM(VM, EM) = g(k) (v(k) e(K)). We then disconnect all edges between different fragments and add the symbols "*" to the discon- nected positions (see Figure 2(b, c)). The fragments with "x" symbols are connection-aware. and we denote the connection-awe -ware version of fragment a F. The motif vocabulary is the collection of all such connection-and aware motifs: Vocab = (g F VM During generating, model connects the motifs together by directly merging the connection sites (i.e., the "*"s) to generate new molecules.
Time Complexity The time complexity of learning merging operations is (KK)D))) where K is the number of iterations, [D] is the molecule library size, and e maXg(v.E)eD [E). In practice, the time cost decreases rapidly as the iteration step l: increases. It takes less than 10 minutes and about 90 minutes to run 3, 000 iterations on the QM9 (~ 133K molecules) (Ruddigkeit et al., 2012) and ZINC (~> 219K molecules) (Irwin al., 2012) datasets, respectively, using CPU cores (see Appendix A.I). The time complexity of fragmentizing an arbitrary molecule G(V,)) E) into motif's is O(K]E)), linear with the number of bonds.
2.2 MOLECULAR GENERATION WITH CONNECTION-AWAR MOTIFS
The generation procedure of MiCaM shown in Figure and Algorithm 2 in Appendix A.2. Mi- CaM generates molecules by gradually adding new motifs to current partial molecule (denoted as Gr. where is the generation step), or merging two connection sites in G, to form a new ring.
For of reference, we define the following notations. We denote the node representation of any atom v (including the connection sites) as hgs and denote the graph representation of any graph G (either molecule partial molecule or a motif), and let Cvucab Up be the set of all connection sites from the motif vocabulary.
motif) hg. Let Cg denote the connection sites from graph (either
Generation Steps In the tth generation step, MiCaM modifies the partial molecule G, as follows:
(1) Focus connection site 0r from G. We use queue Q to manage the the orders of connection sites in Cg.- At the th step, we pop the head of Q to get a connection site 12 The Q is maintained follows: after selecting new motif F to connect with G,, canonical order of the atoms in F², and then put the connection sites into Q following this order. After merging two connection sites together, just remove them from Q.
the RDKit library to give
4

---

Published as a conference paper at ICLR 2023
(2) Encode i/ and candidate connections. We employ graph neural network GNNpmol to encode the partial molecule G1 and obtain the representations and the graph. The node representations h., of 12 and the graph representation hg. of G, will be jointly used to query another connection site. For the motifs F E Vocab, we use another GNN, denoted as GNN representations h. of all connection sites v E CVoaa The candidate connections are either from CVoca or from Cs. {vz).
all atoms (including the connection sites)
encode their atoms and connection sites. In this way we obtain the node
(3) Query another connection site. We employ two neural networks, NN to make a query vec- tor, and NNyk to make key vectors, respectively. Specifically, the probability P. of picking every connection sites is calculated by: softmax vECrenUCo, Mor)
P.
QNuq ((z, hg. hu,,)) NNNka (h.)),
(1)
where is latent vector as used in variational auto-encoder (VAE) (Kingma & Welling, 2013). Using different x results in diverse molecules. During training, & is sampled from a posterior distri- bution given by encoder, while during inference, is sampled from prior distribution.
For inference, we make a constraint on the bond type of picked e by only considering the connection sites whose adjacent edges have the same bond type U₁- This practice guarantees the validity of the generated molecules. We also implement two generation modes, i.e., greedy mode that picks the connection arg max P.,, and distributional mode that samples the connection from P..
(4) Connect a new motif or cyclize. After the model picks a connection v, it turns into a connecting phase or cyclizing phase, depending on whether E CVocah that v E Cr=, then we connect F with G, by directly merging n, and v. Otherwise, when v E Cg.- we merge Ut and v together to form a new ring, and thus the molecule cyclizes itself. Notice that, allowing the picked connection site to come from the partial molecule is important, because with this mechanism MiCaM theoretically can generate novel rings that are not in the motif vocabulary.
Cg.- If CVucab, and suppose
We repeat these steps until Q is empty and thus there is no con-terminal connection site in the partial molecule, which indicates that we have generated an entire molecule.
Starting As in the beginning (i.e., the 0th step), the partial graph is empty, we implement this step exceptionally. Specifically, we use another neural network NN to pick up the first motif from the vocabulary as Go. The probability Py. of picking every motifs is calculated by:
Py-
softmax (NNsma(z) (NNey(1x-)) hy- GNNmour
Vocab
2.3 TRAINING MICAM
We train our model in VAE (Kingma& & Welling, 2013) paradigm.* 4 A standard VAE has an encoder and a decoder. The encoder maps the input molecule G to its representation hg, and then builds a posterior distribution of the latent vector z based on hg. The decoder takes z as input and tries to reconstruct the G. VAE usually has a reconstruction loss term (between the original input & and the reconstructed G) and regularization term (to control the posterior distribution of 2).
In our work, we use GNN model GNN as the encoder to encode a molecule & and obtain its representation hg GNN mot (G). The latent vecotr is then sampled from posterior distribution q(-19)= = N (w(hgg) exp( [hgg))), where μ and Σ output the mean and log variance, respectively. How to use is explained in Equation (1) and (2). The decoder consists of GNN GNN NN query. NNEey and NN that jointly work to generate molecules.
pmool GNN moirs
The overall training objective function is defined as:
EgnD [C(G)] = Eg-D (9) + +Brrior Cprior + Syrop Eprop(G)].
(3)
In Equation (3): (1) (g) is the reconstruction loss as that in a standard VAE. It uses cross entropy loss to evaluate the likelihood of the reconstructed graph compared with the input g. (2) The loss
Duringg inference, the motif representations can calculated offline to avoid additional computation time. 'MiCMM can be naturally paired with other paradigms such as GAN or RL., which we plan to explore in future works.
5

---

Published as a conference paper at ICLR 2023
I)) is used to regularize the posterior distribution in the latent space. (3) Following Maziarz et al. (2021), we add property prediction loss Ppoop (g) to ensure the continuity of the latent space with respect to some simple molecule properties. Specifically, build another network prprop to predict the properties from the latent vector z. (4) Sprior and Sprop hyperparameters to be determined according validation performances.
Sprior (G)
Here we emphasize two useful details. (1) Since MiCaM is an iterative method, in the training procedure, we need determine the orders of motifs to be processed. The orders of the intermediate motifs (i.e., & 2 1) are determined by the queue @ introduced in Section 2.2. The only remaining item is the first motif, and we choose the motif with the largest number of atoms the first one. The intuition is that the largest motif mostly reflects the molecule's 's properties. The generation order then determined according to Algorithm 2. (2) In the training procedure, we provide supervisions for every individual generation steps. We implement the reconstruction loss (9) by viewing the generation steps as parallel classification tasks. In practice, as the vocabulary size large due to various possible connections, (G) is costly to compute. To tackle this problem, we subsample the vocabulary and modify (9) via contrastive learning (He et al., 2020). More details can be found in Appendix A.4.
2.4 DISCUSSION AND RELATED WORK
Molecular Generation plethora of existing generative models are available and they fall into two categories: (1) string- based models (Kusner et al., 2017;
et al., 2018; Sanchez- Lengeli- & Aspuru- Guzik, 2018; Segler et al., 2018), which rely on string representa- tions of molecules such as SMILES (Weininger, 1988) and do not utilize the structual information of molecules, (2) and graph-based based models (Liu et al., 2018; Guo et al., 2021) that are naturally based on molecule graphs. Graph-based based approaches mainly include models that generate molecular graphs (I) atom-by-atom (Li et al., 2018; Mercado et al., 202 I), and (2) mngment-by-fragmeen
(Kong et al., 2021; Maziarz et al., 2021; Zhang et al., 2021; Guo et al., 2021). This work is mainly related to fragment-basee based methods.
Motif Mining Our motif mining algorithm inspired by Byte Pair Encoding (BPE) (Gage, 1994), which is widely adapted in natural language processing (NLP) to tokenize words into subwords (Sen- nrich et al., 2015). Compared with BPE in NLP, molecules have much more complex structures due to different connections and bond types, which solve by building the merging graphs. Another related class of algorithms are Frequent Subgraph Mining (FSM) algorithms (Kuramochi & Karypis, 2001: Jiang al., 2013), which also aim to mine frequent motifs from graphs. However, these al- gorithms do not provide a "tokenizer" to fragmentize an arbitrary molecule into disjoint fragments, like what is done by BPE. Thus we cannot directly apply them in molecular generation tasks. Kong et al. (2021) also try to mine motifs, but they do not incorporate the connection information into the motif vocabulary and they apply totally different generation procedure, which are important to the performance (see Appendix B.3). See Appendix D for more discussions.
Motif Representation Different from many prior works that view motif's as discrete tokens, represent all the motifs in the motif vocabulary as graphs, and we apply the GNNmotif to obtain the representations of motifs. This novel approach has three advantages. (1) The GNN mour obtains similar representations for similar motifs, which thus maintains the graph structure information of handle large size of motif vocabulary in training, which allows us to construct a large motif vocabulary (see Section 2.3 and Apppendix A.4). (3) The model can be easily transferred to another motif vocabulary. Thus we can jointly tune the motif vocabulary and the network parameters on a new dataset, improving the capability of the model to fit new data (see Section 3.2).
the motifs (see Appendix C.3). (2) The GNNmotff combined with contrastive learning.
3 EXPERIMENTS
3.1 DISTRIBUTIONAL LEARNING RESULTS
To demonstrate the effectiveness of MiCaM, we test it on the benchmarks from GuacaMol (Brown et al., 2019), commonly used evaluation framework assess de novo molecular generation mod-

---

Published as a conference paper at ICLR 2023
Table 1: Distributional results QM9, ZINC, and GuacaMol. The higher the better for all metries. The results of JT-VAE, GCPN and GP-VAE are from Kong et al. (2021). For MoLeR, we use the released code from Maziarz al. (2021) with no changes.
Dadaset
Model
Validity
Uniqueness
Novelty
KL Div
FCD
JT-VAR
1.0
0.549
0.386
0.891
0.588
GCPN
1.0
0.533
0.320
0.552
0.174
QM9
GP-VAE
1.0
0.673
0.523
0.921
0.659
MoLeR
1.0
0.940
0.355
0,969
0,931
MiCaM (Ours)
1.0
0,932
0,493
0.980
0.945
JT-VAE
1.0
0,988
0,988
0.882
0.263
GCPN
1.0
0.982
0.982
0.456
0.003
ZINC
GP-VAE
1.0
0,997
0.997
0.850
0.318
MoLeR
1.0
0.996
0.993
0.984
0.721
MiCaM (Ours)
1.0
0.998
0.997
0.988
0.791
MoLeR
1.0
1.000
0.991
0.964
0.625
GuacaMol
MiCaM (Ours)
1.0
0.994
0,986
0.989
0.731
0.60
0.90
MoLalt
0.40
0.56
FCD Sanee
0.55
Divrggen
Nowelyy
0,92
0,20
0.91
0.45
0,92
distibattionll
0.50
diseiibational
guady
0.40
discributional
graaly
0.36
200
2000 3000
100 200
560 600 1000
800 600)
Number of Margiug Operation
Numbber of Mirging Operation
Number Mergiu Operations
(a) KL Divergence
(b) FCD
(c) Novelty
Figure 4: KI. Divergence and FCD scores (higher is better) for different numbers of merging opera- tions and different choices of generation modes.
els. We first consider the distribution learning benchmarks, which assess how well the models learn generate novel molecules that resemble the distribution of the training set.
Experimental Setup Following Brown et al. (2019), we consider five metrics for distribution learning: validity, uniqueness, novelty, KL divergence (KL Div) and Fréchet ChemNet Distance (FCD). The first three metrics measure if the model can generate chemically valid, unique, and novel candidate molecules, while last two are designed to measure the distributional similarity be- tween the generated molecules and the training set. For the KL Div benchmark, probability distributions variety physicochemial descriptors. higher KL Div score, i.c., lower KI. divergences for the descriptors, means that the generated molecules resemble the training in terms of these descriptors. The FCD calculane lated from the hidden representations of molecules neural network called ChemNet (Preuer al., 2018), which can capture important chemiacal and biological features of molecules. A higher FCD score, i.e., a lower Fréchet ChemNet Distance means that the generated molecules have similar chemical and biological properties to those from the training set.
compare the
We evaluate our method on three datasets: QM9 (Ruddigkeit et al., 2012), ZINC (Irwin al., 2012), and GuacaMol (a post-processed ChEMBL (Mendez et al., 2019) dataset proposed by Brown et al. (2019)). These three datasets cover different molecule complexities and different data sizes. The results across them demonstrate the capability of our model to handle different kinds of data.
'The code of MiCaM is available at http://ghhhiccmmmmmtaaaaaasasAaa

---

Published as a conference paper at ICLR 2023
Table 2: Goal directed generation results on five GuacaMol benchmarks. The higher the better for all metrics. The results of other baselines are from Brown et al. (2019) and Ahn et al. (2020). The benchmarks are: a. Celecoxib Rediscovery; b. Aripiprazole Similarity; c. C1 Isomers; d.
Ranolazine MPO; and e. Sitagliptin MPO.
SMILES
Graph MCTS
Graph GA
SMILES LSTM
MiCaM (Ours)
Benchmark Dataset
MSO
GA
b. 0.505
0.732
0.355
1.000
1.000
1.000
1.000
0.595
0.834
0.380
1.000
1.000
1.000
1.000
0.684
0.829
0.410
0.971
0.993
0.997
0.999
d. 0.792
0.881
0.616
0.920
0.855
0.931
0.932
0.509
0.689
0.458
0.891
0.545
0.868
0.914
Quantitative Results Table presents our experimental results of distribution learning tasks. We compare our model with several -fate-of-the-arl models: JT-VAE (Jin et al., 2018), GCPN (You al., 2018), GP-VAE (Kong et al., 2021), and MoL.eR (Maziarz et al., 2021). Since all the models are graph-based, they obtain 100% validity by introducing chemical valence check during generation. We can see that MiCaM achieves the best performances all the three datasets, which demonstrates that it can well resemble the distributions of training sets. Meanwhile it keeps high uniqueness and novelty, comparable to previous best results. In this exper- iment, we set the number of merging operations to be 1000 for QM9, due to the results in Figure 4. For ZINC and GuacaMol, performances that outperform all the baselines. This indicates that existing methods tend to per- form well the sets of relatively simple molecules such QM9, while MiCaM performs well on datasets with variant complexity. We further visualize the distributions in Figure 7 of Appendix B.2.
the KL Divergence and FCD scores on
simply set the number to be 500 and find that MiCaM has achieved
Number of Merging Operations We conduct experiments on different choices of the number of merging operations. Figure 4 presents the experimental results on QM9. shows that FCD score and KL Divergence score, which measure the similarity between the generated molecules and the training set, increase as the number of merging operations grows. Meanwhile, the novelty decreases as the number grows. Intuitively, the more merging operations we use for motif vocabulary con- struction, the larger motifs will be contained the vocabulary, and thus will induce more structural information from the training set. We can achieve trade-off between the similarity and novelty by controlling the number of merging operations. Empirically, a medium number (about 500) of operations is enough to achieve high similarity.
Generation Modes We also compare two different generation modes, i.e., the greedy mode and the distributional mode. With the greedy mode, the model always picks the motif or the connection site with the highest probability. While the distributional mode allows picking motifs connection sites according to a distribution. The results show that the greedy mode leads to little higher KI. Divergence and FCD scores, while the distributional mode leads to higher novelty.
3.2 GOAL DIRECTED GENERATION RESULTS
We further demonstrate the capability of our model generate molecules with wanted properties. In such goal directed generation tasks, we aim to generate molecules that have high scores which are predefined by rules.
Iteratively Tuning We combine MiCaM with iterative target augmentation (ITA) (Yang et al., 2020) and generate optimized molecules by iteratively generating new molecules and tuning the model on molecules with highest scores. Specifically, we first pick out N molecules with top scores from the GuacaMol dataset and store them in training buffer. Iteratively, we tune our model on the training buffer and generate new molecules. In each iteration, we update the training buffer to store the top N molecules that either newly generated or from the training buffer in the last iteration. In order to accelerate the model to explore the latent space, we pair MiCaM with Molecular Swarm Optimization (MSO) (Winter et al., 2019) in each iteration to generate new molecules.
8

---

Published as a conference paper at ICLR 2023
ump urr moof
Scone
0.0
0.4
Sim
0.2
TPSA
Num_Pe
Miou
Step
(a) Generation trajectory
(b) Scores of each properties
generation trajectory for Ranolazine MPO benchmark. In each generation step, the query connection marked in red, and the newly added motif is marked in yellow.
Figure 5:
novelty of our approach that when tuning on a new dataset, we jointly update the motif vo- cabulary and the network parameters. We can do this because we apply GNN to obtain motif representations, which can be transferred to newly built motif vocabulary. The representations of the new motif's are calculated by GNN and then optimize the network parameters using both new data and newly constructed motif vocabulary.
Experimental Setup We test MiCaM several goal directed generation tasks from GuacaMol benchmarks. Specifically, we consider four different categories of tasks: a Rediscovery task (Cele- coxib Rediscovery), Similarity task (Aripiprazole Similarity), Isomers task (Cn H24 Isomers), and two multi-prppoa property objective (MPO) tasks (Ranolazine MPO and Sitagliptin MPO). We compare MiCaM with several strong baselines.
Quantitative Results For each benchmark, we run 7 iterations. In each iteration, we apply MSO to generate 80, 000 molecules, and store 10, 000 molecules with highest scores in the training buffer to tune MiCaM. We tune the model pretrained
the GuacaMol dataset and Table 2 presents the results. MiCaM achieves 1.0 scores on some relatively easy benchmarks. achieves high scores on several difficult benchmarks such MPO tasks, outperforming the baselines.
Case Studies There are some domain -senaif- motifs that are beneficial to the target properties in different goal directed generation tasks, which likely be the pharmacophores in drug molecules. We conduct case studies to demonstrate the ability of MiCaM to mine such favorable motifs for domain-specific tasks.
In Figure 5 we present cases for the Ranolazine MPO benchmark, which tries to discover molecules similar Ranolazine, known drug molecule, but with additional requirements erties. This benchmark calculates the geometric mean of four scores: Sim (the similarity between the molecule and Ranolazine). logP, TPSA. and Num (the number of fluorine atoms). We present generation trajectory well as the scores in each generation step. Due to the domain- specific motif vocabulary and the connection query mechanism, requires only few steps to generate such complex molecule. Moreover, we can see that the scores increase some key motifs are added to the molecule, which implics that the picked motifs are relevant to the target properties. See Figure in Appendix B.4 for more case studies.
some other prop-
4 CONCLUSION
In this work, we proposed MiCaM, a novel model that generates molecules based on mined connection-aware motifs. Specifically, the contributions include (1) data-driven algorithm to mine motifs by iteratively merging most frequent subgraph patterns and (2) a connection-awee aware generator for de molecular generation. It achieve of-te-oftthe-an results distribution learning tasks and on three different datasets. Combined with iterative target augmentation, it can learn domain- specific motifs related to some properties and performs well on goal directed benchmarks.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
The authors would like to thank all the anonymous reviewers for their insightful comments. This work was supported in part by National Nature Science Foundations of China grants U19B2026, U19B2044, 61836011, 62021001, and 61836006, and the Fundamental Research Funds for the Central Universities grant WK3490000004.
REFERENCES
Sungsoo Ahn, Junsu Kim, Hankook Lee, and Jinwoo Shin. Guiding deep molecular optimization with genetic exploration. Advances in neural information processing systems, 2020.
Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F Jensen. Gener- ative models for molecular discovery: Recent advances and challenges. Wiley Interdisciplinury Reviews:
Computational Molecular Science, pp. e1608, 2022.
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowiez, and Samy Ben- gio. Generating sentences from a continuous space. arXiv preprint arXiv:1511.0334 2015.
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3): 1096-1108, 2019.
Binghong Chen, Tianzhe Wang, Chengtao Li, Hanjun Dai, and Le Song. Molecule optimization by explainable evolution. In International Conference on Learning Representation (ICLR). 2021.
Jörg Degen, Christof Begscheid-Gerlachh Andrea Zaliani, and Matthias Rarey. On the art of compil- ing and using drug-like' chemical fragment spaces. ChemMedChem: Chemistry Enabling Drug Discovery, 3(10): 150--1507, 2008.
Philip Gage, A new algorithm for data compression. Users Journal, 12(2):23-38, 1994.
Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sänchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparaquuiree Timothy D Hirzel, Ryan P Adams, and Alán Sspuru-Guzik Automatic chemical design using data-driven contin- uous representation of molecules. ACS central science, 4(2):268 276, 2018.
Minghao Guo, Veronika Thost, Beichen Li, Payel Das, Jie Chen, and Wojciech Matusik. Data- efficient graph grammar learning for molecular generation. In International Conference Learn- ing Representations, 2021.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729- 2020.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. arXiv preprint arXiv:/90s. /2265, 2019.
James P Hughes, Stephen Rees, S Barrett Kalindjian, and Karen Philpott. Principles of early drug discovery. British journal of pharmacology. (62(6)):339-1244 2011.
John J Irwin, Teague Sterling, Michael Mysinger, Erin Bolstad, and Ryan Coleman. Zine: free tool to discover chemistry for biology. Journal of chemical information and modeling, 52 (7):1777-1768, 2012.
Jan H Jensen. A graph- based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12)3356-33777 2019.
Chuntao Jiang, Frans Coenen, and Michele Zito. A survey of frequent subgraph mining algorithms.
The Knowledge Engineering Review, 28(1 281) 2013.
10

---

Published as a conference paper at ICLR 2023
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International conference on machine leamiing, pp. 2323-2332. PMLR. 2018.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In International Conference on Muchine Learning, pp. 4839-4848. PMLR, 2020a.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. In International conference on machine learning, pp. 4849-4855 PMLR, 2020b.
Diederik Kingma and Max Welling. arXiv:1312.6114, 2013.
Auto-encoding variational bayes.
arXiv preprint
Xiangzhe Kong. Zhixing Tan, and Yang Liu. Graphpiece: Efficiently generating high-quality molec- ular graph with substructures. arXiv preprint 2tti::2000,,09999, 2021.
Michihiro Kuramochi and George Karypis. Frequent subgraph discovery. In Proceedings 2001 IEEE international conference on data mining, pp. 313 320. IEEE, 2001.
Matt J Kusner, Brooks Paige, and José Miguel Hemández--Lobto. Grammar variational autoen- coder. In International conference on machine learning, pp. 1945-1954, PMLR, 2017.
Greg Landrum et al. Rdkit: Open-source cheminformatics, 2006.
Xiao Qing Lewell, Duncan B Judd, Stephen P Watson, and Michael M Hann. Recap retrosynthetic combinatorial analysis procedure: a powerful new technique for identifying privileged molecular fragments with useful applications in combinatorial chemistry. Journal of chemical information and computer sciences, 38(3):511-522 1998.
Yibo Li, Liangren Zhang, and Zhenming Liu. Multi-objective de novo drug design with conditional graph generative model. Journal of cheminformatics, 10(1):1-24. 2018.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander Gaunt. Constrained graph varia- tional autoencoders for molecule design. Advances in neural information processing systems, 31, 2018.
Krzysztof Maziarz, Henry Jackson-Flux, Pashmina Cameron, Finton Sirockin, Nadine Schneider, Nikolaus Stiefl, Marwin Segler, and Mare Brockschmidt. Learning to extend molecular scaffolds with structural motifs. arXiv preprint arXiv:2103.0328 2021.
David Mendez, Anna Gaulton, Patrícia Bento, Jon Chambers, Marleen De Veij, Eloy Félix, María Paula Magariños, Juan F Mosquera, Prudence Mutowo, Michal Nowotka, et al. Chembl: towards direct deposition of bioassay data. Nucleic acids research, 47(D1):D930 D940, 2019.
Rocío Mercado, Tobias Rastemo, Edvard Lindelöf, Günter Klambauer, Ola Engkvist, Hongming Chen, and Fshen Jannik Bjerrum. Graph networks for molecular design. Machine Learning: Science and Technology, 2(2):025023, 2021.
Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, and Gunter Klambauer. Fréchet chemnet distance: a metric for generative models for molecules in drug discovery. Journal of chemical information and modeling, 15(9))1736-1741,, 2018.
David Rogers and Mathew Hahn. Extendded-connectvvity fingerprints. Journal of chemical informa- tion and modeling, 50(5):742-754, 2010.
Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Loouui Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of chemical information and modeling. 52(1 1):28)4422777 2012.
Benjamin Menchez-Lengelingg and Alán Aspuru-Guzik. Inverse molecular design using machine learning: Generative models for matter engineering. Science, (6400))33366-335, 2018.

---

Published as a conference paper at ICLR 2023
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science, 4(1): 120-131, 2018.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.0755 2015.
Jonathan M Stokes, Kevin Yang. Kyle Swanson, Wengong Jin, Andres Cubillos-Ruiz, Nina M Donghia, Craig MacNair, Shawn French, Lindsey Carfrae, Zohar Bloom-Ackermann, et al. A deep learning approach to antibiotic discovery. Cell, 180(4))688-702 2020.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
David Weininger. Smiles, chemical language and information system. 1. introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-23) 1-36, 1988.
Robin Winter, Floriane Montanari, Andreas Steffen, Hans Briem, Frank Noé, and Djork-Aré Clev- ert. Efficient multi-objective molecular optimization continuous latent space. Chemical science, 10(34)800668024 8024, 2019.
Kevin Yang. Wengong Jin, Kyle Swanson, Regina Barzilay, and Tommi Jaakkola. Improving molec- ular design by stochastic iterative target augmentation. In International Conference on Machine Learming, pp. 10716-10726. PMLR, 2020.
Soojung Yang, Doyeong Hwang, Seul Lee, Seongok Ryu, and Sung Ju Hwang. Hit and lead discov- ery with explorative rl and fragment- based molecule generation. Advances in Neural Information Processing Systems, 34:7924-797 19779477936, 2021.
Naruki Yoshikawa, Kei Terayama, Masato Sumita, Teruki Homma, Kenta Oono, and Koji Tsuda. Population- based de novo molecule generation, using grammatical evolution. Chemistry Letters, (7(11):1411-1444 2018.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems, 31, 2018.
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self- supervised learning for molecular property prediction. Advances in Neural Information Process- ing Systems, 34:15870-15588 2021.
12

---

Published as a conference paper at ICLR 2023
A IMPLEMENTATION DETAILS
A.1 MOTIF MINING ALGORITHM
Algorithm 1: Connection-awwred Motif Mining
Input: A set of molecule graphs D = -9,, (pp)), the number K of iterations.
Output: The merging operations {M(() Sk=0 and the motif vocabulary Vocab (F*).
(0) t ft: 2 for E do
// Merging-operation learning phase
A (V/)) c(0)
M t g(V.));
D t D
M U ( 1 do
5 for k 0 to
Reset Count() to
for (hk) (V E E D
M do
((k) for (Fi, E &M do
M - F; + Fjj Count(M + Count(M) 1;
10
M(R + arg max (M): 11
D(ktt) +
for (1)
M (k) elk)) E D do
13
Is ( eeM
clktll) t g(k) (Vil) {
14
M
15
for (Fi, F2) E Eas
c(k)
do
if Fj) E E&M cl++l) && F, FF = M then
16
Merge Fi and F, in (++1)
(k+1) t (1+1) U (tg(k+))
18
M
19
Vocab
< f]:
// Motif-vocabulary construction phase
for E(V,)) do
21
1 (2:00 c(0)
t G(V.E);
for l = to K 16-10 do
23
((++)) (V)) (k+1) c(k+l) 1)) t ((k) (Vi,, (
M E
24
for (Fi, E Es do
(k)
if (Fi E E ckk+l) && F; + F5 = ((k) then
25
Merge Fi and F, in 9 (lk+t).
Vocab Vocab U (F* for F E v M };
(K)
27
Our connection-awart motif mining algorithm is in Algorithm 1. In the merging graph (Vr,MM, Ex), each node F E VM represents a fragment F(V,E) of the molecule graph 9(V E), where V C V and & C E are atoms and bonds in F, respectivelyy The edge set EM is defined by: for any two fragments F((2,, &,), (V,, E,) E VM. (F,.F,) E EM + Ea E V., E V,, (a, b) e E. The merging operation "@" is defined to create Fig OV Enj) = F.FFF by merging two fragments Fi(V,, &,) and F,;;,,), Ey). Formally,
Vi V,U,, Eg EE,
E Ela V.. b V,},
which means the new fragment F.y contains all nodes and edges from F,, F, and the edges between them. Notice that when we traverse the edges of graphs, we always follow the orders determined by RDKit. In the motif- construction phase, disconnect bonds between different frag- ments and add *** atoms create connection-awar motifs. Specifically, for each F(V, E) VM. we define its corresponding connection aware motif F= F*' as
n = úu (a E V, b V2, (a, E),, E ((a,b) Ela b")
13

---

Published as a conference paper at ICLR 2023
where "a"" denotes that change the label of the atom to "x". The symbol "*" can be seen as a dummy atom or a connection site, which indicates that the bond is non-ten non-terminal and we will grow the molecule here.
Efficiency Figure 6 presents the time costs of learning operations from QM9, which demonstrates that our algorithm is fast, and the time cost of each iteration decreases rapidly as the motif frequency decreases.
10
10F
10F
Minnte
Minnte
100-1
Proguarcy
10
10-
J0P
10:01
2000
3000
100
250
350
310
100
200
3041
400
500
Iseration
Iocation
Trenatio
(a) Accumulative time costs
(b) Time costs of each iteration
(c) Frequencies
Figure 6: Time costs of learning merging operations QM9. (a) and (b) show the accumulative time costs and time costs of each iteration, respectively. (c) shows the frequencies the learnt merging operations.
Sequential Merging Operations The merging operations are useful they work a "tokenizer" that can be applied to fragmentize an arbitrary molecule, which is unavailable for other methods like Frequent Subgraph Mining, Another choice of using the merging operations is not applying them sequentially, but traversing the molecule edges to find if there are patterns that appear in the learnt frequent motifs. However, this is sub-optimal as it does not work reasonably any arbitrary molecule outside the dataset to learn the operations. For example, consider a trivial dataset D = (CC, CN, CNN, CN=O, CC=O). When (CN, CC). Then we use them to fragmentize a new molecule "CCN". If we apply the merging operations sequentially, the molecule will be decomposed into {C. CN). However, the molecule edges to find the patterns, the molecule will be decomposed into {CC, NJ, which is sub-optimal because "CN" appears with higher frequency in the dataset.
two iterations, the learnt merging operations
traverse
A.2 GENERATING PROCEDURE
Algorithm 2 presents the pseudo code of our generation procedure. For ease of reference, let Cg denote the connection sites from a graph g (either a partial molecule or a motif), and let Cvocab = Up-evecabCy. be the set of all connection sites from the motif vocabulary.
A.3 NETWORKS
The backbone of MiCaM is VAE. The encoder is GNN followed by three MLPs: μ and Σ for resampling, and NN for property prediction. The decoder consists of GNN GNN moutf. which are GNNs, and NN NNquery. NNkey. which are MIPs. All MLPs have 3 layers and use ReLU as the activation function. The latent size and the hidden size are both 256.
We employ GINE (Hu et al., 2019) as the GNN structures, and in each GNN layer we employ a 3- layer MLP for messaage aggregation. For all GNNs, we use five atom-level features as inputs: atom symbol, is arommtiic, formal charge, num.explicit.HH Hs, num num_implicit_H.. The features are embedded with the dimension 192, 16, 16, 16, 16, respectively, and thus the node embedding size is 256. Four edges, consider four types of bonds (single bonds, double bonds, triple bonds and aromatic bond), and the embedding size is 256. GNN mo! and GNN have 15 layers and GNN motif has layers. You can released code for more details.
14

---

Published as a conference paper at ICLR 2023
Algorithm 2: Generating a molecule
Input:
connection-awes aware motif vocabulary Vocab = }. A latent vector 2. Output: A molecule graph G.
Vucau f Uy-eVocab
Eocab Cr+;
2 Calculate Py. by Equation 2 and sample F Py+;
& +F', Q 0;
// Pick the starting motif
4 for U E Cx- do
Q.pu((v);
while Q & do vr + Q.get() C Crecah Ugg \ {
// Connection querying step
Calculate Po over C by Equation and sample v Pv;
10
v E Cvocab and in F E Vocab then < (.AddMoti((F")): G - G.Merge v);
11
// Connecting a new motif
12
13
for E Cr. {a} do Q.put(v');
15
else
16
Assert v E Co \ {ur): G - G.Merge(ur, v); Q t (u)
17
// Cyclizing itself
18
A.4 EXPERIMENT DETAILS
Learning Merging Operations For QM9, we apply 1000 merging operations due to the compara- tive results in 4. For ZINC and GuacaMol, simply use 500 merging operations without elaborate searching, and find that has achieved good results. Due to the large size of GuacaMol dataset, randomly sample 100, 000 molecules from learn merging operations, and then apply these merging operations on all molecules to obtain the motif vocabulary.
Training MiCaM We preprocess all the molecules provide supervision signals for decoder to rebuild the molecules. We provide the true indices of picked connections in every steps so that the model can learn the ground truth. each optimization step, we update the network parameters by optimizing the loss function a batch B of molecules:
Cg = EgaB [3prior (prior(() )+) + Crec(g) + Sprop (Pr)
Specifically, the reconstruction loss free is written the partial graphs G. at each step t, conditioned on z and the last steps:
the negative log probabilities of
(r) -Ez
logp(Golz)
Σ
log 10g((9++12z.9,
-B (g)
log Σ
logp(ue|2,0rr
where Fú. and ui the first motif, the focused query connection site and the picked connec- tion at the tth step, respectively. practice, since the motif vocabulary is large due to different connections, modify the loss via contrastive learning for efficient training:
exp(NNNsar(2) NNkoy (hF;))
logp(Folz) log
Σx.ety. st (z) NNkey (hr=))
exp(NN querry (Iz, hg. hvvl)) NNkey (ha,))
log pogp(ulee 12. v2) f- log
ΣnET, exp ([z, log, fbw,)) NNNey (h+))
where I,-- and I, are the sets of motifs and connections, respectively, containing a positive sample and negative samples from the batch B
15

---

Published as a conference paper at ICLR 2023
We find that proper choice of Sprior is essentially important to the performances distribution learning benchmarks, especially for FCD scores. For QM9, we use a short warm-up (3, 000 steps), and use long sigmoid schedule (400, 000 steps) (Bowman et al., 2015) to let Sprior to reach 0.4.
For property prediction, we predict four simple properties of molecules, including molecular weight, synthetic accessibility (SA) score, octanol- water partition coefficient (logP) and quantitative esti- mate of drug-likeness (QED). The target values are computed using the RDKit library. Empirically, for distribution learning benchmarks, small Sprop (about 0.3) is beneficial.
A.5 VALIDITY CHECK
We conduct validity check during generation avoid the model generating invalid aromatic rings (e.g., merging two "#:c::::::::*" into "clecccccel") Specifically, when the model tries to generate such an invalid aromatic ring, we simply remove the aromaticity of this ring so that the molecule is still valid (e.g.. "cl "eleccccec]" will be replaced with "CICCCCCCC1") Without this chemical validity check, the validity rates on QM9, ZINC, and GuacaMol are 99. 69.68% 98.6%, and 98. 28%, respectively. The high validity rates indicate that MiCaM learns to generate valid aromatic rings.
B ADDITIONAL RESULTS
B. EFFICIENCY
Thanks to the mined motifs and the connection- decoder, MiCaM is very efficient. We measure the training and sampling speed on a single GeForce RTX 3090. For training, trains on 325.7 molecules per second. For inference, it generates 54.4 molecules per second.
B.2 DISTRIBUTION VISUALIZATION
200d
(a) Training sets
(b) Results on QM9
(c) Results on ZINC
(d) Results on GuacaMol
Figure 7: Visualization of the probability distributions of training sets (QM9, ZINC and GuacaMol) and the generated molecules. The postfix "rref" means reference, i.e., the training sets (shown in green, blue and red, respectively). and the postfix gen" means the sets of molecules generated by model (shown in grey). We obtain the representations of molecules by calculating their molecular fingerprints, and we then apply t-SNE dimensionality reduction for visualization. The curves represent the contour lines of the probability distributions of the datasets. (a) shows the distributional shift over the three training sets. (b), (c) and (d) demonstrate that our generative model can properly fit the different distributions respectively.
We visualize the distributional results on the three datasets in Figure 7. Specifically, we first cal- culate the Morgan fingerprints of all molecules. Morgan fingerprint has been used for a long time in drug discovery, as it can represents the structual information of molecules (Rogers & Hahn, 2010). For visualization, we apply the t-distributed stochastic neighbor embedding (t-SNE) algo- rithm (Van der Maaten & Hinton, 2008) nonlinear dimensionality reduction technique to keep the similar high-dimg high-dimensional vectors close in lower-dimensional space- to represent molecules in the two-dimensional plane, and then plot the distributions of molecules.
16

---

Published as a conference paper at ICLR 2023
B.3 ABLATION STUDY
Trade-off Among Metrics As KL Divergence and FCD negatively correlate with Uniqueness and Novelty, there is trade-off among the metries in distribution learning tasks. We can achieve this trade-off via some hyperparameters. For example, and use distributional mode (sampling from top 5 choices) for sampling, MiCaM achieves higher uniqueness and novelty and outperforms MoLeR in terms of all the metrics. The results are in Table
QM9, if we conduct 500 merging operations,
Table MiCaM can achieve trade-off among the distribution learning metries. With 1000 merging operations and greedy mode, MiCaM significantly outperforms MoLeR in terms of KL Divergence and FCD, While with 500 merging operations and distribution mode, MiCaM outperforms MoLeR
in terms of all the metries.
Model
Validity
Uniqueness
Novelty
KL Div
FCD
MoLeR MiCaM-100-greedy
1.0
0.940
0.355
0.969
0.931
1.0
0.932
0.493
0,980
0.945
MiCaM-500-distr
1.0
0.941
0.495
0.978
0,940
Motif Vocabulary We conduct two more experiments on QM9 to verify the effect of the motif vo- cabulary. Specifically, we use the generation procedure of MiCaM, but replace the motif vocabulary with the vocabularies MoLeR (Maziarz et al., 2021) and MGSSL (Zhang et al., 2021), respec- tively. For fair comparison, vocabularies. We name the two models MiCaM-moler and MiCaM-brics,, respectively, as MGSSL. applies BRICS (Degen et al., 2008) with further decomposition. The results are in Table 4.
preserve the connection information (i.e.. the "s*") in the two new
Table 4: Ablation studies on different motif vocabularies on QM9.
Model
Validity
Uniqueness
Novelty
KL Div
FCD
MiCaM-moler
1.0
0.926
0.468
0.973
0.934
MiCaM-brics
1.0
0.927
0.485
0.978
0.938
MiCaM
1.0
0.932
0.493
0.980
0.945
Generating Procedure Besides the molecule fragmentation strategy, two components contribute to the performance of MiCaM. First is the connection information preserved in the motif vocabulary and corresponding connection-awant decoder. Second is the GNN which captures the graph structures of motifs, and allows efficient training on a large motif vocabulary via contrastive learning. We conduct ablation studies demonstrate the importance of the two components. Specifically, implement three different versions of MiCaM. MiCaM-v1 does not apply NN and does not use connection information for generation. In each step, it first picks up the motif (without connection information) by viewing them as discrete tokens, and then determines the connecting points and bonds. MiCaM-v2 employs the NN to pick up motifs, but does directly query the connection sites. The results demonstrate that leveraging connection information and employing the GNN motif actually bring performance improvements.
Table 5; Ablation studies on different versions of MiCaM. The results are from a set of 100, 000 molecules randomly sampled from GuacaMol.
Model
Validity
Uniqueness
Novelty
KL Div
FCD
MiCaM-v1
1.0
1.000
0.993
0.926
0.685
MiCaM-v2
1.0
0.989
0.989
0.965
0.617
MiCaM
1.0
0.998
0.995
0.957
0.754
17

---

Published as a conference paper at ICLR 2023
Itre
sc00e-0.150
$20ro-41.200
Ther
20000=0.2000
Sc000=0
soaresd 750
S000000 DEB
Ieer S:
SOOTSHO 750
Figure 8: Generation trajectories for Celecoxib Rediscovery. We show the trajectories of the best molecules in three different iterations. In each generation step, the query connection is marked in red, and the newly added motif marked in yellow.
B.4 CASE STUDIES
Figure presents cases for Celecoxib Rediscovery task, which aims discover molecules similar to Celecoxid, a known drug molecule. Specifically, we present the trajectories to generate molecules with the highest scores the 1st, 3rd and 5th iterations. As the number of iteration increases, the motifs learnt by MiCaM tend to be specific and more adaptive the target, leading to the model to generate molecules with higher scores while costing fewer generation steps.
B.5 GENERATED MOLECULES
Some examples of the generated molecules are in Figure 9. For further comparison, we visualize the probability distributions of GuacaMol, the molecules generated by MiCaM and MoLeR, respec- tively, in Figure 10. We can see that MiCaM fits the reference distribution better than MoL.R. Moreover, from the visualization, we find that the outermost contour line of MiCaM covers more area than MoLeR and fits that the reference data better. This indicates that some reasonable chemical spaces explored more by MiCaM than MoLeR. We then find that such cases include molecules with large rings or complex ring systems. See Figure 11 for some concrete examples.
18

---

Published as a conference paper at ICLR 2023
g
fooo
oho
gla
foor
Figure 9: Samples from molecules randomly generated by MiCaM model, trained on GuacaMol.
(a) Results of MiCaM
(b) Results of MoLeR
Comparison
Figure 10: Visualization of the distributions of the molecule sets. We obtain the representations of molecules by calculating their molecular fingerprints, and we then apply t-SNE dimensionality reduction for visualization.
(a) and (b) visualize the contour lines of the probability distributions of the molecule sets. Red represents the GucaMol datasets, while gray represents the generated molecules. (c) shows the samples from MiCaM and MoLeR in orange and blue, respectively. Each point represents molecule.
Figure 11: Molecules with large rings or complex ring systems generated by MiCaM, which likely from MoLeR
19

---

Published as a conference paper at ICLR 2023
C MOTIF VOCABULARY
C.I MERGING OPERATIONS
We present the learnt merging operations from QM9 and GuacaMol in Figure 12. The merging operations can efficiently merge molecules into few disjoint units. For statistics, each molecule in GuacaMol has 27.8 27.899 atoms on average. After applying 500 merging operations, each they represented only 8.499 subgraphs average. After applying 1000 merging operations, the number is 8.282. As a comparison, MoLeR decomposes each molecule into 9.667 fragments on average, when taking 4096 as the vocabulary size.
(a) Learnt merging operations QM9.
(b) Learnt merging operations for GuacaMol.
Figure 12: Merging operations. Note that instead of kekulizing all molecules, we maintain the aromaticity of atoms and bonds. Therefore some of the patters seem half-baked.
20

---

Published as a conference paper at ICLR 2023
C.2 MINED MOTIFS
Our algorithm is able mine common graph motifs with high frequency in large number of molecules, including some motifs with complex structures and domain specific motifs. We present some mined motif's Figure C.2.
(a) Mined motifs for QM9.
(b) Mined motifs for GuacaMol.
Mind motifs in the last iteration for Ranolazine MPO benchmark.
Figure 13: Some mined motifs.
C.3 MOTIF REPRESENTATIONS
Since we apply GNN mosif to encode motif representations, instead of viewing motifs as discrete tokens, the learnt motif representations maintain structural information in the sense that sim- ilar motifs have close representations. To show this, we visualize some motif representations in Figure 14.
21

---

Published as a conference paper at ICLR 2023
Oclecetel
Colocceel
CCNC
Celoral
cloodhnoodd
10
-10
Figure 14: The t-SNE visualization Motif representations. Each color represents collection of motifs with common structure but different connections. Each point represents motif.
D DISCUSSION AND RELATED WORK
Motif Vocabulary Construction Many previous works explored motif construction methods. JT- VAE (Jin et al., 2018) decomposes molecules into rings, chemical bonds, and individual atoms. HierVAE (Jin et al., 2020a) and MoLeR (Maziarz et al., 2021) decompose molecules into ring sys- tems and acyclic linkers or functional groups. MGSSL (Zhang et al., 2021) first uses BRICS (Degen al., 2008), which is built upon chemical rules, to split molecules into fragments. After that, it manually designs another two rules to further decompose the molecules into rings and chains. Some molecule fragmentation tools such BRICS and RECAP (Lewell al., 1998) also well de- veloped, though outside the ML community. However, as the motif vocabulary obtained by those methods is large and long-tail, the combination of the methods with ML models is nontrivial. The aforementioned methods are mainly based upon pre-defined rules and templates. Guo et al. (2021) proposed DEG, which learns graph grammars to generate molecules and similar to motif-based methods. However, DEG applies REINFORCE and MCTS to search grammars, the learned gram- mars are only for specific metrics, and DEG cannot mine motifs from large datasets. MiCaM mines the most frequent motifs directly from the dataset. The built motif vocabulary is promising to be used in more tasks such as large-scale pre-training, which we leave as future works.
Goal Directed Generation Many frameworks have been developed for goal-directed generation tasks, including iterative methods such as ITA (Yang et al., 2020), genetic methods such as SMILES GA (Yoshikawa al., 2018) and Graph GA (Jensen, 2019), and latent space optimization meth- ods such as MSO (Winter et al., 2019). RationaleRL (Jin et al., 2020b) proposes to extract ratio- nales from a collection of molecules, and then learns to expand the rationales into full molecules. MolEvol (Chen et al., 2021) then proposes novel EM-like on rationales. Such frameworks designed for dire-did learning tasks can be naturally combined with our proposed generative procedure, which we leave future works.
algorithm bsed
22


Reviews:
[{"review_id": "wNHR0jFAYE", "reviewer": "Reviewer_5Fqs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper develops a novel molecular generation method called MiCaM that simultaneously selects motifs from a motif library and determines how they are connected (or terminate the generation). The key feature of MiCaM is that the motif library (motif vocabulary), the collection of frequent substructural fragments, is built directly from the given set of molecules by iteratively merging subgraphs based on their frequency, also with storing the connection information (this was motivated by byte-pair encoding in MLP). Using this pre-acquired motif library with connection information, MiCaM was trained in a VAE manner to generate novel molecules by selecting motifs and combining them in learned ways. The empirical comparisons to several existing baselines such as JT-VAE, GCPN, and GP-VAE, as well as MoLeR (Maziarz et al., 2021) that generates molecules by extending scaffolds with substructural motifs, demonstrate that it can well resemble the distributions of training sets, keeping high uniqueness and novelty, and it can also perform well on goal-directed benchmarks of the GuacaMol benchmarks.", "summary_of_the_review": "The paper presented an interesting method that generates molecules by selecting and combining \"mined\" and \"connection-aware\" motifs. But possible alternative methods for this vocabulary building, in particular widely ones such as BRICS (also used in the NeurIPS'21 GNN paper [1]), are not evaluated, and the claim of novelty and significance would be unconvincing, at least in its present form.", "strength_and_weaknesses": "[Strength]\n\n- The paper presented an interesting method that generates molecules based on \"mined\" and \"connection-aware\" motifs. Due to these considerations, the generator part was effectively designed as a VAE model to just select motifs and determine how they are combined with each other.\n\n- The experiments include two demonstrative tasks of distributional learning results (to produce a set of new molecules that are structurally similar to the given training set) as well as the goal-directed generation tasks (to produce new molecules better than the given training set with respect to the target property) using the GuacaMol benchmarks, which are one of the well-designed benchmarking datasets.\n\n[Weaknesses]\n\n- One significant feature of the proposed method is the way to construct motif vocabulary that is not predefined but is directly mined from the given data, keeping the connection information for further generation uses. But this main claim would sound very incremental when considering a previous work [1]. (Disclaimer: I have no relationship at all with the authors of this paper [1][2][3] below)\n\nThe paper [1] uses a traditional chemoinformatics algorithm BRICS [2] with further refinements to construct a motif tree (motifs at nodes and connection information forms a tree hierarchy on motif vocabulary). The authors could check Figure 2 of [1] to describe the motif vocabulary procedure and would find that this traditional procedure can also be pure data-driven (\"directly mined from the given dataset\") and connection-aware (due to the constructed motif tree), i.e., both \"mined\" and \"connection-aware.\" \n\n[1] Zhang et al. Motif-based Graph Self-Supervised Learning for Molecular Property Prediction. (NeurIPS'21)\n    https://arxiv.org/abs/2110.00987\n\n[2] Degen et al. On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces. (2008)\n    http://doi.org/10.1002/cmdc.200800178\n\nBRICS is a well-known way for molecule fragmentation (and further use of it for molecule library construction), and the original paper [2] was highly cited (it was also cited in the GuacaMol paper, for example). Furthermore, the examples of MiCaM motif vocabulary shown in Appendix C.1 would look like those from typical BRICS (or RECAP) fragmentation.\n\nSo to claim the novelty of the \"mined\" and \"connection-aware\" motifs, it would be highly appreciated to compare the difference and investigate the performance and properties of MiCaM over BRICS motifs (or other motif construction methods). It is already well-established to generate new molecules using BRICS motifs, and it would be informative to check the RDKit document to get a BRICS-based generator.\n\nhttp://www.rdkit.org/docs/GettingStartedInPython.html#brics-implementation\n\nGetting these building blocks is practically quite important and well-investigated (outside the ML community, though) because they can subsequently be used to construct virtual screening libraries for targeted drug discovery. For example, a tool called eMolFrag [3] was published in a chemoinformatics journal, and in this paper, the authors can check the related literature on motif vocabulary construction and their use for generating new molecules.\n\n[3] Liu et al., Break Down in Order To Build Up: Decomposing Small Molecules for Fragment-Based Drug Design with eMolFrag. (2017)\n    https://doi.org/10.1021/acs.jcim.6b00596\n\n- Related to the above point, some ablation study would be needed to confirm that this paper's motif-vocabulary algorithm contributes to the reported good performance by defining the MiCaM generation procedure with existing motif vocabularies other than the proposed one. For example, reporting any concrete examples that can be generated from MiCaM but not likely from MoLeR would be very informative to readers.\n\n- From the reported results of Table 1 (GuacaMol), KL Divergence and FCD were better than MoLeR, but Uniqueness and Novelty were worse than MoLeR. This could also suggest that generated molecules from MiCaM are more like the training set and less valuable in the sense of molecular structure exploration. So it would require a more detailed investigation to systematically understand how this difference is in actual examples. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "mI9xdCgHahU", "reviewer": "Reviewer_zHsw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is concerned about generation of molecules by connecting motifs extracted from the dataset. The main idea of the proposed method is to mine motifs from the dataset. The algorithm, illustrated in Figure 2, first contracts molecular graphs by contracting the most frequent edge, iteratively. Then, motifs are extracted from the contracted graphs as shown in Figure 2 (b), resulting a vocabulary of motifs. \n\nThe authors propose a generative model using the vocabulary of motifs. At each iteration of the generation process, a connection site $v_t$ from partial graph $\\mathcal{G}_t$ is obtained from the queue, and the other site to be connected is queried by Eq. 1, where the query vector is computed from the partial graph and the input $z$, and the key vector is computed from the other connection site, which may be in the vocabulary or in the partial graph itself. The model is trained in a similar way to VAE, minimizing the loss function in Eq. 3.\n\nThe proposed method is evaluated on GuacaMol benchmarks (distributional and goal-directed benchmarks).\nFor  the distributional learning task, the proposed method achieves relatively higher scores than the baselines. In addition, the authors provide how the scores change as we change the number of merging operations, $K$. The results suggest that as we increase $K$, we get larger motifs, which improves the similarity between the generated molecules and those in the dataset, at the cost of the decreased novelty score, as expected.\n\nFor the goal-directed generation result, the proposed model is combined with off-the-shelf optimization modules to generate optimized molecules. The results show that the proposed method achieves better scores than the baseline methods in GuacaMol benchmark. The authors also analyze the generation trajectory to highlight the benefit of utilizing the motif-based approach.", "summary_of_the_review": "I consider this paper is on borderline, leaning to rejection. While I like the proposed method itself, the relationship between the proposed method and the method by Guo et al. is not clear, which is essential to differentiate the proposed method from the methods in the literature. I would like to re-evaluate the novelty and significance after the authors clarify the relationship.", "strength_and_weaknesses": "# Strengths\n- The proposed approach is reasonable and the algorithm is mostly easy to understand (except for the details elaborated in the clarity section of my review).\n- In the empirical studies, the authors not only evaluate and compare the performance, but also highlight the properties of the proposed method (Figs. 4 and 5), which are insightful.\n\n# Weaknesses\nRelationship to the work by Guo et al. (ICLR 2022), which is cited in the paper, is not clear. As far as I understand, the method proposed by them generates a molecule using a grammar, which can be interpreted as connecting (connection-aware) motifs, and they also optimize the grammar (equivalently, the set of motifs) so as to bias the generative model towards the direction users specify, which implies that the resultant graph grammar is not heuristically designed but is optimal in some sense. Thus, I consider the method by Guo et al. satisfies the two desirable properties suggested in the introduction. I would like to see in-depth discussion on the relationship between their method and the method proposed in this paper, and if they have similar capabilities, they should be compared empirically.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "iSWHQ_VVRY", "reviewer": "Reviewer_3hrY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackles the problem of fragment-based molecular generation. The authors propose MiCaM that includes a data-driven algorithm to mine a connection-aware motif vocabulary from a molecule library, as well as a connection-aware generator for de novo molecular generation. For motif vocabulary, the authors evaluated MiCaM on three different datasets and achieved improvement in some metrics. For fragment-based molecular generation,  MiCaM also obtains improvement on goal directed benchmarks.\n\n", "summary_of_the_review": "Expression of this paper is not good enough, which makes it hard to read. \nThe insight of the proposed model is reasonable but the implementation may neglect chemical rules.\n", "strength_and_weaknesses": "Strength:\n  1) Building fragment vocabulary by data mining rather than pre-defining, which can avoid artificial bias.\n  2) Selecting new connection sites from both motif vocabulary and the intermediate molecule itself, which allows generating larger rings.\n  3) The authors conduct different experiments on various datasets and the problem formulation is clear. \n\nWeaknesses:\n  1) The validity rate of the model without chemical validity check was not provided. Some fragments must combine with each other to form chemical groups (e. g. aromatic systems), therefore only depending on data mining to build motif may neglect this entirety. For example, motif ‘Cc1ccnn1’ in Figure 11 is chemically invalid, since it violate the Hückel's rule.\n 2) Results in Table 1 indicate that the proposed model did not outperform other baseline models significantly in terms of uniqueness and novelty. The improvements in some metrics for the primary results are very weak, which makes me wonder if the authors' many choices are well justified. The authors could conduct more downstream tasks on the motif-vocabulary to demonstrate the benefits.\n 3) How does the model control the size of the molecules generated in the goal directed generation task?\n 4) How does the molecule scoring in the Table 2  define?   Can you demonstrate that the improvement comes from the selection of the motif vocabulary? \n 5)  Careless writing, e. g. the “ZINC” dataset sometimes is written as “ZINK”.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2ciwDZmEXj", "reviewer": "Reviewer_Gyc8", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presented a novel fragment (motif) based generative model for molecules. The motif vocabulary is mined from the molecule dataset by iterative merging small motifs into larger ones and keeping the most frequent ones from each iteration. Then a generative model is learned to construct the motifs into molecules. In each generation step, the model embeds the motif candidates, the connection site, and the partial molecule into the embedding space and uses a pointer mechanism to select the next motif to connect to the site. The proposed method achieved competitive distributional learning performance on several molecule datasets. It can also be used in goal-directed generation tasks.", "summary_of_the_review": "This paper presented a novel fragment-based generative model for molecules. Empirically this method achieved competitive distributional learning performance. Overall the contribution of this paper is solid.", "strength_and_weaknesses": "Strength:\n- The paper presented a Byte-Pair-Encoding-like motif mining algorithm for molecules and demonstrated its usage together with a VAE-based generative model using the mined motifs as building blocks. To my knowledge, such a subgraph mining method is novel and could be used as a generic graph generative model.\n- The pointer mechanism in Equation (1) allows the model to select from a set of candidates with varying sizes. Compared with using a trainable embedding vector for each motif, using a GNN to embed the motifs allows the model to generalize to less seen motifs. Similarly, such a method allows the sharing of information between different connection sites.\n- Negative sampling (or contrastive learning) is employed to reduce computational costs during training. \n- The distributional learning experiment demonstrated the competitive performance of the proposed method.\n\nWeaknesses:\n- Lack of baseline comparisons in the goal-directed generation experiments.\n- Several motif-based goal-directed molecule generation methods are not included and discussed in the paper. Jin, Wengong, Regina Barzilay, and Tommi Jaakkola. \"Multi-objective molecule generation using interpretable substructures.\" International conference on machine learning. PMLR, 2020. Chen, Binghong, et al. \"Molecule optimization by explainable evolution.\" International Conference on Learning Representation (ICLR). 2021.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster