Graph Piece: Efficiently Generating High-Quality Molecular Graphs with Substructures

Molecule generation , which requires generating valid molecules with desired properties , is a fundamental but challenging task . Recent years have witnessed the rapid development of atom-level auto-regressive models , which usually construct graphs following sequential actions of adding atom-level nodes and edges . However , these atom-level models ignore high-frequency substructures , which not only capture the regularities of atomic combination in molecules but are also often related to desired chemical properties , and therefore may be sub-optimal for generating high-quality molecules . In this paper , we propose a method to automatically discover such common substructures , which we call graph pieces , from given molecular graphs . We also present a graph piece variational autoencoder ( GPVAE ) for generating molecular graphs based on graph pieces . Experiments show that our GP-VAE models not only achieve better performance than the state-ofthe-art baseline for distribution-learning , property optimization , and constrained property optimization tasks but are also computationally efficient . 1 INTRODUCTION . Molecule generation is a task that aims to produce chemically valid molecules with optimized properties . It is important for a variety of applications , such as drug discovery and material science . Graph-based molecule generation models , which are robust to molecule substructures ( You et al. , 2018 ; Kwon et al. , 2019 ) , have gained increasing attention recently ( Jin et al. , 2018 ; Li et al. , 2018a ; You et al. , 2018 ; Kwon et al. , 2019 ; De Cao & Kipf , 2018 ; Shi et al. , 2020 ; Jin et al. , 2020b ) . Graph-based molecule generation models typically decompose molecular graphs into sequential actions of generating atoms and bonds autoregressively ( Li et al. , 2018b ; You et al. , 2018 ; Li et al. , 2018a ; Jin et al. , 2020b ) . While this decomposition is natural and straightforward , it inevitably ignores the existence of common substructures in molecular graphs , as illustrated in Figure 1 . Compared with using atoms for generating molecules , using graph substructures for generating molecules have three potential benefits . First , using substructures can capture the regularities of atomic combination in molecules , and therefore is more capable of generating realistic molecules . Second , using substructures can better capture chemical properties , as there is a correlation between substructures and chemical properties ( Murray & Rees , 2009 ; Jin et al. , 2020b ) . Third , using substructures enables efficient training and inference . It is evident that using substructures to represent molecular graphs can result in much shorter sequences , therefore the training and inference process can be accelerated . As a result , we believe that models using substructures to represent molecular graphs can generate more realistic molecules with better-optimized properties efficiently . 83.9 % 60.5 % 41.3 % 32.6 % 27.5 % . In this paper , we present an iterative algorithm to automatically discover common substructures in molecules , which we call graph pieces . Initially , graph pieces correspond to single atoms that appear in graphs of a given dataset . Then for each iteration , we count the occurrence of neighboring pieces in graphs and merge the most frequent neighboring pieces into a new graph piece . Since substructures can be seen as small molecules , we use SMILES ( Weininger , 1988 ) , a text-based representation for molecules , to efficiently judge whether two graph pieces are identical . To effectively utilize these substructures , we also propose a graph piece variational autoencoder ( GP-VAE ) . Our model consists of a graph neural network ( GNN , Scarselli et al. , 2008 ) encoder and a two-step decoder . The two-step decoder first generates graph pieces auto-regressively and then predicts atom-level bonds between graph pieces in parallel . As a result , our GP-VAE decouples the alternated generation of nodes and edges , achieving significant computational efficiency for both training and generation . We conduct extensive experiments on ZINC250K ( Irwin et al. , 2012 ) and QM9 ( Blum & Reymond , 2009 ; Rupp et al. , 2012 ) datasets . Results demonstrate that our GP-VAE models outperform state-of-the-art models on distribution-learning , property optimization , and constrained property optimization tasks , and are about six times faster than the fastest baseline . 2 RELATED WORK . Molecule Generation Based on different representations for molecules , molecule generation models can be divided into two categories : text-based and graph-based . Text-based models ( GómezBombarelli et al. , 2018 ; Kusner et al. , 2017 ; Bjerrum & Threlfall , 2017 ) , which usually adopt the Simplified Molecular-Input Line-entry System ( SMILES ) ( Weininger , 1988 ) representation , are simple and efficient methods for generating molecules . However , these models are not robust because a single perturbation in the text molecule representation can result in significant changes in molecule structure ( You et al. , 2018 ; Kwon et al. , 2019 ) . Graph-based models ( De Cao & Kipf , 2018 ; Shi et al. , 2020 ; Jin et al. , 2020b ) , therefore , have gained increasing attention recently . Li et al . ( 2018b ) proposed a generation model of graphs and demonstrated it performed better than text-based generation models on molecule generation . You et al . ( 2018 ) used reinforcement learning to fuse rewards of chemical validity and property scores into each step of generating a molecule . Popova et al . ( 2019 ) proposed an MRNN to autoregressively generate nodes and edges based on the generated graph . Shi et al . ( 2020 ) proposed a flow-based autoregressive model and use reinforcement learning for the goal-directed molecular graph generation . However , these models use atomlevel graph representation , which results in very long sequences , and therefore the training process is typically time-consuming . Our method is graph-based and uses substructure-level representation for graphs , which not only captures chemical properties but also is computationally efficient . Substructure-level Graph Representation Jin et al . ( 2018 ) proposed to generate molecules in the form of junction trees where each node is a ring or edge . Jin et al . ( 2020a ) decomposed molecules into substructures by breaking all the bridge bonds . It used a complex hierarchical model for polymer generation and graph-to-graph translation . Jin et al . ( 2020b ) proposed to extract the smallest substructure which maintains the original chemical property . The extracted substructures usually include most atoms of the original molecules , which are too coarse-grained and exert limitations on the search space . The models proposed by Jin et al . ( 2020a ) and Jin et al . ( 2020b ) are not suitable for the experiments in this paper since they either need graph-to-graph supervised data or are incompatible with continuous properties . There exists various methods to discover frequent subgraphs ( Inokuchi et al. , 2000 ; Yan & Han , 2002 ; Nijssen & Kok , 2004 ) . However , they have difficulty decomposing graphs into frequent subgraphs ( Jazayeri & Yang , 2021 ) since they mainly aim to discover frequent subgraphs as additional features for downstream network analysis . Therefore they can hardly be applied to substructure-level molecular graph representation . Different from Jin et al . ( 2018 ; 2020a ) which use manual rules to extract substructures , we automatically extract common substructures which better capture the regularities in molecules for substructure-level decomposition . 3 APPROACH . We first give the definition of graph pieces and algorithms for graph piece extraction in Section 3.1 . Then we describe the encoder of our graph piece variational autoencoder ( GP-VAE ) model in Section 3.2 . Finally , we descibe the two-step decoder of GP-VAE in Section 3.3 . Algorithm 1 : Graph Piece Extraction Input : A set of graphs D and the desired number N of graph pieces to learn . Result : A set of graph pieces S and the counter F of graph pieces . 1 begin 2 S ← { GraphToSMILES ( 〈 { a } , ∅〉 ) } ; . Initially , S corresponds to all atoms a that appear in D. 3 N ′ ← max ( N , |S| ) ; 4 while |S| < N ′ do 5 F ← EmptyMap ( ) ; . Initialize a counter . 6 foreach G in D do 7 forall 〈Pi , Pj , Ẽij〉 in G do 8 P ← Merge ( 〈Pi , Pj , Ẽij〉 ) ; . Merge neighboring graph pieces into a new graph piece . 9 s← GraphToSMILES ( P ) ; . Convert a graph to SMILES representation . 10 F [ s ] = F [ s ] + 1 ; . Update the counter , the default value for a new s is 0 . 11 end 12 end 13 s = TopElem ( counter ) ; . Find the most frequent merged graph piece . 14 P ← SMILESToGraph ( s ) ; . Convert the SMILES string to graph representation . 15 S ← S ∪ { s } ; D′ ← { } ; 16 foreach G in D do 17 G′ ← MergeSubGraph ( G , P ) ; . Update the graph representation if possible . 18 D′ ← D′ ∪ { G′ } ; 19 end 20 D ← D′ 21 end 22 end 3.1 GRAPH PIECE A molecule can be represented as a graph G = 〈V , E〉 , where V is a set of nodes that correspond to atoms and E is a set of edges that correspond to chemical bonds . Instead of using atoms , we use substructures , which we call graph pieces , as building blocks . We define a graph pieceP as a subgraph 〈Ṽ , Ẽ〉 that appears in a graph G , where Ṽ ⊆ V and Ẽ ⊆ E . It should be noted that either a single atom or a whole graph is a valid graph piece . Given a set of graph pieces S , suppose the graph G can be decomposed into n graph pieces in S , then G can be alternatively represented as 〈 { Pi } , { Ẽij } 〉 , where Ẽij denotes the set of edges between two neighboring graph pieces Pi and Pj . The decomposition of a graph G into graph pieces satisfies the following constraints : ( 1 ) the union of all atoms in the graph pieces equals to all atoms in the molecule , namely ⋃ i Vi = V ; ( 2 ) there is no intersection between any two graph pieces , namely ∀i 6= j , Vi ∩Vj = ∅ , and Ẽi ∩ Ẽj = ∅ ; ( 3 ) the union of all connections within and between graph pieces equals to all bonds in the molecule , namely ⋃ i , j ( Ẽi ∪ Ẽij ) = E , where i range from 1 to n and j range from i+ 1 to n. Figure 2 shows an decomposed molecule . The algorithm for extracting graph pieces from a given set of graphs D is given in Algorithm 1 . Our algorithm draws inspiration from Byte Pair Encoding ( Gage , 1994 , BPE ) . Initially , a graph G inD is decomposed into atom-level graph pieces and the vocabulary S of graph pieces is composed of all unique atom-level graph pieces that appear in D. Given the number N of graph pieces to learn , at each iteration , our algorithm enumerates all neighboring graph pieces and edges that connect the two graph pieces in G , namely 〈Pi , Pj , Ẽij〉 . As 〈Pi , Pj , Ẽij〉 is also a valid subgraph , we merge it into a graph piece and count its occurrence . We find the most frequent merged graph pieceP and add it into the vocabulary S. After that , we also update graphs G in D that contain P by merging 〈Pi , Pj , Ẽij〉 into P . The algorithm terminates when the vocabulary size exceeds the predefined number N . Note that we use SMILES ( Weininger , 1988 ) to represent a graph piece in our algorithm1 , therefore we ensure the uniqueness of a graph piece . A running example of our graph piece extraction algorithm is illustrated in Figure 3 . At test time , we first decompose a molecular graph into atom-level graph pieces , then apply the learned operations to merge the graph pieces into larger ones . This process ensures there is a piece-level decomposition for an arbitrary molecule . We provide the pseudo code for the piece-level decomposition in Appendix A for better understanding . We provide the complexity analysis for both algorithms in Appendix B . 1We use RDKit ( www.rdkit.org ) to perform the conversion between molecular graph and SMILES .

Summary: This paper proposes a new molecular graph generation method and empirically shows its effectiveness. The proposed method first decomposes molecular graphs into smaller parts (called graph pieces), followed by training a variational autoencoder so that it can generate the collected graph pieces. The trained model is expected to generate a variety of graphs with desirable properties.