Deep Generative Model for Periodic Graphs
Anonymous Author(s)
AffiliationAddress email
Abstract
Periodic graphs are graphs consisting of repetitive local structures, such as crystalnets and polygon mesh.
Their generative modeling has great potential in real-worldapplications such as material design and graphics synthesis.
Classical models eitherrely on domain-specific predefined generation principles (e.g., in crystal net design),or follow geometry-based prescribed rules.
Recently, deep generative models haveshown great promise in automatically generating general graphs.
However, theiradvancement into periodic graphs has not been well explored due to several keychallenges in 1) maintaining graph periodicity; 2) disentangling local and globalpatterns; and 3) efficiency in learning repetitive patterns.
To address them, thispaper proposes Periodical-Graph Disentangled Variational Auto-encoder (PGDVAE), a new deep generative model for periodic graphs that can automaticallylearn, disentangle, and generate local and global graph patterns.
Specifically, wedevelop a new periodic graph encoder consisting of global-pattern encoder andlocal-pattern encoder that ensures to disentangle the representation into global andlocal semantics.
We then propose a new periodic graph decoder consisting of localstructure decoder, neighborhood decoder, and global structure decoder, as well asthe assembler of their outputs that guarantees periodicity.
Moreover, we design anew model learning objective that helps ensure the invariance of local-semanticrepresentations for the graphs with the same local structure.
Comprehensiveexperimental evaluations have been conducted to demonstrate the effectivenessof the proposed method.
The code and appendix is available in the Supplemental
Material.
Introduction
[Figure]
Graphs is a ubiquitous data structure that describes objects and their relations.
As a specialtype of graphs, periodic graphs are graphs thatconsist of repetitive local structures.
They naturally characterize many real-world applicationsranging from crystal nets containing repetitiveunit cells to polygon mesh data containing repetitive grids (Figure 1).
Hence, understanding,modeling, and generating periodic graphs havegreat potential in real-world applications suchas material design and graphics synthesis.
Different from general graphs, modeling generative process of periodic graphs requires to ensure theperiodicity.
Therefore, conventional graph models such as random graphs, small-world models cannotbe used effectively.
Classical studies on periodic graph generative models have a long history, whereconventional methods can be categorized into two types: domain specific-based and geometry-based.
Specifically, domain specific-based models generate periodic graphs relying on properties handcraftedbased on domain knowledge [49, 5].
In addition, geometry-based methods generate periodic graphseither according to predefined geometric rules to craft the targeted periodic graphs [18, 45] or simplygluing the initial fragment into periodic graphs [33].
As mentioned above, both domain specific-based and geometry-based generative models for periodicgraphs require predefined properties that are either domain-specific or geometric-based, enablingthem to fit well towards the properties that the predefined principles are tailored for, but usuallythey cannot do well for the others.
For example, a geometry-based generative model can generategraphs within the predefined geometric rules but cannot generalize to other geometric properties.
However, in many domains, network properties and generation principles are largely unknown,such as those explaining the thermal probability [42] and solubilities [15] of materials.
The lack ofpredefined principles leads to the incapability of existing methods to model periodic graphs.
Hence, itis imperative to fill the gap between the lack of powerful models and the complex generative process.
Expressive generative models that can directly learn the underlying data generative process automatically have been an extremely challenging task to accomplish, until very recent years when theadvancement of deep generative models have exhibited great success in data such as images andgraphs [39, 21].
Under this area, deep graph generative models aim at learning the distribution ofgeneral graph data representation extracted by graph neural networks.
However, despite the progressin graph neural networks for general graphs, they cannot be directly applied to handle periodic graphsdue to the additional requirements of the generated graphs such as periodicity.
It requires significantefforts to advance the current deep graph generative models in order to handle periodic graphs dueto the following unique challenges: (1) Difficulty in ensuring the periodicity of the generatedgraphs .
Existing works for generic graph generative models typically are not able to identify orgenerate the “basic unit”, namely the repetitive local subgraphs.
The conventional learning objectivesuch as reconstruction errors or adversarial errors can only measure the generated graph qualityglobally but cannot ensure the identity of local structures as well as how different local structuresare connected.
(2) How to jointly learn local and global topological patterns .
In periodic graph,there are two key patterns, namely the local structure which specifies the basic unit and the globaltopology which characterizes how local structures are connected to form the whole graph topology.
Therefore, it is imperative for the generative models to be able to factorize these two different patternsfor respective characterization and control.
However, existing works for general graphs only learn arepresentation for the graph instead of disentangling them into different representations for local andglobal patterns; and (3) Efficiency in learning large periodic graphs .
Due to the ubiquitousness ofrepetitive structures, periodic graphs are full of redundant information which makes existing worksfor generic graphs highly inefficient to model them.
It is imperative to establish models that canautomatically deduplicate the modeled generative process of periodic graphs.
To address these challenges, we developed Periodic-Graph Disentangled Variational Auto-encoder(PGD-VAE), a deep generative model for periodic graphs that can automatically learn, disentangleand generate local and global graph patterns.
Specially, we proposed a novel encoder that containsa local-pattern encoder and a global-pattern encoder to disentangle the local and global semanticsfrom the periodic graph representation.
Besides, we designed a periodic graph decoder consisting ofa local structure decoder, a global structure decoder, a neighborhood decoder and an assembler toassemble local and global patterns while preserving periodicity.
To achieve the disentanglement oflocal and global semantics, we proposed a new learning objective that can simultaneously ensurethe invariance of the local-semantic representations of graphs that have the same basic unit.
Ourcontributions can be summarized as follows:
• We propose a new problem of end-to-end periodic graph generation by deep generative models.
• We develop PGD-VAE, a novel variational autoencoder that can decompose and assemble periodic graphs with the learned patterns of basic units, their pairwise relations, and their local
neighborhoods.
• We propose a new learning objective to achieve the disentanglement of the local and globalsemantics by ensuring the invariance of the local-semantic representations of graphs that containthe same local structure.• We conducted extensive experiments to demonstrate the effectiveness and efficiency of the proposedmethods with multiple real-world and synthetic datasets.
Related works
2.1 Graph Representation Learning and Graph Generation
Surging development of deep-learning techniques, recent work on graph representation learning[25, 9,26], especially graph neural networks (GNNs)
[47, 59], is attracting considerable attention.
Significantsuccess has been achieved on developing various GNNs models to handle domain-specific graphs,such as traffic network [53, 55, 8], social network [57, 16], knowledge graph [2, 12, 43] and materials structure [37, 48, 34].
Graph generation is one of the most important topics among the domain ofgraph representation learning.
Due to the development of deep generative models, most GNN-basedgraph generation models borrow the framework of VAE [29, 30, 27] or generative adversarial nets(GANs)
For example, GraphVAE learns the distribution of given graphs and generatessmall graphs under the framework of VAE [41].
By contrast, MolGAN was proposed to generatesmall molecular graphs by means of GANs [13].
2.2 Disentangled Representation Learning
Disentangled representation learning has been attracting considerable attentions of the community,especially in the domain of computer vision [7, 44, 56].
The goal of disentangled representationlearning is to separate underlying semantic factors accounting for the variation of the data in thelearned representation.
This disentangled representation has been shown to be resilient to thecomplex factors involved [3], and can enhance the generalizability and improve robustness againstadversarial attack [22, 1].
Intuitively, disentangled representation learning can achieve superiorinterpretability over regular graph representation learning tasks to better understand the graphs invarious domains [23].
This motivates the surge of a few VAE-based approaches that modify the VAEobjective by adding, removing or adjusting the weight of individual terms for deep graph generationtasks [22, 10, 28].
Moreover, Martinkus et al.
[36] proposed GAN-based model global and localgraph structure via graph spectrum.
Disentangled representation learning is important in modelingperiodical graphs where the process requires to distinguish the repeatable patterns from the others.
2.3 Periodical Graph Learning
Periodic graph is the graph that consists of repetitive basic units in its structure, and can fit into thedata structure in a number of domains, such as crystal structures in materials science [4, 14], polygonmesh in computer graphics [17] and motion planning in robotics [19].
Several graph representationlearning tasks have been conducted on this type of graph.
For example, periodic graph-structuredinformation in the airport network can be formatted into a periodic graph and a graph convolutionalnetwork-based (GCN-based) was developed for flight delay prediction [6].
The multiple periodicspatial-temporal graphs of human mobility was employed to learn the community structure based ona collective embedding model [46].
In addition, graph generation model in periodic graphs can becategorized in to two scenarios: (1) domain specific-based models and (2) geometry-based models.
For example, CDVAE borrows specific bonding preferences between different atoms and generatesthe periodic structure of materials in a diffusion process [49].
Moreover, [38] achieved geometricconstruction of periodic graphs by projecting two-dimensional hyperbolic tilings onto triply periodicminimal surfaces.
They both need predefined rules to generate valid periodic graphs.
Besides, theperiodic graphs are usually large, the large time or space complexity of existing graph generativemodels limits the advance of periodic graph generation, motivating us to propose an end-to-endperiodic graph generative model, PGD-VAE, that bears superior scalability on large periodic graphs.
Methodology
In this section, the problem formulation is first provided before moving on to derive the overallobjective, following which the novel encoder of PGD-VAE that can respectively extract local andglobal information of the graph structure is introduced.
Afterwards, the decoder that generates localand global structure, and the algorithm of the assembler are described.
3.1 Notations and Problem Definition
3.1.1 Periodic graphs
Define a periodic graph as G =
( V , E , A ) , where V is the set of nodes and E ⊆
V × V is the setof edges of the graph G .
The repeated local structures can be defined as basic units of size n (i.e.,number of nodes is n ).
Suppose the periodic graph G consists of m basic units as well as theirconnections, this means the node set V = ∪ mu =1 v i,u is the union of nodes in all basic units, and
E = {∪ mu =1 e ij,u } ∪ {∪ mu ̸ = v e ij,i ∈ u,j ∈ v } is the union of all edges within and across basic units of thegraph.
A ∈ R N × N corresponds to the adjacency matrix of the graph,
in which A ij = 1 if e ij ∈
Eand
A ij = 0 otherwise.
In addition, we define three matrices: (1) A ( l ) ∈
R n × n is the adjacencymatrix of the basic unit of the graph; (2) A ( g ) ∈
R m × m is the adjacency matrix denoting the pairwiserelations among basic units; and (3) A ( n ) ∈ R n × n is the incidence matrix regarding how the nodes inadjacent basic units are connected.
Without loss of generality, all the adjacency matrices stick to aspecific and intrinsic node ordering (i.e., permutation) where A ( l ) ’s occupy the diagonal blocks in A .
[Figure]
Finally, the generated graph is obtained by assembling A ( l ) , A ( n ) and A ( g ) .
Note that A sticks to the intrinsic node ordering and permutation naturally followed by the graphs in the training set.
The reason that we say it is "intrinsic" is because such node ordering is usually naturally provided bythe data (see the dataset description in Section 4.1 for details).
3.1.2 Represent A by A ( l ) , A ( g ) , and A ( n )
As the adjacency matrix A of the periodic graph contains repeated local structures, we want to use 157 an alternative “deduplicated” way to represent A efficiently.
This will be achieved by conciselyrepresenting A with a function of A ( l ) , A ( g ) , and A ( n ) :
A = f ( A ( l ) , A ( g ) , A ( n ) ) , which is detailedin the following and illustrated in the right part of Figure 2.
First, define a binary block matrix
P ∈ { 0 , 1 } nm × m s.t.
P · P T =
J , where J is the block diagonal matrix whose diagonal blocks areall 1’s.
Then, ˆ A ( g ) , the augmentation of A ( g ) can be achieved as:
[Equation]
Additionally, we define the matrix Q ∈ R nm × n as the concatenation of m identity matrices ofsize n .
Then we can obtain ˆ A ( l ) and ˆ A ( n ) , the repetition of A ( l ) and A ( n ) , respectively, along bothdimensions by m :
[Equation]
Consequently, we can deterministically obtain A from A ( l ) , A ( g ) and A ( n ) via the function f :
[Equation]
where M ( l ) and M ( n ) are two mask matrices.
Specifically, the off-diagonal blocks of size n in M ( l )are 0’s while the diagonal blocks in M ( l ) are 1’s.
We also specify that the upper-triangular blocks ofsize n in M ( n ) are 1’s and other blocks in M ( n ) are 0’s.
In the following, we may exchangeably use
G and A to denote the whole graph since it is determined by its topology.
3.1.3 Deep generative models of periodic graphs
Learning generative models for a periodic graph aims to learn the conditional distribution p ( G | Z ) .
Define Z =
( z l , z g ) as the learned graph representation based on the structure of graphs.
z l isdesigned to capture the local semantic factors that characterize the topological structure of basic units,while z g is designed to represent the global semantics on how basic units are topologically connected.
3.2 Learning Deep Periodic Graph Generative models
We first introduce the variational inference of the generative model and learning objective.
Then weintroduce our local and global encoders, as well as the new periodic graph decoder.
3.2.1 Periodic graph generative model inference
As discussed in the problem definition, we aim to learn the conditional distribution of G given 181 Z
= ( z l , z g ) .
So we aim to maximize the marginal likelihood of the observed periodic graph G inexpectation over the distribution of the latent variables ( z l , z g ) .
For a given periodic graph G , wedefine the prior distribution of the latent representation as p ( z l , z g ) which is intractable to infer.
As 184 a result, we leverage variational inference, in which the posterior distribution is approximated byq ϕ
( z l , z g | G ) .
Hence, another goal is to minimize the Kullback–Leibler (KL) divergence between the 186 true prior and the approximated posterior distributions.
To encourage the disentanglement of z l andz g , we introduce a constraint by matching the inferred posterior configurations of the latent factorsq ϕ ( z l , z g | G ) to the prior distribution p θ ( z l , z g ) :
[Equation]
4) where D refers to the observed graphs and I is a constant.
The detailed derivation of the VAE 190 objective can be seen at Appendix A.
Then, we can decompose the constraint term in Eq.
4 given theassumption that z l and z g are independent, leading to:
[Equation]
Based on the problem formulation, G can be represented as { A ( l ) , A ( g ) , A ( n ) } .
Since z l shouldcapture and only capture the local patterns, thus A ( l ) and A ( n ) are dependent on z l and conditionallyindependent from each other given z l , namely A ( l ) ⊥ A ( n ) | z l .
And z g should capture and onlycapture the global patterns, we have
[Equation]
Then, the objective function can be written as:
[Equation]
where I l and I g are the information bottleneck to control the information captured by the latent vector 198 z l and z g .
Based on the objective we derived above, we propose a novel objective for PGD-VAE by 199 extending Eq.
7 in order to not only fulfill the optimization objective of the general VAE framework(Eq. 4), but also secure the disentanglement of local and global semantics of the graph.
Specifically, 201 we force the z l equal for graphs with the same basic unit (Eq. 8).
At the meanwhile, we distinguish 202 z l ’s of graphs with different basic units (Eq. 8).
[Equation]
where z ( j ) l,i and z ( j ) g,i are respectively the local and global representations of the j -th graph for thei -th basic unit.
The derived objective function for PGD-VAE (Eq.
8) can meet the objective in ourproblem formulation, as stated in Theorem 3.1.
The proof of Theorem 3.1 is provided in Appendix B.
Theorem 3.1.
The proposed objective function in Eq.
8 guarantees that z l and z g satisfy:
z l capture and only capture the local semantic of the graph;
z g capture and only capture the global semantic of the graph.
3.2.2 Learning objectives
To optimize the overall objective in Eq.
8 while satisfying the constraint, we transform the inequality 211 constraint into tractable form.
Given that I l and I g are constants, we rewrite the first two constraintsof Eq.
8 using Lagrangian algorithm under KKT condition as:
[Equation]
(9) where G represents a set of observed graphs, the Lagrangian multipliers β 1 and β 2 are regularizationcoefficients that constraint the capacity of latent information channels z l and z g , respectively, and putimplicit pressure of independence on the learned posterior.
Additionally, the goal of the last two constraints of Eq. 8 is to enforce the graphs that contain the 217 same repeat patterns (i.e., local information) sharing the same z l and the graphs that contain differentrepeat patterns having different z l .
Thus, to both minimize the distances of z l among the graphs with the same unit cell and maximize the distances of z l among the graphs with different unit cells, we 220 propose to minimize a contrastive loss L contra as:
[Equation]
where τ is a predefined temperature parameter, and β 3 is the weights added on the contrastive loss toadjust the convergence of z l in graphs with the same basic unit and the divergence in graphs withdifferent basic units.
sim () is the cosine similarity borrowed in computing the contrastive loss [52].
In summary, the final implemented loss function minimized L includes three parts: (1) L rec to 225 minimize the reconstruction loss of the input and generated graphs; (2) L KL to minimize the KLdivergence of the posterior and its approximation; and (3) L contra , the contrastive loss to force z l 227 equal for graphs with the same basic unit and different for those with different basic units:
[Equation]
229 3.3 Architecture of PGD-VAE
Based on the above periodic graph generative model and its learning objective, we are proposingour new Periodic Graph Disentangled VAE model (PGD-VAE).
In general, PGD-VAE takes thewhole adjacency matrix A as the input and can automatically identify A ( l ) , A ( g ) and A ( n ) by modelarchitecture and learning objective.
Specifically, PGD-VAE contains a local-pattern encoder anda global-pattern encoder to disentangle the graph representation into local and global semantics,respectively.
It also contains a local structure decoder, a global structure decoder, a neighborhooddecoder together with an assembler to formalize the graph while ensuring the periodicity.
An overview of PGD-VAE is shown in Figure 2.
The proposed framework has two encoders, eachof which models one of the distributions q θ
( z g | G ) and q θ ( z l | G ) ; and three decoders to modelp θ ( A ( l ) | z l ) , p θ ( A ( n ) | z l ) , and p θ ( A ( g ) | z g ) , respectively.
Each type of representations is sampledusing its own inferred mean and standard derivation.
For example, the representation vectors z l aresampled as z l = µ l + σ l ∗ η , where η follows a standard normal distribution.
3.3.1 Encoder
The encoder of PGD-VAE is designed to learn z l and z g such that z l captures only the local semanticsof the graph and z g captures only the global semantics of the graph.
Thus, we propose a local-patternencoder to encode the local information and a global-pattern encoder to encode the global information,as illustrated in Figure 2.
Local-pattern encoder We designed the encoder to learn an assignment matrix
A rep ∈ R C × Nto identify representative nodes of the graph (Eq. 10) where C is predefined as the number of 249 representative nodes to be assigned.
This node embedding clustering module can enforce z l to onlycapture the local information, namely excluding the global information (e.g., size of the graph) from 251 z l , because: 1)
Each node embedding preserves local pattern since GIN embeds local information 252 of nodes; 2) Since node embedding of periodic graph is periodic, embedding of a node in a basicunit is the same as the corresponding node in another basic unit.
By clustering all node embedding, 254 each cluster preserves repetitive node embedding patterns across different local regions.
In oursetting, these representative nodes capture different embedding with each other.
The column of A rep 256 corresponding to the node v can be calculated:
[Equation]
where h ( K ) v is the embedding of node v at the last layer of Graph Isomorphism Network (GIN) 258 [50].
GIN is provably the most expressive among the class of GNNs and is as powerful as the
WeisfeilerLehman graph isomorphism test [50].
To preserve all information contained by the 260 representative nodes, We concatenate all representative node embedding to obtain µ l and σ l :
[Equation]
where D A rep has the degree of each cluster in its diagonal.
z l can then be sampled according the
VAE framework: z l ∼ N ( µ l , σ l ) .
Global-pattern encoder To enforce that z g captures all the global information, GIN is leveraged to 264 generate node-level embedding of the graph.
Additionally, PGD-VAE leverages sum aggregator asthe graph-level readout (Eq. 12) to obtain the graph-level embedding z g while achieving maximal 266 discriminative power [50]. [Table] [Table]
[Equation]
The h ( K ) v represents the embedding of node v obtained at the last layer of GIN.
z g can then be sampleaccording to the VAE framework:
z g ∼ N ( µ g , σ g ) .
3.3.2 Decoder
As shown in Figure 2, the decoder contains four parts: (1) the local-structure decoder to generate
A ( l ) ; (2) the global-structure decoder to generate A ( g ) ; (3) the neighborhood decoder to generate
A ( n ) ; and (4) an assembler f : { A ( l ) , A ( g ) , A ( n ) }
→ A which generates A according to Eq.
The local structure decoder takes the local-semantic representation z l as the input to generate thestructure of the basic unit.
The global structure decoder takes the global-semantic representation z gas the input to generate the global organization of basic units.
Since how basic units are connectedis regarded as the local structure of the graph, the neighborhood decoder takes the local-semanticrepresentation
z l as the input to generate the connection between adjacent basic units.
The assemblerf is a closed-form function defined in Section 3.1 to assemble A ( l ) , A ( g ) and A ( n ) into A :
[Equation]
Here f l , f g , f n are deterministic functions serving as three decoders to generate A ( l ) , A ( g ) and A ( n ) ,respectively.
They are approximated by three multilayer perceptrons (MLPs) in implementation.
Edges are sampled from the Bernoulli distribution given the probability output of MLP.
3.4 Complexity Analysis
We theoretically evaluate the scalability of our model and comparison models in the number ofgeneration steps, similar to the analysis in [35].
Since the decoder of PGD-VAE follows an one-shotstrategy using MLP, it requires O (1) generation steps.
Additionally, the space complexity of PGDVAE is O ( max { n 2 , m 2 } ) , since the generated graph is determined by basic units, the connection ofadjacent basic units and the global pattern.
The graph size can be calculated as mn .
The number ofgeneration steps of PGD-VAE is aligned with that of VGAE, but is lower than that of GraphVAE and
GRAN.
Nevertheless, PGD-VAE has the smallest space complexity.
For time complexity, PGD-VAEcan achieve O ( m 2 + n 2 + mn ) by using matrix replication and concatenation tricks in implementation,better than that of other comparison models.
The number of generation steps, time complexity andthe space complexity of PGD-VAE and comparison models are summarized in Table 1.
Experiments
This section reports the results of both quantitative and qualitative experiments that were performedto evaluate PGD-VAE with other comparison models.
Two real-world datasets and one syntheticdatasets of periodic graphs were employed in the experiments.
All experiments were conducted onthe 64-bit machine with an NVIDIA GPU, NVIDIA GeForce RTX 3090.
4.1 Data
Two real-world datasets and one synthetic dataset were employed to evaluate the performance of PGD-VAE and other comparison models. All the comparison and our proposed methods only take the [Table]
Also, in all the datasets, the adjacency matrix of each graph followed its intrinsic permutation whenthey were generated, namely all the diagonal blocks are occupied by the basic unit A ( l ) , as shown in
Figure 2.
So all the graphs generated by all the models trained by these datasets also follow suchfixed permutation which naturally saves them from considering node permutation.
1) QMOF is apublicaly available database of computed quantum-chemical properties and molecular structures [40].
We selected metal–organic frameworks (MOFs) from QMOF consisting of unit cells as the basicunit (i.e., A ( l ) ).
A ( g ) and A ( n ) can be automatically determined by Atomic Simulation Environment(ASE)
Finally, the whole adjacency matrix A was the assemble of A ( l ) , A ( g ) and A ( n ) via Eq.3. 2) MeshSeg contains 380 meshes for quantitative analysis of how people decompose objects intoparts and for comparison of mesh segmentation algorithms [11].
We extracted meshes from MeshSeqand made 10 replicates for each mesh.
The whole adjacency matrix A was directly obtained from thegraphs in this dataset. 3)
Synthetic dataset .
Three synthetic datasets were generated based on threedifferent basic units (i.e., A ( l ) ), namely triangle, grid and hexagon.
For each of the above basic unittypes, different numbers of basic units were used to compose the whole graph (i.e., represented bythe whole adjacency matrix A ) via Equation (4), with self-designed A ( g ) and A ( n ) .
All adjacencymatrices were padded into a fixed size with zeroes in implementation.
The statistics of the threedatasets have been summarized in Appendix, Table 1 and more details can be found in Appendix C.
4.2 Comparison Models
Three state-of-the-art deep generative models were employed as comparison models to comparewith PGD-VAE 1 : (1) VGAE , a VAE-based graph generative model [30] by learning the graph-levelrepresentation and using MLP to generate the adjacent matrix; (2) GraphRNN , a deep autoregressivemodel that accounts for non-local dependencies of edges by decomposing generation process intoa sequence of node and edge conformations [51]; and (3) GRAN , a deep autoregressive model [35]which generates a block of nodes and associated edges at a time, and captures the auto-regressiveconditioning between the already-generated and to-be-generated pieces of the graph using GNNs withattention.
PGD-VAE and all comparison methods only take the whole adjacency matrix A as input,without extra information on A ( l ) , A ( g ) , or A ( n ) .
Noticeably, we used fixed permutation that restrictsthe adjacency matrix A to fixed node ordering as illustrated in Figure 2.
Graphs with predefinedpermutation can easily be provided in periodic graph datasets as aforementioned.
In addition, three ablation study are conducted by evaluating four variants of PGD-VAE: (1) PGD-VAEwithout regularization term; (2) PGD-VAE replacing GIN with GCN in the encoder; (3) PGD-VAEwith a single MLP as decoder.
The comparison models are implemented using their default settings.
The implementation details of PGD-VAE are presented in Appendix D.
4.3 Evaluating the Generated Graphs
Quantitative evaluation .
To quantitatively evaluate generation performance of PGD-VAE andcomparison models, we compute the KL Divergence (KLD) between generated and ground-truthgraphs to measure the similarity of their distributions regarding: (1) average clustering coefficient and(2) density of graphs [24].
Moreover, we adopt uniqueness and novelty to evaluate generated graphs.
The uniqueness measures the diversity of generated graphs.
The novelty reflects whether the modelhas generated unseen graphs by measuring the proportion of generated graphs that are not in the setof training graphs.
The KLD-based results were calculated and shown in Table 2.
The results showthat, PGD-VAE achieves KLD smaller than comparison models by 0.8191 for clustering coefficientand 0.5062 for graph density on average on MeshSeq dataset, and 2.4275 for clustering coefficient aswell as 1.7611 for graph density on average on Synthetic dataset.
Interestingly, on QMOF dataset,
PGD-VAE that replaces GIN with GCN achieves KLD of 0.8508 regarding clustering coefficientand 0.9332 regarding graph density, outperforming other models, which is likely due to the fact thatthe basic unit of MOF is usually more complex and needs more powerful GNN to capture the localinformation of the graph.
The measuring of uniqueness and novelty of generated graphs by differentmodels has been summarized in Table 3.
PGD-VAE and all its variants achieve better performancethan GraphRNN and GRAN, as they tend to generate unique graphs, which is desired for tasks suchas materials designing and object synthesis.
Qualitative evaluation for disentanglement To validate the disentanglement of local and globalsemantics in the graph representation, we qualitatively explore whether PGD-VAE can uncover anddisentangle these two latent factors.
To this end, we continuously changing the value of z l or z g whilekeeping another latent vector fixed and simultaneously visualizing the change of generated graphs.
Figure 3 shows the variation of generated graphs when traversing two latent factors z l and z g .
[Figure]
358 A case study of generating MOFs We showcase several representative periodic graph structures generated by the proposed PGD-VAE for MOF design in this section.
As shown in Figure 4, there are threediverse and novel generated MOFs.
The first one can be interpreted as C 16 H 8 Na 8 Cu 4 N 16 S 16 , forwhich PGD-VAE have generated the basic unit (i.e., C 4 H 2 Na 2 CuN 4 S 4 ) that are connected witheach other by incident chemical bonds.
The second one can be interpreted as C 24 H 32 Cl 8 Cu 4 N 8 ,which is highly different from the first one in terms of atom diversity.
The third generated MOF is
C 16 H
Mg 8 O 20 S 20 , which has a larger basic unit, namely, C 4 H
Mg 2 O 5 S 5 .
In addition, thebasic units of these generated structures are all constructed in a periodic way with high validity anddiversity, which demonstrates the effectiveness of the proposed PGD-VAE.
More details of the MOFdecoration and visualization are provided in Appendix E.
Evaluating scalability We evaluated the time complexity of PGD-VAE and comparison modelsby running 10 epochs on graphs in stratified synthesis dataset.
Expectedly, all comparison modelsconsume more time than PGD-VAE.
The details are presented in Appendix F.
Conclusion
We propose PGD-VAE, a novel deep generative model for periodic graphs that can automaticallylearn, disentangle, and generate local and global patterns.
Our method can guarantee the periodicityof generated graphs and outperforms existing graph generative models in periodic graph generation,both quantitatively and qualitatively.
In the future, we attempt to introduce the node and edge featuresinto the model to advance domains such as materials design and object synthesis.
Checklist
For all authors.
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
Our work
does not have potential negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Please see the
code and instructions in the Supplemental material.
(b) Did you specify all the training details (e g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] As shown in Table ??
(d) Did you include the total amount of compute and the type of resources used (e g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] Please see the experiments section
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The details to train the comparison models are presented the Table ??
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A] They are data of graphics and materials that
do not have personally identifiable information or offensive content
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
We do not use crowdsourcing or conducted research with human
subjects in the work.
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
We do not use crowdsourcing or conducted
[Table]
research with human subjects in the work.
[Table] (c) Did you include the estimated hourly wage paid to participants and the total amount [Table]
spent on participant compensation?
[N/A] We do not use crowdsourcing or conducted [Table]
research with human subjects in the work.
[Table]