Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work proposed a novel VAE-based method to generate periodic graphs with applications to MOF, mesh and synthetic graphs generation. The proposed VAE-based method (PGD-VAE) disentangles the graph representation into local and global semantics by generating $A^{(l)}, A^{(g)}, A^{(n)} $ and then assembles them into a complete periodic graph. Overall, I enjoyed reading the paper. But there are some concerns to be addressed.
strengths_and_weaknesses: Strengths:
* The paper is well-written and easy to follow
* To the best of my knowledge, the proposed PGD-VAE is novel in generating periodic graphs, and the proposed disentangled periodic graph representation is interesting.
* The results are impressive on the evaluation metrics.

Weaknesses:
* Why not use the disentangled periodic graph representation as input to the PGD-VAE as well?
* More Ablation studies on the necessity of disentanglement should be added to validate the claim of the benefits of using disentangled generation.
* The generation results look good on the evaluation metrics. However, it is unknown how the generated graphs can be applicable to real-world applications. For instance, are the generated MOF structures stable and synthesizable? 
questions: Please refer to the *weaknesses* for my questions.
limitations: The limitation of this work is that the generated periodic graphs may not be applicable to real-world applications.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: Generating novel periodic graphs is an interesting and important task as the application examples given in the paper. Three challenges towards this task are provided by the authors, which seem reasonable to me. To address these challenges, the authors propose a novel model to generate periodic graphs by automatically learning, disentangling, generating and assembling local and global patterns of periodic graphs. This idea of “hierarchically”generating periodic graphs is novel and it is suitable and efficient to handle this type of graphs as suggested by the complexity analysis part. In addition, the experimental results show the effectiveness of this model compared with other comparison models.

strengths_and_weaknesses: Strengths:
1 The framework proposed by the paper solves an important task to generate periodic graphs and is novel to me as it generates periodic graphs in a hierarchical fashion. This strategy looks natural since the periodic graph (e.g., graph structure of materials) can be very huge and is intuitively more efficient since it only needs to learn and generate local and global patterns. Also, to my understanding, the node embedding clustering module embedded in the local-pattern encoder can learn the local structure of the graph and perform at least as well as the sum pooling on the whole graph. 
2 The analysis on the model is comprehensive both theoretically and experimentally. The learning objective has been derived properly. The complexity of the model is better than the comparison models as shown in Table 1. The proposed model also has better performance than the comparison models.
3 The presentation of the paper is good to me. Figure 2 is organized very well and is intuitive to show how the framework functions. The ability to disentangle local and global patterns has been demonstrated by Figure 3.
4 The authors published the code.


Weakness:
1 Some typos: for example, in Figure 2, it should be A^{(l)}, A^{(g)} and A^{(n)} rather than A_l, A_g and A_n.
2 Missing references: for example, [1] below aims to generate periodic structures, although seems using different strategies and different type of input data.
 
[1] Xie, Tian, et al. "Crystal diffusion variational autoencoder for periodic material generation." arXiv preprint arXiv:2110.06197 (2021).

questions: 1. The author can add and discuss the missing references, such as [1] in the Related Works session.
2. Please check and fix the typos, e.g., that in Figure 2.

limitations: The limitations of the proposed work are discussed in the Conclusion session.
This work does not have potential negative societal impacts.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper focuses on the problem of periodic graph generation, and proposes PGD-VAE, a Variational AutoEncoder based generative model for periodic graphs that can automatically learn, disentangle, and generate local and global patterns. The authors formulate the generation of periodic graphs as the generation of a finite number of basic units.
strengths_and_weaknesses: ## Strengths
* originality: This paper formulates a new problem setting for periodic graph generation. Instead of generating basic units and repeating patterns, they generate the periodic graph consist of a fixed number of basic units. 

## Weaknesses
* According to the authors, this paper's main idea of the generation process is to make the generation of local structures and global repeating patterns independent. But they introduce a contrastive loss to make the generation of local structures actually **conditioned** on the global repeating patterns.
* In this paper, the generation of node type such as atom type in QMOF is ignored. Only the structure is generated.
* Permutation invariance is ignored. The proposed method can not satisfy permutation invariance when the node type needs to be considered. The application of the proposed method is limited because of breaking permutation invariance. 
* Because of above considerations, the quality and significance of the proposed method is limited.
* Clarity: Hard to read. There are variables coming from nowhere and not shown in the equation, such as Equation (8). Many sentences hard to understand.

questions: * If the goal is to make the generation of local structures and global repeating patterns independent, why do you  introduce a contrastive loss to make the generation of local structures conditioned on the global repeating patterns?
limitations: Yes, they addressed the limitations of the proposed method.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes to tackle periodic graph generation, by learning to generate basic structural units and to compose them into a larger graph. This is achieved by using a hierarchical VAE approach with a loss term which encourages disentanglement.
strengths_and_weaknesses: Mostly, current deep graph generation approaches aim to generate the graph by focusing on local structure so any attempt to introduce more focus on the global structure or decouple global and local structure generation is very welcome. Recently, a more general approach to decouple local and global structure generation, not focused on periodic graphs  has been proposed [1]. I think this should be mentioned in the related work. I haven't seen other work that focuses in particular on periodic graph generation using deep learning. Which makes this particular problem quite novel and potentially useful in problems such as crystal structure design.

What I find problematic, is the assumption that the node order in the graph is known and can be consistently determined.
First, it limits us to a small set of problems where we can even hope to determine such an ordering (e.g. atom lattices). Further, even for the problems considered here it's not entirely clear to me that we can determine such an ordering. For example lets consider the MeshSeq dataset. In this dataset we have a sequence of cuts people used to partition a given mesh. Even though the cuts are sequential, they gives us two equivalent parts (we build a binary tree, not a sequence) and depending on the particular mesh of the same type of object or its orientation which part is the "left" of the split and which is "right" could differ. Also, In general for mesh processing various equivariant architectures are normally used, due to the problem that in general we don't have a way to find a canonical orientation (ordering) of a mesh. So I don't see how we can get a canonical representation of a mesh here to use a loss that is not invariant to permutations.
In context of this an important reference [2] discusses, that for graph generation, you need a permutation invariant loss function. Which we do not have in the approach proposed here. I also think this reference should be included in the paper.
The second problem I see with assuming that we can get this canonical graph structure (ordering) is that in this case we are essentially just dealing with a set of matrices. So we can just use a traditional and very powerful matrix processing models such as convolutional neural networks. In fact the proposed architecture is a bit like a CNN already as it considers a bunch of patches and how they combine/overlap. So under this assumption I'm not even sure we need a GNN-based architecture.

A small note on the Global pattern encoder - it's true that using GIN and global pooling gives you 1-WL representation power, but 1-WL is inherently a local procedure, so I'm not sure that's the best architecture to capture the global graph structure. Models that usually perform better on graph-level tasks could be considered, such as the more expressive GNN architectures or graph transformers, which do not suffer from having only local interactions.

Why only two graph similarity measures (density and clustering coefficient) are used? Even the baselines that were used such as GRAN used more advanced measures, such as orbital counts or spectral features. Using only density and clustering coefficient are determined by local features, so there are no measures that show how well the global graph structure is reproduced. Which is one of the main goals of the work.

I'm also confused why the GraphVAE baseline was not used. It certainly has the closest overall architecture (GNN encoder + MLP decoder). Authors argue that it is too expensive, but the expense comes from the graph matching used to compute a permutation invariant loss. But authors claim, that for the graphs they consider we can assume the node order to fixed as otherwise their proposed method does not work. Also, GRAN used GraphVAE as a baseline on datasets with graphs as large as the ones considered in this paper (Protein graphs used in GRAN paper had up to 500 nodes). In GRAN paper they also just discarded GraphVAE matching loss and used the node ordering as given to GRAN model itself, which worked fine.

Writing quality could certainly be improved. The text is wordy, there's a lot of typos, sometimes words are missing in the sentences, even multiple words per sentence (e.g. line 125). I would strongly suggest the authors to at least use some sort of a spelling and grammar correction software (e.g. Grammarly).

[1] Martinkus et al. "SPECTRE: Spectral Conditioning Helps to Overcome the Expressivity Limits of One-shot Graph Generators." ICML 2022
[2] Vignac and Frossard "Top-N: Equivariant Set and Graph Generation without Exchangeability" ICLR 2022
questions: How exactly is m an n selected for each dataset? As I understood it, you use analytic methods (domain knowledge) to determine the optimal substructure sizes. This already seems to give a lot of ground-truth information for the model.

What ordering do the GraphRNN and GRAN use? The default BFS/DFS orderings that were originally used for these models or do you also provide them with the true graph ordering you use for your model?

In Figure 3, are the trees in the lower left corner part of the dataset? It is unclear which dataset this figure comes from. I presume it to be the synthetic one, but it does not have trees. If it is the synthetic dataset, it seems strange trees are generated even though all base units in the dataset where polygons.
limitations: As I mentioned in the weaknesses, the assumption that the adjacency matrix has a true canonical ordering is quite problematic. Even though for regular graphs in some cases we have something close to a canonical ordering. It essentially goes against all of the graph machine learning wisdom that is based on equivariance to the point that it is unclear we even need graph machine learning to solve this problem when this assumption trully holds.
review_title: null

Meta-review: metareview: This paper proposed an interesting model for generating periodic graphs.  The hierarchical representation of periodic graphs decomposes into local structure and global structure, and greatly reduces the size of the structure to be modeled, which is an interesting contribution.  All reviewers liked the idea of this representation and lean toward acceptance.

I would, however, encourage the authors to include some discussion of the limitations of this approach, in particular the dependence on a stable node ordering, and the fact that the model can only generate perfectly periodic graphs (what if the graph is mostly periodic but with some noise?).
confidence: Certain
Decision: Accept