Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper emphasizes learning disentangled representations from graphs. In contrast to prior works, the authors formulate new definitions of unconditional/conditional symmetry-based graph disentanglements and developed a Lie group-based framework for learning with Graph VAE. The authors evaluate their methods on disentanglement metrics constructed on synthetic datasets and downstream tasks, including few-shot classiﬁcation, compression and molecular graph generation.
strengths_and_weaknesses: 

- ***Originality***: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited. 

  Learning disentangled representaitons on graphs is a 

- ***Quality***: The work is complete, my concerns on the theoretical analysis and experimental results are put at the *clarity* and **Questions**.

- ***Clarity***: The submission is clearly written and well organized. The experimental descriptions are not clear to me. My suggestions are as follows:

  **[Adjacency in GNN architechture]** In lines 88-99, the notation of adjacency should be introduced at the beginning, otherwise how to conduct the graph convolution proceducre? The current formuation sees to be have space for improving readability.

  **[Spelling]** (2 c) -> (2 classes) in line 252; in line 319 "Since this is and open problem in representation learning" is not a sentence, need to be rephrased.

- ***Significance***: With the code provided in the supplementary, others researchers or practitioners can use the ideas or build on them. The constructed dataset should also be released (lines 217-219). Besides demonstrating the effectiveness of the proposed method, I have a question about the weakness of comparisons in the molecular generation and property optimization tasks. It seems that all the ways perform equally well/bad in these tasks, and many scores are precisely the same. Given the usages of different models and methods, it is a bit unclear to validate the improvements induced by disentangling the learned representations on these tasks.
questions: **[Experimental setup]**

- The derivation in section 3.2 use a single 2-layer GCN, does this architecture is used across all the experiments in section 5.3? I didn't find a clear description in the section. 5 nor the appendix. 
- The setting value of the communitative penalty is not mentioned.
- What are the comparisons on the similarity and success rates (in line 307)? Are all the methods follow the same setting of distance threshold  $\delta$ ? How to control that threshold in JT-VAE?

**[Interpretations MIA on Table.2]** Its turns out MIA is a strong baseline that achieves the best MSE performance across all the datasets. Any interpretations on this?

**[Scaling Up]** How would the proposed method perform on large graph task (macromolecules, community graphs)?
limitations: The authors adequately addressed the limitations (disentanglements can be measured only on synthetic datasets) of their work in section.6
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes two (conditional and unconditional) symmetry-based and Lie-algebras-based disentanglement on Graph Neural Networks within the framework of Variational Autoencoder. As is defined in symmetry-based disentanglement, a symmetry group, which can be decomposed into subgroups, can help disentangle a vector representation affected by it if specific demands are satisfied. Therefore, the authors introduce a Lie group as the symmetry group to graph disentanglement. Besides, they add commutative penalty and hessian penalty to enforce further disentanglement. In the experiment part, they evaluate systematic disentanglement on multiple metrics based on synthetic datasets and model performance based on real datasets.
strengths_and_weaknesses: Strengths:
1.	The disentanglement on graph is an important topic and the paper provides a new insight from symmetry-based one and introduces the Lie group to it.
2.	The consideration of commutative penalty, hessian penalty is comprehensive.

Weaknesses:
1.	As is written in Conclusion, this paper doesn’t present the disentangled factors on real-world datasets. But both [1] and [2], which are mentioned in Related Work, have disentanglement evaluations based on feature correlation analysis. Therefore, it is possible to evaluate disentanglement on real-world datasets, although it is still open which one is the most convincing.
[1] Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Disentangled graph convolutional networks. In International conference on machine learning, pages 4212–4221. PMLR, 2019.
[2] Yiding Yang, Zunlei Feng, Mingli Song, and Xinchao Wang. Factorizable graph convolutional networks. Advances in Neural Information Processing Systems, 33:20286–20296, 2020.

questions: 1.	There are two minor errors in Conclusion. In line 319, “and” should be “an”. In line 320, “real-word” should be “real-world”.
limitations: N/A
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors first propose a formalization of graph disentanglement in terms of graph embeddings building on existing definitions. They then propose a graph+group VAE architecture incorporating the graph disentanglement definition. This autoencoder first uses a graph encoder to map the original graph features to k latent features, which are then independently passed to k GCNs and aggregated by the group encoder to obtain the group coordinates. They are then decoded by the group decoder, combining the basis elements and passing them to the graph decoder. The authors then demonstrate significant improvements in several metrics, particularly disentanglement metrics. 
strengths_and_weaknesses: The paper in general is clear, well-written, well-motivated, and sufficiently rigorous. Although I am not especially familiar with previous work on graph disentanglement in particular, having reviewed the related works, the proposed method seems novel in the context of graphs. The translation of their proposed definition to their GVAE seems particularly clever and does not seem to obviously follow from the definition alone: it captures the key features of their definition while still being highly general and customizable. The experiments are fairly convincing as well, with some impressive disentanglement performance on the datasets analyzed (though more disentanglement datasets would have been helpful). 

My primary (albeit limited) concern with this paper’s definition is that it appears to more-or-less directly translate the concept of disentanglement in terms of underlying generative latent variables to graphs, embeddings and all, while seemingly largely disregarding graph-specific elements. However, graphs are potentially meaningfully different from the unstructured spaces for which disentanglement has previously been formalized. For example, graphs have often been used to express latent variable models, of which VAEs are a specific kind - though latent variable graphs are only a specific kind, it is not obvious that this framework would be the best way to characterize disentanglement in contexts where the graphs themselves represent generative processes. This is not to say that the proposed definition is not useful in this context: generative processes can themselves be the product of an underlying generative process. However, it is likely worth exploring and discussing the kinds of graphs where this definition may not cleanly apply. 

In addition, it would also be helpful to provide some additional justification for the metrics used. While they are standard disentanglement metrics, it does not appear that many prior works specifically on graphs aside from a 2019 workshop paper use them (including the ones cited). In particular, a workshop paper that may be worth mentioning is “Disentangling Interpretable Generative Parameters of Random and Real-World Graphs” (Stoehr et al 2019) which much less formally proposes evaluating graph disentanglement in terms of equivariance between the underlying generative and corresponding encoding maps. Finally, some visualizations of the kinds of variations learned by disentangled models would have been helpful. 

questions: How universal is the proposed definition - are there classes of graphs for which it may not be ideal?

What are some practical implications of this kind of disentanglement on graphs? In what contexts might this be useful?
limitations: Aside from what is discussed above, I think the authors reasonably discussed their limitations.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper brings group-based disentanglement learning to graph data domain. Two disentanglement cases, unconditional and conditional, are defined under this framework in this paper. The proposed architecture encodes group data into group latent representations with Lie algebra and exponential map, and leverages techniques like Hessian penalty to enforce disentanglement. Experiments on various data have shown the model’s effectiveness in disentanglement on graph data.
strengths_and_weaknesses: Pros:
1. The idea of group-based disentanglement is relatively new, and it is interesting to see it being brought to the graph data domain.
2. The proposed architecture is quite simple and the shown experimental results validate its effectiveness.

Cons:
1. In Fig. 1 right, why is the addition from Z is used in the conditional case? Does it mean Z will contain extra information that is not encoded in the latent T?
2. Is it possible to qualitatively show what information has been disentangled in the graph data?
3. The section 2.2 GNN architecture part seems a little redundant to me as Sec. 3.2 and 3.3 will introduce the architecture in detail.
questions: Please see weaknesses.
limitations: Yes
review_title: null

Meta-review: metareview: The paper presents an algorithm for learning disentangled representations for graph data, in the graph VAE framework. The reviewers appreciate the cleanly defined problem formulation with background knowledge (sec 2), which leads to a practical training objective and regularization (sec 3). Experimental results show that better disentanglement metrics are obtained with the proposed method compared to alternative ones, and better few short classification performance is achieved indicating important structures being learned to provide better generalization. The authors are encouraged to discuss the computational complexity aspects of the method, and provide visualization to help readers interpret group operations on graph and symmetry. 
confidence: Less certain
Decision: Accept