Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: Contributions:
(1) A neural architecture that preserves two symmetries (i) sign invariance and (ii) basis invariance, built based on the existing equivariant neural networks.
(2) Extensive experiments on various settings.
strengths_and_weaknesses: Strengths:
(1) Paper is easy to follow and well-written.
(2) The proposed methods are simple to implement using the existing codes of permutation equivariant neural networks.
(3) Many experiments.

Weaknesses: It is not clear "why" to preserve the two symmetries this paper proposes to preserve including sign invariance and basis invariance. In graph & molecule setting, two important symmetries are permutation (i.e. if we permute the nodes/atoms, the output stays the same) and rotation (i.e. if we rotate the 3D coordinates associating with each atom, the prediction for a physical property stays the same). I don't see a strong evidence that suggests preserving sign & basis invariances on top of permutation & rotation invariances brings any benefits. 
questions: I don't see the theoretical value of Theorem 2. 

A quick review of spectral graph convolution: The spectral graph convolution is built based on Graph Fourier Transform (GFT) as in [Bruna et al., 2014] and [Defferrard et al., 2016]. GFT constructs the Fourier basis as the eigenvectors of the (normalized) graph Laplacian. GFT allows us to do Fourier transform of a graph signal or function f: V -> R where V is the set of nodes (i.e. each node is associated with a number) into the spectral domain. Then, based on the convolution theorem, we can define the graph convolution between a graph signal and a filter based on their Fourier transforms in the spectral domain.

In this paper, given the construction of SignNet and BasisNet using the eigenvectors, I think it is just a tweaked form of spectral graph convolution. The authors use the existing permutation equivariant neural networks and add the minus sign to the eigenvectors, but I find this modification brings no novelty.
limitations: The reasons we want a neural network to preserve symmetries and build equivariant neural networks are because these symmetries arise in the data. In graph and molecule data, only two symmetries are important: permutation and rotation. We want permutation- and rotation-equivariant neural networks to learn and generate graph and molecule data, because both theoretical and empirical evidences have shown that preserving the two symmetries reduces the amount of training data, makes the model robust against data noise and smaller (i.e. less parameters), and importantly allows us to embed physical knowledge into the model. These are the fundamental motivations for preserving permutation and rotation symmetries that obviously arise from the data.

The fundamental limitation of this work is a lack of motivation and evidence for the two symmetries they propose to preserve (i) sign invariance and (ii) basis invariance. I highly doubt if these two symmetries are relevant and important. Given this limitation, I vote to reject this paper. However, I keep the rating to be 4 (borderline reject) because I am willing to listen to the authors' rebuttals.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work proposes a node position encoding scheme for graph representation learning, which respects two symmetric properties of an eigensystem, such as sign invariance, permutation invariance, and basis invariance. The new graph representation is believed spectral invariant and they are more capable for bio- or chemo-applications, where the graph substructures and global graph properties are important.
strengths_and_weaknesses: strengths:
- originality: The authors value the important spectral information provided by eigenvectors of the graph Laplacian, which are usually ignored by message passing methods. On top of that, they make transformations to the eigenvectors to encode position information of a graph while preserving certain properties. Such symmetric properties, to the best of the reviewer’s knowledge, have never been analyzed by existing work. 

- quality: Abundant analyses were provided in the main text and in appendices to consolidate the encoding scheme. In addition, experiments are designed from different perspectives to support the main claims of the work.

weaknesses:
- clarity: While the reviewer acknowledges that there are too many details to show in a limited number of pages, introducing the main algorithm in the appendix indeed prevents a reader from immediately understanding the workflow of the proposed method in a representation learning task. For the general audience, especially those without a solid mathematical background, to build a holistic view effortlessly, the reviewer highly recommends that the authors could consider rearranging some parts of the paper. In addition, some paradigms are used before a proper definition, which makes the paper more difficult to follow. For instance, the “higher dimensional basis” in the introduction was discussed without first explaining what is it.

- significance: The reviewer was mainly concerned with the empirical significance of the framework. Although it is claimed that symmetries are important for an eigensystem, and capturing graph properties (such as substructures) are important for bio- or chemo-applications, the experiments did not show a significant underperformance of the baseline methods over the proposed SignNet. A simple demonstration showing how ignoring such properties cause trouble in graph representation learning would be helpful. In addition, as the model does not outperform other methods significantly on the three benchmark datasets, it is questionable how much more computational cost was made to trade for a smaller MAE score?  

questions: 1. Figure 1, why there are 0-vector rows/columns in the P matrix? (the reviewer assumes black boxes are 1 and the rest gray area are filled with 0 values.)
2. Line 91, it seems (2) defines basis invariant, why here is discussing permutation invariance instead?
3. Line 122, does any arbitrary GNNs satisfy permutation equivariant? If not, this statement is inaccurate.
4. Line 162 (corollary 1), why sign invariant is not discussed here? Does basis invariant always imply sign invariance?
5. Line 176, why the spectral convolution can be written in this form? Usually a spectral graph convolution is defined as $f(X) =V^{\top}\theta VX$. The result keeps the dimension of $n\times q$. However here $f(V,\Lambda,X)\in\R^{q}$ is a vector (the understanding might be incorrect here, but the reviewer assumes $\theta$ is a vector. Even though, unless $\theta\in\R^{n\times n}$, the new dimension is inconsistent with the former form.
6. Line 201, does it imply that spectral invariant equals basis invariant and permutation invariant?
7. In the experiment, why BasisNet is not studied?
8. Line 241, is $k$ a hyper-parameter?
9. Figure 5 (it is not a question though), consider changing orange to another color, as both node feature and position encoding modules are using the orange color, but they are not directly connected. 

limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes SignNet and BasisNet, two neural architectures, which can approximate any continuous function of eigenvectors with  invariances. The authors demonstrate theoretically that the proposed mothods  can approximate spectral graph convolutions and compute powerful spectral invariants. Experiments on several tasks illustrate the empirical benefits of the models’ approximation power and invariances.
strengths_and_weaknesses: ### Strengths:
1) The symmetries and  invariances of eigenspaces are significant and promising problems.
2) The advantages of using SignNet and BasisNet is clear,  making sign flips and choices of eigenspace bases invariant.
3) The theoretical analysis and proofs of the proposed method looks sound to me.
4) The experiments reveal some intriguing findings. SignNet, for example, encodes bilateral symmetry and structural information on a cat model, which seems to demonstrate its great performance.

### Weaknesses:
1) The technical section of the paper is relatively dense, which may be difficult for non-experts in the invariant Networks to understand.
2) The papers writing may need to double check. For instance, the references (1) and (2) in lines 92, 93, and 98 are invalid.
questions: Should eigendecomposition be carried out if SignNet is to be utilized for GNNs, learning arbitrary Spectral Graph Convolutions? If so, does its high time cost limit its scalability?
limitations: Yes. The authors suggest that a potential limitation of the proposed method is that it only consider eigenspaces, not general subspaces.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a SignNet and Basisnet to obtain invariant graph neural network by using eigenspaces. The paper provide significant improvements with the proposed method with several applications.  
strengths_and_weaknesses: This paper makes an a useful and novel contribution to invaraint graph neural networks. Overall, the paper is well written and easy to understand. Experiments are extensive and covers several domains. All experiments show improved performances. However, the paper does not indicate any evidence where the proposed method could provide inferior performance.   
questions: What are the limitations of the proposed methods?


All experiments show improved performance compared to baseline methods. I am wondering if there are any data that may not give inferior performance. Are there any datasets or settings where the proposed method may not give good performance? 


Is it possible to directly compare permutation based invariant/equivariant graph neural networks with the proposed method?   

limitations: Limitation are not clearly stated.
review_title: null

Meta-review: metareview: This paper proposes new neural architectures that are invariant to sign and basis, named SignNet and BasisNet. Some reviewers raised the question of why such invariances are needed, and they noticed that only permutation and rotation symmetry arise in all the data mentioned in the paper. One of the reviewers strongly believes that the proposed approach is a simple tweak from the prior work "Invariant and Equivariant Graph Networks" (IGN) and the complexity of the proposed method is much higher than the spectral model using the eigenvectors. In their response, the authors provided some explanation and theoretical analysis on why SignNet and BasisNet are different from spectral graph convolutions. They argue that they can simulate spectral graph convolutions, and approximate other functions that spectral convolutions cannot express.
confidence: Less certain
Decision: Reject