SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 3. Under Assumption 1, our MAP-sign algorithm gives a sign-invariant, permutation-equivariant and universal canonization of  $\mathbf{u}$ .
One would wonder how restrictive the non-orthogonality condition (Assumption 1) is for sign canonization. In the following theorem, we establish a strong result showing that sign canonizability is equivalent to non-orthogonality. In other words, any sign canonizable eigenvectors can be canonized by MAP-sign. Due to this equivalence, MAP-sign can also serve as a complete algorithm to determine sign canonizability: a vector  $\mathbf{u}$  is sign canonizable iff  $\mathbf{x}_h$  is not <none> in equation 1.
Theorem 4. A vector  $\mathbf{u} \in \mathbb{R}^n$  is sign canonizable iff there exists a summary vector  $\mathbf{x}_h$  s.t.  $\mathbf{u}^\top \mathbf{x}_h \neq 0$ .
Remark. Theorem 3 is proved in Appendix K.5 and experimentally verified in Appendix C.2. We also show that MAP is not the only complete canonization algorithm for sign sinvariance by proposing another polynomial-based algorithm in Appendix K.9. We observe that most eigenvectors in real-world datasets are sign canonizable. For instance, on the ogbg-molhiv dataset, the ratio of non-canonical eigenvectors is only  $2.8\%$ . A thorough discussion on the feasibility of Assumption 1 is in Appendix G.1. For the left non-canonical eigenvectors, we could apply previous methods (like RandSign, SAN, and SignNet) to further eliminate their ambiguity, which could save more compute since there are only a few non-canonical eigenvectors. In practice, we also obtain good performance by simply ignoring these non-canonical features.
3.3.2 Basis Canonization with MAP
We further extend MAP-sign to solve the more challenging basis ambiguity with multiple eigenvectors, named MAP-basis. Now, MAP relies on two conditions to produce canonical eigenvectors. The first one requires there are enough distinctive summary vectors to determine  $d$  canonical eigenvectors.
Assumption 2. The number of distinctive angles  $k$  (i.e., the number of summary vectors  $\{\pmb{x}_i\}$ ) is larger or equal to the multiplicity  $d$ , i.e.,  $k \geq d$ .
Under this condition, we can determine each  $\pmb{u}_i$  iteratively with the corresponding summary vector  $\pmb{x}_i$ . At the  $i$ -th step where  $\pmb{u}_1, \dots, \pmb{u}_{i-1}$  have already been determined, we choose  $\pmb{u}_i$  to be the vector that is closest to the summary vector  $\pmb{x}_i$  in the orthogonal complement space of  $\langle \pmb{u}_1, \dots, \pmb{u}_{i-1} \rangle$  in  $V$ :
$$
\boldsymbol {u} _ {i} = \underset {\boldsymbol {u}} {\arg \max } \boldsymbol {u} ^ {\top} \boldsymbol {x} _ {i}, \tag {3}
$$
$$
s. t. \boldsymbol {u} \in \left\langle \boldsymbol {u} _ {1}, \dots , \boldsymbol {u} _ {i - 1} \right\rangle^ {\perp}, | \boldsymbol {u} | = 1.
$$
With a compact feasibility region, the maximum is attainable, and we can further show that the solution  $\pmb{u}_i$  is unique (c.f., the proof of Theorem 5). Equation 3 can be directly solved using the projection matrix of  $\langle \pmb{u}_1, \dots, \pmb{u}_{i-1} \rangle^\perp$  (see details in Algorithm 3 in Appendix C.1). Repeating this process gives us a canonical basis of  $V$ . We also require non-orthogonality condition at each step to obtain a valid eigenvector.
Assumption 3. For any  $1 \leq i \leq d$ ,  $\pmb{x}_i$  is not perpendicular to  $\langle \pmb{u}_1, \dots, \pmb{u}_{i-1} \rangle^\perp$ .
We summarize MAP-basis in Algorithm 2 in Appendix C.1. In Theorem 5, we prove under Assumption 2 and Assumption 3, MAP-basis gives a basis-invariant, permutation-equivariant and universal canonization of  $U$ . Though, in this scenario, MAP-basis cannot canonize all basis canonizable features, and whether such an algorithm exists remains an open problem for future research.
Theorem 5. Under Assumption 2 and Assumption 3, our MAP-basis algorithm gives a basis-invariant, permutation-equivariant and universal canonization of  $\mathbf{U}$ .
Remark. Assumption 2 and Assumption 3 exclude some symmetries of the eigenspace, which we discuss in details in Appendix G.2. For random orthonormal matrices, the possibility that either assumption is violated is equal to 0, which we verify with random simulation in Appendix C.3. For real-world datasets, these assumptions are not restrictive either. For instance, on the large ogbvmolpcba dataset, the eigenvalues violating Assumption 2 or Assumption 3 make up  $0.87\%$  of all eigenvalues. More statistics and discussions are provided in Appendix G.2.
3.3.3 Summary
We provide the complete pseudo-code of MAP to address both sign and basis invariance in Appendix C.1, along with a detailed time complexity analysis. Overall, the extra complexity introduced by MAP is  $\mathcal{O}(n^2\log n)$ , which is better than eigendecomposition itself with  $\mathcal{O}(n^3)$ . It only needs to be computed once per dataset and can be easily incorporated with various GNN architectures.
Combining Theorem 1, Theorem 3 and Theorem 5 gives us the universality of first-order GNNs [33] such that it respects all graph symmetries under certain assumptions (Assumptions 1, 2 & 3). We show that the probability of violating these assumptions asymptotically converges to zero on random graphs (see Appendix H). We also count the ratio of violation in real-world datasets in Table 2, and observe that the ratio of uncanonizable eigenvectors is less than  $10\%$  on all datasets. Thus, MAP greatly eases the harm caused by sign and basis ambiguities.
So far only GNNs using higher-order tensors have achieved universality while respecting all graph symmetries [33, 24], but they are typically computationally prohibitive in practice. It is still an open problem whether it is also possible for first-order GNNs. The proposed Laplacian Canonization presents a new approach in this direction trying to alleviate the harm of sign and basis ambiguities in a minimalist approach. By establishing universality-invariance results for first-order GNNs under certain assumptions, LC could hopefully could bring some insights to the GNN community.
We evaluate the proposed MAP positional encoding on sparse MP-GNNs and Transformer GNNs using PyTorch [40] and DGL [53]. For sparse MP-GNNs we consider GatedGCN [11] and PNA [15], and for Transformer GNNs we consider SAN [25] and GraphiT [34]. We conduct experiments on three standard molecular benchmarks—ZINC [22], OGBG-MOLTOX21 and OGBG-MOLPCBA [20]. ZINC and MOLTOX21 are of medium scale with 12K and 7.8K graphs respectively, whereas MOLPCBA is of large scale with 437.9K graphs. Details about these datasets are described in Appendix L. We follow the same protocol as in Lim et al. [29], where we replace their SignNet with a normal GNN and the Laplacian eigenvectors with our proposed MAP. We fairly compare several models on a fixed number of 500K model parameters on ZINC and relax the model sizes to larger parameters for evaluation on the two OGB datasets, as being practised on their leaderboards [20]. We also compare our results with GNNs using LapPE and random sign (RS) [17], GNNs using SignNet [29], and GNNs with no PE. For a fair comparison, all models use a limited number  $k$  of eigenvectors for positional encodings. The results of all our experiments are presented in Table 3, 4 & 5. Further implementation details are included in Appendix M.
4.1 Performance on Benchmark Datasets
As shown in Table 3, 4 & 5, using MAP improves the performance of all GNNs on all datasets, demonstrating that removing ambiguities of the eigenvectors is beneficial for graph-level tasks. First, by comparing models with LapPE and RS with models with no PE, we observe that the use of LapPE significantly improves the performance on ZINC, showing the benefits of incorporating expressive
PEs with GNNs, especially with MP-GNNs whose expressive power is limited by the 1-WL test. However, on MOLTOX21 and MOLPCBA, using LapPE has no significant effects. This is because unlike ZINC, OGB-MOL* datasets contain additional structural features that are informative, e.g., if an atom is in ring, among others [18]. Thus the performance gain by providing more positional information is less obvious. Second, MAP outperms LapPE with RS by a large margin especially on ZINC. Although RS also alleviates sign ambiguity by randomly flipping signs during training, MAP removes such ambiguity before training, enabling the network to focus on the real meaningful features and achieves a better performance. Third, we also observe that MAP and SignNet achieve comparable performance. This is because both methods aim at the same goal—eliminating ambiguity. However, SignNet does so in the training stage while MAP does so in the pre-processing stage, thus the latter is more computationally efficient. Lastly, we would also like to highlight that as a kind of positional encoding, MAP can be easily incorporated with any GNN architecture by passing the pre_transform function to the dataset class with a single line of code.
Table 3: Results on ZINC. All scores are averaged over 4 runs with 4 different seeds.
Model	PE	k	#Param	MAE ↓
GatedGCN	None	0	504K	0.251 ± 0.009
GatedGCN	LapPE + RS	8	505K	0.202 ± 0.006
GatedGCN	SignNet (φ(v) only)	8	495K	0.148 ± 0.007
GatedGCN	SignNet	8	495K	0.121 ± 0.005
GatedGCN	MAP	8	486K	0.120 ± 0.002
PNA	None	0	369K	0.141 ± 0.004
PNA	LapPE + RS	8	474K	0.132 ± 0.010
PNA	SignNet	8	476K	0.105 ± 0.007
PNA	MAP	8	462K	0.101 ± 0.005
SAN	None	0	501K	0.181 ± 0.004
SAN	MAP	16	230K	0.170 ± 0.012
GraphiT	None	0	501K	0.181 ± 0.006
GraphiT	MAP	16	329K	0.160 ± 0.006
Table 4: Results on MOLTOX21. All scores are averaged over 4 runs with 4 different seeds.
Model	PE	k	#Param	ROCAUC ↑
GatedGCN	None	0	1004K	0.772 ± 0.006
GatedGCN	LapPE + RS	3	1004K	0.774 ± 0.007
GatedGCN	MAP	3	1505K	0.784 ± 0.005
PNA	None	0	5245K	0.755 ± 0.008
PNA	MAP	16	1951K	0.761 ± 0.002
SAN	None	0	958K	0.744 ± 0.007
SAN	MAP	12	1152K	0.766 ± 0.007
GraphiT	None	0	958K	0.743 ± 0.003
GraphiT	MAP	16	590K	0.769 ± 0.011
Table 5: Results on MOLPCBA. All scores are averaged over 4 runs with 4 different seeds.
Model	PE	k	#Param	AP ↑
GatedGCN	None	0	1008K	0.262 ± 0.001
GatedGCN	LapPE + RS	3	1009K	0.266 ± 0.002
GatedGCN	MAP	3	2658K	0.268 ± 0.002
PNA	None	0	6551K	0.279 ± 0.003
PNA	MAP	16	4612K	0.281 ± 0.003
4.2 Empirical Understandings
Computation time. We demonstrate the efficiency of MAP by measuring the pre-processing time and training time on the large OGBG-MOLPCBA dataset, and compare them with SignNet. For a fair comparison, we use identical model size, hyperparameters and random seed and conduct experiments on the same NVIDIA 3090 GPU. The results are shown in Table 6. We observe that model with MAP train  $41\%$  faster than its SignNet counterpart, saving 44 hours of training time. Since SignNet takes the form  $\rho (\phi (\pmb {u}) + \phi (-\pmb {u}))$  while we use models taking the form  $\rho (\phi (\pmb {u}))$ , models with MAP would always train faster than those with SignNet under the same hyperparameters. We also observe that the pre-processing time is negligible compared with training time  $(< 3\%)$ , since pre-processing only needs to be done once. This makes MAP overall more efficient while achieving the same goal of tackling ambiguities.
Table 6: Comparison of pre-processing and training time between models with MAP or SignNet as PE, on the MOLPCBA dataset. Experiments are run with the same model size and hyperparameters, the same random seed, on the same NVIDIA 3090 GPU.
Model	Pre-processing time	Training Time	Total Time
GatedGCN + MAP	1.70 h	63.02 h	64.72 h
GatedGCN + SignNet	0.27 h	108.51 h	108.78 h
Spectral embedding dimension. Next, we study the effects of  $k$ . We train GatedGCN with MAP on ZINC with different number of eigenvectors used in the PE and report the results in Table 7. The hyperparameters are the same across these experiments. It can be observed that the performance drops when  $k$  is too small, meaning that SE provides crucial structural information to the model. Using larger  $k$  has limited influence on the performance, meaning that the model relies more on low-frequency information of spectral embedding.
Table 7: Effects of  $k$  ,where  $k$  is the number of eigenvectors used.
k	0	4	8	16
Test MAE	0.256 ± 0.012	0.138 ± 0.004	0.120 ± 0.002	0.124 ± 0.002
Ablation study. Finally, we conduct ablation study of MAP by removing each component of MAP and evaluate the performance. The results are shown in Table 8. Removing sign invariance hurts the performance the most, because most eigenvectors are single and thus sign ambiguity has the most influence on the model's performance. Removing basis invariance also has negative effects since a small portion of eigenvalues are multiple. Removing eigenvalues has moderate negative effects, showing that the incorporation of eigenvalue information is beneficial for the network.
Table 8: Effects of the three components of MAP (GatedGCN on ZINC).
PE	full MAP	without can. sign	without can. basis	without eigenvalues
Test MAE	0.120 ± 0.002	0.131 ± 0.003	0.122 ± 0.003	0.125 ± 0.001
In this paper, we explored a new approach called Laplacian Canonization for addressing sign and basis ambiguities of Laplacian eigenvectors while also preserving permutation invariance and universal expressive power. We developed a novel theoretical framework to characterize canonization and the canonizability of eigenvectors. Then we proposed a practical canonization algorithm called Maximal Axis Projection (MAP). Theoretically, it is sign/basis-invariant, permutation-equivariant and universal. Empirically, it canonizes most eigenvectors on synthetic and real-world data while showing promising performance on various datasets and GNN architectures.
[1] Ralph Abboud, Ismail Ilkan Ceylan, Martin Grohe, and Thomas Lukasiewicz. The Surprising Power of Graph Neural Networks with Random Node Initialization. In *IJCAI*, 2021.
[2] Rohith Agaram, Shaurya Dewan, Rahul Sajnani, Adrien Poulenard, Madhava Krishna, and Srinath Sridhar. Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields. arXiv preprint arXiv:2212.02493, 2022.
[3] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD, 2019.
[4] Greg W Anderson, Alice Guionnet, and Ofer Zeitouni. An Introduction to Random Matrices. Cambridge university press, 2010.
[5] Muhammet Balcilar, Renton Guillaume, Pierre Héroux, Benoit Gaüzère, Sébastien Adam, and Paul Honeine. Analyzing the Expressive Power of Graph Neural Networks in a Spectral Perspective. In ICLR, 2021.
[6] Dominique Beaini, Saro Passaro, Vincent Létourneau, Will Hamilton, Gabriele Corso, and Pietro Liò. Directional Graph Networks. In ICML, 2021.
[7] Mikhail Belkin and Partha Niyogi. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, 15(6):1373-1396, 2003.
[8] Florent Benaych-Georges and Raj Rao Nadakuditi. The singular values and vectors of low rank perturbations of large rectangular random matrices. Journal of Multivariate Analysis, 111: 120–135, 2012.
[9] Beatrice Bevilacqua, Fabrizio Frasca, Derek Lim, Balasubramaniam Srinivasan, Chen Cai, Gopinath Balamurugan, Michael M Bronstein, and Haggai Maron. Equivariant Subgraph Aggregation Networks. In ICLR, 2022.
[10] Avrim Blum and Ronald Rivest. Training a 3-Node Neural Network is NP-Complete. In NeurIPS, 1988.
[11] Xavier Bresson and Thomas Laurent. Residual Gated Graph ConvNets. arXiv preprint arXiv:1711.07553, 2017.
[12] Rasmus Bro, Evrim Acar, and Tamara G Kolda. Resolving the Sign Ambiguity in the Singular Value Decomposition. Journal of Chemometrics: A Journal of the Chemometrics Society, 22(2): 135-140, 2008.
[13] Qi Chen, Yifei Wang, Yisen Wang, Jiansheng Yang, and Zhouchen Lin. Optimization-Induced Graph Implicit Nonlinear Diffusion. In ICML, 2022.
[14] Zhengdao Chen, Soledad Villar, Lei Chen, and Joan Bruna. On the equivalence between graph isomorphism testing and function approximation with GNNs. In NeurIPS, 2019.
[15] Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Lio, and Petar Velicković. Principal Neighbourhood Aggregation for Graph Nets. In NeurIPS, 2020.
[16] Vijay Prakash Dwivedi and Xavier Bresson. A Generalization of Transformer Networks to Graphs. In AAAI Workshop on Deep Learning on Graphs: Methods and Applications, 2021.
[17] Vijay Prakash Dwivedi, Chaitanya K Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking Graph Neural Networks. arXiv preprint arXiv:2003.00982, 2020.
[18] Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Graph Neural Networks with Learnable Structural and Positional Representations. In ICLR, 2022.
[19] NT Hoang and Takanori Maehara. Frequency Analysis for Graph Convolution Network. In ICLR, 2020.
[20] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open Graph Benchmark: Datasets for Machine Learning on Graphs. arXiv preprint arXiv:2005.00687, 2020.
[21] Yinan Huang, William Lu, Joshua Robinson, Yu Yang, Muhan Zhang, Stefanie Jegelka, and Pan Li. On the Stability of Expressive Positional Encodings for Graph Neural Networks. arXiv preprint arXiv:2310.02579, 2023.
[22] John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman. ZINC: A Free Tool to Discover Chemistry for Biology. Journal of chemical information and modeling, 52(7):1757-1768, 2012.
[23] Sekou-Oumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. Equivariance with Learned Canonicalization Functions. In NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2022.
[24] Nicolas Keriven and Gabriel Peyre. Universal Invariant and Equivariant Graph Neural Networks. In NeurIPS, 2019.
[25] Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. In NeurIPS, 2021.
[26] Rongjie Lai and Hongkai Zhao. Multiscale Nonrigid Point Cloud Registration Using Rotation-Invariant Sliced-Wasserstein Distance via Laplace-Beltrami Eigenmap. SIAM Journal on Imaging Sciences, 10(2):449-483, 2017.
[27] Mingjie Li, Xiaojun Guo, Yifei Wang, Yisen Wang, and Zhouchen Lin. G2CN: Graph Gaussian Convolution Networks with Concentrated Graph Filters. In ICML, 2022.
[28] Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec. Distance Encoding—Design Provably More Powerful GNNs for Structural Representation Learning. In NeurIPS, 2020.
[29] Derek Lim, Joshua Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka. Sign and Basis Invariant Networks for Spectral Graph Representation Learning. In ICLR, 2023.
[30] Andreas Loukas. What Graph Neural Networks Cannot Learn: Depth vs Width. In ICLR, 2020.
[31] Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, and Yaron Lipman. Provably Powerful Graph Networks. In NeurIPS, 2019.
[32] Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman. Invariant and Equivariant Graph Networks. In ICLR, 2019.
[33] Haggai Maron, Ethan Fetaya, Nimrod Segol, and Yaron Lipman. On the Universality of Invariant Networks. In ICML, 2019.
[34] Grégoire Mialon, Dexiong Chen, Margot Selosse, and Julien Mairal. GraphiT: Encoding Graph Structure in Transformers. arXiv preprint arXiv:2106.05667, 2021.
[35] Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks. In AAAI, 2019.
[36] Ryan Murphy, Balasubramaniam Srinivasan, Vinayak Rao, and Bruno Ribeiro. Relational Pooling for Graph Representations. In ICML, 2019.
[56] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying Graph Convolutional Networks. In ICML, 2019.
[57] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful are Graph Neural Networks? In ICLR, 2019.
[58] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do Transformers Really Perform Badly for Graph Representation? In NeurIPS, 2019.
[59] Jiaxuan You, Rex Ying, and Jure Leskovec. Position-aware Graph Neural Networks. In ICML, 2019.
[60] Jiaxuan You, Jonathan M Gomes-Selman, Rex Ying, and Jure Leskovec. Identity-aware Graph Neural Networks. In AAAI, 2021.
[61] Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J Reddi, and Sanjiv Kumar. Are Transformers universal approximators of sequence-to-sequence functions? In ICLR, 2020.
[62] Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. Graph Transformer Networks. In NeurIPS, 2019.
[63] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep Sets. In NeurIPS, 2017.
A Why sign and basis invariance matters 15
B Related work 18
C Implementation and verification of MAP 19
C.1 Complete pseudo-code of MAP 19
C.2 Verifying the correctness of Theorem 3 20
C.3 Verifying the correctness of Theorem 5 23
C.4 Verifying the correctness of Theorem 1 23
D Synthetic experiment on basis invariance 24
E Other positional encoding methods 25
F Toy Examples 26
F.1 Toy examples for Algorithm 1 26
F.2 Toy examples for Algorithm 2 27
G Discussions on the assumptions 27
G.1 Discussions on Assumption 1 27
G.2 Discussions on Assumption 2 and Assumption 3 28
H Discussions on random graphs 29
I Further weakening the assumptions 32
I.1 Further weakening Assumption 1 32
I.2 Further weakening Assumption 2 and Assumption 3 32
J An alternative approach to dealing with sign ambiguity 33
K Proofs 34
K.1 Proof of Theorem 1 34
K.2 Proof of Theorem 2 35
K.3 Proof of Corollary 1 35
K.4 Proof of Corollary 2 35
K.5 Proof of Theorem 3 35
K.6 Proof of Theorem 4 36
K.7 Proof of Theorem 5 36
K.8 Proof of Theorem 7 37
K.9 Proof of Theorem 14 38
K.10 Proof of Theorem 15 39
L Dataset details 40
M Hyperparameter settings 40
M.1 Real-world tasks 40
M.2 Synthetic tasks 42
A Why sign and basis invariance matters
It is perfectly understandable that one may not see the significance and benefits of sign and basis invariance at once. For instance, below is a quote from one of the reviews of Lim et al. [29] when they first submitted their paper to NeurIPS 2022:
It is not clear "why" to preserve the two symmetries this paper proposes to preserve including sign invariance and basis invariance. In graph & molecule setting, two important symmetries are permutation (i.e. if we permute the nodes/atoms, the output stays the same) and rotation (i.e. if we rotate the 3D coordinates associating with each atom, the prediction for a physical property stays the same). I don't see a strong evidence that suggests preserving sign & basis invariances on top of permutation & rotation invariances brings any benefits.
Indeed, it is not easy to see why issues related with something matters if they don't even use it. One may overlook the importance of sign and basis invariance for two reasons: (1) people not familiar with Laplacian PE may not even realize sign/basis ambiguity is a thing; (2) people that does have some knowledge about Laplacian PE may not realize sign and basis ambiguities are markedly hindering its performance. Nonetheless, Laplacian PE does have advantages over other commonly used PEs that it is worth the efforts addressing their drawbacks. For these reasons we feel it is of great significance that we explain the motivation behind preserving sign and basis invariances in more details.