SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We consider the following symmetry groups: the permutation group  $S_{16}$ , the graph automorphism  $\mathcal{A}_G$ , and the trivial symmetry. Note that the human skeleton graph has  $\mathcal{A}_G = (S_2)^2$  (corresponding to the arm flip and leg flip). Additionally, we investigate an approximate symmetry called Relax- $S_{16}$ ; This relaxes the global weight sharing in  $S_{16}$  (that only learns 2 scalars per equivariant linear layer  $f_{S_{16}}: \mathbb{R}^N \to \mathbb{R}^N$ , where  $f_{S_{16}}[i,i] = a$ ,  $f_{S_{16}}[i,j] = b$  for  $i \neq j$ ) to local weight sharing (that learns  $2 \times 16 = 32$  scalars, where  $f_{\mathrm{Relax - }S_{16}}[i,i] = a_i$ ,  $f_{\mathrm{Relax - }S_{16}}[i,j] = b_i$  for  $i \neq j$ ). Table 4 shows that the best performance is achieved at the hypothesis class induced from Relax- $S_{16}$ , which is smaller than  $S_{16}$  but differs from  $\mathcal{A}_G$ . Furthermore, using  $\mathcal{G}\text{-Net}(\mathrm{gc + ew})$  with Relax- $S_{16}$  gives a substantial improvement over  $S_{16}$  (representing SemGCN in [78]). This demonstrates the utility of enforcing active symmetries in GNNs that results in more expressive models. More details and additional ablation studies can be found in Appendix D.4.
In this paper, we focus on learning tasks where the graph is fixed, and the dataset consists of different signals on the graph. We developed an approach for designing GNNs based on active symmetries and approximate symmetries induced by the symmetries of the graph and its coarse-grained versions. A layer of an approximately equivariant graph network uses a linear map that is equivariant to the chosen symmetry group; the graph is not used as an input but rather induces a hypothesis class. In practice, we further break the symmetry by incorporating the graph in the model computation, thus combining symmetric and asymmetric components.
We theoretically show a bias-variance tradeoff between the loss of expressivity due to imposing symmetries, and the gain in regularity, for settings where the target map is assumed to be (approximately) equivariant. For simplicity, the theoretical analysis focuses on the equivariant models without symmetry breaking; Theoretically analyzing the combination of symmetric and asymmetric components in machine learning models is an interesting open problem. The bias-variance formula is computed only for a simple linear regression model with white noise and in the underparameterized setting; Extending it to more realistic models and overparameterized settings is a promising direction.
As a proof of concept, our approximately equivariant graph networks only consider symmetries of the fixed graph. An interesting future direction is to extend our approach to also account for (approximate) symmetries in node features and labels, using suitably generalized cut distance (e.g., graphon-signal cut distance in [79]; see Appendix C.2 for an overview). Our network architecture consists only of linear layers and pointwise nonlinearity. Another promising direction is to incorporate pooling layers (e.g., [22, 80]) and investigate them through the lens of approximate symmetries. Finally, extending our analysis to general groups is a natural next step. Our techniques are based on compact groups with orthogonal representation. While we believe that the orthogonality requirement can be lifted straightforwardly, relaxing the compactness requirement appears to be more challenging (see related discussion in [46]).
This research project was started at the BIRS 2022 Workshop "Deep Exploration of non-Euclidean Data with Geometric and Topological Representation Learning" held at the UBC Okanagan campus in Kelowna, B.C. The authors thank Ben Blum-Smith (JHU), David Hogg (NYU), Erik Thiede (Cornell), Wenda Zhou (Open AI), Carey Priebe (JHU), and Rene Vidal (UPenn) for valuable discussions, and Hong Ye Tan (Cambridge) for pointing out typos in an earlier version of the manuscript. The authors also thank the anonymous NeurIPS reviewers and area chair for helpful feedback. Ningyuan Huang is partially supported by the MINDS Data Science Fellowship from Johns Hopkins University. Ron Levie is partially funded by ISF (Israel Science Foundation) grant #1937/23. Soledad Villar is partially funded by the NSF-Simons Research Collaboration on the Mathematical and Scientific Foundations of Deep Learning (MoDL) (NSF DMS 2031985), NSF CISE 2212457, ONR N00014-22-1-2126 and an Amazon AI2AI Faculty Research Award.
[1] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst. Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.
[2] Fernando Gama, Joan Bruna, and Alejandro Ribeiro. Stability properties of graph neural networks. IEEE Transactions on Signal Processing, 68:5680-5695, 2020.
[3] Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Velicković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges, 2021.
[4] Carlo Rovelli and Marcus Gaul. Loop quantum gravity and the meaning of diffeomorphism invariance. In Towards Quantum Gravity: Proceeding of the XXXV International Winter School on Theoretical Physics Held in Polanica, Poland, 2-11 February 1999, pages 277-324. Springer, 2000.
[5] Soledad Villar, David W Hogg, Weichi Yao, George A Kevrekidis, and Bernhard Scholkopf. The passive symmetries of machine learning. arXiv preprint arXiv:2301.13724, 2023.
[6] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.
[7] Alberto Bietti, Luca Venturi, and Joan Bruna. On the sample complexity of learning under invariance and geometric stability, 2021.
[8] Bryn Elesedy and Sheheryar Zaidi. Provably strict generalisation benefit for equivariant models. In International Conference on Machine Learning, pages 2959-2969. PMLR, 2021.
[9] Song Mei, Theodor Misiakiewicz, and Andrea Montanari. Learning with invariances in random features and kernel models. arXiv preprint arXiv:2102.13219, 2021.
[10] Clare Lyle, Mark van der Wilk, Marta Kwiatkowska, Yarin Gal, and Benjamin Bloem-Reddy. On the benefits of invariance in neural networks. arXiv preprint arXiv:2005.00178, 2020.
[11] Bryn Elesedy. Group symmetry in pac learning. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning, 2022.
[12] Bryn Elesedy. Provably strict generalisation benefit for invariance in kernel methods. Advances in Neural Information Processing Systems, 34:17273-17283, 2021.
[13] Arash Behboodi, Gabriele Cesa, and Taco Cohen. A pac-bayesian generalization bound for equivariant networks. arXiv preprint arXiv:2210.13150, 2022.
[14] Gregory Benton, Marc Finzi, Pavel Izmailov, and Andrew G Wilson. Learning invariances in neural networks from training data. Advances in neural information processing systems, 33: 17605-17616, 2020.
[15] Jameson Cahill, Dustin G Mixon, and Hans Parshall. Lie pca: Density estimation for symmetric manifolds. Applied and Computational Harmonic Analysis, 65:279-295, 2023.
[16] Vasco Portilheiro. A tradeoff between universality of equivariant models and learnability of symmetries. arXiv preprint arXiv:2210.09444, 2022.
[17] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-80, 2009.
[78] Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, and Dimitris N Metaxas. Semantic graph convolutional networks for 3d human pose regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3425-3435, 2019.
[79] Ron Levie. A graphon-signal analysis of graph neural networks, 2023.
[80] Alejandro Parada-Mayorga, Zhiyang Wang, and Alejandro Ribeiro. Graphon pooling for reducing dimensionality of signals and convolutional operators on graphs, 2022.
[81] Daniel A. Spielman. Testing isomorphism of graphs with distinct eigenvalues, 2018.
[82] Bruce Sagan. The symmetric group: representations, combinatorial algorithms, and symmetric functions, volume 203. Springer Science & Business Media, 2001.
[83] Jean-Pierre Serre et al. Linear representations of finite groups, volume 42. Springer, 1977.
[84] Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan J Tibshirani. Surprises in high-dimensional ridgeless least squares interpolation. The Annals of Statistics, 50(2):949-986, 2022.
[85] Ningyuan Huang, David W. Hogg, and Soledad Villar. Dimensionality reduction, regularization, and generalization in overparameterized regressions. SIAM Journal on Mathematics of Data Science, 4(1):126-152, feb 2022. doi: 10.1137/20m1387821. URL https://doi.org/10.1137%2F20m1387821.
[86] Somesh Das Gupta. Some aspects of discrimination function coefficients. Sankhya: The Indian Journal of Statistics, Series A, pages 387-400, 1968.
[87] Hosagrahar V Jagadish, Johannes Gehrke, Alexandros Labrinidis, Yannis Papakonstantinou, Jignesh M Patel, Raghu Ramakrishnan, and Cyrus Shahabi. Big data and its technical challenges. Communications of the ACM, 57(7):86-94, 2014.
[88] Dario Pavllo, Christoph Feichtenhofer, David Grangier, and Michael Auli. 3d human pose estimation in video with temporal convolutions and semi-supervised training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7753-7762, 2019.
A Parameterization of equivariant linear Maps
We consider a group  $\mathcal{G}$  acting on spaces  $\mathcal{X}$  and  $\mathcal{Y}$  via representations  $\phi$  and  $\psi$ , respectively. Our goal is to find the equivariant linear maps  $f: \mathcal{X} \to \mathcal{Y}$  such that  $f(\phi(g)x) = \psi(g)f(x)$  for all  $g \in \mathcal{G}$  and  $x \in \mathcal{X}$ . The standard way to do this, used extensively in the equivariant machine learning literature (e.g. [40, 43]), is to decompose  $\phi$  and  $\psi$  in irreducibles and use Schur's lemma.
In a nutshell, a group representation  $\varphi$  is an homomorphism  $\mathcal{G} \to \mathrm{GL}(V)$ , where  $\mathrm{GL}(V)$  denotes the General Linear group of the vector space  $V$  (sometimes mathematicians say that  $V$  is a representation of  $\mathcal{G}$ , but we need to know the homomorphism  $\varphi$  too). One way to interpret the group homomorphism (i.e.  $\varphi(gh) = \varphi(g) \circ \varphi(h)$ ) is that the group multiplication corresponds to the composition of linear invertible maps (i.e. matrix multiplication). A linear subspace  $W$  of  $V$  is said to be a subrepresentation of  $\varphi$  if  $\varphi(\mathcal{G})(W) \subset W$ . An irreducible representation is one that only has itself and the trivial subspace as subrepresentations.
Schur's lemma states that if  $V$ ,  $W$  are vector spaces over  $\mathbb{C}$  and  $\varphi_V, \varphi_W$  are irreducible representations, then either (1)  $\varphi_V$  and  $\varphi_W$  are not isomorphic as representations (and the only equivariant linear map between  $V$ ,  $W$  is the zero map), or (2)  $\varphi_V$  and  $\varphi_W$  are isomorphic and the only non-trivial equivariant maps are of the form  $\lambda I$  where  $\lambda \in \mathbb{C}$  and  $I$  is the identity (See Chapter 1 of [68]).
Given  $\mathcal{G}$  acting on spaces  $\mathcal{X}$  and  $\mathcal{Y}$  via representations  $\phi$  and  $\psi$ , respectively, one can decompose  $\phi$  and  $\psi$  in irreducibles over  $\mathbb{C}$
$$
\phi = \oplus_ {k = 1} ^ {\ell} a _ {k} \mathcal {T} _ {k} \quad \psi = \oplus_ {k = 1} ^ {\ell} b _ {k} \mathcal {T} _ {k}
$$
(this notation assumes the same irreducibles appear in both decompositions, which can be done if we allow some of the  $a_{k}$  and  $b_{k}$  to be zero). Then one can parameterize the equivariant linear maps by having one complex parameter per irreducible that appears in both decompositions.
These ideas can be applied to real spaces too. By Maschke's theorem we can decompose the representation in irreducibles over  $\mathbb{R}$ . Then we can check further how to decompose these irreducibles over  $\mathbb{C}$ , and apply Schur's lemma. We have 3 cases for the decomposition:
1. The irreducible over  $\mathbb{R}$  is also irreducible over  $\mathbb{C}$ . In this case we directly apply Schur's lemma.
2. The irreducible over  $\mathbb{R}$  decomposes in two different irreducibles over  $\mathbb{C}$ . In this case we can send each  $\mathbb{C}$ -irreducible to their isomorphic counterpart.
3. The irreducibles over  $\mathbb{R}$  decompose in two copies of the same irreducible over  $\mathbb{C}$ . In this case we can send each irreducible to any isomorphic copy independently.
Therefore, finding the equivariant linear maps reduces to decomposing the corresponding representations in irreducibles. In the next sections we explain in detail how to do this for the specific problems described in this paper. The appendix is organized as follows: We first show how to parameterize equivariant linear layers for abelian group using isotypical decomposition (Section A.1), and then discuss the case for the symmetry group induced by graph coarsening, which further considers parameterizing permutation-equivariant linear maps to model the within-cluster symmetries (Section A.2).
A.1 Equivariant Linear Maps via Isotypical Decomposition
In this section, we assume that the graph adjacency matrix  $A$  has distinct eigenvalues  $\lambda_1 > \lambda_2 > \ldots > \lambda_n$ . Then  $\mathcal{A}_G$  is an abelian group [81, Lemma 3.8.1]. Under this assumption, we present the construction of  $\mathcal{A}_G$ -equivariant linear maps using isotypical decomposition (i.e. decomposition into isomorphism classes of irreducible representations) and group characters. We remark that such construction extends to non-abelian groups and refer the interested reader to [82], but we omit it here for the ease of exposition.
We consider the simplest setting where  $f: \mathbb{R}^N \to \mathbb{R}^N$  is a linear function that maps graph signals. Let  $x \in \mathbb{R}^N$  be the node features, then  $\mathcal{A}_G$ -equivariance requires
$$
f (g x) = g f (x) \quad \text {f o r a l l} g \in \mathcal {A} _ {G}. \tag {9}
$$
To construct equivariant linear functions  $f$ , our roadmap is outlined as follows:
1. Decompose the vector space  $\mathcal{X} = \mathbb{R}^N$  into a sum of components such that different components cannot be mapped to each other equivariantly (also known as the isotypic decomposition);
2. Given  $\mathcal{X} = \oplus_{i}\mathcal{X}_{i}$  an isotopic representation, we then parameterize  $f$  by linear maps at each  $\mathcal{X}_i$  such that for all  $i$ ,  $f(\mathcal{X}_i)\subseteq \mathcal{X}_i$ .
To this end, we need the following definitions.
Definition 5. ( $\mathcal{G}$ -module, [82, Defn 1.3.1]) Let  $\mathcal{X}$  be a vector space and  $\mathcal{G}$  be a group. We say the vector space  $\mathcal{X}$  is a  $\mathcal{G}$ -module or  $\mathcal{X}$  carries a representation of  $\mathcal{G}$  if there is a group homomorphism  $\rho: \mathcal{G} \to GL(\mathcal{X})$ , where  $GL$  denotes the General Linear group. Equivalently, if the following holds:
1.  $gv\in \mathcal{X}$
2.  $g(cv + dw) = c(gv) + d(gw)$
3.  $(gh)v = g(hv)$
4.  $e v = v$
for all  $g, h \in \mathcal{G}$ ;  $v, w \in \mathcal{X}$  and scalars  $c, d \in \mathbb{C}$  ( $e \in \mathcal{G}$  denotes the identity element).
In what follows, we consider  $\mathcal{X} = \mathbb{R}^N$  carries a representation of  $G$ .
Definition 6. (Group characters) Given a group  $\mathcal{G}$  and a representation  $\phi : G \to GL(V)$ , the group character is a function  $\chi : \mathcal{G} \to \mathbb{R}$  (or  $\mathbb{C}$ ) defined as  $\chi(g) \coloneqq \operatorname{tr} \phi(g)$ .
Definition 7. (Group orbits) Let  $\mathcal{X}$  be a vector space and  $\mathcal{G}$  be a group. The group orbit of an element  $x\in \mathcal{X}$  is  $O(x)\coloneqq \{gx:g\in \mathcal{G}\}$ .
Let  $g_1, \ldots, g_s$  be the generators of  $\mathcal{A}_G \subset (\mathcal{S}_2)^n$ , or simply  $\mathcal{A}_G \equiv (\mathcal{S}_2)^k$  for some  $k \leq n$ . Since  $\mathcal{A}_G$  is abelian, any irreducible representation is 1-dimensional [68, p.8]. In other words, the irreducible representations of an abelian group are homomorphisms
$$
\rho : \mathcal {A} _ {G} \rightarrow \mathbb {C}. \tag {10}
$$
Since all the elements of the group  $\mathcal{A}_G = (S_2)^k$  is of order 1 or 2, the homomorphisms are  $\rho : \mathcal{A}_G \to \{\pm 1\} \subset \mathbb{R}$ . By Definition 6, the irreducible characters (i.e., characters of irreducible matrix representation) are also homomorphisms  $\rho : \mathcal{A}_G \to \{\pm 1\}$ . In other words,  $\chi(g) \in \{\pm 1\}$  for all  $g \in \mathcal{A}_G$ . Then we can write down the  $2^k \times 2^k$  character table, where the rows are the characters  $\chi$ , and the columns are the group elements  $g \in \mathcal{A}_G$  (see Table 1 as an example). Now, define the projection onto the isotypic component of the representation  $X$  as
$$
P _ {\chi} := \frac {\deg (X)}{| \mathcal {A} _ {G} |} \sum_ {g \in \mathcal {A} _ {G}} \overline {{\chi (g)}} g = \frac {1}{| \mathcal {A} _ {G} |} \sum_ {g \in \mathcal {A} _ {G}} \chi (g) g, \tag {11}
$$
where the second equality uses the fact that  $\mathcal{A}_G$  is abelian.
Intuitively, applying  $P_{\chi}$  on  $\mathcal{X} = \operatorname{span}(\{e_1, \ldots, e_N\})$  picks out all  $v \in \mathcal{X}$  that stays in the same subspace defined by the group character  $\chi$ . (Note that for the  $(\mathcal{S}_2)^k$  case  $\chi^{-1}(g) = \chi(g)$  since  $\chi(g) \in \{\pm 1\}$ ).
Algorithm 1 presents the construction of equivariant linear map  $f$  with respect to an abelian group. Such construction can parameterize all abelian  $\mathcal{A}_G$ -equivariant linear maps, as shown in Lemma 5.
Lemma 5.  $f$  is linear, equivariant with respect to the abelian group  $\mathcal{A}_G$  if and only if  $f$  can be written as (13) in Algorithm 1.
Proof. By construction in Algorithm 1,  $f$  is linear and equivariant. To show the converse, we first recall some useful facts and notations. Since  $\mathcal{A}_G$  is abelian with all irreducible representations being one-dimensional, for isotypic components  $\mathcal{X}_{\chi_1} \neq \mathcal{X}_{\chi_2}$ , we have
$$
g v _ {1} = \lambda_ {1} (g) v _ {1}, \quad \text {f o r a l l} g \in \mathcal {G}, v _ {1} \in \mathcal {X} _ {\chi_ {1}}, \tag {14}
$$
$$
g v _ {2} = \lambda_ {2} (g) v _ {2}, \quad \text {f o r a l l} g \in \mathcal {G}, v _ {2} \in \mathcal {X} _ {\chi_ {2}}, \tag {15}
$$
where there exists some  $g \in \mathcal{G}$  such that  $\lambda_1(g) \neq \lambda_2(g)$ . To show  $f$  being linear and equivariant implies for all  $v \in \mathcal{X}_{\chi}$ ,  $f(v) \in \mathcal{X}_{\chi}$ , we prove by contradiction. Without loss of generality, suppose
$$
f \left(v _ {\chi_ {1}}\right) = \alpha_ {1} v _ {\chi_ {1}} + \alpha_ {2} v _ {\chi_ {2}}, \tag {16}
$$
Algorithm 1 Parameterizing equivariant linear functions  $f:\mathbb{R}^N\to \mathbb{R}^N$  for abelian group
Require: Abelian group  $\mathcal{A}_G = (\mathcal{S}_2)^k$
1. Construct the character table of  $\chi_{\mathrm{irreps}}$  for  $\mathcal{A}_G$ , i.e.  $\chi_i: \mathcal{A}_G \to \{\pm 1\} i = 1, \ldots, \ell$ ;
2. For each character  $\chi_{i}$  in the character table, compute the projection matrix
$$
P _ {\chi_ {i}} (\mathcal {X}) = \left[ P _ {\chi_ {i}} \left(e _ {1}\right); \dots ; P _ {\chi_ {i}} \left(e _ {N}\right) \right] \in \mathbb {R} ^ {N \times N}; \tag {12}
$$
followed by computing the basis from  $P_{\chi_i}(\mathcal{X})$  and call it  $\mathcal{X}_{\chi_i} = [b_{\chi_i}^{(1)},\ldots ,b_{\chi_i}^{(K_i)}]$
3.  $\mathcal{X} = \bigoplus_{i=1}^{\ell} \mathcal{X}_{\chi_i}$  where  $\mathcal{X}_{\chi_i}$  are the isotypic component. Then  $f$  is any linear function satisfying that  $f(\mathcal{X}_{\chi_i}) \subseteq \mathcal{X}_{\chi_i}$  for all  $i = 1, \ldots, \ell$ . In particular, in the basis  $[b_{\chi_i}^{(s)}]_{1 \leq i \leq \ell, 1 \leq s \leq K_i}$ $f$  can be written as a block diagonal matrix  $\mathbb{R}^{n \times n}$  with each block  $M_{\chi_i}$  being the linear map from  $\mathcal{X}_{\chi_i} \to \mathcal{X}_{\chi_i}$ ,
$$
f = \left[ \begin{array}{c c c c} M _ {\chi_ {1}} & & & \\ & M _ {\chi_ {2}} & & \\ & & \ddots & \\ & & & M _ {\chi_ {\ell}} \end{array} \right]. \tag {13}
$$
return  $f$
e	σ
χe	1	1
χ2	1	-1
Table 1: Character table for  $\operatorname{aut}\left( {P}_{4}\right)  \cong  {Z}_{2}$
for some scalars  $\alpha_{1},\alpha_{2}$  and  $v_{\chi_1}\in \mathcal{X}_{\chi_1},v_{\chi_2}\in \mathcal{X}_{\chi_2}$ . Then by (14), for all  $g\in \mathcal{G}$
$$
f (g v _ {\chi_ {1}}) = f \left(\lambda_ {1} (g) v _ {\chi_ {1}}\right) = \lambda_ {1} (g) f \left(v _ {\chi_ {1}}\right) = \lambda_ {1} (g) \alpha_ {1} v _ {\chi_ {1}} + \lambda_ {1} (g) \alpha_ {2} v _ {\chi_ {2}}. \tag {17}
$$
Now, since  $f$  is equivariant, for all  $g \in \mathcal{G}$ ,
$$
f \left(g v _ {\chi_ {1}}\right) = g f \left(v _ {\chi_ {1}}\right) = g \left(\alpha_ {1} v _ {\chi_ {1}} + \alpha_ {2} v _ {\chi_ {2}}\right) = \lambda_ {1} (g) \alpha_ {1} v _ {\chi_ {1}} + \lambda_ {2} (g) \alpha_ {2} v _ {\chi_ {2}}. \tag {18}
$$
But there exists some  $g' \in \mathcal{G}$  such that  $\lambda_1(g') \neq \lambda_2(g')$ , which leads to  $f(g'v_{\chi_1}) \neq f(g'v_{\chi_1})$ , a contradiction. One can easily extend the proof strategy to the general case for  $f(v_{\chi_1}) = \sum_{i=1}^{l} v_{\chi_i}$ .
Example A.1. Consider the path graph on 4 nodes (i.e.,  $P_4$ ). We have  $\operatorname{aut}(P_4) = \{e, (14)(23)\} \cong Z_2$ .
Steps 1: Note that  $Z_{2}$  is abelian and thus all irreducible characters  $\chi(g) \in \{\pm 1\}$ , for all  $g \in Z_{2}$ . The character table is shown in Table 1.
Step 2: using (11) we have
$$
P _ {\chi_ {e}} \left[ e _ {1}; e _ {2}; e _ {3}; e _ {4} \right] = \frac {1}{2} \left[ \begin{array}{c c c c} 1 & 0 & 0 & 1 \\ 0 & 1 & 1 & 0 \\ 0 & 1 & 1 & 0 \\ 1 & 0 & 0 & 1 \end{array} \right] \text {w h i c h y i e l d s b a s i s} \mathcal {B} \left(P _ {\chi_ {e}}\right) = \left[ e _ {1} + e _ {4}; e _ {2} + e _ {3} \right].
$$
$$
P _ {\chi_ {2}} \left[ e _ {1}; e _ {2}; e _ {3}; e _ {4} \right] = \frac {1}{2} \left[ \begin{array}{c c c c} 1 & 0 & 0 & - 1 \\ 0 & 1 & - 1 & 0 \\ 0 & - 1 & 1 & 0 \\ - 1 & 0 & 0 & 1 \end{array} \right] \text {w h i c h y i e l d s b a s i s} \mathcal {B} \left(P _ {\chi_ {2}}\right) = \left[ e _ {1} - e _ {4}; e _ {2} - e _ {3} \right].
$$
Step 3: Parameterize  $f: \mathbb{R}^4 \to \mathbb{R}^4$  by  $f: \mathcal{B}(P_{\chi_e}) \to \mathcal{B}(P_{\chi_e})$  and  $f: \mathcal{B}(P_{\chi_2}) \to \mathcal{B}(P_{\chi_2})$ . For all  $v \in \mathbb{R}^4$ , write  $v = c_1(e_1 + e_4) + c_2(e_2 + e_3) + c_3(e_1 - e_4) + c_4(e_2 - e_3)$ , then
$$
f (v) = \left[ \begin{array}{l l} \alpha_ {1} & \alpha_ {2} \\ \alpha_ {3} & \alpha_ {4} \end{array} \right] \left[ \begin{array}{l} c _ {1} \\ c _ {2} \end{array} \right] + \left[ \begin{array}{l l} \alpha_ {5} & \alpha_ {6} \\ \alpha_ {7} & \alpha_ {8} \end{array} \right] \left[ \begin{array}{l} c _ {3} \\ c _ {4} \end{array} \right], \tag {19}
$$
where  $\alpha_{1},\ldots ,\alpha_{8}$  are (learnable) real scalars. Now  $f$  is linear, equivariant by construction.
A.2 Equivariant Linear Map for Symmetries Induced by Graph Coarsening
In this section, we provide additional details of constructing equivariant linear maps for using the symmetry group induced by graph coarsening (Definition 3). Recall the symmetry group with  $M$  clusters of  $G$  (with the associated coarsened graph  $G^{\prime}$ ) is given by
$$
\mathcal {G} _ {G \rightarrow G ^ {\prime}} := \left(\mathcal {S} _ {1} \times \mathcal {S} _ {2} \dots \times \mathcal {S} _ {M}\right) \rtimes \overline {{\mathcal {A}}} _ {G ^ {\prime}} \subset \mathcal {S} _ {N}.
$$
We first assume that  $\overline{\mathcal{A}}_{G'}$  is trivial and show how to parameterize equivariant functions with respect to products of permutations. Then we discuss more general cases, for instance if  $\overline{\mathcal{A}}_{G'}$  is abelian. For the ease of exposition, we consider  $X \in \mathbb{R}^N$ ,  $Y \in \mathbb{R}^N$ .
Suppose  $\overline{\mathcal{A}}_{G'}$  is trivial, we claim that a linear function  $f: \mathbb{R}^N \to \mathbb{R}^N$  is equivariant with respect to  $\mathcal{G}_{G \to G'}$  if and only if it admits the following block-matrix form: