SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \phi_ {\theta , \omega} \left(\rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}\right) = \mathbb {E} _ {p _ {\omega} (h | \mathbf {x})} \left[ \rho_ {2} \left(g ^ {\prime} h\right) f _ {\theta} \left(\rho_ {1} \left(g ^ {\prime} h\right) ^ {- 1} \rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}\right) \right] \\ = \mathbb {E} _ {p _ {\omega} (h | \mathbf {x})} \left[ \rho_ {2} (g ^ {\prime}) \rho_ {2} (h) f _ {\theta} (\rho_ {1} (h) ^ {- 1} \rho_ {1} (g ^ {\prime}) ^ {- 1} \rho_ {1} (g ^ {\prime}) \mathbf {x}) \right] \\ = \rho_ {2} (g ^ {\prime}) \mathbb {E} _ {p _ {\omega} (h | \mathbf {x})} \left[ \rho_ {2} (h) f _ {\theta} (\rho_ {1} (h) ^ {- 1} \mathbf {x}) \right] \\ = \rho_ {2} \left(g ^ {\prime}\right) \phi_ {\theta , \omega} (\mathbf {x}), \tag {12} \\ \end{array}
$$
showing the  $G$  equivariance of  $\phi_{\theta ,\omega}$  for arbitrary  $f_{\theta}$
A.1.2 Proof of Theorem 2 (Section 2.1)
Theorem 2. If  $p_{\omega}$  is  $G$  equivariant and  $f_{\theta}$  is a universal approximator, then  $\phi_{\theta, \omega}$  is a universal approximator of  $G$  equivariant functions.
Proof. The proof is inspired by universality proofs of prior symmetrization approaches [102, 74, 41]. Let  $\psi : \mathcal{X} \to \mathcal{Y}$  be an arbitrary  $G$  equivariant function. By equivariance of  $\psi$ , we have:
$$
\begin{array}{l} \left\| \psi (\mathbf {x}) - \phi_ {\theta , \omega} (\mathbf {x}) \right\| = \left\| \psi (\mathbf {x}) - \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} \left[ \rho_ {2} (g) f _ {\theta} \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) \right] \right\| \\ = \left\| \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} [ \psi (\mathbf {x}) ] - \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} \left[ \rho_ {2} (g) f _ {\theta} \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) \right] \right\| \\ = \left\| \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} \left[ \rho_ {2} (g) \rho_ {2} (g) ^ {- 1} \psi (\mathbf {x}) \right] - \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} \left[ \rho_ {2} (g) f _ {\theta} \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) \right] \right\| \\ = \left\| \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} [ \rho_ {2} (g) \psi (\rho_ {1} (g) ^ {- 1} \mathbf {x}) ] - \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} [ \rho_ {2} (g) f _ {\theta} (\rho_ {1} (g) ^ {- 1} \mathbf {x}) ] \right\| \\ = \left\| \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} [ \rho_ {2} (g) \psi (\rho_ {1} (g) ^ {- 1} \mathbf {x}) - \rho_ {2} (g) f _ {\theta} (\rho_ {1} (g) ^ {- 1} \mathbf {x}) ] \right\|. \tag {13} \\ \end{array}
$$
As  $\mathcal{V}$  is finite-dimensional, we can assume that the linear operators in  $\mathrm{GL}(\mathcal{V})$  are bounded and so is the induced operator norm of group representation  $\| \rho_2(g)\|$  for all  $g\in G$ . Thus, we have:
$$
\begin{array}{l} \left\| \psi (\mathbf {x}) - \phi_ {\theta , \omega} (\mathbf {x}) \right\| \leq \max  _ {h \in G} \| \rho_ {2} (h) \| \left\| \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} \left[ \psi \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) - f _ {\theta} \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) \right] \right\| \\ \leq c \left\| \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} \left[ \psi \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) - f _ {\theta} \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) \right] \right\|. \tag {14} \\ \end{array}
$$
for some  $c > 0$ . If  $f_{\theta}$  is a universal approximator, for any compact set  $\mathcal{K} \subseteq \mathcal{X}$  and any  $\epsilon > 0$ , there exists some  $\theta$  such that  $\| \psi(\mathbf{x}) - f_{\theta}(\mathbf{x}) \| \leq \epsilon$  for all  $\mathbf{x} \in \mathcal{K}$ . Consider the set  $\mathcal{K}_{\mathrm{sym}} = \cup_{g \in G} \rho_1(g) \mathcal{K}$  where  $\rho_1(g) \mathcal{K}$  denotes the image of the set  $\mathcal{K}$  under linear transformation by  $\rho_1(g)$ . We use the fact that  $\mathcal{K}_{\mathrm{sym}}$  is also a compact set since it is the image of the compact set  $G \times \mathcal{K}$  under continuous map  $(g, \mathbf{x}) \mapsto \rho_1(g) \mathbf{x}$ . As a consequence, for any compact set  $\mathcal{K} \subseteq \mathcal{X}$  and any  $\epsilon / c > 0$ , there exists some  $\theta$  such that  $\max_{g \in G} \| \psi(\rho_1(g) \mathbf{x}) - f_{\theta}(\rho_1(g) \mathbf{x}) \| \leq \epsilon / c$  for all  $\mathbf{x} \in \mathcal{K}$ . Since a group is closed under inverse, for any compact set  $\mathcal{K} \subseteq \mathcal{X}$  and any  $\epsilon > 0$ , there exists some  $\theta$  such that:
$$
\begin{array}{l} \left\| \psi (\mathbf {x}) - \phi_ {\theta , \omega} (\mathbf {x}) \right\| \leq c \left\| \mathbb {E} _ {p _ {\omega} (g | \mathbf {x})} \left[ \psi \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) - f _ {\theta} \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) \right] \right\| \\ \leq c \max  _ {g \in G} \left\| \psi \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) - f _ {\theta} \left(\rho_ {1} (g) ^ {- 1} \mathbf {x}\right) \right\| \\ = \epsilon , \tag {15} \\ \end{array}
$$
for all  $\mathbf{x} \in \mathcal{K}$ , showing that  $\phi_{\theta, \omega}$  is a universal approximator of  $G$  equivariant functions.
While we have assumed that the group  $G$  is compact in the proof, we conjecture that the results can be extended to non-compact groups if we make an alternative assumption that the distribution  $p_{\omega}(g|\mathbf{x})$  is compactly supported for all  $\mathbf{x} \in \mathcal{K}$ . We leave proving this as a future work.
A.1.3 Proof of Theorem 3 (Section 2.1)
Proof. We prove  $p_{\omega}(g'g|\rho_1(g')\mathbf{x}) = p_{\omega}(g|\mathbf{x})$  for all  $\mathbf{x} \in \mathcal{X}$  and  $g, g' \in G$ . In general, we are interested in obtaining a faithful representation  $\rho$ , i.e., such that  $\rho(g)$  is distinct for each  $g$ . We can interpret the probability  $p_{\omega}(g|\mathbf{x}, \epsilon)$  as a delta distribution centered at the group representation  $\rho(g)$ :
$$
p _ {\omega} (g | \mathbf {x}, \epsilon) = \delta (\rho (g) = q _ {\omega} (\mathbf {x}, \epsilon)). \tag {16}
$$
To obtain  $p_{\omega}(g|\mathbf{x})$  , we marginalize over  $p(\epsilon)$  ..
$$
\begin{array}{l} p _ {\omega} (g | \mathbf {x}) = \int_ {\epsilon} p _ {\omega} (g | \mathbf {x}, \epsilon) p (\epsilon) d \epsilon \\ = \int_ {\epsilon} \delta (\rho (g) = q _ {\omega} (\mathbf {x}, \epsilon)) p (\epsilon) d \epsilon . \tag {17} \\ \end{array}
$$
Let us consider  $p_{\omega}(g'g|\rho_1(g')\mathbf{x})$
$$
p _ {\omega} \left(g ^ {\prime} g \mid \rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}\right) = \int_ {\epsilon} \delta \left(\rho \left(g ^ {\prime} g\right) = q _ {\omega} \left(\rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}, \epsilon\right)\right) p (\epsilon) d \epsilon . \tag {18}
$$
Using the  $G$  equivariance of  $q_{\omega}$ , we have:
$$
\begin{array}{l} q _ {\omega} \left(\rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}, \boldsymbol {\epsilon}\right) = \rho \left(g ^ {\prime}\right) q _ {\omega} \left(\rho_ {1} \left(g ^ {\prime - 1}\right) \rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}, \rho^ {\prime} \left(g ^ {\prime - 1}\right) \boldsymbol {\epsilon}\right) \\ = \rho \left(g ^ {\prime}\right) q _ {\omega} \left(\mathbf {x}, \rho^ {\prime} \left(g ^ {\prime - 1}\right) \epsilon\right) \tag {19} \\ \end{array}
$$
which leads to the following:
$$
\begin{array}{l} p _ {\omega} \left(g ^ {\prime} g \mid \rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}\right) = \int_ {\epsilon} \delta \left(\rho \left(g ^ {\prime} g\right) = \rho \left(g ^ {\prime}\right) q _ {\omega} \left(\mathbf {x}, \rho^ {\prime} \left(g ^ {\prime - 1}\right) \boldsymbol {\epsilon}\right)\right) p (\boldsymbol {\epsilon}) d \boldsymbol {\epsilon} \\ = \int_ {\epsilon} \delta (\rho (g) = q _ {\omega} (\mathbf {x}, \rho^ {\prime} \left(g ^ {\prime - 1}\right) \epsilon)) p (\epsilon) d \epsilon . \tag {20} \\ \end{array}
$$
Note that the second equality follows from invertibility of  $\rho(g')$ . We now introduce a change of variables  $\epsilon' = \rho'(g'^{-1})\epsilon$  that  $\epsilon = \rho'(g')\epsilon'$ :
$$
p _ {\omega} \left(g ^ {\prime} g \mid \rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}\right) = \int_ {\boldsymbol {\epsilon} ^ {\prime}} \delta \left(\rho (g) = q _ {\omega} (\mathbf {x}, \boldsymbol {\epsilon} ^ {\prime})\right) p \left(\rho^ {\prime} \left(g ^ {\prime}\right) \boldsymbol {\epsilon} ^ {\prime}\right) \frac {1}{| \det  \rho^ {\prime} \left(g ^ {\prime - 1}\right) |} d \boldsymbol {\epsilon} ^ {\prime}. \tag {21}
$$
With  $|\operatorname{det}\rho'(g'^{-1})| = 1$ , and  $G$  invariance of  $p(\epsilon)$  which gives  $p(\rho'(g')\epsilon') = p(\epsilon')$ , we get:
$$
\begin{array}{l} p _ {\omega} \left(g ^ {\prime} g \mid \rho_ {1} \left(g ^ {\prime}\right) \mathbf {x}\right) = \int_ {\epsilon^ {\prime}} \delta \left(\rho (g) = q _ {\omega} (\mathbf {x}, \epsilon^ {\prime})\right) p \left(\epsilon^ {\prime}\right) d \epsilon^ {\prime} \\ = p _ {\omega} (g | \mathbf {x}), \tag {22} \\ \end{array}
$$
showing the  $G$  equivariance of  $p_{\omega}(g|\mathbf{x})$ .
A.1.4 Proof of Validity for Implemented Equivariant Distributions  $p_{\omega}$  (Section 2.2)
We formally show  $G$  equivariance of the implemented distributions  $p_{\omega}(g|\mathbf{x})$  presented in Section 2.2. All implementations have a form of noise-outsourced function  $q_{\omega}:(\mathbf{x},\epsilon)\mapsto \rho (g)$  using distribution  $\epsilon \sim p(\epsilon)$  and map  $q_{\omega}$  which is composed of  $G$  equivariant neural network and postprocessing to  $\rho (g)$ . From Theorem 3, for  $G$  equivariance of  $p_{\omega}(g|\mathbf{x})$ , it is sufficient to show  $G$  invariance of  $p(\epsilon)$  under a representation  $\rho^{\prime}$  such that  $|\operatorname*{det}\rho '(g)| = 1$  along with  $G$  equivariance of  $q_{\omega}$ , which we show below.
Symmetric Group  $\mathbf{S}_n$  We recall that  $p_{\omega}(g|\mathbf{x})$  for the symmetric group  $\mathbf{S}_n$  is implemented as below:
1. Sample node-level noise  $\pmb {\epsilon}\in \mathbb{R}^{n\times d}$  from i.i.d. uniform Unif[0,  $\eta$
2. Use a GNN to obtain node-level scalar features  $(\mathbf{x},\pmb {\epsilon})\mapsto \mathbf{Z}\in \mathbb{R}^n$
3. Assuming  $\mathbf{Z}$  is tie-free, use argsort [98] to obtain group representation  $\mathbf{Z} \mapsto \mathbf{P}_g = \rho(g)$ .
$$
\mathbf {P} _ {g} = \operatorname {e q} \left(\mathbf {Z} \mathbf {1} ^ {\top}, \mathbf {1} \operatorname {s o r t} (\mathbf {Z}) ^ {\top}\right), \tag {23}
$$
where eq denotes elementwise equality indicator.
We now show the following:
Proposition 3. The proposed distribution  $p_{\omega}(g|\mathbf{x})$  for the symmetric group  $S_{n}$  is equivariant.
Proof. Given  $p(\epsilon)$  is elementwise i.i.d., it is  $S_{n}$  invariant under the base representation  $\rho'(g) = \mathbf{P}_{g}$  which satisfies  $|\det \mathbf{P}_{g}| = 1$  from orthogonality. As a GNN is  $S_{n}$  equivariant, we only need to show  $S_{n}$  equivariance of  $\text{argsort} : \mathbf{Z} \mapsto \mathbf{P}_{g}$ . This can be shown by transforming  $\mathbf{Z}$  with any permutation matrix  $\mathbf{P}_{g'}$ . Since sort operator and any row replicated matrices are invariant to  $\mathbf{P}_{g'}$ , we have:
$$
\begin{array}{l} \operatorname {e q} \left(\mathbf {P} _ {g ^ {\prime}} \mathbf {Z 1} ^ {\top}, \mathbf {1 s o r t} \left(\mathbf {P} _ {g ^ {\prime}} \mathbf {Z}\right) ^ {\top}\right) = \operatorname {e q} \left(\mathbf {P} _ {g ^ {\prime}} \mathbf {Z 1} ^ {\top}, \mathbf {1 s o r t} (\mathbf {Z}) ^ {\top}\right) \\ = \operatorname {e q} \left(\mathbf {P} _ {g ^ {\prime}} \mathbf {Z} \mathbf {1} ^ {\top}, \mathbf {P} _ {g ^ {\prime}} \mathbf {1} \operatorname {s o r t} (\mathbf {Z}) ^ {\top}\right). \tag {24} \\ \end{array}
$$
Since eq commutes with  $\mathbf{P}_{g'}$ , we have:
$$
\begin{array}{l} \operatorname {e q} \left(\mathbf {P} _ {g ^ {\prime}} \mathbf {Z 1} ^ {\top}, \mathbf {1 s o r t} \left(\mathbf {P} _ {g ^ {\prime}} \mathbf {Z}\right) ^ {\top}\right) = \operatorname {e q} \left(\mathbf {P} _ {g ^ {\prime}} \mathbf {Z 1} ^ {\top}, \mathbf {P} _ {g ^ {\prime}} \mathbf {1 s o r t} (\mathbf {Z}) ^ {\top}\right) \\ = \mathbf {P} _ {g ^ {\prime}} \mathbf {e q} (\mathbf {Z} \mathbf {1} ^ {\top}, \mathbf {1} \operatorname {s o r t} (\mathbf {Z}) ^ {\top}) \\ = \mathbf {P} _ {g ^ {\prime}} \mathbf {P} _ {g}, \tag {25} \\ \end{array}
$$
showing that  $\operatorname{argsort}$  is  $\mathbf{S}_n$  equivariant, i.e., it maps  $\mathbf{P}_{g'}\mathbf{Z} \mapsto \mathbf{P}_{g'}\mathbf{P}_g$  for all  $\mathbf{P}_{g'} \in \mathbf{S}_n$ . Combining the above, by Theorem 3, the distribution  $p_{\omega}(g|\mathbf{x})$  is  $\mathbf{S}_n$  equivariant.
Orthogonal Group  $\mathrm{O}(n)$ ,  $\mathrm{SO}(n)$  We recall that  $p_{\omega}(g|\mathbf{x})$  for the orthogonal group  $\mathrm{O}(n)$  or special orthogonal group  $\mathrm{SO}(n)$  is implemented as follows:
1. Sample noise  $\epsilon \in \mathbb{R}^{n\times d}$  from i.i.d. normal  $\mathcal{N}(0,\eta^2)$ .
2. Use an  $\mathrm{O}(n) / \mathrm{SO}(n)$  equivariant neural network to obtain  $n$  features  $(\mathbf{x},\epsilon)\mapsto \mathbf{Z}\in \mathbb{R}^{n\times n}$ .
3. Assuming  $\mathbf{Z}$  is full-rank, use Gram-Schmidt process [41] to obtain an orthogonal matrix  $\mathbf{Z} \mapsto \mathbf{Q}$ .
4. For the  $\mathrm{O}(n)$  group, use the obtained matrix as group representation  $\mathbf{Q} = \mathbf{Q}_g = \rho(g)$ .
5. For the  $\mathrm{SO}(n)$  group, use below scale operator to obtain group representation  $\mathbf{Q} \mapsto \mathbf{Q}_g^+ = \rho(g)$ .
$$
\operatorname {s c a l e}: \left[ \begin{array}{c c c} \mathbf {Q} _ {1} & \dots & \mathbf {Q} _ {n} \end{array} \right] \mapsto \left[ \begin{array}{c c c} \det  (\mathbf {Q}) \cdot \mathbf {Q} _ {1} & \dots & \mathbf {Q} _ {n} \end{array} \right]. \tag {26}
$$
Proposition 4. The proposed distribution  $p_{\omega}(g|\mathbf{x})$  for the orthogonal group  $\mathrm{O}(n)$  is equivariant.
Proof. Without loss of generality, let us omit the scale  $\eta$  for brevity, which gives that each column  $\epsilon_{i}\in \mathbb{R}^{n}$  of the noise  $\pmb{\epsilon}$  independently follows multivariate standard normal  $\epsilon_{i}\sim \mathcal{N}(0,\mathbf{I}_{n})$ . Then, the density  $p(\epsilon_i) = (2\pi)^{-n / 2}\exp (-\| \epsilon_i\| _2^2 /2)$  is invariant under orthogonal transformation  $\mathbf{Q}$  since  $\| \mathbf{Q}\pmb {\epsilon}_i\| _2^2 = (\mathbf{Q}\pmb {\epsilon}_i)^\top \mathbf{Q}\pmb {\epsilon}_i = \pmb {\epsilon}_i^\top \mathbf{Q}^\top \mathbf{Q}\pmb {\epsilon}_i = \pmb {\epsilon}_i^\top \pmb {\epsilon}_i = \| \pmb {\epsilon}_i\| _2^2$ . Therefore, the distribution  $p(\epsilon)$  is invariant under the base representation  $\rho^{\prime}(g) = \mathbf{Q}_{g}$  which satisfies  $|\operatorname *{det}\rho '(g)| = 1$  from orthogonality. As we use an equivariant neural network to obtain  $\mathbf{Z}$ , and Gram-Schmidt procedure  $\mathbf{Z}\mapsto \mathbf{Q}_g$  is  $\mathrm{O}(n)$  equivariant (Theorem 5 of [41]), by Theorem 3, the distribution  $p_{\omega}(g|\mathbf{x})$  is  $\mathrm{O}(n)$  equivariant.
Proposition 5. The proposed distribution  $p_{\omega}(g|\mathbf{x})$  for special orthogonal group  $\mathrm{SO}(n)$  is equivariant.
Proof. From the proof of Proposition 4, it follows that the distribution  $p(\epsilon)$  is invariant under the base representation  $\rho'(g) = \mathbf{Q}_g^+$  which satisfies  $|\operatorname{det} \rho'(g)| = 1$  due to orthogonality. As we use an equivariant neural network to obtain  $\mathbf{Z}$ , and Gram-Schmidt procedure  $\mathbf{Z} \mapsto \mathbf{Q}$  has  $\mathrm{O}(n)$  equivariance which implies  $\mathrm{SO}(n)$  equivariance because of  $\mathrm{SO}(n) \leq \mathrm{O}(n)$ , we only need to show
$\mathrm{SO}(n)$  equivariance of scale:  $\mathbf{Q} \mapsto \mathbf{Q}_g^+$ . This can be done by transforming  $\mathbf{Q}$  with an orthogonal  $\mathbf{Q}_{g'}^+$  of determinant  $+1$ . Since  $\operatorname*{det}(\mathbf{Q}_{g'}^+\mathbf{Q}) = \operatorname*{det}(\mathbf{Q}_{g'}^+) \operatorname*{det}(\mathbf{Q}) = \operatorname*{det}(\mathbf{Q})$ , we have the following:
$$
\begin{array}{l} \operatorname {s c a l e} \left(\mathbf {Q} _ {g ^ {\prime}} ^ {+} \mathbf {Q}\right) = \left[ \det  \left(\mathbf {Q} _ {g ^ {\prime}} ^ {+} \mathbf {Q}\right) \cdot \left(\mathbf {Q} _ {g ^ {\prime}} ^ {+} \mathbf {Q}\right) _ {1} \quad \dots \quad \left(\mathbf {Q} _ {g ^ {\prime}} ^ {+} \mathbf {Q}\right) _ {n} \right] \\ = \left[ \det (\mathbf {Q}) \cdot \left(\mathbf {Q} _ {g ^ {\prime}} ^ {+} \mathbf {Q}\right) _ {1} \left| \dots \right| \left(\mathbf {Q} _ {g ^ {\prime}} ^ {+} \mathbf {Q}\right) _ {n} \right]. \tag {27} \\ \end{array}
$$
Also, scaling the first column of the product  $\mathbf{Q}_{g'}^+ \mathbf{Q}$  with  $\operatorname*{det}(\mathbf{Q})$  is equivalent to scaling the first column of  $\mathbf{Q}$  with  $\operatorname*{det}(\mathbf{Q})$  then computing the product since  $(\mathbf{Q}_{g'}^+ \mathbf{Q})_{ij} = \sum_k \mathbf{Q}_{g'ik}^+ \mathbf{Q}_{kj}$ . This gives:
$$
\begin{array}{l} \operatorname {s c a l e} \left(\mathbf {Q} _ {g ^ {\prime}} ^ {+} \mathbf {Q}\right) = \mathbf {Q} _ {g ^ {\prime}} ^ {+} \left[ \det  (\mathbf {Q}) \cdot \mathbf {Q} _ {1} \left| \dots \right| \mathbf {Q} _ {n} \right] \\ = \mathbf {Q} _ {g ^ {\prime}} ^ {+} \operatorname {s c a l e} (\mathbf {Q}), \tag {28} \\ \end{array}
$$
showing that scale operator is  $\mathrm{SO}(n)$  equivariant. We also note that  $\mathrm{scale}(\mathbf{Q})$  gives orthogonal matrix of determinant  $+1$ , as it returns  $\mathbf{Q}$  if  $\operatorname*{det}(\mathbf{Q}) = +1$ , otherwise  $(\operatorname*{det}(\mathbf{Q}) = -1$  since  $\mathbf{Q}$  is orthogonal) scales the first column by  $-1$  which flips determinant to  $+1$  while not affecting orthogonality. Combining the above, by Theorem 3, the distribution  $p_{\omega}(g|\mathbf{x})$  is  $\mathrm{SO}(n)$  equivariant.
Euclidean Group  $\mathrm{E}(n)$ ,  $\mathrm{SE}(n)$  We recall that, unlike the other groups, we handle the Euclidean group  $\mathrm{E}(n)$  and special Euclidean group  $\mathrm{SE}(n)$  at symmetrization level as the translation component  $\mathrm{T}(n)$  in  $\mathrm{E}(n) = \mathrm{O}(n) \ltimes \mathrm{T}(n)$  and  $\mathrm{SE}(n) = \mathrm{SO}(n) \ltimes \mathrm{T}(n)$  is non-compact. This is done as follows:
$$
\phi_ {\theta , \omega} (\mathbf {x}) = \mathbb {E} _ {p _ {\omega} (g | \mathbf {x} - \bar {\mathbf {x}} \mathbf {1} ^ {\top})} \left[ \bar {\mathbf {x}} \mathbf {1} ^ {\top} + g \cdot f _ {\theta} \left(g ^ {- 1} \cdot (\mathbf {x} - \bar {\mathbf {x}} \mathbf {1} ^ {\top})\right) \right], \tag {29}
$$
where  $\bar{\mathbf{x}}\in \mathbb{R}^n$  is centroid (mean over channels) of data  $\mathbf{x}\in \mathbb{R}^{n\times d}$  and distribution  $p_{\omega}$  is  $\mathrm{O}(n) / \mathrm{SO}(n)$  equivariant for  $\mathrm{E}(n) / \mathrm{SE}(n)$  equivariant symmetrization, respectively. We now show the following:
Proposition 6. The proposed symmetrization  $\phi_{\theta ,\omega}$  for the Euclidean group  $\operatorname {E}(n)$  is equivariant.
Proof. We prove  $\phi_{\theta, \omega}(g' \cdot \mathbf{x}) = g' \cdot \phi_{\theta, \omega}(\mathbf{x})$  for all  $\mathbf{x} \in \mathcal{X}$  and  $g' \in \operatorname{E}(n)$ . From Eq. (29), we have:
$$
\phi_ {\theta , \omega} \left(g ^ {\prime} \cdot \mathbf {x}\right) = \mathbb {E} _ {p _ {\omega} \left(g | g ^ {\prime} \cdot \mathbf {x} - \overline {{g ^ {\prime} \cdot \mathbf {x}}} \mathbf {1} ^ {\top}\right)} \left[ \overline {{g ^ {\prime} \cdot \mathbf {x}}} \mathbf {1} ^ {\top} + g \cdot f _ {\theta} \left(g ^ {- 1} \cdot \left(g ^ {\prime} \cdot \mathbf {x} - \overline {{g ^ {\prime} \cdot \mathbf {x}}} \mathbf {1} ^ {\top}\right)\right) \right]. \tag {30}
$$
In general, an element of Euclidean group  $g' \in \operatorname{E}(n)$  acts on data  $\mathbf{x} \in \mathbb{R}^{n \times d}$  via  $g' \cdot \mathbf{x} = \mathbf{Q}_{g'} \mathbf{x} + \mathbf{t}_{g'} \mathbf{1}^\top$  where  $\mathbf{Q}_{g'} \in \mathrm{O}(n)$  is its rotation component and  $\mathbf{t}_{g'} \in \mathbb{R}^n$  is its translation component [74, 41]. With this, the centroid of the transformed data  $g' \cdot \mathbf{x}$  is given as follows:
$$
\overline {{g ^ {\prime} \cdot \mathbf {x}}} = \overline {{\mathbf {Q} _ {g ^ {\prime}} \mathbf {x} + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top}}} = \overline {{\mathbf {Q} _ {g ^ {\prime}} \mathbf {x}}} + \mathbf {t} _ {g ^ {\prime}} = \mathbf {Q} _ {g ^ {\prime}} \bar {\mathbf {x}} + \mathbf {t} _ {g ^ {\prime}}, \tag {31}
$$
$$
\begin{array}{l} g ^ {\prime} \cdot \mathbf {x} - \overline {{g ^ {\prime} \cdot \mathbf {x}}} \mathbf {1} ^ {\top} = \mathbf {Q} _ {g ^ {\prime}} \mathbf {x} + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top} - \mathbf {Q} _ {g ^ {\prime}} \overline {{\mathbf {x}}} \mathbf {1} ^ {\top} - \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top} \\ = \mathbf {Q} _ {g ^ {\prime}} (\mathbf {x} - \bar {\mathbf {x}} \mathbf {1} ^ {\top}). \tag {32} \\ \end{array}
$$
Above shows that subtracting centroid eliminates the translation component of the problem and leaves  $\mathrm{O}(n)$  equivariance component. Based on that, we have the following:
$$
\begin{array}{l} \phi_ {\theta , \omega} \left(g ^ {\prime} \cdot \mathbf {x}\right) = \mathbb {E} _ {p _ {\omega} \left(g | \mathbf {Q} _ {g ^ {\prime}} (\mathbf {x} - \overline {{\mathbf {x}}} \mathbf {1} ^ {\top})\right)} \left[ \mathbf {Q} _ {g ^ {\prime}} \bar {\mathbf {x}} \mathbf {1} ^ {\top} + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top} + g \cdot f _ {\theta} \left(g ^ {- 1} \cdot \left(\mathbf {Q} _ {g ^ {\prime}} (\mathbf {x} - \overline {{\mathbf {x}}} \mathbf {1} ^ {\top})\right)\right) \right] \\ = \mathbb {E} _ {p _ {\omega} (g | g ^ {\prime} \cdot (\mathbf {x} - \bar {\mathbf {x}} \mathbf {1} ^ {\top}))} \left[ g ^ {\prime} \cdot \bar {\mathbf {x}} \mathbf {1} ^ {\top} + g \cdot f _ {\theta} \left(g ^ {- 1} g ^ {\prime} \cdot (\mathbf {x} - \bar {\mathbf {x}} \mathbf {1} ^ {\top})\right) \right] + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top}. \tag {33} \\ \end{array}
$$
Note that, inside the expectation, we interpret the rotation component of  $g'$  as an element of the orthogonal group  $\mathrm{O}(n)$ . Similar as in the proof of Theorem 1, we introduce transformed random variable  $h = g'^{-1}g \in \mathrm{O}(n)$  that  $g = g'h$ . Since the distribution  $p_{\omega}$  is  $\mathrm{O}(n)$  equivariant, we can see
that  $p_{\omega}(g|g^{\prime}\cdot (\mathbf{x} - \overline{\mathbf{x}}\mathbf{1}^{\top})) = p_{\omega}(g^{\prime -1}g|g^{\prime -1}g^{\prime}\cdot (\mathbf{x} - \overline{\mathbf{x}}\mathbf{1}^{\top})) = p_{\omega}(g^{\prime -1}g|\mathbf{x} - \overline{\mathbf{x}}\mathbf{1}^{\top}) = p_{\omega}(h|\mathbf{x} - \overline{\mathbf{x}}\mathbf{1}^{\top})$  Thus we can rewrite the above expectation with respect to  $h$  as follows:
$$
\begin{array}{l} \phi_ {\theta , \omega} (g ^ {\prime} \cdot \mathbf {x}) = \mathbb {E} _ {p _ {\omega} (h | \mathbf {x} - \bar {\mathbf {x}} \mathbf {1} ^ {\top})} [ g ^ {\prime} \cdot \bar {\mathbf {x}} \mathbf {1} ^ {\top} + g ^ {\prime} h \cdot f _ {\theta} ((g ^ {\prime} h) ^ {- 1} g ^ {\prime} \cdot (\mathbf {x} - \bar {\mathbf {x}} \mathbf {1} ^ {\top})) ] + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top} \\ = \mathbb {E} _ {p _ {\omega} (h | \mathbf {x} - \overline {{\mathbf {x}}} \mathbf {1} ^ {\top})} \left[ g ^ {\prime} \cdot \overline {{\mathbf {x}}} \mathbf {1} ^ {\top} + g ^ {\prime} h \cdot f _ {\theta} \left(h ^ {- 1} \cdot (\mathbf {x} - \overline {{\mathbf {x}}} \mathbf {1} ^ {\top})\right) \right] + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top} \\ = \mathbf {Q} _ {g ^ {\prime}} \mathbb {E} _ {p _ {\omega} (h | \mathbf {x} - \overline {{\mathbf {x}}} \mathbf {1} ^ {\top})} \left[ \bar {\mathbf {x}} \mathbf {1} ^ {\top} + h \cdot f _ {\theta} \left(h ^ {- 1} \cdot (\mathbf {x} - \overline {{\mathbf {x}}} \mathbf {1} ^ {\top})\right) \right] + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top} \\ = \mathbf {Q} _ {g ^ {\prime}} \phi_ {\theta , \omega} (\mathbf {x}) + \mathbf {t} _ {g ^ {\prime}} \mathbf {1} ^ {\top} \\ = g ^ {\prime} \cdot \phi_ {\theta , \omega} (\mathbf {x}), \tag {34} \\ \end{array}
$$