SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Brandon Anderson, Truong-Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[2] A. K. Aniyan and Kshitij Thorat. Classifying radio galaxies with the convolutional neural network. The Astrophysical Journal Supplement Series, 2017.
[3] Thierry Aubin. Some nonlinear problems in Riemannian geometry. Springer Science & Business Media, 1998.
[4] Francis Bach. Learning Theory from First Principles (draft). 2021.
[5] Alberto Bietti, Luca Venturi, and Joan Bruna. On the sample complexity of learning under geometric stability. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[6] Benjamin Bloem-Reddy and Yee Whye Teh. Probabilistic symmetries and invariant neural networks. Journal of Machine Learning Research, 2020.
[7] Glen E Bredon. Introduction to compact transformation groups. Academic press, 1972.
[8] Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Velickovic. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021.
[9] Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2013.
[10] Sylvain Calinon. Gaussians on Riemannian manifolds: Applications for robot learning and adaptive control. IEEE Robotics & Automation Magazine, 2020.
[11] Yaiza Canzani. Analysis on manifolds via the Laplacian. Lecture notes, 2013.
[12] Isaac Chavel. Eigenvalues in Riemannian geometry. Academic press, 1984.
[13] Shuxiao Chen, Edgar Dobriban, and Jane H. Lee. A group-theoretic framework for data augmentation. Journal of Machine Learning Research, 2020.
[14] Carlo Ciliberto, Francis Bach, and Alessandro Rudi. Localized structured prediction. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[15] H. Dominguez Sánchez, M. Huertas-Company, M. Bernardi, D. Tuccillo, and J. L. Fischer. Improving galaxy morphologies for SDSS with deep learning. Monthly Notices of the Royal Astronomical Society, 2018.
[16] Harold Donnelly. Bounds for eigenfunctions of the laplacian on compact riemannian manifolds. Journal of Functional Analysis, 2001.
[17] Simon S. Du, Yining Wang, Xiyu Zhai, Sivaraman Balakrishnan, Russ R. Salakhutdinov, and Aarti Singh. How many samples are needed to estimate a convolutional neural network? In Advances in Neural Information Processing Systems (NeurIPS), 2018.
[18] Nadav Dym and Haggai Maron. On the universality of rotation equivariant point cloud networks. In Int. Conference on Learning Representations (ICLR), 2021.
[19] Bryn Elesedy. Provably strict generalisation benefit for invariance in kernel methods. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[20] Bryn Elesedy and Sheheryar Zaidi. Provably strict generalisation benefit for equivariant models. In Int. Conference on Machine Learning (ICML), 2021.
[21] Michael James Fenton, Alexander Shmakov, Ta-Wei Ho, Shih-Chieh Hsu, Daniel Whiteson, and Pierre Baldi. Permutationless many-jet event reconstruction with symmetry preserving attention networks. Physical Review D, 2022.
[62] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3D point clouds. arXiv preprint arXiv:1802.08219, 2018.
[63] Soledad Villar, David W Hogg, Kate Storey-Fisher, Weichi Yao, and Ben Blum-Smith. Scalars are universal: Equivariant machine learning, structured like classical physics. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[64] S. Vichy N. Vishwanathan, Nicol N. Schraudolph, Risi Kondor, and Karsten M. Borgwardt. Graph kernels. Journal of Machine Learning Research, 2010.
[65] Martin J. Wainwright. High-dimensional statistics: A non-asymptotic viewpoint. 2019.
[66] Rui Wang, Robin Walters, and Rose Yu. Incorporating symmetry into deep dynamics models for improved generalization. In Int. Conference on Learning Representations (ICLR), 2021.
[67] Jared Willard, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar. Integrating physics-based modeling with machine learning: A survey. arXiv preprint arXiv:2003.04919, 2020.
[68] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In Int. Conference on Learning Representations (ICLR), 2019.
[69] Dmitry Yarotsky. Universal approximations of invariant maps by neural networks. Constructive Approximation, 2022.
[70] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R. Salakhutdinov, and Alexander J. Smola. Deep sets. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
[71] Sicheng Zhu, Bang An, and Furong Huang. Understanding the generalization benefit of model invariance from a data perspective. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[72] Aaron Zweig and Joan Bruna. A functional perspective on learning symmetric functions with neural networks. In Int. Conference on Machine Learning (ICML), 2021.
A.1 Manifolds
A completely metrizable second countable topological space  $\mathcal{M}$  that is locally homeomorphic to the Euclidean space  $\mathbb{R}^{\dim(\mathcal{M})}$  is called a manifold of dimension  $\dim(\mathcal{M})$ . Sphere, torus, and  $\mathbb{R}^d$  are some examples of manifolds. A Riemannian manifold  $(\mathcal{M}, g)$  is a manifold that is equipped with a smooth inner product  $g$  on the tangent space  $T_x\mathcal{M}$  around each point  $x \in \mathcal{M}$ . A Riemannian metric tensor  $g$  allows the use of the following two important tools in the study of manifolds: (i) the geodesic distance  $d_{\mathcal{M}}(x, y)$  between any points  $x, y \in \mathcal{M}$ , and (ii) the volume element  $d\operatorname{vol}_g(x)$ , that defines a Borel measure on the manifold (on the Borel sigma-algebra of open subsets of the manifold). The distance function and the volume element depend on the Riemannian metric tensor  $g$  and make the manifold a metric-measure space. The functional spaces on manifolds, such as  $L^p(\mathcal{M})$ ,  $p \in [1, \infty]$ , and the Sobolev spaces  $\mathcal{H}^s(\mathcal{M})$ ,  $s \geq 0$ , are defined similar to the Euclidean spaces.
A.2 Isometries
A bijection  $\tau : \mathcal{M} \to \mathcal{M}$  is called an isometry of  $(\mathcal{M}, g)$ , if and only if  $d_{\mathcal{M}}(\tau(x), \tau(y)) = d_{\mathcal{M}}(x, y)$  for all  $x, y \in \mathcal{M}$ . The space of isometries of  $(\mathcal{M}, g)$  forms a group under composition, denoted by  $\mathrm{ISO}(\mathcal{M}, g)$  or simply by  $\mathrm{ISO}(\mathcal{M})$ . By the Myers-Steenrod theorem, for connected manifolds, each  $\tau \in \mathrm{ISO}(M, g)$  is not only bijective but indeed smooth (i.e., infinitely many times differentiable) [52]. Moreover,  $\mathrm{ISO}(\mathcal{M}, g)$  is a Lie group (i.e., a smooth manifold which is simultaneously a group) of dimension at most  $\frac{\dim(\mathcal{M})(\dim(\mathcal{M}) + 1)}{2}$ . Another characterization of isometries on manifolds is based on metric tensors, where a function  $\tau : \mathcal{M} \to \mathcal{M}$  is an isometry if and only if the pullback of the metric tensor  $g$  by  $\tau$  is itself, i.e.,  $g = \tau^* g$ . This means that  $\tau$  locally preserved inner products of tangent vectors. For the applications in this paper, whenever it is not mentioned, a  $\dim(\mathcal{M})$ -dimensional Riemannian manifolds  $(\mathcal{M}, g)$  is smooth, connected, and compact boundaryless.
A.3 Laplacain on Manifolds
The Laplace-Beltrami operator is the unique continuous linear operator  $\Delta_g: \mathcal{H}^s(\mathcal{M}) \to \mathcal{H}^{(s-2)}(\mathcal{M})$  satisfying the property
$$
\int_ {\mathcal {M}} \psi (x) \Delta_ {g} \phi (x) d \operatorname {v o l} _ {g} (x) = - \int_ {\mathcal {M}} \left\langle \nabla_ {g} \phi (x), \nabla_ {g} \psi (x) \right\rangle_ {g} d \operatorname {v o l} _ {g} (x), \tag {24}
$$
for any  $\phi, \psi \in \mathcal{H}^s(\mathcal{M})$ . This generalizes the usual definition of the Laplacian operator  $\Delta = \partial_1^2 + \dots + \partial_d^2$ , defined on the Euclidean space  $\mathbb{R}^d$ , which satisfies this property by integration by parts. The kernel of the operator  $\Delta_g$  includes the so-called harmonic functions. In the case of compact boundaryless manifolds, the only harmonic functions are constants.
The operator  $(- \Delta_g)$  is elliptic, self-adjoint, and can be diagonalized in  $L^2(\mathcal{M})$  [12]. In particular, there exists an orthonormal basis  $\{\phi_\ell(x)\}_{\ell=0}^\infty$  of  $L^2(\mathcal{M})$  starting from the constant function  $\phi_0 \equiv 1$  such that  $\Delta_g \phi_\ell + \lambda_\ell \phi_\ell = 0$ , for the discrete spectrum  $0 = \lambda_0 < \lambda_1 \leq \lambda_2 \leq \dots$ . Note that eigenvalues appear in this sequence with their multiplicities. Let us call  $\{\phi_\ell(x)\}_{\ell=0}^\infty$  the Laplace-Beltrami basis for  $L^2(\mathcal{M})$ . Using the above identity, for an arbitrary  $f \in L^2(\mathcal{M})$  with the expansion  $f = \sum_{\ell=0}^\infty \langle f, \phi_\ell \rangle_{L^2(\mathcal{M})} \phi_\ell$  and  $L^2(\mathcal{M})$ -norm  $\|f\|_{L^2(\mathcal{M})}^2 = \sum_{\ell=0}^\infty |\langle f, \phi_\ell \rangle_{L^2(\mathcal{M})}|^2$ , we have  $\Delta_g f = -\sum_{\ell=0}^\infty \lambda_\ell \langle f, \phi_\ell \rangle \phi_\ell$  and
$$
\begin{array}{l} \left\| \nabla_ {g} f \right\| _ {L ^ {2} (\mathcal {M})} ^ {2} = \int_ {\mathcal {M}} \left\langle \nabla_ {g} f (x), \nabla_ {g} f (x) \right\rangle_ {g} d \operatorname {v o l} _ {g} (x) (25) \\ = - \int_ {\mathcal {M}} f (x) \Delta_ {g} f (x) d \operatorname {v o l} _ {g} (x) (26) \\ = \sum_ {\ell = 0} ^ {\infty} \lambda_ {\ell} \left\langle f, \phi_ {\ell} \right\rangle_ {L ^ {2} (\mathcal {M})} ^ {2}, (27) \\ \end{array}
$$
whenever the sums converge.
A.4 (Local) Weyl's Law
It is known that the Laplace-Beltrami spectrum can encode many manifold geometric properties, such as dimension, volume, etc. However, it is also known that isospectral (non-isomorphic) manifolds exist. Still, it provides rich information about the manifold. Let us denote the set of distinct eigenvalues of a manifold by  $\operatorname{Spec}(\mathcal{M}) \coloneqq \{\lambda_0, \lambda_1, \ldots\} \subset \mathbb{R}_{\geq 0}$ .
Weyl's law characterizes the asymptotic distribution of the eigenvalues in a closed-form formula. Let us denote the number of eigenvalues of the Laplace-Beltrami operator up to  $\lambda$  as  $N(\lambda) \coloneqq \# \{\ell : \lambda_{\ell} \leq \lambda\}$ .
Proposition A.1 (Local Weyl's law, [11, 25, 60]). Let  $(\mathcal{M},g)$  denote an arbitrary  $\dim (\mathcal{M})$  -dimensional compact boundaryless Riemannian manifold. Then, for all  $x\in \mathcal{M}$
$$
N _ {x} (\lambda) := \sum_ {\ell : \lambda_ {\ell} \leq \lambda} | \phi_ {\ell} (x) | ^ {2} = \frac {\omega_ {\dim (\mathcal {M})}}{(2 \pi) ^ {\dim (\mathcal {M})}} \operatorname {v o l} (\mathcal {M}) \lambda^ {\dim (\mathcal {M}) / 2} + \mathcal {O} \left(\lambda^ {\frac {\dim (\mathcal {M}) - 1}{2}}\right), \tag {28}
$$
as  $\lambda \to \infty$ , where  $\omega_{d} = \frac{\pi^{d / 2}}{\Gamma\left(\frac{d}{2} + 1\right)}$  is the volume of the unit  $d$ -ball in the Euclidean space  $\mathbb{R}^d$ . The constant in the error term is indeed independent of  $x$  and may only depend on the sectional curvature and the injectivity radius of the manifold [16]. As a byproduct, the following  $L^{\infty}$  upper bound also holds:
$$
\max  _ {\lambda_ {\ell} \leq \lambda} \max  _ {x \in \mathcal {M}} | \phi_ {\ell} (x) | = \mathcal {O} \left(\lambda^ {\frac {\dim (\mathcal {M}) - 1}{4}}\right). \tag {29}
$$
By definition,  $N(\lambda) = \int_{\mathcal{M}} N_x(\lambda) d\mathrm{vol}_g(x)$  and thus, while the above result is called the local Weyl's law, the traditional Weyl's law can be easily derived as  $N(\lambda) = \frac{\omega_{\dim(\mathcal{M})}}{(2\pi)^{\dim(\mathcal{M})}} \mathrm{vol}(\mathcal{M}) \lambda^{\dim(\mathcal{M})/2} + \mathcal{O}(\lambda^{\frac{\dim(\mathcal{M}) - 1}{2}})$ . For compact Riemannian manifolds  $(\mathcal{M}, g)$  with boundary, to define the eigenfunctions of the Laplace-Beltrami operator (i.e., the solutions to the equation  $\Delta_g \phi + \lambda \phi = 0$ ), one has to consider boundary conditions. For the Dirichlet boundary condition (i.e., assuming the solution vanishes on the boundary), or the Neumann boundary condition (i.e., assuming the solution's gradient vanishes on the outward normal vector at each point of the boundary), the above claim on the local behavior of eigenfunctions still holds.
The asymptotic distribution of eigenvalues allows us to define the Minakshisundaram-Pleijel zeta function of the manifold  $(\mathcal{M},g)$  as follows [47]:
$$
\mathcal {Z} _ {\mathcal {M}} (s) = \sum_ {\ell = 1} ^ {\infty} \lambda_ {\ell} ^ {- s} = \int_ {0 +} ^ {\infty} \lambda^ {- s} d N (\lambda); \quad \Im (s) > \frac {\dim (\mathcal {M})}{2}, \tag {30}
$$
where the sum converges absolutely by Weyl's law. Note that the integral must be understood as a Riemann-Stieltjes integral. The zeta function can be analytically continued to a meromorphic function on the complex plane and has a functional equation. By integration by parts,
$$
\begin{array}{l} \mathcal {Z} _ {\mathcal {M}} (s) = \int_ {0 +} ^ {\infty} \lambda^ {- s} d N (\lambda) (31) \\ = \lambda^ {- s} N (\lambda) \big | _ {0 +} ^ {\infty} - \int_ {0 ^ {+}} ^ {\infty} N (\lambda) (- s) \lambda^ {- s - 1} d \lambda (32) \\ = s \int_ {0 +} ^ {\infty} N (\lambda) \lambda^ {- s - 1} d \lambda , (33) \\ \end{array}
$$
by  $N(\lambda) = \mathcal{O}(\lambda^{\dim (\mathcal{M}) / 2})$  and the assumption  $\Im (s) > \frac{\dim(\mathcal{M})}{2}$ .
For more information on the spectral theory of Riemannian manifolds, see [29].
A.5 Quotient Manifold Theorem
This part and the next subsection review some classical results about group actions on manifolds (mostly from [33, 27, 7]). Let  $G$  be an arbitrary group. The action of  $G$  on the manifold  $\mathcal{M}$  is a mapping  $\theta : G \times \mathcal{M} \to \mathcal{M}$  such that  $\theta(\mathrm{id}_G,.) = \mathrm{id}_{\mathcal{M}}$  and  $\theta(\tau_1\tau_2,.) = \theta(\tau_1,\theta(\tau_2,))$  for any
$\tau_{1}, \tau_{2} \in G$ . In particular, any group action gives a  $G$ -indexed set of bijections on  $\mathcal{M}$  with respect to the group law of  $G$ . For example,  $\mathrm{ISO}(\mathcal{M})$  acts on  $\mathcal{M}$  by the isometric transformations (which are bijections by definition).
For each  $x \in \mathcal{M}$ , the orbit of  $x$  is defined as the set of all images of the group transformations at  $x$  on the manifold:
$$
[ x ] := \left\{\theta (\tau , x) \in \mathcal {M}: \tau \in G \right\}. \tag {34}
$$
Also, for any  $x \in \mathcal{M}$ , define the isotropy group  $G_{x} := \{\tau \in G : \theta(\tau, x) = x\}$ . In other words, the isotropy group  $G_{x}$ , as a subgroup of  $G$ , includes all the transformations  $\tau \in G$  which have  $x \in \mathcal{M}$  as a fixed point. The set of all orbits is denoted by  $\mathcal{M}/G$  and is called the orbit space (or sometimes the fundamental domain) of the action on the manifold:
$$
\mathcal {M} / G := \{[ x ]: x \in \mathcal {M} \}. \tag {35}
$$
It is known that  $\mathcal{M} / G$  admits a unique topological structure coming from the topology of the manifold, making the projection (or the quotient) map  $\pi : \mathcal{M} \to \mathcal{M} / G$  continuous.
However,  $\mathcal{M} / G$  is not always a manifold. For example, if  $\mathcal{M} = \mathbb{R}^d$  and  $G = \mathrm{GL}_d(\mathbb{R})$ , then the resulting orbit space  $\mathcal{M} / G$  is not Hausdorff. Even in cases where it is a manifold, the orbit space  $\mathcal{M} / G$  may have boundaries while  $\mathcal{M}$  is boundaryless. For example, consider the action of the orthogonal group  $\mathcal{O}(d) \coloneqq \{A \in \mathrm{GL}_d(\mathbb{R}) : A^TA = A^TA = I_d\}$  on the manifold  $\mathcal{M} = \mathbb{R}^d$ , where the orbit space becomes  $\mathcal{M} / G = \mathbb{R}_{\geq 0}$ . For the purpose of this paper, boundaries are important and affect the results drastically.
The quotient manifold theorem gives a number of sufficient conditions on the manifold/group, such that the orbit space is always a manifold. To introduce the theorem, we need to review a few classical definitions as follows.
A group action is called free, if and only if it has no non-trivial fixed points, i.e.,  $\theta(\tau, x) \neq x$  for all  $\tau \neq \mathrm{id}_G$  (equivalently,  $G_x = \{\mathrm{id}_G\}$  for all  $x \in \mathcal{M}$ ). For example, the action of the group of linear transformations  $\theta(r, x) = x + r$ , for each  $x, r \in \mathbb{R}^d$ , on the manifold  $\mathbb{R}^d$  is a free action. An action is called smooth if and only if for each  $\tau \in G$ , the mapping  $\theta(\tau, .): \mathcal{M} \to \mathcal{M}$  is smooth. An action is called proper, if and only if the map  $(\theta(\tau, x), x): G \times \mathcal{M} \to \mathcal{M} \times \mathcal{M}$  is a proper map. As a sufficient condition, every continuous action of a compact Lie group  $G$  on a manifold is proper. To simplify our notation, let us define  $\tau(x) := \theta(\tau, x)$ .
Theorem A.2 (Quotient Manifold Theorem, [33]). Let  $G$  be a Lie group acting smoothly, freely, and properly on a smooth manifold  $(\mathcal{M}, g)$ . Then, the orbit space (or the fundamental domain)  $\mathcal{M} / G$  is a smooth manifold of dimension  $\dim(\mathcal{M}) - \dim(G)$  with a unique smooth structure such that the projection (or the quotient) map  $\pi: \mathcal{M} \to \mathcal{M} / G$  is a smooth submersion.
Corollary A.3. Suppose  $\mathcal{M}$  is a connected smooth compact boundaryless manifold. Then, the isometry group  $\mathrm{ISO}(\mathcal{M})$  is a compact Lie group acting smoothly on  $\mathcal{M}$ . Thus, if  $G$  is a closed subgroup of  $\mathrm{ISO}(\mathcal{M})$ , then the action of  $G$  on  $\mathcal{M}$  is smooth and proper. In addition, if this action is free, then the orbit space  $\mathcal{M}/G$  becomes a connected closed (i.e., compact boundaryless) manifold of dimension  $\dim(\mathcal{M}) - \dim(G)$ .
Furthermore, assuming  $(\mathcal{M},g)$  is a Riemannian manifold, there exists a unique Riemannian metric  $\tilde{g}$  such that the projection map  $\pi :(\mathcal{M},g)\to (\mathcal{M} / G,\tilde{g})$  is a Riemannian submersion.
There is indeed a natural way to interpret these results. Given a manifold  $(\mathcal{M},g)$ , the orbit space is somehow the shrinkage of the manifold to represent exactly one representative from each orbit, and the metric  $\tilde{g}$  is just identical to the original metric if the tangent lines survive; otherwise, the tangent lines are killed, and the inner product defined by  $\tilde{g}$  is zero.
A.6 Principal Orbit Theorem
The quotient manifold theorem is, however, restricted to free actions. Unfortunately, this assumption is generally necessary to prove that the quotient space is a compact boundaryless manifold. In the lack of freeness, it is still possible to show that the quotient space is almost a manifold (possibly with boundary). First, let us introduce a few definitions. Two isotropy groups  $G_{x}$  and  $G_{y}$  are called conjugate (with respect to  $G$ ), if and only if  $\tau^{-1}G_{x}\tau = G_{y}$  for some  $\tau \in G$ . This is a relation on the
manifold, and the isotropy type of any  $x \in \mathcal{M}$  is defined as the equivalence class of its isotropy group, denoted as  $[G_x]$ . Since all the points on an orbit have the same isotropy type (by definition), one can also define the isotropy type of an orbit in the same way. Given the conjugacy relation, a partial order can be naturally defined as follows:  $H_1 \preceq H_2$  if and only if  $H_1$  is conjugate to a subgroup of  $H_2$ . Since the definition is unchanged modulo conjugate groups, one can also restrict the partial order to the conjugacy class of subgroups. This allows us to define a partial order on orbits as well: for any two orbits  $[x], [y]$ , we write  $[x] \leq [y]$  if and only if  $G_y \preceq G_x$ . For example, if  $G_x = \{\mathrm{id}_G\}$ , then  $[y] \leq [x]$  for all  $y \in \mathcal{M}$ .
Given the above formal definitions, the quotient manifold theorem assumes that all the orbits have a unique maximal orbit type, namely, the orbit type  $\{\mathrm{id}_G\}$ . By removing the freeness assumption, however, there might be several orbit types that are not necessarily comparable with respect to the partial order. However, the principal orbit type theorem shows that there always exists a unique maximal orbit type, where when the action is restricted to those orbits, it defines a Riemannian submersion (thus the image is a manifold), and moreover, the principal orbits (those orbit with the maximal orbit types) are dense in the manifold  $\mathcal{M}$ . This shows that we almost get a nice space, which suffices for our subsequent proofs in the next sections.
Theorem A.4 (Principal Orbit Theorem). Let  $G$  be a compact Lie group acting isometrically on a Riemannian manifold  $(\mathcal{M}, g)$ . Then, there exists an open dense subset  $\mathcal{M}_0 \subseteq \mathcal{M}$ , such that  $[x] \geq [y]$  for all  $x \in \mathcal{M}_0$  and  $y \in \mathcal{M}$ . Moreover, the natural projection  $\pi : \mathcal{M}_0 \to \mathcal{M}_0 / G \subseteq \mathcal{M} / G$  is a Riemannian submersion. Also, the set  $\mathcal{M}_0 / G \subseteq \mathcal{M} / G$  is open, dense, and connected in  $\mathcal{M} / G$ .
Corollary A.5. One has the decomposition
$$
\mathcal {M} / G = \bigsqcup_ {[ H ] \preceq G} \mathcal {M} _ {[ H ]} / G, \tag {36}
$$
where  $\mathcal{M}_{[H]} := \{x \in \mathcal{M} : [G_x] = [H]\}$  is a submanifold of  $\mathcal{M}$ . The disjoint union is taken over all isotropy types of the group action on the manifold. The map  $\pi: \mathcal{M}_{[H]} \to \mathcal{M}_{[H]} / G$  is a Riemannian submersion; therefore its image is a manifold. By an application of the Slice theorem, one can observe that only finitely many isotropy types can exist when  $G$  and  $\mathcal{M}$  are both compact. Thus, the disjoint union is indeed over finitely many precompact smooth manifolds. Among those, the principal part  $\mathcal{M}_0 / G := \mathcal{M}_{[H_0]} / G$  is dense in  $\mathcal{M} / G$ , where  $[H_0]$  is the unique maximal isotropy type of the group action.
Intuitively, the above corollary shows that the quotient space can be decomposed to finitely many "pieces," and each piece has a nice smooth structure. In the case of a free action, the decomposition above reduces to just one "piece" with the unique trivial maximal orbit type (i.e., having a trivial isotropy group). The dimension of each "piece"  $\mathcal{M}_{[H]} / G$  can be computed as
$$
\dim (\mathcal {M} _ {[ H ]} / G) = \dim (\mathcal {M}) - \dim (G) + \dim (H). \tag {37}
$$
The effective dimension of the quotient space is then defined as
$$
d := \dim (\mathcal {M} _ {[ H _ {0} ]} / G) = \dim (\mathcal {M} _ {0} / G) = \dim (\mathcal {M}) - \dim (G) + \dim (H _ {0}), \tag {38}
$$
where  $[H_0]$  is the unique maximal isotropy type of the group action.
A.7 Isometries and Laplace-Beltrami Eigenspaces
In Weyl's law, the eigenvalues are counted with their multiplicities. Let us define  $V_{\lambda}$  as the eigenspace of the eigenvalue  $\lambda$  with the finite dimension  $\dim(V_{\lambda})$ . Then,  $N(\lambda) = \sum_{\lambda' \leq \lambda} \dim(V_{\lambda'})$ .
As a well-known fact from differential geometry, the Laplace-Beltrami operator  $\Delta_g$  commutes with all isometries  $\tau \in \mathrm{ISO}(\mathcal{M})$ . Thus,
$$
\Delta_ {g} \phi_ {\ell} + \lambda_ {\ell} \phi_ {\ell} = 0 \Longleftrightarrow \Delta_ {g} \left(\phi_ {\ell} \circ \tau\right) + \lambda_ {\ell} \left(\phi_ {\ell} \circ \tau\right) = 0. \tag {39}
$$
This means that the eigenspaces of the Laplace-Beltrami operator are invariant with respect to the action of the isometry group on the manifold.