SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Corollary A.6.  $L^2(\mathcal{M}) = \bigoplus_{\lambda \in \operatorname{Spec}(\mathcal{M})} V_\lambda$ , and the isometry group of the manifold  $\operatorname{ISO}(\mathcal{M})$  (and thus all its closed subgroups) acts on each eigenspace  $V_\lambda$ .
Now consider an arbitrary closed subgroup  $G$  of the isometry group  $\mathrm{ISO}(\mathcal{M})$ . Then  $G$  acts on the eigenspaces of the Laplace-Beltrami operator and each  $\tau \in G$  corresponds to a bijective linear transformation  $f \mapsto f \circ \tau$ , denoted as  $\tau_{\lambda}^{*}: V_{\lambda} \to V_{\lambda}$ . There is a natural way to extend this operator into the whole space  $L^2 (\mathcal{M})$  so one may consider  $\tau_{\lambda}^{*}: L^{2}(\mathcal{M}) \to L^{2}(\mathcal{M})$ . Since  $\tau$  is an isometry, for any  $\phi \in V_{\lambda}$ ,
$$
\begin{array}{l} \left\| \tau_ {\lambda} ^ {*} \phi \right\| _ {L ^ {2} (\mathcal {M})} ^ {2} = \int_ {\mathcal {M}} \left| \tau_ {\lambda} ^ {*} \phi (x) \right| ^ {2} d \operatorname {v o l} _ {g} (x) (40) \\ = \int_ {\mathcal {M}} | \phi (x) | ^ {2} d \operatorname {v o l} _ {g} (x) (41) \\ = \| \phi \| _ {L ^ {2} (\mathcal {M})} ^ {2} (42) \\ = 1, (43) \\ \end{array}
$$
since  $d\operatorname{vol}_g$  is invariant with respect to any isometry  $\tau \in G$ . This shows that the bijective linear transformation  $\tau_{\lambda}^{*}$  is indeed a representation of the group  $G$  into the orthogonal group  $\mathcal{O}(\dim(V_{\lambda}))$  for each eigenvalue  $\lambda$ .
In this paper, we are interested in the space of invariant functions defined with respect to an arbitrary closed subgroup  $G$  of the isometry group  $\mathrm{ISO}(\mathcal{M})$ .
Definition A.7. The space of invariant functions with respect to a closed subgroup  $G$  of the isometry group  $\mathrm{ISO}(\mathcal{M})$  is defined as
$$
L _ {\operatorname {i n v}} ^ {2} (\mathcal {M}, G) := \left\{f \in L ^ {2} (\mathcal {M}): \forall \tau \in G: \tau^ {*} f := f \circ \tau = f \right\} \subseteq L ^ {2} (\mathcal {M}), \tag {44}
$$
as a closed subspace of  $L^2 (\mathcal{M})$
Let  $d\tau$  denote the Haar measure (i.e., the uniform measure) associated with a closed subgroup  $G$  of the isometry group  $\mathrm{ISO}(\mathcal{M})$ . Let the projection operator  $\mathcal{P}_G: L^2(\mathcal{M}) \to L_{\mathrm{inv}}^2(\mathcal{M}, G)$  be defined as  $f(x) \mapsto \int_G (f \circ \tau)(x)d\tau = \int_G \tau^* f(x)d\tau$ . Clearly,  $f \in L_{\mathrm{inv}}^2(\mathcal{M}, G)$  if and only if  $f \in \ker(I - \mathcal{P}_G)$ .
Proposition A.8. For any closed subgroup  $G$  of the isometry group  $\mathrm{ISO}(\mathcal{M})$ , the following decomposition holds:
$$
\begin{array}{l} L _ {\text {i n v}} ^ {2} (\mathcal {M}, G) = \ker (I - \mathcal {P} _ {G}) (45) \\ = \bigcap_ {\substack {\lambda \in \operatorname {Spec} (\mathcal{M}) \\ \tau \in G}} \ker (I - \tau_ {\lambda} ^ {*}) (46) \\ = \bigoplus_ {\lambda \in \operatorname {S p e c} (\mathcal {M})} V _ {\lambda , G}, (47) \\ \end{array}
$$
where each  $V_{\lambda ,G}$  is a linear subspace of  $V_{\lambda}$  defined as
$$
\begin{array}{l} V _ {\lambda , G} := \left\{f \in V _ {\lambda}: \forall \tau \in G: \tau_ {\lambda} ^ {*} f := f \circ \tau = f \right\} (48) \\ = \bigcap_ {\tau \in G} \ker \left(I - \tau_ {\lambda} ^ {*}\right). (49) \\ \end{array}
$$
Clearly,  $\dim (V_{\lambda ,G})\leq \dim (V_{\lambda})$
Moreover, the (restricted) projection operator  $\mathcal{P}_G:V_\lambda \to V_\lambda$  has the image  $V_{\lambda ,G}$  and it can be diagonalized in a basis for  $V_{\lambda}$  such as  $\phi_{\lambda ,\ell}\in V_{\lambda},\ell = 1,2,\ldots ,\dim (V_{\lambda})$ , such that for each  $f = \sum_{\ell = 1}^{\dim (V_{\lambda})}\alpha_{\ell}\phi_{\lambda ,\ell}\in V_{\lambda}$
$$
f \in V _ {\lambda , G} \Longleftrightarrow \forall \ell > \dim \left(V _ {\lambda , G}\right): \alpha_ {\ell} = 0 \tag {50}
$$
$$
\dim (V _ {\lambda , G})
$$
$$
\mathcal {P} _ {\lambda , G} f = \sum_ {\ell = 1} \alpha_ {\ell} \phi_ {\lambda , \ell} \in V _ {\lambda , G}. \tag {51}
$$
Due to its simplicity, we omit the proof of this proposition. We always consider the diagonalized basis in this paper, as it always exists for appropriate eigenfunctions.
A.8 Reproducing Kernel Hilbert Spaces on Manifolds
A smooth connected compact boundaryless Riemannian manifold  $(\mathcal{M},g)$  is indeed a compact metric measure space, and a kernel  $K:\mathcal{M}\times \mathcal{M}\to \mathbb{R}$  can be thought of as a measure of similarity between points on the manifold. We assume that  $K$  is continuous, symmetric, and positive-definite, meaning that  $K(x,y) = K(y,x)$  and  $\sum_{i,j = 1}^{n}a_{i}a_{j}K(x_{i},y_{j})\geq 0$  for any  $a_i\in \mathbb{R},x_i,y_j\in \mathcal{M},i,j = 1,2,\ldots ,n$ , and the equality happens only when  $a_1 = a_2 = \dots = a_n = 0$  (assuming the points on the manifold are distinct). The Reproducing Kernel Hilbert Space (RKHS) of  $K$  is a Hilbert space  $\mathcal{H}\subseteq L^2 (\mathcal{M})$  that is achieved by the completion of the span of functions  $K(:,y)\in \mathcal{H}$  for each  $y\in \mathcal{M}$ , satisfying the following property: for all  $f\in \mathcal{H}$ ,  $f(x) = \langle f,K(x,.)\rangle_{\mathcal{H}}$ . Associated with the kernel  $K$ , there exists an integral operator  $\mathcal{K}:L^2 (\mathcal{M})\to L^2 (\mathcal{M})$  defined as  $\mathcal{K}(f) = \int_{\mathcal{M}}K(x,y)f(y)d\mathrm{vol}_g(y)$ . It can be shown that  $K$  can be diagonalized in an appropriate orthonormal basis of functions in  $L^2 (\mathcal{M})$  (Mercer's theorem). Indeed, with a number of appropriate assumptions, it can be diagonalized in the Laplace-Beltrami spectrum.
Proposition A.9. Consider a symmetric positive-definite kernel  $K:\mathcal{M}\times \mathcal{M}\to \mathbb{R}$ , and assume that  $K\in C^2 (\mathcal{M}\times \mathcal{M})$  satisfies the differential equation  $\Delta_{g,x}(K(x,y)) = \Delta_{g,y}(K(x,y))$ . Then,  $K$  can be diagonalized in the basis of the eigenfunctions of the Laplace-Beltrami operator; there exist appropriate  $\mu_{\ell}\geq 0,\ell = 0,1,\ldots$ , such that
$$
K (x, y) = \sum_ {\ell = 0} ^ {\infty} \mu_ {\ell} \phi_ {\ell} (x) \phi_ {\ell} (y), \tag {52}
$$
where  $\phi_{\ell},\ell = 0,1,\ldots$  , form an orthonormal basis for  $L^2 (\mathcal{M})$  such that  $\Delta_g\phi_\ell +\lambda_\ell \phi_\ell = 0$  for each  $\ell$
Proof. Note that  $\mathcal{K}(\phi_{\ell}(x)) = \int_{\mathcal{M}} K(x,y) \phi_{\ell}(y) d\mathrm{vol}_g(y)$ . Therefore,
$$
\begin{array}{l} \Delta_ {g, x} \left(\mathcal {K} \left(\phi_ {\ell}\right)\right) = \Delta_ {g, x} \left(\int_ {\mathcal {M}} K (x, y) \phi_ {\ell} (y) d \operatorname {v o l} _ {g} (y)\right) (53) \\ = \int_ {\mathcal {M}} \Delta_ {g, x} K (x, y) \phi_ {\ell} (y) d \operatorname {v o l} _ {g} (y) (54) \\ = \int_ {\mathcal {M}} \Delta_ {g, y} K (x, y) \phi_ {\ell} (y) d \operatorname {v o l} _ {g} (y) (55) \\ = \int_ {\mathcal {M}} K (x, y) \Delta_ {g, y} \phi_ {\ell} (y) d \operatorname {v o l} _ {g} (y) (56) \\ = \lambda_ {\ell} \int_ {\mathcal {M}} K (x, y) \phi_ {\ell} (y) d \operatorname {v o l} _ {g} (y) (57) \\ = \lambda_ {\ell} \mathcal {K} \left(\phi_ {\ell}\right), (58) \\ \end{array}
$$
where we used the symmetry of the kernel, the regularity condition of the kernel (allowing the interchange of the differentiation and the integral sign), and also the self-adjointness of the Laplace-Beltrami operator. Now since  $\mathcal{K}(\phi_{\ell})$  satisfies the equation  $\Delta_g(\mathcal{K}(\phi_\ell)) + \lambda_i\mathcal{K}(\phi_\ell) = 0$ , we conclude that  $\mathcal{K}(\phi_{\ell})$  is indeed an eigenfunction with respect to the eigenvalue  $\lambda_{\ell}$ , or equivalently  $\mathcal{K}(\phi_{\ell})\in V_{\lambda_{\ell}}$ . In other words,  $V_{\lambda}$ ,  $\lambda \in \operatorname {Spec}(\mathcal{M})$ , are the invariant subspaces of the integral operator associated with the kernel. This means that one can choose an appropriate basis of eigenfunctions in each eigenspace, such that the kernel is diagonalized in each eigenspace (Mercer's theorem).
Remark A.10. While Proposition A.9 holds for a kernel  $K \in C^2(\mathcal{M} \times \mathcal{M})$ , it can be shown that it holds under a weaker assumption that  $K$  is just continuous. The identity  $\Delta_{g,x}(K(x,y)) = \Delta_{g,y}(K(x,y))$  should then be understood as the identity of two distributions.
In this paper, we always consider the diagonalized kernels in the Laplace-Beltrami spectrum. An example of a kernel of this form is the heat kernel with  $\mu_{\ell} = e^{-\lambda_{\ell}t}$ ,  $t \in \mathbb{R}$ . Given a diagonalized kernel  $K(x,y) = \sum_{\ell=0}^{\infty} \mu_{\ell} \phi_{\ell}(x) \phi_{\ell}(y)$ , one can explicitly define the RKHS associated with  $K$  as
$$
\mathcal {H} = \left\{f = \sum_ {\ell = 0} ^ {\infty} \alpha_ {\ell} \phi_ {\ell}: \sum_ {\ell = 0} ^ {\infty} \frac {\left| \alpha_ {\ell} \right| ^ {2}}{\mu_ {\ell}} <   \infty \right\}, \tag {59}
$$
with the inner-product
$$
\left\langle \sum_ {\ell = 0} ^ {\infty} \alpha_ {\ell} \phi_ {\ell}, \sum_ {\ell = 0} ^ {\infty} \beta_ {\ell} \phi_ {\ell} \right\rangle_ {\mathcal {H}} = \sum_ {\ell = 0} ^ {\infty} \frac {\alpha_ {\ell} \beta_ {\ell}}{\mu_ {\ell}}, \tag {60}
$$
where the sum is considered convergent whenever it converges absolutely. The feature map is therefore given as  $\Phi_x = K(x,.) = \sum_{\ell=0}^{\infty} \mu_\ell \phi_\ell(x) \phi_\ell(.)$  for any  $x \in \mathcal{M}$ . The covariance operator  $\Sigma: \mathcal{H} \to \mathcal{H}$  is also defined as  $\Sigma = \mathbb{E}_{x \sim \mu}[\Phi_x \otimes_{\mathcal{H}} \Phi_x]$  where the expectation is with respect to the uniform sample  $x \in \mathcal{M}$  (with respect to the normalized volume element  $d\mu = \frac{1}{\operatorname{vol}(\mathcal{M})} d\operatorname{vol}_g(x)$ ). It is worth mentioning the identity  $\| \mathcal{K}^{1/2}(f) \|_{\mathcal{H}} = \| f \|_{L^2(\mathcal{M})}$ . Also note if  $f^\star = \sum_{\ell=0}^{\infty} \langle f^\star, \phi_\ell \rangle_{L^2(\mathcal{M})} \phi_\ell$ , then the effective ridge regression estimator (Equation 4) is given by the closed-form formula
$$
\hat {f} _ {\text {e f f}} = \sum_ {\ell = 0} ^ {\infty} \frac {\mu_ {\ell}}{\mu_ {\ell} + \eta} \left\langle f ^ {\star}, \phi_ {\ell} \right\rangle_ {L ^ {2} (\mathcal {M})} \phi_ {\ell}. \tag {61}
$$
A.9 Invariant Kernels
A kernel  $K:\mathcal{M}\times \mathcal{M}\to \mathbb{R}$  is called  $G$  -invariant with respect to a closed subgroup  $G$  of  $\mathrm{ISO}(\mathcal{M})$  if and only if  $K(x,y) = K(\tau (x),\tau '(y))$  for any  $\tau ,\tau^{\prime}\in G$  . Equivalently, one has  $K(x,y) =$ $K([x],[y])$  for any  $x,y\in \mathcal{M}$  . In the previous section, it is observed that  $K$  can be written as  $K(x,y) = \sum_{\ell = 0}^{\infty}\mu_{\ell}\phi_{\ell}(x)\phi_{\ell}(y)$  , under a few conditions. Since  $K$  is  $G$  -invariant, a new basis of eigenfunctions exists that allows a more compact representation of the kernel.
Proposition A.11. For any closed subgroup  $G$  of  $\mathrm{ISO}(\mathcal{M})$ , consider a symmetric positive-definite  $G$ -invariant kernel  $K:\mathcal{M}\times \mathcal{M}\to \mathbb{R}$ , and assume that  $K\in C^{2}(\mathcal{M}\times \mathcal{M})$  satisfies the differential equation  $\Delta_{g,x}(K(x,y)) = \Delta_{g,y}(K(x,y))$ . Then,  $K$  can be diagonalized in the basis of eigenfunctions of the Laplace-Beltrami operator:
$$
K (x, y) = \sum_ {\lambda \in \operatorname {S p e c} (\mathcal {M})} \sum_ {\ell = 1} ^ {\dim \left(V _ {\lambda , G}\right)} \mu_ {\lambda , \ell} \phi_ {\lambda , \ell} (x) \phi_ {\lambda , \ell} (y), \tag {62}
$$
where the functions  $\phi_{\lambda,\ell}$ , for any  $\lambda \in \operatorname{Spec}(\mathcal{M})$ , and any  $\ell = 1, \ldots, \dim(V_{\lambda})$ , form a basis for  $L^2(\mathcal{M})$  such that  $\Delta_g(\phi_{\lambda,\ell}) + \lambda(\phi_{\lambda,\ell}) = 0$  for each  $\ell, \lambda$ . Moreover, the functions  $\phi_{\lambda,\ell}$ , for any  $\lambda \in \operatorname{Spec}(\mathcal{M})$ , and any  $\ell = 1, \ldots, \dim(V_{\lambda,G})$ , form an orthonormal basis for  $L_{\mathrm{inv}}^2(\mathcal{M},G)$ .
Therefore, the RKHS of a  $G$ -invariant kernel  $K$  can be defined as
$$
\mathcal {H} = \left\{f = \sum_ {\lambda \in \operatorname {S p e c} (\mathcal {M})} \sum_ {\ell = 1} ^ {\dim (V _ {\lambda , G})} \alpha_ {\lambda , \ell} \phi_ {\lambda , \ell}: \sum_ {\lambda \in \operatorname {S p e c} (\mathcal {M})} \sum_ {\ell = 1} ^ {\dim (V _ {\lambda , G})} \frac {\left| \alpha_ {\lambda , \ell} \right| ^ {2}}{\mu_ {\lambda , \ell}} <   \infty \right\}, \tag {63}
$$
$$
\left\langle \sum_ {\lambda \in \operatorname {S p e c} (\mathcal {M})} \sum_ {\ell = 1} ^ {\dim (V _ {\lambda , G})} \alpha_ {\lambda , \ell} \phi_ {\lambda , \ell}, \sum_ {\lambda \operatorname {S p e c} (\mathcal {M})} \sum_ {\ell = 1} ^ {\dim (V _ {\lambda , G})} \beta_ {\lambda , \ell} \phi_ {\lambda , \ell} \right\rangle_ {\mathcal {H}} = \sum_ {\lambda \in \operatorname {S p e c} (\mathcal {M})} \sum_ {\ell = 1} ^ {\dim (V _ {\lambda , G})} \frac {\alpha_ {\lambda , \ell} \beta_ {\lambda , \ell}}{\mu_ {\lambda , \ell}}. \tag {64}
$$
Whenever  $G$  is the trivial group, the above identities reduce to what is proposed for general (not necessarily invariant) kernels on manifolds in the previous section. Once again, the assumption  $K \in C^2(\mathcal{M} \times \mathcal{M})$  can be weakened to just the continuity of  $K$ .
A.10 Sobolev Spaces of Functions on Manifolds
For any integer  $s \geq 0$ , the Sobolev space  $\mathcal{H}^s(\mathcal{M})$  is the Hilbert space of measurable functions on  $\mathcal{M}$  with square-integrable partial derivatives up to order  $s$ . More generally,  $\mathcal{H}^{s,q}(\mathcal{M})$  denotes the Banach space of measurable functions with  $L^p$  bounded partial derivatives up to order  $s$ . As
5The partial derivatives on manifolds are defined locally in each coordinate.
observed in [24], one can define the Sobolev space  $\mathcal{H}^s (\mathcal{M})\subset L^2 (\mathcal{M})$  using the eigenfunctions of the Laplace-Beltrami operator as
$$
\mathcal {H} ^ {s} (\mathcal {M}) := \left\{f = \sum_ {\ell = 0} ^ {\infty} \alpha_ {\ell} \phi_ {\ell}: \| f \| _ {\mathcal {H} ^ {s} (\mathcal {M})} ^ {2} = \sum_ {\ell = 0} ^ {\infty} \max  \left(1, \lambda_ {\ell} ^ {s}\right) | \alpha_ {\ell} | ^ {2} <   \infty \right\}. \tag {65}
$$
The inner-product on  $\mathcal{H}^s (\mathcal{M})$  is defined as
$$
\left\langle \sum_ {\ell = 1} ^ {\infty} \alpha_ {\ell} \phi_ {\ell}, \sum_ {\ell = 1} ^ {\infty} \alpha_ {\ell} \phi_ {\ell} \right\rangle_ {\mathcal {H} ^ {s} (\mathcal {M})} = \sum_ {\ell = 1} ^ {\infty} \max  \left(1, \lambda_ {\ell} ^ {s}\right) \alpha_ {\ell} \beta_ {\ell}. \tag {66}
$$
This makes  $\mathcal{H}^s (\mathcal{M})$  an RKHS with the Sobolev kernel defined as
$$
K _ {\mathcal {H} ^ {s} (\mathcal {M})} (x, y) = \sum_ {\lambda \in \operatorname {S p e c} (\mathcal {M})} \sum_ {\ell = 1} ^ {\dim (V _ {\lambda})} \min  \left(1, \lambda_ {\ell} ^ {- s}\right) \phi_ {\lambda , \ell} (x) \phi_ {\lambda , \ell} (y), \tag {67}
$$
For  $G$ -invariant functions, as before, the above sum must be truncated to  $\dim(V_{\lambda, G})$  instead of  $\dim(V_{\lambda})$ . Therefore,  $\mathcal{H}_{\mathrm{inv}}^s(\mathcal{M}) = \mathcal{H}^s(\mathcal{M}) \cap L_{\mathrm{inv}}^2(\mathcal{M}, G)$  is well-defined.
We note that  $\mathcal{H}^s (\mathcal{M})$  includes only continuous functions when  $s > d / 2$ . Moreover, it contains only continuously differentiable functions up to order  $k$  when  $s > d / 2 + k$ ; see the Sobolev inequality:
Proposition A.12 ([3], Sobolev inequality). Let  $\frac{1}{2} - \frac{s}{d} = \frac{1}{q} - \frac{\ell}{d}$  with  $s \geq \ell \geq 0$  and  $q > 2$ , where  $d$  is the dimension of the smooth compact closed manifold  $\mathcal{M}$ . Then,
$$
\| f \| _ {\mathcal {H} ^ {\ell , q} (\mathcal {M})} \leq C \| f \| _ {\mathcal {H} ^ {s} (\mathcal {M})}. \tag {68}
$$
The constant  $C$  may depend only on the manifold and the parameters but is independent of the function  $f \in L^{2}(\mathcal{M})$ .
B Proof of Theorem 4.4
We first prove Theorem 4.4 for the cases that the group action on the manifold is free (Proposition B.1), and then we extend it to the general case. We use the preset notation/definitions introduced in Appendix A (specifically, Proposition A.8) in this section.
Proposition B.1. Let  $(\mathcal{M},g)$  be a smooth connected compact boundaryless Riemannian manifold of dimension  $\dim (\mathcal{M})$ . Let  $G$  be a Lie subgroup of  $\mathrm{ISO}(\mathcal{M})$  of dimension  $\dim (G)$ , and assume that  $G$  acts freely on  $\mathcal{M}$  (i.e., having no non-trivial fixed point), and let  $d\coloneqq \dim (\mathcal{M}) - \dim (G)$  denote the effective dimension of the quotient space. Then,
$$
N _ {x} (\lambda ; G) := \sum_ {\lambda^ {\prime} \leq \lambda} \sum_ {\ell = 1} ^ {\dim \left(V _ {\lambda^ {\prime}, G}\right)} | \phi_ {\lambda^ {\prime}, \ell} (x) | ^ {2} = \frac {\omega_ {d}}{(2 \pi) ^ {d}} \operatorname {v o l} (\mathcal {M} / G) \lambda^ {d / 2} + \mathcal {O} \left(\lambda^ {\frac {d - 1}{2}}\right), \tag {69}
$$
as  $\lambda \to \infty$ , where  $\omega_{d} = \frac{\pi^{d / 2}}{\Gamma(\frac{d}{2} + 1)}$  is the volume of the unit  $d$ -ball in the Euclidean space  $\mathbb{R}^d$ .
Remark B.2. Note that the above proposition provides a much stronger result than Theorem 4.4; it is local. Observe that  $N(\lambda; G) = \int_{\mathcal{M}} N_x(\lambda; G) d\mathrm{vol}_g(x)$ , and thus integrating the left-hand side of the above equation proves Theorem 4.4 for the special case of free actions. We will later prove the same local result (Equation (69)) for the general smooth compact Lie group actions on a manifold, presuming the assumptions in Theorem 4.4.
Proof of Proposition B.1. By the quotient manifold theorem (Theorem A.2) and Corollary A.3, the orbit space  $\mathcal{M} / G$  is a connected closed (i.e., compact boundaryless) manifold of dimension  $d = \dim (\mathcal{M}) - \dim (G)$ . Let  $\Delta_g$  and  $\Delta_{\tilde{g}}$  denote the Laplace-Beltrami operators on  $\mathcal{M}$  and  $\mathcal{M} / G$ , respectively, where  $\tilde{g}$  is the induced Riemannian metric on  $\mathcal{M} / G$  from  $g$ . Consider two arbitrary smooth functions  $\phi :\mathcal{M}\to \mathbb{R}$  and  $\tilde{\phi}:\mathcal{M} / G\to \mathbb{R}$  such that  $\phi (x) = \tilde{\phi} ([x])$ . Note that  $\phi$  is smooth on  $\mathcal{M}$ , if and only if  $\tilde{\phi}$  is smooth on  $\mathcal{M} / G$ , and also,  $\phi$  is invariant by definition. Fix an arbitrary  $\lambda$ . We claim that
$$
\Delta_ {\tilde {g}} \tilde {\phi} + \lambda \tilde {\phi} = 0 \Longleftrightarrow \Delta_ {g} \phi + \lambda \phi = 0 \tag {70}
$$
Given the above identity, the desired result follows immediately by an application of the local Weyl's law (Proposition A.1) on the manifold  $\mathcal{M} / G$  of dimension  $d = \dim (\mathcal{M}) - \dim (G)$ .
To prove the claim, we only need to show that  $\Delta_{\tilde{g}}\tilde{\phi} = \Delta_g\phi$ . First assume that  $G$  is a finite group (i.e.,  $\dim(G) = 0$ ). Note that in local coordinates  $(x^1, x^2, \ldots, x^{\dim(\mathcal{M})})$  we have
$$
\Delta_ {g} \phi = \frac {1}{\sqrt {| \det  g |}} \partial_ {i} \left(\sqrt {| \det  g |} g ^ {i j} \partial_ {j} \phi\right). \tag {71}
$$
However, the projection map  $\pi : \mathcal{M} \to \mathcal{M} / G$  is a Riemannian submersion, with differential  $d\pi_x : T_x\mathcal{M} \to T_{[x]}(\mathcal{M} / G)$  being an invertible linear map from a  $\dim(\mathcal{M})$ -dimensional vector space to another  $\dim(\mathcal{M})$ -dimensional vector space. This shows that the local coordinates  $(x^1, x^2, \ldots, x^{\dim(\mathcal{M})})$  are also simultaneously some local coordinates for  $\mathcal{M} / G$ , and since  $\tilde{g}$  is induced by the metric  $g$ , the result holds by the above identity for the Laplace-Beltrami operator.
Now assume  $\dim(G) \geq 1$ . In this case, for the projection map  $\pi: \mathcal{M} \to \mathcal{M}/G$ , the differential map  $d\pi_x: T_x\mathcal{M} \to T_{[x]}(\mathcal{M}/G)$  is a surjective linear map from a  $\dim(\mathcal{M})$ -dimensional vector space to a  $(\dim(\mathcal{M}) - \dim(G))$ -dimensional vector space. Indeed,  $T_x\mathcal{M} = \ker(d\pi_x) \oplus \ker^\perp(d\pi_x)$ , with respect to the inner product defined by  $g$ . This means that with an appropriate choice of local coordinates such as  $(x^1, x^2, \ldots, x^{\dim(\mathcal{M})})$  around a point  $x \in \mathcal{M}$ , satisfying
$$
g _ {i j} = g \left(\frac {\partial}{\partial_ {i}}, \frac {\partial}{\partial_ {j}}\right) = \mathbb {1} \{i = j \}, \tag {72}
$$
for any  $i,j\in \{1,2,\ldots ,\dim (\mathcal{M})\}$ , we have  $\frac{\partial}{\partial_i}\in \ker^\perp (d\pi_x)$  for any  $i = 1,2,\dots ,\dim (\mathcal{M})-$ $\dim (G)$ , and  $\frac{\partial}{\partial_i}\in \ker (d\pi_x)$  for any  $i > \dim (\mathcal{M}) - \dim (G)$ . In particular, the restriction of the local coordinates to the first  $\dim (\mathcal{M}) - \dim (G)$  elements is assumed to be some local coordinates for  $\mathcal{M} / G$ . This is always possible for an appropriate choice of local coordinates.
In these specific local coordinates, by definition,
$$
\Delta_ {g} \phi = \sum_ {i = 1} ^ {\dim (\mathcal {M})} \partial_ {i} ^ {2} \phi \tag {73}
$$
$$
\Delta_ {\tilde {g}} \tilde {\phi} = \sum_ {i = 1} ^ {\dim (\mathcal {M}) - \dim (G)} \partial_ {i} ^ {2} \tilde {\phi}. \tag {74}
$$
Note that  $\partial_i^2\tilde{\phi} = \partial_i^2\phi$  for  $i = 1,2,\ldots ,\dim (\mathcal{M}) - \dim (G)$ . Thus, the proof is complete if we show  $\partial_i\phi \equiv 0$ , for all  $i > \dim (\mathcal{M}) - \dim (G)$ , for a neighborhood around  $x$  in the local coordinates  $(x^{\dim (\mathcal{M}) - \dim (G) + 1},\dots,x^{\dim (\mathcal{M})})$ , while the other coordinates are kept the same as  $x$ . But note that for any  $x^{\prime}$  sufficiently close to  $x$  with the same coordinates  $(x^{1},x^{2},\ldots ,x^{\dim (\mathcal{M}) - \dim (G)})$ , one has  $[x^{\prime}] = [x]$ , by definition. This means that  $\phi (x) = \phi (x^{\prime})$  and this completes the proof.
To extend Proposition B.1 to a general smooth compact Lie group action  $G$ , we need to use the principal orbit theorem (Theorem A.4) and its consequences (see Appendix A.6). Again, we prove that the generalized local result (Equation (69)) holds, presuming the assumptions in Theorem 4.4.
Proof of Theorem 4.4. According to Corollary A.5, one has the following decomposition of the quotient space:  $\mathcal{M} / G = \bigsqcup_{[H] \preceq G} \mathcal{M}_{[H]} / G$ . In other words, the quotient space is the disjoint union of finitely many manifolds, and among them,  $\mathcal{M}_0 / G$  is open and dense in  $\mathcal{M} / G$ . As a first step towards the proof, we show that  $\Delta_g \tilde{\phi} = \Delta_g \phi$  for any two smooth functions  $\phi : \mathcal{M}_0 \to \mathbb{R}$  and  $\tilde{\phi} : \mathcal{M}_0 / G \to \mathbb{R}$  such that  $\phi(x) = \tilde{\phi}([x])$ , for any  $x \in \mathcal{M}_0$  and  $[x] \in \mathcal{M}_0 / G$ . However, the proof of this claim is exactly the same as what is presented in the proof of Proposition B.1; thus, we skip it.
Recall that the effective dimension of the quotient space is defined as  $d \coloneqq \dim(\mathcal{M}_{[H_0]} / G) = \dim(\mathcal{M}_0 / G) = \dim(\mathcal{M}) - \dim(G) + \dim(H_0)$ , where  $[H_0]$  is the unique maximal isotropy type (corresponding to  $\mathcal{M}_0$ ). We claim that there exists a connected compact manifold (possibly with boundary)  $\widetilde{\mathcal{M}} \subseteq \mathcal{M} / G$  such that (1) it includes the principal part, i.e.,  $\widetilde{\mathcal{M}} \supseteq \mathcal{M}_0 / G$ , and (2) the projected invariant functions on  $\widetilde{\mathcal{M}}$  satisfy the Neumann boundary condition on its boundary  $\partial(\widetilde{\mathcal{M}})$ .