SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Amir Zandieh
Independent Researcher amir@zed512@gmail.com
Insu Han
Yale University insu.han@yale.edu
Haim Avron
Tel Aviv University haimav@tauex.tau.ac.il
We propose an algorithm for robust recovery of the spherical harmonic expansion of functions defined on the  $d$ -dimensional unit sphere  $\mathbb{S}^{d-1}$  using a near-optimal number of function evaluations. We show that for any  $f \in L^2(\mathbb{S}^{d-1})$ , the number of evaluations of  $f$  needed to recover its degree- $q$  spherical harmonic expansion equals the dimension of the space of spherical harmonics of degree at most  $q$ , up to a logarithmic factor. Moreover, we develop a simple yet efficient kernel regression-based algorithm to recover degree- $q$  expansion of  $f$  by only evaluating the function on uniformly sampled points on  $\mathbb{S}^{d-1}$ . Our algorithm is built upon the connections between spherical harmonics and Gegenbauer polynomials. Unlike the prior results on fast spherical harmonic transform, our proposed algorithm works efficiently using a nearly optimal number of samples in any dimension  $d$ . Furthermore, we illustrate the empirical performance of our algorithm on numerical examples.
We consider the fundamental problem of recovering a function from a finite number of (noisy) observations. To provide accurate and reliable predictions at unobserved points we need to avoid overfitting which is typically achieved through restricting our estimator or interpolant to a family of smooth or structured functions. In this paper, we focus on interpolating square-integrable functions on the  $d$ -dimensional unit sphere, with low-degree spherical harmonics, a critical task in scenarios where rotational invariance is a fundamental property. Spherical harmonics are essential in various theoretical and practical applications, including the representation of electromagnetic fields [Wei95], gravitational potential [Wer97], cosmic microwave background radiation [KKS97] and medical imaging [CLL15], as well as modelling of 3D shapes in computer graphics [KFR03] and computer vision  $\mathrm{[GCL^{+}22]}$ . Further notable real-life applications include molecular/atom systems, where understanding the underlying functions within a spherical context can significantly enhance predictive modeling and simulation accuracy [EEHM17, LWL  $^+$  21, ZDK  $^+$  22].
We begin by observing that any function  $f$  in  $L^2(\mathbb{S}^{d-1})$ , i.e., the family of square-integrable functions on the sphere  $\mathbb{S}^{d-1}$ , can be uniquely decomposed into orthogonal spherical harmonic components. Specifically, if we denote the space of spherical harmonics of degree  $\ell$  in dimension  $d$  by  $\mathcal{H}_{\ell}(\mathbb{S}^{d-1})$ , any  $f \in L^2(\mathbb{S}^{d-1})$  has a unique orthogonal expansion  $f = \sum_{\ell=0}^{\infty} f_{\ell}$  with  $f_{\ell} \in \mathcal{H}_{\ell}(\mathbb{S}^{d-1})$  (Lemma 2). With this observation, we aim to find the best spherical harmonic approximation of degree  $\leq q$  to  $f$  using minimal number of samples (essentially treating higher order terms in  $f$ 's expansion as noise).
Problem 1 (Informal Version of Problem 2). For an unknown function  $f \in L^{2}(\mathbb{S}^{d - 1})$  and an integer  $q \geq 1$ , efficiently (both in terms of number of samples from  $f$  and computations) learn the first  $q + 1$  spherical harmonic components  $\left\{f_{\ell} \in \mathcal{H}_{\ell}(\mathbb{S}^{d - 1})\right\}_{\ell = 0}^{q}$  of  $f$  which minimize
$$
\left\| \sum_ {\ell = 0} ^ {q} f _ {\ell} - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} := \int_ {\mathbb {S} ^ {d - 1}} \left| \sum_ {\ell = 0} ^ {q} f _ {\ell} (w) - f (w) \right| ^ {2} d w. \tag {1}
$$
The angular power spectrum of  $f$  commonly obeys a power law decay of the form  $\| f_{\ell}\|_{\mathbb{S}^{d - 1}}^{2}\leq \mathcal{O}(\ell^{-s})$ , for some  $s > 0$ , depending on the order of differentiability of  $f$ . In fact, for any infinitely differentiable  $f$ ,  $\| f_{\ell}\|_{\mathbb{S}^{d - 1}}^{2}$  decays asymptotically faster than any rational function of  $\ell$ . Furthermore, for any real analytic  $f$  on the sphere,  $\| f_{\ell}\|_{\mathbb{S}^{d - 1}}^{2}$  decays exponentially. Thus, the first  $q + 1$  spherical harmonic components of  $f$  typically well approximate  $f$  for even modest  $q$ , and answering Problem 1 is meaningful for a wide range of differentiable functions.
1.1 Our Main Results
We reformulate Problem 1 as a least-squares regression and then solve it using randomized numerical linear algebra techniques. We first consider an orthonormal projection operator that maps functions in  $L^2(\mathbb{S}^{d-1})$  onto the space of bounded-degree spherical harmonics  $\bigoplus_{\ell=0}^{q} \mathcal{H}_{\ell}(\mathbb{S}^{d-1})$ . Specifically, if  $\mathcal{K}_d^{(q)}$  is an operator that maps any  $f$  with expansion  $f = \sum_{\ell=0}^{\infty} f_{\ell}$  where  $f_{\ell} \in \mathcal{H}_{\ell}(\mathbb{S}^{d-1})$ , onto  $f$ 's first  $q+1$  expansion components, i.e.,  $\mathcal{K}_d^{(q)}f = \sum_{\ell=0}^{q} f_{\ell}$ , then Problem 1 can be formulated as
$$
\min  _ {g \in L ^ {2} (\mathbb {S} ^ {d - 1})} \left\| \mathcal {K} _ {d} ^ {(q)} g - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}.
$$
However, solving this regression problem with "continuous" cost function is challenging. To avoid such continuous optimizations, we adopt the approach of  $\mathrm{[AKM^{+}19]}$  which discretizes the aforementioned regression problem according to the leverage scores of the operator  $\mathcal{K}_d^{(q)}$ . It turns out that if we can draw random samples with probabilities proportional to the leverage scores of  $\mathcal{K}_d^{(q)}$  then we can recover the degree-  $q$  spherical harmonic expansion of  $f$ , i.e.  $\sum_{\ell = 0}^{q}f_{\ell}$ , with finite number of observations. Particularly, by exploiting the connections between spherical harmonics and Zonal (Gegenbauer) Harmonics and the fact that zonal harmonics are the reproducing kernels of  $\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$  (Lemma 3), we prove that the leverage scores of  $\mathcal{K}_d^{(q)}$  are constant everywhere on the sphere  $\mathbb{S}^{d - 1}$ . Thus, solving a discrete regression problem with uniformly sampled observations yields a near-optimal solution to Problem 1. Informal statements of our results are as follows.
Theorem 1 (Informal Version of Theorem 5). Let  $\beta_{q,d}$  be the dimension of spherical harmonics of degree at most  $q$ , i.e.,  $\beta_{q,d} \equiv \dim \left(\bigoplus_{\ell=0}^{q} \mathcal{H}_{\ell}(\mathbb{S}^{d-1})\right)$ . There exists an algorithm that finds a  $(1+\varepsilon)$ -approximation to the optimal solution of Problem 1, given  $s = \mathcal{O}(\beta_{q,d} \log \beta_{q,d} + \varepsilon^{-1} \beta_{q,d})$  observations of function  $f$  at uniformly sampled points on  $\mathbb{S}^{d-1}$ , with  $\mathcal{O}(s^2 d + s^\omega)^1$  runtime.
We also prove that our bound on the number of required samples is optimal up to a logarithmic factor.
Theorem 2 (Informal Version of Theorem 6). Any (randomized) algorithm that takes  $s < \beta_{q,d}$  samples on any input fails with probability greater than  $9/10$ , where  $\beta_{q,d} \equiv \dim \left( \bigoplus_{\ell=0}^{q} \mathcal{H}_{\ell}(\mathbb{S}^{d-1}) \right)$ .
Efficient reconstruction of functions as per Problem 1 has been extensively studied in various fields. Many prior papers considered reconstructing 1-dimensional functions from finite number of samples on a finite interval under smoothness assumption about the underlying function. Notably, the influential line of work of [SP61, LP61, LP62, XRY01] focused on reconstructing Fourier-bandlimited functions and [CKPS16, KVZ19, EMM20,  $\mathrm{BKM}^{+}23$  ] considered interpolating Fourier-sparse signals. Recently,  $[\mathrm{AKM}^{+}19]$  unified these reconstruction methods in dimension  $d = 1$  and gave a universal sampling framework for reconstructing nearly all classes of functions with Fourier-based smoothness constraints. One can view 1-dimensional functions on a finite interval as functions on the unit circle
$^{1}\omega < 2.3727$  is the exponent of the fast matrix multiplication algorithm [Wil12]
$\mathbb{S}^1$ . Thus, Problem 1 is indeed a generalization of the aforementioned prior work to high dimensions under the assumption that the generalized Fourier series (Lemma 2) of the underlying function only contains bounded-degree spherical harmonics. This degree constraint on spherical harmonic expansions can be viewed as the  $d$ -dimensional analog of Fourier-bandlimitedness on circle  $\mathbb{S}^1$ .
Computing the spherical harmonic expansion in dimension  $d = 3$  has received considerable attention in physics and applied mathematics communities. The algorithms for this special case of Problem 1 are known in the literature as "fast spherical harmonic transform" [SS00, ST02]. Most notably, [RT06] proposed an algorithm for computing spherical harmonic expansion of degree  $\leq q$  to precision  $\varepsilon$  using  $\mathcal{O}(\beta_{q,3})$  samples and  $\mathcal{O}(\beta_{q,3}\log \beta_{q,3}\cdot \log (1 / \varepsilon))$  time. These fast algorithms were developed based on fast Fourier and associated Legendre transforms and make use of a (well-conditioned) orthogonal basis for  $\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$ , which happened to be the associated Legendre polynomials when  $d = 3$ . However, it is in general very difficult to compute an orthogonal basis for spherical harmonics [MNY06], so unlike our Theorem 1, it is inefficient to extend these prior results to higher  $d$ .
From the techniques point of view, a related work is [GMMM21], which employs harmonic analysis over  $\mathbb{S}^{d - 1}$  to analyze the generalization of two-layered neural tangent kernels. They show that an unknown function defined on  $\mathbb{S}^{d - 1}$  can be efficiently recovered using kernel regression w.r.t. neural tangent kernel on uniform random samples from the function. However, the number of samples that [GMMM21] requires for recovering bounded degree spherical harmonics, especially when the degrees are high, is sub-optimal and is strictly worse than our result. Additionally, [GMMM21] does not guarantee recovery with relative error, while our Theorem 5 provides relative error guarantees.
Furthermore, recent applications of Gegenbauer polynomials, along with other orthonormal polynomials like Hermite polynomials, have been found in designing efficient random features for approximating various kernel functions. These applications extend to dot-product kernels [HZA22], Neural Tangent Kernels  $\left[\mathrm{ZHA}^{+}21\right.$ ,  $\left.\mathrm{HZL}^{+}22\right]$ , and Gaussian kernels [WZ22,  $\mathrm{AKK}^{+}20$ ].
2 Mathematical Preliminaries
We denote by  $\mathbb{S}^{d - 1}$  the unit sphere in  $d$  dimension. We use  $|\mathbb{S}^{d - 1}| = \frac{2\pi^{d / 2}}{\Gamma(d / 2)}$  to denote the surface area of sphere  $\mathbb{S}^{d - 1}$  and  $\mathcal{U}(\mathbb{S}^{d - 1})$  to denote the uniform probability distribution on  $\mathbb{S}^{d - 1}$ . We denote by  $L^2\left(\mathbb{S}^{d - 1}\right)$  the set of all square-integrable real-valued functions on sphere  $\mathbb{S}^{d - 1}$ . Furthermore, for any  $f,g\in L^{2}\left(\mathbb{S}^{d - 1}\right)$  we use the following definition of the inner product on the unit sphere<sup>2</sup>,
$$
\langle f, g \rangle_ {\mathbb {S} ^ {d - 1}} := \int_ {\mathbb {S} ^ {d - 1}} f (w) g (w) d w = \left| \mathbb {S} ^ {d - 1} \right| \cdot \underset {w \sim \mathcal {U} \left(\mathbb {S} ^ {d - 1}\right)} {\mathbb {E}} [ f (w) g (w) ]. \tag {2}
$$
The function space  $L^2(\mathbb{S}^{d-1})$  is complete with respect to the norm induced by the above inner product, i.e.  $\| f \|_{\mathbb{S}^{d-1}} \coloneqq \sqrt{\langle f, f \rangle_{\mathbb{S}^{d-1}}}$ , so  $L^2(\mathbb{S}^{d-1})$  is a Hilbert space.
We often use the term quasi-matrix which is informally defined as a "matrix" in which one dimension is finite while the other is infinite. A quasi-matrix can be tall (or wide) meaning that there is a finite number of columns (or rows) where each one is a functional operator. For a more formal definition, see [SA22].
Spherical Harmonics are the solutions of Laplace's equation in spherical domains and can be thought of as functions defined on  $\mathbb{S}^{d - 1}$  employed in solving partial differential equations. Formally,
Definition 1 (Spherical Harmonics). For integers  $\ell \geq 0$  and  $d \geq 1$ , let  $\mathcal{P}_{\ell}(d)$  be the space of degree- $\ell$  homogeneous polynomials with  $d$  variables and real coefficients. Let  $\mathcal{H}_{\ell}(d)$  denote the space of degree- $\ell$  harmonic polynomials in dimension  $d$ , i.e., homogeneous polynomial solutions of Laplace's equation:
$$
\mathcal {H} _ {\ell} (d) := \left\{P \in \mathcal {P} _ {\ell} (d): \Delta P = 0 \right\},
$$
where  $\Delta = \frac{\partial^2}{\partial x_1^2} + \dots + \frac{\partial^2}{\partial x_d^2}$  is the Laplace operator on  $\mathbb{R}^d$ . Finally, let  $\mathcal{H}_{\ell}(\mathbb{S}^{d-1})$  be the space of (real) Spherical Harmonics of order  $\ell$  in dimension  $d$ , i.e. restrictions of harmonic polynomials in
Formally,  $L^2 \left( \mathbb{S}^{d-1} \right)$  is a space of equivalence classes of functions that differ at a set of points with measure 0. For notational simplicity, here and throughout we use  $f$  to denote the specific representative of the equivalence class  $f \in L^2 \left( \mathbb{S}^{d-1} \right)$ . In this way, we can consider the point-wise value  $f(w)$  for every  $w \in \mathbb{S}^{d-1}$ .
$\mathcal{H}_{\ell}(d)$  to the sphere  $\mathbb{S}^{d - 1}$ . The dimension of this space,  $\alpha_{\ell ,d}\equiv \dim \left(\mathcal{H}_{\ell}\left(\mathbb{S}^{d - 1}\right)\right)$ , is
$$
\alpha_ {0, d} = 1, \quad \alpha_ {1, d} = d, \quad \alpha_ {\ell , d} = \left( \begin{array}{c} d + \ell - 1 \\ \ell \end{array} \right) - \left( \begin{array}{c} d + \ell - 3 \\ \ell - 2 \end{array} \right) \text {f o r} \ell \geq 2.
$$
2.1 Gegenbauer Polynomials
The Gegenbauer (a.k.a. ultraspherical) polynomial of degree  $\ell \geq 0$  in dimension  $d\geq 2$  is given by
$$
P _ {d} ^ {\ell} (t) := \sum_ {j = 0} ^ {\lfloor \ell / 2 \rfloor} c _ {j} \cdot t ^ {\ell - 2 j} \cdot (1 - t ^ {2}) ^ {j}, \tag {3}
$$
where  $c_0 = 1$  and  $c_{j + 1} = -\frac{(\ell - 2j)(\ell - 2j - 1)}{2(j + 1)(d - 1 + 2j)} \cdot c_j$  for  $j = 0,1,\ldots ,\lfloor \ell /2\rfloor -1$ . These polynomials are orthogonal on the interval  $[-1,1]$  with respect to the measure  $(1 - t^{2})^{\frac{d - 3}{2}}$ , i.e.,
$$
\int_ {- 1} ^ {1} P _ {d} ^ {\ell} (t) \cdot P _ {d} ^ {\ell^ {\prime}} (t) \cdot \left(1 - t ^ {2}\right) ^ {\frac {d - 3}{2}} d t = \frac {\left| \mathbb {S} ^ {d - 1} \right|}{\alpha_ {\ell , d} \cdot | \mathbb {S} ^ {d - 2} |} \cdot \mathbb {1} _ {\{\ell = \ell^ {\prime} \}}. \tag {4}
$$
Zonal Harmonics. The Gegenbauer polynomials naturally provide positive definite dot-product kernels on  $\mathbb{S}^{d - 1}$  known as Zonal Harmonics, which are closely related to the spherical harmonics. The following reproducing property of zonal harmonics plays a crucial role in our analysis.
Lemma 1 (Reproducing Property of Zonal Harmonics). Let  $P_d^\ell(\cdot)$  be the Gengenbauer polynomial of degree  $\ell$  in dimension  $d$ . For any  $x, y \in \mathbb{S}^{d-1}$ :
$$
P _ {d} ^ {\ell} (\langle x, y \rangle) = \alpha_ {\ell , d} \cdot \mathbb {E} _ {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} \left[ P _ {d} ^ {\ell} (\langle x, w \rangle) P _ {d} ^ {\ell} (\langle y, w \rangle) \right],
$$
Furthermore, for any  $\ell' \neq \ell$ :
$$
\mathbb {E} _ {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} \left[ P _ {d} ^ {\ell} \left(\langle x, w \rangle\right) \cdot P _ {d} ^ {\ell^ {\prime}} \left(\langle y, w \rangle\right) \right] = 0.
$$
The proof of this and all subsequent results can be found in the appendix. The following useful fact, known as the addition theorem, connects Gegenbauer polynomials and spherical harmonics.
Theorem 3 (Addition Theorem). For every integer  $\ell \geq 0$ , if  $\left\{y_1^\ell, y_2^\ell, \ldots, y_{\alpha_{\ell,d}}^\ell\right\}$  is an orthonormal basis for  $\mathcal{H}_{\ell}\left(\mathbb{S}^{d-1}\right)$ , then for any  $\sigma, w \in \mathbb{S}^{d-1}$  we have
$$
\frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |} \cdot P _ {d} ^ {\ell} (\langle \sigma , w \rangle) = \sum_ {j = 1} ^ {\alpha_ {\ell , d}} y _ {j} ^ {\ell} (\sigma) \cdot y _ {j} ^ {\ell} (w).
$$
3 Reconstructing  $L^2(\mathbb{S}^{d-1})$  Functions via Spherical Harmonics
In this section we show how to approximate any function  $f \in L^{2}(\mathbb{S}^{d - 1})$  by spherical harmonics using the optimal number of samples. We begin with the fact that spherical harmonics form a complete set of orthonormal functions and thus form an orthonormal basis for the Hilbert space of square-integrable functions on sphere  $\mathbb{S}^{d - 1}$ . This is analogous to periodic functions, viewed as functions defined on the circle  $\mathbb{S}^1$ , which can be expressed as a linear combination of circular functions (sines and cosines) via the Fourier series.
Lemma 2 (Direct Sum Decomposition of  $L^2(\mathbb{S}^{d-1})$ ). The family of spaces  $\mathcal{H}_{\ell}(\mathbb{S}^{d-1})$  yields a Hilbert space direct sum decomposition  $L^2(\mathbb{S}^{d-1}) = \bigoplus_{\ell=0}^{\infty} \mathcal{H}_{\ell}(\mathbb{S}^{d-1})$ : the summands are closed and pairwise orthogonal, and every  $f \in L^2(\mathbb{S}^{d-1})$  is the sum of a converging series (in the sense of mean-square convergence with the  $L^2$ -norm defined in Eq. (2)),
$$
f = \sum_ {\ell = 0} ^ {\infty} f _ {\ell},
$$
where  $f_{\ell} \in \mathcal{H}_{\ell}\left(\mathbb{S}^{d-1}\right)$  are uniquely determined functions. Furthermore, given any orthonormal basis  $\left\{y_1^\ell, y_2^\ell, \ldots, y_{\alpha_{\ell,d}}^\ell\right\}$  of  $\mathcal{H}_{\ell}\left(\mathbb{S}^{d-1}\right)$  we have  $f_{\ell} = \sum_{j=1}^{\alpha_{\ell,d}} \langle f, y_j^\ell \rangle_{\mathbb{S}^{d-1}} \cdot y_j^\ell$ .
The series expansion in Lemma 2 is the analog of the Fourier expansion of periodic functions, and is known as "generalized Fourier series" [Pen30] with respect to the Hilbert basis  $\{y_j^\ell : j \in [\alpha_{\ell,d}], \ell \geq 0\}$ . We remark that it is in general intractable to compute an orthogonal basis for the space of spherical harmonics [MNY06], which renders the generalized Fourier series expansion in Lemma 2 primarily existential. While finding the generalized Fourier expansion of a function  $f \in L^2(\mathbb{S}^{d-1})$  is computationally intractable, our goal is to answer the next fundamental question, which is about finding the projection of a function  $f$  onto the space of spherical harmonics, i.e., the  $f_\ell$ 's in Lemma 2. Concretely, we seek to solve the following problem.
Problem 2. For an integer  $q \geq 0$  and a given function  $f \in L^{2}(\mathbb{S}^{d - 1})$  whose decomposition over the Hilbert sum  $\bigoplus_{\ell = 0}^{\infty}\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$  is  $f = \sum_{\ell = 0}^{\infty}f_{\ell}$  as per Lemma 2, let us define the low-degree expansion of this function as  $f^{(q)} := \sum_{\ell = 0}^{q}f_{\ell}$ . How efficiently can we learn  $f^{(q)} \in \bigoplus_{\ell = 0}^{q}\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$ ? More precisely, we want to find a set  $\{w_{1},w_{2},\ldots ,w_{s}\} \subseteq \mathbb{S}^{d - 1}$  with minimal cardinality  $s$  along with an efficient algorithm that given samples  $\{f(w_i)\}_{i = 1}^s$  can interpolate  $f(\cdot)$  with a function  $\tilde{f}^{(q)} \in \bigoplus_{\ell = 0}^{q}\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$  such that:
$$
\left\| \tilde {f} ^ {(q)} - f ^ {(q)} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \leq \varepsilon \cdot \left\| f ^ {(q)} - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}.
$$
For ease of notation, we denote the Hilbert space of spherical harmonics of degree at most  $q$  by  $\mathcal{H}^{(q)}(\mathbb{S}^{d - 1}) \coloneqq \bigoplus_{\ell = 0}^{q}\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$ . To answer Problem 2 we exploit the close connection between the spherical harmonics and Gengenbauer polynomials, and in particular the fact that zonal harmonics are the reproducing kernels of the Hilbert spaces  $\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$ .
Lemma 3 (A Reproducing Kernel for  $\mathcal{H}_{\ell}\left(\mathbb{S}^{d - 1}\right)$ ). For every  $f\in L^{2}\left(\mathbb{S}^{d - 1}\right)$ , if  $f = \sum_{\ell = 0}^{\infty}f_{\ell}$  is the unique decomposition of  $f$  over  $\bigoplus_{\ell = 0}^{\infty}\mathcal{H}_{\ell}\left(\mathbb{S}^{d - 1}\right)$  as per Lemma 2, then  $f_{\ell}$  is given by
$$
f _ {\ell} (\sigma) = \alpha_ {\ell , d} \cdot \underset {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ f (w) P _ {d} ^ {\ell} \left(\langle \sigma , w \rangle\right) \right] \quad f o r   \sigma \in \mathbb {S} ^ {d - 1}.
$$
Now we define a kernel operator, based on the low-degree Gegenbauer polynomials, which projects functions onto their low-degree spherical harmonic expansion.
Definition 2 (Projection Operator onto  $\mathcal{H}^{(q)}(\mathbb{S}^{d - 1})$ ). For any integers  $q\geq 0$  and  $d\geq 2$ , define the kernel operator  $\mathcal{K}_d^{(q)}:L^2\left(\mathbb{S}^{d - 1}\right)\to L^2\left(\mathbb{S}^{d - 1}\right)$  as follows: for  $f\in L^{2}\left(\mathbb{S}^{d - 1}\right)$  and  $\sigma \in \mathbb{S}^{d - 1}$ ,
$$
\left[ \mathcal {K} _ {d} ^ {(q)} f \right] (\sigma) := \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |} \left\langle f, P _ {d} ^ {\ell} (\langle \sigma , \cdot \rangle) \right\rangle_ {\mathbb {S} ^ {d - 1}} = \sum_ {\ell = 0} ^ {q} \alpha_ {\ell , d} \cdot \underset {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ f (w) P _ {d} ^ {\ell} (\langle \sigma , w \rangle) \right]. \tag {5}
$$
This is an integral operator with kernel function  $k_{q,d}(\sigma ,w)\coloneqq \sum_{\ell = 0}^{q}\frac{\alpha_{\ell,d}}{|\mathbb{S}^{d - 1}|}\cdot P_d^\ell (\langle \sigma ,w\rangle)$ .
Note that the operator  $\mathcal{K}_d^{(q)}$  is self-adjoint and positive semi-definite. Moreover, using the reproducing property of this kernel we can establish that  $\mathcal{K}_d^{(q)}$  is a projection operator.
Claim 1. The operator  $\mathcal{K}_d^{(q)}$  defined in Definition 2 satisfies the property  $\left(\mathcal{K}_d^{(q)}\right)^2 = \mathcal{K}_d^{(q)}$ .
Furthermore, by the addition theorem (Theorem 3),  $\mathcal{K}_d^{(q)}$  is trace-class (i.e., the trace is finite and independent of the choice of basis) because:
$$
\begin{array}{l} \operatorname {t r a c e} \left(\mathcal {K} _ {d} ^ {(q)}\right) = \int_ {\mathbb {S} ^ {d - 1}} k _ {q, d} (w, w) d w = \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |} \cdot \int_ {\mathbb {S} ^ {d - 1}} P _ {d} ^ {\ell} (\langle w, w \rangle) d w \\ = \sum_ {\ell = 0} ^ {q} \alpha_ {\ell , d} = \binom {d + q - 1} {q} + \binom {d + q - 2} {q - 1} - 1. \tag {6} \\ \end{array}
$$
By combining Theorem 3 and Lemma 2, and using the definition of the projection operator  $\mathcal{K}_d^{(q)}$ , it follows that for any function  $f \in L^2(\mathbb{S}^{d-1})$  with Hilbert sum decomposition  $f = \sum_{\ell=0}^{\infty} f_\ell$ , the low-degree component  $f^{(q)} = \sum_{\ell=0}^q f_\ell \in \mathcal{H}^{(q)}(\mathbb{S}^{d-1})$  can be computed as  $f^{(q)} = \mathcal{K}_d^{(q)}f$ . Equivalently,
in order to learn  $f^{(q)}$ , it suffices to solve the following least-squares regression problem,
$$
\min  _ {g \in L ^ {2} \left(\mathbb {S} ^ {d - 1}\right)} \left\| \mathcal {K} _ {d} ^ {(q)} g - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}. \tag {7}
$$
If  $g^{*}$  is an optimal solution to the above regression problem then  $f^{(q)} = \mathcal{K}_d^{(q)}g^*$ . In the next claim we show that solving the least squares problem in Eq. (7), even to a coarse approximation, is sufficient to solve our interpolation problem (i.e., Problem 2):
Claim 2. For any  $f \in L^2(\mathbb{S}^{d-1})$ , any integer  $q \geq 0$ , and any  $C \geq 1$ , if  $\tilde{g} \in L^2(\mathbb{S}^{d-1})$  satisfies,