SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section we prove the basic properties of the Gegenbauer Polynomials as well as the Spherical Harmonics and establish the connection between the two. We start by the direct sum decomposition of the Hilbert space  $L^2 (\mathbb{S}^{d - 1})$  in terms of the spherical harmonics,
Lemma 2 (Direct Sum Decomposition of  $L^2(\mathbb{S}^{d-1})$ ). The family of spaces  $\mathcal{H}_{\ell}\left(\mathbb{S}^{d-1}\right)$  yields a Hilbert space direct sum decomposition  $L^2\left(\mathbb{S}^{d-1}\right) = \bigoplus_{\ell=0}^{\infty} \mathcal{H}_{\ell}\left(\mathbb{S}^{d-1}\right)$ : the summands are closed and pairwise orthogonal, and every  $f \in L^2\left(\mathbb{S}^{d-1}\right)$  is the sum of a converging series (in the sense of mean-square convergence with the  $L^2$ -norm defined in Eq. (2)),
where  $f_{\ell} \in \mathcal{H}_{\ell}\left(\mathbb{S}^{d - 1}\right)$  are uniquely determined functions. Furthermore, given any orthonormal basis  $\left\{y_1^\ell ,y_2^\ell ,\ldots ,y_{\alpha_{\ell ,d}}^\ell \right\}$  of  $\mathcal{H}_{\ell}\left(\mathbb{S}^{d - 1}\right)$  we have  $f_{\ell} = \sum_{j = 1}^{\alpha_{\ell ,d}}\langle f,y_j^\ell \rangle_{\mathbb{S}^{d - 1}}\cdot y_j^\ell$ .
Proof. This is in fact a standard result. For example, see [Lan12] for a proof.
Now we show that the Gegenbauer polynomials and spherical harmonics are related through the so-called the addition theorem,
Proof. The result can be proven analytically, using the properties of the Poisson kernel in the unit ball. This is classic and the proof can be found in [AH12, Theorem 2.9].
Next we show that the Gegenbauer kernels can project any function into the space of their corresponding spherical harmonics,
Proof. This is a classic textbook result, see [Mor98].
Now we prove that the Gegenbauer kernels satisfy the reproducing property for the Hilbert space  $\mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$ .
Proof. This result follows directly from the Funk-Hecke formula (See [AH12]). However, we provide another proof here. First note that for every  $x \in \mathbb{S}^{d - 1}$  the function  $P_{d}^{\ell}(\langle x,\cdot \rangle) \in \mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$ . Therefore the first claim follows by applying Lemma 3 on function  $f(\sigma) = P_{d}^{\ell}(\langle x,\sigma \rangle)$  which also satisfies  $f_{\ell} = f$ . On the other hand,  $P_{d}^{\ell^{\prime}}(\langle y,\cdot \rangle) \in \mathcal{H}_{\ell^{\prime}}(\mathbb{S}^{d - 1})$  for every  $y \in \mathbb{S}^{d - 1}$ . Thus, for  $\ell' \neq \ell$ , using the fact that spherical harmonics are orthogonal spaces of functions,  $P_{d}^{\ell^{\prime}}(\langle y,\cdot \rangle) \perp \mathcal{H}_{\ell}(\mathbb{S}^{d - 1})$ , which gives the second claim.
Next we prove that the kernel operator defined in Definition 2 is in fact a projection operator,
Proof. For every  $f \in L^2(\mathbb{S}^{d-1})$  and every  $\sigma \in \mathbb{S}^{d-1}$ , using Definition 2 we have,
$$
\begin{array}{l} \left[ \left(\mathcal {K} _ {d} ^ {(q)}\right) ^ {2} f \right] (\sigma) = \sum_ {\ell^ {\prime} = 0} ^ {q} \frac {\alpha_ {\ell^ {\prime} , d}}{| \mathbb {S} ^ {d - 1} |} \left\langle \mathcal {K} _ {d} ^ {(q)} f, P _ {d} ^ {\ell^ {\prime}} (\langle \sigma , \cdot \rangle) \right\rangle_ {\mathbb {S} ^ {d - 1}} \\ = \sum_ {\ell = 0} ^ {q} \sum_ {\ell^ {\prime} = 0} ^ {q} \alpha_ {\ell , d} \alpha_ {\ell^ {\prime}, d} \cdot \underset {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ P _ {d} ^ {\ell^ {\prime}} (\langle \sigma , w \rangle) \cdot \underset {\tau \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ P _ {d} ^ {\ell} (\langle \tau , w \rangle) f (\tau) \right] \right] \\ = \sum_ {\ell = 0} ^ {q} \sum_ {\ell^ {\prime} = 0} ^ {q} \alpha_ {\ell , d} \alpha_ {\ell^ {\prime}, d} \cdot \underset {\tau \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ f (\tau) \cdot \underset {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ P _ {d} ^ {\ell^ {\prime}} (\langle \sigma , w \rangle) P _ {d} ^ {\ell} (\langle \tau , w \rangle) \right] \right] \\ = \sum_ {\ell = 0} ^ {q} \alpha_ {\ell , d} \cdot \underset {\tau \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ f (\tau) \cdot P _ {d} ^ {\ell} \left(\langle \sigma , \tau \rangle\right) \right] \\ = \left[ \mathcal {K} _ {d} ^ {(q)} f \right] (\sigma), \\ \end{array}
$$
where the fourth line above follows from Lemma 1. This proves the claim.
B Reducing the Interpolation Problem to a Least-Squares Regression
In this section we show that our spherical harmonic interpolation problem, i.e., Problem 2, can be solved by approximately solving a least-squares problem as claimed in Claim 2. We start by showing that for any function  $f \in L^{2}(\mathbb{S}^{d - 1})$ ,  $\mathcal{K}_d^{(q)}f$  gives its low-degree component. More precisely, let  $f = \sum_{\ell = 0}^{\infty}f_{\ell}$  be the decomposition of  $f$  over the Hilbert sum  $\bigoplus_{\ell = 0}^{\infty}\mathcal{H}_{\ell}\left(\mathbb{S}^{d - 1}\right)$  as per Lemma 2. Now if we let  $\mathcal{K}_d^{(q)}$  be the kernel operator from Definition 2 and if  $\left\{y_1^\ell ,y_2^\ell ,\ldots ,y_{\alpha_{\ell ,d}}^\ell \right\}$  is an orthonormal basis for  $\mathcal{H}_{\ell}\left(\mathbb{S}^{d - 1}\right)$ , then by Theorem 3 we have,
$$
\begin{array}{l} \left[ \mathcal {K} _ {d} ^ {(q)} f \right] (\sigma) = \sum_ {\ell = 0} ^ {q} \alpha_ {\ell , d} \cdot \underset {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ f (w) P _ {d} ^ {\ell} \left(\langle \sigma , w \rangle\right) \right] \\ = \sum_ {\ell = 0} ^ {q} \left| \mathbb {S} ^ {d - 1} \right| \cdot \underset {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ f (w) \cdot \sum_ {j = 1} ^ {\alpha_ {\ell , d}} y _ {j} ^ {\ell} (\sigma) \cdot y _ {j} ^ {\ell} (w) \right] \\ = \sum_ {\ell = 0} ^ {q} \sum_ {j = 1} ^ {\alpha_ {\ell , d}} y _ {j} ^ {\ell} (\sigma) \cdot \left| \mathbb {S} ^ {d - 1} \right| \cdot \underset {w \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} [ f (w) \cdot y _ {j} ^ {\ell} (w) ] \\ = \sum_ {\ell = 0} ^ {q} \sum_ {j = 1} ^ {\alpha_ {\ell , d}} \langle f, y _ {j} ^ {\ell} (w) \rangle_ {\mathbb {S} ^ {d - 1}} \cdot y _ {j} ^ {\ell} (\sigma) \\ = \sum_ {\ell = 0} ^ {q} f _ {\ell} (\sigma) = f ^ {(q)} (\sigma), \\ \end{array}
$$
where the second line above follows from Theorem 3, the fourth line follows from Eq. (2), and the last line follows from Lemma 2. This proves that the low-degree component  $f^{(q)} = \mathcal{K}_d^{(q)}f$ .
$$
\left\| \mathcal {K} _ {d} ^ {(q)} \tilde {g} - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \leq C \cdot \min _ {g \in L ^ {2} (\mathbb {S} ^ {d - 1})} \left\| \mathcal {K} _ {d} ^ {(q)} g - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2},
$$
Proof. First, note that  $g^{*} = f$  is an optimal solution to the least-squares problem in Eq. (7). Thus we have,
$$
\min  _ {g \in L ^ {2} (\mathbb {S} ^ {d - 1})} \left\| \mathcal {K} _ {d} ^ {(q)} g - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} = \left\| \mathcal {K} _ {d} ^ {(q)} f - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} = \left\| f ^ {(q)} - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}.
$$
Next, we can write,
$$
\begin{array}{l} \left\| \mathcal {K} _ {d} ^ {(q)} \tilde {g} - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} = \left\| \mathcal {K} _ {d} ^ {(q)} \tilde {g} - \mathcal {K} _ {d} ^ {(q)} f + \left(\mathcal {K} _ {d} ^ {(q)} f - f\right) \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \\ = \left\| \mathcal {K} _ {d} ^ {(q)} (\tilde {g} - f) + \left(\mathcal {K} _ {d} ^ {(q)} f - f\right) \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \\ = \left\| \mathcal {K} _ {d} ^ {(q)} (\tilde {g} - f) \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} + \left\| \mathcal {K} _ {d} ^ {(q)} f - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \\ = \left\| \mathcal {K} _ {d} ^ {(q)} \tilde {g} - f ^ {(q)} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} + \left\| f ^ {(q)} - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}, \\ \end{array}
$$
where the third line follows from the Pythagorean theorem because  $\mathcal{K}_d^{(q)}(\tilde{g} -f)\in \mathcal{H}^{(q)}\left(\mathbb{S}^{d - 1}\right)$  while  $\mathcal{K}_d^{(q)}f - f = -\sum_{\ell >q}f_\ell$  thus  $\left(\mathcal{K}_d^{(q)}f - f\right)\perp \mathcal{H}^{(q)}\left(\mathbb{S}^{d - 1}\right)$ . Combining the two equalities above with inequality  $\left\| \mathcal{K}_d^{(q)}\tilde{g} -f\right\|_{\mathbb{S}^{d - 1}}^2\leq C\cdot \min_{g\in L^2 (\mathbb{S}^{d - 1})}\left\| \mathcal{K}_d^{(q)}g - f\right\|_{\mathbb{S}^{d - 1}}^2$  given in the statement of the claim, proves Claim 2.
C Approximate Regression via Leverage Score Sampling
In this section we ultimately prove our main result of Theorem 4. We start by proving useful properties of the leverage function given in Definition 3. First, we show the fact that the leverage function can be characterized in terms of a least-squares minimization problem, which is crucial for computing the leverage scores distribution. This fact was previously exploited in [AKM+17] and [AKM+19] in the context of Fourier operators.
Lemma 4 (Min Characterization of the Leverage Function). For any  $w \in \mathbb{S}^{d-1}$ , let  $\tau_q(w)$  be the leverage function (Definition 3) and define  $\phi_w \in L^2(\mathbb{S}^{d-1})$  by  $\phi_w(\sigma) \equiv \sum_{\ell=0}^q \frac{\alpha_{\ell,d}}{|\mathbb{S}^{d-1}|} P_d^\ell(\langle \sigma, w \rangle)$ . We have the following minimization characterization of the leverage function:
We remark that this lemma is in fact an adaptation and generalization of Theorem 5 of  $\left[\mathrm{AKM}^{+}19\right]$ . We prove this lemma here for the sake of completeness.
Proof. First we show that the right hand side of Eq. (9) is never smaller than the leverage function in Definition 3. Let  $g_w^* \in L^2(\mathbb{S}^{d-1})$  be the optimal solution of Eq. (9) for any  $w \in \mathbb{S}^{d-1}$ . Note that the
optimal solution satisfies  $\mathcal{K}_d^{(q)}g_w^* = \phi_w$ . Thus, for any function  $f\in L^{2}(\mathbb{S}^{d - 1})$ , using Definition 2, we can write
$$
\begin{array}{l} \left| \left[ \mathcal {K} _ {d} ^ {(q)} f \right] (w) \right| ^ {2} = \left| \sum_ {\ell = 0} ^ {q} \alpha_ {\ell , d} \cdot \underset {\sigma \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ P _ {d} ^ {\ell} (\langle \sigma , w \rangle) \cdot f (\sigma) \right] \right| ^ {2} \\ = \left| \langle \phi_ {w}, f \rangle_ {\mathbb {S} ^ {d - 1}} \right| ^ {2} = \left| \left\langle \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*}, f \right\rangle_ {\mathbb {S} ^ {d - 1}} \right| ^ {2} \\ = \left| \left\langle g _ {w} ^ {*}, \mathcal {K} _ {d} ^ {(q)} f \right\rangle_ {\mathbb {S} ^ {d - 1}} \right| ^ {2} \\ \leq \left\| g _ {w} ^ {*} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \cdot \left\| \mathcal {K} _ {d} ^ {(q)} f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \\ \end{array}
$$
(because  $\mathcal{K}_d^{(q)}$  is self-adjoint)
(by Cauchy-Schwarz inequality)
Therefore, for any  $f \in L^{2}(\mathbb{S}^{d - 1})$  with  $\left\| \mathcal{K}_d^{(q)}f\right\|_{\mathbb{S}^{d - 1}} > 0$ , we have
$$
\frac {\left| \left[ \mathcal {K} _ {d} ^ {(q)} f \right] (w) \right| ^ {2}}{\left\| \mathcal {K} _ {d} ^ {(q)} f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}} \leq \left\| g _ {w} ^ {*} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}. \tag {11}
$$
We conclude the proof by showing that the maximum value is attained. First, we show that the optimal solution  $g_{w}^{*}$  of Eq. (9) satisfies the property that  $\mathcal{K}_d^{(q)}g_w^* = g_w^*$ . Suppose for the sake of contradiction that  $\mathcal{K}_d^{(q)}g_w^*\neq g_w^*$ . In this case, Claim 1 implies that,
$$
\mathcal {K} _ {d} ^ {(q)} \left(\mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} - g _ {w} ^ {*}\right) = \left(\mathcal {K} _ {d} ^ {(q)}\right) ^ {2} g _ {w} ^ {*} - \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} = \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} - \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} = 0.
$$
Thus, the function  $g = \mathcal{K}_d^{(q)}g_w^*$  satisfies the constraint of the minimization problem in Eq. (9). Now, using the above and the fact that  $\mathcal{K}_d^{(q)}$  is self-adjoint we can write,
$$
\left\langle \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*}, \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} - g _ {w} ^ {*} \right\rangle_ {\mathbb {S} ^ {d - 1}} = \left\langle g _ {w} ^ {*}, \mathcal {K} _ {d} ^ {(q)} \left(\mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} - g _ {w} ^ {*}\right) \right\rangle_ {\mathbb {S} ^ {d - 1}} = 0.
$$
This shows that  $\mathcal{K}_d^{(q)}g_w^*\perp \left(\mathcal{K}_d^{(q)}g_w^* -g_w^*\right)$ , hence by Pythagorean theorem we have,
$$
\left\| g _ {w} ^ {*} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} = \left\| \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} + \left\| \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} - g _ {w} ^ {*} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} > \left\| \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} = \| g \| _ {\mathbb {S} ^ {d - 1}} ^ {2},
$$
which is in contrast with the assumption that  $g_w^*$  is the optimal solution of Eq. (9). Therefore, our claim that  $\mathcal{K}_d^{(\ell)}g_w^* = g_w^*$  holds.
Now, we show that for  $f = g_w^*$ , the maximum value in inequality Eq. (11) is attained. For any  $w \in \mathbb{S}^{d-1}$  we have the following
$$
\left[ \mathcal {K} _ {d} ^ {(q)} f \right] (w) = \left\langle \mathcal {K} _ {d} ^ {(q)} g _ {w} ^ {*}, f \right\rangle_ {\mathbb {S} ^ {d - 1}} = \left\langle g _ {w} ^ {*}, g _ {w} ^ {*} \right\rangle_ {\mathbb {S} ^ {d - 1}} = \| g _ {w} ^ {*} \| _ {\mathbb {S} ^ {d - 1}} ^ {2}.
$$
On the other hand we have  $\left\| \mathcal{K}_d^{(q)}f\right\|_{\mathbb{S}^{d - 1}}^2 = \| g_w^*\|_{\mathbb{S}^{d - 1}}^2$ . Thus,  $\left\| \mathcal{K}_d^{(q)}f\right\|_{\mathbb{S}^{d - 1}}^{-2}\cdot \left|\left[\mathcal{K}_d^{(q)}f\right](w)\right|^2 =$ $\| g_w^*\|_{\mathbb{S}^{d - 1}}^2$  which implies that  $\tau_q(w) = \| g_w^*\|_{\mathbb{S}^{d - 1}}^2$  and thus proves the lemma.
Next we prove that the leverage function is constant.
Proof. First we prove that  $\tau_q(w) \leq \sum_{\ell=0}^q \frac{\alpha_{\ell,d}}{|\mathbb{S}^{d-1}|}$  using the min-characterization. If we let  $\phi_w \in L^2(\mathbb{S}^{d-1})$  be defined as  $\phi_w(\sigma) := \sum_{\ell=0}^q \frac{\alpha_{\ell,d}}{|\mathbb{S}^{d-1}|} P_d^\ell(\langle \sigma, w \rangle)$ , then by Definition 2, for every  $\sigma \in \mathbb{S}^{d-1}$
we can write,
$$
\begin{array}{l} \left[ \mathcal {K} _ {d} ^ {(q)} \phi_ {w} \right] (\sigma) = \sum_ {\ell = 0} ^ {q} \alpha_ {\ell , d} \cdot \underset {v \sim \mathcal {U} \left(\mathbb {S} ^ {d - 1}\right)} {\mathbb {E}} \left[ P _ {d} ^ {\ell} (\langle \sigma , v \rangle) \cdot \phi_ {w} (v) \right] \\ = \sum_ {\ell = 0} ^ {q} \sum_ {\ell^ {\prime} = 0} ^ {q} \frac {\alpha_ {\ell , d} \alpha_ {\ell^ {\prime} , d}}{| \mathbb {S} ^ {d - 1} |} \cdot \underset {v \sim \mathcal {U} (\mathbb {S} ^ {d - 1})} {\mathbb {E}} \left[ P _ {d} ^ {\ell} \left(\langle \sigma , v \rangle\right) \cdot P _ {d} ^ {\ell^ {\prime}} \left(\langle v, w \rangle\right) \right] \\ = \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |} P _ {d} ^ {\ell} (\langle \sigma , w \rangle) = \phi_ {w} (\sigma), \tag {12} \\ \end{array}
$$
where the third line above follows from Lemma 1. Therefore, the test function  $g \coloneqq \phi_w$  satisfies the constraint of the minimization in Eq. (9), i.e.,  $\mathcal{K}_d^{(q)}g = \phi_w$ . Thus, Lemma 4 implies that,
$$
\tau_ {q} (w) \leq \| g \| _ {\mathbb {S} ^ {d - 1}} ^ {2} = \| \phi_ {w} \| _ {\mathbb {S} ^ {d - 1}} ^ {2} = \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |},
$$
where the equality above follows from Lemma 1 along with Eq. (2). This establishes the upper bound on the leverage function that we sought to prove.
Now, using the maximization characterization of the leverage function in Definition 3, we prove that  $\tau_q(w) \geq \sum_{\ell=0}^{q} \frac{\alpha_{\ell,d}}{|\mathbb{S}^{d-1}|}$ . Again, we consider the same test function  $g = \phi_w$  and write,
$$
\begin{array}{l} \left| \left| \mathcal {K} _ {d} ^ {(q)} \phi_ {w} \right| \right| _ {\mathbb {S} ^ {d - 1}} ^ {- 2} \cdot \left| \left[ \mathcal {K} _ {d} ^ {(q)} \phi_ {w} \right] (w) \right| ^ {2} = \frac {\left| \phi_ {w} (w) \right| ^ {2}}{\left| \phi_ {w} \right| _ {\mathbb {S} ^ {d - 1}} ^ {2}} \\ = \frac {\left| \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |} P _ {d} ^ {\ell} (\langle w , w \rangle) \right| ^ {2}}{\sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |}} \\ = \frac {\left| \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |} P _ {d} ^ {\ell} (1) \right| ^ {2}}{\sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |}} = \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |}, \\ \end{array}
$$
where the first and second line above follow from Eq. (12) and Lemma 1, respectively. Therefore, the max characterization of the leverage function in Definition 3 implies that,
$$
\tau_ {q} (w) \geq \left\| \mathcal {K} _ {d} ^ {(q)} \phi_ {w} \right\| _ {\mathbb {S} ^ {d - 1}} ^ {- 2} \cdot \left| \left[ \mathcal {K} _ {d} ^ {(q)} \phi_ {w} \right] (w) \right| ^ {2} = \sum_ {\ell = 0} ^ {q} \frac {\alpha_ {\ell , d}}{| \mathbb {S} ^ {d - 1} |}.
$$
This completes the proof of Lemma 5 and establishes that  $\tau_q(w)$  is uniformly equal to  $\sum_{\ell = 0}^{q}\frac{\alpha_{\ell,d}}{|\mathbb{S}^{d - 1}|}$ .
To prove Theorem 4, we need to use prior results about solving linear systems in continuous setting via leverage score sampling. In particular, we use Theorem 6.3 from [CP19], which is restated below,
Theorem 7 (Theorem 6.3 of [CP19]). Consider any dimension  $n$  linear space  $\mathcal{F}$  of functions from a domain  $G$  to  $\mathbb{R}$ . Let  $D$  be a distribution over  $G$  and  $f$  be some function from  $G$  to  $\mathbb{R}$ . Also, define the norm with respect to  $D$  of any function  $h: G \to \mathbb{R}$  as  $\| h \|_D^2 := \mathbb{E}_{x \sim D} [|h(x)|^2]$  and let  $y = \arg \min_{h \in \mathcal{F}} \| h - f \|_D^2$ . Fix any distribution  $D'$  over  $G$  and let  $K_{D'} := \sup_{x \in G} \sup_{h \in \mathcal{F}} \left\{ \frac{D(x)}{D'(x)} \cdot \frac{|h(x)|^2}{\| h \|_D^2} \right\}$ .
For i.i.d. sample query points  $x_{1}, x_{2}, \ldots, x_{s} \sim D'$  and weights  $w_{i} = \frac{D(x_{i})}{s \cdot D'(x_{i})}$  for  $i \in [s]$ , let the weighted ERM estimator  $\widetilde{f}_{s}$  be defined as  $\widetilde{f}_{s} := \arg \min_{h \in \mathcal{F}} \sum_{i=1}^{s} w_{i} \cdot |h(x_{i}) - f(x_{i})|^{2}$ . For any  $\varepsilon > 0$  and a sufficiently large fixed constant  $c$ , if the number of queries  $s \geq c \cdot \left( K_{D'} \log n + \frac{K_{D'}}{\varepsilon} \right)$ , then the weighted ERM estimator  $\widetilde{f}_{s}$  satisfies,
$$
\Pr \left[ \left\| \widetilde {f _ {s}} - y \right\| _ {D} ^ {2} \leq \varepsilon \cdot \| f - y \| _ {D} ^ {2} \right] \geq 1 - 1 0 ^ {- 4}.
$$
Now we are ready to prove Theorem 4. Our approach is to apply Theorem 7 to the space of degree  $q$  spherical harmonics  $\mathcal{H}^{(q)}(\mathbb{S}^{d - 1})$  and use the fact that the leverage scores distribution of the operator  $\mathcal{K}_d^{(q)}$  are uniform on the unit sphere  $\mathbb{S}^{d - 1}$ .
Proof. We prove this theorem by invoking Theorem 7. To do so, we first let the space  $\mathcal{F}$  of function from  $\mathbb{S}^{d - 1}$  to  $\mathbb{R}$  be  $\mathcal{F} \coloneqq \mathcal{H}^{(q)}\left(\mathbb{S}^{d - 1}\right)$ . It is clear that the space of spherical harmonics is a linear space of functions because of the existence of the kernel operator  $\kappa_{d}^{(q)}$  which is a projection operator onto  $\mathcal{H}^{(q)}\left(\mathbb{S}^{d - 1}\right)$ , so  $\mathcal{F}$  satisfies the first precondition of Theorem 7. Additionally, the dimension of this space of functions is  $n = \beta_{q,d}$ .
Also, let  $D$  be a uniform distribution over the unit sphere  $\mathbb{S}^{d-1}$ . For this distribution, the norm defined in Theorem 7 satisfies  $\| h\|_{\mathbb{S}^{d-1}}^2 = |\mathbb{S}^{d-1}| \cdot \| h\|_D^2$ , for any  $h \in L^2(\mathbb{S}^{d-1})$ . Moreover, let  $D'$  be a uniform distribution over the unit sphere  $\mathbb{S}^{d-1}$  as well.
Now we show that for these choices of  $\mathcal{F}, D, D'$ , the condition number  $K_{D'}$  defined as per Theorem 7 is equal to  $\beta_{q,d}$ . We can write,
$$
\begin{array}{l} K _ {D ^ {\prime}} := \sup  _ {x \in \mathbb {S} ^ {d - 1}} \sup  _ {h \in \mathcal {F}} \left\{\frac {D (x)}{D ^ {\prime} (x)} \cdot \frac {| h (x) | ^ {2}}{\| h \| _ {D} ^ {2}} \right\} \\ = | \mathbb {S} ^ {d - 1} | \cdot \sup  _ {x \in \mathbb {S} ^ {d - 1}} \sup  _ {h \in \mathcal {F}} \left\{\frac {| h (x) | ^ {2}}{\| h \| _ {\mathbb {S} ^ {d - 1}} ^ {2}} \right\} \\ = \left| \mathbb {S} ^ {d - 1} \right| \cdot \sup  _ {x \in \mathbb {S} ^ {d - 1}} \sup  _ {g \in L ^ {2} (\mathbb {S} ^ {d - 1})} \left\{\frac {\left| \left[ \mathcal {K} _ {d} ^ {(q)} g \right] (x) \right| ^ {2}}{\left| \left| \mathcal {K} _ {d} ^ {(q)} g \right| \right| _ {\mathbb {S} ^ {d - 1}} ^ {2}} \right\} \\ = | \mathbb {S} ^ {d - 1} | \cdot \sup  _ {x \in \mathbb {S} ^ {d - 1}} \tau_ {q} (x) \\ = \beta_ {q, d}, \\ \end{array}
$$
where the second line above follows from the fact that  $D, D'$  are both equal to the uniform distribution over  $\mathbb{S}^{d-1}$  and  $\|h\|_{\mathbb{S}^{d-1}}^2 = |\mathbb{S}^{d-1}| \cdot \|h\|_D^2$ . The third line above follows from the fact that any function  $h \in \mathcal{H}^{(q)}(\mathbb{S}^{d-1})$  can be expressed as  $h = \mathcal{K}_d^{(q)}g$  for some  $g \in L^2(\mathbb{S}^{d-1})$  because  $\mathcal{K}_d^{(q)}$  is the projection operator onto  $\mathcal{H}^{(q)}(\mathbb{S}^{d-1})$ . The fourth line follows from Definition 3 and last line follows from Lemma 5.
Finally, in order to invoke Theorem 7, we can write that the weighted ERM estimator  $\widetilde{f}_s$  is equal to the following,
$$
\begin{array}{l} \widetilde {f} _ {s} := \underset {h \in \mathcal {F}} {\arg \min } \sum_ {i = 1} ^ {s} w _ {i} \cdot | h (x _ {i}) - f (x _ {i}) | ^ {2} \\ = \underset {g \in L ^ {2} (\mathbb {S} ^ {d - 1})} {\arg \min} \sum_ {i = 1} ^ {s} w _ {i} \cdot \left| \left[ \mathcal {K} _ {d} ^ {(q)} g \right] (x _ {i}) - f (x _ {i}) \right| ^ {2} \\ = \underset {g \in L ^ {2} (\mathbb {S} ^ {d - 1})} {\arg \min} \sum_ {i = 1} ^ {s} \left| \frac {1}{\sqrt {s}} \cdot \left[ \mathcal {K} _ {d} ^ {(q)} g \right] (x _ {i}) - \boldsymbol {f} _ {i} \right| ^ {2} \\ = \operatorname *{arg  min}_{g\in L^{2}(\mathbb{S}^{d - 1})}\| P^{*}g - f\|_{2}^{2}, \\ \end{array}
$$
where the third line above uses the definition of  $f_{i} = \frac{1}{\sqrt{s}} f(x_{i})$ , and the last line follows from the definition of adjoint of the quasi-matrix  $P$ . Therefore, the theorem follows by invoking Theorem 7.
D Efficient Algorithm for Spherical Harmonic Interpolation
In this section we prove our main theorem about our spherical harmonic interpolation algorithm.
Proof. First note that the random points  $w_{1}, w_{2}, \ldots, w_{s}$  in line 3 of Algorithm 1 are i.i.d. sample with uniform distribution on the surface of  $\mathbb{S}^{d-1}$ . Therefore, we can invoke Theorem 4. More specifically, if we let  $P$  be the quasi-matrix defined in Theorem 4 corresponding to the random points  $w_{1}, w_{2}, \ldots, w_{s}$  sampled in line 3 and if we let  $f$  be the vector of function samples defined in line 5 of the algorithm, then with probability at least  $1 - 10^{-4}$ , any optimal solution to the following least-squares problem
$$
\tilde {g} \in \arg \min  _ {g \in L ^ {2} \left(\mathbb {S} ^ {d - 1}\right)} \| P ^ {*} g - f \| _ {2} ^ {2}, \tag {13}
$$
satisfies the following,
$$
\left\| \mathcal {K} _ {d} ^ {(q)} \tilde {g} - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2} \leq (1 + \varepsilon) \cdot \min  _ {g \in L ^ {2} (\mathbb {S} ^ {d - 1})} \left\| \mathcal {K} _ {d} ^ {(q)} g - f \right\| _ {\mathbb {S} ^ {d - 1}} ^ {2}. \tag {14}
$$