Title: Near Optimal Reconstruction of Spherical Harmonic Expansions
Venue: NeurIPS 2023
Accepted: yes
Keywords: spherical harmonic expansion

Gegenbauer polynomials

interpolation

leverage score sampling
Abstract: We propose an algorithm for robust recovery of the spherical harmonic expansion of functions defined on the $d$-dimensional unit sphere $\mathbb{S}^{d-1}$ using a near-optimal number of function evaluations. We show that for any $f\in L^2(\mathbb{S}^{d-1})$, the number of evaluations of $f$ needed to recover its degree-$q$ spherical harmonic expansion equals the dimension of the space of spherical harmonics of degree at most $q$, up to a logarithmic factor. Moreover, we develop a simple yet efficient kernel regression-based algorithm to recover degree-$q$ expansion of $f$ by only evaluating the function on uniformly sampled points on $\mathbb{S}^{d-1}$. Our algorithm is built upon the connections between spherical harmonics and Gegenbauer polynomials. Unlike the prior results on fast spherical harmonic transform, our proposed algorithm works efficiently using a nearly optimal number of samples in any dimension $d$. Furthermore, we illustrate the empirical performance of our algorithm on numerical examples.

Near Optimal Reconstruction of Spherical Harmonic Expansions

Abstract

We propose an algorithm for robust recovery of the spherical harmonic expansion of functions defined on the d -dimensional unit sphere S d -1 using a near-optimal number of function evaluations. We show that for any f ∈ L 2 ( S d -1 ) , the number of evaluations of f needed to recover its degreeq spherical harmonic expansion equals the dimension of the space of spherical harmonics of degree at most q , up to a logarithmic factor. Moreover, we develop a simple yet efficient kernel regressionbased algorithm to recover degreeq expansion of f by only evaluating the function on uniformly sampled points on S d -1 . Our algorithm is built upon the connections between spherical harmonics and Gegenbauer polynomials. Unlike the prior results on fast spherical harmonic transform, our proposed algorithm works efficiently using a nearly optimal number of samples in any dimension d . Furthermore, we illustrate the empirical performance of our algorithm on numerical examples.

1 Introduction

We consider the fundamental problem of recovering a function from a finite number of (noisy) observations. To provide accurate and reliable predictions at unobserved points we need to avoid overfitting which is typically achieved through restricting our estimator or interpolant to a family of smooth or structured functions. In this paper, we focus on interpolating square-integrable functions on the d -dimensional unit sphere, with low-degree spherical harmonics, a critical task in scenarios where rotational invariance is a fundamental property. Spherical harmonics are essential in various theoretical and practical applications, including the representation of electromagnetic fields [Wei95], gravitational potential [Wer97], cosmic microwave background radiation [KKS97] and medical imaging [CLL15], as well as modelling of 3D shapes in computer graphics [KFR03] and computer vision [GCL + 22]. Further notable real-life applications include molecular/atom systems, where understanding the underlying functions within a spherical context can significantly enhance predictive modeling and simulation accuracy [EEHM17, LWL + 21, ZDK + 22].

We begin by observing that any function f in L 2 ( S d -1 ) , i.e., the family of square-integrable functions on the sphere S d -1 , can be uniquely decomposed into orthogonal spherical harmonic components. Specifically, if we denote the space of spherical harmonics of degree glyph[lscript] in dimension d by H glyph[lscript] ( S d -1 ) , any f ∈ L 2 ( S d -1 ) has a unique orthogonal expansion f = ∑ ∞ glyph[lscript] =0 f glyph[lscript] with f glyph[lscript] ∈ H glyph[lscript] ( S d -1 ) (Lemma 2). With this observation, we aim to find the best spherical harmonic approximation of degree ≤ q to f using minimal number of samples (essentially treating higher order terms in f 's expansion as noise).

Insu Han

Problem 1 (Informal Version of Problem 2) . For an unknown function f ∈ L 2 ( S d -1 ) and an integer q ≥ 1 , efficiently (both in terms of number of samples from f and computations) learn the first q +1 spherical harmonic components { f glyph[lscript] ∈ H glyph[lscript] ( S d -1 ) } q glyph[lscript] =0 of f which minimize

The angular power spectrum of f commonly obeys a power law decay of the form ‖ f glyph[lscript] ‖ 2 S d -1 ≤ O ( glyph[lscript] -s ) , for some s > 0 , depending on the order of differentiability of f . In fact, for any infinitely differentiable f , ‖ f glyph[lscript] ‖ 2 S d -1 decays asymptotically faster than any rational function of glyph[lscript] . Furthermore, for any real analytic f on the sphere, ‖ f glyph[lscript] ‖ 2 S d -1 decays exponentially. Thus, the first q +1 spherical harmonic components of f typically well approximate f for even modest q , and answering Problem 1 is meaningful for a wide range of differentiable functions.

1.1 Our Main Results

We reformulate Problem 1 as a least-squares regression and then solve it using randomized numerical linear algebra techniques. We first consider an orthonormal projection operator that maps functions in L 2 ( S d -1 ) onto the space of bounded-degree spherical harmonics ⊕ q glyph[lscript] =0 H glyph[lscript] ( S d -1 ) . Specifically, if K ( q ) d is an operator that maps any f with expansion f = ∑ ∞ glyph[lscript] =0 f glyph[lscript] where f glyph[lscript] ∈ H glyph[lscript] ( S d -1 ) , onto f 's first q +1 expansion components, i.e., K ( q ) d f = ∑ q glyph[lscript] =0 f glyph[lscript] , then Problem 1 can be formulated as

However, solving this regression problem with 'continuous' cost function is challenging. To avoid such continuous optimizations, we adopt the approach of [AKM + 19] which discretizes the aforementioned regression problem according to the leverage scores of the operator K ( q ) d . It turns out that if we can draw random samples with probabilities proportional to the leverage scores of K ( q ) d then we can recover the degreeq spherical harmonic expansion of f , i.e. ∑ q glyph[lscript] =0 f glyph[lscript] , with finite number of observations. Particularly, by exploiting the connections between spherical harmonics and Zonal (Gegenbauer) Harmonics and the fact that zonal harmonics are the reproducing kernels of H glyph[lscript] ( S d -1 ) (Lemma 3), we prove that the leverage scores of K ( q ) d are constant everywhere on the sphere S d -1 . Thus, solving a discrete regression problem with uniformly sampled observations yields a near-optimal solution to Problem 1. Informal statements of our results are as follows.

Theorem 1 (Informal Version of Theorem 5) . Let β q,d be the dimension of spherical harmonics of degree at most q , i.e., β q,d ≡ dim (⊕ q glyph[lscript] =0 H glyph[lscript] ( S d -1 ) ) . There exists an algorithm that finds a (1 + ε ) -approximation to the optimal solution of Problem 1, given s = O ( β q,d log β q,d + ε -1 β q,d ) observations of function f at uniformly sampled points on S d -1 , with O ( s 2 d + s ω ) 1 runtime.

We also prove that our bound on the number of required samples is optimal up to a logarithmic factor. Theorem 2 (Informal Version of Theorem 6) . Any (randomized) algorithm that takes s < β q,d samples on any input fails with probability greater than 9 / 10 , where β q,d ≡ dim (⊕ q glyph[lscript] =0 H glyph[lscript] ( S d -1 ) ) .

1.2 Related Work

Efficient reconstruction of functions as per Problem 1 has been extensively studied in various fields. Many prior papers considered reconstructing 1 -dimensional functions from finite number of samples on a finite interval under smoothness assumption about the underlying function. Notably, the influential line of work of [SP61, LP61, LP62, XRY01] focused on reconstructing Fourier-bandlimited functions and [CKPS16, KVZ19, EMM20, BKM + 23] considered interpolating Fourier-sparse signals. Recently, [AKM + 19] unified these reconstruction methods in dimension d = 1 and gave a universal sampling framework for reconstructing nearly all classes of functions with Fourier-based smoothness constraints. One can view 1 -dimensional functions on a finite interval as functions on the unit circle

S 1 . Thus, Problem 1 is indeed a generalization of the aforementioned prior work to high dimensions under the assumption that the generalized Fourier series (Lemma 2) of the underlying function only contains bounded-degree spherical harmonics. This degree constraint on spherical harmonic expansions can be viewed as the d -dimensional analog of Fourier-bandlimitedness on circle S 1 .

Computing the spherical harmonic expansion in dimension d = 3 has received considerable attention in physics and applied mathematics communities. The algorithms for this special case of Problem 1 are known in the literature as 'fast spherical harmonic transform' [SS00, ST02]. Most notably, [RT06] proposed an algorithm for computing spherical harmonic expansion of degree ≤ q to precision ε using O ( β q, 3 ) samples and O ( β q, 3 log β q, 3 · log(1 /ε )) time. These fast algorithms were developed based on fast Fourier and associated Legendre transforms and make use of a (well-conditioned) orthogonal basis for H glyph[lscript] ( S d -1 ) , which happened to be the associated Legendre polynomials when d = 3 . However, it is in general very difficult to compute an orthogonal basis for spherical harmonics [MNY06], so unlike our Theorem 1, it is inefficient to extend these prior results to higher d .

From the techniques point of view, a related work is [GMMM21], which employs harmonic analysis over S d -1 to analyze the generalization of two-layered neural tangent kernels. They show that an unknown function defined on S d -1 can be efficiently recovered using kernel regression w.r.t. neural tangent kernel on uniform random samples from the function. However, the number of samples that [GMMM21] requires for recovering bounded degree spherical harmonics, especially when the degrees are high, is sub-optimal and is strictly worse than our result. Additionally, [GMMM21] does not guarantee recovery with relative error, while our Theorem 5 provides relative error guarantees.

Furthermore, recent applications of Gegenbauer polynomials, along with other orthonormal polynomials like Hermite polynomials, have been found in designing efficient random features for approximating various kernel functions. These applications extend to dot-product kernels [HZA22], Neural Tangent Kernels [ZHA + 21, HZL + 22], and Gaussian kernels [WZ22, AKK + 20].

2 Mathematical Preliminaries

We denote by S d -1 the unit sphere in d dimension. We use | S d -1 | = 2 π d/ 2 Γ( d/ 2) to denote the surface area of sphere S d -1 and U ( S d -1 ) to denote the uniform probability distribution on S d -1 . We denote by L 2 ( S d -1 ) the set of all square-integrable real-valued functions on sphere S d -1 . Furthermore, for any f, g ∈ L 2 ( S d -1 ) we use the following definition of the inner product on the unit sphere 2 ,

The function space L 2 ( S d -1 ) is complete with respect to the norm induced by the above inner product, i.e. ‖ f ‖ S d -1 := √ 〈 f, f 〉 S d -1 , so L 2 ( S d -1 ) is a Hilbert space .

We often use the term quasi-matrix which is informally defined as a 'matrix' in which one dimension is finite while the other is infinite. A quasi-matrix can be tall (or wide ) meaning that there is a finite number of columns (or rows) where each one is a functional operator. For a more formal definition, see [SA22].

Spherical Harmonics are the solutions of Laplace's equation in spherical domains and can be thought of as functions defined on S d -1 employed in solving partial differential equations. Formally,

Definition 1 (Spherical Harmonics) . For integers glyph[lscript] ≥ 0 and d ≥ 1 , let P glyph[lscript] ( d ) be the space of degreeglyph[lscript] homogeneous polynomials with d variables and real coefficients. Let H glyph[lscript] ( d ) denote the space of degreeglyph[lscript] harmonic polynomials in dimension d , i.e., homogeneous polynomial solutions of Laplace's equation:

where ∆ = ∂ 2 ∂x 2 1 + · · · + ∂ 2 ∂x 2 d is the Laplace operator on R d . Finally, let H glyph[lscript] ( S d -1 ) be the space of (real) Spherical Harmonics of order glyph[lscript] in dimension d , i.e. restrictions of harmonic polynomials in

H glyph[lscript] ( d ) to the sphere S d -1 . The dimension of this space, α glyph[lscript],d ≡ dim ( H glyph[lscript] ( S d -1 )) , is

2.1 Gegenbauer Polynomials

The Gegenbauer (a.k.a. ultraspherical ) polynomial of degree glyph[lscript] ≥ 0 in dimension d ≥ 2 is given by

where c 0 = 1 and c j +1 = -( glyph[lscript] -2 j )( glyph[lscript] -2 j -1) 2( j +1)( d -1+2 j ) · c j for j = 0 , 1 , . . . , glyph[floorleft] glyph[lscript]/ 2 glyph[floorright]1 . These polynomials are orthogonal on the interval [ -1 , 1] with respect to the measure (1 -t 2 ) d -3 2 , i.e.,

Zonal Harmonics. The Gegenbauer polynomials naturally provide positive definite dot-product kernels on S d -1 known as Zonal Harmonics , which are closely related to the spherical harmonics. The following reproducing property of zonal harmonics plays a crucial role in our analysis.

Lemma 1 (Reproducing Property of Zonal Harmonics) . Let P glyph[lscript] d ( · ) be the Gengenbauer polynomial of degree glyph[lscript] in dimension d . For any x, y ∈ S d -1 :

glyph[negationslash]

Furthermore, for any glyph[lscript] ′ = glyph[lscript] :

The proof of this and all subsequent results can be found in the appendix. The following useful fact, known as the addition theorem, connects Gegenbauer polynomials and spherical harmonics.

Theorem 3 (Addition Theorem) . For every integer glyph[lscript] ≥ 0 , if { y glyph[lscript] 1 , y glyph[lscript] 2 , . . . , y glyph[lscript] α glyph[lscript],d } is an orthonormal basis for H glyph[lscript] ( S d -1 ) , then for any σ, w ∈ S d -1 we have

3 Reconstructing L 2 ( S d -1 ) Functions via Spherical Harmonics

In this section we show how to approximate any function f ∈ L 2 ( S d -1 ) by spherical harmonics using the optimal number of samples. We begin with the fact that spherical harmonics form a complete set of orthonormal functions and thus form an orthonormal basis for the Hilbert space of square-integrable functions on sphere S d -1 . This is analogous to periodic functions, viewed as functions defined on the circle S 1 , which can be expressed as a linear combination of circular functions (sines and cosines) via the Fourier series.

Lemma 2 (Direct Sum Decomposition of L 2 ( S d -1 ) ) . The family of spaces H glyph[lscript] ( S d -1 ) yields a Hilbert space direct sum decomposition L 2 ( S d -1 ) = ⊕ ∞ glyph[lscript] =0 H glyph[lscript] ( S d -1 ) : the summands are closed and pairwise orthogonal, and every f ∈ L 2 ( S d -1 ) is the sum of a converging series (in the sense of mean-square convergence with the L 2 -norm defined in Eq. (2) ),

where f glyph[lscript] ∈ H glyph[lscript] ( S d -1 ) are uniquely determined functions. Furthermore, given any orthonormal basis { y glyph[lscript] 1 , y glyph[lscript] 2 , . . . , y glyph[lscript] α glyph[lscript],d } of H glyph[lscript] ( S d -1 ) we have f glyph[lscript] = ∑ α glyph[lscript],d j =1 〈 f, y glyph[lscript] j 〉 S d -1 · y glyph[lscript] j .

The series expansion in Lemma 2 is the analog of the Fourier expansion of periodic functions, and is known as ' generalized Fourier series ' [Pen30] with respect to the Hilbert basis { y glyph[lscript] j : j ∈ [ α glyph[lscript],d ] , glyph[lscript] ≥ 0 } . We remark that it is in general intractable to compute an orthogonal basis for the space of spherical harmonics [MNY06], which renders the generalized Fourier series expansion in Lemma 2 primarily existential. While finding the generalized Fourier expansion of a function f ∈ L 2 ( S d -1 ) is computationally intractable, our goal is to answer the next fundamental question, which is about finding the projection of a function f onto the space of spherical harmonics, i.e., the f glyph[lscript] 's in Lemma 2. Concretely, we seek to solve the following problem.

Problem 2. For an integer q ≥ 0 and a given function f ∈ L 2 ( S d -1 ) whose decomposition over the Hilbert sum ⊕ ∞ glyph[lscript] =0 H glyph[lscript] ( S d -1 ) is f = ∑ ∞ glyph[lscript] =0 f glyph[lscript] as per Lemma 2, let us define the low-degree expansion of this function as f ( q ) := ∑ q glyph[lscript] =0 f glyph[lscript] . How efficiently can we learn f ( q ) ∈ ⊕ q glyph[lscript] =0 H glyph[lscript] ( S d -1 ) ? More precisely, we want to find a set { w 1 , w 2 , . . . , w s } ⊆ S d -1 with minimal cardinality s along with an efficient algorithm that given samples { f ( w i ) } s i =1 can interpolate f ( · ) with a function ˜ f ( q ) ∈ ⊕ q glyph[lscript] =0 H glyph[lscript] ( S d -1 ) such that:

For ease of notation, we denote the Hilbert space of spherical harmonics of degree at most q by H ( q ) ( S d -1 ) := ⊕ q glyph[lscript] =0 H glyph[lscript] ( S d -1 ) . To answer Problem 2 we exploit the close connection between the spherical harmonics and Gengenbauer polynomials, and in particular the fact that zonal harmonics are the reproducing kernels of the Hilbert spaces H glyph[lscript] ( S d -1 ) .

Lemma 3 (A Reproducing Kernel for H glyph[lscript] ( S d -1 ) ) . For every f ∈ L 2 ( S d -1 ) , if f = ∑ ∞ glyph[lscript] =0 f glyph[lscript] is the unique decomposition of f over ⊕ ∞ glyph[lscript] =0 H glyph[lscript] ( S d -1 ) as per Lemma 2, then f glyph[lscript] is given by

Now we define a kernel operator, based on the low-degree Gegenbauer polynomials, which projects functions onto their low-degree spherical harmonic expansion.

Definition 2 (Projection Operator onto H ( q ) ( S d -1 ) ) . For any integers q ≥ 0 and d ≥ 2 , define the kernel operator K ( q ) d : L 2 ( S d -1 ) → L 2 ( S d -1 ) as follows: for f ∈ L 2 ( S d -1 ) and σ ∈ S d -1 ,

This is an integral operator with kernel function k q,d ( σ, w ) := ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | · P glyph[lscript] d ( 〈 σ, w 〉 ) .

Note that the operator K ( q ) d is self-adjoint and positive semi-definite. Moreover, using the reproducing property of this kernel we can establish that K ( q ) d is a projection operator.

Furthermore, by the addition theorem (Theorem 3), K ( q ) d is trace-class (i.e., the trace is finite and independent of the choice of basis) because:

By combining Theorem 3 and Lemma 2, and using the definition of the projection operator K ( q ) d , it follows that for any function f ∈ L 2 ( S d -1 ) with Hilbert sum decomposition f = ∑ ∞ glyph[lscript] =0 f glyph[lscript] , the lowdegree component f ( q ) = ∑ q glyph[lscript] =0 f glyph[lscript] ∈ H ( q ) ( S d -1 ) can be computed as f ( q ) = K ( q ) d f . Equivalently,

in order to learn f ( q ) , it suffices to solve the following least-squares regression problem,

If g ∗ is an optimal solution to the above regression problem then f ( q ) = K ( q ) d g ∗ . In the next claim we show that solving the least squares problem in Eq. (7), even to a coarse approximation, is sufficient to solve our interpolation problem (i.e., Problem 2):

Claim 2. For any f ∈ L 2 ( S d -1 ) , any integer q ≥ 0 , and any C ≥ 1 , if ˜ g ∈ L 2 ( S d -1 ) satisfies,

and if we let f ( q ) := K ( q ) d f , where K ( q ) d is defined as per Definition 2, then the following holds

Claim 2 shows that solving the regression problem in Eq. (7) approximately provides a solution to our spherical harmonics interpolation problem (Problem 2). But how can we solve this least-squares problem efficiently? Not only does the problem involve a possibly infinite dimensional parameter vector g , but the objective function also involves the continuous domain on the surface of S d -1 .

3.1 Randomized Discretization via Leverage Function Sampling

We solve the continuous regression in Eq. (7) by randomly discretizing the sphere S d -1 , thereby reducing our problem to a regression on a finite set of points w 1 , w 2 , . . . , w s ∈ S d -1 . In particular, we propose to sample points on S d -1 with probability proportional to the so-called leverage function , a specific distribution that has been widely applied in randomized algorithms for linear algebra problems on discrete matrices [LMP13]. We start with the definition of the leverage function for compact operators such as K ( q ) d :

Definition 3 (Leverage Function) . For integers q ≥ 0 and d > 0 , we define the leverage function of the operator K ( q ) d (see Definition 2) for every w ∈ S d -1 as follows,

Intuitively, τ q ( w ) is an upper bound of how much a function that is spanned by the eigenfunctions of the operator K ( q ) d can 'blow up' at w . The larger the leverage function τ q ( w ) implies the higher the probability we will be required to sample w . This ensures that our sample points well reflect any possibly significant components, or 'spikes', of the function. Ultimately, the integral ∫ S d -1 τ q ( w ) dw determines how many samples we require to solve the regression problem Eq. (7) to a given accuracy. It is a known fact that the leverage function integrates to the rank of the operator K ( q ) d (which is equal to the dimensionality of the Hilbert space H ( q ) ( S d -1 ) ). This ultimately allows us to achieve a ˜ O ( ∑ q glyph[lscript] =0 α glyph[lscript],d ) sample complexity bound for solving Problem 2. To compute the leverage function, we make use of the following useful alternative characterization of the leverage function.

Lemma 4 (Min Characterization of the Leverage Function) . For any w ∈ S d -1 , let τ q ( w ) be the leverage function (Definition 3) and define φ w ∈ L 2 ( S d -1 ) by φ w ( σ ) ≡ ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | P glyph[lscript] d ( 〈 σ, w 〉 ) . We have the following minimization characterization of the leverage function:

Using the min and max characterizations of the leverage function we can find upper and lower bounds on this function. Surprisingly, in this case the upper and lower bounds match, so we actually have an exact value for the leverage function.

Lemma 5 (Leverage Function is Constant) . The leverage function given in Definition 3 is equal to τ q ( w ) = ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | for every w ∈ S d -1 .

Algorithm 1 Efficient Spherical Harmonic Expansion

1: Input: accuracy parameter ε > 0 , integer q ≥ 0

2: Set s = c · ( β q,d log β q,d + β q,d /ε ) for sufficiently large fixed constant c

3: Sample i.i.d. random points w 1 , w 2 , . . . , w s from U ( S d -1 )

4: Compute K ∈ R s × s with K i,j = ∑ q glyph[lscript] =0 α glyph[lscript],d s ·| S d -1 | · P glyph[lscript] d ( 〈 w i , w j 〉 ) for i, j ∈ [ s ]

5: Compute f ∈ R s with f j = 1 √ s · f ( w j ) for j ∈ [ s ]

6: Solve the regression by computing z = K † f

7: Return: y ∈ H ( q ) ( S d -1 ) with y ( σ ) := ∑ q glyph[lscript] =0 α glyph[lscript],d √ s ·| S d -1 | · ∑ s j =1 z j · P glyph[lscript] d ( 〈 w j , σ 〉 ) for σ ∈ S d -1

We prove this lemma in Appendix C. The integral of the leverage function, which determines the total samples needed to solve our least-squares regression, is therefore equal to the dimensionality of the Hilbert space H ( q ) ( S d -1 ) .

Corollary 1. The leverage function defined in Definition 3 integrates to the dimensionality of the Hilbert space H ( q ) ( S d -1 ) , which we denote by β q,d , i.e.,

We now show that the leverage function can be used to randomly sample the points on the unit sphere to discretize the regression problem in Eq. (7) and solve it approximately.

Theorem 4 (Approximate Regression via Leverage Function Sampling) . For any ε > 0 , let s = c · ( β q,d log β q,d + β q,d ε ) , for sufficiently large fixed constant c , and let x 1 , x 2 , . . . , x s be i.i.d. uniform samples on S d -1 . Define the quasi-matrix P : R s → L 2 ( S d -1 ) as follows, for every v ∈ R d :

Also let f ∈ R s be a vector with f j := 1 √ s · f ( x j ) for j = 1 , 2 , . . . , s and let P ∗ be the adjoint of P . If ˜ g is an optimal solution to the least-squares problem ˜ g ∈ arg min g ∈ L 2 ( S d -1 ) ‖ P ∗ g -f ‖ 2 2 , then with probability at least 1 -10 -4 the following holds,

We prove Theorem 4 in Appendix C. This theorem shows that the function ˜ g obtained from solving the discretized regression problem provides an approximate solution to Eq. (7).

3.2 Efficient Solution for the Discretized Least-Squares Problem

In this section, we demonstrate how to apply Theorem 4 algorithmically to approximately solve the regression problem of Eq. (7). To achieve this, we leverage the kernel trick , following a similar approach to previous works such as [AKM + 19, CP19], which allows us to efficiently address the randomly discretized least squares problem as detailed in Algorithm 1. The associated guarantee for this approach is provided in Theorem 5.

Theorem 5 (Efficient Spherical Harmonic Interpolation) . Algorithm 1 returns a function y ∈ H ( q ) ( S d -1 ) such that, with probability at least 1 -10 -4 :

Suppose we can compute the Gegenbauer polynomial P glyph[lscript] d ( t ) at every point t ∈ [ -1 , 1] in constant time. Then Algorithm 1 queries the function f at s = O ( β q,d log β q,d + β q,d ε ) points on the sphere S d -1 and runs in O ( s 2 · d + s ω ) time. This algorithm evaluates y ( σ ) in O ( d · s ) time for any σ ∈ S d -1 .

We provide the proof of this theorem in Appendix D.

Remark 1 (Noise Robustness) . In Theorem 5, our method's robustness is demonstrated under a noise model where the function f is not strictly a low-degree spherical harmonic and may include high-degree components. In this scenario, the higher-degree components are considered as noise added to the input function.

However, our algorithm is robust against alternative noise models, particularly additive i.i.d. Normal noise that corrupts the function evaluations f in Algorithm 1 with iid Normal noise. More precisely, suppose that we observe samples from the function f ( q ) contaminated by Gaussian noise, i.e., f ( w j ) = f ( q ) ( w j ) + n j in Algorithm 1 for i.i.d. n 1 , n 2 , . . . n s ∼ N (0 , 1) . The expected value of perturbation's norm in the output y of our algorithm caused by this noise is:

By Markov's inequality, with 0 . 99 probability ∥ ∥ y -f ( q ) ∥ ∥ 2 S d -1 = O (1 /s ) · trace ( K † KK † ) . If we let P be the operator defined in Theorem 4, then we can see that K = P ∗ P . Using the properties of the trace, one can see that trace ( K † KK † ) = 1 /λ 1 +1 /λ 2 + . . . , where λ i 's are the singular values of the operator PP ∗ . By matrix Chernoff inequalities one can show that all singular values of the operator PP ∗ closely approximate the singular values of the projection operator K ( q ) d up to a constant factor. So, we have trace ( K † KK † ) = O (rank( K ( q ) d )) = O ( β q,d ) . Thus, because s ≥ Ω( β/ε ) and using union bound, with 0 . 98 probability:

4 Lower Bound on The Number of Required Observations

We conclude by showing that the dimensionality of the Hilbert space H ( q ) ( S d -1 ) tightly characterizes the sample complexity of Problem 2. Thus, our Theorem 5 is optimal up to a logarithmic factor. Intuitively, there are β q,d degrees of freedom for specifying a spherical harmonic. Consequently, any deterministic algorithm attempting to reconstruct such polynomials would need at least β q,d samples. We aim to demonstrate that even a 'randomized' algorithm, which succeeds with only a constant probability, must still gather β q,d samples. This complements our upper bound, which is established using a randomized algorithm. The crucial fact we use for proving the lower bound is that all (non-zero) eigenvalues of the operator K ( q ) d are equal to one. This fact follows from the addition theorem presented in Theorem 3, i.e., if { y glyph[lscript] 1 , y glyph[lscript] 2 , . . . , y glyph[lscript] α glyph[lscript],d } is an orthonormal basis of H glyph[lscript] ( S d -1 ) , then for any function f ∈ L 2 ( S d -1 ) ,

Theorem 6 (Lower Bound) . Consider an error parameter ε > 0 , and any (possibly randomized) algorithm that solves Problem 2 with probability greater than 1 / 10 for any input function f and makes at most r (possibly adaptive) queries on any input. Then r ≥ β q,d .

We describe a distribution on input function f on which any deterministic algorithm that takes r < β q,d samples fails with probability ≥ 9 / 10 . The theorem then follows by Yao's principle.

Hard Input Distribution. For integer glyph[lscript] ≤ q , consider an orthonormal basis of H glyph[lscript] ( S d -1 ) and denote it by { y glyph[lscript] 1 , y glyph[lscript] 2 , . . . , y glyph[lscript] α glyph[lscript],d } . Let Y glyph[lscript] : R α glyph[lscript],d → H glyph[lscript] ( S d -1 ) be the quasi-matrix with y glyph[lscript] j as its j th column, i.e., [ Y glyph[lscript] · u ]( σ ) := ∑ α glyph[lscript],d j =1 u j · y glyph[lscript] j ( σ ) for any u ∈ R α glyph[lscript],d and σ ∈ S d -1 . Let v (0) ∈ R α 0 ,d , v (1) ∈ R α 1 ,d , . . . , v ( q ) ∈ R α q,d be independent random vectors with i.i.d. Gaussian entries: v ( glyph[lscript] ) j ∼ N (0 , 1) . The random input is defined to be f := ∑ q glyph[lscript] =0 Y glyph[lscript] · v ( glyph[lscript] ) . In other words, f = ∑ q glyph[lscript] =0 Y glyph[lscript] · v ( glyph[lscript] ) is a random linear combination of the eigenfunctions of K ( q ) d . We prove that accurate

reconstruction of f drawn from the aforementioned distribution yields accurate reconstruction of random vectors v (0) , v (1) , . . . , v ( q ) . Since each v ( glyph[lscript] ) is α glyph[lscript],d -dimensional, this reconstruction requires Ω( ∑ q glyph[lscript] =0 α glyph[lscript],d ) = Ω( β q,d ) samples, giving us a lower bound for accurate reconstruction of f .

First we show that finding an ˜ f ( q ) satisfying the condition of Problem 2 is at least as hard as accurately finding all vectors v (0) , v (1) , . . . , v ( q ) . The following lemma is proved in Appendix E.

Lemma 6. If a deterministic algorithm solves Problem 2 with probability at least 1 / 10 over our random input distribution f = ∑ q glyph[lscript] =0 Y glyph[lscript] · v ( glyph[lscript] ) , then with probability at least 1 / 10 , the output of the algorithm ˜ f ( q ) satisfies Y ∗ glyph[lscript] ˜ f ( q ) = v ( glyph[lscript] ) for all integers glyph[lscript] ≤ q .

Finally, we complete the proof of Theorem 6 by arguing that if ˜ f ( q ) is formed using less than β q,d queries from f , then ∑ q glyph[lscript] =0 ∥ ∥ ∥ Y ∗ glyph[lscript] ˜ f ( q ) -v ( glyph[lscript] ) ∥ ∥ ∥ 2 2 > 0 with good probability, thus the bound of Lemma 6 cannot hold and ˜ f ( q ) cannot be a solution to Problem 2. Assume for sake of contradiction that there is a deterministic algorithm which solves Problem 2 with probability ≥ 1 / 10 over the random input f = ∑ q glyph[lscript] =0 Y glyph[lscript] · v ( glyph[lscript] ) that makes r = β q,d -1 queries on any input (we can modify an algorithm that makes fewer queries to make exactly β glyph[lscript],d -1 queries). For every σ ∈ S d -1 and integer glyph[lscript] ≤ q define u glyph[lscript] σ := [ y glyph[lscript] 1 ( σ ) , y glyph[lscript] 2 ( σ ) , . . . , y glyph[lscript] α glyph[lscript],d ( σ ) ] . Also define u σ := [ u 0 σ , u 1 σ , . . . , u q σ ] ∈ R β q,d and v ∈ R β q,d as v := ( v (0) , v (1) , . . . , v ( q ) ) . Additionally, define the quasi-matrix Y := [ Y 0 , . . . , Y q ] .

Using the above notations and the definition of the hard input instance f , each query to f is in fact a query to the random vector v in the form of f ( σ ) = 〈 u σ , v 〉 . Now consider a deterministic function Q , that is given input V ∈ R i × β q,d (for any positive integer i ) and outputs Q ( V ) ∈ R β q,d × β q,d such that Q ( V ) has orthonormal rows with the first i rows spanning the i rows of V . If σ 1 , σ 2 , . . . , σ r ∈ S d -1 denote the points where our algorithm queries the input f , for any integer i ∈ [ r ] , let Q i be an orthonormal matrix whose first i rows span the first i queries of the algorithm, i.e., Q i := Q ( [ u σ 1 , u σ 2 , . . . , u σ i ] glyph[latticetop] ) . Since the algorithm is deterministic, Q i is a deterministic function of input v . The following claim is proved in [AKM + 19]:

Claim 3 (Claim 23 of [AKM + 19]) . Conditioned on the queries f ( σ 1 ) , f ( σ 2 ) , . . . , f ( σ r ) for r < β q,d , the variable [ Q r · v ]( β q,d ) is distributed as N (0 , 1) .

Now using Claim 3 we can write,

where the expectation in the last line is taken over the randomness of f ( σ 1 ) , . . . , f ( σ r ) . Conditioned on f ( σ 1 ) , . . . , f ( σ r ) , [ Q r Y ∗ ˜ f ( q ) ] ( β q,d ) is a fixed quantity because the algorithm determines ˜ f ( q ) given the knowledge of the queries f ( σ 1 ) , . . . , f ( σ r ) . Furthermore, by Claim 3, [ Q r · v ]( β q,d ) is a random variable distributed as N (0 , 1) , conditioned on f ( σ 1 ) , . . . , f ( σ r ) . This implies that,

Thus, Pr [ ∑ q glyph[lscript] =0 ∥ ∥ ∥ v ( glyph[lscript] ) -Y ∗ glyph[lscript] ˜ f ( q ) ∥ ∥ ∥ 2 2 = 0 ] = E f ( σ 1 ) ,...,f ( σ r ) [0] = 0 . However, we have assumed that this algorithm solves Problem 2 with probability at least 1 / 10 , and hence, by Lemma 6, Pr [ ∑ q glyph[lscript] =0 ‖ v ( glyph[lscript] ) -Y ∗ glyph[lscript] ˜ f ( q ) ‖ 2 2 = 0 ] ≥ 1 / 10 . This is a contradiction, yielding Theorem 6.

5 Numerical Evaluation

Noise-free Setting. For a fixed q , we generate a random function f ( σ ) = ∑ q glyph[lscript] =0 c glyph[lscript] P glyph[lscript] d ( 〈 σ, v 〉 ) where v ∼ U ( S d -1 ) and c glyph[lscript] 's are i.i.d. samples from N (0 , 1) . Then, f is recovered by running Algorithm 1 with s random evaluations of f on S d -1 . Note that ‖K ( q ) d f -f ‖ S d -1 = 0 since f ∈ H ( q ) ( S d -1 ) , thus,

(a)

d

= 3

(b)

d

= 4

600

-1.0

600

2400

-1.0

2400

3

le

p

520

440

-0.8

β

3

9

520

440

3

p

1760

2080

-0.8

β

4

2080

1760

iS

360

-0.6

360

S

1440

0.6

0

n

1440

umber

200

120

280

0.2

0.4

dime

120

280

200

f

numb

1120

800

480

-0.4

0.2

气

S

E

e

1120

800

480

inu

40

0.0

40

160

0.0

160

6810121416182022

6810121416182022

6 810121416182022

6810121416182022

degree q

degree q

degreeq

degreeq

(a)

d

= 3

(b)

d

= 4

ples

S

520

600

440

noisescale10-6

-1.0

-0.8

errors

101

102

100

noise scale 10-6

q=7

noise

p

E

S

2400

1760

2080

noise scale 10-6

-1.0

-0.8

f

errors

10-1

102

100

noisescale10-6

noise

q=

360

280

0.4

0.6

norm

104

103

q=9

q=13

f

S

1440

1120

0.6

0.4

norm

103

10

—4

=b

=b

=

6

11

13

number

120

200

0.2

1

2

10-5

10

-0

q=15

q = 17

number

800

480

-0.2

2

105

106

q=

15

17

b0

10-7

40

6810121416182022

0.0

80

200

320

440

560

160

6810121416182022

0.0

480

9601440 19202400

degree q

number ofsampless

degreeq

numberofsampless

as shown in Theorem 4, Algorithm 1 can recover f 'exactly' using s = O ( β q,d log β q,d ) evaluations, where β q,d is the dimension of the Hilbert space H ( q ) ( S d -1 ) . We predict f 's value on a random test set on S d -1 and consider the algorithm fails if the testing error is greater than 10 -12 . We count the number of failures among 100 independent random trials with different choices of d ∈ { 3 , 4 } , q ∈ { 5 , . . . , 22 } , and s ∈ { 40 , . . . , 2400 } . The empirical success probabilities for d = 3 and 4 are reported in Fig. 1(a) and Fig. 1(b), respectively. Fig. 1 illustrates that the success probabilities of our algorithm sharply transition to 1 as soon as the number of samples approaches s ≈ β q,d for a wide range of q and both d = 3 , 4 . These experimental results complement our Theorem 4 along with the lower bound analysis in Section 4 and empirically verify the performance of our algorithm.

Noisy Setting. We repeated our experiments in the presence of an additive noise which is a linear combination of random spherical harmonics of degrees q +1 to 2 q . More precisely, we let the noise be n ( σ ) = ∑ 2 q glyph[lscript] = q +1 c glyph[lscript] P glyph[lscript] d ( 〈 v, σ 〉 ) ∈ H (2 q ) ( S d -1 ) \ H ( q ) ( S d -1 ) for c glyph[lscript] 's that are i.i.d. samples from N (0 , 1) . We then re-scale the noise to have norm ‖ n ‖ S d -1 = 10 -6 . Furthermore, the function f is defined as before, and f is recovered by Algorithm 1 with s random evaluations of f + n on S d -1 . The heat-maps in Fig. 2 are generated by considering an instance of our algorithm as a 'success' if the error's energy is below the noise level, ∥ ∥ ∥ ˜ f ( q ) -f ∥ ∥ ∥ S d -1 ≤ ‖ n ‖ S d -1 = 10 -6 . The success probability transitions less sharply than the noiseless setting but the shift of probabilities starts at β s,q samples.

6 Conclusion

We studied the problem of robustly recovering spherical harmonic expansion of a function defined on the sphere. The number of function evaluations needed to recover its degreeq expansion is the dimension of spherical harmonics of degree at most q , up to a logarithmic factor. We develop a simple yet efficient kernel regression-based algorithm to recover degreeq expansion of the function by only evaluating the function on uniformly sampled points on the sphere. Unlike the prior results on fast spherical harmonic transform, our algorithm works efficiently using a nearly optimal number of samples in any dimension. We believe our findings would appeal to the readership of the community.

References

[AH12] Kendall Atkinson and Weimin Han. Spherical harmonics and approximations on the unit sphere: an introduction . Springer Science & Business Media, 2012.

[AKK + 20] Thomas D Ahle, Michael Kapralov, Jakob BT Knudsen, Rasmus Pagh, Ameya Velingker, David P Woodruff, and Amir Zandieh. Oblivious sketching of high-degree polynomial kernels. In Symposium on Discrete Algorithms (SODA) , 2020.

[AKM + 17] Haim Avron, Michael Kapralov, Cameron Musco, Christopher Musco, Ameya Velingker, and Amir Zandieh. Random Fourier features for kernel ridge regression: Approximation bounds and statistical guarantees. In International Conference on Machine Learning (ICML) , 2017.

[AKM + 19] Haim Avron, Michael Kapralov, Cameron Musco, Christopher Musco, Ameya Velingker, and Amir Zandieh. A universal sampling method for reconstructing signals with simple fourier transforms. In Symposium on the Theory of Computing (STOC) , 2019.

[BKM + 23] Karl Bringmann, Michael Kapralov, Mikhail Makarov, Vasileios Nakos, Amir Yagudin, and Amir Zandieh. Traversing the fft computation tree for dimension-independent sparse fourier transforms. In Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 4768-4845. SIAM, 2023.

[CKPS16] Xue Chen, Daniel M Kane, Eric Price, and Zhao Song. Fourier-sparse interpolation without a frequency gap. In Foundations of Computer Science (FOCS) , 2016.

[CLL15] Pui Tung Choi, Ka Chun Lam, and Lok Ming Lui. FLASH: Fast landmark aligned spherical harmonic parameterization for genus-0 closed brain surfaces. SIAM Journal on Imaging Sciences , 2015.

[CP19] Xue Chen and Eric Price. Active Regression via Linear-Sample Sparsification. In Conference on Learning Theory (COLT) , 2019.

[EEHM17] Michael Eickenberg, Georgios Exarchakis, Matthew Hirn, and Stéphane Mallat. Solid harmonic wavelet scattering: Predicting quantum molecular energy from invariant descriptors of 3d electronic densities. Advances in Neural Information Processing Systems , 30, 2017.

[EMM20] Tamás Erdélyi, Cameron Musco, and Christopher Musco. Fourier sparse leverage scores and approximate kernel learning. Neural Information Processing Systems (NeurIPS) , 2020.

[GCL + 22] Jan Gerken, Oscar Carlsson, Hampus Linander, Fredrik Ohlsson, Christoffer Petersson, and Daniel Persson. Equivariance versus augmentation for spherical images. In International Conference on Machine Learning , pages 7404-7421. PMLR, 2022.

[GMMM21] B Ghorbani, S Mei, T Misiakiewicz, and A Montanari. Linearized two-layers neural networks in high dimension. The Annals of Statistics , 2021.

[HZA22] Insu Han, Amir Zandieh, and Haim Avron. Random Gegenbauer Features for Scalable Kernel Methods. In International Conference on Machine Learning , pages 8330-8358. PMLR, 2022.

[HZL + 22] Insu Han, Amir Zandieh, Jaehoon Lee, Roman Novak, Lechao Xiao, and Amin Karbasi. Fast neural kernel embeddings for general activations. Advances in neural information processing systems , 35:35657-35671, 2022.

[KFR03] Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz. Rotation invariant spherical harmonic representation of 3 d shape descriptors. In Symposium on Geometry Processing , 2003.

[KKS97] Marc Kamionkowski, Arthur Kosowsky, and Albert Stebbins. Statistics of cosmic microwave background polarization. Physical Review D , 1997.

[KVZ19] Michael Kapralov, Ameya Velingker, and Amir Zandieh. Dimension-independent sparse fourier transform. In Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 2709-2728. SIAM, 2019.

[Lan12] Serge Lang. Real and functional analysis . Springer Science & Business Media, 2012.

[LMP13] Mu Li, Gary L Miller, and Richard Peng. Iterative row sampling. In Foundations of Computer Science (FOCS) , 2013.

[LP61] Henry J Landau and Henry O Pollak. Prolate spheroidal wave functions, Fourier analysis and uncertainty-II. Bell System Technical Journal , 1961.

[LP62] Henry J Landau and Henry O Pollak. Prolate spheroidal wave functions, fourier analysis and uncertainty-III: the dimension of the space of essentially time-and band-limited signals. Bell System Technical Journal , 1962.

[LWL + 21] Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations , 2021.

[MNY06] Ha Quang Minh, Partha Niyogi, and Yuan Yao. Mercer's theorem, feature maps, and smoothing. In Conference on Learning Theory (COLT) , 2006.

[Mor98] Mitsuo Morimoto. Analytic functionals on the sphere . American Mathematical Soc., 1998.

[Pen30] WOPennell. A generalized Fourier series representation of a function. The American Mathematical Monthly , 1930.

[RT06] Vladimir Rokhlin and Mark Tygert. Fast algorithms for spherical harmonic expansions. SIAM Journal on Scientific Computing (SISC) , 2006.

[SA22] Paz Fink Shustin and Haim Avron. Semi-Infinite Linear Regression and Its Applications. SIAM Journal on Matrix Analysis and Applications (SIMAX) , 2022.

[SP61] David Slepian and Henry O Pollak. Prolate spheroidal wave functions, Fourier analysis and uncertainty-I. Bell System Technical Journal , 1961.

[SS00] Paul N Swarztrauber and William F Spotz. Generalized discrete spherical harmonic transforms. Journal of Computational Physics , 2000.

[ST02] Reiji Suda and Masayasu Takami. A fast spherical harmonics transform algorithm. Mathematics of Computation , 2002.

[Wei95] DR Weimer. Models of high-latitude electric potentials derived with a least error fit of spherical harmonic coefficients. Journal of Geophysical Research: Space Physics , 1995.

[Wer97] Robert A Werner. Spherical harmonic coefficients for the potential of a constant-density polyhedron. Computers & Geosciences , 1997.

[Wil12] Virginia Vassilevska Williams. Multiplying matrices faster than Coppersmith-Winograd. In Symposium on the Theory of Computing (STOC) , 2012.

[WZ22] David Woodruff and Amir Zandieh. Leverage score sampling for tensor product matrices in input sparsity time. In International Conference on Machine Learning , pages 23933-23964. PMLR, 2022.

[XRY01] Hong Xiao, Vladimir Rokhlin, and Norman Yarvin. Prolate spheroidal wavefunctions, quadrature and interpolation. Inverse problems , 2001.

[ZDK + 22] Larry Zitnick, Abhishek Das, Adeesh Kolluru, Janice Lan, Muhammed Shuaibi, Anuroop Sriram, Zachary Ulissi, and Brandon Wood. Spherical channels for modeling atomic interactions. Advances in Neural Information Processing Systems , 35:80548067, 2022.

[ZHA + 21] Amir Zandieh, Insu Han, Haim Avron, Neta Shoham, Chaewon Kim, and Jinwoo Shin. Scaling Neural Tangent Kernels via Sketching and Random Features. In Neural Information Processing Systems (NeurIPS) , 2021.

A Properties of Gegenbauer Polynomials and Spherical Harmonics

In this section we prove the basic properties of the Gegenbauer Polynomials as well as the Spherical Harmonics and establish the connection between the two. We start by the direct sum decomposition of the Hilbert space L 2 ( S d -1 ) in terms of the spherical harmonics,

Lemma 2 (Direct Sum Decomposition of L 2 ( S d -1 ) ) . The family of spaces H glyph[lscript] ( S d -1 ) yields a Hilbert space direct sum decomposition L 2 ( S d -1 ) = ⊕ ∞ glyph[lscript] =0 H glyph[lscript] ( S d -1 ) : the summands are closed and pairwise orthogonal, and every f ∈ L 2 ( S d -1 ) is the sum of a converging series (in the sense of mean-square convergence with the L 2 -norm defined in Eq. (2) ),

where f glyph[lscript] ∈ H glyph[lscript] ( S d -1 ) are uniquely determined functions. Furthermore, given any orthonormal basis { y glyph[lscript] 1 , y glyph[lscript] 2 , . . . , y glyph[lscript] α glyph[lscript],d } of H glyph[lscript] ( S d -1 ) we have f glyph[lscript] = ∑ α glyph[lscript],d j =1 〈 f, y glyph[lscript] j 〉 S d -1 · y glyph[lscript] j .

Proof. This is in fact a standard result. For example, see [Lan12] for a proof.

Now we show that the Gegenbauer polynomials and spherical harmonics are related through the so called the addition theorem,

Theorem 3 (Addition Theorem) . For every integer glyph[lscript] ≥ 0 , if { y glyph[lscript] 1 , y glyph[lscript] 2 , . . . , y glyph[lscript] α glyph[lscript],d } is an orthonormal basis for H glyph[lscript] ( S d -1 ) , then for any σ, w ∈ S d -1 we have

Proof. The result can be proven analytically, using the properties of the Poisson kernel in the unit ball. This is classic and the proof can be found in [AH12, Theorem 2.9].

Next we show that the Gegenbauer kernels can project any function into the space of their corresponding spherical harmonics,

Lemma 3 (A Reproducing Kernel for H glyph[lscript] ( S d -1 ) ) . For every f ∈ L 2 ( S d -1 ) , if f = ∑ ∞ glyph[lscript] =0 f glyph[lscript] is the unique decomposition of f over ⊕ ∞ glyph[lscript] =0 H glyph[lscript] ( S d -1 ) as per Lemma 2, then f glyph[lscript] is given by

Proof. This is a classic textbook result, see [Mor98].

Now we prove that the Gegenbauer kernels satisfy the reproducing property for the Hilbert space H glyph[lscript] ( S d -1 ) .

Lemma 1 (Reproducing Property of Zonal Harmonics) . Let P glyph[lscript] d ( · ) be the Gengenbauer polynomial of degree glyph[lscript] in dimension d . For any x, y ∈ S d -1 :

glyph[negationslash]

Furthermore, for any glyph[lscript] ′ = glyph[lscript] :

glyph[negationslash]

Proof. This result follows directly from the Funk-Hecke formula (See [AH12]). However, we provide another proof here. First note that for every x ∈ S d -1 the function P glyph[lscript] d ( 〈 x, ·〉 ) ∈ H glyph[lscript] ( S d -1 ) . Therefore the first claim follow by applying Lemma 3 on function f ( σ ) = P glyph[lscript] d ( 〈 x, σ 〉 ) which also satisfies f glyph[lscript] = f . On the other hand, P glyph[lscript] ′ d ( 〈 y, ·〉 ) ∈ H glyph[lscript] ′ ( S d -1 ) for every y ∈ S d -1 . Thus, for glyph[lscript] ′ = glyph[lscript] , using the fact that spherical harmonics are orthogonal spaces of functions, P glyph[lscript] ′ d ( 〈 y, ·〉 ) ⊥ H glyph[lscript] ( S d -1 ) , which gives the second claim.

Next we prove that the kernel operator defined in Definition 2 is in fact a projection operator,

Proof. For every f ∈ L 2 ( S d -1 ) and every σ ∈ S d -1 , using Definition 2 we have,

where the fourth line above follows from Lemma 1. This proves the claim.

B Reducing the Interpolation Problem to a Least-Squares Regression

In this section we show that our spherical harmonic interpolation problem, i.e., Problem 2, can be solved by approximately solving a least-squares problem as claimed in Claim 2. We start by showing that for any function f ∈ L 2 ( S d -1 ) , K ( q ) d f gives its low-degree component. More precisely, let f = ∑ ∞ glyph[lscript] =0 f glyph[lscript] be the decomposition of f over the Hilbert sum ⊕ ∞ glyph[lscript] =0 H glyph[lscript] ( S d -1 ) as per Lemma 2. Now if we let K ( q ) d be the kernel operator from Definition 2 and if { y glyph[lscript] 1 , y glyph[lscript] 2 , . . . , y glyph[lscript] α glyph[lscript],d } is an orthonormal basis for H glyph[lscript] ( S d -1 ) , then by Theorem 3 we have,

where the the second line above follows from Theorem 3, the fourth line follows from Eq. (2), and the last line follows from Lemma 2. This proves that the low-degree component f ( q ) = K ( q ) d f .

Claim 2. For any f ∈ L 2 ( S d -1 ) , any integer q ≥ 0 , and any C ≥ 1 , if ˜ g ∈ L 2 ( S d -1 ) satisfies,

and if we let f ( q ) := K ( q ) d f , where K ( q ) d is defined as per Definition 2, then the following holds

Proof. First, note that g ∗ = f is an optimal solution to the least-squares problem in Eq. (7). Thus we have,

Next, we can write,

where the third line follows from the Pythagorean theorem because K ( q ) d (˜ g -f ) ∈ H ( q ) ( S d -1 ) while K ( q ) d f -f = -∑ glyph[lscript]>q f glyph[lscript] , thus ( K ( q ) d f -f ) ⊥ H ( q ) ( S d -1 ) . Combining the two equalities above with inequality ∥ ∥ ∥ K ( q ) d ˜ g -f ∥ ∥ ∥ 2 S d -1 ≤ C · min g ∈ L 2 ( S d -1 ) ∥ ∥ ∥ K ( q ) d g -f ∥ ∥ ∥ 2 S d -1 given in the statement of the claim, proves Claim 2.

C Approximate Regression via Leverage Score Sampling

In this section we ultimately prove our main result of Theorem 4. We start by proving useful properties of the leverage function given in Definition 3. First, we show the fact that the leverage function can be characterized in terms of a least-squares minimization problem, which is crucial for computing the leverage scores distribution. This fact was previously exploited in [AKM + 17] and [AKM + 19] in the context of Fourier operators.

Lemma 4 (Min Characterization of the Leverage Function) . For any w ∈ S d -1 , let τ q ( w ) be the leverage function (Definition 3) and define φ w ∈ L 2 ( S d -1 ) by φ w ( σ ) ≡ ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | P glyph[lscript] d ( 〈 σ, w 〉 ) . We have the following minimization characterization of the leverage function:

We remark that this lemma is in fact an adaptation and generalization of Theorem 5 of [AKM + 19]. We prove this lemma here for the sake of completeness.

Proof. First we show that the right hand side of Eq. (9) is never smaller than the leverage function in Definition 3. Let g ∗ w ∈ L 2 ( S d -1 ) be the optimal solution of Eq. (9) for any w ∈ S d -1 . Note that the

optimal solution satisfies K ( q ) d g ∗ w = φ w . Thus, for any function f ∈ L 2 ( S d -1 ) , using Definition 2, we can write

Therefore, for any f ∈ L 2 ( S d -1 ) with ∥ ∥ ∥ K ( q ) d f ∥ ∥ ∥ S d -1 > 0 , we have

We conclude the proof by showing that the maximum value is attained. First, we show that the optimal solution g ∗ w of Eq. (9) satisfies the property that K ( q ) d g ∗ w = g ∗ w . Suppose for the sake of contradiction that K ( q ) d g ∗ w = g ∗ w . In this case, Claim 1 implies that, glyph[negationslash]

Thus, the function g = K ( q ) d g ∗ w satisfies the constraint of the minimization problem in Eq. (9). Now, using the above and the fact that K ( q ) d is self-adjoint we can write,

This shows that K ( q ) d g ∗ w ⊥ ( K ( q ) d g ∗ w -g ∗ w ) , hence by Pythagorean theorem we have,

which is in contrast with the assumption that g ∗ w is the optimal solution of Eq. (9). Therefore, our claim that K ( glyph[lscript] ) d g ∗ w = g ∗ w holds.

Now, we show that for f = g ∗ w , the maximum value in inequality Eq. (11) is attained. For any w ∈ S d -1 we have the following

On the other hand we have ∥ ∥ ∥ K ( q ) d f ∥ ∥ ∥ 2 S d -1 = ‖ g ∗ w ‖ 2 S d -1 . Thus, ∥ ∥ ∥ K ( q ) d f ∥ ∥ ∥ -2 S d -1 · ∣ ∣ ∣ [ K ( q ) d f ] ( w ) ∣ ∣ ∣ 2 = ‖ g ∗ w ‖ 2 S d -1 which implies that τ q ( w ) = ‖ g ∗ w ‖ 2 S d -1 and thus proves the lemma.

Next we prove that the leverage function is constant.

Lemma 5 (Leverage Function is Constant) . The leverage function given in Definition 3 is equal to τ q ( w ) = ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | for every w ∈ S d -1 .

Proof. First we prove that τ q ( w ) ≤ ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | using the min-characterization. If we let φ w ∈ L 2 ( S d -1 ) be defined as φ w ( σ ) := ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | P glyph[lscript] d ( 〈 σ, w 〉 ) , then by Definition 2, for every σ ∈ S d -1

we can write,

where the third line above follows from Lemma 1. Therefore, the test function g := φ w satisfies the constraint of the minimization in Eq. (9), i.e., K ( q ) d g = φ w . Thus, Lemma 4 implies that,

where the equality above follows from Lemma 1 along with Eq. (2). This establishes the upper bound on the leverage function that we sought to prove.

Now, using the maximization characterization of the leverage function in Definition 3, we prove that τ q ( w ) ≥ ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | . Again, we consider the same test function g = φ w and write,

where the first and second line above follow from Eq. (12) and Lemma 1, respectively. Therefore, the max characterization of the leverage function in Definition 3 implies that,

This completes the proof of Lemma 5 and establishes that τ q ( w ) is uniformly equal to ∑ q glyph[lscript] =0 α glyph[lscript],d | S d -1 | .

To prove Theorem 4, we need to use prior results about solving linear systems in continuous setting via leverage score sampling. In particular, we use Theorem 6.3 from [CP19], which is restated below,

Theorem 7 (Theorem 6.3 of [CP19]) . Consider any dimension n linear space F of functions from a domain G to R . Let D be a distribution over G and f be some function from G to R . Also, define the norm with respect to D of any function h : G → R as ‖ h ‖ 2 D := E x ∼ D [ | h ( x ) | 2 ] and let y = arg min h ∈F ‖ h -f ‖ 2 D . Fix any distribution D ′ over G and let K D ′ := sup x ∈ G sup h ∈F { D ( x ) D ′ ( x ) · | h ( x ) | 2 ‖ h ‖ 2 D } .

For i.i.d. sample query points x 1 , x 2 , . . . x s ∼ D ′ and weights w i = D ( x i ) s · D ′ ( x i ) for i ∈ [ s ] , let the weighted ERM estimator ˜ f s be defined as ˜ f s := arg min h ∈F ∑ s i =1 w i · | h ( x i ) -f ( x i ) | 2 . For any ε > 0 and a sufficiently large fixed constant c , if the number of queries s ≥ c · ( K D ′ log n + K D ′ ε ) , then the weighted ERM estimator ˜ f s satisfies,

Now we are ready to prove Theorem 4. Our approach is to apply Theorem 7 to the space of degree q spherical harmonics H ( q ) ( S d -1 ) and use the fact that the leverage scores distribution of the operator K ( q ) d are uniform on the unit sphere S d -1 .

Theorem 4 (Approximate Regression via Leverage Function Sampling) . For any ε > 0 , let s = c · ( β q,d log β q,d + β q,d ε ) , for sufficiently large fixed constant c , and let x 1 , x 2 , . . . , x s be i.i.d. uniform samples on S d -1 . Define the quasi-matrix P : R s → L 2 ( S d -1 ) as follows, for every v ∈ R d :

Also let f ∈ R s be a vector with f j := 1 √ s · f ( x j ) for j = 1 , 2 , . . . , s and let P ∗ be the adjoint of P . If ˜ g is an optimal solution to the least-squares problem ˜ g ∈ arg min g ∈ L 2 ( S d -1 ) ‖ P ∗ g -f ‖ 2 2 , then with probability at least 1 -10 -4 the following holds,

Proof. We prove this theorem by invoking Theorem 7. To do so, we first let the space F of function from S d -1 to R be F := H ( q ) ( S d -1 ) . It is clear that the space of spherical harmonics is a linear space of functions because of the existence of the kernel operator K ( q ) d which is a projection operator onto H ( q ) ( S d -1 ) , so F satisfies the first precondition of Theorem 7. Additionally, the dimension of this space of functions is n = β q,d .

Also, let D be a uniform distribution over the unit sphere S d -1 . For this distribution, the norm defined in Theorem 7 satisfies ‖ h ‖ 2 S d -1 = | S d -1 | · ‖ h ‖ 2 D , for any h ∈ L 2 ( S d -1 ) . Moreover, let D ′ be a uniform distribution over the unit sphere S d -1 as well.

Now we show that for these choices of F , D, D ′ , the condition number K D ′ defined as per Theorem 7 is equal to β q,d . We can write,

where the second line above follows from the fact that D,D ′ are both equal to the uniform distribution over S d -1 and ‖ h ‖ 2 S d -1 = | S d -1 | · ‖ h ‖ 2 D . The third line above follows from the fact that any function h ∈ H ( q ) ( S d -1 ) can be expressed as h = K ( q ) d g for some g ∈ L 2 ( S d -1 ) because K ( q ) d is the projection operator onto H ( q ) ( S d -1 ) . The fourth line follows from Definition 3 and last line follows from Lemma 5.

Finally, in order to invoke Theorem 7, we can write that the weighted ERM estimator ˜ f s is equal to the following,

where the third line above uses the definition of f i = 1 √ s f ( x i ) , and the last line follows from the definition of adjoint of the quasi-matrix P . Therefore, the theorem follows by invoking Theorem 7.

D Efficient Algorithm for Spherical Harmonic Interpolation

In this section we prove our main theorem about our spherical harmonic interpolation algorithm.

Theorem 5 (Efficient Spherical Harmonic Interpolation) . Algorithm 1 returns a function y ∈ H ( q ) ( S d -1 ) such that, with probability at least 1 -10 -4 :

Suppose we can compute the Gegenbauer polynomial P glyph[lscript] d ( t ) at every point t ∈ [ -1 , 1] in constant time. Then Algorithm 1 queries the function f at s = O ( β q,d log β q,d + β q,d ε ) points on the sphere S d -1 and runs in O ( s 2 · d + s ω ) time. This algorithm evaluates y ( σ ) in O ( d · s ) time for any σ ∈ S d -1 .

Proof. First note that the random points w 1 , w 2 , . . . , w s in line 3 of Algorithm 1 are i.i.d. sample with uniform distribution on the surface of S d -1 . Therefore, we can invoke Theorem 4. More specifically, if we let P be the quasi-matrix defined in Theorem 4 corresponding to the random points w 1 , w 2 , . . . , w s sampled in line 3 and if we let f be the vector of function samples defined in line 5 of the algorithm, then with probability at least 1 -10 -4 , any optimal solution to the following least-squares problem

satisfies the following,

Now note that the least-squares problem in Eq. (13) has at least one optimal solution ˜ g which is in the eigenspace of the operator PP ∗ . More specifically, there exists a vector z ∈ R s such that ˜ g = P · z is an optimal solution for Eq. (13). Therefore, we can focus on finding this optimal solution by solving the following least-squares problem

and then letting ˜ g = P · z . This ˜ g is guaranteed to be an optimal solution for Eq. (13), thus it satisfies Eq. (14). We solve the above least-squares problem using the kernel trick. In fact we show that P ∗ P is equal to the kernel matrix K computed in line 4 of Algorithm 1. To see why, note that for any

i, j ∈ [ s ] we have,

where the fourth line above follows from Lemma 1. Therefore, we are interested in the optimal solution of the following least-squares problem

The least-squares solution to the above problem is z = K † f which is exactly what is computed in line 6 of the algorithm. Now note that, the function ˜ g = P · z satisfies Eq. (14). Because ˜ g = P · z ∈ H ( q ) ( S d -1 ) and because K ( q ) d is an orthonormal projection operator into H ( q ) ( S d -1 ) , we have K ( q ) d · ˜ g = ˜ g = P · z . This together with Eq. (14) imply that,

Now if we invoke Claim 2 with C = 1 + ε on the above inequality we find that,

Finally, one can easily see that the function y ∈ H ( q ) ( S d -1 ) that Algorithm 1 outputs in line 7 is exactly equal to y = P · z . This completes the accuracy bound of the theorem.

Runtime and Sample Complexity. these bounds follow from observing that:

s · d time is needed to generate w 1 , w 2 , . . . , w s in line 3 of the algorithm. To do this, we first generate random Gaussian points in R d and then project then onto S d -1 by normalizing them.

s 2 · d operations are needed to form the kernel matrix K in line 4 of the algorithm.

s queries to function f are needed to form the samples vector f in line 5 of the algorithm.

s ω time is needed to compute the least-squares solution z = K † f in line 6 of the algorithm.

s · d operations are needed to evaluate the output function y ( σ ) in line 7 of the algorithm.

This completes the proof of Theorem 5.

E Lower Bound: Claims and Lemmas

In this section we prove the Claims and Lemmas used in our lower bound analysis for proving Theorem 6.

Claim 4. Given the random input f = ∑ q glyph[lscript] =0 Y glyph[lscript] · v ( glyph[lscript] ) generated as described in Section 4, to solve Problem 2, an algorithm must return a function ˜ f ( q ) ∈ H ( q ) ( S d -1 ) such that ‖ ˜ f ( q ) -f ‖ 2 S d -1 = 0 .

Proof. Note that Problem 2 requires recovering a function ˜ f ( q ) ∈ H ( q ) ( S d -1 ) such that:

where f ( q ) = K ( q ) d f . Using the definition of the input function f = ∑ q glyph[lscript] =0 Y glyph[lscript] · v ( glyph[lscript] ) , we can write,

where the equality in the second line above follows from Eq. (10) and the addition theorem in Theorem 3, and the third line follows because the operator Y glyph[lscript] has orthonormal columns and thus Y ∗ glyph[lscript] ′ Y glyph[lscript] = I α glyph[lscript],d · { glyph[lscript] = glyph[lscript] ′ } . Therefore, plugging this into Eq. (15) gives,

Lemma 6. If a deterministic algorithm solves Problem 2 with probability at least 1 / 10 over our random input distribution f = ∑ q glyph[lscript] =0 Y glyph[lscript] · v ( glyph[lscript] ) , then with probability at least 1 / 10 , the output of the algorithm ˜ f ( q ) satisfies Y ∗ glyph[lscript] ˜ f ( q ) = v ( glyph[lscript] ) for all integers glyph[lscript] ≤ q .

Proof. By Claim 4, the output of the algorithm that solves Problem 2, satisfies ∥ ∥ ∥ ˜ f ( q ) -f ∥ ∥ ∥ 2 S d -1 = 0 . Therefore, by orthonormality of the columns of the operator Y glyph[lscript] , we can write,

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n### Result ###\n\nThe paper studies the problem of recovering a function from a finite number of noisy observations, for the class of \"square-integrable functions on the unit sphere\" (denoted by $L^2(\\mathbb{S}^{d-1})$ when the sphere in question is of degree $d$).\n\nThe paper shows, by developing an algorithm, that the number of samples required for recovery up to an $\\epsilon$-multiplicative error is proportional to $\\beta_{q, d}$, where $q$ is the degree of spherical harmonics desired and $\\beta_{q, d}$ is the dimension of the space of degree $q$ spherical harmonics on the sphere of dimension $d-1$. This sample complexity is optimal, as shown by the paper via lower bounds. \n\n---------------\n\n### Broader Contributions ###\n\nTo achieve the result above, the paper makes the following conceptual and technical contributions. \n\nFirst, the problem at hand is a regression problem with potentially infinite-dimensional continuous cost functions. Inspired by the approach of Avron, Kapralov, Musco, Musco, Velingker, and Zandieh (AKMMVZ19), which discretizes this problem via the leverage scores of the regression matrix. \n\nSecond, the authors utilize connections between spherical harmonics and zonal harmonics to enable the implementation of leverage score sampling of the regression matrix in question. This leverage score sampling result is novel.\n\nSTRENGTHS:\n### Result's Strength ### \n\nI think the optimality of the stated result of the paper makes the paper mathematically strong.  \n\n----------------------------------------\n\n# During the Rebuttal Phase # \n\nThe author's explanations, repeated readings of the paper, and Reviewer BcP8's review and questions have helped me understand and appreciate the paper better. I'm therefore raising the score from 4 to 6 and confidence from 2 to 3.\n\nWEAKNESSES:\n### Difficulty in reading. ### \n\nI found the paper quite difficult to read though I could tell it was written well. I think this is simply because of the deeply technical nature of the problem and my unfamiliarity with the topic (at least to the level of generality of this paper) within the context of machine learning. \n\nI am not sure if I have any concrete feedback to this end but due to this particular reason I feel this paper might be a much better fit (in terms of reaching a wide audience who'd actually understand and appreciate the results) at a mathematical/optimization journal. That said, I acknowledge, based on my unfamiliarity with the topic, that my judgement could be completely misplaced. \n\nI'll be grateful to the authors if they could situate their work more in the context of machine learning. I'll also be happy to keep reading the submission and improving my understanding through the rebuttal period.\n\n----------------------\n\nQUESTIONS:\n### Questions ### \n\n1. Would the authors be able to situate their results in the context of machine learning? For instance, in lines 20-21, the potential applications mentioned seem to be leaning towards physics/astronomy; it would be quite interesting to see such applications in machine learning (e.g., published in prior ICML/NeurIPS conferences) and also some that might be somewhat more recent. \n\n2. I'm actually slightly confused about the main result (Theorem 1 in the full version of the submission; lines 53-56 in the main_full.pdf) for the following reason: In \"Randomized algorithms for matrices and data\" by Mahoney (2011), we see that the (essentially optimal) sample complexity of least-squares regression is $d/\\epsilon^2$, obtained by leverage score sampling. Here, $d$ can be thought of as the \"intrinsic dimension\" of the projection matrix in question; I'm surprised the result in the paper (Theorem 1 in the full submission) seems to have a better dependence of $\\epsilon^{-1}$ on the sample complexity despite (seemingly) being a more general result. Would the authors be able to comment on this? \n\n--------------------------\n\n### Suggestions ### \n\n1.  I think it would really help with readability if the authors could present \"specific cases\" of their statements wherever possible. As an example, the Definition 3, Equation 8 is essentially the definition of the sensitivities function (introduced by Langberg and Schulman; for a recent paper with this fact explicitly stated, see, for example, \"Sharper Bounds for $\\ell_p$ Sensitivity Sampling\" by Woodruff and Yasuda); similarly, the minimum characterization of the leverage score function is essentially the one seen in Lemma 2 of \"Uniform Sampling for Matrix Approximation\" by Cohen, Lee, Musco, Musco, Peng, and Sidford). It would make it easier for readers to see these (possibly more familiar) statements first followed by the generalized versions stated in the paper.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA technique is proposed to recover spherical harmonic expansions for functions defined on a d-dimensional sphere from a set of function evaluations.  Spherical harmonic expansions are recovered by solving an optimizaiton problem via a kernal approach, which is accompanied by theoretical guarantees.  Numerical experiments demonstrate phase transitions close to the theoretical bounds.\n\nSTRENGTHS:\nA solid theoretical analysis is presented to derive a new approach to recovering spherical harmonic expansions from the evaluation of functions on the d-dimensional sphere.  It is shown that functions should be evalued uniformly randomly over the sphere.  The approach presented is accompanied by numerous theoretical results and guarantees.  Experimental results validate the theory presented, with phase transitions in success probabilities close to the theoretical bounds.\n\nWEAKNESSES:\nWhile the method is interesting, it does not seem that NeurIPS is the appropriate venue for this work.  While the field of deep learning on the sphere is an active area of research (e.g. [Cohen et al.](https://arxiv.org/abs/1801.10130)), this contribution would appear to be somewhat orthogonal to that body of literature.  No connection is made in the submitted manuscript beyond the final, somewhat cryptic, comment: \"We believe our finding would appeal to the readership of the community\".\n\nFurthermore, the connection to previous literature of the topic of fast spherical harmonic transforms and efficient sampling on the sphere is very poor.  No reference is made to the central works of [Driscoll & Healy](https://www.sciencedirect.com/science/article/pii/S0196885884710086) and [McEwen & Wiaux](https://ieeexplore.ieee.org/document/6006544) and the follow-up articles of their groups.\n\nThe use of the term \"near optimal\" in the title and throughout the article is somewhat overstated.  The proposed method is \"near\" optimal up to a logarithmic factor.  I would typically expect near optimal to be up to a constant factor.\n\nQUESTIONS:\nHow is the proposed method relevant for the field of deep learning on the sphere?  Can it offer a differentiable transform that can be integrated into approaches such as  [Cohen et al.](https://arxiv.org/abs/1801.10130)?\n\nWhat is meant by the comment: \"it is generally intractable to compute an orthogonal basis for the space of spherical harmonics, which renders the generalized Foureri series expansion in Lemma 2 primarily existential\"?  This comment is repeated twice in the manuscript. \n Could this please be elaborated?\n\nLIMITATIONS:\nNo special negative societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the approximation of a function $f \\in L_2(\\mathbb{S}^{d-1})$ from its evaluations, via a degree-q spherical harmonic expansion. To this end, an efficient kernel regression based algorithm is proposed which recovers such a degree-q expansion of f, from the evaluations of f on $\\mathbb{S}^{d-1})$. In particular, the number of evaluations needed scales nearly linearly in the dimension of the space of spherical harmonics of degree at most $q$. The main idea is to exploit connections between spherical harmonics and zonal harmonics, and the fact that the zonal harmonics are the reproducing kernels of the space of degree $l$ spherical harmonics. Some numerical simulations are provided on synthetic examples to demonstrate the performance of the algorithm.\n\nSTRENGTHS:\n1. The paper is written well overall with a well-defined problem statement, and a clear description of related work.\n\n\n2. The results hold for any dimension $d$ which does not seem to have been handled previously. As noted in the related work, previous results typically applied to small, fixed values of $d$. In that respect, I think the results are quite strong.\n\nWEAKNESSES:\n1.  It’s a bit unclear to me whether the results hold only in the noiseless case, or is the method actually robust to noise. For instance, if we the function evaluations are corrupted with iid centred Gaussian noise, what can be said about the recovery error? The abstract mentions that the algorithm provides robust recovery, but I am not sure if this is what is proven.\n\n2.  While I understand the main contributions are theoretical, are there any real examples on which the method can be evaluated? At the moment, experiments are only conducted on synthetical examples.\n\nQUESTIONS:\n1. In the optimization problem after line 42, isn’t the solution non-unique? If $g^*$ is a solution, then $g^* + g’$ for any g’ in the null space of the operator is also a solution? Also, isn’t f a solution of this optimization problem?\n\n2. Just to clarify my understanding, my first thought was to simply do linear regression in the basis of the lower degree spherical harmonics. But I suppose this is intractable since computing such a basis is computationally hard for even moderate values of q as noted in the paper. Is this correct? And so, this is why we can only hope for an approximate solution of Problem 1?\n\n3. In Definition 3, couldn’t we define the leverage function for any operator mapping $ L_2(\\mathbb{S}^{d-1})$ to itself? Or is it specifically defined for $\\mathcal{K}_d^{(q)}$?\n\nLIMITATIONS:\nI do not see the limitations discussed anywhere but this probably does not apply to this paper since it is essentially theoretical, and the sample complexity bounds are shown to be nearly optimal. Perhaps the running time which is super-quadratic in the number of samples could be improved upon?", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

comments: SUMMARY:
To develop kernel regression based algorithm to recover degree-q expansion of f \in L^2(S^{}d-1| - by only evaluating on uniformly sampled points on S^{d-1}.

STRENGTHS:
The ideas used in this paper is deeply technical and ideas are complicated. It first re-formulates the problem as least squares regression and then uses the sampling based on leverage scores to decide which samples to pick. Then the rest of the paper is proving the estimated bound is satisfied.

WEAKNESSES:
motivate the problem - how it could be applied in real life

** after rebuttal -- thank you for providing all these papers regarding contributions of this workstream in terms of application side. I am happy to increase my score regarding soundness of the paper.

QUESTIONS:
-

LIMITATIONS:
-
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nConsider the $d$-dimensional unit sphere $\\mathbb{S}^{d-1}$, and any function $f:\\mathbb{S}^{d-1}\\rightarrow\\mathbb{R}$ defined on the sphere with bounded L2 norm $\\|\\|f\\|\\|_{\\mathbb{S}^{d-1}}$. This function $f$ can be evaluated at any point $\\vec w \\in \\mathbb{S}^{d-1}$ on the sphere, but it is expensive to evaluate $f$ so we wish to do this as little as possible.\n\nThe goal of this paper is to recover a near-optimal polynomial $\\tilde f$ approximation to $f$, where $\\tilde f$ is constrained to be a multivariate polynomial where the sum-of-degrees of each term is at most $q$. (For example, the sum-of-degrees of each term in $\\tilde f(x,y,z) = x^2y^3 + z + x^5$ is at most 5).\n\nLetting $\\beta_{d,q}$ denote the number of free parameters in any $d$-variate polynomial with terms whose sum-of-degrees is at most $q$, this paper shows that $O(\\beta_{d,q} \\log \\beta_{d,q} + \\frac{\\beta_{d,q}}{\\varepsilon})$ evaluations of $f$ chosen uniformly at random on the sphere suffice to recover such a near-optimal $f$. This is proven in three parts:\n1. The leverage function associated with this polynomial recovery problem is **exactly** the uniform distribution on the sphere\n2. The randomized linear algebra toolkit shows this near-linear sample complexity suffices\n3. A small kernel ridge regression problem can be solved to recover this $\\tilde f$ from uniform random samples on the sphere\n\nThey also prove an $\\Omega(\\beta_{d,q})$ sample complexity lower bound. Lastly, they include some synthetic experiments.\n\nSTRENGTHS:\nThe paper is a nice application of well understood tools from randomized linear algebra (RandNLA) to spherical harmonics, which has not been explored by the RandNLA literature afaik. The real core strength of this paper is its novelty in connecting these two literatures in a simple and elegant way.\n\nSimplicity and elegance really are words that mark this paper. Almost everything is extremely clear and well written. There are basically no typos even! The flow of logic in the paper is very clear, the problem they solve seems important in the spherical harmonics literature (though I'm no expert in that domain), and the proofs are even pretty clean. To that last point, the theorems in this paper are proven either by very clean techniques that are now standard in the RandNLA literature, or by using well established classical facts about polynomials and spherical harmonics. I verified a decent amount of the math, and it all struck me as rather nice and clean.\n\nSince I'm not an expert in spherical harmonics, I can't perfectly speak to the significance of the result. Taking the authors' words at face value, it seems that prior work in spherical harmonics did not use such simple algorithms (Kernel Ridge Regression) and did not achieve optimal sample complexities.\n\nThe paper may not carry a huge amount of new technical ideas to get their sample complexity and simple algorithm, but this result would only exist if someone who knew enough about both spherical harmonics and RandNLA decided to sit down and figure out if this all works together. For that view of novelty, in addition to the simplicity and elegance of the paper, I recommend accepting this paper.\n\nThe experiments are perfectly fine for a theoretical paper. Nothing particularly strong about them, but nothing I'm left looking for.\n\nWEAKNESSES:\nThe novelty, simplicity, and elegance is strong. But, the proof techniques are not especially novel. The proof techniques are standard relationships between operators and algorithms as explored by the RandNLA community for a good few years now.\n\nThe only proof that doesn't seem directly tied to something already proven in the RandNLA literature is the proof that the leverage function for the regression problem is uniform on the sphere. Even then, this claim is pretty intuitive, since the problem statement is rotationally invariant. There's no reason for a sampling algorithm to care more about one point of the sphere than another. (This is in contrast to learning on an interval, where an algorithm may prefer to sample near the edges of the interval.) So, even this most novel proof is not terribly surprising, and the proof is short and simple.\n\nThat said, I hesitate to really call this a \"weakness\". This isn't clearly a downside of the paper. While it's certainly nice for a paper to overcome new technical issues and proof difficulties, it's also nice for a paper to show that spherical harmonics smoothly fit into the RandNLA framework.\n\nAll this is to say that the technical weight of the paper truly is in understanding two literatures and connecting them. The effort to connect the literatures seems to be low, but that's given that someone actually understands both literatures well. I'd say this overall comes out as a slight strength for the paper, but it's certainly a tradeoff in the reviewing process.\n\nQUESTIONS:\nI list minor questions, typos, and recommended edits here. These are all soft recommendations, adapt whatever you want to.\n\n1. [Line 53] Consider mentioning a failure probability here.\n1. [Line 87] Replace \"this paper\" with \"[GMMM21]\" since the language is a bit too ambiguous at a glance.\n1. [Line 173] Is this not a bijection? At least for traditional least squares regression, this seems to be a bijection? RandNLA people tend to use the former form, so it's nice to see if the RandNLA form is exactly equivalent to the form of Problem 2.\n1. [Line 213] Consider removing the $\\cdot$ between P and v? This notation feels a bit clunky and odd to me?\n1. [Line 220] Cite [AKM+] or [SA] or [CP from COLT19] here, or some other paper that draws a connection between the semi-infinite regression and kernel ridge regression. This section reads like the idea of the kernel trick here is a new contribution.\n1. [Line 226] Add \"Then,\" before \"Algorithm 1\"\n1. [Line 236] Discuss why this isn't trivial that we need $\\beta_{d,q}$ samples to learn $\\beta_{d,q}$ parameters in a polynomial. Certainly, when we're looking at interpolating a polynomial on the real line, we need at least $q+1$ samples to learn a degree $q$ polynomial exactly. This feels similar to the argument made in this lower bound, but I don't understand what the rest of the formalization is needed for.\n1. [Line 250] Replace \"accurately\" with \"exactly\" (I think that's more technically accurate?)\n1. [Lines 263-274] I got really lost here. I think that If I'd read the lower bound of AKM, then I might be able to stitch together the parts of this lower bound, but the reason any of this construction is made is unclear to me. The notation is also pretty hard to track. I don't really get what the construction is trying to get at. I don't really see why the top rows of Q need to span the queries made so far. This all feels odd to me.\n1. You don't need to do this. But, if you want to, I think you could make a lower bound of $\\Omega(\\frac1\\varepsilon)$ by following the technique in Section 5.4 of [here](https://arxiv.org/pdf/2211.06790.pdf). It'd be interesting to see if this technique holds on the sphere, because at a glance it seems like it should. It'd give an overall lower bound of only $\\Omega(\\beta_{d,q} + \\frac1{\\varepsilon})$ though, which isn't the most compelling rate.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-27"}