SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lisheng Ren
University of Wisconsin-Madison lren29@wisc.edu
University of Wisconsin-Madison yxsun@cs.wisc.edu
We study the complexity of Non-Gaussian Component Analysis (NGCA) in the Statistical Query (SQ) model. Prior work developed a general methodology to prove SQ lower bounds for this task that have been applicable to a wide range of contexts. In particular, it was known that for any univariate distribution  $A$  satisfying certain conditions, distinguishing between a standard multivariate Gaussian and a distribution that behaves like  $A$  in a random hidden direction and like a standard Gaussian in the orthogonal complement, is SQ-hard. The required conditions were that (1)  $A$  matches many low-order moments with the standard univariate Gaussian, and (2) the chi-squared norm of  $A$  with respect to the standard Gaussian is finite. While the moment-matching condition is necessary for hardness, the chi-squared condition was only required for technical reasons. In this work, we establish that the latter condition is indeed not necessary. In particular, we prove near-optimal SQ lower bounds for NGCA under the moment-matching condition only. Our result naturally generalizes to the setting of a hidden subspace. Leveraging our general SQ lower bound, we obtain near-optimal SQ lower bounds for a range of concrete estimation tasks where existing techniques provide sub-optimal or even vacuous guarantees.
Non-Gaussian Component Analysis (NGCA) is a statistical estimation task first considered in the signal processing literature  $\left[\mathrm{BKS}^{+}06\right]$ . As the name suggests, the objective is to find a non-gaussian direction (or, more generally, low-dimensional subspace) in a high-dimensional dataset. Since its introduction, the NGCA problem has been studied in a range of works from an algorithmic standpoint; see [TV18, GS19] and references therein. Here we explore this problem from a hardness perspective in the Statistical Query (SQ) model. Before we motivate and state our results, we require basic background on the SQ model.
SQ Model SQ algorithms are a class of algorithms that are allowed to query expectations of bounded functions on the underlying distribution through the SQ oracle rather than directly access samples. The model was introduced by Kearns [Kea98] as a natural restriction of the PAC model [Val84] in the context of learning Boolean functions. Since then, the SQ model has been extensively studied in a range of settings, including unsupervised learning [Fel16]. The class of SQ algorithms is broad and captures a range of known algorithmic techniques in machine learning in-
cluding spectral techniques, moment and tensor methods, local search (e.g., EM), and many others (see, e.g., [FGR $^{+}$ 17, FGV17] and references therein).
Definition 1.1 (SQ Model). Let  $D$  be a distribution on  $\mathbb{R}^n$ . A statistical query is a bounded function  $q: \mathbb{R}^n \to [-1,1]$ . We define  $\mathrm{STAT}(\tau)$  to be the oracle that given any such query  $q$ , outputs a value  $v$  such that  $|v - \mathbf{E}_{\mathbf{x} \sim D}[q(\mathbf{x})]| \leq \tau$ , where  $\tau > 0$  is the tolerance parameter of the query. A statistical query (SQ) algorithm is an algorithm whose objective is to learn some information about an unknown distribution  $D$  by making adaptive calls to the corresponding  $\mathrm{STAT}(\tau)$  oracle.
The following family of high-dimensional distributions forms the basis for the definition of the NGCA problem.
Definition 1.2 (Hidden-Subspace Distribution). For a distribution  $A$  supported on  $\mathbb{R}^m$  and a matrix  $\mathbf{V}\in \mathbb{R}^{n\times m}$  with  $\mathbf{V}^{\intercal}\mathbf{V} = \mathbf{I}_m$ , we define the distribution  $\mathbf{P}_{\mathbf{V}}^{A}$  supported on  $\mathbb{R}^n$  such that it is distributed according to  $A$  in the subspace  $\mathrm{span}(\mathbf{v}_1,\dots ,\mathbf{v}_m)$  and is an independent standard Gaussian in the orthogonal directions, where  $\mathbf{v}_1,\ldots ,\mathbf{v}_m$  denote the column vectors of  $\mathbf{V}$ . In particular, if  $A$  is a continuous distribution with probability density function  $A(\mathbf{y})$ , then  $\mathbf{P}_{\mathbf{V}}^{A}$  is the distribution over  $\mathbb{R}^n$  with probability density function
$$
\mathbf {P} _ {\mathbf {V}} ^ {A} (\mathbf {x}) = A \left(\left\langle \mathbf {v} _ {1}, \mathbf {x} \right\rangle , \dots , \left\langle \mathbf {v} _ {m}, \mathbf {x} \right\rangle\right) \exp \left(- \| \mathbf {x} - \mathbf {V V} ^ {\mathsf {T}} \mathbf {x} \| _ {2} ^ {2} / 2\right) / (2 \pi) ^ {(n - m) / 2}.
$$
That is,  $\mathbf{P}_{\mathbf{V}}^{A}$  is the product distribution whose orthogonal projection onto the subspace of  $\mathbf{V}$  is  $A$  and onto the subspace perpendicular to  $\mathbf{V}$  is the standard  $(n - m)$ -dimensional normal distribution. An important special case of the above definition considers to  $m = 1$  (i.e., the case when the hidden subspace is a hidden direction); for this setting, we will use the notation  $\mathbf{P}_{\mathbf{v}}^{A}$  for such a distribution, where  $A$  is a one-dimensional distribution and  $\mathbf{v} \in \mathbb{R}^n$  is a unit vector.
Since we are focusing on establishing hardness, we will consider the following hypothesis testing version of NGCA (since the learning/search version typically reduces to the testing problem). We use  $\mathcal{N}_n$  to denote the standard  $n$ -dimensional Gaussian distribution  $\mathcal{N}(0,\mathbf{I}_n)$ . We use  $U(\mathbf{O}_{n,m})$  to denote the uniform distribution over the set of all orthogonal matrices  $\mathbf{V}\in \mathbb{R}^{n\times m}$ ; namely, this is the distribution obtained by taking  $\mathbf{RV}'$ , where  $\mathbf{R}\in \mathbb{R}^{n\times n}$  is a random rotation matrix and  $\mathbf{V}'\in \mathbb{R}^{n\times m}$  is an arbitrary orthogonal matrix.
Definition 1.3 (Hypothesis Testing Version of NGCA). Let  $n > m \geq 1$  be integers. For a distribution  $A$  supported on  $\mathbb{R}^m$ , one is given access to a distribution  $D$  such that either:
-  $H_0\colon D = \mathcal{N}_n$
-  $H_{1}$ :  $D$  is given by  $\mathbf{P}_{\mathbf{V}}^{A}$ , where  $\mathbf{V} \sim U(\mathbf{O}_{n,m})$ .
The goal is to distinguish between these two cases  $H_0$  and  $H_1$ .
For the special case that  $m = 1$  (i.e., for a univariate distribution  $A$ ), prior work [DKS17] established SQ-hardness of NGCA $^{1}$  under the following condition:
Condition 1.4. Let  $d \in \mathbb{Z}_+$ . The distribution  $A$  on  $\mathbb{R}$  is such that (i) the first  $d$  moments of  $A$  agree with the first  $d$  moments of  $\mathcal{N}(0,1)$ , and (ii) the chi-squared distance  $\chi^2(A,\mathcal{N})$  is finite, where the chi-squared distance for two distributions (with probability density functions)  $P,Q: \mathbb{R}^n \to \mathbb{R}_+$  is defined as  $\chi^2(P,Q) \stackrel{\mathrm{def}}{=} \int_{\mathbf{x} \in \mathbb{R}^n} P(\mathbf{x})^2 / Q(\mathbf{x}) d\mathbf{x} - 1$ .
Specifically, the main result of [DKS17] shows that any SQ algorithm that solves the testing version of NGCA requires either  $2^{n^{\Omega(1)}}$  many SQ queries or at least one query with accuracy
$$
n ^ {- \Omega (d)} \sqrt {\chi^ {2} (A , \mathcal {N} (0 , 1))}.
$$
It is worth noting that subsequent works (see [DK22a] and [DKPZ21]) generalized this result so that it only requires that (i)  $A$  approximately matches moments with the standard Gaussian, and (ii)  $A$  is a low-dimensional distribution embedded in a hidden low-dimensional subspace, instead of a one-dimensional distribution.
<sup>1</sup>While the SQ lower bound result of [DKS17] was phrased for the search version of NGCA, they can be directly translated to the testing version; see, e.g., Chapter 8 of [DK23].
The starting point of our investigation is a key technical limitation of this line of work. Specifically, if  $\chi^2 (A,\mathcal{N}(0,1))$  is very large (or infinite), e.g., if  $A$  has constant probability mass on a discrete set, the aforementioned SQ lower bound of [DKS17] can be very weak (or even vacuous). It is thus natural to ask if the finite chi-squared assumption is in fact necessary for the corresponding SQ lower bounds to hold.
A concrete motivation to answer this question comes from the applications of a generic SQ lower bound for NGCA to various learning problems. The SQ-hardness of NGCA can be used to obtain similar hardness for a number of well-studied learning problems that superficially appear very different. These include learning mixture models [DKS17, DKPZ23, DKS23], robust mean/covariance estimation [DKS17], robust linear regression [DKS19], learning halfspaces and other natural concepts with adversarial or semi-random label noise [DKZ20, GGG20, DK22a, DKPZ21, \(\mathrm{DKK^{+}22}\)], list-decodable mean estimation and linear regression [DKS18, \(\mathrm{DKP^{+}21}\)], learning simple neural networks [DKKZ20, \(\mathrm{GGJ^{+}20}\]), and even learning simple generative models [CLL22]. In several of these applications, the requirement of bounded chi-squared distance is somewhat problematic, in some cases leading to quantitatively sub-optimal results. Moreover, in certain applications, this restriction leads to vacuous guarantees.
1.1.1 Main Result
Our main result is a qualitatively near-optimal (up to the constant factor in the exponent) SQ lower bound for the NGCA problem, assuming only the moment-matching condition (i.e., without the chi-squared distance restriction). Informally, we essentially show that in order to solve the NGCA in  $n$  dimensions with an  $m$ -dimensional distribution  $A$  that (approximately) matches moments with the standard Gaussian up to degree  $d$ , any SQ algorithm will either require one query with accuracy  $O_{m,d}(n^{-\Omega(d)})$  or exponential in  $n$  many queries.
Formally, we establish the following theorem.
Theorem 1.5 (Main SQ Lower Bound Result). Let  $\lambda \in (0,1)$  and  $n,m,d\in \mathbb{N}$  with  $d$  even and  $m,d\leq n^{\lambda}$ . Let  $\nu \in \mathbb{R}_{+}$  and  $A$  be a distribution on  $\mathbb{R}^m$  such that for any polynomial  $f:\mathbb{R}^m\to \mathbb{R}$  of degree at most  $d$  and  $\mathbf{E}_{\mathbf{x}\sim \mathcal{N}_m}[f(\mathbf{x})^2 ] = 1$ , the following holds:  $|\mathbf{E}_{\mathbf{x}\sim A}[f(\mathbf{x})] - \mathbf{E}_{\mathbf{x}\sim \mathcal{N}_m}[f(\mathbf{x})]|\leq \nu$ . Let  $0 < c < (1 - \lambda) / 8$  and  $n$  be at least a sufficiently large constant depending on  $c$ . Then any SQ algorithm solving the  $n$ -dimensional NGCA problem (as in Definition 1.3) with  $2 / 3$  success probability requires either (i) a query of tolerance  $O_{m,d}\left(n^{-(1 - \lambda) / 8 - c)d}\right) + (1 + o(1))\nu$ , or (ii)  $2^{n^{\Omega (c)}}$  many queries.
A few comments regarding the parameters are in order here. We first note that the constant in  $O_{m,d}(n^{-(1 - \lambda) / 8 - c)d})$  is roughly the size of the Binomial coefficient  $\binom{(d+m)/2-1}{m/2-1}$ . Furthermore, we would like to point out that for most applications we will have  $m, d < 1/poly(n)$ . Therefore, the parameters  $\lambda$  and  $c$  can be taken to be arbitrarily close to 0. So, informally, our lower bound on the query accuracy can be roughly thought of as  $\binom{(d+m)/2-1}{m/2-1} n^{-d/8}$ .
For all the applications given in this paper, the above theorem will be applied for the special case that  $m = 1$ ; namely, the case that the hidden subspace is a hidden direction and  $A$  is a univariate distribution.
Relation to LLL-based Algorithms for NGCA Consider the special case of the NGCA problem corresponding to  $m = 1$  where  $A$  is a discrete distribution that matches its first  $d$  moments with the standard Gaussian. Theorem 1.5 implies that any SQ algorithm for this version of the problem either uses a query with accuracy  $n^{-\Omega (d)}$  or exponential many queries. On the other hand, recent works[DK22b, ZSWB22] gave polynomial-time algorithms for this problem with sample complexity  $O(n)$ , regardless of the degree  $d$ . It is worth noting that the existence of these algorithms does not contradict our SQ lower bound, as these algorithms are based on the LLL-method for lattice basis reduction that is not captured by the SQ framework. An an implication, it follows that LLL-based methods surpass any efficient SQ algorithm for these settings of NGCA. A similar observation was previously made in [DH23] for the special case that the discrete distribution  $A$  is supported on  $\{0,\pm 1\}$ ; consequently, [DH23] could only obtain a quadratic separation. Finally, we note that this
limitation of SQ algorithms is also shared by two other prominent restricted families of algorithms (namely, SoS algorithms and low-degree polynomial tests).
1.1.2 Applications
We believe that Theorem 1.5 is interesting in its own right, as it elucidates the SQ complexity of a natural and well-studied statistical problem. Here we discuss concrete applications to some natural statistical tasks. We note that the SQ lower bound in the prior work [DKS17] cannot give optimal (or even nontrivial) lower bounds for these applications.
List-decodable Gaussian Mean Estimation One can leverage our result to obtain a sharper SQ lower bound for the task of list-decodable Gaussian mean estimation. In this task, the algorithm is given as input points from  $\mathbb{R}^n$  where an  $\alpha < 1/2$  fraction of the points are drawn from an unknown mean and identity covariance Gaussian  $\mathcal{N}(\boldsymbol{\mu}, \mathbf{I})$ , and the remaining points are arbitrary. The goal of the algorithm is to output a list of  $O(1/\alpha)$  many hypothesis vectors at least one of which is close to  $\boldsymbol{\mu}$  in  $\ell_2$ -norm with probability at least  $2/3$ . [DKS18] established the following SQ lower bound for this problem (see also [DK23] for a different exposition).
Fact 1.6 ([DKS18]). For each  $d \in \mathbb{Z}_+$  and  $c \in (0,1/2)$ , there exists  $c_d > 0$  such that for any  $\alpha > 0$  sufficiently small the following holds. Any SQ algorithm that is given access to a  $(1 - \alpha)$ -corrupted Gaussian  $\mathcal{N}(\boldsymbol{\mu},\mathbf{I})$  in  $n > d^{3/c}$  dimensions and returns a list of hypotheses such that with probability at least  $2/3$  one of the hypotheses is within  $\ell_2$ -distance  $c_d\alpha^{-1/d}$  of the true mean  $\boldsymbol{\mu}$ , does one of the following: (i) Uses queries with error tolerance at most  $\exp(O(\alpha^{-2/d}))\Omega(n)^{-(d+1)(1/4-c/2)}$ . (ii) Uses at least  $\exp(\Omega(n^c))$  many queries. (iii) Returns a list of at least  $\exp(\Omega(n))$  many hypotheses.
The above statement is obtained using the framework of [DKS17] by considering the distribution testing problem between  $\mathcal{N}(\boldsymbol{\mu},\mathbf{I})$  and  $\mathbf{P}_{\mathbf{v}}^{A}$ , where  $A$  is the one-dimensional moment-matching distribution of the following form.
Fact 1.7 ([DKS18], see Lemma 8.21 in [DK23]). For  $d \in \mathbb{Z}_+$ , there exists a distribution  $A = \alpha \mathcal{N}(\mu, 1) + (1 - \alpha)E$ , for some distribution  $E$  and  $\mu = 10c_d\alpha^{-1/d}$ , such that the first  $d$  moments of  $A$  agree with those of  $\mathcal{N}(0,1)$ . Furthermore, the probability density function of  $E$  can be taken to be pointwise at most twice the pdf of the standard Gaussian.
We note that  $\chi^2 (A,\mathcal{N}) = O(\exp (\mu^2))$ , and this results in the  $\exp (O(\alpha^{-2 / d}))$  term in the error tolerance of Fact 1.6. Consequently, if  $\alpha \ll \log (n)^{-d / 2}$ , the error tolerance is greater than one and Fact 1.6 fails to give a non-trivial bound. It is worth noting that the setting of "small  $\alpha$  (e.g.,  $\alpha$  is sub-constant in the dimension) is of significant interest in various applications, including in mean estimation. A concrete application is in the related crowd-sourcing setting of [MV18] dealing with this parameter regime.
We can circumvent this technical problem by combining our main result (Theorem 1.5) with Fact 1.7 to obtain the following sharper SQ lower bound (see Appendix C for the proof).
Theorem 1.8 (SQ Lower Bound for List-Decoding the Mean). Let  $\lambda \in (0,1)$  and  $n,d\in \mathbb{N}$  with  $d$  be even and  $d\leq n^{\lambda}$ . Let  $c > 0$  and  $n$  be at least a sufficiently large constant depending on  $c$ . There exists  $c_{d} > 0$  such that for any  $\alpha >0$  sufficiently small the following holds. Any SQ algorithm that is given access to a  $(1 - \alpha)$ -corrupted Gaussian  $\mathcal{N}(\boldsymbol {\mu},\mathbf{I})$  in  $n$  dimensions and returns a list of hypotheses such that with probability at least  $2 / 3$  one of the hypotheses is within  $\ell_2$ -distance  $c_{d}\alpha^{-1 / d}$  of the true mean  $\pmb{\mu}$ , does one of the following: (i) Uses queries with error tolerance at most  $O_{d}(n^{-(\left(1 - \lambda\right) / 8 - c)d})$ . (ii) Uses at least  $2^{n^{\Omega (c)}}$  many queries. (iii) Returns a list of at least  $\exp (\Omega (n))$  many hypotheses.
Anti-concentration Detection Anti-concentration (AC) detection is the following hypothesis testing problem: given access to an unknown distribution  $D$  over  $\mathbb{R}^n$ , where the input distribution  $D$  is promised to satisfy either (i)  $D$  is the standard Gaussian; or (ii)  $D$  has at least  $\alpha < 1$  of its probability mass residing inside a dimension  $(n - 1)$  subspace  $V \subset \mathbb{R}^n$ . The goal is to distinguish between the two cases with success probability  $2/3$ .
In order to use our main result to derive an SQ lower bound for this task, we require the following lemma on univariate moment-matching (see Appendix C for the proof).
Lemma 1.9. Let  $D_0$  denote the distribution that outputs 0 with probability 1. For  $d \in \mathbb{Z}_+$ , there exists  $\alpha_d \in (0,1)$  and a univariate distribution  $A = \alpha_d D_0 + (1 - \alpha_d) E$  for some distribution  $E$  such that the first  $d$  moments of  $A$  agree with those of  $\mathcal{N}(0,1)$ . Furthermore, the pdf of  $E$  can be taken to be pointwise at most twice the pdf of the standard Gaussian.
Using the above lemma and our main result, we can deduce the following SQ lower bound on the anti-concentration detection problem (see Appendix C for the proof).
Theorem 1.10 (SQ Lower Bound for AC Detection). For any  $\alpha \in (0,1/2)$ , any SQ algorithm that has access to a  $n$ -dimensional distribution that is either (i) the standard Gaussian; or (ii) a distribution that has at least  $\alpha < 1$  probability mass in a  $(n-1)$ -dimensional subspace  $V \subset \mathbb{R}^n$ , and distinguishes the two cases with success probability at least  $2/3$ , either requires a query with error at most  $n^{-\omega_{\alpha}(1)}$ , or uses at least  $2^{n^{\Omega(1)}}$  many queries.
Learning Periodic Functions Another application is for the well-studied problem of learning periodic functions, see, e.g., [SVWX17] and [SZB21], which is closely related to the continuous Learning with Errors (cLWE) problem [BRST21]. In the task of learning periodic functions, the algorithm is given sample access to a distribution  $D$  of  $(\mathbf{x},y)$  over  $\mathbb{R}^n\times \mathbb{R}$ . The distribution  $D$  is such that  $\mathbf{x}\sim \mathcal{N}(0,\mathbf{I}_n)$  and  $y = \cos (2\pi (\delta \langle \mathbf{w},\mathbf{x}\rangle +\zeta))$  with noise  $\zeta \sim \mathcal{N}(0,\sigma^2)$ . This implies that  $y$  is a periodic function of  $\mathbf{x}$  along the direction of  $\mathbf{w}$  with a small amount of noise. While the frequency and noise scale parameters  $\delta ,\sigma \in \mathbb{R}_{+}$  are known to the algorithm, the parameter  $\mathbf{w}\in \mathbb{S}^{n - 1}$  is unknown; the goal of the algorithm is to output a hypothesis  $h:\mathbb{R}^n\to \mathbb{R}$  such that  $\mathbf{E}_{\mathbf{x}\sim D}[(h(\mathbf{x}) - y)^2 ]$  is minimized.
In order to use our main theorem to derive an SQ lower bound for this problem, we need to show that for any  $t \in [-1, 1]$ ,  $D$  conditioned on  $y = t$  approximately matches moments with  $\mathcal{N}(0, \mathbf{I}_n)$ . To this end, we introduce the following definition and fact of discrete Gaussian measure from [DK22a].
Definition 1.11. For  $s \in \mathbb{R}_+$  and  $\theta \in \mathbb{R}$ , let  $G_{s,\theta}$  denote the measure of the "s-spaced discrete Gaussian distribution". In particular, for each  $n \in \mathbb{Z}$ ,  $G_{s,\theta}$  assigns mass  $sg(ns + \theta)$  to the point  $ns + \theta$ , where  $g$  is the pdf function of  $\mathcal{N}(0,1)$ .
Note that although  $G_{s,\theta}$  is not a probability measure (as the total measure is not one), it can be thought of as a probability distribution since the total measure is close to one for small  $\sigma$ . To see this, we introduce the following fact.
Fact 1.12 (Lemma 3.12 from [DK22a]). For all  $k \in \mathbb{N}$ ,  $s > 0$  and all  $\theta \in \mathbb{R}$ , we have that  $|\mathbf{E}_{t \sim \mathcal{N}(0,1)}[t^k] - \mathbf{E}_{t \sim G_{s,\theta}}[t^k]| = k! O(s)^k \exp(-\Omega(1/s^2))$ .
Using the above fact, we are now ready to prove our SQ lower bound for learning periodic functions (see Appendix C for the proof).
Theorem 1.13 (SQ Lower Bound for Learning Periodic Functions). Let  $c > 0$  and  $n$  be at least a sufficiently large constant depending on  $c$ . Let  $D$  be the distribution of  $(\mathbf{x}, y)$  over  $\mathbb{R}^n \times \mathbb{R}$  that is generated by  $\mathbf{x} \sim \mathcal{N}(0, \mathbf{I}_n)$  and  $y = \cos(2\pi (\delta \langle \mathbf{w}, \mathbf{x} \rangle + \zeta))$  with noise  $\zeta \sim \mathcal{N}(0, \sigma^2)$ . Let  $\delta > n^c$ ,  $\sigma$  be known and  $\mathbf{w}$  be unknown to the algorithm. Then any SQ algorithm that has access to the distribution  $D$  and returns a hypothesis  $h: \mathbb{R}^n \to \mathbb{R}$  such that  $\mathbf{E}_{(\mathbf{x}, y) \sim D}[(h(\mathbf{x}) - y)^2] = o(1)$ , either requires a query with error at most  $\exp(-n^{c'})$  for  $c' < \min(2c, 1/10)$ , or uses at least  $2^{n^{\Omega(1)}}$  many queries.
We start by noting that we cannot use the standard SQ dimension argument  $\mathrm{[FGR^{+}17]}$  to prove our result, due to the unbounded chi-squared norm. To handle this issue, we need to revisit the underlying ideas of that proof. In particular, we will show that for any bounded query function  $f:\mathbb{R}^n\to [-1,1]$ , with high probability over some  $\mathbf{V}\sim U(\mathbf{O}_{n,m})$ , it holds that  $|\mathbf{E}_{\mathbf{x}\sim \mathcal{N}_n}[f(\mathbf{x})] - \mathbf{E}_{\mathbf{x}\sim \mathbf{P}_{\mathbf{V}}^A}[f(\mathbf{x})]|$  will be small. This allows an adversarial oracle to return  $\mathbf{E}_{\mathbf{x}\sim \mathcal{N}_n}[f(\mathbf{x})]$  to every query  $f$  regardless of which case we are in, unless the algorithm is lucky enough (or uses high accuracy) to find an  $f$  that causes  $|\mathbf{E}_{\mathbf{x}\sim \mathcal{N}_n}[f(\mathbf{x})] - \mathbf{E}_{\mathbf{x}\sim \mathbf{P}_{\mathbf{V}}^A}[f(\mathbf{x})]|$  to be at least  $\tau$ .
Our approach for calculating  $\mathbf{E}_{\mathbf{x}\sim \mathbf{P}_{\mathbf{V}}^{A}}[f(\mathbf{x})]$  will be via Fourier analysis. In particular, using the Fourier decomposition of  $f$ , we can write  $f(\mathbf{x}) = \sum_{k = 0}^{\infty}\langle \mathbf{T}_k,\mathbf{H}_k(\mathbf{x})\rangle$ , where  $\mathbf{H}_k(\mathbf{x})$  is the
properly normalized degree-  $k$  Hermite polynomial tensor and  $\mathbf{T}_k$  is the degree-  $k$  Fourier coefficients of  $f$ . Taking the inner product with the distribution  $\mathbf{P}_{\mathbf{V}}^{A}$  involves computing the expectation of  $\mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A}}[\mathbf{H}_k(\mathbf{x})]$ , which can be seen to be  $\langle \mathbf{V}^{\otimes k} \mathbf{A}_k, \mathbf{T}_k \rangle$ , where  $\mathbf{A}_k = \mathbf{E}_{\mathbf{x} \sim A}[\mathbf{H}_k(\mathbf{x})]$ . Thus, we obtain (at least morally speaking) that