SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma 3.10. Under the conditions of Proposition 3.1, and further assume  $d,m\leq n^{\lambda} / \log n$ $\nu < 2$  and  $\left(\frac{\Gamma(d / 2 + m / 2)}{\Gamma(m / 2)}\right)n^{-(1 - \lambda) / 8 - c)d} <   2.$  Then for any  $n$  that is at least a sufficiently small constant depending on  $c,$  there is an  $\alpha^\prime <  (1 - \lambda) / 8$  such that for any  $B =$ $n^{\alpha}$  where  $\alpha^{\prime} <   \alpha <  (1 - \lambda) / 8$  the following holds. Let  $A^{\prime}$  be the truncated distribution defined as the distribution of  $\mathbf{x}\sim A$  conditioned on  $\mathbf{x}\in \mathbb{B}^{m}(B)$  . Then  $d_{\mathrm{TV}}(A,A^{\prime})\leq$ $\left(\frac{\Gamma(d / 2 + m / 2)}{\Gamma(m / 2)}\right)n^{-(1 - \lambda) / 8 - c)d}.$  Furthermore for any  $\ell \in \mathbb{N},$  except with probability at most  $2^{-n^{\Omega (c)}}$  w.r.t.  $\mathbf{V}\sim U(\mathbf{O}_{n,m}),\sum_{k = 1}^{\ell}|\langle \mathbf{A}_k,(\mathbf{V}^{\intercal})^{\otimes k}\mathbf{T}_k\rangle | = \left(\frac{\Gamma(d / 2 + m / 2)}{\Gamma(m / 2)}\right)n^{-(1 - \lambda) / 8 - c)d} + (1 + o(1))\nu$  where  $\mathbf{A}_k = \mathbf{E}_{\mathbf{x}\sim A'}[\mathbf{H}_k(\mathbf{x})]$  and  ${\bf T}_k = {\bf E}_{\mathbf{x}\sim \mathcal{N}_n}[f(\mathbf{x}){\bf H}_k(\mathbf{x})].$
3.2 Proof for Proposition 3.1
We are now ready to prove Proposition 3.1 which is the main technical ingredient of our main result. Proposition 3.1 states that  $|\mathbf{E}_{\mathbf{x}\sim \mathbf{P}_{\mathbf{V}}^{A}}[f(\mathbf{x})] - \mathbf{E}_{\mathbf{x}\sim \mathcal{N}_{n}}[f(\mathbf{x})]|$  is small with high probability. The main idea of the proof is to use Fourier analysis on  $\mathbf{E}_{\mathbf{x}\sim \mathbf{P}_{\mathbf{V}}^{A'}}[f(\mathbf{x})]$  as we discussed in the last section, where  $A^{\prime}$  is the distribution obtained by truncating  $A$  inside  $\mathbb{B}^m (B)$ .
Proof for Proposition 3.1. For convenience, we let  $\zeta = (1 - \lambda) / 8 - c$ . We will analyze  $\mathbf{E}_{\mathbf{x} \sim A}[\mathbf{H}_k(\mathbf{x})]$  by truncating  $A$ . Therefore, we will apply Lemma 3.10 here. However, notice Lemma 3.10 additionally assumes  $d, m \leq n^{\lambda} / \log n, \nu < 2$  and  $\left(\frac{\Gamma(d / 2 + m / 2)}{\Gamma(m / 2)}\right)n^{-\zeta d} < 2$ . We show that all these three conditions can be assumed true WLOG. If either the second or the third condition is not true, then our lower bound here is trivialized and is always true since  $f$  is bounded between  $[-1, +1]$ . For  $d, m \leq n^{\lambda} / \log n$ , consider a  $\lambda' > \lambda$  such that  $(1 - \lambda') / 8 - \zeta = \frac{(1 - \lambda) / 8 - \zeta}{2}$ . Then it is easy to see for any sufficiently large  $n$  depending on  $(1 - \lambda) / 8 - \zeta$ , we have  $d, m \leq n^{\lambda'} / \log n$  and  $\zeta \leq (1 - \lambda) / 8 - \zeta$ . Therefore, we can WLOG apply Lemma 3.10 for  $\lambda'$ .
Now let  $B = n^{\alpha}$  where  $\alpha < (1 - \lambda)/8$  is the constant in Lemma 3.10. Then we consider the truncated distribution  $A'$  defined as the distribution of  $\mathbf{x} \sim A$  conditioned on  $\mathbf{x} \in \mathbb{B}^{m}(B)$ . By Lemma 3.10, we have  $d_{\mathrm{TV}}(A, A') \leq \left(\frac{\Gamma(d/2 + m/2)}{\Gamma(m/2)}\right)n^{-\zeta d}$ . Given that  $f$  is bounded between  $[-1,1]$ , this implies  $|\mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A}}[f(\mathbf{x})] - \mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A'}}[f(\mathbf{x})]| \leq 2d_{\mathrm{TV}}(\mathbf{P}_{\mathbf{V}}^{A}, \mathbf{P}_{\mathbf{V}'}^{A'}) = 2d_{\mathrm{TV}}(A, A') \leq 2\left(\frac{\Gamma(d/2 + m/2)}{\Gamma(m/2)}\right)n^{-\zeta d}$ . Thus, it suffices for us to analyze  $\mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A'}}[f(\mathbf{x})]$  instead of  $\mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A}}[f(\mathbf{x})]$ .
Let  $\ell = \ell_f(n) \in \mathbb{N}$  be a function depending only on the query function  $f$  and the dimension  $n$  ( $\ell$  to be specified later). By Lemma 3.3, we have that  $\mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A'}}[f(\mathbf{x})] = \sum_{k=0}^{\ell} |\langle \mathbf{A}_k, (\mathbf{V}^\intercal)^{\otimes k} \mathbf{T}_k \rangle| + \mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A'}}[f^{> \ell}(\mathbf{x})]$ . Recall that we want to bound  $|\mathbf{E}_{\mathbf{x} \sim \mathbf{P}_{\mathbf{V}}^{A'}}[f(\mathbf{x})] - \mathbf{E}_{\mathbf{x} \sim \mathcal{N}_n}[f(\mathbf{x})]|$  with high probability, where we note that  $\mathbf{E}_{\mathbf{x} \sim \mathcal{N}_n}[f(\mathbf{x})] = \langle \mathbf{A}_0, \mathbf{T}_0 \rangle$ . Therefore, we can write
$$
| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f (\mathbf {x}) ] - \mathbf {E} _ {\mathbf {x} \sim \mathcal {N} _ {n}} [ f (\mathbf {x}) ] | \leq \left| \sum_ {k = 1} ^ {\ell} \langle \mathbf {A} _ {k}, (\mathbf {V} ^ {\intercal}) ^ {\otimes k} \mathbf {T} _ {k} \rangle \right| + | \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f ^ {> \ell} (\mathbf {x}) ] |.
$$
For the first term, by Lemma 3.10, we have that  $\left|\sum_{k=1}^{\ell}\langle\mathbf{A}_{k},(\mathbf{V}^{\intercal})^{\otimes k}\mathbf{T}_{k}\rangle\right| = \left(\frac{\Gamma(d/2+m/2)}{\Gamma(m/2)}\right)n^{-\zeta d} + (1 + o(1))\nu$ , except with probability  $2^{-n^{\Omega(c)}}$ .
It now remains for us to show that  $|\mathbf{E}_{\mathbf{x}\sim \mathbf{P}_{\mathbf{V}}^{A^{\prime}}}[f^{> \ell}(\mathbf{x})]|$  is also small with high probability. Consider the distribution  $D = \mathbf{E}_{\mathbf{v}\sim U(\mathbf{O}_{n,m})}[\mathbf{P}_{\mathbf{V}}^{A^{\prime}}]$ . The following lemma shows that  $D$  is continuous
and  $\chi^2 (D,\mathcal{N}_n)$  is at most a constant only depending on  $n$  (independent of the choice of the one dimensional distribution  $A$ ).
Lemma 3.11. Let  $A$  be any distribution supported on  $\mathbb{B}^m (n)$  for  $n\in \mathbb{N}$  which is at least a sufficiently large universal constant. Let  $D = \mathbf{E}_{\mathbf{V}\sim U(\mathbf{O}_{n,m})}[\mathbf{P}_{\mathbf{V}}^{A'}]$ . Then,  $D$  is a continuous distribution and  $\chi^2 (D,\mathcal{N}_n) = O_n(1)$ .
Roughly speaking, the proof of the lemma follows by noting that the average over  $\mathbf{V}$  of  $\mathbf{P}_{\mathbf{V}}^{A}$  is spherically symmetric. We defer its proof to Appendix B.2. Using this lemma, we can write
$$
\begin{array}{l} \mathbf {E} _ {\mathbf {V} \sim U (\mathbf {O} _ {n, m})} \left[ \left| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f ^ {> \ell} (\mathbf {x}) ] \right| \right] \leq \mathbf {E} _ {\mathbf {V} \sim U (\mathbf {O} _ {n, m})} \left[ \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ | f ^ {> \ell} (\mathbf {x}) | ] \right] = \mathbf {E} _ {\mathbf {x} \sim D} [ | f ^ {> \ell} (\mathbf {x}) | ] \\ = \mathbf {E} _ {\mathbf {x} \sim \mathcal {N} _ {n}} \left[ \frac {D (\mathbf {x})}{\mathcal {N} _ {n} (\mathbf {x})} | f ^ {> \ell} (\mathbf {x}) | \right] \leq \sqrt {1 + \chi^ {2} (D , \mathcal {N} _ {n})} \| f ^ {> \ell} \| _ {2}, \\ \end{array}
$$
where the first equality holds due to Fubini's theorem. Furthermore, since  $\chi^2 (D,\mathcal{N}_n) = O_n(1)$ , there is a function  $\delta :\mathbb{R}\to \mathbb{R}$  such that  $1 + \chi^{2}(D,\mathcal{N}_{n})\leq \delta (n)$ . Therefore, we have that
$$
\mathbf {E} _ {\mathbf {V} \sim U (\mathbf {O} _ {n, m})} \left[ \left| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f ^ {> \ell} (\mathbf {x}) ] \right| \right] \leq \sqrt {1 + \chi^ {2} (D , \mathcal {N} _ {n})} \| f ^ {> \ell} \| _ {2} \leq \delta (n) \| f ^ {> \ell} \| _ {2}.
$$
We can take  $\ell = \ell_f(n)$  ( $\ell$  only depends on the query function  $f$  and dimension  $n$ ) to be a sufficiently large function such that  $\| f^{> \ell} \|_2 \leq \left( \frac{e^{-n}}{\delta(n)} \right) \left( \frac{\Gamma(d/2 + m/2)}{\Gamma(m/2)} \right) n^{-\zeta d}$ . Then we get
$$
\mathbf {E} _ {\mathbf {V} \sim U (\mathbf {O} _ {n, m})} \left[ \left| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f ^ {> \ell} (\mathbf {x}) ] \right| \right] \leq \delta (n) \| f ^ {> \ell} \| _ {2} \leq e ^ {- n} \left(\frac {\Gamma (d / 2 + m / 2)}{\Gamma (m / 2)}\right) n ^ {- \zeta d}.
$$
This gives the tail bound  $\mathbf{Pr}_{\mathbf{V}\sim U(\mathbf{O}_{n,m})}\left[\left|\mathbf{E}_{\mathbf{x}\sim \mathbf{P}_{\mathbf{V}}^{A^{\prime}}}[f^{> \ell}(\mathbf{x})]\right|\geq \left(\frac{\Gamma(d / 2 + m / 2)}{\Gamma(m / 2)}\right)n^{-\zeta d}\right]\leq e^{-n}.$
Using the above upper bounds, we have
$$
\begin{array}{l} \left| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f (\mathbf {x}) ] - \mathbf {E} _ {\mathbf {x} \sim \mathcal {N} _ {n}} [ f (\mathbf {x}) ] \right| \leq \left| \sum_ {k = 1} ^ {\ell} \langle \mathbf {A} _ {k}, (\mathbf {V} ^ {\intercal}) ^ {\otimes k} \mathbf {T} _ {k} \rangle \right| + | \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f ^ {> \ell} (\mathbf {x}) ] | \\ = 2 \left(\frac {\Gamma (d / 2 + m / 2)}{\Gamma (m / 2)}\right) n ^ {- \zeta d} + (1 + o (1)) \nu , \\ \end{array}
$$
except with probability  $2^{-n^{\Omega(1)}}$ . As we have argued at the beginning of the proof,
$$
\left| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A}} [ f (\mathbf {x}) ] - \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A ^ {\prime}}} [ f (\mathbf {x}) ] \right| \leq 2 \left(\frac {\Gamma (d / 2 + m / 2)}{\Gamma (m / 2)}\right) n ^ {- \zeta d}.
$$
$$
\left| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {v}} ^ {A}} [ f (\mathbf {x}) ] - \mathbf {E} _ {\mathbf {x} \sim \mathcal {N} _ {n}} [ f (\mathbf {x}) ] \right| \leq 3 \left(\frac {\Gamma (d / 2 + m / 2)}{\Gamma (m / 2)}\right) n ^ {- \zeta d} + (1 + o (1)) \nu ,
$$
except with probability  $2^{-n^{\Omega(1)}} < 2^{-n^{\Omega(c)}}$  given  $c = O(1)$ .
In the end, notice that the above argument is still true if we take  $\zeta' > \zeta$  such that  $(1 - \lambda) / 8 - \zeta' = \frac{(1 - \lambda) / 8 - \zeta}{2}$ . Using the above argument for  $\zeta'$  and given  $n$  is a sufficiently large constant depending on  $(1 - \lambda) / 8 - \zeta = 2((1 - \lambda) / 8 - \zeta')$ , we get
$$
| \mathbf {E} _ {\mathbf {x} \sim \mathbf {P} _ {\mathbf {V}} ^ {A}} [ f (\mathbf {x}) ] - \mathbf {E} _ {\mathbf {x} \sim \mathcal {N} _ {n}} [ f (\mathbf {x}) ] | \leq \left(\frac {\Gamma (d / 2 + m / 2)}{\Gamma (m / 2)}\right) n ^ {- \zeta d} + (1 + o (1)) \nu ,
$$
except with probability  $2^{-n^{\Omega((1 - \lambda)/8 - \zeta')} = 2^{-n^{\Omega(c)}}}$ . Replacing  $\zeta$  with  $(1 - \lambda)/8 - c$  completes the proof.
[ZSWB22] I. Zadik, M. J. Song, A. S. Wein, and J. Bruna. Lattice-based methods surpass sum-of-squares in clustering. In Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pages 1247-1248. PMLR, 2022.