SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[ABL17] Pranjal Awasthi, Maria Florina Balcan, and Philip M Long. The power of localization for efficiently learning linear separators with noise. Journal of the ACM (JACM), 63(6):1-27, 2017. 1
[BH21] Maria-Florina Balcan and Nika Haghtalab. Noise in classification. Beyond the Worst-Case Analysis of Algorithms, page 361, 2021. 1
[BK21] Ainesh Bakshi and Pravesh K Kothari. List-decodable subspace recovery: Dimension independent error in polynomial time. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 1279-1297. SIAM, 2021. 1
[BZ17] Maria-Florina F Balcan and Hongyang Zhang. Sample and computationally efficient learning algorithms under s-concave distributions. Advances in Neural Information Processing Systems, 30, 2017. 1
[Dan15] Amit Daniely. A ptas for agnostically learning halfspaces. In Conference on Learning Theory, pages 484–502. PMLR, 2015. 1
$\left[\mathrm{DKK}^{+}22\right]$  Ilias Diakonikolas, Daniel M Kane, Vasilis Kontonis, Christos Tzamos, and Nikos Zarifis. Learning general halfspaces with general massart noise under the gaussian distribution. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 874-885, 2022. 1
$\left[\mathrm{DKK}^{+}23\right]$  Ilias Diakonikolas, Daniel M Kane, Vasilis Kontonis, Sihan Liu, and Nikos Zarifis. Efficient testable learning of halfspaces with adversarial label noise. arXiv preprint arXiv:2303.05485, 2023. 1, 1
[DKPZ21] Ilias Diakonikolas, Daniel M Kane, Thanasis Pittas, and Nikos Zarifis. The optimality of polynomial regression for agnostic learning under gaussian marginals in the sq model. In Conference on Learning Theory, pages 1552–1584. PMLR, 2021. 1
[DKR23] Ilias Diakonikolas, Daniel M Kane, and Lisheng Ren. Near-optimal cryptographic hardness of agnostically learning halfspaces and relu regression under gaussian marginals. arXiv preprint arXiv:2302.06512, 2023. 1
[DKTZ20a] Ilias Diakonikolas, Vasilis Kontonis, Christos Tzamos, and Nikos Zarifis. Learning halfspaces with massart noise under structured distributions. In Conference on Learning Theory, pages 1486–1513. PMLR, 2020. 1, 2, 4, 4, 4.2, A.5, C.1
[DKTZ20b] Ilias Diakonikolas, Vasilis Kontonis, Christos Tzamos, and Nikos Zarifis. Non-convex sgd learns halfspaces with adversarial label noise. Advances in Neural Information Processing Systems, 33:18540–18549, 2020. 1, 2, 2, 4, 4, 4.2, C.1
[DKZ20] Ilias Diakonikolas, Daniel Kane, and Nikos Zarifis. Near-optimal sq lower bounds for agnostically learning halfspaces and relus under gaussian marginals. Advances in Neural Information Processing Systems, 33:13586-13596, 2020. 1
$\left[\mathrm{FKP}^{+}19\right]$  Noah Fleming, Pravesh Kothari, Toniann Pitassi, et al. Semialgebraic proofs and efficient algorithm design. Foundations and Trends® in Theoretical Computer Science, 14(1-2):1-221, 2019. 1
[GGK20] Surbhi Goel, Aravind Gollakota, and Adam Klivans. Statistical-query lower bounds via functional gradients. Advances in Neural Information Processing Systems, 33:2147-2158, 2020. 1
[GKK23] Aravind Gollakota, Adam R Klivans, and Pravesh K Kothari. A moment-matching approach to testable learning and a new characterization of rademacher complexity. Proceedings of the fifty-fifth annual ACM Symposium on Theory of Computing, 2023. To appear. 1, 1, 1
[GKSV23] Aravind Gollakota, Adam R Klivans, Konstantinos Stavropoulos, and Arsen Vasilyan. An efficient tester-learner for halfspaces. arXiv preprint arXiv:2302.14853, 2023. 1, 1, 1, 2, 2, 3.1, 4, 4, 4.2, A.4, A.5, C.1
[HL13] Christopher J Hillar and Lek-Heng Lim. Most tensor problems are np-hard. Journal of the ACM (JACM), 60(6):1-39, 2013. 3.2
[KKK19] Sushrut Karmalkar, Adam Klivans, and Pravesh Kothari. List-decodable linear regression. Advances in neural information processing systems, 32, 2019. 1
[KLS95] Ravi Kannan, László Lovász, and Miklós Simonovits. Isoperimetric problems for convex bodies and a localization lemma. Discrete & Computational Geometry, 13:541-559, 1995. 2, 2.6
[KLS09] Adam R Klivans, Philip M Long, and Rocco A Servedio. Learning halfspaces with malicious noise. Journal of Machine Learning Research, 10(12), 2009. 1
[KS17] Pravesh K Kothari and Jacob Steinhardt. Better agnostic clustering via relaxed tensor norms. arXiv preprint arXiv:1711.07465, 2017. 1, 3.2, 3.4, 3.2
[Las01] Jean B Lasserre. New positive semidefinite relaxations for nonconvex quadratic programs. Advances in Convex Analysis and Global Optimization: Honoring the Memory of C. Caratheodory (1873-1950), pages 319-331, 2001. 3.2
[LV07] László Lovász and Santosh Vempala. The geometry of logconcave functions and sampling algorithms. Random Structures & Algorithms, 30(3):307-358, 2007. 2.3
[LV18] Yin Tat Lee and Santosh S Vempala. The Kannan-Lovász-Simonovits conjecture. arXiv preprint arXiv:1807.03465, 2018. 2.6
[Nes00] Yurii Nesterov. Squared functional systems and optimization problems. High performance optimization, pages 405-440, 2000. 3.2
[Par00] Pablo A Parrilo. Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization. California Institute of Technology, 2000. 3.2
[RV23] Ronitt Rubinfeld and Arsen Vasilyan. Testing distributional assumptions of learning algorithms. Proceedings of the fifty-fifth annual ACM Symposium on Theory of Computing, 2023. To appear. 1, 1
[RY20] Prasad Raghavendra and Morris Yau. List decodable learning via sum of squares. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 161-180. SIAM, 2020. 1
[Sho87] N.Z. Shor. Quadratic optimization problems. Izv. Akad. Nauk SSSR Tekhn. Kibernet., 1987(1):128-139, 222, 1987. 3.2
[SW14] Adrien Saumard and Jon A Wellner. Log-concavity and strong log-concavity: a review. Statistics surveys, 8:45, 2014. 2.2, 2.5
[YZ17] Songbai Yan and Chicheng Zhang. Revisiting perceptron: Efficient and label-optimal learning of halfspaces. Advances in Neural Information Processing Systems, 30, 2017. 1
[Zha18] Chicheng Zhang. Efficient active learning of sparse halfspaces. In Conference on Learning Theory, pages 1856-1880. PMLR, 2018. 1
[ZL21] Chicheng Zhang and Yinan Li. Improved algorithms for efficient active learning halfspaces with massart and tsybakov noise. In Conference on Learning Theory, pages 4526-4527. PMLR, 2021. 1
[ZSA20] Chicheng Zhang, Jie Shen, and Pranjal Awasthi. Efficient active learning of sparse halfspaces with arbitrary bounded noise. Advances in Neural Information Processing Systems, 33:7184-7197, 2020. 1
In this section, we provide a list of technical results that we use in our proofs.
Lemma A.1 (Preservation of Poincaré constant). Let  $I$  be an open interval in  $\mathbb{R}$  and  $q:\mathbb{R}^d\to \mathbb{R}_+$  the density of a  $\gamma$ -Poincaré distribution. Let  $\mathbf{v}\in \mathbb{S}^{d - 1}$  and  $q_{\mathbf{v}}^{\prime}:\mathbb{R}^{d - 1}\to \mathbb{R}_{+}$  be the density of the distribution resulting from conditioning  $q$  to  $\mathbf{x}\cdot \mathbf{v}\in I$  and projecting on the subspace perpendicular to  $\mathbf{v}$ . Then, the distribution corresponding to  $q_{\mathbf{v}}^{\prime}$  is  $\gamma$ -Poincaré.
Proof. Assume, without loss of generality, that  $\mathbf{v} = \mathbf{e}_d$ . We have that
$$
q _ {\mathbf {v}} ^ {\prime} (\mathbf {x} _ {<   d}) = \frac {\int_ {x _ {d} \in I} q (\mathbf {x} _ {<   d} , x _ {d}) d x _ {d}}{\int_ {\mathbf {x} _ {<   d}} \int_ {x _ {d} \in I} q (\mathbf {x}) d \mathbf {x}}, \text {f o r a n y} \mathbf {x} _ {<   d} \in \mathbb {R} ^ {d - 1}.
$$
Let  $f: \mathbb{R}^{d-1} \to \mathbb{R}$  be any differentiable function. In order to show that  $q_{\mathbf{v}}'$  is  $\gamma$ -Poincaré, it is sufficient to show that under no further assumptions on  $f$ , the quantity  $\operatorname{Var}_{q_{\mathbf{v}}'}(f(\mathbf{x}_{<d}))$  is upper bounded by the product of  $\gamma$  and  $\mathbb{E}_{q_{\mathbf{v}}'}[\|\nabla f(\mathbf{x}_{<d})\|_2^2]$ . We expand the quantity  $\operatorname{Var}_{q_{\mathbf{v}}'}(f(\mathbf{x}_{<d}))$  as follows
$$
\begin{array}{l} \mathrm {V a r} _ {q _ {\mathbf {v}} ^ {\prime}} (f (\mathbf {x} _ {<   d})) = \inf _ {\tau} \int_ {\mathbf {x} _ {<   d}} (f (\mathbf {x} _ {<   d}) - \tau) ^ {2} q _ {\mathbf {v}} ^ {\prime} (\mathbf {x} _ {<   d}) d \mathbf {x} _ {<   d} \\ = \inf _ {\tau} \int_ {\mathbf {x} _ {<   d}} (f (\mathbf {x} _ {<   d}) - \tau) ^ {2} \cdot \frac {\int_ {x _ {d} \in I} q (\mathbf {x} _ {<   d} , x _ {d}) d x _ {d}}{\int_ {\mathbf {x} _ {<   d}} \int_ {x _ {d} \in I} q (\mathbf {x}) d \mathbf {x}} d \mathbf {x} _ {<   d} \\ = \frac {\inf _ {\tau} \int_ {\mathbf {x} _ {<   d}} \int_ {x _ {d} \in I} (f (\mathbf {x} _ {<   d}) - \tau) ^ {2} \cdot q (\mathbf {x}) d x _ {d} d \mathbf {x} _ {<   d}}{\int_ {\mathbf {x} _ {<   d}} \int_ {x _ {d} \in I} q (\mathbf {x}) d \mathbf {x}} \\ \leq \frac {\gamma \cdot \int_ {\mathbf {x} <   d} \int_ {x _ {d} \in I} \| \nabla_ {\mathbf {x}} f (\mathbf {x} <   d) \| _ {2} ^ {2} \cdot q (\mathbf {x}) d \mathbf {x}}{\int_ {\mathbf {x} <   d} \int_ {x _ {d} \in I} q (\mathbf {x}) d \mathbf {x}} \quad (s i n c e q i s s i n g a r e) \\ = \gamma \cdot \int_ {\mathbf {x} _ {<   d}} \| \nabla_ {\mathbf {x} _ {<   d}} f (\mathbf {x} _ {<   d}) \| _ {2} ^ {2} \cdot q _ {\mathbf {v}} ^ {\prime} (\mathbf {x} _ {<   d}) d \mathbf {x} _ {<   d} \quad \left(\text {s i n c e} \frac {\partial f}{\partial x _ {d}} \equiv 0\right) \\ = \gamma \cdot \underset {q _ {\mathbf {v}} ^ {\prime}} {\mathbb {E}} [ \| \nabla f (\mathbf {x} _ {<   d}) \| _ {2} ^ {2} ], \\ \end{array}
$$
Proposition A.2 (Spectral Tester). Let  $D$  be a distribution over  $\mathbb{R}^d$ . Then, there is a tester that given  $\delta \in (0,1)$ ,  $\lambda \geq 1$ ,  $\theta > 0$  and a set  $S$  of i.i.d. samples from  $D$  with size at least  $\frac{2\lambda d^4}{\theta^2\delta}$ , runs in time  $\mathrm{poly}(d,\frac{1}{\theta},|S|)$  and satisfies the following specifications
(a) If the tester accepts, then, for  $\mathbf{z} \sim S$ ,  $\mathbb{E}_S[\mathbf{z}\mathbf{z}^T] \succeq \frac{\theta}{2} I_d$  (resp.  $\mathbb{E}_S[\mathbf{z}\mathbf{z}^T] \preceq 2\theta I_d$ ).
(b) If, for  $\mathbf{z} \sim D$ ,  $\mathbb{E}_D[(\mathbf{z}_i\mathbf{z}_j)^2] \leq \lambda$  and  $\mathbb{E}_D[\mathbf{z}\mathbf{z}^T] \succeq \theta I_d$  (resp.  $\mathbb{E}_D[\mathbf{z}\mathbf{z}^T] \preceq \theta I_d$ ), then the tester accepts with probability at least  $1 - \delta$ .
Proof. The tester receives  $\lambda$ , a set  $S$  and  $\delta \in (0,1)$  and does the following:
1. Compute the matrix  $M_S = \mathbb{E}_S[\mathbf{z}\mathbf{z}^T]$
2. If the minimum (resp. maximum) eigenvalue of  $M_S$  is larger than  $\frac{\theta}{2}$  (resp. smaller than  $2\theta$ ), then accept. Otherwise reject.
Clearly, if the tester accepts, then the desired property is satisfied by construction. If the distribution  $D$  satisfies the conditions of part (b), we can show that for  $M_D = \mathbb{E}_{\mathbf{z} \sim D}[\mathbf{z}\mathbf{z}^T]$  we have
$$
\left\| M _ {S} - M _ {D} \right\| _ {\mathrm {o p}} \leq \frac {\theta}{2}, \text {w i t h p r o b a b i l i t y a t l e a s t} 1 - \delta
$$
which implies that  $M_S \succeq \frac{\theta}{2} I_d$  (and  $M_S \preceq (\theta + \frac{\theta}{2}) I_d \preceq 2\theta I_d$ ). In particular, we have that  $(M_S)_{ij} = \mathbb{E}_S[\mathbf{z}_i\mathbf{z}_j]$ , and by Chebyshev's inequality we have
$$
\mathbb {P} \left[ | (M _ {S}) _ {i j} - (M _ {D}) _ {i j} | > \frac {\theta}{2 d} \right] \leq \frac {4 d ^ {2}}{\theta^ {2} | S |} \underset {\mathbf {z} \sim_ {D}} {\mathbb {E}} [ (\mathbf {z} _ {i} \mathbf {z} _ {j}) ^ {2} ] \leq \frac {4 \lambda d ^ {2}}{\theta^ {2} | S |} \leq \frac {\delta}{\binom {d} {2}}
$$
By a union bound, we get that  $\| M_S - M_D\|_{\max} \leq \frac{\theta}{2d}$  with probability at least  $1 - \delta$  and hence  $\| M_S - M_D\|_{\mathrm{op}} \leq d\| M_S - M_D\|_{\max} \leq \frac{\theta}{2}$ , which concludes the proof.
Proposition A.3. Let  $c \geq 0$ ,  $\lambda \geq 1$ ,  $\sigma \leq \frac{1}{2\lambda}$  and  $D$  be a  $\lambda$ -nice distribution over  $\mathbb{R}^d$ . Then, for any unit vectors  $\mathbf{w}, \mathbf{v}, \mathbf{v}', \mathbf{u}, \mathbf{u}' \in \mathbb{R}^d$  with  $\langle \mathbf{w}, \mathbf{v} \rangle = \langle \mathbf{w}, \mathbf{v}' \rangle = 0$  and for some universal constant  $C > 0$  we have
(i)  $\mathbb{P}[|\langle \mathbf{w},\mathbf{x}\rangle |\leq \sigma ] = 2\sigma \cdot \alpha^C$  ,for some  $\alpha \in [\frac{1}{C\lambda},C\lambda ]$
(ii)  $\mathbb{E}[\langle \mathbf{v},\mathbf{x}\rangle^2\cdot \mathbb{1}\{||\mathbf{w},\mathbf{x}\rangle |\leq \sigma \} ] = 2\sigma \cdot \alpha^C$  ,for some  $\alpha \in [\frac{1}{C\lambda},C\lambda ]$
(iii)  $\mathbb{E}[\langle \mathbf{x},\mathbf{u}\rangle^2\langle \mathbf{x},\mathbf{u}'\rangle^2 ] = \alpha^C$  , for some  $\alpha \leq C\lambda$
(iv)  $\mathbb{E}[\langle \mathbf{v},\mathbf{x}\rangle^2\cdot \mathbb{1}\{\lvert |\langle \mathbf{w},\mathbf{x}\rangle \rvert \in [c,c + \sigma ]\} \} ]\leq 2\sigma \cdot \alpha^C$  , for some  $\alpha \leq C\lambda$
Proof. We start by deriving property (i). Recall the function  $Q$  from the definition of a  $\lambda$ -nice distribution, which upper-bounds the density of any two-dimensional projection of a  $\lambda$ -nice distribution we see that:
$$
\begin{array}{l} \mathbb {P} [ | \langle \mathbf {w}, \mathbf {x} \rangle | \leq \sigma ] = \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} q _ {\operatorname {s p a n} (\mathbf {v}, \mathbf {w})} (x _ {1}, x _ {2}) d x _ {1} d x _ {2} \\ \leq \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \\ \end{array}
$$
Now, note that the region  $\{(x_1,x_2):|x_1|\leq \sigma \}$  is a subset of the set
$$
\left\{\left(x _ {1}, x _ {2}\right): \left| x _ {2} \right| \leq \sigma \left| x _ {1} \right| \right\} \cup \left\{\left(x _ {1}, x _ {2}\right): \left| x _ {1} \right| \leq \sigma \& \left| x _ {2} \right| \leq 1 \right\}.
$$
$$
\begin{array}{l} \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \leq \\ 4 \arcsin (\sigma) \cdot \int_ {r = 0} ^ {\infty} 2 \pi r Q (r) d r + \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - 1} ^ {1} Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \leq O (\sigma \lambda) \\ \end{array}
$$
Note that in the last line above, we bounded the first term via the bound  $\int_{r=0}^{\infty} rQ(r) dr \leq \lambda$  from the definition of  $\lambda$ -nice distributions. Likewise, we bounded the second term via the inequality  $Q(r) \leq \lambda$  from the definition of  $\lambda$ -nice distributions. Overall, we get
$$
\mathbb {E} [ \langle \mathbf {v}, \mathbf {x} \rangle^ {2} \cdot \mathbb {1} \{| \langle \mathbf {w}, \mathbf {x} \rangle | \leq \sigma \} ] \leq O (\sigma \lambda)
$$
Now, we shall lower-bound the same quantity. We have
$$
\begin{array}{l} \mathbb {P} [ | \langle \mathbf {w}, \mathbf {x} \rangle | \leq \sigma ] = \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} q _ {\operatorname {s p a n} (\mathbf {v}, \mathbf {w})} (x _ {1}, x _ {2}) d x _ {1} d x _ {2} \\ \geq \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \frac {1}{2 \lambda}} ^ {\frac {1}{2 \lambda}} q _ {\operatorname {s p a n} (\mathbf {v}, \mathbf {w})} (x _ {1}, x _ {2}) d x _ {1} d x _ {2} \\ \end{array}
$$
Now, since  $\sigma \leq \frac{1}{2\lambda}$  via the premise of the lemma, we see that the whole region of integration on the right side of the set  $\{(x_1,x_2):\sqrt{x_1^2 + x_2^2}\leq \frac{1}{\lambda}\}$ . From the definition of  $\lambda$ -nice distributions, the density  $q_{\mathrm{span}(\mathbf{v},\mathbf{w})}$  is lower-bounded by  $1 / \lambda$  in this region. Therefore, we have
$$
\mathbb {P} [ | \langle \mathbf {w}, \mathbf {x} \rangle | \leq \sigma ] \geq \frac {2 \sigma}{\lambda} \cdot \frac {1}{\lambda} = \frac {2 \sigma}{\lambda^ {2}},
$$
which finishes the proof of property (i).
Now, we derive property (ii). Recall the function  $Q$  from the definition of a  $\lambda$ -nice distribution, which upper-bounds the density of any two-dimensional projection of a  $\lambda$ -nice distribution we see that:
$$
\begin{array}{l} \mathbb {E} [ \langle \mathbf {v}, \mathbf {x} \rangle^ {2} \cdot \mathbb {1} \{| \langle \mathbf {w}, \mathbf {x} \rangle | \leq \sigma \} ] = \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {2} ^ {2} \cdot q _ {\operatorname {s p a n} (\mathbf {v}, \mathbf {w})} (x _ {1}, x _ {2}) d x _ {1} d x _ {2} \\ \leq \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {2} ^ {2} \cdot Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \\ \end{array}
$$
$$
\begin{array}{l} \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {2} ^ {2} \cdot Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \leq \\ 4 \arcsin (\sigma) \cdot \int_ {r = 0} ^ {\infty} 2 \pi r ^ {3} Q (r) d r + \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - 1} ^ {1} x _ {2} ^ {2} \cdot Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \leq O (\sigma \lambda) \\ \end{array}
$$
Note that in the last line above, we bounded the first term via the bound on  $\int_{r=0}^{\infty} r^{3} Q(r) dr$  from the definition of  $\lambda$ -nice distributions. Likewise, we bounded the second term via the inequality  $Q(r) \leq \lambda$  from the definition of  $\lambda$ -nice distributions. Therefore, we get
$$
\begin{array}{l} \mathbb {E} [ \langle \mathbf {v}, \mathbf {x} \rangle^ {2} \cdot \mathbb {1} \{| \langle \mathbf {w}, \mathbf {x} \rangle | \leq \sigma \} ] = \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {2} ^ {2} \cdot q _ {\operatorname {s p a n} (\mathbf {v}, \mathbf {w})} (x _ {1}, x _ {2}) d x _ {1} d x _ {2} \\ \geq \int_ {x _ {1} = - \sigma} ^ {\sigma} \int_ {x _ {2} = - \frac {1}{2 \lambda}} ^ {\frac {1}{2 \lambda}} x _ {2} ^ {2} \cdot q _ {\operatorname {s p a n} (\mathbf {v}, \mathbf {w})} (x _ {1}, x _ {2}) d x _ {1} d x _ {2} \\ \end{array}
$$
$$
\mathbb {E} [ \langle \mathbf {v}, \mathbf {x} \rangle^ {2} \cdot \mathbb {1} \{| \langle \mathbf {w}, \mathbf {x} \rangle | \leq \sigma \} ] \geq \frac {2 \sigma}{\lambda} \cdot \frac {1}{4 \lambda^ {2}} \cdot \frac {1}{\lambda} = \frac {\sigma}{2 \lambda^ {4}},
$$
which finishes the proof of property (ii).
We proceed to property (iii). We will denote the angle between  $\mathbf{v}$  and  $\mathbf{v}'$  as  $\beta$ , which allows us to write
$$
\begin{array}{l} \mathbb {E} [ \langle \mathbf {x}, \mathbf {v} \rangle^ {2} \langle \mathbf {x}, \mathbf {v} ^ {\prime} \rangle^ {2} ] = \int_ {x _ {1} = - \infty} ^ {\infty} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {1} ^ {2} (x _ {1} \cos \beta + x _ {2} \sin \beta) ^ {2} q _ {\mathrm {s p a n} (\mathbf {v}, \mathbf {w})} (x _ {1}, x _ {2}) d x _ {1} d x _ {2} \\ \leq \int_ {x _ {1} = - \infty} ^ {\infty} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {1} ^ {2} (x _ {1} \cos \beta + x _ {2} \sin \beta) ^ {2} \cdot Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \\ \leq \int_ {x _ {1} = - \infty} ^ {\infty} \int_ {x _ {2} = - \infty} ^ {\infty} (x _ {1} ^ {2} + x _ {2} ^ {2}) ^ {2} \cdot Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \\ = \int_ {r = 0} ^ {\infty} 2 \pi r ^ {5} Q (r) d r \leq 2 \pi \lambda , \\ \end{array}
$$
which finishes the proof of property (iii).
Finally, we prove property (iv). For  $\beta \geq 0$  we have
$$
\begin{array}{l} \int_ {r = 0} ^ {\infty} r ^ {2} Q \left(\sqrt {r ^ {2} + \beta}\right) d r = \int_ {r = 0} ^ {1} r ^ {2} Q \left(\sqrt {r ^ {2} + \beta}\right) d r + \int_ {r = 1} ^ {\infty} r ^ {2} Q \left(\sqrt {r ^ {2} + \beta}\right) d r \\ \leq \lambda + \int_ {r = 1} ^ {\infty} r ^ {3} Q (\sqrt {r ^ {2} + \beta}) d r \quad \text {(s i n c e} \sup  _ {r \geq 0} Q (r) \leq \lambda) \\ \leq \lambda + \int_ {r ^ {\prime} = \sqrt {1 + \beta}} ^ {\infty} \left(r ^ {\prime 3} - \beta r ^ {\prime}\right) Q \left(r ^ {\prime}\right) d r ^ {\prime} \quad (\text {b y}) \\ \leq \lambda + \int_ {r = 0} ^ {\infty} r ^ {3} Q (r) d r \quad (\text {s i n c e} \beta r Q (r) \geq 0 \text {f o r a n y} r \geq 0) \\ \leq 2 \lambda \\ \end{array}
$$
Applying the above inequality to the quantity of property (iv), we get the desired result.
$$
\begin{array}{l} \mathbb {E} [ \langle \mathbf {v}, \mathbf {x} \rangle^ {2} \cdot \mathbb {1} \{| \langle \mathbf {w}, \mathbf {x} \rangle | \in [ c, c + \sigma ] \} ] = \int_ {| x _ {1} | \in [ c, c + \sigma ]} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {2} ^ {2} \cdot q _ {\operatorname {s p a n} (\mathbf {v}, \mathbf {w})} d x _ {1} d x _ {2} \\ \leq \int_ {| x _ {1} | \in [ c, c + \sigma ]} \int_ {x _ {2} = - \infty} ^ {\infty} x _ {2} ^ {2} \cdot Q \left(\sqrt {x _ {1} ^ {2} + x _ {2} ^ {2}}\right) d x _ {1} d x _ {2} \\ = \int_ {| x _ {1} | \in [ c, c + \sigma ]} \left(2 \int_ {r = 0} ^ {\infty} r ^ {2} \cdot Q \left(\sqrt {x _ {1} ^ {2} + r ^ {2}}\right) d r\right) d x _ {1} \\ \leq \int_ {| x _ {1} | \in [ c, c + \sigma ]} (4 \lambda) d x _ {1} \leq 8 \lambda \sigma \\ \end{array}
$$
This concludes the proof of Proposition A.3.
Proposition A.4 (Proposition 4.2 of [GKSV23]). There is a universal constant  $C > 0$ , such that for any  $\sigma > 0$ , there exists a continuously differentiable function  $\ell_{\sigma} : \mathbb{R} \to [0,1]$  with the following properties.
1. For any  $t \in [-\sigma / 6, \sigma / 6]$ ,  $\ell_{\sigma}(t) = \frac{1}{2} + \frac{t}{\sigma}$ .
2. For any  $t > \sigma / 2$ ,  $\ell_{\sigma}(t) = 1$  and for any  $t < -\sigma / 2$ ,  $\ell_{\sigma}(t) = 0$ .
3. For any  $t\in \mathbb{R}$ $\ell_{\sigma}^{\prime}(t)\in [0,C / \sigma ]$ $\ell_{\sigma}^{\prime}(t) = \ell_{\sigma}^{\prime}(-t)$  and  $|\ell_{\sigma}^{\prime \prime}(t)|\leq C / \sigma^{2}$
Proposition A.5 (PSGD Convergence [DKTZ20a], restated in [GKSV23]). Let  $\mathcal{L}_{\sigma}$  be as in Equation (4.1) with  $\sigma \in (0,1]$ ,  $\ell_{\sigma}$  as described in Proposition A.4,  $\lambda \geq 1$  and  $D_{\mathcal{X}\mathcal{Y}}$  such that the marginal  $D_{\mathcal{X}}$  on  $\mathbb{R}^d$  is  $\lambda$ -nice. Then for some universal constant  $C > 0$  and for any  $\epsilon > 0$  and  $\delta \in (0,1)$ , there is an algorithm whose time and sample complexity is  $O\left(\frac{\lambda^C d}{\sigma^4} + \frac{\lambda^C \log(1 / \delta)}{\epsilon^4 \sigma^4}\right)$ , which, having access to samples from  $D_{\mathcal{X}\mathcal{Y}}$ , outputs a list  $L$  of vectors  $\mathbf{w} \in \mathbb{S}^{d-1}$  with  $|L| = O\left(\frac{\lambda^C d}{\sigma^4} + \frac{\lambda^C \log(1 / \delta)}{\epsilon^4 \sigma^4}\right)$  so that there exists  $\mathbf{w} \in L$  with
$$
\left\| \nabla_ {\mathbf {w}} \mathcal {L} _ {\sigma} (\mathbf {w}; D _ {\mathcal {X Y}}) \right\| _ {2} \leq \epsilon , w i t h p r o b a b i l i t y a t l e a s t 1 - \delta .
$$
In particular, the algorithm performs Stochastic Gradient Descent on  $\mathcal{L}_{\sigma}$  Projected on  $\mathbb{S}^{d - 1}$  (PSGD).
B Proofs from Section 3
We restate Lemma 3.1 here for convenience.
Lemma B.1 (Lemma 3.1). Let  $D_{\mathcal{X}\mathcal{Y}}$  be a distribution over  $\mathbb{R}^d \times \{\pm 1\}$ ,  $\mathbf{w} \in \mathbb{S}^{d - 1}$ ,  $\theta \in (0,\pi /4]$ ,  $\lambda \geq 1$  and  $\delta \in (0,1)$ . Then, for a sufficiently large constant  $C$ , there is a tester that given  $\delta$ ,  $\theta$ ,  $\mathbf{w}$  and a set  $S$  of samples from  $D_{\mathcal{X}}$  with size at least  $C \cdot \left(\frac{d^4}{\theta^2\delta}\right)$ , runs in time poly  $(d,\frac{1}{\theta},\frac{1}{\delta})$  and satisfies the following specifications:
Proof. The testing algorithm receives integer  $d$ , set  $S \subset \mathbb{R}^d$ ,  $\mathbf{w} \in \mathbb{S}^{d-1}$ ,  $\theta \in (0, \pi/4]$ ,  $\lambda \geq 1$  and  $\delta \in (0, 1)$  and does the following for some sufficiently large universal constant  $C_1 > 0$ :