SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of Theorem 1.3. Denote by  $\mathbf{v}^*$ , a unit vector with error at most opt, i.e.,  $\mathbf{Pr}_{(\mathbf{x},y)\sim D}[\mathrm{sign}(\mathbf{v}^*\cdot \mathbf{x})\neq y]\leq \mathrm{opt}$ . We start by analyzing Algorithm 5. In Line 2, Algorithm 5 uses Algorithm 2 with parameter  $\eta = 1 / (20000C_{\mathrm{A}}^{2})$  to get a hypothesis  $\mathbf{v}^{(0)}$  with small distance with  $\mathbf{v}^*$ . From Proposition 2.1, Algorithm 2 either reports that  $\mathbf{x}$ -marginal of  $D$  is not  $\mathcal{N}(\mathbf{0},\mathbf{I})$  or outputs a vector  $\mathbf{v}^{(0)}$  small distance with  $\mathbf{v}^*$ . If Algorithm 2 reports that the  $\mathbf{x}$ -marginal of  $D$  is not  $\mathcal{N}(\mathbf{0},\mathbf{I})$ , we can terminate the algorithm. Conditioned on the event that the algorithm did not terminate, then we have
$$
\left\| \mathbf {v} ^ {(0)} - \mathbf {v} ^ {*} \right\| _ {2} \leq C _ {\mathrm {A}} \sqrt {\operatorname {o p t} + \eta}. \tag {17}
$$
We consider two cases depending on how large the value of opt is. If  $\mathrm{opt} > 1 / (20000C_{\mathrm{A}}^2)$ , then any unit vector achieves constant error; therefore, conditioned that the algorithm did not terminate on any proceeding test, any vector we output will satisfy the guarantees of Theorem 1.3. For the rest of the proof, we consider the case where  $\mathrm{opt}\leq 1 / (20000C_{\mathrm{A}}^{2})$ . In this case,  $\left\| \mathbf{v}^{(0)} - \mathbf{v}^{*}\right\|_{2}\leq 1 / 100$ , this means that Algorithm 5 on Lines 4-4 will decrease the distance between the current hypothesis and  $\mathbf{v}^{*}$ .
Conditioned on the event that the algorithm did not terminate at Lines 4-4 of Algorithm 5, we claim that there must have some  $0 \leq t^{*} \leq \log(1 / \epsilon)$  such that  $\left\| \mathbf{v}^{(t^{*})} - \mathbf{v}^{*} \right\|_{2} \leq O(\mathrm{opt} + \epsilon)$ . Let  $t' \in \mathbb{N}$  be
the maximum value so that  $2^{-t'} / 100 \geq 40000\mathrm{opt}$ , then, for all  $t \leq \min(t', \log(1/\epsilon))$  it holds that  $2^{-t} / 100 \geq 40000\mathrm{opt}$ . From Lemma 3.5, we have that for all  $t \leq \min(t', \log(1/\epsilon))$  it holds that
$$
\left\| \mathbf {v} ^ {*} - \mathbf {v} ^ {(t)} \right\| _ {2} \leq 2 ^ {- t - 1} / 1 0 0.
$$
From the above, note that if  $t' > \log(1/\epsilon)$  then  $\left\| \mathbf{v}^{(\log(1/\epsilon))} - \mathbf{v}^* \right\|_2 \leq \epsilon / 100$ . If  $t' \leq \log(1/\epsilon)$ , we have that  $\left\| \mathbf{v}^{(t')} - \mathbf{v}^* \right\|_2 \leq O(\mathrm{opt})$ , which proves our claim. It remains to show that Algorithm 5 will return a vector  $\mathbf{v}'$  so that  $\operatorname{Pr}_{(\mathbf{x}, y) \sim D}[\operatorname{sign}(\mathbf{v}' \cdot \mathbf{x}) \neq y] = O(\operatorname{opt} + \epsilon)$ . From Lemma 3.1, conditioned that Algorithm 5 did not terminate on Lines 5-5b, we have that for all vectors  $\mathbf{v}^{(0)}, \ldots, \mathbf{v}^{(\log(1/\epsilon))}$  generated on Lines 4-4 of Algorithm 5, we have that
$$
\mathop{\Pr}\limits_{(\mathbf{x},y)\sim D}[\operatorname {sign}(\mathbf{v}^{(t)}\cdot \mathbf{x})\neq y]\leq O\left(\left\| \mathbf{v}^{(t)} - \mathbf{v}^{*}\right\|_{2}\right).
$$
Hence, we can conclude that  $\mathbf{Pr}_{(\mathbf{x},y)\sim D}[\mathrm{sign}(\mathbf{v}^{(t')}\cdot \mathbf{x})\neq y]\leq O(\mathrm{opt} + \epsilon)$ . Then, applying Lemma 3.6 gives that the returned halfspace will have error at most  $10\cdot \mathbf{Pr}_{(\mathbf{x},y)\sim D}[\mathrm{sign}(\mathbf{v}^{(t')}\cdot \mathbf{x})\neq y] + \epsilon = O(\mathrm{opt} + \epsilon)$ .
To conclude the proof, note that each invocation of Algorithms 1, 2 and 3 consumes  $\mathrm{poly}(d,1 / \epsilon)\log (1 / \tau)$  unlabeled samples and  $O\left(d\left(\log (d / \tau) + \log \log (1 / \epsilon)\right)\right)$  label queries. Moreover, the total number of invocations is upper bounded by  $\Theta (\log (1 / \epsilon))$  For algorithm Algorithm 4, it draws  $\mathrm{poly}(1 / \epsilon)$  many unlabeled samples and use  $\Theta \left(\log^2 (1 / \epsilon)\left(\log \log (1 / \epsilon) + \log (1 / \tau)\right)\right)$  label queries. Therefore the total unlabeled samples consumed by Algorithm 5 is  $N = \mathrm{poly}(d,1 / \epsilon)$  and the total label quereis used is at most
$$
q = O \left(d \log (1 / \epsilon) (\log (d / \tau) + \log \log (1 / \epsilon)) + \log^ {2} (1 / \epsilon) (\log \log (1 / \epsilon) + \log (1 / \tau))\right).
$$
All the procedures run in time that scales polynomially with the number of samples drawn. This concludes the proof.