SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For convenience we denote the normalized projection of any vector  $\mathbf{r}$  as  $\tilde{\mathbf{r}} := \frac{\pmb{\Sigma}^{1/2}\mathbf{r}}{\|\pmb{\Sigma}^{1/2}\mathbf{r}\|_2}$ . Now let  $\tilde{\mathbf{r}} \in \mathbb{S}^{d-1}$  be a vector such that  $\min\{\|\tilde{\mathbf{r}} - \tilde{\mathbf{r}}_*\|_2, \|\tilde{\mathbf{r}} + \tilde{\mathbf{r}}_*\|_2\} \geq \varepsilon_2$ . Hence, using the definitions from Lemma H.4,  $|\gamma(\mathbf{r})| \leq 1 - \varepsilon_2^2/2$  while  $\gamma(\mathbf{r}_*) = 1$  which implies  $\rho(\mathbf{r}_*) - \rho(\mathbf{r}) \geq \kappa_3(\mathbf{p})\varepsilon_2^2/2$ . Note that  $\rho(\mathbf{r}) \leq \rho(\mathbf{r}_*) = 2 + \kappa_3(\mathbf{p})$ . Choosing  $\varepsilon_1 < \frac{\kappa_3(\mathbf{p})}{4\theta(2 + \kappa_3(\mathbf{p}))}\varepsilon_2^2$ , we obtain that  $\rho(\mathbf{r}_*)(1 - \theta(\mathbf{p})\varepsilon_1) > \rho(\mathbf{r})(1 + \theta(\mathbf{p})\varepsilon_1)$ . Using this along with the bound (*) we obtain that w.p. at least  $1 - \delta$ ,  $\hat{\rho}(\mathbf{r}_*) > \hat{\rho}(\mathbf{r})$  when  $\varepsilon_2 > 0$ . Since our algorithm returns  $\hat{\mathbf{r}}$  as the maximizer of  $\hat{\rho}$ , w.p. at least  $1 - \delta$  we get  $\min\{\|\tilde{\mathbf{r}} - \tilde{\mathbf{r}}_*\|_2, \| \tilde{\mathbf{r}} + \tilde{\mathbf{r}}_*\|_2\} \leq \varepsilon_2$ . Using Lemma 2.1,  $\min\{\|\hat{\mathbf{r}} - \mathbf{r}_*\|_2, \| \hat{\mathbf{r}} + \mathbf{r}_*\|_2\} \leq 4\sqrt{\frac{\lambda_{\max}}{\lambda_{\min}}} \varepsilon_2$ . Substituting  $\varepsilon_2 = \frac{\varepsilon}{4}\sqrt{\frac{\lambda_{\min}}{\lambda_{\max}}}$ ,  $\| \mathbf{r} - \mathbf{r}_*\|_2 \leq \varepsilon$  w.p. at least  $1 - \delta$ . The conditions on  $\varepsilon_1$  are satisfied by taking it to be  $\leq O\left(\frac{\kappa_3(\mathbf{p})\varepsilon^2\lambda_{\min}}{\theta(\mathbf{p})(2 + \kappa_3(\mathbf{p}))\lambda_{\max}}\right)$ , and thus we can take  $m \geq O\left((d/\varepsilon^4)\log(d/\delta)\left(\frac{\lambda_{\max}}{\lambda_{\min}}\right)^2\theta(\mathbf{p})^2\left(\frac{1}{\kappa_3(\mathbf{p})}\right)^2\right)$ . Taking  $m \geq O\left((d/\varepsilon^4)\log(d/\delta)\left(\frac{\lambda_{\max}}{\lambda_{\min}}\right)^4q^4\left(1/\sum_{k=1}^{q-1}p_k^2\right)^2\right)$  satisfies this using bounds in Defn. H.3. This completes the proof.
We observe that the sample complexity bound is worse for  $\{p_1, \ldots, p_{q-1}\}$  which are not concentrated i.e., the probability of the label sum is supported over many different values. This occurs for e.g. in the noisy setting when the label flip noise is large (see Appendix F).