SlowGuess/NeurIPS_2023_OCR huggingface dataset:
**Hardness:** To see that the list size must be at least  $\exp(1/\eta)$ , consider the set of inlier distributions given by  $\{D_s \mid s \in \{\pm 1\}^d\}$  where each  $D_s$  is defined as follows:  $D_s$  is a distribution over  $\mathbb{R}^d$  such that each coordinate independently takes the value  $s_i$  with probability  $1/d$ , and 0 otherwise.
Each  $D_{s}$  defined above belongs to  $\mathcal{D}_{\sqrt{1 - 1 / d}}$  since  $\mathbf{E}_{x\sim D_s}[x] = s / d$  and
$$
\begin{array}{l} \sigma^ {2} := \mathbf {E} _ {x \sim D _ {s}} [ \| x - s / d \| ^ {2} ] = \sum_ {i = 1} ^ {d} \mathbf {E} _ {x _ {i} \sim (D _ {s}) _ {i}} [ (x _ {i} - s _ {i} / d) ^ {2} ] \\ = \sum_ {i = 1} ^ {d} (1 - 1 / d) (1 / d) ^ {2} + (1 / d) (1 - 1 / d) ^ {2} = (1 - 1 / d). \\ \end{array}
$$
We will set the oblivious noise distribution for each  $D_{s}$  to be  $-D_{s}$ . Our objective is to demonstrate that the distribution of  $D_{s} - D_{s}$  is the same for all  $s$  and is independent of  $s$ . This means that we cannot identify  $s$  by seeing samples from  $D_{s} - D_{s}$ .
Then, since the means of  $D_{s}$  and  $D_{s}^{\prime}$  for any distinct pair  $s, s^{\prime} \in \{\pm 1\}^{d}$  differ by at least  $1 / d$ , if we set  $d = 1 / \eta$  we see that there are  $2^{1 / \eta}$  possible different values of the original mean, each pair being at least  $\eta$  far apart, which is larger than  $\eta \sigma^{2} = \eta (1 - \eta)$ .
We can assume, without loss of generality, that  $s = 1$ , where  $\mathbf{1}$  represents the all-ones vector. Each coordinate of  $D_{s}$  can be viewed as a random coin flip, taking the value 0 with probability  $1 - 1 / d$  and 1 with probability  $1 / d$ .
The probability of obtaining the all-zeros vector is given by  $(1 - 1 / d)^d$ , which approaches a constant value for sufficiently large  $d$ , and so,  $\mathbf{Pr}_{x\sim D_s}[x = 0]\geq 0.001$ , i.e., the  $\alpha$  for the oblivious noise is at least a constant. In fact, it can be as large as  $1 / e > 0.35$  for large enough  $d$ .
Let the oblivious noise be  $-D$ . Now, consider the distribution of  $x + y$ , where  $x$  follows the distribution  $D$  and  $y$  follows the distribution  $-D$ . If we focus on the first coordinate,  $x_{1} + y_{1}$ , we observe that it follows a symmetric distribution over  $\{-1,0,1\}$  which does not depend on  $s_1$ . Also, each coordinate exhibits the same distribution, and they are drawn independently of one another. Hence, the final distribution is independent of  $s$ , so we get our result.
E Proof of Corollary 4.2
Below, we restate Corollary 4.2 for convenience.
Corollary E.1. Given access to oblivious noise oracle  $\mathcal{O}_{\alpha,\sigma,f}$ , a  $(O(\eta\sigma),\epsilon)$ -inexact-learner  $\mathcal{A}_G$  running in time  $T_G$ , there exists an algorithm that takes  $\mathrm{poly}((1/\alpha)^{1/\eta^2},(O(1)/\eta)^{1/\eta},\log (T_G / \delta \eta \alpha))$  samples, runs in time  $T_G\cdot \mathrm{poly}(d,(1/\alpha)^{1/\eta^2},(O(1)/\eta)^{1/\eta},\log (1/\eta \alpha \delta))$ , and with probability  $1 - \delta$  returns a list  $\mathcal{L}$  of size  $\tilde{O}((1/\alpha)^{1/\eta^2})$  such that  $\min_{x\in \mathcal{L}}\|\nabla f(x)\| \leq O(\eta \sigma) + \epsilon$ . Additionally, the exponential dependence on  $1/\eta$  in the list size is necessary.
Proof. This follows by substituting the guarantees of Fact 2.1 for the algorithm  $\mathcal{A}_{ME}$  in Theorem 1.4.
F Varying Noise Assumptions
In this section, we demonstrate that changing some distributional assumptions of the oblivious noise  $\xi$  can yield better guarantees or simpler algorithms.
F.1 Stronger Concentration on  $\xi$  Yields Polynomial Dependence on  $1 / \eta$
We show that if the oblivious noise has bounded tails we can avoid the exponential dependence on  $1 / \eta$ , which we have shown in Appendix D to be necessary under our standard assumptions.
For the sake of demonstration, we will assume the standard assumptions of heavy-tailed stochastic optimization where the mean of  $\xi$  is equal to zero and the variance of  $\xi$  is upper bounded by  $\sigma$ , as is done in Gorbunov et al. [2020]. In this specific setting, our algorithm remains the same but our analysis becomes tighter in Item 2 of Claim C.4. Our original analysis yields an exponential dependence on  $1 / \eta$  due to the upper bound on  $k$ , yet with concentration on the noise, we can show that  $k = \mathrm{poly}(1 / \alpha, 1 / \eta)$ .
Due to our specific construction of  $\tilde{P}(k)$  in the proof of Claim C.4 of Item 1, we have that  $k\tilde{P}(k) \leq O(k^{-1/3})$  and  $\alpha/2 \leq \tilde{P}(1) \leq \sum_{j=1}^{k} \tilde{P}(j)$ . Thus, when  $k \geq O(1 / (\alpha \eta)^3)$ , the inequality of Item 2 is satisfied. The fact that  $\alpha/2 \leq \tilde{P}(1)$  is given, and thus, what remains is to show  $\tilde{P}(k) \leq k^{-4/3}$ , which follows by splitting the summation in  $\tilde{P}(i, z)$  over  $j = 1, \ldots, k-1$  into  $j \in [1, k^{2/3}]$  and  $j \in (k^{2/3}, k-1]$ . The sum over  $[1, k^{2/3}]$  is upper bounded by  $O(k^{-4/3})$  and the sum over  $(k^{2/3}, k-1]$  is upper bounded by  $O(A^{-2}k^{-4/3}) = O(\alpha k^{-4/3})$  using Chebyshev's inequality. Therefore,  $\tilde{P}(k) \leq k^{-4/3}$  and  $k = \mathrm{poly}(1 / \alpha, 1 / \eta)$  in Claim 3.3.
Ultimately, it would be an interesting direction to identify conditions necessary and sufficient for a polynomial dependence on  $1 / \eta$ .
F.2 Small Fraction of Adversarial Corruptions
If, on the other hand, at each point you have access to gradients  $v(x)$  such that with probability  $1 - \epsilon$  you see  $\nabla f(x,\gamma)$ , which concentrates around  $\nabla f(x,\gamma)$  and with probability  $\epsilon$  you see an arbitrary point, then one can use robust mean estimation instead of our location estimation algorithm and recover guarantees which are correct up to an error of  $O(\sqrt{\epsilon})$ .
In this case, there is no need for list-decoding or robust location estimation.